Commit Graph

166 Commits

Author SHA1 Message Date
Aart Bik e585605488 [dart/fuzzer] Ignore output directory option
Rationale:
Step towards parallel execution.
Change-Id: I1c6ba3eaa33834d6f4b7ba5489acca567fc00b3d
Reviewed-on: https://dart-review.googlesource.com/c/83041
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-11-06 17:43:40 +00:00
Aart Bik 751cf1974a [dart/fuzzer] Add DBC as execution mode.
Rationale:
More coverage is better!
Change-Id: Ic04f4caf7d02e1db9398f3d51ae44f6b51d6542d
Reviewed-on: https://dart-review.googlesource.com/c/78862
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-09 18:56:28 +00:00
Aart Bik 5b1444503e [dart/fuzzer] Minor output tweaks
Rationale:
(1) does not print output difference on clusters
    to avoid cluttering logs (difference can be
    reproduced locally anyway after the fact)
(2) prints total # divergences at end in summary
Change-Id: I6fb736178569bfc0fa41afc525dee8f76ea7b465
Reviewed-on: https://dart-review.googlesource.com/c/78581
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-08 17:59:09 +00:00
Aart Bik 4d69ecf4be [dart/fuzzer] Avoid OOM situations that may yield false divergence
Bug:
https://github.com/dart-lang/sdk/issues/34688
https://github.com/dart-lang/sdk/issues/34681

Change-Id: Ia6426e2f6415294352d2a3bd3c10ed575ad8c261
Reviewed-on: https://dart-review.googlesource.com/c/78384
Reviewed-by: Aart Bik <ajcbik@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-06 00:13:20 +00:00
Aart Bik f048ef3da8 [dart/fuzzer] Periodic output to keep cluster logs happy
Rationale:
Even though I tried to keep fuzz testing log output to a
bare minimum, not showing something at least every 20 minutes
may cause a cluster run timeouts. Therefore, show some progress
indicators during run to keep cluster logs happy.
Change-Id: I59ea0e50d2ccbe78cf09fc2ee397eeb404ec72a2
Reviewed-on: https://dart-review.googlesource.com/c/78321
Reviewed-by: Aart Bik <ajcbik@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-05 23:53:27 +00:00
Aart Bik d31db6581c [dart/fuzzer] Add kbc execution mode
Rationale:
This enables testing of bytecode interpretation,
compilation, or mixed mode.


Change-Id: Ie4322c62f1f6a1abe4896ec98c4edf40d25b7135
Reviewed-on: https://dart-review.googlesource.com/c/78186
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-05 05:39:09 +00:00
Aart Bik e1421f3d08 [dart/fuzzer] avoid timeouts
Rationale:
Based on monitoring several runs, AOT on SIMARM(64)
simply takes too much time, resulting in sessions
that only have timeouts. Until we fix the runtime,
this CL disables these runs just to avoid wasting
cluster cycles.
Change-Id: I626ab07c64bd097dc8d654dbfc6a8030bc7803d1
Reviewed-on: https://dart-review.googlesource.com/c/78181
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-04 21:42:50 +00:00
Aart Bik 8d0281e03b [dart/fuzzer] Shard fuzzer support
Change-Id: I727498e5fbc321bbcd9546f43b25cf3c47209dd1
Reviewed-on: https://dart-review.googlesource.com/c/78180
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-04 20:47:20 +00:00
Aart Bik 74f2348941 [dart/fuzzer] DartFuzz Improvements
Included:
(1) Added more expression variants in generator.
(2) Added --isolates option in driver (enables parallel execution).


Change-Id: I4e04490acc3046fd4771dd07502ba0fcdceb7df5
Reviewed-on: https://dart-review.googlesource.com/c/77720
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-04 16:47:58 +00:00
Aart Bik 0016f80467 [dart/fuzzer] Pick execution modes at random by default
Rationale:
Provides randomness for our bots without having to
explicitly code all execution modes.
Change-Id: Ida680ece59dd05e6fc11aa60ab9d4deacdbf3d05
Reviewed-on: https://dart-review.googlesource.com/c/77664
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-02 20:10:07 +00:00
Aart Bik bdeb6a3f95 [dart/fuzzer] Remove use of PATH in env
Hardcoding the paths to dart compiler and runtime
ensures the proper binaries are used or an error
is reported (before we could silently fall back
to another toolchain if the first part of PATH
failed, but alternative followed in a later part).

Change-Id: Ib8d2c4f3fd7ab9b78c8d11585017d506e2812333
Reviewed-on: https://dart-review.googlesource.com/77001
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-01 19:25:14 +00:00
Aart Bik 19e844ed5b [dart/fuzzing] Use current directory as SDK top (by default)
Rationale:
DART_TOP can now be provided through switch, as
an environment variable or, by default, using the
current directory from which script is started.
This simplifies set up in cluster runs.
Change-Id: Icb4f720b7a7bb7b349ce158d39574c271132c224
Reviewed-on: https://dart-review.googlesource.com/76201
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-09-24 19:02:01 +00:00
Aart Bik cbf60679eb [vm/fuzzer] Migrated driver from Python to Dart.
Rationale:
Remove last part of Python in favor of Dart. Note that
this is still under heavy development, with many new
features planned. But it is a runnable system now which
we be deployed from the comfort of your own desktop or
on a cluster if needed.
Change-Id: I96fc8973bcc00c69849434087c57ff897d01b79a
Reviewed-on: https://dart-review.googlesource.com/75621
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-09-20 17:41:59 +00:00
Aart Bik dd2d4af442 [vm/fuzzer] Migrated from Python to Dart.
Rationale:
I suppose that "Drinking the Kool-Aid" requires me to migrate
the Dart fuzzer test tools from Python to Dart. This CL is a first
step, migrating dartfuzz.py to dartfuzz.dart. In addition, more
constructs were added to the generation, although it is still
far from finished of course!

Todo:
Migrate the driver (run_dartfuzz_test.py) to Dart as well.
Keep on adding more constructs to generate.
Change-Id: I9a8cb4fa9482d0d41c9af3e9bd3ac2e59286c949
Reviewed-on: https://dart-review.googlesource.com/72361
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-11 17:52:32 +00:00
Aart Bik 85350b0c53 [vm/fuzzer] Added configuration support in DartFuzz.
Rationale:
This change allows running Dart in various
different configurations (e.g. jit-ia32 and
aot-arm64).
Change-Id: I16700cd462873766f47a33bd866c7fa7cba603d1
Reviewed-on: https://dart-review.googlesource.com/63985
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-07-11 05:15:26 +00:00
Aart Bik bde2a8a981 [vm/fuzzer] Start of Dart fuzz tester.
Rationale:
Generation-based fuzz testing for Dart that
finds divergences for different execution paths
(e.g. AOT vs JIT) on randomly generated Dart programs.
This project is still under heavy development, this
first CL just introduces the concepts.

Note:
The JS mode may report quite a few divergences due to
dart2js - VM semantic differences on integer ops.

Change-Id: I2854341d9c2744556f19e44a825cddb7437405ee
Reviewed-on: https://dart-review.googlesource.com/63323
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-07-02 17:59:19 +00:00