Commit Graph

18 Commits

Author SHA1 Message Date
Aart Bik ec15148db2 [dart/fuzzer] no target on unsupported build targets
Rationale:
This if "hides" the dart_libfuzzer target for
build targets that do not support the fuzzer tool
(e.g. no asan, ia32, etc.). This makes "build all"
possible again, and yields "unknown target 'dart_libfuzzer'"
for unsupported case. Also updates README accordingly
and fixes some formatting.

https://github.com/dart-lang/sdk/issues/36774

Change-Id: I61d85ae00d13a0eda3818d40b992cd81f5ee0357
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100630
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2019-04-26 20:51:15 +00:00
Aart Bik d136a35244 [dart/fuzzer] Add O3 as random command line switch
Also further refactors switch selection from naming.
The JIT stress execution modes don't really need
their own visible name (you cannot really control
which stress test to use anyway with --mode), so it's
cleaner to remove the idea of controlling switches
altogether.

Change-Id: I98aa71ce8653013fb45a4c482d7b4c5a02143a55
Reviewed-on: https://dart-review.googlesource.com/c/92386
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2019-02-08 18:04:05 +00:00
Aart Bik b05b8777d4 [dart/fuzzer] Refactor switch setup, add VFP to ARM
Rationale:
The fuzz test constructors were getting a bit convoluted
with mode specific setup. This CL refactors the flag
and mode name setup into the factory method, making
all constructors small again.
Change-Id: I5fd34e089315c89e674c7b28a4f298143fbeaeea
Reviewed-on: https://dart-review.googlesource.com/c/92380
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2019-02-07 22:41:18 +00:00
Aart Bik bc0cbac5cc [dart/fuzzer] remove obsolete reference to py file
Change-Id: I4532e425904d95d3f4883422da943bc5a9ab9c7c
Reviewed-on: https://dart-review.googlesource.com/c/90782
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2019-01-23 19:13:36 +00:00
Aart Bik a16f1bd67b [dart/fuzzer] Fix js execution mode specification
Rationale:
With recent refactoring, we need to give js a tag,
which actually makes the naming more consistent
(even though path for js is computed slightly differently).
Change-Id: I5649f38ae41c9bf133629f8a0602f03be99ac523
Reviewed-on: https://dart-review.googlesource.com/c/86945
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-12-11 19:18:10 +00:00
Ryan Macnak f6b5f9ce2e [vm] Add --gc_every to trigger major GC every N stack overflow checks.
Add DartFuzz configurations for various stress flags.

Change-Id: I2278ffcd1ea4193067c4c9f12ec669e48a182f00
Reviewed-on: https://dart-review.googlesource.com/c/86822
Reviewed-by: Aart Bik <ajcbik@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-12-11 01:28:02 +00:00
Aart Bik bbe9333af5 [dart/fuzzer] Report stderr on non-output divergence
Rationale:
The output for output divergences is too lenghty to
report (and should be reproducable anyway). However,
for any other divergence, we want to have a better
clue on what went wrong, since this may indicate
an infrastructure problem that is harder to reproduce.
Change-Id: I20e14a753048406fad6cbd55a270bf6958920915
Reviewed-on: https://dart-review.googlesource.com/c/86000
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-12-04 19:06:26 +00:00
Aart Bik 81d899ac30 [dart/fuzzer] Add mode to stress test JIT optimization
Rationale:
We get pretty good coverage on unoptimized path for JIT,
but the optimized path depends heavily on whether functions
are called often in the fuzz generated code. This mode
adds coverage of going quicker into the optimized path.

Note:
This mode already exposed an issue with DEBUG JIT.

https://github.com/dart-lang/sdk/issues/35196

Change-Id: Ib32b4ba449cb7b2b777d669ef264742aa44a9ccc
Reviewed-on: https://dart-review.googlesource.com/c/84680
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-11-16 22:25:24 +00:00
Aart Bik c60cff1d33 [dart/fuzzer] Introduce time-based test runs
Rationale:
To maximize CPU utilitization during fuzzing runs,
we now use a time-out on all the individual fuzz
testing runs (rather than repeat based). This way,
the "faster" tests will cover more ground, while
the "slower" straggler tests will not break the test
with unnecessary time-outs at cluster level.
Change-Id: I5f0f9646da3fbd8b361a7ea43444b1896a45b57f
Reviewed-on: https://dart-review.googlesource.com/c/84047
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-11-12 18:47:59 +00:00
Aart Bik 751cf1974a [dart/fuzzer] Add DBC as execution mode.
Rationale:
More coverage is better!
Change-Id: Ic04f4caf7d02e1db9398f3d51ae44f6b51d6542d
Reviewed-on: https://dart-review.googlesource.com/c/78862
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-09 18:56:28 +00:00
Aart Bik d31db6581c [dart/fuzzer] Add kbc execution mode
Rationale:
This enables testing of bytecode interpretation,
compilation, or mixed mode.


Change-Id: Ie4322c62f1f6a1abe4896ec98c4edf40d25b7135
Reviewed-on: https://dart-review.googlesource.com/c/78186
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-05 05:39:09 +00:00
Aart Bik 74f2348941 [dart/fuzzer] DartFuzz Improvements
Included:
(1) Added more expression variants in generator.
(2) Added --isolates option in driver (enables parallel execution).


Change-Id: I4e04490acc3046fd4771dd07502ba0fcdceb7df5
Reviewed-on: https://dart-review.googlesource.com/c/77720
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-04 16:47:58 +00:00
Aart Bik 0016f80467 [dart/fuzzer] Pick execution modes at random by default
Rationale:
Provides randomness for our bots without having to
explicitly code all execution modes.
Change-Id: Ida680ece59dd05e6fc11aa60ab9d4deacdbf3d05
Reviewed-on: https://dart-review.googlesource.com/c/77664
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-10-02 20:10:07 +00:00
Aart Bik 19e844ed5b [dart/fuzzing] Use current directory as SDK top (by default)
Rationale:
DART_TOP can now be provided through switch, as
an environment variable or, by default, using the
current directory from which script is started.
This simplifies set up in cluster runs.
Change-Id: Icb4f720b7a7bb7b349ce158d39574c271132c224
Reviewed-on: https://dart-review.googlesource.com/76201
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-09-24 19:02:01 +00:00
Aart Bik cbf60679eb [vm/fuzzer] Migrated driver from Python to Dart.
Rationale:
Remove last part of Python in favor of Dart. Note that
this is still under heavy development, with many new
features planned. But it is a runnable system now which
we be deployed from the comfort of your own desktop or
on a cluster if needed.
Change-Id: I96fc8973bcc00c69849434087c57ff897d01b79a
Reviewed-on: https://dart-review.googlesource.com/75621
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-09-20 17:41:59 +00:00
Aart Bik dd2d4af442 [vm/fuzzer] Migrated from Python to Dart.
Rationale:
I suppose that "Drinking the Kool-Aid" requires me to migrate
the Dart fuzzer test tools from Python to Dart. This CL is a first
step, migrating dartfuzz.py to dartfuzz.dart. In addition, more
constructs were added to the generation, although it is still
far from finished of course!

Todo:
Migrate the driver (run_dartfuzz_test.py) to Dart as well.
Keep on adding more constructs to generate.
Change-Id: I9a8cb4fa9482d0d41c9af3e9bd3ac2e59286c949
Reviewed-on: https://dart-review.googlesource.com/72361
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-11 17:52:32 +00:00
Aart Bik 85350b0c53 [vm/fuzzer] Added configuration support in DartFuzz.
Rationale:
This change allows running Dart in various
different configurations (e.g. jit-ia32 and
aot-arm64).
Change-Id: I16700cd462873766f47a33bd866c7fa7cba603d1
Reviewed-on: https://dart-review.googlesource.com/63985
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-07-11 05:15:26 +00:00
Aart Bik bde2a8a981 [vm/fuzzer] Start of Dart fuzz tester.
Rationale:
Generation-based fuzz testing for Dart that
finds divergences for different execution paths
(e.g. AOT vs JIT) on randomly generated Dart programs.
This project is still under heavy development, this
first CL just introduces the concepts.

Note:
The JS mode may report quite a few divergences due to
dart2js - VM semantic differences on integer ops.

Change-Id: I2854341d9c2744556f19e44a825cddb7437405ee
Reviewed-on: https://dart-review.googlesource.com/63323
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-07-02 17:59:19 +00:00