Aart Bik
ec15148db2
Rationale: This if "hides" the dart_libfuzzer target for build targets that do not support the fuzzer tool (e.g. no asan, ia32, etc.). This makes "build all" possible again, and yields "unknown target 'dart_libfuzzer'" for unsupported case. Also updates README accordingly and fixes some formatting. https://github.com/dart-lang/sdk/issues/36774 Change-Id: I61d85ae00d13a0eda3818d40b992cd81f5ee0357 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100630 Commit-Queue: Aart Bik <ajcbik@google.com> Reviewed-by: Alexander Aprelev <aam@google.com>
109 lines
4.7 KiB
Markdown
109 lines
4.7 KiB
Markdown
DartFuzz
|
|
========
|
|
|
|
DartFuzz is a tool for generating random programs with the objective
|
|
of fuzz testing the Dart project. Each randomly generated program
|
|
can be run under various modes of execution, such as using JIT,
|
|
using AOT, using JavaScript after dart2js, and using various target
|
|
architectures (x86, arm, etc.). Any difference between the outputs
|
|
(**divergence**) may indicate a bug in one of the execution modes.
|
|
|
|
How to run DartFuzz
|
|
===================
|
|
To generate a single random Dart program, run
|
|
|
|
dart dartfuzz.dart [--help] [--seed SEED] FILENAME
|
|
|
|
where
|
|
|
|
--help : prints help and exits
|
|
--seed : defines random seed (system-set by default)
|
|
|
|
The tool provides a runnable main isolate. A typical single
|
|
test run looks as:
|
|
|
|
dart dartfuzz.dart fuzz.dart
|
|
dart fuzz.dart
|
|
|
|
How to start DartFuzz testing
|
|
=============================
|
|
To start a fuzz testing session, run
|
|
|
|
dart dartfuzz_test.dart [--help]
|
|
[--isolates ISOLATES ]
|
|
[--repeat REPEAT]
|
|
[--time TIME]
|
|
[--true_divergence]
|
|
[--mode1 MODE]
|
|
[--mode2 MODE]
|
|
|
|
where
|
|
|
|
--help : prints help and exits
|
|
--isolates : number of isolates in the session (1 by default)
|
|
--repeat : number of tests to run (1000 by default)
|
|
--time : time limit in seconds (none by default)
|
|
--show-stats : show statistics during session (true by default)
|
|
--true-divergence : only report true divergences (true by default)
|
|
--dart-top : sets DART_TOP explicitly through command line
|
|
--mode1 : m1
|
|
--mode2 : m2, and values one of
|
|
jit-[debug-]ia32 = Dart JIT (ia32)
|
|
jit-[debug-]x64 = Dart JIT (x64)
|
|
jit-[debug-]arm32 = Dart JIT (simarm)
|
|
jit-[debug-]arm64 = Dart JIT (simarm64)
|
|
jit-[debug-]dbc = Dart JIT (simdbc)
|
|
jit-[debug-]dbc64 = Dart JIT (simdbc64)
|
|
aot-[debug-]x64 = Dart AOT (x64)
|
|
aot-[debug-]arm32 = Dart AOT (simarm)
|
|
aot-[debug-]arm64 = Dart AOT (simarm64)
|
|
kbc-int-[debug-]x64 = Dart KBC (interpreted bytecode)
|
|
kbc-mix-[debug-]x64 = Dart KBC (mixed-mode bytecode)
|
|
kbc-cmp-[debug-]x64 = Dart KBC (compiled bytecode)
|
|
djs-x64 = dart2js + Node.JS
|
|
|
|
If no modes are given, a random JIT and/or AOT combination is used.
|
|
|
|
This fuzz testing tool must have access to the top of a Dart SDK
|
|
development tree (DART_TOP) in which all proper binaries have been
|
|
built already (for example, testing jit-ia32 will invoke the binary
|
|
${DART_TOP}/out/ReleaseIA32/dart to start the Dart VM). The DART_TOP
|
|
can be provided through the --dart-top option, as an environment
|
|
variable, or, by default, as the current directory by invoking the
|
|
fuzz testing tool from the Dart SDK top.
|
|
|
|
Background
|
|
==========
|
|
|
|
Although test suites are extremely useful to validate the correctness of a
|
|
system and to ensure that no regressions occur, any test suite is necessarily
|
|
finite in size and scope. Tests typically focus on validating particular
|
|
features by means of code sequences most programmers would expect. Regression
|
|
tests often use slightly less idiomatic code sequences, since they reflect
|
|
problems that were not anticipated originally, but occurred “in the field”.
|
|
Still, any test suite leaves the developer wondering whether undetected bugs
|
|
and flaws still linger in the system.
|
|
|
|
Over the years, fuzz testing has gained popularity as a testing technique for
|
|
discovering such lingering bugs, including bugs that can bring down a system
|
|
in an unexpected way. Fuzzing refers to feeding a large amount of random data
|
|
as input to a system in an attempt to find bugs or make it crash.
|
|
Generation-based fuzz testing constructs random, but properly formatted input
|
|
data. Mutation-based fuzz testing applies small random changes to existing
|
|
inputs in order to detect shortcomings in a system. Profile-guided or
|
|
coverage-guided fuzz testing adds a direction to the way these random changes
|
|
are applied. Multi-layered approaches generate random inputs that are
|
|
subsequently mutated at various stages of execution.
|
|
|
|
The randomness of fuzz testing implies that the size and scope of testing is
|
|
no longer bounded. Every new run can potentially discover bugs and crashes
|
|
that were hereto undetected.
|
|
|
|
Links
|
|
=====
|
|
|
|
* [Dart bugs found with fuzzing](https://github.com/dart-lang/sdk/issues?utf8=%E2%9C%93&q=label%3Adartfuzz+)
|
|
* [DartFuzz](https://github.com/dart-lang/sdk/tree/master/runtime/tools/dartfuzz)
|
|
* [DartLibFuzzer](https://github.com/dart-lang/sdk/tree/master/runtime/vm/libfuzzer)
|
|
* [LibFuzzer](https://llvm.org/docs/LibFuzzer.html)
|