Currently tests in the app_jit-debug configuration are given 16 minutes
timeout. This caused us to run into a global timeout of test.dart (it's
"debug timer" which is set to 10 minutes of inactivity).
Even the slowest tests run in the app_jit-debug configuration within around 1:30 m on the buildbot.
This CL changes the timeout from 16 minutes to 4 minutes.
R=vegorov@google.com
Review-Url: https://codereview.chromium.org/2685303002 .
We are not switching yet, but now everybody can try fasta by doing:
$ dart -DDFE_USE_FASTA=true --dfe=utils/kernel-service.dart hello.dart
Note that when using fasta we expect that patched_sdk contains platform.dill
Kernel binary containing compiled patched sdk. This file can be obtained by
doing:
$ export DART_AOT_SDK=<path-to-patched_sdk>
$ dart pkg/front_end/lib/src/fasta/bin/compile_platform.dart \
${DART_AOT_SDK}/platform.dill
We are also adding --use-fasta to testing script to allow end-to-end testing
of fasta (though platform.dill file needs to be generated manually prior to
running tests):
$ tools/test.py -c dartk -m release -a x64 --nobatch --use-fasta
Current test status:
$ tools/test.py -c dartk -m release -a x64 --nobatch --use-fasta
[27:34 | 100% | +12566 | - 1941]
BUG=
R=ahe@google.com, kustermann@google.com
Review-Url: https://codereview.chromium.org/2684943003 .
The former doesn't seem to tell us anything useful, and is broken. The
latter is no longer needed as a flag without the former. The pkgbuild
suite implies it and other suites don't support it.
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2673133002 .
Uses cdb.exe (part of Windows SDK, comes with depot_tools) to dump stack traces of all threads when a test times out similar
to what we already do on Linux and MacOS.
Note that cdb.exe comes in two version (32- and 64-bit). For proper stacktraces the correct version must be in the PATH.
Fixes#28242.
R=kustermann@google.com
Review-Url: https://codereview.chromium.org/2670763002 .
Change UnexpectedCrashDumpArchiver to generate coredumps file in the current
directory. Each line of this file specifies a core and a binary to be
uploaded in the following format:
test-name,core-file,binary-file
CoreDumpArchiver reads this file and uploads the first 10 crash dumps.
R=kustermann@google.com
BUG=
Review-Url: https://codereview.chromium.org/2645963004 .
Guarded by a GN argument, this CL builds a Fuchsia OS image
that includes Dart tree, including the test suite, packages,
test harness, and sufficient build output to run through the
test suite.
R=asiva@google.com
Review-Url: https://codereview.chromium.org/2547013003 .
- Fix dartanalyzer warnings about unused variables and missing returns
- Remove trailing whitespace
- Add a dummy pubspec.yaml so editors recognize it as a Dart project
R=whesse@google.com
Review-Url: https://codereview.chromium.org/2633213003 .
The test suite can be invoked as :
tools/test.py -rflutter --packages=<path_to_packages_dir>/.packages --flutter=<path_to_flutter_engine_workspace>/sky-shell --exclude-suite=pkg
Currently the sky_shell executable does not yet support the proper return
codes:
Success - 0
API error - 253
Compilation error - 254
Unhandled error - 255
Once this is fixed we can start running the tests against our
dart/flutter engine bleeding edge build bot.
R=zra@google.com
Review-Url: https://codereview.chromium.org/2602783002 .
Some of the kernel unit tests can now be run using:
tools/test.py -cdartk -rself_check language co19
This will search the pkg/ folder for files matching *_self_check.dart
and run each program with the compiled output as argument. If there is
no compiler, the test case itself is given as argument. These testers
are always run in batch-mode.
This type of test has no expected output, but is intended to check
itself by testing that certain invariants are not violated while
processing the given data set.
The 'self_check' runtime is not specifically tied to kernel,
although only kernel is using it at the moment.
There is also a new option --skip-compilation which skips the
compiler step. It doesn't interact nicely with the status files,
but can still be useful for a quick offline test.
Current limitations:
- All self-check tests are treated as the same test case. If one fails,
the remaining self-check testers don't run for that input.
- There is no way to run a subset of the self-check tests, or filter
them based on what compiler was used.
- Tests that are expected to fail in the compiler show up as
errors when skipping compilation.
BUG=
R=kustermann@google.com, whesse@google.com
Review-Url: https://codereview.chromium.org/2549793002 .
This adds status files for skipping tests that are not yet strong, and
a status file with the tests that fail with the dartk-vm configuration.
To test:
tools/test.py -cdartk --strong {language,lib,corelib}_strong
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2583453002 .
Every crash of a dartk invocation will cause 1 more re-try. This is independent
of whether the crash was caused by the entire batch runner or just that single
compilation caused an exception.
The unfortunate side-effect of this is that ./tools/test.py will now print
warnings every time it retries the command.
R=vegorov@google.com
Review-Url: https://codereview.chromium.org/2576693002 .
To make it clear in the status file whether a crash (or other issues) are coming
from the dartk compiler or from the vm, this CL introduces new status file
markers which are the normal markers prefixed with Dartk (e.g. DartkCrash).
Furthermore this CL groups common failures
* compiler: dartk & dartkp
* runtime: vm & dart_precompiled
into shared status file blocks, thereby deduplicating failures (which removes around 500 lines).
R=vegorov@google.com
Review URL: https://codereview.chromium.org/2543973002 .
Currently failures on the buildbot will look for example like this:
Short reproduction command (experimental):
python tools/test.py -cdartk -t120 --write-debug-log --write-test-outcome-log --copy-coredumps --exclude-suite pkg language/transitive_private_library_access_test
But developers only would like to see
python tools/test.py -cdartk -t120 language/transitive_private_library_access_test
R=whesse@google.com
Review URL: https://codereview.chromium.org/2532123002 .
This replaces the old --sanity-check flag from dartk. Some files have
been renamed to avoid the wording "sanity check".
Compared to --sanity-check, the following checks have been added:
- variables are not referenced out of scope
- variables are not redeclared
- class type parameters are not referenced from static context
A unit test has been added to check that the verifier rejects certain
invalid ASTs.
BUG=
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2531873002 .
Previously the testing script was just printing interleaved stack traces with
all the other output. Which means it's not associatable with a single test and
is highly confusing!
This CL associates the stack trace information of tests which time out to the
particular test which timed out.
R=whesse@google.com
Review URL: https://codereview.chromium.org/2524733004 .
Previously it was emitted in the directory used for intermediate files, which
is build system specific.
test.py script needs to locate patched_sdk thus its location must be aligned
between build systems.
Use checked in SDK if available and fallback on dart_boostrap if not available to speed up Debug builds
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org/2469683002 .
Martin Kustermann