The test suite can be invoked as :
tools/test.py -rflutter --packages=<path_to_packages_dir>/.packages --flutter=<path_to_flutter_engine_workspace>/sky-shell --exclude-suite=pkg
Currently the sky_shell executable does not yet support the proper return
codes:
Success - 0
API error - 253
Compilation error - 254
Unhandled error - 255
Once this is fixed we can start running the tests against our
dart/flutter engine bleeding edge build bot.
R=zra@google.com
Review-Url: https://codereview.chromium.org/2602783002 .
Some of the kernel unit tests can now be run using:
tools/test.py -cdartk -rself_check language co19
This will search the pkg/ folder for files matching *_self_check.dart
and run each program with the compiled output as argument. If there is
no compiler, the test case itself is given as argument. These testers
are always run in batch-mode.
This type of test has no expected output, but is intended to check
itself by testing that certain invariants are not violated while
processing the given data set.
The 'self_check' runtime is not specifically tied to kernel,
although only kernel is using it at the moment.
There is also a new option --skip-compilation which skips the
compiler step. It doesn't interact nicely with the status files,
but can still be useful for a quick offline test.
Current limitations:
- All self-check tests are treated as the same test case. If one fails,
the remaining self-check testers don't run for that input.
- There is no way to run a subset of the self-check tests, or filter
them based on what compiler was used.
- Tests that are expected to fail in the compiler show up as
errors when skipping compilation.
BUG=
R=kustermann@google.com, whesse@google.com
Review-Url: https://codereview.chromium.org/2549793002 .
This adds status files for skipping tests that are not yet strong, and
a status file with the tests that fail with the dartk-vm configuration.
To test:
tools/test.py -cdartk --strong {language,lib,corelib}_strong
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2583453002 .
Every crash of a dartk invocation will cause 1 more re-try. This is independent
of whether the crash was caused by the entire batch runner or just that single
compilation caused an exception.
The unfortunate side-effect of this is that ./tools/test.py will now print
warnings every time it retries the command.
R=vegorov@google.com
Review-Url: https://codereview.chromium.org/2576693002 .
To make it clear in the status file whether a crash (or other issues) are coming
from the dartk compiler or from the vm, this CL introduces new status file
markers which are the normal markers prefixed with Dartk (e.g. DartkCrash).
Furthermore this CL groups common failures
* compiler: dartk & dartkp
* runtime: vm & dart_precompiled
into shared status file blocks, thereby deduplicating failures (which removes around 500 lines).
R=vegorov@google.com
Review URL: https://codereview.chromium.org/2543973002 .
Currently failures on the buildbot will look for example like this:
Short reproduction command (experimental):
python tools/test.py -cdartk -t120 --write-debug-log --write-test-outcome-log --copy-coredumps --exclude-suite pkg language/transitive_private_library_access_test
But developers only would like to see
python tools/test.py -cdartk -t120 language/transitive_private_library_access_test
R=whesse@google.com
Review URL: https://codereview.chromium.org/2532123002 .
This replaces the old --sanity-check flag from dartk. Some files have
been renamed to avoid the wording "sanity check".
Compared to --sanity-check, the following checks have been added:
- variables are not referenced out of scope
- variables are not redeclared
- class type parameters are not referenced from static context
A unit test has been added to check that the verifier rejects certain
invalid ASTs.
BUG=
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2531873002 .
Previously the testing script was just printing interleaved stack traces with
all the other output. Which means it's not associatable with a single test and
is highly confusing!
This CL associates the stack trace information of tests which time out to the
particular test which timed out.
R=whesse@google.com
Review URL: https://codereview.chromium.org/2524733004 .
Previously it was emitted in the directory used for intermediate files, which
is build system specific.
test.py script needs to locate patched_sdk thus its location must be aligned
between build systems.
Use checked in SDK if available and fallback on dart_boostrap if not available to speed up Debug builds
BUG=
R=kustermann@google.com
Review URL: https://codereview.chromium.org/2469683002 .
- bring patched SDK generation scripts and VM patch tweaks that allow VM patch files to be parsed by analyzer front-end;
Patched SDK is an SDK with all VM patches spliced into it. Kernel compiler is based on the analyzer front-end which does
not have any patch files support/model so for it to produce Kernel files that match VM we need to generate a such patched SDKs.
- bring test script modifications that allow to test Kernel pipeline
BUG=
R=asiva@google.com, kmillikin@google.com, whesse@google.com, zra@google.com
Review URL: https://codereview.chromium.org/2434123003 .
- unified kAppAfterRun and kAppJITAfterRun to kAppJIT
- adjusted the test configuration to make dart2app and dart2appjit to mean
the same
- delete GenerateFullSnapshot function as it is not used anymore
R=rmacnak@google.com
Review URL: https://codereview.chromium.org/2429023002 .
For blobs, put all four pieces into a single file with a header describing their offsets.
For dynamic libraries, move the vm isolate and isolate pieces into the dynamic library as additional read-only sections.
Automatically detect if the script argument is an app snapshot and initialize the VM appropriately, similar to automatic detection of script snapshots.
R=asiva@google.com
Review URL: https://codereview.chromium.org/2405393002 .
copying what I see for the cps-ir flag. Turns out we were not passing the flag
when running the html tests.
BUG=
Review URL: https://codereview.chromium.org/2284793002 .
Siva Annamalai