* Move FFI tests into a separate test suite.
They never belonged in standalone_2/ since they are not only available in
the standalone VM. Also, we want to have a separate status file.
* Add new "SharedObjects" option to test files to copy needed shared objects
to the Android device for testing.
* Add support to compiler/runtime_configuration.dart for testing JIT-mode on Android.
* Add new configurations and builders to test_matrix.json to test JIT-mode on Android.
* Clean up status file entries for FFI (we didn't need to special-case stress & subtype tests).
Change-Id: Ifb32ef7051754f477d00ecd7a0f9b19ca8a66eae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97334
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
All configurations are created with the same named configuration,
if there is a named configuration argument.
Change-Id: I2de34a4a503c34e6b24d3edcf0dd06567ed4f0ed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97327
Reviewed-by: Jonas Termansen <sortie@google.com>
This option makes the available named configurations much more discoverable.
Additionally this change expands the available named configurations by
generalizing the operating system and processor architecture patterns
in the named configurations. This change should ensure that nobody is doing
any local testing that isn't covered by a named configuration.
Change-Id: I776105955a86e9f0403ce07a3cdf971e4213646f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96320
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Previously, messages would contain "Warning:" twice and failed to print
which file the errors came from.
Removed newlines from messages because the logger only prints the first
line.
Fixes#34405
Change-Id: I34d3679a4d769d7c7e2488ca55ffd15f08af9b16
Reviewed-on: https://dart-review.googlesource.com/c/93990
Commit-Queue: Alexander Thomas <athom@google.com>
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
All of DDC's supported platforms have Promises, so we can use them
instead of MutationObservers (web) and timers (node.js).
See issue #20055 (same issue, but for dart2js).
Change-Id: Id635a4a9fa104a2ab19dd20824d209f682f831f9
Reviewed-on: https://dart-review.googlesource.com/c/91765
Reviewed-by: Mark Zhou <markzipan@google.com>
Commit-Queue: Jenny Messerly <jmesserly@google.com>
Specify extra shared options to the compiler and runtime via the
--shared-options= commandline option to test.py. For instance,
--shared-options=--enable-experiment=constant-update-2018 will enable
the front-end constant evaluation experiment for all tests run.
To avoid conflicts with tests explicitly enabling experiments, the
front-end parsing of experimental flags is changed to allow the same
flag to be mentioned several times, as long as the value is the same.
This matches the existing behavior of the analyzer.
Change-Id: I0c79e62b0b23ec7bc3824064dd27ef86b4451f9a
Reviewed-on: https://dart-review.googlesource.com/c/93501
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Previously, this only file names were allowed. However, deflaking may
run on a different machine than determinining which tests flaked. Allow
callers to specify a list of tests instead of a file, simplifies sharded
deflaking considerably.
Change-Id: I710f9aa8a8ef696a6982851106c0e171343dc190
Reviewed-on: https://dart-review.googlesource.com/c/92561
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Support for the flutter runtime (sky_shell) was added to the Dart
test framework in the hope that one could run the flutter engine
through the Dart test suite. We never got around to running these
tests on the build bots, the sky_shell executable has been deprecated
and there are plans in the flutter engine team to have their own
unit test frame work.
See Issue https://github.com/flutter/flutter/issues/9115 for more
details.
Change-Id: I6ee9e8c919721dccecf4202ab0778939aeb79174
Reviewed-on: https://dart-review.googlesource.com/c/91103
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
All multitests expecting errors were just being reported as expecting "fail", not as expecting compile-time
or runtime error. Cleaned up legacy checked-only expectations as a side effect.
Change-Id: I7785db0e094e3af3464700fbc71aac30b8bb4282
Reviewed-on: https://dart-review.googlesource.com/c/90684
Reviewed-by: Jonas Termansen <sortie@google.com>
Auto-Submit: William Hesse <whesse@google.com>
This reverts commit 8044e32fc5.
Reason for revert: The device was fixed, and there is now one device per bot. Blacklisting a single device is no longer the right approach because it effectively destroys a bot.
Original change's description:
> [infra] Blacklist android device id in attempt to make buildbot builder more stable
>
> This particular device sometimes fails with errors such as:
>
> * "protocol error"
> * "protocol error (no status)"
> * "device not found"
>
> Change-Id: Iced97ae8f5e891c43436a3d745004517dc59b42c
> Reviewed-on: https://dart-review.googlesource.com/c/84905
> Reviewed-by: Alexander Thomas <athom@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
TBR=whesse@google.com,vegorov@google.com,kustermann@google.com,athom@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I8422a75d10518ec06de737a0a719f9648279fbf8
Reviewed-on: https://dart-review.googlesource.com/c/89500
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
It's simply an alias for `dartdevc --kernel`, and now that build_runner
is switching to that, there's no reason to keep it around anymore.
Change-Id: I43d30a582314a236bcfa2e3a0b50bb48ddac79f8
Reviewed-on: https://dart-review.googlesource.com/c/88281
Auto-Submit: Jenny Messerly <jmesserly@google.com>
Commit-Queue: Vijay Menon <vsm@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Batch mode crashes were treated as if they exited -10 (SIGUSR1 on Linux),
however these are all Dart unhandled exceptions, and no crash dumps were
produced. Exit 253 instead, which dart2js uses to signify an internal
compiler error, which is considered a crash but inhibits uploading crash
reports.
Change-Id: I3779ec1c511c1bcd31967254d3e0cee9a347a9bd
Reviewed-on: https://dart-review.googlesource.com/c/87440
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
This will result in:
tools/test.py -n foobar
The named configuration "foobar" does not exist. The following configurations are available:
* analyzer-asserts-linux
* analyzer-asserts-mac
* analyzer-asserts-win
* ...
Change-Id: I2fadc2ab3ae6f2b34dd373d84365d44ef27c7b8b
Reviewed-on: https://dart-review.googlesource.com/c/87325
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
dart2js internal compiler errors exit 253 and don't produce a core dump, but
rather dump the relevant debug information as a stacktrace on stdio. This
change disables uploading crashes in that case, which would otherwise fail
because core dumps can't be located for the dart2js process, causing the test
step to fail.
Change-Id: I6154bd205c8cccb9c62d007e6071fea8b4196a96
Reviewed-on: https://dart-review.googlesource.com/c/87075
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
We prevent our crashing tests from hitting timeouts on the bots
by disabling Windows Error Reporting UI via SetErrorMode.
However this also disables builtin crash dump generation functionality
that WER has.
This change moves WER suppression for GP faults under a flag to
make sure that we can collect crash dumps when VM crashes on
user machines.
We also make sure that our exception handler call abort()
instead of calling exit() - because exit would not cause
WER to generate a dump.
Bug: https://github.com/flutter/flutter/issues/22558
Change-Id: I42f3e31cfaaa578f6a040b8f10621e5663cddc09
Reviewed-on: https://dart-review.googlesource.com/c/87061
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
This change renames the --no-status option to --clean-exit, as that better
describes what it does, which is useful. We'll want to add a proper option
--no-status-files in the future that ignores status file entries except skip
and slow, which will be useful for the reproduction commands provided by
test.dart in the transition period until the status files are deleted.
Bug: https://github.com/dart-lang/sdk/issues/35358
Change-Id: Idc22a6c1b109508dadafb8a225cb91ad06287c0b
Reviewed-on: https://dart-review.googlesource.com/c/86565
Reviewed-by: William Hesse <whesse@google.com>
Currently reproduction commands often look like this:
tools/test.py -m release -n dartk-linux-release-x64 language_2/...
This has duplicate information in it, namely "--mode=release".
This CL
a) validates that the implied options are not passed in addition to -n
b) the reproduction commands will no longer print the redundant information.
Change-Id: I1a70ad021d5dcfb7a514cd9bbefbb8d0f77edb0a
Reviewed-on: https://dart-review.googlesource.com/c/85944
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Makes the --keep-generated-files flag an option for the test.py command
only, not part of a named configuration passed in with the -n flag
and defined in tools/bots/test_matrix.json.
Change-Id: If67c2848cf6fc15e4cb7167ce23eeb3eb9bd1684
Reviewed-on: https://dart-review.googlesource.com/c/85705
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: William Hesse <whesse@google.com>
We do not archive core dumps generated by tests that are expected
to crash - so there is no need to generate those dumps.
It seems that generating core dumps is very slow on Mac and causes
timeouts.
Fixes#35275
Remove Timeout expectations from Crashing tests
Change-Id: I36c89fd583a6a94521560544163f02414cbbf41a
Reviewed-on: https://dart-review.googlesource.com/c/85680
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Stevie Strickland