Rationale:
Deprecating DBC cluster runs in fuzzer since the
focus will move away from this mode. Our resources
are better spent on running all other modes.
Change-Id: I6938381fa59c5d40928d891b08725fdf7c54cece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106921
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
This makes it an actual Pub package like most other code inside the SDK
repo. The main goal is to make it easier to write tests for the test
runner itself.
This change:
- Moves all of the code from tools/testing/dart/ over to
pkg/test_runner. Most of it ends up under test_runner/lib/src.
- Move tools/testing/dart/main.dart to
pkg/test_runner/bin/test_runner.dart.
- Move standalone_2/io/test_runner_test.dart to
pkg/test_runner/test/test_runner_test.dart. I don't think it currently
works, but it wasn't being run in its old location either.
- Add test_runner to the analysis-server bot. This ensures the
test_runner package is static error clean.
- Remove standalone_2/io/test_runner_analyze_test.dart which used to
attempt to do the above and is no longer needed.
- Update test.py to look for the test runner at its new location.
- Add test_runner to the repo .packages file and remove the weird
test_dart pseudo-package. (I think this fixes #35279.)
- Remove status file entries for the removed standalone_2 tests.
There are no code changes to the test runner itself aside from fixing
up import paths.
Change-Id: I3d05d50d222b291848fa5a30de2846e803bc81e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105821
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
For this to work properly, we first need the test infrastructure to have support
for the `--output-directory` flag (see base CL)
Change-Id: I75f788d19ad3b4e9523830250c4a1c9de8418cda
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104400
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This adds support for BabelJS to test.py's dart2js compiler
configuration. When --babel is specified directly or through a named
configuration, test.py will run an additional step after dart2js that
post-processes the javascript output by running it through BabelJS with
the specified Babel configuration. BabelJS is added to the DEPS in its
standalone form. d8 is used to run BabelJS standalone to avoid adding
a dependency on NodeJS. d8 can only write to stdout but not to files or
stderr, which makes it necessary to change the test_runner to handle
commands that expect their output to be piped to a file.
Changes:
* Add --babel option to test.py.
* Add babel option to pkg/smith.
* Switch IE11 builder to use babel transformation.
* Fix option list comparison bugs in pkg/smith.
* Change dart2js compiler configuration to generate files using the
test name rather than just "out.js" (update test that relied on this).
* Remove runtime_configuration dependency on test_suite.
* Remove obsolete blocks adding --preview-dart-2 dart2js arguments.
* Make dart2js' compiler configuration more like DDC's.
* Remove createCommand method that is no longer used.
* Remove support for "OtherResources" which was only used for
dart:isolate tests on dart2js and DDC.
* Skip co19_2 tests that are slow to transform with babel.
* Simplify the timeout handling in the test runner with Future.timeout.
Change-Id: I32e4917b2a57ecbe684538e40d744f0101c552a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/90402
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This change adds a disguised test.py which the testing infrastructure does
not recognize, effectively ensuring the test results from the front end
unit tests are not uploaded to the database, preventing them from being
approved. As such those tests will be invisible to the testing
infrastructure and will not show up on dashboards.
Change-Id: Ia28976e7cd94284eee968b215a0023a6db52d9c1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94753
Reviewed-by: William Hesse <whesse@google.com>
This CL includes:
* A dart script that - given a flutter checkout - finds and compiles
tests to ensure we actually can.
* A shell script that checks out flutter, compiles the platform
(to cope with any changes to the kernel format) and runs the above
script.
Change-Id: I02806a375943361c3c111d9aa12a76d745391ca3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98140
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Bytecode configurations should have the same test coverage as default mode.
The following tests are excluded in dartkb mode as there are too many crashes:
- service/* (debugging capabilities are not supported);
- lib_2/mirrors/* (mirrors are not supported);
- vm/cc/* (bytecode modes are not wired up in run_vm_tests).
Change-Id: I11eead94b96f43ab454acd6e6f6861af943c0488
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97847
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Adds:
- dart2aot, a script similar to dart2js which compiles my.dart to my.dart.aot.
- dartaotruntime, a minimal Dart runtime that only runs AOT blobs.
- some extra tooling like gen_kernel and gen_snapshot used by the above.
- build rules for all of the above, including adding it to the full SDK builds.
Bug:https://github.com/dart-lang/sdk/issues/27596
Change-Id: Ic35f832b2b86be959212b8d21cfc5a082da5ced4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97627
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
This change adds a commit queue test that the bytecode benchmarks continue
to work in order to avoid any future unintended breakage.
The new linux-x64-bytecode build is exactly the same as the linux-x64 build
except without the SIMDBC build.
This change also moves the build from the script out to the test matrix,
which let's Goma work correctly, speeding the builder considerably up.
Change-Id: Ifc275c55e405f8dcd7e98a8f15c651602a6dfc4f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98326
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Move FFI tests into a separate test suite.
They never belonged in standalone_2/ since they are not only available in
the standalone VM. Also, we want to have a separate status file.
* Add new "SharedObjects" option to test files to copy needed shared objects
to the Android device for testing.
* Add support to compiler/runtime_configuration.dart for testing JIT-mode on Android.
* Add new configurations and builders to test_matrix.json to test JIT-mode on Android.
* Clean up status file entries for FFI (we didn't need to special-case stress & subtype tests).
Change-Id: Ifb32ef7051754f477d00ecd7a0f9b19ca8a66eae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97334
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This change adds a 'preapprovals' field to approved_results.json, which is
a map from gerrit Change-Id values to a pre-approval record, containing:
* 'from': What the previous approval was.
* 'result': What the new approval is.
* 'matches': Whether the new approval matches the expectation.
* 'expected': The expectation for the new approval.
* 'preapprover': Who did the preapproval.
* 'preapproved_at': When the preapproval occurred.
tools/approve_results.dart now produces pre-approval records instead of
updating the actual approvals. This ensures pre-approvals don't take
effect immediately and fixes the problem of pre-approving one kind of
failure becoming another kind of failure. The script needed to query the
gerrit API in order to determine the Change-Id of changelists. This query
also lets approve_results know what the latest changelist and lifts the
restriction that the patchset must be supplied when pre-approving.
The innards of approve_results has been refactored so it better can handle
selective modification of approve_results.dart.
approve_results now also checks for race conditions with any other
concurrent approval in interactive mode. This should catch most cases where
people leave approve_results running for long. There still is a small race
condition, so there is no check done in the non-interactive mode as the
results would just have been downloaded anyway.
The new tools/bots/apply_preapprovals.dart script will be used by the
recipe. It locates the outstanding pre-approved changelists and checks
the git history for whether they have landed by searching for their
Change-Id. It then applies their pre-approvals in order, warning about any
merge conflicts. Optionally it uploads the pre-approvals. This will work
for the commit queue where the pre-approvals can be applied temporarily,
and in the continuous integration where the pre-approvals become permanent
approvals.
This change allows empty approval records that only contain preapprovals but
with no actual base approval. tools/bots/compare_results.dart has been
updared to handle that case.
This changelist addresses https://github.com/dart-lang/sdk/issues/36279 by
ensuring only verified fields make it into new approved_results.json records
and adds a temporary workaround for 14 days to remove existing needless
existing fields.
Change-Id: I7b3ff59756dc0d28721c6db480782b4ea983b55d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97308
Reviewed-by: William Hesse <whesse@google.com>
This option makes the available named configurations much more discoverable.
Additionally this change expands the available named configurations by
generalizing the operating system and processor architecture patterns
in the named configurations. This change should ensure that nobody is doing
any local testing that isn't covered by a named configuration.
Change-Id: I776105955a86e9f0403ce07a3cdf971e4213646f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96320
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
* Run Debian package creation in a Docker container.
* Upload archives from Docker host (the container lacks credentials).
* Remove BUILDBOT_BUILDERNAME environment variables from test matrix:
They are overwritten in the recipe.
Change-Id: I90d7747e64231bdb745c85030a9cecf256adbaf0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94542
Reviewed-by: William Hesse <whesse@google.com>
It was using the wrong 'expectation' field instead of the correct 'expected'
field. This change uses the right field. However as the matches field has
now been initialized, this change forcefully recomputes it instead of
politely using putIfAbsent. The matches result for the current result will
still take precedence.
Change-Id: I0775c1ff43b8f6691e457b5155ac9335338b209d
Reviewed-on: https://dart-review.googlesource.com/c/95182
Reviewed-by: William Hesse <whesse@google.com>
The expected field can be used to be displayed in the flakiness dashboard.
The matches fields can be used in the flakiness dashboards to decide whether
to include links to logs for a given outcome.
Change-Id: I19454e7cc8fb49871dbb0e7a11478d8888b56662
Reviewed-on: https://dart-review.googlesource.com/c/95024
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Aart Bik