This change adds a disguised test.py which the testing infrastructure does
not recognize, effectively ensuring the test results from the front end
unit tests are not uploaded to the database, preventing them from being
approved. As such those tests will be invisible to the testing
infrastructure and will not show up on dashboards.
Change-Id: Ia28976e7cd94284eee968b215a0023a6db52d9c1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94753
Reviewed-by: William Hesse <whesse@google.com>
This CL includes:
* A dart script that - given a flutter checkout - finds and compiles
tests to ensure we actually can.
* A shell script that checks out flutter, compiles the platform
(to cope with any changes to the kernel format) and runs the above
script.
Change-Id: I02806a375943361c3c111d9aa12a76d745391ca3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98140
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Bytecode configurations should have the same test coverage as default mode.
The following tests are excluded in dartkb mode as there are too many crashes:
- service/* (debugging capabilities are not supported);
- lib_2/mirrors/* (mirrors are not supported);
- vm/cc/* (bytecode modes are not wired up in run_vm_tests).
Change-Id: I11eead94b96f43ab454acd6e6f6861af943c0488
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97847
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Adds:
- dart2aot, a script similar to dart2js which compiles my.dart to my.dart.aot.
- dartaotruntime, a minimal Dart runtime that only runs AOT blobs.
- some extra tooling like gen_kernel and gen_snapshot used by the above.
- build rules for all of the above, including adding it to the full SDK builds.
Bug:https://github.com/dart-lang/sdk/issues/27596
Change-Id: Ic35f832b2b86be959212b8d21cfc5a082da5ced4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97627
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
This change adds a commit queue test that the bytecode benchmarks continue
to work in order to avoid any future unintended breakage.
The new linux-x64-bytecode build is exactly the same as the linux-x64 build
except without the SIMDBC build.
This change also moves the build from the script out to the test matrix,
which let's Goma work correctly, speeding the builder considerably up.
Change-Id: Ifc275c55e405f8dcd7e98a8f15c651602a6dfc4f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98326
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Move FFI tests into a separate test suite.
They never belonged in standalone_2/ since they are not only available in
the standalone VM. Also, we want to have a separate status file.
* Add new "SharedObjects" option to test files to copy needed shared objects
to the Android device for testing.
* Add support to compiler/runtime_configuration.dart for testing JIT-mode on Android.
* Add new configurations and builders to test_matrix.json to test JIT-mode on Android.
* Clean up status file entries for FFI (we didn't need to special-case stress & subtype tests).
Change-Id: Ifb32ef7051754f477d00ecd7a0f9b19ca8a66eae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97334
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This change adds a 'preapprovals' field to approved_results.json, which is
a map from gerrit Change-Id values to a pre-approval record, containing:
* 'from': What the previous approval was.
* 'result': What the new approval is.
* 'matches': Whether the new approval matches the expectation.
* 'expected': The expectation for the new approval.
* 'preapprover': Who did the preapproval.
* 'preapproved_at': When the preapproval occurred.
tools/approve_results.dart now produces pre-approval records instead of
updating the actual approvals. This ensures pre-approvals don't take
effect immediately and fixes the problem of pre-approving one kind of
failure becoming another kind of failure. The script needed to query the
gerrit API in order to determine the Change-Id of changelists. This query
also lets approve_results know what the latest changelist and lifts the
restriction that the patchset must be supplied when pre-approving.
The innards of approve_results has been refactored so it better can handle
selective modification of approve_results.dart.
approve_results now also checks for race conditions with any other
concurrent approval in interactive mode. This should catch most cases where
people leave approve_results running for long. There still is a small race
condition, so there is no check done in the non-interactive mode as the
results would just have been downloaded anyway.
The new tools/bots/apply_preapprovals.dart script will be used by the
recipe. It locates the outstanding pre-approved changelists and checks
the git history for whether they have landed by searching for their
Change-Id. It then applies their pre-approvals in order, warning about any
merge conflicts. Optionally it uploads the pre-approvals. This will work
for the commit queue where the pre-approvals can be applied temporarily,
and in the continuous integration where the pre-approvals become permanent
approvals.
This change allows empty approval records that only contain preapprovals but
with no actual base approval. tools/bots/compare_results.dart has been
updared to handle that case.
This changelist addresses https://github.com/dart-lang/sdk/issues/36279 by
ensuring only verified fields make it into new approved_results.json records
and adds a temporary workaround for 14 days to remove existing needless
existing fields.
Change-Id: I7b3ff59756dc0d28721c6db480782b4ea983b55d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97308
Reviewed-by: William Hesse <whesse@google.com>
This option makes the available named configurations much more discoverable.
Additionally this change expands the available named configurations by
generalizing the operating system and processor architecture patterns
in the named configurations. This change should ensure that nobody is doing
any local testing that isn't covered by a named configuration.
Change-Id: I776105955a86e9f0403ce07a3cdf971e4213646f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96320
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
* Run Debian package creation in a Docker container.
* Upload archives from Docker host (the container lacks credentials).
* Remove BUILDBOT_BUILDERNAME environment variables from test matrix:
They are overwritten in the recipe.
Change-Id: I90d7747e64231bdb745c85030a9cecf256adbaf0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94542
Reviewed-by: William Hesse <whesse@google.com>
It was using the wrong 'expectation' field instead of the correct 'expected'
field. This change uses the right field. However as the matches field has
now been initialized, this change forcefully recomputes it instead of
politely using putIfAbsent. The matches result for the current result will
still take precedence.
Change-Id: I0775c1ff43b8f6691e457b5155ac9335338b209d
Reviewed-on: https://dart-review.googlesource.com/c/95182
Reviewed-by: William Hesse <whesse@google.com>
The expected field can be used to be displayed in the flakiness dashboard.
The matches fields can be used in the flakiness dashboards to decide whether
to include links to logs for a given outcome.
Change-Id: I19454e7cc8fb49871dbb0e7a11478d8888b56662
Reviewed-on: https://dart-review.googlesource.com/c/95024
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
This change makes named configurations central in test.dart. Previously
test.dart would attempt to emulate the steps taken by a builder. Instead it
now runs a single named configuration as a single step and invokes test.py
directly without any other arguments from the test steps. This better
matches the behavior of test.py and what the developers expect.
Either a named configuration or a builder must be specified. If both are
provided, then the results for that builder is downloaded and used to
compare with when running the named configuration. Otherwise if the only the
named configuration is provided, then it downloads results from all the
builders using the named configuration. Finally if only the builder is
provided, then the named configuration defaults to the one tested by the
builder. If the builder has multiple named configuration, the user is
asked to clarify which one should be tested.
This change adds support for branches to the test matrix. test.dart needs to
know which builders are on which branch, so it can download the relevant
results, and not look on builders that won't be building the right commit.
Fixes https://github.com/dart-lang/sdk/issues/35873
Change-Id: Ie7b75445b954250493528299a0b45eca4e0bb2e5
Reviewed-on: https://dart-review.googlesource.com/c/92780
Reviewed-by: William Hesse <whesse@google.com>
This change assigns systematic names to the builders for branches, where
those builders end in their branch name as the suffix. This will be used to
identify to which branch the builders belong.
Change-Id: Ifa3be5dbbd45d4f623f70dd36d12426d360b0c30
Reviewed-on: https://dart-review.googlesource.com/c/92741
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
The shards are currently defaulting to use j4 because the docker host
has 4 cores. 6 devices share one host and are therefore trying to run 20
processes competing for the only 4 physical cores. The docker hosts will
soon be upgraded to have 8 cores. j2 will result in 10 processes
competing for those 8 cores which should result in a better balance
between utilization and latency.
Change-Id: Ieb67cb92f17317108d1a1d3da710c58400484a7c
Reviewed-on: https://dart-review.googlesource.com/c/91540
Reviewed-by: William Hesse <whesse@google.com>
Nate Bosch