* Move FFI tests into a separate test suite.
They never belonged in standalone_2/ since they are not only available in
the standalone VM. Also, we want to have a separate status file.
* Add new "SharedObjects" option to test files to copy needed shared objects
to the Android device for testing.
* Add support to compiler/runtime_configuration.dart for testing JIT-mode on Android.
* Add new configurations and builders to test_matrix.json to test JIT-mode on Android.
* Clean up status file entries for FFI (we didn't need to special-case stress & subtype tests).
Change-Id: Ifb32ef7051754f477d00ecd7a0f9b19ca8a66eae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97334
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This change adds a 'preapprovals' field to approved_results.json, which is
a map from gerrit Change-Id values to a pre-approval record, containing:
* 'from': What the previous approval was.
* 'result': What the new approval is.
* 'matches': Whether the new approval matches the expectation.
* 'expected': The expectation for the new approval.
* 'preapprover': Who did the preapproval.
* 'preapproved_at': When the preapproval occurred.
tools/approve_results.dart now produces pre-approval records instead of
updating the actual approvals. This ensures pre-approvals don't take
effect immediately and fixes the problem of pre-approving one kind of
failure becoming another kind of failure. The script needed to query the
gerrit API in order to determine the Change-Id of changelists. This query
also lets approve_results know what the latest changelist and lifts the
restriction that the patchset must be supplied when pre-approving.
The innards of approve_results has been refactored so it better can handle
selective modification of approve_results.dart.
approve_results now also checks for race conditions with any other
concurrent approval in interactive mode. This should catch most cases where
people leave approve_results running for long. There still is a small race
condition, so there is no check done in the non-interactive mode as the
results would just have been downloaded anyway.
The new tools/bots/apply_preapprovals.dart script will be used by the
recipe. It locates the outstanding pre-approved changelists and checks
the git history for whether they have landed by searching for their
Change-Id. It then applies their pre-approvals in order, warning about any
merge conflicts. Optionally it uploads the pre-approvals. This will work
for the commit queue where the pre-approvals can be applied temporarily,
and in the continuous integration where the pre-approvals become permanent
approvals.
This change allows empty approval records that only contain preapprovals but
with no actual base approval. tools/bots/compare_results.dart has been
updared to handle that case.
This changelist addresses https://github.com/dart-lang/sdk/issues/36279 by
ensuring only verified fields make it into new approved_results.json records
and adds a temporary workaround for 14 days to remove existing needless
existing fields.
Change-Id: I7b3ff59756dc0d28721c6db480782b4ea983b55d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97308
Reviewed-by: William Hesse <whesse@google.com>
This option makes the available named configurations much more discoverable.
Additionally this change expands the available named configurations by
generalizing the operating system and processor architecture patterns
in the named configurations. This change should ensure that nobody is doing
any local testing that isn't covered by a named configuration.
Change-Id: I776105955a86e9f0403ce07a3cdf971e4213646f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96320
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
* Run Debian package creation in a Docker container.
* Upload archives from Docker host (the container lacks credentials).
* Remove BUILDBOT_BUILDERNAME environment variables from test matrix:
They are overwritten in the recipe.
Change-Id: I90d7747e64231bdb745c85030a9cecf256adbaf0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/94542
Reviewed-by: William Hesse <whesse@google.com>
It was using the wrong 'expectation' field instead of the correct 'expected'
field. This change uses the right field. However as the matches field has
now been initialized, this change forcefully recomputes it instead of
politely using putIfAbsent. The matches result for the current result will
still take precedence.
Change-Id: I0775c1ff43b8f6691e457b5155ac9335338b209d
Reviewed-on: https://dart-review.googlesource.com/c/95182
Reviewed-by: William Hesse <whesse@google.com>
The expected field can be used to be displayed in the flakiness dashboard.
The matches fields can be used in the flakiness dashboards to decide whether
to include links to logs for a given outcome.
Change-Id: I19454e7cc8fb49871dbb0e7a11478d8888b56662
Reviewed-on: https://dart-review.googlesource.com/c/95024
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
This change makes named configurations central in test.dart. Previously
test.dart would attempt to emulate the steps taken by a builder. Instead it
now runs a single named configuration as a single step and invokes test.py
directly without any other arguments from the test steps. This better
matches the behavior of test.py and what the developers expect.
Either a named configuration or a builder must be specified. If both are
provided, then the results for that builder is downloaded and used to
compare with when running the named configuration. Otherwise if the only the
named configuration is provided, then it downloads results from all the
builders using the named configuration. Finally if only the builder is
provided, then the named configuration defaults to the one tested by the
builder. If the builder has multiple named configuration, the user is
asked to clarify which one should be tested.
This change adds support for branches to the test matrix. test.dart needs to
know which builders are on which branch, so it can download the relevant
results, and not look on builders that won't be building the right commit.
Fixes https://github.com/dart-lang/sdk/issues/35873
Change-Id: Ie7b75445b954250493528299a0b45eca4e0bb2e5
Reviewed-on: https://dart-review.googlesource.com/c/92780
Reviewed-by: William Hesse <whesse@google.com>
This change assigns systematic names to the builders for branches, where
those builders end in their branch name as the suffix. This will be used to
identify to which branch the builders belong.
Change-Id: Ifa3be5dbbd45d4f623f70dd36d12426d360b0c30
Reviewed-on: https://dart-review.googlesource.com/c/92741
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
The shards are currently defaulting to use j4 because the docker host
has 4 cores. 6 devices share one host and are therefore trying to run 20
processes competing for the only 4 physical cores. The docker hosts will
soon be upgraded to have 8 cores. j2 will result in 10 processes
competing for those 8 cores which should result in a better balance
between utilization and latency.
Change-Id: Ieb67cb92f17317108d1a1d3da710c58400484a7c
Reviewed-on: https://dart-review.googlesource.com/c/91540
Reviewed-by: William Hesse <whesse@google.com>
Context: we finally removed the old emitter and fast startup is on by default.
This CL removes from the test matrix duplicate steps that now are doing the same
thing and the faststartup builders that are no longer needed.
Change-Id: Id65ea3178bcd7a9554bfa127a09938ca910c57e8
Reviewed-on: https://dart-review.googlesource.com/c/89181
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
The buildbucket API is not always reliable and responds within 30 seconds.
Rather than spuriously failing, try again. This should improve the
reliability of the base branch update bot.
Change-Id: Iade2d78372743b8d843c0dead591c28f8ee8f3ae
Reviewed-on: https://dart-review.googlesource.com/c/88445
Reviewed-by: William Hesse <whesse@google.com>
Ryan Macnak