The buildbucket API is not always reliable and responds within 30 seconds.
Rather than spuriously failing, try again. This should improve the
reliability of the base branch update bot.
Change-Id: Iade2d78372743b8d843c0dead591c28f8ee8f3ae
Reviewed-on: https://dart-review.googlesource.com/c/88445
Reviewed-by: William Hesse <whesse@google.com>
The output of a compare-results run showing logs will enclose
the failing test name in a box, as in:
/=======================================================================================\
| dart2js_extra/no_such_method_test is new and failed (CompileTimeError, expected Pass) |
\=======================================================================================/
--- Command "dart2js" (took 244ms):
DART_CONFIGURATION=ReleaseX64 out/ReleaseX64/dart-sdk/bin/dart2js --generate-code-with-compile-time-errors --test-mode --packages=/b/s/w/ir/cache/builder/sdk/.packages --out=/b/s/w/ir/cache/builder/sdk/out/ReleaseX64/generated_compilations/dart2js-sdk/tests_compiler_dart2js_extra_no_such_method_test/test.js /b/s/w/ir/cache/builder/sdk/tests/compiler/dart2js_extra/no_such_method_test.dart --preview-dart-2
exit code:
1
Change-Id: Ic63fa41381bab320636340c1e7919e22cd424c1d
Reviewed-on: https://dart-review.googlesource.com/c/86925
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Rationale:
With 100 machines, 8 isolates each, we get a theoretical
peak of 100,000 fuzzing tests per nightly run (actual
number depends on time-based execution mode run).
Change-Id: I5872f681a636e6b5369f684ceef0f37f2311c8d7
Reviewed-on: https://dart-review.googlesource.com/c/84664
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
This change makes update_flakiness count how many times each outcome has
happened. If the same outcome happens more than a 100 times in a row, the
test is forgiven. If that was a mistake, and it happens again, the flakiness
frequency is less than 3.65% with 95% certainty, and the probability of
deflaking not catching it is astronomical.
The number of occurences of each kind of outcome mode is tracked. This lets
us track how probable each flakiness mode is. It also lets us answer
important questions such as how many flakes only happen a single time (not
fault of test) versus multiple times (something to do with the test).
Bug: https://github.com/dart-lang/sdk/issues/35318
Change-Id: Ieccd4afb96ada8053f59fbdc005fd853f0d1f996
Reviewed-on: https://dart-review.googlesource.com/c/85952
Reviewed-by: William Hesse <whesse@google.com>
This includes:
* Selecting front-end target.
* Support for multi-root virtual file system.
* Not linking platform into resulting kernel file.
* Specifying input as URI (instead of file path) on command line.
* Automatically converting input script URI to package URI.
* Writing ninja dependencies file.
* Writing package-split kernel binaries.
After this change Fuchsia's compiler.dart will become a small wrapper
over pkg/vm, sharing most logic and even most command line options
with pkg/vm gen_kernel tool.
Also, this CL attempts to share some pieces of code between frontend
server and gen_kernel.
In addition, seperate bytecode generation for package-split binaries
is implemented (needed for https://dart-review.googlesource.com/c/sdk/+/85469).
Corresponding Fuchsia CL: https://fuchsia-review.googlesource.com/c/topaz/+/229964
Change-Id: I12d7b2f6401357b3c9df2e31bc736af5a9dc5fd2
Reviewed-on: https://dart-review.googlesource.com/c/85721
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Switch the Dart SDK builders to use the dart_sdk.py script to build
and upload the SDK.
Change-Id: I8d8fc41e8d90d14507d5337bd6dcc573ee09bb6a
Reviewed-on: https://dart-review.googlesource.com/c/85701
Reviewed-by: Jonas Termansen <sortie@google.com>
test.dart locates where the current branch branched off master and compares
the local testing results with the appropriate mainline builder results,
letting you know how the current change compares without the need for status
files.
Bug: https://github.com/dart-lang/sdk/issues/35086
Change-Id: Ib79479b867c5ac131302fea1bdf7effd0422a83a
Reviewed-on: https://dart-review.googlesource.com/c/83281
Reviewed-by: Alexander Thomas <athom@google.com>
Rationale:
To maximize CPU utilitization during fuzzing runs,
we now use a time-out on all the individual fuzz
testing runs (rather than repeat based). This way,
the "faster" tests will cover more ground, while
the "slower" straggler tests will not break the test
with unnecessary time-outs at cluster level.
Change-Id: I5f0f9646da3fbd8b361a7ea43444b1896a45b57f
Reviewed-on: https://dart-review.googlesource.com/c/84047
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Rationale:
Now that we are running on multiple shards, worker
timeouts are becoming a bit more frequent due to
(1) file copying overhead
(2) more stragglers due to law of larger numbers.
Change-Id: I36169c0b2033390b1c577f1dbafb1b34c3dd382d
Reviewed-on: https://dart-review.googlesource.com/c/84040
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
The approve_results program downloads the current results, the approved
results, and flakiness information for the specified bots. It presents
tables of the tests that have been fixed and the ones that have broken since
the last approval. Finally, it provides the option to approve the new
changes.
The approve_results program is part of the new workflow and only affects the
greenness of bots enrolled in it.
You can list the available bots by running:
dart tools/approve_results.dart -l
You can approve the results on a bot by running:
dart tools/approve_results.dart -b vm-kernel-linux-release-x64
Globs are supported, e.g. to approve all vm linux bots on ia32 and x64:
dart tools/approve_results.dart -b 'vm-*linux*{ia32,x64}*'
By default the tool is interactive and asks whether the changes should be
approved. Approving the results turns the bots green on the next commit.
It's assumed that bugs have been filed prior to approving the test changes.
The -y option approves the changes non-interactively while the -n option
does a try run and shows what needs to be approved.
This tool is meant to be used early and often to turn the bot green. Emails
are not set about additional failures until the regressions has been
approved.
Bug: https://github.com/dart-lang/sdk/issues/35000
Change-Id: I6479edaa76556aa5ef499509c04b92ae770d1727
Reviewed-on: https://dart-review.googlesource.com/c/82222
Reviewed-by: William Hesse <whesse@google.com>
Martin Kustermann