Commit Graph

603 Commits

Author SHA1 Message Date
Martin Kustermann 8c439c7e84 [VM/AOT] Turn bare instructions on by default
Issue https://github.com/dart-lang/sdk/issues/33274

Change-Id: I44850d5906143a53d6b1872eb729364c3a509f9a
Reviewed-on: https://dart-review.googlesource.com/c/91060
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2019-01-25 10:25:18 +00:00
Alexander Thomas 957e194735 [infra] Switch co19 DEP to cipd
This means ~20000 files less need to be transferred to test shards.

TBR=whesse@google.com

Change-Id: Ic89e5c71e341dfa7872e82c1b1868df19480fd2f
Reviewed-on: https://dart-review.googlesource.com/c/90743
Reviewed-by: Alexander Thomas <athom@google.com>
2019-01-24 07:34:58 +00:00
Jonas Termansen fb7d075cfa [infra] Use third_party/gsutil in the status file free workflow.
Change-Id: I13d68092874121acedec305a186867ee88863806
Reviewed-on: https://dart-review.googlesource.com/c/90463
Reviewed-by: Alexander Thomas <athom@google.com>
2019-01-23 12:52:41 +00:00
Alexander Thomas 770ab5275a [infra] Upgrade gsutil to version 4.34
Change-Id: Id353fa11a8f86415569a505ef5ecd7bf03fb51e1
Reviewed-on: https://dart-review.googlesource.com/c/90487
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-01-22 16:58:58 +00:00
Jonas Termansen 0f8b58f48f [infra] Output logs for flaky tests in compare_results.
Change-Id: I532924a61aa19bb41c8e4467e87c92bac8523cee
Reviewed-on: https://dart-review.googlesource.com/c/90483
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2019-01-22 14:24:19 +00:00
William Hesse 9d25cc93e8 Add build_id for flaky builds to log of flaky tests.
Change-Id: Idb9409cceb46343d3c07270c61a595a9a3cdedaf
Reviewed-on: https://dart-review.googlesource.com/c/89530
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-01-21 11:56:00 +00:00
William Hesse a533753bec Remove --fast-startup flag support from testing scripts and status files
Change-Id: Icfb4aa228c0f13bf9fce25aeb7bacbf1e38497ee
Reviewed-on: https://dart-review.googlesource.com/c/89582
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2019-01-16 11:32:44 +00:00
William Hesse 59300164af Remove test flakiness records with misspelled field
Change-Id: I80f29ebc9aaed33df27fa0ac75d5a38cfd90d1d5
Reviewed-on: https://dart-review.googlesource.com/c/88727
Auto-Submit: William Hesse <whesse@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-01-15 11:31:57 +00:00
Sigmund Cherem 0d6402c1ec Remove faststartup from the test matrix
Context: we finally removed the old emitter and fast startup is on by default.
This CL removes from the test matrix duplicate steps that now are doing the same
thing and the faststartup builders that are no longer needed.

Change-Id: Id65ea3178bcd7a9554bfa127a09938ca910c57e8
Reviewed-on: https://dart-review.googlesource.com/c/89181
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2019-01-14 18:22:47 +00:00
Alexander Thomas b0855ffd35 [infra] Fix testing on Android
* Add adb to vm-kernel fileset.
* Add out directory for Android to fileset.

TBR=whesse@google.com

Change-Id: I95906d6fa5cb06ef53773754dc9c03a1a1bc5146
Reviewed-on: https://dart-review.googlesource.com/c/89240
Reviewed-by: Alexander Thomas <athom@google.com>
2019-01-13 22:45:13 +00:00
Martin Kustermann 736ac8469d Use correct way of enabling blobs on dartk-bare-{simarm,simarm64}
Change-Id: Icce8bda1f186fe01f86514f697adc1f88fc500cd
Reviewed-on: https://dart-review.googlesource.com/c/89120
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-01-11 11:14:59 +00:00
Martin Kustermann d9c5bc131d [VM] Use --use-blobs for dartk-precomp-simarm/simarm64 builders
Change-Id: I1e7b27c586500cc265e55c6c84f2bec971458e6c
Reviewed-on: https://dart-review.googlesource.com/c/89001
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Auto-Submit: Martin Kustermann <kustermann@google.com>
2019-01-11 02:02:29 +00:00
Alexander Thomas 0bb0072f5f [infra] Add vm-kernel-precomp-android builder to the test matrix
Change-Id: Id80e9ef4ce94eb2e79ad215d401f7ee8a3c37cc3
Reviewed-on: https://dart-review.googlesource.com/c/88958
Reviewed-by: William Hesse <whesse@google.com>
2019-01-10 13:07:37 +00:00
Martin Kustermann d21b852b02 Extend bare to have coverage across all AOT configurations
Change-Id: I6a0aa22dbdca9daf85c879defa64859441f35e31
Reviewed-on: https://dart-review.googlesource.com/c/88713
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-01-09 09:53:41 +00:00
William Hesse e2471dd363 Fix spelling of "occurrences" in flaky test results output.
Change-Id: I9028298127d69893d1402080150a5bf24f3867f2
Reviewed-on: https://dart-review.googlesource.com/c/88460
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-01-04 14:44:00 +00:00
Jonas Termansen 5e808aa048 [infra] Add retry logic to find_base_commit.
The buildbucket API is not always reliable and responds within 30 seconds.
Rather than spuriously failing, try again. This should improve the
reliability of the base branch update bot.

Change-Id: Iade2d78372743b8d843c0dead591c28f8ee8f3ae
Reviewed-on: https://dart-review.googlesource.com/c/88445
Reviewed-by: William Hesse <whesse@google.com>
2019-01-04 13:11:35 +00:00
Jonas Termansen 0a0eac5038 [infra] Build Linux ARM architectures in tools/bots/dart_sdk.py.
Change-Id: I0ab9a2c798bf95866f3139b2d3bf25fea5475a17
Reviewed-on: https://dart-review.googlesource.com/c/88327
Reviewed-by: William Hesse <whesse@google.com>
2019-01-03 14:37:35 +00:00
Alexander Aprelev 8bbf240078 [infra] Explicitly pass 'debug' to pub_integration_test step from pkg_linux_debug config.
This is follow-up to https://dart-review.googlesource.com/c/sdk/+/87984 with temporary workaround.

Change-Id: I8077f636608c4e9c223c679ffc03151f7c5b83d8
Reviewed-on: https://dart-review.googlesource.com/c/88040
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2018-12-26 15:53:55 +00:00
Alexander Aprelev 0a0ee50501 [infra] Add debug mode support to pub_integration_test.
This is follow-up to https://dart-review.googlesource.com/c/sdk/+/87921.

Change-Id: I32eca90b5b4f708b9238f1aa475b967fd5c242fd
Reviewed-on: https://dart-review.googlesource.com/c/87984
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-12-21 23:47:23 +00:00
Alexander Aprelev 0f523f2462 [infra] Adds pkg linux debug buildbot
Change-Id: I2327f20f61688abcbb4c430e7bebf4ee45672816
Reviewed-on: https://dart-review.googlesource.com/c/87921
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2018-12-21 18:37:49 +00:00
Martin Kustermann b9170ffb83 Add new "dartkp-bare-linux-release-x64" configuration to test_matrix.json
This configuration will run the AOT compiler/runtime with the new
--use-bare-instructions support.

Change-Id: I029c4098f17e194d14e1b213173f2c6907e69f98
Reviewed-on: https://dart-review.googlesource.com/c/87408
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-12-18 14:14:34 +00:00
Kevin Moore 1337062779 Remove pkg:microlytics
Never used, not touched in 4+ years

Change-Id: Ia1568969b2d07c56b5cbe3258e6a13e10b80813d
Reviewed-on: https://dart-review.googlesource.com/c/86941
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Kevin Moore <kevmoo@google.com>
2018-12-11 17:18:00 +00:00
William Hesse 78b53ff6b2 [testing] Format logs with more distinctive separators
The output of a compare-results run showing logs will enclose
the failing test name in a box, as in:

/=======================================================================================\
| dart2js_extra/no_such_method_test is new and failed (CompileTimeError, expected Pass) |
\=======================================================================================/

--- Command "dart2js" (took 244ms):
DART_CONFIGURATION=ReleaseX64 out/ReleaseX64/dart-sdk/bin/dart2js --generate-code-with-compile-time-errors --test-mode --packages=/b/s/w/ir/cache/builder/sdk/.packages --out=/b/s/w/ir/cache/builder/sdk/out/ReleaseX64/generated_compilations/dart2js-sdk/tests_compiler_dart2js_extra_no_such_method_test/test.js /b/s/w/ir/cache/builder/sdk/tests/compiler/dart2js_extra/no_such_method_test.dart --preview-dart-2

exit code:
1

Change-Id: Ic63fa41381bab320636340c1e7919e22cd424c1d
Reviewed-on: https://dart-review.googlesource.com/c/86925
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-12-11 14:19:27 +00:00
Jonas Termansen 0e08c267c8 [infra] Add Windows support for gsutil.py.
The Windows PATH search finds gsutil.py and attempts to invoke that, which
is not a valid Windows application. Instead invoke it's accompanying batch
file.

Bug: https://github.com/dart-lang/sdk/issues/35367
Change-Id: I396809e18dabdc59664bbda34e0832cc7dd4082f
Reviewed-on: https://dart-review.googlesource.com/c/86767
Reviewed-by: William Hesse <whesse@google.com>
2018-12-11 12:08:51 +00:00
Aart Bik b2d4a95cb0 [dart/fuzzer] Use 100 machines for DartFuzz tests
Rationale:
With 100 machines, 8 isolates each, we get a theoretical
peak of 100,000 fuzzing tests per nightly run (actual
number depends on time-based execution mode run).
Change-Id: I5872f681a636e6b5369f684ceef0f37f2311c8d7
Reviewed-on: https://dart-review.googlesource.com/c/84664
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-12-07 18:05:24 +00:00
Paul Berry 09f2736d9a Remove support analyzer-use-fasta-parser from the test matrix.
The bot was removed in d258ee768908e12d7c0abeba9c00776a21b982a1.

Change-Id: Ice51ef23fa7b26e0a8117701062eca453f0d9166
Reviewed-on: https://dart-review.googlesource.com/c/86501
Auto-Submit: Paul Berry <paulberry@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-12-07 08:52:51 +00:00
Martin Kustermann 8ea5ac12c0 [Gardening] Remove redundant --compiler=dartkb from test_matrix.json
Change-Id: I1bfc4fee0250cd087404f3593df812b7b7d96f79
Reviewed-on: https://dart-review.googlesource.com/c/86382
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Auto-Submit: Martin Kustermann <kustermann@google.com>
2018-12-06 15:37:33 +00:00
William Hesse e4cbeab852 Remove redundant compiler option from dartkb builders
Change-Id: Ie7a518a4c7ba9b42d4392a3d8fc7abb638e26d1c
Reviewed-on: https://dart-review.googlesource.com/c/86354
Reviewed-by: William Hesse <whesse@google.com>
2018-12-06 12:38:18 +00:00
Jonas Termansen afb8906699 [infra] Fix compare_results non-human output being null.
TBR=whesse@google.com

Change-Id: I8a028bac9fac4200c8fe49249f8ba43d4dd8964e
Reviewed-on: https://dart-review.googlesource.com/c/86207
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-12-05 16:19:01 +00:00
Jonas Termansen 1ca3f3d2cb [infra] Fix compare_results test name in human readable mode.
TBR=whesse@google.com

Change-Id: Iab9b756a946283abd800f9a0303440880092e729
Reviewed-on: https://dart-review.googlesource.com/c/86202
Reviewed-by: Jonas Termansen <sortie@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2018-12-05 14:14:10 +00:00
Jonas Termansen e1e3b78e3a [infra] Fix update_flakiness missing current_counter null check.
Change-Id: I6ee8b1266f242ec2251dd869b3c775f9509d5fb2
Reviewed-on: https://dart-review.googlesource.com/c/86180
Reviewed-by: William Hesse <whesse@google.com>
2018-12-05 12:14:47 +00:00
Jonas Termansen 90c7cf254e [infra] Implement update_flakiness forgiveness and statistics.
This change makes update_flakiness count how many times each outcome has
happened. If the same outcome happens more than a 100 times in a row, the
test is forgiven. If that was a mistake, and it happens again, the flakiness
frequency is less than 3.65% with 95% certainty, and the probability of
deflaking not catching it is astronomical.

The number of occurences of each kind of outcome mode is tracked. This lets
us track how probable each flakiness mode is. It also lets us answer
important questions such as how many flakes only happen a single time (not
fault of test) versus multiple times (something to do with the test).

Bug: https://github.com/dart-lang/sdk/issues/35318
Change-Id: Ieccd4afb96ada8053f59fbdc005fd853f0d1f996
Reviewed-on: https://dart-review.googlesource.com/c/85952
Reviewed-by: William Hesse <whesse@google.com>
2018-12-05 11:42:23 +00:00
Paul Berry 670f9ab49b Merge commit '65401da92251a7318e3264b816ce0c4e14701751' into analyzer 2018-12-04 15:38:21 -08:00
Aart Bik 65401da922 [dart/fuzzer] fix bot setup
Rationale:
Nightly showed many failures due to missing dependence.

Bug:
https://github.com/dart-lang/sdk/issues/35319


Change-Id: I775df9870a8fbac65e29a4c0b3aab2867b2a0c8a
Reviewed-on: https://dart-review.googlesource.com/c/86104
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
2018-12-04 22:51:10 +00:00
Paul Berry 0a4873bb35 Merge commit '191a09f540ac0516c4088fb8454c693743be8b3f' into analyzer 2018-12-04 09:41:55 -08:00
William Hesse b35aebde1c Add --logs parameter to compare_results.dart, and output logs
The logs of tests that start failing will be output at the end of stdout.

Change-Id: I93a85c983be278e93aaf7d7447438667c54b5185
Reviewed-on: https://dart-review.googlesource.com/c/85500
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-12-04 16:07:42 +00:00
Jonas Termansen 52183f2afa [infra] Implement 3-way comparison with approved_results in compare_results.
Bug: https://github.com/dart-lang/sdk/issues/35298
Change-Id: Ie5bd1101cae8eddd65ad5aa53c8f2d97c6548b2b
Reviewed-on: https://dart-review.googlesource.com/c/85702
Reviewed-by: William Hesse <whesse@google.com>
2018-12-04 13:46:59 +00:00
Alexander Markov 170d5578c7 [vm/kernel] Move functionality of Fuchsia's compiler.dart into pkg/vm
This includes:
* Selecting front-end target.
* Support for multi-root virtual file system.
* Not linking platform into resulting kernel file.
* Specifying input as URI (instead of file path) on command line.
* Automatically converting input script URI to package URI.
* Writing ninja dependencies file.
* Writing package-split kernel binaries.

After this change Fuchsia's compiler.dart will become a small wrapper
over pkg/vm, sharing most logic and even most command line options
with pkg/vm gen_kernel tool.

Also, this CL attempts to share some pieces of code between frontend
server and gen_kernel.

In addition, seperate bytecode generation for package-split binaries
is implemented (needed for https://dart-review.googlesource.com/c/sdk/+/85469).

Corresponding Fuchsia CL: https://fuchsia-review.googlesource.com/c/topaz/+/229964

Change-Id: I12d7b2f6401357b3c9df2e31bc736af5a9dc5fd2
Reviewed-on: https://dart-review.googlesource.com/c/85721
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-12-04 01:08:03 +00:00
Paul Berry 7af0ba8dbf Merge commit '4088bb1302cebbb6922e1292037bc60127f1f852' into analyzer 2018-12-03 07:19:08 -08:00
William Hesse 7abe999437 Enable uploads from Dart SDK swarming builders
Switch the Dart SDK builders to use the dart_sdk.py script to build
and upload the SDK.

Change-Id: I8d8fc41e8d90d14507d5337bd6dcc573ee09bb6a
Reviewed-on: https://dart-review.googlesource.com/c/85701
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-12-03 11:26:30 +00:00
Paul Berry 5949a3f984 Merge commit '5a54f0eb577f0f4a1708ddbd4f69bf70458ff647' into analyzer 2018-11-27 08:34:40 -08:00
Alexander Thomas 9903d86d50 [infra] Apply flutter patches in flutter-analyze builds as well
Change-Id: Ibc4b4f1267ee533040cf81d940395af1985a3b76
Reviewed-on: https://dart-review.googlesource.com/c/85386
Reviewed-by: Paul Berry <paulberry@google.com>
2018-11-26 18:42:39 +00:00
Paul Berry b14b19e13c Merge commit '9e3d3cf765ba3f761b2507950039bcdd2dd3ece8' into analyzer 2018-11-15 09:17:48 -08:00
Alexander Thomas bc2294c004 [infra] Shard reload builders
Change-Id: I6e1577ec7a8f0d7aec73bbee71d8d3a3a0febdd8
Reviewed-on: https://dart-review.googlesource.com/c/84520
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-11-15 14:01:18 +00:00
Paul Berry 1b5996ae67 Merge commit '30c5b39d4e0e7672d05df6aebba73254dd7ee770' into analyzer 2018-11-14 09:37:06 -08:00
Alexander Thomas 1ade3b624f [infra] Switch cross-arm64 builder to use a single shard
Triggering on arm64 fails because some kitchen CIPD packages
are unavailable on arm64.

TBR=whesse@google.com

Change-Id: I26b05d000dde712ab1b1d55a33be1e4a02f22247
Reviewed-on: https://dart-review.googlesource.com/c/84424
Reviewed-by: Alexander Thomas <athom@google.com>
2018-11-14 15:45:08 +00:00
Alexander Thomas eb4f9290de [infra] Add arm64 builders to the test matrix
TBR=whesse@google.com

Change-Id: If432f9fe2c12ed5a0d7a5af8b15e4daaa13e68f5
Reviewed-on: https://dart-review.googlesource.com/c/84420
Reviewed-by: Alexander Thomas <athom@google.com>
2018-11-14 11:08:00 +00:00
Paul Berry fc2888982c Merge commit 'c92f9dce2357d0da7909c0c5f630ad59882ccf29' into analyzer 2018-11-13 12:35:13 -08:00
Jonas Termansen 927f0937bd [infra] Add test.dart script for local testing.
test.dart locates where the current branch branched off master and compares
the local testing results with the appropriate mainline builder results,
letting you know how the current change compares without the need for status
files.

Bug: https://github.com/dart-lang/sdk/issues/35086
Change-Id: Ib79479b867c5ac131302fea1bdf7effd0422a83a
Reviewed-on: https://dart-review.googlesource.com/c/83281
Reviewed-by: Alexander Thomas <athom@google.com>
2018-11-12 21:51:48 +00:00
Aart Bik c60cff1d33 [dart/fuzzer] Introduce time-based test runs
Rationale:
To maximize CPU utilitization during fuzzing runs,
we now use a time-out on all the individual fuzz
testing runs (rather than repeat based). This way,
the "faster" tests will cover more ground, while
the "slower" straggler tests will not break the test
with unnecessary time-outs at cluster level.
Change-Id: I5f0f9646da3fbd8b361a7ea43444b1896a45b57f
Reviewed-on: https://dart-review.googlesource.com/c/84047
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-11-12 18:47:59 +00:00