Commit Graph

511 Commits

Author SHA1 Message Date
William Hesse 0fd93a5c6e Test.dart exits with non-zero exit code if named configuration disagrees with flags.
Change-Id: I05e869c3f37aa786cd39185b51dda627133aa5e9
Reviewed-on: https://dart-review.googlesource.com/67805
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-01 16:01:16 +00:00
William Hesse f10b4005cd Code to load a named configuration and check it against the existing options.
First stab at loading named configuration, plus temporary test code

Change-Id: I5835fe34a024a6052021c8189629a1578bed2355
Reviewed-on: https://dart-review.googlesource.com/63880
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2018-08-01 12:55:13 +00:00
William Hesse 7413644de8 Change builder names from -ff- to -firefox-
Change-Id: I69317cb92364de2b38f7967cb3ce9e3b307c7678
Reviewed-on: https://dart-review.googlesource.com/67780
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-01 12:28:13 +00:00
William Hesse cf77a3b00f [Test.dart] Use Compiler, Runtime, etc. classes from pkg:smith.
Change-Id: I1251c94e87fe6953f7604b5ef8f6ae5c49ab36d3
Reviewed-on: https://dart-review.googlesource.com/67522
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-01 09:45:17 +00:00
Danny Tuppeny 3dc201e168 Update ChangeBuilder to not include empty edits (Take 2)
Combination of Brian's change:
https://dart-review.googlesource.com/c/sdk/+/63901

and Devon's change:
https://dart-review.googlesource.com/c/sdk/+/66405

Change-Id: I57e5d1f7d3a653f225aba2d690190fe1ff18a5b1
Reviewed-on: https://dart-review.googlesource.com/67344
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Danny Tuppeny <dantup@google.com>
2018-07-31 13:44:06 +00:00
William Hesse 21db860271 Remove non-strong dart2js builders from test_matrix.json
Change-Id: I76c4a1f0df8f6e8f6ddd03fd35c9a069845687ef
Reviewed-on: https://dart-review.googlesource.com/67346
Reviewed-by: Sigmund Cherem <sigmund@google.com>
2018-07-30 17:47:44 +00:00
Alexander Thomas 3665cad81c [infra] Shard analyzer unit tests
TBR=whesse@google.com

Change-Id: I8616b3674258cb4406ace7bd81d4a91df124ff83
Reviewed-on: https://dart-review.googlesource.com/66960
Reviewed-by: Alexander Thomas <athom@google.com>
2018-07-27 08:24:12 +00:00
Zach Anderson 1336802dee [test] Adds a KBC configuration to the test matrix
This is to run on FYI

Change-Id: Ic62e858853a3ccb67d844ec454fd1819aa35b87d
Reviewed-on: https://dart-review.googlesource.com/65780
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-07-26 22:05:06 +00:00
Devon Carew 11784f6ef9 Run the dartanalyzer with --use-cfe.
Change-Id: Ib87c04d3469d0546ebbc6a5c903d5f022e9fb224
Reviewed-on: https://dart-review.googlesource.com/66342
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-07-24 14:07:33 +00:00
Devon Carew c4d92dc5cf Analyze the observatory on the bots.
Change-Id: I5e5992310b3886c95cfb0e7b13f177e09e721541
Reviewed-on: https://dart-review.googlesource.com/66182
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-07-23 18:15:49 +00:00
Devon Carew 188ef5e3d5 Analyze all the pkg/ code on the bots.
Bug: https://github.com/dart-lang/sdk/issues/29318
Change-Id: Iccb16ddbccbb43fa7a12eb818c2dde3a75700b97
Reviewed-on: https://dart-review.googlesource.com/64663
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-07-12 16:35:03 +00:00
William Hesse 6b01ba7112 Update status for vm --checked (enable asserts) tests.
Remove accidental legacy test section from front_end bots.
Remove no-op --strong arg from test matrix.

Change-Id: Icafc7eeb3c25f5816596affeaf8f75a7328fb907
Reviewed-on: https://dart-review.googlesource.com/64520
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-07-12 09:31:10 +00:00
Sergey G. Grekhov 54f66eebf2 [co19] co19 roll: DEPS and test_matrix update
Cq-Include-Trybots:luci.dart.try:front-end-mac-release-x64;luci.dart.try:front-end-win-release-x6;luci.dart.try:analyzer-mac-release;luci.dart.try:analyzer-win-release

Change-Id: I37f3b78180133c31f6267b76c3eeb16f8c00325a
Reviewed-on: https://dart-review.googlesource.com/63623
Reviewed-by: Erik Ernst <eernst@google.com>
2018-07-10 11:07:36 +00:00
Devon Carew 550f294dfc Move some analysis to analyzer-analysis-server-linux and make analyzer-analysis-server-linux a try bot.
Change-Id: Iabac78eb7bdbb3b1bc3ff83ae4ba241af1c8a6ec
Reviewed-on: https://dart-review.googlesource.com/64020
Reviewed-by: William Hesse <whesse@google.com>
2018-07-09 20:18:20 +00:00
Sigmund Cherem c1f6c9d8c5 Fix dart2js-hostassert configuration and enable it on the CQ
TBR=athom@google.com,whesse@google.com

Change-Id: I1dec1386b823bc72bc8f482c7400a129a3037b4a
Reviewed-on: https://dart-review.googlesource.com/63704
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2018-07-07 00:15:37 +00:00
Devon Carew 1a08800dd6 Run analyzer --use-fasta-parser on the language2 tests; update status file.
Change-Id: I35ca9dceb211a30f7e211177956e3e62575f7e57
Reviewed-on: https://dart-review.googlesource.com/63686
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-07-06 14:20:26 +00:00
William Hesse 1400cf66e7 Add named configurations to test matrix.
Change-Id: I259cd854a6ceb2ab2c2813184bfaae1fbc7b3f4e
Reviewed-on: https://dart-review.googlesource.com/62140
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2018-07-04 13:57:19 +00:00
Alexander Thomas 9871ad23c9 [infra] Move fasta legacy mode SDK tests to their own builder
* This will take them of the CQ and lower CQ cycle time by ~15mins.
* The new builder will be enabled by:
https://dart-review.googlesource.com/c/sdk/+/63627

Change-Id: I9318f73a670aee1c67ae9ed19a1bc0a100c14411
Reviewed-on: https://dart-review.googlesource.com/63626
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-07-04 11:08:48 +00:00
Devon Carew 6e8332bf77 Remove older analyzer bots.
Change-Id: I375f6a3f76bdb1fd8e9000e5edbc6bf21a148490
Reviewed-on: https://dart-review.googlesource.com/63643
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-07-04 03:43:07 +00:00
Sigmund Cherem 7007137dd0 Adjustments to dart2js test matrix.
* uses 6 shards for try bots like the recipes used to do
* add a strong-hostasserts configuration. Currently the test.py flag is called
  --host-checked, but all it does is enable assertions (no more checked mode in
  Dart2)

Change-Id: I50c565bb210fbb98334b7d84556c3ab98c38b22a
Reviewed-on: https://dart-review.googlesource.com/63682
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
2018-07-03 20:30:56 +00:00
Devon Carew 46bfa7297e Have the regular analyzer bots test in dart 2 mode.
Change-Id: I91bf09600d0e5f099737be0363228cad6d0b5329
Reviewed-on: https://dart-review.googlesource.com/63642
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-07-03 19:12:36 +00:00
Sigmund Cherem aa009339b3 Update test matrix for dart2js
After reviewing all recipes, we needed a few changes to bring the test matrix to parity:
- we needed to add strong-mode coverage (mostly in d8 at this time)
- we needed to add fast-startup coverage on all browsers

This however includes a bit more coverage than the existing recipes:
- strong-mode coverage in d8 now includes all suites, not just language_2 and corelib_2
- legacy coverage in ie11 includes all suites, not just html and co19


Next steps:
- move the remaining dart2js bots to the test matrix using the existing and new builders
- fix .status files on builders that include more suites (d8-strong + ie11)
- fix .status files on new Dart 2.0 builders
- add new Dart 2.0 builders to the waterfall
- eventually remove Dart1 builders.

Change-Id: Iec0a7fef7176cc81d3ced5e34c8dd63d33b7079e
Reviewed-on: https://dart-review.googlesource.com/61086
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-07-03 18:32:53 +00:00
William Hesse c407e83d0e Delete duplicate key "configurations" in test_matrix.json
The builder configurations have been moved to the "builder_configurations" key,
and the "configurations" key will be used for test configurations.

Change-Id: Ic61f1646378a13526dc59b713f272cea51d9781b
Reviewed-on: https://dart-review.googlesource.com/63580
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2018-07-03 12:02:31 +00:00
William Hesse 45a0ced146 Duplicate the "configurations" section of test_matrix, under the new name "builder_configurations".
The key "configurations" will be used for named configurations, after
the recipes are updated to use the new "builder_configurations" key.

Change-Id: Ifec6803f3aaee83cbc48c591ac306af3ef951933
Reviewed-on: https://dart-review.googlesource.com/62147
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-07-02 15:57:43 +00:00
William Hesse 1bdef38057 Add pinned firefox version to the general settings in test_matrix.json
Change-Id: I376126ed28a2c90e3a164f0e6c36b9e2004575ef
Reviewed-on: https://dart-review.googlesource.com/62809
Reviewed-by: Alexander Thomas <athom@google.com>
2018-07-02 12:30:08 +00:00
Devon Carew 4af2a9d914 Some normalization of the test_matrix.json file.
Change-Id: I146877b60a5711487c4aceda55e07055a935ee24
Reviewed-on: https://dart-review.googlesource.com/63200
Reviewed-by: Dan Rubel <danrubel@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-06-29 16:40:51 +00:00
Alexander Thomas 583a81ade3 [infra] Make test.py's --strong option a no-op to prepare for removing it
Remove --preview-dart-2 option.


Change-Id: I4ca07e00391ee9a9f7f454d0b56e82898c38250c
Reviewed-on: https://dart-review.googlesource.com/62601
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2018-06-28 09:39:03 +00:00
Ryan Macnak fa7c38dec5 [vm, standalone] Do not copy when loading kernel.
In a topaz release build with JIT apps,
mondrian(flutter_runner):      PSS 44.4M -> 35.8M (-19.4%)
ledger_dashboard(dart_runner): PSS 15.2M -> 12.0M (-21.1%)

Change-Id: I501c24de56a5a7c65a2f03b87709cb09bb95d465
Reviewed-on: https://dart-review.googlesource.com/60202
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-06-21 23:27:53 +00:00
Devon Carew 7d9ac9dca5 Add a --use-fasta-parse flag to test.py; add a 'analyzer-use-fasta-parser' buildbot configuration.
Change-Id: I802df6b5fbfe963a466ce49f29d2bb0c84f14d25
Reviewed-on: https://dart-review.googlesource.com/60810
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Dan Rubel <danrubel@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-06-20 15:10:54 +00:00
asiva bc7220a4fd [VM] Turn dart2 on by default on the command line VM
- make Dart2 the default option for the command line VM
- add option --no-preview-dart-2 as a fallback option to run dart1
- change test scripts to use the executable dart for testing dart2 mode instead of
pkg/vm/tool/dart2
- adjust numerous build and test configurations

Change-Id: Id813fa5b71a89c7ec9335d3f6e83cfc9f35f86e7
Reviewed-on: https://dart-review.googlesource.com/58240
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
2018-06-18 22:31:32 +00:00
Jenny Messerly 1ef4399df0 Run dartfmt --fix for dart2 on pkg/dev_compiler
This uses optional new/const and `=` in named argument defaults.

All changes are automated, except for:

- utils/dartdevc/BUILD.gn: run DDC build scripts with --preview-dart-2
- pkg/dev_compiler/tool/patch_sdk.dart: add a TODO that Analyzer doesn't
  supporting implicit const in libraries.dart
- pkg/dev_compiler/tool/input_sdk/libraries.dart: was not formatted due
  to the aforementioned Analyzer bug
- tools/bots/test_matrix.json: run DDC sourcemap suite in Dart 2 mode
- pkg/pkg.status: skip pkg/dev_compiler if running in Dart 1 mode

Change-Id: I9b80ccba0c2cc7b66efc662a0b16562e3660aee3
Reviewed-on: https://dart-review.googlesource.com/60402
Commit-Queue: Jenny Messerly <jmesserly@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2018-06-15 00:28:13 +00:00
Alexander Thomas f3ce474673 [infra] Remove obsolete analyzer test configurations
The --preview-dart-2 flag flip makes it unecessary to:
* Test Dart 2 configs in host-checked mode.
* Test --checked and --strong together.
* Test --preview-dart-2 and --strong separately.

Change-Id: I1738d351c866fb95fd9e1249f5f4ffb88169e106
Reviewed-on: https://dart-review.googlesource.com/59900
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2018-06-13 04:17:16 +00:00
Alexander Thomas 305c166154 [infra] Add --no-preview-dart-2 to all Dart 1 configurations in the test matrix
Change-Id: Ib51adc2587c573598ceddd343cb48d0f8136a71b
Reviewed-on: https://dart-review.googlesource.com/59801
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-06-11 21:36:15 +00:00
Alexander Thomas cf319e484a [infra] Remove checked tests from VM product builders in test matrix
Change-Id: I32a68f18eaad0d999d1c3f51795da5f50ab84e50
Reviewed-on: https://dart-review.googlesource.com/59081
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-06-07 12:23:53 +00:00
Alexander Thomas 94091b10f2 [infra] Use the correct runtime for dartjs-csp-minified builder in the test matrix
TBR=whesse@google.com

Change-Id: I925f16ef8c143a743f5ca4e12dc15bd21098834f
Reviewed-on: https://dart-review.googlesource.com/58861
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-06-06 19:40:34 +00:00
Alexander Thomas ff8100d1e3 [infra] Add dartk reload/rollback configurations to test matrix (#33126)
Change-Id: Iedff74fbdfd04165755f2609333742f62e48f18f
Reviewed-on: https://dart-review.googlesource.com/56114
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-05-31 10:48:23 +00:00
Alexander Thomas d28c5499fc [infra] Add support for app_jitk compiler to test.py (#33126)
* Add app_jitk configuration to test matrix.
* Skip it on non-Dart 2 test suites.

Change-Id: I05f75dbbd33d34a4acf4df1fe63104933d39d2fe
Reviewed-on: https://dart-review.googlesource.com/56113
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-05-28 11:54:56 +00:00
Alexander Thomas 96cfc632bb [infra] Use a less specific browser version that works across platforms
TBR=whesse@google.com

Change-Id: Ieff493c187dc5da5dbee49887454cf25e1dc6eef
Reviewed-on: https://dart-review.googlesource.com/56115
Reviewed-by: Alexander Thomas <athom@google.com>
2018-05-22 21:11:03 +00:00
Alexander Thomas 2a8eb8089d [infra] Specify chrome version used by builds running on the test matrix
Prepares the test matrix for the corresponding recipe change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/1064632

Change-Id: I21d380f6ad452e3eca6e6f20133297d31347872c
Reviewed-on: https://dart-review.googlesource.com/56109
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-05-22 16:17:34 +00:00
Alexander Thomas 4d22c0de58 [infra] Improve cycle time of dart2js builders
* Remove co19_2 from file sets for now.
* Skip extremely slow test.
* Remove redundant build target from test matrix.

#33150

Change-Id: Ib17d2e7438dc2ab473e751f1d79c617cbc26275d
Reviewed-on: https://dart-review.googlesource.com/55661
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-05-17 10:21:23 +00:00
Alexander Thomas 243f3e5333 [infra] Add basic pub integration test
Closes #32656

Change-Id: Id3646e7b9567e6f60e449a90c355d3e535ac2a6a
Reviewed-on: https://dart-review.googlesource.com/51661
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-05-15 07:09:58 +00:00
William Hesse 006fda374b Update test matrix with correct path to dart on Mac OS
Change-Id: Ie50148e5519bd4695d547727bb2fdaf3263f01d1
Reviewed-on: https://dart-review.googlesource.com/49641
Reviewed-by: Alexander Thomas <athom@google.com>
2018-04-05 11:19:12 +00:00
William Hesse 1c913801a6 Add third_party packages to dart2js testing shards
The http_server in the test.py scripts uses package_resolver, which
uses 10 packages from third_party/pkg and third_party/pkg_tested.

Change-Id: I1e9ca660b9bd0a6270bcc8e4121da6a3bf9713bd
Reviewed-on: https://dart-review.googlesource.com/47781
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-03-23 12:56:06 +00:00
Zach Anderson aac5d04198 [infra] Add optimization_counter_threshold builder tag to the test matrix
related #32619

Change-Id: I0fc20f21cf3a6e231e0f3d962f139fd072acf3ab
Reviewed-on: https://dart-review.googlesource.com/47540
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-03-21 18:39:12 +00:00
Alexander Thomas c8d044e27d [infra] Remove non-strong mode vm kernel tests steps from test matrix
https://github.com/dart-lang/sdk/issues/32499

TBR=whesse@google.com

Change-Id: I161b13bff4d6b4464d83eb39da9cb8e6d4051374
Reviewed-on: https://dart-review.googlesource.com/46142
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-03-15 16:53:06 +00:00
Alexander Thomas 5c90b1fd98 [infra] Add vm-kernel-legacy builder to the test matrix
https://github.com/dart-lang/sdk/issues/32499

TBR=whesse@google.com

Change-Id: Ia9829fc106162b161e9e31bc918feebc691b3621
Reviewed-on: https://dart-review.googlesource.com/46742
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-03-15 14:31:13 +00:00
Alexander Thomas 019dae1904 [infra] Remove front end unit tests from other builders
The new front end builders will continue to run them.

Change-Id: Iffd5af76bd3a082256ea4fb146981737462938f9
Reviewed-on: https://dart-review.googlesource.com/46042
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Peter von der Ahé <ahe@google.com>
2018-03-13 12:40:39 +00:00
Paul Berry 440b58d644 Start running analyzer tests with --preview-dart-2.
The changes to the CQ bot should take effect immediately.  The changes
to the other bots will take effect once those bots are switched over
to use the test matrix.

Change-Id: Ibfe59817a11f730fd1847cdc60b3244b4a71a480
Reviewed-on: https://dart-review.googlesource.com/45300
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
2018-03-07 15:04:33 +00:00
Alexander Thomas d0eb2d63e8 [infra] Add front-end builder to test matrix
#32123

Change-Id: I996ed57edc78f46aac6fa65a45e07500ad3758a0
Reviewed-on: https://dart-review.googlesource.com/44160
Reviewed-by: Peter von der Ahé <ahe@google.com>
2018-03-05 14:53:03 +00:00
Alexander Thomas 926ef7d742 [infra] Remove dart2js jsshell builder from the test matrix
Change-Id: I850f0d1d75e030da71593e56292fe986e15cc846
Reviewed-on: https://dart-review.googlesource.com/44461
Reviewed-by: William Hesse <whesse@google.com>
2018-03-01 11:56:39 +00:00