Commit Graph

5874 Commits

Author SHA1 Message Date
William Hesse 30596c2f03 Output a results.json file containing the expected and actual results of each test
Also outputs a run.json file with information about this run of test.py,
such as the commit hash and the named configuration.

Bug: https://github.com/dart-lang/sdk/issues/34348
Change-Id: Idb4d2a6b892d13314ed81e6163933d23c1b8b609
Reviewed-on: https://dart-review.googlesource.com/72682
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-09-07 15:34:46 +00:00
Alexander Aprelev fa862aa490 [hhh] Fix string comparison in apply.sh.
This is follow-up to https://dart-review.googlesource.com/73686.

Change-Id: Id2e6d16deef5d3655f7e80ea50181d39173069a5
Reviewed-on: https://dart-review.googlesource.com/73842
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-09-07 07:00:59 +00:00
Alexander Aprelev df1afc516e [hhh] Add flutter/flutter patching capability to hhh buildbot.
Change-Id: I6b7d96a335c7a8f130ad0c2388c7fb6ba38e2c83
Reviewed-on: https://dart-review.googlesource.com/73686
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-09-07 06:00:46 +00:00
Alexander Aprelev a1b1b8dc06 [hhh] Patch engine until next dart roll lands
Incompatible change landed https://dart-review.googlesource.com/c/sdk/+/72540.

Change-Id: I618650964ec953ae8f0efe9c45e59ecd356a2840
Reviewed-on: https://dart-review.googlesource.com/73685
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2018-09-06 21:08:04 +00:00
Alexander Thomas c0697ad738 [co19] Roll co19_2 and enable for dart2js
Change-Id: I72b493a65ef5c854b122d76d80549d30141157fd
Reviewed-on: https://dart-review.googlesource.com/71243
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-09-06 09:09:29 +00:00
Alexander Thomas 38c0287019 [infra] Add vm-kernel-checked builder to the test matrix
Change-Id: If83bc9f95cf02f44d4cf88d8076fa490f78e6cd5
Reviewed-on: https://dart-review.googlesource.com/73200
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-09-05 15:31:58 +00:00
Alexander Thomas a6bac2ed29 [infra] Run experimental mac builder less often to conserve capacity
Change-Id: I84e52d66cdf7c24b347614babe584d89abea9aee
Reviewed-on: https://dart-review.googlesource.com/72840
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2018-09-04 12:53:10 +00:00
William Hesse 56957ff171 Do not crash testing scripts if a test requests a bad URI
Bug: https://github.com/dart-lang/sdk/issues/34347
Change-Id: I505479fcb224332a79dc7e77eab723316d18b3f3
Reviewed-on: https://dart-review.googlesource.com/72684
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2018-09-04 08:54:59 +00:00
Alexander Thomas 14dda998dc [infra] Add dartkp-obfuscate builder to the test matrix (#32229)
Change-Id: I866e0ac6a4ce848d31d36667c712b381a5723098
Reviewed-on: https://dart-review.googlesource.com/72600
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-09-03 08:04:21 +00:00
William Hesse 0999a7b75e Remove special handling of _htmltest.html tests.
This removes the special category of html tests which was added in 2014, in cl
https://codereview.chromium.org//695893002

Change-Id: I42d2992e79e7b508465a7fffb4864b545f3af452
Reviewed-on: https://dart-review.googlesource.com/72000
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
2018-08-30 14:37:57 +00:00
William Hesse 4db04c9f56 Add realResult and realExpected fields to test case and command output.
These will be the values reported to the test result database.
There was previously no value with type Expectation that reflected
the desired semantics of realExpected.  The result was processed
using flags like "hasCompileTimeError" and "isNegative" to produce
a processed result.  The new realResult field avoids that processing.

Change-Id: Ic59a9bf2c94cc0ccdaa31db3c7f5fde004918004
Reviewed-on: https://dart-review.googlesource.com/72105
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-08-30 13:53:47 +00:00
William Hesse 3603bfaebd Remove remaining traces of content_shell and DumpRenderTree
Change-Id: Iae15260fa588b101929095c865807efecfb678a2
Reviewed-on: https://dart-review.googlesource.com/71960
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-08-30 08:15:30 +00:00
Paul Berry 9a243b35eb Remove summary-related code from pkg/front_end/tool/perf.dart.
This was needed back when we thought the front end would be based on
the analyzer, and so we thought its performance characteristics would
depend on those of the analyzer summary mechanism.  But since the
front end is not based on the analyzer, and the performance of the
analyzer summary mechanism is performance tested elsewhere, we can
remove this code.  Removing it will make it easier to do some re-work
of summaries that is needed for Dart 2.1 support.

Change-Id: I4aa8b5251f26ba7fd59b99d325c66da642d410ed
Reviewed-on: https://dart-review.googlesource.com/71661
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2018-08-30 00:09:55 +00:00
Devon Carew 0e908e9520 Simplify the language_2_analyzer.status status file.
Change-Id: Iad994ad10262f8ac73d8498fada8ff33d0ee4fd3
Reviewed-on: https://dart-review.googlesource.com/71926
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-08-29 18:31:12 +00:00
Janice Collins f40ff153a9 Enable --use-fasta-parser with the analyzer by default.
Based on Devon's change at:

https://dart-review.googlesource.com/c/sdk/+/67090

Change-Id: I80b7943826e248bf37d9bbbb769330d24b0dd6ae
Reviewed-on: https://dart-review.googlesource.com/70160
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Janice Collins <jcollins@google.com>
2018-08-28 19:29:17 +00:00
Devon Carew 28f902744b Disable the analyzer cfe bots.
Change-Id: I4cbe50cd552347a2be9098f1f583fae4a3a3daaa
Reviewed-on: https://dart-review.googlesource.com/71880
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-08-28 16:25:16 +00:00
asiva 0a29264117 [VM] Remove vm_platform from the dart binary
- Remove vm_platform from the dart binary
- kernel isolate always compiles in strong mode

Change-Id: I4e3973fdaee0bda1c43244ea04673f253eaae249
Reviewed-on: https://dart-review.googlesource.com/65788
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-08-24 15:45:04 +00:00
asiva d1bd887d03 [VM/SDK] Switch kernel isolate to run in Dart 2 mode
- Switch the kernel isolate to run in Dart2 mode
- Refactor build rules so bootstrapping of the front end happens
  with a prebuilt dart sdk
- Refactor application snapshot build rules so the kernel application
  snapshot rule does not get into a cyclic dependency during the
  bootstrap stage

Change-Id: Iccca3233f6cb8d6b307b4cf1ca3262f293f794a6
Reviewed-on: https://dart-review.googlesource.com/70841
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2018-08-21 20:28:06 +00:00
Zach Anderson 51bed8a0ed [test matrix] Add back --compiler=dartkb
This was removed by https://dart-review.googlesource.com/c/sdk/+/69970
but the --compiler flag isn't being picked up automatically. See e.g.:
https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8937641495482850608/+/steps/vm_tests/0/stdout

Change-Id: Ia0bd7d708c8e7f8ada536915b9147423b37bc4cf
Reviewed-on: https://dart-review.googlesource.com/70783
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-08-20 21:45:09 +00:00
asiva 1f5f197e90 [VM] Do not start the kernel isolate when doing an app JIT snapshot training run if a kernel file is specified as the application.
Change-Id: Ib2d57accfb369676d907b0811b8c689dd1d5efbe
Reviewed-on: https://dart-review.googlesource.com/70782
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2018-08-20 21:32:21 +00:00
William Hesse 3aa40c9625 Revert "Disable windows builder on commit queue while windows machines are down"
This reverts commit e08286da9d.

Change-Id: If7724bda467167d61e5c8ff807ccda4b5fbe1dc1
Reviewed-on: https://dart-review.googlesource.com/70518
Reviewed-by: William Hesse <whesse@google.com>
2018-08-20 08:09:44 +00:00
Vyacheslav Egorov 032037681e Revert "[VM/SDK] Switch kernel isolate to run in Dart 2 mode"
This reverts commit 499761f0da.

Reason for revert: crashes when building SDK, red bots, broken Flutter 3xHEAD bot

Original change's description:
> [VM/SDK] Switch kernel isolate to run in Dart 2 mode
> 
> - Switch the kernel isolate to run in Dart2 mode
> - Refactor build rules so bootstrapping of the front end happens
>   with a prebuilt dart sdk
> - Refactor application snapshot build rules so the kernel application
>   snapshot rule does not get into a cyclic dependency during the
>   bootstrap stage
> 
> Change-Id: I067c5f1165fb93811dcc8e390a8bf381db721e5b
> Reviewed-on: https://dart-review.googlesource.com/70461
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=rmacnak@google.com,asiva@google.com

Change-Id: Ibf6d526fb5132b766f063fcf873cde661ae895d7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/70516
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2018-08-19 19:58:32 +00:00
asiva 499761f0da [VM/SDK] Switch kernel isolate to run in Dart 2 mode
- Switch the kernel isolate to run in Dart2 mode
- Refactor build rules so bootstrapping of the front end happens
  with a prebuilt dart sdk
- Refactor application snapshot build rules so the kernel application
  snapshot rule does not get into a cyclic dependency during the
  bootstrap stage

Change-Id: I067c5f1165fb93811dcc8e390a8bf381db721e5b
Reviewed-on: https://dart-review.googlesource.com/70461
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2018-08-18 22:50:09 +00:00
Keerti Parthasarathy d5d7869f76 Update tools/VERSION to 2.1, to indicate the 2.1.x builds.
Change-Id: Iac66b914c9b355b2afb7882ba375eb38794aefb4
Reviewed-on: https://dart-review.googlesource.com/70580
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Keerti Parthasarathy <keertip@google.com>
2018-08-17 19:32:42 +00:00
William Hesse e29176ea0d Add --use-cfe and --analyzer-use-fasta-parser flags to smith:configuration
Use them in test_matrix.json for the analyzer builders.

Bug: https://github.com/dart-lang/sdk/issues/34151
Change-Id: I4abe0bcc8b62ee372df32c0402fad3cd9042cb27
Reviewed-on: https://dart-review.googlesource.com/70282
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2018-08-17 14:42:43 +00:00
William Hesse 09ce852bd3 Remove obsolete field from CQ configuration
Bug: 875204
Change-Id: I70f0236d5a014dad2145ffba50828d4642b808aa
Reviewed-on: https://dart-review.googlesource.com/70504
Reviewed-by: William Hesse <whesse@google.com>
2018-08-17 11:44:30 +00:00
William Hesse e08286da9d Disable windows builder on commit queue while windows machines are down
Change-Id: Ib9482824b144ed70bbb653ec913b29497a73579f
Reviewed-on: https://dart-review.googlesource.com/70503
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-08-17 11:07:24 +00:00
Régis Crelier 2c6ad6e76d [VM interpreter] Disable code optimization when running interpreted tests.
Change-Id: Ie8be83b5223db5c73d0d98ddc03e26d757d1ea29
Reviewed-on: https://dart-review.googlesource.com/70480
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2018-08-17 01:53:26 +00:00
Alexander Thomas a558e71b19 [infra] Re-add --dart2js-batch to dart2js builders in the test matrix
TBR=whesse@google.com

Change-Id: Icb64186f3fe9b6a2b7319334ec7e2b9e169fac66
Reviewed-on: https://dart-review.googlesource.com/70341
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-16 20:32:10 +00:00
Alexander Thomas 54fc71ddca [infra] Fix dart2js-csp-chrome builder
TBR=whesse@google.com

Change-Id: I9da1267d27e017ff4e13b27e823a7f27640c4561
Reviewed-on: https://dart-review.googlesource.com/70240
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-16 07:23:01 +00:00
William Hesse e5dd9c44b4 Remove test.py flags from builders in test_matrix, use configuration instead.
The -n<named configuration> flag is being used instead of the
command-line flags in the test.py builder steps.  Remove the unused flags.

Bug: https://github.com/dart-lang/sdk/issues/34066
Change-Id: I07a451273741618e23f85e900eac040e29c84c0a
Reviewed-on: https://dart-review.googlesource.com/69970
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-15 15:26:42 +00:00
Devon Carew 59a68b5310 Merge the analyzer-use-cfe-linux, analyzer-use-cfe-mac, and analyzer-use-cfe-win bots.
Change-Id: I72c3e4c8154b5950793f8df9bcc3d48a793f73eb
Reviewed-on: https://dart-review.googlesource.com/70142
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-08-15 14:35:11 +00:00
Alexander Thomas 94f80d6ee5 [infra] Remove Dart 1 front-end builder from the test matrix
TBR=whesse@google.com

Change-Id: I55598120c81083cb83caa497d7792bb0438bf5d8
Reviewed-on: https://dart-review.googlesource.com/69964
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2018-08-15 10:29:01 +00:00
Jonas Termansen 23dd9a0f66 Fix 3xhead_flutter_hooks.sh script not being executable.
Change-Id: If4fc09d64afd80881f10875f77f8aeca9d05fb2f
Reviewed-on: https://dart-review.googlesource.com/69963
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Jonas Termansen <sortie@google.com>
2018-08-15 09:50:03 +00:00
William Hesse 14f6fbb457 Fix test_matrix.json for dartkp product mode builder.
Mark a test slow that otherwise times out on dartdevc kernel mode.

Change-Id: Id27683c54982f9a6afd1e41f566dd12aa67040de
Reviewed-on: https://dart-review.googlesource.com/69962
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2018-08-14 17:18:02 +00:00
William Hesse 81d47a84c5 Use the named configuration in test.dart, if it is present
This switches the test scripts over to using the test options
gotten from the -n [configuration_name] option instead of those
options given on the command line.  Command line options that
are part of the named configuration class will be taken from
that object.

Change-Id: I19b29c144a5b1546a75d560c1136a6f06306a01b
Reviewed-on: https://dart-review.googlesource.com/69941
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-14 11:37:05 +00:00
Devon Carew 580f809949 Convert the current analyzer --use-cfe test results to language2 status file entries.
Change-Id: I39a1573b0c0e5c85524cb654a5febea09fb9577a
Reviewed-on: https://dart-review.googlesource.com/69720
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-08-13 19:46:56 +00:00
William Hesse d8e01be7d5 Update named configurations to agree with builder flags
Change-Id: I437c741ebeb684ce1660b3fcd80ab2aa756e501b
Reviewed-on: https://dart-review.googlesource.com/68882
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-13 12:09:47 +00:00
Alexander Thomas ba10df6611 [infra] Switch pkg bots to Dart 2
* Update named configs for unit tests to Dart 2.
* Bump json_rpc_2 in front_end package.
* Use named configurations in unit tests.
* Fix typing issues.

Change-Id: Id3b393a0d2abe92c8eaec10fa8057fb35d18a4c6
Reviewed-on: https://dart-review.googlesource.com/67341
Reviewed-by: Jens Johansen <jensj@google.com>
2018-08-13 07:04:06 +00:00
William Hesse be7022f6d9 Remove named configuration template expansion code from test.py.
Named configuration templates are expanded by the recipe before being
passed to test.py.

Change-Id: I164df7acf0a0aff76f050b871101590efae51954
Reviewed-on: https://dart-review.googlesource.com/69300
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2018-08-10 13:12:49 +00:00
Vyacheslav Egorov bde6bbca96 [kernel] Move VmTarget and FlutterTarget to package:vm
Similar to how dart2js keeps its own target in package compiler.

This allows VmTarget to use package vm specific transformations and metadata.

Change-Id: I41dd2ae241b828224fb2c9a51e6ad5073b6fdea8
Reviewed-on: https://dart-review.googlesource.com/69160
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
2018-08-10 12:55:57 +00:00
Devon Carew be6309690f Analyze pkg/dart2js_tools on the bots.
Change-Id: I727012f8584a4d92d5333da303147a1c947d4fb6
Reviewed-on: https://dart-review.googlesource.com/69247
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
2018-08-10 06:04:10 +00:00
Alexander Thomas af02ccae83 [infra] Fix filesets for new vm-kernel-precomp builders
TBR=whesse@google.com

Change-Id: I81a836d532a355db5e3a74a244af71605e0c04c3
Reviewed-on: https://dart-review.googlesource.com/69102
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-09 11:50:36 +00:00
William Hesse 8e2f28e264 Update homebrew to drop the @2 tab and fix for 2.0
Change-Id: I1ece391c9c7cdcec9fd0662841bdbdaee7a7a373
Reviewed-on: https://dart-review.googlesource.com/69101
Reviewed-by: Alexander Thomas <athom@google.com>
2018-08-09 11:49:32 +00:00
Alexander Thomas 7eda513405 [infra] Add new vm-kernel-precomp builders to the test matrix
TBR=whesse@google.com

Change-Id: Ia88f17cee0fb1f69d4bf17eb35dbdaa606aee71b
Reviewed-on: https://dart-review.googlesource.com/68942
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
2018-08-08 17:19:27 +00:00
Samir Jindel de408202a5 [vm/precomp] Take 4 for procedure- and class-entrypoints.
This reverts commit 08f59e5de3.

There are no changes to take 3 except fixing merge conflicts.

Change-Id: I45d5a16274c80f469827c19f4e4f4ca724e85a67
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-release-x64-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/68363
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2018-08-07 13:54:27 +00:00
William Hesse 0986b2bfa2 Clean up test.py named configurations code
Drops the debug printing of configurations, and the checking
that the named configurations agree with the flags passed.

Change-Id: Id19ca002668a757d44017873a5d0c8bb06472564
Reviewed-on: https://dart-review.googlesource.com/68680
Reviewed-by: William Hesse <whesse@google.com>
2018-08-07 13:53:37 +00:00
Terry Lucas d9a07b09ff Fixed createObjectStore to keyPath.
TBR=vsm@google.com

Change-Id: I709e2741adf1dc222dd94e5052360a35cf21d917
Reviewed-on: https://dart-review.googlesource.com/68480
Reviewed-by: Terry Lucas <terry@google.com>
Commit-Queue: Terry Lucas <terry@google.com>
2018-08-06 20:52:35 +00:00
William Hesse e0f6fdfff1 Refactor testing:TestConfiguration to use smith:Configuration by inclusion.
Change-Id: I5c116ad082a24c25a07b9ceb6aaf8c9cbe3f11e3
Reviewed-on: https://dart-review.googlesource.com/68361
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2018-08-06 16:19:08 +00:00
Devon Carew 42b02e02e7 Add more test steps to the analyzer --use-cfe bots.
Change-Id: Ifb756a9faf4c57bd8426b8d94a88e69d9cb890a0
Reviewed-on: https://dart-review.googlesource.com/68421
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2018-08-04 06:35:53 +00:00