- option --no-preview-dart-2 will now result in an error
- change aot-assembly build rule to generate AOT snapshot using Dart 2
- generate coresnapshot using Dart 2 (this snapshot is not used yet, next CL which switch the isolate create code to use this snapshot)
- by pass all Dart1 test runs in the status file
- change the default compiler setting in test.py to use dartk
- have test.py not pick up any configuration for --no-preview-dart-2
Change-Id: Ia136943ebfd0fed0c52683b330745b3e2c7a7ce6
Reviewed-on: https://dart-review.googlesource.com/75820
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
tools/bots/compare_results.dart compares the previous and current test
results in the results.json format and lists the differences, taking the
flakiness data into account.
tools/bots/update_flakiness.dart reads new result.json files and updates the
flakiness data in the flaky.json format file. The updated flakiness data
contains the list of tests that were already known to be flaky, plus any
new tests with multiple different outcomes in the provided results.json
files.
For instance, after running the tests, to find the list of tests that
changed result and needs to be deflaked, excluding tests that are already
known to be flaky:
compare_results.dart \
--flakiness-data flaky.json
--changed \
--passing \
--failing \
previous.json results.json
After the tests needing deflakinghas been run again, the flakiness data can
be updated:
update_flakiness.dart -i flaky.json -o flaky.json results.json more.json
Finally a human readable report can explain what happened, exiting 1 if any
tests started failing in a non-flaky manner:
compare_results.dart \
--flakiness-data flaky.json \
--judgement \
--human \
--verbose \
--changed \
--failing \
--flaky \
previous.json results.json
Bug: https://github.com/dart-lang/sdk/issues/34517
Bug: https://github.com/dart-lang/sdk/issues/34518
Change-Id: I156a8a49b8df09c0aebcb77376b69d365d0aa2ac
Reviewed-on: https://dart-review.googlesource.com/75540
Reviewed-by: William Hesse <whesse@google.com>
This change modifies the test matrix to run with --compiler=dartkb with
--enable_interpreter and --use_bytecode_compiler in sequence.
Change-Id: Ie1be3ffdd97e65ceafb8838fd1b159746ba4c2cb
Reviewed-on: https://dart-review.googlesource.com/75206
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The placeholder executable is replaced by a shell script that invokes
pkg/analyzer_fe_comparison (in a similar way to how
sdk/bin/dartanalyzer invokes pkg/analyzer_cli). This required some
rewriting of pkg/analyzer_fe_comparison to support analysis of a
single test case (as opposed to analysis of a package, which is what
it did before).
It's now possible to run the comparison tool over the test suite in
language_2. There are many failures, which I'll begin addressing in
follow-up CLs.
Change-Id: I52d8f3b53064e8e29028571d416f2573c6cba35d
Reviewed-on: https://dart-review.googlesource.com/74820
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
The "vm" test suite is cauing the kernel bytecode compiler to
OOM on the bot, so this CL removes it from the test matrix for
dartkb. This CL also protects against non-utf8 data coming in
on the error message pipe during process spawning.
Change-Id: Iab110a49a49ea12d21037719716a228776343602
Reviewed-on: https://dart-review.googlesource.com/74490
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
This allows dartdevk to be used in the context of an existing build
system based on Analyzer summary files. This will ease migration of
existing dartdevc users over to dartdevk.
Change-Id: I8413f906e384f3f33c98e450adc0860f6fa7bc53
Reviewed-on: https://dart-review.googlesource.com/65641
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Commit-Queue: Jenny Messerly <jmesserly@google.com>
This CL just adds the compiler=compareAnalyzerCfe option to the test
framework, and updates the status files so that no language_2 failures
are expected when this "compiler" is used. A placeholder executable
has been added to sdk/bin, but it is not wired up yet--that will be
done in a future CL.
In a future CL I will wire up the placeholder executable to the code
in pkg/analyzer_fe_comparison, and fill in the methods in
CompareAnalyzerCfeCommandOutput to process the output accordingly.
Change-Id: I27439de4ac609cb8b003f157a0ff88d181bfbcae
Reviewed-on: https://dart-review.googlesource.com/73561
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
After this CL, the interpreter is included by default in the
JIT VM under the flag --enable-interpreter.
Reland with fix to NativeArgument setup in simulator_arm.cc
Change-Id: Ib9b4df6eb4d997dfbe361188b8a127828c1d9c6f
Reviewed-on: https://dart-review.googlesource.com/74003
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
These will be the values reported to the test result database.
There was previously no value with type Expectation that reflected
the desired semantics of realExpected. The result was processed
using flags like "hasCompileTimeError" and "isNegative" to produce
a processed result. The new realResult field avoids that processing.
Change-Id: Ic59a9bf2c94cc0ccdaa31db3c7f5fde004918004
Reviewed-on: https://dart-review.googlesource.com/72105
Reviewed-by: Jonas Termansen <sortie@google.com>
This was needed back when we thought the front end would be based on
the analyzer, and so we thought its performance characteristics would
depend on those of the analyzer summary mechanism. But since the
front end is not based on the analyzer, and the performance of the
analyzer summary mechanism is performance tested elsewhere, we can
remove this code. Removing it will make it easier to do some re-work
of summaries that is needed for Dart 2.1 support.
Change-Id: I4aa8b5251f26ba7fd59b99d325c66da642d410ed
Reviewed-on: https://dart-review.googlesource.com/71661
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
- Switch the kernel isolate to run in Dart2 mode
- Refactor build rules so bootstrapping of the front end happens
with a prebuilt dart sdk
- Refactor application snapshot build rules so the kernel application
snapshot rule does not get into a cyclic dependency during the
bootstrap stage
Change-Id: Iccca3233f6cb8d6b307b4cf1ca3262f293f794a6
Reviewed-on: https://dart-review.googlesource.com/70841
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This reverts commit 499761f0da.
Reason for revert: crashes when building SDK, red bots, broken Flutter 3xHEAD bot
Original change's description:
> [VM/SDK] Switch kernel isolate to run in Dart 2 mode
>
> - Switch the kernel isolate to run in Dart2 mode
> - Refactor build rules so bootstrapping of the front end happens
> with a prebuilt dart sdk
> - Refactor application snapshot build rules so the kernel application
> snapshot rule does not get into a cyclic dependency during the
> bootstrap stage
>
> Change-Id: I067c5f1165fb93811dcc8e390a8bf381db721e5b
> Reviewed-on: https://dart-review.googlesource.com/70461
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>
TBR=rmacnak@google.com,asiva@google.com
Change-Id: Ibf6d526fb5132b766f063fcf873cde661ae895d7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/70516
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
- Switch the kernel isolate to run in Dart2 mode
- Refactor build rules so bootstrapping of the front end happens
with a prebuilt dart sdk
- Refactor application snapshot build rules so the kernel application
snapshot rule does not get into a cyclic dependency during the
bootstrap stage
Change-Id: I067c5f1165fb93811dcc8e390a8bf381db721e5b
Reviewed-on: https://dart-review.googlesource.com/70461
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Alexander Thomas