This fixes an issue introduced when DDS testing support was added to the
test suite. Previously, tests with multiple VMOptions lines would get
the same name for both DDS and service variants. This caused incorrect
behavior in the results processing that broke deflaking, but also
caused one of the results to be ignored by the infrastructure.
Also adds some tests for VMOptions handling in test_suite.dart.
Fixes https://github.com/dart-lang/sdk/issues/43768
Change-Id: I3c0f9cbc1807fe814aed5ecb7531ef4289e95683
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166858
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
The arguments were missing for some dartkp subcommands and for AppJit.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-debug-x64-try
Change-Id: I467b10eea1187d44b978170b12a0b2b5927f5b1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164380
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
test.py invocations that specify a null safety mode will now work
without tool specific options.
For instance, `test.py --nnbd=strong -cdartkp -mrelease` will now add
the `non-nullable` experiment flag as well as the pass
`--sound-null-safety` to the relevant subcommands.
Fixes https://github.com/dart-lang/sdk/issues/42370
Change-Id: I1ae5bf569cc77ed6f26411c171c2cafe0a12eaff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164160
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
If test resources have read-only permissions and are copied to
generated_compilations copySync will fail the next time because it can't
overwrite the write protected file.
Creating the file first ensures that it has default permissions that
won't be changed when copySync overwrites it.
Change-Id: I1b4a07c4644dc6d0c25275ebc84ba4ebc07a56c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162754
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
The static error updater used to run analyzer using the scripts in
sdk/bin/, but those are no longer reliable now that analyzer looks for
a nearby VERSION file. Instead, look for the most recently built
analyzer in the build directory.
Change-Id: If2b437b12d036e15f5503aba816dc2a007e39cfe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160943
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
I got tired of wading through the giant list and guessing at which one
matches the options I want to run the tests on, so I added filtering.
If you pass any of the common options like "-m", "-r", etc. when also
passing "--list-configurations", then it only prints configurations
that match those options.
Also, by default it only prints configurations that match the current
host OS.
Eventually, I would like *running* tests to work the same way, where
passing "-c" means "find me a config in the test matrix with this
compiler. But this seems like a good start.
Also, I removed the slow way that test.dart calls test.py to handle
--list-configurations now that those are all in the same package.
Change-Id: Ifabb415a9fad889afc12cfcd7dd81bd02c918612
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158980
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Karl Klose <karlklose@google.com>
When reading a secondary snapshot, re-run the dispatch table initialization found in the root snapshot. Since each loading unit's contribution to the dispatch table is scattered, it would take more snapshot size to try to serialize their contributions separately. We'll revisit this when we can defer Class, Function and Code, which will prevent the root snapshot from referencing all Codes.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Iefd2b98647b96ae59a7efe92897538f5cf8c2426
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158923
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
If a static error test contains "[web]" error markers, this runs them
on DDC and dart2js, parses the output, and tests that the compilers
report the expected errors.
Change-Id: I8b846f75828a0078f61c2fc1f3da0a4a587349cb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158100
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
- Don't use PC-relative calls when calling between loading units.
- Sort the contents of the code cluster by loading unit, then by text offset.
- Handle binding PC-relatives calls and inserting trampolines per loading unit.
- Create one code order table per loading unit to implement PC -> Code lookup.
- Read code order tables directly, instead of copying into malloc'd memory.
--use_table_dispatch still not yet supported.
This slightly shrinks non-split binaries (~2% clustered part, 0.4% total snapshot) due to the new delta encoding when Code references Instructions.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I51052ebc7b4968b554897d1710135a6c41821302
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157820
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
If a static error test only has expectations for *warnings* then it
still has well-defined runtime semantics that we want to test.
This distinguishes those tests and runs them on non-front-end
configurations.
Change-Id: I41b8d84a229ba53ad0db0271b28a9b9482ad582d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155305
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Karl Klose <karlklose@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
This script takes a list of test results and identifies blamelists
in the result feed data that include the commit of the test results
and tries to narrow the blamelist, if possible.
This CL also adds a small library to use the firestore REST API,
which contains mostly the functionality used in the script, but
should be easy enough to extend for other scripts.
Change-Id: If3c8272438e2a9bbf24891d9f5b62c342ea77cc6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153966
Commit-Queue: Karl Klose <karlklose@google.com>
Reviewed-by: William Hesse <whesse@google.com>
I don't like having a large volume of Dart code sitting under tools/
where it is hard to analyze, lint, test, and reuse. Also, eventually
we want to merge test.dart and test.py. This seems like an easy mostly
mechanical first step.
All I did was:
1. Move the contents of tools/test.dart to
pkg/test_runner/lib/test_runner.dart. (That's not a great file name
since we already have pkg/test_runner/bin/test_runner.dart, but it
was the best I could come up with.
2. Copy tools/bots/results to pkg/test_runner/bot_results.dart. I
don't like duplicating this, but there are other scripts under tools
that import the old location. Eventually, we should have those
scripts import it from package:test_runner/bot_results.dart, but I
didn't want to do that here since I'm not familiar with those other
scripts.
3. Make tools/test.dart import and forward to
pkg/test_runner/lib/test_runner.dart.
4. Fix any linter and type errors. The test_runner package has a bunch
of strictness checks and lints enable to keep it cleaner.
5. Run dartfmt --fix to format and get rid of "new", etc.
Change-Id: Ifc89817508d3fc147fa78dbc6744d547aeaf4c55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155240
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
This doesn't actually run DDC to generate the web errors yet, but it
changes the CLI in anticipation of that, and adds tests to verify that
once web errors are reported that the updater handles them correctly.
Change-Id: I31264e3d468969b07f9eb60353a9b02a93bec7ea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155102
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
The IO tests aren't working yet, but basic tests work:
tools/test.py -n dartk-fuchsia-debug-x64 language_2/list/literal3_test
You may need to run this first:
sudo chmod 666 /dev/kvm
Change-Id: I04915ce11f671f1d493f9eeb6bc832089ba9bfa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154828
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: William Hesse <whesse@google.com>
(Assignment of libraries to loading units is already done in the kernel generation step.)
After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.
All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.
Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.
Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.
Embedder's guide:
At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.
At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Now that the sound null safety mode is a compile time flag
instead of a runtime flag there is no need to set the value in
the bootstrapping code before running main(). The sound and weak
versions of the sdk now have it already set.
Add compileTimeFlag() method that will inline the constant value
directly into the generated output.
Change-Id: I5b9243c47cffc421067ac820ef49342b9d0efff9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154100
Reviewed-by: Mark Zhou <markzipan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
The bootstrapping was still loading the wrong version of the sdk and
pre-compiled packages when running tests for DDC.
Update constant_null_safety_mode_test to test a constant that is
defined within the sdk itself in addition to one defined in the test.
Change-Id: I8938ed92d866cfca0ce42567f03bed5f3dd9f10c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154121
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
This is a reland of 40c79c8ed7
The build recipes now use forward slashes on Windows to pass the path to
the browser executable to test.py.
Original change's description:
> [test] Add paths to chrome and firefox to the environment when present
>
> This adds a CHROME_PATH or FIREFOX_PATH environment variable to the
> test's evnironment which contains a path to the browser's executable.
> This is only set if the --chrome or --firefox argument was passed to
> test.py.
>
> https://github.com/dart-lang/sdk/issues/42417
>
> Change-Id: I711de6ab91b8ffbbe72c7fb1530b68c4189a4b8a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153209
> Commit-Queue: Alexander Thomas <athom@google.com>
> Reviewed-by: William Hesse <whesse@google.com>
Change-Id: Ib61b8c6424fa7bce525d76d8b94756a215d28c1e
Cq-Include-Trybots: dart/try:dart2js-strong-win-x64-chrome-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153980
Commit-Queue: Alexander Thomas <athom@google.com>
Commit-Queue: William Hesse <whesse@google.com>
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
* Setting the nonNullAsserts flag in the bootstrapping logic will
enable/disable failing when a null value is passed to a non-nullable
method parameter when running with weak null safety.
* Move the --null-assertions from VM options to the shared options in
the test and use it to set the flag in DDC the entry point.
* Configure other backends to ignore the flag.
Change-Id: Ia2670514bed7fa981564e99b85d74f6bae6dd9fc
Fixes: https://github.com/dart-lang/sdk/issues/42404
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151306
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Mark Zhou <markzipan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This reverts commit 65e3c9d3b1.
Reason for revert: package:logging, package:stack_trace and package:usage need to be migrated first.
Original change's description:
> [vm, service, observatory] Bang Bang (My Type System Shot Me Down).
>
> Port the service tests and Observatory to Dart 3.
>
> Changes from the original landing:
>
> - The old tests are copied to observatory_2 / service_2 so the service can still be tested when running a legacy mode program
> - The test harness is taught about 'service_2'
> - Observatory's package is added to front end's opt-in list
> - Fixed some places in the bot configuration matrix so ensure 'service' runs on legacy bots and 'service_2' on weak-mode bots
>
> The ported tests themselves are not changed.
>
> Change-Id: I1d7e5cc61cdc044e1985e851bea7fd8a18f7d810
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149720
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>
TBR=bkonyi@google.com,rmacnak@google.com
Change-Id: I43b72f149d8d7e9fe06006cdd8593fed1726aa3a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152004
Reviewed-by: David Morgan <davidmorgan@google.com>
Commit-Queue: David Morgan <davidmorgan@google.com>
Port the service tests and Observatory to Dart 3.
Changes from the original landing:
- The old tests are copied to observatory_2 / service_2 so the service can still be tested when running a legacy mode program
- The test harness is taught about 'service_2'
- Observatory's package is added to front end's opt-in list
- Fixed some places in the bot configuration matrix so ensure 'service' runs on legacy bots and 'service_2' on weak-mode bots
The ported tests themselves are not changed.
Change-Id: I1d7e5cc61cdc044e1985e851bea7fd8a18f7d810
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149720
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Previously it tested if it was at the end of file (defined as
_currentLine >= _lines.length), if it was not it would try to read the
_next_ line (_peek(1)), meaning that if _currentLine = _lines.length - 1
it would _peek(1) and try to read _lines[lines.length] and crash with a
range error. This CL fixes it by actually checking if it can safely peek
the amount it wants to.
Change-Id: I2e66c539e823f0b39bd0da39e2b2277a9e6c9501
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150624
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Ryan Macnak