Extend test_runner VMOptions support with an ability to specify
paths relative to temporary compilation directory.
// VMOptions=--foo=$TEST_COMPILATION_DIR/foo.file
The same directory will also be passed as an environment variable
to execution command.
Migrate most of the tests which used to write stuff into the SDK
root to use this feature. I am leaving vm/dart/causal/* tests
unmigrated because migrating requires time consuming manual
update of expectations (which encode raw line numbers). I have
a follow up CL which changes how these tests are written which
will make migration trivial.
Change-Id: Id53008be66de8ff18623efac27ff15750f407749
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300600
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Also picks up keys from `package:native_assets_cli` instead of coding
them which could lead to typos.
Change-Id: I5c72c26c9a45a38ac41f3513c2b5b10e8863ff83
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300221
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
We use the MSVC from Depot Tools to build C files.
When using clang from DEPS, we need to pass the right INCLUDE / LIB
environment variables. So we might as well use the MSVC instead.
Moves the `nativeCompilerEnvironmentVariables` to the
`TestConfiguration` so that it's cached.
Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: I3576cd0ed07081afb9ae5ba737b8bfe1b21941e5
Cq-Include-Trybots: luci.dart.try:pkg-win-release-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-linux-release-try,pkg-linux-debug-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/298580
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
These were added during the Dart 2 test migration as a way to indicate
why a test was expected to fail. But, like negative tests, they have
very poorly granularity.
Static error tests are strictly superior.
The co19 tests have been migrated off of these markers for a couple of
years, so our own language tests were the only holdouts. I see no uses
of "@syntax-error", "@runtime-error", or "@static-warning". I have now
migrated all of the tests that contained "@compile-error" to be proper
static error tests.
This simplifies the test runner and makes our tests more precise.
Fix#45634.
Change-Id: I0f46d110b6f322d98187e734195ecba7524574af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296720
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Multitests aren't valid Dart files that can be processed by a Dart
implementation so the updater generally does a poor job if it tries to
update one.
It's probably not worth supporting because, in practice, a test should
either be a multitest or a static error test, but not both.
Change the tool to skip over any multitests it encounters. If this
results in it doing nothing at all, it reports that as an error.
Otherwise, it just lists the multitests it didn't process.
Close#37721.
Change-Id: Icfb1ff9fe63f2c249b3ccfba65166b97654a9918
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296760
Commit-Queue: Leaf Petersen <leafp@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Pulls in a new versions of https://github.com/dart-lang/native with
support for MacOS and enables the tests for these on the MacOS pkg
bots.
Fixes the clang paths in pkg/test_runner/lib/src/options.dart for
mac and win.
Since a relatively recent XCode is available on the bots, use XCode
to build rather than the clang from DEPS.
test/native_toolchain/recognizer_test was refactored to only
run tests for tools installed on the host. So it's unskipped.
Bug: https://github.com/dart-lang/sdk/issues/50565
Change-Id: I3804b01da7a4e74d3e49a1b2ba3c8425132417d7
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-mac-release-arm64-try,pkg-mac-release-try,pkg-win-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296180
Reviewed-by: Liam Appelbe <liama@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Sometimes failing tests crash or terminate with arbitrary exit codes.
For example, MSAN can abort a test with exit code 1.
This should not be treated as an infra failure by the test harness,
and should not prevent running other tests.
Change-Id: I97450d9cfb1e117451c35ab8a8cd01074fbef8db
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291304
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Eventually, these hints should probably be moved over to warnings. But
for now, this makes it possible to write static error language tests
that validate that analyzer produces unreachable case warnings/hints
where expected.
Also updated the patterns/ and switch/ tests now that those errors must
be expected by the test.
Change-Id: If1fb92602c4bde2819b9eec73598033009054947
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/291967
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
- Remove unsound platform .dill files from the packaged SDK.
- Throw and error referencing the sound null safety requirements of
Dart 3 when compiling with `--no-sound-null-safety` and an unsound
platform .dill file is not found.
Change-Id: I703a446e3e92c76a6b48168577fc0161eea83854
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289280
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
For DDC "legacy" code requires the same flags as compiling
and running modern code with unsound null safety.
During compilation all the nnbd "weak" options are already passed when
the mode is "legacy". Now the same runtime flags are used as well.
Change-Id: I8610a786adabab4d85f47dc3d0989b280376bece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283133
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Use "ddcOptions" in test files to closer resemble the name 'ddc-options'
from the test matrix or command line option.
I don't like that this option is in camelCase when written in a test
file but uses a dash when passed on the command line but that matches
all the other implementations (dart2jsOptions, sharedOptionn, etc).
Hopefully it is the most discoverable or least surprising choice.
Change-Id: I8f0d53d50416b952c9ea74406c18bb9c700f98dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281820
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Make test.py pick migrated suites by default when the `NnbdMode` is not `NnbdMode.legacy`.
* Remove non-existing benchmark_smoke suite.
* Remove broken analyzer_library suite from default suites.
* Deprecated observatory_ui is not added to the default migrated suites.
* Remove remaining references to samples-dev.
* Remove unnecessary suite specifications from the test_matrix.json.
* Remove broken references to observatory_ui from the test_matrix.json.
* Remove defunct observatory_ui hack from test.py.
Fixes: b/268474066
Change-Id: I6c3635bcb396021ca86498d60c9efc77ab39589e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281701
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
* The new compiler name "ddc" will be used for all configurations
in an upcoming change.
* Aliases "dartdevc" and "dartdevk" will be removed in the future.
Change-Id: I2fa48e0ec043b1ba7fb70f883b39e40f338b6928
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280862
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Use explicit ddc-options to pass `--canary` flag to the compiler instead
of sniffing for a builder tag.
This just feels more expected and makes more sense if you are trying
to understand how the canary configurations compile tests differently
from the stable configurations.
Change-Id: I59b1e62d3839f6f992eea917ecbae240640fcd76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280783
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
The --detect-host flag is provided as a convenience for running tests
locally with named configurations. It alters the provided configurations
if needed to match the host system and architecture.
It is an error to use the flag without named configurations or as an
option in the test_matrix.json file.
Add a the `Configuration._cloneHelper()` constructor with all parameters
as required arguments. This provides some compile time feedback when
new fields are added to the class and the clone methods haven't been
updated.
Change-Id: I58e2ead1c3324c2cc893a4766d7daccd2787f631
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280139
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Updated strictness on build configuration for analyze_snapshot
Will run only under Linux/Android 64 bit arm, arm64c, simarm64,
simarm64c and x64 architectures.
analyze_snapshot just has a single target that links into product precompiled programatically
Testing suite updated to comply with null-safety requirements and
fix for simulated platform failing to use gen_snapshot with assembly.
TEST=ci
Change-Id: I3d58400db2e26a441a40fe7197b22510a52732b6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279391
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Chris Evans <cmevans@google.com>
* ARGUMENT_TYPE_NOT_ASSIGNABLE_TO_ERROR_HANDLER
* BODY_MIGHT_COMPLETE_NORMALLY_CATCH_ERROR
* BODY_MIGHT_COMPLETE_NORMALLY_NULLABLE
* DEPRECATED_EXTENDS_FUNCTION
* DEPRECATED_IMPLEMENTS_FUNCTION
* DEPRECATED_MIXIN_FUNCTION
* DEPRECATED_NEW_IN_COMMENT_REFERENCE
This requires allowing these warnings to be reported in language tests without being expected.
Also, observatory now must comply with BODY_MIGHT_COMPLETE_NORMALLY_NULLABLE (or we could ignore the diagnostic).
TESTED=unit tests
Change-Id: I8facfa48a8645bac5bf8037163076a8f0a0ad05e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275740
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Samuel Rawlins <srawlins@google.com>
This is necessary so we can start specializing JS method trampolines. Specializing JS methods will help reduce polymorphic behavior in JS trampolines for JS interop methods.
In the follow on CL we will start injecting specialized JS method trampolines directly into the runtime.
Change-Id: Ib12fb286f54cfb8a64c9f84e238433e8da2eeaa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/272441
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Joshua Litt <joshualitt@google.com>
- Flip flag to make strong null safety the default
- Remove code that auto detects null safety mode from source files,
it is necessary to specify --no-strong-null-safety to opt out.
- Retains sniffing of AOT/JIT snapshots and kernel files to determine
null safety mode, the opt out has to be done when generating these
file.
TEST=ci
Change-Id: If2c9608eedb7c46d9c3cd85e261ee9640e0d28eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261140
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Vyacheslav Egorov