When the test runner runs a legacy test with the non-nullable experiment
enabled, this passes an extra command line flag to tell analyzer to
treat the test library as opted out of NNBD.
This CL also removes the guard that made the test runner skip all legacy
tests when running analyzer with the non-nullable experiment enabled.
We want to run those tests now and (when analyzer supports the new
command line flag) they should be correctly treated as opted out.
This change also removes the tests package config file since that no
longer needs to be used to set the language version. It also removes the
code to generate a package config for the multitests. This CL replaces
both of those mechanisms.
Change-Id: Iedc1e6111bc0623ac21263499f3f5d11bd6fdf7c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135346
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Closes: https://github.com/dart-lang/sdk/issues/40233
This CL creates nnbd versions of the tests and runs them on the nnbd sdk.
This CL does not (1) migrate sdk_nnbd/lib/ffi fully yet, and does not (2) fix all the tests/ffi (which is NNBD tests) yet.
Uncovered new issues:
Issue: https://github.com/dart-lang/sdk/issues/40234 nullptr should have type Pointer<Never>.
Issue: https://github.com/dart-lang/sdk/issues/40247 Structs need external fields.
Issue: https://github.com/dart-lang/sdk/issues/40271 Callbacks hit assert in debug.
Change-Id: Icb1b83577e03ed283165eb17703fc8dfc7fa5960
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,analyzer-nnbd-linux-release-try,dart2js-nnbd-linux-x64-chrome-try,ddc-nnbd-linux-release-chrome-try,front-end-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132604
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
This works on all browsers, in d8, and supports both dartdevk and
dart2js. In the case of dart2js, this also expands minified names and
inlined data.
The result is presented to users as a separate section, so the original
JS stack trace is printed as is, and a second deobfuscated stack trace
is printed afterwards.
Change-Id: I10af5e0241a8c87ad8db48234ce52cfb51a27411
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132966
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Sometimes, the batch runners need the experiment flags set. This is
especially true when running in NNBD mode which needs to be enabled in
the VM running the batch runner as well as in the test code.
Previously this was only done for the CFE batch runner, this CL moves
the logic to apply to all compilation commands using batch runners.
Change-Id: I9358ae7cfed4f082075ef91b59408edcb2912361
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131831
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
* Correct mistake in the named configuration.
* Add missing test directories to the fileset.
* Add configurations for all operating systems (for local testing).
* Avoid duplicate experiments specified by both configuration and test.
* Pass experiment flags also to VM running the batch runner.
Change-Id: I47f2c1b07e91e2f8120a2712ce362689b68e9ce4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130900
Reviewed-by: Bob Nystrom <rnystrom@google.com>
It seems that we need to pass this flag to the VM now based on the
following message that appears when trying to run tests with the
experiment enabled:
```
VM initialization failed: Snapshot not compatible with the current VM
configuration: the snapshot requires 'release no-dwarf_stack_traces
use_bare_instructions nnbd-experiment' but the VM has
'release no-dwarf_stack_traces use_bare_instructions'
```
Alternatively, we could use the script from the sdk_nnbd/bin/ directory
where the experiment is hardcoded on, but that could be more confusing
when we unfork. WDYT?
Bug: #38665
Change-Id: I033e9e2e4a38f04c5cb8227efeee5a66826b46e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128570
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Allows us to more easily start running tests that use the NNBD sdk and do not
depend on these packages. We can add them back when they are migrated or can
easily be opted out and they build without errors.
Also cleans up a little of the ddk/ddc split and the location of the package
.dill files when running without --use-sdk.
Change-Id: I12fcbf18b47ad3189040f7e42c3ce9a4a06ed838
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/127966
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
The script verifies that the multi-test only contains static annotations
and that converting it to a single test produces the same error messages
as testing the sub-tests generated by the multi-test.
If that is the case, it uses
pkg/test_runner/tool/update_static_error_tests.dart
to insert the expected errors and removes the multi-test markers.
Additionally, a copy of the '/none' test and of all the sub-tests with an
'OK' marker are created to preserve the testof runtime behavior.
Change-Id: I6eb67246f209dfbdf9b4ba8705c368c6f1978938
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124128
Commit-Queue: Karl Klose <karlklose@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
It doesn't do anything useful yet because the forked co19 tests aren't
getting pulled into the repo, but once gclient syncs them to
tests/co19, this should start doing something.
Change-Id: I5ee54f9afc19353c7a7a379178df1084c6f2e8d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125681
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
- Copies corelib_2/a* -> corelib/
- Copies language_2/ab* -> language/
- Copies lib_2/math/ -> lib/math/
- Copies standalone_2/a* -> standalone/
And also copies over and renames all of the status files in those
directories.
Then it migrates those tests to be static error free in NNBD.
Finally, adds support to the test_runner for the new suites.
Note that this review is split into multiple patchsets. The first
patchset is a straight copy of the existing files. Then the later
patchsets have the interesting changes.
Change-Id: Icec2ff850a3aee30b653066ac184495d1e3814d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125467
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
This change allows to run test.dart and pkg/test_runner with multiple arguments
for the -n option to run tests for multiple configurations with one invocation.
Change-Id: If62e0bfc364460fa415c7f700f7e449b0de56987
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122395
Commit-Queue: Karl Klose <karlklose@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Both a test file and now a test configuration can specify that
experiment flags should be enabled. Since they both can, passing those
to the tools is a little tricky because we need to merge the two
command line arguments. It's made worse by the fact that this corner of
test_runner is riddled with poorly factored code.
This doesn't fix the redundant code, but does merge and pass the flags
through. Right now, it parses them from the test file by looking for
them in the SharedOptions line, which is where they currently appear.
That's pretty hacky. Eventually, we should change these tests to use:
// Requirements=
Fixes#38386, #38387, #38388, and #38389.
Change-Id: I1aea1482b30b9ba19c427319f895822d81ee12ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121262
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
If the current test configuration enables the "non-nullable" experiment,
the test runner will read from the NNBD build directory instead of the
normal one.
This only kicks in when the experiment is enabled by the
*configuration*. If the test file enables it using:
// SharedOptions=--enable-experiment=non-nullable
It has no effect on which directory is used. I think the right fix for
this is to use a different mechanism for these tests to specify that
they require NNBD. I'll send an email about that.
Change-Id: I4be2de8d0f7301c93dfb3f7e81c12372fc5d154a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121143
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
This gives us a way to specify that a test is only meaningful for
certain NNBD modes and should be skipped on other configurations. My
hope is we can extend this for other platform capabilities and then
eventually use this to express "skips" instead of relying on status
files for them.
Change-Id: I99548c326ee6fbe6db64e28e7f7f07d2fc1fd23c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117080
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
- Make it synchronous. I did some benchmarking and the async didn't seem
to make a measurable difference and made the code harder to follow.
- Fix a bunch of small-scale idiom things: "var", names, etc.
- Move TestCase.hash into TestFile since that lets us access it before
we've created a TestCase.
- Remove dead code.
- Make it clearer which functions work with TestFiles and which with
TestCases.
- Clarify the code that determines whether or not to enqueue a test.
The last point is the motivating one. Soon, I'll be extending this code
to take NNBD into account when determining which tests to skip, so I
wanted to clean it up some first.
Change-Id: I488ed6c7d2453535968ac43389a2fd8370ead57d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116662
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
- Better name than "optedIn".
- Actually hook up the command-line option to the configuration.
- Add tests, which would have caught the previous mistake.
- Don't allow comma-separated values for "--progress" and "--nnbd".
- Remove unused dead "--strong" option.
Change-Id: I57d7cb0d81af50d662dcf3f7f4c9ca1f2b102f2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116544
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
Robert Nystrom