Commit Graph

12 Commits

Author SHA1 Message Date
Robert Nystrom f9f005cdb9 Tool to automatically update expectations in static error tests.
Change-Id: Ie1faacd66448efe209b35dfd66fded622e59812a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/110766
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2019-07-30 01:23:22 +00:00
Robert Nystrom 94b34b2b30 Refine how unspecified static error expectations are defined.
The previous system let you use this to define an "unspecified error":

   int i = "bad";
   // [unspecified error]

The marker comment meant that *both* front ends had to report some error
on the previous line, but any error at any position was considered
acceptable.

Unfortunately, this doesn't let us incrementally specify an error for
one front end while leaving it unspecified for the other. This means
that when, say, the analyzer implementation lands first, we can't pin
down its errors while still leaving the CFE errors unspecified until
its ready.

This addresses that by making "unspecified" a property of each front
end, not the error itself. The new syntax is:

   int i = "bad";
   //       ^^^
   // [analyzer] unspecified
   // [cfe] unspecified

This means the same thing as the previous example: both front ends must
report some error on the previous line. The column information is
authored as a best effort, but ignored.

Now, let's say the CFE implementation lands. The above can be changed
to:

   int i = "bad";
   //       ^^^
   // [analyzer] unspecified
   // [cfe] Real error message.

At this point, the CFE test must report that exact error at that exact
position and message. But the analyzer is still free to report any
error code anywhere on that line.

This syntax is a little more verbose, but it's simpler and more
flexible.

Change-Id: I37f937d245fd8ec8054acb6128256f9fff6241e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/109728
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2019-07-24 02:04:22 +00:00
Robert Nystrom 6c03c1a47c Support static error tests on Fasta.
Change-Id: I1e20032c91af6aaa6a4a481403f2899f9f1b872f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108562
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
2019-07-12 23:37:13 +00:00
Robert Nystrom 0104a62595 Enable "prefer final fields" lint and fix violations.
I also removed a field that did nothing useful, fixed a couple of types,
and removed some unnecessary getters.

Change-Id: I95af91850f5971a22046db4a7d746bc0a43a643c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108700
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
2019-07-10 22:04:07 +00:00
Robert Nystrom 81b677cd80 Turn on a bunch of linter rules for the test_runner.
I pulled this list from the build package and turned on the ones that
only had a handful of violations. Left the others that I eventually want
to enable commented out.

Change-Id: I09ec037982e29d3682bc0eaa6af28adfad91b941
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108563
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
2019-07-10 17:53:48 +00:00
Robert Nystrom 5450d08ca1 Add support for analyzer static error tests.
If a test has the static error markers and is run on analyzer, the
test runner verifies that the reported errors exactly match the expected
ones. A test with error markers is skipped on all other configurations.

Change-Id: Ib921acde517688f5b47937a5b100c0507e0e9138
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107823
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2019-07-09 00:18:22 +00:00
Robert Nystrom 13b470ac46 Parse error expectations in tests.
They don't do anything yet, but the syntax is supported and tested.

Change-Id: I67e3c72babd8e272b28b434dfb0cf6028f472676
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107568
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
2019-06-29 00:54:11 +00:00
Robert Nystrom 690ab40377 Fix path separator in test for Windows.
Change-Id: I542dedcf46336d38c2c641e508c6d8c9f7bb4adb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107565
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
2019-06-27 22:35:53 +00:00
Robert Nystrom 0f70ef1a6d Tests for TestFile class.
Also a few minor tweaks to TestFile to make it more testable and pin
down some edge case behavior.

Change-Id: Icf56be74b68b1e9073b892bc441f04a433e437a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107366
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
2019-06-27 18:27:20 +00:00
Robert Nystrom 15cac4204e Remove deprecated package-root flag.
Change-Id: I519d4751b596751f930884d0320c60dbdbcbce48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106202
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2019-06-19 20:54:41 +00:00
Robert Nystrom c1b56a9ea0 Catch the test_runner codebase up to somewhat modern practices.
- Run dartfmt --fix. This converts JavaDoc comments to "///", removes
  "new" and extraneous "const", and a couple of other things.
- Fix SCREAMING_CAPS constants to lowerCamelCase.
- Use collection literals where possible.
- Use UI-as-code in a couple of places where it seemed obvious.
- Use "var" for more local variables.
- Use "const" instead of "final" when possible.
- Make members private when possible. Deleted a few that then became
  obviously unused.
- ".length > 0" -> ".isNotEmpty".

There are no meaningful changes.

Change-Id: Ic6c5a74b2af9b3ebcbe881dbed69f65488bdef09
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105880
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
2019-06-15 01:02:50 +00:00
Robert Nystrom 99e8a9fba5 Move the test runner (i.e. "test.dart"/"test.py") to pkg/.
This makes it an actual Pub package like most other code inside the SDK
repo. The main goal is to make it easier to write tests for the test
runner itself.

This change:

- Moves all of the code from tools/testing/dart/ over to
  pkg/test_runner. Most of it ends up under test_runner/lib/src.

- Move tools/testing/dart/main.dart to
  pkg/test_runner/bin/test_runner.dart.

- Move standalone_2/io/test_runner_test.dart to
  pkg/test_runner/test/test_runner_test.dart. I don't think it currently
  works, but it wasn't being run in its old location either.

- Add test_runner to the analysis-server bot. This ensures the
  test_runner package is static error clean.

- Remove standalone_2/io/test_runner_analyze_test.dart which used to
  attempt to do the above and is no longer needed.

- Update test.py to look for the test runner at its new location.

- Add test_runner to the repo .packages file and remove the weird
  test_dart pseudo-package. (I think this fixes #35279.)

- Remove status file entries for the removed standalone_2 tests.

There are no code changes to the test runner itself aside from fixing
up import paths.

Change-Id: I3d05d50d222b291848fa5a30de2846e803bc81e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105821
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2019-06-14 23:35:10 +00:00