Currently only CFE ("Fasta") tests have their context message output
parsed. It should be easy to extend that to dart2js and DDC if that's
useful. Analyzer might be more work.
This also adds support to the test updater for inserting context
messages when updating tests. By default, that flag is off, so the
existing behavior is preserved where context messages are ignnored. If
you want them, pass "-c" when updating a test.
When validating test output, if the test file contains context messages,
then they are validated. Otherwise, any context messages in the CFE
output are ignored. This way existing tests still pass.
Change StaticError to represent a single error for a single front end.
Before, the data model collapsed errors for different front-ends at the
same location into a single StaticError object which tracked different
messages for each front end. The idea was to move towards a world where
they really are the "same" error with eventually the same message.
But this adds a lot of complexity with things like merging errors and
doesn't reflect the reality that each error from each front end is
basically its own thing. Also, critically, it makes it much harder to
attach context messages to a specific front end's error object.
This changes it so that an instance of StaticError represents a single
error for a single front end. The test file syntax is unchanged and the
updated tool behaves the same. In a static error test, multiple
expectations can still share the same "// ^^^" marker line. They are
just expanded to multiple StaticError objects at parse time.
This eliminates all of the complexity around merging and simplifying
errors.
Change-Id: Ida1736bfcde436fc2d1ce2963d91fa9cb154afa8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193281
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Before, the data model collapsed errors for different front-ends at the
same location into a single StaticError object which tracked different
messages for each front end. The idea was to move towards a world where
they really are the "same" error with eventually the same message.
But this adds a lot of complexity with things like merging errors and
doesn't reflect the reality that each error from each front end is
basically its own thing. Also, critically, it makes it much harder to
attach context messages to a specific front end's error object.
This changes it so that an instance of StaticError represents a single
error for a single front end. The test file syntax is unchanged and the
updated tool behaves the same. In a static error test, multiple
expectations can still share the same "// ^^^" marker line. They are
just expanded to multiple StaticError objects at parse time.
This eliminates all of the complexity around merging and simplifying
errors.
Change-Id: I1d55a6e885e12cc9c438f928297fc0db7dd5ce85
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193280
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
If a static error test only has expectations for *warnings* then it
still has well-defined runtime semantics that we want to test.
This distinguishes those tests and runs them on non-front-end
configurations.
Change-Id: I41b8d84a229ba53ad0db0271b28a9b9482ad582d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155305
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Karl Klose <karlklose@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
This enables parsing and validation for tests that contain:
// ^^^
// [web] Some web-specific error.
This doesn't run those tests on DDC and dart2js yet, and the test
updating tool doesn't handle web tests yet. I'll do those in follow-up
patches.
Change-Id: Id06397a20a06c00e48801a16c01d5878e0264f6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146462
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Auto-Submit: Bob Nystrom <rnystrom@google.com>
The old code to report the difference between a tests's expected and
actual errors was easily confused by missing or unexpected errors.
Those would misalign the lists of errors causing all subsequent errors
to be reported as wrong. This fixes that. Before, this test containing
a single unexpected error:
main() {
int a = "s";
// ^^^
// [analyzer] STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
unknownFunction();
int b = "s";
// ^^^
// [analyzer] STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
int c = "s";
// ^^^
// [analyzer] STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
}
Would cause the test runner to output:
incorrect static errors:
Error at line 6, column 3, length 15
- Expected on line 8 but was on 6.
- Expected on column 11 but was on 3.
- Expected length 3 but was 15.
- Expected error code STATIC_TYPE_WARNING.INVALID_ASSIGNMENT but was STATIC_TYPE_WARNING.UNDEFINED_FUNCTION.
Error at line 8, column 11, length 3
- Expected on line 12 but was on 8.
reported unexpected static errors:
Error at line 12, column 11, length 3
STATIC_TYPE_WARNING.INVALID_ASSIGNMENT
Oof. Now it prints:
static error failures:
Unexpected static error at line 6, column 3, length 15:
- Had error code STATIC_TYPE_WARNING.UNDEFINED_FUNCTION.
Change-Id: I1b22b2a542c6f8049f92fda3627ae69144bfd52a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114240
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
The previous system let you use this to define an "unspecified error":
int i = "bad";
// [unspecified error]
The marker comment meant that *both* front ends had to report some error
on the previous line, but any error at any position was considered
acceptable.
Unfortunately, this doesn't let us incrementally specify an error for
one front end while leaving it unspecified for the other. This means
that when, say, the analyzer implementation lands first, we can't pin
down its errors while still leaving the CFE errors unspecified until
its ready.
This addresses that by making "unspecified" a property of each front
end, not the error itself. The new syntax is:
int i = "bad";
// ^^^
// [analyzer] unspecified
// [cfe] unspecified
This means the same thing as the previous example: both front ends must
report some error on the previous line. The column information is
authored as a best effort, but ignored.
Now, let's say the CFE implementation lands. The above can be changed
to:
int i = "bad";
// ^^^
// [analyzer] unspecified
// [cfe] Real error message.
At this point, the CFE test must report that exact error at that exact
position and message. But the analyzer is still free to report any
error code anywhere on that line.
This syntax is a little more verbose, but it's simpler and more
flexible.
Change-Id: I37f937d245fd8ec8054acb6128256f9fff6241e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/109728
Auto-Submit: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
If a test has the static error markers and is run on analyzer, the
test runner verifies that the reported errors exactly match the expected
ones. A test with error markers is skipped on all other configurations.
Change-Id: Ib921acde517688f5b47937a5b100c0507e0e9138
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107823
Commit-Queue: Bob Nystrom <rnystrom@google.com>
Reviewed-by: William Hesse <whesse@google.com>
Robert Nystrom