Make the parser handle metadata properly according to the spec.
There has been some confusion in regards to metadata because
https://github.com/dart-lang/language/blob/master/accepted/future-releases/small-features-21Q1/feature-specification.md
(used to) say we needed a spec update.
That was done in the parser in
https://dart-review.googlesource.com/c/sdk/+/182668
but really that made the parser adhere to the spec as it was which is
also correct (as the spec change wasn't needed at all).
The parser also needs to be able to skip metadata, though, and that
wasn't done it that CL which caused the issues in
https://github.com/dart-lang/sdk/issues/45120
The work directly in this CL is:
* Add lots of comments to parseMetadata to make them more up-to-date with
the spec.
* Fix skipMetadata to correspond to parseMetadata.
* Tests verifying that 45120 is fixed in the parser and that metadata in
general works as expected in the parser.
(In working on this, the future specification was corrected to say that the
spec did not need updating. Thanks to Erik for the quick turn-around
there).
Bug: #45120
Change-Id: Id1e4912cac7e5d9113efe083b3ecd06e6834a16e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190880
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This CL avoids the infinite loop created by allowing the parser
to parse it as a complex type (despite no actual name after it) and
adds a few asserts in an attempt to catch rewriting on eof (setting a
token after eof).
Fixes#42229Fixes#44477
Change-Id: Ie2da0957894067a3d4748cb6384bc9d6cf32c215
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150521
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
* Add mechanisms to try out a recovery and only perform it if is
successful.
* Recover written out binary operators, e.g. "a xor b", "a or b" etc.
This fixes#26810 and is also how these operators are written in e.g.
Kotlin. This might be somewhat controversial though.
* Adds a mechanism to _replace_ a token by another token.
This might be controversial.
It is done because just inserting the operator causes the same rewrite
to be attempted on the same token stream several times (at least with
the way the token stream is parsed via the CFE) in turn causing it to
be recovered *sometimes*. This is now avoided by actually replacing
the token.
Change-Id: Icfa806045575d2aa2e5f35126708651b275bcf84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162003
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This CL improves recovery in situations like this:
typedef c = foo(int x);
=> recovers like typedef c = foo Function(int x);
typedef d = (int x);
=> recovers like typedef d = Function(int x);
typedef e = foo<F>(int x);
=> recovers like typedef e = foo<F> Function(int x);
typedef f = <F>(int x);
=> recovers like Function<F>(int x);
typedef g = foo<F, G, H, I, J>(int x);
=> recovers like typedef g = foo<F, G, H, I, J> Function(int x);
typedef h = <F, G, H, I, J>(int x);
=> recovers like typedef h = Function<F, G, H, I, J>(int x);
typedef i = <F, G, H, I, J>;
=> recovers like typedef i = Function<F, G, H, I, J>();
And appropriate error messages are given:
"Expected 'Function' before this." and (when inserting parenthesis)
"A typedef needs an explicit list of parameters.".
Fixes#26073.
Change-Id: I368f36f2993033d62b36315198bc993eed74bc92
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161485
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
This CL:
* Changes the events so beginX and endX events always comes in pairs
(though technically not right as some specific events can be beginX
endY --- but in those cases it is at least documented and used in code
that actually tests it).
-> This entails adding some events and converting something from
"beginX" to "handleX" instead.
* Adds a utility that can generate an AST of sorts directly from the
parser via a listener using the begin/end matching (and knowing of the
specific ones that doesn't match directly).
* Adds a test that checks that - at least for all tested (50,000+) files
- the AST actually generate "correctly", i.e. matches up begin/ends
and ends up with a single top entry "CompilationUnit".
* Adds a different visualization to the parser listener events by
displaying the "AST directly from the parser" in a UI that can be
navigated. The visualization may not be the best, but it's certainly
a stepping stone.
Change-Id: I9b27f7bbf3be442adc92f357c7b3c46da6f84cf7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159664
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
If an identifier starts with a non-ascii-character and has a comment
an assert was triggered about the comment not being attached to the
token. This CL attaches the comment to the token (which it should be)
and thus avoiding the assert trigger.
Change-Id: Id261970b88ca721d4b3a996abfb8ff43f0ec8341
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161102
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
E.g. class Foo extends Bar, Baz {} would before have parsed weirdly,
but it will parse as you'd (probably) expect with an error saying
you cannot do that. For now at least both CFE and Analyzer will
pretend like you just specified the first one but I suppose Analyzer
could for instance use the extra information to propose converting
other ones to implements clauses or similar.
Fixes https://github.com/dart-lang/sdk/issues/22313
Change-Id: I180cdd8ab07143dd74fd21c9976ec2a46c428d8e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158261
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
There is no functional change to the analyzer or CFE yet; we simply
report the error in the parser listeners now rather than the parser
itself. In follow up CLs when we add full support for the feature, we
will want to keep the error reporting logic, but only report the
errors in pre-NNBD code.
Change-Id: Ifd14dd97d7d1d57586a8e14a25103356d54f0ea1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154364
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Jens Johansen