Before, when parsing a new style typedef without an identifier (e.g.
`typedef = whatnot`) it concluded that the token after the equal-sign
wasn't an equal sign an that it thus had to be an old style typedef,
followed by recovery there.
This CL makes the recovery insert an identifier after `typedef` (what
the recovery of the old style actually did too), and then parse it as a
new style typedef.
This causes the previous many errors to just be `Expected an identifier,
but got '='.`
It is furthermore verified that the parsing of such a case is ~the same
as when having an indentifier.
Fixes https://github.com/dart-lang/sdk/issues/56912
Change-Id: I5cde1f29839555b1d6027a7d040dc6f60ac614a5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/392560
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
The entire scanner/parser now no longer uses `optional` or `optional2`,
the latter being completely removed.
Furthermore, when compiling a fixed version of the CFE with the CFE
this CL reduces the instructions used by ~13.6 mio.
```
page-faults:u: -0.3695% +/- 0.0676% (-384.20 +/- 70.23)
instructions:u: -0.0645% +/- 0.0007% (-13671097.80 +/- 153289.78)
```
Change-Id: Ib93dd2983aa1b4df0610d099010ac10f55f357f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/388323
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This adds the enclosingDeclarationName to the beginMethod of the
parser listener.
This enables the removal BuilderFactory.currentTypeParameterScopeBuilder.
Change-Id: Ie2bec9432c20b8bdbd62a14e8a65c272179d4698
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383182
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
This changes the BodyBuilder to create FunctionTypeParameter instead
of FormalParameterBuilder for "parameters" in function types. This
avoids the creation of unnecessary VariableDeclaration nodes
function types.
The types of these VariableDeclarations where created before the type
variable scope was completed, introducing an artificial dependency on
the pending nullability computation.
Change-Id: Ie1203fa4c78a27f3e7a0dfad16725cbbde24a6a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381143
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
When the parser sends events to the listener some events has a pair of
parameters, `beginToken` and `endToken`. Most of these are constructed
in such a way that the `beginToken` is the first token and the
`endToken` is the last token in that construct. This is for instance the
case with `endClassDeclaration`.
It was, however, not the case for `endMetadata` where `endToken` instead
was the next token *not* in the metadata.
In this CL I've found a changed the following to point to the last token
in the construct instead of the next token not in the construct and
renamed the parameter where it made sense:
* `endAssert` --- and renamed `semicolonToken` (which only pointed to
a semicolon for statements) to `endToken`.
* `endAwaitExpression`
* `endConstLiteral` --- and renamed `token` to `endToken`
* `endConstructorReference`
* `endFieldInitializer` --- and renamed `token` to `endToken`
* `endForIn`
* `endForInBody` --- and renamed `token` to `endToken`
* `endForStatement`
* `endForStatementBody` --- and renamed `token` to `endToken`
* `endFunctionExpression` --- and renamed `token` to `endToken`
* `endInitializer` --- and renamed `token` to `endToken`
* `endInitializers`
* `endInvalidAwaitExpression`
* `endMetadata`
* `endSwitchCase`
* `endTopLevelDeclaration` --- and renamed `nextToken` to `endToken`
* `endWhileStatement`
* `endWhileStatementBody` --- and renamed `token` to `endToken`
* `handleNoConstructorReferenceContinuationAfterTypeArguments`
In the few places in listeners where these values were used I've mostly
updated to do e.g. `endToken.text!` to retain the current behavior.
Change-Id: I25495e160d1eec5c75bcf1313b512cd04bcb1533
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364322
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Sometimes when the parser is performing error recovery it skips tokens
until it finds the next `,` or `;`, to try to get back on track. This
is handled by the `Parser.findNextCommaOrSemicolon` method.
When this method encounters a `BeginToken` (i.e., `(`, `[`, `{`, or
`<`), it needs to skip to the matching end token (using the `endToken`
getter), so that it doesn't try to resume parsing inside some nested
structure.
However, sometimes there is no matching end token (either because the
user has failed to properly matched `()`, `[]`, or `{}`, or because
the `BeginToken` is `<`, which doesn't always have a matching `>`). To
avoid a crash when this happens, `Parser.findNextCommaOrSemicolon`
needs to treat an unmatched `BeginToken` like an ordinary token, and
just advance to the next token.
Fixes https://github.com/dart-lang/sdk/issues/54236.
Bug: https://github.com/dart-lang/sdk/issues/54236
Change-Id: Id208c7a46c9c00b69f7f460a638d59486ebaffea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339980
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Includes a cleanup of the errors reported for duplicate instance field.
Previously, in addition to reporting the duplicate, we would report that
the field cannot be used because it is a duplicate and that the
duplicate field hasn't been initialized.
Closes#53467Closes#53320
Change-Id: Ifd94223500809f3be36a1345d9d0c409778eed8a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336885
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
This passes the first token of these top level declaration directly
to the endX listener method.
In the CFE this is used to ensure that can handle the new modifiers in
the textual outline. Furthermore, support for extension types is added
and having an "unknown chunk" now results in an error. The latter should
help us keep the textual outline up-to-date wrt new features.
Change-Id: I813d6162b6cba0a2bf550ed33a6091abf9bf49f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/324702
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This changes the parser to always interpret `extension type` as the
start of an extension type declaration.
Recovery is added to handle missing identifier, primary constructor,
parameter constructor parameters, and extension type declaration body.
Change-Id: I98cac0b2641167c4708fa20b22f0de0a70336457
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321704
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
This is not the flag you're looking for.
The 'extension-types' flag was used for an early experiment that is
not directly related to the Extension Types feature currently being
developed. The current feature uses the 'inline-class' flag.
Change-Id: Icbb6c3828c41e743e726161b17da4c7784a2c677
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316380
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
This adds parser support for the optional 'const' keyword in extension
type declaration. The token is passed on through the endPrimaryConstructor
listener call since it is used to make the primary constructor constant.
Change-Id: I518a8e397fb62272002c424e7b69affe6123f4af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316221
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
This changes the parser to call endInvalidYieldStatement for
yield statement both in sync and async methods. This ensures that
the CFE creates an invalid expression for yield in async methods,
making the generated AST verifiable.
Change-Id: I1bfe922878fcf3dc19c823cb89ee869af3dd2ce3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309200
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Two of these tests (`recordPattern_nullable_beforeAs` and
`recordPattern_nullable_beforeWhen`) verify that the fix for #52439
(Destructuring with explicit type of nullable record is a parsing
error), which causes `(...)? identifier` to be recognized as a
variable pattern, doesn't get confused by the pseudo-identifiers `as`
and `when`, and so it continues to correctly parse `(...)? as ...` as a
cast pattern and `(...)? when ...` as a guarded pattern.
The other two tests (`recordPattern_nonNullable_beforeAs` and
`recordPattern_nonNullable_beforeWhen`) verify that `(...) as ...` and
`(...) when ...` are parsed correctly. These forms were never broken,
but they were not well tested either.
Bug: https://github.com/dart-lang/sdk/issues/52439
Change-Id: I866e1c7a6a8e47e0cd91a1a77654405f395b50da
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305844
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Previously, if a record-typed variable pattern lacked a `var` or
`final` keyword, and was followed by one of the tokens `||`, `&&`,
`as`, `?`, `!`, `when`, or `=>`, the parser would fail to recognize
it. This happened because the call to `computeVariablePatternType`
wasn't passing `true` for the optional parameter `required`; that in
turn placed `computeType` in a mode where it believed it was parsing a
potentially ambiguous construct in a top-level declaration, and hence
it would only accept the record type if the variable name was followed
by something that looked like part of a correct declaration (e.g. a
comma).
The fix is to pass `true` for the optional parameter `required` of
`computeVariablePatternType`. This places `computeType` in a mode
where it accepts the record type regardless of what follows it. This
is correct behavior since at the point where a variable pattern is
being parsed, ambiguities have already been taken care of and the
construct being parsed is most definitely a variable pattern.
Fixes#52521.
Change-Id: If20772ad914827a29df45c27eefb382ec1f470d2
Bug: https://github.com/dart-lang/sdk/issues/52521
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305848
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Record parsing is generally done by accepting what comes after as
something that can follow a record type.
To support something like `List<List<(int, int)>>` we have to accept
`>>` after a record type.
When parsing something like `(whatever here) >> 42;` we thus say that
it's a record --- even if the stuff inside is a recovered type. We
inherit the `recovered` mark, but will say we're a record type.
This CL fixes parsing this by returning that it's _not_ a record type
when seing the `>>` (an a few others) if what's inside is recovered.
This _might_ mean worse recovery in some situations but fixes parsing
valid code such as `(() => print("hello")) >> 42;`.
Fix: https://github.com/dart-lang/sdk/issues/52365
Change-Id: Ie56958f5503e74d37c181c0b71003b8e61209c6c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303242
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
In rare circumstances involving syntax errors, the `parsePattern`
method inserts a synthetic token but does not consume any
tokens. Usually when this happens it's not a problem, because whatever
method is calling `parsePattern` consumes some tokens, so the parser
always makes progress. However, when parsing list patterns, after
calling `parsePattern`, the parser would look for a `,`, and if it
didn't find one, it would supply a synthetic `,` and call
`parsePattern` again, resulting in an infinite loop. A similar
situation happened with map patterns, though the situation was more
complex because in between the calls to `parsePattern`, the parser
would also create synthetic key expressions and `:`s.
To fix the problem, when parsing a list or map pattern, after the call
to `parsePattern`, the parser checks whether any tokens were
consumed. If no tokens were consumed, it ignores the next token from
the input stream in order to make progress.
I also investigated whether there were similar issues with
parenthesized/record patterns and switch expressions, since those
constructs also consist of a sequence of patterns separated by tokens
and other things that could in principle be supplied
synthetically. Fortunately, parser recovery doesn't get into an
infinite loop in those cases, so I didn't make any further
changes. But I did include test cases to make sure.
Fixes#52352.
Bug: https://github.com/dart-lang/sdk/issues/52352
Change-Id: Idc8140236f6054deb1fd3c862036fe47dd84f30b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302803
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
In https://dart-review.googlesource.com/c/sdk/+/299400, the parser was
adjusted so that it no longer accepts `when` and `as` as the names for
variable patterns in cases where there is a possible ambiguity
(e.g. `int when` is not accepted as a pattern because `int` is a
legitimate pattern, therefore `when` could introduce a guard
clause). This change further prohibits `when` and `as` from being the
names of variable patterns or identifier patterns even in the case
where there is no ambiguity. This is in line with the discussion at
https://github.com/dart-lang/sdk/issues/52199#issuecomment-1526297771,
and the spec change at
https://github.com/dart-lang/language/pull/3033.
Three new error codes are introduced, to cover the three circumstances
in which `when` or `as` might be used illegally: in a declared
variable pattern, in an assigned variable pattern, or in an identifier
pattern. I've also added analyzer tests to ensure that the parser
recovers from these errors nicely. Unfortunately, nice error recovery
is only feasible in the non-ambiguous cases.
I've also updated the language test expectations in
`tests/language/patterns/version_2_32_changes_error_test.dart` to
reflect the new error messages, and added a few more examples of uses
of `when` and `as` that are still permitted.
Fixes#52260.
Bug: https://github.com/dart-lang/sdk/issues/52260
Change-Id: I229f627aa639659c30b83c74895759207da279f7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301482
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
This change fixes parsing of case clauses such as:
case foo when !flag:
Constructions like this require some lookahead in order to parse
correctly, because the token `when` is valid both as an identifier and
as a part of the grammar for a case clause. Therefore, at the time
`foo` is encountered, the parser must decide whether it is looking at
a variable pattern (`foo when`, where `when` is the name of the
variable) or an identifier pattern (`foo`, where `when` begins the
case's guard clause). Previous to this fix, the algorithm for
disambiguating these two choices was as follows:
- If the token sequence starting at `foo` looked like a type, and the
token that follows was an identifier, the parser assumed it was
looking at a variable pattern with a type; otherwise it assumed it
was looking at an identifier pattern.
- EXCEPT that if the token that followed the supposed type was `when`
or `as` (both of which are valid identifiers), then it probed
further:
- If the token that followed `when` or `as` was a token that could
legitimately follow a pattern, then it assumed that it was looking
at a variable pattern with a type. (The tokens that could
legitimately follow a pattern are `,`, `:`, `||`, `&&`, `)`, `}`,
`]`, `as`, `when`, `?`, `!`).
- Otherwise it assumed that it was looking at an identifier pattern.
This didn't fully disambiguate, because the third bullet didn't
account for the fact that the tokens `as`, `when`, and `!` could
_either_ legitimately follow a pattern _or_ legitimately begin an
expression (or, in the case of `when`, a type), therefore constructs
like the following were incorrectly parsed:
- `case foo when as:` (where `as` is a local boolean variable)
- `case foo when when:` (where `when` is a local boolean variable)
- `case foo when !flag:` (where `flag` is a local boolean variable)
- `case foo as when:` (where `when` is the name of a type)
The solution is to simplify the disambiguation logic so that if if the
token that follows the supposed type is `when` or `as`, then the
parser assumes that it's looking at an identifier pattern, _not_ a
typed variable pattern.
The consequence of this is that the above four constructions are
parsed correctly; however it is no longer possible for a typed
variable pattern to name a variable `when` or `as`.
For consistency we would like to prohibit _any_ variable pattern from
naming a variable `when` or `as`, however to keep this change as small
as possible (and reduce the risk involved in a possible cherry-pick)
that will be postponed until a later CL.
Fixes#52199.
Bug: https://github.com/dart-lang/sdk/issues/52199
Change-Id: Ibab9b92f01e3e4020d7d64f1ff000a9b964a4564
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299400
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Jens Johansen