The entire scanner/parser now no longer uses `optional` or `optional2`,
the latter being completely removed.
Furthermore, when compiling a fixed version of the CFE with the CFE
this CL reduces the instructions used by ~13.6 mio.
```
page-faults:u: -0.3695% +/- 0.0676% (-384.20 +/- 70.23)
instructions:u: -0.0645% +/- 0.0007% (-13671097.80 +/- 153289.78)
```
Change-Id: Ib93dd2983aa1b4df0610d099010ac10f55f357f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/388323
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Short explanation: For whatever reason, when using `identical` on `int`s
the ints are first boxed (`BoxInt64`) before being compared
(`StrictCompare`) whereas just doing `==` just does a compare
(`EqualityCompare`).
Results:
With the CFE compiling (a fixed version of) itself I get these results:
```
instructions:u: -0.5756% +/- 0.0003% (-124825401.80 +/- 64013.17)
```
i.e. almost 125 mio instructions saved.
Another run - with 100 iterations each - I get
```
msec task-clock:u: -0.4927% +/- 0.2585% (-20.85 +/- 10.94)
page-faults:u: 0.0174% +/- 0.0139% (18.80 +/- 15.00)
cycles:u: -0.5233% +/- 0.2683% (-91305451.82 +/- 46815747.30)
instructions:u: -0.5754% +/- 0.0002% (-124793061.49 +/- 37426.30)
branch-misses:u: -1.6903% +/- 1.1207% (-1091410.69 +/- 723627.04)
seconds time elapsed: -0.4863% +/- 0.2581% (-0.02 +/- 0.01)
seconds user: -0.4547% +/- 0.3253% (-0.02 +/- 0.01)
```
In the scanner benchmark with `--string` (i.e. using string scanner) I
get these results:
```
msec task-clock:u: -3.7992% +/- 0.3316% (-190.54 +/- 16.63)
cycles:u: -4.1423% +/- 0.3566% (-836808313.28 +/- 72033424.19)
instructions:u: -3.3524% +/- 0.0000% (-1480262370.08 +/- 828.58)
branch-misses:u: -1.7591% +/- 0.9582% (-1781144.28 +/- 970258.82)
seconds time elapsed: -3.7988% +/- 0.3303% (-0.19 +/- 0.02)
seconds user: -4.0211% +/- 0.4161% (-0.19 +/- 0.02)
```
(Just running the benchmark also sees the characters/µs go from ~93 to
~97).
In the scanner benchmark with `--bytes` (i.e. using the utf8 scanner) I
get these results:
```
msec task-clock:u: -4.2872% +/- 0.4467% (-185.64 +/- 19.34)
cycles:u: -4.2972% +/- 0.4382% (-812955454.92 +/- 82892232.23)
instructions:u: -3.4867% +/- 0.0000% (-1479744935.28 +/- 297.12)
seconds time elapsed: -4.2872% +/- 0.4470% (-0.19 +/- 0.02)
seconds user: -4.2204% +/- 0.4730% (-0.18 +/- 0.02)
```
(Just running the benchmark also sees the bytes/µs go from ~108 to ~113).
In both cases we notice how the actual time, cycles and instructions
agree pretty well.
Combining the data for the compile and the benchmark I assume this CL
actually reduces the runtime of the CFE compiling itself by a about
half a percent.
Change-Id: I67d056837240aef61b6707d02507ab4121b31715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/385940
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
This adds the enclosingDeclarationName to the beginMethod of the
parser listener.
This enables the removal BuilderFactory.currentTypeParameterScopeBuilder.
Change-Id: Ie2bec9432c20b8bdbd62a14e8a65c272179d4698
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383182
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
This changes the BodyBuilder to create FunctionTypeParameter instead
of FormalParameterBuilder for "parameters" in function types. This
avoids the creation of unnecessary VariableDeclaration nodes
function types.
The types of these VariableDeclarations where created before the type
variable scope was completed, introducing an artificial dependency on
the pending nullability computation.
Change-Id: Ie1203fa4c78a27f3e7a0dfad16725cbbde24a6a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381143
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Paul Berry <paulberry@google.com>
Fix bracketed references in doc comments that previously pointed to
nowhere, in the shared parser
(pkg/_fe_analyzer_shared/lib/src/parser/...) and scanner
(pkg/_fe_analyzer_shared/lib/src/scanner/...).
This is part of a larger effort to clean up _fe_analyzer_shared to the
point where the `comment_references` lint can be enabled.
Change-Id: I60c402d9b4df50208ed51587e71a0663e369d622
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/375221
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Work towards https://github.com/dart-lang/language/issues/2
The feature is well-specified at the issue, but I will also follow
up with a specification to check into the language repo.
This change implements the feature more-or-less from front to back
(because the back is very close to the front in this case :P; no
"backend" work in the VM, etc). Digit separators are made available
via a new experiment, `digit-separators`.
Care is taken to report a single error when an underscore appears in
an unexpected position (see new `separators_error_test.dart`).
Three test files are added:
* `separators_test.dart` is run with the experiment enabled, and has
no compile-time errors.
* `separators_error_test.dart` is run with the experiment enabled, and
has many compile-time errors.
* `separators_error_no_experiment_test.dart` is run with the
experiment _disabled_.
Change-Id: I7f1b1305d28b708b5ddf83f26188cd6e9ce3dd58
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/365181
Commit-Queue: Samuel Rawlins <srawlins@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Kevin Moore <kevmoo@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The [StackListener.importUri] property is problematic because it assumes
or implies that when a compilation unit is being parsed, the import URI
for the containing library is known. This might not be the case if a part file is read before the main library file.
Currently the [StackListener.importUri] is only used to detect whether
the current file is part of a `dart:` library, so the property is replace with [isDartLibrary] which does just that.
Change-Id: I2d2baf2fe20bb62fd1922864c0e5af95e8fd1ca7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/372084
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
The parser didn't support 'new' as identifier in this context.
The CL also adds the reporting of tear-offs as metadata for the CFE.
This was already handled by the analyzer.
Change-Id: I7ab5868fa83e5f216d0e7be7ae9cec4a2c865e80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364480
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Erik Ernst <eernst@google.com>
When the parser sends events to the listener some events has a pair of
parameters, `beginToken` and `endToken`. Most of these are constructed
in such a way that the `beginToken` is the first token and the
`endToken` is the last token in that construct. This is for instance the
case with `endClassDeclaration`.
It was, however, not the case for `endMetadata` where `endToken` instead
was the next token *not* in the metadata.
In this CL I've found a changed the following to point to the last token
in the construct instead of the next token not in the construct and
renamed the parameter where it made sense:
* `endAssert` --- and renamed `semicolonToken` (which only pointed to
a semicolon for statements) to `endToken`.
* `endAwaitExpression`
* `endConstLiteral` --- and renamed `token` to `endToken`
* `endConstructorReference`
* `endFieldInitializer` --- and renamed `token` to `endToken`
* `endForIn`
* `endForInBody` --- and renamed `token` to `endToken`
* `endForStatement`
* `endForStatementBody` --- and renamed `token` to `endToken`
* `endFunctionExpression` --- and renamed `token` to `endToken`
* `endInitializer` --- and renamed `token` to `endToken`
* `endInitializers`
* `endInvalidAwaitExpression`
* `endMetadata`
* `endSwitchCase`
* `endTopLevelDeclaration` --- and renamed `nextToken` to `endToken`
* `endWhileStatement`
* `endWhileStatementBody` --- and renamed `token` to `endToken`
* `handleNoConstructorReferenceContinuationAfterTypeArguments`
In the few places in listeners where these values were used I've mostly
updated to do e.g. `endToken.text!` to retain the current behavior.
Change-Id: I25495e160d1eec5c75bcf1313b512cd04bcb1533
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364322
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Sometimes when the parser is performing error recovery it skips tokens
until it finds the next `,` or `;`, to try to get back on track. This
is handled by the `Parser.findNextCommaOrSemicolon` method.
When this method encounters a `BeginToken` (i.e., `(`, `[`, `{`, or
`<`), it needs to skip to the matching end token (using the `endToken`
getter), so that it doesn't try to resume parsing inside some nested
structure.
However, sometimes there is no matching end token (either because the
user has failed to properly matched `()`, `[]`, or `{}`, or because
the `BeginToken` is `<`, which doesn't always have a matching `>`). To
avoid a crash when this happens, `Parser.findNextCommaOrSemicolon`
needs to treat an unmatched `BeginToken` like an ordinary token, and
just advance to the next token.
Fixes https://github.com/dart-lang/sdk/issues/54236.
Bug: https://github.com/dart-lang/sdk/issues/54236
Change-Id: Id208c7a46c9c00b69f7f460a638d59486ebaffea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339980
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
The analyzer/CFE shared error template framework requires name
substitutions to be non-empty. However, sometimes, due to parser error
recovery, an identifier token can be an empty string.
One of the circumstances in which this occurs is when parsing text
like the following:
final b = (final g, final ) = 55;
In this example, there is a parse error due to the fact that the user
is trying to declare variables inside a pattern assignment. But one of
those variables doesn't have a name yet, so the parser has created a
synthetic token for it whose name is an empty string.
To avoid a crash, we must supply some name to the error message
template, so the name we supply is `(unnamed)`.
Fixes https://github.com/dart-lang/sdk/issues/54178.
Bug: https://github.com/dart-lang/sdk/issues/54178
Change-Id: Iabf3263ee1f56a89d1a69bcd74a17a296d470b9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339661
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Includes a cleanup of the errors reported for duplicate instance field.
Previously, in addition to reporting the duplicate, we would report that
the field cannot be used because it is a duplicate and that the
duplicate field hasn't been initialized.
Closes#53467Closes#53320
Change-Id: Ifd94223500809f3be36a1345d9d0c409778eed8a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336885
Reviewed-by: Paul Berry <paulberry@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Previously the nominal variable builders were created in all cases and
then converted to structural if that was required. It induced some
unnecessary type substitutions and complicated the logic behind the
builders. This CL introduces NominalVariableBuilder and
StructuralVariableBuilder as two possible subclasses of
TypeVariableBuilderBase, and the appropriate builders are created from
start.
This is a follow-up to
https://dart-review.googlesource.com/c/sdk/+/312264
Change-Id: I4e40de158201462dcef96b75445b8f179e1f75b5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330081
Commit-Queue: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This passes the first token of these top level declaration directly
to the endX listener method.
In the CFE this is used to ensure that can handle the new modifiers in
the textual outline. Furthermore, support for extension types is added
and having an "unknown chunk" now results in an error. The latter should
help us keep the textual outline up-to-date wrt new features.
Change-Id: I813d6162b6cba0a2bf550ed33a6091abf9bf49f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/324702
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This changes the parser to always interpret `extension type` as the
start of an extension type declaration.
Recovery is added to handle missing identifier, primary constructor,
parameter constructor parameters, and extension type declaration body.
Change-Id: I98cac0b2641167c4708fa20b22f0de0a70336457
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321704
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Johnni Winther