Commit Graph

30 Commits

Author SHA1 Message Date
Jens Johansen b5da327ed9 [parser] Parse and skip metadata according to spec
Make the parser handle metadata properly according to the spec.

There has been some confusion in regards to metadata because
https://github.com/dart-lang/language/blob/master/accepted/future-releases/small-features-21Q1/feature-specification.md
(used to) say we needed a spec update.

That was done in the parser in
https://dart-review.googlesource.com/c/sdk/+/182668
but really that made the parser adhere to the spec as it was which is
also correct (as the spec change wasn't needed at all).

The parser also needs to be able to skip metadata, though, and that
wasn't done it that CL which caused the issues in
https://github.com/dart-lang/sdk/issues/45120

The work directly in this CL is:
* Add lots of comments to parseMetadata to make them more up-to-date with
  the spec.
* Fix skipMetadata to correspond to parseMetadata.
* Tests verifying that 45120 is fixed in the parser and that metadata in
  general works as expected in the parser.

(In working on this, the future specification was corrected to say that the
spec did not need updating. Thanks to Erik for the quick turn-around
there).

Bug: #45120
Change-Id: Id1e4912cac7e5d9113efe083b3ecd06e6834a16e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190880
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2021-03-15 08:40:40 +00:00
Johnni Winther 63373b9e9c [cfe,analyzer] Pass message arguments directly to AnalysisError
This allows for supporting customized message for scanner/parser
errors.

Change-Id: Ib9470c3d8231de69611bc672570ae8902e9c0e6e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177940
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-11 16:45:26 +00:00
Jens Johansen 3a1f732c0f [parser] Make begin/end events always come in pairs
This CL:
* Changes the events so beginX and endX events always comes in pairs
  (though technically not right as some specific events can be beginX
  endY --- but in those cases it is at least documented and used in code
  that actually tests it).
  -> This entails adding some events and converting something from
    "beginX" to "handleX" instead.
* Adds a utility that can generate an AST of sorts directly from the
  parser via a listener using the begin/end matching (and knowing of the
  specific ones that doesn't match directly).
* Adds a test that checks that - at least for all tested (50,000+) files
  - the AST actually generate "correctly", i.e. matches up begin/ends
  and ends up with a single top entry "CompilationUnit".
* Adds a different visualization to the parser listener events by
  displaying the "AST directly from the parser" in a UI that can be
  navigated. The visualization may not be the best, but it's certainly
  a stepping stone.

Change-Id: I9b27f7bbf3be442adc92f357c7b3c46da6f84cf7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159664
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2020-09-01 09:30:43 +00:00
Jens Johansen 7e42cbf582 [parser] Change recovery of multiple extends with comma
E.g. class Foo extends Bar, Baz {} would before have parsed weirdly,
but it will parse as you'd (probably) expect with an error saying
you cannot do that. For now at least both CFE and Analyzer will
pretend like you just specified the first one but I suppose Analyzer
could for instance use the extra information to propose converting
other ones to implements clauses or similar.

Fixes https://github.com/dart-lang/sdk/issues/22313

Change-Id: I180cdd8ab07143dd74fd21c9976ec2a46c428d8e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158261
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-08-13 08:49:53 +00:00
Jens Johansen b6423f1f81 [CFE] Make parser_test_listener up-to-date
This also changes lots of expect files.
I don't know how it got out-of-date --- I actually thought that was
tested. It seems it's not...

Change-Id: I6dd524908b8735a5889ed6fdb59be5383e4e8237
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158301
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-08-12 14:28:54 +00:00
Johnni Winther a021b99fc3 [cfe] Fix scope setup for generic function types
Closes #36870
Closes #41951
Closes #42364

Change-Id: Id323bfdc26c688bfeb756ef8c1e0530283b44e34
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/156028
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2020-07-29 09:04:24 +00:00
Johnni Winther 92f4eb58c8 [cfe] Support abstract fields
Only positive implementation, no error states are checked yet.

Change-Id: Iad79da1dd7e8be9229ee9d47c5301d39ebc3debc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154003
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2020-07-16 10:13:23 +00:00
Paul Berry 8cb7dfa70f Modify parser to handle abstract fields.
There is no functional change to the analyzer or CFE yet; we simply
report the error in the parser listeners now rather than the parser
itself.  In follow up CLs when we add full support for the feature, we
will want to keep the error reporting logic, but only report the
errors in pre-NNBD code.

Change-Id: Ifd14dd97d7d1d57586a8e14a25103356d54f0ea1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154364
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-07-15 22:48:01 +00:00
Johnni Winther 9ab7a5edef [cfe] Use variable name in error message for missing initializer
Closes #42610

Change-Id: I70a5ac9302bd6545cd5e73f2dab61a29bc6a3000
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153462
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2020-07-08 12:45:15 +00:00
Paul Berry 3ca37fb66e Re-apply some recent parser fixes.
My recent revert (275b922ee8) mistakenly
reverted too much (I incorrectly attributed the angular failure to
1f9b5c1996 when it was actually due to
96df1f9e6c).  Reapply the commits that
are ok.

This re-applies commit 1f9b5c1996.
This re-applies commit c40b24da48.

Change-Id: I4d64b1511b92d87197a5e939903a31e01f0b2e70
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147920
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
2020-05-13 16:00:01 +00:00
Paul Berry 275b922ee8 Revert recent parser fixes
Commit 1f9b5c1996 causes tests to fail
in angular when rolled into internal sources; the other two commits
need to be reverted as well to avoid merge conflicts.

This reverts commit 1f9b5c1996.
This reverts commit c40b24da48.
This reverts commit 96df1f9e6c.

Change-Id: Ic4e55181ef60e825ce4409a9a9528ecbf19c39c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147822
Reviewed-by: Mike Fairhurst <mfairhurst@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
2020-05-12 22:02:09 +00:00
Johnni Winther d5650235a2 [cfe] Handle ProblemBuilder in binary expression
Closes #41787

Change-Id: I81fcf0aee975b58b3b44d0de94054ae87bdd265d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147140
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2020-05-12 11:44:19 +00:00
Jens Johansen c40b24da48 [parser] Show errors from 'handleErrorToken' in parser suite tests
Change-Id: I3528136b6c220f0fe72d9d7b2aa591737c4c7096
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146225
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-05-07 11:23:48 +00:00
Jens Johansen 5b76fc2172 [parser] Better recovery of methods/fields called 'const', 'final', 'var' or 'void'
Change-Id: I370089a6de8c585a60785fe069de5da869ae23f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145820
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-05-06 11:56:24 +00:00
Jens Johansen a840da4617 [parser] Tests of using build in identifier and reserved keywords as method/field names
Bug: https://github.com/dart-lang/sdk/issues/40419
Change-Id: Ide0fabc4466714a7f9bb9a4474c80a3dfafde56f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145584
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-05-01 07:40:46 +00:00
Jens Johansen 4ab3bda5db [parser] Change recovery for constructor; report errors in bad constructors
Change-Id: I3d656afce4a3ec14c7c099893a434abba3945ebf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140862
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-03-27 11:57:56 +00:00
Jens Johansen 3969356f1a [parser] Parse bangs before type parameters correctly
Change-Id: Ie05e57d12cdbdeac037785ef6b94497c77d264ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137789
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-03-02 11:21:40 +00:00
Jens Johansen 399c68e884 [parser] Parse chained calls with type arguments correctly
Change-Id: Ifaecd7f74c4e2d3d332fc759f5e2ae55de47b8a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137784
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-03-02 10:28:47 +00:00
Jens Johansen 1b534d04a8 [CFE] Change parser test output slightly
Special handling of beginBinaryExpression/endBinaryExpression as they
don't always comes in pairs.

Change-Id: I74323dd45615fbb340acc1c5144b0ba94ea19f5a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137783
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-03-02 08:22:03 +00:00
Jens Johansen 3660b9ecc7 [parser] New syntax for null aware index (2nd try)
This reverts commit bc4893a3b3.

The previous attempt failed because when we check if something is a
conditional expression we used a rewriter that didn't actually rewrite.
This further down the line (still checking if we can parse the expression)
results in errors occurring (that wouldn't have occurred if the rewrite
had happened) and the conclusion was thus that it wasn't a conditional
expression and we parsed it as a null-aware index operator (which gave
errors because it wasn't).

This 2nd try solves it by having a rewriter that actually rewrites,
but then allows to undo the rewrites after we're done parsing.

Change-Id: Ide3c0d6a870d4dc18b6d9475b0227d93c733dc7f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136624
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2020-02-26 13:31:52 +00:00
Jens Johansen bc4893a3b3 Revert "[parser] New syntax for null aware index"
This reverts commit cf4e044e07.

Reason for revert: Breaks internal customer.

Original change's description:
> [parser] New syntax for null aware index
> 
> Change-Id: Iea6b29eb897eb01863a92b223e270d225c3ca1ba
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135903
> Commit-Queue: Jens Johansen <jensj@google.com>
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>

TBR=brianwilkerson@google.com,jensj@google.com,johnniwinther@google.com

Change-Id: Ic0ff607c944e489f29bda7da37afd63c62d368eb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136600
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2020-02-20 14:11:08 +00:00
Jens Johansen cf4e044e07 [parser] New syntax for null aware index
Change-Id: Iea6b29eb897eb01863a92b223e270d225c3ca1ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135903
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2020-02-20 12:33:28 +00:00
Johnni Winther f521734ce7 [cfe] Handle setter import conflicts
Fixes the underlying problem for the crash reported in
https://github.com/flutter/flutter/issues/46823

Change-Id: I392c7bae3679cae52b694f3bd08b0b700ee9c56f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128540
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2019-12-16 12:02:47 +00:00
Karl Klose bd06f1bf51 [cfe/infra] Unit test suites: print re-run command when a test fails
Closes #39423

Change-Id: Ifa3fbe4d670f4f95beceb2aae8769a553a9fbe7b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125604
Commit-Queue: Karl Klose <karlklose@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2019-11-22 10:45:54 +00:00
Johnni Winther d76ffbc149 [cfe] Handle assigned variables in for and for-in
Change-Id: I06deda7f017dcd40e5c6987c61261cf3be2be0ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123410
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2019-10-31 11:36:43 +00:00
Jens Johansen fa05fedd89 [CFE] Parser test suite prints messages prettily at top of expect file
Change-Id: I9af28f57f8ab54b38e34b06019000b720e5fdecb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123524
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2019-10-30 14:25:54 +00:00
Johnni Winther c8ff1b0c21 [cfe] Add BlockKind to parse/begin/endBlock
... and use this to computed AssignedVariables data for try statements

Change-Id: I235575b055cb89b553baf0ac728d9954d8d0aaa6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123251
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2019-10-29 09:17:06 +00:00
Johnni Winther 688454c4a7 [cfe] Compute AssignedVariables in body builder
For-loops and try statements currently not supported.

Change-Id: I52a591b55149ddfd5d0c5792239555d72b8146fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123248
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2019-10-29 09:17:06 +00:00
Jens Johansen 0f7343e1a9 [parser] Reproduce analyser crash _OperatorName in endClassConstructor
This will crash the analyser:

class operator {
  operator^(int x) {}
}

The output from the parser looks reasonable to me though.

Change-Id: I956daf43670decca979d9dad6614e139007cda26
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121321
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2019-10-28 14:00:37 +00:00
Jens Johansen 5b7c981086 [CFE] Parser test framework also outputs token stream
This makes it much easier to see what the parser is actually working
with in the face of token stream rewrites.

Change-Id: Ic336f18fab7c8fd7a42020b32af4a480bbde25bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122865
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2019-10-28 10:25:30 +00:00