* add new SyntheticSymbolToken class
* whenever fasta inserts a closer ( '}', ']', ')' ) that is not part
of the original source, ensure that the inserted closer is
synthetic (isSynthetic returns true) and in the token stream
* make EOF a synthetic token
R=ahe@google.com, scheglov@google.com
Review-Url: https://codereview.chromium.org/2777153002 .
* cleanup defaultRecoveryStrategy next/previous and eof
* add SymbolToken.eof() constructor to ensure that EOF tokens
are correctly self referencing
* add fasta.scanString method to match scan method
* add fasta.ScannerResult.hasErrors field
so that scan and scanString callers can quickly check for errors
* new scanner replacement test
which scans using fasta then asserts as if it were from analyzer
* improve SymbolToken.toString() for EOF
* update analyzer test :: fasta scanner considers "<" to be an opener
* extract and move translateErrorToken into front_end
* address comments in https://codereview.chromium.org/2763833002R=ahe@google.com, paulberry@google.com
Review-Url: https://codereview.chromium.org/2767083002 .
* fasta.BeginGroupToken implement analyzer.BeginToken
* update token translator initialization of head and tail EOF
* update fasta.Token value() and type accessor to match analyzer
R=paulberry@google.com
Review-Url: https://codereview.chromium.org/2762103003 .
* update the fasta scanner to generate comment and dartdoc tokens
* replace fasta.preceedingComments field
with preceedingCommentTokens field and preceedingComments getter/setter
* update token translator
R=paulberry@google.com
Review-Url: https://codereview.chromium.org/2763833002 .
This updates PrecedenceInfo to implement TokenType
so that fasta.Token.type can return info for the token type.
In addition, the TokenType consts are now reference the corresponding
PrecedenceInfo consts so that parsing using either scanner will result
in token types that are identical.
R=ahe@google.com, paulberry@google.com
Review-Url: https://codereview.chromium.org/2758113002 .
This CL moves many of the files from
pkg/front_end/lib/src/fasta/analyzer/ to pkg/analyzer/lib/src/fasta/.
It also moves two files from pkg/kernel/lib/analyzer/ to
pkg/analyzer/lib/src/kernel/.
This reduces the amount of circularity between analyzer and
front_end/kernel so that there are no files in front_end or kernel
which are both dependend upon by analyzer and depend upon analyzer. I
will clean up the remaining circularities in future CLs.
There should be no functional change.
R=ahe@google.com, asgerf@google.com, kmillikin@google.com, scheglov@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2756593004 .
This CL adds checks to ensure that the assertions in
subpackage_relationships_test.dart aren't too generous. For example,
if a subpackage is allowed to import analyzer, we now check that it
actually does import analyzer.
R=danrubel@google.com
Review-Url: https://codereview.chromium.org/2748073007 .
There are still failing tests because we cannot access the comma between
the exception and the stack trace parameters. I will make a change
to Analyzer AST to remember commas in parameters, and send another CL
to use it in AST builder.
R=paulberry@google.com, ahe@google.com
BUG=
Review-Url: https://codereview.chromium.org/2752543005 .
These utilities are needed for generating the code that supports
summaries; in order to move summary logic into front_end, we will need
these code generation tools in front_end as well.
Also make a minor change to summary code generation logic so that it
talks to the file system directly rather than depending on analyzer's
file system abstraction.
R=brianwilkerson@google.com
Review-Url: https://codereview.chromium.org/2742333005 .
This class will allow parser error recovery to insert synthetic tokens
into the token stream. It makes use of Token.previousToken when it's
available, but does not depend on it; when it's not available, it
falls back on heuristics which should still be fast, and which don't
impose any performance burden until the first synthetic token is
inserted.
R=danrubel@google.com
Review-Url: https://codereview.chromium.org/2732193007 .
Konstantin Shcheglov