Brian Wilkerson
f50c52f3dd
Change-Id: I8edb453f638223464ffb8563c8d4fe0dd6999991 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269360 Reviewed-by: Samuel Rawlins <srawlins@google.com> Commit-Queue: Brian Wilkerson <brianwilkerson@google.com>
13 lines
701 B
Markdown
13 lines
701 B
Markdown
# The Token stream
|
|
|
|
The token stream is a doubly linked list with special 'EOF' tokens at the
|
|
beginning and end. The EOF token at the end points to itself as the `next`
|
|
token, and the one at the beginning points to itself as the `previous` token.
|
|
So, for a correctly formed token stream (which we routinely assume will always
|
|
be the case) neither `previous` nor `next` can ever return `null`. We can't
|
|
express that in the type system because those fields will be `null` temporarily
|
|
while the list is being built. As a result, the convention throughout the code
|
|
base, is to suffix those two getters with a bang (`!`) everywhere they're
|
|
invoked.
|
|
|
|
Tokens are typically accessed through the [AST](ast.md). |