This CL allows for streaming big parts of the binary,
i.e. without using the AST nodes.
It is thus a stepping-stone in getting rid of the AST nodes in the VM.
Generally, all Expressions except "FunctionExpression",
and all Statements except "FunctionDeclaration" can be streamed.
There are currently not streamed because they create new functions,
which has a pointer to an AstNode (which we don't have when streaming).
Once we no longer need AstNodes at all these can be streamed as well.
This is, I think, mostly a matter of streaming the ScopeBuilder as well,
something that is not currently done.
The way the streaming is build, one has to stream an entire subtree.
That means, that if an expression (or statement), A, that is generally
streamable contains an expression or a statement, B, that is not streamable,
A cannot be streamed.
The way this is build is by marking AstNodes as streamable or not
("cannot_stream_" field). That way we know up front whether we can stream
a subtree or not.
The streaming is done via "kernel_binary_flowgraph".
In this file there are many obvious comments, e.g.
```
TokenPosition position = ReadPosition(); // read position.
```
This has been done in an attempt to add a comment to everything that
reads from the binary to make it stand out more.
All changes from kernel_to_il up to and including May 2nd 2017
should be included.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2854393002 .
1. A --platform flag is added to dart to give a path to a Kernel
binary for the platform libraries (as produced by building the
runtime_kernel target).
2. This binary is used for bootstrapping. Since it contains libraries
other then the VM's bootstrap libraries, they are also loaded.
3. The frontend does not send any library with a dart: import URI
scheme. Note that it does not (yet) prune the canonical name
table, which will contain a lot of unnecessary names used for
internal linkages in the platform libraries.
4. There is a single dependency in the platform libraries on the
script: _getMainClosure in dart:_builtin. This is patched after
the script is loaded.
BUG=
R=ahe@google.com, kustermann@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2786083002 .
Instead of a list of strings given by their length and UTF-8 encoding,
restructure the string table to consist of a list of string ending
offsets followed by a blob of UTF-8 encoded strings without lengths.
This is a step toward copying the string blob into the VM's heap and
building a heap-allocated structure to give random access to them.
BUG=
R=asgerf@google.com, jensj@google.com
Review-Url: https://codereview.chromium.org/2790073004 .
A new AST node 'ClosureCreation' is added. It takes a name of a
top-level function, a context, and a closure function type and creates a
closure of the given type. The effect of this closure invocation is the
same as of the invocation of the given top-level function with the
contexts as the first argument.
In order to use 'ClosureCreation', the closure conversion pass now
transforms closures into top-level functions, not closure classes. These
functions receive the context as the first argument.
The type of the expression represented by 'ClosureCreation' is its third
parameter. It its the responsibility of closure conversion pass to
create the correct types for 'ClosureCreation' nodes based on types of
closures transformed into top-level functions.
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2778223002 .
There are four operations that work on Vectors: Vector creation, looking
up an item in a Vector, assigning a value to an item in a Vector, and
copying a Vector. The first three operations are allowed to only use
integer literals as number operands (length for Vector creation, index
for item lookup and assignment). Corresponding AST nodes are created for
these operations.
Vectors are used to represent contexts in Closure Conversion. The parent
context is stored as item 0 in its children contexts. The "golden" tests
for this transformation are adjusted accordingly.
The support for Vectors is added to ast-to-text, ast-to-binary, and
binary-to-ast transformations.
R=asgerf@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2767773004 .
- Offsets on more stuff
- Status file for kernel
-> Testing with (before fasta)
```
python tools/test.py -mrelease -cdartk --no-tree-shake --no-dfe service
```
now shows everything as passing.
Reload stuff, and evaluation stuff (e.g.) has been disabled though.
- Lots of new service tests that tests what positions we stop at when
saying next.
As fasta has now landed debugging does not currently work, but this is still needed in order to both test better and allow for proper kernel debugging support.
R=hausner@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2680303002 .
This adds a class CanonicalName that can represent a library, class,
or member. All references now go through a Reference object, which is
linked to both the AST node and its CanonicalName, so either can be
created first.
dartk now accepts multiple input files:
- If multiple dart files are given, they are all compiled.
- If multiple binaries are given, they are linked together.
Mixed dart and binary input is not supported by dartk.
dartk now has a flag --include-sdk which includes the entire SDK in
the output. This is so the SDK can be compiled alone and then linked.
Example of compiling separately and then linking:
dartk foo.dart -o foo.dill
dartk main.dart -o main.dill
dartk --include-sdk -o sdk.dill
dartk main.dill foo.dill sdk.dill --target=vm --link -o program.dill
dartk still has incredibly slow cold start due to the analyzer loading
the dart sdk, so this does not actually speed things up at the moment.
BUG=
R=ahe@google.com, kmillikin@google.com, kustermann@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2665723002 .
Previously a functions debuggable field was set to false in the async transformation (and because it wasn't done yet, not in async* and sync* transformation, though it should have been). Now instead, set the original async status and set debuggable on the c++ side equivalently.
Also, on the c++ side, set a functions modifier to async etc if they were originally async etc.
BUG=
R=asgerf@google.com, kmillikin@google.com, kustermann@google.com
Review-Url: https://codereview.chromium.org/2697193008 .
When a class from a different build unit is mixed in, the instance
members of the mixed-in class are retained in the external library
definition, so the mixin resolution pass can clone them.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2669303002 .
Added offsets to more nodes.
Added end offsets to some nodes.
Added functionnode debuggability flag.
This changes the dill format.
The new offsets et al. are read on the C++ side, but not used
for anything usefull yet.
This is step #2 in introducing these things, next step(s) will be
using it on the C++ side.
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2626613002 .
- For now include source uncompressed.
- When running from kernel, use token position 0
(i.e. dummy, but 'real' position) as start and end on functions
and classes to enable Observatory to run with the dill file.
- Debugging does not work, but one can browse the source in
Observatory.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2587673004 .
This is the result of:
- taking the diff of the branch closure_conversion to master in the kernel
repository
- updating the file paths
- applying the diff to the Dart SDK
- fixing conflicts between the changes to pkg/kernel in the Dart SDK and the master branch in the kernel repository
R=asgerf@google.com
Review-Url: https://codereview.chromium.org/2561723003 .
FunctionType now has a List<NamedType> which must be sorted by name.
Previously, named parameters were stored in a Map<String, DartType>.
FunctionNode still has a List<VariableDeclaration>, but this list must
now be sorted by name.
BUG=
R=ahe@google.com
Review URL: https://codereview.chromium.org/2502343002 .
Since offsets and uris was added to the dill file, compiling to dill
without linking hasn't worked because of external libraries.
The external libraries doesn't add to the line starts table and thus
the uris from libraries, classes and so on isn't recorded.
When trying to write an unrecorded uri to the dill file it fails.
The simple solution applied here is to fall-back to the empty
string when the uri hasn't been recorded.
BUG=
R=asgerf@google.com
Review URL: https://codereview.chromium.org/2483243002 .
BlockExpression allowed statements to appear in an expression context.
It is unsafe for its original use case, allowing 'yield' to appear as an
expression, because the VM's yield implementation does not work when
there are live expression intermediate values.
Eliminate it by changing the async ExpressionLifter to return a
transformed expression and emit statements into a given list as a side
effect. This requires implementing the transformation for all
statements to account for how they may contain a nested expression.
BUG=
R=asgerf@google.com
Review URL: https://codereview.chromium.org/2460373002 .
Type annotations in the supertype clauses of a class are now Supertypes
instead of InterfaceTypes. In the current version, the two classes
contain the same information, but they are about to diverge in the
following ways:
- An InterfaceType may be nullable, whereas a supertype cannot.
- An InterfaceType may represent the exact class, a subclass, or
a subtype of the given class.
- The type arguments to an interface type represent bounds, whereas
the arguments to a supertype are always exact.
We also introduce a class Substitution that represents an operator
that replaces type parameters with types, depending on the variance
of their use site.
A substitution can be applied to a DartType or a Supertype, and can
be generated independently of how it will be applied.
BUG=
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2439043002 .
The concept of a binary library file no longer exists.
A kernel file can contain any number of libraries, and some of these
libraries can be "external". To reference a class or member from
another build, the class or member must be declared in an external
library.
Members in an external library contain all their type information,
but have no body.
Classes in an external library have their hierarchy information
present, but are not guaranteed to contain all their actual members.
The idea is that references themselves don't really cross module
boundaries, but rather refer to a local definition whose body is
contributed from elsewhere, much like 'external' members in Dart.
A modular backend such as DDC should be able to compile from one of
these kernel files without needing to load auxiliary information from
summaries or other kernel files.
For whole program transformations or backends, a linking step, which
is not yet implemented, must merge classes and members in external
libraries based on their name.
External libraries share the same IR and binary format as ordinary
libraries. Transformations that affect the interface for a member
or class should transform the external libraries alongside with the
internal ones, ideally without needing to treat them any different.
R=kmillikin@google.com
Review URL: https://chromereviews.googleplex.com/516847013 .
For now the focus is on getting stacktraces to look right.
What that basically means is to include source positions for throws,
calls (in many forms).
Includes source file uri for libraries, field, classes and methods.
Methods and fields for instance needs it because of top-level
procedures that might be included via 'parts'.
Corresponding kernel-sdk change in review at
https://chromereviews.googleplex.com/516617014R=asgerf@google.com, kasperl@google.com
Review URL: https://chromereviews.googleplex.com/509247013 .
dartk and other command-line tools no longer rely on analyzer classes
like AnalysisContext or DartSdk, and instead use only the interface
defined in loader.dart.
The class AnalyzerLoader has been renamed to DartLoader.
The sharing of state in batch mode is now owned by loader.dart, so
dartk.dart does not deal with DartSdk and package_config.
Repository no longer has a package or SDK path. These concepts are
now specific to the Dart frontend. Consequently, loading a binary
file no longer relies on URI resolution.
The Repository class is now more clearly focused on linking symbolic
names to their IR objects, which for the time being is only
URI-to-library bindings, but should be extended to support linking
class and member references across different files and different
loaders.
BUG=
R=kmillikin@google.com
Review URL: https://chromereviews.googleplex.com/512907013 .
The interface target can now be stored on PropertyGet, PropertySet, and
MethodInvocation If set, we know the concrete target overrides or
implements that member.
All expressions have a method getStaticType for computing its type,
which relies on interface targets for the expressions that have one.
Expressions whose type is a least upper bound have the type stored
explicitly, so the definition of least upper bounds is contained only
in the frontend.
This is a work in progress towards strong mode support, it is still
not complete.
Still missing in the frontend:
- checks from implicit downcasts
- parameter checks from covariant override or covariant generics
Implemented but not part of this CL:
- subtype tests
- IR type checker (for debugging)
BUG=
R=kmillikin@google.com
Review URL: https://chromereviews.googleplex.com/496717014 .
This makes super calls name-based (no target) and add direct calls with
both receiver and explicit target.
Super call resolution translates the name-based super calls into direct
calls after they have been cloned into the mixin application.
For JS backends, it should suffice to clone mixed-in methods that
contain super calls, but that transformation is not part of this CL.
This also adds a --target option to dartk designating the target to
which the kernel IR should be specialized. The new transformation
is run when --target=vm.
BUG=
R=kmillikin@google.com
Review URL: https://chromereviews.googleplex.com/486537013 .
We need the ability to efficiently transform a mixin application class
into a regular class, and the separation was obstructing it. It also
simplifies the IR a bit, although it no longer enforces the invariant
that mixin applications cannot contain fields and procedures.
The binary format is unchanged, so the separation still shows up in
there. This ensures the CL can land without any changes to the SDK.
BUG=
R=vegorov@google.com
Review URL: https://chromereviews.googleplex.com/488407013 .
Konstantin Shcheglov