This CL copies the kernel bodies for all functions and
fields into the VM heap. The function bodies in the VM
heap are then used when compiling the flowgraphs.
This theoretically means that the malloc'd data can be
freed and that snapshotting from kernel could possibly
work, though it hasn't been tested.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2972343002 .
Prior to this CL we carried around information about the containing class
and member, both of which was fetched by reading out-of-line in the binary
(i.e. while reading the current member, start reading something from the
parent member etc).
It had also required the introduction of extra fields in the kernel
binary file (dill file).
This CL cleans that up, by
a) Setting type parameters on functions as needed (in kernel_reader.cc)
b) Using the VM Class and VM Function to get the required information
(with a above the information is all available).
(in kernel_binary_flowgraph.cc.) This means that
c) We don't have to read the binary out-of-line (for TypeParameterType
to work at least), and that
d) We can remove the previously introduced extra fields from the
kernel binary file (dill file).
R=dmitryas@google.com, kmillikin@google.com
Review-Url: https://codereview.chromium.org/2973633002 .
Via kernel we create PatchClasses to make sure the correct script is set
for functions and fields. Currently we create a new PatchClass each time
we need one, but often an old one could easily be reused.
This CL implements a simple caching-scheme and often uses a cached version.
Because some are requested out-of-order "duplicate" PatchClasses can still
be created though.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2967593002 .
Currently we run execute "library.AddFieldMetadata",
"library.AddFunctionMetadata" etc for all fields and methods,
even if they do not contain any annotations
(which is what will eventually be extracted from it if ever used).
This CL changes that to only execute the "AddXMetadata" if there
are any annotations.
This saves the VM from creating strings and fields that aren't really
used anyway, while - as before - returning an empty array when
answering a request for metadata for something without any.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2979653002 .
Mostly stream kernel_reader, i.e. the code that sets up the libraries,
classes, methods etc.
Mostly because it still takes a "Program" ast node, and looks at the
"Library" ast nodes to get their kernel offset in the binary.
Currently the scripts (containing breakable points etc) are also created
from the ast nodes.
The rest is now streamed.
This also means that more ast visitors could be deleted.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2931813002 .
2. Get rid of _getMainClosure()
3. Adjust the AOT compiler to ensure it retains the function associated with
the main closure without having to include '_getMainClosure()' in the list
of embedder specified entry points
4. Get rid of the hack in kernel reader to do a delayed patch of
'_getMainClosure()' in the builtin library.
BUG=
R=aam@google.com, rmacnak@google.com
Review-Url: https://codereview.chromium.org/2933603002 .
- Put pointer to kernel data into Script.
- Replace function.kernel_function pointer to AstNode with
kernel_offset():
- Replace field.kernel_field pointer to AstNode with kernel_offest().
- Stream the previously unstreamed AstNodes: FunctionDeclaration and
FunctionExpression.
- Move special handling for _buildin.getMainClosure into the streaming
flowgraph builder.
- Delete big parts of kernel_to_il.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2901533002 .
This CL allows for streaming big parts of the binary,
i.e. without using the AST nodes.
It is thus a stepping-stone in getting rid of the AST nodes in the VM.
Generally, all Expressions except "FunctionExpression",
and all Statements except "FunctionDeclaration" can be streamed.
There are currently not streamed because they create new functions,
which has a pointer to an AstNode (which we don't have when streaming).
Once we no longer need AstNodes at all these can be streamed as well.
This is, I think, mostly a matter of streaming the ScopeBuilder as well,
something that is not currently done.
The way the streaming is build, one has to stream an entire subtree.
That means, that if an expression (or statement), A, that is generally
streamable contains an expression or a statement, B, that is not streamable,
A cannot be streamed.
The way this is build is by marking AstNodes as streamable or not
("cannot_stream_" field). That way we know up front whether we can stream
a subtree or not.
The streaming is done via "kernel_binary_flowgraph".
In this file there are many obvious comments, e.g.
```
TokenPosition position = ReadPosition(); // read position.
```
This has been done in an attempt to add a comment to everything that
reads from the binary to make it stand out more.
All changes from kernel_to_il up to and including May 2nd 2017
should be included.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2854393002 .
This transformation replaces the body of _getMainClosure in
dart:_builtin with `return main` for the script's main if there is one
and removes the body if there is no main.
Without this transformation fasta will compile `return null` and dartk
will compile `return throw new NoSuchMethodError...`. Both of these
patterns will be patched in the VM when dart:_builtin is compiled from
Kernel.
BUG=
R=sigmund@google.com
Review-Url: https://codereview.chromium.org/2861023002 .
In the VM's Kernel representation, introduce wrapper classes for
string and name indexes so it is obvious which one is which. For
convenience there is an implicit conversion so that they can each be
used where an int is allowed. However, there is no implicit
conversion _to_ either of these types.
BUG=
R=vegorov@google.com
Review-Url: https://codereview.chromium.org/2860823002 .
The canonical name table is copied into a typed data array in the VM's
heap. The encoding is the same as in the binary except that the
integer indexes are fixed-size.
Canonical names are now integer indexes instead of objects allocated
in the C++ heap.
BUG=
R=jensj@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2853423002 .
Copy the Kernel string offsets into a uint32 array in the VM's heap.
This avoids allocating small string objects with new and avoids having
a table of the canonical strings.
Instead of an offset and a size, strings are now represented as
indexes into the string table in the heap. The start offset of string
N is found at byte offset N*4 because it is a uint32, and the end
offset is found at byte offset (N+1)*4. The strings themselves are
just integer indexes instead of pointers.
In the stream flow graph builder, string access is all random access.
R=jensj@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2852943003 .
The front end tests run in a mode where the script is compiled to
Kernel, linked with the platform libraries, and written to disk as a
single binary containing both the script and platfrom libraries.
Ensure that this mode continues to work.
BUG=
R=asiva@google.com
Review-Url: https://codereview.chromium.org/2842913002 .
1. A --platform flag is added to dart to give a path to a Kernel
binary for the platform libraries (as produced by building the
runtime_kernel target).
2. This binary is used for bootstrapping. Since it contains libraries
other then the VM's bootstrap libraries, they are also loaded.
3. The frontend does not send any library with a dart: import URI
scheme. Note that it does not (yet) prune the canonical name
table, which will contain a lot of unnecessary names used for
internal linkages in the platform libraries.
4. There is a single dependency in the platform libraries on the
script: _getMainClosure in dart:_builtin. This is patched after
the script is loaded.
BUG=
R=ahe@google.com, kustermann@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2786083002 .
- Instead of a pointer to new'd memory, Kernel strings now have an
offset from the start of the string data.
- When the streaming reader encounters the string data it records the
offset from the start of the binary. This offset is stored in the
Kernel Program and is used to compute the offset for strings.
- When a KernelReader is constructed, the string data is copied into a
Uint8 array in the VM's heap.
- A pointer to the string data is put into every Kernel script so it
can be used for constructing VM strings at compile time.
The source table does not use Kernel strings any more because those
strings are not found in the raw string data. Instead, the source
table uses new'd buffers for strings (but this will be cleaned up
separately).
R=jensj@google.com
Review-Url: https://codereview.chromium.org/2820363002 .
Add function_type_arguments field in closure instances.
Lots of other smaller changes, also related to generic function semantics.
This is still work in progress, with a change of direction in the design:
The type argument vector of a generic function will be prepended with the type
arguments of enclosing generic functions. The re-allocation and concatenation
will be done in nested generic function's prolog. This will greatly simplify
instantiation of types at run time without having to search the context for
parent function's type arguments. However, a closure instance now requires an
additional field. On the other hand, type parameters do not require a
parent_level field anymore.
R=rmacnak@google.com
Review-Url: https://codereview.chromium.org/2818273002 .
Before: Canonical names contained pointers to the corresponding Kernel
tree which assumed that the whole tree was in memory whenever the
canonical names were.
Now: Canonical names do not contain these pointers. They were only
really used to perform name-based lookup in the VM's heap so the
canonical name itself is enough.
If we later find that we need to get from a canonical name to its
Kernel tree we can add an offset in the binary (for instance) to the
canonical name or in a separate mapping on the side.
BUG=
R=asgerf@google.com, jensj@google.com, vegorov@google.com
Committed: https://github.com/dart-lang/sdk/commit/ed77783cd32d55fdad61bf9bc749030847ba9384
Review-Url: https://codereview.chromium.org/2781893004 .
Before: Canonical names contained pointers to the corresponding Kernel
tree which assumed that the whole tree was in memory whenever the
canonical names were.
Now: Canonical names do not contain these pointers. They were only
really used to perform name-based lookup in the VM's heap so the
canonical name itself is enough.
If we later find that we need to get from a canonical name to its
Kernel tree we can add an offset in the binary (for instance) to the
canonical name or in a separate mapping on the side.
BUG=
R=asgerf@google.com, vegorov@google.com
Review-Url: https://codereview.chromium.org/2781893004 .
For compiling the platform libraries separately and for hot reload, we
will not link canonical names of Kernel libraries, classes, and
members to their Kernel definitions. Those definitions will not be
persistent.
As a first step, change name-based lookup of libraries and classes to
be be based on the canonical name instead of the Kernel definition.
BUG=
R=kustermann@google.com
Review-Url: https://codereview.chromium.org/2762213002 .
Don't create getters for uninitialized static fields.
This fixes two assert hits that was hit when running service tests in debug mode via kernel. These can thus now be enabled.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2771443003 .
Previously a functions debuggable field was set to false in the async transformation (and because it wasn't done yet, not in async* and sync* transformation, though it should have been). Now instead, set the original async status and set debuggable on the c++ side equivalently.
Also, on the c++ side, set a functions modifier to async etc if they were originally async etc.
BUG=
R=asgerf@google.com, kmillikin@google.com, kustermann@google.com
Review-Url: https://codereview.chromium.org/2697193008 .
When we were implementing bootstraping from Kernel in https://github.com/dart-lang/sdk/commit/23fd1a184b393825f29444243780f3593ce0b3c1 we switched ClassID.cid* fields to become 'static final' lazily initialized fields instead of constants as they were before. This was mainly done to allow dartk compile patched_sdk - because these fields were previously injected in runtime and never existed in the text form.
However this regressed code quality for app-jit and app-aot snapshots because 'static final' fields are reset by snapshotting so resulting code contains InitStaticField and LoadStaticField instructions.
BUG=
R=kustermann@google.com
Review-Url: https://codereview.chromium.org/2693863006 .
This introduces
- a list of valid token positions, so debugging actually
starts to work, and the observatory can be loaded without crash.
- a list of valid yield positions, so stepping over await stuff
works as expected.
- Adding "DebugStepCheckInstr" to the kernel generated il so stepping
over await stuff works as expected, we can break in empty methods etc.
With this, approximately 80% of the service tests pass in kernel mode.
R=kmillikin@google.com
Committed: https://github.com/dart-lang/sdk/commit/2d5147be9d7435c06dd892d145d8717b6d1f62d5
Review-Url: https://codereview.chromium.org/2632183002 .
This introduces
- a list of valid token positions, so debugging actually
starts to work, and the observatory can be loaded without crash.
- a list of valid yield positions, so stepping over await stuff
works as expected.
- Adding "DebugStepCheckInstr" to the kernel generated il so stepping
over await stuff works as expected, we can break in empty methods etc.
With this, approximately 80% of the service tests pass in kernel mode.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2632183002 .
The CL adds metadata support for Fields/Classes/Functions/Constructors. There
are other places where annotations can be put in the Dart language but the
Kernel IR currently does not store it in the AST.
Whenever a element (e.g. a function) gets annotated, the VM will make a [Field] for this
element whos value is an array of evaluated constant metadata annotations.
This CL attaches to these [Fields] a `kernel_function` which is then used to do the constant
evaluation.
R=vegorov@google.com
Review-Url: https://codereview.chromium.org/2632253002 .
Utilize the offsets added in previous CLs.
While this doesn't by it self give anything that resembles
a good debugging experience, it should now be possible to
set *some* breakpoints and actually break on them.
Because of the way observatory works (and because of an
unfinished Script::GenerateLineNumberArray (for kernel))
the formatting of the code you try to load up and 'debug'
has to be quite specific (e.g. no indentation at all),
and even then it mostly doesn't work.
This is step #3 in introducing these things, next step(s) will be
fixing stuff like the above.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2628693004 .
- The flow graph builder is only used to visit Statements and
Expressions so it does not need TreeVisitor as a base class. This
will make refactoring it easier.
- The TranslationHelper is bound to a Thread so we do not need
Thread::Current where we have a TranslationHelper.
- BlockExpression does not exist in Kernel any more, so it can be
removed from the C++ AST implementation.
BUG=
R=kustermann@google.com
Review-Url: https://codereview.chromium.org/2624513005 .
- For now include source uncompressed.
- When running from kernel, use token position 0
(i.e. dummy, but 'real' position) as start and end on functions
and classes to enable Observatory to run with the dill file.
- Debugging does not work, but one can browse the source in
Observatory.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/2587673004 .
- Kernel-to-IL translation leaves behind some unused constants without temp indices assigned.
- Let compile time errors propagate instead of failing with FATAL error in the post load class finalization.
R=kustermann@google.com
BUG=
Review URL: https://codereview.chromium.org/2528763002 .
E.g.
file1:
------------
library crash;
part "file2.dart";
main() {
print(field2);
}
crash() {
try {
throw "Crashing!";
} catch(exception, stacktrace) {
print(stacktrace);
}
}
file2:
------------
part of crash;
var field2 = crash();
We want something like
#0 crash (whatnot/file1.dart:11:5)
#1 field2 (whatnot/file2.dart:3:14) <-- good line
#2 field2 (whatnot/file2.dart:3:5)
#3 main (whatnot/file1.dart:6:9)
and not like
#0 crash (whatnot/file1.dart:11:5)
#1 field2 (whatnot/file1.dart:3:14) <-- bad line
#2 field2 (whatnot/file2.dart:3:5)
#3 main (whatnot/file1.dart:6:9)
R=kmillikin@google.com
Review URL: https://codereview.chromium.org/2519133004 .
Jens Johansen