Dart_LoadKernel is only used in Flutter engine and dart_runner, where it is always used as Dart_LoadKernel(Dart_ReadKernelBinary). These uses should be replaced with Dart_LoadScriptFromKernel.
Dart_LoadLibraryFromKernel is needed for loading split kernel files and implementing IsolateMirror.loadUri.
Change-Id: Ib505350eff53ec889406747f8f99393ebbdb4c7d
Reviewed-on: https://dart-review.googlesource.com/46220
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This CL changes:
* package:kernel/clone.dart: To not clone file offsets if the
file-uri we copy from is unknown.
* package:kernel/binary/ast_to_binary.dart: To not write out file
offsets if the file-uri is not written/indexed (which happens if
the file uri is not in the `Program.uriToSource` map).
* It reverts the file renaming part of 60a2cfa219 which
fixed the name collision problem only partially (it missed to
rename `runtime/lib/double.dart`, which is not possible because
there's already a `double_patch.dart` file).
Instead of renaming, we solve the problem by mangling the names as follows:
.../sdk/lib/core/double.dart -> dart:core/double.dart
.../runtime/lib/double.dart -> dart:core/runtime/lib/double.dart
.../runtime/lib/double_patch.dart -> dart:core/runtime/lib/double_patch.dart
* It changes the test to ignore any negative token positions. The vm
service does not just expose `-1` token positions for places where
there is no source, but also e.g. `-9` (which means TokenPosition::kMethodExtractor),
this was one reason why the appjit test failed.
* It changes the test to try to force-compile everything (unsure if that works).
Issue https://github.com/dart-lang/sdk/issues/32489
Change-Id: Ia9f42ca9d56d987e3041175add549bb7a133f269
Reviewed-on: https://dart-review.googlesource.com/46341
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This makes sure we can distinguish e.g. 2 different timeline.dart files
(from runtime/lib/timeline.dart and sdk/lib/developer/timeline.dart)
This CL also fixes the owner of constructors to be patch classes, if
the constructors come from a patch.
This CL also adds a service/valid_source_locations_test, which loops
over libraries/classes/fields/functions and obtains source locations for
them, including line numbers. This ensures that if there is a source
location attached to a member, we can use it's token position to get to
the line number.
This CL also changes package:kernel's [Cloner] to clone fileOffsets (and
not just fileEndOffsets). This is important for mixin resolution, where
we copy members into mixin application classes.
Issue https://github.com/dart-lang/sdk/issues/32489
Change-Id: I4fea5cd646d81f47e1c4ede1e86d477ba6de3e82
Reviewed-on: https://dart-review.googlesource.com/46141
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Static field initializers are not added as members of their owning class, so they must be pre-emptively given a patch class to avoid the meaning of their kernel/token position changing during a reload. Member functions and fields have their owner changed to patch class only when the class is changed during a reload.
Curiously, this was originally done in the C++ front end before hot reload: https://codereview.chromium.org//1317753004
Bug: https://github.com/dart-lang/sdk/issues/32497
Change-Id: I236c188a4b9e05a1e3c011d27e123979db1ca2e5
Reviewed-on: https://dart-review.googlesource.com/46124
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This simplifies figuring out what went wrong from just
debugging output in some cases.
Change-Id: Ie7a775c57fb691d6bd3ca1c70a04fb6cbe9e0875
Reviewed-on: https://dart-review.googlesource.com/43060
Reviewed-by: Jens Johansen <jensj@google.com>
While working on #32274 I noticed that #32087 also impacted debugging
experience in the external debuggers like VS Code - IDE attempts to
load non-existent local sources for core libraries, instead of
asking Service for their source.
To resolve this I moved work-around we placed in stack trace formatting
futher up the flow into kernel loader, which will now rewrite urls
for all core library scripts as classes are being created.
Bug: 32087
Change-Id: I2175a041e5fdcc9958bfccb2af724c37c2435f94
Reviewed-on: https://dart-review.googlesource.com/43021
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Writing "break Foo" in observatory with a file including
```
class Foo { }
```
would crash them VM.
This CL introduces a test that reproduces it and fixes the issue.
Bug:
Change-Id: I24780d9204117ade19a0cb590c45aa31b7c04f7e
Reviewed-on: https://dart-review.googlesource.com/30445
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
The "main" method can also be a getter, which this assertion doesn't
take into account. The assertion was just recently introduced in a
large CL and is not really necessary.
Revert "[gardening] Mark Language/Libraries_and_Scripts/Scripts/top_level_main_t05 crashing on debug dartk[p]"
Fixes https://github.com/dart-lang/sdk/issues/31389
Change-Id: I0fd45cfa5b885bb1942dcafe33afb922b7a75d40
Reviewed-on: https://dart-review.googlesource.com/21442
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
The introduced "constants" transformation can evaluate constant expressions. The
original use-sites of constant expressions are replaced by a new [ConstantExpression]
node, which points to a subclass of a new [Constant] class hierarchy. Constant
[Field]s and [VariableDeclarations]s will be removed, since all use-sites are
re-written.
The [Constant] class hierarchy is, similarly to the [DartType] class hierarchy, not
part of the AST tree (also has no parent pointer). The constants form a
DAG (directed acyclic graph).
There is no canonicalization requirement of the [Constant] objects referenced by the
AST (via [ConstantExpression]). Although it is beneficial to canonicalize them during
construction, since it reduces time spent in operator==/hashCode.
This CL furthermore adds support for a constant table in the binary format. Similarly
to [String]s, we canonicalize the constants before writing the table to the binary.
The constant table entries in the binary are written in a post-order way, to ensure
easy construction on the backend side.
The text format will be augmented with a "constants { ... }" section at the end,
which lists the constants in the same order as in the binary format.
The transformation can be used by those backends who choose to do so. It is not
enabled by default atm. It should therefore not affect analyzer, fasta or other
components.
Change-Id: I57cd9624fedcf537ab6870db76246149647bed21
Reviewed-on: https://dart-review.googlesource.com/14382
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
It was previously an array of Smi values. By converting to TypedData
array of int32_t values, we shave off 70+KB from the kernel based core
snapshots. While 70KB might not seem too high, it is 5% of the excess
size we have wrt to the token stream based core snapshots.
Change-Id: I5b58a1d1ac27391eca1efff179301bb04162e97b
Reviewed-on: https://dart-review.googlesource.com/19800
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This change brings down core snapshot size by ~750KB, and brings
down app-jit snapshot size of simple "Hello, World" dart script by
~650KB. The bot cycle times will also come down by around ~20%.
Change-Id: I2a01c98bedc7ebfa2a653983995486a71504daf3
Reviewed-on: https://dart-review.googlesource.com/16323
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The kernel format has been designed so that one can concatenate several
dill files into one file and then load it. For instance the dart
function BinaryBuilder.readProgram supports this.
Currently a dill file contains one or more programs.
In the VM each of these programs are called either program or subprogram.
Technically a dill "program" isn't necessarily a program at all (e.g. it
could be missing a library).
This naming snafu should probably be cleaned up at some point, but that's
for another CL.
When loading a dill file via BinaryBuilder.readProgram what happens is
this:
- Each program in the dill file ends in 4 bytes that indicates the size
of the program.
- Reading the input from the end one can then read the size, skip back
that amount of bytes, if we have more data (i.e. there's another
program), read another size and so on, and continue until we have
accounted for all bytes in the input.
- We then read each program from the start, and basically overwrite any
library, class, procedure etc. we find.
The first main reference found is the one used though.
(Saying that we overwrite is not completely true, but when the library
is a non-external library that's basically what happens).
This CL introduces (some) support on the C++ side for the same thing.
So far the C++ side could only handle single-program-dills, and trying
to load anything else would probably crash the VM.
The support added is this:
- Assume the SDK (i.e. vm_platform.dill) is not a concatenated file
(error out if it is).
- For user provided input, loop over each contained program one-by-one,
for each individual one behave as normal.
- The way LibraryLoad is implemented (i.e. it skips if the library is
already loaded) this means that it currently would behave differently
than the dart version (i.e. the first one is used, not the last one).
For now it is assumed that that's not a problem.
- There is a possibly snafu if the same script is included several times.
This could probably mostly be remedied by not creating scripts up front,
but only as needed. By the "keep only one" (and fixing the above point,
probably by simply loading in the opposite order, i.e. last program
in the binary first) the (theoretical) problem would probably do away.
Note that we will have separate string tables, canonical name tables etc
per "sub program" and that there might be some duplication.
The implementation was tested as indicated below, but introduces no tests.
$ cat test_lib1.dart
import "test_lib2.dart" as lib2;
String lib1field = "lib #1 field!!";
main() {
foo();
lib2.foo();
print("From lib2: ${lib2.lib2field}");
}
foo() {
print("Hello, Foo, from test_lib1!");
var x = 42;
print(x);
}
$ cat test_lib2.dart
String lib2field = "Lib #2 field!!!!";
foo() {
print("Hello, Foo, from test_lib2!");
var y = 34;
print(y);
}
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart --packages=.packages --platform=out/ReleaseX64/vm_platform.dill test_lib1.dart
$ ls -lha test_lib1.dart.dill
[...] 4.2M Oct 26 14:42 test_lib1.dart.dill
$ dart pkg/kernel/bin/split.dart test_lib1.dart.dill
Wrote test_lib1.dart.dill.part1.dill
Wrote test_lib1.dart.dill.part2.dill
$ ls -lha test_lib1.dart.dill.part{1,2}.dill
[...] 811 Oct 26 14:42 test_lib1.dart.dill.part1.dill
[...] 582 Oct 26 14:42 test_lib1.dart.dill.part2.dill
$ cat test_lib1.dart.dill.part1.dill test_lib1.dart.dill.part2.dill > test_lib1.dart.dill.concat.dill
$ ls -lha test_lib1.dart.dill.concat.dill
[...] 1.4K Oct 26 14:44 test_lib1.dart.dill.concat.dill
$ out/ReleaseX64/dart --kernel-binaries=out/ReleaseX64 --packages=.packages test_lib1.dart.dill.concat.dill
Hello, Foo, from test_lib1!
42
Hello, Foo, from test_lib2!
34
From lib2: Lib #2 field!!!!
Change-Id: I233a033aa3042b202dd4708908a5be3089474588
Reviewed-on: https://dart-review.googlesource.com/16820
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This new object points to program wide data like string offsets,
string data etc. It also holds an array of pointers to all Script
objects corresponding scripts in the program's source table.
This new object type is required for two reasons:
1. The Script objects now have a number of fields which point to
program wide data. All Script objects point to the same data in the
VM heap. By introducing an indirection via this new object, we reduce
the number of pointers in Script objects.
2. Lazy loading of VM objects - Kernel nodes of
entities like fields and functions have a field which point to the
source file in which they are defined. This entry is an index into
the program wide source table and helps in associating
functions/fields with their actual source location. When lazy loading
functions and fields, the pre-loaded script objects in the
program's KernelProgramInfo help in associating the functions and
fields with the correct source script at load time.
Change-Id: Id863284ae7dd98b0832e5dfc115dabad1ed762d8
Reviewed-on: https://dart-review.googlesource.com/13920
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Devirtualization optimization now adds metadata to kernel AST instead
of transforming nodes to Direct* ones. The direct call metadata
provides information about checking receiver for null, while
Direct* kernel nodes do not support null checking.
VM's kernel binary loader is extended to extract arbitrary metadata
from kernel binaries and keep it for flow graph builder.
Kernel flow graph builder is extended to take direct call metadata
into account and generate CheckNull/StaticCall instructions
for devirtualized PropertyGet, PropertySet and MethodInvocation nodes.
Issue: https://github.com/dart-lang/sdk/issues/30480
Change-Id: I57f56fbf4a8981d33b1571c0d93105cf8ca71d76
Reviewed-on: https://dart-review.googlesource.com/12260
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Before this change, each function and field had its own kernel data blob
in the VM heap. With this change, the entire kernel data of a library is
stored as one single blob in the VM heap. Functions and fields store an
offset which points to the kernel data, specific to them, in that single
blob.
The pointer to the kernel data for a library is saved in two places:
1. With the library objects themselves.
2. With all the patch classes of the library.
3. With the patch classes created during hot reload.
Change-Id: Ie03e738c4d20f16056a5ef04341b75506fda9c60
Bug:
Reviewed-on: https://dart-review.googlesource.com/6601
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Before this CL we skipped procedure bodies in kernel_loader.cc by
parsing the body (but not storing anything).
With this CL we now skip them directly (i.e. don't read them at all)
in kernel_loader.cc by using the newly available extra indexes in kernel.
Change-Id: I48cf0599b2a85102c9008ff7c455785151ef3c9c
Reviewed-on: https://dart-review.googlesource.com/5764
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
- Weaken assert for identity reloads to account for lazy finalization.
- Store actual field end positions instead of computing from a terminating semicolon.
- Consider unfinalized classes to be unchanged if they have same sequence of tokens.
Change-Id: I3fcd7fed924bfac47dc382702ce63207bb8aa031
Reviewed-on: https://dart-review.googlesource.com/8164
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This adds more indexes to the kernel format so we know where classes
and procedures starts and stops. This allows for more random access.
E.g. one could now read the program index and jump directly to
library $i_1$, then read the library index and jump directly to class $i_2$,
read the class index and jump directly to procedure $i_3$.
The utilization (in this CL) is to not (always) read the procedure body
up front when loading kernel code on the dart side (ast_from_binary).
The observation is that - when running through the VM - almost none
of the bodies from the platform file are actually used.
This lowers the start-up cost which is noticeable for small programs
(e.g. hello world, or tests).
In this CL this is only done on the dart side and not on the C++ side,
that's for another CL.
Startup time:
dart2js: -1.84253% +/- 1.22157%
hello world: -11.1188% +/- 5.57892%
Running "time python tools/test.py -m release -cdartk language -j6":
real: -11.72% +/- 0.35%
user: -14.59% +/- 0.24%
sys: -12.71% +/- 0.61%
File size change (compiling with fasta to dill file incl. platform):
hello world: 0.88% (35,934 bytes).
dart2js: 0.97% (200,967 bytes).
Change-Id: I1f0ec121bc75bb17f11d3fade03da9815037d0bb
Reviewed-on: https://dart-review.googlesource.com/5262
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Originally these passes were a single confusingly named flow_graph_optimizer
pass, they later were completely split by duplicating flow_graph_optimizer
twice as jit_optimizer.cc and aot_optimizer.cc and tweaking them in few
places.
Duplication has been done in anticipation that these passes will diverge
considerably, however simple diffing reveals that in 2 years they have
not actually diverged much at all.
Diffing also reveals numerous bugs and inconsistencies between jit_optimizer
and aot_optimizer.
It does not make sense to keep these files duplicated so this change merges
them back together - extracting most of their common behaviour into
CallSpecializer base class.
The separation is not entirely clean - for simplicity we introduce
the knowledge about precompiled_mode into CallSpecializer base class.
You can look for FLAG_precompiled_mode and FLAG_use_field_guards in the
call_specializer.cc to see where base class has to be aware about both AOT and
JIT mode.
This change also renames XyzOptimizer to XyzCallSpecializer because the main
optimization these classes were doing was specialization of calls based on
type feedback or inferred types.
Bug: https://github.com/dart-lang/sdk/issues/30575
Change-Id: I0b062c4b7549d08b1bee9303d92a3fb549f54e21
Reviewed-on: https://dart-review.googlesource.com/3640
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
New folder structure (nested under vm/):
- compiler/
- jit/ - JIT specific code
- aot/ - AOT specific code
- backend/ - all middle-end and back-end code (IL, flow graph)
- assembler/ - assemblers and disassemblers
- frontend/ - front ends (AST -> IL, Kernel -> IL)
compiler/README.md would be the documentation root for the compiler
pipeline
Bug: https://github.com/dart-lang/sdk/issues/30575
Change-Id: I2dfd9688793bff737f7632ddc77fca766875ce36
Reviewed-on: https://dart-review.googlesource.com/2940
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
There already exists a class named "Reader" in the "kernel" namespace
which reads bytes from a kernel binary. The class KernelLoader
(KernelReader before this change) creates VM heap objects and populates
them.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/3010543002 .
Ryan Macnak