This change brings down core snapshot size by ~750KB, and brings
down app-jit snapshot size of simple "Hello, World" dart script by
~650KB. The bot cycle times will also come down by around ~20%.
Change-Id: I2a01c98bedc7ebfa2a653983995486a71504daf3
Reviewed-on: https://dart-review.googlesource.com/16323
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The kernel format has been designed so that one can concatenate several
dill files into one file and then load it. For instance the dart
function BinaryBuilder.readProgram supports this.
Currently a dill file contains one or more programs.
In the VM each of these programs are called either program or subprogram.
Technically a dill "program" isn't necessarily a program at all (e.g. it
could be missing a library).
This naming snafu should probably be cleaned up at some point, but that's
for another CL.
When loading a dill file via BinaryBuilder.readProgram what happens is
this:
- Each program in the dill file ends in 4 bytes that indicates the size
of the program.
- Reading the input from the end one can then read the size, skip back
that amount of bytes, if we have more data (i.e. there's another
program), read another size and so on, and continue until we have
accounted for all bytes in the input.
- We then read each program from the start, and basically overwrite any
library, class, procedure etc. we find.
The first main reference found is the one used though.
(Saying that we overwrite is not completely true, but when the library
is a non-external library that's basically what happens).
This CL introduces (some) support on the C++ side for the same thing.
So far the C++ side could only handle single-program-dills, and trying
to load anything else would probably crash the VM.
The support added is this:
- Assume the SDK (i.e. vm_platform.dill) is not a concatenated file
(error out if it is).
- For user provided input, loop over each contained program one-by-one,
for each individual one behave as normal.
- The way LibraryLoad is implemented (i.e. it skips if the library is
already loaded) this means that it currently would behave differently
than the dart version (i.e. the first one is used, not the last one).
For now it is assumed that that's not a problem.
- There is a possibly snafu if the same script is included several times.
This could probably mostly be remedied by not creating scripts up front,
but only as needed. By the "keep only one" (and fixing the above point,
probably by simply loading in the opposite order, i.e. last program
in the binary first) the (theoretical) problem would probably do away.
Note that we will have separate string tables, canonical name tables etc
per "sub program" and that there might be some duplication.
The implementation was tested as indicated below, but introduces no tests.
$ cat test_lib1.dart
import "test_lib2.dart" as lib2;
String lib1field = "lib #1 field!!";
main() {
foo();
lib2.foo();
print("From lib2: ${lib2.lib2field}");
}
foo() {
print("Hello, Foo, from test_lib1!");
var x = 42;
print(x);
}
$ cat test_lib2.dart
String lib2field = "Lib #2 field!!!!";
foo() {
print("Hello, Foo, from test_lib2!");
var y = 34;
print(y);
}
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart --packages=.packages --platform=out/ReleaseX64/vm_platform.dill test_lib1.dart
$ ls -lha test_lib1.dart.dill
[...] 4.2M Oct 26 14:42 test_lib1.dart.dill
$ dart pkg/kernel/bin/split.dart test_lib1.dart.dill
Wrote test_lib1.dart.dill.part1.dill
Wrote test_lib1.dart.dill.part2.dill
$ ls -lha test_lib1.dart.dill.part{1,2}.dill
[...] 811 Oct 26 14:42 test_lib1.dart.dill.part1.dill
[...] 582 Oct 26 14:42 test_lib1.dart.dill.part2.dill
$ cat test_lib1.dart.dill.part1.dill test_lib1.dart.dill.part2.dill > test_lib1.dart.dill.concat.dill
$ ls -lha test_lib1.dart.dill.concat.dill
[...] 1.4K Oct 26 14:44 test_lib1.dart.dill.concat.dill
$ out/ReleaseX64/dart --kernel-binaries=out/ReleaseX64 --packages=.packages test_lib1.dart.dill.concat.dill
Hello, Foo, from test_lib1!
42
Hello, Foo, from test_lib2!
34
From lib2: Lib #2 field!!!!
Change-Id: I233a033aa3042b202dd4708908a5be3089474588
Reviewed-on: https://dart-review.googlesource.com/16820
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This new object points to program wide data like string offsets,
string data etc. It also holds an array of pointers to all Script
objects corresponding scripts in the program's source table.
This new object type is required for two reasons:
1. The Script objects now have a number of fields which point to
program wide data. All Script objects point to the same data in the
VM heap. By introducing an indirection via this new object, we reduce
the number of pointers in Script objects.
2. Lazy loading of VM objects - Kernel nodes of
entities like fields and functions have a field which point to the
source file in which they are defined. This entry is an index into
the program wide source table and helps in associating
functions/fields with their actual source location. When lazy loading
functions and fields, the pre-loaded script objects in the
program's KernelProgramInfo help in associating the functions and
fields with the correct source script at load time.
Change-Id: Id863284ae7dd98b0832e5dfc115dabad1ed762d8
Reviewed-on: https://dart-review.googlesource.com/13920
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Devirtualization optimization now adds metadata to kernel AST instead
of transforming nodes to Direct* ones. The direct call metadata
provides information about checking receiver for null, while
Direct* kernel nodes do not support null checking.
VM's kernel binary loader is extended to extract arbitrary metadata
from kernel binaries and keep it for flow graph builder.
Kernel flow graph builder is extended to take direct call metadata
into account and generate CheckNull/StaticCall instructions
for devirtualized PropertyGet, PropertySet and MethodInvocation nodes.
Issue: https://github.com/dart-lang/sdk/issues/30480
Change-Id: I57f56fbf4a8981d33b1571c0d93105cf8ca71d76
Reviewed-on: https://dart-review.googlesource.com/12260
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Before this change, each function and field had its own kernel data blob
in the VM heap. With this change, the entire kernel data of a library is
stored as one single blob in the VM heap. Functions and fields store an
offset which points to the kernel data, specific to them, in that single
blob.
The pointer to the kernel data for a library is saved in two places:
1. With the library objects themselves.
2. With all the patch classes of the library.
3. With the patch classes created during hot reload.
Change-Id: Ie03e738c4d20f16056a5ef04341b75506fda9c60
Bug:
Reviewed-on: https://dart-review.googlesource.com/6601
Commit-Queue: Siva Chandra <sivachandra@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Before this CL we skipped procedure bodies in kernel_loader.cc by
parsing the body (but not storing anything).
With this CL we now skip them directly (i.e. don't read them at all)
in kernel_loader.cc by using the newly available extra indexes in kernel.
Change-Id: I48cf0599b2a85102c9008ff7c455785151ef3c9c
Reviewed-on: https://dart-review.googlesource.com/5764
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
- Weaken assert for identity reloads to account for lazy finalization.
- Store actual field end positions instead of computing from a terminating semicolon.
- Consider unfinalized classes to be unchanged if they have same sequence of tokens.
Change-Id: I3fcd7fed924bfac47dc382702ce63207bb8aa031
Reviewed-on: https://dart-review.googlesource.com/8164
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This adds more indexes to the kernel format so we know where classes
and procedures starts and stops. This allows for more random access.
E.g. one could now read the program index and jump directly to
library $i_1$, then read the library index and jump directly to class $i_2$,
read the class index and jump directly to procedure $i_3$.
The utilization (in this CL) is to not (always) read the procedure body
up front when loading kernel code on the dart side (ast_from_binary).
The observation is that - when running through the VM - almost none
of the bodies from the platform file are actually used.
This lowers the start-up cost which is noticeable for small programs
(e.g. hello world, or tests).
In this CL this is only done on the dart side and not on the C++ side,
that's for another CL.
Startup time:
dart2js: -1.84253% +/- 1.22157%
hello world: -11.1188% +/- 5.57892%
Running "time python tools/test.py -m release -cdartk language -j6":
real: -11.72% +/- 0.35%
user: -14.59% +/- 0.24%
sys: -12.71% +/- 0.61%
File size change (compiling with fasta to dill file incl. platform):
hello world: 0.88% (35,934 bytes).
dart2js: 0.97% (200,967 bytes).
Change-Id: I1f0ec121bc75bb17f11d3fade03da9815037d0bb
Reviewed-on: https://dart-review.googlesource.com/5262
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Originally these passes were a single confusingly named flow_graph_optimizer
pass, they later were completely split by duplicating flow_graph_optimizer
twice as jit_optimizer.cc and aot_optimizer.cc and tweaking them in few
places.
Duplication has been done in anticipation that these passes will diverge
considerably, however simple diffing reveals that in 2 years they have
not actually diverged much at all.
Diffing also reveals numerous bugs and inconsistencies between jit_optimizer
and aot_optimizer.
It does not make sense to keep these files duplicated so this change merges
them back together - extracting most of their common behaviour into
CallSpecializer base class.
The separation is not entirely clean - for simplicity we introduce
the knowledge about precompiled_mode into CallSpecializer base class.
You can look for FLAG_precompiled_mode and FLAG_use_field_guards in the
call_specializer.cc to see where base class has to be aware about both AOT and
JIT mode.
This change also renames XyzOptimizer to XyzCallSpecializer because the main
optimization these classes were doing was specialization of calls based on
type feedback or inferred types.
Bug: https://github.com/dart-lang/sdk/issues/30575
Change-Id: I0b062c4b7549d08b1bee9303d92a3fb549f54e21
Reviewed-on: https://dart-review.googlesource.com/3640
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
New folder structure (nested under vm/):
- compiler/
- jit/ - JIT specific code
- aot/ - AOT specific code
- backend/ - all middle-end and back-end code (IL, flow graph)
- assembler/ - assemblers and disassemblers
- frontend/ - front ends (AST -> IL, Kernel -> IL)
compiler/README.md would be the documentation root for the compiler
pipeline
Bug: https://github.com/dart-lang/sdk/issues/30575
Change-Id: I2dfd9688793bff737f7632ddc77fca766875ce36
Reviewed-on: https://dart-review.googlesource.com/2940
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
There already exists a class named "Reader" in the "kernel" namespace
which reads bytes from a kernel binary. The class KernelLoader
(KernelReader before this change) creates VM heap objects and populates
them.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/3010543002 .
Vyacheslav Egorov