The kernel format has been designed so that one can concatenate several
dill files into one file and then load it. For instance the dart
function BinaryBuilder.readProgram supports this.
Currently a dill file contains one or more programs.
In the VM each of these programs are called either program or subprogram.
Technically a dill "program" isn't necessarily a program at all (e.g. it
could be missing a library).
This naming snafu should probably be cleaned up at some point, but that's
for another CL.
When loading a dill file via BinaryBuilder.readProgram what happens is
this:
- Each program in the dill file ends in 4 bytes that indicates the size
of the program.
- Reading the input from the end one can then read the size, skip back
that amount of bytes, if we have more data (i.e. there's another
program), read another size and so on, and continue until we have
accounted for all bytes in the input.
- We then read each program from the start, and basically overwrite any
library, class, procedure etc. we find.
The first main reference found is the one used though.
(Saying that we overwrite is not completely true, but when the library
is a non-external library that's basically what happens).
This CL introduces (some) support on the C++ side for the same thing.
So far the C++ side could only handle single-program-dills, and trying
to load anything else would probably crash the VM.
The support added is this:
- Assume the SDK (i.e. vm_platform.dill) is not a concatenated file
(error out if it is).
- For user provided input, loop over each contained program one-by-one,
for each individual one behave as normal.
- The way LibraryLoad is implemented (i.e. it skips if the library is
already loaded) this means that it currently would behave differently
than the dart version (i.e. the first one is used, not the last one).
For now it is assumed that that's not a problem.
- There is a possibly snafu if the same script is included several times.
This could probably mostly be remedied by not creating scripts up front,
but only as needed. By the "keep only one" (and fixing the above point,
probably by simply loading in the opposite order, i.e. last program
in the binary first) the (theoretical) problem would probably do away.
Note that we will have separate string tables, canonical name tables etc
per "sub program" and that there might be some duplication.
The implementation was tested as indicated below, but introduces no tests.
$ cat test_lib1.dart
import "test_lib2.dart" as lib2;
String lib1field = "lib #1 field!!";
main() {
foo();
lib2.foo();
print("From lib2: ${lib2.lib2field}");
}
foo() {
print("Hello, Foo, from test_lib1!");
var x = 42;
print(x);
}
$ cat test_lib2.dart
String lib2field = "Lib #2 field!!!!";
foo() {
print("Hello, Foo, from test_lib2!");
var y = 34;
print(y);
}
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart --packages=.packages --platform=out/ReleaseX64/vm_platform.dill test_lib1.dart
$ ls -lha test_lib1.dart.dill
[...] 4.2M Oct 26 14:42 test_lib1.dart.dill
$ dart pkg/kernel/bin/split.dart test_lib1.dart.dill
Wrote test_lib1.dart.dill.part1.dill
Wrote test_lib1.dart.dill.part2.dill
$ ls -lha test_lib1.dart.dill.part{1,2}.dill
[...] 811 Oct 26 14:42 test_lib1.dart.dill.part1.dill
[...] 582 Oct 26 14:42 test_lib1.dart.dill.part2.dill
$ cat test_lib1.dart.dill.part1.dill test_lib1.dart.dill.part2.dill > test_lib1.dart.dill.concat.dill
$ ls -lha test_lib1.dart.dill.concat.dill
[...] 1.4K Oct 26 14:44 test_lib1.dart.dill.concat.dill
$ out/ReleaseX64/dart --kernel-binaries=out/ReleaseX64 --packages=.packages test_lib1.dart.dill.concat.dill
Hello, Foo, from test_lib1!
42
Hello, Foo, from test_lib2!
34
From lib2: Lib #2 field!!!!
Change-Id: I233a033aa3042b202dd4708908a5be3089474588
Reviewed-on: https://dart-review.googlesource.com/16820
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Includes the following changes:
- Disable type checking in the service isolate when strong mode is
turned on. We don't yet have strong mode clean vmservice Kernel
binary available (Issue #31203);
- Introduce new Dart VM API to allow creating List<T> (T in {String, int}).
Current API only allows to create List<dynamic> and in strong mode
List<dynamic> is not assignable to List<T>. For now we limit this API to
a fixed number of core types for performance considerations (type
arguments can be canonicalized and cache ahead of time). We will
extend the API allowing creation of arbitrary List<T> if we will
discover the need for it later;
- Use this new API to fix CreateRuntimeOptions to create List<String>
and not List<dynamic>;
- Likewise fix OneByteString_splitWithCharCode to return List<String>
and not List<dynamic>;
Additionally this CL refactors how getters and setters are created
in object_store.h removing 500 lines of largely boilerplate code.
Bug: https://github.com/dart-lang/sdk/issues/31052
Change-Id: I3fdcd2d54ff3f307db0913c67a7c2f009ea9d7a7
Reviewed-on: https://dart-review.googlesource.com/15884
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This is needed to support cases where memory with kernel binary was
allocated outside of Dart VM(for example, in Flutter Engine) has to be
released using appropriate mechanism too.
Bug:
Change-Id: I863db9e5a33469ba8fb5c1f8789c6498ddfbb385
Reviewed-on: https://dart-review.googlesource.com/5121
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
- Refactoring of the TimelinePage in order to remove the
dependencies from the Service objects.
- Implemented TimelineRepository to abstract the communication layer
- Structured new TimelineDashboardPage with simplified UI and event
filtering
The VM returns lists of events which are forwarded to Catapult.
Some events are white-listed for the common user.
Due to limitations in the way the events are generated the only way to
attach the white-list flag to the events is by adding an extra argument.
By setting the hash of timeline.html to #basic we inform JavaScript that
we require the activation of the filter which forwards just the events
white-listed to Captapult, while preserving all the others during
dumping operations.
See https://docs.google.com/document/d/1sDKI3ROYM1ORaKhelt063ayfveF_6WC8aq9MiqHjYBs/edit#
Change-Id: I38435d8d32f37305ce3c366daf55f82e8287959d
Reviewed-on: https://dart-review.googlesource.com/3346
Commit-Queue: Carlo Bernaschina <cbernaschina@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
New folder structure (nested under vm/):
- compiler/
- jit/ - JIT specific code
- aot/ - AOT specific code
- backend/ - all middle-end and back-end code (IL, flow graph)
- assembler/ - assemblers and disassemblers
- frontend/ - front ends (AST -> IL, Kernel -> IL)
compiler/README.md would be the documentation root for the compiler
pipeline
Bug: https://github.com/dart-lang/sdk/issues/30575
Change-Id: I2dfd9688793bff737f7632ddc77fca766875ce36
Reviewed-on: https://dart-review.googlesource.com/2940
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Use the speed of previous scavenges to estimate the time required for the next scavenge. When we receive an initial notification, perform a scavenge if we have allocated enough and we estimate we can complete the scavenge before the deadline.
Expand the time recorded for a scavenge to include safepointing and iterating roots.
R=asiva@google.com
Review-Url: https://codereview.chromium.org/3001423002 .
There already exists a class named "Reader" in the "kernel" namespace
which reads bytes from a kernel binary. The class KernelLoader
(KernelReader before this change) creates VM heap objects and populates
them.
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/3010543002 .
At the moment isolated has a name field which is on the main_port the
only exception is the vm-service isolate.
The name is used just for logging.
The service protocol though uses an extra field called debugger_name,
that at this moment is redundant and is optimized away during
compilation.
BUG=
R=rmacnak@google.com, zra@google.com
This CL:
- Removes the debugger_name in favor of a modifiable name
- Adds to all the log where the name was logged the main_port (that is
the only real identifier)
- Changes the default format of the isolate name from:
<script_uri>$<main>
to:
<script_uri>:<main>()
Review-Url: https://codereview.chromium.org/3004563003 .
Obfuscation is controlled by obfuscate flag in Dart_IsolateFlags.
Obfuscation of identifiers is performed during script tokenization - when TokenStream is generated from the source. All kIDENT and kINTERPOL_VAR tokens are renamed consistently using a persistent obfuscation map stored in ObjectStore::obfuscation_map.
Some identifiers (pseudo-keywords, arithmetic operators, builtin recognized methods and entry-points) are not renamed to keep name based lookups from breaking. All other identifiers are renamed.
Constant instances of Symbol-s (both created via literal syntax #ident and using constant constructor const Symbol("ident")) are renamed consistently with corresponding identifiers.
Script urls and Library urls and names are also obfuscated.
Obfuscation map can be dumped as a JSON array at the end of precompilation using Dart_GetObfuscationMap API.
BUG=https://github.com/dart-lang/sdk/issues/30524R=rmacnak@google.com
Review-Url: https://codereview.chromium.org/3003583002 .
Moved the responsibility to provide MaxRSS and CurrentRSS from the VM to
the embedder.
The embedder can opt-it by setting a Dart_GetEmbedderInformation
callback using the public Dart_SetEmbedderInformationCallback API.
The implementation of the Dart_GetEmbedderInformation should mandatory
fill the version field and the ones it is able to fill.
The name field must reference a constant C style string, it will not be
freed by the VM code.
R=zra@google.com
Review-Url: https://codereview.chromium.org/2996803002 .
This should cleanup user script kernel data and service isolate kernel data,
but leave the platform kernel data alone as it is used several times,
and it is unclear to me when it is safe to free it.
BUG=
R=kmillikin@google.com
Review-Url: https://codereview.chromium.org/3000623002 .
Reapply 3d4838fc70 with the following
modifications:
* define PRIX64 to fix compilation errors on Windows;
* add and use short form format specifiers PX and PX64 to make it
uniform with other format specifiers.
Original review: https://codereview.chromium.org/2987183002
Description of the original CL:
Until this change, implementation of Dart_IntegerToHexCString()
always created Bigint objects and used Bigint::ToHexCString() to do
the work.
In --limit-ints-to-64-bits mode (with Dart integers limited to 64
bits) Bigints are banned and should not be used.
This CL revises Dart_IntegerToHexCString implementation to avoid
creating Bigint objects for Smi and Mint arguments, so it would
work in --limit-ints-to-64-bits mode.
Also, this CL adds test case to lock down the behavior of
Dart_IntegerToHexCString.
R=zra@google.com
Issue: https://github.com/dart-lang/sdk/issues/30103
Review-Url: https://codereview.chromium.org/2985423002 .
Until this change, implementation of Dart_IntegerToHexCString()
always created Bigint objects and used Bigint::ToHexCString() to do
the work.
In --limit-ints-to-64-bits mode (with Dart integers limited to 64
bits) Bigints are banned and should not be used.
This CL revises Dart_IntegerToHexCString implementation to avoid
creating Bigint objects for Smi and Mint arguments, so it would
work in --limit-ints-to-64-bits mode.
Also, this CL adds test case to lock down the behavior of
Dart_IntegerToHexCString.
R=asiva@google.com
Issue: https://github.com/dart-lang/sdk/issues/30103
Review-Url: https://codereview.chromium.org/2987183002 .
The TypedData_getUint64 native method is used to implement Uint64List
and ByteData.getUint64().
When running in --limit-ints-to-64-bits mode (with Dart integers
limited to 64 bits), this native method and its clients should
silently cast values (with wrap-around) to signed 64-bit integer.
This is the intended behavior, chosen to make it possible to work
with arbitrary data.
This CL changes underlying Integer::NewFromUint64() to do the silent
conversion instead of returning Integer::null().
Another client of Integer::NewFromUint64, Dart_NewIntegerFromUint64 is
changed to handle error without relying on Integer::NewFromUint64().
R=zra@google.com
Issue: https://github.com/dart-lang/sdk/issues/30103
Review-Url: https://codereview.chromium.org/2992013002 .
2. Adjust the Dart front end to not require a 'main' method when the in memory filesystem is used, this allows execution of VM unit tests which all do not necessarily have a 'main' method (integrated CL from siggi)
3. Change kernel reader to not look for a 'main' methos when Dart_LoadKernel is called.
4. Change Dart_LoadScript to also support loading of dart programs when Dart front end is used
R=aam@google.com, sigmund@google.com
Review-Url: https://codereview.chromium.org/2948273002 .
This is the 2nd attempt to land the CL
abed3c2a70
Original review: https://codereview.chromium.org/2982823002
No extra changes were made. Large literals in the Dart core library
which caused failures during the previous attempt to land this CL have
been fixed separately.
Description of the CL:
In --limit-ints-to-64-bits mode:
* Integer constructors return Integer::null if integer is out of range.
* Error is reported for integer literals which are out of range.
* Dart API is revised to return errors if integers are out of range.
Bigint::IsDisabled() method is introduced to be able to enable/disable
Bigints independently of --limit-ints-to-64-bits mode in future.
Deprecated constructor Integer::NewFromUint64 is replaced with
Integer::New in certain cases.
R=regis@google.com
Issue: https://github.com/dart-lang/sdk/issues/30103
Review-Url: https://codereview.chromium.org/2980223002 .
In --limit-ints-to-64-bits mode:
* Integer constructors return Integer::null if integer is out of range.
* Error is reported for integer literals which are out of range.
* Dart API is revised to return errors if integers are out of range.
Bigint::IsDisabled() method is introduced to be able to enable/disable
Bigints independently of --limit-ints-to-64-bits mode in future.
Deprecated constructor Integer::NewFromUint64 is replaced with
Integer::New in certain cases.
R=zra@google.com
Issue: https://github.com/dart-lang/sdk/issues/30103
Review-Url: https://codereview.chromium.org/2982823002 .
There are a few problems with this function:
* It returns uint64_t instead of int64_t, which is different from
core Dart library function identityHashCode()
* Dart_IdentityHash truncates negative Mint and Bigint values to 0,
while negative Smi values are converted to uint64_t.
The Dart_IdentityHash function is not used, so we decided to remove it
instead of fixing these problems.
R=zra@google.com
Review-Url: https://codereview.chromium.org/2980043002 .
I collected statistics for the sizes and capacities of growable arrays which are promoted to old-space or survive an old-space gc when running dart2js and Fasta. For these applications, the vast majority of arrays stay empty. More than half of the total object size of promoted backing arrays is backing for empty growable arrays.
Furthermore, since the overhead for an array is 3 words (header, type parameters and length), and object sizes are rounded up to an even number of words, we waste one word for all even-sized arrays.
This CL changes the growth strategy so that empty growable arrays are created with a shared, zero-sized array as backing, avoiding the allocation of a backing array if no elements are added. When the array needs to grow, it starts out at 3 and grows to double size plus one each time: 7, 15, 31, ...
A few places in the VM code need to handle these shared, zero-sized arrays specially. In particular, the Array::MakeArray function needs to allocate a new, empty array if its result is to be returned to Dart code.
Benchmarks suggest that the change improves memory usage by a few percent overall and does not significantly affect run time.
BUG=
R=erikcorry@google.com
Review-Url: https://codereview.chromium.org/2949803002 .
Ryan Macnak