Weaken assertion in gen_snapshot requiring all libraries to have a loading unit as there can still be unreachable libraries:
- Google3 and Fuchsia will compile all the sources in a package to a single dill file, then present multiple input dill files to the AOT compilation. Since the set of libraries was derived from package membership instead of imports, many can be unreachable.
- When the root library's main comes from an export, the frontend's representation will incorrectly report the library containing main as the root library and the true root library may be unreachable from it.
Instead, assert only that surviving compiled code is assigned a loading unit.
TEST=gallery
Bug: https://github.com/flutter/gallery/issues/545
Bug: https://github.com/dart-lang/sdk/issues/49325
Bug: https://github.com/dart-lang/sdk/issues/41974
Bug: b/237016312
Change-Id: Ia52563a6f517308d041368be11dcc85270f19acc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249724
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This field is only used only by vm-service (including
hot reload and debugging), Kernel loading and mirrors.
So drop it to prevent putting it into the AOT snapshot.
Saves at 125Kb of snapshot size on large Flutter app.
Related to https://github.com/dart-lang/sdk/issues/48910
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try
Change-Id: Id7c47496e87482d462c2b318307f617d48758ad1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242864
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This change improves a few implementations of hashcodes in compiler.
Slightly improves AOT compilation time:
gen_snapshot 60.736s -> 58.920s (-2.9%)
(on a large Flutter app, compiled in release mode for arm64).
Also, the same large app can be now compiled with
--hash_map_probes_limit=1500, meaning that all hash maps in
the compiler perform less than 1500 probes when looking for an
element.
This change also adds a test which verifies that kernel compiler
itself can be compiled with --hash_map_probes_limit=1000.
This is a sanity check to ensure we do not have a very
badly distributed hashcode in the compiler.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/43299
Change-Id: I7a802709727a33760c4f1d13f7b2c8cb263852d7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229940
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change improves hash code implementations in multiple places in
the compiler. That reduces number of probes during lookups in hash maps
and improves AOT compilation time of large applications.
On a large Flutter app, compiled in release mode for arm64:
Total gen_snapshot time 89.184s -> 60.736s (-31.9%)
Also, this change adds --hash_map_probes_limit=N option which sets
a hard limit for the number of probes in hash maps. This option
makes it easy to find hash maps where there are many collisions
due to poor hash code implementation.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/43299
Bug: b/154155290
Change-Id: Ibf6f37d4b9f3bf42dd6731bfb4095a7305b98b2d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change bakes binary search table which maps PC ranges
to corresponding stack maps and Code objects (if still present
in the snapshot) into RO data section of the snapshot - instead
of constructing it at load time.
This allows to considerably reduce amount of work done when
loading Code cluster for programs which have majority of their
Code objects discarded (i.e. in DWARF stack traces mode): as
we no longer write / read any information for discarded Code
objects.
This CL also changes program visitor to deduplicate Code objects
if their instructions are deduplicated in AOT mode. Only a single
Code object can be choose as a representative for the given
PC range so it does not make sense to write multiple Code objects
into the snapshot which refer to the same Instructions.
The overall improvement is hard to quantify but ReadProgramSnapshot
shows the following improvement when starting a large
Flutter application on a slow Android device:
before 223.55±59.94 (192.02 .. 391.74) ms
after 178.06±47.03 (151.31 .. 291.34) ms
This CL packs CompressedStackMaps next to the binary search table
itself allowing us to address them via offsets instead of
pointers.
Snapshot sizes are actually affected positively by this change. On
the same large Flutter application I see
DWARF stack traces on: -1.34% total SO size
DWARF stack traces off: -1.63% total SO size
Issue https://github.com/dart-lang/sdk/issues/46116
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ic997045a33daa81ec68df462a0792915885df66b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220766
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Every AOT user out there has been using bare instructions mode
and continuing to maintaining non-bare instructions mode simply
adds costs (both in terms of time spent making changes to work
in a mode that is not used and CI resources spent on testing it).
This change removes FLAG_use_bare_instructions and changes the code
to assume that FLAG_precompiled_mode implies bare instructions.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try
Change-Id: I5032b13bfcb613f79865f2cfa139cca8d1b42556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220964
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Rename PointerKeyValueTrait<T>, which is used to create sets of pointers
to T instances where T <: Object and T has appropriate Hash and Equals
instance methods, to the more specific name PointerSetKeyValueTrait.
Create a PointerSet<T> alias for using this trait with
DirectChainedHashMap and use that alias in other code as a shorthand.
Remove PointerSetKeyValueTrait<const char> as a superclass of
CStringKeyValueTrait, as the only reuse from the former are the two
methods KeyOf and ValueOf which just return their argument, and having
this relationship is odd since const char is not a subtype of Object.
TEST=Renaming/refactoring, so existing tests.
Change-Id: I0274b16cb9fcb3939a28fb109fb8626c1ac8c0e9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215761
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit b5922e6db4.
Reason for revert: vm/cc/AllocateZone fails on Mac
Original change's description:
> [vm] Streamline Zones.
>
> - Don't put a HANDLESCOPE right after a StackZone, as there's no state worth restoring
> - Remove remaining zone bookkeeping dead since e3a9d70591.
> - Remove unnecessary resetting of handle blocks before zone deletion.
> - Shrink initial chunk size to make Zone's overall size fit in a faster malloc size class.
>
> TEST=ci
> Change-Id: I920e12f5af93aa71fb876435764c60df485215f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212261
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
TBR=rmacnak@google.com,asiva@google.com
Change-Id: Id23dcebb781791616e48e54d2dcd7e58159178bc
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215943
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Don't put a HANDLESCOPE right after a StackZone, as there's no state worth restoring
- Remove remaining zone bookkeeping dead since e3a9d70591.
- Remove unnecessary resetting of handle blocks before zone deletion.
- Shrink initial chunk size to make Zone's overall size fit in a faster malloc size class.
TEST=ci
Change-Id: I920e12f5af93aa71fb876435764c60df485215f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212261
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Use more zones and handle scopes to prevent temporary allocations from having unnecessarily long lifespans.
/usr/bin/time -v gen_snapshot --snapshot-kind=app-aot-elf ... dart2js.dill
Maximum resident set size (kbytes): 821344 -> 443688 (-46.0%)
TEST=ci
Bug: b/196510517
Change-Id: I41cdcac69f485b2f33adf2c94bb11f466e6a54fb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210944
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Previously VM had a hardcoded constant values for context indices of
_future variables in _Future.timeout and Future.wait (used while
collecting stack traces). As those variables are allocated in
the context after captured parameters, their indices depend on
whether the parameters are captured.
I'd like to improve tree shaking so that it will remove
part of Future.wait method and make one of the parameters
not captured, which changes the context index (and it may vary
between modes and for different programs).
So, instead of hardcoding those indices, they are now calculated
at compile time and stored in ObjectStore. This is more flexible
and accommodates for future changes in tree shaker.
TEST=existing tests
Change-Id: I47629b41e497843fe3e998e813e793cf77c18bdd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207202
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Previously there were several pieces of information shared between
both FunctionType and Function, mostly in the packed fields, but
named argument names were also kept in both places.
Now the FunctionType is the primary source for this information, with
the Function only keeping the names of positional arguments, which are
discarded in AOT snapshots.
This does mean extra work to access this information via the function
object, but for the most part, this information is only accessed in the
compiler or during dynamic lookups or checks in the runtime.
After adding the count of type parameters to the packed information
in FunctionType, the packed information has been split into two pieces:
one for parameter counts, another for type parameter counts. This
split does not increase the size of UntaggedFunctionType, as there
were 2 bytes available in the existing padding.
Changes on flutter gallery in release mode:
* ARM7 code size: total -0.91%, readonly -0.22%, isolate -4.32%
* ARM7 heap size: total -2.00%
* ARM8 code size: total -0.93%, readonly -0.22%, isolate -4.32%
* ARM8 heap size: total -2.12%
Changes on flutter gallery in release-sizeopt mode:
* ARM7 code size: total -0.24%, readonly -0.08%, isolate -1.49%
* ARM7 heap size: total -0.88%
* ARM8 code size: total -0.26%, readonly -0.11%, isolate -1.49%
* ARM8 heap size: total -1.01%
TEST=Refactoring, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ic4d59a7b4acca039a5647f9163e716f6019163f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203241
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
This reverts commit 80102c981b.
Reason for revert: b/186359854
Original change's description:
> [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
>
> This reduces the number of variable-length integer encodings in our
> database from 2 to 1, and chooses the more standard one.
>
> TEST=Existing tests, in particular any that involve snapshots.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Idf32bdd879cf8bb7407f6dae764312140ad6eeb2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196920
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
This reduces the number of variable-length integer encodings in our
database from 2 to 1, and chooses the more standard one.
TEST=Existing tests, in particular any that involve snapshots.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of 2852408881
Original change's description:
> [vm] Reduce size of Class objects in AOT and PRODUCT modes
>
> This change conditionally removes certain fields from Class objects
> in AOT and PRODUCT modes.
>
> Flutter gallery in release-sizeopt mode:
> Snapshot size -0.47% (arm64), -0.44% (arm)
> Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
>
> On a large Flutter application compiled in --dwarf-stack-traces mode:
> Heap size of Class objects -26%
> Heap size of all snapshot objects -3.6%
>
> TEST=ci
> Issue: https://github.com/dart-lang/sdk/issues/44020
> Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TEST=ci
Change-Id: Ib473a959a2c610b7a0e6f4000b2101e5256c6119
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196103
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This change conditionally removes certain fields from Class objects
in AOT and PRODUCT modes.
Flutter gallery in release-sizeopt mode:
Snapshot size -0.47% (arm64), -0.44% (arm)
Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
On a large Flutter application compiled in --dwarf-stack-traces mode:
Heap size of Class objects -26%
Heap size of all snapshot objects -3.6%
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44020
Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
In the codebase, we have several different interfaces for equality
instance methods on a type T:
- bool Equals(const T&) const;
- bool Equals(const T*) const;
- bool Equals(T*) const;
This CL standardizes on `bool Equals(const T&) const;`, and adjusts any
related functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: I2ebcf2b861732ce65c6f1c673b2df2a2c7f5dd4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195071
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In the codebase, we have several different interfaces for instance
methods that return a hash:
- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;
This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This CL removes asserts that prevented background compiler from doing class finalization.
It adds program_lock synchronization for when object properties are updated during class finalization.
It also encloses code instructions installation into safepoint scopes to ensure that no code is executed during installation.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=existing test suite
Change-Id: If297e44f51b187242eca2cdcfff066c8f6386d97
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178163
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This is a step toward deferring literals, to effectively split binary size for libraries that are more data than code, such as localizations. Bare instructions mode still has all constants in the root snapshot because it has one global ObjectPool.
dart2js+analyzer product X64 non-bare:
app.so 10072728 -> 8479824 (-15.8%)
app.so-2.part.so 14074512 -> 15016976 (6.69%)
app.so-3.part.so 9225936 -> 9888984 (7.18%)
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: If61f6156e3fc2dd539331335b3bf0458102b7c75
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168991
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup` (or to another place).
The access of `ObjectStore::closure_functions()` was spread out over our
codebase, including from `Isolate`. This CL moves it to a centralized
place - ClosureFunctionsCache.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I3b282623f07614cdc2c2a863bd5b9d6c7748d36f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177130
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Unlike cfc8e6de, this does _not_ replace the default variable length
encoding for {Read,Write}Streams, but insteads adds separate
{Read,Write}{S,}LEB128 methods to the appropriate classes. If we later
find the cause of the issues that led to the revert of cfc8e6de, it'll
be easy to switch over then.
Note that WriteLEB128 asserts that the value is non-negative if used
with a signed type (since negative values suggests that SLEB128 should
be used instead for minimal encoding).
Also removes the various other encoding and decoding methods for
(S)LEB128 across the codebase and changes those clients to use
{Read,Write}Streams instead.
Other cleanups:
* Various constant-related cleanups in datastream.h.
* Adds DART_FORCE_INLINE to ReadStream::ReadByte and uses it in the
default variable length decoding methods for retrieving bytes
from the stream instead of managing current_ by hand.
* Creates a canonical empty CompressedStackMaps instance and uses
that instead of the null CompressedStackMaps instance in most cases.
The only remaining (expected) use of the null CompressedStackMaps
instance is for the global table in the object store when no global
table exists (e.g., in JIT mode before any snapshotting).
* Moves CompressedStackMapsIterator from code_descriptors.h to an
Iterator class within CompressedStackMaps in object.h (similar to
PcDescriptors::Iterator), to limit friend declarations and because it
conceptually makes more sense as part of CompressedStackMaps.
* Removed CompressedStackMaps::PayloadByte, since existing clients
(CompressedStackMaps::Iterator, StackMapEntry in program_visitor.cc)
are better served by just operating on the payload buffer directly
(with appropriate NoSafepointScopes).
* WriteStreams no longer allocate their initial space on construction,
but rather on the first write, so no allocation is performed by
constructing a never-used WriteStream.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-ubsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: Ice63321abaa79157fbe9f230a864c8bba0e6dea9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166421
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Code objects might be connected by static calls and these references
were not included into the profile. This lead to some objects being
unreachable from the root.
This CL also adds a test which computes dominators for all nodes
in the snapshot (this in turn verifies that there are no
unreachable nodes).
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I2af4107fdb7f875624192e892ce1cec78cbf0dd0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161709
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
With --no-use-bare-instructions or with splitting, not all calls are pc-relative. We must replace the Function with its Code in the static call table before detaching the Function's code as part of removing Function objects.
Also handle WeakSerializationReferences in AssignLoadingUnitsCodeVisitor.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I29f9258c240f5a2b1b8dca52f74146dfe44d6401
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158165
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Don't use PC-relative calls when calling between loading units.
- Sort the contents of the code cluster by loading unit, then by text offset.
- Handle binding PC-relatives calls and inserting trampolines per loading unit.
- Create one code order table per loading unit to implement PC -> Code lookup.
- Read code order tables directly, instead of copying into malloc'd memory.
--use_table_dispatch still not yet supported.
This slightly shrinks non-split binaries (~2% clustered part, 0.4% total snapshot) due to the new delta encoding when Code references Instructions.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I51052ebc7b4968b554897d1710135a6c41821302
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157820
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
(Assignment of libraries to loading units is already done in the kernel generation step.)
After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.
All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.
Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.
Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.
Embedder's guide:
At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.
At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL adds --trace-precompiler-to option which generates a machine readable
precompiler trace (list of all compiled functions and their dependencies).
It also expands package:vm_snapshot_analysis with tools for reading and
analysing this trace.
For example snapshot_analysis explain dynamic-calls command allows
to list all dynamic calls sorted by their impact on the size of the AOT
snapshot.
Issue https://github.com/dart-lang/sdk/issues/41249
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: Ie49143f4da375067991991e2ad20a41ec67bb1c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152851
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Previously, we used the owner field to find the function whose entry
points to fix up after deduplicating instructions/code objects. However,
this means retrieving and checking the owning Function for each Code
object that has one, even if that Code object isn't the current code
object for the owning function. Instead, now we extend the instruction
deduplication visitors to Functions, where visiting a Function means
visiting its active Code object before fixing up the entry point cache.
Since the ProgramWalker also visits Code objects, this means visiting
some Code objects twice (once in the ProgramWalker and once when
visiting the Function for which it is the current code). However, since
the deduplication process is idempotent, the additional visits will not
cause the entry points for the Code object to change further (which
would otherwise invalidate the caches of already visited Functions).
Thus, the order of the visits does not affect the results.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try
Change-Id: Idd12fc94c5fb98f1fc9eb669d09b47822b0eac1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144700
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In most cases, Code objects in the static calls tables are reachable
through other means. Thus, only add the overhead of walking the static
calls tables when we are in precompiled mode and
--retain-function-objects is disabled.
Also, when deduplicating Instructions objects in Code objects, only
adjust the entry points for the owning function if the Code object is
the current code for that function.
Fixes https://github.com/dart-lang/sdk/issues/41619
Bug: 41619
Change-Id: I51621ccfe1d1c8402088a1c24c56e0a74a02e1ed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144560
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Alexander Markov