Original revert in patchset 1, fix for deadlock issue in patchset 2: avoid reentrant calls to RunWithStoppedMutator.
Further review comments addressed in successive patchsets.
This reverts commit 8ef508ba36.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Change-Id: I6a0279d9812020ad7a5c2b7851980b6a29b95b9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143327
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I64c86525f4ef9cb30567a49a106bfe700355942b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136780
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
These changes were originally submitted separately on different days,
and a major performance regression was seen after the first change
when creating snapshots that led to both being reverted. However,
that performance regression should be addressed by the followup.
First change:
"[vm] Treat the dispatch table as a root in the snapshot.
Additional changes:
* Only serialize a dispatch table in precompiled snapshots.
* Add information in v8 snapshot profiles for the dispatch table.
* Fix a typo in a field name.
* Print the number of Instructions objects (or payloads, for
precompiled bare instructions mode) in the fake cluster for
the data section.
* Fix v8 snapshots profiles so objects in memory mapped segments
and only those are prefixed with "(RO) ".
* Add names for Instructions objects in v8 snapshot profiles
when we can use the assembly namer.
* Add command line flag for old #define'd false flag."
Second change:
"[vm/aot] Keep GC-visible references to dispatch table Code entries.
This change splits dispatch table handling into four distinct
parts:
* The dispatch table generator does not make a dispatch table
directly, but rather creates an Array that contains the Code
objects for dispatch table entries.
* The precompiler takes this Array and puts it in the object
store, which makes it a new GC root.
* The serializer takes this information and serializes the
dispatch table information in the same form as before.
* The deserializer creates a DispatchTable object and populates
it using the serialized information.
The change in the precompiler ensures that the Code objects
used in the dispatch table have GC-visible references. Thus,
even if all other references to them from the other GC roots
were removed, they would be accessible in the serializer in
the case of a GC pass between the precompiler and serializer.
This change also means that the serializer can retrieve and
trace the Code objects directly rather than first looking up
the Code objects by their entry point."
Bug: https://github.com/dart-lang/sdk/issues/41022
Change-Id: I52c83b0536fc588da0bef9aed1f0c72e8ee4663f
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139285
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This change splits dispatch table handling into four distinct
parts:
* The dispatch table generator does not make a dispatch table
directly, but rather creates an Array that contains the Code
objects for dispatch table entries.
* The precompiler takes this Array and puts it in the object
store, which makes it a new GC root.
* The serializer takes this information and serializes the
dispatch table information in the same form as before.
* The deserializer creates a DispatchTable object and populates
it using the serialized information.
The change in the precompiler ensures that the Code objects
used in the dispatch table have GC-visible references. Thus,
even if all other references to them from the other GC roots
were removed, they would be accessible in the serializer in
the case of a GC pass between the precompiler and serializer.
This change also means that the serializer can retrieve and
trace the Code objects directly rather than first looking up
the Code objects by their entry point.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ic9ba29dfd64945a6e9a51f9de73f74d9202b58b7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138288
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Normalize FutureOr<T> and Never? after instantiation.
Use mutual subtyping of bounds in generic function type parameters instead of structural equality.
Change-Id: I78b836d8bd2f091bd56e5a0f9c2089a411fa6f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135787
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
NullCheck pc descriptors are added to all dispatch table calls where the
receiver may be null (and the selector is not one implemented by null).
All null entries in the table go to the NullError runtime entry, which
reads the NullCheck pc descriptor to get the name of the called member
for the error message.
Change-Id: I9d2847d0ccdfdb735b06e879916920ec299f39bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134294
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL is responsible for inlining Float32x4 and Float64x2 arithmetic
operations in order to reduce their run-time cost.
Null-checks were required in order to allow the unbox instructions to
be non-speculative.
CheckNullInstr was modified to throw ArgumentError when checking
arguments for those operations.
Change-Id: I18b36d96a36011647480f301eb6dc6bb478f59a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128589
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
When we are writing a snapshot, canonicalize all entries within
CompressedStackMaps payloads. We do this by creating a global table
of stack map information (bit counts + bit payload) that we store
in the isolate object store, and then for each existing CSM, we
replace it with an alternate version that refers to the global table.
This gets back some of the canonicalization we lost when we moved from
individual StackMap objects to per-Code CompressedStackMaps objects.
Here, we also represent the global table as a CompressedStackMaps
object. This means that there are three types of CompressedStackMaps:
* The original version that directly contains all entry information.
* The version where each entry is just the PC offset delta and an
offset into the global table payload.
* A version representing the global table where the entries are like
the original version except they don't contain a PC offset delta.
-----
The impact on AOT snapshot size when compiling the Flutter Gallery
in release mode:
armv7: Total size -0.88% (Isolate RO: -5.05%, Isolate snapshot: +0.00%)
armv8: Total size -1.04% (Isolate RO: -5.28%, Isolate snapshot: +0.00%)
-----
Bug: https://github.com/dart-lang/sdk/issues/35274
Change-Id: I1ce0f8b3cc58e2f11584f3c218e0fdf8984b799b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120667
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Currently the switchable calls require there to be a unique PC -> Code
-> Function mapping to re-construct the arguments descriptor which was
lost when transitioning from unlinked to monormophic.
=> This prevents instruction deuplication in bare mode for those
functions, because the PC -> Function mapping is amgiguous (the
frames don't have a Code object on the stack we can use)
This CL preserves the (name, args-descriptor) per switchable call site when we
go from unlinked to monomorphic state by remembering it in a map. When
transitioning out of monomorphic we can use this map to re-construct the
name and arguments descriptor (and delete the map entry).
Flutter gallery release size impact:
* arm64: total -2.33 % (-3.62% RX)
* arm: total -1.72 % (-2.65% RX)
From local testing it seems like running flutter gallery in profile mode
and connecting via observatory the memory consumed by UnlinkedCall
instances seems minimal.
Change-Id: I55902e6f11c9941493c610ed18f26bc5fc3dcf59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121421
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Like the constant pool, this pool is populated when the flag
`--populate-llvm-constant-pool` is used. This pool contains the `Function`
objects that were compiled, and each `FlowGraph` node in the serialized flow
graphs will contain extra information labeled by `llvm-index` which is the
index of its corresponding `Function` object in the pool.
Also fix up printing of the constant and function pools when the flag
`--print-llvm-constant-pool` is used so that the output is buffered to avoid
overlapping output from concurrently running isolates.
Bug: https://github.com/dart-lang/sdk/issues/38297
Change-Id: I67739500c665e509ed16becc2e6bca7fe4928eb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119543
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This function has been unused for a long time.
Also, computing fingerprints for bytecode function causes bytecode to be loaded, which can lead to an old enum class being initialized with enum values for the new enum class, causing a forwarding loop when migrating enum values.
Change-Id: Iad9b2bb11aaf161d66f7752cb99cc4b4fc9fb310
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116442
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Since Dart 2, library prefixes are always loaded eagerly on the VM.
Also remove reload check for deferred prefixes. This check was added to avoid behavior of library prefixes giving spurious "not loaded" errors after a reload, but now this error can never occur. Resolves a difference in reload between AST and bytecode modes, since bytecode mode isn't surfacing the deferred property.
Change-Id: Ide5fb6cac2efc90ca1b108a35bc09d342cbd60de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116051
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Move generation of Function objects for native trampolines to the Precompiler, so they can be generated during AOT and tree-shaken if possible.
Issue dartbug.com/35765
Change-Id: I0e69b7e0b22db73e3a40f2fe445660e57ddb6fa9
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-bare-linux-release-simarm64-try, vm-kernel-precomp-bare-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-dartkb-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107407
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Rationale:
The problem was that we were evaluating field
initializers while still loading the constant
table. This hit two asserts (1) having pending
potential_natives() when entering codegen and
(2) not being able to evaluate constants
since the table has not been loaded yet. The
solution is to simply postpone enum field
initialization (forced to avoid hot reload
issues) while loading the constant table
until right after the table is ready.
https://github.com/dart-lang/sdk/issues/36153
Change-Id: I5060476e5b3b49e555cfc507fb398ef0144b44d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96844
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Prototype for `dart:ffi` on Linux/MacOS x64 in JIT mode.
`dart:ffi` is experimental and its API is likely to change in the future.
Progress and design decisions are tracked in https://github.com/dart-lang/sdk/projects/13
issue: https://github.com/dart-lang/sdk/issues/34452
Change-Id: Ifa4566388e42c8757f154741d11e303465ef305d
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try, vm-kernel-mac-debug-x64-try, vm-kernel-asan-linux-release-x64
Reviewed-on: https://dart-review.googlesource.com/c/80124
Reviewed-by: Samir Jindel <sjindel@google.com>
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Two fixes:
- Make entrypoints_verification_test work on configurations where CFE compilation
is separate from execution.
- Add missing entry-point annotation for Windows.
The original revision is in patchset 1.
Change-Id: I6c4b52b1bae7bc730546dad6a3e31d8625f850b1
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/90942
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Directly referencing VM isolate instructions from an isolate snapshot causes the instructions to be copied into the isolate's image page because instructions do not use the indirection of the ref array. Duplicating the instructions caused TypeTestingStubFinder::LookupByAddresss to be unable to identity the duplicate copies of the VM isolate type testing stubs.
Fixes identity of StubCode::*TypeTest broken by duplication in snapshot.
Fixes generating AppJIT snapshots after loading from AppJIT snapshots.
Fixes attribution of ticks for type test stubs in the AOT profiler.
Change-Id: I9879a85bd548e694ee36e14f795898582ccdddb0
Reviewed-on: https://dart-review.googlesource.com/c/90501
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This reverts commit 0203243381.
Reason for revert: breaking several bots
Original change's description:
> [vm] Prevent access to non-annotated members through native API.
>
> We will issue warnings and throw an API error when the native API is used to access members which were not appropriately
> annotated as entry-points, if the flag "--verify-entry-points" is passed.
>
> Also, fixes Class::Invoke against fields (isn't allowed through native API but
> could be allowed with mirrors, and is inconsistent with Library::Invoke behavior).
>
> I've checked locally that this would have caught the bug in
> "Wrap the user entrypoint function in a zone with native exception callback. (#7512)".
>
> Change-Id: I617c71e1965457c956c97761359765bb9bb18c1c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/90060
> Commit-Queue: Samir Jindel <sjindel@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
TBR=alexmarkov@google.com,sjindel@google.com
Change-Id: I554b389780e4ba652183c044b040b0c524beb5c2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/90841
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
We will issue warnings and throw an API error when the native API is used to access members which were not appropriately
annotated as entry-points, if the flag "--verify-entry-points" is passed.
Also, fixes Class::Invoke against fields (isn't allowed through native API but
could be allowed with mirrors, and is inconsistent with Library::Invoke behavior).
I've checked locally that this would have caught the bug in
"Wrap the user entrypoint function in a zone with native exception callback. (#7512)".
Change-Id: I617c71e1965457c956c97761359765bb9bb18c1c
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/90060
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
ARM write barrier goes from
ldrne lr, [thr, #+508]
blxne lr
to
blne <offset>
ARM64 write barrier goes from (similarly X64)
beq +8
ldrx lr, [thr, #1056]
blr lr
to
beq +4
bl <offset>
It reduces RX on arm/arm64 by around 0.9%
Though the write barrier wrappers stub has multiple entrypoints (one for
each available register). Because of this, we modify the relocation
logic to support per-call offsets into the target.
To avoid making the assembler code depend on StubCode/FlowGraphCompiler,
we set a closure, which the assembler can call.
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I9e3d68260cab7ef19ea88f1235c78d6031819d6d
Reviewed-on: https://dart-review.googlesource.com/c/90063
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL improves AOT code for StackOverflowInstr/CheckNullInstr:
* On ARM we can do a conditional pc-relative calls for the stack overflow
checks, getting rid of the slow-paths entirely.
* On ARM64 we can do pc-relative calls on the slow path, avoiding an
extra load.
Flutter gallery size impact (in bare instructions mode):
* ARM: -3.7% RX
* ARM64: -1.4% RX
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: Ia1acd76ac6efa26642f99e1ce3e417100aa357f3
Reviewed-on: https://dart-review.googlesource.com/c/89620
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This is the final CL which adds a new --use-bare-instructions flag to
the VM.
If this flag is set during AOT compilation, we will:
* Build one global object pool (abbr: GOP) which all code objects
share. This gop will be stored in the object store. The PP register
is populated in the enter dart stub and it is restored when
returning from native calls.
* Gets rid of the CODE_REG/PP slots from the dart frames. Instead the
compiled code uses the global object pool, which is always in PP.
* Starts emitting pc-relative calls for calls between two dart
functions or when invoking a stub.
Limitation: We only emit pc-relative calls between two code objects
in the same isolate (this is because the image writer is writing
instruction objects for vm-isolate/main-isolate seperately)
* We do compile-time relocation of those static calls after the
precompiler has finished its work, but before writing the snapshot.
This patches all the instruction objects with pc-relative calls to
have the right .text distance.
* We emit a sorted list of code objects in ObjectStore::reverse_code_table,
which will be used by the AOT runtime to go back from PC to Code
objects (where all metadata, e.g. stack maps, catch entry moves, pc
descriptors are available).
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I6c5dd2b1571e3a889b27e804a24c2986c71e03b6
Reviewed-on: https://dart-review.googlesource.com/c/85769
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL adds a [ReversePcLookupCache] which, based on a list of code
objects, builds up a binary searchable table for mapping PCs to Code
objects (where all the metadata is available, like stackmaps, ...).
This CL also adds stack walking support for "bare instruction" frames.
In a later part we will start emitting the sorted list of code objects
(via the new field in the object store) under a flag.
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I3c06c12bc0fb266dc1bd843a4a11b5208773151d
Reviewed-on: https://dart-review.googlesource.com/c/85746
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Martin Kustermann