There was a missing check for whether a backwards call is out-of-range.
The CL also removes leftover code related to [max_offset_into_target_] -
which wasn't used. The intended use is already over approximated via
[max_instructions_size_].
The CL makes the pc-relative call/tail-call distances pluggable so a
newly added test can modify them for testing in-range/out-of-range
forward/backwards calls.
TEST=vm/cc/CodeRelocator_*
Fixes https://github.com/flutter/flutter/issues/80043
Change-Id: Id4bdb7176108b61235dafb7ffc125da4a2bf07fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195682
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This CL adds a new PAYLOAD_SIZEOF specification to
runtime_offsets_list.h which defines InstanceSize methods given an
method name to invoke to get the header size (i.e., the size of
the object portion before the payload).
It uses this new specification to create appropriate InstanceSize()
methods for objects written to read-only sections of snapshots,
instead of needing separate size calculations for SIMARM_X64.
It adds more methods to Instructions to avoid special casing for bare
instructions mode. It also removes the special casing for SIMARM_X64,
serializing all read-only objects in the same manner even when not
in a crossword situation.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie3e4009f4bc03688998c32281e42fa22a255731d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165501
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Right now AssertAssignable will not call the TTS of a TypeParameter but
instead load the value for the type parameter inline and then call it's
TTS.
To prepare for future changes where we want to call TTS of an arbitrary
type object, we need to be able to install a functioning TTS on
TypeParameter, which this CL does.
Issue https://github.com/dart-lang/sdk/issues/40813
Change-Id: I0792cc1a796fe8afb2167b7665580b3f1385005b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162007
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Code objects might be connected by static calls and these references
were not included into the profile. This lead to some objects being
unreachable from the root.
This CL also adds a test which computes dominators for all nodes
in the snapshot (this in turn verifies that there are no
unreachable nodes).
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I2af4107fdb7f875624192e892ce1cec78cbf0dd0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161709
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Before, use of a function in the static calls table of a Code object
would cause it to be retained. Now, like dispatch table use, we only
retain such functions if there are other uses like dynamic function
lookup that needs the function object or if the
--retain-function-objects flag is enabled.
In most modes, --retain-function-objects is enabled by default.
It is only disabled by default in product mode when
--dwarf-stack-traces is enabled, since otherwise the removed
function objects may be needed for debugging platforms like the
Observatory or when creating symbolic stack traces.
Changes on flutter gallery in release mode:
Default:
arm7: isolate: +6.65%, total: +1.30%
arm8: isolate: +6.67%, total: +1.28%
(The increase is due to changing back to all function objects
being retained as the default when using symbolic stack traces.)
With --no-retain-function-objects:
arm7: isolate: -9.95%, total: -1.73%
arm8: isolate: -10.04%, total: -1.77%
(This measures how much dropping static call function objects
when possible affects the size compared to just dropping
function objects for dispatch table entries when possible.)
Bug: https://github.com/dart-lang/sdk/issues/41052
Change-Id: I3c834b14b0c58ccfdbaca3f154df536df29c13ef
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142146
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now, when writing an AOT snapshot in bare instructions mode, only
the actual instructions in the RawInstructions payload are serialized
instead of the entire RawInstructions object.
Since there are no longer RawInstructions objects in these AOT
snapshots, we also change how Code objects are serialized. Instead
of just containing a reference to the RawInstructions object, we
serialize two pieces of information: where the instructions
payload for this Code object begins and whether there was a single
entry for the instructions payload. (To save space, the single
entry bit is serialized as the low order bit of the unchecked
offset, which was already being serialized).
While we also need the length of the instructions payload, we
approximate it for all but the last Code object by subtracting
the next Code object's payload start from this Code object's
payload start. For the last Code object, we assume it extends
to the end of the instructions image.
Changes on flutter gallery in release mode:
armv7: instructions size -2.70%, total size -1.73%
armv8: instructions size -6.04%, total size -3.63%
Fixes https://github.com/dart-lang/sdk/issues/38451.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ia0a5c4e5e47c956776dc62503da38ec55a143c04
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134325
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 7475c637c3.
Reason for revert: Breakage on simarm_x64 (and thus arm_x64)
Original change's description:
> [vm/aot] Remove object wrapping of bare instructions for AOT snapshots.
>
> Now, when writing an AOT snapshot in bare instructions mode, only
> the actual instructions in the RawInstructions payload are serialized
> instead of the entire RawInstructions object.
>
> Since there are no longer RawInstructions objects in these AOT
> snapshots, we also change how Code objects are serialized. Instead
> of just containing a reference to the RawInstructions object, we
> serialize two pieces of information: where the instructions
> payload for this Code object begins and whether there was a single
> entry for the instructions payload. (To save space, the single
> entry bit is serialized as the low order bit of the unchecked
> offset, which was already being serialized).
>
> While we also need the length of the instructions payload, we
> approximate it for all but the last Code object by subtracting
> the next Code object's payload start from this Code object's
> payload start. For the last Code object, we assume it extends
> to the end of the instructions image.
>
> Changes on flutter gallery in release mode:
> armv7: instructions size -2.66%, total size -1.68%
> armv8: instructions size -5.81%, total size -3.49%
>
> Fixes https://github.com/dart-lang/sdk/issues/38451.
>
> Change-Id: Ia458e8d99bae18f5c3b6e849df2519027f06f574
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131067
> Commit-Queue: Teagan Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com
Change-Id: If2c8ca6b0993211b2509b275af7ff11a8fa2baa3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-simarm_x64-try, vm-kernel-precomp-android-release-arm64-try, vm-kernel-precomp-android-release-arm_x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134322
Reviewed-by: Teagan Strickland <sstrickl@google.com>
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Now, when writing an AOT snapshot in bare instructions mode, only
the actual instructions in the RawInstructions payload are serialized
instead of the entire RawInstructions object.
Since there are no longer RawInstructions objects in these AOT
snapshots, we also change how Code objects are serialized. Instead
of just containing a reference to the RawInstructions object, we
serialize two pieces of information: where the instructions
payload for this Code object begins and whether there was a single
entry for the instructions payload. (To save space, the single
entry bit is serialized as the low order bit of the unchecked
offset, which was already being serialized).
While we also need the length of the instructions payload, we
approximate it for all but the last Code object by subtracting
the next Code object's payload start from this Code object's
payload start. For the last Code object, we assume it extends
to the end of the instructions image.
Changes on flutter gallery in release mode:
armv7: instructions size -2.66%, total size -1.68%
armv8: instructions size -5.81%, total size -3.49%
Fixes https://github.com/dart-lang/sdk/issues/38451.
Change-Id: Ia458e8d99bae18f5c3b6e849df2519027f06f574
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131067
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The pc-relative calls were recorded with pc-offsets pointing to the start of the
call instruction, where as code-based calls were recorded with
pc-offsets pointing to the next instruction (i.e. return address).
This is inconsistent and caused us to hit an assert in
`Code::set_static_calls_target_table`. The assert was benign, but it is
good to maintain the uniqueness guarantee in the static calls table, so
we'll unify the encoding to use offsets to the instruction after the
call in both cases.
Closes https://github.com/dart-lang/sdk/issues/39811
Change-Id: Id0305befd78f09ed0b0e100f39641bca9e764442
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129717
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
There's just one case where we can't get the offset for Instructions
from somewhere else, and that's when we reset active_instructions_ to
instructions_ in Code::Enable(). Thus, we add a caching field for the
uncached offset to RawCode, but only when the active_instructions_
field is added (when not running in the precompiled runtime).
We also output the unchecked offset in snapshots for Code objects so
we can recalculate the entry point cache on loading. (For full JIT
snapshots, we output both the cached offset and the offset for
active_instructions_.)
Bug: https://github.com/dart-lang/sdk/issues/38451
Change-Id: I78572ec83d1574b00820df148bccb57e5c79eaaa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125400
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Teagan Strickland <sstrickl@google.com>
To avoid confusion between the heap size (includes header and rounding up) and number-of-elements size or payload size.
Change-Id: I714c4e8cec92d7ef963de39474d2a1bb23e5b4a3
Reviewed-on: https://dart-review.googlesource.com/c/92400
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Aart Bik <ajcbik@google.com>
ARM write barrier goes from
ldrne lr, [thr, #+508]
blxne lr
to
blne <offset>
ARM64 write barrier goes from (similarly X64)
beq +8
ldrx lr, [thr, #1056]
blr lr
to
beq +4
bl <offset>
It reduces RX on arm/arm64 by around 0.9%
Though the write barrier wrappers stub has multiple entrypoints (one for
each available register). Because of this, we modify the relocation
logic to support per-call offsets into the target.
To avoid making the assembler code depend on StubCode/FlowGraphCompiler,
we set a closure, which the assembler can call.
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I9e3d68260cab7ef19ea88f1235c78d6031819d6d
Reviewed-on: https://dart-review.googlesource.com/c/90063
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL adds support to the ImageWriter to write opaque blocks of
trampoline bytes
The AOT code relocator is adapted to allow limited range calls and
inserts trampolines if need be. The algorithm tries to minimize the
number of trampolines added, which for small applications will be 0.
The unconditional pc-relative calls have limited range:
* on ARM (+/-32 MB)
* on ARM64 (+/-128 MB)
To avoid verbose code for doubly-linked list, this CL adds double_list.h
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I0354cf4b2dd58ed5de25d67fc818f0603a2ec501
Reviewed-on: https://dart-review.googlesource.com/c/89283
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This is the final CL which adds a new --use-bare-instructions flag to
the VM.
If this flag is set during AOT compilation, we will:
* Build one global object pool (abbr: GOP) which all code objects
share. This gop will be stored in the object store. The PP register
is populated in the enter dart stub and it is restored when
returning from native calls.
* Gets rid of the CODE_REG/PP slots from the dart frames. Instead the
compiled code uses the global object pool, which is always in PP.
* Starts emitting pc-relative calls for calls between two dart
functions or when invoking a stub.
Limitation: We only emit pc-relative calls between two code objects
in the same isolate (this is because the image writer is writing
instruction objects for vm-isolate/main-isolate seperately)
* We do compile-time relocation of those static calls after the
precompiler has finished its work, but before writing the snapshot.
This patches all the instruction objects with pc-relative calls to
have the right .text distance.
* We emit a sorted list of code objects in ObjectStore::reverse_code_table,
which will be used by the AOT runtime to go back from PC to Code
objects (where all metadata, e.g. stack maps, catch entry moves, pc
descriptors are available).
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I6c5dd2b1571e3a889b27e804a24c2986c71e03b6
Reviewed-on: https://dart-review.googlesource.com/c/85769
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Start ordering instructions before writing them out in AOT snapshots.
This is done by:
* discovery of relevant code objects
* ordering of the code objects (currently simply discovery order)
* relocate any pc-relative calls by patching up pc-relative calls with
the actual distance
* producing image writer commands which tell it in which order and at
which .text offsets to write out instructions
Note: Our compiler currently does not emit any pc-relative calls. This
will be added later on.
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I0153c32a7427ea395e80489efadd5513fe7ca371
Reviewed-on: https://dart-review.googlesource.com/c/85745
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Martin Kustermann