Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.
The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.
Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.
If reload succeeds we drop original class table and
set program class table as heap walk one.
If reload fails we drop the program class table and
restore original one from heap walk table.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This change introduces separate stubs for suspending sync* functions
at start and at yield/yield*. Suspend stub for yield/yield*
no longer calls Dart callback (in order to make it faster).
Also, ReturnSyncStar stub is removed - sync* functions now directly
return false instead of going through the stub.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Iee9a1f48cab2812cf0f9f0e4e6d8e847547e49f7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250420
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.
Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:
* Suspending variable-length frames (for unoptimized code).
* Handling of Code and pool pointers in Dart stack frames.
* OSR.
* Deoptimization.
* Hot reload.
* Debugger.
The new implementation is not enabled in JIT mode yet.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Passing handles in FFI calls has significant overhead due to
how each handle requires a runtime entry to allocate in the
handle scope.
This change removes that runtime entry by relying on the
register allocator to allocate all handle arguments on the stack,
so that we don't need to allocate them separately.
To pass the stack handles to the native call we then pass a pointer
to the stack slot as the native argument.
Testing:
- We already have comprehensive tests for correctness in the form
of the FFI tests. These make calls with various combinations of
Handle and non-handle arguments.
- Correct GC behaviour is likewise covered in
`vmspecific_handle_test.dart` which makes calls with handles
arguments and triggers GC in-flight.
In case we do not correctly pass the handles on the stack, the
GC will trash them during the call.
TEST=Existing.
Bug: https://github.com/dart-lang/sdk/issues/47624
Change-Id: Ic837bad5484daaa5534b7c2e8707ac2c5dfa480f
Cq-Include-Trybots: luci.dart.try:vm-kernel-gcc-linux-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243320
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
1/2 or 1/4 branches executed per object initialization for uncompressed and compressed pointers respectively.
Note ARM32 already has double-wide initialization.
TEST=ci
Change-Id: Iacc61cac39f74191bb2e5e06cd48b90c006b8585
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241622
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL implements `NativeFinalizer` in the GC.
`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.
On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.
In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.
When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.
In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*
Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Original CL in patchset 1.
Split-off https://dart-review.googlesource.com/c/sdk/+/238341
And pulled in fix https://dart-review.googlesource.com/c/sdk/+/238582
(Should merge cleanly when this lands later.)
This CL implements the `Finalizer` in the GC.
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: Ibdfeadc16d5d69ade50aae5b9f794284c4c4dbab
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238086
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
- Fixes SIMD registers clobbered by write barrier on ARM64.
- Moves code generation out of non-compiler directory.
- Removes unnecessary building of Dart frames on leaf runtime calls.
- Removes unnecessary Threads slots for write barrier Code objects.
- Removes duplicate saves of SP in leaf runtime calls on ARM64 and RISC-V.
- Avoids some redundant SP updates on RISC-V.
TEST=ci
Change-Id: Idb92127658edc90b320923ef3d882a7219a450ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236842
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 7dca34c235.
Reason for revert: b/226085355 dart_vm_test crashing. Unclear what
the cause is. Reverting so we can triage the issue.
TEST=This is a revert.
Original change's description:
> [vm] Implement `Finalizer`
>
> This CL implements the `Finalizer` in the GC.
>
> (This CL does not yet implement `NativeFinalizer`.)
>
> The GC is specially aware of two types of objects for the purposes of
> running finalizers.
>
> 1) `FinalizerEntry`
> 2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
>
> A `FinalizerEntry` contains the `value`, the optional `detach` key, and
> the `token`, and a reference to the `finalizer`.
> An entry only holds on weakly to the value, detach key, and finalizer.
> (Similar to how `WeakReference` only holds on weakly to target).
>
> A `Finalizer` contains all entries, a list of entries of which the value
> is collected, and a reference to the isolate.
>
> When a the value of an entry is GCed, the enry is added over to the
> collected list.
> If any entry is moved to the collected list, a message is sent that
> invokes the finalizer to call the callback on all entries in that list.
>
> When a finalizer is detached by the user, the entry token is set to the
> entry itself and is removed from the all entries set.
> This ensures that if the entry was already moved to the collected list,
> the finalizer is not executed.
>
> To speed up detaching, we use a weak map from detach keys to list of
> entries. This ensures entries can be GCed.
>
> Both the scavenger and marker tasks process finalizer entries in
> parallel.
> Parallel tasks use an atomic exchange on the head of the collected
> entries list, ensuring no entries get lost.
> The mutator thread is guaranteed to be stopped when processing entries.
> This ensures that we do not need barriers for moving entries into the
> finalizers collected list.
> Dart reads and replaces the collected entries list also with an atomic
> exchange, ensuring the GC doesn't run in between a load/store.
>
> When a finalizer gets posted a message to process finalized objects, it
> is being kept alive by the message.
> An alternative design would be to pre-allocate a `WeakReference` in the
> finalizer pointing to the finalizer, and send that itself.
> This would be at the cost of an extra object.
>
> Send and exit is not supported in this CL, support will be added in a
> follow up CL. Trying to send will throw.
>
> Bug: https://github.com/dart-lang/sdk/issues/47777
>
> TEST=runtime/tests/vm/dart/finalizer/*
> TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
> TEST=runtime/vm/object_test.cc
>
> Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,rmacnak@google.com,dacoharkes@google.com
Change-Id: I991f6e49896d18a8d70210cf315d858b462d66c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/47777
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238080
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL implements the `Finalizer` in the GC.
(This CL does not yet implement `NativeFinalizer`.)
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL implements `WeakReference` in the VM.
* This reduces the size of weak references from 2 objects using 8 words
to 1 object using 4 words.
* This makes loads of weak reference targets a single load instead of
two.
* This avoids the fix-point in the GC and message object copying for
weak references. (N.b. Weak references need to be processed _after_
the fix-point for weak properties.)
The semantics of weak references in messages is that their target gets
set to `null` if the target is not included in the message by a strong
reference.
The tests take particular care to exercise the case where a weak
reference's target is only kept alive because a weak property key is
alive and it refers to the target in its value. This exercises the fact
that weak references need to be processed last.
Does not add support for weak references in the app snapshot. It would
be dead code until we start using weak references in for example the
CFE.
This CL does not try to unify weak references and weak properties in
the GC or messaging (as proposed in go/dart-vm-weakreference), because
their semantics differ enough.
Closes: https://github.com/dart-lang/sdk/issues/48162
TEST=runtime/tests/vm/dart/finalizer/weak_reference_run_gc_test.dart
TEST=runtime/tests/vm/dart/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/lib/isolate/weak_reference_message_1_test.dart
TEST=tests/lib/isolate/weak_reference_message_2_test.dart
Change-Id: I3810e919a5866f3ae8a95bd9aa23a880a0b0921c
Cq-Include-Trybots: luci.dart.try:app-kernel-linux-debug-x64-try,dart-sdk-mac-arm64-try,vm-canary-linux-debug-try,vm-fuchsia-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-release-x64-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/232087
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This change removes distinct Slots for the same field:
PointerBase_data_field -> PointerBase_data
Pointer_data_field -> PointerBase_data
TypedDataBase_data_field -> PointerBase_data
Also, the following slot is renamed to match field name:
TypedDataView_data -> TypedDataView_typed_data
The following offsets are renamed to match field names /
declarations in Untagged* classes:
PointerBase::data_field_offset -> PointerBase::data_offset
TypedData::data_offset -> TypedData::payload_offset
TypedDataView::data_offset -> TypedDataView::typed_data_offset
TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/48273
Change-Id: I602545fc43018494bcb24d8692292fdbc6a8f3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231160
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change improves hash code implementations in multiple places in
the compiler. That reduces number of probes during lookups in hash maps
and improves AOT compilation time of large applications.
On a large Flutter app, compiled in release mode for arm64:
Total gen_snapshot time 89.184s -> 60.736s (-31.9%)
Also, this change adds --hash_map_probes_limit=N option which sets
a hard limit for the number of probes in hash maps. This option
makes it easy to find hash maps where there are many collisions
due to poor hash code implementation.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/43299
Bug: b/154155290
Change-Id: Ibf6f37d4b9f3bf42dd6731bfb4095a7305b98b2d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When ffi code calls Dart code that requests the isolate to exit(via Isolate.exit, for example), we ensure that isolate indeed exits on return from ffi call.
This is implemented by introduction of new safepoint bit which, once set, forces ExitSafepoint to propagate unwind error.
TEST=isolate_exit_sandwich_test
Change-Id: I2e8f5ecec7f4e59ae5f99b9525cc566f20d4b6a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219846
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
This change bakes binary search table which maps PC ranges
to corresponding stack maps and Code objects (if still present
in the snapshot) into RO data section of the snapshot - instead
of constructing it at load time.
This allows to considerably reduce amount of work done when
loading Code cluster for programs which have majority of their
Code objects discarded (i.e. in DWARF stack traces mode): as
we no longer write / read any information for discarded Code
objects.
This CL also changes program visitor to deduplicate Code objects
if their instructions are deduplicated in AOT mode. Only a single
Code object can be choose as a representative for the given
PC range so it does not make sense to write multiple Code objects
into the snapshot which refer to the same Instructions.
The overall improvement is hard to quantify but ReadProgramSnapshot
shows the following improvement when starting a large
Flutter application on a slow Android device:
before 223.55±59.94 (192.02 .. 391.74) ms
after 178.06±47.03 (151.31 .. 291.34) ms
This CL packs CompressedStackMaps next to the binary search table
itself allowing us to address them via offsets instead of
pointers.
Snapshot sizes are actually affected positively by this change. On
the same large Flutter application I see
DWARF stack traces on: -1.34% total SO size
DWARF stack traces off: -1.63% total SO size
Issue https://github.com/dart-lang/sdk/issues/46116
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ic997045a33daa81ec68df462a0792915885df66b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220766
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
TSAN instruments C++ code by adding prologue/epilogue code which
maintains a shadow stack. Using setjmp()/longjmp() is intercepted by
TSAN and correspondingly unwinds the shadow stack.
When Dart VM throws exceptions we call the JumpToFrame stub
from C++ which will directly reset the stack to the exception handler
catch entry. This leaves the TSAN shadow stack unchanged.
This means whenever an exception is thrown we leak frames in TSAN's
shadow stack. Due to using a fixed-size shadow stack, it will cause a
buffer-overflow in TSAN when too many such frame leaks happen. This can
cause arbitrary memory to be overriden, leading to awkward crashes.
This is especially an issue on the "iso-stres" builder because it
launches - in the same process - *many* small tests, more easily hitting
that limit.
This CL will workaround the TSAN issue by making runtime call save it's
state via setjmp() and make exception throughing process go via
longjmp() (which TSAN will intercept) before actually calling the
JumpToFrame stub.
=> This will ensure the TSAN shadow stack is correctly maintained.
The [jmp_buf]'s encoding of register state is non-trivial (e.g. it uses
XOR'ing of the actual saved state under certain glibc versions). So we
store any state we need to pass to the target of the `longjmp()` on the
[Thread] instead of overriding the [jmp_buf]s register state with the
arguments.
Issue https://github.com/dart-lang/sdk/issues/47472#issuecomment-948235479
TEST=vm/dart{,_2}/regress47472_test.dart
Change-Id: Ifbf6580aa15bcce54d0584cdc3cd18cc19be0a9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222300
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
TSAN instruments C++ code by adding prologue/epilogue code which
maintains a shadow stack. Using setjmp()/longjmp() is intercepted by
TSAN and correspondingly unwinds the shadow stack.
When Dart VM throws exceptions we call the JumpToFrame stub
from C++ which will directly reset the stack to the exception handler
catch entry. This leaves the TSAN shadow stack unchanged.
This means whenever an exception is thrown we leak frames in TSAN's
shadow stack. Due to using a fixed-size shadow stack, it will cause a
buffer-overflow in TSAN when too many such frame leaks happen. This can
cause arbitrary memory to be overriden, leading to awkward crashes.
This is especially an issue on the "iso-stres" builder because it
launches - in the same process - *many* small tests, more easily hitting
that limit.
This CL will workaround the TSAN issue by making runtime call save it's
state via setjmp() and make exception throughing process go via
longjmp() (which TSAN will intercept) before actually calling the
JumpToFrame stub.
=> This will ensure the TSAN shadow stack is correctly maintained.
The [jmp_buf]'s encoding of register state is non-trivial (e.g. it uses
XOR'ing of the actual saved state under certain glibc versions). So we
store any state we need to pass to the target of the `longjmp()` on the
[Thread] instead of overriding the [jmp_buf]s register state with the
arguments.
Issue https://github.com/dart-lang/sdk/issues/47472#issuecomment-948235479
TEST=vm/dart{,_2}/regress47472_test.dart
Change-Id: I4b6f0d4eacef85487c55999021d72a6d932facad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221083
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Now that Types are no longer created for top level classes, the type
class id is guaranteed to fit in ClassIdTagType. That means we can
reduce the size of Type objects slightly as well as avoid the need to
tag and untag the type class id.
Also fixes an issue uncovered in GenerateSubtype1TestCacheLookup where
the retrieved super type was not first checked to be non-null.
TEST=Refactoring, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-ia32-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try,vm-kernel-precomp-linux-debug-simarm64c-try
Change-Id: I851a827843e34eac6961e0f06f888b40bf16df55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217221
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
1) double.truncate() now simply calls toInt(), omitting
truncateToDouble() call.
2) double.floor() and ceil() are now intrinsified on arm64 and AOT/x64
and generated using DoubleToInteger instruction.
3) DoubleToInteger instruction is extended to support floor and ceil.
On arm64 DoubleToInteger is implemented using fcvtms and fcvtps
instructions. On x64 DoubleToInteger is implemented using roundsd
under a check if it is supported (with a fallback to a stub and a
runtime call).
AOT/x64:
Before: BenchFloor(RunTime): 318.82148549569655 us.
After: BenchFloor(RunTime): 133.29430189936687 us.
TEST=ci
Closes https://github.com/dart-lang/sdk/issues/46876
Closes https://github.com/dart-lang/sdk/issues/46650
Change-Id: I16ca18faf97954f8e8e25f0b72a2bbfac5bdc672
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212381
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This CL introduces immutable maps and sets in the VM backend but does
not yet target them from the frontend. The changes are tested by unit
tests constructing these immutable maps and sets.
This CL introduces immutable variants of the hash map and set in
compact_hash.dart and recognizes them in the VM. The immutable ones
use a different mixin with a different recognized method for accessing
members.
* Data list is an immutable list with a different cid. (Otherwise the
optimizer notices that immutable and mutable lists cannot be equal.)
* Index is a nullable mutable typed data. (Otherwise optimizer removes
necessary null checks.)
* Index should use a store-release barrier when written to.
Multiple isolates might lazily compute the index for const sets and
maps. This is fine because all identityHashCodes and hashCodes are
guaranteed to be race-free. The later isolates will override the index
pointer with an identical index.
This CL does not introduce support for using these immutable maps and
sets in AOT (clustered_snapshot) and in messages to other isolates
(message_snapshot) because that is harder to test with unit tests. That
will be added in the follow-up CL.
Design doc: go/dart-vm-const-maps
TEST=runtime/vm/object_test.cc
Bug: https://github.com/dart-lang/sdk/issues/45908
Change-Id: I4042179c15e8b37692d3255655351c01c7124991
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210860
Reviewed-by: Martin Kustermann <kustermann@google.com>
In the unoptimized code unboxed double inputs are now unboxed in the
instruction prologue, and unboxed double output is boxed in the
instruction epilogue. Double values are still always boxed on the
expression stack in the unoptimized mode.
Support for unboxed doubles in unoptimized code allows us to have
one implementation of double and dart:math built-in functions,
shared across optimized and unoptimized modes.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/46650
Change-Id: Ic1e42b72ee80cb2c12019b7abab8111240b05efd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211302
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Adds GetNativeField(o,i) to access NativeFieldWrapperClass{1-4}
native fields directly in Dart instead of through VM API.
This will allow us to pass `this` as a raw pointer instead of as
a Dart handle which needs to be unwrapped.
This will in turn allow ffi native calls for instance methods.
TEST=Adds vm/cc/DartAPI_NativeFieldAccess
Bug: https://github.com/dart-lang/sdk/issues/43889
Change-Id: I8006c735569fcbe5e49915a4c7e72c4ca85b356e
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205482
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
`_CompactLinkedHashSet` now extends `_HashVMBase`. The class hierarchy
is organized as mixins similar to LinkedHashMap to accomodate for the
other Sets not extending `_HashVMBase`.
Also, rearranges some code so that introducing ImmutableHashMap and
ImmutableHashSet is easier.
1) snapshot.h and snapshot.cc now have a MapReadFrom, MapWriteTo,
SetReadFrom, and SetWriteTo to facilitate code sharing between
mutable and immutable implementations similar to ArrayReadFrom and
ArrayWriteTo.
2) Macros for CLASS_LIST_MAPS and CLASS_LIST_SETS to facilitate
treating mutable and immutable implementations with the same handle.
Also similar to Array.
Clustered snapshots for HashMaps is currently dead code. This CL makes
it explicit by marking these as unreachable. Immutable maps and sets
will end up in the clustered snapshot in follow up CLs.
Bug: https://github.com/dart-lang/sdk/issues/36077
Bug: https://github.com/dart-lang/sdk/issues/45908
TEST=runtime/vm/object_test.cc
TEST=tests/**_test.dart on many bots
Change-Id: If3cc5ebb3138535aeb0d5e06d9da3d1c9fb2deb2
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206222
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Previously there were several pieces of information shared between
both FunctionType and Function, mostly in the packed fields, but
named argument names were also kept in both places.
Now the FunctionType is the primary source for this information, with
the Function only keeping the names of positional arguments, which are
discarded in AOT snapshots.
This does mean extra work to access this information via the function
object, but for the most part, this information is only accessed in the
compiler or during dynamic lookups or checks in the runtime.
After adding the count of type parameters to the packed information
in FunctionType, the packed information has been split into two pieces:
one for parameter counts, another for type parameter counts. This
split does not increase the size of UntaggedFunctionType, as there
were 2 bytes available in the existing padding.
Changes on flutter gallery in release mode:
* ARM7 code size: total -0.91%, readonly -0.22%, isolate -4.32%
* ARM7 heap size: total -2.00%
* ARM8 code size: total -0.93%, readonly -0.22%, isolate -4.32%
* ARM8 heap size: total -2.12%
Changes on flutter gallery in release-sizeopt mode:
* ARM7 code size: total -0.24%, readonly -0.08%, isolate -1.49%
* ARM7 heap size: total -0.88%
* ARM8 code size: total -0.26%, readonly -0.11%, isolate -1.49%
* ARM8 heap size: total -1.01%
TEST=Refactoring, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ic4d59a7b4acca039a5647f9163e716f6019163f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203241
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Vyacheslav Egorov