The CallMarshaller is used not only from the compiler, but also
from the FfiCall runtime entry used by the interpreter. Since it
only has one use of the thread's CompilerState, looking up the
TypedData class, move the storage of that class from the compiler
state to the object store and remove this dependency.
TEST=ffi/address_of_array_generated_test
ffi/address_of_cast_test
fii/address_of_struct_generated_test
ffi/address_of_typeddata_generated_test
Fixes: https://github.com/dart-lang/sdk/issues/61913
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Ia4fc5d9ecef370aa9476b998e37cc4ae94ee447f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498563
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This is a reland of commit 88496ba1c3
Fixes on top of the original change:
* Closure layout is changed to avoid gap between fixed fields and
variable-length elements on compressed pointers architecture.
This gap was causing crashes in the GC when scanning closure
objects.
* pkg/vm_snapshot_analysis/test/instruction_sizes_test is fixed
on arm64 by decreasing threshold for detecting size changes.
Original change's description:
> [vm,dart2bytecode,modular_aot] Variable-length closure objects
>
> Extend closure objects with variable number of elements to capture.
> This is needed to support capturing multiple independent contexts
> after capturing is computed in the front-end.
>
> The following fixed Closure fields are moved into variable-length
> elements:
> - delayed type arguments;
> - instantiator type arguments;
> - function type arguments;
> - context.
>
> Number of elements and presence/indices of various type arguments
> are encoded into the new length_and_flags field in the Closure.
>
> Most closure objects don't need any of the type arguments so this
> change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I8685e632e2d0832766ecdc470f3cf9a6b880de48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494243
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Extend closure objects with variable number of elements to capture.
This is needed to support capturing multiple independent contexts
after capturing is computed in the front-end.
The following fixed Closure fields are moved into variable-length
elements:
- delayed type arguments;
- instantiator type arguments;
- function type arguments;
- context.
Number of elements and presence/indices of various type arguments
are encoded into the new length_and_flags field in the Closure.
Most closure objects don't need any of the type arguments so this
change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I7ca5cec0fd8725c432a01d51781fb14e803997dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489482
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Add a mutex to guard execution of vm:shared field initialization.
TEST=isolate_group_shared_init_test
BUG=dartbug.com/60699
Change-Id: If544351fc26bfcc7fb9703954efe785989d488bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/431742
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
With b32e5e5a91 under --experimental_shared_data flag, `LoadStaticField` calls initializer stub to confirm presence of isolate when accessing non-shared static fields. This breaks the role that `calls_initializer` attribute/parameter used to play where it was specifically describing neccessity to call initializer, which is important in how `LoadStaticField` instruction is used in static field setters - it should not attempt to call initialzer.
This CL fixes the problem by renaming current `calls_initializer` `LoadStaticFieldInstr` attribute to `does_slow_checks`, and "adding" separate `calls_initializer` parameter to the instruction to signify specifically whether initializer has to be called.
TEST=ci
Change-Id: Ib81424864c98d889fb6dbb9dd15dac9a39c530f8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429062
Reviewed-by: Slava Egorov <vegorov@google.com>
For out-of-memory events vm reserves (and if used, then replenishes) small chunk of memory for handling those, so preallocation of oom and stacktrace is not needed.
Unwind error can be created once as part of vm launch, kept in vm isolate.
This should reduce overhead of starting new isolate.
TEST=ci
Change-Id: Ibe2c85f250a521a8e838cc5690984fe9fa287a98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/419261
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
This reverts commit 562af5de35.
Reason for revert: regression, loses exact type for initial value?
Original change's description:
> [vm, compiler] Treat _uninitializedData/Index as effectively const.
>
> This avoids static field initialization checks in the default map and set constructors, which in turn avoids write barriers and the frame build.
>
> TEST=ci
> Change-Id: Ie12840ae1799cd97645b2132ad94ec6525126f74
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/398760
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: I41af0247277f6456ba9c5dfcdb8da43cebc64cc0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/400222
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This avoids static field initialization checks in the default map and set constructors, which in turn avoids write barriers and the frame build.
TEST=ci
Change-Id: Ie12840ae1799cd97645b2132ad94ec6525126f74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/398760
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
With sound null safety front-end guarantees that all logical
expressions have a non-nullable bool type, so legacy AssertBoolean
checks are no longer needed and can be removed.
TEST=ci
Change-Id: If952da7bd0ac83c43de3e5d98845c5e5d8d29f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382744
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This opens possibility for other core libraries to access implementation
details of compact hash maps/sets and call special "core-library-only"
methods.
TEST=ci
CoreLibraryReviewExempt: VM only library change
Change-Id: I1d7524932c34e6fbe2428853dd547d58bef2d061
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/379840
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Decorating a field with 'vm:shared' pragma makes values in this field accessible to all isolates in an isolate group.
Introduce `channel` to the `Version` class so that the pragma can only be enabled on main and dev channels.
TEST=shared_test, shared_fail_without_flag_test
BUG=https://github.com/dart-lang/sdk/issues/55991
Change-Id: I843c9f0d2ffc9f2ced7ddc4006bb6f9ca4e2ddf4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/370064
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
This change removes support for external strings from the VM along with
Dart_NewExternalLatin1String, Dart_NewExternalUTF16String and
Dart_IsExternalString Dart C API functions.
External strings are not used by the VM nor any known embedder, but
Dart VM was paying the maintenance and performance price for
the external string implementation classes.
TEST=ci
Change-Id: I094cd2d2b7ec0840e9f09e1ca9e5a7acd4e78c28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358760
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Previously, method extractors (getter functions which return
tear-offs) where implemented in the following way:
Method extractor body is an asm intrinsic which loads a few registers
and jumps to BuildMethodExtractor stubs.
BuildMethodExtractor stub loads more registers, calls AllocateClosure
stub and initializes a few fields of a closure (this stub also used
to call AllocateContext but it is no longer needed for tear-offs).
AllocateClosure allocates a closure object and initializes closure
fields.
Instead of doing 2 hops to the object allocation, method extractor
body now calls AllocateClosure stubs directly which initializes
all closure fields. This saves a jump and a handful of instructions
to load certain things on registers and set closure fields.
Assembler intrinsic for method extractor body and
architecture-specific BuildMethodExtractor stubs are now removed.
Also, as method extractors are really tiny, they are now always
inlined.
TEST=ci
Change-Id: Icfeed18414659a76c7d24d3f6f53a76c91e5bbd3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356302
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Previously, they were implemented as native methods with special
replacements in the inliner.
Instead, create force-compiled versions of the original inliner
replacements and use those instead of native methods, unless the
flow graph compiler doesn't support unboxing the requested element type.
In that case, the force-compiled version just calls a native method,
and we only keep the native methods that might be needed (that is,
for double/SIMD element access).
Also, revert the change in 26911a6176, since now the _getX/_setX
methods are appropriately inlined instead of failing to inline due
to being native methods.
TEST=vm/dart/typed_list_index_checkbound_il_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-simarm_x64-try,vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try
Change-Id: I4840883d1fc12b36a450803da339406bec149044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330786
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Original change in patchset 1.
The reland contains a fix for handles. The objects passed in handles
must live on the stack (or in the heap) so that that memory location
can be used as a handle.
This CL introduces a new allocation policy `RequiresRegister` which
forces constants to be spilled to the stack during register
allocation.
Note: Parameters to FfiCall instructions are currently not used in a
way that can make the parameters assigned to registers. So only
constants are treated currently. If we have a way to reproduce params
being assigned to registers, then we can force spilling for
non-consts as well in the register allocator.
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
TEST=tests/ffi/vmspecific_ffi_native_handles_test.dart
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
CoreLibraryReviewExempt: Internal FFI implementation changes
Change-Id: Ia1401b335524dcbf50c663a8770d9a02802923df
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333841
Reviewed-by: Slava Egorov <vegorov@google.com>
This reverts commit e16bb210d2.
Reason for revert: Indication this caused engine test
failures of the kind:
```
[0;32m[ RUN ] [mEmbedderA11yTest.A11yTreeIsConsistentUsingV1Callbacks
../../flutter/shell/platform/embedder/tests/embedder_a11y_unittests.cc:639: Failure
Expected equality of these values:
std::strncmp(kTooltip, node->tooltip, sizeof(kTooltip) - 1)
Which is: 116
0
```
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
>
> TEST=runtime/vm/compiler/backend/il_test.cc
> TEST=tests/ffi/function_*_native_(leaf_)test.dart
> TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/47625
> Closes: https://github.com/dart-lang/sdk/issues/51618
> Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
> CoreLibraryReviewExempt: Internal FFI implementation changes
> Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Icc87a6ca33bffecabb15c6b168a06ccc38c2fe5b
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333840
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL removes static fields for storing the `@Native`'s function
addresses. Instead, the function addresses are stored in the object
pool for all archs except for ia32. ia32 has no AOT and no AppJit
snapshots, so the addresses are directly embedded in the code.
This CL removes the closure wrapping for `@Native`s. Instead of
`pointer.asFunctionInternal()()` where `asFunction` returns a closure
which contains the trampoline, the function is compiled to a body
which contains the trampoline `Native()`. This is possible for
`@Native`s because the dylib and symbol names are known statically.
Doing the compilation in kernel_to_il instead of a CFE transform
enables supporting static linking later. (The alternative would have
been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
instead of constructing the IL in kernel_to_il.
To enable running resolution in ia32 in kernel_to_il.cc, the
resolution function has been made available via
`runtime/lib/ffi_dynamic_library.h`.
Because the new calls are simply static calls, the TFA can figure
out const arguments flowing to these calls. This leads to constant
locations in the parameters to FfiCalls. So, this CL also introduces
logic to move constants into `NativeLocation`s.
TEST=runtime/vm/compiler/backend/il_test.cc
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
Closes: https://github.com/dart-lang/sdk/issues/51618
Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
CoreLibraryReviewExempt: Internal FFI implementation changes
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The main contribution of this CL is unification of disparate
handling of various functions like `Future.timeout`,
`Future.wait`, `_SuspendState.createAsyncCallbacks` and
`_SuspendState._createAsyncStarCallback` into a single
`@pragma('vm:awaiter-link')` which allows Dart developers
to specify where awaiter unwinder should look for the next
awaiter.
For example this allows unwinding to succeed for the code like this:
Future<int> outer(Future<int> inner) {
@pragma('vm:awaiter-link')
final completer = Completer<int>();
inner.then((v) => completer.complete(v));
return completer.future;
}
This refactoring also ensures that we preserve information
(including Function & Code objects) required for awaiter
unwinding across all modes (JIT, AOT and AOT with DWARF stack
traces). This guarantees users will get the same information
no matter which mode they are running in. Previously
we have been disabling awaiter_stacks tests in some AOT
modes - which led to regressions in the quality of produced
stacks.
This CL also cleans up relationship between debugger and awaiter
stack returned by StackTrace.current - which makes stack trace
displayed by debugger (used for stepping out and determinining
whether exception is caught or not) and `StackTrace.current`
consistent.
Finally we make one user visible change to the stack trace:
awaiter stack will no always include intermediate listeners
created through `Future.then`. Previously we would sometimes
include these listeners at the tail of the stack trace,
which was inconsistent.
Ultimately this means that code like this:
Future<int> inner() async {
await null; // asynchronous gap
print(StackTrace.current); // (*)
return 0;
}
Future<int> outer() async {
int process(int v) {
return v + 1;
}
return await inner().then(process);
}
void main() async {
await outer();
}
Produces stack trace like this:
inner
<asynchronous suspension>
outer.process
<asynchronous suspension>
outer
<asynchronous suspension>
main
<asynchronous suspension>
And when stepping out of `inner` execution will stop at `outer.process`
first and the next step out will bring execution to `outer` next.
Fixes https://github.com/dart-lang/sdk/issues/52797
Fixes https://github.com/dart-lang/sdk/issues/52203
Issue https://github.com/dart-lang/sdk/issues/47985
TEST=ci
Bug: b/279929839
CoreLibraryReviewExempt: CL just adds @pragma to facilitate unwinding
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-dwarf-linux-product-x64-try
Change-Id: If377d5329d6a11c86effb9369dc603a7ae616fe7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311680
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Previously, inliner was able to inline closure calls only when closure
was allocated in the current function (or in a function which was
inlined so far). This is not true for local functions declared in outer
functions (they are allocated at declaration site).
Now, when calling local functions, target of closure call is always
known regardless of where the function is declared, so calls to local
functions can be inlined.
TEST=runtime/tests/vm/dart/inline_local_functions_il_test.dart
Fixes https://github.com/dart-lang/sdk/issues/52695
Fixes https://github.com/dart-lang/sdk/issues/15558
Change-Id: I0e51dbaf63a6d4427bff366b40414ca5fac3d418
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311465
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is a reland of commit 6f4bed30cc
Original change's description:
> [vm] Fix CompileType::CanBeFuture()
>
> In certain cases compiler can omit a type check in 'await e' if it can
> prove that expression 'e' cannot be evaluated to Future.
>
> Previously, if actual type of 'e' is unknown, CompileType::CanBeFuture()
> tested if static type of 'e' is a subtype of Future.
>
> This is not correct, as static type could have a subtype which is also
> a subtype of Future, but static type itself is not a subtype of Future:
>
> class A {}
> class B implements A, Future<A> {}
>
> A e = confuse(B()); // 'e' is B, but has a static type A.
> await e; // A is not a subtype of Future, but 'e' is Future
> // and should be awaited.
>
TEST=language/async/await_flatten_test
Fixes https://github.com/dart-lang/sdk/issues/52585
Change-Id: I37add9021261c95dee5930e031187868e821c161
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309481
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 6f4bed30cc.
Reason for revert: g3 failure (b/287294480).
Original change's description:
> [vm] Fix CompileType::CanBeFuture()
>
> In certain cases compiler can omit a type check in 'await e' if it can
> prove that expression 'e' cannot be evaluated to Future.
>
> Previously, if actual type of 'e' is unknown, CompileType::CanBeFuture()
> tested if static type of 'e' is a subtype of Future.
>
> This is not correct, as static type could have a subtype which is also
> a subtype of Future, but static type itself is not a subtype of Future:
>
> class A {}
> class B implements A, Future<A> {}
>
> A e = confuse(B()); // 'e' is B, but has a static type A.
> await e; // A is not a subtype of Future, but 'e' is Future
> // and should be awaited.
>
> TEST=language/async/await_flatten_test
> Fixes https://github.com/dart-lang/sdk/issues/52585
>
> Change-Id: I270a8260224246e1f8c16eff57231363a0f25ae6
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309380
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
Change-Id: I87a08e46b4af6c57efa828c0f9041e6b5e51708e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309480
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
In certain cases compiler can omit a type check in 'await e' if it can
prove that expression 'e' cannot be evaluated to Future.
Previously, if actual type of 'e' is unknown, CompileType::CanBeFuture()
tested if static type of 'e' is a subtype of Future.
This is not correct, as static type could have a subtype which is also
a subtype of Future, but static type itself is not a subtype of Future:
class A {}
class B implements A, Future<A> {}
A e = confuse(B()); // 'e' is B, but has a static type A.
await e; // A is not a subtype of Future, but 'e' is Future
// and should be awaited.
TEST=language/async/await_flatten_test
Fixes https://github.com/dart-lang/sdk/issues/52585
Change-Id: I270a8260224246e1f8c16eff57231363a0f25ae6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309380
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.
More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing
I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.
Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The ffi-callback related information on the [Thread] object is metadata
corresponding to ffi-callback-trampoline [Function] objects. There is
nothing thread or isolate specific about it.
Moving it away from [Thread] is needed because an [Isolate] can
have different [Thread] objects across its lifetime (see [0]): When
the stack of an isolate is empty, we reserve now the right to
re-cycle the [Thread]. If the isolate later runs again, it may
get a new [Thread] object.
This CL moves this information from [Thread] to the [ObjectStore]. In
addition we make the compiler be responsible for populating this
metadata - instead of doing this per-call site of
`Pointer.fromFunction()`. It will be preserved across snapshot writing
& snapshot reading (for AppJIT as well as AppAOT).
Similarly the JIT trampolines that are on Isolate aren't isolate
specific and can go to [IsolateGroup]. This simplifies doing the above
as the compiler can allocate those as well.
The effect is that [Thread] object gets smaller, GC doesn't have to
visit the 2 slots per-thread. It comes at expense of 2 more loads
when invoking the callback.
[0] https://dart-review.googlesource.com/c/sdk/+/297920
TEST=Regression test is vm/ffi{,_2}/invoke_callback_after_suspension_test
Change-Id: Ifde46a9f6e79819b5c0e359c3d3998d1d93b9b1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303700
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL introduces `VarArgs` to `NativeFunction` signatures. The
`VarArgs` type takes a single type argument. This type argument is a
subtype of `NativeType` if there is a single variadic argument, and a
record with native types if there are multiple variadic arguments.
For example:
`NativeFunction<Void Function(Pointer<Char>, VarArgs<(Int32,Int32)>)>`
for calling refering to a `printf` binding with two `int32_t` arguments
passed as variadic arguments.
The logic of the native calling conventions are detailed in
https://dart-review.googlesource.com/c/sdk/+/278342.
Here we explain how this influences the FFI pipeline.
First, now that `VarArgs` is part of signatures, we have to unwrap
that when with the C types in the CFE transform and checking (analyzer
is in a separate CL), and also in the marshaller when looking up the
C type of arguments.
Second, we have to deal with `BothNativeLocations`. On windows x64,
floating point arguments must be passed both in FPU _and_ CPU
registers. For FFI calls, we solve this in the argument moves by just
copying to both locations. For FFI callbacks, we just take the FPU
register location (which avoids an extra bitcast).
Third, on System-V, we have to pass an upper bound of the number of
XMM registers used in AL. This means we instead RAX, we use R13 for the
target address. For variadic calls, we always pass 8 in AL as the valid
upper bound. We could consider passing the actual number of XMM
registers used.
We keep using RAX as default register for the function address on non-
variadic calls, because changing to R13 (the first free) register
creates more spilling in leaf calls. R13 is callee-saved while RAX is
not, so using R13 instead of RAX causes us to have to spill the value
from RAX on leaf calls.
Fourth, on both x64 and RISC-V, we pass floats in integer locations.
`EmitNativeMove` has been modified to deal with this, so that we do not
have to insert more `BitCastInstr`s.
The tests are generated by a test generator: `tests/ffi/generator/`.
The formatter doesn't support records yet, so the tests are not properly
formatted.
Bug: https://github.com/dart-lang/sdk/issues/50798
TEST=tests/ffi/*_varargs_*
Closes: https://github.com/dart-lang/sdk/issues/38578
Closes: https://github.com/dart-lang/sdk/issues/49460
Closes: https://github.com/dart-lang/sdk/issues/50858
Change-Id: I6a6296fe972527f8a54ac75a630131769e3cc540
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-win-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-debug-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-android-release-arm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,app-kernel-linux-debug-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276921
Reviewed-by: Devon Carew <devoncarew@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Adds support for `asset: 'asset-id'` in `FfiNative`s.
Symbols in assets are resolved by
1. the provided [asset] (the bulk of this CL),
2. the native resolver set with `Dart_SetFfiNativeResolver` in
`dart_api.h` (old behavior), and
3. the current process.
Symbols in assets are resolved by the Dart VM through the mapping
provided in the `vm:ffi:native_assets` library:
```
@pragma('vm:ffi:native-assets', {
"linux_x64": {
"<asset1>": [
"absolute",
"<absolute-path>"
],
"<asset2>": [
"relative",
"<relative-path>"
],
"<asset3>": [
"system",
"<system-path>"
],
"<asset4>": [
"process",
],
"<asset5>": [
"executable",
],
}
})
library;
```
There are 5 asset path types. Symbols resolved in these asset types are
resolved as follows.
A. Absolute paths are dlopened, after which the symbol is looked up.
B. Relative paths are resolved relative to `Platform.script.uri`, and
dlopened, after which the symbol is looked up.
C. System paths are treated as absolute paths. (But we might explicitly
try to use `PATH` in the future.)
D. Symbols looked up in Process assets are looked up in the current
process.
E. Symbols looked up in Executable assets are looked up in the current
executable.
The native assets mapping can be provided in three ways.
X. In the invocation to `gen_kernel` with the `--native-assets`
argument. This uses the gen_kernel entry point for compiling
kernel.
Y. By placing a `native_assets.yaml` next to the
`package_config.json` used by the isolate group. This works for
`dart <source file>` and `Isolate.spawnUri(<dart source file>)`.
This uses the kernel_service entry point for compiling kernel.
Z. By sending a String containing the native assets mapping when
invoking the kernel_service directly from within the VM. Currently,
only used for unit tests.
TEST=tests/ffi/native_assets/asset_*_test.dart (X)
TEST=tests/ffi/native_assets/infer_native_assets_yaml_*.dart (Y)
TEST=runtime/vm/ffi/native_assets_test.cc (Z)
The library is synthesized from yaml in pkg:vm as AST.
Design doc: go/dart-native-assets
Bug: https://github.com/dart-lang/sdk/issues/49803
Change-Id: I8bf7000bfcc03b362948efd3baf168838948e6b9
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264842
Reviewed-by: Martin Kustermann <kustermann@google.com>
Tess Strickland