Unlike other calls from Dynamic Modules, dynamic calls cannot be
validated entirely at compile time. While we check that the selector
used matches a selector that was allowed (either because a method with
that selector name was exposed as dynamically callable or because the
selector was allowlisted during bytecode compilation), the compiler
doesn't know statically whether the target of the call is exposed.
In prior changes we modified the annotator to add a pragma indicating
whether a member is dynamically-callable or
implicitly-dynamically-callable. Here we use that information to set a
bit on functions and their corresponding dynamic invocation forwarders,
which is verified by the interpreter to make sure the dynamic call is
still allowed.
TEST=none yet - will be added in subsequent CL (see CL chain)
Bug: b/448095881
Change-Id: I27acb4e690a68e08fe1f1ca94e0d77cc7dc4d11e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498300
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
When mixin is applied, its instance members are cloned and copies
of the members would override original members.
This change fixes bugs and improved usability of extendable mixins with
private members:
- All instance members of extendable mixins and mixin classes are
marked as can-be-overridden to allow overriding by cloned members
when mixin is applied.
- TFA no longer takes privacy into account, as private members of
mixins can be overridden by their clones in other libraries.
- Dynamic module validator always unwraps cloned members to originals
before verifying if overriding is allowed. This eliminates overriding
errors between clone(s) and original members of mixins.
TEST=pkg/dynamic_modules/test/data/mixin_private_member1,
pkg/dynamic_modules/test/data/mixin_private_member2
Fixes b/470461203
Fixes b/469094721
Change-Id: If369c42c58e5ea4d707be83b1e7078ee9a8cc3ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/470803
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Consider the following situation: member M1 is overridden by another
member M2; M2 is overridden in a dynamic module.
Members which can be overridden in a dynamic module (such as M2)
should be specified as 'can-be-overridden' in the dynamic interface.
Members which are overridden implicitly/transitively (such as M1)
are not required to be mentioned in the dynamic interface.
However, when determining possible targets for a call with
interface target M1, compiler should treat it as potentially
overridden in a dynamic module.
This change adds such handling to the VM/AOT. Dynamic interface
annotator now marks members such as M1 with
'dyn-module:can-be-overridden-implicitly' pragma, and
VM/AOT takes both can-be-overridden and can-be-overridden-implicitly
into account.
This change also simplifies handling of implicitly extenable classes
in the VM/AOT - now VM handles both extendable and implicitly-extendable
pragmas (from dynamic interface annotator) instead of recalculating
implicitly extendable classes on its own.
TEST=pkg/vm/test/transformations/dynamic_interface_annotator_test.dart
TEST=dynamic_modules_suite/implicitly_extendable
Fixes https://github.com/dart-lang/sdk/issues/59880
Change-Id: Id4570cc86303f8e45a061d696e9bca0d0b2b4b81
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/403951
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
This is a reland of commit bfc1a44527
Change how the partial specializations for AtomicBitFieldContainer
are written so that Visual Studio properly chooses them instead of
the base BitField template when appropriate.
TEST=Windows ARM64 CI trybots
Original change's description:
> [vm] Improvements in the BitField API.
>
> If no position is specified, then the bitfield starts at bit 0.
>
> The default size for bool BitFields is 1 instead of the remaining
> bits in the container.
>
> If the size of the value type is smaller than the remaining bits
> in the container, then the size of the value type is used as
> the default size instead.
>
> If a signed value is used in a non-sign-extended BitField, only
> the magnitude of the value is stored, not the sign bit. This means
> the actual size of the bitfield may be one less than the requested
> size in this case.
>
> If the requested size of the bitfield is larger than the size of the
> value type, a compile-time error is thrown. (For signed types, the
> requested size is allowed to be the size of the entire value, even if
> only the magnitude bits are stored.)
>
> Rework uses of BitFields to avoid using separate constants for
> bit positions/sizes except for macro-defined bitfields (which now
> are universally bool, and so size 1).
>
> TEST=vm/cc/BitFields_Defaults
>
> Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
> Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I5e3a9e3e2a80d5689a23a0603f8f81fac1576cd3
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-win-debug-x64c-try,vm-win-release-x64-try,vm-aot-win-release-x64-try,vm-win-release-arm64-try,vm-aot-win-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382383
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit bfc1a44527.
Reason for revert: Broke Windows build.
Original change's description:
> [vm] Improvements in the BitField API.
>
> If no position is specified, then the bitfield starts at bit 0.
>
> The default size for bool BitFields is 1 instead of the remaining
> bits in the container.
>
> If the size of the value type is smaller than the remaining bits
> in the container, then the size of the value type is used as
> the default size instead.
>
> If a signed value is used in a non-sign-extended BitField, only
> the magnitude of the value is stored, not the sign bit. This means
> the actual size of the bitfield may be one less than the requested
> size in this case.
>
> If the requested size of the bitfield is larger than the size of the
> value type, a compile-time error is thrown. (For signed types, the
> requested size is allowed to be the size of the entire value, even if
> only the magnitude bits are stored.)
>
> Rework uses of BitFields to avoid using separate constants for
> bit positions/sizes except for macro-defined bitfields (which now
> are universally bool, and so size 1).
>
> TEST=vm/cc/BitFields_Defaults
>
> Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
> Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I13bf2218ecc47a42f3a6091821037b51b56618fb
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382380
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
If no position is specified, then the bitfield starts at bit 0.
The default size for bool BitFields is 1 instead of the remaining
bits in the container.
If the size of the value type is smaller than the remaining bits
in the container, then the size of the value type is used as
the default size instead.
If a signed value is used in a non-sign-extended BitField, only
the magnitude of the value is stored, not the sign bit. This means
the actual size of the bitfield may be one less than the requested
size in this case.
If the requested size of the bitfield is larger than the size of the
value type, a compile-time error is thrown. (For signed types, the
requested size is allowed to be the size of the entire value, even if
only the magnitude bits are stored.)
Rework uses of BitFields to avoid using separate constants for
bit positions/sizes except for macro-defined bitfields (which now
are universally bool, and so size 1).
TEST=vm/cc/BitFields_Defaults
Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Decorating a field with 'vm:shared' pragma makes values in this field accessible to all isolates in an isolate group.
Introduce `channel` to the `Version` class so that the pragma can only be enabled on main and dev channels.
TEST=shared_test, shared_fail_without_flag_test
BUG=https://github.com/dart-lang/sdk/issues/55991
Change-Id: I843c9f0d2ffc9f2ced7ddc4006bb6f9ca4e2ddf4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/370064
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
gen_snapshot can now consume a concatenated dill file.
Restrictions:
* The first dill file must be the main program.
* The subsequent dill files must only contain dummy libraries with
objects which are recognized classes in the VM and not treeshaken
in the main program.
This will enable compiling Dart AOT first, then using the tree-shaking
information to tree-shake native assets, and then embedding the native
assets mapping in the kernel file before running gen_snapshot.
TEST=tests/ffi/native_assets/asset_relative_test.dart
Closes: https://github.com/dart-lang/sdk/issues/50152
Bug: https://github.com/dart-lang/sdk/issues/55377
Change-Id: Id562aa39840d5eb467198efaa4cf3152d860f8b5
q-Include-Trybots: dart/try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362381
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL introduces a way to mark all instances of a class as deeply
immutable.
In order to statically verify that all instances of a deeply immutable
class are immutable, a deeply immutable classes must have the following
properties:
1. All instance fields must
1. have a deeply immutable type,
2. be final, and
3. be non-late.
2. The class must be `final` or `sealed`. This ensures no
non-deeply-immutable subtypes are added by external code.
3. All subtypes must be deeply immutable. This ensures 1.1 can be
trusted.
4. The super type must be deeply immutable (except for Object).
Note that instances of some classes in the VM are deeply immutable
while their class cannot be marked immutable.
* SendPort, Capability, RegExp, and StackTrace are not `final` and
can be implemented by external code.
* UnmodifiableTypedDataViews do not have a public type. (It was
recently deprecated.)
See runtime/docs/deeply_immutable.md for more details.
Use case:
This enables attaching a `Dart_FinalizableHandle` to a deeply immutable
object and the deeply immutable object with other isolates in the same
isolate group.
(Note that `NativeFinalizer`s live in an isolate, and not an isolate
group. So this should currently _not_ be used with `NativeFinalizer`s.
See https://github.com/dart-lang/sdk/issues/55062 for making a
`NativeFinalizer.shared(` that would live in an isolate group instead
of in an isolate.)
Implementation details:
Before this CL, the `ImmutableBit` in the object header was only ever
set to true for predefined class ids (and for const objects). After
this CL, the bit can also be set to true for non const instances of
user-defined classes. The object allocation and initialization code has
been changed to deal with this new case. The immutability of a class is
saved in the class state bits. On object allocation and initialization
the immutability bit is read from the class for non-predefined class
ids.
TEST=runtime/tests/vm/dart/isolates/fast_object_copy2_test.dart
TEST=runtime/vm/isolate_reload_test.cc
TEST=tests/lib/isolate/deeply_immutable_*
Bug: https://github.com/dart-lang/sdk/issues/55120
Bug: https://github.com/dart-lang/sdk/issues/54885
Change-Id: Ib97fe589cb4f81673cb928c93e3093838d82132d
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/354902
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Original change in patchset 1.
The reland contains a fix for handles. The objects passed in handles
must live on the stack (or in the heap) so that that memory location
can be used as a handle.
This CL introduces a new allocation policy `RequiresRegister` which
forces constants to be spilled to the stack during register
allocation.
Note: Parameters to FfiCall instructions are currently not used in a
way that can make the parameters assigned to registers. So only
constants are treated currently. If we have a way to reproduce params
being assigned to registers, then we can force spilling for
non-consts as well in the register allocator.
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
TEST=tests/ffi/vmspecific_ffi_native_handles_test.dart
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
CoreLibraryReviewExempt: Internal FFI implementation changes
Change-Id: Ia1401b335524dcbf50c663a8770d9a02802923df
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333841
Reviewed-by: Slava Egorov <vegorov@google.com>
This reverts commit e16bb210d2.
Reason for revert: Indication this caused engine test
failures of the kind:
```
[0;32m[ RUN ] [mEmbedderA11yTest.A11yTreeIsConsistentUsingV1Callbacks
../../flutter/shell/platform/embedder/tests/embedder_a11y_unittests.cc:639: Failure
Expected equality of these values:
std::strncmp(kTooltip, node->tooltip, sizeof(kTooltip) - 1)
Which is: 116
0
```
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
>
> TEST=runtime/vm/compiler/backend/il_test.cc
> TEST=tests/ffi/function_*_native_(leaf_)test.dart
> TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/47625
> Closes: https://github.com/dart-lang/sdk/issues/51618
> Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
> CoreLibraryReviewExempt: Internal FFI implementation changes
> Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Icc87a6ca33bffecabb15c6b168a06ccc38c2fe5b
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333840
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL removes static fields for storing the `@Native`'s function
addresses. Instead, the function addresses are stored in the object
pool for all archs except for ia32. ia32 has no AOT and no AppJit
snapshots, so the addresses are directly embedded in the code.
This CL removes the closure wrapping for `@Native`s. Instead of
`pointer.asFunctionInternal()()` where `asFunction` returns a closure
which contains the trampoline, the function is compiled to a body
which contains the trampoline `Native()`. This is possible for
`@Native`s because the dylib and symbol names are known statically.
Doing the compilation in kernel_to_il instead of a CFE transform
enables supporting static linking later. (The alternative would have
been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
instead of constructing the IL in kernel_to_il.
To enable running resolution in ia32 in kernel_to_il.cc, the
resolution function has been made available via
`runtime/lib/ffi_dynamic_library.h`.
Because the new calls are simply static calls, the TFA can figure
out const arguments flowing to these calls. This leads to constant
locations in the parameters to FfiCalls. So, this CL also introduces
logic to move constants into `NativeLocation`s.
TEST=runtime/vm/compiler/backend/il_test.cc
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
Closes: https://github.com/dart-lang/sdk/issues/51618
Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
CoreLibraryReviewExempt: Internal FFI implementation changes
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
a) Remove 2 fields in our object representation:
* PatchClass::library_data_
* Library::kernel_data_
=> This saves O(#libraries + #patch-classes) which can amount up to
10+ MB for big apps, as we save not just the slots but the
[ExternalTypedData] objects they used to reference.
Instead we'll compute the KernelLibraryData when we need it by making a
[TypedDataView] of sub parts of the component.
b) We make a kernel binary be represented by a single [ExternalTypedData].
Whenever we need a sub-part of a kernel binary (e.g. one for each
component for concatinated kernels, or one for a library inside a
component) we use proper [TypedDataView]s for that.
c) As we sometimes need to create a view of only a particular library
within a component we need to find start/end of a library based on
index.
=> We store the library index instead of the library offset on
Library/PatchClass.
=> We can easily derive the start/end of a library from it's index by
looking at the kernel component encoding.
d) We make the [Reader] object work purely based on a pointer - instead
of making it have if/else when reading bytes (either from pointer or
from a view).
e) We make the [KernelProgramInfo] store the kernel_component and
various TD views into it.
TEST=ci
Change-Id: Ibe160881ff48635e834c3d647a977a144b5d0565
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313561
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
** KernelProgramInfo
The VM can load multiple kernel files into an isolate. Each kernel file
is represented as a [KernelProgramInfo] object. Loading a kernel blob
will result in a tree of objects
Library
- TopLevelClass
+ Field
+ Function
+ Class
+ Field
+ Function
+ ... closure functions ...
The entire tree belongs to one [KernelProgramInfo] object. There are
two exceptions
* after a hot-reload we inject [PatchClass]es (as owners of
old [Field]/[Function] objects) that identify the old kernel data
* expression evaluation functions will get a special data array with
information about the expression evaluation kernel data
=> We attach the [KernelProgramInfo] to the root of the tree, namely
the [Library]
=> We introduce `{Field,Function,Class,Library}::KernelProgramInfo()`
that will get that object - by walking up the chain.
=> We introduce `UntaggedPatchClass::kernel_program_info_` field that
allows saving the old [KernelProgramInfo] (on which the old
[Field]/[Function] objects are based upon) in the [PatchClass].
=> We include the [KernelProgramInfo] in the expression evaluation
data array.
** Scripts
A [Script] object should represent a .dart source file. It contains the
source, line offsets, etc. of that dart source file. The contents of a
dart source file may be used by many classes/libraries or even many
kernel blobs (e.g. via our copying mixin application).
When we build a tree like above, we'll attach a reference to a
[Script] object to [Class] objects. But members of a class may come from
a different script, in which case we inject an intermediary [PatchClass]
that references that other script.
When the compiler frontend loads kernel, builds flow graph, etc we walk
down the kernel binary (the tree above) - all belonging to the same
[KernelProgramInfo] object, but sub-parts may belong to different
[Script]s.
=> We use the existing [ActiveClass] class that keeps track of the
current class & member inside the class we're operating on.
=> Whenever we need the script we get it from the active member
function/class.
** Misc
We make kernel related classes (e.g. [KernelReaderHelper]) independent
of [Script].
We make the [Script] objects hold on to the [KernelProgramInfo] they are
based upon (e.g. for lazy source reading). But since a given [Script]
doesn't belong to a particular kernel blob (multiple kernel blobs may
contain information about the same script) we don't expose this in the
public API.
=> In the future we could have a unique [Script] object per url (instead
of having several as we do now)
TEST=ci
Change-Id: I7bd698f497b0bd8ab8e546540bb1dad5b5a6dd48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313381
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Decorate Zone, Future, Completer, Timer and Stream with newly-introduced pragma to make sure that message verification stops earlier, produces shorter retaining path for the user.
BUG=https://github.com/dart-lang/sdk/issues/51722
TEST=send_unsupported_objects_test,isolate_exit_unsendable_test.dart
CoreLibraryReviewExempt: vm-specific pragmas
Change-Id: I499eea542d228ac9cf0797a682664f93f360dc80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289027
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead of peeking into constant table and then delaying scanning of
constants by putting it in an array, which is walked again in some
future point, we simply read the annotation constants entirely without
requiring const evaluation. This works fine for pragma annotations the
VM is interested in - as there's no user-defined classes involved.
-> Loading kernel will no longer require constant evaluation to work.
Motivation for this is that [0] wants to make this delayed annotation
scanning logic even more complicated, so I prefer to remove it entirely.
[0] https://dart-review.googlesource.com/c/sdk/+/289027
TEST=ci
Change-Id: Ib859480107b6cf119d66035e66ec161ed11ddb32
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290502
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The pragma can be used to annotate functions which should not appear in
stack traces unless --show-invisible-frames is passed to the VM.
TEST=vm/dart{,_2}/invisible_function_pragma_test
Change-Id: I807359146f16b67913697f07416fbe8ce9aaa6e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233381
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
As part of deprecating support for native extensions we are also
migrating away from legacy VM-specific `native 'name'` syntax
towards metadata based encoding which does not require any special
syntax.
This CL is a step 1 in migration:
- introduces support for `@pragma('vm:external-name', 'name')`
which serves as a direct replacement for `native 'name'`;
- all core libraries and tests are migrated to use the annotation;
Once this CL lands and rolls we will edit internal and external embedders
to eliminate uses of the native keyword (step 2) and finally remove
support for native keyword across our parsers (step 3).
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/28791
Change-Id: Id6dea878db82dd4fd81149243c425b5c5dc6df86
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212461
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now that support for native extensions (aka dart-ext:*) has been
removed, the kernel loader is guaranteed not to call out to embedder
anymore (it did so before for loading shared libraries).
This allos us to clean up some code as well as extend the scope of our
NoActiveIsolateScope.
TEST=ci
Change-Id: I5c8c9ee47298fdd316583ba3aed74772c1c7783f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212296
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
In the codebase, we have several different interfaces for instance
methods that return a hash:
- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;
This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Since loading kernel will affect all isolates within a group, there
should be nothing isolate-specific inside the kernel loader. To ensure
we don't have (or introduce) any accidental uses, we add a
NoActiveIsolateScope to the main parts of kernel loading.
Only loading of native extensions - done by the kernel loader -
currently requires an active isolate. The reason for this is that
loading native extensions happen by calling out to embedder, which calls
back into the VM using the our embedding API (which currently requires
an active isolate)
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing implementation.
Change-Id: I96e64dbfe7148b76b8fa006fe5dbde8c1f904504
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184269
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
When handling hot reload via library tag handler's Kernel tag
VM must guarantee that external typed data it receives from the
embedder stays alive for as long as VM has any structures
referencing this kernel binary. This is achieved by attaching
original typed data to KernelProgramInfo objects containing
the views into it.
Unfortunately kernel::Reader APIs were somewhat unsafe: allowing
to construct reader object from Program's raw buffer and forget to
set the link betwen the reader and original typed data.
This created a situation where reloading using a multicomponent
Kernel binary would result in KernelProgramInfo objects without
link to the original typed data, which in turn leads to premature
finalization of external typed data and subsequent crashes
when trying to use delete kernel binary.
This CL reworks kernel::Reader API (by introducing kernel::ProgramBinary
wrapper) to make it safer and make sure that connection is preserved.
TEST=vm/cc/IsolateReload_RegressB179030011
Bug: b/179030011
Change-Id: I05f8c31c3cb8e67de6e94a20d9501a5f476b7e27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182280
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
As part of making lightweight isolates work in JIT, we will need to
ensure that whenever a new static field is loaded, it will get a new
slot on every isolate and the possibly initial static value is set in
all isolates's field state.
In order to achieve this we first move field registration out of
Field::New*() - to the place where the field is created and the initial
static field value is known.
In a future CL we will make a field registration be performed on all
isolates in a group.
TEST=CL is internal refactoring, existing test coverage.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I802ae55b841be173733dbd0ad3dbffeebd5c1889
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173260
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Loading unit metadata is expanded to create an object representing each loading unit, and each library to reference the loading unit it is a part of.
To be used at compile-time to help assign objects to snapshots.
To be used at runtime to track which units have been loading and the base objects for cross-snapshot references.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icd3fa95ffd5949bd8f7f84a6f750c4c515de0dba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151865
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Static methods and constructors may receive unboxed parameters.
This change adds a bitmap to the RawFunction object indicating
which parameters are unboxed and their types. Also, the stack frame
bitmap was updated to include new bits for those parameters.
The ArgumentsDescriptor object was updated to store not only the number
of arguments, but also their size, since 32-bit platforms would use two
words for each int64 and double value.
In addition, static methods that return non-nullable integers
and doubles may return these values unboxed.
Dart AOT x64:
TypedData.Uint8ClampedListViewVarBench +11.73%
DeltaBlue +9.32%
Meteor (Intel Xeon) +4.64%
Flutter release android-armv8:
velocity_tracker_bench -2.32% (less is better)
flutter_gallery_readonlydata_size +0.18%
flutter_gallery_total_size +0.13%
flutter_gallery_instructions_size +0.12%
Issue https://github.com/dart-lang/sdk/issues/33549
Change-Id: I97e310f8f30213920af2333fbdedf193d03db5c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135313
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Sigmund Cherem