Enables passing inner pointers to typed data in FFI leaf calls.
This works for typed data's in the Dart heap, external typed datas
(constructed from `Pointer.asTypedList`), and typed data views.
Notable implementation details:
1. The Dart signature is used in the Marshaller now. This means it
needs to keep track of whether there's a pointer argument in the
signature (`asFunction`) or not (`@Native`).
2. Unwrapping is done in `FfiCallInstr::EmitNativeCode` before moving
the arguments to their native location. This ensures we can use
the assembler logic to load the `TypedDataBase::data` field.
3. The `XXXList` user visible classes don't have predefined cids.
So the implementation uses symbols for comparison.
4. The type checking logic takes `isLeaf` as input to reject typed
data. This leads to an error message about the type not accepted.
Alternatively, we could consider adding an error message that
specifically says the function should be leaf.
5. To cover all calling convention variants, tests are generated with
up to 20 arguments.
TEST=pkg/analyzer/test/src/diagnostics/ffi_unwrap_typed_data_test.dart
TEST=tests/ffi/unwrap_typeddata_generated_native_test.dart
TEST=tests/ffi/unwrap_typeddata_generated_test.dart
TEST=tests/ffi/vmspecific_static_checks_typeddata_test.dart
Closes: https://github.com/dart-lang/sdk/issues/44589
Change-Id: Ia78f18bf3238d42ac6882929b441f6dc432fcefe
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338620
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This adds the `Native.addressOf` function, which takes a constant tear-
off of a method annotated with `@Native` as a parameter and returns a
pointer to the underlying C function.
The CFE will resolve these calls in two steps: First, the existing
transformer for `@Native` methods adds a pragma describing the fully-
resolved native annotation (with the asset id inferred from the library
if not given explicitly). Then, the FFI use sites transformer rewrites
calls to `Native.addressOf` by searching for this pragma on the passed
function and passing the `Native` constants to `Native._addressOf`. The
latter method is implemented in the VM, which can re-use existing parts
already used for `@Native` calls.
An alternative implementation strategy would have been to forward
`addressOf` calls to `Native.ffi_resolver_function` directly without
any special consideration in the VM. However, the chosen approach makes
it easier to support static linking in the future, as this requires
unresolved symbols in the generated assembly.
Closes https://github.com/dart-lang/sdk/issues/50552
TEST=pkg/vm/testcases/transformations/ffi/ffinative.dart
TEST=tests/ffi/native_assets/asset_*_test.dart
TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart
TEST=pkg/analyzer/test/src/diagnostics/ffi_native_test.dart
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-kernel-precomp-linux-release-x64-try
CoreLibraryReviewExempt: VM & dart2wasm only feature
Change-Id: Ic8e3a390146dffd44c95578f975a4472db79a0ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333920
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Allows FFI to reuse FlowGraphCompiler::EmitMove in contexts where the delta between SP and FP isn't accurately described by StackSize.
TEST=dart2js codesize unchanged
Change-Id: Idf0643af3debfc77564672d6d16491990b87611c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/339040
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Conditional constant propagation doesn't update environments and may
also replace redefinition instructions with constants
without updating environment uses of their original definitions.
This causes discrepancy between arguments of a call and its environment.
It is harmless and corresponding check in the flow graph checker can be
relaxed.
TEST=runtime/tests/vm/dart/regress_54128_test.dart
Fixes https://github.com/dart-lang/sdk/issues/54128
Change-Id: I6e68e0525b7f110400435ac071934dd9a02eebd4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338941
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Implementation of stores and their inlinings had some
stuff left over their since Dart 1:
* No need to insert null-checks (in sound null-safety mode).
* Since Dart 2 there is no need to insert (speculative)
cid checks. Inputs are guaranteed to be a value of a
supported implementation type. Inserting narrow speculative
checks for Smis is actually leads to worse code in JIT.
* There is no need to convert incomming integer values to
smaller representation - the store will take care of it. This was
left over from Dart 1 times when incomming integer could be _Bigint.
TEST=existing tests
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-x64-try,vm-aot-mac-product-arm64-try,vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-android-release-arm_x64-try,vm-aot-android-release-arm64c-try
Change-Id: I72cdaaecc524f1dccc63825df4f7b71241ab47a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338600
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
When generating code for deferred units compiler
can't emit PC relative call to the shared Mint allocation
stub. This causes compiler to emit an indirect call
through a Code object. Such calls clobber CODE_REG
which is actually an allocatable register.
Fix this by using non-allocatable register instead of
CODE_REG when generating indirect calls through
Code object in AOT mode. Callees don't expect
anything useful in CODE_REG anyway because AOT
calling convetion does not use it.
TEST=vm/cc/{BranchLinkPreservesRegisters,JumpAndLinkPreservesRegisters,CallCodePreservesRegisters}
Bug: b/242559057
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-linux-debug-simriscv64-try,vm-ffi-qemu-linux-release-riscv64-try
Change-Id: Ib1fdc1c104d0269d41bb1ab9cbe292ad28c7cd49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338127
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Original change in patchset 1.
The reland contains a fix for handles. The objects passed in handles
must live on the stack (or in the heap) so that that memory location
can be used as a handle.
This CL introduces a new allocation policy `RequiresRegister` which
forces constants to be spilled to the stack during register
allocation.
Note: Parameters to FfiCall instructions are currently not used in a
way that can make the parameters assigned to registers. So only
constants are treated currently. If we have a way to reproduce params
being assigned to registers, then we can force spilling for
non-consts as well in the register allocator.
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
TEST=tests/ffi/vmspecific_ffi_native_handles_test.dart
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
CoreLibraryReviewExempt: Internal FFI implementation changes
Change-Id: Ia1401b335524dcbf50c663a8770d9a02802923df
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333841
Reviewed-by: Slava Egorov <vegorov@google.com>
Replacing index checks with a single GenericCheckBound instruction
currently causes performance regressions on 32-bit ARM. Undo that
optimization there for now, instead inlining the original Dart code
into the indexing operations.
TEST=ci
Issue: https://github.com/flutter/flutter/issues/138689
Change-Id: I7e67966b576a9d1b5f510e17b44f853f23c98a91
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/337480
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Create a _typedDataIndexCheck abstraction for the conditional throwing
of IndexError in TypedData indexing operations. Recognize that
abstraction in the inliner and replace it with the equivalent CheckBound
instruction.
Do the same for byte offset checking in ByteData operations.
TEST=vm/dart/typed_list_index_checkbound
Change-Id: Ia6c4a3b7ae4667af69aa5bf214d4f187557dac06
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-linux-debug-x64-try,vm-eager-optimization-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330800
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Switch the Windows ARM64 builds to use MSVC. Clang disagrees with itself about handling of small structs in variadic functions, allowing splitting between the last argument register and the stack as the callee but not as the caller.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-linux-release-arm64-try,vm-mac-debug-arm64-try,vm-mac-release-arm64-try,vm-win-debug-arm64-try,vm-win-release-arm64-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-release-ia32-try,vm-win-release-ia32-try,vm-linux-debug-x64-try,vm-linux-release-x64-try,vm-mac-debug-x64-try,vm-mac-release-x64-try,vm-win-debug-x64-try,vm-win-release-x64-try
Bug: https://github.com/dart-lang/sdk/issues/52644
Bug: https://github.com/dart-lang/sdk/issues/53829
Change-Id: I2fd6c40620a885479f11bb8528ca1e9df3948a2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331209
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit e16bb210d2.
Reason for revert: Indication this caused engine test
failures of the kind:
```
[0;32m[ RUN ] [mEmbedderA11yTest.A11yTreeIsConsistentUsingV1Callbacks
../../flutter/shell/platform/embedder/tests/embedder_a11y_unittests.cc:639: Failure
Expected equality of these values:
std::strncmp(kTooltip, node->tooltip, sizeof(kTooltip) - 1)
Which is: 116
0
```
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
>
> TEST=runtime/vm/compiler/backend/il_test.cc
> TEST=tests/ffi/function_*_native_(leaf_)test.dart
> TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/47625
> Closes: https://github.com/dart-lang/sdk/issues/51618
> Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
> CoreLibraryReviewExempt: Internal FFI implementation changes
> Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Icc87a6ca33bffecabb15c6b168a06ccc38c2fe5b
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333840
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL removes static fields for storing the `@Native`'s function
addresses. Instead, the function addresses are stored in the object
pool for all archs except for ia32. ia32 has no AOT and no AppJit
snapshots, so the addresses are directly embedded in the code.
This CL removes the closure wrapping for `@Native`s. Instead of
`pointer.asFunctionInternal()()` where `asFunction` returns a closure
which contains the trampoline, the function is compiled to a body
which contains the trampoline `Native()`. This is possible for
`@Native`s because the dylib and symbol names are known statically.
Doing the compilation in kernel_to_il instead of a CFE transform
enables supporting static linking later. (The alternative would have
been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
instead of constructing the IL in kernel_to_il.
To enable running resolution in ia32 in kernel_to_il.cc, the
resolution function has been made available via
`runtime/lib/ffi_dynamic_library.h`.
Because the new calls are simply static calls, the TFA can figure
out const arguments flowing to these calls. This leads to constant
locations in the parameters to FfiCalls. So, this CL also introduces
logic to move constants into `NativeLocation`s.
TEST=runtime/vm/compiler/backend/il_test.cc
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
Closes: https://github.com/dart-lang/sdk/issues/51618
Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
CoreLibraryReviewExempt: Internal FFI implementation changes
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
If range analysis has determined that a integer operation cannot throw
or deoptimize and can only result in a single possible value that
matches the operation's representation, replace the operation with that
constant value.
Be more precise about when a BinaryInt64OpInstr can throw. Previously
any MOD or TRUNCDIV operation was marked as possibly throwing, but now
in those cases we only mark it as possibly throwing if the right hand
side can be 0.
TEST=vm/dart/unsigned_truncated_division
Change-Id: I0f4fd754199240715606b90a6a423dbeb268c90d
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64c-try,vm-linux-release-simarm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331283
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit baa18e6d29
Fixes IsRepresentable to take a int64_t, not a uint64_t.
Original change's description:
> [vm/compiler] Unify methods for UnboxIntegerOpInstr subclasses.
>
> Unify the Canonicalize() methods on subclasses of UnboxIntegerOp
> so that there aren't subtle differences between canonicalizing
> UnboxInt32 and UnboxUint32 instructions. Also unify the
> ComputeCanDeoptimize() methods for similar reasons.
>
> If canonicalizing a truncating unbox of a constant, then create an
> unboxed constant of the truncated value instead of an unboxed
> constant of the original untruncated value.
>
> Previously, a subclass of Definition that didn't override InferRange
> got a default range solely based on its type. Now, if the representation
> of the definition is an unboxed integer, the default range is the
> intersection of the range inferred from its type with the range
> inferred from its representation.
>
> TEST=ci
>
> Change-Id: Ib022c366904ee6f8a81995bd4c16b87bd876176d
> Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-win-release-x64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332400
> Reviewed-by: Slava Egorov <vegorov@google.com>
TEST=ci (including simarm, not just simarm_x64)
Change-Id: I6f3d3976634da9725d1a81faa62ab9b718d6663e
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-win-release-x64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm-try,vm-linux-release-simarm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333001
Reviewed-by: Slava Egorov <vegorov@google.com>
Unify the Canonicalize() methods on subclasses of UnboxIntegerOp
so that there aren't subtle differences between canonicalizing
UnboxInt32 and UnboxUint32 instructions. Also unify the
ComputeCanDeoptimize() methods for similar reasons.
If canonicalizing a truncating unbox of a constant, then create an
unboxed constant of the truncated value instead of an unboxed
constant of the original untruncated value.
Previously, a subclass of Definition that didn't override InferRange
got a default range solely based on its type. Now, if the representation
of the definition is an unboxed integer, the default range is the
intersection of the range inferred from its type with the range
inferred from its representation.
TEST=ci
Change-Id: Ib022c366904ee6f8a81995bd4c16b87bd876176d
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-win-release-x64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332400
Reviewed-by: Slava Egorov <vegorov@google.com>
Adds a `CachableIdempotentCallInstr` that can be invoked via
`@pragma('vm:cachable-idempotent')` if the call-sites is force
optimized.
The object pool is not visited by the scavenger. So, we store the
results as unboxed integers. Consequently, only dart functions that
return integers can be cached.
Cachable idempotent calls should never be inlined. After the first
call the function not be called again.
The call itself is on a slow path to avoid register spilling on the
fast path.
TEST=vm/cc/IRTest_CachableIdempotentCall
TEST=runtime/tests/vm/dart/cachable_idempotent_test.dart
Bug: https://github.com/dart-lang/sdk/issues/51618
Change-Id: I612e896f27add76f57796c060157e14cc687a0fd
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301601
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
We can simplify and improve the range analysis in IntConverter if we
generalize based on the sizes and signs of the source and target
representations and whether the operation truncates or deoptimizes.
Adds RepresentationUtils::MinValue() and ::MaxValue(), which return the
minimum and maximum value (as an int64_t) for an unboxed integer
representation.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-win-release-x64-try,vm-gcc-linux-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm-try
Change-Id: Ic8dc0d58059e30d85fb2816e15ab3c19fbee461c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332203
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
If we have constant type arguments include them
into the type.
This CL also expands IL tests framework with the
capability to match CompileTypes.
TEST=vm/dart/regress_53817_il_test
Bug: https://github.com/dart-lang/sdk/issues/53817
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-obfuscate-linux-release-x64-try
Change-Id: I48722b002d9cd436dbc8ca3c9afbefa124f18996
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331900
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This reverts commit 5c4fd50667.
Reason for revert: breaks ffi/data_test on ARM bots
Original change's description:
> [vm/compiler] Consistently use PointerBase.data values as FFiIntPtr.
>
> Also if converting an unboxed int with only non-negative values
> that fit in 32 bits to a uint32, then keep the range from the value.
>
> TEST=regress_306327173_il_test
>
> Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try
> Change-Id: Id9e7c2d5f477e560822a02574739c57d77b5a6d1
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332202
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I35f6b3b03c9d2dec0f35005f850dab7f2dd54515
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332205
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Also if converting an unboxed int with only non-negative values
that fit in 32 bits to a uint32, then keep the range from the value.
TEST=regress_306327173_il_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try
Change-Id: Id9e7c2d5f477e560822a02574739c57d77b5a6d1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332202
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Before, converting a Uint32 to a Int32 would just clamp the range
to be within the possible Int32 range. However, that's not what the
IntConverterInstr represents in this case: in this case, it's just
a reinterpretation of the bits as signed instead of unsigned.
For simplicity's sake, the initial fix for this is to keep the range
[x, y] if it is 0 <= x <= y <= kMaxInt32, and otherwise just use
the full Int32 range.
Also add a version of Range::Full that takes a Representation instead
of a RangeSize, so that the ranges for unsigned unboxed ints can be
made more precise. Previously, they'd be overapproximated by the next
largest signed representation instead.
Also check that we're never trying to create a range for x->untagged,
and that untagged->x just uses the full range for x.
TEST=vm/dart/regress_306327173_il_test
Fixes: b/306327173
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try
Change-Id: Ic60d24be016844deda56ee477a6950a315b8506f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332040
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Follow up of:
* https://dart-review.googlesource.com/c/sdk/+/328861
* https://dart-review.googlesource.com/c/sdk/+/331087
* https://dart-review.googlesource.com/c/sdk/+/331120
The pool entries for switchable calls have their pool entries
transformed during snapshot loading in AOT. This behavior is now folded
into `SnapshotBehavior`.
Because snapshot behavior is set on the call sites, and the call sites
are already AOT specific, no conditionals inside app_snapshot.cc are
needed anymore.
The Arm64 implementation does not use the assembler for implicitly
adding pool entries, it does so explictly and loads two pool entries
in one load.
We could consider doing these explicit loads in the other architectures
as well. This would avoid adding the snapshot_behavior argument to
`LoadUnqiueObject` for the other architectures.
IA32 has no AOT support.
TEST=precomp bots
Change-Id: I75f70e00ca44f6545694d9b59e3ad08ad8bb3f8e
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-simriscv64-try,vm-aot-mac-product-arm64-try,vm-aot-win-debug-arm64-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331101
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
AOT no longer emit call sites that immediately use the megamorphic dispatch stub. Such call sites are now generated as unlinked switchable call sites that may eventually transition to megamorphic.
TEST=ci
Change-Id: Iec342a2b3bc8fe27612bfa14b516e3c0512c1134
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331087
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Fix IntConverterInstr::ComputeType() to return the Object CompileType,
not Int, if converting to a kUntagged representation. (This matches
the CompileType of a LoadFieldInstr loading a native untagged address
field.)
Similarly, change the type of LoadUntagged from Int to Object and
LoadThread from Dynamic to Object, since they also generate untagged
values.
In PhiUnboxingHeuristic::Process, start with the Phi node's existing
representation instead of kTagged. If any input of a Phi node is an untagged address, mark the Phi node as untagged, and let later
attempts to insert conversions fail if any other input is not.
TEST=vm/cc/ConstantPropagator_Regress35371
vm/dart/address_local_pointer_il_test
vm/dart/pointer_as_typed_list_il_test
ffi/extension_methods_test
ffi_2/extension_methods_test
Fixes: https://github.com/dart-lang/sdk/issues/53789
Cq-Include-Trybots: luci.dart.try:vm-linux-debug-x64c-try,vm-kernel-precomp-linux-release-x64-try,vm-win-debug-x64c-try,vm-asan-linux-release-x64-try,vm-aot-asan-linux-release-x64-try
Change-Id: I173b56022758413225c3fbf2b2ed8ac6b80ad7eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331060
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
`NativeCallInstr::EmitNativeCode` `link_lazily` code path uses the
new snapshotting behavior, and the non-`link_lazily` checks that it's
never snapshotted.
The snapshot behavior argument is threaded from the the EmitNativeCode
to the places where the pool entry is constructed.
Does not yet migrate `kSwitchableCallMissEntryPoint` and
`kMegamorphicCallEntryPoint`.
TEST=CI
Change-Id: I8dd1dc5aa525f888b493856c60ae385087463d02
Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-simriscv64-try,vm-aot-linux-release-arm64-try,vm-aot-android-release-arm64c-try,vm-aot-win-debug-arm64-try,vm-aot-mac-product-arm64-try,vm-mac-debug-arm64-try,vm-ffi-android-debug-arm-try,vm-ffi-qemu-linux-release-arm-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328861
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of commit 06d7a2352e
This version fixes an issue when a phi node has multiple inputs with
different unboxed integer representations. The original CL made a
change where only the representations were considered, not the range
of values for the Phi calculated by range analysis. The reland goes
back to the old behavior for this case.
Also fixes the new tests on 32-bit architectures.
Original change's description:
> [vm/compiler] Change MemoryCopy to also take untagged addresses.
>
> This CL adds the ability to pass the payload address of the source
> and destination directly to the MemoryCopy instruction as an untagged
> value.
>
> The new translation of the _TypedListBase._memMoveN methods use the new
> MemoryCopy constructor, retrieving the untagged value of the data field
> of both the source and destination. This way, if inlining exposes the
> allocation of the object from which the data field is being retrieved,
> then allocation sinking can remove the intermediate allocation if there
> are no escaping uses of the object.
>
> Since Pointer.asTypedList allocates such ExternalTypedData objects,
> this CL makes that method inlined if at all possible, which removes
> the intermediate allocation if the only use of the TypedData object
> is to call setRange for memory copying purposes.
>
> This CL also separates unboxed native slots into two groups: those
> that contain untagged addresses and those that do not. The former
> group now have the kUntagged representation, which mimics the old
> use of LoadUntagged for the PointerBase data field and also ensures
> that any arithmetic operations on untagged addresses must first be
> explicitly converted to an unboxed integer and then explicitly converted
> back to untagged before being stored in a slot that contains untagged
> addresses.
>
> When a unboxed native slot that contains untagged addresses is defined,
> the definition also includes a boolean which represents whether
> addresses that may be moved by the GC can be stored in this slot or not.
> The redundancy eliminator uses this to decide whether it is safe to
> eliminate a duplicate load, replace a load with the value originally
> stored in the slot, or lift a load out of a loop.
>
> In particular, the PointerBase data field may contain GC-moveable
> addresses, but only for internal TypedData objects and views, not
> for external TypedData objects or Pointers. To allow load optimizations
> involving the latter, the LoadField and StoreField instructions now
> take boolean flags for whether loads or stores from the slot are
> guaranteed to not be GC-moveable, to override the information from
> the slot argument.
>
> Notable benchmark changes on x64 (similar for other archs unless noted):
>
> JIT:
> * FfiMemory.PointerPointer: 250.7%
> * FfiStructCopy.Copy1Bytes: -26.73% (only x64)
> * FfiStructCopy.Copy32Bytes: -25.18% (only x64)
> * MemoryCopy.64.setRange.Pointer.Uint8: 19.36%
> * MemoryCopy.64.setRange.Pointer.Double: 18.96%
> * MemoryCopy.8.setRange.Pointer.Double: 17.59%
> * MemoryCopy.8.setRange.Pointer.Uint8: 19.46%
>
> AOT:
> * FfiMemory.PointerPointer: 323.5%
> * FfiStruct.FieldLoadStore: 483.3%
> * FileIO_readwrite_64kb: 15.39%
> * FileIO_readwrite_512kb (Intel Xeon): 46.22%
> * MemoryCopy.512.setRange.Pointer.Uint8: 35.20%
> * MemoryCopy.64.setRange.Pointer.Uint8: 55.40%
> * MemoryCopy.512.setRange.Pointer.Double: 29.45%
> * MemoryCopy.64.setRange.Pointer.Double: 60.37%
> * MemoryCopy.8.setRange.Pointer.Double: 59.54%
> * MemoryCopy.8.setRange.Pointer.Uint8: 55.40%
> * FfiStructCopy.Copy32Bytes: 398.3%
> * FfiStructCopy.Copy1Bytes: 1233%
>
> TEST=vm/dart/address_local_pointer, vm/dart/pointer_as_typed_list
>
> Issue: https://github.com/dart-lang/sdk/issues/42072
> Fixes: https://github.com/dart-lang/sdk/issues/53124
>
> Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-arm-try,vm-eager-optimization-linux-release-x64-try,vm-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try
> Change-Id: I563e0bfac5b1ac6cf1111649934067c12891b631
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/324820
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TEST=vm/dart/address_local_pointer, vm/dart/pointer_as_typed_list
Issue: https://github.com/dart-lang/sdk/issues/42072
Fixes: https://github.com/dart-lang/sdk/issues/53124
Change-Id: Iabb0e910f12636d0ff51e711c8c9c98ad40e5811
Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-arm-try,vm-eager-optimization-linux-release-x64-try,vm-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330600
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 06d7a2352e.
Reason for revert: everything crashes on vm-aot-linux-debug-simarm_x64
Original change's description:
> [vm/compiler] Change MemoryCopy to also take untagged addresses.
>
> This CL adds the ability to pass the payload address of the source
> and destination directly to the MemoryCopy instruction as an untagged
> value.
>
> The new translation of the _TypedListBase._memMoveN methods use the new
> MemoryCopy constructor, retrieving the untagged value of the data field
> of both the source and destination. This way, if inlining exposes the
> allocation of the object from which the data field is being retrieved,
> then allocation sinking can remove the intermediate allocation if there
> are no escaping uses of the object.
>
> Since Pointer.asTypedList allocates such ExternalTypedData objects,
> this CL makes that method inlined if at all possible, which removes
> the intermediate allocation if the only use of the TypedData object
> is to call setRange for memory copying purposes.
>
> This CL also separates unboxed native slots into two groups: those
> that contain untagged addresses and those that do not. The former
> group now have the kUntagged representation, which mimics the old
> use of LoadUntagged for the PointerBase data field and also ensures
> that any arithmetic operations on untagged addresses must first be
> explicitly converted to an unboxed integer and then explicitly converted
> back to untagged before being stored in a slot that contains untagged
> addresses.
>
> When a unboxed native slot that contains untagged addresses is defined,
> the definition also includes a boolean which represents whether
> addresses that may be moved by the GC can be stored in this slot or not.
> The redundancy eliminator uses this to decide whether it is safe to
> eliminate a duplicate load, replace a load with the value originally
> stored in the slot, or lift a load out of a loop.
>
> In particular, the PointerBase data field may contain GC-moveable
> addresses, but only for internal TypedData objects and views, not
> for external TypedData objects or Pointers. To allow load optimizations
> involving the latter, the LoadField and StoreField instructions now
> take boolean flags for whether loads or stores from the slot are
> guaranteed to not be GC-moveable, to override the information from
> the slot argument.
>
> Notable benchmark changes on x64 (similar for other archs unless noted):
>
> JIT:
> * FfiMemory.PointerPointer: 250.7%
> * FfiStructCopy.Copy1Bytes: -26.73% (only x64)
> * FfiStructCopy.Copy32Bytes: -25.18% (only x64)
> * MemoryCopy.64.setRange.Pointer.Uint8: 19.36%
> * MemoryCopy.64.setRange.Pointer.Double: 18.96%
> * MemoryCopy.8.setRange.Pointer.Double: 17.59%
> * MemoryCopy.8.setRange.Pointer.Uint8: 19.46%
>
> AOT:
> * FfiMemory.PointerPointer: 323.5%
> * FfiStruct.FieldLoadStore: 483.3%
> * FileIO_readwrite_64kb: 15.39%
> * FileIO_readwrite_512kb (Intel Xeon): 46.22%
> * MemoryCopy.512.setRange.Pointer.Uint8: 35.20%
> * MemoryCopy.64.setRange.Pointer.Uint8: 55.40%
> * MemoryCopy.512.setRange.Pointer.Double: 29.45%
> * MemoryCopy.64.setRange.Pointer.Double: 60.37%
> * MemoryCopy.8.setRange.Pointer.Double: 59.54%
> * MemoryCopy.8.setRange.Pointer.Uint8: 55.40%
> * FfiStructCopy.Copy32Bytes: 398.3%
> * FfiStructCopy.Copy1Bytes: 1233%
>
> TEST=vm/dart/address_local_pointer, vm/dart/pointer_as_typed_list
>
> Issue: https://github.com/dart-lang/sdk/issues/42072
> Fixes: https://github.com/dart-lang/sdk/issues/53124
>
> Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-arm-try,vm-eager-optimization-linux-release-x64-try,vm-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try
> Change-Id: I563e0bfac5b1ac6cf1111649934067c12891b631
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/324820
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Issue: https://github.com/dart-lang/sdk/issues/42072
Change-Id: I7c31434e01108487de69a32154bbefd1538c6f0f
Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-arm-try,vm-eager-optimization-linux-release-x64-try,vm-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330523
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This CL adds the ability to pass the payload address of the source
and destination directly to the MemoryCopy instruction as an untagged
value.
The new translation of the _TypedListBase._memMoveN methods use the new
MemoryCopy constructor, retrieving the untagged value of the data field
of both the source and destination. This way, if inlining exposes the
allocation of the object from which the data field is being retrieved,
then allocation sinking can remove the intermediate allocation if there
are no escaping uses of the object.
Since Pointer.asTypedList allocates such ExternalTypedData objects,
this CL makes that method inlined if at all possible, which removes
the intermediate allocation if the only use of the TypedData object
is to call setRange for memory copying purposes.
This CL also separates unboxed native slots into two groups: those
that contain untagged addresses and those that do not. The former
group now have the kUntagged representation, which mimics the old
use of LoadUntagged for the PointerBase data field and also ensures
that any arithmetic operations on untagged addresses must first be
explicitly converted to an unboxed integer and then explicitly converted
back to untagged before being stored in a slot that contains untagged
addresses.
When a unboxed native slot that contains untagged addresses is defined,
the definition also includes a boolean which represents whether
addresses that may be moved by the GC can be stored in this slot or not.
The redundancy eliminator uses this to decide whether it is safe to
eliminate a duplicate load, replace a load with the value originally
stored in the slot, or lift a load out of a loop.
In particular, the PointerBase data field may contain GC-moveable
addresses, but only for internal TypedData objects and views, not
for external TypedData objects or Pointers. To allow load optimizations
involving the latter, the LoadField and StoreField instructions now
take boolean flags for whether loads or stores from the slot are
guaranteed to not be GC-moveable, to override the information from
the slot argument.
Notable benchmark changes on x64 (similar for other archs unless noted):
JIT:
* FfiMemory.PointerPointer: 250.7%
* FfiStructCopy.Copy1Bytes: -26.73% (only x64)
* FfiStructCopy.Copy32Bytes: -25.18% (only x64)
* MemoryCopy.64.setRange.Pointer.Uint8: 19.36%
* MemoryCopy.64.setRange.Pointer.Double: 18.96%
* MemoryCopy.8.setRange.Pointer.Double: 17.59%
* MemoryCopy.8.setRange.Pointer.Uint8: 19.46%
AOT:
* FfiMemory.PointerPointer: 323.5%
* FfiStruct.FieldLoadStore: 483.3%
* FileIO_readwrite_64kb: 15.39%
* FileIO_readwrite_512kb (Intel Xeon): 46.22%
* MemoryCopy.512.setRange.Pointer.Uint8: 35.20%
* MemoryCopy.64.setRange.Pointer.Uint8: 55.40%
* MemoryCopy.512.setRange.Pointer.Double: 29.45%
* MemoryCopy.64.setRange.Pointer.Double: 60.37%
* MemoryCopy.8.setRange.Pointer.Double: 59.54%
* MemoryCopy.8.setRange.Pointer.Uint8: 55.40%
* FfiStructCopy.Copy32Bytes: 398.3%
* FfiStructCopy.Copy1Bytes: 1233%
TEST=vm/dart/address_local_pointer, vm/dart/pointer_as_typed_list
Issue: https://github.com/dart-lang/sdk/issues/42072
Fixes: https://github.com/dart-lang/sdk/issues/53124
Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-arm-try,vm-eager-optimization-linux-release-x64-try,vm-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try
Change-Id: I563e0bfac5b1ac6cf1111649934067c12891b631
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/324820
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Adds `EnterTestFrame` and `LeaveTestFrame` to ia32 assembler tests
to give access to the thread and code object.
Moves `LoadInt32FromBoxOrSmi` from the unbox instruction to the
assembler and implements `LoadWordFromBoxOrSmi`. And adds unit tests.
Note: Loading from boxed mints was broken. It didn't multiply the
SmiUntagged value by two when loading. The implementation is now in
line with the x64.
TEST=vm/cc/Assembler_LoadWordFromBoxOrSmi
Closes: https://github.com/dart-lang/sdk/issues/53622
Change-Id: I5e5d8dca5fa5717b9bd475a3c319a7de4ee0d2f3
Cq-Include-Trybots: luci.dart.try:vm-linux-debug-ia32-try,vm-linux-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328520
Reviewed-by: Tess Strickland <sstrickl@google.com>
Alexander Markov