This CL implements `NativeFinalizer` in the GC.
`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.
On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.
In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.
When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.
In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*
Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The FFI use sites transformation generates code accessing some late
final fields of the Array class. By changing these to be getters, the
transformation works both with and without late field lowering.
Tested: ci
Change-Id: Ib30d18e0830f7bb597214701272b1d0a922ac06a
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/237363
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
These transformations are maintained by the VM team and keeping them
in pkg/kernel would put a burden on the CFE team after code reviews
are tightened to require OWNERS approval.
TEST=ci
Change-Id: I931033ba987d17a2c9c6d80a16dad09bfbaa0572
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/234905
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Use `hasScheme` in place of comparing against the empty string, and
`isScheme` to compare against all other schemes.
TEST=No behavior changes.
Change-Id: Ifc9fd13c6cf37933ebd4a754c4b500dedbcb291b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231185
Reviewed-by: Kevin Moore <kevmoo@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
This CL implements `Finalizable` by means of a CFE transform.
The transform is implemented in the existing FFI transform to avoid the
overhead of traversing the AST an extra time. The FFI transform slows
down ~15-30% on Flutter Gallery. For more info see the design doc.
TEST=`dart pkg/vm/test/transformations/ffi_test.dart`
Design doc: go/dart-vm-finalizable
Bug: https://github.com/dart-lang/sdk/issues/47777
Change-Id: I0bf9dead90a61631b7f215dc19fbaa9e40e63c8d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227501
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The VM patch files contain many annotations that guide the TFA and the
VM compilation pipeline. Most of these are not relevant to dart2wasm,
and having the TFA look at these when running inside dart2wasm leads to
sub-optimal, and sometimes even unsound, results.
In order to enable sharing of patch files between the VM and dart2wasm,
the annotation parser is given a target flag to control which
annotations it should recognize and how to parse them. The annotation
parser thus becomes an abstraction layer between the concrete
annotations in the patch files and the parsed annotations that the TFA
sees.
Additionally, external members used by dart2wasm don't always have
external names. To support this, the signature shaker must skip all
external members, not just the ones with an external name. This makes no
difference for the VM, since all of its external members either have an
external name or are implemented in a patch file, making them no longer
external.
TEST=ci + upcoming dart2wasm CL
Change-Id: Id425dd1ccc5560721034ae83c8cdc1593801a4e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231102
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL does two things:
* "Reenable" the skipping of the dart:ffi transformation entirely when
the application does not use dart:ffi. This was originally added in
ad596359a6 but was logically disabled in 4558112105 when dart:core
started depending on it. Here we now ignore (real) dart:* libraries
when looking for dependencies of dart:ffi.
* Find the subset of the just compiled libraries that transitively
depend on dart:ffi and just run the dart:ffi transformation on those
libraries.
For dart2js that doesn't use dart:ffi at all we go from:
```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.003428: Transformed ffi natives in 51ms.
0:00:09.203442: Transformed ffi annotations in 199ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.098175: Transformed ffi natives in 52ms.
0:00:09.296847: Transformed ffi annotations in 198ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:08.854341: Transformed ffi natives in 57ms.
0:00:09.061804: Transformed ffi annotations in 207ms.
```
to
```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.138651: Skipped ffi transformation in 1ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.242590: Skipped ffi transformation in 1ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.163885: Skipped ffi transformation in 1ms.
```
When compiling the flutter gallery app that does depend on dart:ffi we go from:
```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:16.344955: Transformed ffi natives in 78ms.
0:00:16.777218: Transformed ffi annotations in 432ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:15.994674: Transformed ffi natives in 80ms.
0:00:16.472779: Transformed ffi annotations in 478ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:16.027488: Transformed ffi natives in 81ms.
0:00:16.491362: Transformed ffi annotations in 463ms.
```
to
```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:15.991628: Transformed ffi natives in 30ms.
0:00:16.226564: Transformed ffi annotations in 234ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:16.184984: Transformed ffi natives in 29ms.
0:00:16.404439: Transformed ffi annotations in 219ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:15.933513: Transformed ffi natives in 27ms.
0:00:16.145640: Transformed ffi annotations in 212ms.
```
TEST=Assuming tests already exists.
Change-Id: Ie13be1924d3439cb710c782af81e1e42cda2a838
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228001
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Flutter in google3 uses --delete-tostring-package-uri compiler option
to remove toString methods in package:flutter and dart:ui to reduce
size in release mode.
This has unfortunate effect of removing toString methods from exception
classes which may provide valuable information for investigating
problems seen in the wild.
This change adds a new @pragma('flutter:keep-to-string-in-subtypes')
on classes to keep toString methods on all subtypes of the annotated
classes. This pragma is now used on Exception and Error classes in
dart:core.
TEST=pkg/vm/test/transformations/to_string_transformer_test.dart
Issue: https://github.com/flutter/flutter/issues/61562
Change-Id: Ib739c83cdf6b539208f705ba198e63b8bc54fa61
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227920
Reviewed-by: Dan Field <dnfield@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Before this CL there was a pattern of looking up size and alignment for
a specific abi by using `size[abi]` and `alignment[abi]`. The problem
being that both are often dynamic map creations, meaning that it created
tons of maps that it looked up a specific value in and threw away.
This CL instead adds handles for getting size and alignment for a
specific abi, skipping the map creation.
On my machine this more than doubles the speed of the ffi transformation
when compiling the flutter gallery app:
Before:
```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.aD6a8pEOZW/flutter_patched_sdk/platform_strong.dill /tmp/tmp.aD6a8pEOZW/gallery/lib/main.dart | grep "Transformed ffi annotations"
0:00:17.861255: Transformed ffi annotations in 990ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.aD6a8pEOZW/flutter_patched_sdk/platform_strong.dill /tmp/tmp.aD6a8pEOZW/gallery/lib/main.dart | grep "Transformed ffi annotations"
0:00:18.029886: Transformed ffi annotations in 1055ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.aD6a8pEOZW/flutter_patched_sdk/platform_strong.dill /tmp/tmp.aD6a8pEOZW/gallery/lib/main.dart | grep "Transformed ffi annotations"
0:00:17.807917: Transformed ffi annotations in 1012ms.
```
With this fix:
```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.aD6a8pEOZW/flutter_patched_sdk/platform_strong.dill /tmp/tmp.aD6a8pEOZW/gallery/lib/main.dart | grep "Transformed ffi annotations"
0:00:17.159482: Transformed ffi annotations in 457ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.aD6a8pEOZW/flutter_patched_sdk/platform_strong.dill /tmp/tmp.aD6a8pEOZW/gallery/lib/main.dart | grep "Transformed ffi annotations"
0:00:17.385509: Transformed ffi annotations in 410ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.aD6a8pEOZW/flutter_patched_sdk/platform_strong.dill /tmp/tmp.aD6a8pEOZW/gallery/lib/main.dart | grep "Transformed ffi annotations"
0:00:17.216748: Transformed ffi annotations in 407ms.
```
TEST=This shouldn't change functionality, assuming tests already exists.
Change-Id: I2013659376e249622d75ebd8b77f2bf70685d4f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/226688
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
In order to make sure static types in kernel AST are still correct
after tree shaking, tree shaker needs to insert no-op unsafeCast calls
when removing conditional expressions, as ConditionalExpression nodes
have explicit 'staticType' which may be different from static types of
sub-expressions.
Otherwise we might end up with an untyped function call
(FunctionInvocation with FunctionAccessKind.Function)
performed from a receiver with a known function type.
That would violate VM's assumptions about static type of receiver
being checked by the front-end and trigger assertion in
pkg/vm/lib/transformations/call_site_annotator.dart.
TEST=runtime/tests/vm/dart/untyped_function_invocation_with_known_function_type_test.dart
TEST=pkg/vm/testcases/transformations/type_flow/transformer/null_test_elimination2_nnbd_strong.dart
Change-Id: I6bf201a979d1b71eb48ed04f154adf2b62dac922
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/226741
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL adds support for users defining integers which are mapped to
differing sizes and signedness based on the application binary interface
the Dart VM is running on.
Notable implementation design decisions:
- ABIs are open world, so that adding an ABI to the Dart VM does not
break existing definitions. Thus, we only figure out in the VM that
we're missing a mapping. We throw compile-time errors.
- In AOT, these show up in the precompilation step.
- In JIT, these show up as `_CompileTimeError` at runtime. Note that
these can be caught. So in subsequent compilation steps we need to
ensure that we also throw the same compile-time error.
- We match on the call-sites (streaming_flowgraph_builder) rather than
method bodies (kernel_to_il) of AbiSpecific loads and stores so that
we can compile for the int-size of the call site.
API design decisions:
https://github.com/dart-lang/sdk/issues/42563#issuecomment-981774001
Closes: https://github.com/dart-lang/sdk/issues/42563
TEST=tests/ffi_2/abi_*_test.dart
TEST=tests/ffi/function_*_generated_test.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart
Change-Id: I8c8df36fab939b6fb614c5f1ee8e1bf46b6e9521
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,benchmark-linux-try,dart-sdk-linux-try,front-end-linux-release-x64-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-fuchsia-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64c-try,vm-kernel-mac-debug-x64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221501
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This change introduces Rapid Type Analysis (RTA) and uses it to
calculate initial set of allocated classes for Type Flow Analysis (TFA).
As a result, TFA converges much faster and AOT compilation time
improves.
RTA is less precise than TFA, so the set of allocated classes is
larger compared to the one calculated by TFA. However, it has only
marginal effect on the size of resulting AOT snapshots.
Time of AOT compilation step 2 on a large Flutter application
118.652s -> 59.907s (-49.5% / improved by a factor of 1.98x)
Snapshot size on armv8: -0.13%
Flutter gallery snapshot size in release and release-sizeopt modes
armv7 +0.19%, armv8 +0.2%
Just in case, RTA can be disabled using --no-rta option.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/42442
Issue: b/154155290
Change-Id: Iffbdabe7d486cad2e138f7592bffcb70474ddc34
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222500
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
If tree shaker doesn't visit TypeParameterType.promotedBound which
contains the only reference to a particular class it may remove that
class, leaving a dangling reference from promotedBound. After that
kernel AST serialization would crash with "Missing canonical name for
Reference" error.
This change also fixes TypeParameterType.visitChildren.
TEST=pkg/vm/testcases/transformations/type_flow/transformer/regress_47878.dart
Fixes https://github.com/dart-lang/sdk/issues/47878
Change-Id: I555261713240bf0be412db27efb68f32cfd8008b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222781
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
During the fix for https://github.com/dart-lang/language/issues/1785,
an explicit `bool` type was added to the variable `nullable` to ensure
that it would properly participate in type promotion while the fix was
still being rolled out. Now that the fix is in place, this explicit
type is no longer needed.
TEST=standard trybots, TAP global presubmit, Golem
Change-Id: Ib81ad436876e576f85c929c205d5831214e6a05a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214821
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
When ABI-specific integers are introduced, their mappings can be
partial. We need to account for this in the transformation and the code
we generate.
In the transformation, all sizes and offsets become nullable.
In the generated code we add `null` constants and a call to check
whether the value is not-null at runtime.
Note that with only this CL we can not generate nulls yet, because all
size and offset mappings are still complete.
TEST=pkg/front_end/testcases/nnbd/ffi*
TEST=tests/ffi*
Bug: https://github.com/dart-lang/sdk/issues/42563
Change-Id: I80d45f3f52001670bc0679a033f7daa22198d55e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221631
Reviewed-by: Ryan Macnak <rmacnak@google.com>
We're going to extend this logic in subsequent CLs, so lets give this
abstraction its own place to live.
TEST=just moving file contents
Change-Id: Ie6b267e77dd3a772bb3bbf0b79b33946ee767b0e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221630
Reviewed-by: Ryan Macnak <rmacnak@google.com>
In order to avoid spending too much time analyzing multiple
invocations of huge methods with different combinations of
argument types, the following heuristic is added to type
flow analysis:
- if size of a method summary >= N operations, and
- number of invocations for this method >= M, then
create and use an approximate invocation with raw argument types
for all subsequent invocations (so that we would have to analyze
at most M+1 invocations per huge method).
Constants N = 300 and M = 10 are selected so that size of
Flutter gallery AOT snapshot is not affected, and at the same
time compilation time of example involving huge auto-generated
methods from issue https://github.com/dart-lang/sdk/issues/47546 is
not regressed very much (<10s).
TEST=ci
Issue https://github.com/dart-lang/sdk/issues/47546
Change-Id: I6a452872b13bd32928585fce4431ec1176833e6c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219749
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL expands the ABIs in the FFI transform and VM from 3 groups to
18 individual ABIs in preparation of adding ABI-specific integer sizes
and structs.
This increases const lists with offsets in the kernel representation
from 3 to 18 elements. We do not expect significant size increases or
performance regressions from this. The constants are deduplicated in
Dart and the optimizer should optimize indexed lookups in const lists
away. However, landing this separately will enable us to verify that
assumption.
This also moves `Abi` to its own file and makes it an opaque class with
a predefined set of instances so that we do not depend on its internals.
That will enable us to keep `pkg/vm/lib/transformations/ffi/abi.dart`
consistent with the `Abi` abstraction to be introduced in `dart:ffi`
later for specifying ABI-specific integer sizes.
Bug: https://github.com/dart-lang/sdk/issues/42563
Bug: https://github.com/dart-lang/sdk/issues/42816
List of ABIs decided based on what's currently used (as Dart SDK target
platform, or Flutter target platform, or G3 target) and added
windows_arm64 in anticipation of
(https://github.com/flutter/flutter/issues/53120).
Excluded are macos_ia32 (https://github.com/dart-lang/sdk/issues/39810)
because we discontinued support; and windows_arm, fuchsia_arm,
fuchsia_ia32, ios_ia32, and macos_arm because these are
unlikely to be added.
TEST=pkg/front_end/testcases/*.expect
TEST=tests/ffi/*
Change-Id: I437707c18d8667490c063272a5f8a33d846e6061
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-ffi-android-debug-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-ffi-android-debug-arm64c-try,vm-kernel-mac-release-arm64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-android-release-arm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217184
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
This change makes the FfiNative closure inherit the nullability
of the associated Dart function.
Example:
@FfiNative<Handle Function()>('MaybeStr')
external String? maybeStr();
Generates a closure of type `() → core::Object?`.
This enables @FfiNatives to return nullable types where they
previously would be forced non-nullable.
TEST=Updated expect file.
Change-Id: I85cf9f2f43749e97ab97779f911789034e6b9ab1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/218101
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
This change contains a few small improvements which reduce
time of type flow analysis (AOT compilation step 2):
* Faster construction of _DirectInvocation objects
* Eager approximation of arguments of operator==
* Eager approximation of arguments of identical
Improves time of AOT compilation step 2 (TFA) on a large
Flutter application 137s -> 124s (-9.4%).
Doesn't affect size of Flutter gallery AOT snapshot.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/42442
Change-Id: I9da0b0e68d1ee8062d86094fb5cdb9462fb7ea6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217741
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Adds explicit types to list literals used in the ffi/native
transform, as the list type inferred will otherwise be too
narrow.
This triggers a problem later on when a different transform
then tries to add a broader typed value into these lists,
triggering a typing error:
`type 'ConstantExpression' is not a subtype of type 'VariableGet'`
TEST=ffi/vmspecific_ffi_native_test
Bug: https://github.com/dart-lang/sdk/issues/47512
Change-Id: Ie3ba10da02083e16e7325b225febdc43b31077f5
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217302
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Auto-Submit: Clement Skau <cskau@google.com>
FFI functions return Handles as Object so we have to add a cast
to get the expected return type. E.g.:
@FfiNative<Handle Function()>(..)
external String getSomeStr();
Will have a synthetic FFI function of type `Object Function()`,
which we need to add a `as String` to.
This CL also refactors the overlapping parts of
`_transformInstanceMethod` and `_transformInstanceFunction` into
two shared functions.
TEST=Updated .expect file.
Change-Id: I0f74d34ee3eef4928f7beb38a1180180c13672ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216286
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Having the transforms in a dedicated folder enables us to add more files
without cluttering the transformations folder itself.
TEST=This only moves files. Bots test whether imports are correct.
Change-Id: Icf1d64b5ec09a7d19915b1cb72464fa67b6cfe15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217180
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL adds the `Bool` `NativeType` with support for loading/storing
from `Pointer<Bool>`, `Array<Bool>`, and `Struct` and `Union` fields,
and support for passing booleans through FFI calls and callbacks.
The assumption is that `bool` is always treated as `uint8_t` in the
native ABIs. Including: (1) whether there can be garbage in the upper
bytes in CPU registers, (2) stack alignment, and (3) alignment in
compounds.
The conversion from `bool` to `uint8_t` is implemented as follows:
- bool to int: `value ? 1 : 0`
- int to bool: `value != 0`
The conversion is implemented in Dart in patch files for memory loads
and stores (pointer, array, and struct fields) and kernel_to_il for
FFI call and callback arguments and return value.
TEST=runtime/vm/compiler/ffi/native_type_vm_test.cc
TEST=tests/ffi/bool_test.dart
TEST=tests/ffi/function_callbacks_structs_by_value_generated_test.dart
TEST=tests/ffi/function_structs_by_value_generated_test.dart
Closes: https://github.com/dart-lang/sdk/issues/36855
Change-Id: I75d100340ba41771abfb41c598ca92066a89370b
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-ffi-android-debug-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-ffi-android-debug-arm64c-try,vm-kernel-mac-release-arm64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-android-release-arm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/216900
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
- Allows FfiNative annotation to be attached to non-static
methods of classes.
- Transforms non-static instance methods to static, adding
wrappers with the receiver as an extra, implicit first paramters.
- Transform all parameters and arguments to Pointer if the object
being passed supports it (i.e. extends NativeFieldWrapperClass1).
- Adds compile time errors for cases where the FfiNative annotation
doesn't align with the annotated function. Taking into account
implicit receivers and converted Pointers.
- Adds complimentary Analyzer checks for the above errors as well.
- Adds tests for the transforms, compile time errors and analyzer
changes.
TEST=Adds new tests for instance methods, analyzer changes.
Change-Id: Idf54430acf2728a650008333b149b254941290ad
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/213773
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Previously, TFA used ClosedWorldClassHierarchy from CFE to look for
dispatch targets of invocations, and cached those lookups. However,
doing lookups via ClosedWorldClassHierarchy for the first time is still
slow. Now TFA builds maps of dispatch targets from AST and no longer
uses ClosedWorldClassHierarchy for the lookup.
Improves time of AOT compilation step 2 (TFA) on a large
Flutter application 170s -> 152s (-10.5%).
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/42442
Change-Id: I1a22d298e5b2c0ead57c38ddfbf5ebbd1876732f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215985
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
We can't rely on GC to trigger the finalizer of a given test object,
so the previous liveness test was unreliable.
Instead we add an expect test to verify we generate the necessary
`reachabilityFence(..)`s to ensure liveness.
TEST=Adds expect.
Bug: https://github.com/dart-lang/sdk/issues/47362
Change-Id: Ia57a07522c8b8265b24780f00f3339b50534eb60
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-mac-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215542
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This adds a third reference to Field declaration, so that we know have
three references, one for each use case:
1) fieldReference used for initialization; in FieldInitializer and as
the key in the InstanceConstant.fieldValues map.
2) getterReference used for reading; in InstanceGet, StaticGet and
SuperPropertyGet
3) setterReference used for writing; in InstanceSet, StaticSet and
SuperPropertySet
TEST=existing
Change-Id: I223f130e808e7f19a831c1fe5e3a4725d1bcdc3b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203770
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This makes it more closely mirror the Dart_NativeEntryResolver,
and acts as an extra sanity check that signatures (roughly)
align between the FfiNative decl. and the native function.
TEST=Updated runtime/vm/dart_api_impl_test.cc
Change-Id: I40799dc583ec14db14dc453afed4e2d1eb06fced
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212566
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This change removes handling of StaticGet and ConstructorInvocation
nodes in annotations. This code became redundant since constant
evaluation was moved into the front-end.
TEST=ci
Change-Id: Idfa16624d28cdf73da09b52aec429d881e43a36e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212871
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
As part of deprecating support for native extensions we are also
migrating away from legacy VM-specific `native 'name'` syntax
towards metadata based encoding which does not require any special
syntax.
This CL is a step 1 in migration:
- introduces support for `@pragma('vm:external-name', 'name')`
which serves as a direct replacement for `native 'name'`;
- all core libraries and tests are migrated to use the annotation;
Once this CL lands and rolls we will edit internal and external embedders
to eliminate uses of the native keyword (step 2) and finally remove
support for native keyword across our parsers (step 3).
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/28791
Change-Id: Id6dea878db82dd4fd81149243c425b5c5dc6df86
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212461
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Daco Harkes