`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Cleans up unused object_store->ffi_struct_class.
Reland 1: Allow importing `dart:ffi` from `dart:core` with
`--enable-ffi=false`.
Reland 2: Allow the new `_Compound` class to extend `NativeType`. (This
got triggered in this CL on the build because of the import
from `dart:core`.)
Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072
TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart
Change-Id: I133278e16bd75cd2bb6234e7ddf042ffa0a54fd6
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195079
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Cleans up unused object_store->ffi_struct_class.
Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072
TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart
Change-Id: I9af6d0173db60614091068c218391f73756c135f
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195061
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
In preparation of having `Union` besides `Struct`, renames all mentions
of `Struct` to `Compound`.
Also, changes the type checking to have `allowXYZ` named arguments
always default to false, and remove redundant arguments.
Bug: https://github.com/dart-lang/sdk/issues/38491
tools/test.py ffi ffi_2
TEST=tests/ffi(_2)/(.*)by_value_(*.)_test.dart
Change-Id: Ie5f7cf4189dc315f896e3b3933ff0e0580ac540f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194424
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Cleans up unused object_store->ffi_struct_class.
Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072
TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart
Change-Id: I1f11adfa073c7d2c979f3c2bb15c7444c7c767a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186280
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Adds a way to express native effects in Dart expressions in kernel.
This CL adds a `void _nativeEffect(Object)` to `dart:internal`.
The semantics of `_nativeEffect` are to not execute its arguments and
return `null`.
This CL uses this `_nativeEffect` to make sure that we never execute
the struct constructor invocations used to simulate the native behavior
of FFI trampolines.
Closes: https://github.com/dart-lang/sdk/issues/45607
TEST=tests/ffi(_2)/native_effect_test.dart
Change-Id: Ie06de145e49f8b1cae9e148c2d5d97d5cd8e6878
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,dart-sdk-linux-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194421
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL enables tree shaking of `Struct` sub classes by simulating the
native behavior in Dart code.
We call the struct constructors in let expressions where FFI trampolines
are created which allocate these struct objects in native code. This way
TFA is instructed about the native behavior. The VM recognizes these
constructor calls as dead code and removes them.
For more info see go/dart-ffi-struct-treeshaking.
Closes: https://github.com/dart-lang/sdk/issues/38721
This CL fixes the types on the generated #fromTypedDataBase constructors.
Also, this CL also cleans up the struct naming:
* _addressOf -> _typedDataBase
* _fromPointer -> _fromTypedDataBase
* #pointer -> #typedDataBase
These cleanups are not split into a separate CL to prevent updating
the .expect files multiple times.
Finally, this CL enables running a single transformer test through:
`dart pkg/vm/test/transformations/type_flow/transformer_test.dart name`
TEST=pkg/vm/testcases/transformations/type_flow/transformer/ffi_struct_constructors.dart
TEST=tests/ffi(_2)/function_callbacks_structs_by_value_generated_test.dart
TEST=tests/ffi(_2)/function_structs_by_value_generated_test.dart
Change-Id: I418d0d73bc86b234dfe5b7b04ae726c33d2b8aeb
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,dart-sdk-linux-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193661
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL only changes dart:ffi API, CFE, and analyzer. No VM changes
were needed because the dimensions of inline arrays can be flattened
before passing them to the VM. The multi-dimensionality does not
impact the ABI.
Closes: https://github.com/dart-lang/sdk/issues/45023
TEST=pkg/analyzer/test/src/diagnostics/size_annotation_dimensions_test.dart
TEST=pkg/front_end/testcases/nnbd/ffi_struct_inline_array_multi_dimensional.dart
TEST=tests/ffi/function_structs_by_value_generated_test.dart
TEST=tests/ffi/inline_array_multi_dimensional_test.dart
Change-Id: Ica2c01fccbea7e513879365b34086d8968b54c5b
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188286
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Adds support for single dimension inline arrays in structs. Multi-
dimensional arrays will be supported in a future CL.
This CL adds:
- CFE static error checks for inline arrays.
- CFE transformations for inline arrays.
- VM consumption of inline array fields for NativeType.
- Test generator support for inline arrays + generated tests.
Previous CLs added support for inline arrays in:
- analyzer https://dart-review.googlesource.com/c/sdk/+/183684
- updated in this CL to new API.
- ABI calculation https://dart-review.googlesource.com/c/sdk/+/183682
Closes: https://github.com/dart-lang/sdk/issues/35763
Open issue: https://github.com/dart-lang/sdk/issues/45101
CFE transformations are tested with expectation files:
TEST=pkg/front_end/testcases/(.*)/ffi_struct_inline_array.dart
Trampolines and CArray API are tested with end-to-end Dart tests:
TEST=tests/ffi(_2)/(.*)by_value(.*)test.dart
TEST=tests/ffi(_2)/inline_array_test.dart
Compile-time errors (both CFE and analyzer) are tested in:
TEST=tests/ffi(_2)/vmspecific_static_checks_test.dart
Change-Id: I014c0e4153f1b885638adce80de6ab3cac8e6bb2
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183640
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
This CL changes the semantics of
`Pointer<T extends NativeType>.elementAt` and
`sizeOf<T extends NativeType>` to use the compile-time `T` rather than
the runtime `T`.
Issue: https://github.com/dart-lang/sdk/issues/38721
TEST=tests/ffi/data_test.dart
TEST=tests/ffi/sizeof_test.dart
TEST=tests/ffi/structs_test.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart
Change-Id: Ifb25a4bd66d50a385d3db6dec9213b96dff21722
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,analyzer-nnbd-linux-release-try,front-end-linux-release-x64-try,front-end-nnbd-linux-release-x64-try,benchmark-linux-try,dart-sdk-linux-try,pkg-linux-release-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178200
Reviewed-by: Aske Simon Christensen <askesc@google.com>
This CL changes the semantics of `Pointer<T extends Struct>.ref` to use
the compile-time `T` rather than the runtime `T`.
This enables tree shaking of subtypes of Struct and optimizing `.ref`.
Bug: https://github.com/dart-lang/sdk/issues/38721
TEST=tests/ffi/vmspecific_static_checks_test.dart
TEST=tests/ffi/*struct*test_.dart
Change-Id: Ie19bc3259d1cb721d0ce56d68e82d09dc3a4ad0e
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180190
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
This rewrites `Pointer<Struct>.ref` and `Pointer<Struct>[]` calls in
the CFE to skip the runtime entry when the type argument is constant.
The runtime entry is still used when the type argument is generic.
Forcing the type argument to be constant and removing the runtime entry
will be done in follow up CLs.
Bug: https://github.com/dart-lang/sdk/issues/38721
Bug: https://github.com/dart-lang/sdk/issues/44621
Removing the runtime entry speeds up `.ref` significantly.
before: FfiStruct.FieldLoadStore(RunTime): 18868.140186915887 us.
after: FfiStruct.FieldLoadStore(RunTime): 270.5877976190476 us.
Measurements from Linux x64 in JIT mode.
Closes: https://github.com/dart-lang/sdk/issues/38648
TEST=tests/ffi/structs_test.dart
Change-Id: I82abd930b5a9c5c78a8999c2bc49802d67d37534
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182265
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
The analyzer and CFE now report a warning on calls to
`Pointer<T extends NativeType>.elementAt()` and
`sizeOf<T extends NativeType>()` where `T` is a generic.
Adapted from https://dart-review.googlesource.com/c/sdk/+/178200 to
only deprecate but not error out on generic calls.
Does not roll forward `package:ffi` to a version with the `Allocator`
but keeps the generic `sizeOf` invocation. This causes extra warnings
in the pkg/front_end testcases.
Issue: https://github.com/dart-lang/sdk/issues/38721
TEST=tests/ffi/data_test.dart
TEST=tests/ffi/sizeof_test.dart
TEST=tests/ffi/structs_test.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart
Change-Id: I8f41c4dc04fc44e7e6c540ba87a3f41604130fe9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180560
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL changes the semantics of `Pointer<T extends Struct>.ref` to use
the compile-time `T` rather than the runtime `T`.
This enables tree shaking of subtypes of Struct and optimizing `.ref`.
Issue: https://github.com/dart-lang/sdk/issues/38721
TEST=tests/ffi/vmspecific_static_checks_test.dart
TEST=tests/ffi/*struct*test_.dart
Change-Id: I3f5b08c08ec0799ef8aab3c4177e2ac70d25501c
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,analyzer-nnbd-linux-release-try,front-end-linux-release-x64-try,front-end-nnbd-linux-release-x64-try,benchmark-linux-try,dart-sdk-linux-try,pkg-linux-release-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177862
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Introduces the Allocator API in `dart:ffi`.
This CL does not yet roll `package:ffi` to use `Allocator`, because that
breaks the checked in Dart in Fluter in g3. Instead, this coppies
`_MallocAllocator` from `package:ffi` into the ffi tests for testing.
This CL does not yet migrate off `allocate` and `free` in the SDK. That
is done in a dependent CL.
Issue: https://github.com/dart-lang/sdk/issues/44621
Issue: https://github.com/dart-lang/sdk/issues/38721
TEST=tests/ffi/allocator_test.dart
TEST=tests/ffi/calloc_test.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart
Change-Id: I173e213a750b8b3f594bb8d4fc72575f2b6b91f7
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,analyzer-nnbd-linux-release-try,front-end-linux-release-x64-try,front-end-nnbd-linux-release-x64-try,benchmark-linux-try,dart-sdk-linux-try,pkg-linux-release-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177705
Reviewed-by: Clement Skau <cskau@google.com>
Before this CL, the map we use to "reuse" references when doing
experimental invalidation mapped from String to Reference.
In bug #44523 two abstract member-signatures have the same textual
(String) name meaning that via the lookup one would get the correct
one, one would get the wrong one, they would get the same one
which eventually causes a crash when trying to serialize.
This CL fixes the issue by mapping via the Name instead, which
basically - for private names - wraps the name and the library
which disambiguates it.
This CL also includes the reproduction of #44523.
Fixes#44523.
TEST=Mostly relying on existing test coverage.
Change-Id: Ib62ebca0b7f5092f0b8410d3c458663c3032eca1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177704
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Changes fields to be either mutable or immutable by construction.
This ensure that we don't create setter references for fields that
cannot be assigned to and is a prerequisite for replacing @fields/
@fields= canonical names with @getters/@setters.
TEST=existing expectation tests and verification
Change-Id: I70b9a504ee6f221b7c334ac02620feb0d5f7ae01
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176665
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This CL adds support for nested structs in FFI calls, callbacks, and
memory loads and stores through the Struct classes itself.
Nesting empty structs and nesting a structs in themselves (directly or
indirectly) is reported as error.
This feature is almost fully implemented in the CFE transformation.
Because structs depend on the sizes of their nested structs, the structs
now need to be processed in topological order.
Field access to nested structs branches at runtime on making a derived
Pointer if the backing memory of the outer struct was a Pointer or
making a TypedDataView if the backing memory of the outer struct was
a TypedData.
Assigning to a nested struct is a byte for byte copy from the source.
The only changes in the VM are contained in the native calling
convention calculation which now recursively needs to reason about
fundamental types instead of just 1 struct deep.
Because of the amount of corner cases in the calling conventions that
need to be covered, the tests are generated, rather than hand-written.
ABIs tested on CQ: x64 (Linux, MacOS, Windows), ia32 (Linux, Windows),
arm (Android softFP, Linux hardFP), arm64 Android.
ABIs tested locally through Flutter: arm64 iOS.
ABIs not tested: ia32 Android (emulator), x64 iOS (simulator), arm iOS.
TEST=runtime/bin/ffi_test/ffi_test_functions_generated.cc
TEST=runtime/bin/ffi_test/ffi_test_functions.cc
TEST=tests/{ffi,ffi_2}/function_structs_by_value_generated_test.dart
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_generated_tes
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_test.dart
TEST=tests/{ffi,ffi_2}/vmspecific_static_checks_test.dart
Closes https://github.com/dart-lang/sdk/issues/37271.
Contains a temporary workaround for
https://github.com/dart-lang/sdk/issues/44454.
Change-Id: I5e5d10e09e5c3fc209f5f7e997efe17bd362214d
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/169221
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
This CL adds passing structs by value in FFI trampolines.
Nested structs and inline arrays are future work.
C defines passing empty structs as undefined behavior, so that is not
supported in this CL.
Suggested review order:
1) commit message
2) ffi/marshaller (decisions for what is done in IL and what in MC)
3) frontend/kernel_to_il (IL construction)
4) backend/il (MC generation from IL)
5) rest in VM
Overall architecture is that structs are split up into word-size chunks
in IL when this is possible: 1 definition in IL per chunk, 1 Location in
IL per chunk, and 1 NativeLocation for the backend per chunk.
In some cases it is not possible or less convenient to split into
chunks. In these cases TypedDataBase objects are stored into and loaded
from directly in machine code.
The various cases:
- FFI call arguments which are not passed as pointers: pass individual
chunks to FFI call which already have the right location.
- FFI call arguments which are passed as pointers: Pass in TypedDataBase
to FFI call, allocate space on the stack, and make a copy on the stack
and pass the copies' address to the callee.
- FFI call return value: pass in TypedData to FFI call, and copy result
in machine code.
- FFI callback arguments which are not passed as pointers: IL definition
for each chunk, and populate a new TypedData with those chunks.
- FFI callback arguments which are passed as pointer: IL definition for
the pointer, and copying of contents in IL.
- FFI return value when location is pointer: Copy data to callee result
location in IL.
- FFI return value when location is not a pointer: Copy data in machine
code to the right registers.
Some other notes about the implementation:
- Due to Store/LoadIndexed loading doubles from float arrays, we use
a int32 instead and use the BitCastInstr.
- Linux ia32 uses `ret 4` when returning structs by value. This requires
special casing in the FFI callback trampolines to either use `ret` or
`ret 4` when returning.
- The 1 IL definition, 1 Location, and 1 NativeLocation approach does
not remove the need for special casing PairLocations in the machine
code generation because they are 1 Location belonging to 1 definition.
Because of the amount of corner cases in the calling conventions that
need to be covered, the tests are generated, rather than hand-written.
ABIs tested on CQ: x64 (Linux, MacOS, Windows), ia32 (Linux, Windows),
arm (Android softFP, Linux hardFP), arm64 Android.
ABIs tested locally through Flutter: ia32 Android (emulator), x64 iOS
(simulator), arm64 iOS.
ABIs not tested: arm iOS.
TEST=runtime/bin/ffi_test/ffi_test_functions_generated.cc
TEST=runtime/bin/ffi_test/ffi_test_functions.cc
TEST=tests/{ffi,ffi_2}/function_structs_by_value_generated_test.dart
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_generated_tes
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_test.dart
TEST=tests/{ffi,ffi_2}/vmspecific_static_checks_test.dart
Closes https://github.com/dart-lang/sdk/issues/36730.
Change-Id: I474d3a4ee1faadbe767ddadd1b696e24d8dc364c
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140290
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Most of the time one already knows at the call site if it's a Field,
a Procedure etc --- so use that instead.
For now I'll leave the corresponding getters that are basically
documented as "don't use" ("[...] for convenience, not efficiency.
Consider manually iterating the members [...]").
Change-Id: Ib732759432c62963e6645f85f6df301c4281df9d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168826
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Changes the kernel representation of structs in two ways.
1. `_addressOf` field of `Struct` gets type `Object` because it will be
either `TypedData` or `Pointer` in structs by value CL.
2. Subtypes of `Struct` get a pragma `vm:ffi:struct-fields` which
contains a const list of the native types of the fields of the struct
which will be read in the VM to do compute the locations of structs
in the target ABI.
Split off from https://dart-review.googlesource.com/c/sdk/+/140290/23
to make that CL smaller. That CL will no longer have changes to the
kernel representation of FFI code after this CL lands separately.
These changes are not consumed in the VM in this CL, but they are tested
by the expect files.
Issue: https://github.com/dart-lang/sdk/issues/36730.
Change-Id: I5d3babd5be07f78c6d2bd80bbc1fd492c51bc01f
Cq-Include-Trybots: luci.dart.try:front-end-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167280
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL introduces dart_api_dl.h which exposes a subset of dart_api.h
and dart_native_api.h for dynamic linking at runtime through the FFI.
Dynamic linking is done through including dart_api_dl.cc in a shared
library and passing NativeApi.initializeApiDLData to the init function.
This CL also includes Native API versioning to deal with possible
version skew between native api version against which native libraries
are compiled and the version in the DartVM the code is run on.
The subset of symbols in the CL includes handle related symbols, error
related symbols, handle scope symbols, and native port sumbols.
Design: http://go/dart-ffi-expose-dart-api
Closes: https://github.com/dart-lang/sdk/issues/40607
Closes: https://github.com/dart-lang/sdk/issues/36858
Closes: https://github.com/dart-lang/sdk/issues/41319
Closes: https://github.com/flutter/flutter/issues/46887
Closes: https://github.com/flutter/flutter/issues/47061
Misc:
Closes: https://github.com/dart-lang/sdk/issues/42260
Change-Id: I9e557808dbc99b341f23964cbddbb05f26d7a6c5
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145592
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This prevents them from being called dynamically.
Moreover, it prevents asFunction from being called on a non-NativeFunction type argument, simplifying the amount of manual checks.
Note that this CL had to change the CFE and analzyer, and their tests (including mock_sdk) as well.
This can potentially be a breaking change, as the extension methods are only visible when `dart:ffi` is imported, while methods on objects are always visible.
Issue: https://github.com/dart-lang/sdk/issues/35903
Change-Id: I1e291f154228d5d9a34b21a022088bf493f6557d
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,analyzer-nnbd-linux-release-try,dart2js-nnbd-linux-x64-chrome-try,ddc-nnbd-linux-release-chrome-try,front-end-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135463
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL changes the message template generator to add an extra
'isNonNullableByDefault' parameter to 'withArguments' for all
messages that include types, thus forcing the reporter to specify
whether the library that the message belongs to is opted in to NNBD.
When the message is formatted, a '?' is added to all nullable types
if the message is reported for an opt-in library.
Change-Id: I4e1861eba48043c570afad3f40bc3dccfd2e5668
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124984
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Change Expression.getStaticType to take a StaticTypeContext instead
of a TypeEnvironment.
The StaticTypeContext provides access to the TypeEnvironment and the
current 'this type' as well as determining the nullability state of the
enclosing library.
This change is needed to support nnbd types for getStaticType and also
serves as a step towards supporting caching during static type
computations to avoid repeated computations of the same (complex)
expressions.
Change-Id: Ied974dff7f6f7c3c8f262aa80c8dea5c674662f7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124683
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Removes deprecated `load`, `store`, `allocate`, `free`, and `asExternalTypedData` methods from Pointer because these have been replace by extension methods and `package:ffi`.
Closes: https://github.com/dart-lang/sdk/issues/38860
Change-Id: I76d21bcfd2fdd6ee504a51db821345125beb92d2
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-dartkb-linux-debug-simarm64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-dartkb-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-reload-mac-release-simdbc64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-mac-release-simarm_x64-try,dart-sdk-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121708
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Daco Harkes