Commit Graph

125 Commits

Author SHA1 Message Date
Simon Binder a44a524bb0 [vm/ffi]: Add Native.addressOf
This adds the `Native.addressOf` function, which takes a constant tear-
off of a method annotated with `@Native` as a parameter and returns a
pointer to the underlying C function.

The CFE will resolve these calls in two steps: First, the existing
transformer for `@Native` methods adds a pragma describing the fully-
resolved native annotation (with the asset id inferred from the library
if not given explicitly). Then, the FFI use sites transformer rewrites
calls to `Native.addressOf` by searching for this pragma on the passed
function and passing the `Native` constants to `Native._addressOf`. The
latter method is implemented in the VM, which can re-use existing parts
already used for `@Native` calls.
An alternative implementation strategy would have been to forward
`addressOf` calls to `Native.ffi_resolver_function` directly without
any special consideration in the VM. However, the chosen approach makes
it easier to support static linking in the future, as this requires
unresolved symbols in the generated assembly.

Closes https://github.com/dart-lang/sdk/issues/50552

TEST=pkg/vm/testcases/transformations/ffi/ffinative.dart
TEST=tests/ffi/native_assets/asset_*_test.dart
TEST=tests/ffi/vmspecific_static_checks_ffinative_test.dart
TEST=pkg/analyzer/test/src/diagnostics/ffi_native_test.dart

Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-kernel-precomp-linux-release-x64-try
CoreLibraryReviewExempt: VM & dart2wasm only feature
Change-Id: Ic8e3a390146dffd44c95578f975a4472db79a0ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333920
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2023-12-05 13:10:36 +00:00
Alexander Markov b15e1bf19d [vm] Rename nullSafety to soundNullSafety in pkg/vm
Also:
* Clean up obsolete code for returning null safety from kernel
service.
* Enable sound null safety in TFA unit tests.

TEST=ci

Change-Id: I1e008ed661823e9cb8004670d6b66bef35089a19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/334081
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-11-07 14:32:20 +00:00
Daco Harkes f5983e4214 [ffi] Update some FfiNative docs to Native
The old FfiNatives have been deprecated for a while.

TEST=Only comments updated.

Change-Id: Ideb6e904841e5162d30df479e83b463b94682afe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332780
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-10-30 18:43:14 +00:00
Alexander Markov 293cc6475a [vm/aot/tfa] Whole-program propagation of closure values
This change adds propagation of closure values to TFA.
For now, inferred closure values are used only when they are
constant (tear-off of a static method or a constructor).

Arbitrary closures can now be referenced from unrelated
members via closure-id metadata.

In addition, this change fixes an incorrect stack trace when
an implicit closure (tear-off) was propagated and its call was
inlined. Inlining interval was not recorded because of the missing
source position of a call to a target method within implicit closure.

TEST=pkg/vm/testcases/transformations/type_flow/transformer/closures.dart
Issue: https://github.com/dart-lang/sdk/issues/39692

Change-Id: I3590da91b6057e0b55a8614382dba1bbcc267b39
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325447
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-10-12 16:16:21 +00:00
Martin Kustermann 587e6308bf [vm] Fix expression evaluation in context of closures
When evaluating an expressino at a breakpoint that's inside a closure,
the closure may refer to anything it closes over. That includes the
`this` from an enclosing method.

So depending on whether a closure's parent chain is an instance method
or a static method, the expression evaluation function is going to be an
instance method or a static method.

=> We walk up the parent chain to determine the correct enclosing class.
=> This avoids making a Type object with a cid from a top-level class (see [0])

Handling this correctly will now try to get the `this` when invoking an
eval function that has an enclosing instance method. Though we may often
have "<optimized out>" the `this` (e.g. due to not capturing it in
closure context chain).

=> We still allow running the expression evaluation function in this
case, but only if the expression being evaluated doesn't access `this`.

A similar issue occurs when trying to use variables in the eval
expression that the closure didn't capture. This results in a confusing
CFE compile-time error. This is a separate issue and tracked in [1].

=> We update the test to distinuish the cases that this CL makes passing
and those that are failing due to [1].

Fixes [0] https://github.com/dart-lang/sdk/issues/53061
See also [1] https://github.com/dart-lang/sdk/issues/53087

TEST=Fixes part of service/evaluate_activation_test failures

Change-Id: I3bb24e7338c7b2f12d5340311d944cb59a455641
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/317540
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-08-04 11:07:46 +00:00
Alexander Markov 110b0f0eba [tfa,dart2wasm] Create separate class for each record shape in TFA
This allows us to better track all kinds of accesses to record
implementation classes in dart2wasm, which generates separate
record implementation class per record shape.

This change also allows us to remove mutable dispatch targets
which were used to implement dynamic accesses to record fields,
and make tracking of record field types more accurate
(record fields are now versioned per shape).

This is also a step towards inferring actual record types.

TEST=pkg/vm/testcases/transformations/type_flow/transformer/records.dart
TEST=pkg/vm/testcases/transformations/type_flow/transformer/records_dart2wasm.dart
TEST=language/records/simple/dynamic_field_access_test

Issue https://github.com/dart-lang/sdk/issues/49719
Fixes https://github.com/dart-lang/sdk/issues/51363

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Icba62a7ca8cfd8ddbc7f2b7c38aeabbef5caec4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286950
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Ömer Ağacan <omersa@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-03-07 18:06:42 +00:00
Alexander Markov e27923a5a0 [dart2js] Static weak references to method tearoffs
TEST=language/static_weak_reference_test
TEST=language/static_weak_reference_error_test

Bug: b/269223463

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I760476a7c81751f6c302f21251b525cb5c916c02
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284489
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
2023-03-02 20:04:19 +00:00
Alexander Markov 143e4e9dc8 [vm/aot/tfa] Static weak references to method tearoffs
TEST=pkg/vm/testcases/transformations/type_flow/transformer/weak.dart
TEST=language/vm/static_weak_reference_test
TEST=language/vm/static_weak_reference_error_test

Bug: b/269223463
Change-Id: I23c8229c39217aa1c3f9fb576d8eefa5ceb1d8ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/283421
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2023-02-22 01:01:29 +00:00
Daco Harkes dc2c739a6c [vm/ffi] Fix Struct use in @Natives
The FfiNative transform was trying to access the generated struct
constructor which is added to the AST in the FFI definitions transform.

The FFI definitions transform has to run _before_ the FFI native
transform.

TEST=tests/ffi/regress_51321_test.dart

Closes: https://github.com/dart-lang/sdk/issues/51321
Change-Id: I6b5caf6e5851455cece6aec13ef73d85cc201ebf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/282103
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2023-02-13 09:40:24 +00:00
Alexander Markov d7e4ad02d7 [kernel] Cleanup Target.enableSuperMixins flag
This flag is no longer used by the front-end and can be removed.

TEST=ci

Change-Id: Ia19e9927d48d4ae486ee7ffa110abb9960a0840d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281461
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
2023-02-09 16:17:11 +00:00
Alexander Markov a9d9b7395b [kernel] Enable sound null safety in TargetFlags by default
TargetFlags.enableNullSafety is set to true by default and
also renamed to TargetFlags.soundNullSafety to better reflect its
meaning.

TEST=ci

Change-Id: I2c2f30c2af6502fd9a96141dc60e4afbf8c524fd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280216
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-02-02 14:54:18 +00:00
Ryan Macnak a2de36e708 [vm] Rename the default implementation classes of Map and Set.
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet

This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.

TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2022-11-15 17:15:58 +00:00
Leaf Petersen 131da0516b Refactor standalone tests to avoid including platform library code via parts
Add testing only stubs to dart:_http, and make it available for use in
the standalone tests so that the tests can stop including platform
library code directly using "part" directives (which can cause
surprising breakage, since the same code gets included twice via
different uris).

TEST=Existing tests

Change-Id: I1b5a5061008ef36980bd21b46a9d0fd701286f66
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262780
Reviewed-by: Kallen Tu <kallentu@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
2022-10-29 00:29:09 +00:00
Alexander Markov 671717b578 [vm/aot] Support dynamic record field access in TFA
TEST=language/records/simple/dynamic_field_access_test

Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I811db5c649988cbadf7ab29e5c4c70366f55e86b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262845
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-10-19 17:49:47 +00:00
Paul Berry 54906759b9 Don't delegate foreign private names to noSuchMethod.
If a concrete class implements an interface containing a name that's
private to a different library, any attempt to invoke that name will
result in an exception getting thrown.  Previously, such attempts
would result in the call being diverted to noSuchMethod.

This change closes a loophole in Dart's privacy system, and paves the way for
a future implementation of promotion for private final fields (see
https://github.com/dart-lang/language/issues/2020).

Bug: https://github.com/dart-lang/sdk/issues/49687
Change-Id: Ie55805e0fc77dc39713761a80a42c28bd0504722
Tested: language tests
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255640
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
2022-09-27 21:20:35 +00:00
Alexander Markov e5f3b0dc0e [vm/aot] Initial support for records in TFA
This change adds minimal support for RecordType, RecordLiteral,
RecordIndexGet, RecordNameGet and RecordConstant kernel nodes to TFA.
TFA is *not* yet extended with the ability to infer record types.

TEST=language/records/simple

Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I7c5eb860c6a5cb263e4d1bb55ad230e5c51f47c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/259520
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-09-16 18:43:50 +00:00
Aske Simon Christensen f2bc59738c Rename the VM's internal hash set classes
Rename `_CompactLinkedHashSet` to `_InternalLinkedHashSet` and
`_CompactImmutableLinkedHashSet` to `_InternalImmutableLinkedHashSet`
to make the names consistent with the corresponding `Map` class names
and to distinguish them from the non-VM-specific hash maps and sets,
which are generally named starting with `_Compact`.

Tested: ci
Change-Id: Ic1ebc44ba8beba34ceab382a4edddc6624fe3fb3
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258002
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2022-09-09 14:04:34 +00:00
Lasse R.H. Nielsen 8a883fa54d Change : to = for default values in pkg.
Leaves some in parser test:
 pkg/front_end/parser_testcases/error_recovery/keyword_named_formal_parameter_prime.dart

TEST=Refactoring, covered by existing tests.

Change-Id: I7a83ef95df3cbd283878b3685b5c747bd89a1b16
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256125
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
2022-08-24 15:57:16 +00:00
Alexander Markov 94c120a6ea [vm] Cleanup old async/async*/sync* implementation from kernel
This change removes kernel transformation which was used to
desugar async/async*/sync* functions in the old implementation of
async/async*/sync*.

The useful part of the transformation is retained in
pkg/vm/lib/transformations/for_in_lowering.dart.

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ic70c1fb35162a31bcc22eac3a8f6488b61e945b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249944
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-07-11 18:12:41 +00:00
Devon Carew 77c79d678e [pkg/vm] analyze using package:lints
Tested: via static analysis on the CI
Change-Id: Icffb8d52001d5099cca3ab1f153d3f6e5300177d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250770
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2022-07-07 23:22:49 +00:00
Alexander Markov 77ea9820aa [vm] New implementation of sync* based on suspend/resume stubs
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I7f4b6b56d914a617dfd7ac724cd4414532073b4c
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249141
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-06-27 18:19:12 +00:00
Alexander Markov 801bbb551b [flutter] Mark dart:mirrors library as unsupported in Flutter
Flutter doesn't support dart:mirrors (in all modes).
With this change CFE will be able to correctly select conditional
imports 'if (dart.library.mirrors)' and evaluate
const bool.fromEnvironment('dart.library.mirrors') in Flutter.

Issue: https://github.com/dart-lang/sdk/issues/49266
Change-Id: Ibde848d4a99ac818b736fe0d957cf23d920aa249
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249021
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2022-06-17 21:53:49 +00:00
Alexander Markov 2afc34112c [vm,kernel] New async/await implementation in the VM, part 1 - kernel
This change includes kernel-related changes:

* --compact-async option is added to kernel compilers (front-end
  server and gen_kernel). This option disables desugaring of async
  and await on kernel AST. Note that 'await for' is still desugared.

* File offset of the 'await' is now written for AwaitExpression nodes
  in the kernel binaries (will be used for async stack traces).

* Async/async*/sync* functions and AwaitExpression nodes are supported
  in TFA.

Design doc: go/compact-async-await.

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I4233086b7434bc48347f4220645b0be5f9133456
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241842
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-04-21 15:11:08 +00:00
Johnni Winther f26c0e2056 [cfe] Add explicit type argument to .map(...).toList()
Expressions like

  List<VariableDeclaration> variables = ...
  List<Expression> reads = variables.map((v) => VariableGet(v)).toList()

don't get the intended type argument `<Expression>` through inference,
but instead the type inferred from the return type of the closure, in
this case `<VariableGet>`.

This causes problems when used with the AST which assumes that
expressions are interchangeable wherever they are used, and is likely
the cause of https://github.com/flutter/flutter/issues/102077 .

This CL adds an explicit type argument in these cases.

TEST=general/infer_map

Change-Id: I0bcae21f06c54f290dc20686b0d84c065d8ea04c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241605
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2022-04-20 06:56:58 +00:00
Daco Harkes 532c116cd2 [vm] Implement NativeFinalizer
This CL implements `NativeFinalizer` in the GC.

`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.

On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.

In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.

When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.

In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.

Bug: https://github.com/dart-lang/sdk/issues/47777

TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*

Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-03-26 09:41:21 +00:00
Alexander Markov d0e8ef9816 [vm,kernel] Move transformations maintained by the VM team from pkg/kernel to pkg/vm
These transformations are maintained by the VM team and keeping them
in pkg/kernel would put a burden on the CFE team after code reviews
are tightened to require OWNERS approval.

TEST=ci

Change-Id: I931033ba987d17a2c9c6d80a16dad09bfbaa0572
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/234905
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-03-03 15:04:10 +00:00
Aske Simon Christensen 8df763f266 Prepare for sharing TFA and patch files between the VM and dart2wasm.
The VM patch files contain many annotations that guide the TFA and the
VM compilation pipeline. Most of these are not relevant to dart2wasm,
and having the TFA look at these when running inside dart2wasm leads to
sub-optimal, and sometimes even unsound, results.

In order to enable sharing of patch files between the VM and dart2wasm,
the annotation parser is given a target flag to control which
annotations it should recognize and how to parse them. The annotation
parser thus becomes an abstraction layer between the concrete
annotations in the patch files and the parsed annotations that the TFA
sees.

Additionally, external members used by dart2wasm don't always have
external names. To support this, the signature shaker must skip all
external members, not just the ones with an external name. This makes no
difference for the VM, since all of its external members either have an
external name or are implemented in a patch file, making them no longer
external.

TEST=ci + upcoming dart2wasm CL
Change-Id: Id425dd1ccc5560721034ae83c8cdc1593801a4e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231102
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
2022-02-01 21:08:46 +00:00
Jens Johansen ae593598f8 [dart:ffi] Only operate on transitive dependencies of dart:ffi
This CL does two things:
* "Reenable" the skipping of the dart:ffi transformation entirely when
  the application does not use dart:ffi. This was originally added in
  ad596359a6 but was logically disabled in 4558112105 when dart:core
  started depending on it. Here we now ignore (real) dart:* libraries
  when looking for dependencies of dart:ffi.
* Find the subset of the just compiled libraries that transitively
  depend on dart:ffi and just run the dart:ffi transformation on those
  libraries.


For dart2js that doesn't use dart:ffi at all we go from:

```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.003428: Transformed ffi natives in 51ms.
0:00:09.203442: Transformed ffi annotations in 199ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.098175: Transformed ffi natives in 52ms.
0:00:09.296847: Transformed ffi annotations in 198ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:08.854341: Transformed ffi natives in 57ms.
0:00:09.061804: Transformed ffi annotations in 207ms.
```

to

```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.138651: Skipped ffi transformation in 1ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.242590: Skipped ffi transformation in 1ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v pkg/compiler/bin/dart2js.dart | grep -i "ffi"
0:00:09.163885: Skipped ffi transformation in 1ms.
```

When compiling the flutter gallery app that does depend on dart:ffi we go from:

```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:16.344955: Transformed ffi natives in 78ms.
0:00:16.777218: Transformed ffi annotations in 432ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:15.994674: Transformed ffi natives in 80ms.
0:00:16.472779: Transformed ffi annotations in 478ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:16.027488: Transformed ffi natives in 81ms.
0:00:16.491362: Transformed ffi annotations in 463ms.
```

to

```
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:15.991628: Transformed ffi natives in 30ms.
0:00:16.226564: Transformed ffi annotations in 234ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:16.184984: Transformed ffi natives in 29ms.
0:00:16.404439: Transformed ffi annotations in 219ms.
$ out/ReleaseX64/dart pkg/front_end/tool/_fasta/compile.dart -v --target=flutter --platform=/tmp/tmp.6KV5psVGIG/flutter_patched_sdk/platform_strong.dill /tmp/tmp.6KV5psVGIG/gallery/lib/main.dart | grep -i "ffi"
0:00:15.933513: Transformed ffi natives in 27ms.
0:00:16.145640: Transformed ffi annotations in 212ms.
```

TEST=Assuming tests already exists.

Change-Id: Ie13be1924d3439cb710c782af81e1e42cda2a838
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228001
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2022-01-17 09:43:14 +00:00
Johnni Winther 8510f27d37 [cfe] Change encoding of supported dart: libraries
This CL changes the way dart: libraries are considered supported when
used in conditional imports or bool.fromEnvironment constant using
the "dart.library.*" values.

Library nodes now has an isUnsupported flag which is set according to
the "supported" property in the libraries specification. Furthermore
the Target now supplies a DartLibrarySupport interface that allows
targets to override whether dart: libraries are unsupported.

This allows the JIT/AOT to use the same platform file but still
consider dart:mirrors unsupported in AOT mode, and dart2js to consider
the internal library `dart:_dart2js_runtime_metrics` supported.

Furthermore, the internal handling is changed so that condition imports
and bool.fromEnvironments constants are computed through the same logic
for "dart.library.*" values, avoiding the need for passing these values
through the environment.

TEST=pkg/front_end/testcases/general/supported_libraries/main

Closes https://github.com/dart-lang/sdk/issues/48057
Closes https://github.com/dart-lang/sdk/issues/47814
Closes https://github.com/dart-lang/sdk/issues/47243
Closes https://github.com/dart-lang/sdk/issues/32657
Closes https://github.com/dart-lang/sdk/issues/36460

Change-Id: Ie8f8dff99167de64ced51b71d89918bf0f3bbd13
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227020
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2022-01-11 14:52:38 +00:00
Daco Harkes 0cd586486b [cfe/ffi] Move FFI transforms into folder
Having the transforms in a dedicated folder enables us to add more files
without cluttering the transformations folder itself.

TEST=This only moves files. Bots test whether imports are correct.

Change-Id: Icf1d64b5ec09a7d19915b1cb72464fa67b6cfe15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217180
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-10-19 07:25:09 +00:00
Clement Skau 998d5f5a9b [vm/ffi] Adds FFI transform FfiNative instance methods.
- Allows FfiNative annotation to be attached to non-static
  methods of classes.
- Transforms non-static instance methods to static, adding
  wrappers with the receiver as an extra, implicit first paramters.
- Transform all parameters and arguments to Pointer if the object
  being passed supports it (i.e. extends NativeFieldWrapperClass1).
- Adds compile time errors for cases where the FfiNative annotation
  doesn't align with the annotated function. Taking into account
  implicit receivers and converted Pointers.
- Adds complimentary Analyzer checks for the above errors as well.
- Adds tests for the transforms, compile time errors and analyzer
  changes.

TEST=Adds new tests for instance methods, analyzer changes.

Change-Id: Idf54430acf2728a650008333b149b254941290ad
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/213773
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-10-12 10:58:44 +00:00
Johnni Winther bc1c71fd9b [cfe] Handle un-inlined fields/locals in eager instantiation constantification
Closes #47108

TEST=existing

Change-Id: I590939161cde1316eede7d828dff5fe6ac2790fd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212577
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2021-09-07 15:14:36 +00:00
Daco Harkes 2b5adb7231 [vm] Implement constant Maps in the VM
This CL changes the frontend to emit map and set constants which are
then processed by the constant reader in the VM.

This CL also introduces support for sending the const maps and sets
in messages between isolates and saving it in snapshots.

TEST=tests/language/const/map_test.dart (et al, for lookups)
TEST=tests/lib/isolate/message3_test.dart (et al, for isolate messages)
TEST=tools/test.py -c dartkp (for consts from clustered snapshot)
TEST=tools/test.py -n app_jitk-linux-debug-x64 (for consts from app
     jit snapshots)
TEST=Building the SDK which uses const Maps in clustered snapshots.

Closes: https://github.com/dart-lang/sdk/issues/45908

Change-Id: I1f8150a8aba8298c4e64d2571dd147743526135a
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203765
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-09-06 06:28:59 +00:00
Johnni Winther 93c531f1a0 [cfe] Move force* flags from TargetFlags to TestTargetFlags
TEST=existing

Change-Id: I3db6f5bdd72ca6303766b6524df19e66b1ba1cc7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212001
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-09-01 13:29:26 +00:00
Alexander Markov 0b1ddaa598 [vm] Enable lowering of typedef constructor tear-offs
TEST=language/typedef/aliased_constructor_tear_off_test
Issue https://github.com/dart-lang/sdk/issues/46231

Change-Id: I10df62d4d170a4ae018a0e943ac6fc37d669d439
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208984
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-08-09 18:06:39 +00:00
Johnni Winther cb3a9b0043 [cfe] Remove old method invocation encoding nodes
TEST=existing

Change-Id: I05ee649ecfa7945e2e0f5e5d09441d8916a9c46e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208185
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2021-07-28 15:44:04 +00:00
Alexander Markov 45ae00676a Migrate pkg/vm to null safety, part 4
TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/46620
Change-Id: I74c45ed1525248447e769cc832366728e7a017fc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208020
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-07-27 17:52:18 +00:00
Clement Skau 2f0b352e98 [VM] Adds compile-time error for non-static FfiNative
This makes it a compile-time error to add @FfiNative annotation
to any non-static function, such as an instance method.

TEST=tests/ffi/ffi_native_test.dart,pkg/analyzer/test/src/diagnostics/ffi_native_test.dart

Bug: https://github.com/dart-lang/sdk/issues/43889
Change-Id: Ib9ec61345bb47e735ed635c5ceea15ab643f65a6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207306
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-07-20 11:54:30 +00:00
Alexander Markov 8e1fbc9ab3 Migrate pkg/vm to null safety, part 1
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/46620

Change-Id: I2ad3f3aac03c8c13d2562a385a5b6636675401f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207362
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2021-07-20 06:49:20 +00:00
Johnni Winther a0d57b0a4d [cfe] Initial implementation for constructor tear off lowering
TEST=pkg/front_end/testcases/constructor_tearoffs/lowering/*

Change-Id: Ie8a36df4068093dc4d239b48c31f47d03054ba2b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206080
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2021-07-08 11:05:04 +00:00
Clement Skau 2f916ad807 [vm/ffi] Adds @FfiNative() support
Adds support for marking external functions as @FfiNative's,
which will be called using fast FFI calls.

Resolution happens by calling out to a new embedder provided
Dart_FfiNativeResolver which the embedder can specify via
Dart_SetFfiNativeResolver.

TEST=vm/cc/DartAPI_FfiNativeResolver

Bug: https://github.com/dart-lang/sdk/issues/43889
Change-Id: I3cfff360b05314499a81444b90f4ea0a1b937b0b
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170092
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-06-15 05:58:39 +00:00
Alexander Markov 88021fd67a [vm] Enable new kernel AST invocation nodes for the VM, take 2
This change was already reviewed at
https://dart-review.googlesource.com/c/sdk/+/197586.

Bugfix is in the separate change:
https://dart-review.googlesource.com/c/sdk/+/199365.

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/45340
Change-Id: I4311909e4893af53e88895512f03d3ef84c6bc5f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/199366
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2021-05-20 18:53:45 +00:00
Martin Kustermann 31364331e9 Revert "[vm] Enable new kernel AST invocation nodes for the VM"
This reverts commit 53e3c079b7.

Reason for revert: This change has seemingly caused errors of the form
```
stderr >> Unhandled exception:
stderr >> Invalid argument(s): Missing canonical name for Reference to <XXX>
stderr >> #0      BinaryPrinter.writeNullAllowedReference (package:kernel/binary/ast_to_binary.dart:894:9)
stderr >> #1      BinaryPrinter.visitTypedefType (package:kernel/binary/ast_to_binary.dart:2410:5)
stderr >> #2      TypedefType.accept (package:kernel/ast.dart:11115:42)
stderr >> #3      BinaryPrinter.writeNode (package:kernel/binary/ast_to_binary.dart:403:10)
stderr >> #4      BinaryPrinter.writeOptionalNode (package:kernel/binary/ast_to_binary.dart:496:7)
```

Issue https://github.com/dart-lang/sdk/issues/45966

Original change's description:
> [vm] Enable new kernel AST invocation nodes for the VM
>
> TEST=ci
>
> Issue: https://github.com/dart-lang/sdk/issues/45340
> Change-Id: Ibb307d8df8ff4ae8f2efad177880634ec9e27905
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197586
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Aske Simon Christensen <askesc@google.com>

TBR=vegorov@google.com,alexmarkov@google.com,johnniwinther@google.com,askesc@google.com

Change-Id: I05d010378cfcc5ba40f2be58b01b3ed27a8fc31e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Issue: https://github.com/dart-lang/sdk/issues/45340
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/199000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-05-10 12:32:38 +00:00
Alexander Markov 53e3c079b7 [vm] Enable new kernel AST invocation nodes for the VM
TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/45340
Change-Id: Ibb307d8df8ff4ae8f2efad177880634ec9e27905
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197586
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2021-05-09 21:00:28 +00:00
Daco Harkes cf647f5aad [cfe/ffi] Remove empty Struct use site check
After https://dart-review.googlesource.com/c/sdk/+/197740, I realized
we also don't need the other data structure anymore.

In https://dart-review.googlesource.com/c/sdk/+/180189, we disallow
empty `Struct`s on the definition sites, making a check on the use sites
superfluous.

This removes the last dependency between the ffi definitions and ffi
use sites transformers.

Bug: https://github.com/dart-lang/sdk/issues/45899

TEST=tests/ffi/vmspecific_static_checks_test.dart

Change-Id: I6eb5a26d4ece713107ba2a9e6bf601a6e029baa7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198047
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-05-05 07:19:02 +00:00
Johnni Winther 8f92864ca3 [kernel] Rename MapEntry to MapLiteralEntry
- to avoid collisions with MapEntry from dart:core

TEST=existing

Change-Id: I7b9592844345313e79ac18d18017c74de7c02106
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196930
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2021-04-27 10:36:38 +00:00
Johnni Winther 9d80c4db57 [kernel,vm] Migrate vm transformations in package:kernel
TEST=existing

Change-Id: I86dfda23fa32ecb924fa6c202e7142cb8a258dc6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192145
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2021-04-07 08:30:47 +00:00
Jens Johansen ad596359a6 [VM] Only run FFI transformation when dart:ffi is imported
Before this CL the FFI transformation was always run (for target VM),
so if for instance running dart2js (from source), e.g. like
`out/ReleaseX64/dart -DDFE_VERBOSE=true pkg/compiler/bin/dart2js.dart --help`
one could see how it spends something like 120-140 ms, which is
something along the lines of 2.5% of the entire request processing
(i.e. compile and serialization).

This CL first checks if dart:ffi is imported at all. If it's not the
transformation is skipped.
When compiling dart2js (where, at least currently, dart:ffi is not used)
this skips the transformation, thus saving the 120-140 ms (or ~2.5%).

All measurements on my machine. Your mileage may vary.

TEST=Existing tests.

Change-Id: Ia5d3d7b989f549ca5da257660c204e8c59f15d4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185543
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-02-19 09:44:08 +00:00
Johnni Winther 07ddd00e1b [cfe] Remove unneeded hide combinators
The front end previously enforced a stricter-than-spec requirement on
conflicting imports on its own code. The check was included of the
kernel snapshot and therefore always enforced, even in published sdks.

The extra check was removed a month ago and now tools/sdks/ have been
updated to use a later version of the sdk, so the unneeded hide
combinators can now be removed from the source code.

Closes #44667

TEST=existing

Change-Id: I1d1053b1ef9a40b6a918eef515a02d7b404906c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185084
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-02-18 14:23:10 +00:00
Alexander Markov 9d1a307954 [vm] Add specialized factory constructors for small list literals
Flutter gallery in release mode:
AOT snapshot size arm64 -0.22%, arm -0.21%
instructions size arm64 -0.46%, arm -0.4%

TEST=existing tests

Fixes https://github.com/dart-lang/sdk/issues/44391

Change-Id: I4733e91ecf4601c0bcde725cf9e97f5db0b8bc13
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175821
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-11 19:32:30 +00:00