This CL prevents treeshaking of members that have type `Finalizable`, `Future<Finalizable>` or `FutureOr<Finalizable>`.
All `Finalizable` members are kept. Even the ones which are not
members of `Finalizable`s. This is in line with the logic to keep
all `Finalizable` arguments/variables alive.
Moreover, this CL adds AOT tests to the FFI transformer tests so that
we catch differences between AOT/JIT in the future.
Closes: https://github.com/dart-lang/sdk/issues/49643
TEST=pkg/vm/test/transformations/ffi_test.dart
TEST=pkg/vm/testcases/transformations/ffi/finalizable_member.dart
Change-Id: I14003314b9f23692fee30d1c3eef1bdcd27ed1ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254904
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Weaken assertion in gen_snapshot requiring all libraries to have a loading unit as there can still be unreachable libraries:
- Google3 and Fuchsia will compile all the sources in a package to a single dill file, then present multiple input dill files to the AOT compilation. Since the set of libraries was derived from package membership instead of imports, many can be unreachable.
- When the root library's main comes from an export, the frontend's representation will incorrectly report the library containing main as the root library and the true root library may be unreachable from it.
Instead, assert only that surviving compiled code is assigned a loading unit.
TEST=gallery
Bug: https://github.com/flutter/gallery/issues/545
Bug: https://github.com/dart-lang/sdk/issues/49325
Bug: https://github.com/dart-lang/sdk/issues/41974
Bug: b/237016312
Change-Id: Ia52563a6f517308d041368be11dcc85270f19acc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249724
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
If a library contained an entry point but was not reachable from the root library, it was not assigned to any loading unit and caused a null dereference in gen_snapshot. This is not possible with the standalone embedder, but is possible in Flutter because it passes multiple sources to frontend_server. E.g., `--source dart_plugin_registrant.dart`.
TEST=gallery
Bug: https://github.com/flutter/gallery/issues/545
Change-Id: I9c67f0e39f7509094ee873610d80851a702a0cf2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249640
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Use `hasScheme` in place of comparing against the empty string, and
`isScheme` to compare against all other schemes.
TEST=No behavior changes.
Change-Id: Ifc9fd13c6cf37933ebd4a754c4b500dedbcb291b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231185
Reviewed-by: Kevin Moore <kevmoo@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
This CL implements `Finalizable` by means of a CFE transform.
The transform is implemented in the existing FFI transform to avoid the
overhead of traversing the AST an extra time. The FFI transform slows
down ~15-30% on Flutter Gallery. For more info see the design doc.
TEST=`dart pkg/vm/test/transformations/ffi_test.dart`
Design doc: go/dart-vm-finalizable
Bug: https://github.com/dart-lang/sdk/issues/47777
Change-Id: I0bf9dead90a61631b7f215dc19fbaa9e40e63c8d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227501
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The VM patch files contain many annotations that guide the TFA and the
VM compilation pipeline. Most of these are not relevant to dart2wasm,
and having the TFA look at these when running inside dart2wasm leads to
sub-optimal, and sometimes even unsound, results.
In order to enable sharing of patch files between the VM and dart2wasm,
the annotation parser is given a target flag to control which
annotations it should recognize and how to parse them. The annotation
parser thus becomes an abstraction layer between the concrete
annotations in the patch files and the parsed annotations that the TFA
sees.
Additionally, external members used by dart2wasm don't always have
external names. To support this, the signature shaker must skip all
external members, not just the ones with an external name. This makes no
difference for the VM, since all of its external members either have an
external name or are implemented in a patch file, making them no longer
external.
TEST=ci + upcoming dart2wasm CL
Change-Id: Id425dd1ccc5560721034ae83c8cdc1593801a4e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231102
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
This CL changes the way dart: libraries are considered supported when
used in conditional imports or bool.fromEnvironment constant using
the "dart.library.*" values.
Library nodes now has an isUnsupported flag which is set according to
the "supported" property in the libraries specification. Furthermore
the Target now supplies a DartLibrarySupport interface that allows
targets to override whether dart: libraries are unsupported.
This allows the JIT/AOT to use the same platform file but still
consider dart:mirrors unsupported in AOT mode, and dart2js to consider
the internal library `dart:_dart2js_runtime_metrics` supported.
Furthermore, the internal handling is changed so that condition imports
and bool.fromEnvironments constants are computed through the same logic
for "dart.library.*" values, avoiding the need for passing these values
through the environment.
TEST=pkg/front_end/testcases/general/supported_libraries/main
Closes https://github.com/dart-lang/sdk/issues/48057
Closes https://github.com/dart-lang/sdk/issues/47814
Closes https://github.com/dart-lang/sdk/issues/47243
Closes https://github.com/dart-lang/sdk/issues/32657
Closes https://github.com/dart-lang/sdk/issues/36460
Change-Id: Ie8f8dff99167de64ced51b71d89918bf0f3bbd13
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227020
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Having the transforms in a dedicated folder enables us to add more files
without cluttering the transformations folder itself.
TEST=This only moves files. Bots test whether imports are correct.
Change-Id: Icf1d64b5ec09a7d19915b1cb72464fa67b6cfe15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217180
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
- Allows FfiNative annotation to be attached to non-static
methods of classes.
- Transforms non-static instance methods to static, adding
wrappers with the receiver as an extra, implicit first paramters.
- Transform all parameters and arguments to Pointer if the object
being passed supports it (i.e. extends NativeFieldWrapperClass1).
- Adds compile time errors for cases where the FfiNative annotation
doesn't align with the annotated function. Taking into account
implicit receivers and converted Pointers.
- Adds complimentary Analyzer checks for the above errors as well.
- Adds tests for the transforms, compile time errors and analyzer
changes.
TEST=Adds new tests for instance methods, analyzer changes.
Change-Id: Idf54430acf2728a650008333b149b254941290ad
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/213773
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
We can't rely on GC to trigger the finalizer of a given test object,
so the previous liveness test was unreliable.
Instead we add an expect test to verify we generate the necessary
`reachabilityFence(..)`s to ensure liveness.
TEST=Adds expect.
Bug: https://github.com/dart-lang/sdk/issues/47362
Change-Id: Ia57a07522c8b8265b24780f00f3339b50534eb60
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-mac-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215542
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This change adds toString transformation to gen_kernel and
frontend_server tools in Dart SDK.
The implementation and tests are derived from
pkg/frontend_server/lib/src/to_string_transformer.dart and
https://github.com/flutter/engine/tree/master/flutter_frontend_server/test.
In addition to _KeepToString in dart:ui, toString transformation
now supports @pragma('flutter:keep-to-string') to exclude
certain toString methods from the transformation without depending
on dart:ui.
pkg/frontend_server/lib/src/to_string_transformer.dart is not
cleaned up in this change yet as it is still used by Flutter
engine. Cleanup will be done after Flutter engine is cleaned up,
after it is switched to the implementation of toString transformer
added in this change.
This is also a step towards a unified kernel compiler
(https://github.com/dart-lang/sdk/issues/39126).
TEST=pkg/vm/test/transformations/to_string_transformer_test.dart
Issue: https://github.com/dart-lang/sdk/issues/46022
Issue: https://github.com/dart-lang/sdk/issues/39126
Change-Id: Icbfd3fa193d54f1fc6b2d7fa0bace82b3704f91f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/200525
Reviewed-by: Dan Field <dnfield@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL enables tree shaking of `Struct` sub classes by simulating the
native behavior in Dart code.
We call the struct constructors in let expressions where FFI trampolines
are created which allocate these struct objects in native code. This way
TFA is instructed about the native behavior. The VM recognizes these
constructor calls as dead code and removes them.
For more info see go/dart-ffi-struct-treeshaking.
Closes: https://github.com/dart-lang/sdk/issues/38721
This CL fixes the types on the generated #fromTypedDataBase constructors.
Also, this CL also cleans up the struct naming:
* _addressOf -> _typedDataBase
* _fromPointer -> _fromTypedDataBase
* #pointer -> #typedDataBase
These cleanups are not split into a separate CL to prevent updating
the .expect files multiple times.
Finally, this CL enables running a single transformer test through:
`dart pkg/vm/test/transformations/type_flow/transformer_test.dart name`
TEST=pkg/vm/testcases/transformations/type_flow/transformer/ffi_struct_constructors.dart
TEST=tests/ffi(_2)/function_callbacks_structs_by_value_generated_test.dart
TEST=tests/ffi(_2)/function_structs_by_value_generated_test.dart
Change-Id: I418d0d73bc86b234dfe5b7b04ae726c33d2b8aeb
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,dart-sdk-linux-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193661
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Evaluated constant instances in kernel are represented as a list of
field->value pairs. Tree shaker should not remove fields used in
constants as it may potentially affect identity of constant instances
(non-identical constants may become identical).
Previously, fields used in constants were only marked as written,
so they could still be removed as write-only fields.
Furthermore, if such field was also used as an interface target
it could be converted to an abstract getter, causing the conflict
during kernel AST serialization.
This fix marks fields used in constants as also read, effectively
preventing their tree shaking.
Flutter gallery AOT snapshot size is not affected.
TEST=pkg/vm/testcases/transformations/type_flow/transformer/regress_45324.dart
Fixes https://github.com/dart-lang/sdk/issues/45324
Change-Id: Id6d35604757ad6ba1e23f7227aa9b769448c2688
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191280
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is in preparation to migrate package:kernel to null safety.
For the visitor interfaces to support non-nullable return types, the
implementations must avoid using `null` as return value in its base case.
TEST=Refactoring
Change-Id: Ie8fa5d41b99850d9e4abb59634c72920c64128d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183691
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
This is in preparation to migrate package:kernel to null safety.
For the visitor interfaces to support non-nullable return types, the
implementations must avoid using `null` as return value in its base case.
TEST=Refactoring
Change-Id: Ie5e4153f8d3779d94957bb13b3d2d2a942040ff2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179760
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
In certain cases involving auto-generated Dart sources there could be a
huge number of allocated classes which are subtypes of a certain class.
Specializing such cone types to set types, as well as intersection and
union operations on such types may be slow and may severely affect
compilation time. Also, gradually discovering allocated classes in
such cone types may cause a lot of invalidations during analysis.
In order to avoid servere degradation of compilation time in such case,
this change adds WideConeType which works like a ConeType when number of
allocated classes is large, but it doesn't specialize to a SetType and
has more efficient but approximate implementation of union and
intersection.
Uncompressed size of Flutter gallery AOT snapshot (android/arm64):
WideConeType approximation for types with
>32 allocated subtypes: +0.1176%
>64 allocated subtypes: +0.0956%
>128 allocated subtypes: +0.0027%
For now conservative approximation is used when number of allocated
types >128.
TFA time of large app #1: 175s -> 119s (-32%)
TFA time of large app #2: 211s -> 81s (-61.6%)
Snapshot size changes are insignificant.
TEST=Stress tested on precomp bots with
maxAllocatedTypesInSetSpecializations = 3 and 0.
Issue: https://github.com/dart-lang/sdk/issues/42442
Issue: https://github.com/dart-lang/sdk/issues/43299
Change-Id: Idae33205ddda81714e4aeccc7ae292e0164be651
b/154155290, b/177498788, b/177497864
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179200
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
- Global parameter use analysis: when a parameter is only used directly
as arguments to other calls, only consider it used if any of the
target parameters are used.
- A parameter can be eliminated if the TFA infers that it has a constant
value in every implementation.
This change also extends the test of the protobuf-aware tree shaker to
run the AOT global transformations on all its test cases in order to
check that the protobuf handling in that code path is not broken by
signature shaking.
Reduces ARM64 .so size of Flutter Gallery by 0.25% uncompressed, 0.37%
with gzip and 0.45% with brotli.
Issue: https://github.com/dart-lang/sdk/issues/40488
TEST=Existing test suite, expectation files for TFA test, extended test
of protobuf-aware tree shaker.
Change-Id: Ib00fe362f798c92592a260f72e7bccad04246bb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173727
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is in preparation to migrate package:kernel to null safety.
For the visitor interfaces to support non-nullable return types, the
implementations must avoid using `null` as return value in its base case.
TEST=Refactoring
Change-Id: I9f9b318982148d844be9826a5f8c88374a9fc402
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172180
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The existing signature shaker transforms optional and named parameters
of static and top-level methods and constructors into required
parameters whenever they are always passed.
This change improves signature shaking in a number of areas:
- Extend signature shaking to cover instance methods.
- Remove parameters which are never passed.
- Remove parameters which are not used.
- Transform signatures before unboxing, enabling transformed parameters
to be unboxed.
TFA summaries for transformed functions are updated to stay in sync with
the program for later transformations.
Reduces instructions size of Flutter Gallery on ARM64 by 2.2%.
Issue: https://github.com/dart-lang/sdk/issues/40488
Change-Id: I39e364bc23b01e4502864d7369a745a9278cfb92
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161175
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This change introduces handling of protobufs while doing type flow
analysis. Metadata in protobuf message classes is updated dynamically
according to the set of called accessors, invalidating and rebuilding
TFA summaries as needed.
Previously, protobuf-aware tree shaker required the 2nd run of TFA
in order to do the actual tree-shaking after protobuf messages are
pruned. This significantly increases compilation time.
AOT compilation time of a large app (--from-dill): 274s -> 152s
New tree shaker is available in kernel compilers under the flag
--protobuf-tree-shaker-v2.
Issue https://github.com/dart-lang/sdk/issues/42442
Fixes https://github.com/dart-lang/sdk/issues/40785
Change-Id: I4347896737b9b0f7407b845e614dda9ba7621921
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
The partitioning has the property that for every reference from loading unit S to unit T, either
- the reference is through a deferred prefix, or
- T is guarenteed to be loaded before S
The partitioning is constructed by cutting the dominator tree of the library import graph at deferred imports. It is suboptimal if there are deferred imports to multiple libraries in a connected component, but the expected use case involves loading units with a single deferred entry.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I3c49044ae19112525f197b077b36e4ce874b81ac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151470
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL adds a Node.toText method together with an AstPrinter. These
facility and better toString implementation on AST nodes while allowing
for toString independent printing of AST to use in testing. This also
add support for an integrated toString of custom/internal nodes.
Some work is still needed in bringing the toString implementation on
all nodes to the old quality, and not all internal nodes have
customized textual representations yet. This work is left for future
CLs.
Change-Id: Ib0bf8a0bc02f489dfacdc8aa5f96da9c52f26058
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150923
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
When performing protobuf-aware treeshaking, we may generate a kernel
file which will later be used in conjunction with a non-transformed
platform kernel. In this case signature shaking is unsafe.
The original revision is in patchset 1.
This reverts commit b59dad38df.
Change-Id: I1a85e42f245fcdbd821fa1f07b2e9966b8c2d958
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139100
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
So far tree shaking was removing fields which are not used at all.
This change improves tree shaking of fields so fields
which are only written or used as interface targets can be removed.
The following limitations apply:
* Field is not removed if there is a constant object with that field, as
it may impact identity of constant objects which is an observable
behavior.
* Field is not removed if it has a non-trivial initializer as it may
have side-effects.
* Late final fields are not removed, as writing such fields may have
side-effect.
* When field is removed, we may need to introduce an abstract getter
or abstract setter if field is used as a target of an interface call.
If a field was written, then setter would be non-abstract (but empty).
Fixes https://github.com/dart-lang/sdk/issues/35310
Change-Id: I79c00158b8eb658081a647c5dbdecde481fddb41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134204
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
New class representation is used in ConcreteType and ConeType instead of
DartType. Uses of pre-NNBD legacy raw types are eliminated in most cases.
This also simplifies subtype checks.
TypesBuilder class is created to translate static types into
TFA types. It holds CoreTypes which is used to recognize Null class
and supersedes Type.cone and Type.fromStatic factory constructors.
Change-Id: Ia1cd48cf8c738173a06dd04864f6cec8ae6def83
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124286
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Daco Harkes