This CL enables tree shaking of `Struct` sub classes by simulating the
native behavior in Dart code.
We call the struct constructors in let expressions where FFI trampolines
are created which allocate these struct objects in native code. This way
TFA is instructed about the native behavior. The VM recognizes these
constructor calls as dead code and removes them.
For more info see go/dart-ffi-struct-treeshaking.
Closes: https://github.com/dart-lang/sdk/issues/38721
This CL fixes the types on the generated #fromTypedDataBase constructors.
Also, this CL also cleans up the struct naming:
* _addressOf -> _typedDataBase
* _fromPointer -> _fromTypedDataBase
* #pointer -> #typedDataBase
These cleanups are not split into a separate CL to prevent updating
the .expect files multiple times.
Finally, this CL enables running a single transformer test through:
`dart pkg/vm/test/transformations/type_flow/transformer_test.dart name`
TEST=pkg/vm/testcases/transformations/type_flow/transformer/ffi_struct_constructors.dart
TEST=tests/ffi(_2)/function_callbacks_structs_by_value_generated_test.dart
TEST=tests/ffi(_2)/function_structs_by_value_generated_test.dart
Change-Id: I418d0d73bc86b234dfe5b7b04ae726c33d2b8aeb
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,dart-sdk-linux-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193661
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Evaluated constant instances in kernel are represented as a list of
field->value pairs. Tree shaker should not remove fields used in
constants as it may potentially affect identity of constant instances
(non-identical constants may become identical).
Previously, fields used in constants were only marked as written,
so they could still be removed as write-only fields.
Furthermore, if such field was also used as an interface target
it could be converted to an abstract getter, causing the conflict
during kernel AST serialization.
This fix marks fields used in constants as also read, effectively
preventing their tree shaking.
Flutter gallery AOT snapshot size is not affected.
TEST=pkg/vm/testcases/transformations/type_flow/transformer/regress_45324.dart
Fixes https://github.com/dart-lang/sdk/issues/45324
Change-Id: Id6d35604757ad6ba1e23f7227aa9b769448c2688
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191280
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is in preparation to migrate package:kernel to null safety.
For the visitor interfaces to support non-nullable return types, the
implementations must avoid using `null` as return value in its base case.
TEST=Refactoring
Change-Id: Ie8fa5d41b99850d9e4abb59634c72920c64128d9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183691
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
This is in preparation to migrate package:kernel to null safety.
For the visitor interfaces to support non-nullable return types, the
implementations must avoid using `null` as return value in its base case.
TEST=Refactoring
Change-Id: Ie5e4153f8d3779d94957bb13b3d2d2a942040ff2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179760
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
In certain cases involving auto-generated Dart sources there could be a
huge number of allocated classes which are subtypes of a certain class.
Specializing such cone types to set types, as well as intersection and
union operations on such types may be slow and may severely affect
compilation time. Also, gradually discovering allocated classes in
such cone types may cause a lot of invalidations during analysis.
In order to avoid servere degradation of compilation time in such case,
this change adds WideConeType which works like a ConeType when number of
allocated classes is large, but it doesn't specialize to a SetType and
has more efficient but approximate implementation of union and
intersection.
Uncompressed size of Flutter gallery AOT snapshot (android/arm64):
WideConeType approximation for types with
>32 allocated subtypes: +0.1176%
>64 allocated subtypes: +0.0956%
>128 allocated subtypes: +0.0027%
For now conservative approximation is used when number of allocated
types >128.
TFA time of large app #1: 175s -> 119s (-32%)
TFA time of large app #2: 211s -> 81s (-61.6%)
Snapshot size changes are insignificant.
TEST=Stress tested on precomp bots with
maxAllocatedTypesInSetSpecializations = 3 and 0.
Issue: https://github.com/dart-lang/sdk/issues/42442
Issue: https://github.com/dart-lang/sdk/issues/43299
Change-Id: Idae33205ddda81714e4aeccc7ae292e0164be651
b/154155290, b/177498788, b/177497864
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179200
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
- Global parameter use analysis: when a parameter is only used directly
as arguments to other calls, only consider it used if any of the
target parameters are used.
- A parameter can be eliminated if the TFA infers that it has a constant
value in every implementation.
This change also extends the test of the protobuf-aware tree shaker to
run the AOT global transformations on all its test cases in order to
check that the protobuf handling in that code path is not broken by
signature shaking.
Reduces ARM64 .so size of Flutter Gallery by 0.25% uncompressed, 0.37%
with gzip and 0.45% with brotli.
Issue: https://github.com/dart-lang/sdk/issues/40488
TEST=Existing test suite, expectation files for TFA test, extended test
of protobuf-aware tree shaker.
Change-Id: Ib00fe362f798c92592a260f72e7bccad04246bb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173727
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is in preparation to migrate package:kernel to null safety.
For the visitor interfaces to support non-nullable return types, the
implementations must avoid using `null` as return value in its base case.
TEST=Refactoring
Change-Id: I9f9b318982148d844be9826a5f8c88374a9fc402
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172180
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
The existing signature shaker transforms optional and named parameters
of static and top-level methods and constructors into required
parameters whenever they are always passed.
This change improves signature shaking in a number of areas:
- Extend signature shaking to cover instance methods.
- Remove parameters which are never passed.
- Remove parameters which are not used.
- Transform signatures before unboxing, enabling transformed parameters
to be unboxed.
TFA summaries for transformed functions are updated to stay in sync with
the program for later transformations.
Reduces instructions size of Flutter Gallery on ARM64 by 2.2%.
Issue: https://github.com/dart-lang/sdk/issues/40488
Change-Id: I39e364bc23b01e4502864d7369a745a9278cfb92
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161175
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This change introduces handling of protobufs while doing type flow
analysis. Metadata in protobuf message classes is updated dynamically
according to the set of called accessors, invalidating and rebuilding
TFA summaries as needed.
Previously, protobuf-aware tree shaker required the 2nd run of TFA
in order to do the actual tree-shaking after protobuf messages are
pruned. This significantly increases compilation time.
AOT compilation time of a large app (--from-dill): 274s -> 152s
New tree shaker is available in kernel compilers under the flag
--protobuf-tree-shaker-v2.
Issue https://github.com/dart-lang/sdk/issues/42442
Fixes https://github.com/dart-lang/sdk/issues/40785
Change-Id: I4347896737b9b0f7407b845e614dda9ba7621921
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
The partitioning has the property that for every reference from loading unit S to unit T, either
- the reference is through a deferred prefix, or
- T is guarenteed to be loaded before S
The partitioning is constructed by cutting the dominator tree of the library import graph at deferred imports. It is suboptimal if there are deferred imports to multiple libraries in a connected component, but the expected use case involves loading units with a single deferred entry.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I3c49044ae19112525f197b077b36e4ce874b81ac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151470
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL adds a Node.toText method together with an AstPrinter. These
facility and better toString implementation on AST nodes while allowing
for toString independent printing of AST to use in testing. This also
add support for an integrated toString of custom/internal nodes.
Some work is still needed in bringing the toString implementation on
all nodes to the old quality, and not all internal nodes have
customized textual representations yet. This work is left for future
CLs.
Change-Id: Ib0bf8a0bc02f489dfacdc8aa5f96da9c52f26058
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150923
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
When performing protobuf-aware treeshaking, we may generate a kernel
file which will later be used in conjunction with a non-transformed
platform kernel. In this case signature shaking is unsafe.
The original revision is in patchset 1.
This reverts commit b59dad38df.
Change-Id: I1a85e42f245fcdbd821fa1f07b2e9966b8c2d958
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139100
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
So far tree shaking was removing fields which are not used at all.
This change improves tree shaking of fields so fields
which are only written or used as interface targets can be removed.
The following limitations apply:
* Field is not removed if there is a constant object with that field, as
it may impact identity of constant objects which is an observable
behavior.
* Field is not removed if it has a non-trivial initializer as it may
have side-effects.
* Late final fields are not removed, as writing such fields may have
side-effect.
* When field is removed, we may need to introduce an abstract getter
or abstract setter if field is used as a target of an interface call.
If a field was written, then setter would be non-abstract (but empty).
Fixes https://github.com/dart-lang/sdk/issues/35310
Change-Id: I79c00158b8eb658081a647c5dbdecde481fddb41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134204
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
New class representation is used in ConcreteType and ConeType instead of
DartType. Uses of pre-NNBD legacy raw types are eliminated in most cases.
This also simplifies subtype checks.
TypesBuilder class is created to translate static types into
TFA types. It holds CoreTypes which is used to recognize Null class
and supersedes Type.cone and Type.fromStatic factory constructors.
Change-Id: Ia1cd48cf8c738173a06dd04864f6cec8ae6def83
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124286
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is a reland of 56991ad94b
Original change's description:
> [vm,aot] Simple unreachable code elimination before type-flow analysis, take 2
>
> This is the relanding of https://dart-review.googlesource.com/c/sdk/+/121500
> with fixes.
>
> Original CL description:
>
> This change adds transformation for very early cleanup of unreachable
> code such as code guarded by if statements with constant conditions or
> code used in assert statements when assertions are disabled.
>
> The advantage of cleaning such code early is that type-flow analysis
> won't be looking at it and TFA-based tree shaker is able to remove
> more code.
>
> flutter_gallery_total_size -0.5663% (arm), -0.5409% (arm64)
> build_bench_total_size -2.533% (arm), -2.449% (arm64)
> gesture_detector_total_size -4.183% (arm), -4.072% (arm64)
>
> Change-Id: Ief8ebd0cd828c0e7a847a824f44d2d97a3595b87
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121901
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
Change-Id: I82ea0f70845851f85ec159c366f408aa7f9e5788
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122166
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit 56991ad94b.
Reason for revert: broke building Flutter apps in release mode
Original change's description:
> [vm,aot] Simple unreachable code elimination before type-flow analysis, take 2
>
> This is the relanding of https://dart-review.googlesource.com/c/sdk/+/121500
> with fixes.
>
> Original CL description:
>
> This change adds transformation for very early cleanup of unreachable
> code such as code guarded by if statements with constant conditions or
> code used in assert statements when assertions are disabled.
>
> The advantage of cleaning such code early is that type-flow analysis
> won't be looking at it and TFA-based tree shaker is able to remove
> more code.
>
> flutter_gallery_total_size -0.5663% (arm), -0.5409% (arm64)
> build_bench_total_size -2.533% (arm), -2.449% (arm64)
> gesture_detector_total_size -4.183% (arm), -4.072% (arm64)
>
> Change-Id: Ief8ebd0cd828c0e7a847a824f44d2d97a3595b87
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121901
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
TBR=vegorov@google.com,kustermann@google.com,rmacnak@google.com,alexmarkov@google.com
Change-Id: I25fe5d4457bfcc5ee25506810aa7eee01ca1b28a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122023
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is the relanding of https://dart-review.googlesource.com/c/sdk/+/121500
with fixes.
Original CL description:
This change adds transformation for very early cleanup of unreachable
code such as code guarded by if statements with constant conditions or
code used in assert statements when assertions are disabled.
The advantage of cleaning such code early is that type-flow analysis
won't be looking at it and TFA-based tree shaker is able to remove
more code.
flutter_gallery_total_size -0.5663% (arm), -0.5409% (arm64)
build_bench_total_size -2.533% (arm), -2.449% (arm64)
gesture_detector_total_size -4.183% (arm), -4.072% (arm64)
Change-Id: Ief8ebd0cd828c0e7a847a824f44d2d97a3595b87
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121901
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit 41b65b27c2.
Reason for revert: broke protobuf-aot benchmarks.
Original change's description:
> [vm,aot] Simple unreachable code elimination before type-flow analysis
>
> This change adds transformation for very early cleanup of unreachable
> code such as code guarded by if statements with constant conditions or
> code used in assert statements when assertions are disabled.
>
> The advantage of cleaning such code early is that type-flow analysis
> won't be looking at it and TFA-based tree shaker is able to remove
> more code.
>
> flutter_gallery_total_size -0.5663% (arm), -0.5409% (arm64)
> build_bench_total_size -2.533% (arm), -2.449% (arm64)
> gesture_detector_total_size -4.183% (arm), -4.072% (arm64)
>
> Change-Id: Ic764c056b6594164dc32eed4fd5679a7077ea2f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121500
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=vegorov@google.com,kustermann@google.com,rmacnak@google.com,alexmarkov@google.com
Change-Id: I3e5668176a7a90d2891126bf79139ac5859bc630
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121824
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change adds transformation for very early cleanup of unreachable
code such as code guarded by if statements with constant conditions or
code used in assert statements when assertions are disabled.
The advantage of cleaning such code early is that type-flow analysis
won't be looking at it and TFA-based tree shaker is able to remove
more code.
flutter_gallery_total_size -0.5663% (arm), -0.5409% (arm64)
build_bench_total_size -2.533% (arm), -2.449% (arm64)
gesture_detector_total_size -4.183% (arm), -4.072% (arm64)
Change-Id: Ic764c056b6594164dc32eed4fd5679a7077ea2f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121500
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Normally, TFA runs after constant evaluation and it expects @pragma
annotations in the evaluated form. In unit tests TFA runs after
front-end and it was expecting @pragma annotations in the expression
form. With constants-update-2018 front-end starts evaluating constants,
so TFA unit tests should switch from ExpressionPragmaAnnotationParser
to ConstantPragmaAnnotationParser.
This CL does not update test expectations as flag is not flipped yet.
Test expectations should be updated when flipping a flag using
tools/test.py -m release --vm-options -DupdateExpectations=true pkg/vm
Change-Id: I94ee2de9f70328ce3ed3ffd9902a92eeb26d3a29
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108840
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Pkg/vm tests were adjusted after constant evaluation was enabled in the front-end.
As constant evaluation was reverted (4b7d256a06),
we also need to revert corresponding changes in pkg/vm unit tests.
Revert "[vm/bytecode] Update expectations for bytecode generator unit test"
This reverts commit 734c50cbf2.
Revert "[vm/aot/tfa] Update TFA tests after front-end starts evaluating constants"
This reverts commit 25cb45c02f.
Change-Id: I2fcc30571bec2dd904925724ddb7105567c7ecda
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99782
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Usually TFA takes kernel AST with all constants evaluated.
However, in order to avoid calling constant evaluator pass in TFA unit tests
there is a special implementation of pragma annotation parser (for testing)
which accepts annotations which are not ConstantExpressions.
After CFE starts evaluating all constants (constant-update-2018 was enabled in
e9ca6a5335), TFA unit tests need be
switched to use ConstantPragmaAnnotationParser.
Also, empty environment defines are passed to CFE to avoid unevaluated constants
and unit test expectations are updated.
Closes https://github.com/dart-lang/sdk/issues/36602
Change-Id: I2e7b0647e065cca129449b4531c6278c1d98061d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99269
Reviewed-by: Samir Jindel <sjindel@google.com>
Validating the class hierarchy requires subtype tests. Previously,
Fasta has worked around this by computing the hierarchy twice, as
well as visiting the hierarchy several times. In order to improve
performance of modular compilation, we're working on changing this.
To enable creating a TypeEnvironment without a hierarchy, we're
removing the field from the API.
Change-Id: I7c9f88d5c6495aaa8223eaeff7d2dfd5669e9e95
Reviewed-on: https://dart-review.googlesource.com/c/88823
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Also allow using the unchecked entry-point for invocations of generic functions where
there are no bounds or the bounds don't require dynamic checks.
Change-Id: I6ca1ebec777ecf2989c4fb77425d65d542d5adf2
Cq-Include-Trybots: luci.dart.try:vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-mac-release-simarm64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/87181
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Johnni Winther