We reconstruct ProgramInfo tree structure from the snapshot
by applying a simple graph algorithm, which takes parts of the
snapshot with clear ownership (e.g. library, class, function objects)
and then attempts to attribute the rest of the snapshot to these
clearly owned objects based on reachability.
Issue https://github.com/dart-lang/sdk/issues/41249
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: I17c0f8323ee9092a2214b18bd948ff51fa2ccc49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151384
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The partitioning has the property that for every reference from loading unit S to unit T, either
- the reference is through a deferred prefix, or
- T is guarenteed to be loaded before S
The partitioning is constructed by cutting the dominator tree of the library import graph at deferred imports. It is suboptimal if there are deferred imports to multiple libraries in a connected component, but the expected use case involves loading units with a single deferred entry.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I3c49044ae19112525f197b077b36e4ce874b81ac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151470
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL also refactors ProgramInfo making it less structured - we remove
individual node types (LibraryInfo/FunctionInfo) and represent
the information using ProgramInfoNode class instead.
Issue https://github.com/dart-lang/sdk/issues/41249
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: I0156ae75f48a41b7689790860267ac88706d55c0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151227
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL introduces dart_api_dl.h which exposes a subset of dart_api.h
and dart_native_api.h for dynamic linking at runtime through the FFI.
Dynamic linking is done through including dart_api_dl.cc in a shared
library and passing NativeApi.initializeApiDLData to the init function.
This CL also includes Native API versioning to deal with possible
version skew between native api version against which native libraries
are compiled and the version in the DartVM the code is run on.
The subset of symbols in the CL includes handle related symbols, error
related symbols, handle scope symbols, and native port sumbols.
Design: http://go/dart-ffi-expose-dart-api
Closes: https://github.com/dart-lang/sdk/issues/40607
Closes: https://github.com/dart-lang/sdk/issues/36858
Closes: https://github.com/dart-lang/sdk/issues/41319
Closes: https://github.com/flutter/flutter/issues/46887
Closes: https://github.com/flutter/flutter/issues/47061
Misc:
Closes: https://github.com/dart-lang/sdk/issues/42260
Change-Id: I9e557808dbc99b341f23964cbddbb05f26d7a6c5
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145592
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Disassembling on every run makes it difficult to get an idea of which
specific configuration failed. Instead, just pick a configuration that
already succeeded and test that configuration with disassembly
separately so that it's clear if the disassembly was what caused the
failure.
Also add reasons to Expect.isTrue/isFalse uses in
package:vm/v8_snapshot_profile.dart. Now we don't have to track down the
specific line to find out what failed when reading logs. Another benefit
is that the error message now includes the values involved in the
failure and not just "Expect.isTrue(false)" or "Expect.isFalse(true)".
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try
Change-Id: I3d4b533f4df041dfcced4eaeb8bc5729dfad0b82
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151502
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This CL adds a Node.toText method together with an AstPrinter. These
facility and better toString implementation on AST nodes while allowing
for toString independent printing of AST to use in testing. This also
add support for an integrated toString of custom/internal nodes.
Some work is still needed in bringing the toString implementation on
all nodes to the old quality, and not all internal nodes have
customized textual representations yet. This work is left for future
CLs.
Change-Id: Ib0bf8a0bc02f489dfacdc8aa5f96da9c52f26058
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150923
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
AOT transformations (mixin deduplication and TFA tree shaker) were
using CanonicalName.unbind() in order to make sure that kernel
writer will not be able to write dangling references to the deleted
classes or members (to detect such dangling references earlier).
This conflicts with lazy reading from a kernel file if --from-dill
option is used: lazy reader may read a reference to such class after
unbind() and it creates a new Reference object for such canonical name.
This causes crash later when the fresh Reference is used as it doesn't
have Class node filled in.
The solution is to avoid calling CanonicalName.unbind().
Instead, it is enough to nullify node.reference.canonicalName, which
won't break CanonicalName.reference used by lazy reader.
This crash can be reproduced on tests/language_2/vm/regress_flutter_55345_test.dart
if it is compiled in AOT mode using 2-step kernel generation.
Bug: b/158853113
Change-Id: Ib2004dbbbda85d9f56adc56b48882f4ef08869a7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151120
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Introduce helper classes and methods for constructing program
hierarchy from a flat list of symbols as well as methods for
diffing and bucketing the program hierarchies.
Issue https://github.com/dart-lang/sdk/issues/41249
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: Ieb6ad1ccd8bc7fa1181d31e78ac351369ede5f81
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150101
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This change adds options
--manifest
--component-name
--data-dir
to gen_kernel tool.
After that, gen_kernel becomes a drop-in replacement for Fuchsia's
kernel compiler.
Change-Id: I8370b4224d1fc1973c8a850ea1e46defdb2cbac4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150690
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This option can be used to split AOT kernel generation into two rounds:
* The 1st round uses CFE to parse source files and generate intermediate
dill file in *non-AOT* mode.
* The 2nd round loads intermediate dill file and applies AOT global
transformations, then generates final dill file.
Each round is faster than one-shot generation of a dill file in AOT mode.
In order to use this 2-round compilation, command
gen_kernel <other-options> --aot -o output.dill main.dart
should be replaced with the following 2 commands:
gen_kernel <other-options> --no-aot -o intermediate.dill main.dart
gen_kernel <other-options> --aot --from-dill intermediate.dill -o output.dill main.dart
Bug: b/158101962
Change-Id: I2d4deef45b50eab11df47f4a372598e6cf1d0edf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150272
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The first compilation result is leaked in two ways:
1) Directly by saving the component in a variable; and
2) Via an unfortunate context thing, probably a variation of
http://dartbug.com/36983. I will update that bug with a reproduction
example later.
The reason this creates a (big) leak is illustrated with an example:
* Say the first component (A) has 10 libraries in it. Each of these
libraries has parent pointers and points (currently) to A, which again
points to all of the 10 libraries.
* We then do a recompilation, say 5 libraries are reused and 5 are new.
They are put into a component (B). We really should have 10 libraries,
the 5 old ones and the 5 new ones (and for simplicity lets say these are
the ones in B). Notice that the 5 old ones will have their parent
pointers updated and also still be in the list of libraries in A.
We keep 15 libraries alive because we have the 10 original ones saved via
A and the 10 (where 5 is new) saved via B.
* We then do a recompilation, say 2 of the same libraries as was also
recompiled before, these end up in compnent C which has 5 libraries from A,
3 libraries from B and the 2 new ones. All of these libraries will have
their parent pointers updated to point to C.
* Because we saved A we keep all the 10 libraries in A though.
Because we saved A and some of the libraries in A had parent pointers
updated to point to B we also keep B and all libraries in B.
Because we keep B and some of the libraries in B had parent pointers
updated to point to C we also keep C and all libraries in C.
So instead of only having the 10 "live" libraries, we have 10 + 5 + 2 = 17
libraries, a leak of 7. With more compilations this keeps happening and
the leak keeps growing.
This CL stops the leak by not holding on to A (which, in turn, stops holding
on to B etc.)
Change-Id: If4f8b1e240b7c39f084df9cb2690570ff26fa9b3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149280
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
The functionality of these two separate tools will now be available as
$ pkg/vm/bin/snapshot_analysis.dart treemap ...
$ pkg/vm/bin/snapshot_analysis.dart compare ...
respectively.
This is first step towards making these tools share more infrastructure
and also work with --print-instructions-sizes-to and
--write-v8-snapshot-profile-to outputs.
Change-Id: I16a2d7d51680397ce22b931c00af6eeda446e634
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148420
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Add a kernel transformation which removes redundant casts and reduces
some casts to null checks. This transformation assumes that type
system is sound and subtyping relation is transitive. So in NNBD weak
mode this transformation is always correct. In NNBD strong mode
this transformation is correct only if opted-out libraries are not
allowed.
The following patterns are handled:
If S <: T (this includes S <: T? in weak mode)
S x; x as T => x
If S <: T? in strong mode
S x; x as T => (x == null) ? x as T : x
This kernel transformation is not specific to the VM and can be reused
by other back-ends such as dart2js and DDC.
Fixes https://github.com/dart-lang/sdk/issues/40892
Issue https://github.com/dart-lang/sdk/issues/40893
Change-Id: I466ba96d763d5358011c768400c5225fa7bf5e67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139314
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This enables adding the `external` keyword to struct fields, which
enables use in nnbd (weak mode).
Closes: https://github.com/dart-lang/sdk/issues/40247
External fields are not implemented in the analyzer yet:
https://github.com/dart-lang/sdk/issues/41940
Change-Id: I9d88acbdabf73ca63a6ad3d549930aa3c97cb53f
Cq-Include-Trybots: luci.dart.try: analyzer-nnbd-linux-release-try,dart2js-nnbd-linux-x64-chrome-try,ddc-nnbd-linux-release-chrome-try,front-end-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148242
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
For some types it makes sense to distinguish between nullability as a
property of including null and the declared nullability, such as for
type-parameter types and FutureOr types. This CL introduces the
separation between the two concepts. The plan is to use that when
implementing FutureOrType.
Bug: https://github.com/dart-lang/sdk/issues/40123
Change-Id: I90b45bea768e89c213d021d360a481d38377c682
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147903
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
When building scopes, VM looks for variables using their names in the
current scope. If signature tree shaking is applied multiple times,
then we can end up with multiple #argN variables with the same names
in the same scope (in constructor initializer list). After that,
VM crashes when using the incorrect variable.
This change removes names from temporary variables introduced by
signature tree shaking in order to avoid these conflicts.
Currently, TFA (and signature tree shaking) is applied multiple times
only when protobuf-aware tree shaking is enabled, and this bug doesn't
appear without protobuf-aware tree shaking.
Bug: b/156025352
Change-Id: Icc4e39a22711cecc0b4399acdd266e2ca2276a3d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147620
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Previously the output used scrubbed qualified names which hides some
of the information and causes names of various synthetic functions
collide. We change it to use internal qualified names instead and add
additional information for disambiguation:
- For functions this means adding a prefix for various synthetic
methods;
- For anonymous closures we add their token position;
- For types we prefix type parameters with their owner's name.
We also add a unit test covering parsing of
--print-instructions-sizes-to output.
As part of adding a test dart2native was changed to support passing
extra arguments to gen_snapshot binary.
Fixes https://github.com/dart-lang/sdk/issues/41137
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: I299ab0fb5993c8af9b51246483608d1e65970d67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146581
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Removes the Closure.clone() in sync* by adding an additional closure
around the rewritten sync* function body.
This is a re-worked version of the now reverted:
- https://dart-review.googlesource.com/c/sdk/+/136190
- https://dart-review.googlesource.com/c/sdk/+/144948
This CL includes changes from both the above CLs, in addition to:
- A new 'ShadowRewriter' transformer to do deep rewriting of
shadowed parameters.
Previous versions only rewrote the immediate sync* function, missing
references in nested rewritten functions.
Note: This CL is re-landing largely unchanged since the roll-blocking
issues suspected to be caused by this was later attributed to a
different set of changes.
I have independently re-run all failing external tests to verify
them passing for this CL.
Bug: https://github.com/dart-lang/sdk/issues/37753
Change-Id: Id1670a93961180e8558d7c7eca65fe7e9115f07b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145402
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
CFE may use anonymous mixin application classes in types after
applying LUB (Least Upper Bound) algorithm in conditional expressions,
and then those types can spread via type inference.
After anonymous mixin application classes are deduplicated, all
references to removed mixin applications from types should be
updated to point to the remaining mixin application.
Fixes https://github.com/flutter/flutter/issues/55345
Change-Id: I7860b2868cb153d125fec363c1653b7aa16521e6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145501
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead, this CL wraps sync*'s sync_op in an additional function
that captures local copies of the necessary closure state.
Notably this state includes the parameters of the original sync* function
as these also need to be cloned for each instance.
Note: a large number of .expect files have a small number of changes
due to the additional wrapper function, as well as some extra plumbing.
Bug: https://github.com/dart-lang/sdk/issues/37753
Change-Id: If099dbfe9e5d62979626b852386fd0173fc76de5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136190
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Async transformation sometimes inserts additional unsafeCast calls in
order to make sure kernel AST is typed.
We can recognize these calls in TFA and handle them more accurately,
maintaining type flow through these native (and otherwise opaque) calls.
Change-Id: I3e316507e0a37f36dcf8a9621dbd7b342ce5f19d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138661
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Adds another interface target to instance calls, pointing to the torn
off method for tearoff getter calls. This is then used to direct the
call to the method extractor for the method.
Increases the fraction of dynamically dispatched instance calls in
Flutter Gallery that are handled by the dispatch table from 95.8% to
98.4%. The remaining 92 IC calls are all on static receiver type
dynamic or Function (34 function calls, 22 method calls, 21 operator
calls, 14 getter calls and 1 setter call).
Change-Id: I0bde1a8271c8563f4631534bd39d1254cd779380
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142155
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This relands commit ff34fd8110,
but puts tree shaking of write-only fields under the flag which is
disabled by default.
So far tree shaking was removing fields which are not used at all.
This change improves tree shaking of fields so fields
which are only written or used as interface targets can be removed.
The following limitations apply:
* Field is not removed if there is a constant object with that field, as
it may impact identity of constant objects which is an observable
behavior.
* Instance field is not removed if it has a non-trivial initializer as
it may have side-effects when executed by constructors.
* Late final fields are not removed, as writing such fields may have
side-effect.
* When field is removed, we may need to introduce an abstract getter
or abstract setter if field is used as a target of an interface call.
If a field was written, then setter would be non-abstract (but empty).
Issue https://github.com/dart-lang/sdk/issues/35310
Change-Id: Iec75b8301892664f4f955a01e5960b17e6620531
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143286
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This change contains the following fixes for NNBD strong mode in TFA:
* TypesBuilder.fromStaticType now correctly applies nullability of
TypeParameterType even if bound is non-nullable.
* UnknownType is introduced to represent a type argument not known at
compile time. Previously AnyType was used for that purpose, and it
could pass isSubtypeOfRuntimeType check for non-nullable Object type.
Fixes https://github.com/dart-lang/sdk/issues/41452
Change-Id: Ie836c99b38ad288aa68d9aae48e50844f9f949d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143310
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
New option can be used instead of --aot to produce a tree-shaken
minimal .dill file which can run on the VM in JIT mode.
Can be used along with --no-link-platform and --gen-bytecode to further
decrease size of the .dill file.
Change-Id: I35e8d81263a5e223decf160d79322c06cfd27321
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140740
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Vyacheslav Egorov