Some IR instructions (e.g. RawStoreField) can never deopt by-design. Some
instructions are calling runtime entries but those are leaf calls (e.g.
InvokeMathCFunction). Some are synthetic IR instructions that never
actually emit code (e.g. Constraint). Some IR instructions cannot deopt
due to inputs (e.g. Boxing instructions whose value is guaranteed to fit
into Smi).
Part of https://dart-review.googlesource.com/c/sdk/+/190481
TEST=Existing test suite.
Change-Id: I770b7d094ef36221d357157c736710476fd6bfb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190482
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Also ensure the number of parent type arguments is appropriately
adjusted in the returned FunctionType when partially instantiating
a FunctionType, and that free parent type arguments are _not_
included in types built while reading constants.
Since the number of parent type arguments is now cached there, we
no longer need to also cache them in ClosureData. We can also remove
the parent walking in Function::NumParentTypeArguments().
Previously, a FunctionType where the component types did not use
parent type parameters was considered instantiated. This CL changes
it so that FunctionType with free parent type parameters are never
considered instantiated. This is necessary because otherwise,
when instantiating the parent type parameters, a FunctionType that
does not use its parent type parameters will be used unchanged
instead of creating a copy with fewer parent type parameters.
Because of this, places where IsInstantiated was used to simply
check for generic parent functions (namely, for implicit closure
creation) has been appropriately weakened to check
!HasGenericParent() instead.
TEST=Existing tests on CI.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Ifb4a0a1273d8d01908cdf4ffc3c4c28a1c33ffa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190021
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
We originally sized the compressed heap at 2GB to avoid implementing alignment offsets. (When decompression is sign-extension + base, base should be the middle of a 4GB-aligned region.) After 8db0a975c1, the needed alignment offset is zero, and we no longer need to avoid the high bit being set.
TEST=ci
Change-Id: Ib6e6c40bcb9638e01867edd527b3acaaccec570b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191321
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
This CL adds code to detect situations when we hit
null error through GDT call with non-null receiver.
We achieve this by making receiver's cid part of the
GDT calling convention and checking this cid
in runtime entry responsible for throwing the null
error.
This CL is an attempt to narrow down b/179632636
and collapse various impossible crash reports
into a single native crash cluster by crashing VM
instead of throwing a null error.
TEST=tested manually
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: If2ed4646c4c0f403016266e4e83e296a7b234cb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191141
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead of being part of the --trace-precompiler output, there is
is a new flag --write-retained-reasons-to=<path>.
To allow for non-functions to have reasons for retention, the
JSON objects in the array now have one additional field:
* type: a string, the C++ type of the Dart object.
This CL also wraps all the machinery in a RetainedReasonsWriter
class instead of putting it in the Precompiler interface.
TEST=vm/dart{,2}/use_trace_precompiler_flag
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Ifa40ac64fdcbdad5114499425c7f8b08e327fa76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190525
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
There are two fixes:
1) If a function was compiled but not retained, precompiler
should still trace through types used in exception handlers.
2) Certain auxiliary functions such as implicit closure functions
and field initializers were added to 'possibly_retained_functions_'
set but were not traced through AddTypesOf and were not added to
'functions_to_retain_' set.
Flutter gallery AOT snapshot size
release - no changes
release-sizeopt arm +0.13%, arm64 +0.14%
TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/45179
Issue https://github.com/dart-lang/sdk/issues/44852
Change-Id: Ide9a7e82a7bb9098dc90bc41a76e527e7a5cb603
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190781
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
More specifically on 64-bit platforms, when other heuristic fail:
- in AOT mode unbox all int typed phis.
- in JIT mode unbox all non-Smi int typed phis i.e. mixed
Mint/Smi phis.
This brings treatment of int phis closer to how we treat other
boxed phis.
This considerably improves AOT code quality for loops which
our current conservative heuristic does not currently hit,
for example:
void foo(int start, int end) {
for (var i = start; i < end; i++) {
}
}
Code like this would use a boxed Phi previously which does not make
sense.
In JIT mode to really benefit from aggressive unboxing we
might need to shift from over-reliance on Smi operations, e.g.
when phi is unboxed we want to cascade that through and unbox
operations which are performed on it transitively. This requires
much more sophisticated algorithm and is outside of scope for this
change.
In AOT we in general treat all arithmetic operations as operations
over 64-bit int values so unboxing phis makes more sense.
As a result of this change we see small improvement in code size:
flutter_gallery_app_so_gzip_size -0.2392%
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Change-Id: Iae9a16f353e53f649da4e665d2812ad47099a674
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187860
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Allows removing a branch when decompressing pointers and removing fixup sign-extension after Smi ops that use 32-bit ops to detect overflow. Increases the code size of indexed and stores for some widths due to addressing mode limitations.
TEST=ci
Change-Id: Ia48353cccbbc586dd0d2e055a843c65e37c63a30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185660
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
This is a follow-up to https://dart-review.googlesource.com/c/sdk/+/185381.
That change introduced a more compact serialization of canonical sets,
but it didn't include correct counting of heap size of tables
allocated during deserialization of canonical sets.
This change adds counting of heap size of canonical tables.
The heap size benchmark numbers will regress with this change,
but this is only due to a more accurate counting, not due to a real
regression.
TEST=ci
Change-Id: I42ee3bd91f0420bf41c660d2eaa51e9901d9b198
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190580
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Unfortunately, we never capitalized on having it (e.g. original plan was
to use it for writing tests, among other things) and continuing to
maintain it does not make sense.
TEST=ci
Change-Id: I02b2c35149298a295aa0de75916d48434d4a3ea9
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190220
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
To get this information, enable --trace-precompiler.
There are two version of the output: a human readable version
and a JSON version (preceded by 'JSON for function decisions:')
for easier tabulation. The JSON version is an array of objects,
where the fields of the object are as follows:
* name: a string, the function name
* kind: a string, the function kind
* retained: a boolean, whether the function was retained
* reasons: an array of strings, reasons for retaining
(only for retained objects)
TEST=vm/dart{,_2}/use_trace_precompiler_flag_test.dart
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I9b006a49e1410a832950a1bd29a0abaabe9431f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190000
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
AssertBoolean may assume its input is nullable bool. BooleanNegate may assume its input is non-nullable bool. Use fixed alignment bits to distinguish true, false and null.
TEST=ci
Change-Id: Ife694e66c1225d943881c2325cd67f18575a47e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189240
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The deoptimization environment of AssertAssignable should not have it's
inputs in it, since those get poped before the call to the
TypeTestingStub.
A lazy deopt should therefore continue in unoptimized code after the TTS
call, without the inputs.
This was discovered by running many lightweight isolates concurrently on
the JIT.
To simplify a regression test we'll add new VM flags that can trigger
deoptimization on any runtime call (optionally filtered by name of
runtime call).
We can use this mechanism to possibly discover more bugs of the same
kind.
Closes https://github.com/dart-lang/sdk/issues/45207
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=runtime/tests/vm/dart{,_2}/regress_45207_test
Change-Id: Ib713fa8be5914eaf4c7395d2e1d8dd88714f1ce2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189202
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This removes the @fields and @=fields canonical name
encodings that allowed for encoding of conflicting members
and didn't support field<->getter/setter conversion
between dills or between outline and full dill.
TEST=existing tests+add aot expectation tests
Change-Id: I119b0c95f90e456356146cdc2d9241de4c1b4fff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186680
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
This CL changes how canonical sets for some specific types are written
into the root snapshot: instead of writing canonical set as a separate
object we reorder objects within a canonical cluster in such a way that
the order matches order of elements in the backing store of a canonical
set and then we write canonical set layout out using differential
encoding (essentially writing gaps between elements instead of writing
absolute indices).
This significantly reduces the overhead of having canonical sets in the
snapshot while maintaining fast deserialisation: for example on build
microbenchmark this brings regression in the snapshot size from 4% to
.3%.
On sizeopt benchmarks:
flutter_gallery_app_so_gzip_size -1.5%
flutter_gallery_app_so_size -4.7%
flutter_gallery_total_heap_size -16.2%
TEST=ci
Change-Id: I2be7fd073668e9b52098e2acda9f11d128cfda95
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185381
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
* Add InstanceGetterInvocation for getter/field invocation in web
backends
* Add localFunction getter to LocalFunctionInvocation
* Remove isNot from EqualsCall and EqualsNull - the encoding didn't
carry its weight.
* Remove uses of Name.name
* Remove BottomType code from VM
TEST=existing
Change-Id: I99d05d35b9ef193d092cc151c99ad472dbd60834
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188725
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
The lazily calculated value can be calculated concurrently and installed
in arbitratry order as long as all writers write the same value (which
they should).
There's no need to guard it with a lock.
Closes https://github.com/dart-lang/sdk/issues/45175
TEST=Refactoring of existing code, fixes flaky tsan reports.
Change-Id: Iaa45cf3b70300958acc04953e37599836436bc59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188720
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
To avoid a O(n) traversal of isolates in order to find pending deopts,
we'll move the pending deopt state to the Thread. It's really a property
of a thread: when the thread's stack gets unwound - via return or
exceptional flow - we remove pending deopts.
This CL also encapsulates the pending deopt logic into a [PendingDeopts]
class - thereby avoiding logic around manipulating this array being
spread around. It simplifies the already very big [Isolate] class.
TEST=Refactoring of existing code.
Issue https://github.com/dart-lang/sdk/issues/36097
Closes https://github.com/dart-lang/sdk/issues/45168
Change-Id: I5a9b18c577f21a0f2a88332b1bc26641ec39d2af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188521
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This is the initial implementation of hot reload with multi-isolate
groups.
Implementation:
As before, when a service API call triggers a reload it will be routed
as an OOB message to a specific isolate (**). As opposed to before, that
isolate has now to coordinate with all other isolates, ensuring that it
"owns" the reload and all other isolates are waiting in a state that
allows reload.
This is implemented as a [ReloadOperationScope] which first participates
in other reloads (if there are any) and then owns the reload. It will
send a new kind of service message to all other registered isolates. All
of them have to check in before reload can proceed. If a new isolate
is about to join the group, it will participate when registering the
isolate. If an old isolate wants to die, it will participate when
unregistering the isolate.
This means that in addition to the existing StackOverFlow checks that
can process OOB messages and therefore reload, we'll have isolate
registration and unregistration as well as a new
Isolate::kCheckForReload OOB message handler where an isolate can
participate in a reload.
We consider the isolate group to be reloadable if the main isolate has
loaded the program and set the root library. Helper isolates don't need
to load any more kernel code and only initialize core libraries, so it's
fine to reload them during this time.
(**) The reason we continue to send reload service API calls to any
isolate in an isolate group is that re-loading might involve calling out
to the embedder's tag handler. Doing so currently requires an active
isolate.
If we allowed a subset of dart_api.h (the subset needed by the tag
handler) to be used only with an active IsolateGroup instead of an
active Isolate we could remove this requirement.
Edge cases:
There's various edge cases to consider: The main edge case is, we currently
maintain an upper limit to the number of isolates executing in parallel
(to ensure each can have big enough chunk of new space, i.e. TLAB).
If there are more isolates with active work they are waiting until one
of the exiting ones "yields". To ensure progress, if any such actively
running isolate gets a request to participate in a reload, it will mark
its own thread as "blocked" and therefore "yields", so another isolate
can make progress until all isolates are participating and the reload
can start.
Marking an isolate as "blocked" happens by exiting that isolate. It will
free up it's TLAB, decrease active mutator count and (if running on VM's
thread pool) also temporarily increase the thread pool size.
The side-effect of this is that it will use one pthread per isolate
during reload. In the future we can extend this first implementation, by
specially handling isolates that don't have a message handler running.
Doing so would require careful consideration to avoid races.
Testing:
In order to test this we use a small helper framework for reload tests.
The helper framework will, similar to real world reload e.g. in flutter,
will spawn a subprocess. It will use the service API to trigger reloads
in this subproces.
To synchronize between the reload driver and the application being
reloaded it allows watching for events to be printed to stdout/stderr.
The reload test itself can be written - similar to multitests - with
annotations such as `// @include-in-relload-0` in them. The testing
framework will then generate multiple application versions that all get
compiled to kernel.
For simplicity we generate the kernel using the standalone VM with
`--snapshot-kind=kernel` and avoid using the incremental compiler.
There are 4 different tests exercising different aspects of
multi-isolate reload:
vm/dart_2/isolates/reload_active_stack_test:
Performs a reload while a fixed number of isolates have an active
stack, thereby ensuring e.g. that all frames of all isolate mutator
stacks get deoptimized, ...
vm/dart_2/isolates/reload_no_active_stack_test:
Similar to the test above, but instead of having an active stack the
isolates can yield to the event loop, possibly be even descheduled
vm/dart_2/isolates/reload_many_isolates_test:
Similar to the test above, but this test uses many more isolates.
vm/dart_2/isolates/reload_many_isolates_live_and_die_test:
Performs a reload where isolates get spawned and die all the time.
There are always P isolates alive at any given point in time, each
of them spawns children when their parent has died.
Performing a reload catches isolates as various stages of their
lifecycle and can therefore cover a lot of corner cases.
TEST=vm/dart_2/isolates/reload_*_test.dart
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I97039b4084de040b7f2e22f5832a40d57ba398d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
The kernel loader works without an active isolate. It no longer causes
execution of Dart code. Constants are evaluated in CFE and are simply
"deserialized" by the kernel loader.
Since kernel loader doesn't execute Dart code and also doesn't process
OOB messages at any point, there's no reason to have a [NoReloadScope]
in it.
Formatting language errors seems to be done purely in the VM and should
therefore also require no [NoReloadScope]
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing code.
Change-Id: I80b2888b60a4d7d179988b8606cd7ca0893a2fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187400
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
For now our decision is to not synchronize access to usage counters and
instead let there be possible non-observable writes (if two concurrent writes
happen).
Though TSAN will report concurrent access in C++ to the usage counters
as a race, so we'll use a relaxed atomic.
(We do have non-synchronized access to the usage counters already:
If the mutator kicks of a background compilation, the BG compiler
might reset the counters while mutator is running.)
Closes https://github.com/dart-lang/sdk/issues/45134
TEST=Fixes flaky tsan errors.
Change-Id: I7919747536d43094e7201e03f464e4ac7b1a4740
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187923
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Martin Kustermann