Generally speaking, the optimizing compiler should be able to use
(deopt-id, deopt-environ) of any IR instruction and use it as an eager
deopt target (e.g. optimizing compiler might insert TestCidsInstr
with eager (deopt-id, deopt-environ) from AssertAssignable).
Currently we prune the environment eagerly during SSA construction for
some instructions This pruning breaks the above mechanism, since the
deopt-environ isn't usable as eager deopt target. (It is effectively
changing the environment on the IR instruction to be a lazy-deopt
environment).
This CL makes the [Environment] represent both the eager deopt target as
well as the lazy deopt target. It distinguishes the two by remembering
how many slots the eager deopt target needs to be pruned to come to the
lazy deopt target. The SSA construction will populate this information.
Effectively we move the deopt env pruning from SSA construction to the
place when we need it (e.g. inlining, emitting after-call metadata).
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=Refactoring of existing code.
Change-Id: I6c2a117b33f35764e556372484e4beaa294b708d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192141
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
More specifically on 64-bit platforms, when other heuristic fail:
- in AOT mode unbox all int typed phis.
- in JIT mode unbox all non-Smi int typed phis i.e. mixed
Mint/Smi phis.
This brings treatment of int phis closer to how we treat other
boxed phis.
This considerably improves AOT code quality for loops which
our current conservative heuristic does not currently hit,
for example:
void foo(int start, int end) {
for (var i = start; i < end; i++) {
}
}
Code like this would use a boxed Phi previously which does not make
sense.
In JIT mode to really benefit from aggressive unboxing we
might need to shift from over-reliance on Smi operations, e.g.
when phi is unboxed we want to cascade that through and unbox
operations which are performed on it transitively. This requires
much more sophisticated algorithm and is outside of scope for this
change.
In AOT we in general treat all arithmetic operations as operations
over 64-bit int values so unboxing phis makes more sense.
As a result of this change we see small improvement in code size:
flutter_gallery_app_so_gzip_size -0.2392%
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Change-Id: Iae9a16f353e53f649da4e665d2812ad47099a674
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187860
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When creating new instructions that inherit a token position that
represents a source location from another instruction, the inheriting
instruction must also have the same inlining ID in order for the source
position represented by the token position to be looked up in the
correct script.
Force this by wrapping both in a single InstructionSource struct which
is taken by instructions which take token positions instead of just a
token position. That way, it's more work to manually transfer the token
position separately from the inlining ID of the instruction than doing
the right thing of transfering both at once.
To ensure this information is kept consistent, we pass InstructionSource
structs through the FlowGraphCompiler all the way down to the
CodeSourceMapBuilder.
This CL also makes the following changes:
* Cache the upper bound of source positions in scripts and use it to add
a check for if a given real token position is valid for the script
without iterating over the line starts data for each token position.
* Start inlining intervals appropriately when adding descriptor and
null check information to the code source map.
Code size changes are minimal on Flutter gallery in release mode
(<0.05% decrease).
TEST=Existing tests on trybots, with manual checking with
--check-token-positions that previous errors are now removed.
Bug: https://github.com/dart-lang/sdk/issues/44436
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-ia32-try
Change-Id: I23ced262cb4e9fe9d81356f409e7e8d220d63ee0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173967
Reviewed-by: Régis Crelier <regis@google.com>
The `ICData::AddTarget()` is always used directly after allocation of an
ICData (if needed). We'll remove it and add a `ICData::NewForStaticCall()`
which guarantees the ICData is correctly initialized from the start.
This avoids mental overhead of needing to think about whether
`AddTarget()` needs locking or not.
TEST=Refactoring of already tested code.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I3529dd881887afb281c83b72117bdb9247a8dff0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173760
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
The initial release assert was added when removing assembler intrinsics
for implicit getters and replacing them with graph intrinsics in:
https://dart-review.googlesource.com/c/sdk/+/155900
Normally if the return value of a implicit field getter is unboxed, the
field itself must be unboxable. That is true, though just because we can
unbox a field doesn't mean we always will:
The unboxing field bitmap - which e.g. the GC uses - only allows 64-bits.
Classes with more fields might have some fields boxed even though they
could theoretically be unboxed.
Closes https://github.com/flutter/flutter/issues/67803
Change-Id: I39717d356e293ed85becd64c8a664465d2abd31d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168100
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is to make sure that lookup can be done without getting a safepoint, which might be needed if class needs to be finalized.
Change-Id: I6ed52a56d53369327df0b85d7b04cf41a46ac690
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159188
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When converting InstanceCall to LoadField don't adjust deoptimization
environment because when we deoptimize from LoadField back to
InstanceCall we would expect to have receiver on the stack.
This is a different situation from LoadField to LoadField lazy
deoptimization in which case receiver should not be on the stack.
Regression test provided by Ryan.
Fixed: 42900
Change-Id: I6041ff02abf22a2bae0fdcd65d5b49ef8aec6204
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157182
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Instead of having identical cases for every instruction that implements
RedefinedValue, just have one generic piece of code.
Change-Id: If3f4905a578eb345ae3d740d1f539d346c6223af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150985
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
When replacing an unboxed phi with its contant value we should not
try to insert unboxing instruction before the phi - as phis exist
outside of the normal stream of instructions and InsertBefore does
not work on them.
Instead we should insert unboxing into the block which contains
the phi.
Original test is expanded to cover this case as well now.
Fixes https://github.com/dart-lang/sdk/issues/42164
TEST=vm/cc/ConstantPropagator_Regress35371
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try
Fixed: 42164
Change-Id: I53f984216e53195206fb8247b0ed8999590415a3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149842
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Load*Field and Init*Field IL instructions are fused into load
instructions with 'calls_initializer' flag.
A new, more powerful elimination of lazy initializer calls uses
data flow analysis and load forwarding (place numbering).
In addition to improved elimination of initializer calls,
fusing instructions avoids extra load after field is already
loaded for lazy initialization check.
Fixes https://github.com/dart-lang/sdk/issues/41417
Change-Id: Ibb2a63f84b9b0c970db67b3d6684ec15384a64e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148283
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Before this change we would replace unboxed definitions (those with
representation not equal to kTagged) which is too loose and causes issues
in subsequent passes, which assume that replacements have normalised
representation.
After this change we make sure that replacement has a matching
representation.
Closes https://github.com/dart-lang/sdk/issues/35371
Fixes https://github.com/dart-lang/sdk/issues/41971
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try
Change-Id: I48543893562acf4f8516651196eba7c0c0769a22
Fixed: 35371,41971
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103139
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
It might be too late to remove redundant phis in EliminateDeadPhis
because EliminateDeadPhis runs after OptimizeBranches, and
OptimizeBranches may be able to eliminate more branches if redundant
phis are cleaned up.
This change moves elimination of redundant phis into canonicalization
pass which is repeated more frequently.
Fixes recent regressions on several micro-benchmarks after
https://dart-review.googlesource.com/c/sdk/+/139314
Change-Id: Ib0148564615dbd0adb88e93b938ca9f3f59cabfd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148792
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The following options are removed:
--use_strong_mode_types
--experimental_unsafe_mode_use_at_your_own_risk
The following flags and predicates are removed:
Dart_IsolateFlags::unsafe_trust_strong_mode_types
Isolate::argument_type_checks()
Isolate::can_use_strong_mode_types()
Isolate::should_emit_strong_mode_checks()
Also, everything depending on these flags is cleaned up.
Change-Id: I9328009ad5a42ea2173842386d612c465e3ebec1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147325
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
#if defined(DART_PRECOMPILED_RUNTIME)
#error "AOT runtime should not use compiler sources (including header files)"
#endif // defined(DART_PRECOMPILED_RUNTIME)
Remove #if defined(DART_PRECOMPILED_RUNTIME) from most compiler sources.
Change-Id: Id175c83fdbea38d9d5e1371ff433e3888f2afe8a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143523
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Allocate the canonical true and false objects right after the null
object such that their pointers are the same except for a single bit.
This enables a number of optimizations:
- Test of a bool value becomes a bit test when it is known that the
value is a bool. on ARM64, this bit test can be combined with the
branch.
- Negation of a bool value becomes an xor when it is known that the
value is a bool.
- Load of a bool value becomes an immediate add with the null register
on ARM64.
This is a reland of f8ead09ba2
Change-Id: Iadb2169f80d3731153df3a93772f6142c3197260
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142375
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
This reverts commit 87b4848bfc as it broke simarm, simarm64 builds.
Revert "[vm] Distinguish between bool objects by a single bit in their pointers."
This reverts commit f8ead09ba2 being collateral damage for the revert above.
Change-Id: I0a12fe8095566430823c094b294c490b59cb3c39
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142404
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Allocate the canonical true and false objects right after the null
object such that their pointers are the same except for a single bit.
This enables a number of optimizations:
- Test of a bool value becomes a bit test when it is known that the
value is a bool. on ARM64, this bit test can be combined with the
branch.
- Negation of a bool value becomes an xor when it is known that the
value is a bool.
- Load of a bool value becomes an immediate add with the null register
on ARM64.
Change-Id: I70db024437e0f9cb15c401354e5e35802c354172
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137100
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
In order to have instance calls with unboxed parameters, all
the possible targets need to be able to those parameters unboxed.
Then, the methods were partitioned according to the class hierarchy,
and the inferred types from TFA are used to determine the parameters
that could be unboxed.
This is done for non-nullable integer and double parameters, and
return value.
dart-aot linux x64:
Bench2D 8.028%
DeltaBlue 4.688%
ParticleSystemPaint 27.79%
Flutter release android-armv8:
velocity_tracker_bench -2.026% (less is better)
flutter_gallery_total_size -0.1460%
flutter_gallery_instructions_size -0.2715%
flutter_gallery_readonlydata_size 0.0539%
Issue https://github.com/dart-lang/sdk/issues/33549
Change-Id: Iedd9b4321c3d78b406ea1708e15c747754433027
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138501
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
Static methods and constructors may receive unboxed parameters.
This change adds a bitmap to the RawFunction object indicating
which parameters are unboxed and their types. Also, the stack frame
bitmap was updated to include new bits for those parameters.
The ArgumentsDescriptor object was updated to store not only the number
of arguments, but also their size, since 32-bit platforms would use two
words for each int64 and double value.
In addition, static methods that return non-nullable integers
and doubles may return these values unboxed.
Dart AOT x64:
TypedData.Uint8ClampedListViewVarBench +11.73%
DeltaBlue +9.32%
Meteor (Intel Xeon) +4.64%
Flutter release android-armv8:
velocity_tracker_bench -2.32% (less is better)
flutter_gallery_readonlydata_size +0.18%
flutter_gallery_total_size +0.13%
flutter_gallery_instructions_size +0.12%
Issue https://github.com/dart-lang/sdk/issues/33549
Change-Id: I97e310f8f30213920af2333fbdedf193d03db5c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135313
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The same instruction should be able to set different speculative
modes for its inputs when they are inserted by select representations,
e.g. static calls that have unboxed parameters might set unbox
instructions for those parameters as non speculative.
Change-Id: Ie450f0431e4dbbf0269869f520f59d5d5e946887
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134334
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Rationale:
Leaving a push argument on the stack in slightly more
complex flow graphs (try-catch, loop back, conditinals)
required a bit more bookkeeping. This CL also introduces
more integrity checks to make sure the stack is as
expected at each block and instruction.
Note:
Fuzzer is regression test for new feature.
https://github.com/dart-lang/sdk/issues/38231
Change-Id: I8f7e605937d76c1a429df601e69380111737dd94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116545
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Since Dart 2, library prefixes are always loaded eagerly on the VM.
Also remove reload check for deferred prefixes. This check was added to avoid behavior of library prefixes giving spurious "not loaded" errors after a reload, but now this error can never occur. Resolves a difference in reload between AST and bytecode modes, since bytecode mode isn't surfacing the deferred property.
Change-Id: Ide5fb6cac2efc90ca1b108a35bc09d342cbd60de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116051
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
* Adds non-static final field printing for `Instance`s
* Adds specialized printing for `Closure`s (tearoffs).
* Instances of `Mint` and `Type` can also appear in constant
pools, so handle both in ObjectToSExp.
* DartValueToSExp is a wrapper method for ObjectToSExp that
also checks to see if the value is defined in the constant
pool. If it is, the object is serialized as a reference to
that definition.
* Reworks the `tmp_*_` fields used for non-reentrant cases
to be per function to make it easier to understand how they
interact. (Only cross-function tmp field that remains is
`tmp_string_`, since its contents are always immediately
converted to a C string and then no longer needed.)
Bug: https://github.com/dart-lang/sdk/issues/36882
Change-Id: Ibad4bc2497dd1f580d1daa536c4a267818981c14
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/111425
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
We do not generate explicit control flow and phis for exceptional
edges. Instead CatchBlockEntry has a Parameter instruction for
each local variable. These Parameter-s represent that state
of local variables on entry into the catch block. This means
that we should mark all variables assigned on entry to the
catch block.
Previous version of the code was incorrectly marking only
direct parameters as stored.
Fixes https://github.com/dart-lang/sdk/issues/36953
Bug: 36953
Change-Id: I2e8040d75ba23d2483e451300f44bbdcef458655
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103802
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Type propagator adds Redefinition instructions to provide more accurate
type information only for dominated uses (for example, to mark receiver
as non-nullable after instance calls). However, adding Redefinition
instructions may interfere with branch simplification, which looks only
at empty join blocks. This change avoids adding unused Redefinition
instructions (if there are no dominated uses).
This particular situation happens in Flutter in RenderObject.layout() where
constraints.isTight is inlined with multiple polymorphic targets, including
SliverConstraints.isTight which returns false.
Branch simplification was not happening due to Redefinition instruction
for 'constraints'. That Redefinition instruction is not used.
Change-Id: I15082bacd7cc6ea1445a4dadc8ef97853142ba74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103023
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
LoadOptimizer has a builtin alias-analysis which recognized only
Redefinition and AssertAssignable as instructions which must
be followed through when traversing use lists for a given
definition. However after recent changes CheckNull should be
treated the same way as these other two instructions.
This lead to incorrect computation of aliasing identity and
consequently incorrect load forwarding.
This change fixes this bug and also changes implementation of
OriginalDefinition in a way that makes aliasing analysis code
more future-proof.
We introduce virtual method Definition::RedefinedValue and
reimplement Definition::OriginalDefinition through RedefinedValue.
Aliasing analysis is changed to use RedefinedValue to detect
if an instruction is a redefinition.
Fixes https://github.com/flutter/flutter/issues/30619
Bug: https://github.com/flutter/flutter/issues/30619
Change-Id: I8019c93779491a4e6f127119d1d56eb037a3bb56
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99170
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit d79787771e.
Reason for revert:
This caused significant performance regressions on our protobuf_decode* benchmarks (probably on more benchmarks, but we don't yet have results due to some perf infra issue).
To avoid letting the regressions land for downstream users we'll revert it.
We should measure performance impact before landing this again.
Original change's description:
> [vm] Repair the resolver abstraction, take 2.
>
> - Move resolution logic that accumulated in the runtime entries into Resolver.
> - Improve NoSuchMethodError for closures when --no-lazy-dispatchers
> - Fix concurrent modification of Null class by nullability propagation.
>
> Change-Id: Ib05b431a289d847785032dda46e1bbcb524b7343
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98428
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
TBR=rmacnak@google.com,alexmarkov@google.com,regis@google.com
Change-Id: I4f0f0a0d7c6018fc5968aafdce30f6d2e7495059
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99140
Reviewed-by: Martin Kustermann <kustermann@google.com>
Ryan Macnak