Previously, compiler canonicalized Phi instructions which reference
only one value (modulo redefinitions):
v1 = Phi(v2, v2, ..., v2)
This change extends the canonicalization rule to also allow the Phi
to reference itself:
v1 = Phi(v2, v2, ...., v2, v1, ...., v1)
Such Phi instructions may occur in loops when local variable is not
mutated (remains the same on the back-edge).
TEST=ci
Change-Id: I2838c4307a6dd3067d52624db66487ba4682538c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195043
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
In the codebase, we have several different interfaces for equality
instance methods on a type T:
- bool Equals(const T&) const;
- bool Equals(const T*) const;
- bool Equals(T*) const;
This CL standardizes on `bool Equals(const T&) const;`, and adjusts any
related functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: I2ebcf2b861732ce65c6f1c673b2df2a2c7f5dd4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195071
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In the codebase, we have several different interfaces for instance
methods that return a hash:
- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;
This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
The StoreIndexedInstr::EmitNativeCode will use the "index" register
as is and not zero-extend it. That means the "index" register
cannot have kUnboxedInt32 representation (which allows upper 32
bits to contain garbage)
TEST=Existing test suite.
Change-Id: I413847ec9a43ff446a4f89c78dce7c4fdbceca10
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195066
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This change removes code duplication between
LoadIndexedInstr::representation() and
StoreIndexedInstr::RequiredInputRepresentation(), multiplied by 4
target architectures.
This change also removes duplicated logic from FFI load and store
intrinsics.
TEST=ci
Change-Id: I9f10cc8c70288ecaea33a337a880c181ecbfcd6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194281
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Before value is stored into Float32List, it is converted to 32-bit
float using DoubleToFloat instruction. If allocation sinking
eliminated the allocation of Float32List but we need to deoptimize,
the list is materialized and elements are filled. In such case,
we shouldn't perform double->float conversion as it already happened.
This change also updates the assertion in DoubleToFloatInstr::Canonicalize
which verifies that DoubleToFloat instruction can be used only
in certain cases.
TEST=tests/language/vm/allocation_sinking_arrays_test.dart
Fixes https://github.com/dart-lang/sdk/issues/45547
Change-Id: I9a1bd28a9fc09bccad6aa3c91a7880abd002f7c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193831
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change extends EqualityCompare IL instruction to handle case
when operands are nullable ints.
Unlike CheckedSmiComparison, EqualityCompare performs comparison
without making a dynamic call on a slow path. Dynamic call in
CheckedSmiComparison is a problem because it is not present in
the original program and not properly tracked in TFA and precompiler,
so precompiler is not able to drop Code objects (needed for
dynamic calls) reliably.
This change removes the remaining case when CheckedSmiComparison
instruction was used. The cleanup will follow in a separate CL.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/32166
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I8114a2616becf632462d4cb3bfc255691f24e1ef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193447
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Loosening of assertion:
There is no guarantee that a static-final field has been initialized by
the time a function is compiled (in optimized mode) that uses such a
field.
We should therefore loosen the ASSERT to not require the field to be
initialized and rather allows CSE
Enabling of CSE:
In the past we had separate InitStaticField and LoadStaticField. The
load itself had no side-effects and could therefore be moved
arbitrarily. Though we couldn't allow it to be moved before it's
InitStaticField. This dependency was not explicitly made and we
therefore disabled CSE / LICM if the actual field was not initialized
(or field may be reset) - see [0].
Though after merging of InitStaticField and LoadStaticField in [1] there
is no longer a need for tracking any dependencies: The side-effects
of InitStaticField are now reported by LoadStaticField.
=> We can therefore always allow CSE of LoadStaticFieldinstr and
any code motion would respect side-effects of the instruction.
[0] https://codereview.chromium.org/1497783002
[1] https://dart-review.googlesource.com/c/sdk/+/148283
TEST=Fixes flaky test.
Closes https://github.com/dart-lang/sdk/issues/45133
Closes https://github.com/dart-lang/sdk/issues/45446
Change-Id: I4a699e9b1dc9dfec9a91208ed78ed0a0c41f5cad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192924
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This reverts commit ff1c9c9916.
Reason for revert: Most likely cause of failures in AOT, e.g.
python tools/test.py -n dartkp-strong-linux-release-x64 co19/LanguageFeatures/nnbd/late_A06_t05
Original change's description:
> [vm/compiler] Always allow CSE of LoadStaticField & loosen assertion in AllowsCSE
>
> Loosening of assertion:
>
> There is no guarantee that a static-final field has been initialized by
> the time a function is compiled (in optimized mode) that uses such a
> field.
>
> We should therefore loosen the ASSERT to not require the field to be
> initialized and rather allows CSE
>
> Enabling of CSE:
>
> In the past we had separate InitStaticField and LoadStaticField. The
> load itself had no side-effects and could therefore be moved
> arbitrarily. Though we couldn't allow it to be moved before it's
> InitStaticField. This dependency was not explicitly made and we
> therefore disabled CSE / LICM if the actual field was not initialized
> (or field may be reset) - see [0].
>
> Though after merging of InitStaticField and LoadStaticField in [1] there
> is no longer a need for tracking any dependencies: The side-effects
> of InitStaticField are now reported by LoadStaticField.
> => We can therefore always allow CSE of LoadStaticFieldinstr and
> any code motion would respect side-effects of the instruction.
>
> [0] https://codereview.chromium.org/1497783002
> [1] https://dart-review.googlesource.com/c/sdk/+/148283
>
> TEST=Fixes flaky test.
>
> Closes https://github.com/dart-lang/sdk/issues/45133
>
> Change-Id: I547702586d980f5d02fde0e8d12b057ca4d8c363
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192740
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
TBR=vegorov@google.com,kustermann@google.com
Change-Id: I921c6e8929bbc9f58392d6be6db599dddb8e4b4c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192922
Reviewed-by: Martin Kustermann <kustermann@google.com>
Since allocation instructions can throw OOM exceptions, they require
deopt information. In general it's rather hard to test that this deopt
information is correct.
In order to test this, this CL makes those IR instructions support lazy
deopt and add tests that exercise this deopt sequence, thereby ensuring
the deoptimization environment is correct.
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=New vm/dart_2/isolates/deopt/*_test.
Change-Id: I6a02dcf5a0c47636f1f0aa4cd8cc0d2b4f032ca0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192687
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
An allocation can - in the slow path - go to runtime which can cause a
pre-allocated OutOfMemory exception to be thrown. It therefore requires
correct deopt information.
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=Existing test suite. A follow-up CL will add actual deoptimization tests for those instructions.
Change-Id: Iaf44904b82ec021919352483f58e9e8fed473955
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192685
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Loosening of assertion:
There is no guarantee that a static-final field has been initialized by
the time a function is compiled (in optimized mode) that uses such a
field.
We should therefore loosen the ASSERT to not require the field to be
initialized and rather allows CSE
Enabling of CSE:
In the past we had separate InitStaticField and LoadStaticField. The
load itself had no side-effects and could therefore be moved
arbitrarily. Though we couldn't allow it to be moved before it's
InitStaticField. This dependency was not explicitly made and we
therefore disabled CSE / LICM if the actual field was not initialized
(or field may be reset) - see [0].
Though after merging of InitStaticField and LoadStaticField in [1] there
is no longer a need for tracking any dependencies: The side-effects
of InitStaticField are now reported by LoadStaticField.
=> We can therefore always allow CSE of LoadStaticFieldinstr and
any code motion would respect side-effects of the instruction.
[0] https://codereview.chromium.org/1497783002
[1] https://dart-review.googlesource.com/c/sdk/+/148283
TEST=Fixes flaky test.
Closes https://github.com/dart-lang/sdk/issues/45133
Change-Id: I547702586d980f5d02fde0e8d12b057ca4d8c363
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192740
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Generally speaking, the optimizing compiler should be able to use
(deopt-id, deopt-environ) of any IR instruction and use it as an eager
deopt target (e.g. optimizing compiler might insert TestCidsInstr
with eager (deopt-id, deopt-environ) from AssertAssignable).
Currently we prune the environment eagerly during SSA construction for
some instructions This pruning breaks the above mechanism, since the
deopt-environ isn't usable as eager deopt target. (It is effectively
changing the environment on the IR instruction to be a lazy-deopt
environment).
This CL makes the [Environment] represent both the eager deopt target as
well as the lazy deopt target. It distinguishes the two by remembering
how many slots the eager deopt target needs to be pruned to come to the
lazy deopt target. The SSA construction will populate this information.
Effectively we move the deopt env pruning from SSA construction to the
place when we need it (e.g. inlining, emitting after-call metadata).
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=Refactoring of existing code.
Change-Id: I6c2a117b33f35764e556372484e4beaa294b708d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192141
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Unboxed int64 are supported on all architectures which Dart VM supports.
The only was to disable support for unboxed int64 is to use
--no-unbox_mints flag, which is not used and not tested.
This change cleans up this flag and corresponding compiler predicates.
TEST=ci
Change-Id: Ib667319eaf87f0ded8bf9d9505272a3693a9e63e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191761
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Some IR instructions (e.g. RawStoreField) can never deopt by-design. Some
instructions are calling runtime entries but those are leaf calls (e.g.
InvokeMathCFunction). Some are synthetic IR instructions that never
actually emit code (e.g. Constraint). Some IR instructions cannot deopt
due to inputs (e.g. Boxing instructions whose value is guaranteed to fit
into Smi).
Part of https://dart-review.googlesource.com/c/sdk/+/190481
TEST=Existing test suite.
Change-Id: I770b7d094ef36221d357157c736710476fd6bfb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190482
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Also ensure the number of parent type arguments is appropriately
adjusted in the returned FunctionType when partially instantiating
a FunctionType, and that free parent type arguments are _not_
included in types built while reading constants.
Since the number of parent type arguments is now cached there, we
no longer need to also cache them in ClosureData. We can also remove
the parent walking in Function::NumParentTypeArguments().
Previously, a FunctionType where the component types did not use
parent type parameters was considered instantiated. This CL changes
it so that FunctionType with free parent type parameters are never
considered instantiated. This is necessary because otherwise,
when instantiating the parent type parameters, a FunctionType that
does not use its parent type parameters will be used unchanged
instead of creating a copy with fewer parent type parameters.
Because of this, places where IsInstantiated was used to simply
check for generic parent functions (namely, for implicit closure
creation) has been appropriately weakened to check
!HasGenericParent() instead.
TEST=Existing tests on CI.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Ifb4a0a1273d8d01908cdf4ffc3c4c28a1c33ffa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190021
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This CL adds code to detect situations when we hit
null error through GDT call with non-null receiver.
We achieve this by making receiver's cid part of the
GDT calling convention and checking this cid
in runtime entry responsible for throwing the null
error.
This CL is an attempt to narrow down b/179632636
and collapse various impossible crash reports
into a single native crash cluster by crashing VM
instead of throwing a null error.
TEST=tested manually
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: If2ed4646c4c0f403016266e4e83e296a7b234cb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191141
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
AssertBoolean may assume its input is nullable bool. BooleanNegate may assume its input is non-nullable bool. Use fixed alignment bits to distinguish true, false and null.
TEST=ci
Change-Id: Ife694e66c1225d943881c2325cd67f18575a47e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189240
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The deoptimization environment of AssertAssignable should not have it's
inputs in it, since those get poped before the call to the
TypeTestingStub.
A lazy deopt should therefore continue in unoptimized code after the TTS
call, without the inputs.
This was discovered by running many lightweight isolates concurrently on
the JIT.
To simplify a regression test we'll add new VM flags that can trigger
deoptimization on any runtime call (optionally filtered by name of
runtime call).
We can use this mechanism to possibly discover more bugs of the same
kind.
Closes https://github.com/dart-lang/sdk/issues/45207
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=runtime/tests/vm/dart{,_2}/regress_45207_test
Change-Id: Ib713fa8be5914eaf4c7395d2e1d8dd88714f1ce2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189202
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
When going from Int64 through another representation back into Int64
you can discard a conversion if and only if it does not lose some
bits on the way. Previously we were checking for truncation flag, but
this is incorrect because it does not actually tell us anything about
about whether or not bits are going to be lost.
Instead, we should be checking the range of the source value.
In addition to fixing potential correctness issues, this CL actually
improves quality of code generated for Uint8 loads because it removes
a redundant chain of Int64->Uint32->Int64 conversions that follows
the load (which compiled 2 two redundant moves).
Issue https://github.com/dart-lang/sdk/issues/40308
TEST=vm/cc/IL_IntConverterCanonicalization
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Change-Id: Ifef0943a50d414bde41b2194e30b58ad3a8c106c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187921
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
The range of Smis must be reduced so that pointer compression is not lossy.
Smis are kept in sign-extended form. In the future, we might track down all places that perform Smi comparisons or otherwise assume sign-extension. Allowing garbage in the upper half would make it safe to unconditionally add the heap base during pointer decompression.
Based on cf78da8a48.
TEST=ci
Change-Id: If8c76da3166c170618e5b3f3991353bab5c84e6f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181760
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
As part of making the compiler independent of the current isolate (since
JITed code will be shared across all isolate) we make the compiler not
depend on global field state, with one exception: we preserve an existing
optimization that utilizes knowledge whether a global final field was
already initialized (if --enable-isolate-groups is turned off) - we do
so by asking the isolate group for it's own isolate member.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing code.
Change-Id: I2f08d854af6102e05e5a1df8d5b66b514d6f3ce4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182540
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When creating new instructions that inherit a token position that
represents a source location from another instruction, the inheriting
instruction must also have the same inlining ID in order for the source
position represented by the token position to be looked up in the
correct script.
Force this by wrapping both in a single InstructionSource struct which
is taken by instructions which take token positions instead of just a
token position. That way, it's more work to manually transfer the token
position separately from the inlining ID of the instruction than doing
the right thing of transfering both at once.
To ensure this information is kept consistent, we pass InstructionSource
structs through the FlowGraphCompiler all the way down to the
CodeSourceMapBuilder.
This CL also makes the following changes:
* Cache the upper bound of source positions in scripts and use it to add
a check for if a given real token position is valid for the script
without iterating over the line starts data for each token position.
* Start inlining intervals appropriately when adding descriptor and
null check information to the code source map.
Code size changes are minimal on Flutter gallery in release mode
(<0.05% decrease).
TEST=Existing tests on trybots, with manual checking with
--check-token-positions that previous errors are now removed.
Bug: https://github.com/dart-lang/sdk/issues/44436
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-ia32-try
Change-Id: I23ced262cb4e9fe9d81356f409e7e8d220d63ee0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173967
Reviewed-by: Régis Crelier <regis@google.com>
This CL adds passing structs by value in FFI trampolines.
Nested structs and inline arrays are future work.
C defines passing empty structs as undefined behavior, so that is not
supported in this CL.
Suggested review order:
1) commit message
2) ffi/marshaller (decisions for what is done in IL and what in MC)
3) frontend/kernel_to_il (IL construction)
4) backend/il (MC generation from IL)
5) rest in VM
Overall architecture is that structs are split up into word-size chunks
in IL when this is possible: 1 definition in IL per chunk, 1 Location in
IL per chunk, and 1 NativeLocation for the backend per chunk.
In some cases it is not possible or less convenient to split into
chunks. In these cases TypedDataBase objects are stored into and loaded
from directly in machine code.
The various cases:
- FFI call arguments which are not passed as pointers: pass individual
chunks to FFI call which already have the right location.
- FFI call arguments which are passed as pointers: Pass in TypedDataBase
to FFI call, allocate space on the stack, and make a copy on the stack
and pass the copies' address to the callee.
- FFI call return value: pass in TypedData to FFI call, and copy result
in machine code.
- FFI callback arguments which are not passed as pointers: IL definition
for each chunk, and populate a new TypedData with those chunks.
- FFI callback arguments which are passed as pointer: IL definition for
the pointer, and copying of contents in IL.
- FFI return value when location is pointer: Copy data to callee result
location in IL.
- FFI return value when location is not a pointer: Copy data in machine
code to the right registers.
Some other notes about the implementation:
- Due to Store/LoadIndexed loading doubles from float arrays, we use
a int32 instead and use the BitCastInstr.
- Linux ia32 uses `ret 4` when returning structs by value. This requires
special casing in the FFI callback trampolines to either use `ret` or
`ret 4` when returning.
- The 1 IL definition, 1 Location, and 1 NativeLocation approach does
not remove the need for special casing PairLocations in the machine
code generation because they are 1 Location belonging to 1 definition.
Because of the amount of corner cases in the calling conventions that
need to be covered, the tests are generated, rather than hand-written.
ABIs tested on CQ: x64 (Linux, MacOS, Windows), ia32 (Linux, Windows),
arm (Android softFP, Linux hardFP), arm64 Android.
ABIs tested locally through Flutter: ia32 Android (emulator), x64 iOS
(simulator), arm64 iOS.
ABIs not tested: arm iOS.
TEST=runtime/bin/ffi_test/ffi_test_functions_generated.cc
TEST=runtime/bin/ffi_test/ffi_test_functions.cc
TEST=tests/{ffi,ffi_2}/function_structs_by_value_generated_test.dart
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_generated_tes
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_test.dart
TEST=tests/{ffi,ffi_2}/vmspecific_static_checks_test.dart
Closes https://github.com/dart-lang/sdk/issues/36730.
Change-Id: I474d3a4ee1faadbe767ddadd1b696e24d8dc364c
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140290
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL enables dart:ffi on Fuchsia.
It has the necessary VM changes for x64 and arm64.
- Allocate VM callback trampoline memory pages with RX.
- Do not use x18 on arm64.
- Save x18 on arm64 in Thread to be able to restore it on exceptions.
We do not have a bot testing this
https://github.com/dart-lang/sdk/issues/38752.
The bot only tests the build, it does not run the emulator.
Manual build and test steps x64 Fuchsia emulator:
```
tools/build.py --os=fuchsia -m debug fuchsia_ffi_test_package
xvfb-run third_party/fuchsia/sdk/linux/bin/femu.sh --image qemu-x64 -N --headless
third_party/fuchsia/sdk/linux/bin/fserve.sh --bucket fuchsia-sdk --image qemu-x64 --device-name step-atom-yard-juicy
third_party/fuchsia/sdk/linux/bin/fpublish.sh out/DebugFuchsiaX64/gen/dart_ffi_test_debug/dart_ffi_test_debug.far
third_party/fuchsia/sdk/linux/bin/fssh.sh --device-name step-atom-yard-juicy run fuchsia-pkg://fuchsia.com/dart_ffi_test_debug#meta/dart.cmx --ignore-unrecognized-flags --packages=/pkg/data/.packages --disable-dart-dev /pkg/data/tests/ffi/all_positive.dart
```
Manual build and test steps arm64 Fuchsia emulator:
```
tools/build.py --no-start-goma --os=fuchsia -m debug -a arm64 fuchsia_ffi_test_package
xvfb-run third_party/fuchsia/sdk/linux/bin/femu.sh --image qemu-arm64 -N --headless --experiment-arm64
third_party/fuchsia/sdk/linux/bin/fserve.sh --bucket fuchsia-sdk --image qemu-arm64 --device-name step-atom-yard-juicy
third_party/fuchsia/sdk/linux/bin/fpublish.sh out/DebugFuchsiaARM64/gen/dart_ffi_test_debug/dart_ffi_test_debug.far
third_party/fuchsia/sdk/linux/bin/fssh.sh --device-name step-atom-yard-juicy run fuchsia-pkg://fuchsia.com/dart_ffi_test_debug#meta/dart.cmx --ignore-unrecognized-flags --packages=/pkg/data/.packages --disable-dart-dev /pkg/data/tests/ffi/all_positive.dart
```
Note that running on the arm64 emulator takes over an hour.
This has been tested on an Estelle device.
TEST=Manual test steps above.
The AOT mode has seen an initial test in g3 by rolling gen_snapshot and
the flutter runner.
Arm64 x18 treatment along the lines of
https://dart-review.googlesource.com/c/sdk/+/119481:
- FfiCallInstr, does not modify x18 at all. We do not have to do
anything special when having a C++ frame on top of a Dart frame.
- NativeEntryInstr and NativeReturnInstr, saves and restores x18 with
PushNativeCalleeSavedRegisters && PopNativeCalleeSavedRegisters. Also
save to the thread field in both places. That way the JumpToFrame stub
can load from the thread field.
Closes: https://github.com/dart-lang/sdk/issues/44434
Change-Id: Ide29369c878fed5909a0d6f7d4c3f7508e179f25
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173273
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The layout of WeakProperty is defined in C++ instead of Dart. Add missing recognition of its accessors so the compiler can use direct field access instead of always going through natives. Compare LinkedHashMap.
Change the sentinel for WeakProperty lists to object null so that no special constructor is needed.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/44333
Change-Id: I8686383053c2c345c972b6615b514a1381e79fda
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175001
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
We serialize all changes to patchable call state to ensure we always
make progress (e.g. transition from Monomorphic -> Polymorphic).
Only if the miss handler actually needs to patch the patchable call do
we stop mutators - if it only updates ICData we don't have to do that.
=> In the future we might introduce a more clever mechanism to avoid
stopping mutators.
The CL also moves some locks from Isolate to IsolateGroup to ensure
all mutators use the same lock, since they operate on shared data
structures.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Existing isolate tests that --enable-isolate-groups.
Change-Id: I4a11a8b8bb65581c6aba11cdf73acdea0f24a502
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174469
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
With lightweights isolates enabled we need to ensure to update entries
in MetamorphicCache in sequence with mutual exclusion.
This CL adds appropriate locking for the MegamorphicCache.
Right now this is safe in AOT because the entire SwitchableCallMiss
handler runs with stopped mutators. A future CL will relax this. That
future CL will also move the megamorphic mutex from Isolate to
IsolateGroup, since the megamoprhic caches are shared within an isolate
group.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=CL is purely a refactoring.
Change-Id: I30305b04027d2bf9b4d5f6b1700bf158d38bdba7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174467
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
To prevent unoptimized compilations of functions from modifying
ICData objects, we'll ensure to populate the ICData with the right
information when it's created.
Furthermore this moves out resolution related code from ICData which
really doesn't belong there.
TEST=Refactoring of already tested code.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I4b71118c02687a047077f2709c188b5d8a825de2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173968
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This is a reland of 42c76fd910
Main issue was due to the parameter order of the invoke field dispatcher
not matching the argument order described by its saved arguments
descriptor. There's no reason for it not to, so now it does.
Also fixes some issues with stack trace tests that failed due to
increased deduplication of closures by forbidding deduplication for
those tests.
TEST=Run on trybots of all architectures as well as flutter engine
trybot, new test added for downstream issues seen after initial landing.
Original change's description:
> [vm/compiler] Move AssertAssignables out of closure bodies.
>
> This CL moves the final set of checks out of closure bodies and into
> dynamic closure call dispatchers. It also adds stubs for checking top
> types and null assignability for types only known at runtime.
>
> Fixes https://github.com/dart-lang/sdk/issues/40813 .
>
> Changes in Flutter gallery in release mode:
>
> * arm7: -3.05% total, +0.99% vmisolate, -0.89% isolate,
> -1.20% readonly, -4.43% instructions
> * arm8: -3.20% total, +0.99% vmisolate, -0.88% isolate,
> -1.18% readonly, -5.05% instructions
>
> TEST=Run on trybots of all architectures, includes test adjustments where needed.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
> Change-Id: Ifb136c64339be76a642ecbb4fda26b6ce8f871f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166622
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
Change-Id: Ic5ec59cf355f7779bb82db798d97d762ba1e5556
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-product-x64-try,flutter-engine-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172644
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 42c76fd910.
Reason for revert: Failures in flutter/google3. Initial hypothesis is related to product mode and instruction deduplication.
Original change's description:
> [vm/compiler] Move AssertAssignables out of closure bodies.
>
> This CL moves the final set of checks out of closure bodies and into
> dynamic closure call dispatchers. It also adds stubs for checking top
> types and null assignability for types only known at runtime.
>
> Fixes https://github.com/dart-lang/sdk/issues/40813 .
>
> Changes in Flutter gallery in release mode:
>
> * arm7: -3.05% total, +0.99% vmisolate, -0.89% isolate,
> -1.20% readonly, -4.43% instructions
> * arm8: -3.20% total, +0.99% vmisolate, -0.88% isolate,
> -1.18% readonly, -5.05% instructions
>
> TEST=Run on trybots of all architectures, includes test adjustments where needed.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
> Change-Id: Ifb136c64339be76a642ecbb4fda26b6ce8f871f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166622
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
TBR=kustermann@google.com,regis@google.com,sstrickl@google.com
Change-Id: Iaf79acddcf18fb3699a894950b31515d6f759349
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172643
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit 3c87c14457.
Reason for revert: Reverting parent CL.
Original change's description:
> [vm/compiler] Check non-constant supertypes for top types in AssertSubtype.
>
> This restores the invariant that the runtime entry is never run on
> instantiated top types.
>
> TEST=Run on trybots of all architectures, since existing tests cover static and dynamic AssertSubtype well.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
> Change-Id: Id0ec9a90f1bcc70748af8d8d2b1e8d6198b66152
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171942
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,regis@google.com,sstrickl@google.com
Change-Id: Ia7316b6e9c2aa226171ab09e066a707b4c8a4a76
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172642
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This restores the invariant that the runtime entry is never run on
instantiated top types.
TEST=Run on trybots of all architectures, since existing tests cover static and dynamic AssertSubtype well.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Id0ec9a90f1bcc70748af8d8d2b1e8d6198b66152
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171942
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL moves the final set of checks out of closure bodies and into
dynamic closure call dispatchers. It also adds stubs for checking top
types and null assignability for types only known at runtime.
Fixes https://github.com/dart-lang/sdk/issues/40813 .
Changes in Flutter gallery in release mode:
* arm7: -3.05% total, +0.99% vmisolate, -0.89% isolate,
-1.20% readonly, -4.43% instructions
* arm8: -3.20% total, +0.99% vmisolate, -0.88% isolate,
-1.18% readonly, -5.05% instructions
TEST=Run on trybots of all architectures, includes test adjustments where needed.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ifb136c64339be76a642ecbb4fda26b6ce8f871f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166622
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Ryan Macnak