This also lowers the threshold for converting from a linear cache
to a hash-based cache from 100 to 30 on non-IA32 architectures.
(IA32 still goes to runtime if there's a hash-based cache.)
For SubtypeTestCache benchmarks above this threshold, we see an
improvement varying from ~50-100% to ~700-800% on all architectures,
from lowest number of checks (50) to highest (1000).
For SubtypeTestCache benchmarks below this threshold, no major
changes are seen: generally <5% improvement or <5% regression per
check at most.
TEST=vm/cc/TTS
Change-Id: I83aa7c085ab5a411e944ec660d6b8eba7a788ee0
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/308941
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In particular, avoid storing/comparing the destination type unless the
SubtypeTestCache was created for a dynamic AssertAssignable instruction.
TEST=vm/cc/STC_{Linear,Hash}Lookup, vm/cc/TTS
Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: I279c69a668ce19785785cf71707acf951f2b2133
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310340
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Also eliminates the need to store the delayed type arguments on the
stack during the search loop on ARM.
TEST=vm/cc/TTS
Change-Id: Ife01cc8d255724c272bc277eec397486cedc07a7
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307681
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.
More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing
I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.
Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The ffi-callback related information on the [Thread] object is metadata
corresponding to ffi-callback-trampoline [Function] objects. There is
nothing thread or isolate specific about it.
Moving it away from [Thread] is needed because an [Isolate] can
have different [Thread] objects across its lifetime (see [0]): When
the stack of an isolate is empty, we reserve now the right to
re-cycle the [Thread]. If the isolate later runs again, it may
get a new [Thread] object.
This CL moves this information from [Thread] to the [ObjectStore]. In
addition we make the compiler be responsible for populating this
metadata - instead of doing this per-call site of
`Pointer.fromFunction()`. It will be preserved across snapshot writing
& snapshot reading (for AppJIT as well as AppAOT).
Similarly the JIT trampolines that are on Isolate aren't isolate
specific and can go to [IsolateGroup]. This simplifies doing the above
as the compiler can allocate those as well.
The effect is that [Thread] object gets smaller, GC doesn't have to
visit the 2 slots per-thread. It comes at expense of 2 more loads
when invoking the callback.
[0] https://dart-review.googlesource.com/c/sdk/+/297920
TEST=Regression test is vm/ffi{,_2}/invoke_callback_after_suspension_test
Change-Id: Ifde46a9f6e79819b5c0e359c3d3998d1d93b9b1e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/303700
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This is a reland of commit afdf640866
Original change's description:
> [vm] Refactor StubCodeCompiler to be a real class.
>
> Previously, StubCodeCompiler was just a set of static methods, all of
> which take an assembler as their first arg. This makes it hard to pass
> additional state to the ~160 stub macro defined stub generators.
>
> This refactor makes StubCodeCompiler a real class, with assembler as a
> field. So we can easily add new fields to the class later, to pass new
> state without having to update every stub generator.
>
> assembler is declared as a public field for a few reasons:
> - There's one place where it needs to be accessed by a non-member
> function (in the ia32 file).
> - If it's private, it has to be named assembler_, which would mean a lot
> more insignificant diffs.
> - Non-member functions that take assembler would have to take assembler_,
> for consistency with the __ macro, which would be weird.
>
> Change-Id: I142f0803a07c7839753188065c69c334d4d1798a
> TEST=CI
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289924
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>
Change-Id: Ib5be28c46a0a80b84e31aea60893ab5bbc02e2ea
TEST=CI, including all sanitizers and architectures
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290681
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
This reverts commit afdf640866.
Reason for revert: breaks google3. See b/274712076.
Original change's description:
> [vm] Refactor StubCodeCompiler to be a real class.
>
> Previously, StubCodeCompiler was just a set of static methods, all of
> which take an assembler as their first arg. This makes it hard to pass
> additional state to the ~160 stub macro defined stub generators.
>
> This refactor makes StubCodeCompiler a real class, with assembler as a
> field. So we can easily add new fields to the class later, to pass new
> state without having to update every stub generator.
>
> assembler is declared as a public field for a few reasons:
> - There's one place where it needs to be accessed by a non-member
> function (in the ia32 file).
> - If it's private, it has to be named assembler_, which would mean a lot
> more insignificant diffs.
> - Non-member functions that take assembler would have to take assembler_,
> for consistency with the __ macro, which would be weird.
>
> Change-Id: I142f0803a07c7839753188065c69c334d4d1798a
> TEST=CI
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289924
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>
Change-Id: If36a9122e1a55d86673d05afbbd21dfb27d7acd5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290522
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Emmanuel Pellereau <emmanuelp@google.com>
Previously, StubCodeCompiler was just a set of static methods, all of
which take an assembler as their first arg. This makes it hard to pass
additional state to the ~160 stub macro defined stub generators.
This refactor makes StubCodeCompiler a real class, with assembler as a
field. So we can easily add new fields to the class later, to pass new
state without having to update every stub generator.
assembler is declared as a public field for a few reasons:
- There's one place where it needs to be accessed by a non-member
function (in the ia32 file).
- If it's private, it has to be named assembler_, which would mean a lot
more insignificant diffs.
- Non-member functions that take assembler would have to take assembler_,
for consistency with the __ macro, which would be weird.
Change-Id: I142f0803a07c7839753188065c69c334d4d1798a
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289924
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
This change introduces specialized stubs and IL instruction for
allocating records with 2 or 3 fields. This makes allocation of
small records slightly faster compared to a construction of similar
class instances and makes code size of record allocation smaller.
Benchmark:
MultipleReturns.NotInlined.Record(RunTime) 77150 -> 66222
MultipleReturns.NotInlined.RecordNamed(RunTime) 78073 -> 67044
MultipleReturns.Forwarded.Record(RunTime) 97130 -> 77635
MultipleReturns.Forwarded.RecordNamed(RunTime) 96495 -> 77904
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I8ed7add06b39ba79dfd78bbe2afaefe606cc505b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266420
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This change introduces separate stubs for suspending sync* functions
at start and at yield/yield*. Suspend stub for yield/yield*
no longer calls Dart callback (in order to make it faster).
Also, ReturnSyncStar stub is removed - sync* functions now directly
return false instead of going through the stub.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Iee9a1f48cab2812cf0f9f0e4e6d8e847547e49f7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250420
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.
Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:
* Suspending variable-length frames (for unoptimized code).
* Handling of Code and pool pointers in Dart stack frames.
* OSR.
* Deoptimization.
* Hot reload.
* Debugger.
The new implementation is not enabled in JIT mode yet.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The RISC-V frame pointer convention is different from that of x86 and ARM. FP is the caller's SP, and the saved FP is at FP[-2] instead of FP[0].
Making Dart frames match the convention of C frames allows stack walkers to continue their traversals through transitions between Dart and C (e.g., for stack dumps or profiling).
TEST=ci
Change-Id: I463348beba70c1a75bfb0d902b3391be524de0fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/235960
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.
The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.
Quirks:
- RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
- There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
- The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
- For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
- Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
- There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
- The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
- We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.
Unimplemented:
- non-trivial FFI cases
- Compressed pointers - No intention to implement.
- Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
- BigInt intrinsics
TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This CL changes the handling of late static fields to be similar to the
handling of late instance fields. That is, if the field does not need a
load guard and has an initializer, the new InitLateStaticField and
InitLateFinalStaticField stubs are used, which directly call the
initializer function instead of going to the runtime. Thus, the only
runtime call left in these cases is when a late final static field is
modified by the initializer function, in which case an appropiate
exception is thrown.
In the case where a late static field might call the initializer
function but no initializer exists, a LateInitializationErrorSlowPath is
generated to be called if the field value is the sentinel, also similar
to the late instance field case.
TEST=Refactoring, so existing tests.
Change-Id: I95887022d8ece1164a8f7c34b77d9de8c53aa898
Bug: https://github.com/dart-lang/sdk/issues/45138
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-ia32-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220009
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Closures created via Closure::New() set the delayed type arguments to
the null type argument vector for non-generic closure functions and to
the empty type argument vector for generic closure functions. However,
closures created via the method extractor stub would always have the
empty type argument vector for delayed type arguments. Make the two
consistent by creating two method extractor stubs, one for generic
extracted methods and one for non-generic ones.
With this change, Closure::IsGeneric simply compares the delayed type
arguments against the empty type argument vector. This change also makes
method extraction for non-generic functions slightly faster, since the
delayed type arguments field is not changed post-allocation.
The old version of Closure::IsGeneric was the only use of
Closure::NumTypeParameters, so remove the latter method.
Related cleanups:
* Use similar checks for Class::IsGeneric and FunctionType::IsGeneric,
since they only have non-null type_parameters fields when generic.
For Class::IsGeneric, this avoids having to fetch the current thread
if the class declaration has been loaded.
* Refactor methods that read packed fields in FunctionType into two
methods, a static method that takes a FunctionTypePtr and an instance
method that calls the static method. Replace the methods in Function
that also directly read FunctionType packed fields with delegations to
the static methods.
* Rework Closure_equals to avoid unneeded handle allocation in certain
cases for non-equal closures.
* Replace cases that allocates a handle for the closure function only
for retrieving the signature with calls to Closure::signature().
TEST=language_2/closure/tearoff_dynamic_test
language_2/regress/regress45890_test
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try
Change-Id: I29c8859c3350ed7b3f1a8f71d82a4393496b7c2b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206820
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
When building method extractors in a given isolate group, the copy of
the AllocateContext stub in the isolate group should be used. Otherwise,
calls to that stub cannot be turned into PC relative calls when
appropriate.
Also retrieve the closure allocation stub from the object store directly
instead of going through GetAllocationStubForClass, now that closure
allocation is handled via a distinct stub.
Be more specific about the types of the stub objects passed to
GenerateBuildMethodExtractorStub, as they're guaranteed to be Code
objects.
TEST=Existing test cases.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try
Change-Id: I05bc5e8a133834e66af3e1003d8a6597ee11c312
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/200868
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Also create a subclass of AllocationInstr, AllocateClosureInstr,
which is used when allocating closures.
Followup CLs add inputs to this instruction and to the AllocateClosure
stub, starting with the closure function. Adding these inputs allows
the related field to be set within the stub, instead of using
StoreInstanceField manually at each closure allocation point.
Since this CL only adds the initial stub/instruction implementation,
the overhead on snapshots is minimal: just the addition of the new
stub to each isolate.
-----
This CL also adds virtual and non-virtual methods to assembler_base.h
revolving around field loads and stores and attempted inline object
allocation, to ensure all architectures have these methods.
It also adds LoadFromSlot/StoreToSlot/StoreToSlotNoBarrier, which
appropriately calls one of the other methods based on whether the slot
is compressed or not and whether it is a boxed or unboxed field.
With these additions, the AllocateClosure stub generator can be defined
in an architecture-independent way.
TEST=Basically a refactoring, so check using current test suites.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-x64-try,vm-kernel-linux-debug-x64c-try
Change-Id: I71f5691307679f8d5e3604007699de4706f86eb8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198284
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL adds passing structs by value in FFI trampolines.
Nested structs and inline arrays are future work.
C defines passing empty structs as undefined behavior, so that is not
supported in this CL.
Suggested review order:
1) commit message
2) ffi/marshaller (decisions for what is done in IL and what in MC)
3) frontend/kernel_to_il (IL construction)
4) backend/il (MC generation from IL)
5) rest in VM
Overall architecture is that structs are split up into word-size chunks
in IL when this is possible: 1 definition in IL per chunk, 1 Location in
IL per chunk, and 1 NativeLocation for the backend per chunk.
In some cases it is not possible or less convenient to split into
chunks. In these cases TypedDataBase objects are stored into and loaded
from directly in machine code.
The various cases:
- FFI call arguments which are not passed as pointers: pass individual
chunks to FFI call which already have the right location.
- FFI call arguments which are passed as pointers: Pass in TypedDataBase
to FFI call, allocate space on the stack, and make a copy on the stack
and pass the copies' address to the callee.
- FFI call return value: pass in TypedData to FFI call, and copy result
in machine code.
- FFI callback arguments which are not passed as pointers: IL definition
for each chunk, and populate a new TypedData with those chunks.
- FFI callback arguments which are passed as pointer: IL definition for
the pointer, and copying of contents in IL.
- FFI return value when location is pointer: Copy data to callee result
location in IL.
- FFI return value when location is not a pointer: Copy data in machine
code to the right registers.
Some other notes about the implementation:
- Due to Store/LoadIndexed loading doubles from float arrays, we use
a int32 instead and use the BitCastInstr.
- Linux ia32 uses `ret 4` when returning structs by value. This requires
special casing in the FFI callback trampolines to either use `ret` or
`ret 4` when returning.
- The 1 IL definition, 1 Location, and 1 NativeLocation approach does
not remove the need for special casing PairLocations in the machine
code generation because they are 1 Location belonging to 1 definition.
Because of the amount of corner cases in the calling conventions that
need to be covered, the tests are generated, rather than hand-written.
ABIs tested on CQ: x64 (Linux, MacOS, Windows), ia32 (Linux, Windows),
arm (Android softFP, Linux hardFP), arm64 Android.
ABIs tested locally through Flutter: ia32 Android (emulator), x64 iOS
(simulator), arm64 iOS.
ABIs not tested: arm iOS.
TEST=runtime/bin/ffi_test/ffi_test_functions_generated.cc
TEST=runtime/bin/ffi_test/ffi_test_functions.cc
TEST=tests/{ffi,ffi_2}/function_structs_by_value_generated_test.dart
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_generated_tes
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_test.dart
TEST=tests/{ffi,ffi_2}/vmspecific_static_checks_test.dart
Closes https://github.com/dart-lang/sdk/issues/36730.
Change-Id: I474d3a4ee1faadbe767ddadd1b696e24d8dc364c
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140290
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
When loading a late field without initializer, we check if it was not
initialized and throw an exception. This change adds slow path (shared
if supported) for throwing LateInitializationError instead of calling
an initializer which would throw an exception. This improves register
allocation around accesses to such late fields.
In addition, this CL removes duplicated code for generating shared
stubs.
Size of Flutter gallery snapshot on armv8 -0.45%
(gzip -0.34%, brotli -0.41%)
Null-safe version of ParticleSystemUpdate benchmark on x64
JIT +9-17%, AOT +3-4%.
Change-Id: Ia12c1efec0871a2fd7acc6db75f255853af52d0f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168993
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is the first step towards optimizing typed data array allocations,
which is useful for e.g. package:vector_math.
Opaque call to a typed data factory constructor is replaced with IL
instruction which doesn't have side-effects and doesn't prevent
other optimizations (e.g. load forwarding) from happening.
Asm intrinsics for typed data factories are replaced with stubs.
Issue: https://github.com/dart-lang/sdk/issues/43228
Change-Id: Icc7d4863f2f94ea5d8bb4620d041f7b6fe664098
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161840
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
#if defined(DART_PRECOMPILED_RUNTIME)
#error "AOT runtime should not use compiler sources (including header files)"
#endif // defined(DART_PRECOMPILED_RUNTIME)
Remove #if defined(DART_PRECOMPILED_RUNTIME) from most compiler sources.
Change-Id: Id175c83fdbea38d9d5e1371ff433e3888f2afe8a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143523
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.
Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.
This change tries to address this inconsistency.
A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.
Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Ryan Macnak