The operand size is dropped from Address/FieldAddress constructors on
arm64. This makes code using Address/FieldAddress less fragile, avoids
repeating size both in the Address/FieldAddress and load/store
instructions, and better aligns arm64 Address/FieldAddress with other
architectures.
TEST=ci
Change-Id: I92d7c5c8f0239333f022deebc0472136018bb0fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260072
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.
The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.
Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.
If reload succeeds we drop original class table and
set program class table as heap walk one.
If reload fails we drop the program class table and
restore original one from heap walk table.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
In tsan mode, the load-acquire/store-release assembler implementation
will make a runtime call to let TSAN know about the operation.
In order to perform the runtime call the THR register needs to be
populated, which it wasn't.
TEST=Fixes crashing vm/cc/StoreReleaseLoadAcquire tests in tsan mode
Change-Id: I1f45fee9c8e40b8069bf32b13caa97a02f13a6a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256261
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.
Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:
* Suspending variable-length frames (for unoptimized code).
* Handling of Code and pool pointers in Dart stack frames.
* OSR.
* Deoptimization.
* Hot reload.
* Debugger.
The new implementation is not enabled in JIT mode yet.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The CCallInstr takes unboxed word-sized integers and does a call to C
without any VM transitions.
This can be used for the following.
1) Leaf runtime entries. This CL removes the EnterHandleScope,
ExitHandleScope, and AllocateHandle instructions.
2) Compilation of isolate independent code where all definitions in IL
are unboxed integers.
Currently, the CCallInstr expects all it's arguments and the return type
to be word-sized.
This CL also:
- Adds loadImmediate and CCall with a register for each assembler.
- Adds a CSE pass to force-optimized so that LoadThreadInstr gets de-
duplicated.
TEST=tests/ffi(_2)/*
Change-Id: I67c6aaa1b7d6aa7d60e274477686a54e5a331216
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
(Macro-)assembler helper PushRegistersInOrder generates series of
register pushes in an architecture-specific way:
* Using STMDB on ARM (if possible).
* Using STP on ARM64.
* Updating SP only once on RISC-V.
TEST=ci
Change-Id: I6cbd1dd5f8acf8ce8087ed899a311264a873ad37
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
1/2 or 1/4 branches executed per object initialization for uncompressed and compressed pointers respectively.
Note ARM32 already has double-wide initialization.
TEST=ci
Change-Id: Iacc61cac39f74191bb2e5e06cd48b90c006b8585
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241622
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Adds an additional `contains_call` mode specifying that the
register allocator should block all non-callee-save ABI registers.
As opposed to `kCall`, this frees the use of callee-save registers
across FfiCallInstr, meaning that live values do not always have
to be spilled.
For architectures with sufficient free callee-save registers
this should result in less overhead for each FfiCall.
This change also removes RequiresStackSlot as it was only being
used for FfiCall's TypedData, and wasn't implemented in the
Register Allocator.
Since FfiCall no longer blocks all registers TypedData can now
live in a register, so the previous assumptions about it always
being on stack have to be relaxed.
This change also works around an issue where the register allocator
might allocate an FPU register that does not have an associated S
register. This causes problems when FFI tries to pass 32 bit floats
in said register.
As a work around we block all FPU regs. above the 8 that have S regs.
for FFI calls in VFPv3-D32 mode.
This change also adds an additional temp registers which was previous
used but not being reserved in FfiCall IL on Arm64.
Some additional prereq. fixes were branched out into:
https://dart-review.googlesource.com/c/sdk/+/237690
TEST=Adds IRTest_FfiCallInstrLeafDoesntSpill.
Bug: https://github.com/dart-lang/sdk/issues/45468
Change-Id: Icdeccb0de77e46f5bc34dd5bd4e308d27bc1ef99
Cq-Do-Not-Cancel-Tryjobs: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-win-release-x64-try, vm-precomp-ffi-qemu-linux-release-arm-try, vm-kernel-mac-release-arm64-try, vm-kernel-nnbd-mac-debug-arm64-try, vm-kernel-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221624
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Check the right register set when deciding if there is a leftover pop single. This worked by accident on non-Fuchsia ARM64 because there is an even number for both CPU and FPU registers, but Fuchsia has a different number of preserved CPU registers due to the shadow call stack.
TEST=no Fuchsia ARM64 bots; inspect disassembly of the entry stub
Bug: b/226647006
Change-Id: Ie0c7fa4a3b713d5b9fb21e9e778d620c7728bc39
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239160
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Alexander Markov