In tsan mode, the load-acquire/store-release assembler implementation
will make a runtime call to let TSAN know about the operation.
In order to perform the runtime call the THR register needs to be
populated, which it wasn't.
TEST=Fixes crashing vm/cc/StoreReleaseLoadAcquire tests in tsan mode
Change-Id: I1f45fee9c8e40b8069bf32b13caa97a02f13a6a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256261
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.
Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:
* Suspending variable-length frames (for unoptimized code).
* Handling of Code and pool pointers in Dart stack frames.
* OSR.
* Deoptimization.
* Hot reload.
* Debugger.
The new implementation is not enabled in JIT mode yet.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The CCallInstr takes unboxed word-sized integers and does a call to C
without any VM transitions.
This can be used for the following.
1) Leaf runtime entries. This CL removes the EnterHandleScope,
ExitHandleScope, and AllocateHandle instructions.
2) Compilation of isolate independent code where all definitions in IL
are unboxed integers.
Currently, the CCallInstr expects all it's arguments and the return type
to be word-sized.
This CL also:
- Adds loadImmediate and CCall with a register for each assembler.
- Adds a CSE pass to force-optimized so that LoadThreadInstr gets de-
duplicated.
TEST=tests/ffi(_2)/*
Change-Id: I67c6aaa1b7d6aa7d60e274477686a54e5a331216
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
(Macro-)assembler helper PushRegistersInOrder generates series of
register pushes in an architecture-specific way:
* Using STMDB on ARM (if possible).
* Using STP on ARM64.
* Updating SP only once on RISC-V.
TEST=ci
Change-Id: I6cbd1dd5f8acf8ce8087ed899a311264a873ad37
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
1/2 or 1/4 branches executed per object initialization for uncompressed and compressed pointers respectively.
Note ARM32 already has double-wide initialization.
TEST=ci
Change-Id: Iacc61cac39f74191bb2e5e06cd48b90c006b8585
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241622
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Adds an additional `contains_call` mode specifying that the
register allocator should block all non-callee-save ABI registers.
As opposed to `kCall`, this frees the use of callee-save registers
across FfiCallInstr, meaning that live values do not always have
to be spilled.
For architectures with sufficient free callee-save registers
this should result in less overhead for each FfiCall.
This change also removes RequiresStackSlot as it was only being
used for FfiCall's TypedData, and wasn't implemented in the
Register Allocator.
Since FfiCall no longer blocks all registers TypedData can now
live in a register, so the previous assumptions about it always
being on stack have to be relaxed.
This change also works around an issue where the register allocator
might allocate an FPU register that does not have an associated S
register. This causes problems when FFI tries to pass 32 bit floats
in said register.
As a work around we block all FPU regs. above the 8 that have S regs.
for FFI calls in VFPv3-D32 mode.
This change also adds an additional temp registers which was previous
used but not being reserved in FfiCall IL on Arm64.
Some additional prereq. fixes were branched out into:
https://dart-review.googlesource.com/c/sdk/+/237690
TEST=Adds IRTest_FfiCallInstrLeafDoesntSpill.
Bug: https://github.com/dart-lang/sdk/issues/45468
Change-Id: Icdeccb0de77e46f5bc34dd5bd4e308d27bc1ef99
Cq-Do-Not-Cancel-Tryjobs: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-win-release-x64-try, vm-precomp-ffi-qemu-linux-release-arm-try, vm-kernel-mac-release-arm64-try, vm-kernel-nnbd-mac-debug-arm64-try, vm-kernel-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221624
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Check the right register set when deciding if there is a leftover pop single. This worked by accident on non-Fuchsia ARM64 because there is an even number for both CPU and FPU registers, but Fuchsia has a different number of preserved CPU registers due to the shadow call stack.
TEST=no Fuchsia ARM64 bots; inspect disassembly of the entry stub
Bug: b/226647006
Change-Id: Ie0c7fa4a3b713d5b9fb21e9e778d620c7728bc39
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239160
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Fixes SIMD registers clobbered by write barrier on ARM64.
- Moves code generation out of non-compiler directory.
- Removes unnecessary building of Dart frames on leaf runtime calls.
- Removes unnecessary Threads slots for write barrier Code objects.
- Removes duplicate saves of SP in leaf runtime calls on ARM64 and RISC-V.
- Avoids some redundant SP updates on RISC-V.
TEST=ci
Change-Id: Idb92127658edc90b320923ef3d882a7219a450ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236842
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This change fixes a bug in the IA32 FG Moves, where any moves
of Int8 and Uint8 from a register other than the first four general
purpose registers would be bogus.
The moves would generate to move the second byte of the first four
GP registers because the other four registers do not have byte
registers.
This change also fixes an issue where X64 byte register moves would
generate incorrect bytecode for otherwise correct instruction prints.
It would for instance print "movb [rsp],rdi", but emit "movb [rsp],bh".
This change also fixes a bug in NativeFpuRegistersLocation::Equals
that cause different FPU regs. to compare equal despite being different
kinds.
This in turn caused moves between these different regs. to be dropped
because they looked like no-ops.
See also:
https://dart-review.googlesource.com/c/sdk/+/221624
TEST=Existing.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-win-release-ia32-try
Change-Id: I09e69295c8a522d74f8a8fd817188b4de656c619
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/237690
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
* Use CompilerPass::GenerateCode in AOT to ensure we print
the graph if the code generation crashes.
* Enable Dart_DumpNativeStackTrace in gen_snapshot even in
PRODUCT builds.
Additionally enable disassembler in gen_snapshot even in
PRODUCT builds.
TEST=tested manually
Change-Id: I951d62ca07c7882fa1d8e765d4b5ffd26edd9456
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233880
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Martin Kustermann