Instead of handling FFI related execution state and safepoint
in assembly handle it in runtime code.
The transition needs to be done before JumpToFrame unwinds
stack because unwinding destroys exit frame and this can't
be done at safepoint as GC might be traversing the stack.
An incorrect order of operation was manifesting as crashes in
GC when one isolate in a group was encountering a lot of
exceptions thrown from an FFI call and another isolate is
triggering GCs.
To catch this in the future added a bit of validation to
ExitSafepoint runtime call which triggers when --use-slow-path
is enabled. Though after refactoring this code does not
trigger this code path anymore because it was completely
removed - but it is better than nothing.
This CL also removes a lot of unnecessary complexity which
was associated with handling this transition in the stub
itself.
TEST=ffi/vmspecific_handle_test
Bug: b/408377905
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-mac-debug-simarm64_arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Change-Id: Ia073cb6bb9e1b5a0ea8514c7e048cee6019b84d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420324
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
On architectures with a weak memory model, there is the possibility that the concurrent marker can see the publishing store of a new object before it sees the initializing store of that object's header. On an M1, the barrier to prevent reordering these stores is fairly cheap, so we emit this barrier on Mac/iOS ARM64. Otherwise, this barrier is very expensive (or at least expensive for some hardware within the ABI), so instead we avoid the race by deferring marking of objects inside an active TLAB.
Disable TSAN instrumentation on the marker setting the mark bit, as TSAN does not understand fences.
TEST=ooo arm64 machines
Bug: https://github.com/dart-lang/sdk/issues/56845
Change-Id: I0676661a7cf941fdc6b451e516d890c26826bb3b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/389265
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
In 67f93d3840, the simulator was extended to simulate signal handlers clobbering below CSP, but Simulator::JumpToFrame didn't kick CSP back near the stack limit.
TEST=fuzzer (anything with exceptions in debug simarm64)
Change-Id: I233388c9a73181521d6760e7a2f9c9dfe57e33ef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238463
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Fixes SIMD registers clobbered by write barrier on ARM64.
- Moves code generation out of non-compiler directory.
- Removes unnecessary building of Dart frames on leaf runtime calls.
- Removes unnecessary Threads slots for write barrier Code objects.
- Removes duplicate saves of SP in leaf runtime calls on ARM64 and RISC-V.
- Avoids some redundant SP updates on RISC-V.
TEST=ci
Change-Id: Idb92127658edc90b320923ef3d882a7219a450ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236842
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Every AOT user out there has been using bare instructions mode
and continuing to maintaining non-bare instructions mode simply
adds costs (both in terms of time spent making changes to work
in a mode that is not used and CI resources spent on testing it).
This change removes FLAG_use_bare_instructions and changes the code
to assume that FLAG_precompiled_mode implies bare instructions.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try
Change-Id: I5032b13bfcb613f79865f2cfa139cca8d1b42556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220964
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
1) double.truncate() now simply calls toInt(), omitting
truncateToDouble() call.
2) double.floor() and ceil() are now intrinsified on arm64 and AOT/x64
and generated using DoubleToInteger instruction.
3) DoubleToInteger instruction is extended to support floor and ceil.
On arm64 DoubleToInteger is implemented using fcvtms and fcvtps
instructions. On x64 DoubleToInteger is implemented using roundsd
under a check if it is supported (with a fallback to a stub and a
runtime call).
AOT/x64:
Before: BenchFloor(RunTime): 318.82148549569655 us.
After: BenchFloor(RunTime): 133.29430189936687 us.
TEST=ci
Closes https://github.com/dart-lang/sdk/issues/46876
Closes https://github.com/dart-lang/sdk/issues/46650
Change-Id: I16ca18faf97954f8e8e25f0b72a2bbfac5bdc672
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212381
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This relands https://dart-review.googlesource.com/c/sdk/+/205633
but without renaming TARGET_OS_IPHONE to DART_TARGET_OS_IPHONE.
It also changes uses of TARGET_OS_IOS to
DART_TARGET_OS_MACOS_IOS to be consistent with the rest of the
VM.
TargetConditionals.h for XCode 13 defines several
TARGET_OS_* preprocessor symbols that confuse the
Dart build. There is probably a more targeted fix
for this, but renaming the symbols that Dart uses
will also prevent this problem if more symbols
are added to the platform headers in the future.
See: https://github.com/dart-lang/sdk/issues/46499
TEST=It builds.
Change-Id: Ie775c19dd23cfdf5f65e5ebc6ee4ec3a561676fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205860
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
TargetConditionals.h for XCode 13 defines several
TARGET_OS_* preprocessor symbols that confuse the
Dart build. There is probably a more targeted fix
for this, but renaming the symbols that Dart uses
will also prevent this problem if more symbols
are added to the platform headers in the future.
See: https://github.com/dart-lang/sdk/issues/46499
TEST=It builds.
Change-Id: I3b33a03b4a9a14b76d55fe12f8cdefec4b3c3664
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/205633
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
BARRIER_MASK requires 8 low bits and HEAP_BASE requires 32 high bits. Combine them into a single register HEAP_BITS, and use ARM's shifted operands to continue accessing the relevant bits in a single instruction.
TEST=ci
Change-Id: I5a4cd43fec2d19615239ec5cec2ac282d17c461c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198100
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
First, change LocationForPosition and TokenRangeAtLine to return
whether they successfully located the offset/line. Also change the
wrappers in Script (GetTokenLocation and TokenRangeAtLine) to return
whether or not any information was successfully located.
These methods now only change the out parameters in the case that
appropriate information was found, so any unconditional uses of the out
parameters need the out parameters to be appropriately initialized.
We now allow negative lines to be requested in TokenRangeAtLine (with
the failure behavior described above) so the line returned by
LocationAtPosition can be fed into TokenRangeAtLine without intermediate
checking.
The calculation of the token length, which uses the script source and
not the line starts array, has been moved to a new method,
Script::GetTokenLength. The new method returns the length (or a
negative value if the token length could not be determined) instead of
using an out parameter.
TEST=Existing tests on trybots, to ensure tests using current
line/number info aren't affected.
Bug: https://github.com/dart-lang/sdk/issues/44436
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try
Change-Id: Ibc048a226d11ff9a340a8d249654d07720fdf115
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175365
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Assembler now always knows whether LR contains return address
or can be safely cloberred.
TEST=tested through normal test suite
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-ffi-android-release-arm64-try,vm-ffi-android-release-arm-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64-try
Change-Id: I0b436509389cfd29ee413fd28930fb2d6b07616e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174468
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
When running the simulator on precompiled mode, source information
is lost, so only the filename is valid with the line and column
information printed as -1. For each frame, this CL adds the symbol name
of the instructions section and a hexadecimal offset from the start of
the section, which can be used with the native_stack_traces package. The
absolute address of the VM and isolate instructions are also included
at the start of the backtrace.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
Change-Id: Ie67f914b156ead48fccc3069ced6a26940b80673
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168822
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
In --enable-isolate group is
* turned off, then we have 1<->1 mapping between isolate and isolate
group, so moving the locks should have no impact
* turned on, then we have N<->1 mapping between isolates and isolate
group, mutators need to use the same locks to guarantee exclusive
access (e.g. to symbol table, type canonicalization, type arguments
canonicalization, ...)
This is a follow-up to share the program structure in AOT mode across
multiple isoltes.
In order to move the existing `Isolate::type_canonicalization_mutex_` to
`IsolateGroup` (and thereby make all mutators use the same lock), we need
to remove the `RunWithMutatorsStopped()` usage in the runtime entry.
=> Without this it can lead to deadlocks.
In order to remove usage of `RunWithMutatorsStopped()` we will change
access to the cached type arguments array to use load-acquire barriers
in generated code and store-release barriers in runtime code.
A nice side effect is that it reduces `sizeof(dart::Isolate)`.
Closes https://github.com/dart-lang/sdk/issues/41912
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ifd42691524fe41ffe8bb4e2623c5b8c1151de973
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148539
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Vyacheslav Egorov