Instead of switching over each of the different callback types and duplicating the call-dart-spill-call-runtime-restore in the stub, have the initial runtime call return a function pointer for the runtime call to end that kind of callback. A separate case for tail calls is still needed.
TEST=ci
Change-Id: I691487ca12337ddef1adaeec4c163c56e2feb09e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/488261
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Adds a new code path for NativeCallable.isolateLocal invocations. If
the current thread is not entered into any isolate, but owns the
target isolate, then it enters the target isolate, invokes, then exits
the isolate.
Fixes: https://github.com/dart-lang/sdk/issues/61623
TEST=tests/ffi/function_callbacks_isolate_ownership_test.dart
Change-Id: I401f185fadf7d2a55190dafd15387e1c418c67c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/452380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Thread::OffsetFromThread from runtime_api.cc assumed that all fields
between leaf and non-leaf runtime entries in Thread have word size,
so the delta between runtime entries can be translated from host to
target by multiplying by the ratio between target and host word size.
Since leaf and non-leaf runtime entries are not co-located in Thread,
this assumption is incorrect. It is now fixed by using separate
bases when calculating offsets of leaf and non-leaf runtime entries.
Also, add FatalError runtime entry which is useful for
bootstrapping of the new compiler.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: Iea4762b21285aa6423876b34328c863227ee1434
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/467900
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
- Reduce header surface area.
- Remove runtime_entry_<arch>.cc, which no longer include code generation.
- Fix signature of DLRT_Enter/ExitSafepoint.
- Give non-leaf runtime functions C linkage, as C++ does not have a defined ABI.
TEST=ci
Change-Id: I21e21a50ad9e4f9b32b1204d755bfa69a2ae9b17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/430981
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Make the free and allocated lists simply linked lists of `Metadata*`
(instead of `Metadata*` pointing to Trampoline, which - via some
logic - can be translated into next `Metadata*`).
Make the layout of the virtual address space mapping and
offsets `constexpr` functions instead of computing them at
runtime & caching in fields.
Use `uword` to represent `Trampoline` entrypoint (as we generally
use `uword` for `Code.EntryPoint()` / ...)
TEST=ci
Change-Id: If4ffa11712acc46c9295b609caff7576d2354fe4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/305983
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This is the same metadata system that will support async callbacks. The
main thing this does is take the trampolines that used to be JIT only,
and use them in AOT too. This is partly a safety thing (there's some
extra checks that used to be skipped on AOT), but mostly just so that
the metadata system is unified between the sync and async callbacks.
More details about the design:
https://docs.google.com/document/d/1QDjyY_6wOTOgURwpeYMKU9qEz0gKxx2MUrdruC6Kp6c/edit?usp=sharing
I split this off the async CL. Some of the comments refer to async
stuff that doesn't exist yet, but it's coming immediately after this so
I didn't update them.
Change-Id: Icd5e86934ee9ae34c2c0e2ed2bbd1b928a7184ac
TEST=ffi_callback_metadata_test.cc and CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302903
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Fixes SIMD registers clobbered by write barrier on ARM64.
- Moves code generation out of non-compiler directory.
- Removes unnecessary building of Dart frames on leaf runtime calls.
- Removes unnecessary Threads slots for write barrier Code objects.
- Removes duplicate saves of SP in leaf runtime calls on ARM64 and RISC-V.
- Avoids some redundant SP updates on RISC-V.
TEST=ci
Change-Id: Idb92127658edc90b320923ef3d882a7219a450ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236842
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit b5922e6db4.
Reason for revert: vm/cc/AllocateZone fails on Mac
Original change's description:
> [vm] Streamline Zones.
>
> - Don't put a HANDLESCOPE right after a StackZone, as there's no state worth restoring
> - Remove remaining zone bookkeeping dead since e3a9d70591.
> - Remove unnecessary resetting of handle blocks before zone deletion.
> - Shrink initial chunk size to make Zone's overall size fit in a faster malloc size class.
>
> TEST=ci
> Change-Id: I920e12f5af93aa71fb876435764c60df485215f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212261
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
TBR=rmacnak@google.com,asiva@google.com
Change-Id: Id23dcebb781791616e48e54d2dcd7e58159178bc
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215943
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Don't put a HANDLESCOPE right after a StackZone, as there's no state worth restoring
- Remove remaining zone bookkeeping dead since e3a9d70591.
- Remove unnecessary resetting of handle blocks before zone deletion.
- Shrink initial chunk size to make Zone's overall size fit in a faster malloc size class.
TEST=ci
Change-Id: I920e12f5af93aa71fb876435764c60df485215f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212261
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
To avoid accidental code that calls stubs without emitting proper
(deopt-id, deopt-env) metadata, we enforce that it's always present for
stub calls and add another GenerateNonLazyDeoptableStubCall for cases
where it's intentionally omitted.
The environment has in many cases been still emitted before, due to the
usage of `pending_deoptimization_env` though we make code pass it
explicitly (just as the deopt-id). We may want to consider deprecating
this `pending_deoptimization_env`.
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=Existing test suite.
Change-Id: I93f1d5ba4d74da5f9afa4b526ad57b9d032ca99e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197164
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
The "RunWithMutatorsStopped" wrapper is used in places where the invoked
callback can cause deoptimization of code. It should therefore ensure
it's running with a "deopt safpoint operation scope" to ensure mutators
are stopped at well-defined places that allow lazy-deopt.
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=Existing code base, will add "fuzzer"-like test to nightly/weekend builders.
Change-Id: Icb9a4183c13fab0f084e481c10dfc56a0308126a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197162
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As a pre-requisite of partitioning our safepoints into "gc only
safepoints" and "lazy-deopt'able safepoints" we'll let mutators keep
track of whether a runtime call into the VM is deoptable.
A mutator is lazy-deopt'able except for one particular case: When the
mutator calls into the VM from a place where no deopt-id/deopt-env
gets recorded (e.g. StoreInstanceField's box allocation goes to
runtime).
If a mutator is in the VM in a non-lazy-deopt'able state, it cannot call
out to the embedder, transition from "vm" to "native" state, ...
A future change will make new non-lazy-deopt'able runtime entries taking
advantage of this CL.
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=Existing test suite & tests in future CLs.
Change-Id: I30dbee02dc839b2ba9632b20ffa233aff4d58f64
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196922
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
The deoptimization environment of AssertAssignable should not have it's
inputs in it, since those get poped before the call to the
TypeTestingStub.
A lazy deopt should therefore continue in unoptimized code after the TTS
call, without the inputs.
This was discovered by running many lightweight isolates concurrently on
the JIT.
To simplify a regression test we'll add new VM flags that can trigger
deoptimization on any runtime call (optionally filtered by name of
runtime call).
We can use this mechanism to possibly discover more bugs of the same
kind.
Closes https://github.com/dart-lang/sdk/issues/45207
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=runtime/tests/vm/dart{,_2}/regress_45207_test
Change-Id: Ib713fa8be5914eaf4c7395d2e1d8dd88714f1ce2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189202
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
To avoid a O(n) traversal of isolates in order to find pending deopts,
we'll move the pending deopt state to the Thread. It's really a property
of a thread: when the thread's stack gets unwound - via return or
exceptional flow - we remove pending deopts.
This CL also encapsulates the pending deopt logic into a [PendingDeopts]
class - thereby avoiding logic around manipulating this array being
spread around. It simplifies the already very big [Isolate] class.
TEST=Refactoring of existing code.
Issue https://github.com/dart-lang/sdk/issues/36097
Closes https://github.com/dart-lang/sdk/issues/45168
Change-Id: I5a9b18c577f21a0f2a88332b1bc26641ec39d2af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188521
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
The background compiler can cause finalization of classes which can
cause deoptimization of code.
Any deoptimization will iterate over all mutator threads and marks
frames on the stack to be deoptimized. This requires saving old
(fp, pc) state. This state is currently saved on the current isolate's
pending deopt array (via `Thread::Current()->isolate()->AddPendingDeopt()`)
instead of the isolate whose stack got walked.
As part of making the compiler independent of `Isolate::Current()` we
pass the isolate explicitly.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing code.
Change-Id: I9327f3b76981fc16c1879a873edf37df1cbdd8bf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182380
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
On simulated architectures (simarm, simarm64) machine code checks stack
overflow of the simulated (arm/arm64) stack. However, it is also
possible to overflow C++ stack used by simulator and runtime if runtime
code is called recursively.
This change adds C++ stack overflow check on entry into the runtime
code (on simulated architectures).
Fixes https://github.com/dart-lang/sdk/issues/42936
Change-Id: If8f9388a1edc06044402f052c1d10d79063477e9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157103
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.
Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.
This change tries to address this inconsistency.
A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.
Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
--verify_gc_contains is roughly O(heap^2) and gives false errors in some cases involving remembered cards
--verify_on_transition takes over 30 minutes for even hello world
Change-Id: Idee7e535bb96b11f0fa620532debbd21ca9d7135
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123329
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
The original revision is in patchset 1.
Three bugs are fixed:
1. Fix SIMARM_X64 build: no need to generate trampolines for AOT or simulated JIT.
2. Hot-reload: Fix hot-reload: don't invalidate Code for force-optimized functions.
3. Windows: Provide shadow space to runtime routines.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-mac-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm_x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-win-release-x64-try
Change-Id: I326009cfacb51a84e9de4ddf9ff2d6d415460f91
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/113829
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is due to how Windows destroys objects as destroyed objects have
their vtables reset to the vtable of the base class. If this happens
when an isolate hasn't finished shutting down and a virtual function
which is overriding a pure virtual function from the base class is
called, _purecall() is invoked and attempts to call exit() again which
leads to a deadlock.
See issue #35855 for more details.
Change-Id: I93a0e00549e7ee87a717a6cde01bf0848050384d
Reviewed-on: https://dart-review.googlesource.com/c/92000
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
This is the next step towards preventing compiler from directly peeking
into runtime and instead interact with runtime through a well defined
surface. The goal of the refactoring to locate all places where compiler
accesses some runtime information and partion those accesses into two
categories:
- creating objects in the host runtime (e.g. allocating strings, numbers, etc)
during compilation;
- accessing properties of the target runtime (e.g. offsets of fields) to
embed those into the generated code;
This change introduces dart::compiler and dart::compiler::target namespaces.
All code in the compiler will gradually be moved into dart::compiler namespace.
One of the motivations for this change is to be able to prevent access to
globally defined host constants like kWordSize by shadowing them in the
dart::compiler namespace.
The nested namespace dart::compiler::target hosts all information about
target runtime that compiler could access, e.g. compiler::target::kWordSize
defines word size of the target which will eventually be made different
from the host kWordSize (defined by dart::kWordSize).
The API for compiler to runtime interaction is placed into compiler_api.h.
Note that we still permit runtime to access compiler internals directly -
this is not going to be decoupled as part of this work.
Issue https://github.com/dart-lang/sdk/issues/31709
Change-Id: If4396d295879391becfa6c38d4802bbff81f5b20
Reviewed-on: https://dart-review.googlesource.com/c/90242
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
- Remove dead Isolate::IsTopLevelParsing.
- Remove dead TraceFunctionEntry/Exit.
- Migrate more tracing flags to flag_list.h to allow code elimination in non-debug modes.
- Require an explicit Zone parameter for CheckedHandles.
- Remove side-effects from RuntimeEntry constructor.
out/ProductX64/exe.stripped/dart_precompiled_runtime 4063464 -> 4046952 (-16k)
Change-Id: Iffd3de25a03d2354cdecf2d79aa761c33ab08bd3
Reviewed-on: https://dart-review.googlesource.com/c/83120
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Ryan Macnak