- Updated conditional compilation flags throughout the runtime codebase to transition from DART_DYNAMIC_MODULES to DART_BYTECODE_INTERPRETER.
- Adjusted logic in various files including object_graph_copy.cc, object_reload.cc, profiler.cc, and others to ensure compatibility with the new interpreter model.
- Ensured that all references to dynamic modules are replaced with bytecode interpreter checks, maintaining functionality for interpreted code execution.
- Modified stack frame handling and service-related code to align with the new interpreter architecture.
- Updated tests and service implementations to reflect the changes in the runtime environment.
Signed-off-by: Tony <tonylu@tony-cloud.com>
Thread::saved_shadow_call_stack_ has the value for the top-most Dart entry, and needs to be saved and restored for nested Dart entries.
TEST=fuchsia
Change-Id: Id035bec7e798458fdfcf85678a7246d5f7eef850
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/464603
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
HWASAN tags stack regions (and stack pointers) with 0,
but also selectively tags some stack allocations with
non-zero tags. These tags need to be cleared when
allocation goes out of scope so that stack can be
later safely reused - otherwise stale tags confuse
HWASAN instrumentation later.
This CL teaches Exceptions::JumpToFrame to do that.
CL also adds basic build changes to support building
with HWASAN, though I don't wire it into CI for now
as it can only really run on Android ARM64
TEST=manually on Android phone and on internal tests
Bug: b/374433249
Change-Id: I678443dc1de693c999a226bee0d71a6f3582d4f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421180
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Auto-Submit: Slava Egorov <vegorov@google.com>
This should help speeding up new isolate creation (or running dart code on mutator thread when it is supported).
TEST=ci
Change-Id: I88b5deeb33e72694cb3e2727cf5668ce15fd8c6b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420361
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Change 8085a97a63 moved commonly
invoked StackResource::Unwind(...) from several different code
paths (simulators & interpreter) into Exceptions::JumpToFrame.
However the change did not account for the order of operations
in Exceptions::JumpToFrame which broke interpreter. This CL
fixes that by moving Interpreter::JumpToFrame under
StackResource::Unwind(...).
We also forgot to remove StackResource::Unwind from RISC-V
simulator - but that's benign because calling it the second
time simply does nothing.
TEST=vm-aot-dyn-linux-debug-x64
Change-Id: Ia93fc6997b768180fbbb5e54bb0c0fd5c713a244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/421060
Commit-Queue: Slava Egorov <vegorov@google.com>
Auto-Submit: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Instead of handling FFI related execution state and safepoint
in assembly handle it in runtime code.
The transition needs to be done before JumpToFrame unwinds
stack because unwinding destroys exit frame and this can't
be done at safepoint as GC might be traversing the stack.
An incorrect order of operation was manifesting as crashes in
GC when one isolate in a group was encountering a lot of
exceptions thrown from an FFI call and another isolate is
triggering GCs.
To catch this in the future added a bit of validation to
ExitSafepoint runtime call which triggers when --use-slow-path
is enabled. Though after refactoring this code does not
trigger this code path anymore because it was completely
removed - but it is better than nothing.
This CL also removes a lot of unnecessary complexity which
was associated with handling this transition in the stub
itself.
TEST=ffi/vmspecific_handle_test
Bug: b/408377905
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-mac-debug-simarm64_arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Change-Id: Ia073cb6bb9e1b5a0ea8514c7e048cee6019b84d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420324
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
For out-of-memory events vm reserves (and if used, then replenishes) small chunk of memory for handling those, so preallocation of oom and stacktrace is not needed.
Unwind error can be created once as part of vm launch, kept in vm isolate.
This should reduce overhead of starting new isolate.
TEST=ci
Change-Id: Ibe2c85f250a521a8e838cc5690984fe9fa287a98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/419261
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Before this change, an error reaching an FFI callback would attempt to execute the normal invocation stub from the beginning in the FFI callback's frame, which quickly crashes. After this change, the runtime recognizes this marker use of the invocation stub and returns to the FFI callback function instead.
TEST=ffi/unwind
Bug: https://github.com/dart-lang/sdk/issues/39487
Change-Id: I477cfcfc236e6cf518ebfe52860ba49e466ebf8b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/409562
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Also removed a bunch of `CheckNotNullable`s that shouldn't be necessary any more. Any remaining non-sound-null-safety code runs today, and no more should be written. (And if it is, it'll mostly just err somewhere else, with a worse error message.)
Tested: New test added. Removed older tests checking for unsound null-safety.
Change-Id: I28626909cd8c1f91db6c61fc2b93042ed1b085dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380780
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
Since null safety, all static fields with initializers are implicitly
late. This change cleans up transition_sentinel which was used in
the detection of cyclic initialization of legacy static fields.
TEST=ci
Change-Id: I6a990dc8ba030f5bd40eb0b86706cbfb0f725e33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/373520
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Instead of having special support for targeting catch entry as a
lazy deoptimization target, split this into two independent
operations:
* Perform lazy deoptimization at call.
* Rethrow the exception once deoptimization is complete.
TEST=ci
Change-Id: Ib275d615bc6717e1184e88602ae5459d5b771f2b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361580
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This reverts commit 63179aa49d.
See commit 1ab083ae48 for the original
description of the change.
The original change was reverted because it did not take
into account that in Dart it is legal to make some
parameters optional when you override a method. Such parameters
should not be passed in registers - which means for each
possible interface target we need to compute the minimum
number of direct parameters accepted by all overrides.
I have also added two comprehensive tests for different
combinations of direct, optional and named parameters.
TEST=vm/dart/calling_conventions_test.dart,vm/dart/calling_conventions2_test.dart
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try
Change-Id: Ib76e2512d3e4077a40265d315483ebe3b0f5551c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357862
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Following the lead of unboxed calling conventions augment
out backend to support passing arguments in registers on
ARM, ARM64 and X64.
Similar to unboxing the calling convention change only applies
to functions which are invoked statically or through
global dispatch table. Functions which are invoked from native
code or directly invoked by dynamic call sites are not allowed
to use register calling convention.
This CL avoid messing too much with call instructions and
instead leans on adjusting existing machinery: MoveArgument
instructions corresponding to register arguments are simply
detached from the graph and are processed as input constraints
by the register allocator.
The computation of the calling convention is centralized in
`ComputeCallingConvention` in `flow_graph.cc` and is used
by callees and callers.
While working on this CL I have also discovered an existing bug
in catch move handling (see added test runtime/tests/vm/dart/try_catch_parameter_sync_test.dart).
On a large Flutter app this decreases the size of generated code
by 1.8% (380Kb) and there a mixed bag of improvements in runing time, e.g.
```
build_bench (Pixel 6a)
-3.536% (0.2 noise)423.16 438.68
animation_bench_close_first_frame (Pixel 6a)
-3.717% (0.0 noise)253.64 263.43
rrect_contains_bench (Pixel 6a)
-3.796% (0.0 noise)0.088700 0.092200
sync_star_sync_star (Pixel 6a)
-5.039% (0.0 noise)2.4310 2.5600
sync_star_list (Pixel 6a)
-5.329% (0.0 noise)1.0660 1.1260
velocity_tracker_bench (Pixel 6a)
-5.731% (0.1 noise)62.039 65.811
sync_star_generated (Pixel 6a)
-6.438% (0.0 noise)1.090 01.1650
matrix_utils_transform_point_perspective (Pixel 6a)
-6.844% (0.2 noise)0.012520 0.013439
matrix_utils_transform_rect_perspective (Pixel 6a)
-7.818% (0.2 noise)0.028117 0.030502
matrix_utils_transform_rect_affine
-27.05% (0.7 noise) 0.017494 0.023981
```
TEST=ci
Issue https://github.com/dart-lang/sdk/issues/54955
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try
Change-Id: I9a134a764666a541d4693e6ca9f6813025ecd63e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/353404
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This adds the `close()` method to `DynamicLibrary`, which uses
`dlclose()` on Unix and `FreeLibrary` on Windows to close a dynamic FFI
library.
TEST=tests/ffi/dylib_close_test.dart
Closes https://github.com/dart-lang/sdk/issues/40159
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,dart2wasm-linux-d8-try,dart2wasm-linux-chrome-try,vm-reload-rollback-linux-debug-x64-try,vm-reload-linux-debug-x64-try,vm-win-debug-x64c-try,vm-win-debug-x64-try,vm-aot-android-release-arm_x64-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-asan-linux-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try
CoreLibraryReviewExempt: `dart:ffi` VM only API.
Change-Id: I73af98677b481902fe548bdcee56964a0195faf0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307680
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This deprecated class cannot be removed just yet because
we still need to throw something in case of cyclic initialization
in Dart legacy libraries (which are still supported).
Class CyclicInitializationError is copied for each implementation
(VM, dart2js and DDC) and made private. Each implementation now has
an independent private class which can be removed separately.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49529
Change-Id: I8100bbe16636c12c4cbabbb5fe770f4c648c4249
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275120
Commit-Queue: Michael Thomsen <mit@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Adds a stack map to the catch block entry pc.
The alternative would be to prevent yielding to a safepoint between these two point. For throws coming from Dart or the embedding API, this is possible by unwinding the transitions before the first point and doing only a non-yielding VM->Generated between the two points. For FFI, this does not work because the final state is Native instead of Generated, and VM->Native is a blocking transition.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/48668
Change-Id: I8164fddae018ed95d9147fafa65785b7638e5301
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269746
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
The error has not been thrown since Dart 2.0,
where being able to reach the end of a switch case
became a compile-time error.
TEST=Removes tests depending on discontinued behavior.
Change-Id: I76292e7c73f2b3aaf071bbb290e97db493b75477
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261860
Reviewed-by: Michael Thomsen <mit@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
Auto-Submit: Lasse Nielsen <lrn@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.
Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:
* Suspending variable-length frames (for unoptimized code).
* Handling of Code and pool pointers in Dart stack frames.
* OSR.
* Deoptimization.
* Hot reload.
* Debugger.
The new implementation is not enabled in JIT mode yet.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
When ffi code calls Dart code that requests the isolate to exit(via Isolate.exit, for example), we ensure that isolate indeed exits on return from ffi call.
This is implemented by introduction of new safepoint bit which, once set, forces ExitSafepoint to propagate unwind error.
TEST=isolate_exit_sandwich_test
Change-Id: I2e8f5ecec7f4e59ae5f99b9525cc566f20d4b6a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219846
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
TSAN instruments C++ code by adding prologue/epilogue code which
maintains a shadow stack. Using setjmp()/longjmp() is intercepted by
TSAN and correspondingly unwinds the shadow stack.
When Dart VM throws exceptions we call the JumpToFrame stub
from C++ which will directly reset the stack to the exception handler
catch entry. This leaves the TSAN shadow stack unchanged.
This means whenever an exception is thrown we leak frames in TSAN's
shadow stack. Due to using a fixed-size shadow stack, it will cause a
buffer-overflow in TSAN when too many such frame leaks happen. This can
cause arbitrary memory to be overriden, leading to awkward crashes.
This is especially an issue on the "iso-stres" builder because it
launches - in the same process - *many* small tests, more easily hitting
that limit.
This CL will workaround the TSAN issue by making runtime call save it's
state via setjmp() and make exception throughing process go via
longjmp() (which TSAN will intercept) before actually calling the
JumpToFrame stub.
=> This will ensure the TSAN shadow stack is correctly maintained.
The [jmp_buf]'s encoding of register state is non-trivial (e.g. it uses
XOR'ing of the actual saved state under certain glibc versions). So we
store any state we need to pass to the target of the `longjmp()` on the
[Thread] instead of overriding the [jmp_buf]s register state with the
arguments.
Issue https://github.com/dart-lang/sdk/issues/47472#issuecomment-948235479
TEST=vm/dart{,_2}/regress47472_test.dart
Change-Id: Ifbf6580aa15bcce54d0584cdc3cd18cc19be0a9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222300
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
TSAN instruments C++ code by adding prologue/epilogue code which
maintains a shadow stack. Using setjmp()/longjmp() is intercepted by
TSAN and correspondingly unwinds the shadow stack.
When Dart VM throws exceptions we call the JumpToFrame stub
from C++ which will directly reset the stack to the exception handler
catch entry. This leaves the TSAN shadow stack unchanged.
This means whenever an exception is thrown we leak frames in TSAN's
shadow stack. Due to using a fixed-size shadow stack, it will cause a
buffer-overflow in TSAN when too many such frame leaks happen. This can
cause arbitrary memory to be overriden, leading to awkward crashes.
This is especially an issue on the "iso-stres" builder because it
launches - in the same process - *many* small tests, more easily hitting
that limit.
This CL will workaround the TSAN issue by making runtime call save it's
state via setjmp() and make exception throughing process go via
longjmp() (which TSAN will intercept) before actually calling the
JumpToFrame stub.
=> This will ensure the TSAN shadow stack is correctly maintained.
The [jmp_buf]'s encoding of register state is non-trivial (e.g. it uses
XOR'ing of the actual saved state under certain glibc versions). So we
store any state we need to pass to the target of the `longjmp()` on the
[Thread] instead of overriding the [jmp_buf]s register state with the
arguments.
Issue https://github.com/dart-lang/sdk/issues/47472#issuecomment-948235479
TEST=vm/dart{,_2}/regress47472_test.dart
Change-Id: I4b6f0d4eacef85487c55999021d72a6d932facad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221083
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Tony