Commit Graph

500 Commits

Author SHA1 Message Date
Ryan Macnak 0a6fa7ac03 [vm] Fix --gc_at_throw, etc for isolate group callbacks.
TEST=ffi/isolate_group_shared_callback_test
Change-Id: Ib4c241d59a36544495c264c51848753a0bc22bf4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/429460
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2025-05-19 14:07:50 -07:00
Alexander Aprelev 50e0e0d99d [vm/shared] Introduce NativeCallable.isolateGroupShared
This method allows for synchronous execution of dart callbacks from native code. The execution happens on dart mutator thread, from which dart code can only access isolate-group variables - those which are tagged with .

Bug: https://github.com/dart-lang/sdk/issues/54530
Bug: https://github.com/dart-lang/sdk/issues/56841
Change-Id: Ia1a6b01327be493f003f1eea82e558bb6b147dd3
CoreLibraryReviewExempt: only internal library change
TEST=isolate_group_shared_callback_test
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/422920
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2025-05-06 13:28:44 -07:00
Alexander Aprelev b32e5e5a91 [vm/shared] Throw AccessError when isolategroup mutator accesses static fields.
Sample snapshot size comparison before/after:

===
dart2js_aot.dart.snapshot before: 19946368
                          after:  19998800  (with flag turned on)
                          delta:     52432 0.26%
===

Bug: https://github.com/dart-lang/sdk/issues/54530
Bug: https://github.com/dart-lang/sdk/issues/56841
CoreLibraryReviewExempt: internal library change only
Change-Id: I34b1945c040bbad22cb3ec6fdb6e6776df31a82f
TEST=run_isolate_group_run_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/422360
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2025-05-06 10:42:44 -07:00
Alexander Aprelev fddb38ecab [vm/shared] Introduce IsolateGroup.runSync()
Allow dart code execution on mutator thread, do not require an isolate.

It moves some states that was kept on an isolate to thread or isolate group.

Bug: https://github.com/dart-lang/sdk/issues/54530
Bug: https://github.com/dart-lang/sdk/issues/56841

TEST=run_isolate_group_run_test

CoreLibraryReviewExempt: only internal library is being updated
Change-Id: I99df09e23954755387ea6230bfd166493d78e989
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/418503
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2025-05-05 14:52:07 -07:00
Ryan Macnak a9dd011f94 [vm] Make out-of-memory scenarios a bit more robust.
- Don't scavenge when there is no OOM reservation, so refilling the OOM reservation will get the first go once free space is available instead it being consumed by promotions.
 - Increase the desired free space in new space needed to continue scavenging. Since new space became paged, 1 KB / 64 pages only guarentees the ability to allocate a 16 byte object.
 - Stop issuing stress test reloads after an OOM is encountered.
 - Add a HANDLESCOPE to reload to make memory collectible sooner after an error.

TEST=vm/out_of_memory
Bug: https://github.com/dart-lang/sdk/issues/48920
Bug: https://github.com/dart-lang/sdk/issues/53455
Bug: https://github.com/dart-lang/sdk/issues/43642
Change-Id: Ife8bed52e341648a66f78d55bfa60b3944be5d46
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/422324
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2025-04-15 10:39:32 -07:00
Vyacheslav Egorov 8085a97a63 [vm] Fix JumpToFrame execution state transition
Instead of handling FFI related execution state and safepoint
in assembly handle it in runtime code.

The transition needs to be done before JumpToFrame unwinds
stack because unwinding destroys exit frame and this can't
be done at safepoint as GC might be traversing the stack.

An incorrect order of operation was manifesting as crashes in
GC when one isolate in a group was encountering a lot of
exceptions thrown from an FFI call and another isolate is
triggering GCs.

To catch this in the future added a bit of validation to
ExitSafepoint runtime call which triggers when --use-slow-path
is enabled. Though after refactoring this code does not
trigger this code path anymore because it was completely
removed - but it is better than nothing.

This CL also removes a lot of unnecessary complexity which
was associated with handling this transition in the stub
itself.

TEST=ffi/vmspecific_handle_test

Bug: b/408377905
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-mac-debug-simarm64_arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Change-Id: Ia073cb6bb9e1b5a0ea8514c7e048cee6019b84d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420324
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2025-04-07 05:22:56 -07:00
Alexander Aprelev 4a46bed718 [vm/prealloc] Don't preallocate oom, unwind errors, stacktrace object during isolate startup.
For out-of-memory events vm reserves (and if used, then replenishes) small chunk of memory for handling those, so preallocation of oom and stacktrace is not needed.
Unwind error can be created once as part of vm launch, kept in vm isolate.

This should reduce overhead of starting new isolate.

TEST=ci

Change-Id: Ibe2c85f250a521a8e838cc5690984fe9fa287a98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/419261
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2025-03-31 11:17:18 -07:00
Ryan Macnak 101516eb18 [vm] Allow reload during FFI and native calls.
TEST=many_isolates_blocked_* under reload stress testing
Bug: https://github.com/dart-lang/sdk/issues/60098
Change-Id: Ifb9838735e1846236e30820d7d957e3a51fd9e99
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/410960
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2025-03-12 12:33:50 -07:00
Alexander Markov 151e9dd509 [vm, dynamic_modules] Fix instantiated tear-offs of generic static methods and factories
TEST=language/function_subtype/bound_closure7_test
TEST=language/named_arguments_anywhere/order_side_effects_ok_test
TEST=language/regress/regress52243_test

Change-Id: I09765f5a272941388fccac65c083d2239da27b1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/414320
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2025-03-12 07:47:40 -07:00
Ryan Macnak 18c0848513 [test] Directly check for blocked mutator instead of a somewhat related state.
TEST=ffi/vmspecific_callback_functions
Change-Id: I45564b22d3c6a0d6a4300f6821a0bfa492cbee4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/414824
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2025-03-11 16:12:33 -07:00
Alexander Markov 22b26e5a29 [vm, dynamic modules] Bugfixes in the bytecode reader and interpreter runtime
* Never unbox fields loaded from bytecode as interpreter works with
  boxed fields only.

* Ensure classes are allocate-finalized in AllocateObject
  runtime entry as interpreter may allocate instances of certain
  built-in classes (such as _Closure, _Double etc) without prior
  allocate-finalization.

* Fix type arguments vector in constant instances when class doesn't
  have type parameters but extends a generic class (so its instances
  have type arguments vector).

TEST=ci

Change-Id: I488287f84572a79ca7f1fddbd53aed1c3a038cea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/412981
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2025-03-03 11:36:50 -08:00
Ryan Macnak 2a08770912 [vm] If waiting for an active mutator slot for too long, steal a slot from a thread at a native safepoint.
The thread whose slot was stolen will take the slow path attempting to exit its safepoint and wait to reacquire an active mutator slot.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/54687
Change-Id: I49c4c12b38d38d7afa4df6b346c2a464f75ef6e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/406404
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2025-02-10 09:38:42 -08:00
Ryan Macnak 4617198d58 [vm] Add option for GC at [re]throw.
Add a reduced test for crash / false ConcurrentModificationError observed by the isolate stress test.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/60017
Change-Id: I665a7ec48c150d270e751ab13393384613d44674
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/408201
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2025-02-05 14:24:32 -08:00
Alexander Markov 430554158e [vm] Support implicit dynamic calls
According to the spec, a call in the form

  e(a0,...,aN)

where static type of 'e' is 'dynamic' should succeed only if
(1) 'e' evaluates to a function, or
(2) runtime type of 'e' has a 'call' *method*.

If runtime type of 'e' has a 'call' getter this invocation should
fail with NSM.

This behavior is different from 'e.call(a0,...,aN)' which accepts
'call' getters.

---

In order to implement this behavior in the VM, a special
'dyn:implicit:call' selector is added. It behaves similarly to
'dyn:call' except when looking for a getter target.

This selector is used when CFE sets FlagImplicitCall on a
DynamicInvocation node.


TEST=co19/Language/Expressions/Function_Invocation/Function_Expression_Invocation/call_A04_t01
TEST=co19/Language/Expressions/Function_Invocation/Function_Expression_Invocation/call_A04_t02

Fixes https://github.com/dart-lang/sdk/issues/59965
Issue https://github.com/dart-lang/sdk/issues/59952
Issue https://github.com/dart-lang/sdk/issues/51517
Issue https://github.com/dart-lang/language/issues/3482

Change-Id: Ic45f7743ad75571476642dcec9c91e6a77e8e321
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/407161
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2025-02-04 08:13:29 -08:00
Daco Harkes 74f2d4b1af [vm/ffi] Error on invoking callback from leaf call
All callbacks flow via the single runtime entry, so we add a check
there.

TEST=tests/ffi/function_callbacks_leaf_test.dart

(The transition verification in
https://dart-review.googlesource.com/c/sdk/+/407021
doesn't catch the error. The callbacks already transitioned before
reaching the generated code in that CL.)

Closes: https://github.com/dart-lang/sdk/issues/60021
Change-Id: Ia6bf8af186c9c8c5a66cffa537d80421467330d4
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-arm64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/407023
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2025-02-04 03:36:29 -08:00
Alexander Markov 5fd11d360d [vm] Cleanup legacy differences between 'is' and 'as' checks
With sound null safety both 'is' and 'as' type checks use subtyping
without any extra differences.

Replace Instance::IsAssignableTo with IsInstanceOf (which is now
the same as RuntimeTypeIsSubtypeOf).

Remove Instance::NullIsInstanceOf as it is no longer used.

Replace CompileType::IsAssignableTo and IsInstanceOf with IsSubtypeOf.

TEST=ci

Change-Id: I3a071d114c5f86d990255416b244961d506257a3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/401861
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2024-12-19 11:03:59 -08:00
Ryan Macnak 810b842577 [vm, reload] Fix allocation on the wrong side of a safepoint transition during a failed reload.
Ignore identity reload failures during VM shutdown.

TEST=ci, rr chaos mode
Change-Id: I8ba8ccae4294f96f2e0f2b57baa74a600c8d4080
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/399840
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2024-12-09 23:02:48 +00:00
Liam Appelbe 50f367b283 [vm] Early-out in NativeCallable trampoline if isolate creation disabled
I haven't been able to get a solid repro the tsan flake locally, and I'm not sure how to write a test for it, but this change is pretty uncontroversial anyway, so I'm inclined to just land it and see what happens.

I have some weak evidence that this fixes the issue though. Before this change I ran the flaky command about 500 times before I saw a failure (though I didn't get any useful data from the failure, and it didn't look quite the same as the bug). With the change, the command ran 3500 times before flaking. I'm not sure if it's the same flake though (rr isn't working for me atm).

Bug: https://github.com/dart-lang/sdk/issues/56900
Fixes: https://github.com/dart-lang/sdk/issues/56900
Change-Id: I939e8ceca4051ec91e942ca38a2b01477b67ddd8
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/392162
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2024-10-29 22:26:59 +00:00
Alexander Markov 1225b45bc7 [dart2bytecode, vm/interpreter] Async exceptions
TEST=ci (vm-aot-dyn-linux-debug-x64)

Change-Id: I9d5bb0f7f2544e41078ec9aeb75bce6224087976
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383706
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2024-09-09 13:44:04 +00:00
Alexander Markov a17bc048b4 [dart2bytecode, vm/interpreter] Late final fields and variables
* Also, add implicitly overridden _Enum._enumToString to dynamic
  interface.

TEST=ci (vm-aot-dyn-linux-debug-x64)

Change-Id: I9d9d368715d0837d8b1039a46451152e03be7eec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383024
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-09-04 14:18:09 +00:00
Alexander Markov 52304b29d0 [vm/interpreter] Fix type checks and async stack traces
TEST=ci (vm-aot-dyn-linux-debug-x64)

Change-Id: Ia8d5dff98ef1b1afa97108a40f1835545e08790f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382962
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
2024-09-04 08:14:18 +00:00
Ryan Macnak b87378f8a5 [vm] Avoid race on TTS testing flag.
TEST=tsan
Change-Id: I939fbd756f114dbf0db027b4eb05a63b8b940833
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/383421
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2024-09-03 21:58:20 +00:00
Alexander Markov ce571036e1 [vm] Cleanup legacy AssertBoolean checks
With sound null safety front-end guarantees that all logical
expressions have a non-nullable bool type, so legacy AssertBoolean
checks are no longer needed and can be removed.

TEST=ci

Change-Id: If952da7bd0ac83c43de3e5d98845c5e5d8d29f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382744
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-08-29 16:45:00 +00:00
Alexander Markov 062b0738e3 [dart2bytecode, vm/interpreter] await/yield/yield*
TEST=language tests in vm-aot-dyn-linux-debug-x64 configuration

Change-Id: I205bec19c2072fe9ac11a3211123bba43cb99d5e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381945
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-08-26 20:35:18 +00:00
Alexander Markov 84fd647969 [vm] Refactor access to Integer value
Add methods to provide uniform access to values of Dart integers:

  Integer::Value()
  Integer::Value(IntegerPtr)

  Smi::Value()
  Smi::Value(SmiPtr)

  Mint::Value()
  Mint::Value(MintPtr)

Remove

  AsInt64Value()
  AsTruncatedInt64Value()
  AsTruncatedUint32Value()
  GetInt64Value(IntegerPtr)

Also, rename AsDoubleValue() to ToDouble() and
remove unused (FitsIntoSmi, AsValidInteger) and
value-based methods (IsZero, IsNegative).

TEST=ci

Change-Id: I28786ec3a14703574b7a192ead42eeefdbd09106
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380586
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-08-19 21:46:26 +00:00
Vyacheslav Egorov 49ed2007e5 [vm] Fix deadlock in AllocateSuspendState
AllocateSuspendState was calling `Instance::SetField` but did not allow lazy deopt to occur. Possibility of lazy deopt from those
field stores is only theoretical, but it manifested as a
deadlock between background compiler and main thread: if main
thread calls `SetField` which tries to acquire write access to
the program lock, while background compiler has already
acquired write access to the program lock and is trying to
stop all mutators at a GC+Deopt safepoint then we will
deadlock as `AllocateSuspendState` does not allow Deopts
(safepoint level was lowered by 5bc107c29d).

We fix this problem by bypassing field guard and simply
writing affected fields directly in AllocateSuspendState.
We make sure to initialize guarded state for these fields
eagerly, so it never needs to change.

TEST=added assertion which validates that attempting to acquire program lock for write can only occur where GC+Deopt are permitted.

Change-Id: I6ee6b82f3296f49f799c0069e42850711c9320ac
Bug: b/355226004
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2024-08-19 14:04:47 +00:00
Alexander Markov 758c490ae5 [vm] Update dynamic modules for the recent changes in the VM
* Revise uses of ObjectPtr::GetClassId which now handles Smis.
* Accomodate for _Map and _Set moved to dart:_compact_hash.

Also, a bit of code cleanup in the interpreter:
* Replace RAW_CAST with Handle::RawCast
* Replace NULL with nullptr.

TEST=Manual

Change-Id: Ib49f66cfcbac26200294d3ee55edc1b99c94fcf6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380685
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2024-08-16 08:27:22 +00:00
Alexander Markov 8fbca8ba67 [vm] Initial implementation of dynamic modules in the VM/AOT
TEST=Manually tested dynamic modules

Change-Id: Icb2616e414167bd1fbd10f01dea64c57dbdeeac7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380281
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-08-15 14:09:52 +00:00
Ryan Macnak 5bc107c29d [vm] Mark AllocateSuspendState runtime entry as not allowing a deopt safepoint.
The suspend stubs need to save the real pc, not the lazy deopt stub entry, otherwise pointer visiting for the SuspendState will incorrectly use the stackmap for the lazy deopt stub.

TEST=iso-stress
Bug: https://github.com/dart-lang/sdk/issues/56230
Change-Id: Ie6f9fe5744849f6e6b5bbcbafcb82e6cfb4c4500
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/375763
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2024-07-16 16:52:48 +00:00
Alexander Markov 00eac588c2 [vm] Remove transition_sentinel and detection of cyclic initialization of legacy static fields
Since null safety, all static fields with initializers are implicitly
late. This change cleans up transition_sentinel which was used in
the detection of cyclic initialization of legacy static fields.

TEST=ci

Change-Id: I6a990dc8ba030f5bd40eb0b86706cbfb0f725e33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/373520
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-06-27 18:29:58 +00:00
Ryan Macnak 3180d95453 [vm, gc] Don't add large arrays to the deferred marking list.
Write barriers are not eliminated for large arrays, and deferred marking them is expensive.

TEST=IsolateSendExitLatency
Change-Id: Ifeefb0d86e15d096ae45f8e8ff6aa3d9db4cb9ed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/373141
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2024-06-25 21:12:28 +00:00
Tess Strickland d06d627c79 [vm] Remove --[no-]lazy-dispatchers flag.
No client of the VM uses this flag, only tests, and this flag was always
set to false in AOT mode. Thus, remove uses of this flag and instead
always lazily create dispatchers as needed when resolving method names
in JIT mode.

Remove the implicit value of `allow_add` for some Resolver
static methods. For callers that previously depended on the implicit
`true` value (which includes the AOT precompilier), pass `true` for
uses in the compiler and pass `!FLAG_precompiled_mode` for uses in the
runtime. Assert that `allow_add` is false when these methods are invoked
from the precompiled runtime.

Remove Resolver static methods that are no longer used.

TEST=ci

Change-Id: Ib6a7354f7a859e86743c381513a4129c14895753
Cq-Include-Trybots: luci.dart.try:vm-linux-debug-x64-try,vm-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-mac-debug-arm64-try,vm-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/366668
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2024-06-06 10:56:12 +00:00
Tess Strickland 8b8ec2aa3d [gardening] Fix TrySwitchInstanceCall for --no-lazy-dispatchers.
TEST=Manual run of the following command line found by the dart
     fuzzer:

out/DebugX64/dart --profiler --max_profile_depth=189 \
  --sample_buffer_duration=51 --marker_tasks=2 \
  --optimization_counter_threshold=31050 --no_load_cse \
  --no_lazy_dispatchers pkg/compiler/lib/src/dart2js.dart \
  --invoker=test --platform-binaries=out/ReleaseX64 \
  --out=out/dartfuzz/5.js --no-source-maps \
  pkg/compiler/lib/src/util/memory_compiler.dart

Change-Id: If7620b7b7bb76d28f66ffde01af52a62392302c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/366780
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2024-05-16 10:33:58 +00:00
Alexander Markov ac3a99b760 [vm] Optimize double.remainder
Bypass runtime entry and directly call `fmod` from generated code,
similarly to double.operator %.

On a micro-benchmark (from #55479):

JIT
Before: Remainder(RunTime): 1347507.0 us.
After:  Remainder(RunTime): 19465.037735849055 us. (69x faster)

AOT
Before: Remainder(RunTime): 1232950.0 us.
After: Remainder(RunTime): 19465.377358490565 us. (63x faster)

TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/55479

Change-Id: I5b35b15f21da74aa4e2c92d7aa2804dae9efff99
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-04-29 17:47:21 +00:00
Ryan Macnak 2d0e514583 Update clang to 0f61051f541a5b8cfce25c84262dfdbadb9ca688.
Account for new -Werror,-Wcast-function-type-mismatch.

Re-run clang-format.

TEST=ci
Change-Id: I70d30548cf2e2f86b3d654962f0e73561b6e6de2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/363722
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2024-04-19 19:46:47 +00:00
Vyacheslav Egorov 7eed880d53 [vm/compiler] Refactor implementation of throw-into-pending-lazy-deopt
Instead of having special support for targeting catch entry as a
lazy deoptimization target, split this into two independent
operations:

* Perform lazy deoptimization at call.
* Rethrow the exception once deoptimization is complete.

TEST=ci

Change-Id: Ib275d615bc6717e1184e88602ae5459d5b771f2b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361580
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2024-04-15 11:10:23 +00:00
Alexander Markov a6ded2fd08 [vm] Unsound mode cleanup: remove --strict_null_safety_checks
This flag only makes sense without sound null safety. VM no longer
supports unsound mode, so it can be removed.

In sound mode this flag is always true.

TEST=ci

Change-Id: I4a5fa60d9b351b6281efab9462ef2dd2c5565a7c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362182
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-04-10 19:19:31 +00:00
Daco Harkes eb50c2683c [vm/ffi] Throw on attaching NativeFinalizer to deeply immutable object
This CL

* adds a `kind` to `CheckWritableInstr`,
* starts using `CheckWritableInstr` in JIT mode, and
* adds support for `CheckWritableInstr` ia32.

TEST=tests/ffi/vmspecific_native_finalizer_deeply_immutable_test.dart

Closes: https://github.com/dart-lang/sdk/issues/55067
Change-Id: I0b397daba12cfc8b885401169889f7cd7c040166
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359223
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2024-03-28 11:24:36 +00:00
Daco Harkes 9426e47d73 [vm] Add receiver argument to WriteError runtime entry
Note this changes the error in AOT to tell what is the typed data that
is not writable.

This CL eases the follow up CL which starts reusing the
`CheckWritableInstr` and accompanying slow path and runtime entry for
other messages.

TEST=tests/lib/typed_data/polymorphic_unmodifiable_typed_data_test.dart

Bug: https://github.com/dart-lang/sdk/issues/55067
Change-Id: Icf26d119501059e2ea4e885897539ebcb1063da0
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-simarm-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359063
Reviewed-by: Ilya Yanok <yanok@google.com>
2024-03-28 11:24:36 +00:00
Alexander Markov 35d8630176 [vm] Make String.codeUnitAt monomorphic
The new implementation of String.codeUnitAt is always inlined and
doesn't depend on the polymorphic inlining of recognized methods.

String.codeUnitAt now has a custom body in the flow graph builder
which  performs non-speculative bounds check and then branches between
OneByteString and TwoByteString. Corresponding graph intrinsic and
native method are removed.

This change also fixes passing of unboxed arguments to runtime
in the slow path of GenericCheckBound instruction in JIT mode
(when shared stubs are not used).

TEST=ci

Change-Id: Iab2805fc752df84c37089165f828e31aca5f043f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359000
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-03-24 14:50:17 +00:00
Alexander Markov b9b341f4a7 [vm] Simplify and optimize method extractors
Previously, method extractors (getter functions which return
tear-offs) where implemented in the following way:

Method extractor body is an asm intrinsic which loads a few registers
and jumps to BuildMethodExtractor stubs.
BuildMethodExtractor stub loads more registers, calls AllocateClosure
stub and initializes a few fields of a closure (this stub also used
to call AllocateContext but it is no longer needed for tear-offs).
AllocateClosure allocates a closure object and initializes closure
fields.

Instead of doing 2 hops to the object allocation, method extractor
body now calls AllocateClosure stubs directly which initializes
all closure fields. This saves a jump and a handful of instructions
to load certain things on registers and set closure fields.

Assembler intrinsic for method extractor body and
architecture-specific BuildMethodExtractor stubs are now removed.

Also, as method extractors are really tiny, they are now always
inlined.

TEST=ci

Change-Id: Icfeed18414659a76c7d24d3f6f53a76c91e5bbd3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/356302
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-03-11 15:56:33 +00:00
Alexander Markov a261196ea7 [vm] Remove context allocations for tear-offs
Previously, when taking a tear-off, a separate Context object was
allocated in order to capture receiver. Now, receiver is stored directly
in the Closure object in the 'context' field. This saves 1 object
allocation per tear-off and makes tear-offs cheaper compared to
explicit closures which can share context with other closures.

Benchmarks in AOT mode:

x64:
TearOff.NotInlined +40%
TearOff.NotInlined.InTry +43%
TearOff.Inlined.InTry +47%

arm64:
TearOff.NotInlined +27-43%
TearOff.NotInlined.InTry +29-43%
TearOff.Inlined.InTry +58-94%

arm64c:
TearOff.NotInlined +71%
TearOff.NotInlined.InTry +72%
TearOff.Inlined.InTry +96%

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/54808
Change-Id: I3ad95e8a8a4fc23f856bbc0fe238da58a9d25b8d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350945
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2024-02-13 15:33:38 +00:00
Ryan Macnak 1501d688c0 [vm] Avoid allocation when printing type names.
Avoids recursive assertion failures when a GC assertion failure occurs during a compile.

TEST=ci
Change-Id: I2fd2b29c1a7d83c426ceb437f3880746a4c8d8b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/349142
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2024-01-29 23:53:19 +00:00
Ryan Macnak 4e6cd29eef [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I1e653c9b5d3e02e45b280302c832157a75788db6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345350
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2024-01-26 18:58:54 +00:00
Siva Annamalai 94c8d341d7 Revert "[vm, gc] Mark through new-space."
This reverts commit a3b7c323b0.

Reason for revert: We are seeing crashes in some internal testing
si_signo=Segmentation fault(11), si_code=SEGV_ACCERR(2), si_addr=0x7f1cadb85bc6
version=3.3.0-edge+google3-v2 (google3) (google3) on "linux_x64"
pid=8967, thread=8969, isolate_group=main(0x5583571aaaf0), isolate=(nil)((nil))
os=linux, arch=x64, comp=no, sim=no
isolate_instructions=7f1cad8b7e40, vm_instructions=7f1cad8b1000
fp=7f1cacc7ebc0, sp=7f1cacc7eb90, pc=5583568155bc
  pc 0x00005583568155bc fp 0x00007f1cacc7ebc0 dart::MarkingVisitorBase<true>::VisitPointers(dart::ObjectPtr*, dart::ObjectPtr*)+0x15c
  pc 0x0000558356975a1d fp 0x00007f1cacc7ec60 dart::StackFrame::VisitObjectPointers(dart::ObjectPointerVisitor*)+0x1bd
  pc 0x0000558356935c29 fp 0x00007f1cacc7ecc0 dart::UntaggedSuspendState::VisitSuspendStatePointers(dart::SuspendStatePtr, dart::ObjectPointerVisitor*)+0x89
  pc 0x0000558356815948 fp 0x00007f1cacc7ed10 dart::MarkingVisitorBase<true>::DrainMarkingStackWithPauseChecks()+0x128
  pc 0x0000558356815749 fp 0x00007f1cacc7ed80 dart::ConcurrentMarkTask::Run()+0x69

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: Ice07b4eb5bef3b41c9618ef0ca7759de006ffe00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/343060
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-12-21 21:02:38 +00:00
Ryan Macnak 2d01bf3779 [vm, gc] Interrupt to finalize concurrent marking.
STW marking is no longer O(new-space), so there's no longer a reason to delay finalizing marking hoping for a scavenge to make new-space mostly empty.

TEST=ci
Change-Id: Ie782e88852714d30e0c75aa9aecac62e56c434ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319880
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-12-14 17:27:49 +00:00
Ryan Macnak a3b7c323b0 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-12-12 01:56:15 +00:00
Ryan Macnak fb1d821bbe [vm] Fix flaky race with --runtime_allocate_spill_tlab.
TEST=tsan
Change-Id: Ib37a3812a122e3688a1fd9310d4cfda5f4780c21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336602
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-11-16 18:43:24 +00:00
Slava Egorov 425a42e3be Revert "[vm, gc] Mark through new-space."
This reverts commit 5daaa7d9eb.

Reason for revert: internal crashes

Original change's description:
> [vm, gc] Mark through new-space.
>
>   - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>   - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>   - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>   - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>   - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I8a50074db343c63c14f0487ae8b4f5fee2c4ae76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330720
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2023-10-18 07:47:39 +00:00
Ryan Macnak 5daaa7d9eb [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
  - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-10-10 19:15:12 +00:00