- Updated conditional compilation flags throughout the runtime codebase to transition from DART_DYNAMIC_MODULES to DART_BYTECODE_INTERPRETER.
- Adjusted logic in various files including object_graph_copy.cc, object_reload.cc, profiler.cc, and others to ensure compatibility with the new interpreter model.
- Ensured that all references to dynamic modules are replaced with bytecode interpreter checks, maintaining functionality for interpreted code execution.
- Modified stack frame handling and service-related code to align with the new interpreter architecture.
- Updated tests and service implementations to reflect the changes in the runtime environment.
Signed-off-by: Tony <tonylu@tony-cloud.com>
The CallMarshaller is used not only from the compiler, but also
from the FfiCall runtime entry used by the interpreter. Since it
only has one use of the thread's CompilerState, looking up the
TypedData class, move the storage of that class from the compiler
state to the object store and remove this dependency.
TEST=ffi/address_of_array_generated_test
ffi/address_of_cast_test
fii/address_of_struct_generated_test
ffi/address_of_typeddata_generated_test
Fixes: https://github.com/dart-lang/sdk/issues/61913
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Ia4fc5d9ecef370aa9476b998e37cc4ae94ee447f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498563
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Also load/store canonical hashes in the heap for non-empty TypedData
instances in the same manner as canonical hashes for Arrays.
TEST=ci (refactoring only)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I54274b558fa9f0c8e304198b18cb3f0e9c3e0dfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504600
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
The RecordCoverage instruction has an A/E encoding. The A argument
is the type of coverage being recorded, whereas the E argument is
the logical index into the coverage array for updating whether that
source position has been hit.
Also adds new metadata to the bytecode component for the coverage
arrays associated with bytecode containing RecordCoverage instructions
and a new runtime entry for lazily allocate the coverage array for
an interpreted function when needed.
The type of coverage is encoded in the RecordCoverage instruction,
despite being redundant with the information in the coverage array, so that checking whether that type of coverage is currently enabled at
runtime doesn't require either accessing the coverage array (which may
be lazily allocated), forcing allocation of the coverage array just to
discover that type of coverage is currently disabled, or reading the
serialized bytecode component to avoid that forced allocation.
------
Other changes:
Source reporting now treats unexecuted interpreted functions when
not forcing compilation as if they were uncompiled native functions,
so that the source report from running the same code gives the same
result whether using the interpreter or the native compiler.
Bytecode closures are no longer skipped in source reports. Previously
any closure without a context scope was skipped, but bytecode closures
don't have those.
TEST=vm/cc/SourceReport_Coverage
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I7557e5dd4c98331c7ca2f5c867dd5f6d03e9d756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Unlike other calls from Dynamic Modules, dynamic calls cannot be
validated entirely at compile time. While we check that the selector
used matches a selector that was allowed (either because a method with
that selector name was exposed as dynamically callable or because the
selector was allowlisted during bytecode compilation), the compiler
doesn't know statically whether the target of the call is exposed.
In prior changes we modified the annotator to add a pragma indicating
whether a member is dynamically-callable or
implicitly-dynamically-callable. Here we use that information to set a
bit on functions and their corresponding dynamic invocation forwarders,
which is verified by the interpreter to make sure the dynamic call is
still allowed.
TEST=none yet - will be added in subsequent CL (see CL chain)
Bug: b/448095881
Change-Id: I27acb4e690a68e08fe1f1ca94e0d77cc7dc4d11e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498300
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
In the VM, factory constructors always had an extra "type arguments"
parameter, even if class is not generic. Factory constructor bodies
were using class type parameters instead of function type parameters.
This results in extra code when calling non-generic factories
which is slightly inefficient in terms of code size and performance.
Also, it creates an additional complexity throughout the system as
factories should be special cased in many places.
This change removes artificial "type arguments" parameter, treating
factory constructors basically as static methods. This matches
kernel AST representation.
TEST=ci
Change-Id: I957583cb2ce9a3c408699880a04036e06b01dd31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501762
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Dynamic modules may make dynamic calls to compiled methods that had no
dynamic calls in the original whole world compilation. If this happens,
then dynamically create an interpreted dynamic invocation forwarder that
then appropiately checks and delegates to the compiled function.
TEST=co19/LibTest
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I283ca501f50118606650cf434b8debe30a43f676
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit 88496ba1c3
Fixes on top of the original change:
* Closure layout is changed to avoid gap between fixed fields and
variable-length elements on compressed pointers architecture.
This gap was causing crashes in the GC when scanning closure
objects.
* pkg/vm_snapshot_analysis/test/instruction_sizes_test is fixed
on arm64 by decreasing threshold for detecting size changes.
Original change's description:
> [vm,dart2bytecode,modular_aot] Variable-length closure objects
>
> Extend closure objects with variable number of elements to capture.
> This is needed to support capturing multiple independent contexts
> after capturing is computed in the front-end.
>
> The following fixed Closure fields are moved into variable-length
> elements:
> - delayed type arguments;
> - instantiator type arguments;
> - function type arguments;
> - context.
>
> Number of elements and presence/indices of various type arguments
> are encoded into the new length_and_flags field in the Closure.
>
> Most closure objects don't need any of the type arguments so this
> change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I8685e632e2d0832766ecdc470f3cf9a6b880de48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494243
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Extend closure objects with variable number of elements to capture.
This is needed to support capturing multiple independent contexts
after capturing is computed in the front-end.
The following fixed Closure fields are moved into variable-length
elements:
- delayed type arguments;
- instantiator type arguments;
- function type arguments;
- context.
Number of elements and presence/indices of various type arguments
are encoded into the new length_and_flags field in the Closure.
Most closure objects don't need any of the type arguments so this
change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I7ca5cec0fd8725c432a01d51781fb14e803997dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489482
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
In AOT mode, when call site is transitioned from monomorphic state
(with receiver cid1) to polymorphic (with receiver cid2),
there is an optimization which checks if all _allocated_ classes
in the class id range cid1..cid2 have the same dispatch target.
If so, a specialized SingleTargetCall stub is used.
The problem is that 'allocated' bit is only set during precompilation,
and dynamically loaded classes were not considered as valid receiver
classes by this optimization.
As a result, the following situation could happen:
cid1 < cid3 < cid2,
cid1 dispatches to target1
cid2 dispatches to target1
cid3 should dispatch to target2, but it is still in range cid1..cid2 and
SingleTargetCall stub would incorrectly dispatch it to target1.
The fix is to treat all dynamically loaded classes as allocated when
checking for single target optimization.
TEST=pkg/dynamic_modules/test/data/single_target_cid_range_dispatch
Fixes b/493677699
Change-Id: I42e407385a5d9b0a4a1017f713b47587c3c6a818
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/488920
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Instead of switching over each of the different callback types and duplicating the call-dart-spill-call-runtime-restore in the stub, have the initial runtime call return a function pointer for the runtime call to end that kind of callback. A separate case for tail calls is still needed.
TEST=ci
Change-Id: I691487ca12337ddef1adaeec4c163c56e2feb09e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/488261
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Adds a new code path for NativeCallable.isolateLocal invocations. If
the current thread is not entered into any isolate, but owns the
target isolate, then it enters the target isolate, invokes, then exits
the isolate.
Fixes: https://github.com/dart-lang/sdk/issues/61623
TEST=tests/ffi/function_callbacks_isolate_ownership_test.dart
Change-Id: I401f185fadf7d2a55190dafd15387e1c418c67c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/452380
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
Closures going into shared variables in particular have to be confirmed to capture appropriate values.
BUG=https://github.com/dart-lang/sdk/issues/62179
TEST=run_isolate_group_run_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try,vm-aot-dyn-linux-debug-x64-try
Change-Id: I27416773fd77077018739ea4dbcc6e4695e67be8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/469103
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Inform ASAN and MSAN about reads and writes through FFI pointers or external typed data. Previously we only informed MSAN about writes, which was enough to prevent false positives in foreign code, but gave false negatives for Dart code. This mostly uses the same machinery used for TSAN, and replaces some of the existing MSAN unpoison calls.
Also fixes some marshalling of compounds by value that generate loads extending past the end of the compound.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/62221
Change-Id: Ia3573edb5a0aec32b6a57035a63e8f323a655ecc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/467401
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Thread::OffsetFromThread from runtime_api.cc assumed that all fields
between leaf and non-leaf runtime entries in Thread have word size,
so the delta between runtime entries can be translated from host to
target by multiplying by the ratio between target and host word size.
Since leaf and non-leaf runtime entries are not co-located in Thread,
this assumption is incorrect. It is now fixed by using separate
bases when calculating offsets of leaf and non-leaf runtime entries.
Also, add FatalError runtime entry which is useful for
bootstrapping of the new compiler.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: Iea4762b21285aa6423876b34328c863227ee1434
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/467900
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Fix dart_stream_ and service_extension_stream_ appearing after the end-of-accessed-from-generated-code comment.
TEST=ci
Change-Id: I8f10d32b082125d7ed090cd118089309022c0f3e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/459700
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Remove the locks in DLRT_GetFfiCallbackMetadata, and switch from
safepoint locks to ordinary locks in FfiCallbackMetadata. This fixes the
deadlock bugs at the cost of reducing thread safety in error cases. Some
cases that would have failed gracefully will now have undefined
behavior.
Also, FATAL instead of no-op if a dead callback is invoked.
Fixes: https://github.com/dart-lang/sdk/issues/61372
Change-Id: Ie09fca3c629ad61b2ffbdd029269338f2706df4b
TEST=CI, particularly many_listener_callbacks_test on reload bot
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/449160
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Detects yield points in Debugger::IsAtAsyncJump for bytecode by seeing
if the currently executing instruction is a direct call to an await
or yield compiled stub.
Adds a ResumptionBreakpointHandler runtime entry that is called
during Interpreter::Resume() if the current isolate has resumption
breakpoints.
Similarly, all the places where a DebugCheck could be emitted if
debugging stops are requested now include an explicit source position
emission when source positions are requested but debugger stops are not,
to ensure the debugger has appropriate information.
Fixes CompareTopDartFrameTo returning kSelf for non-top frames when
the top frame was interpreted but the stepping frame was not or
vice versa.
TEST=pkg/vm_service/test
Change-Id: I88cdc37cf745f30e8dfb6b14c19fc9b2c4cbaf2d
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/446300
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Call __tsan_func_entry/__tsan_func_exit in functions that use __tsan_read/__tsan_write or call other functions. Call __tsan_func_exit once per frame when unwinding for exceptions. Do so only in AOT, since TSAN won't be able to symbolize JIT functions anyway.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/61352
Cq-Include-Trybots: luci.dart.try:vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try
Change-Id: Ie52c978c25664d78b834e9b72ecf7eb2a12cc2ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/444181
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Adds a set of new VMInternal_Breakpoint instructions, one for each
possible instruction size, and adds a bytecode_ and saved_opcode_
field to code breakpoints.
When enabling a breakpoint, the original opcode of the instruction is
replaced with the same-sized VmInternal_Breakpoint opcode and stored in
the saved_opcode_ field of the CodeBreakpoint. When disabling it, the
original opcode is replaced.
New labels are added to the dispatch loop for single stepping
purposes. Both the computed goto and switch dispatch versions
of the dispatch loop are appropriately altered to dispatch to
the single stepping labels instead of the original ones if single
stepping is currently enabled.
Fix up more parts of the debugger that assumed functions had Code
objects to handle functions with Bytecode objects as well. In
particular, instead of using the PcDescriptors to find safepoint
source locations in Bytecode objects, the source positions information
is used instead (since the PcDescriptors for Bytecode objects only
stores information about the start and end of try blocks at the moment).
Todo (from looking at the remaining failing tests):
* Handle async jumps.
* Handle coverage information.
TEST=now-passing tests from pkg/vm_service like
pkg/vm_service/test/break_on_function_test
Change-Id: Icbd4b818e00508d9a4e74c81520aad2363b26d41
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/444880
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Tony