- Updated conditional compilation flags throughout the runtime codebase to transition from DART_DYNAMIC_MODULES to DART_BYTECODE_INTERPRETER.
- Adjusted logic in various files including object_graph_copy.cc, object_reload.cc, profiler.cc, and others to ensure compatibility with the new interpreter model.
- Ensured that all references to dynamic modules are replaced with bytecode interpreter checks, maintaining functionality for interpreted code execution.
- Modified stack frame handling and service-related code to align with the new interpreter architecture.
- Updated tests and service implementations to reflect the changes in the runtime environment.
Signed-off-by: Tony <tonylu@tony-cloud.com>
The original iteration of adding coverage arrays originally eagerly
allocated coverage arrays for bytecode methods with recorded coverage.
Now that these coverage arrays are lazily allocated,
Function::GetCoverageArray() needs to ensure that the appropriate
coverage array has been allocated. Otherwise, no misses within the
function are recorded in cases where no RecordCoverage instruction in
the method has yet been executed.
TEST=vm/cc/SourceReport_Coverage_UnusedClass_ForceCompile
pkg/vm_service/test/branch_coverage_test
pkg/vm_service/test/coverage_async_test
pkg/vm_service/test/coverage_closure_call_after_optimization_test
pkg/vm_service/test/coverage_closure_call_test
pkg/vm_service/test/coverage_instance_call_after_optimization_test
pkg/vm_service/test/coverage_static_call_after_optimization_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Iff4ddaf583d0f9583bc415142bdee52c046a6712
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507440
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The previous attempt was reverted because it broke unoptimized JIT
on ARM 32. This reland force-optimizes the two getters.
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Fixes https://github.com/dart-lang/sdk/issues/63436
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: Ib812cbaec6e371b9720df7a543411f78e524cac1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506060
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Also load/store canonical hashes in the heap for non-empty TypedData
instances in the same manner as canonical hashes for Arrays.
TEST=ci (refactoring only)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I54274b558fa9f0c8e304198b18cb3f0e9c3e0dfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504600
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
On mac with two WorkSet classes, WorkSet destructor from lib/isolate.cc is invoked instead of WorkSet destructor from vm/object.cc. This results in consistent assertion failures on flutter dart-debug builds on mac.
Might help with https://github.com/flutter/flutter/issues/186134
TEST=flutter test with dart-debug build
Change-Id: I9a0a0c9d02ee013da900807b8726df26a3660cc7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505661
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Previously, the loop threw away the calculated hash value, so the
finalized hash was only the combined hash from the length plus last
byte.
Fix this, and also use 32-bit chunks as much as possible to reduce the
number of iterations for larger typed data objects.
TEST=ci
Change-Id: I0c033701232f1a1299356cab0b09079f9f86c026
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504980
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
The RecordCoverage instruction has an A/E encoding. The A argument
is the type of coverage being recorded, whereas the E argument is
the logical index into the coverage array for updating whether that
source position has been hit.
Also adds new metadata to the bytecode component for the coverage
arrays associated with bytecode containing RecordCoverage instructions
and a new runtime entry for lazily allocate the coverage array for
an interpreted function when needed.
The type of coverage is encoded in the RecordCoverage instruction,
despite being redundant with the information in the coverage array, so that checking whether that type of coverage is currently enabled at
runtime doesn't require either accessing the coverage array (which may
be lazily allocated), forcing allocation of the coverage array just to
discover that type of coverage is currently disabled, or reading the
serialized bytecode component to avoid that forced allocation.
------
Other changes:
Source reporting now treats unexecuted interpreted functions when
not forcing compilation as if they were uncompiled native functions,
so that the source report from running the same code gives the same
result whether using the interpreter or the native compiler.
Bytecode closures are no longer skipped in source reports. Previously
any closure without a context scope was skipped, but bytecode closures
don't have those.
TEST=vm/cc/SourceReport_Coverage
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I7557e5dd4c98331c7ca2f5c867dd5f6d03e9d756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Unlike other calls from Dynamic Modules, dynamic calls cannot be
validated entirely at compile time. While we check that the selector
used matches a selector that was allowed (either because a method with
that selector name was exposed as dynamically callable or because the
selector was allowlisted during bytecode compilation), the compiler
doesn't know statically whether the target of the call is exposed.
In prior changes we modified the annotator to add a pragma indicating
whether a member is dynamically-callable or
implicitly-dynamically-callable. Here we use that information to set a
bit on functions and their corresponding dynamic invocation forwarders,
which is verified by the interpreter to make sure the dynamic call is
still allowed.
TEST=none yet - will be added in subsequent CL (see CL chain)
Bug: b/448095881
Change-Id: I27acb4e690a68e08fe1f1ca94e0d77cc7dc4d11e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498300
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
In the VM, factory constructors always had an extra "type arguments"
parameter, even if class is not generic. Factory constructor bodies
were using class type parameters instead of function type parameters.
This results in extra code when calling non-generic factories
which is slightly inefficient in terms of code size and performance.
Also, it creates an additional complexity throughout the system as
factories should be special cased in many places.
This change removes artificial "type arguments" parameter, treating
factory constructors basically as static methods. This matches
kernel AST representation.
TEST=ci
Change-Id: I957583cb2ce9a3c408699880a04036e06b01dd31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501762
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
These will be base objects even for kFullAOT and kFullJIT snapshots, so their allocation needs to be separated from the rest of Object::Init.
TEST=ci
Change-Id: I1ad03cf9900a925d3b084843206bba7f67c5daa9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/499083
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Call sites targeting a procedure annotated with `external-effect` will
not produce any code, including the argument which will not be
evaluated.
However, the single parameter will be treated as 'live' for the purposes
of any global analysis the backends do. This is useful for things like
protobuf shaking where a user may want to retain certain protobuf
messages without actually emitting the code that retains those messages.
Today this functionality is available internally in the vm and wasm SDK
libraries. dart2js has similar functionality represented via the
opaqueTrue and opaqueFalse booleans (which will cause conditional
branches to get shaken after analysis). This will replace dart2js's
opaque(True/False).
This also adds validation to the frontend to ensure a method annotated
with 'external-effect' is well-formed.
Change-Id: If1c4096673e655c58fe7638840a16125003e7809
Tested: Backend tests for codegen were added. A frontend test was added for the validation. A language test was added to confirm the behavior.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476020
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Some objects have their canonical bit set but don't have the deeply immutable bit set. It's simpler if they have this bit clear so there is only cluster need, and these objects are instances of VM-internal classes that don't really do canonicalization.
Also give the oddballs their header hash while doing the rest of their initialization. This makes it easier to use the as snapshot base objects when there is no finalize VM isolate step.
TEST=ci
Change-Id: I7bdf7dc59b04ae30f49c5c67c0dc4f69c2d2524f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/497121
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Dynamic modules may make dynamic calls to compiled methods that had no
dynamic calls in the original whole world compilation. If this happens,
then dynamically create an interpreted dynamic invocation forwarder that
then appropiately checks and delegates to the compiled function.
TEST=co19/LibTest
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I283ca501f50118606650cf434b8debe30a43f676
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This class is structurally equivalent to _HashVMBase and the only reason
for it to exist for inability of normal Dart classes to extend
_HashVMBase - but this can worked around by properly configuring
_HashVMBase class in bootstrapping.
TEST=ci
Change-Id: I5ca401e274920d2b4739424c6b6595306a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494140
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is a reland of commit 88496ba1c3
Fixes on top of the original change:
* Closure layout is changed to avoid gap between fixed fields and
variable-length elements on compressed pointers architecture.
This gap was causing crashes in the GC when scanning closure
objects.
* pkg/vm_snapshot_analysis/test/instruction_sizes_test is fixed
on arm64 by decreasing threshold for detecting size changes.
Original change's description:
> [vm,dart2bytecode,modular_aot] Variable-length closure objects
>
> Extend closure objects with variable number of elements to capture.
> This is needed to support capturing multiple independent contexts
> after capturing is computed in the front-end.
>
> The following fixed Closure fields are moved into variable-length
> elements:
> - delayed type arguments;
> - instantiator type arguments;
> - function type arguments;
> - context.
>
> Number of elements and presence/indices of various type arguments
> are encoded into the new length_and_flags field in the Closure.
>
> Most closure objects don't need any of the type arguments so this
> change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I8685e632e2d0832766ecdc470f3cf9a6b880de48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494243
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Extend closure objects with variable number of elements to capture.
This is needed to support capturing multiple independent contexts
after capturing is computed in the front-end.
The following fixed Closure fields are moved into variable-length
elements:
- delayed type arguments;
- instantiator type arguments;
- function type arguments;
- context.
Number of elements and presence/indices of various type arguments
are encoded into the new length_and_flags field in the Closure.
Most closure objects don't need any of the type arguments so this
change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I7ca5cec0fd8725c432a01d51781fb14e803997dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489482
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Remove special case for null in message snapshots; snapshots are sometimes read or written with no current isolate group.
Currently still all copies pointing into the VM isolate.
TEST=ci
Change-Id: I4d2e35a01880885d4e92e1c623c0f39a35e06065
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493866
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Instead of using the Function String-creating methods, use the
Function CString-creating methods which avoids allocation that
causes issues when called from within the interpreter (especially
in debug mode, where asserts are triggered).
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Ic43a831a6ec1ee4327e1fdf4dfc64bd09af5c04a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493801
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
This CL adds a new `can-be-used-as-type` section in the dynamic
interface to expose the type of a class or extension type, without
making them callable or exposing their members.
Details:
* add the new section to the documentation
* add support in the TFA transformation to annotate kernel nodes
* add support to the validator to check for proper usage from dynamic modules
* add CFE, transformation, and end2end tests.
The CL ignores DDC and Dart2wasm at the moment and we intend to approve
any failures. It's possible that DDC works due to its modular nature.
BUG=b/488343137
TEST=cfe, annotator, and e2e tests
Change-Id: I900b22b85950f8ac379eabaa2ef58f075e18519c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486340
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Add __VA_OPT__ to guard the trailing comma, since we use
the /Zc:preprocessor flag.
Remove the conditional guard, since __VA_OPT__ is the
portable way of doing this in C++20, and we target that
version of the language standard when compiling.
TEST=build on vm-msvc-win-x64,vm-msvc-win-arm64
Cq-Include-Trybots: luci.dart.try:vm-msvc-win-arm64-try,vm-msvc-win-x64-try
Change-Id: I5b718858300d72679e9c8bced5f1419e206a2e94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/484621
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Previously there were two separate implementations of the same
initialization code, one in object.cc and one in interpreter.cc.
Now they both use Object::InitializeHeader.
Make TryAllocate a static method of Interpreter instead of a file-local
function in interpreter.cc so that it gets access to the private method
Object::InitializeHeader.
TEST=ci (just code refactoring)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I4e042fbf0db84e02c88f7ca0f3e025723660001b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476120
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
The previous check can fail if marker reads the old tags, then the mutator truncates the array, then the marker reads the length field. Rereading the length in the marker won't lead to consistency, and rereading the tags is not permitted when running in the scavenger.
Also avoid a compiler bug where the compare_exchange_weak desired value was not reloaded on failure, resulting in the CAS infinitely looping after a failure.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/62373
Change-Id: If1bfeeef648a909509558401b25ce5085d5f1727
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/475924
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Closures going into shared variables in particular have to be confirmed to capture appropriate values.
BUG=https://github.com/dart-lang/sdk/issues/62179
TEST=run_isolate_group_run_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try,vm-aot-dyn-linux-debug-x64-try
Change-Id: I27416773fd77077018739ea4dbcc6e4695e67be8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/469103
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
* Make sure to start and stop profiler as mutator pauses and resumes.
* Allow profiler to call Code::GetPrologOffset without crashing
* Make sure to emit mapping with id 0 into the perfetto output if
necessary, otherwise Perfetto UI can't load the profile
The last item seems a bug in Perfetto UI because 0 is supposed
to mean "value not set".
TEST=tested by running _perf_witness tests
Change-Id: I57578ba3efafd2aaf83c2c41b0e0b1f36a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/459744
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Tony