- Updated conditional compilation flags throughout the runtime codebase to transition from DART_DYNAMIC_MODULES to DART_BYTECODE_INTERPRETER.
- Adjusted logic in various files including object_graph_copy.cc, object_reload.cc, profiler.cc, and others to ensure compatibility with the new interpreter model.
- Ensured that all references to dynamic modules are replaced with bytecode interpreter checks, maintaining functionality for interpreted code execution.
- Modified stack frame handling and service-related code to align with the new interpreter architecture.
- Updated tests and service implementations to reflect the changes in the runtime environment.
Signed-off-by: Tony <tonylu@tony-cloud.com>
The previous attempt was reverted because it broke unoptimized JIT
on ARM 32. This reland force-optimizes the two getters.
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Fixes https://github.com/dart-lang/sdk/issues/63436
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: Ib812cbaec6e371b9720df7a543411f78e524cac1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506060
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: I9c2c4225fd63d54e190398a0fb6649366317dd17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504221
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
When enabled, indirect calls and jumps must set X7 to a label value that matches the label in a so-called landing pad at the call target. This check does not apply if the indirect call is made through X1/RA, X5/RA2 or X7/T2, with the expectation that the compiler will limit use of these registers for logically direct calls like an AUPIC+JALR pair, or for software guarded branches like a bounds-checked jump table. We use FAR_TMP for logically direct jumps that exceed J-type range, and blocking it from register allocation limits the availablity of gadgets that set X7.
Compare x64's IBT and arm64's BTI.
TEST=ci
Change-Id: Ic00195eb3c04cc10379f9965e05ef84052e02933
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494861
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This is a reland of commit 88496ba1c3
Fixes on top of the original change:
* Closure layout is changed to avoid gap between fixed fields and
variable-length elements on compressed pointers architecture.
This gap was causing crashes in the GC when scanning closure
objects.
* pkg/vm_snapshot_analysis/test/instruction_sizes_test is fixed
on arm64 by decreasing threshold for detecting size changes.
Original change's description:
> [vm,dart2bytecode,modular_aot] Variable-length closure objects
>
> Extend closure objects with variable number of elements to capture.
> This is needed to support capturing multiple independent contexts
> after capturing is computed in the front-end.
>
> The following fixed Closure fields are moved into variable-length
> elements:
> - delayed type arguments;
> - instantiator type arguments;
> - function type arguments;
> - context.
>
> Number of elements and presence/indices of various type arguments
> are encoded into the new length_and_flags field in the Closure.
>
> Most closure objects don't need any of the type arguments so this
> change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I8685e632e2d0832766ecdc470f3cf9a6b880de48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494243
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Extend closure objects with variable number of elements to capture.
This is needed to support capturing multiple independent contexts
after capturing is computed in the front-end.
The following fixed Closure fields are moved into variable-length
elements:
- delayed type arguments;
- instantiator type arguments;
- function type arguments;
- context.
Number of elements and presence/indices of various type arguments
are encoded into the new length_and_flags field in the Closure.
Most closure objects don't need any of the type arguments so this
change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I7ca5cec0fd8725c432a01d51781fb14e803997dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489482
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
When using the --dump-kernel-bytecode flag, also print the line and
number of the source position encoded by a synthetic token position.
Also use the decoded value for a synthetic token position when
converting it as a C string for printing, which also makes it easier to
correlate real and synthetic token positions encoding the same source
position.
Also change the long outdated comment about synthetic token positions
not encoding a real source offset, as most if not all uses of them
do actually encode a real source offset.
TEST=manual use during debugging
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Ic16e320e9a10d8343e8a37c8270c7076bb7052c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/484943
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
https://dart-review.googlesource.com/c/sdk/+/481782 removed the tracing
of the original instruction for breakpoints. This CL adds it back.
Also increments icount_ before calling TraceInstruction in the dispatch
loop instead of afterwards so that breakpoint/single step tracing can
just use icount_ without adjusting it. In particular, this means tracing
the original instruction at a breakpoint can just use TraceInstruction
instead of duplicating it.
Also, now both the breakpoint instruction and the original instruction
are written to the instruction trace if requested, instead of only the
breakpoint instruction.
TEST=debugging only changes, so manually tested.
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try
Change-Id: I1999f1d9e7ba55b950c508e848ff12db0c8d5cab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/482320
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Inc/dec instructions partially update flags register,
which creates false dependencies between instructions,
so they should be only used when optimizing for size.
TEST=ci
Change-Id: I0be2a00044820592689884037dd4784ee4765e4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/479700
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Add missing byte and halfword compare-and-swap.
Add missing clear of reservation after SC.
Define XLEN aliases for the AMOs.
Add wait-on-reservation-set.
More systematically print state on fault.
TEST=ci
Change-Id: Ia07e638a6e08cd40eecc1f8f7d242fd94ef2f2d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/467200
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Previously, the jump target was printed as the value of the offset
from the current PC. Instead, print it as the absolute or relative
PC corresponding to the jump target, depending on the value of
FLAG_disassemble_relative.
To facilitate this, pipe the base PC for relative outputs through
the various formatting functions.
TEST=ci (tested manually while debugging)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try
Change-Id: I9db6baa845bf89df70b36635277893686d73d842
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/465921
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Add an additional argument to the base implementation of KernelBytecodeDisassembler::Disassemble that is the base address to use
when FLAG_disassemble_relative is enabled.
For the delegating versions of KernelBytecodeDisassembler::Disassemble
that take a Bytecode object, the payload start of the bytecode object
is retrieved and used as the base.
For the delegating versions of KernelBytecodeDisassembler::Disassemble
that take neither a base or a bytecode object, the start is used as
the base.
When tracing instructions in the interpreter, the bytecode is retrieved
via the frame pointer and then its payload start is passed in as the
base, so that the relative offset printed with the instructions match
the relative offsets printed when dumping function bytecode.
TEST=manual use of --disassemble-relative while tracing/dumping bytecode
Change-Id: I7f931037970acd950a330fa6be024df5beb144a7
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/463022
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
There are two possible flags for each source position currently:
a flag that marks the source position as synthetic and a flag that
marks the source position as within a yield point.
Synthetic source positions in bytecode are treated the same as synthetic
source positions in compiled code. That is, they encode the source
position in the text that caused them to be synthesized, but denote that
the covered instructions are internal and not to be used for debugger
pause points or for call site/branch coverage information.
Adding these flags allow us to mark appropriate parts of the async
machinery as synthetic, and also allow us to mark all the bytecode
involved in yield points as having the same token position.
The latter fixes tests where the code would step over a previous
expression, thus being paused at the start of the await bytecode,
and would record the fp and token position there as the ones to
ignore. However, since a new source position wasn't emitted until the
direct call to the await method, the recorded token position would
be the token position prior to the await call, and so the change
in token position at the await call would trigger an early pause.
TEST=pkg/vm_service/test/async_single_step_exception_test
pkg/vm_service/test/async_single_step_into_test
pkg/vm_service/test/async_single_step_out_test
pkg/vm_service/test/async_star_single_step_into_test
pkg/vm_service/test/async_step_out_test
pkg/vm_service/test/positive_token_pos_test
pkg/vm_service/test/step_into_async_no_await_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-product-x64-try
Change-Id: Ic7642a74fb76227a473f461f360e84dd3d5a45a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/453322
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
The end PC offset for the scope is inclusive, not exclusive.
Also rename PrintDescriptorsError -> PrintContextLevelError and
print the Bytecode local variable information since that is what
is searched for the context level for interpreted frames.
TEST=pkg/vm_service/test/step_through_closure
pkg/vm_service/test/step_through_function_expression
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try
Change-Id: Ib53d0a7cff81de16c957b73ce9ec7dcb0e4aaa34
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/449740
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Instead of using the computed LocalVarDescriptors when the
frame is interpreted, use the LocalVariableInfo in the serialized
bytecode to find which Scope is currently active.
Add printing of local variable information to the bytecode
disassembler.
TEST=pkg/vm_service/test
Change-Id: I7bd15056e4e2a947ad16ffb83a50447c2ba59994
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/449340
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Call __tsan_func_entry/__tsan_func_exit in functions that use __tsan_read/__tsan_write or call other functions. Call __tsan_func_exit once per frame when unwinding for exceptions. Do so only in AOT, since TSAN won't be able to symbolize JIT functions anyway.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/61352
Cq-Include-Trybots: luci.dart.try:vm-tsan-linux-release-arm64-try,vm-tsan-linux-release-x64-try
Change-Id: Ie52c978c25664d78b834e9b72ecf7eb2a12cc2ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/444181
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Tony