- Updated conditional compilation flags throughout the runtime codebase to transition from DART_DYNAMIC_MODULES to DART_BYTECODE_INTERPRETER.
- Adjusted logic in various files including object_graph_copy.cc, object_reload.cc, profiler.cc, and others to ensure compatibility with the new interpreter model.
- Ensured that all references to dynamic modules are replaced with bytecode interpreter checks, maintaining functionality for interpreted code execution.
- Modified stack frame handling and service-related code to align with the new interpreter architecture.
- Updated tests and service implementations to reflect the changes in the runtime environment.
Signed-off-by: Tony <tonylu@tony-cloud.com>
- Introduced Dart_SetObfuscationMap to restore obfuscation maps before AOT precompilation.
- Added Dart_AotPatchInstallOptions structure for AOT patch installation options.
- Implemented Dart_AotPatchingEnabled to check if compact AOT patching is supported.
- Created Dart_SetAotPatchKeyCallback for AES key resolution during AOT patch installation.
- Developed Dart_InstallAotPatch for validating and installing encrypted AOT patches.
- Added Dart_FreeAotPatchPayload to free memory allocated for patch payloads.
- Updated runtime_args.gni to include dart_enable_aot_patching flag.
- Added tests for AOT patching functionality and ensured exported symbols include new APIs.
- Refactored existing code to accommodate new AOT patching features and improve error handling.
The CallMarshaller is used not only from the compiler, but also
from the FfiCall runtime entry used by the interpreter. Since it
only has one use of the thread's CompilerState, looking up the
TypedData class, move the storage of that class from the compiler
state to the object store and remove this dependency.
TEST=ffi/address_of_array_generated_test
ffi/address_of_cast_test
fii/address_of_struct_generated_test
ffi/address_of_typeddata_generated_test
Fixes: https://github.com/dart-lang/sdk/issues/61913
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Ia4fc5d9ecef370aa9476b998e37cc4ae94ee447f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498563
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Implicit setters of covariant fields (both covariant by declaration and
covariant by class aka generic-covariant-impl) need to perform a type
check of their argument.
It means that inferred type of the field cannot be used as inferred
type of parameter of such setter. This change removes such uses of
an inferred type of the field, which restores parameter type check
which was previously incorrectly optimized out.
TEST=runtime/tests/vm/dart/regress_63419_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63419
Change-Id: Ie313dbaab51dff15d60ce4390e7e41bdc66ad59d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/509020
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The previous attempt was reverted because it broke unoptimized JIT
on ARM 32. This reland force-optimizes the two getters.
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Fixes https://github.com/dart-lang/sdk/issues/63436
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: Ib812cbaec6e371b9720df7a543411f78e524cac1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506060
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Add `final` to PredecessorAt and PredecessorCount on the six concrete
BlockEntryInstr subclasses (GraphEntry, JoinEntry, TargetEntry,
FunctionEntry, OsrEntry, CatchBlockEntry), so the compiler can
devirtualize the calls.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: JIT compile drops from
~30.6s to ~27.4s (about 10%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: Ia2ef2761646a4c94748bcd7dfe0591262d93cd69
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505401
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Also load/store canonical hashes in the heap for non-empty TypedData
instances in the same manner as canonical hashes for Arrays.
TEST=ci (refactoring only)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I54274b558fa9f0c8e304198b18cb3f0e9c3e0dfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504600
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This adds serialization of the new local variables, VariableStatement and VariableDeclaration. This prepares for replacing LegacyVariable with the new variables in all backends.
TEST=existing
Change-Id: I9bbebfbfd372042d6b7027f0fabd24c165699832
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
When the unoptimized code of a function exceeds 1 MB, start its
optimized compile with far_branch_level=1 instead of 0. Without this,
the assembler fails on out-of-range conditional branches at level=0,
longjmps out of the optimizing pipeline, and the whole pipeline reruns
at level=1 from scratch.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted: JIT compile drops from ~26.6s to ~13.2s (about 50%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: I2608a23ef899603a46b401b71be7322b7a8dd44b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506120
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: I9c2c4225fd63d54e190398a0fb6649366317dd17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504221
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Generalize DefinitionWorklist into a Worklist<T, E> template and use a
new BlockEntryWorklist (keyed by preorder_number) to drive the WBE
dataflow. The bitvector-backed membership set prevents the same block
from being enqueued many times, which is what caused compile time to
blow up on large generated programs.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: AOT compile drops from
~42s to ~19s.
Work towards https://github.com/dart-lang/sdk/issues/63230
Fixes https://github.com/dart-lang/sdk/issues/63426
TEST=ci
Change-Id: I56c5dc40f534e4b28fe6b608a3b7f49e14957ef7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The RecordCoverage instruction has an A/E encoding. The A argument
is the type of coverage being recorded, whereas the E argument is
the logical index into the coverage array for updating whether that
source position has been hit.
Also adds new metadata to the bytecode component for the coverage
arrays associated with bytecode containing RecordCoverage instructions
and a new runtime entry for lazily allocate the coverage array for
an interpreted function when needed.
The type of coverage is encoded in the RecordCoverage instruction,
despite being redundant with the information in the coverage array, so that checking whether that type of coverage is currently enabled at
runtime doesn't require either accessing the coverage array (which may
be lazily allocated), forcing allocation of the coverage array just to
discover that type of coverage is currently disabled, or reading the
serialized bytecode component to avoid that forced allocation.
------
Other changes:
Source reporting now treats unexecuted interpreted functions when
not forcing compilation as if they were uncompiled native functions,
so that the source report from running the same code gives the same
result whether using the interpreter or the native compiler.
Bytecode closures are no longer skipped in source reports. Previously
any closure without a context scope was skipped, but bytecode closures
don't have those.
TEST=vm/cc/SourceReport_Coverage
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I7557e5dd4c98331c7ca2f5c867dd5f6d03e9d756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Unlike other calls from Dynamic Modules, dynamic calls cannot be
validated entirely at compile time. While we check that the selector
used matches a selector that was allowed (either because a method with
that selector name was exposed as dynamically callable or because the
selector was allowlisted during bytecode compilation), the compiler
doesn't know statically whether the target of the call is exposed.
In prior changes we modified the annotator to add a pragma indicating
whether a member is dynamically-callable or
implicitly-dynamically-callable. Here we use that information to set a
bit on functions and their corresponding dynamic invocation forwarders,
which is verified by the interpreter to make sure the dynamic call is
still allowed.
TEST=none yet - will be added in subsequent CL (see CL chain)
Bug: b/448095881
Change-Id: I27acb4e690a68e08fe1f1ca94e0d77cc7dc4d11e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498300
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
In the VM, factory constructors always had an extra "type arguments"
parameter, even if class is not generic. Factory constructor bodies
were using class type parameters instead of function type parameters.
This results in extra code when calling non-generic factories
which is slightly inefficient in terms of code size and performance.
Also, it creates an additional complexity throughout the system as
factories should be special cased in many places.
This change removes artificial "type arguments" parameter, treating
factory constructors basically as static methods. This matches
kernel AST representation.
TEST=ci
Change-Id: I957583cb2ce9a3c408699880a04036e06b01dd31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501762
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This test verifies how deoptIds and environments are set
when inlining force-optimized functions.
The problem is that it looks at the chain of inlined calls:
foo() {
call1 -> Pointer.fromAddress
}
Pointer.fromAddress() {
call2 -> _fromAddress
}
// force-optimized
_fromAddress() { ... }
During inlining of force-optimized callee, its instructions
should receive deoptId/env from inlined call site.
In this case, the call site is 'call2'.
However, the test has been matching deoptId/env with 'call1'.
It was a mere coincidence that deoptIds of call1 and call2
were the same. Any attempt to make a change which would
affect deoptIds within Pointer.fromAddress breaks this test.
In order to fix this, a simpler setup is now used:
foo() {
call1 -> newHash()
}
// force-optimized
newHash() { ... }
Also, the check for deoptId of outermost environment is dropped as
in this case there are no multiple environments.
In addition, also remove the duplicate pragma from _fromAddress.
TEST=vm/cc/Inliner_InlineForceOptimized
Change-Id: I1f592a74dfb8b7c2f67f3b0a51e7f7b74bf358eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/502801
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When checking number of arguments passed by the caller against
number of parameters in the callee, inliner was not taking
"type arguments" argument into account which resulted in
the rejected inline if all optional parameters are passed.
TEST=runtime/tests/vm/dart/inline_generic_functions_il_test.dart
Change-Id: I38d1fd2a1a4aadfb0196e5a472d224e3da3a1562
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/502800
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
If catch block is included into the loop, we should also include
the whole try body as there are implicit control flow edges between
each block in a try body and corresponding catch block.
If try body is not included, analysis of loop invariant loads
could miss conflicting stores and may conclude that certain loads are
loop invariant although they are not.
TEST=runtime/tests/vm/dart/regress_63336_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63336
Change-Id: I9d6b73e24554ee3bfbcc722fa59634ea90b00abb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501560
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
On 32-bit arm, when generating subtype test cache stub,
[delayed_type_args_reg] could be the same as [TypeTestABI::kInstanceReg]
(due to lack of registers).
As a result, when loading delayed type arguments from an instantiated
generic closure, [TypeTestABI::kInstanceReg] is eagerly overwritten,
which results in garbage delayed type arguments.
With the linear probe cache this would cause repeated cache misses.
With the hash-based cache this would cause a crash.
Bug: b/507666702
Change-Id: I1093f9277d874154a121ef644ecd7aa76320f789
TEST=runtime/tests/vm/dart/regress_b_507666702_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500842
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL enables the primary constructors feature by default in Dart 3.13.
The primary constructors feature is a brevity feature. There are no new semantics, but it allows us to express declarations in a less verbose way.
This feature allows one constructor and a set of instance variables to be specified in the header of a declaration.
Currently a declaration with a constructor and some fields is written as:
```dart
// Current syntax.
class Point {
int x;
int y;
Point(this.x, this.y);
}
```
With a primary constructor, we would write the above as:
```
class Point(var int x, var int y);
```
If a primary constructor needs an initializer list or a body, they can be
specified inside the class using the `this` body syntax:
```dart
class Point(var int x, var int y) {
this : assert(x >= 0) {
print('Point created at $x, $y');
}
}
```
As part of this feature, you can also use the `new` and `factory` keywords to
declare constructors in the class body without repeating the class name:
```dart
class Point {
int x, y;
// Equivalent to Point(this.x, this.y)
new(this.x, this.y);
// Equivalent to Point.origin()
new origin() : x = 0, y = 0;
// Equivalent to factory Point.clone(Point other)
factory clone(Point other) => Point(other.x, other.y);
}
```
To learn more about the feature, check out the feature specification located here: https://github.com/dart-lang/language/blob/main/accepted/future-releases/primary-constructors/feature-specification.md
Tested: Has existing language, CFE, analyzer, analysis server tests.
Bug: https://github.com/dart-lang/sdk/issues/61524
Change-Id: I296f2fcd918b87bf2a1dd00256340759866c2423
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489241
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Michael Thomsen <mit@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Kallen Tu <kallentu@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Require the embedder to place a blob containing the FFI callback stub at pkg/lib/ffi_callback_stub.bin. All blobs in the package can be loaded as executable VMOs.
Also reapply "Remove special case for simulator FFI callback thunks."
TEST=ci
Change-Id: I6ab4a73fc28fb750e45e67797b0bf9deee9d8b53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/495840
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Call sites targeting a procedure annotated with `external-effect` will
not produce any code, including the argument which will not be
evaluated.
However, the single parameter will be treated as 'live' for the purposes
of any global analysis the backends do. This is useful for things like
protobuf shaking where a user may want to retain certain protobuf
messages without actually emitting the code that retains those messages.
Today this functionality is available internally in the vm and wasm SDK
libraries. dart2js has similar functionality represented via the
opaqueTrue and opaqueFalse booleans (which will cause conditional
branches to get shaken after analysis). This will replace dart2js's
opaque(True/False).
This also adds validation to the frontend to ensure a method annotated
with 'external-effect' is well-formed.
Change-Id: If1c4096673e655c58fe7638840a16125003e7809
Tested: Backend tests for codegen were added. A frontend test was added for the validation. A language test was added to confirm the behavior.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476020
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
The 2nd argument of _simpleInstanceOf call is always a
Constant (type). However, a Redefinition can be inserted in the middle.
So, instead of ArgumentAt(1)->AsConstant()->value() it is
more safe to use ArgumentValueAt(1)->BoundConstant() as
BindsToConstant/BoundConstant unwraps Redefinition(s) via
OriginalDefinition().
TEST=runtime/tests/vm/dart/regress_63211_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63211
Change-Id: Ie4a473ebe2deee8562e6634a792f02b0dcefc918
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/497761
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The five binary operators on Int32x4 (+, -, |, &, ^) were never added
to the recognized-method list as graph intrinsics, so calls to them
were left as runtime calls through external-name bodies. In JIT the
call specializer picked kInt32x4Cid from IC feedback and emitted a
native SimdOpInstr, but AOT has no IC feedback and therefore fell back
to boxed calls, making Int32x4List inner loops 10-70x slower than both
the JIT version and a hand-written scalar equivalent.
This CL wires the same specialization paths that already exist for
Float32x4.+,-,*,/:
- recognize the five operators as graph intrinsics and mark them
with `@pragma("vm:recognized", "graph-intrinsic")` plus an
exact-result-type pragma;
- add Build_Int32x4{Add,Sub,BitAnd,BitOr,BitXor} helpers that
delegate to the existing BuildSimdOp;
- extend SimdOpInstr::KindForOperator and CreateFromCall;
- extend CallSpecializer::InlineSimdOp and TryInlineRecognizedMethod
so the non-speculative null-check path used for Float32x4 operators
in AOT also applies here.
Measured on macOS arm64 (M-series), `dart compile exe`:
Issue 63217 orSimd : 12.58 -> 0.32 us/iter (39x)
Issue 63217 andNotSimd : 23.51 -> 0.34 us/iter (69x)
Issue 53662 mandelbrot : 4038.5 -> 55.5 ms (72x)
A new benchmark benchmarks/SimdInt32x4 exercises all five operators
with a scalar and a SIMD variant so the specialization stays covered
by the benchmark bots; it is registered in Omnibus and OmnibusDeferred.
Existing tests/lib/typed_data/simd_*_test.dart still pass in JIT and
AOT.
TEST=tests/lib/typed_data/int32x4_arithmetic_test; benchmarks/SimdInt32x4
Bug: https://github.com/dart-lang/sdk/issues/53662
Bug: https://github.com/dart-lang/sdk/issues/63217
Change-Id: I9b76ab4fff228ff1a5e3d3c86f4bfc059e66a49a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/497000
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
When enabled, indirect calls and jumps must set X7 to a label value that matches the label in a so-called landing pad at the call target. This check does not apply if the indirect call is made through X1/RA, X5/RA2 or X7/T2, with the expectation that the compiler will limit use of these registers for logically direct calls like an AUPIC+JALR pair, or for software guarded branches like a bounds-checked jump table. We use FAR_TMP for logically direct jumps that exceed J-type range, and blocking it from register allocation limits the availablity of gadgets that set X7.
Compare x64's IBT and arm64's BTI.
TEST=ci
Change-Id: Ic00195eb3c04cc10379f9965e05ef84052e02933
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494861
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This class is structurally equivalent to _HashVMBase and the only reason
for it to exist for inability of normal Dart classes to extend
_HashVMBase - but this can worked around by properly configuring
_HashVMBase class in bootstrapping.
TEST=ci
Change-Id: I5ca401e274920d2b4739424c6b6595306a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494140
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is a reland of commit 88496ba1c3
Fixes on top of the original change:
* Closure layout is changed to avoid gap between fixed fields and
variable-length elements on compressed pointers architecture.
This gap was causing crashes in the GC when scanning closure
objects.
* pkg/vm_snapshot_analysis/test/instruction_sizes_test is fixed
on arm64 by decreasing threshold for detecting size changes.
Original change's description:
> [vm,dart2bytecode,modular_aot] Variable-length closure objects
>
> Extend closure objects with variable number of elements to capture.
> This is needed to support capturing multiple independent contexts
> after capturing is computed in the front-end.
>
> The following fixed Closure fields are moved into variable-length
> elements:
> - delayed type arguments;
> - instantiator type arguments;
> - function type arguments;
> - context.
>
> Number of elements and presence/indices of various type arguments
> are encoded into the new length_and_flags field in the Closure.
>
> Most closure objects don't need any of the type arguments so this
> change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I8685e632e2d0832766ecdc470f3cf9a6b880de48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494243
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Extend closure objects with variable number of elements to capture.
This is needed to support capturing multiple independent contexts
after capturing is computed in the front-end.
The following fixed Closure fields are moved into variable-length
elements:
- delayed type arguments;
- instantiator type arguments;
- function type arguments;
- context.
Number of elements and presence/indices of various type arguments
are encoded into the new length_and_flags field in the Closure.
Most closure objects don't need any of the type arguments so this
change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I7ca5cec0fd8725c432a01d51781fb14e803997dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489482
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Fix the following cases of incorrect optimizations:
* In TFA, when anlyzing a dynamic call with unknown receiver type,
do not assume that all possible targets can be computed at compile
time (if there can be dynamically loaded classes).
* In the AOT, disable optimizations for dynamic calls with unique
selectors (if there can be dynamically loaded classes).
TEST=pkg/dynamic_modules/test/data/dyn_call_from_host
Change-Id: I39d620aae3c116de03d4a2a3fd61864d88c48c8e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493960
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Tony