Adds two new getters to int for bit-counting: trailingZeroBitCount
(ctz) and oneBitCount (popcount). On native platforms they operate
on the full 64-bit two's-complement representation; on the web they
operate on the least-significant 32 bits.
Implementations:
- VM: unified C++ natives Integer_trailingZeroBitCount /
Integer_oneBitCount on _IntegerImplementation, using
Utils::CountTrailingZeros64 and Utils::CountOneBits64. The receiver
may be _Smi or _Mint at runtime.
- dart2js / DDC: clz32-based ctz and a SWAR popcount.
- dart2wasm: inlined i64.ctz and i64.popcnt intrinsics.
leadingZeroBitCount (clz) is intentionally excluded from this CL: its
result depends on the platform integer width (e.g. 1.leadingZeroBitCount
is 31 on web, 63 on native), and the same value can be derived from
the existing bitLength getter when needed.
Asm intrinsification on native architectures is intentionally left for
a separate follow-up CL.
Work towards https://github.com/dart-lang/sdk/issues/6486 (this CL
covers popcount and ctz from the bit-twiddling list; clz, rotate,
reverse, and others remain).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Bug: https://github.com/dart-lang/sdk/issues/52673
Bug: https://github.com/dart-lang/sdk/issues/38346
TEST=tests/corelib/int_bit_count_test
Change-Id: I8a5cdb5c91360478f47bbd6b9c84ca1c477aa8c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498041
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
If catch block is included into the loop, we should also include
the whole try body as there are implicit control flow edges between
each block in a try body and corresponding catch block.
If try body is not included, analysis of loop invariant loads
could miss conflicting stores and may conclude that certain loads are
loop invariant although they are not.
TEST=runtime/tests/vm/dart/regress_63336_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63336
Change-Id: I9d6b73e24554ee3bfbcc722fa59634ea90b00abb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501560
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Letting isolate go seems to be problematic. If this native messaging api is used during runtime call, then original isolate might end up on a different worker os thread causing remembered frame pointer(remembered as part of entering runtime call) to become invalid due to different stack bounds of this new thread. Invalid frame pointer causes immediate assertion failures during stack walk done for GC purposes or for exception handling.
TEST=ci, https://dart-review.git.corp.google.com/c/sdk/+/486522/comments/ee54d99f_a5c2a9d4
Change-Id: If69076bb9e107502dfef819de73829d9682bb134
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501220
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
On 32-bit arm, when generating subtype test cache stub,
[delayed_type_args_reg] could be the same as [TypeTestABI::kInstanceReg]
(due to lack of registers).
As a result, when loading delayed type arguments from an instantiated
generic closure, [TypeTestABI::kInstanceReg] is eagerly overwritten,
which results in garbage delayed type arguments.
With the linear probe cache this would cause repeated cache misses.
With the hash-based cache this would cause a crash.
Bug: b/507666702
Change-Id: I1093f9277d874154a121ef644ecd7aa76320f789
TEST=runtime/tests/vm/dart/regress_b_507666702_test.dart
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500842
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL enables the primary constructors feature by default in Dart 3.13.
The primary constructors feature is a brevity feature. There are no new semantics, but it allows us to express declarations in a less verbose way.
This feature allows one constructor and a set of instance variables to be specified in the header of a declaration.
Currently a declaration with a constructor and some fields is written as:
```dart
// Current syntax.
class Point {
int x;
int y;
Point(this.x, this.y);
}
```
With a primary constructor, we would write the above as:
```
class Point(var int x, var int y);
```
If a primary constructor needs an initializer list or a body, they can be
specified inside the class using the `this` body syntax:
```dart
class Point(var int x, var int y) {
this : assert(x >= 0) {
print('Point created at $x, $y');
}
}
```
As part of this feature, you can also use the `new` and `factory` keywords to
declare constructors in the class body without repeating the class name:
```dart
class Point {
int x, y;
// Equivalent to Point(this.x, this.y)
new(this.x, this.y);
// Equivalent to Point.origin()
new origin() : x = 0, y = 0;
// Equivalent to factory Point.clone(Point other)
factory clone(Point other) => Point(other.x, other.y);
}
```
To learn more about the feature, check out the feature specification located here: https://github.com/dart-lang/language/blob/main/accepted/future-releases/primary-constructors/feature-specification.md
Tested: Has existing language, CFE, analyzer, analysis server tests.
Bug: https://github.com/dart-lang/sdk/issues/61524
Change-Id: I296f2fcd918b87bf2a1dd00256340759866c2423
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489241
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Michael Thomsen <mit@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Kallen Tu <kallentu@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Require the embedder to place a blob containing the FFI callback stub at pkg/lib/ffi_callback_stub.bin. All blobs in the package can be loaded as executable VMOs.
Also reapply "Remove special case for simulator FFI callback thunks."
TEST=ci
Change-Id: I6ab4a73fc28fb750e45e67797b0bf9deee9d8b53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/495840
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
These will be base objects even for kFullAOT and kFullJIT snapshots, so their allocation needs to be separated from the rest of Object::Init.
TEST=ci
Change-Id: I1ad03cf9900a925d3b084843206bba7f67c5daa9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/499083
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Call sites targeting a procedure annotated with `external-effect` will
not produce any code, including the argument which will not be
evaluated.
However, the single parameter will be treated as 'live' for the purposes
of any global analysis the backends do. This is useful for things like
protobuf shaking where a user may want to retain certain protobuf
messages without actually emitting the code that retains those messages.
Today this functionality is available internally in the vm and wasm SDK
libraries. dart2js has similar functionality represented via the
opaqueTrue and opaqueFalse booleans (which will cause conditional
branches to get shaken after analysis). This will replace dart2js's
opaque(True/False).
This also adds validation to the frontend to ensure a method annotated
with 'external-effect' is well-formed.
Change-Id: If1c4096673e655c58fe7638840a16125003e7809
Tested: Backend tests for codegen were added. A frontend test was added for the validation. A language test was added to confirm the behavior.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476020
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
To support checking for dynamically-callable targets in dart dynamic
modules, we need to use a bit in the Function header in AOT. Currently
all 32 bits of `kind_tags_` are in use. To make space for that new bit,
we need to evict one of the current properties that is not needed by AOT.
Among them, `IsRedirectingFactory` made the top of the list. It has only
one use in non-AOT logic.
TEST=existing
Bug: b/448095881
Change-Id: I3112d8865523696ed8e906a6c59f53c23db3090a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498281
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The 2nd argument of _simpleInstanceOf call is always a
Constant (type). However, a Redefinition can be inserted in the middle.
So, instead of ArgumentAt(1)->AsConstant()->value() it is
more safe to use ArgumentValueAt(1)->BoundConstant() as
BindsToConstant/BoundConstant unwraps Redefinition(s) via
OriginalDefinition().
TEST=runtime/tests/vm/dart/regress_63211_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63211
Change-Id: Ie4a473ebe2deee8562e6634a792f02b0dcefc918
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/497761
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The five binary operators on Int32x4 (+, -, |, &, ^) were never added
to the recognized-method list as graph intrinsics, so calls to them
were left as runtime calls through external-name bodies. In JIT the
call specializer picked kInt32x4Cid from IC feedback and emitted a
native SimdOpInstr, but AOT has no IC feedback and therefore fell back
to boxed calls, making Int32x4List inner loops 10-70x slower than both
the JIT version and a hand-written scalar equivalent.
This CL wires the same specialization paths that already exist for
Float32x4.+,-,*,/:
- recognize the five operators as graph intrinsics and mark them
with `@pragma("vm:recognized", "graph-intrinsic")` plus an
exact-result-type pragma;
- add Build_Int32x4{Add,Sub,BitAnd,BitOr,BitXor} helpers that
delegate to the existing BuildSimdOp;
- extend SimdOpInstr::KindForOperator and CreateFromCall;
- extend CallSpecializer::InlineSimdOp and TryInlineRecognizedMethod
so the non-speculative null-check path used for Float32x4 operators
in AOT also applies here.
Measured on macOS arm64 (M-series), `dart compile exe`:
Issue 63217 orSimd : 12.58 -> 0.32 us/iter (39x)
Issue 63217 andNotSimd : 23.51 -> 0.34 us/iter (69x)
Issue 53662 mandelbrot : 4038.5 -> 55.5 ms (72x)
A new benchmark benchmarks/SimdInt32x4 exercises all five operators
with a scalar and a SIMD variant so the specialization stays covered
by the benchmark bots; it is registered in Omnibus and OmnibusDeferred.
Existing tests/lib/typed_data/simd_*_test.dart still pass in JIT and
AOT.
TEST=tests/lib/typed_data/int32x4_arithmetic_test; benchmarks/SimdInt32x4
Bug: https://github.com/dart-lang/sdk/issues/53662
Bug: https://github.com/dart-lang/sdk/issues/63217
Change-Id: I9b76ab4fff228ff1a5e3d3c86f4bfc059e66a49a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/497000
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Some objects have their canonical bit set but don't have the deeply immutable bit set. It's simpler if they have this bit clear so there is only cluster need, and these objects are instances of VM-internal classes that don't really do canonicalization.
Also give the oddballs their header hash while doing the rest of their initialization. This makes it easier to use the as snapshot base objects when there is no finalize VM isolate step.
TEST=ci
Change-Id: I7bdf7dc59b04ae30f49c5c67c0dc4f69c2d2524f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/497121
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
When enabled, indirect calls and jumps must set X7 to a label value that matches the label in a so-called landing pad at the call target. This check does not apply if the indirect call is made through X1/RA, X5/RA2 or X7/T2, with the expectation that the compiler will limit use of these registers for logically direct calls like an AUPIC+JALR pair, or for software guarded branches like a bounds-checked jump table. We use FAR_TMP for logically direct jumps that exceed J-type range, and blocking it from register allocation limits the availablity of gadgets that set X7.
Compare x64's IBT and arm64's BTI.
TEST=ci
Change-Id: Ic00195eb3c04cc10379f9965e05ef84052e02933
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494861
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Make VM always return empty array in the response.
Current implementation for this field comes with a bunch of complexity
because it locks message handler and then invokes Dart code which
makes it difficult to reason about various invariants. This code is
furthermore demonstrated to cause deadlocks. Given that nobody uses
it - it is simpler to remove this code altogether.
Fixes https://github.com/flutter/flutter/issues/185156
TEST=ci
Change-Id: I497210e0f1542860caa0d765d634f8ec6a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/496340
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Also allows the --print-classes flag to be used in the AOT runtime if
dynamic modules are enabled and crashes with an appropriate error
message if class finalization fails when loading a member from bytecode.
TEST=co19/LibTest/math/Rectangle co19/LibTest/math/MutableRectangle
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I1921aa0189eb587cd4658c592a779be190af092e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/495724
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
We can do a "native" stack walk starting from the interrupt context, a Dart stack walk starting from the exit frame, or a Dart stack walk starting from the interrupt context / simulator state / interpreter state.
Decide which kind of stack walk to do once instead both inside and outside CollectSample. Remove CollectSample, as we should not need SEH guards now that we have accurate stack bounds.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/63105
Change-Id: I8f46f2ab860220e0be373f02aa882ac096bb2fb1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493969
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Dynamic modules may make dynamic calls to compiled methods that had no
dynamic calls in the original whole world compilation. If this happens,
then dynamically create an interpreted dynamic invocation forwarder that
then appropiately checks and delegates to the compiled function.
TEST=co19/LibTest
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I283ca501f50118606650cf434b8debe30a43f676
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This class is structurally equivalent to _HashVMBase and the only reason
for it to exist for inability of normal Dart classes to extend
_HashVMBase - but this can worked around by properly configuring
_HashVMBase class in bootstrapping.
TEST=ci
Change-Id: I5ca401e274920d2b4739424c6b6595306a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494140
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is a reland of commit 88496ba1c3
Fixes on top of the original change:
* Closure layout is changed to avoid gap between fixed fields and
variable-length elements on compressed pointers architecture.
This gap was causing crashes in the GC when scanning closure
objects.
* pkg/vm_snapshot_analysis/test/instruction_sizes_test is fixed
on arm64 by decreasing threshold for detecting size changes.
Original change's description:
> [vm,dart2bytecode,modular_aot] Variable-length closure objects
>
> Extend closure objects with variable number of elements to capture.
> This is needed to support capturing multiple independent contexts
> after capturing is computed in the front-end.
>
> The following fixed Closure fields are moved into variable-length
> elements:
> - delayed type arguments;
> - instantiator type arguments;
> - function type arguments;
> - context.
>
> Number of elements and presence/indices of various type arguments
> are encoded into the new length_and_flags field in the Closure.
>
> Most closure objects don't need any of the type arguments so this
> change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I8685e632e2d0832766ecdc470f3cf9a6b880de48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/494243
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The handling of entry frame in PrintStackFrames was broken, now it's
fixed.
In addition, the stack printer now print various info about the static
slots in stack frames, as well as printing out the objects in slots.
Now that entry frames are handled properly, printing with no limit
prints _all_ stack frames on the interpreter stack, noting call
boundaries as they are passed, instead of stopping at the closest
entry frame.
Move the separator between frame from the stack pointer for a frame
to the frame pointer for a frame, meaning that the metadata before
the frame pointer is printed as part of the previous frame, not the
current one.
TEST=ci (manual testing while debugging)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: Ia2ad07c832b791f0c2ce2bbdfbfc32d5d8968476
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493402
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Extend closure objects with variable number of elements to capture.
This is needed to support capturing multiple independent contexts
after capturing is computed in the front-end.
The following fixed Closure fields are moved into variable-length
elements:
- delayed type arguments;
- instantiator type arguments;
- function type arguments;
- context.
Number of elements and presence/indices of various type arguments
are encoded into the new length_and_flags field in the Closure.
Most closure objects don't need any of the type arguments so this
change will reduce average Closure object size.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/61572
Issue: https://github.com/dart-lang/sdk/issues/61635
Change-Id: I7ca5cec0fd8725c432a01d51781fb14e803997dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/489482
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Fix the following cases of incorrect optimizations:
* In TFA, when anlyzing a dynamic call with unknown receiver type,
do not assume that all possible targets can be computed at compile
time (if there can be dynamically loaded classes).
* In the AOT, disable optimizations for dynamic calls with unique
selectors (if there can be dynamically loaded classes).
TEST=pkg/dynamic_modules/test/data/dyn_call_from_host
Change-Id: I39d620aae3c116de03d4a2a3fd61864d88c48c8e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/493960
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Modestas Valauskas