This adds serialization of the new local variables, VariableStatement and VariableDeclaration. This prepares for replacing LegacyVariable with the new variables in all backends.
TEST=existing
Change-Id: I9bbebfbfd372042d6b7027f0fabd24c165699832
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
On mac with two WorkSet classes, WorkSet destructor from lib/isolate.cc is invoked instead of WorkSet destructor from vm/object.cc. This results in consistent assertion failures on flutter dart-debug builds on mac.
Might help with https://github.com/flutter/flutter/issues/186134
TEST=flutter test with dart-debug build
Change-Id: I9a0a0c9d02ee013da900807b8726df26a3660cc7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505661
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When the unoptimized code of a function exceeds 1 MB, start its
optimized compile with far_branch_level=1 instead of 0. Without this,
the assembler fails on out-of-range conditional branches at level=0,
longjmps out of the optimizing pipeline, and the whole pipeline reruns
at level=1 from scratch.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted: JIT compile drops from ~26.6s to ~13.2s (about 50%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: I2608a23ef899603a46b401b71be7322b7a8dd44b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506120
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: I9c2c4225fd63d54e190398a0fb6649366317dd17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504221
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Generalize DefinitionWorklist into a Worklist<T, E> template and use a
new BlockEntryWorklist (keyed by preorder_number) to drive the WBE
dataflow. The bitvector-backed membership set prevents the same block
from being enqueued many times, which is what caused compile time to
blow up on large generated programs.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: AOT compile drops from
~42s to ~19s.
Work towards https://github.com/dart-lang/sdk/issues/63230
Fixes https://github.com/dart-lang/sdk/issues/63426
TEST=ci
Change-Id: I56c5dc40f534e4b28fe6b608a3b7f49e14957ef7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This test has been failing with the following errors on
iso-stress-linux-x64 bot:
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:36755/, ]
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:46789/, ]
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:37003/, ]
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:42069/, ]
TEST=ci
Change-Id: Ibebd6fe681ae06a098109557051bc0e8d73e74c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505360
Reviewed-by: Alexander Aprelev <aam@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Previously, the loop threw away the calculated hash value, so the
finalized hash was only the combined hash from the length plus last
byte.
Fix this, and also use 32-bit chunks as much as possible to reduce the
number of iterations for larger typed data objects.
TEST=ci
Change-Id: I0c033701232f1a1299356cab0b09079f9f86c026
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504980
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Coverage information for enhanced enums should not include initializer
functions for enum elements or the values field, so skip over these when
the enum is defined in bytecode.
TEST=vm/cc/SourceReport_Coverage_IssueCov386_EnhancedEnums
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I1dba3c87fdfcc39762bba5f0c35cfd061ec76371
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504240
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The RecordCoverage instruction has an A/E encoding. The A argument
is the type of coverage being recorded, whereas the E argument is
the logical index into the coverage array for updating whether that
source position has been hit.
Also adds new metadata to the bytecode component for the coverage
arrays associated with bytecode containing RecordCoverage instructions
and a new runtime entry for lazily allocate the coverage array for
an interpreted function when needed.
The type of coverage is encoded in the RecordCoverage instruction,
despite being redundant with the information in the coverage array, so that checking whether that type of coverage is currently enabled at
runtime doesn't require either accessing the coverage array (which may
be lazily allocated), forcing allocation of the coverage array just to
discover that type of coverage is currently disabled, or reading the
serialized bytecode component to avoid that forced allocation.
------
Other changes:
Source reporting now treats unexecuted interpreted functions when
not forcing compilation as if they were uncompiled native functions,
so that the source report from running the same code gives the same
result whether using the interpreter or the native compiler.
Bytecode closures are no longer skipped in source reports. Previously
any closure without a context scope was skipped, but bytecode closures
don't have those.
TEST=vm/cc/SourceReport_Coverage
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I7557e5dd4c98331c7ca2f5c867dd5f6d03e9d756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This updates the IOHandle type in eventhandler_fuchsia to use
zx_port_cancel_key to cancel outstanding asynchronous waits instead of
the (older) zx_port_cancel. The cancel_key operation requires only the
key used to register the initial wait and not the originating handle.
This means that the IOHandle operation no longer has to store the
original handle or worry about its lifetime.
Tested: Patched locally in a test embedder using this logic
Change-Id: Iec0ec632a7dd438a34e482a12b5b298be56ce476
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501840
Reviewed-by: Zijie He <zijiehe@google.com>
Auto-Submit: James Robinson <jamesr@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
At this time, this flag is mainly going to be used to enable partial
testing of the JIT runtime when loading applications from kernel
snapshots.
Bug: b/448095881
TEST=none yet, will be added in upcoming CL
Change-Id: I272d43cdab0c9a9c3653cccc9e96961c1122bbd7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/503460
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Unlike other calls from Dynamic Modules, dynamic calls cannot be
validated entirely at compile time. While we check that the selector
used matches a selector that was allowed (either because a method with
that selector name was exposed as dynamically callable or because the
selector was allowlisted during bytecode compilation), the compiler
doesn't know statically whether the target of the call is exposed.
In prior changes we modified the annotator to add a pragma indicating
whether a member is dynamically-callable or
implicitly-dynamically-callable. Here we use that information to set a
bit on functions and their corresponding dynamic invocation forwarders,
which is verified by the interpreter to make sure the dynamic call is
still allowed.
TEST=none yet - will be added in subsequent CL (see CL chain)
Bug: b/448095881
Change-Id: I27acb4e690a68e08fe1f1ca94e0d77cc7dc4d11e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498300
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Adds isSystemIsolate in the isolate startup message sent to the
VM service, allowing for the service to identify the first non-system
isolate as the root isolate.
TEST=Existing tests
CoreLibraryReviewExempt: dart:vmservice is internal.
Change-Id: I0a982a1fc06bd0be0426ad9d1401e89375cbed40
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500000
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Jessy Yameogo <yjessy@google.com>
With the removal of the VM isolate and less bootstrapping being repeated in the AOT runtime, ApiError cannot be instantiated until after snapshot loading succeeds.
TEST=ci
Change-Id: I5be295e87e6d6351ada40d2837825b1f93672ac7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501262
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
In the VM, factory constructors always had an extra "type arguments"
parameter, even if class is not generic. Factory constructor bodies
were using class type parameters instead of function type parameters.
This results in extra code when calling non-generic factories
which is slightly inefficient in terms of code size and performance.
Also, it creates an additional complexity throughout the system as
factories should be special cased in many places.
This change removes artificial "type arguments" parameter, treating
factory constructors basically as static methods. This matches
kernel AST representation.
TEST=ci
Change-Id: I957583cb2ce9a3c408699880a04036e06b01dd31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501762
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This test verifies how deoptIds and environments are set
when inlining force-optimized functions.
The problem is that it looks at the chain of inlined calls:
foo() {
call1 -> Pointer.fromAddress
}
Pointer.fromAddress() {
call2 -> _fromAddress
}
// force-optimized
_fromAddress() { ... }
During inlining of force-optimized callee, its instructions
should receive deoptId/env from inlined call site.
In this case, the call site is 'call2'.
However, the test has been matching deoptId/env with 'call1'.
It was a mere coincidence that deoptIds of call1 and call2
were the same. Any attempt to make a change which would
affect deoptIds within Pointer.fromAddress breaks this test.
In order to fix this, a simpler setup is now used:
foo() {
call1 -> newHash()
}
// force-optimized
newHash() { ... }
Also, the check for deoptId of outermost environment is dropped as
in this case there are no multiple environments.
In addition, also remove the duplicate pragma from _fromAddress.
TEST=vm/cc/Inliner_InlineForceOptimized
Change-Id: I1f592a74dfb8b7c2f67f3b0a51e7f7b74bf358eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/502801
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When checking number of arguments passed by the caller against
number of parameters in the callee, inliner was not taking
"type arguments" argument into account which resulted in
the rejected inline if all optional parameters are passed.
TEST=runtime/tests/vm/dart/inline_generic_functions_il_test.dart
Change-Id: I38d1fd2a1a4aadfb0196e5a472d224e3da3a1562
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/502800
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Adds two new getters to int for bit-counting: trailingZeroBitCount
(ctz) and oneBitCount (popcount). On native platforms they operate
on the full 64-bit two's-complement representation; on the web they
operate on the least-significant 32 bits.
Implementations:
- VM: unified C++ natives Integer_trailingZeroBitCount /
Integer_oneBitCount on _IntegerImplementation, using
Utils::CountTrailingZeros64 and Utils::CountOneBits64. The receiver
may be _Smi or _Mint at runtime.
- dart2js / DDC: clz32-based ctz and a SWAR popcount.
- dart2wasm: inlined i64.ctz and i64.popcnt intrinsics.
leadingZeroBitCount (clz) is intentionally excluded from this CL: its
result depends on the platform integer width (e.g. 1.leadingZeroBitCount
is 31 on web, 63 on native), and the same value can be derived from
the existing bitLength getter when needed.
Asm intrinsification on native architectures is intentionally left for
a separate follow-up CL.
Work towards https://github.com/dart-lang/sdk/issues/6486 (this CL
covers popcount and ctz from the bit-twiddling list; clz, rotate,
reverse, and others remain).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Bug: https://github.com/dart-lang/sdk/issues/52673
Bug: https://github.com/dart-lang/sdk/issues/38346
TEST=tests/corelib/int_bit_count_test
Change-Id: I8a5cdb5c91360478f47bbd6b9c84ca1c477aa8c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498041
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
If catch block is included into the loop, we should also include
the whole try body as there are implicit control flow edges between
each block in a try body and corresponding catch block.
If try body is not included, analysis of loop invariant loads
could miss conflicting stores and may conclude that certain loads are
loop invariant although they are not.
TEST=runtime/tests/vm/dart/regress_63336_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63336
Change-Id: I9d6b73e24554ee3bfbcc722fa59634ea90b00abb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501560
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Ryan Macnak