The original iteration of adding coverage arrays originally eagerly
allocated coverage arrays for bytecode methods with recorded coverage.
Now that these coverage arrays are lazily allocated,
Function::GetCoverageArray() needs to ensure that the appropriate
coverage array has been allocated. Otherwise, no misses within the
function are recorded in cases where no RecordCoverage instruction in
the method has yet been executed.
TEST=vm/cc/SourceReport_Coverage_UnusedClass_ForceCompile
pkg/vm_service/test/branch_coverage_test
pkg/vm_service/test/coverage_async_test
pkg/vm_service/test/coverage_closure_call_after_optimization_test
pkg/vm_service/test/coverage_closure_call_test
pkg/vm_service/test/coverage_instance_call_after_optimization_test
pkg/vm_service/test/coverage_static_call_after_optimization_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Iff4ddaf583d0f9583bc415142bdee52c046a6712
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507440
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Extend --check-clean (enabled on bots) to verify build commands don't include absolute paths.
The non-RBE GCC and MSVC builds still have absolute paths. GCC doesn't have -ffile-compilation-dir so it uses -fdebug-prefix-map, which is not fixable. The MSVC build has absolute path in the toolchain wrappers, which might be fixable.
TEST=ci
Change-Id: I3b984aaab7aefa7ff527f0a039ca42281224a09d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506505
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Avoid a race condition between async setenv/unsetenv calls on the VM thread and concurrent getenv/environ reads on the main thread.
In glibc, setenv and getenv are not thread-safe against each other. VmInteropHandler.setEnvironmentVariable previously sent a message to the VM thread to call setenv/unsetenv asynchronously, while the main thread immediately proceeded to access Platform.environment or Platform.localeName (which calls getenv). This could cause a crash (SIGSEGV) in getenv.
This CL makes VmInteropHandler.setEnvironmentVariable synchronous by awaiting a reply from the VM thread before returning.
TEST=pkg/dartdev/test/environment_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63460
Change-Id: Ic8211897ce26ffbdc142fa594cd397189a61f061
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506800
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Auto-Submit: Ben Konyi <bkonyi@google.com>
The previous attempt was reverted because it broke unoptimized JIT
on ARM 32. This reland force-optimizes the two getters.
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Fixes https://github.com/dart-lang/sdk/issues/63436
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: Ib812cbaec6e371b9720df7a543411f78e524cac1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506060
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The Integer_parse VM native has no Dart-side caller. int.parse is
implemented entirely in Dart in
sdk/lib/_internal/vm_shared/lib/integers_patch.dart (_tryParseSmi
and _parseRadix). Removes the native entry and drops a stale
reference in the comment on the shared ParseInteger helper, which
is still used by Integer_fromEnvironment.
TEST=ci
Change-Id: I8d034e3081c09357094abc6415fc709e3edbea07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507322
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Add `final` to PredecessorAt and PredecessorCount on the six concrete
BlockEntryInstr subclasses (GraphEntry, JoinEntry, TargetEntry,
FunctionEntry, OsrEntry, CatchBlockEntry), so the compiler can
devirtualize the calls.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: JIT compile drops from
~30.6s to ~27.4s (about 10%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: Ia2ef2761646a4c94748bcd7dfe0591262d93cd69
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505401
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
The test tries to keep fields alive by my making a
toString() => field1 + field2 + ...
Though some field values don't have a `+` operator (e.g. instances of `class C`).
Improving TFA precision can infer that some of these `+` operations will throw,
which then makes other `+` operations unreachable, which removes usages
of those other fields, which will tree shake those other fields, which
will make the test not test anymore what it's intended to.
=> Make the test more robust by using string concatenation instead.
Change-Id: I80788af516083ea3d78ad910eb394e4b7e122384
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507000
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Implement api methods to create and shutdown isolates from IsolateGroup-bound callbacks (normally invoked from native code), run dart code on such isolates.
The rest of the api is not implemented yet.
TEST=tests/ffi/threading_test.dart
BUG=https://github.com/dart-lang/sdk/issues/62407
CoreLibraryReviewExempt: vm-only change to isolate library
Change-Id: I0271ead8ba011dfe9d7953769415d6a88a962854
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486522
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Also load/store canonical hashes in the heap for non-empty TypedData
instances in the same manner as canonical hashes for Arrays.
TEST=ci (refactoring only)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I54274b558fa9f0c8e304198b18cb3f0e9c3e0dfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504600
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This adds serialization of the new local variables, VariableStatement and VariableDeclaration. This prepares for replacing LegacyVariable with the new variables in all backends.
TEST=existing
Change-Id: I9bbebfbfd372042d6b7027f0fabd24c165699832
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
On mac with two WorkSet classes, WorkSet destructor from lib/isolate.cc is invoked instead of WorkSet destructor from vm/object.cc. This results in consistent assertion failures on flutter dart-debug builds on mac.
Might help with https://github.com/flutter/flutter/issues/186134
TEST=flutter test with dart-debug build
Change-Id: I9a0a0c9d02ee013da900807b8726df26a3660cc7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505661
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When the unoptimized code of a function exceeds 1 MB, start its
optimized compile with far_branch_level=1 instead of 0. Without this,
the assembler fails on out-of-range conditional branches at level=0,
longjmps out of the optimizing pipeline, and the whole pipeline reruns
at level=1 from scratch.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted: JIT compile drops from ~26.6s to ~13.2s (about 50%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: I2608a23ef899603a46b401b71be7322b7a8dd44b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506120
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: I9c2c4225fd63d54e190398a0fb6649366317dd17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504221
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Generalize DefinitionWorklist into a Worklist<T, E> template and use a
new BlockEntryWorklist (keyed by preorder_number) to drive the WBE
dataflow. The bitvector-backed membership set prevents the same block
from being enqueued many times, which is what caused compile time to
blow up on large generated programs.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: AOT compile drops from
~42s to ~19s.
Work towards https://github.com/dart-lang/sdk/issues/63230
Fixes https://github.com/dart-lang/sdk/issues/63426
TEST=ci
Change-Id: I56c5dc40f534e4b28fe6b608a3b7f49e14957ef7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This test has been failing with the following errors on
iso-stress-linux-x64 bot:
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:36755/, ]
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:46789/, ]
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:37003/, ]
[tests/standalone/io/http_client_connect_test.dart] error: [HttpException: Connection reset by peer, uri = http://127.0.0.1:42069/, ]
TEST=ci
Change-Id: Ibebd6fe681ae06a098109557051bc0e8d73e74c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505360
Reviewed-by: Alexander Aprelev <aam@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Previously, the loop threw away the calculated hash value, so the
finalized hash was only the combined hash from the length plus last
byte.
Fix this, and also use 32-bit chunks as much as possible to reduce the
number of iterations for larger typed data objects.
TEST=ci
Change-Id: I0c033701232f1a1299356cab0b09079f9f86c026
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504980
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Coverage information for enhanced enums should not include initializer
functions for enum elements or the values field, so skip over these when
the enum is defined in bytecode.
TEST=vm/cc/SourceReport_Coverage_IssueCov386_EnhancedEnums
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I1dba3c87fdfcc39762bba5f0c35cfd061ec76371
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504240
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The RecordCoverage instruction has an A/E encoding. The A argument
is the type of coverage being recorded, whereas the E argument is
the logical index into the coverage array for updating whether that
source position has been hit.
Also adds new metadata to the bytecode component for the coverage
arrays associated with bytecode containing RecordCoverage instructions
and a new runtime entry for lazily allocate the coverage array for
an interpreted function when needed.
The type of coverage is encoded in the RecordCoverage instruction,
despite being redundant with the information in the coverage array, so that checking whether that type of coverage is currently enabled at
runtime doesn't require either accessing the coverage array (which may
be lazily allocated), forcing allocation of the coverage array just to
discover that type of coverage is currently disabled, or reading the
serialized bytecode component to avoid that forced allocation.
------
Other changes:
Source reporting now treats unexecuted interpreted functions when
not forcing compilation as if they were uncompiled native functions,
so that the source report from running the same code gives the same
result whether using the interpreter or the native compiler.
Bytecode closures are no longer skipped in source reports. Previously
any closure without a context scope was skipped, but bytecode closures
don't have those.
TEST=vm/cc/SourceReport_Coverage
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I7557e5dd4c98331c7ca2f5c867dd5f6d03e9d756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Ryan Macnak