- Break false dependency of non-VM platforms on VM platform
- Break false dependency of bootstrap gen_kernel on VM platform
- Extend precompile_tools to gen_kernel steps
- Rename intermediate kernel files so JIT versus AOT is visible in ninjatracing
TEST=ci
Change-Id: I07011abe8303597af61d2b8c73e788b661482cc0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/510060
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Implicit setters of covariant fields (both covariant by declaration and
covariant by class aka generic-covariant-impl) need to perform a type
check of their argument.
It means that inferred type of the field cannot be used as inferred
type of parameter of such setter. This change removes such uses of
an inferred type of the field, which restores parameter type check
which was previously incorrectly optimized out.
TEST=runtime/tests/vm/dart/regress_63419_test.dart
Fixes https://github.com/dart-lang/sdk/issues/63419
Change-Id: Ie313dbaab51dff15d60ce4390e7e41bdc66ad59d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/509020
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
- Declare OS::SafeReadMemory in os.h
- Implement using pread64(/proc/self/mem) on Linux and Android
- Add _readNativeMemory VM Service RPC with pre-checks for null
and address overflow
TEST=runtime/vm/service_test.cc
Cq-Include-Trybots: dart/try:vm-asan-linux-release-x64-try,vm-asan-mac-release-arm64-try,vm-asan-win-release-x64-try,vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-arm64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-simarm-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-tsan-linux-release-x64-try,vm-tsan-mac-release-arm64-try,vm-ubsan-linux-release-x64-try,vm-ubsan-mac-release-arm64-try,vm-ubsan-win-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try
Change-Id: Id15a82bf478bc4822c08d7fdf0a5c8bfd71a1fe0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505781
Auto-Submit: Nourhan Hasan <nourhan.m.hasan@gmail.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
The only actual change required to the profiler service is appropriately
retrieving the source position for interpreted frames during processing.
Otherwise, most of the tests just need small changes to account for the
slightly different stack trace generated by interpreted code when
walking the stack as code instead of functions:
* Interpreted code calls the runtime entry DRT_AllocateObject directly
for slow-path allocation instead of via the AllocateObjectSlow stub.
* The current name for bytecode frames when walked as code starts with
"[Bytecode]" instead of "[Unoptimized]".
The test vm/cc/Profiler_GetSourceReport is skipped since:
* It tests compiled code-specific features like classifying source
positions.
* Inserting fake samples for interpreted functions is messier since
many PC offsets may map to the same source position.
The other skipped tests are specific to inlining and/or optimized
compiled code, so there's no point in running them on interpreted code.
TEST=vm/cc/Profiler
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I3e73f42b2562eb6e3e73fe8d5c3f14389039ee04
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507082
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
The debugger assumes a null suspend state variable in an asynchronous
function means that the function is still in the prologue prior
to setting up the suspend state. However, the interpreter clears the
suspend state variable before returning, and the debugger needs to be
able to pause before returning when single stepping, so earlier a hack
was added to the debugger that detects being at the direct call of the
async return method and/or the return instruction with a null suspend
state variable.
However, there's a much simpler way of ensuring the debugger pauses
before returning: just emit the source position for the return prior to
clearing the suspend state variable. This also ensures that the debugger
still has access to the function's suspend state when pausing before
the return, instead of waiting until it has been cleared and thus is no
longer accessible.
TEST=ci (should not change the result of any current tests)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I401cceb169d8692ac379cdc5a531e07cafbe9a65
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500740
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The original iteration of adding coverage arrays originally eagerly
allocated coverage arrays for bytecode methods with recorded coverage.
Now that these coverage arrays are lazily allocated,
Function::GetCoverageArray() needs to ensure that the appropriate
coverage array has been allocated. Otherwise, no misses within the
function are recorded in cases where no RecordCoverage instruction in
the method has yet been executed.
TEST=vm/cc/SourceReport_Coverage_UnusedClass_ForceCompile
pkg/vm_service/test/branch_coverage_test
pkg/vm_service/test/coverage_async_test
pkg/vm_service/test/coverage_closure_call_after_optimization_test
pkg/vm_service/test/coverage_closure_call_test
pkg/vm_service/test/coverage_instance_call_after_optimization_test
pkg/vm_service/test/coverage_static_call_after_optimization_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: Iff4ddaf583d0f9583bc415142bdee52c046a6712
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507440
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Extend --check-clean (enabled on bots) to verify build commands don't include absolute paths.
The non-RBE GCC and MSVC builds still have absolute paths. GCC doesn't have -ffile-compilation-dir so it uses -fdebug-prefix-map, which is not fixable. The MSVC build has absolute path in the toolchain wrappers, which might be fixable.
TEST=ci
Change-Id: I3b984aaab7aefa7ff527f0a039ca42281224a09d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506505
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
The previous attempt was reverted because it broke unoptimized JIT
on ARM 32. This reland force-optimizes the two getters.
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Fixes https://github.com/dart-lang/sdk/issues/63436
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: Ib812cbaec6e371b9720df7a543411f78e524cac1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506060
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The Integer_parse VM native has no Dart-side caller. int.parse is
implemented entirely in Dart in
sdk/lib/_internal/vm_shared/lib/integers_patch.dart (_tryParseSmi
and _parseRadix). Removes the native entry and drops a stale
reference in the comment on the shared ParseInteger helper, which
is still used by Integer_fromEnvironment.
TEST=ci
Change-Id: I8d034e3081c09357094abc6415fc709e3edbea07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507322
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Add `final` to PredecessorAt and PredecessorCount on the six concrete
BlockEntryInstr subclasses (GraphEntry, JoinEntry, TargetEntry,
FunctionEntry, OsrEntry, CatchBlockEntry), so the compiler can
devirtualize the calls.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: JIT compile drops from
~30.6s to ~27.4s (about 10%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: Ia2ef2761646a4c94748bcd7dfe0591262d93cd69
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505401
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Implement api methods to create and shutdown isolates from IsolateGroup-bound callbacks (normally invoked from native code), run dart code on such isolates.
The rest of the api is not implemented yet.
TEST=tests/ffi/threading_test.dart
BUG=https://github.com/dart-lang/sdk/issues/62407
CoreLibraryReviewExempt: vm-only change to isolate library
Change-Id: I0271ead8ba011dfe9d7953769415d6a88a962854
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486522
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Also load/store canonical hashes in the heap for non-empty TypedData
instances in the same manner as canonical hashes for Arrays.
TEST=ci (refactoring only)
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try
Change-Id: I54274b558fa9f0c8e304198b18cb3f0e9c3e0dfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504600
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This adds serialization of the new local variables, VariableStatement and VariableDeclaration. This prepares for replacing LegacyVariable with the new variables in all backends.
TEST=existing
Change-Id: I9bbebfbfd372042d6b7027f0fabd24c165699832
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
On mac with two WorkSet classes, WorkSet destructor from lib/isolate.cc is invoked instead of WorkSet destructor from vm/object.cc. This results in consistent assertion failures on flutter dart-debug builds on mac.
Might help with https://github.com/flutter/flutter/issues/186134
TEST=flutter test with dart-debug build
Change-Id: I9a0a0c9d02ee013da900807b8726df26a3660cc7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505661
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When the unoptimized code of a function exceeds 1 MB, start its
optimized compile with far_branch_level=1 instead of 0. Without this,
the assembler fails on out-of-range conditional branches at level=0,
longjmps out of the optimizing pipeline, and the whole pipeline reruns
at level=1 from scratch.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted: JIT compile drops from ~26.6s to ~13.2s (about 50%).
Work towards https://github.com/dart-lang/sdk/issues/63230
TEST=ci
Change-Id: I2608a23ef899603a46b401b71be7322b7a8dd44b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506120
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: I9c2c4225fd63d54e190398a0fb6649366317dd17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504221
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Generalize DefinitionWorklist into a Worklist<T, E> template and use a
new BlockEntryWorklist (keyed by preorder_number) to drive the WBE
dataflow. The bitvector-backed membership set prevents the same block
from being enqueued many times, which is what caused compile time to
blow up on large generated programs.
Measured on a naive 49 KLOC generated lexer with --huge_method_cutoff
gates lifted and with synchronous compilation: AOT compile drops from
~42s to ~19s.
Work towards https://github.com/dart-lang/sdk/issues/63230
Fixes https://github.com/dart-lang/sdk/issues/63426
TEST=ci
Change-Id: I56c5dc40f534e4b28fe6b608a3b7f49e14957ef7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/505461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Previously, the loop threw away the calculated hash value, so the
finalized hash was only the combined hash from the length plus last
byte.
Fix this, and also use 32-bit chunks as much as possible to reduce the
number of iterations for larger typed data objects.
TEST=ci
Change-Id: I0c033701232f1a1299356cab0b09079f9f86c026
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504980
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Coverage information for enhanced enums should not include initializer
functions for enum elements or the values field, so skip over these when
the enum is defined in bytecode.
TEST=vm/cc/SourceReport_Coverage_IssueCov386_EnhancedEnums
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I1dba3c87fdfcc39762bba5f0c35cfd061ec76371
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504240
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The RecordCoverage instruction has an A/E encoding. The A argument
is the type of coverage being recorded, whereas the E argument is
the logical index into the coverage array for updating whether that
source position has been hit.
Also adds new metadata to the bytecode component for the coverage
arrays associated with bytecode containing RecordCoverage instructions
and a new runtime entry for lazily allocate the coverage array for
an interpreted function when needed.
The type of coverage is encoded in the RecordCoverage instruction,
despite being redundant with the information in the coverage array, so that checking whether that type of coverage is currently enabled at
runtime doesn't require either accessing the coverage array (which may
be lazily allocated), forcing allocation of the coverage array just to
discover that type of coverage is currently disabled, or reading the
serialized bytecode component to avoid that forced allocation.
------
Other changes:
Source reporting now treats unexecuted interpreted functions when
not forcing compilation as if they were uncompiled native functions,
so that the source report from running the same code gives the same
result whether using the interpreter or the native compiler.
Bytecode closures are no longer skipped in source reports. Previously
any closure without a context scope was skipped, but bytecode closures
don't have those.
TEST=vm/cc/SourceReport_Coverage
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try
Change-Id: I7557e5dd4c98331c7ca2f5c867dd5f6d03e9d756
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501520
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The former contents of the VM isolate are now included into each isolate group. This makes each isolate group's heap independent, and in particular allows each heap to be allocated to a separate pointer cage (not done in this CL).
The duplicated stubs that allowed PC relative calls are removed, since the originals can now be the target of PC relative calls.
The bootstrapping needing to load an AppJIT or AppAOT snapshot is reduced to allocating the oddballs. The code is entirely dropped in the AOT runtime, but the JIT runtime still has it to allow for flags to affect the compilation of the stub code. Further refactoring might be able to remove this for the JIT runtime too, with only gen_snapshot knowing how to bootstrap.
Class serialization no longer distinguishes predefined classes.
The page containing null is marked as never-evacuate. null, false and true must not move because the compiler relies on their low bits having certain patterns for some optimizations. (Previously, the entire VM isolate heap never moved.)
Compaction is disabled for IA32. Due to register pressure, some stub calls must not use a scratch register and embed the address of Code.
The page containing the call-through-safepoint stub is frozen when running with --write-protect-code and the stub is created at runtime (instead of loaded from an AppJIT or AppAOT snapshot). This stub must remain executable even during a safepoint, as a foreign call might during return during a safepoint and only block after the stub directs it to the runtime.
The snapshot symbols are renamed to kDartSnapshotData and kDartSnapshotText. There is no need to distinguish the VM isolate's snapshot, and snaphots are per isolate group not per isolate. Aliases with the old names are added to ease migration.
Some global flags that were automatically set based on the VM isolate's snapshot are now isolate group flags and automatically set by the isolate group's snapshot.
TEST=ci
Change-Id: Iee82016057d609112e9b021d178fc3d4d18b5044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/500621
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
SLSA-Policy-Verified: SLSA Policy Verification Service <devtools-gerritcodereview-exitgate@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Ryan Macnak