The previous attempt was reverted because it broke unoptimized JIT
on ARM 32. This reland force-optimizes the two getters.
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Fixes https://github.com/dart-lang/sdk/issues/63436
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: Ib812cbaec6e371b9720df7a543411f78e524cac1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/506060
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The Integer_parse VM native has no Dart-side caller. int.parse is
implemented entirely in Dart in
sdk/lib/_internal/vm_shared/lib/integers_patch.dart (_tryParseSmi
and _parseRadix). Removes the native entry and drops a stale
reference in the comment on the shared ParseInteger helper, which
is still used by Integer_fromEnvironment.
TEST=ci
Change-Id: I8d034e3081c09357094abc6415fc709e3edbea07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507322
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Implement api methods to create and shutdown isolates from IsolateGroup-bound callbacks (normally invoked from native code), run dart code on such isolates.
The rest of the api is not implemented yet.
TEST=tests/ffi/threading_test.dart
BUG=https://github.com/dart-lang/sdk/issues/62407
CoreLibraryReviewExempt: vm-only change to isolate library
Change-Id: I0271ead8ba011dfe9d7953769415d6a88a962854
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/486522
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Stacks on top of the int.{trailingZeroBitCount,oneBitCount} API CL
(commit 754239b077). Both getters route through OTHER_RECOGNIZED_LIST
when a hardware fast path is available; otherwise the newly added
Dart bodies inline at call sites via vm:prefer-inline. The C++
natives are removed.
Backend codegen
---------------
ARM64: NEON CNT + UADDLV (popcount); RBIT + CLZ (ctz).
ARM: NEON CNT + VPADDL chain (popcount); RBIT + CLZ on the
register pair (ctz).
x64: popcntq when TargetCPUFeatures::popcnt_supported();
LoadImmediate(64) + rep_bsfq for ctz (decodes as tzcnt
on BMI1+, preserves dest on zero otherwise).
RISC-V 64: cpop / ctz when RV_baseline includes Zbb.
Per-arch availability is encapsulated in
UnaryInt64OpInstr::IsSupported(Token::Kind).
Apple M-series ARM64, AOT (us/iter, lower is better):
cardinality.swar 371
cardinality.accelerated 154 (2.4x)
forEachSetBit.swar 19031
forEachSetBit.accelerated 4988 (3.8x)
select.swar 199
select.accelerated 77 (2.6x)
complementCardinality.swar 399
complementCardinality.accel 152 (2.6x)
Work towards https://github.com/dart-lang/sdk/issues/6486 (popcount
and ctz intrinsification).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Fixes https://github.com/dart-lang/sdk/issues/52673
Fixes https://github.com/dart-lang/sdk/issues/38346
Issue https://github.com/dart-lang/sdk/issues/10212
Issue https://github.com/dart-lang/sdk/issues/5798
TEST=tests/corelib/int_bit_count_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-simarm_x64-try,dart-sdk-linux-riscv64-try
Change-Id: I9c2c4225fd63d54e190398a0fb6649366317dd17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/504221
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
In the VM, factory constructors always had an extra "type arguments"
parameter, even if class is not generic. Factory constructor bodies
were using class type parameters instead of function type parameters.
This results in extra code when calling non-generic factories
which is slightly inefficient in terms of code size and performance.
Also, it creates an additional complexity throughout the system as
factories should be special cased in many places.
This change removes artificial "type arguments" parameter, treating
factory constructors basically as static methods. This matches
kernel AST representation.
TEST=ci
Change-Id: I957583cb2ce9a3c408699880a04036e06b01dd31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/501762
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Adds two new getters to int for bit-counting: trailingZeroBitCount
(ctz) and oneBitCount (popcount). On native platforms they operate
on the full 64-bit two's-complement representation; on the web they
operate on the least-significant 32 bits.
Implementations:
- VM: unified C++ natives Integer_trailingZeroBitCount /
Integer_oneBitCount on _IntegerImplementation, using
Utils::CountTrailingZeros64 and Utils::CountOneBits64. The receiver
may be _Smi or _Mint at runtime.
- dart2js / DDC: clz32-based ctz and a SWAR popcount.
- dart2wasm: inlined i64.ctz and i64.popcnt intrinsics.
leadingZeroBitCount (clz) is intentionally excluded from this CL: its
result depends on the platform integer width (e.g. 1.leadingZeroBitCount
is 31 on web, 63 on native), and the same value can be derived from
the existing bitLength getter when needed.
Asm intrinsification on native architectures is intentionally left for
a separate follow-up CL.
Work towards https://github.com/dart-lang/sdk/issues/6486 (this CL
covers popcount and ctz from the bit-twiddling list; clz, rotate,
reverse, and others remain).
Work towards https://github.com/dart-lang/sdk/issues/1053 (efficient
BitSet implementation).
Bug: https://github.com/dart-lang/sdk/issues/52673
Bug: https://github.com/dart-lang/sdk/issues/38346
TEST=tests/corelib/int_bit_count_test
Change-Id: I8a5cdb5c91360478f47bbd6b9c84ca1c477aa8c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/498041
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Modestas Valauskas <valauskasmodestas@gmail.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Call sites targeting a procedure annotated with `external-effect` will
not produce any code, including the argument which will not be
evaluated.
However, the single parameter will be treated as 'live' for the purposes
of any global analysis the backends do. This is useful for things like
protobuf shaking where a user may want to retain certain protobuf
messages without actually emitting the code that retains those messages.
Today this functionality is available internally in the vm and wasm SDK
libraries. dart2js has similar functionality represented via the
opaqueTrue and opaqueFalse booleans (which will cause conditional
branches to get shaken after analysis). This will replace dart2js's
opaque(True/False).
This also adds validation to the frontend to ensure a method annotated
with 'external-effect' is well-formed.
Change-Id: If1c4096673e655c58fe7638840a16125003e7809
Tested: Backend tests for codegen were added. A frontend test was added for the validation. A language test was added to confirm the behavior.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/476020
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
When instantiating a closure, allocate a new closure object in
the same boostrap native method as checking bounds.
This simplifies code generation and prepares for variable-length
closure objects by removing the only place where closure of unknown
length was allocated in the generated code.
TEST=ci
Change-Id: I9ac51c862081612bca2160699ebe24526d4fc915
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/490360
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This API allows to programmatically generate timeline information
including profiling samples if timeline format supports it.
This change also includes Perfetto timeline writer and profiler
into non-mobile PRODUCT builds. The size of Mac OS X dartaotruntime
has increased by 34352 bytes (0.7%).
This relands commit 553887bc56 with fixes
for Android builds.
TEST=vm/dart/stream_timeline_to_test
CoreLibraryReviewExempt: VM specific dart:developer API.
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-product-arm-try,vm-ffi-android-release-arm-try,vm-aot-android-release-arm64c-try
Change-Id: Ifb7c66982fe9d17d2f5c9acb163a94c56a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457301
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 553887bc56.
Reason for revert: Broken android builds
Original change's description:
> [vm] Add NativeRuntime.streamTimelineTo
>
> This API allows to programmatically generate timeline information
> including profiling samples if timeline format supports it.
>
> This change also includes Perfetto timeline writer and profiler
> into non-mobile PRODUCT builds. The size of Mac OS X dartaotruntime
> has increased by 34352 bytes (0.7%).
>
> TEST=vm/dart/stream_timeline_to_test
>
> CoreLibraryReviewExempt: VM specific dart:developer API.
> Change-Id: I326f5fd29ee82a503160f5f5a2a857a7c55330ec
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/426220
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Slava Egorov <vegorov@google.com>
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I583ad74b850514fb47054149da464a07874fc2f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This API allows to programmatically generate timeline information
including profiling samples if timeline format supports it.
This change also includes Perfetto timeline writer and profiler
into non-mobile PRODUCT builds. The size of Mac OS X dartaotruntime
has increased by 34352 bytes (0.7%).
TEST=vm/dart/stream_timeline_to_test
CoreLibraryReviewExempt: VM specific dart:developer API.
Change-Id: I326f5fd29ee82a503160f5f5a2a857a7c55330ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/426220
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Also fix return type of _storeAbiSpecificInt / _storeAbiSpecificIntAtIndex (they actually return null,
so declaring return type 'int' violates soundness of types).
TEST=ci
Change-Id: Ic7726e618a5d8dda189f81464c213629d70c23a1
Cq-Include-Trybots: luci.dart.try:vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/447480
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Currently it using legacy stringified parameters which
makes it hard to pass complex structured data to it.
TEST=ci
CoreLibraryReviewExempt: vm-service implementation changes no affecting public corelib APIs.
Change-Id: I1291e0a2971ad51fef4bc4a2d53e7ec0a76b3131
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/437221
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Update isolate_group_shared_callback_test to use that and also use flag to print exceptions.
TEST=ci
CoreLibraryReviewExempt: only internal library is affected
Change-Id: Ia53ca7a6dc847b551229ccf00bbbfe302a260fcb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/427680
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
TEST=pkg/vm_service/test/timeline_events_for_completed_microtasks_test
CoreLibraryReviewExempt: This CL does not include any core library API
changes, it only modifies the implementation of microtasks (by
instrumenting them).
Change-Id: I54d886db9519c73f9e3218a9cc1c46bc9fe9acc3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420221
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
This change makes it so that when the VM detects that a timer is at
least 100 ms overdue, it sends a warning in a VM Service event on the
'Timer' stream.
TEST=Built an Android Flutter app with a modified Engine that contained
a `sleep` call in `eventhandler_linux.cc` and confirmed that the warning
event got sent, built an iOS Flutter app with a modified Engine that
contained a `sleep` call in `eventhandler_macos.cc` and verified that
the warning event got sent,
pkg/vm_service/test/overdue_timer_detection_test
CoreLibraryReviewExempt: This CL does not include any core library API
changes, only VM Service implementation changes.
Change-Id: Ie8db047116b7f63cfb5413f763eaf56c7bdd6975
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/409500
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
Also removed a bunch of `CheckNotNullable`s that shouldn't be necessary any more. Any remaining non-sound-null-safety code runs today, and no more should be written. (And if it is, it'll mostly just err somewhere else, with a worse error message.)
Tested: New test added. Removed older tests checking for unsound null-safety.
Change-Id: I28626909cd8c1f91db6c61fc2b93042ed1b085dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/380780
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
Previously, Internal_randomInstructionsOffsetInsideAllocateObjectStub
assumed the entry point is at the start of the instructions payload when
choosing a random offset into the instructions payload. However, most
of our architectures have non-zero entry point offsets.
Instead, replace it with two methods, one that returns the start
(inclusive) and one that returns the end (exclusive) of the
AllocateObject stub instructions payload and have the test choose
points inside and outside of that range to test.
TEST=vm/dart/use_dwarf_stack_traces_flag_test
Fixes: https://github.com/dart-lang/sdk/issues/50286
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try
Change-Id: Ia8965c72fbc6f6d2c178778a32f8083923a8b243
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/363080
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This is a reland of commit d24b5d1f5e
On top of the original change, the following is fixed and improved:
1) Flow graph builder (FGB) body is now also applied to
dynamic invocation forwarders (similarly to graph
intrinsics).
2) Frame can be omitted for functions which have a
call on a shared slow path. This is needed to make
FGB implementation of GetIndexed frameless, as it has
GenericCheckBound which calls on shared slow path.
(Graph intrinsics are frameless).
3) Range analysis is enabled for force-optimized functions,
so more efficient code can be generated for boxing
instructions. Range analysis is fixed to avoid crashes
and correctly intersect ranges with constant boundaries
(needed for some force-optimized FFI functions).
4) EliminateStackOverflowChecks pass is enabled for
force-optimized functions so CheckStackOverflow can be
eliminated.
Original change's description:
> [vm] Replace array GetIndexed graph intrinsics with flow graph builder implementation
>
> _Array, _GrowableList, internal and external typed data 'operator []'
> are now implemented in the flow graph builder.
>
> Unlike graph intrinsics, flow graph created in the flow graph builder
> can be used by the inliner. Corresponding graph intrinsics and native
> methods are removed.
>
> Also, this change adds missing external typed data indexing operations.
>
> TEST=ci
>
> Change-Id: Ic19784481feadf54c096a587413e67b4e18353dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359940
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
TEST=ci
Change-Id: I04ef008a04238d432683d7543cd047e35bad17c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360560
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit d24b5d1f5e.
Reason for revert: performance regression
(Graph intrinsics are applied to dynamic invocation forwarders,
but flow graph builder implementation isn't.)
Original change's description:
> [vm] Replace array GetIndexed graph intrinsics with flow graph builder implementation
>
> _Array, _GrowableList, internal and external typed data 'operator []'
> are now implemented in the flow graph builder.
>
> Unlike graph intrinsics, flow graph created in the flow graph builder
> can be used by the inliner. Corresponding graph intrinsics and native
> methods are removed.
>
> Also, this change adds missing external typed data indexing operations.
>
> TEST=ci
>
> Change-Id: Ic19784481feadf54c096a587413e67b4e18353dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359940
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
Change-Id: I2b31b06edaa4b8a09d256f25f923d4489e28518b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360305
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
_Array, _GrowableList, internal and external typed data 'operator []'
are now implemented in the flow graph builder.
Unlike graph intrinsics, flow graph created in the flow graph builder
can be used by the inliner. Corresponding graph intrinsics and native
methods are removed.
Also, this change adds missing external typed data indexing operations.
TEST=ci
Change-Id: Ic19784481feadf54c096a587413e67b4e18353dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359940
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The new implementation of String.codeUnitAt is always inlined and
doesn't depend on the polymorphic inlining of recognized methods.
String.codeUnitAt now has a custom body in the flow graph builder
which performs non-speculative bounds check and then branches between
OneByteString and TwoByteString. Corresponding graph intrinsic and
native method are removed.
This change also fixes passing of unboxed arguments to runtime
in the slow path of GenericCheckBound instruction in JIT mode
(when shared stubs are not used).
TEST=ci
Change-Id: Iab2805fc752df84c37089165f828e31aca5f043f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359000
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Previously, they were implemented as native methods with special
replacements in the inliner.
Instead, create force-compiled versions of the original inliner
replacements and use those instead of native methods, unless the
flow graph compiler doesn't support unboxing the requested element type.
In that case, the force-compiled version just calls a native method,
and we only keep the native methods that might be needed (that is,
for double/SIMD element access).
Also, revert the change in 26911a6176, since now the _getX/_setX
methods are appropriately inlined instead of failing to inline due
to being native methods.
TEST=vm/dart/typed_list_index_checkbound_il_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-linux-debug-x64-try,vm-aot-linux-release-simarm_x64-try,vm-linux-release-simarm-try,vm-ffi-qemu-linux-release-arm-try
Change-Id: I4840883d1fc12b36a450803da339406bec149044
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330786
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Shikhar Soni