Add a GCReason for store buffer overflow instead of using a separate print.
This is a pure refactoring that doesn't change GC policy.
TEST=ci
Change-Id: Idcd95b0701a36f45ebaa02db2c2fac9a9081d7c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/215102
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
1) double.truncate() now simply calls toInt(), omitting
truncateToDouble() call.
2) double.floor() and ceil() are now intrinsified on arm64 and AOT/x64
and generated using DoubleToInteger instruction.
3) DoubleToInteger instruction is extended to support floor and ceil.
On arm64 DoubleToInteger is implemented using fcvtms and fcvtps
instructions. On x64 DoubleToInteger is implemented using roundsd
under a check if it is supported (with a fallback to a stub and a
runtime call).
AOT/x64:
Before: BenchFloor(RunTime): 318.82148549569655 us.
After: BenchFloor(RunTime): 133.29430189936687 us.
TEST=ci
Closes https://github.com/dart-lang/sdk/issues/46876
Closes https://github.com/dart-lang/sdk/issues/46650
Change-Id: I16ca18faf97954f8e8e25f0b72a2bbfac5bdc672
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212381
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
In the unoptimized code unboxed double inputs are now unboxed in the
instruction prologue, and unboxed double output is boxed in the
instruction epilogue. Double values are still always boxed on the
expression stack in the unoptimized mode.
Support for unboxed doubles in unoptimized code allows us to have
one implementation of double and dart:math built-in functions,
shared across optimized and unoptimized modes.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/46650
Change-Id: Ic1e42b72ee80cb2c12019b7abab8111240b05efd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211302
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This reverts commit 1796160606.
Reason for revert: Still seeing failures.
Original change's description:
> Reland "[ package:dds ] Add support for caching CPU samples based on UserTag"
>
> This reverts commit 4160747ef6.
>
> It seems like 9397b8ff05 should have fixed the original failures we were seeing and the failing bots in question seem to be passing when running try jobs.
>
> TBR=asiva@ (discussed offline)
>
> TEST=Existing service tests + get_cached_cpu_samples_test
> Change-Id: I1c50c0e79375df819a0bd68e68ac28c3064874c8
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208441
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Ben Konyi <bkonyi@google.com>
TBR=bkonyi@google.com,asiva@google.com
Change-Id: Ic522ab61b430673533c94ecd0c91bee569a0add6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208960
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
This reverts commit 4160747ef6.
It seems like 9397b8ff05 should have fixed the original failures we were seeing and the failing bots in question seem to be passing when running try jobs.
TBR=asiva@ (discussed offline)
TEST=Existing service tests + get_cached_cpu_samples_test
Change-Id: I1c50c0e79375df819a0bd68e68ac28c3064874c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/208441
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
DDS can be configured to listen for CPU sample events and cache samples
that were collected while certain UserTags are active. These cached
samples are stored in a ring buffer and are stored until the isolate
shuts down.
TEST=pkg/dds/test/get_cached_cpu_samples_test.dart
Change-Id: Ib20770f59f1672c703413486f87795b3bb23f676
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207206
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Kenzie Schmoll <kenzieschmoll@google.com>
It's mainly hoisting an expression out of
IsolateGroup::{Increase,Decrease}MutatorCount to callers of it.
Right now we consider a Thread::{Exit,Enter} to be nested if there are
existing Dart frames on the stack. A future CL will have a case where
there's a nested exit&re-enter without Dart frames on the stack.
Trying to do this automatically by checking whether there's an
active api scope / zone on the thread fails due to some
asymmetry in our codebase.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing code.
Change-Id: Id70eda03efb2099487c5f0c8affd25ddff926553
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186287
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This CL adds passing structs by value in FFI trampolines.
Nested structs and inline arrays are future work.
C defines passing empty structs as undefined behavior, so that is not
supported in this CL.
Suggested review order:
1) commit message
2) ffi/marshaller (decisions for what is done in IL and what in MC)
3) frontend/kernel_to_il (IL construction)
4) backend/il (MC generation from IL)
5) rest in VM
Overall architecture is that structs are split up into word-size chunks
in IL when this is possible: 1 definition in IL per chunk, 1 Location in
IL per chunk, and 1 NativeLocation for the backend per chunk.
In some cases it is not possible or less convenient to split into
chunks. In these cases TypedDataBase objects are stored into and loaded
from directly in machine code.
The various cases:
- FFI call arguments which are not passed as pointers: pass individual
chunks to FFI call which already have the right location.
- FFI call arguments which are passed as pointers: Pass in TypedDataBase
to FFI call, allocate space on the stack, and make a copy on the stack
and pass the copies' address to the callee.
- FFI call return value: pass in TypedData to FFI call, and copy result
in machine code.
- FFI callback arguments which are not passed as pointers: IL definition
for each chunk, and populate a new TypedData with those chunks.
- FFI callback arguments which are passed as pointer: IL definition for
the pointer, and copying of contents in IL.
- FFI return value when location is pointer: Copy data to callee result
location in IL.
- FFI return value when location is not a pointer: Copy data in machine
code to the right registers.
Some other notes about the implementation:
- Due to Store/LoadIndexed loading doubles from float arrays, we use
a int32 instead and use the BitCastInstr.
- Linux ia32 uses `ret 4` when returning structs by value. This requires
special casing in the FFI callback trampolines to either use `ret` or
`ret 4` when returning.
- The 1 IL definition, 1 Location, and 1 NativeLocation approach does
not remove the need for special casing PairLocations in the machine
code generation because they are 1 Location belonging to 1 definition.
Because of the amount of corner cases in the calling conventions that
need to be covered, the tests are generated, rather than hand-written.
ABIs tested on CQ: x64 (Linux, MacOS, Windows), ia32 (Linux, Windows),
arm (Android softFP, Linux hardFP), arm64 Android.
ABIs tested locally through Flutter: ia32 Android (emulator), x64 iOS
(simulator), arm64 iOS.
ABIs not tested: arm iOS.
TEST=runtime/bin/ffi_test/ffi_test_functions_generated.cc
TEST=runtime/bin/ffi_test/ffi_test_functions.cc
TEST=tests/{ffi,ffi_2}/function_structs_by_value_generated_test.dart
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_generated_tes
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_test.dart
TEST=tests/{ffi,ffi_2}/vmspecific_static_checks_test.dart
Closes https://github.com/dart-lang/sdk/issues/36730.
Change-Id: I474d3a4ee1faadbe767ddadd1b696e24d8dc364c
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140290
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
All existing embedders have been opted into --lazy-async-stacks, the
VM also uses it as it's default in all configurations.
After this CL, any user of --causal-async-stacks will get an error
message when trying to use it.
=> In any such case, please simply remove the flag.
TEST=Exhaustive CQ.
Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ia440afcf2dba464aa8b8cf381b93bbac8eb9f8dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172564
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Also lower the initial array size to save a tiny bit of memory and
let the existing tests exercise exponential growth
TEST=tests/{ffi,ffi_2}/function_callbacks_many_test.dart
Change-Id: I7b490878035cf2e2264ce7443dfdc2b342bedcc3
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64-try,vm-ffi-android-debug-arm-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170438
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Running tsan build locally on IsolateJson identified concurrent access to isolate_ from NoActiveIsolateScope/CollectNewSpaceGarbage and IsMutatorThread/SafepointThreads/Scavenge.
Change-Id: I4c6f5c83a7f6ddbfee8ede59443720db62a48b6f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151023
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.
Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.
This change tries to address this inconsistency.
A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.
Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Three bugs were fixed:
1. BitVector::Equals was not fully fixed by the original CL.
2. We need to add old objects to the deferred marking queue during
RememberLiveTemporaries().
3. The thread being scanned in RestoreWriteBarrierInvariant may not
be scheduled, so we cannot use its store buffer block.
In addition, this changed uncovered another bug fixed in:
https://dart-review.googlesource.com/c/sdk/+/138960.
Original CL is in patchset 3.
This reverts commit 30a12a349e.
Change-Id: I36169b09563998ed5b3c3eac70ee0ebe78853e62
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138920
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
This reverts commit eff1a9ff97.
Reason for revert:
Causes flaky hits of RELEASE_ASSERT in marker.cc, see b/151131634.
Original change's description:
> Re-land "[vm] Aggressive write-barrier elimination."
>
> The original revision is in Patchset 3.
>
> Four bugs were fixed:
>
> 1. JoinEntryInstr::SuccessorCount() is not the correct way to get the
> number of successor blocks from the Join block;
> JoinEntryInstr::last_instruction()->SuccessorCount() must be used
> instead.
>
> 2. BitVector::Equals() was non-deterministically returning 'false'
> for equal vectors.
>
> 3. All blocks need to be processed at least once during the Analysis
> phase (not only in the SaveResults phase).
>
> 4. We were not removing write barriers from StoreIndexed instructions,
> even though we had support for it.
>
> This reverts commit 7fd8ad5a2d.
>
> Fixes https://github.com/dart-lang/sdk/issues/40780
>
> Change-Id: I9650ec2c547ec49cf88ca0524e14f6c245621f6a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138086
> Commit-Queue: Samir Jindel <sjindel@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sjindel@google.com
Change-Id: If9afd84465175fad2431405a97e9293c8bd5e476
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138808
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The original revision is in Patchset 3.
Four bugs were fixed:
1. JoinEntryInstr::SuccessorCount() is not the correct way to get the
number of successor blocks from the Join block;
JoinEntryInstr::last_instruction()->SuccessorCount() must be used
instead.
2. BitVector::Equals() was non-deterministically returning 'false'
for equal vectors.
3. All blocks need to be processed at least once during the Analysis
phase (not only in the SaveResults phase).
4. We were not removing write barriers from StoreIndexed instructions,
even though we had support for it.
This reverts commit 7fd8ad5a2d.
Fixes https://github.com/dart-lang/sdk/issues/40780
Change-Id: I9650ec2c547ec49cf88ca0524e14f6c245621f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138086
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
It incorrectly assumed that all stores in Dart code write to Instances.
There is actually one exception, Contexts, which do not inherit from Instance.
I've added asserts to ensure this kind of bug cannot resurface.
The original change is in patchset 4.
Change-Id: Ic2d8d05e70a4de738eb9fb5980487b4f27111b8c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136221
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This CL:
* Moves [Heap]/[SharedClassTable] from [Isolate] to [IsolateGroup], which
will make all isolates in the group use the same heap. The GC will use
the shared class table for object size information.
* Adds support for entering/leaving an isolate group as a helper thread
(e.g. via [Thread::EnterIsolateGroupAsHelper]). The current active
isolate group can be accessed via TLS `IsolateGroup::Current()` or
`Thread::isolate_group_`. When entering as a helper thread there will be
no current isolate.
* Changes the GC to use the above mechanism and ensures GC works without
a currently active isolate. The GC will use information purely available via
[IsolateGroup]. The GC will iterate all isolates within an isolate
group e.g. for scanning roots.
* Makes spawning of new isolates start in their own isolate group.
Once the isolate is fully functional it's heap will be merged into
the original isolate group
* Moves ApiState, containing persistent and weak persistent handles,
from [Isolate] to [IsolateGroup], plus adds appropriate locking.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ia8e1d8aa78750e8400864200f4825395a182c004
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126646
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This scope can sneak objects larger than kAllocatablePageSize directly into non-large heap pages.
This scope also holds a heap lock outside of the GC, which is likely to create issues when multiple isolates run on the same heap.
Change-Id: Ic56277c83ffccfe2e1396ed773dd30b8d172bbba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134040
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Ryan Macnak