This is a reland of commit bfc1a44527
Change how the partial specializations for AtomicBitFieldContainer
are written so that Visual Studio properly chooses them instead of
the base BitField template when appropriate.
TEST=Windows ARM64 CI trybots
Original change's description:
> [vm] Improvements in the BitField API.
>
> If no position is specified, then the bitfield starts at bit 0.
>
> The default size for bool BitFields is 1 instead of the remaining
> bits in the container.
>
> If the size of the value type is smaller than the remaining bits
> in the container, then the size of the value type is used as
> the default size instead.
>
> If a signed value is used in a non-sign-extended BitField, only
> the magnitude of the value is stored, not the sign bit. This means
> the actual size of the bitfield may be one less than the requested
> size in this case.
>
> If the requested size of the bitfield is larger than the size of the
> value type, a compile-time error is thrown. (For signed types, the
> requested size is allowed to be the size of the entire value, even if
> only the magnitude bits are stored.)
>
> Rework uses of BitFields to avoid using separate constants for
> bit positions/sizes except for macro-defined bitfields (which now
> are universally bool, and so size 1).
>
> TEST=vm/cc/BitFields_Defaults
>
> Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
> Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I5e3a9e3e2a80d5689a23a0603f8f81fac1576cd3
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-win-debug-x64c-try,vm-win-release-x64-try,vm-aot-win-release-x64-try,vm-win-release-arm64-try,vm-aot-win-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382383
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit bfc1a44527.
Reason for revert: Broke Windows build.
Original change's description:
> [vm] Improvements in the BitField API.
>
> If no position is specified, then the bitfield starts at bit 0.
>
> The default size for bool BitFields is 1 instead of the remaining
> bits in the container.
>
> If the size of the value type is smaller than the remaining bits
> in the container, then the size of the value type is used as
> the default size instead.
>
> If a signed value is used in a non-sign-extended BitField, only
> the magnitude of the value is stored, not the sign bit. This means
> the actual size of the bitfield may be one less than the requested
> size in this case.
>
> If the requested size of the bitfield is larger than the size of the
> value type, a compile-time error is thrown. (For signed types, the
> requested size is allowed to be the size of the entire value, even if
> only the magnitude bits are stored.)
>
> Rework uses of BitFields to avoid using separate constants for
> bit positions/sizes except for macro-defined bitfields (which now
> are universally bool, and so size 1).
>
> TEST=vm/cc/BitFields_Defaults
>
> Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
> Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I13bf2218ecc47a42f3a6091821037b51b56618fb
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382380
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
If no position is specified, then the bitfield starts at bit 0.
The default size for bool BitFields is 1 instead of the remaining
bits in the container.
If the size of the value type is smaller than the remaining bits
in the container, then the size of the value type is used as
the default size instead.
If a signed value is used in a non-sign-extended BitField, only
the magnitude of the value is stored, not the sign bit. This means
the actual size of the bitfield may be one less than the requested
size in this case.
If the requested size of the bitfield is larger than the size of the
value type, a compile-time error is thrown. (For signed types, the
requested size is allowed to be the size of the entire value, even if
only the magnitude bits are stored.)
Rework uses of BitFields to avoid using separate constants for
bit positions/sizes except for macro-defined bitfields (which now
are universally bool, and so size 1).
TEST=vm/cc/BitFields_Defaults
Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This reverts commit bc0f02e4c8.
Reason for revert: https://github.com/dart-lang/sdk/issues/55754
Original change's description:
> [vm, gc] Incremental compaction.
>
> At the beginning of a major GC cycle, select some mostly-empty pages to be evacuated. Mark the pages and the objects on these pages. Apply a write barrier for stores creating old -> evacuation candidate pointers, and discover any such pointers that already exist during marking.
>
> At the end of a major GC cycle, evacuate objects from these pages. Forward pointers of objects in the remembered set and new-space. Free the evacuated pages.
>
> This compaction is incremental in the sense that creating the remembered set is interleaved with mutator execution. The evacuation step, however, is stop-the-world.
>
> Write-barrier elimination for x.slot = x is removed. Write-barrier elimination for x.slot = constant is removed in the JIT, kept for AOT but snapshot pages are marked as never-evacuate.
>
> TEST=ci
> Bug: https://github.com/dart-lang/sdk/issues/52513
> Change-Id: Icbc29ef7cb662ef8759b8c1d7a63b7af60766281
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357760
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Bug: https://github.com/dart-lang/sdk/issues/52513
Change-Id: I565ad6c0fca283d33f605c10f181bc0a59e7d2b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/366965
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Auto-Submit: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
At the beginning of a major GC cycle, select some mostly-empty pages to be evacuated. Mark the pages and the objects on these pages. Apply a write barrier for stores creating old -> evacuation candidate pointers, and discover any such pointers that already exist during marking.
At the end of a major GC cycle, evacuate objects from these pages. Forward pointers of objects in the remembered set and new-space. Free the evacuated pages.
This compaction is incremental in the sense that creating the remembered set is interleaved with mutator execution. The evacuation step, however, is stop-the-world.
Write-barrier elimination for x.slot = x is removed. Write-barrier elimination for x.slot = constant is removed in the JIT, kept for AOT but snapshot pages are marked as never-evacuate.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/52513
Change-Id: Icbc29ef7cb662ef8759b8c1d7a63b7af60766281
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357760
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This pragma forces compiler to align loop headers within the
function by architecture specific boundary: 32 bytes on X64
and ARM64 (with the exception of Apple Silicon, which explicitly
discourages aligning branch targets in the optimization manual).
Current implementation is rather naive and does not do any
attempt to decide whether aligning is actually profitable
based on loop body itself.
I have found this pragma to be helpful both to stabilize
benchmark results and achieve better performance
for tight loops on Intel hardware.
Issue https://github.com/dart-lang/sdk/issues/55522
TEST=vm/dart/align_loops_test
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-mac-product-arm64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-aot-win-product-x64-try,vm-aot-win-release-arm64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-dwarf-linux-product-x64-try,vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try
Change-Id: Ic22fb90d85e7fdebeeaa3908a43328c59436ab58
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/364121
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This change removes support for external strings from the VM along with
Dart_NewExternalLatin1String, Dart_NewExternalUTF16String and
Dart_IsExternalString Dart C API functions.
External strings are not used by the VM nor any known embedder, but
Dart VM was paying the maintenance and performance price for
the external string implementation classes.
TEST=ci
Change-Id: I094cd2d2b7ec0840e9f09e1ca9e5a7acd4e78c28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358760
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL introduces a way to mark all instances of a class as deeply
immutable.
In order to statically verify that all instances of a deeply immutable
class are immutable, a deeply immutable classes must have the following
properties:
1. All instance fields must
1. have a deeply immutable type,
2. be final, and
3. be non-late.
2. The class must be `final` or `sealed`. This ensures no
non-deeply-immutable subtypes are added by external code.
3. All subtypes must be deeply immutable. This ensures 1.1 can be
trusted.
4. The super type must be deeply immutable (except for Object).
Note that instances of some classes in the VM are deeply immutable
while their class cannot be marked immutable.
* SendPort, Capability, RegExp, and StackTrace are not `final` and
can be implemented by external code.
* UnmodifiableTypedDataViews do not have a public type. (It was
recently deprecated.)
See runtime/docs/deeply_immutable.md for more details.
Use case:
This enables attaching a `Dart_FinalizableHandle` to a deeply immutable
object and the deeply immutable object with other isolates in the same
isolate group.
(Note that `NativeFinalizer`s live in an isolate, and not an isolate
group. So this should currently _not_ be used with `NativeFinalizer`s.
See https://github.com/dart-lang/sdk/issues/55062 for making a
`NativeFinalizer.shared(` that would live in an isolate group instead
of in an isolate.)
Implementation details:
Before this CL, the `ImmutableBit` in the object header was only ever
set to true for predefined class ids (and for const objects). After
this CL, the bit can also be set to true for non const instances of
user-defined classes. The object allocation and initialization code has
been changed to deal with this new case. The immutability of a class is
saved in the class state bits. On object allocation and initialization
the immutability bit is read from the class for non-predefined class
ids.
TEST=runtime/tests/vm/dart/isolates/fast_object_copy2_test.dart
TEST=runtime/vm/isolate_reload_test.cc
TEST=tests/lib/isolate/deeply_immutable_*
Bug: https://github.com/dart-lang/sdk/issues/55120
Bug: https://github.com/dart-lang/sdk/issues/54885
Change-Id: Ib97fe589cb4f81673cb928c93e3093838d82132d
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/354902
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I1e653c9b5d3e02e45b280302c832157a75788db6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345350
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit a3b7c323b0.
Reason for revert: We are seeing crashes in some internal testing
si_signo=Segmentation fault(11), si_code=SEGV_ACCERR(2), si_addr=0x7f1cadb85bc6
version=3.3.0-edge+google3-v2 (google3) (google3) on "linux_x64"
pid=8967, thread=8969, isolate_group=main(0x5583571aaaf0), isolate=(nil)((nil))
os=linux, arch=x64, comp=no, sim=no
isolate_instructions=7f1cad8b7e40, vm_instructions=7f1cad8b1000
fp=7f1cacc7ebc0, sp=7f1cacc7eb90, pc=5583568155bc
pc 0x00005583568155bc fp 0x00007f1cacc7ebc0 dart::MarkingVisitorBase<true>::VisitPointers(dart::ObjectPtr*, dart::ObjectPtr*)+0x15c
pc 0x0000558356975a1d fp 0x00007f1cacc7ec60 dart::StackFrame::VisitObjectPointers(dart::ObjectPointerVisitor*)+0x1bd
pc 0x0000558356935c29 fp 0x00007f1cacc7ecc0 dart::UntaggedSuspendState::VisitSuspendStatePointers(dart::SuspendStatePtr, dart::ObjectPointerVisitor*)+0x89
pc 0x0000558356815948 fp 0x00007f1cacc7ed10 dart::MarkingVisitorBase<true>::DrainMarkingStackWithPauseChecks()+0x128
pc 0x0000558356815749 fp 0x00007f1cacc7ed80 dart::ConcurrentMarkTask::Run()+0x69
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: Ice07b4eb5bef3b41c9618ef0ca7759de006ffe00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/343060
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 5daaa7d9eb.
Reason for revert: internal crashes
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: I8a50074db343c63c14f0487ae8b4f5fee2c4ae76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330720
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 3fb88e4c66.
Reason for revert: b/297175670
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
Change-Id: I33075156160dc35861355d738a5776b74dce88b9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322344
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit e95e7b8e96.
Reason for revert: suspected cause to non-deterministic AOT binaries crashes, b/296014654
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
Change-Id: Idda542c7c657d4f14c836423b173c9b067132212
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/320820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ilya Yanok <yanok@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit 6194209b28.
Reason for revert: issues on arm32
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: I434eb595c9e7858efc8c9b07cbca954e5649f506
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319321
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
If we are generating a SubtypeTestCache for a known type that only
needs instantiator type arguments to be fully instantiated, then
don't store and check against the unneeded function type arguments.
TEST=vm/cc/STC, vm/cc/TTS, ci
Change-Id: I370adf820168079322b8a87811057670a47ee6b3
Cq-Include-Trybots: luci.dart.try:vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-mac-release-arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311382
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
In particular, avoid storing/comparing the destination type unless the
SubtypeTestCache was created for a dynamic AssertAssignable instruction.
TEST=vm/cc/STC_{Linear,Hash}Lookup, vm/cc/TTS
Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: I279c69a668ce19785785cf71707acf951f2b2133
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310340
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Previously, SlowTypeTestStub would use the incoming instance and
destination type to determine which `SubtypeNTestCacheStub` to call,
where `N` is the number of inputs checked for each entry. The code
for generating `InstanceOf` instructions performs similar work,
inlining some checks and then falling back to checking and updating
an appropriate `SubtypeNTestCacheStub` if the inlined checks fail.
Meanwhile, the runtime would use all available information to
create a new entry when needed, depending on the `SubtypeNTestCacheStub`
called at runtime to ignore the parts of the entry it didn't need.
To illustrate a situation where the runtime might use more
information to create an entry than needed by the stub, consider
the following class header:
```
class C<T> extends D
```
If no subtype of `D` requires knowing the specific type arguments for
an instance to determine whether that instance is an instance of `D`,
then the compiler can generate a `Subtype1TestCacheStub` call, where
only the class id of the instance is checked.
However, in the runtime, when updating the cache, the runtime will
create an entry that includes the specific type arguments used to
instantiate `C` when the initial cache miss occurred, as the runtime
does not know how many inputs the code that checks that SubtypeTestCache
uses. If the cache has grown to the point that it is hash-based, that
means the runtime uses the instance type arguments of the first cache
miss to determine the correct place to create a new entry.
When the stub only performs a linear search over the cache contents,
this works, as every entry is checked until a matching entry is found,
and thus the stub can ignore unneeded information. However, if a cache
is hash-based, this doesn't work: if the first miss was an instance
of `C<int>`, then the runtime includes all that information in a new
entry. This means the stub has to calculate the instance type
arguments, which it did not have to do before, and if a subsequent
check uses different instance type arguments (e.g., an instance of
`C<String>`), the result may still be a cache miss, and the runtime
adds another entry which has the same information as the previous
entry as far as the stub was originally concerned.
Thus, to limit itself to considering only the inputs used by the
appropriate `SubtypeNTestCacheStub`, the runtime needs to be told how
many inputs are used. This CL adds an additional field to
`SubtypeTestCache`s which contains the number of used inputs, and
changes the places where `SubtypeTestCache`s are allocated in the
flow graph compiler and the `DRT_TypeCheck` runtime entry to pass the
appropriate number of inputs. To ensure that the runtime and stubs
agree on how many inputs should be used, the `SlowTypeTestStub` now
uses the value of this field to determine which `SubtypeNTestCacheStub`
to call.
TEST=vm/cc/STC_{Linear,Hash}Lookup
Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: Ie16f2d12c4c5eae856b668f28178818203662d1c
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310180
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Avoids objects between kAllocatablePageSize and kNewAllocatableSize getting promoted to regular pages and contributing to fragmentation. The snapshot allocator is also updated to be able to assert the invariant that objects larger than kAllocatePageSize never appear on non-large old pages.
TEST=ci
Bug: b/264420546 (speculative)
Change-Id: Ib2d61f0797bad4c8c6229cf4842169fae5c7053e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306906
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
For class type parameters 'parameterized_class_id' duplicates
a recently added 'owner' and can be removed.
'owner' is changed to hold class reference as a class id (Smi).
TEST=ci
Change-Id: I93fc11e6bcfcc00058a5281b497f59c5c5847ea6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299160
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Now that we have an immutability bit, we can set the bit in object
header of newly allocated objects if those objects are always
deeply immutable.
That allows only checking for the immutable bit instead of checking
against a whole range of cids when sending objects via
`SendPort.send()` / `Isolate.exit()`.
TEST=ci
Change-Id: I4451a9fe11f68c7dd874d49ab632daa3102be19e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281842
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Fixes a scaling limitation where compiling N functions using the same guarded field or CHA guarded interface will result in O(N^2) comparisons when registering the dependencies.
Don't use Instructions' address as the hash, as this gets relocated between the snapshot writer and reader.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/51125
Change-Id: I9a13d57455e10865d9c5f7c12009d869a4ef0488
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279753
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
The representation of record shape in record instances and record
types is changed from a pair
int num_fields
Array field_names
to a single integer - packed bitfield
int num_fields(16)
int field_names_index(kSmiBits-16)
where field names index is an index in the array available from
ObjectStore.
With the new representation of record shapes:
1) Size of record instances is reduced.
2) Number of comparisons for a shape test reduced from 2 to 1
(shape test is used during type checks).
3) A few operations removed from Record.hashCode.
4) Type testing stubs (TTS) are now supported for record types with
named fields. Previously it was not possible to check shape of records
with named fields in TTS as TTS cannot access object pool and cannot
load field names array).
TEST=existing
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I7cdcbb53938aba5d561cd24dc99530395dbbea7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet
This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.
TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
'is' tests against record types are split into series of 'is' tests of
record fields. This is more efficient as record instances cannot be
added to subtype test caches (because type of a record instance
depends on types of its fields).
This change also adds canonicalization and constant evaluation of
LoadFieldInstr for record fields.
Performance on a trivial micro-benchmark (on x64):
JIT (RunTime): 4519104.5 -> 20031.5
AOT (RunTime): 4352583.0 -> 26281.6
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I2ed464cd3b31f365b17805f4e7debe1d6d1051fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268080
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
- Prep work for non-moving promotion.
- The generations still have different alignment offsets.
- Old-space pages now participate in the page cache.
- Fix boolean/null bit tricks to assert the right requirements on the alignment of a page's first object.
TEST=ci
Change-Id: I4369d8c6af73228e162c226d411914868bafed33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260401
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Ryan Macnak