Commit Graph

164 Commits

Author SHA1 Message Date
Ryan Macnak a3b7c323b0 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-12-12 01:56:15 +00:00
Slava Egorov 425a42e3be Revert "[vm, gc] Mark through new-space."
This reverts commit 5daaa7d9eb.

Reason for revert: internal crashes

Original change's description:
> [vm, gc] Mark through new-space.
>
>   - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>   - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>   - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>   - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>   - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I8a50074db343c63c14f0487ae8b4f5fee2c4ae76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330720
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2023-10-18 07:47:39 +00:00
Ryan Macnak 5daaa7d9eb [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
  - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-10-10 19:15:12 +00:00
Emmanuel Pellereau a675a79f39 Revert "Reland "[vm, gc] Mark through new-space.""
This reverts commit 095171e937.

Reason for revert: breaks google3 (b/296014654)

Original change's description:
> Reland "[vm, gc] Mark through new-space."
>
>  - Adjust allocation stub write barrier elimination compensation to check if result is no longer in an active TLAB.
>
> TEST=ci
> Change-Id: I5d24602ae76ee861f2d009d67272251b04da3592
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322448
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: I73b60f799dda755028b65653ac21f27c13e009e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322682
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Emmanuel Pellereau <emmanuelp@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
2023-08-28 14:34:21 +00:00
Ryan Macnak 095171e937 Reland "[vm, gc] Mark through new-space."
- Adjust allocation stub write barrier elimination compensation to check if result is no longer in an active TLAB.

TEST=ci
Change-Id: I5d24602ae76ee861f2d009d67272251b04da3592
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322448
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-25 21:13:02 +00:00
Ivan Inozemtsev 3176cfcd21 Revert "[vm, gc] Mark through new-space."
This reverts commit 3fb88e4c66.

Reason for revert: b/297175670

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: I33075156160dc35861355d738a5776b74dce88b9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322344
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-08-23 13:13:28 +00:00
Ryan Macnak 3fb88e4c66 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-22 16:19:06 +00:00
Ilya Yanok 92ca7985a5 Revert "[vm, gc] Mark through new-space."
This reverts commit e95e7b8e96.

Reason for revert: suspected cause to non-deterministic AOT binaries crashes, b/296014654

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: Idda542c7c657d4f14c836423b173c9b067132212
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/320820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ilya Yanok <yanok@google.com>
2023-08-15 11:09:20 +00:00
Ryan Macnak e95e7b8e96 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-14 18:16:05 +00:00
Ryan Macnak a7e20dd2b0 Revert "[vm, gc] Mark through new-space."
This reverts commit 6194209b28.

Reason for revert: issues on arm32

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I434eb595c9e7858efc8c9b07cbca954e5649f506
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319321
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-09 18:13:17 +00:00
Ryan Macnak 6194209b28 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-08-09 15:57:17 +00:00
Tess Strickland 85de9c1d7e [vm] Allow STCs that only store instantiator, not function, type args.
If we are generating a SubtypeTestCache for a known type that only
needs instantiator type arguments to be fully instantiated, then
don't store and check against the unneeded function type arguments.

TEST=vm/cc/STC, vm/cc/TTS, ci

Change-Id: I370adf820168079322b8a87811057670a47ee6b3
Cq-Include-Trybots: luci.dart.try:vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-mac-release-arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311382
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-07-03 10:39:53 +00:00
Tess Strickland 2b1c0f5692 [vm] Reorder SubtypeTestCache inputs to improve non-dynamic cases.
In particular, avoid storing/comparing the destination type unless the
SubtypeTestCache was created for a dynamic AssertAssignable instruction.

TEST=vm/cc/STC_{Linear,Hash}Lookup, vm/cc/TTS

Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: I279c69a668ce19785785cf71707acf951f2b2133
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310340
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2023-06-22 16:47:59 +00:00
Tess Strickland dfc1b6b16b [vm] Store the number of used inputs in SubtypeTestCaches.
Previously, SlowTypeTestStub would use the incoming instance and
destination type to determine which `SubtypeNTestCacheStub` to call,
where `N` is the number of inputs checked for each entry. The code
for generating `InstanceOf` instructions performs similar work,
inlining some checks and then falling back to checking and updating
an appropriate `SubtypeNTestCacheStub` if the inlined checks fail.

Meanwhile, the runtime would use all available information to
create a new entry when needed, depending on the `SubtypeNTestCacheStub`
called at runtime to ignore the parts of the entry it didn't need.

To illustrate a situation where the runtime might use more
information to create an entry than needed by the stub, consider
the following class header:

```
  class C<T> extends D
```

If no subtype of `D` requires knowing the specific type arguments for
an instance to determine whether that instance is an instance of `D`,
then the compiler can generate a `Subtype1TestCacheStub` call, where
only the class id of the instance is checked.

However, in the runtime, when updating the cache, the runtime will
create an entry that includes the specific type arguments used to
instantiate `C` when the initial cache miss occurred, as the runtime
does not know how many inputs the code that checks that SubtypeTestCache
uses. If the cache has grown to the point that it is hash-based, that
means the runtime uses the instance type arguments of the first cache
miss to determine the correct place to create a new entry.

When the stub only performs a linear search over the cache contents,
this works, as every entry is checked until a matching entry is found,
and thus the stub can ignore unneeded information. However, if a cache
is hash-based, this doesn't work: if the first miss was an instance
of `C<int>`, then the runtime includes all that information in a new
entry. This means the stub has to calculate the instance type
arguments, which it did not have to do before, and if a subsequent
check uses different instance type arguments (e.g., an instance of
`C<String>`), the result may still be a cache miss, and the runtime
adds another entry which has the same information as the previous
entry as far as the stub was originally concerned.

Thus, to limit itself to considering only the inputs used by the
appropriate `SubtypeNTestCacheStub`, the runtime needs to be told how
many inputs are used. This CL adds an additional field to
`SubtypeTestCache`s which contains the number of used inputs, and
changes the places where `SubtypeTestCache`s are allocated in the
flow graph compiler and the `DRT_TypeCheck` runtime entry to pass the
appropriate number of inputs. To ensure that the runtime and stubs
agree on how many inputs should be used, the `SlowTypeTestStub` now
uses the value of this field to determine which `SubtypeNTestCacheStub`
to call.

TEST=vm/cc/STC_{Linear,Hash}Lookup

Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: Ie16f2d12c4c5eae856b668f28178818203662d1c
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310180
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-06-21 15:24:30 +00:00
Ryan Macnak 3a598b4c03 [vm, gc] Honor kAllocatablePageSize during promotion and snapshot reading.
Avoids objects between kAllocatablePageSize and kNewAllocatableSize getting promoted to regular pages and contributing to fragmentation. The snapshot allocator is also updated to be able to assert the invariant that objects larger than kAllocatePageSize never appear on non-large old pages.

TEST=ci
Bug: b/264420546 (speculative)
Change-Id: Ib2d61f0797bad4c8c6229cf4842169fae5c7053e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306906
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-06-06 16:20:47 +00:00
Alexander Markov e46b97e1b7 [vm] Cleanup UntaggedTypeParameter::parameterized_class_id_
For class type parameters 'parameterized_class_id' duplicates
a recently added 'owner' and can be removed.
'owner' is changed to hold class reference as a class id (Smi).

TEST=ci

Change-Id: I93fc11e6bcfcc00058a5281b497f59c5c5847ea6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299160
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-04-28 19:37:10 +00:00
Ryan Macnak 63e6041ca9 [vm] Update to variadic FATAL.
TEST=ci
Change-Id: Ic6bc784605e10760bb28ea6df34242336a33b4d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286947
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-03-06 22:06:59 +00:00
Martin Kustermann 7db7bd0573 [vm] Simplify is-deeply-immutable check in transitive object copy
Now that we have an immutability bit, we can set the bit in object
header of newly allocated objects if those objects are always
deeply immutable.

That allows only checking for the immutable bit instead of checking
against a whole range of cids when sending objects via
`SendPort.send()` / `Isolate.exit()`.

TEST=ci

Change-Id: I4451a9fe11f68c7dd874d49ab632daa3102be19e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281842
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-02-09 00:23:22 +00:00
Ryan Macnak 3b57bac701 [vm] Use a weak set to represent dependent code.
Fixes a scaling limitation where compiling N functions using the same guarded field or CHA guarded interface will result in O(N^2) comparisons when registering the dependencies.

Don't use Instructions' address as the hash, as this gets relocated between the snapshot writer and reader.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/51125
Change-Id: I9a13d57455e10865d9c5f7c12009d869a4ef0488
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279753
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-01-26 22:56:48 +00:00
Alexander Markov 3675fd25f4 [vm] Packed representation of record shape
The representation of record shape in record instances and record
types is changed from a pair

  int num_fields
  Array field_names

to a single integer - packed bitfield

  int num_fields(16)
  int field_names_index(kSmiBits-16)

where field names index is an index in the array available from
ObjectStore.

With the new representation of record shapes:
1) Size of record instances is reduced.
2) Number of comparisons for a shape test reduced from 2 to 1
(shape test is used during type checks).
3) A few operations removed from Record.hashCode.
4) Type testing stubs (TTS) are now supported for record types with
named fields. Previously it was not possible to check shape of records
with named fields in TTS as TTS cannot access object pool and cannot
load field names array).

TEST=existing

Issue: https://github.com/dart-lang/sdk/issues/49719

Change-Id: I7cdcbb53938aba5d561cd24dc99530395dbbea7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-12-20 20:11:48 +00:00
Ryan Macnak a2de36e708 [vm] Rename the default implementation classes of Map and Set.
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet

This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.

TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2022-11-15 17:15:58 +00:00
Ryan Macnak 715866aed3 [vm, compiler] Remove quadratic time spent verifying handle lifetime.
Instead of checking whether a handle is a zone handle rather than a scoped handle by iterating the zone handle blocks, reserve an extra slot in debug mode to record the handle's lifetime.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/50408
Change-Id: I121c76d02b03e5d2141450077bc6e9da3cb60a5d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268801
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-11-11 18:37:44 +00:00
Alexander Markov 10e9606861 [vm] More efficient 'is' tests for record types
'is' tests against record types are split into series of 'is' tests of
record fields. This is more efficient as record instances cannot be
added to subtype test caches (because type of a record instance
depends on types of its fields).

This change also adds canonicalization and constant evaluation of
LoadFieldInstr for record fields.

Performance on a trivial micro-benchmark (on x64):
JIT (RunTime): 4519104.5 -> 20031.5
AOT (RunTime): 4352583.0 -> 26281.6

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I2ed464cd3b31f365b17805f4e7debe1d6d1051fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268080
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-11-07 22:27:31 +00:00
Ryan Macnak 7e4f448f78 [vm, gc] Don't use alignment bits when verifying eliminated write barriers.
TEST=ci
Change-Id: I6affde0b9db6ec951458905391670fe602dd3b61
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262266
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-10-04 20:10:15 +00:00
Ryan Macnak efbf68f306 [vm, gc] Common page structure for both generations.
- Prep work for non-moving promotion.
 - The generations still have different alignment offsets.
 - Old-space pages now participate in the page cache.
 - Fix boolean/null bit tricks to assert the right requirements on the alignment of a page's first object.

TEST=ci
Change-Id: I4369d8c6af73228e162c226d411914868bafed33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260401
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-09-27 21:34:05 +00:00
Alexander Markov d592882f49 [vm] Unify nullability and type state among all types
This change moves 'nullability' and 'type_state' from all kinds of
types to AbstractType base class. This removes a lot of code
duplication and allows uniform access to nullability and type state
for all kinds of types.

TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/47034

Change-Id: I1f0dc7fda78426db83fec6a20ebebcd632ad6d99
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260662
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-27 17:30:03 +00:00
Alexander Markov ed593585cc [vm] Record operator== and hashCode
TEST=language/records/simple/equals_and_hashcode_test
TEST=co19/LanguageFeatures/Records/equality_*

Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I63842f980389d63650d00638f1cb1501db88b025
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260442
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-22 21:24:58 +00:00
Vyacheslav Egorov 83ab5d5ca3 [vm] Remove dynamic field unboxing in JIT
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-09-20 13:42:45 +00:00
Ryan Macnak c1e67ac84f [vm] Recognize unmodifiable typed data views.
These types now work with Dart_TypedDataAcquireData.

The presence of these types no longer degrades the performance of typed data indexed loads.

The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Change-Id: Iffad865708501acf96db418985cd5a69bd9afa55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254501
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-11 18:10:48 +00:00
Ryan Macnak 10bf1cfe58 Revert "[vm] Recognize unmodifiabled typed data views."
This reverts commit d1112d37bd.

Reason for revert: b/242043014

Original change's description:
> [vm] Recognize unmodifiabled typed data views.
>
> These types now work with Dart_TypedDataAcquireData.
>
> The presence of these types no longer degrades the performance of typed data indexed loads.
>
> The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.
>
> TEST=ci
> Bug: https://github.com/dart-lang/sdk/issues/32028
> Bug: https://github.com/dart-lang/sdk/issues/40924
> Bug: https://github.com/dart-lang/sdk/issues/42785
> Change-Id: I05ac5c9543f6f61ac37533b9efe511254778caed
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253700
> Reviewed-by: Aske Simon Christensen <askesc@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

TBR=kustermann@google.com,rmacnak@google.com,askesc@google.com

TEST=ci
Change-Id: I32c1c460fc30c51bc0d42e7cfaafe72bf5630069
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254560
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-10 20:08:22 +00:00
Ryan Macnak d1112d37bd [vm] Recognize unmodifiabled typed data views.
These types now work with Dart_TypedDataAcquireData.

The presence of these types no longer degrades the performance of typed data indexed loads.

The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Change-Id: I05ac5c9543f6f61ac37533b9efe511254778caed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253700
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-09 20:57:05 +00:00
Ryan Macnak e5602b0652 [vm, compiler] Fix loading Smis from the pool instead of as immediates due to polluted upper bits under compressed pointers.
Bring other architectures into agreement with arm about the order in which LoadObject should consider its options.

TEST=ci
Change-Id: I277e692020acc6581b3d19a87a85d95c8badd696
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250322
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-07-07 16:08:15 +00:00
Ryan Macnak 0a8fda94b6 [vm, compiler] Use RISC-V's compare-and-branch for a shorter write barrier sequence.
dart2js.aot.rv64 25753840 -> 25721032 (-0.13%)
dart2js.aot.rv32 24891160 -> 24858368 (-0.13%)

TEST=ci
Change-Id: I252e0477f11b9198fe043f105a8745bc283aaaa8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249062
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-06-23 15:58:15 +00:00
Ryan Macnak 3f0461a17c [vm, compiler] Make more use of the zero register on ARM64 and RV.
flutter_gallery
Instructions(CodeSize): 4622928 -> 4611996 (-0.24%)
       Total(CodeSize): 7217938 -> 7207097 (-0.15%)

TEST=ci
Change-Id: I3e0b2bd51a190260ab2c23706815be34000fcb22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/245422
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-05-24 22:12:17 +00:00
Alexander Markov bf4bb95308 [vm] New async/await implementation in the VM, part 2 - vm
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.

Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.

Design doc: go/compact-async-await.

Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2022-04-29 01:03:50 +00:00
Daco Harkes 532c116cd2 [vm] Implement NativeFinalizer
This CL implements `NativeFinalizer` in the GC.

`FinalizerEntry`s are extended to track `external_size` and in which
`Heap::Space` the finalizable value is.

On attaching a native finalizer, the external size is added to the
relevant heap. When the finalizable value is promoted from new to old
space, the external size is promoted as well. And when a native
finalizer is run or is detached, the external size is removed from the
relevant heap again.

In contrast to Dart `Finalizer`s, `NativeFinalizer`s are run on isolate
shutdown.

When the `NativeFinalizer`s themselves are collected, the finalizers are
not run. Users should stick the native finalizer in a global variable to
ensure finalization. We will revisit this design when we add send and
exit support, because there is a design space to explore what to do in
that case. This current solution promises the least to users.

In this implementation native finalizers have a Dart entry to clean up
the entries from the `all_entries` field of the finalizer. We should
consider using another data structure that avoids the need for this Dart
entry. See the TODO left in the code.

Bug: https://github.com/dart-lang/sdk/issues/47777

TEST=runtime/tests/vm/dart(_2)/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/ffi(_2)/vmspecific_native_finalizer_*

Change-Id: I8f594c80c3c344ad83e1f2de10de028eb8456121
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236320
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-03-26 09:41:21 +00:00
Daco Harkes e151a81108 Reland "[vm] Implement Finalizer"
Original CL in patchset 1.
Split-off https://dart-review.googlesource.com/c/sdk/+/238341
And pulled in fix https://dart-review.googlesource.com/c/sdk/+/238582
(Should merge cleanly when this lands later.)

This CL implements the `Finalizer` in the GC.

The GC is specially aware of two types of objects for the purposes of
running finalizers.

1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)

A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).

A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.

When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.

When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.

To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.

Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.

When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.

Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.

Bug: https://github.com/dart-lang/sdk/issues/47777

TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc

Change-Id: Ibdfeadc16d5d69ade50aae5b9f794284c4c4dbab
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238086
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-03-25 10:29:30 +00:00
Ryan Macnak 67f93d3840 [vm] Refactor compilation of runtime calls.
- Fixes SIMD registers clobbered by write barrier on ARM64.
 - Moves code generation out of non-compiler directory.
 - Removes unnecessary building of Dart frames on leaf runtime calls.
 - Removes unnecessary Threads slots for write barrier Code objects.
 - Removes duplicate saves of SP in leaf runtime calls on ARM64 and RISC-V.
 - Avoids some redundant SP updates on RISC-V.

TEST=ci
Change-Id: Idb92127658edc90b320923ef3d882a7219a450ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236842
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-03-22 19:45:03 +00:00
Daco Harkes 37526fc8d7 Revert "[vm] Implement Finalizer"
This reverts commit 7dca34c235.

Reason for revert: b/226085355 dart_vm_test crashing. Unclear what
the cause is. Reverting so we can triage the issue.

TEST=This is a revert.

Original change's description:
> [vm] Implement `Finalizer`
>
> This CL implements the `Finalizer` in the GC.
>
> (This CL does not yet implement `NativeFinalizer`.)
>
> The GC is specially aware of two types of objects for the purposes of
> running finalizers.
>
> 1) `FinalizerEntry`
> 2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
>
> A `FinalizerEntry` contains the `value`, the optional `detach` key, and
> the `token`, and a reference to the `finalizer`.
> An entry only holds on weakly to the value, detach key, and finalizer.
> (Similar to how `WeakReference` only holds on weakly to target).
>
> A `Finalizer` contains all entries, a list of entries of which the value
> is collected, and a reference to the isolate.
>
> When a the value of an entry is GCed, the enry is added over to the
> collected list.
> If any entry is moved to the collected list, a message is sent that
> invokes the finalizer to call the callback on all entries in that list.
>
> When a finalizer is detached by the user, the entry token is set to the
> entry itself and is removed from the all entries set.
> This ensures that if the entry was already moved to the collected list,
> the finalizer is not executed.
>
> To speed up detaching, we use a weak map from detach keys to list of
> entries. This ensures entries can be GCed.
>
> Both the scavenger and marker tasks process finalizer entries in
> parallel.
> Parallel tasks use an atomic exchange on the head of the collected
> entries list, ensuring no entries get lost.
> The mutator thread is guaranteed to be stopped when processing entries.
> This ensures that we do not need barriers for moving entries into the
> finalizers collected list.
> Dart reads and replaces the collected entries list also with an atomic
> exchange, ensuring the GC doesn't run in between a load/store.
>
> When a finalizer gets posted a message to process finalized objects, it
> is being kept alive by the message.
> An alternative design would be to pre-allocate a `WeakReference` in the
> finalizer pointing to the finalizer, and send that itself.
> This would be at the cost of an extra object.
>
> Send and exit is not supported in this CL, support will be added in a
> follow up CL. Trying to send will throw.
>
> Bug: https://github.com/dart-lang/sdk/issues/47777
>
> TEST=runtime/tests/vm/dart/finalizer/*
> TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
> TEST=runtime/vm/object_test.cc
>
> Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,rmacnak@google.com,dacoharkes@google.com

Change-Id: I991f6e49896d18a8d70210cf315d858b462d66c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/47777
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238080
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-03-22 11:55:38 +00:00
Daco Harkes 7dca34c235 [vm] Implement Finalizer
This CL implements the `Finalizer` in the GC.

(This CL does not yet implement `NativeFinalizer`.)

The GC is specially aware of two types of objects for the purposes of
running finalizers.

1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)

A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).

A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.

When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.

When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.

To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.

Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.

When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.

Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.

Bug: https://github.com/dart-lang/sdk/issues/47777

TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc

Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-03-22 10:30:18 +00:00
Daco Harkes 828dcd00e8 [vm] Implement WeakReference in the VM
This CL implements `WeakReference` in the VM.

* This reduces the size of weak references from 2 objects using 8 words
  to 1 object using 4 words.
* This makes loads of weak reference targets a single load instead of
  two.
* This avoids the fix-point in the GC and message object copying for
  weak references. (N.b. Weak references need to be processed _after_
  the fix-point for weak properties.)

The semantics of weak references in messages is that their target gets
set to `null` if the target is not included in the message by a strong
reference.

The tests take particular care to exercise the case where a weak
reference's target is only kept alive because a weak property key is
alive and it refers to the target in its value. This exercises the fact
that weak references need to be processed last.

Does not add support for weak references in the app snapshot. It would
be dead code until we start using weak references in for example the
CFE.

This CL does not try to unify weak references and weak properties in
the GC or messaging (as proposed in go/dart-vm-weakreference), because
their semantics differ enough.

Closes: https://github.com/dart-lang/sdk/issues/48162

TEST=runtime/tests/vm/dart/finalizer/weak_reference_run_gc_test.dart
TEST=runtime/tests/vm/dart/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
TEST=tests/lib/isolate/weak_reference_message_1_test.dart
TEST=tests/lib/isolate/weak_reference_message_2_test.dart

Change-Id: I3810e919a5866f3ae8a95bd9aa23a880a0b0921c
Cq-Include-Trybots: luci.dart.try:app-kernel-linux-debug-x64-try,dart-sdk-mac-arm64-try,vm-canary-linux-debug-try,vm-fuchsia-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-release-x64-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/232087
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-02-10 21:59:41 +00:00
Alexander Markov 45f42d0be8 [vm] Refactor duplicated slots and rename offsets
This change removes distinct Slots for the same field:

PointerBase_data_field -> PointerBase_data
Pointer_data_field -> PointerBase_data
TypedDataBase_data_field -> PointerBase_data

Also, the following slot is renamed to match field name:

TypedDataView_data -> TypedDataView_typed_data

The following offsets are renamed to match field names /
declarations in Untagged* classes:

PointerBase::data_field_offset -> PointerBase::data_offset
TypedData::data_offset -> TypedData::payload_offset
TypedDataView::data_offset -> TypedDataView::typed_data_offset


TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/48273

Change-Id: I602545fc43018494bcb24d8692292fdbc6a8f3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231160
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-02-02 16:30:50 +00:00
Ryan Macnak 6f9289ca30 [vm, compiler] Avoid false positive in assertion for eliminated write barrier.
This adjusts RISC-V to match the other architectures in checking only the generational barrier.

TEST=fuzzer
Change-Id: I86965bb2caaae27aa9770769f392cffc8edb6eb8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231046
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-02-01 17:17:05 +00:00
Alexander Markov 2a34453dd1 [vm/aot] Improve AOT compilation speed by using better hash codes
This change improves hash code implementations in multiple places in
the compiler. That reduces number of probes during lookups in hash maps
and improves AOT compilation time of large applications.

On a large Flutter app, compiled in release mode for arm64:
Total gen_snapshot time 89.184s -> 60.736s (-31.9%)

Also, this change adds --hash_map_probes_limit=N option which sets
a hard limit for the number of probes in hash maps. This option
makes it easy to find hash maps where there are many collisions
due to poor hash code implementation.

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/43299
Bug: b/154155290
Change-Id: Ibf6f37d4b9f3bf42dd6731bfb4095a7305b98b2d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-01-21 02:02:00 +00:00
Ryan Macnak ecdf148428 [vm] Refactor generation of identity hashes.
Generating identity hashes from the runtime no longer calls into Dart. On 32-bit systems, generating identity hashes from Dart now does only one runtime transition.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/47873
Change-Id: Ib21156cb05706f81744eb4e5ccb644f40aa84c96
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222326
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-12-13 22:31:46 +00:00
Martin Kustermann 228c52ed4a Reland "[vm] When run under TSAN use longjmp() to skip over C++ frames before manually unwinding to the catch entry"
TSAN instruments C++ code by adding prologue/epilogue code which
maintains a shadow stack. Using setjmp()/longjmp() is intercepted by
TSAN and correspondingly unwinds the shadow stack.

When Dart VM throws exceptions we call the JumpToFrame stub
from C++ which will directly reset the stack to the exception handler
catch entry. This leaves the TSAN shadow stack unchanged.

This means whenever an exception is thrown we leak frames in TSAN's
shadow stack. Due to using a fixed-size shadow stack, it will cause a
buffer-overflow in TSAN when too many such frame leaks happen. This can
cause arbitrary memory to be overriden, leading to awkward crashes.

This is especially an issue on the "iso-stres" builder because it
launches - in the same process - *many* small tests, more easily hitting
that limit.

This CL will workaround the TSAN issue by making runtime call save it's
state via setjmp() and make exception throughing process go via
longjmp() (which TSAN will intercept) before actually calling the
JumpToFrame stub.

=> This will ensure the TSAN shadow stack is correctly maintained.

The [jmp_buf]'s encoding of register state is non-trivial (e.g. it uses
XOR'ing of the actual saved state under certain glibc versions). So we
store any state we need to pass to the target of the `longjmp()` on the
[Thread] instead of overriding the [jmp_buf]s register state with the
arguments.

Issue https://github.com/dart-lang/sdk/issues/47472#issuecomment-948235479

TEST=vm/dart{,_2}/regress47472_test.dart

Change-Id: Ifbf6580aa15bcce54d0584cdc3cd18cc19be0a9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222300
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-12-08 20:41:18 +00:00
Vyacheslav Egorov 3d34ec1dbf Revert "[vm] When run under TSAN use longjmp() to skip over C++ frames before manually unwinding to the catch entry"
This reverts commit 20e6a4dc92.


Revert "[gardening] Fix Product TSAN build by fixing accidental ReleaseTSANX64 -> ProductTSANX64 issue"

This reverts commit e6c24f934e.

Reason for revert: to unblock Google3 roll.

TEST=revert

Change-Id: If22e17d514a81829a9eeb5585257fc477026ea63
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221465
Auto-Submit: Slava Egorov <vegorov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Emmanuel Pellereau <emmanuelp@google.com>
2021-11-29 09:34:23 +00:00
Martin Kustermann 20e6a4dc92 [vm] When run under TSAN use longjmp() to skip over C++ frames before manually unwinding to the catch entry
TSAN instruments C++ code by adding prologue/epilogue code which
maintains a shadow stack. Using setjmp()/longjmp() is intercepted by
TSAN and correspondingly unwinds the shadow stack.

When Dart VM throws exceptions we call the JumpToFrame stub
from C++ which will directly reset the stack to the exception handler
catch entry. This leaves the TSAN shadow stack unchanged.

This means whenever an exception is thrown we leak frames in TSAN's
shadow stack. Due to using a fixed-size shadow stack, it will cause a
buffer-overflow in TSAN when too many such frame leaks happen. This can
cause arbitrary memory to be overriden, leading to awkward crashes.

This is especially an issue on the "iso-stres" builder because it
launches - in the same process - *many* small tests, more easily hitting
that limit.

This CL will workaround the TSAN issue by making runtime call save it's
state via setjmp() and make exception throughing process go via
longjmp() (which TSAN will intercept) before actually calling the
JumpToFrame stub.

=> This will ensure the TSAN shadow stack is correctly maintained.

The [jmp_buf]'s encoding of register state is non-trivial (e.g. it uses
XOR'ing of the actual saved state under certain glibc versions). So we
store any state we need to pass to the target of the `longjmp()` on the
[Thread] instead of overriding the [jmp_buf]s register state with the
arguments.

Issue https://github.com/dart-lang/sdk/issues/47472#issuecomment-948235479

TEST=vm/dart{,_2}/regress47472_test.dart

Change-Id: I4b6f0d4eacef85487c55999021d72a6d932facad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221083
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-11-26 12:57:04 +00:00
Vyacheslav Egorov 71c9c2cd83 [vm] Remove support for non-bare AOT snapshots
Every AOT user out there has been using bare instructions mode
and continuing to maintaining non-bare instructions mode simply
adds costs (both in terms of time spent making changes to work
in a mode that is not used and CI resources spent on testing it).

This change removes FLAG_use_bare_instructions and changes the code
to assume that FLAG_precompiled_mode implies bare instructions.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,app-kernel-linux-release-x64-try,app-kernel-linux-debug-x64-try
Change-Id: I5032b13bfcb613f79865f2cfa139cca8d1b42556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220964
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-11-24 13:03:52 +00:00
Tess Strickland f1f83c9bdd [vm] Store type class id for Types as ClassIdTagType, not Smi.
Now that Types are no longer created for top level classes, the type
class id is guaranteed to fit in ClassIdTagType. That means we can
reduce the size of Type objects slightly as well as avoid the need to
tag and untag the type class id.

Also fixes an issue uncovered in GenerateSubtype1TestCacheLookup where
the retrieved super type was not first checked to be non-null.

TEST=Refactoring, so existing tests.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-ia32-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try,vm-kernel-precomp-linux-debug-simarm64c-try
Change-Id: I851a827843e34eac6961e0f06f888b40bf16df55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217221
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-10-20 16:31:02 +00:00