Commit Graph

80 Commits

Author SHA1 Message Date
Ryan Macnak a3b7c323b0 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-12-12 01:56:15 +00:00
Slava Egorov 425a42e3be Revert "[vm, gc] Mark through new-space."
This reverts commit 5daaa7d9eb.

Reason for revert: internal crashes

Original change's description:
> [vm, gc] Mark through new-space.
>
>   - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>   - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>   - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>   - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>   - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I8a50074db343c63c14f0487ae8b4f5fee2c4ae76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330720
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2023-10-18 07:47:39 +00:00
Ryan Macnak 5daaa7d9eb [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
  - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-10-10 19:15:12 +00:00
Emmanuel Pellereau a675a79f39 Revert "Reland "[vm, gc] Mark through new-space.""
This reverts commit 095171e937.

Reason for revert: breaks google3 (b/296014654)

Original change's description:
> Reland "[vm, gc] Mark through new-space."
>
>  - Adjust allocation stub write barrier elimination compensation to check if result is no longer in an active TLAB.
>
> TEST=ci
> Change-Id: I5d24602ae76ee861f2d009d67272251b04da3592
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322448
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: I73b60f799dda755028b65653ac21f27c13e009e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322682
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Emmanuel Pellereau <emmanuelp@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
2023-08-28 14:34:21 +00:00
Ryan Macnak 095171e937 Reland "[vm, gc] Mark through new-space."
- Adjust allocation stub write barrier elimination compensation to check if result is no longer in an active TLAB.

TEST=ci
Change-Id: I5d24602ae76ee861f2d009d67272251b04da3592
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322448
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-25 21:13:02 +00:00
Ivan Inozemtsev 3176cfcd21 Revert "[vm, gc] Mark through new-space."
This reverts commit 3fb88e4c66.

Reason for revert: b/297175670

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: I33075156160dc35861355d738a5776b74dce88b9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322344
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-08-23 13:13:28 +00:00
Ryan Macnak 3fb88e4c66 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-22 16:19:06 +00:00
Ilya Yanok 92ca7985a5 Revert "[vm, gc] Mark through new-space."
This reverts commit e95e7b8e96.

Reason for revert: suspected cause to non-deterministic AOT binaries crashes, b/296014654

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: Idda542c7c657d4f14c836423b173c9b067132212
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/320820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ilya Yanok <yanok@google.com>
2023-08-15 11:09:20 +00:00
Ryan Macnak e95e7b8e96 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-14 18:16:05 +00:00
Ryan Macnak a7e20dd2b0 Revert "[vm, gc] Mark through new-space."
This reverts commit 6194209b28.

Reason for revert: issues on arm32

Original change's description:
> [vm, gc] Mark through new-space.
>
>  - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
>  - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
>  - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
>  - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
>  - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

Change-Id: I434eb595c9e7858efc8c9b07cbca954e5649f506
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319321
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-08-09 18:13:17 +00:00
Ryan Macnak 6194209b28 [vm, gc] Mark through new-space.
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
 - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
 - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
 - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
 - reachabilityBarrier is now just a count of major GCs.

TEST=ci
Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-08-09 15:57:17 +00:00
Ryan Macnak f2611b7c8e [vm] Update lingering references to RawObject.
TEST=ci
Change-Id: If01ff0f7a9b10dc4e6f6d3139bd421df9ecc9855
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316528
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-07-26 21:53:49 +00:00
Alexander Aprelev bd8cc7bc93 [gardening] Use correct ObjectPtr in DART_COMPRESSED_POINTERS mode.
This is the follow-up to https://dart.googlesource.com/sdk/+/a5936e404b39f4af7b4fbf51d06ad3a99ff9fa36.

TEST=ci

Change-Id: Id0a21952d9f8b044ca207d8f8286fd76c0630cb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314980
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-07-19 21:17:35 +00:00
Alexander Aprelev a5936e404b [vm] Include field and local names in isolate invalid object retaining path message.
Retrieval of local names only works in JIT as it relies on availability of LocalVarDescriptors.

For example, for this
```
import 'dart:isolate';

foo() {
  final x = RawReceivePort();
  RawReceivePort().sendPort.send(() => print("$x"));
}

main() {
   foo();
}
```

error message says
```
Unhandled exception:
Invalid argument(s): Illegal argument in isolate message: (object is a ReceivePort)
 <- field x in foo.<anonymous closure> (from file:///usr/local/google/home/aam/p/d/dart-sdk1/sdk/s1.dart)
```

Bug: https://github.com/dart-lang/sdk/issues/52957
TEST=ci
Change-Id: I684014fd19ea82829e7df17a8a67d386551a5a82
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/314501
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2023-07-19 19:58:18 +00:00
Tess Strickland 2f63acea22 [vm] Attempt to avoid allocating non-Ptr fields with Object::null().
There have been multiple cases in the past where using Object::null()
as the initializer, even for non-Ptr fields, has caused nondeterministic
snapshot creation. In particular, this can happen if a non-Ptr field
is only used in some versions of an object but not others, and thus
there is no reason to worry about what the value of the field is during
runtime. Only when snapshotting can it become relevant that the field
value is a portion of the address of Object::null(), which can vary
over different runs.

Instead of initializing the entirety of allocated objects (outside
of a few rare cases) with Object::null(), only initialize the portion
of the object corresponding to object pointer fields (in most cases,
between from() and to() inclusively) to Object::null() and initialize
the rest of the object to 0.

With this change, the only special casing in Object::InitializeObject
that remains is to skip initialization for TypedData and Arrays when
the memory is guaranteed to be zero initialized and to initialize the
contents of Instructions objects with the break instruction, not 0.

Note that this behavior does not occur in the following cases, which
all involve copying an arbitrary object:

* Object::Clone(), which zero-initializes the contents of the object
  in all cases. However, the contents of the original object are then
  copied over before the thread can reach a safepoint.
* Inside the ObjectGraphCopier, which uses the old initialization
  behavior. This is safe, as any GC-important fields are immediately
  copied over, and the rest of the contents are eventually copied over
  before the caller receives the root of the copied object graph.

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/52876
Change-Id: Ib09fc562a8b6af97b509b493eb2d64109230ec35
Cq-Include-Trybots: luci.dart.try:vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312900
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2023-07-11 10:15:58 +00:00
Ryan Macnak 3a598b4c03 [vm, gc] Honor kAllocatablePageSize during promotion and snapshot reading.
Avoids objects between kAllocatablePageSize and kNewAllocatableSize getting promoted to regular pages and contributing to fragmentation. The snapshot allocator is also updated to be able to assert the invariant that objects larger than kAllocatePageSize never appear on non-large old pages.

TEST=ci
Bug: b/264420546 (speculative)
Change-Id: Ib2d61f0797bad4c8c6229cf4842169fae5c7053e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/306906
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-06-06 16:20:47 +00:00
Alexander Markov 2ee6fcf514 [vm] Remove TypeRef
TypeRef type wraps around another type and it was used to represent
a graph of recursive types. After [0], the only use of TypeRef is
for TypeParameter.bound which may indirectly reference the same
TypeParameter.

This change replaces TypeParameter.bound with TypeParameter.owner and
removes TypeRef entirely. Various parts of the VM no longer need to
handle and support TypeRefs.

TypeParameter.owner can reference a FunctionType, Class,
or, as an optimization, it can be set to null in order to share
class type parameters among different classes.

With the exception of the 'TypeParameter.owner' back pointer,
VM types are now not recursive and can be visited without
additional tracking.

Caveats:

* Generic FunctionType cannot be cloned in a shallow way:
  when copying a FunctionType, type parameters should be cloned too
  and their owners should be updated. For that reason,
  a mapping between 'from' and 'to' function types
  (FunctionTypeMapping) is maintained during type transformations
  such as InstantiateFrom.
  FunctionType::Clone is used instead of Object::Clone where
  appropriate.

* When testing types for subtyping and equivalence, mapping
  between function types is passed to make sure
  type parameters belong to the equivalent function types.

* IL serializer needs to serialize function types as a whole before
  serializing any types potentially pointing into the middle of a
  function type (such as return type 'List<Y0>' pointing into
  the middle of a function type 'List<Y0> Function<Y0>()').

[0] https://dart-review.googlesource.com/c/sdk/+/296300

TEST=ci

Change-Id: I67c2fd0117c6183a45e183919a7847fd1af70b3e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294165
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2023-04-28 19:29:36 +00:00
Ryan Macnak 96fd0b2208 [vm, gc] Avoid duplicate pointer visiting functions when compressed pointers are disabled.
This reduces code size ~10k, but more importantly it is easier to read profiling results without duplicate entries.

TEST=ci
Change-Id: I14c4a0d78d856242c5493df6de629ce048753c38
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294440
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2023-04-11 17:36:33 +00:00
Ryan Macnak 1eae028526 [vm] Fix zero-as-null-pointer-constant warnings in the VM.
Unfortunately we cannot enable -Wzero-as-null-pointer-constant because of zlib and icu headers we include.

TEST=build
Change-Id: I9ffd7d7e26f2b3dd616ce54d164b92e60a2366dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293882
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-10 21:38:57 +00:00
Alexander Aprelev 67683c3973 [vm/isolates] Introduce 'vm:isolate-unsendable' pragma.
Decorate Zone, Future, Completer, Timer and Stream with newly-introduced pragma to make sure that message verification stops earlier, produces shorter retaining path for the user.

BUG=https://github.com/dart-lang/sdk/issues/51722
TEST=send_unsupported_objects_test,isolate_exit_unsendable_test.dart
CoreLibraryReviewExempt: vm-specific pragmas
Change-Id: I499eea542d228ac9cf0797a682664f93f360dc80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289027
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-03-27 20:54:11 +00:00
Ryan Macnak 63e6041ca9 [vm] Update to variadic FATAL.
TEST=ci
Change-Id: Ic6bc784605e10760bb28ea6df34242336a33b4d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286947
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-03-06 22:06:59 +00:00
Alexander Aprelev 4f30b5c9b5 [vm/isolates] Include retaining path into an illegal send argument exception.
The exception contains retaining path that looks like this:
```
Invalid argument(s): Illegal argument in isolate message: (object extends NativeWrapper - Library:'dart:io' Class: _RandomAccessFileOpsImpl@14069316)
 <- Library:'dart:io' Class: _RandomAccessFile@14069316
 <- Library:'file:///vm/dart/isolates/send_unsupported_objects_test.dart' Class: SomeLog
 <- Library:'file:///vm/dart/isolates/send_unsupported_objects_test.dart' Class: SomeState
 <-  Class: Context
 <- Library:'dart:core' Class: _Closure@0150898

```

BUG=https://github.com/dart-lang/sdk/issues/51115
BUG=https://github.com/dart-lang/sdk/issues/48592
TEST=send_unsupported_objects_test

Change-Id: I022e693adccf43a7d2c95e1c7283fd7f210cf1d7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280523
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-02-23 03:11:08 +00:00
Martin Kustermann 7db7bd0573 [vm] Simplify is-deeply-immutable check in transitive object copy
Now that we have an immutability bit, we can set the bit in object
header of newly allocated objects if those objects are always
deeply immutable.

That allows only checking for the immutable bit instead of checking
against a whole range of cids when sending objects via
`SendPort.send()` / `Isolate.exit()`.

TEST=ci

Change-Id: I4451a9fe11f68c7dd874d49ab632daa3102be19e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281842
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-02-09 00:23:22 +00:00
Martin Kustermann 896e535fe9 [vm] Allow sharing of deeply immutable lists across isolates
We support allocating deeply immutable typed data in the Dart C API
already.
=> This CL adds a test for that.

Furthermore we hook into the already O(n) construction of
`List.unmodifiable(<list>)` and set the objects immutable bit
if all elements of `<list>` are also immutable.
=> This allows then sharing of such lists when sent across isolates.

We do this eagerly to

  - avoid introducing complexity to the deep immutability check in
    transitive copy algorithm

  - avoids re-checking for the subgraph on every send (similar to
    canonicalized bit for constants, we can simply rely on the
    immutable bit)

If we ever optimized `List.unmodifiable()` (which always performs
runtime call atm) we could fold this checking into the place that copies
list elements from `<list>` to the newly allocated list.

Issue https://github.com/dart-lang/sdk/issues/51302

TEST=vm/dart{,_2}/isolates/fast_object_copy2_test

Change-Id: Ie275e65036a4c53d992804972aae741a9f5ed6dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281424
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-02-08 19:14:13 +00:00
Martin Kustermann 63b5f5d4b1 [vm] Ensure we copy shallow (but not deeply) immutable lists
The object graph copy code assumed kImmutableCid is used only for
list constants but it turns out our List.immutable([]) implementation
isn't using a wrapper-view, but instead a fixed-length immutable list as
with constants:

  factory List.unmodifiable(Iterable elements) {
    final result = new List<E>.from(elements, growable: false);
    return makeFixedListUnmodifiable(result);
  }

The list constants should already be handled via canonical bit.

Issue https://github.com/dart-lang/sdk/issues/51302

TEST=vm/dart{,_2}/isolates/regress_51302_test

Change-Id: I5924084ff004dc52bf675897455453d728484d1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/281421
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-02-08 08:58:40 +00:00
Martin Kustermann c9f3aca45b [vm] Ensure we only enqueue maps/sets to be rehashed if we know it got copied successfully
Closes https://github.com/dart-lang/sdk/issues/51226

TEST=vm/dart{,_2}/regress_51226_test

Change-Id: I1da2dfe45863d5f1004efde3a4ecaae89ffd62fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280469
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2023-02-06 11:00:29 +00:00
Ryan Macnak f3067e8740 [vm] Add WeakArray; to be used to create weak canonical sets.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/50648
Change-Id: I788a8e3e5e8383c0b8d6f248d64a33dc67013260
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278860
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-01-17 20:33:03 +00:00
Josh Soref 0633041729 Spelling runtime vm
Closes https://github.com/dart-lang/sdk/pull/50786

TEST=ci

GitOrigin-RevId: 781a38e2925ba505f0223ecd578faa9be8e1485c
Change-Id: Ia3c615012d1bac3f0cf64dc01c2ceb40d50d1193
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276684
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2022-12-29 10:18:02 +00:00
Alexander Markov 3675fd25f4 [vm] Packed representation of record shape
The representation of record shape in record instances and record
types is changed from a pair

  int num_fields
  Array field_names

to a single integer - packed bitfield

  int num_fields(16)
  int field_names_index(kSmiBits-16)

where field names index is an index in the array available from
ObjectStore.

With the new representation of record shapes:
1) Size of record instances is reduced.
2) Number of comparisons for a shape test reduced from 2 to 1
(shape test is used during type checks).
3) A few operations removed from Record.hashCode.
4) Type testing stubs (TTS) are now supported for record types with
named fields. Previously it was not possible to check shape of records
with named fields in TTS as TTS cannot access object pool and cannot
load field names array).

TEST=existing

Issue: https://github.com/dart-lang/sdk/issues/49719

Change-Id: I7cdcbb53938aba5d561cd24dc99530395dbbea7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-12-20 20:11:48 +00:00
Alexander Aprelev 438b81c4f2 [vm/regexp] Share RegExp objects between isolates, keep isolate-specific state in static members.
Revert "[vm/regexp] Copy, rather than share RegExp objects between isolates." is in patchset 1.

Isolate RegEx-sharing benchmarks show 24-100x improvements.

Bug: https://github.com/dart-lang/sdk/issues/50639
TEST=ci
Change-Id: I87943c9b9928869f7a20052cb0dc71d24ad36087
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276240
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-12-20 02:28:04 +00:00
Martin Kustermann 896675c462 [vm] Always share double/float32x4/float64x2/int32x4 in inter-isolate messages
The JIT support for dynamically unboxing has been removed in [0]. As
such all double/float32x4/float64x2/int32x4 objects are immutable and
can therefore be shared across isolates.

[0] https://dart-review.googlesource.com/c/sdk/+/256211

TEST=ci

Change-Id: Ifd4e7c2444415b2e3b5269d9fbeb6570cc5d6768
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273680
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2022-12-07 00:48:28 +00:00
Alexander Aprelev 4275d595b6 [vm/regexp] Copy, rather than share RegExp objects between isolates.
RegExp code objects keep backtracking and registers stacks in object pools, can't be shared between isolates.

BUG=https://github.com/dart-lang/sdk/issues/50082
TEST=one_regexp_many_workers

Change-Id: Ic7db8d7a75a0951178b2f4800f96224d52506545
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273480
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2022-12-02 22:45:21 +00:00
Ryan Macnak a2de36e708 [vm] Rename the default implementation classes of Map and Set.
_InternalLinkedHashMap => _Map
_InternalImmutableLinkedHashMap => _ConstMap
_InternalLinkedHashSet => _Set
_InternalImmutableLinkedHashSet => _ConstSet

This makes things nicer to read in places that display implementation names, such as stack traces, debuggers, profilers and inspectors.

TEST=ci
Change-Id: Iec851c80ea2086cbe79934565dbf35f04809a836
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266303
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
2022-11-15 17:15:58 +00:00
Alexander Markov 10e9606861 [vm] More efficient 'is' tests for record types
'is' tests against record types are split into series of 'is' tests of
record fields. This is more efficient as record instances cannot be
added to subtype test caches (because type of a record instance
depends on types of its fields).

This change also adds canonicalization and constant evaluation of
LoadFieldInstr for record fields.

Performance on a trivial micro-benchmark (on x64):
JIT (RunTime): 4519104.5 -> 20031.5
AOT (RunTime): 4352583.0 -> 26281.6

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I2ed464cd3b31f365b17805f4e7debe1d6d1051fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268080
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-11-07 22:27:31 +00:00
Ryan Macnak 353dc891c8 [vm] Allocate old after isolate messages reach a certain size.
This avoids expensive promotion of large messages.

Cf. 04659de9f05af63d5fe4b8d67d70bb3cdf1c9a7a

TEST=ci
Change-Id: I89d5e560063f7d0c1f791ef299bb74c971f3dccf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/257801
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-09-26 23:45:07 +00:00
Vyacheslav Egorov 83ab5d5ca3 [vm] Remove dynamic field unboxing in JIT
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-09-20 13:42:45 +00:00
Alexander Markov c94103ae05 [vm] Initial implementation of record instances
TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I82ba571d1935d616fe3d4d6d579e59eb57d65a43
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256804
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-12 22:41:57 +00:00
Alexander Markov 9a023aeae9 [vm] Initial implementation of record types
TEST=language/record_type_test

Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: Ib2100c23513395c9fa9c541320eacbb33a2a119e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256802
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-09-12 22:40:57 +00:00
Vyacheslav Egorov 4a4eedd860 [vm] Clean up ClassTable
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.

The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.

Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.

If reload succeeds we drop original class table and
set program class table as heap walk one.

If reload fails we drop the program class table and
restore original one from heap walk table.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-10 15:12:35 +00:00
Derek Xu c9f0120220 Reland "[VM - Runtime] Return nullptr when allocating a FinalizablePersistentHandle fails"
This is a reland of commit b8d4e24338

How the failures were fixed:
1. My ExternalSizeLimit test crashed on msvc because I was using a
0-sized array. I have now changed that array to have size 1.
2. My ExternalSizeLimit test crashed on x64c because
ExternalTypedData::MaxElements(kExternalTypedDataUint8ArrayCid) is much
smaller than kMaxAddrSpaceMB/4 on x64c. I now call
ExternalTypedData::New() with a length argument of 1, and just pretend
that the external allocations are larger when calling
FinalizablePersistentHandle::New().

Original change's description:
> [VM - Runtime] Return nullptr when allocating a
> FinalizablePersistentHandle fails
>
> This CL adds checks to ensure that the tracked total size of
> externally allocated objects never exceeds the amount of memory on the
> system. When the limit is exceeded, then
> FinalizablePersistentHandle::New() will return nullptr.
>
> Resolves https://github.com/dart-lang/sdk/issues/49332
>
> TEST=ci
>
> Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TEST=I ran the tryjobs for the configurations that broke CI.

Change-Id: I813aa74667c59a4dbec7f53440ca8d0bf21256ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256973
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2022-09-06 15:13:16 +00:00
Ryan Macnak a2d5419f73 Revert "[VM - Runtime] Return nullptr when allocating a FinalizablePersistentHandle fails"
This reverts commit b8d4e24338.

Reason for revert: breaks msvc, msan, x64c builds

Original change's description:
> [VM - Runtime] Return nullptr when allocating a FinalizablePersistentHandle fails
>
> This CL adds checks to ensure that the tracked total size of externally
> allocated objects never exceeds the amount of memory on the system. When
> the limit is exceeded, then FinalizablePersistentHandle::New() will
> return nullptr.
>
> Resolves https://github.com/dart-lang/sdk/issues/49332
>
> TEST=ci
>
> Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=bkonyi@google.com,asiva@google.com,dnfield@google.com,derekx@google.com

Change-Id: I934bfbf5dc2e8e2ead5c74fe6b1d84e7b311788c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256972
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2022-08-31 16:41:59 +00:00
Ryan Macnak 3f4db67730 [vm] Avoid pauses from uninterruptible WeakTable::Rehash().
TEST=ci
Change-Id: I39f59931b0607542a76e4b1c64a3126dbd055b74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/247622
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-08-31 16:24:09 +00:00
Derek Xu b8d4e24338 [VM - Runtime] Return nullptr when allocating a FinalizablePersistentHandle fails
This CL adds checks to ensure that the tracked total size of externally
allocated objects never exceeds the amount of memory on the system. When
the limit is exceeded, then FinalizablePersistentHandle::New() will
return nullptr.

Resolves https://github.com/dart-lang/sdk/issues/49332

TEST=ci

Change-Id: Ib6cc92325b1d5efcb2965098fa45cfecc90995e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256201
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2022-08-30 18:51:59 +00:00
Ryan Macnak 938a2c81d2 [vm] Add Dart_NewUnmodifiableExternalTypedDataWithFinalizer.
This foregoes the optimization of removing CheckWritable if the unmodifiable views are not used from Dart code.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/49784
Change-Id: I18f3c36437ef136daf875358278caca4e3e0faa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255816
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-23 16:36:10 +00:00
Ryan Macnak 234128e8d3 Revert "[vm] Allocate old in object-graph-copy's slow path."
This reverts commit d0a60d67c9.

Reason for revert: Slow path transition seems to occur earlier than I expected

Original change's description:
> [vm] Allocate old in object-graph-copy's slow path.
>
> If we have reached the slow path, the message's subgraph is very likely to be large.
>
> TEST=ci
> Change-Id: Ie8a18c78936ae8a53f30dd61da3650e684c09dfa
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249081
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,rmacnak@google.com

Change-Id: I61faddd37e424a742427ae7c270cdb8b0a13c418
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255300
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-16 16:23:47 +00:00
Ryan Macnak a8a94dcc4d [vm] Fix leak of forwarding tables when isolate message sending fails.
TEST=ci
Change-Id: I644834dcb49f197390dd71468b42b74c68a3cdb7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255180
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-16 16:19:07 +00:00
Ryan Macnak d0a60d67c9 [vm] Allocate old in object-graph-copy's slow path.
If we have reached the slow path, the message's subgraph is very likely to be large.

TEST=ci
Change-Id: Ie8a18c78936ae8a53f30dd61da3650e684c09dfa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/249081
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-08-15 20:32:56 +00:00
Ryan Macnak c1e67ac84f [vm] Recognize unmodifiable typed data views.
These types now work with Dart_TypedDataAcquireData.

The presence of these types no longer degrades the performance of typed data indexed loads.

The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Change-Id: Iffad865708501acf96db418985cd5a69bd9afa55
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254501
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-11 18:10:48 +00:00
Ryan Macnak 10bf1cfe58 Revert "[vm] Recognize unmodifiabled typed data views."
This reverts commit d1112d37bd.

Reason for revert: b/242043014

Original change's description:
> [vm] Recognize unmodifiabled typed data views.
>
> These types now work with Dart_TypedDataAcquireData.
>
> The presence of these types no longer degrades the performance of typed data indexed loads.
>
> The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.
>
> TEST=ci
> Bug: https://github.com/dart-lang/sdk/issues/32028
> Bug: https://github.com/dart-lang/sdk/issues/40924
> Bug: https://github.com/dart-lang/sdk/issues/42785
> Change-Id: I05ac5c9543f6f61ac37533b9efe511254778caed
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253700
> Reviewed-by: Aske Simon Christensen <askesc@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

TBR=kustermann@google.com,rmacnak@google.com,askesc@google.com

TEST=ci
Change-Id: I32c1c460fc30c51bc0d42e7cfaafe72bf5630069
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254560
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-10 20:08:22 +00:00
Ryan Macnak d1112d37bd [vm] Recognize unmodifiabled typed data views.
These types now work with Dart_TypedDataAcquireData.

The presence of these types no longer degrades the performance of typed data indexed loads.

The presence of these types degrades the performance of typed data indexed stores much less. The performance of indexed stores is somewhat regressed if these types were not used.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/32028
Bug: https://github.com/dart-lang/sdk/issues/40924
Bug: https://github.com/dart-lang/sdk/issues/42785
Change-Id: I05ac5c9543f6f61ac37533b9efe511254778caed
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/253700
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-08-09 20:57:05 +00:00