- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: Ic7754e8d972763654eae2b7faa8670735d9cda3f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340644
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 5daaa7d9eb.
Reason for revert: internal crashes
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: I8a50074db343c63c14f0487ae8b4f5fee2c4ae76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/330720
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I4a6a23273d8ecb78c640f054731d4ceb737bfc4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325840
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
This reverts commit 3fb88e4c66.
Reason for revert: b/297175670
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
Change-Id: I33075156160dc35861355d738a5776b74dce88b9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322344
Reviewed-by: Martin Kustermann <kustermann@google.com>
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I6362802cd93ba5ba9c39f116ddff82e4feb4c312
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321304
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit e95e7b8e96.
Reason for revert: suspected cause to non-deterministic AOT binaries crashes, b/296014654
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
Change-Id: Idda542c7c657d4f14c836423b173c9b067132212
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/320820
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ilya Yanok <yanok@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I8c2c64b120766571b62d3bd8dab37ae81c2dca98
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319583
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This reverts commit 6194209b28.
Reason for revert: issues on arm32
Original change's description:
> [vm, gc] Mark through new-space.
>
> - Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
> - A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
> - Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
> - Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
> - reachabilityBarrier is now just a count of major GCs.
>
> TEST=ci
> Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: I434eb595c9e7858efc8c9b07cbca954e5649f506
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319321
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
- Initial and final marking no longer visit all of new-space, reducing the STW pause for major GC.
- A scavenge during concurrent marking must forward / filter objects in the marking worklist that are moved / collected, increasing the STW pause for minor GC.
- Unreachable intergenerational cycles and weak references are collected in the next mark-sweep instead of first requiring enough scavenges to promote the whole cycle or weak target into old-space.
- Artificial minor GCs are no longer needed to avoid memory leaks from back-to-back major GCs.
- reachabilityBarrier is now just a count of major GCs.
TEST=ci
Change-Id: I3668a2e56821f9eadf96e38c228dab27be656016
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/309826
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
If we are generating a SubtypeTestCache for a known type that only
needs instantiator type arguments to be fully instantiated, then
don't store and check against the unneeded function type arguments.
TEST=vm/cc/STC, vm/cc/TTS, ci
Change-Id: I370adf820168079322b8a87811057670a47ee6b3
Cq-Include-Trybots: luci.dart.try:vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-mac-release-arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311382
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
These checks convert what would likely be a segfault into a breakpoint
instead, but they're heavyweight enough on certain tests to cause
timeouts. Remove them as issues here are easily debugged without them.
Fixes https://github.com/dart-lang/sdk/issues/52787
TEST=vm/dart/regress_48523 on debug reload bots
Change-Id: Icf9ea6191eb82c254d9a01fbc9f8ba6255602084
Cq-Include-Trybots: luci.dart.try:vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311621
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
If these checks fail, then a `SubtypeNTestCache` stub has been called
with a `SubtypeTestCache` that uses a different number of inputs
than the stub expects, which could lead to unexpected false negatives
or false positives, so catch it early.
TEST=vm/cc/TTS, ci
Change-Id: Ia9c48aad4c35872cad32b7b6f3eead3ee9e59680
Cq-Include-Trybots: luci.dart.try:vm-mac-debug-arm64-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-debug-ia32-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311380
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This also lowers the threshold for converting from a linear cache
to a hash-based cache from 100 to 30 on non-IA32 architectures.
(IA32 still goes to runtime if there's a hash-based cache.)
For SubtypeTestCache benchmarks above this threshold, we see an
improvement varying from ~50-100% to ~700-800% on all architectures,
from lowest number of checks (50) to highest (1000).
For SubtypeTestCache benchmarks below this threshold, no major
changes are seen: generally <5% improvement or <5% regression per
check at most.
TEST=vm/cc/TTS
Change-Id: I83aa7c085ab5a411e944ec660d6b8eba7a788ee0
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/308941
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In particular, avoid storing/comparing the destination type unless the
SubtypeTestCache was created for a dynamic AssertAssignable instruction.
TEST=vm/cc/STC_{Linear,Hash}Lookup, vm/cc/TTS
Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: I279c69a668ce19785785cf71707acf951f2b2133
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310340
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Previously, SlowTypeTestStub would use the incoming instance and
destination type to determine which `SubtypeNTestCacheStub` to call,
where `N` is the number of inputs checked for each entry. The code
for generating `InstanceOf` instructions performs similar work,
inlining some checks and then falling back to checking and updating
an appropriate `SubtypeNTestCacheStub` if the inlined checks fail.
Meanwhile, the runtime would use all available information to
create a new entry when needed, depending on the `SubtypeNTestCacheStub`
called at runtime to ignore the parts of the entry it didn't need.
To illustrate a situation where the runtime might use more
information to create an entry than needed by the stub, consider
the following class header:
```
class C<T> extends D
```
If no subtype of `D` requires knowing the specific type arguments for
an instance to determine whether that instance is an instance of `D`,
then the compiler can generate a `Subtype1TestCacheStub` call, where
only the class id of the instance is checked.
However, in the runtime, when updating the cache, the runtime will
create an entry that includes the specific type arguments used to
instantiate `C` when the initial cache miss occurred, as the runtime
does not know how many inputs the code that checks that SubtypeTestCache
uses. If the cache has grown to the point that it is hash-based, that
means the runtime uses the instance type arguments of the first cache
miss to determine the correct place to create a new entry.
When the stub only performs a linear search over the cache contents,
this works, as every entry is checked until a matching entry is found,
and thus the stub can ignore unneeded information. However, if a cache
is hash-based, this doesn't work: if the first miss was an instance
of `C<int>`, then the runtime includes all that information in a new
entry. This means the stub has to calculate the instance type
arguments, which it did not have to do before, and if a subsequent
check uses different instance type arguments (e.g., an instance of
`C<String>`), the result may still be a cache miss, and the runtime
adds another entry which has the same information as the previous
entry as far as the stub was originally concerned.
Thus, to limit itself to considering only the inputs used by the
appropriate `SubtypeNTestCacheStub`, the runtime needs to be told how
many inputs are used. This CL adds an additional field to
`SubtypeTestCache`s which contains the number of used inputs, and
changes the places where `SubtypeTestCache`s are allocated in the
flow graph compiler and the `DRT_TypeCheck` runtime entry to pass the
appropriate number of inputs. To ensure that the runtime and stubs
agree on how many inputs should be used, the `SlowTypeTestStub` now
uses the value of this field to determine which `SubtypeNTestCacheStub`
to call.
TEST=vm/cc/STC_{Linear,Hash}Lookup
Bug: https://github.com/dart-lang/sdk/issues/48345
Change-Id: Ie16f2d12c4c5eae856b668f28178818203662d1c
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/310180
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Also eliminates the need to store the delayed type arguments on the
stack during the search loop on ARM.
TEST=vm/cc/TTS
Change-Id: Ife01cc8d255724c272bc277eec397486cedc07a7
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307681
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
In d592882f, the old uint8_t `type_state_` and `nullability_` fields
in subclasses of `AbstractType` were merged into a single uint32_t
`flags_` field in `AbstractType` itself.
Accessing the nullability information was abstracted into two methods:
* `LoadAbstractTypeNullability`
* `CompareAbstractTypeNullabilityWith`
However, the code from these methods were based off the old
`CompareTypeNullabilityWith` and `CompareFunctionTypeNullabilityWith`
methods that used unsigned byte loads to access the old field, and kept
that even though the new field is now a 32-bit unsigned integer. This
works for now, because the nullability portion of the `flags_` field
happens to be the lowest two bits, but might lead to confusion if
someone attempts to write similar code for accessing the other parts of
the `flags_` field without realizing it's larger than 8 bits.
Thus, this CL does the following:
* Unifies the implementation of both methods across architectures.
* Adds a native slot definition for the `AbstractType::flags_` field
and uses `LoadFromSlot` to load the `flags_` field instead. Other
places where the `AbstractType::flags_` field is accessed from the
assembler are also changed to use `LoadFromSlot` appropriately.
TEST=The existing suite of tests are sufficient.
Change-Id: Ie578e4dddce810223ca9e767403136cb40d340f3
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-x64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-linux-debug-x64c-try,vm-kernel-precomp-linux-release-simarm-try,vm-linux-release-x64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-linux-release-ia32-try,vm-linux-debug-simriscv64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302187
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
For class type parameters 'parameterized_class_id' duplicates
a recently added 'owner' and can be removed.
'owner' is changed to hold class reference as a class id (Smi).
TEST=ci
Change-Id: I93fc11e6bcfcc00058a5281b497f59c5c5847ea6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299160
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
TypeRef type wraps around another type and it was used to represent
a graph of recursive types. After [0], the only use of TypeRef is
for TypeParameter.bound which may indirectly reference the same
TypeParameter.
This change replaces TypeParameter.bound with TypeParameter.owner and
removes TypeRef entirely. Various parts of the VM no longer need to
handle and support TypeRefs.
TypeParameter.owner can reference a FunctionType, Class,
or, as an optimization, it can be set to null in order to share
class type parameters among different classes.
With the exception of the 'TypeParameter.owner' back pointer,
VM types are now not recursive and can be visited without
additional tracking.
Caveats:
* Generic FunctionType cannot be cloned in a shallow way:
when copying a FunctionType, type parameters should be cloned too
and their owners should be updated. For that reason,
a mapping between 'from' and 'to' function types
(FunctionTypeMapping) is maintained during type transformations
such as InstantiateFrom.
FunctionType::Clone is used instead of Object::Clone where
appropriate.
* When testing types for subtyping and equivalence, mapping
between function types is passed to make sure
type parameters belong to the equivalent function types.
* IL serializer needs to serialize function types as a whole before
serializing any types potentially pointing into the middle of a
function type (such as return type 'List<Y0>' pointing into
the middle of a function type 'List<Y0> Function<Y0>()').
[0] https://dart-review.googlesource.com/c/sdk/+/296300
TEST=ci
Change-Id: I67c2fd0117c6183a45e183919a7847fd1af70b3e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294165
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is a reland of commit afdf640866
Original change's description:
> [vm] Refactor StubCodeCompiler to be a real class.
>
> Previously, StubCodeCompiler was just a set of static methods, all of
> which take an assembler as their first arg. This makes it hard to pass
> additional state to the ~160 stub macro defined stub generators.
>
> This refactor makes StubCodeCompiler a real class, with assembler as a
> field. So we can easily add new fields to the class later, to pass new
> state without having to update every stub generator.
>
> assembler is declared as a public field for a few reasons:
> - There's one place where it needs to be accessed by a non-member
> function (in the ia32 file).
> - If it's private, it has to be named assembler_, which would mean a lot
> more insignificant diffs.
> - Non-member functions that take assembler would have to take assembler_,
> for consistency with the __ macro, which would be weird.
>
> Change-Id: I142f0803a07c7839753188065c69c334d4d1798a
> TEST=CI
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289924
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>
Change-Id: Ib5be28c46a0a80b84e31aea60893ab5bbc02e2ea
TEST=CI, including all sanitizers and architectures
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290681
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
This reverts commit afdf640866.
Reason for revert: breaks google3. See b/274712076.
Original change's description:
> [vm] Refactor StubCodeCompiler to be a real class.
>
> Previously, StubCodeCompiler was just a set of static methods, all of
> which take an assembler as their first arg. This makes it hard to pass
> additional state to the ~160 stub macro defined stub generators.
>
> This refactor makes StubCodeCompiler a real class, with assembler as a
> field. So we can easily add new fields to the class later, to pass new
> state without having to update every stub generator.
>
> assembler is declared as a public field for a few reasons:
> - There's one place where it needs to be accessed by a non-member
> function (in the ia32 file).
> - If it's private, it has to be named assembler_, which would mean a lot
> more insignificant diffs.
> - Non-member functions that take assembler would have to take assembler_,
> for consistency with the __ macro, which would be weird.
>
> Change-Id: I142f0803a07c7839753188065c69c334d4d1798a
> TEST=CI
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289924
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>
Change-Id: If36a9122e1a55d86673d05afbbd21dfb27d7acd5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290522
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Emmanuel Pellereau <emmanuelp@google.com>
Previously, StubCodeCompiler was just a set of static methods, all of
which take an assembler as their first arg. This makes it hard to pass
additional state to the ~160 stub macro defined stub generators.
This refactor makes StubCodeCompiler a real class, with assembler as a
field. So we can easily add new fields to the class later, to pass new
state without having to update every stub generator.
assembler is declared as a public field for a few reasons:
- There's one place where it needs to be accessed by a non-member
function (in the ia32 file).
- If it's private, it has to be named assembler_, which would mean a lot
more insignificant diffs.
- Non-member functions that take assembler would have to take assembler_,
for consistency with the __ macro, which would be weird.
Change-Id: I142f0803a07c7839753188065c69c334d4d1798a
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289924
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
The representation of record shape in record instances and record
types is changed from a pair
int num_fields
Array field_names
to a single integer - packed bitfield
int num_fields(16)
int field_names_index(kSmiBits-16)
where field names index is an index in the array available from
ObjectStore.
With the new representation of record shapes:
1) Size of record instances is reduced.
2) Number of comparisons for a shape test reduced from 2 to 1
(shape test is used during type checks).
3) A few operations removed from Record.hashCode.
4) Type testing stubs (TTS) are now supported for record types with
named fields. Previously it was not possible to check shape of records
with named fields in TTS as TTS cannot access object pool and cannot
load field names array).
TEST=existing
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I7cdcbb53938aba5d561cd24dc99530395dbbea7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
TypeParameter::set_index was already asserting for 16 bits but silently truncating to 8 bits. The size of TypeParameter does not increase due to allocation rounding.
Add explicit range check to class finalization.
TEST=ci
Change-Id: Id4017f74b7dba47b00d64e77e787712c70290dac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275560
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Lower the threshold for converting from a linear to a hash-based cache
on most architectures from 500 to 10.
Due to register pressure, the InstantiateTypeArguments stub on IA32
continues to go to the runtime for hash caches, and so we do not
lower the threshold for converting to a hash-based cache there.
The following are benchmark results for those benchmark that use enough
Instantiations to trigger the use of hash-based caches. In the following
tables, "Results 1" denotes the benchmark results from only this change,
whereas "Results 2" include from comparing to the results prior to
4f925105cf, when only linear caches were used.
Dart AOT:
* InstantiateTypeArguments.Instantiate100
Arch | CPU | Results 1 | Results 2
-------|----------------|-----------------------
ARM | Odroid-C2 | 382.8% | 381.5%
ARM | Raspberry Pi 4 | 486.7% | 449.2%
ARM64 | Odroid-C2 | 328.1% | 372.8%
ARM64 | Raspberry Pi 4 | 1283% | 1281%
ARM64C | Raspberry Pi 4 | 2353% | 2811%
X64 | Intel Xeon | 568.7% | 584.9%
* InstantiateTypeArguments.Instantiate1000
Arch | CPU | Results 1 | Results 2
-------|----------------|------------------------
ARM | Odroid-C2 | 743.7% | 3821%
ARM | Raspberry Pi 4 | 486.7% | 3218%
ARM64 | Odroid-C2 | 584.7% | 3222%
ARM64 | Raspberry Pi 4 | 430.7% | 8172%
ARM64C | Raspberry Pi 4 | 491.4% | 16699%
X64 | Intel Xeon | 954.1% | 5528%
Dart JIT:
* InstantiateTypeArguments.Instantiate100
Arch | CPU | Results 1 | Results 2
-------|----------------|-----------------------
ARM | Raspberry Pi 4 | 315.7% | 295.1%
ARM64 | Raspberry Pi 4 | 1070% | 1058%
ARM64C | Raspberry Pi 4 | 1769% | 2095%
X64 | Intel Xeon | 507.4% | 496.2%
* InstantiateTypeArguments.Instantiate1000
Arch | CPU | Results 1 | Results 2
-------|----------------|-----------------------
ARM | Raspberry Pi 4 | 565.2% | 2550%
ARM64 | Raspberry Pi 4 | 406.8% | 7375%
ARM64C | Raspberry Pi 4 | 379.6% | 12996%
X64 | Intel Xeon | 807.9% | 4459%
During work on this change, an issue was found where cache lookups
in the stub on ARM64C always failed and went to runtime, even with
the old linear-only caches, hence the much larger performance gains
in those rows above.
TEST=vm/cc/TypeArguments_Cache_{Some,Many}Instantiations
Fixes: https://github.com/dart-lang/sdk/issues/48344
Change-Id: I3d29566ba0582502954c9fc59626ceb8fd40317a
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simriscv64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-nnbd-linux-debug-simriscv64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/270702
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Previously, the VM used a linear array to cache previous instantiations
of a type arguments object. Now once the cache hits a certain number
of occupied entries, the VM changes to using a hash-based approach.
The InstantiateTypeArguments stubs have not yet been updated to
traverse the hash-based cache, so once the cache has grown too large,
all attempts at instantiations, even those that are in the cache, go to
the runtime. Thus, until the stubs are updated, this is only an
improvement if the cost of traversing the linear cache dominates the
cost of making a runtime call. Our benchmarks see a ~40% performance
regression for hash-based caches of size 100 but a ~400% performance
improvement for hash-based caches of size 1000. Thus, we currently
split the difference and set the maximum size of linear caches to 500.
TEST=vm/cc/TypeArguments_Cache_ManyInstantiations
Bug: https://github.com/dart-lang/sdk/issues/48344
Change-Id: I7f1376943523bb5bcd8b175cfb1936779ea73d60
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simriscv64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/265325
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Some large applications are bumping up against the 16-bit limit.
The object header changes from hash:32,cid:16,size:8,gc:8 to hash:32,cid:20,size:4,gc:8.
Because the CID field is no longer a power of two, it takes two instructions to load instead of one, increasing code size. The alternative change to cid:32,hash:24,gc:8 was considered and rejected because the reduction in hash bits very noticeably affects performance in applications with large data sets.
TEST=ci
Bug: b/255934984
Change-Id: I5fa36ba48a6852359d994393cf80c6a761c5d84c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269120
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
'is' tests against record types are split into series of 'is' tests of
record fields. This is more efficient as record instances cannot be
added to subtype test caches (because type of a record instance
depends on types of its fields).
This change also adds canonicalization and constant evaluation of
LoadFieldInstr for record fields.
Performance on a trivial micro-benchmark (on x64):
JIT (RunTime): 4519104.5 -> 20031.5
AOT (RunTime): 4352583.0 -> 26281.6
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I2ed464cd3b31f365b17805f4e7debe1d6d1051fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/268080
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change introduces specialized stubs and IL instruction for
allocating records with 2 or 3 fields. This makes allocation of
small records slightly faster compared to a construction of similar
class instances and makes code size of record allocation smaller.
Benchmark:
MultipleReturns.NotInlined.Record(RunTime) 77150 -> 66222
MultipleReturns.NotInlined.RecordNamed(RunTime) 78073 -> 67044
MultipleReturns.Forwarded.Record(RunTime) 97130 -> 77635
MultipleReturns.Forwarded.RecordNamed(RunTime) 96495 -> 77904
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I8ed7add06b39ba79dfd78bbe2afaefe606cc505b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/266420
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This is a reland of commit 9700458975
Original change's description:
> [vm/compiler] Initial implementation of IL binary serialization
>
> This change adds binary serialization/deserialization of flow graphs.
> It supports all IL instructions and certain objects which can be
> referenced from IL instructions. IL binary serialization is a useful
> machanism which would allow us to split compilation into multiple parts
> in order to parallelize AOT compilation.
>
> The program structure (libraries/classes/functions/fields) is not
> serialized. It is assumed that reader and writer use the same
> program structure.
>
> Caveats:
> * FFI callbacks are not supported yet.
> * Closure functions are not re-created when reading flow graph.
> * Flow graph should be in SSA form (unoptimized flow graphs are not
> supported).
> * JIT mode is not supported (serializer currently assumes lazy
> linking of native methods and empty ICData).
>
> In order to test IL serialization, --test_il_serialization VM option is
> added to serialize and deserialize flow graph before generating code.
TEST=vm/dart/splay_test now runs with --test_il_serialization.
TEST=Manual run of vm-kernel-precomp-linux-debug-x64-try with
--test_il_serialization enabled (only ffi tests failed).
TEST=gcc build on dart-sdk-linux-try bot.
Issue: https://github.com/dart-lang/sdk/issues/43299
> Change-Id: I7bbfd9e3a301e00c9cfbffa06b8f1f6c78a78470
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254941
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
Change-Id: I64ff9747f761496a096371e490ef070a14023256
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255840
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit 9700458975.
Reason for revert: breaks Dart SDK build using gcc and dart-sdk-linux-main bot.
Original change's description:
> [vm/compiler] Initial implementation of IL binary serialization
>
> This change adds binary serialization/deserialization of flow graphs.
> It supports all IL instructions and certain objects which can be
> referenced from IL instructions. IL binary serialization is a useful
> machanism which would allow us to split compilation into multiple parts
> in order to parallelize AOT compilation.
>
> The program structure (libraries/classes/functions/fields) is not
> serialized. It is assumed that reader and writer use the same
> program structure.
>
> Caveats:
> * FFI callbacks are not supported yet.
> * Closure functions are not re-created when reading flow graph.
> * Flow graph should be in SSA form (unoptimized flow graphs are not
> supported).
> * JIT mode is not supported (serializer currently assumes lazy
> linking of native methods and empty ICData).
>
> In order to test IL serialization, --test_il_serialization VM option is
> added to serialize and deserialize flow graph before generating code.
>
> TEST=vm/dart/splay_test now runs with --test_il_serialization.
>
> TEST=Manual run of vm-kernel-precomp-linux-debug-x64-try with
> --test_il_serialization enabled (only ffi tests failed).
>
> Issue: https://github.com/dart-lang/sdk/issues/43299
> Change-Id: I7bbfd9e3a301e00c9cfbffa06b8f1f6c78a78470
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254941
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
TBR=vegorov@google.com,kustermann@google.com,rmacnak@google.com,alexmarkov@google.com
Change-Id: Iae4e4868f183815a8fc3cd79597141b3896e23d7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Issue: https://github.com/dart-lang/sdk/issues/43299
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/255780
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change adds binary serialization/deserialization of flow graphs.
It supports all IL instructions and certain objects which can be
referenced from IL instructions. IL binary serialization is a useful
machanism which would allow us to split compilation into multiple parts
in order to parallelize AOT compilation.
The program structure (libraries/classes/functions/fields) is not
serialized. It is assumed that reader and writer use the same
program structure.
Caveats:
* FFI callbacks are not supported yet.
* Closure functions are not re-created when reading flow graph.
* Flow graph should be in SSA form (unoptimized flow graphs are not
supported).
* JIT mode is not supported (serializer currently assumes lazy
linking of native methods and empty ICData).
In order to test IL serialization, --test_il_serialization VM option is
added to serialize and deserialize flow graph before generating code.
TEST=vm/dart/splay_test now runs with --test_il_serialization.
TEST=Manual run of vm-kernel-precomp-linux-debug-x64-try with
--test_il_serialization enabled (only ffi tests failed).
Issue: https://github.com/dart-lang/sdk/issues/43299
Change-Id: I7bbfd9e3a301e00c9cfbffa06b8f1f6c78a78470
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/254941
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Ryan Macnak