The differences between the different implementations are very
minor, so abstract out the few that remain into Assembler:
* Change Load<X>FromPoolOffset to Load<X>FromPoolIndex and do pool
offset calculations internally, since all caller sites have the pool
index. (Especially important because the pool register is tagged on
X64 and ARM and untagged on ARM64, so this abstracts that away.)
* Add default for pool register argument on ARM, since ARM64 already had
one and X64 doesn't take a pool register argument.
Other changes:
* Use specific TestTypeABI registers within the helper method that adds
caller-side checks instead of passing registers as arguments and
document which registers are used for input or output (and when, if
they are used conditionally).
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ifc7a0eaa6aacf7f629aa9647b028500648af653d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167803
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
These checks are now performed in the dynamic closure call dispatcher.
To avoid having to create type argument vectors (TAVs) for default type
arguments at runtime, we cache a compile-time created TAV in the
ClosureData for the closure function which is retrieved by the
dispatcher when needed.
We also keep an associated packed field of information that can also be
determined at compile time:
* Whether the cached TAV needs instantiation or can share its
instantiator or function type arguments.
* The number of parent type parameters.
The former allows the generated IL to keep the invariant that the
InstantiateTypeArguments instruction (and the runtime entry it calls) is
only used for uninstantiated TAVs.
Also changes the destination name to an Value input for the
AssertSubtype instruction and adds handling for non-constant types and
names in that instruction's backend.
Additional changes:
* Adds new slots for ClosureData, Function, and TypeArguments.
* Adds a new kUnboxedUint8 representation (needed for
TypeParameterLayout::flags_, which is of type uint8_t).
* Extends LoadField to handle uint8_t unboxed native fields.
* Adds BoxUint8 for boxing unboxed uint8_t values.
Code size impact on Flutter gallery in release mode:
* arm7: total +0.08%, vmisolate: +0.56%, isolate: +0.42%,
readonly: -0.04%, instructions: -0.03%
* arm8: total +0.12%, vmisolate: +0.56%, isolate: +0.42%,
readonly: -0.002%, instructions: +0.03%
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Bug: https://github.com/dart-lang/sdk/issues/40813
Change-Id: I5a7de27a17e3119e27752bd0d10e1c6bc1b52a16
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158844
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
This change prevents the scaled addressing mode to be wrongly applied by the assembler.
Let's illustrate with an example:
__ ldr(R1, FieldAddress(R1, target::Type::nullability_offset()), kUnsignedByte);
Let's assume that target::Type::nullability_offset() == 57 (it is currently 61, but will be 57 in a forthcoming CL).
FieldAddress substracts 1 and builds an Address with offset 56.
The default operand size passed to Address is kDoubleWord, i.e. a scaling factor of 3.
Because 56 == (56 >> 3) << 3, the scaled addressing mode is chosen with a scaled offset of 7, which would be correct for a kDoubleWord access, but not for a kUnsignedByte access.
It is important to specify the same operand size in the instruction as in FieldAddress:
__ ldr(R1, FieldAddress(R1, target::Type::nullability_offset(), kByte), kUnsignedByte);
Change-Id: I46cf2b6ac103b2acec723189c88e211567b795af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167900
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
Unlike cfc8e6de, this does _not_ replace the default variable length
encoding for {Read,Write}Streams, but insteads adds separate
{Read,Write}{S,}LEB128 methods to the appropriate classes. If we later
find the cause of the issues that led to the revert of cfc8e6de, it'll
be easy to switch over then.
Note that WriteLEB128 asserts that the value is non-negative if used
with a signed type (since negative values suggests that SLEB128 should
be used instead for minimal encoding).
Also removes the various other encoding and decoding methods for
(S)LEB128 across the codebase and changes those clients to use
{Read,Write}Streams instead.
Other cleanups:
* Various constant-related cleanups in datastream.h.
* Adds DART_FORCE_INLINE to ReadStream::ReadByte and uses it in the
default variable length decoding methods for retrieving bytes
from the stream instead of managing current_ by hand.
* Creates a canonical empty CompressedStackMaps instance and uses
that instead of the null CompressedStackMaps instance in most cases.
The only remaining (expected) use of the null CompressedStackMaps
instance is for the global table in the object store when no global
table exists (e.g., in JIT mode before any snapshotting).
* Moves CompressedStackMapsIterator from code_descriptors.h to an
Iterator class within CompressedStackMaps in object.h (similar to
PcDescriptors::Iterator), to limit friend declarations and because it
conceptually makes more sense as part of CompressedStackMaps.
* Removed CompressedStackMaps::PayloadByte, since existing clients
(CompressedStackMaps::Iterator, StackMapEntry in program_visitor.cc)
are better served by just operating on the payload buffer directly
(with appropriate NoSafepointScopes).
* WriteStreams no longer allocate their initial space on construction,
but rather on the first write, so no allocation is performed by
constructing a never-used WriteStream.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-ubsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: Ice63321abaa79157fbe9f230a864c8bba0e6dea9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166421
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Leaf runtime calls a silently clobbering R23 and R25 registers, which
causes bugs when these calls happen from write barriers (which are not
expected to clobber anything outside of R25).
We introduce a special helper class to handle saving and restoring
necessary registers.
The bug originally reported by Joe Lin <zuojian.lzj@alibaba-inc.com>
who proposed a fix[1]. This CL builds on top of that fix and tries to
make it more safe by forcing users to use a more strict API.
[1] https://dart-review.googlesource.com/c/sdk/+/162300
TEST=vm/dart/write_barrier_register_clobber_test
Change-Id: I93e0a13a3c4c38ad28210b35750a66e615e3e44a
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162191
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Add information about the saved arguments descriptor for dispatcher
functions to Function::ToCString and information about both the
function kind and the saved arguments descriptor (when appropriate)
to the flow graph printer and the disassembler.
Change-Id: I41f5db2d80039cb397447eb9b9db47e60469b284
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158007
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This assignment appears to have the intention of performing a comparison
and asserting that the opcode encountered was as expected, rather than
assigning it to be that particular opcode.
Change-Id: I21f3c00f414d4b191f85e9dcf4391a6dd7023495
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157105
Auto-Submit: Drew Fisher <zarvox@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Right now we assign class ids to top-level classes, abstract classes as
well as concrete classes. All of them have allocated from a 16-bit pool
of ids. The VM FATAL()s once it hits that limit.
Customers who run very large programs (significant amount of generated
code) on the Dart VM have started to hit this 16-bit class limit.
Concrete classes can have instances in the heap. Our current heap layout
only allows 16-bit class ids to be encoded in the header word. To avoid
increasing the size of heap objects or shrinking the size of the identity
hash code to 16-bit we keep class ids in object headers to be 16-bit.
Abstract classes cannot have instances in the heap. Though their class
ids are encoded in type objects. Furthermore we sort classes in
AOT/AppJIT mode to perform fast class-id range checks. To avoid impacting
this optimization we treat abstract classes the same way as concrete
classes.
Top-level classes cannot have instances in the heap. Their class ids are
only used in the runtime code, for example for hot-reload as well as
part of the service protocol.
=> We can allocate class ids outside the 16-bit range for top-level
classes, thereby freeing a significant amount of space in the 16-bit
range.
This CL does exactly that: We change classid_t to be int32_t. The
ClassLayout::id_ can now be assigned ids outside 16-bit range for
top-level classes. To do this we keep dart classes and top level classes
as separate arrays in the ClassTable.
Issue https://github.com/dart-lang/sdk/issues/42533
See also b/160229360
Change-Id: I6710a644e7b0ab2d4f4c792bef8e1f91cb117421
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153607
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.
It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.
It allows few warnings still since they are triggered in third_party sources.
Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
In --enable-isolate group is
* turned off, then we have 1<->1 mapping between isolate and isolate
group, so moving the locks should have no impact
* turned on, then we have N<->1 mapping between isolates and isolate
group, mutators need to use the same locks to guarantee exclusive
access (e.g. to symbol table, type canonicalization, type arguments
canonicalization, ...)
This is a follow-up to share the program structure in AOT mode across
multiple isoltes.
In order to move the existing `Isolate::type_canonicalization_mutex_` to
`IsolateGroup` (and thereby make all mutators use the same lock), we need
to remove the `RunWithMutatorsStopped()` usage in the runtime entry.
=> Without this it can lead to deadlocks.
In order to remove usage of `RunWithMutatorsStopped()` we will change
access to the cached type arguments array to use load-acquire barriers
in generated code and store-release barriers in runtime code.
A nice side effect is that it reduces `sizeof(dart::Isolate)`.
Closes https://github.com/dart-lang/sdk/issues/41912
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: Ifd42691524fe41ffe8bb4e2623c5b8c1151de973
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148539
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
* AssertAssignable should not be generated for top types according to
IsTopTypeForSubtyping().
* Arguments descriptor should be passed when calling field initializer
from InitLateInstanceField/InitLateFinalInstanceField stubs.
If interpreter is enabled, InterpretCall stub expects to find
arguments descriptor in a register. Also, lazy compilation stub
may save arguments descriptor in a slot visited by GC, so arguments
descriptor register should contain a GC-safe value.
Fixes https://github.com/dart-lang/sdk/issues/41807
Change-Id: Ia33ed1bc57007d674799fb4453c8ab4841037211
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147383
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Previously the output used scrubbed qualified names which hides some
of the information and causes names of various synthetic functions
collide. We change it to use internal qualified names instead and add
additional information for disambiguation:
- For functions this means adding a prefix for various synthetic
methods;
- For anonymous closures we add their token position;
- For types we prefix type parameters with their owner's name.
We also add a unit test covering parsing of
--print-instructions-sizes-to output.
As part of adding a test dart2native was changed to support passing
extra arguments to gen_snapshot binary.
Fixes https://github.com/dart-lang/sdk/issues/41137
Cq-Include-Trybots: luci.dart.try:pkg-linux-debug-try,pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: I299ab0fb5993c8af9b51246483608d1e65970d67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146581
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit 922ea3e9b6 in patchset 1, fix for assertion triggered in https://ci.chromium.org/b/8883214567628884960 in patchset 2, fix for deadlock around symbols table mutex in patchset 4.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I655e337198214c9dfacbe76f7852b941b5a7e910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143462
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
#if defined(DART_PRECOMPILED_RUNTIME)
#error "AOT runtime should not use compiler sources (including header files)"
#endif // defined(DART_PRECOMPILED_RUNTIME)
Remove #if defined(DART_PRECOMPILED_RUNTIME) from most compiler sources.
Change-Id: Id175c83fdbea38d9d5e1371ff433e3888f2afe8a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143523
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.
Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.
This change tries to address this inconsistency.
A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.
Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Original revert in patchset 1, fix for deadlock issue in patchset 2: avoid reentrant calls to RunWithStoppedMutator.
Further review comments addressed in successive patchsets.
This reverts commit 8ef508ba36.
Original commit description:
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Change-Id: I6a0279d9812020ad7a5c2b7851980b6a29b95b9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143327
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.
To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.
Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I64c86525f4ef9cb30567a49a106bfe700355942b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136780
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Allocate the canonical true and false objects right after the null
object such that their pointers are the same except for a single bit.
This enables a number of optimizations:
- Test of a bool value becomes a bit test when it is known that the
value is a bool. on ARM64, this bit test can be combined with the
branch.
- Negation of a bool value becomes an xor when it is known that the
value is a bool.
- Load of a bool value becomes an immediate add with the null register
on ARM64.
This is a reland of f8ead09ba2
Change-Id: Iadb2169f80d3731153df3a93772f6142c3197260
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142375
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
This reverts commit 87b4848bfc as it broke simarm, simarm64 builds.
Revert "[vm] Distinguish between bool objects by a single bit in their pointers."
This reverts commit f8ead09ba2 being collateral damage for the revert above.
Change-Id: I0a12fe8095566430823c094b294c490b59cb3c39
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142404
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Allocate the canonical true and false objects right after the null
object such that their pointers are the same except for a single bit.
This enables a number of optimizations:
- Test of a bool value becomes a bit test when it is known that the
value is a bool. on ARM64, this bit test can be combined with the
branch.
- Negation of a bool value becomes an xor when it is known that the
value is a bool.
- Load of a bool value becomes an immediate add with the null register
on ARM64.
Change-Id: I70db024437e0f9cb15c401354e5e35802c354172
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137100
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Daco Harkes