Commit Graph

172 Commits

Author SHA1 Message Date
Ryan Macnak 739d987c91 [vm, compiler] Add MSAN support for ARM64.
TEST=local
Bug: https://github.com/dart-lang/sdk/issues/52399
Change-Id: I6af1954c70208534e87b6b87dd5cf62aaf595bd3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329582
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2023-11-28 23:52:49 +00:00
Vyacheslav Egorov dcac60b672 [vm/aot] Fix BoxInt64 in deferred units
When generating code for deferred units compiler
can't emit PC relative call to the shared Mint allocation
stub. This causes compiler to emit an indirect call
through a Code object. Such calls clobber CODE_REG
which is actually an allocatable register.

Fix this by using non-allocatable register instead of
CODE_REG when generating indirect calls through
Code object in AOT mode. Callees don't expect
anything useful in CODE_REG anyway because AOT
calling convetion does not use it.

TEST=vm/cc/{BranchLinkPreservesRegisters,JumpAndLinkPreservesRegisters,CallCodePreservesRegisters}

Bug: b/242559057
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-linux-debug-simriscv64-try,vm-ffi-qemu-linux-release-riscv64-try
Change-Id: Ib1fdc1c104d0269d41bb1ab9cbe292ad28c7cd49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/338127
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-11-28 13:15:56 +00:00
Daco Harkes 0cd55a18c1 [vm] Introduce cachable idempotent calls
Adds a `CachableIdempotentCallInstr` that can be invoked via
`@pragma('vm:cachable-idempotent')` if the call-sites is force
optimized.

The object pool is not visited by the scavenger. So, we store the
results as unboxed integers. Consequently, only dart functions that
return integers can be cached.

Cachable idempotent calls should never be inlined. After the first
call the function not be called again.

The call itself is on a slow path to avoid register spilling on the
fast path.

TEST=vm/cc/IRTest_CachableIdempotentCall
TEST=runtime/tests/vm/dart/cachable_idempotent_test.dart

Bug: https://github.com/dart-lang/sdk/issues/51618
Change-Id: I612e896f27add76f57796c060157e14cc687a0fd
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/301601
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-10-27 07:55:47 +00:00
Daco Harkes 9fbaa62a1a [vm] Use SnapshotBehavior for switchable call pool entries
Follow up of:
* https://dart-review.googlesource.com/c/sdk/+/328861
* https://dart-review.googlesource.com/c/sdk/+/331087
* https://dart-review.googlesource.com/c/sdk/+/331120

The pool entries for switchable calls have their pool entries
transformed during snapshot loading in AOT. This behavior is now folded
into `SnapshotBehavior`.

Because snapshot behavior is set on the call sites, and the call sites
are already AOT specific, no conditionals inside app_snapshot.cc are
needed anymore.

The Arm64 implementation does not use the assembler for implicitly
adding pool entries, it does so explictly and loads two pool entries
in one load.

We could consider doing these explicit loads in the other architectures
as well. This would avoid adding the snapshot_behavior argument to
`LoadUnqiueObject` for the other architectures.

IA32 has no AOT support.

TEST=precomp bots

Change-Id: I75f70e00ca44f6545694d9b59e3ad08ad8bb3f8e
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-simriscv64-try,vm-aot-mac-product-arm64-try,vm-aot-win-debug-arm64-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-debug-arm-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331101
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-10-19 22:11:48 +00:00
Daco Harkes 243cf59776 [vm] Object pool SnapshotBehavior for bootstrapping native functions
`NativeCallInstr::EmitNativeCode` `link_lazily` code path uses the
new snapshotting behavior, and the non-`link_lazily` checks that it's
never snapshotted.

The snapshot behavior argument is threaded from the the EmitNativeCode
to the places where the pool entry is constructed.

Does not yet migrate `kSwitchableCallMissEntryPoint` and
`kMegamorphicCallEntryPoint`.

TEST=CI

Change-Id: I8dd1dc5aa525f888b493856c60ae385087463d02
Cq-Include-Trybots: luci.dart.try:vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-simriscv64-try,vm-aot-linux-release-arm64-try,vm-aot-android-release-arm64c-try,vm-aot-win-debug-arm64-try,vm-aot-mac-product-arm64-try,vm-mac-debug-arm64-try,vm-ffi-android-debug-arm-try,vm-ffi-qemu-linux-release-arm-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328861
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-10-18 13:27:26 +00:00
Daco Harkes 0e9e000264 [vm] Unify LoadInt32FromBoxOrSmi in assemblers
And add unit tests.

TEST=vm/cc/Assembler_LoadInt32

Change-Id: Ib697a0beae97968a0d1ae50c54b5769d9c2a23de
Cq-Include-Trybots: luci.dart.try:vm-linux-release-simarm64-try,vm-aot-linux-release-simarm64-try,vm-linux-debug-x64c-try,vm-aot-linux-debug-simriscv64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/328521
Reviewed-by: Tess Strickland <sstrickl@google.com>
2023-09-30 08:12:16 +00:00
Daco Harkes b64212b111 [vm] Make UnboxInstr logic reusable in assembler
Only unifies wordsize unboxed loads, and skips ia32. Leaves the other
UnboxInstr machine code untouched.

TEST=vm/cc/Assembler_LoadWordFromBoxOrSmi

Change-Id: Ib24f503cef257001838928371c2d442465100b11
Cq-Include-Trybots: luci.dart.try:vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-kernel-linux-debug-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-simarm-try,vm-linux-release-simarm64-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/327680
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-09-27 13:05:49 +00:00
Ryan Macnak 139ff289db [vm, compiler] Allocation canaries.
Before each allocation, check the memory hasn't been touched since page initialization. This more reliably detects the issue fixed in a4381f127c.

TEST=ci
Change-Id: I5c0ed0a18a32515eab2ae90110b5a7e7f48779dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325366
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2023-09-15 17:04:55 +00:00
Tess Strickland 3bbe27eadf [vm/compiler] Perform word-sized copies in MemoryMove when possible.
When copying elements that are smaller than words on non-X86
architectures, only copy element by element until the remaining elements
can be copied in word-sized chunks, and then do so.

TEST=vm/cc/IRTest_Memory, co19{,_2}/LibTest/typed_data,
     lib{,_2}/typed_data, corelib{,_2}/list_test

Issue: https://github.com/dart-lang/sdk/issues/42072

Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-linux-debug-simriscv64-try,vm-mac-debug-arm64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-mac-release-arm64-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-linux-release-simarm-try,vm-linux-release-simarm64-try,vm-mac-release-arm64-try,vm-aot-android-release-arm64c-try,vm-ffi-android-debug-arm64c-try
Change-Id: I61eab310b92a6bc5ebd88fa63d562103d887cb74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/324280
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-09-07 14:11:19 +00:00
Tess Strickland 396796fba9 [vm] Abstract out pieces of SubtypeTestCache stubs across architectures.
Also eliminates the need to store the delayed type arguments on the
stack during the search loop on ARM.

TEST=vm/cc/TTS

Change-Id: Ife01cc8d255724c272bc277eec397486cedc07a7
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-product-x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-debug-x64c-try,vm-aot-tsan-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-aot-dwarf-linux-product-x64-try,vm-linux-release-ia32-try,vm-linux-debug-x64c-try,vm-linux-debug-x64-try,vm-linux-debug-simriscv64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-mac-release-arm64-try,vm-mac-release-x64-try,vm-tsan-linux-release-x64-try,vm-reload-rollback-linux-release-x64-try,vm-reload-linux-release-x64-try,vm-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/307681
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2023-06-13 09:53:12 +00:00
Tess Strickland 4b8aa6a634 [vm] Update assembler to access AbstractType::flags_ as Uint32.
In d592882f, the old uint8_t `type_state_` and `nullability_` fields
in subclasses of `AbstractType` were merged into a single uint32_t
`flags_` field in `AbstractType` itself.

Accessing the nullability information was abstracted into two methods:

* `LoadAbstractTypeNullability`
* `CompareAbstractTypeNullabilityWith`

However, the code from these methods were based off the old
`CompareTypeNullabilityWith` and `CompareFunctionTypeNullabilityWith`
methods that used unsigned byte loads to access the old field, and kept
that even though the new field is now a 32-bit unsigned integer. This
works for now, because the nullability portion of the `flags_` field
happens to be the lowest two bits, but might lead to confusion if
someone attempts to write similar code for accessing the other parts of
the `flags_` field without realizing it's larger than 8 bits.

Thus, this CL does the following:

* Unifies the implementation of both methods across architectures.
* Adds a native slot definition for the `AbstractType::flags_` field
  and uses `LoadFromSlot` to load the `flags_` field instead. Other
  places where the `AbstractType::flags_` field is accessed from the
  assembler are also changed to use `LoadFromSlot` appropriately.

TEST=The existing suite of tests are sufficient.

Change-Id: Ie578e4dddce810223ca9e767403136cb40d340f3
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-release-x64-try,vm-aot-linux-release-simarm_x64-try,vm-aot-linux-release-simarm64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-linux-debug-x64c-try,vm-kernel-precomp-linux-release-simarm-try,vm-linux-release-x64-try,vm-linux-release-simarm64-try,vm-linux-release-simarm-try,vm-linux-release-ia32-try,vm-linux-debug-simriscv64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/302187
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2023-05-10 22:40:58 +00:00
Ryan Macnak ba2b255200 [vm] Fix compilation warnings-as-errors with gcc 12.
TEST=local build
Change-Id: Iaeeef4ff9ca90c020f296026b22a935a9ca39431
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294660
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2023-04-12 01:39:08 +00:00
Alexander Markov 3675fd25f4 [vm] Packed representation of record shape
The representation of record shape in record instances and record
types is changed from a pair

  int num_fields
  Array field_names

to a single integer - packed bitfield

  int num_fields(16)
  int field_names_index(kSmiBits-16)

where field names index is an index in the array available from
ObjectStore.

With the new representation of record shapes:
1) Size of record instances is reduced.
2) Number of comparisons for a shape test reduced from 2 to 1
(shape test is used during type checks).
3) A few operations removed from Record.hashCode.
4) Type testing stubs (TTS) are now supported for record types with
named fields. Previously it was not possible to check shape of records
with named fields in TTS as TTS cannot access object pool and cannot
load field names array).

TEST=existing

Issue: https://github.com/dart-lang/sdk/issues/49719

Change-Id: I7cdcbb53938aba5d561cd24dc99530395dbbea7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276201
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-12-20 20:11:48 +00:00
Josh Soref 1aeece5e31 Spelling runtime vm compiler
Closes https://github.com/dart-lang/sdk/pull/50784

Co-authored-by: Josh Soref 2119212+jsoref@users.noreply.github.com

TEST=ci
GitOrigin-RevId: dbb20e7ee532a3490e86a633b5164ebab54bd38d
Change-Id: I9bd29543c4417efd4f338ff1c0e45c4ddbcc044d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276622
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Erik Ernst <eernst@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2022-12-20 17:01:52 +00:00
Ryan Macnak 8a37b70a53 [vm, compiler] More consistently support large load/store offsets.
TEST=ci
Change-Id: I7691808679c57c6f16f3859641cf52b9e606e304
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275381
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-12-14 22:17:35 +00:00
Alexander Aprelev e07321ec9d [vm] Factor StringEqual out of platform-specific intrinsifiers into single asm_intrinsifier.
TEST=ci
Change-Id: Iaab92c4a4c5a04355778c17ce564672c1e97814c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/272349
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2022-12-11 02:49:28 +00:00
Tess Strickland a43d3cb993 [vm/compiler] Use CombineHashes/FinalizeHash methods in intrinsifier.
AsmIntrinsifier::OneByteString_getHashCode now uses the CombineHashes
and FinalizeHash Assembler methods introduced in 49f998dc31.

To avoid having to check for 0 again after masking the result of
FinalizeHash to fit the desired bit size, we create a more general
FinalizeHashForSize that also takes the desired bit size of the result.

The intrinsified versions for ARM/ARM64/RISCV are also modified to
remove the special casing for the empty string, and instead just check
the size of the string against the loop counter at the start of the
loop, similarly to X64/IA32. This is less efficient for the empty
string case, but avoids the need to create a version of FinalizeHash
that elides the zero check.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-nnbd-linux-debug-simriscv64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-linux-debug-x64c-try
Change-Id: I53db731acb8bf35d16efb55fa2c5a79d8302399f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/272800
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2022-12-08 12:58:28 +00:00
Tess Strickland 49f998dc31 [vm/compiler] Handle hash caches in the InstantiateTypeArguments stubs.
Lower the threshold for converting from a linear to a hash-based cache
on most architectures from 500 to 10.

Due to register pressure, the InstantiateTypeArguments stub on IA32
continues to go to the runtime for hash caches, and so we do not
lower the threshold for converting to a hash-based cache there.

The following are benchmark results for those benchmark that use enough
Instantiations to trigger the use of hash-based caches. In the following
tables, "Results 1" denotes the benchmark results from only this change,
whereas "Results 2" include from comparing to the results prior to
4f925105cf, when only linear caches were used.

Dart AOT:

* InstantiateTypeArguments.Instantiate100

Arch   | CPU            | Results 1 | Results 2
-------|----------------|-----------------------
ARM    | Odroid-C2      |   382.8%  |   381.5%
ARM    | Raspberry Pi 4 |   486.7%  |   449.2%
ARM64  | Odroid-C2      |   328.1%  |   372.8%
ARM64  | Raspberry Pi 4 |  1283%    |  1281%
ARM64C | Raspberry Pi 4 |  2353%    |  2811%
X64    | Intel Xeon     |   568.7%  |   584.9%

* InstantiateTypeArguments.Instantiate1000

Arch   | CPU            | Results 1  | Results 2
-------|----------------|------------------------
ARM    | Odroid-C2      |   743.7%   |  3821%
ARM    | Raspberry Pi 4 |   486.7%   |  3218%
ARM64  | Odroid-C2      |   584.7%   |  3222%
ARM64  | Raspberry Pi 4 |   430.7%   |  8172%
ARM64C | Raspberry Pi 4 |   491.4%   | 16699%
X64    | Intel Xeon     |   954.1%   |  5528%


Dart JIT:

* InstantiateTypeArguments.Instantiate100

Arch   | CPU            | Results 1 | Results 2
-------|----------------|-----------------------
ARM    | Raspberry Pi 4 |   315.7%  |   295.1%
ARM64  | Raspberry Pi 4 |  1070%    |  1058%
ARM64C | Raspberry Pi 4 |  1769%    |  2095%
X64    | Intel Xeon     |   507.4%  |   496.2%

* InstantiateTypeArguments.Instantiate1000

Arch   | CPU            | Results 1 | Results 2
-------|----------------|-----------------------
ARM    | Raspberry Pi 4 |   565.2%  |  2550%
ARM64  | Raspberry Pi 4 |   406.8%  |  7375%
ARM64C | Raspberry Pi 4 |   379.6%  | 12996%
X64    | Intel Xeon     |   807.9%  |  4459%

During work on this change, an issue was found where cache lookups
in the stub on ARM64C always failed and went to runtime, even with
the old linear-only caches, hence the much larger performance gains
in those rows above.

TEST=vm/cc/TypeArguments_Cache_{Some,Many}Instantiations

Fixes: https://github.com/dart-lang/sdk/issues/48344
Change-Id: I3d29566ba0582502954c9fc59626ceb8fd40317a
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simriscv64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-nnbd-linux-debug-simriscv64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-mac-release-arm64-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/270702
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2022-12-01 10:15:58 +00:00
Tess Strickland 333d924504 [vm/compiler] Make InstantiateTypeArgs stubs architecture independent.
TEST=ci

Bug: https://github.com/dart-lang/sdk/issues/48344
Change-Id: I7c0cee827a09d981e6e65802304a361ace7ce501
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simriscv64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/270102
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-11-28 15:00:20 +00:00
Ryan Macnak 9182d5e535 [vm] Raise the limit on the number of classes.
Some large applications are bumping up against the 16-bit limit.

The object header changes from hash:32,cid:16,size:8,gc:8 to hash:32,cid:20,size:4,gc:8.

Because the CID field is no longer a power of two, it takes two instructions to load instead of one, increasing code size. The alternative change to cid:32,hash:24,gc:8 was considered and rejected because the reduction in hash bits very noticeably affects performance in applications with large data sets.

TEST=ci
Bug: b/255934984
Change-Id: I5fa36ba48a6852359d994393cf80c6a761c5d84c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/269120
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-11-17 20:35:47 +00:00
Ryan Macnak 7e4f448f78 [vm, gc] Don't use alignment bits when verifying eliminated write barriers.
TEST=ci
Change-Id: I6affde0b9db6ec951458905391670fe602dd3b61
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/262266
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-10-04 20:10:15 +00:00
Daco Harkes f70ca8ee11 [vm] Make RangeCheck reusable
Moves `RangeCheck` from the intrinsifier to the assembler.

Also, unifies the signature and semantics and adds documentation.

TEST=refactor

Change-Id: I274e5e46fde6a6e8baf30d8b732932f4f46d5855
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261681
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-09-29 15:07:27 +00:00
Alexander Markov d592882f49 [vm] Unify nullability and type state among all types
This change moves 'nullability' and 'type_state' from all kinds of
types to AbstractType base class. This removes a lot of code
duplication and allows uniform access to nullability and type state
for all kinds of types.

TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/47034

Change-Id: I1f0dc7fda78426db83fec6a20ebebcd632ad6d99
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/260662
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-27 17:30:03 +00:00
Vyacheslav Egorov 83ab5d5ca3 [vm] Remove dynamic field unboxing in JIT
Instead apply the same approach as we do in AOT: unbox based on the
static type information. There are no TFA results available in JIT,
but we could still unbox fields when running in sound null-safety.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Ide2e78c6659261ef8d245a4586cf699ea0fbb459
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256211
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-09-20 13:42:45 +00:00
Alexander Markov 3ec7cf9c34 [vm] Record literals
TEST=language/record_literal_test

Issue: https://github.com/dart-lang/sdk/issues/49719
Change-Id: I287586c0adb19fe401d76c7a586133a1fe9f1d1d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/257264
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-12 23:08:17 +00:00
Vyacheslav Egorov 4a4eedd860 [vm] Clean up ClassTable
* Merge ClassTable and SharedClassTable back together;
* Simplify handling of multiple arrays growing in sync;
* Refactor how reload deals with ClassTable.

The last change is the most important because it makes it
much easier to reason about the code. We move away from
copying bits and pieces of the class table and shared
class table into reload contexts.

Having two class table fields in the isolate group makes
it easier to reason about. One field contains program
class table (one modified by kernel loader and accessed
by various program structure cid lookups) and heap
walk class table (used by GC visitors). Normally these
two fields point to the same class table, but during
hot reload we temporary split them apart: original
class table is kept as a heap walk class table, while
program class table is replaced by a clone and updated
by reload.

If reload succeeds we drop original class table and
set program class table as heap walk one.

If reload fails we drop the program class table and
restore original one from heap walk table.

TEST=ci

Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-linux-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-rollback-linux-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-asan-linux-release-x64-try
Change-Id: I8b66259fcc474dea7dd2af063e4772df99be06c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/258361
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-09-10 15:12:35 +00:00
Ahmed Ashour 85700570f6 Fix typos
Fixes #49241

TEST=ci

Change-Id: I6117bf816fc8c4613cce66927f952fef75632725
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/248120
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
2022-06-15 11:08:28 +00:00
Alexander Markov af4da780be [vm] New async/async* implementation in JIT mode
The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.

Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:

 * Suspending variable-length frames (for unoptimized code).
 * Handling of Code and pool pointers in Dart stack frames.
 * OSR.
 * Deoptimization.
 * Hot reload.
 * Debugger.

The new implementation is not enabled in JIT mode yet.

Design doc: go/compact-async-await.
TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2022-06-02 23:39:45 +00:00
Daco Harkes 5380fa5d40 [vm/ffi] Introduce CCallInstr and replace handle instructions
The CCallInstr takes unboxed word-sized integers and does a call to C
without any VM transitions.

This can be used for the following.
1) Leaf runtime entries. This CL removes the EnterHandleScope,
   ExitHandleScope, and AllocateHandle instructions.
2) Compilation of isolate independent code where all definitions in IL
   are unboxed integers.

Currently, the CCallInstr expects all it's arguments and the return type
to be word-sized.

This CL also:
- Adds loadImmediate and CCall with a register for each assembler.
- Adds a CSE pass to force-optimized so that LoadThreadInstr gets de-
  duplicated.

TEST=tests/ffi(_2)/*

Change-Id: I67c6aaa1b7d6aa7d60e274477686a54e5a331216
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-simriscv64-try,vm-kernel-precomp-linux-debug-simriscv64-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246240
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-05-30 20:58:03 +00:00
Ryan Macnak 4ee35ee998 [vm, compiler] Load unboxed doubles directly from the literal pool.
TEST=ci
Change-Id: I4f881bc3e29059bce09eb0aa0ba99928e7fb10a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/244120
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-05-10 21:31:40 +00:00
Ryan Macnak 2cf4bf72a8 [vm, compiler] Handle large offsets with double spill slots on RISC-V.
Fix non-existent addressing mode in ARM fpu spilling.

TEST=ci
Change-Id: I0ecb973059a6cab11dce5b87137b54e0d1ac5ac5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243703
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2022-05-05 22:11:17 +00:00
Alexander Markov c9840a024f [vm] Introduce Assembler::PushRegistersInOrder to generate series of register pushes more efficiently
(Macro-)assembler helper PushRegistersInOrder generates series of
register pushes in an architecture-specific way:
 * Using STMDB on ARM (if possible).
 * Using STP on ARM64.
 * Updating SP only once on RISC-V.

TEST=ci

Change-Id: I6cbd1dd5f8acf8ce8087ed899a311264a873ad37
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2022-05-02 22:34:19 +00:00
Alexander Markov bf4bb95308 [vm] New async/await implementation in the VM, part 2 - vm
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.

Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.

Design doc: go/compact-async-await.

Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842

TEST=ci

Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2022-04-29 01:03:50 +00:00
Ryan Macnak 53c04ff910 [vm] More pieces of Windows ARM support.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/47824
Change-Id: Ie7388b12359b9ae11509a3f1b9e86c72507df38f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/240483
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-04-07 23:15:48 +00:00
Ryan Macnak 67f93d3840 [vm] Refactor compilation of runtime calls.
- Fixes SIMD registers clobbered by write barrier on ARM64.
 - Moves code generation out of non-compiler directory.
 - Removes unnecessary building of Dart frames on leaf runtime calls.
 - Removes unnecessary Threads slots for write barrier Code objects.
 - Removes duplicate saves of SP in leaf runtime calls on ARM64 and RISC-V.
 - Avoids some redundant SP updates on RISC-V.

TEST=ci
Change-Id: Idb92127658edc90b320923ef3d882a7219a450ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/236842
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-03-22 19:45:03 +00:00
Ryan Macnak 04ba20aa98 [vm] Support RISC-V.
Implements a backend targeting RV32GC and RV64GC, based on Linux standardizing around GC. The assembler is written to make it easy to disable usage of C, but because the sizes of some instruction sequences are compile-time constants, an additional build configuration would need to be defined to make use of it.

The assembler and disassembler cover every RV32/64GC instruction. The simulator covers all instructions except accessing CSRs and the floating point state accessible through such, include accrued exceptions and dynamic rounding mode.

Quirks:
  - RISC-V is a compare-and-branch architecture, but some existing "architecture-independent" parts of the Dart compiler assume a condition code architecture. To avoid rewriting these parts, we use a peephole in the assembler to map to compare-and-branch. See Assembler::BranchIf. Luckily nothing depended on taking multiple branches on the same condition code set.
  - There are no hardware overflow checks, so we must use Hacker's Delight style software checks. Often these are very cheap: if the sign of one operand is known, a single branch is needed.
  - The ranges of RISC-V branches and jumps are such that we use 3 levels of generation for forward branches, instead of the 2 levels of near and far branches used on ARM[64]. Nearly all code is handled by the first two levels with 20-bits of range, with enormous regex matchers triggering the third level that uses aupic+jalr to get 32-bits of range.
  - For PC-relative calls in AOT, we always generate auipc+jalr pairs with 32-bits of range, so we never generate trampolines.
  - Only a subset of registers are available in some compressed instructions, so we assign the most popular uses to these registers. In particular, THR, TMP[2], CODE and PP. This has the effect of assigning CODE and PP to volatile registers in the C calling convention, whereas they are assigned preserved registers on the other architectures. As on ARM64, PP is untagged; this is so short indices can be accessed with a compressed instruction.
  - There are no push or pop instructions, so combining pushes and pops is preferred so we can update SP once.
  - The C calling convention has a strongly aligned stack, but unlike on ARM64 we don't need to use an alternate stack pointer. The author ensured language was added to the RISC-V psABI making the OS responsible for realigning the stack pointer for signal handlers, allowing Dart to leave the stack pointer misaligned from the C calling convention's point of view until a foreign call.
  - We don't bother with the link register tracking done on ARM[64]. Instead we make use of an alternate link register to avoid inline spilling in the write barrier.

Unimplemented:
 - non-trivial FFI cases
 - Compressed pointers - No intention to implement.
 - Unboxed SIMD - We might make use of the V extension registers when the V extension is ratified.
 - BigInt intrinsics

TEST=existing tests for IL level, new tests for assembler/disassembler/simulator
Bug: https://github.com/dart-lang/sdk/issues/38587
Bug: https://github.com/dart-lang/sdk/issues/48164
Change-Id: I991d1df4be5bf55efec5371b767b332d37dfa3e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/217289
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2022-01-20 00:57:57 +00:00
Alexander Aprelev cb64a64241 [vm/isolate_exit] Ensure ffi code exits if isolate is exiting.
When ffi code calls Dart code that requests the isolate to exit(via Isolate.exit, for example), we ensure that isolate indeed exits on return from ffi call.
This is implemented by introduction of new safepoint bit which, once set, forces ExitSafepoint to propagate unwind error.

TEST=isolate_exit_sandwich_test

Change-Id: I2e8f5ecec7f4e59ae5f99b9525cc566f20d4b6a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/219846
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2022-01-08 06:01:08 +00:00
Tess Strickland 37d45743e1 [vm] Avoid calling runtime when lazily initializing late static fields.
This CL changes the handling of late static fields to be similar to the
handling of late instance fields. That is, if the field does not need a
load guard and has an initializer, the new InitLateStaticField and
InitLateFinalStaticField stubs are used, which directly call the
initializer function instead of going to the runtime. Thus, the only
runtime call left in these cases is when a late final static field is
modified by the initializer function, in which case an appropiate
exception is thrown.

In the case where a late static field might call the initializer
function but no initializer exists, a LateInitializationErrorSlowPath is
generated to be called if the field value is the sentinel, also similar
to the late instance field case.

TEST=Refactoring, so existing tests.

Change-Id: I95887022d8ece1164a8f7c34b77d9de8c53aa898
Bug: https://github.com/dart-lang/sdk/issues/45138
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-ia32-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/220009
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-12-01 08:51:49 +00:00
Tess Strickland bb1640eec2 Reland "[vm/compiler] Add more checks in TTSes for uninstantiated types."
This is a reland of 1016bbadde

The broken benchmarks causing the revert were not due to this CL.

TEST=vm/cc/TTS

Original change's description:
> [vm/compiler] Add more checks in TTSes for uninstantiated types.
>
> In certain cases, optimized type testing stubs for uninstantiated types
> compare the instantiation of the type arguments for the type against the
> corresponding instance type arguments. Previously, only an identity
> check was generated, so the instantiation had to match the instance type
> argument exactly.
>
> This CL adds checks for the following cases:
> * The instantiated type argument is dynamic, void, or Object.
> * The instance type argument is Null or Never.
>
> When strong null safety is enabled, we also check that the the instance
> type argument is legacy or non-nullable when the instantiated type
> argument is non-nullable Object or that the instantiated type argument
> is nullable or legacy when the instance type argument is Null.
>
> This CL also adds handling for the case where the instantiated or
> instance type argument is not a Type, which is necessary for safely
> retrieving the type class id and nullability, and allocates some
> additional registers in TTSInternalRegs for storing type arguments. On
> ARM7, this requires pushes/pops around type argument checking, because
> we've run out of registers there.
>
> Fixes https://github.com/dart-lang/sdk/issues/40736
>
> TEST=vm/cc/TTS
>
> Bug: https://github.com/dart-lang/sdk/issues/46920
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
> Change-Id: Ib8498fd2b9593b4abb92111f062ed2fc95a45c16
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210681
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>

Bug: https://github.com/dart-lang/sdk/issues/46920
Change-Id: Id84730b3bb292ea7fdfe9ebd8b1fc6e71c64cf22
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211901
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-09-27 10:57:18 +00:00
Daco Harkes 32f6f94ab5 [vm] Add store release barrier option to StoreInstanceInstr
In dependent CL, const maps and sets can be shared between isolates in
the same isolate group before the index is lazily populated. To prevent
one isolate from seeing a non-null index pointer, while not all writes
to the index itself are visible to the isolate (the mutator thread of
that isolate), we use a store release barrier on arm architectures.

This CL adds a store release barrier option to the StoreInstanceField
instruction to be used in the dependent CL.

Even though there is no barrier on x64 and ia32, we still also use the
option on these architectures because on x64 we signal the barrier to
TSAN.

The stlr instruction on arm64 does not feature an address operand.
Therefore we need to branch in the assembler on whether we're using
a store-release before we construct an Address.

Design doc: go/dart-vm-const-maps

TEST=Dependent CL, split off for ease of reviewing.

Bug: https://github.com/dart-lang/sdk/issues/45908

Change-Id: I6563aa3c813319a23e815b862ca5caabb805dc9e
Cq-Include-Trybots: luci.dart.try:analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,dart-sdk-linux-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-tsan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210726
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-09-06 06:28:59 +00:00
Alexander Markov 307bc3ef2c [vm] Faster double.toInt() in AOT mode
double.toInt() micro-benchmark on AOT/x64:
Before: BenchToInt(RunTime): 438.67258771929824 us.
After:  BenchToInt(RunTime): 118.8603434955726 us.

double.floor() micro-benchmark on AOT/x64:
Before: BenchFloor(RunTime): 537.2132688691916 us.
After:  BenchFloor(RunTime): 321.2052352657781 us.

TEST=ci
Issue https://github.com/dart-lang/sdk/issues/46876
Issue https://github.com/dart-lang/sdk/issues/46650

Change-Id: Id37c827bceb7f374ae5b91b36871ccf0d9e92441
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211620
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-09-01 16:57:59 +00:00
Tess Strickland a70ab07a47 Revert "[vm/compiler] Add more checks in TTSes for uninstantiated types."
This reverts commit 1016bbadde.

Reason for revert: Broken benchmarks on golem.

Original change's description:
> [vm/compiler] Add more checks in TTSes for uninstantiated types.
>
> In certain cases, optimized type testing stubs for uninstantiated types
> compare the instantiation of the type arguments for the type against the
> corresponding instance type arguments. Previously, only an identity
> check was generated, so the instantiation had to match the instance type
> argument exactly.
>
> This CL adds checks for the following cases:
> * The instantiated type argument is dynamic, void, or Object.
> * The instance type argument is Null or Never.
>
> When strong null safety is enabled, we also check that the the instance
> type argument is legacy or non-nullable when the instantiated type
> argument is non-nullable Object or that the instantiated type argument
> is nullable or legacy when the instance type argument is Null.
>
> This CL also adds handling for the case where the instantiated or
> instance type argument is not a Type, which is necessary for safely
> retrieving the type class id and nullability, and allocates some
> additional registers in TTSInternalRegs for storing type arguments. On
> ARM7, this requires pushes/pops around type argument checking, because
> we've run out of registers there.
>
> Fixes https://github.com/dart-lang/sdk/issues/40736
>
> TEST=vm/cc/TTS
>
> Bug: https://github.com/dart-lang/sdk/issues/46920
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
> Change-Id: Ib8498fd2b9593b4abb92111f062ed2fc95a45c16
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210681
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>

TBR=kustermann@google.com,alexmarkov@google.com,sstrickl@google.com

Change-Id: I8c2368cbbb3d23aa2d0cb4788737d9b737318bb7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/46920
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211860
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-08-30 15:51:40 +00:00
Tess Strickland 1016bbadde [vm/compiler] Add more checks in TTSes for uninstantiated types.
In certain cases, optimized type testing stubs for uninstantiated types
compare the instantiation of the type arguments for the type against the
corresponding instance type arguments. Previously, only an identity
check was generated, so the instantiation had to match the instance type
argument exactly.

This CL adds checks for the following cases:
* The instantiated type argument is dynamic, void, or Object.
* The instance type argument is Null or Never.

When strong null safety is enabled, we also check that the the instance
type argument is legacy or non-nullable when the instantiated type
argument is non-nullable Object or that the instantiated type argument
is nullable or legacy when the instance type argument is Null.

This CL also adds handling for the case where the instantiated or
instance type argument is not a Type, which is necessary for safely
retrieving the type class id and nullability, and allocates some
additional registers in TTSInternalRegs for storing type arguments. On
ARM7, this requires pushes/pops around type argument checking, because
we've run out of registers there.

Fixes https://github.com/dart-lang/sdk/issues/40736

TEST=vm/cc/TTS

Bug: https://github.com/dart-lang/sdk/issues/46920
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
Change-Id: Ib8498fd2b9593b4abb92111f062ed2fc95a45c16
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210681
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-08-30 14:01:15 +00:00
Alexander Markov 25d0ae2304 [vm/compiler] Support UnboxedDouble representation in unoptimized code
In the unoptimized code unboxed double inputs are now unboxed in the
instruction prologue, and unboxed double output is boxed in the
instruction epilogue. Double values are still always boxed on the
expression stack in the unoptimized mode.

Support for unboxed doubles in unoptimized code allows us to have
one implementation of double and dart:math built-in functions,
shared across optimized and unoptimized modes.

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/46650
Change-Id: Ic1e42b72ee80cb2c12019b7abab8111240b05efd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211302
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-08-26 23:21:56 +00:00
Tess Strickland f709bf12f0 Reland "[platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint."
datastream.h is included in places where we cannot import the compiler
namespace, so we can't make compiler::target::word the type of
WriteTargetWord. That also would possibly silently truncate a
host-word sized value that was passed in instead of alerting to a
possible issue if those bits were important.

However, just using IsInt fails if the value being passed in is
a compiler::target::uword that is larger than the max value
that fits in a compiler::target::word. Instead, check for
truncation by checking the bit length of the value, which works
for both signed and unsigned target word values.

Original description:

Also remove previous (incorrect) uses in datastream.cc and
image_snapshot.cc, now that Utils::IsInt<T>(N, value) can now
be run for values of N >= sizeof(T).

Fixes https://github.com/dart-lang/sdk/issues/46572

TEST=language/generic/super_bounded_types_test passes on NNBD simarm,
     added value that triggered failure to vm/cc/MagnitudeIsUint

Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-precomp-linux-debug-simarm_x64-try
Change-Id: Idbfdda9f28243d206d482a1d9ac7ae76a5022ad2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206201
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-07-08 14:22:54 +00:00
Liam Appelbe ecb6354a79 Revert "[platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint."
This reverts commit 2ea5507fc3.

Reason for revert: Broke vm-kernel-precomp-linux-debug-simarm_x64

Original change's description:
> [platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint.
>
> Also remove previous (incorrect) uses in datastream.cc and
> image_snapshot.cc, now that Utils::IsInt<T>(N, value) can now
> be run for values of N >= sizeof(T).
>
> Fixes https://github.com/dart-lang/sdk/issues/46572
>
> TEST=language/generic/super_bounded_types_test passes on NNBD simarm,
>      added value that triggered failure to vm/cc/MagnitudeIsUint
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try
> Change-Id: Ibc2d2bb5037a8fdee11fc26fa2a313149d3ca274
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206083
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Daco Harkes <dacoharkes@google.com>

TBR=vegorov@google.com,dacoharkes@google.com,sstrickl@google.com

Change-Id: I5194ba777d8811bb5557338c6b071ea152b93d44
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206143
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2021-07-07 20:14:55 +00:00
Tess Strickland 2ea5507fc3 [platform] Fix Utils::IsAbsoluteUint and rename to MagnitudeIsUint.
Also remove previous (incorrect) uses in datastream.cc and
image_snapshot.cc, now that Utils::IsInt<T>(N, value) can now
be run for values of N >= sizeof(T).

Fixes https://github.com/dart-lang/sdk/issues/46572

TEST=language/generic/super_bounded_types_test passes on NNBD simarm,
     added value that triggered failure to vm/cc/MagnitudeIsUint

Cq-Include-Trybots: luci.dart.try:vm-kernel-nnbd-linux-release-simarm-try
Change-Id: Ibc2d2bb5037a8fdee11fc26fa2a313149d3ca274
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/206083
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-07-07 12:39:37 +00:00
Liam Appelbe 155ac440f7 [vm] Migrate Arrays and Instances to compressed pointers
Change-Id: I8264b340ee91d883afc52a52e83cda504f552e37
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191302
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-06-24 17:31:12 +00:00
Tess Strickland b8ecf06943 [vm/compiler] Add kUnboxedUint16 representation.
Adding this representation allows us to create slots for storing
and retrieving values from uint16_t-typed object fields in the runtime.

Also generalize BoxUint8Instr to BoxSmallIntInstr, which holds any
unboxed integer representation where the value is guaranteed to fit
into a Smi.

To unify the implementation of BoxSmallIntInstr across architectures,
we add ExtendValue(Register dst, Register src, OperandSize sz), which
extends the sz-sized value in src to a full register size in dst. If
sz is a signed OperandSize, then the resulting value is sign
extended appropriately.

We also add MoveAndSmiTagRegister / ExtendAndSmiTagValue, which is like
the base operation but also tags the value as a Smi. On some
architectures, this can be performed using a single instruction.

Also adds sbfx/ubfx on ARM for use in ExtendValue.

TEST=Existing tests for BoxSmallIntInstr, followup CL for the new
     representation.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-simarm64c-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Change-Id: I9c0b3a23f5e28480e0bfbde55b9494f7b822dcfa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203761
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-06-24 12:15:48 +00:00
Tess Strickland faa1c9ff98 [vm] Move context setting into AllocateClosure stub.
With this change, allocation instructions can now be direct inputs to
other allocation instructions, so the redundancy eliminator is extended
to handle this possibility.

Methods for working with instruction input-related slots are added to
subclasses of AllocationInstr, so that the redundancy eliminator can be
written more generically in places, instead of needing to add cases for
new allocation instructions and/or instruction inputs.

Code size different in Flutter gallery (release-sizeopt):
* ARM7: Total -0.30%, instructions -0.41%
* ARM8: Total -0.31%, instructions -0.47%

TEST=Current test suite.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-x64-try,vm-kernel-linux-debug-x64c-try
Change-Id: Idc1aa2a1cb8c0c62f0bcb64aee89a7525dd3d1e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/198406
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2021-06-02 18:10:38 +00:00