When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Previously, there were two ways to determine whether a deserialization
cluster was being deserialized for the root loading unit or a non-root
loading unit:
* some deserialization clusters (those deriving from
CanonicalSetDeserializationCluster) took a boolean at construction
time, and
* all deserialization clusters took an additional boolean argument to
ReadFill and PostLoad on whether the loading unit was primary (i.e.,
the root) or not.
Since only clusters that deal with canonical values need to worry about
whether the deserializing loading unit is the root or not, standardize
on the first, where clusters that need to know take the boolean in their
constructor and store it in a field, and remove the boolean argument to
ReadFill and PostLoad.
Have DeserializationCluster::PostLoad just check is_canonical() to
determine whether to throw an error if not overridden in the base class,
so that any deserialization cluster that may operate on canonical
objects needs to explicitly make a choice about whether they need to
recanonicalize or not. This check only happens in the precompiled
runtime, as otherwise there are no non-root loading units.
In particular, this fixes an issue where symbols (canonical strings)
in non-root loading units weren't marked as canonical, even though
they do not need recanonicalization (since symbol tables are
per-isolate group, and thus per-loading unit).
TEST=ci (refactoring, test of symbol fix coming in followup CL)
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-release-x64-try,vm-aot-linux-debug-x64-try,vm-aot-mac-release-arm64-try,vm-aot-mac-product-arm64-try
Change-Id: I411a254767653da82631abc1fcc687ae4259e0b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/363321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Non-variadic arguments on the stack on MacOS Arm64 are aligned to
the value size (which can be smaller than word size). However, for
varargs, the arguments on the stack seem to be aligned to the word
size.
This CL introduces an alignment strategy constant for primitives on
the stack in varargs and uses it in the native calling convention
calculation.
TEST=runtime/vm/compiler/ffi/native_calling_convention_test.cc with
runtime/vm/compiler/ffi/unit_tests/variadic_less_than_word/arm64_macos.expect
TEST=tests/ffi/function_varargs_generated_native_leaf_test.dart
Closes: https://github.com/dart-lang/sdk/issues/55471
Change-Id: I51d20c3933a2cea9b110954ddec92fb91b9c3ecd
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362762
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This is a reland of commit a7e2dce0cc
Fixes the crash during code generation when typed data view
LoadIndexed was used on an internal typed data array
(in the unreachable code path) without loading its payload.
TEST=runtime/tests/vm/dart/regress_b_334128316_test.dart
TEST=manually verified repro from b/334128316
Original change's description:
> [vm] Implement typed data view GetIndexed in flow graph builder
>
> Typed data view 'operator []' is now implemented in the flow
> graph builder. This unifies all typed data GetIndexed operations
> and removes unnecessary address calculations.
>
> TEST=ci
>
> Change-Id: I8d2ca6e7c7bf18b2590536a643f92cad5beb6d95
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361283
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
Change-Id: I3786fa17e971f8ba6cd01a57f1a68504bf24bc47
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362821
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Instead of having special support for targeting catch entry as a
lazy deoptimization target, split this into two independent
operations:
* Perform lazy deoptimization at call.
* Rethrow the exception once deoptimization is complete.
TEST=ci
Change-Id: Ib275d615bc6717e1184e88602ae5459d5b771f2b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361580
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Typed data view 'operator []' is now implemented in the flow
graph builder. This unifies all typed data GetIndexed operations
and removes unnecessary address calculations.
TEST=ci
Change-Id: I8d2ca6e7c7bf18b2590536a643f92cad5beb6d95
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361283
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Canonical type erasure was used when running without sound null
safety. VM no longer supports unsound mode, so canonical type erasure
can be dropped.
This is a preparation to switching platform kernel file from
nnbd-agnostic mode to null-safe.
Also
* pkg/vm/test/modular_kernel_plus_aot_test.dart is migrated to
sound mode;
* test cases for unsound mode are removed from
pkg/dartdev/test/commands/compile_test.dart.
TEST=ci
Change-Id: I2ed0b84cffd650bbaecf4ee99f23da2fdc09de71
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362162
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
gen_snapshot can now consume a concatenated dill file.
Restrictions:
* The first dill file must be the main program.
* The subsequent dill files must only contain dummy libraries with
objects which are recognized classes in the VM and not treeshaken
in the main program.
This will enable compiling Dart AOT first, then using the tree-shaking
information to tree-shake native assets, and then embedding the native
assets mapping in the kernel file before running gen_snapshot.
TEST=tests/ffi/native_assets/asset_relative_test.dart
Closes: https://github.com/dart-lang/sdk/issues/50152
Bug: https://github.com/dart-lang/sdk/issues/55377
Change-Id: Id562aa39840d5eb467198efaa4cf3152d860f8b5
q-Include-Trybots: dart/try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362381
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This flag only makes sense without sound null safety. VM no longer
supports unsound mode, so it can be removed.
In sound mode this flag is always true.
TEST=ci
Change-Id: I4a5fa60d9b351b6281efab9462ef2dd2c5565a7c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362182
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Vm flag --null-assertions only makes sense without sound null safety.
Now that VM no longer supports unsound mode, the flag is obsolete and
can be removed.
TEST=ci
Change-Id: I69eb2012ac483f082fd43ecc9743076e0e1928c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362181
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The MemoryCopyInstr on x64 in MSAN mode when forward copying was
unpoisoning at the wrong pointer. The `rep mov` changes the
destination register.
Now we unpoison _before_ the `rep mov` if it's a forward copy, and
_after_ the `rep mov` for backwards copies. This way the destination
register contains the start of the memory in both cases.
TEST=tests/ffi/msan_test.dart
b/266213262
Change-Id: I37688802e63797f650a8242b6ba8b884813ebbd0
Cq-Include-Trybots: luci.dart.try:vm-msan-linux-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-linux-debug-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361420
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Product builds of gen_snapshot are used by dart cli tools. So if we are to allow --enable-asserts for 'dart compile' commands, we need to allow this flag in product mode.
BUG=dartbug.com/53343
TEST=ci
Change-Id: I6252b6ee42ed45a25df2046f4a6642c880e8ee14
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361122
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
This CL removes support for --no-sound-null-safety option, the VM will
only run sound null safety code.
First part of deleting support for legacy and unsound null safe code.
TEST=CI
Change-Id: I9dbed10a18ef102533d6af516ff5075297d17b74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357183
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
CodeSerializationCluster::Trace uses local object pools to discover
object pool entries used by the Code object, even when code object
uses GOP.
So UnlinkedCall objects should be de-duplicated in the local
object pools to avoid stale references regardless of
--write_v8_snapshot_profile_to and --trace_precompiler_to flags.
TEST=User tested on their app.
Fixes https://github.com/dart-lang/sdk/issues/55343
Change-Id: I781dc6dc75f2fca25c1c107e224dedbaeadbc53a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361240
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Now Slots have an extra flag that describes whether or not the
instance object for that Slot is tagged (Dart objects) or untagged
(non-Dart objects like Thread).
Add new Slot categories for Slots with untagged instances, based on
the return value of the Slot:
* a tagged value (e.g., the finalizers_ field of Isolate),
* an untagged value (e.g., the isolate_ field of Thread), or
* an unboxed value (e.g., the enabled_ field of StreamInfo).
The primary difference in the specification of slots for untagged
instances vs tagged instances is that the slots for untagged instances
do not include an underlying type (as only Dart objects have an
Untagged<X> for each <X> class that specifies the actual object layout).
Also do a little more refactoring of Slot categories to simplify
some switch statements that now only need to be updated if the
categories the switch statement deals with change. For example,
methods for range analysis are only interested in Slots that are
guaranteed to contain integers, so now those methods are written in
a way that they only need to be updated if a new tagged integer
Slot declaration is added.
Adjust LoadFromSlot, StoreToSlot, and StoreToSlotNoBarrier to
appropriately handle Slots on untagged instances, and change
LoadField and StoreField's RequiredInputRepresentation methods
to appropriately returned kTagged or kUntagged based on the slot.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-release-ia32-try
Change-Id: Ic27de4f092c9e087486066459f68a82b4e1685b0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357341
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Specialized compiler pipeline for force-optimized functions is not
needed (force-optimized functions can be inlined into the arbitrary
functions and optimized using ordinary pipeline anyway) and error-prone
(certain optimizations could be missing by mistake).
This change unifies the pipelines by using ordinary pipeline for
force-optimized functions.
TEST=ci
Change-Id: Ia315abded61954e06acfc209d31ccce45c286437
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360960
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Add OperandSize arguments to the non-compressed Store methods so that
the compressed Store methods can be implemented as calls to the
corresponding non-compressed method using kObjectBytes. This makes it
so that all compressed Store methods are now non-virtual.
Add a scratch register argument to all Store methods that involve a
write barrier that defaults to TMP. Originally only the IA32
implementation took a scratch register, with the others using TMP or
TMP2 internally.
Since all architectures can represent a register/offset pair as an
Address without losing information, make all the offset-based load
and store methods have an implementation that creates an Address or
FieldAddress and calls the corresponding address-based store method.
This makes all the offset-based load and store methods non-virtual.
After this, most of the Store methods are non-virtual, with
only the base methods used to implement them as virtual:
* Store
* StoreRelease
* StoreObjectIntoObjectNoBarrier
* StoreBarrier
* ArrayStoreBarrier
* VerifyStoreNeedsNoWriteBarrier (for DEBUG use only)
The Load methods can't be consolidated quite so much, as the
base methods for handling non-Smi compressed pointers must be defined
per-architecture, as each compressed pointer architecture handles
adding the upper bits for the heap separately.
Have LoadFieldInstr::EmitNativeCode use Assembler::LoadToSlot when the
result location is a single integer register and
StoreFieldInstr::EmitNativeCode use Assembler::StoreToSlot and
Assembler::StoreToSlotNoBarrier when the value location is a single
integer register to avoid code duplication between those methods.
Handle both compressed and uncompressed Smi fields in LoadFromSlot.
TEST=ci (refactoring)
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-x64-try,vm-win-release-ia32-try
Change-Id: I60cc03776af220ed87918664bb4b9abafff2788a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360641
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Due to a missing return, two stores would be done for a non-tagged field
in StoreToSlot: the correct store without a write barrier and then a
second store that also applies the write barriers inappropriately.
Rewrite LoadFromSlot/StoreToSlot/StoreToSlotNoBarrier to just use
chained ifs instead of returning in some if statement bodies and not
in others.
TEST=ci
Change-Id: Ia0a204217433056856e4dc355758f34beceb79b6
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360862
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This is a reland of commit d24b5d1f5e
On top of the original change, the following is fixed and improved:
1) Flow graph builder (FGB) body is now also applied to
dynamic invocation forwarders (similarly to graph
intrinsics).
2) Frame can be omitted for functions which have a
call on a shared slow path. This is needed to make
FGB implementation of GetIndexed frameless, as it has
GenericCheckBound which calls on shared slow path.
(Graph intrinsics are frameless).
3) Range analysis is enabled for force-optimized functions,
so more efficient code can be generated for boxing
instructions. Range analysis is fixed to avoid crashes
and correctly intersect ranges with constant boundaries
(needed for some force-optimized FFI functions).
4) EliminateStackOverflowChecks pass is enabled for
force-optimized functions so CheckStackOverflow can be
eliminated.
Original change's description:
> [vm] Replace array GetIndexed graph intrinsics with flow graph builder implementation
>
> _Array, _GrowableList, internal and external typed data 'operator []'
> are now implemented in the flow graph builder.
>
> Unlike graph intrinsics, flow graph created in the flow graph builder
> can be used by the inliner. Corresponding graph intrinsics and native
> methods are removed.
>
> Also, this change adds missing external typed data indexing operations.
>
> TEST=ci
>
> Change-Id: Ic19784481feadf54c096a587413e67b4e18353dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359940
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
TEST=ci
Change-Id: I04ef008a04238d432683d7543cd047e35bad17c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360560
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Remove CompareWithCompressedFieldFromOffset, which has no uses.
Rename the LoadFromOffset and StoreFromOffset methods that took
Addresses to Load and Store, respectively. This makes the names
of the Assembler methods more uniform:
* Takes an address: Load, Store, LoadField, LoadCompressedField,
StoreIntoObject, StoreCompressedIntoObject, LoadSmi,
LoadCompressedSmi, etc.
* Takes a base register and an offset: LoadFromOffset, StoreToOffset,
LoadFieldFromOffset, LoadCompressedFieldFromOffset,
StoreIntoObjectOffset, StoreCompressedIntoObjectOffset,
LoadSmiFromOffset, LoadCompressedSmiFromOffset, etc.
Create AssemblerBase methods for loading and storing compressed
pointers that weren't already there, as well as the corresponding
methods for loading and storing uncompressed values.
Make non-virtual methods that load and store uncompressed fields
that call the corresponding method for loading from and storing to
memory regions, adjusting the address or offset accordingly. This
avoids needing per-architecture overrides for these.
Make non-virtual methods that load compressed fields, calling the
corresponding method for loading a compressed value from a memory
region. (Since compressed pointers are only stored in Dart objects,
and stores into a Dart object may require a barrier, there is no
method for storing a compressed value into an arbitrary memory region.)
Create pure virtual methods for loading from or storing to an Address
or any method that does not have both an Address-taking and a
base register and offset pair-taking version (e.g., LoadAcquire).
Create methods for loading from or storing to a base register
and an offset. The base implementation takes the base register and
offset and creates an Address from it, then calls the Address-taking
equivalent. These methods are non-virtual when the implementation is
the same on all architectures and virtual to allow overriding when
necessary.
Make a non-virtual method for loading uncompressed Smis, since all
architectures have the same code for this, including the DEBUG check.
If compressed pointers are not being used, all the methods for
compressed pointers are non-virtual methods that call the
corresponding method for uncompressed values.
If compressed pointers are being used:
* Install pure virtual methods for loading compressed values from
and storing compressed values to an Address or any method that does
not have both an Address-taking and a base register and offset
pair-taking version (e.g., LoadAcquireCompressed).
* Install virtual methods for loading compressed values from and
storing compressed values to a base register and offset. Like the
uncompressed implementation, the base implementation of these
create an Address and call the Address-taking equivalent, and these
implementations are overridden on ARM64.
* Install a non-virtual method for loading compressed Smis, since the
only difference is that it loads a zero-extended 32-bit value, which
AssemblerBase can do.
TEST=ci (refactoring only)
Change-Id: I934791d26a6e2cdaa6ac5f188b0fd89dbdc491d1
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359861
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This reverts commit d24b5d1f5e.
Reason for revert: performance regression
(Graph intrinsics are applied to dynamic invocation forwarders,
but flow graph builder implementation isn't.)
Original change's description:
> [vm] Replace array GetIndexed graph intrinsics with flow graph builder implementation
>
> _Array, _GrowableList, internal and external typed data 'operator []'
> are now implemented in the flow graph builder.
>
> Unlike graph intrinsics, flow graph created in the flow graph builder
> can be used by the inliner. Corresponding graph intrinsics and native
> methods are removed.
>
> Also, this change adds missing external typed data indexing operations.
>
> TEST=ci
>
> Change-Id: Ic19784481feadf54c096a587413e67b4e18353dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359940
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
Change-Id: I2b31b06edaa4b8a09d256f25f923d4489e28518b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/360305
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
_Array, _GrowableList, internal and external typed data 'operator []'
are now implemented in the flow graph builder.
Unlike graph intrinsics, flow graph created in the flow graph builder
can be used by the inliner. Corresponding graph intrinsics and native
methods are removed.
Also, this change adds missing external typed data indexing operations.
TEST=ci
Change-Id: Ic19784481feadf54c096a587413e67b4e18353dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359940
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL
* adds a `kind` to `CheckWritableInstr`,
* starts using `CheckWritableInstr` in JIT mode, and
* adds support for `CheckWritableInstr` ia32.
TEST=tests/ffi/vmspecific_native_finalizer_deeply_immutable_test.dart
Closes: https://github.com/dart-lang/sdk/issues/55067
Change-Id: I0b397daba12cfc8b885401169889f7cd7c040166
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359223
Auto-Submit: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Note this changes the error in AOT to tell what is the typed data that
is not writable.
This CL eases the follow up CL which starts reusing the
`CheckWritableInstr` and accompanying slow path and runtime entry for
other messages.
TEST=tests/lib/typed_data/polymorphic_unmodifiable_typed_data_test.dart
Bug: https://github.com/dart-lang/sdk/issues/55067
Change-Id: Icf26d119501059e2ea4e885897539ebcb1063da0
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-simarm-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359063
Reviewed-by: Ilya Yanok <yanok@google.com>
This CL changes compounds (structs, unions, and arrays) to be backed
by a TypedDataBase and an int offset.
Before this CL, the compounds where only backed by a TypedDataBase.
This leads to the following issues:
1. Access to nested structs required code for allocating new typed data
views or pointers, which the optimizer then had to prevent from
being allocated after inlining.
2. Runtime branching on whether the TypedDataBase was a Pointer or
TypedData increased code size and prevented inlining.
https://github.com/dart-lang/sdk/issues/54892
This could not be properly optimized if in AOT both typed
data and pointer were flowing into the same compound.
3. Constructing TypedData views required calculating the length of the
view.
After this CL, accessing nested compounds will lead to accesses on the
original TypedDataBase with an extra offset.
This removes the polymorphism on TypedData vs Pointer, because the
final int/float/Pointer accesses in nested compounds operate on
TypedDataBase.
Also, it simplifies creating an `offsetBy` accessor, because it will
no longer have to be polymorphic in typed data vs pointer, nor will it
have to calculate the length of the field.
Implementation details:
* The changes in the CFE and patch files are straightforward.
* VM: Struct-by-value returns (or callback params) are initialized
with an offsetInBytes of 0.
* VM: Struct-by-value arguments (and callback return) need to read out
the offsetInBytes. Before this CL we were passing in the TypedData
as tagged value. With this CL we are passing the TypedData as tagged
value and the offset as unboxed int, from 1 IL input to 2 IL
inputs. (The alternative would have been to take the compound as
a tagged value, but that would have prevented optimizations from not
allocating the compound object in the optimizer.
The FfiCallInstr is updated to have two definitions for the case
where we were passing in the TypedData previously.
The NativeReturnInstr is refactored to be able to take two inputs
instead of 1. (Note that we don't have VariadicInstr only
VariadicDefinition in the code base. So the instruction is _not_
implemented as variadic, rather as having a fixed length of 2.)
* dart2wasm does no longer support nested compounds due to the
compound implementation only storing a pointer address.
https://github.com/dart-lang/sdk/issues/55083
Intending to land this after
https://dart-review.googlesource.com/c/sdk/+/353101.
TEST=test/ffi
CoreLibraryReviewExempt: VM and WASM-only implementation change.
Closes: https://github.com/dart-lang/sdk/issues/54892
Bug: https://github.com/dart-lang/sdk/issues/44589
Change-Id: I8749e21094bf8fa2d5ff1e48b6b002c375232eb5
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/354226
Reviewed-by: Tess Strickland <sstrickl@google.com>
This reverts commit eebc219bab.
Reason for revert: performance regressions
Original change's description:
> [vm/compiler] Move handling of recognized methods from inliner to call specializer
>
> This refactoring reduces number of places where recognized methods are
> handled and cleans up general-purpose inliner pass.
>
> The code is mostly moved as is to simplify reviewing, with a notable
> exception: in a few places handling of recognized method calls is
> removed as it duplicates handling performed during call specialization
> passes (ApplyClassIds and ApplyICData) which run both before and after
> inlining.
>
> TEST=ci
>
> Change-Id: I8d53d23587beb09e6edf64ee95524f6ad3e060de
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358221
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
Change-Id: I9576bc4b385b5d0e0b728556defedb4962ec0a8a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359842
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Renames `ReturnInstr` to `DartReturnInstr`, and introduces a new
`ReturnBaseInstr` to be the common parent of `DartReturnInstr` and
`NativeReturnInstr`. (Before this CL, `NativeReturnInstr` was a
subtype of `ReturnInstr`.)
In a follow up CL, the `NativeReturnInstr` will get up to two inputs.
https://dart-review.googlesource.com/c/sdk/+/354226
Therefore, the `ReturnBaseInstr` does not inherit from `TemplateInstr`
with 1 input, but instead only inherits from `Instruction`.
TEST=SDK build
TEST=*_il_test.dart
Change-Id: I017eb7802ae6c902b64f1cda20edf4a11408dbe1
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358904
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This refactoring reduces number of places where recognized methods are
handled and cleans up general-purpose inliner pass.
The code is mostly moved as is to simplify reviewing, with a notable
exception: in a few places handling of recognized method calls is
removed as it duplicates handling performed during call specialization
passes (ApplyClassIds and ApplyICData) which run both before and after
inlining.
TEST=ci
Change-Id: I8d53d23587beb09e6edf64ee95524f6ad3e060de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358221
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change aligns all typed data implementations with Int8 typed data
and allows compiler to recognized and replace typed data access
operations earlier, before method inlining. Consequently, this increases
chances for [] and []= to be inlined. In JIT, this also prevents
non-inlined calls to _getIndexed/_setIndexed as [] and []= are graph
intrinsics which bypass calls to _getIndexed/_setIndexed.
TEST=ci
Change-Id: I95a85963da07fd1c04c06e0af09f76eb15f93353
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359520
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The new implementation of String.codeUnitAt is always inlined and
doesn't depend on the polymorphic inlining of recognized methods.
String.codeUnitAt now has a custom body in the flow graph builder
which performs non-speculative bounds check and then branches between
OneByteString and TwoByteString. Corresponding graph intrinsic and
native method are removed.
This change also fixes passing of unboxed arguments to runtime
in the slow path of GenericCheckBound instruction in JIT mode
(when shared stubs are not used).
TEST=ci
Change-Id: Iab2805fc752df84c37089165f828e31aca5f043f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/359000
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Daco Harkes