This reverts commit 80102c981b.
Reason for revert: b/186359854
Original change's description:
> [vm] Switch datastream Write/WriteUnsigned to (S)LEB128.
>
> This reduces the number of variable-length integer encodings in our
> database from 2 to 1, and chooses the more standard one.
>
> TEST=Existing tests, in particular any that involve snapshots.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
> Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Idf32bdd879cf8bb7407f6dae764312140ad6eeb2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196920
Reviewed-by: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ivan Inozemtsev <iinozemtsev@google.com>
This reduces the number of variable-length integer encodings in our
database from 2 to 1, and chooses the more standard one.
TEST=Existing tests, in particular any that involve snapshots.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-ubsan-linux-release-x64-try,vm-kernel-precomp-tsan-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-simarm64c-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: Ia700158ac873ad32ac28c1027a669895961bc715
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196321
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This change removes code duplication between
LoadIndexedInstr::representation() and
StoreIndexedInstr::RequiredInputRepresentation(), multiplied by 4
target architectures.
This change also removes duplicated logic from FFI load and store
intrinsics.
TEST=ci
Change-Id: I9f10cc8c70288ecaea33a337a880c181ecbfcd6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194281
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This change extends EqualityCompare IL instruction to handle case
when operands are nullable ints.
Unlike CheckedSmiComparison, EqualityCompare performs comparison
without making a dynamic call on a slow path. Dynamic call in
CheckedSmiComparison is a problem because it is not present in
the original program and not properly tracked in TFA and precompiler,
so precompiler is not able to drop Code objects (needed for
dynamic calls) reliably.
This change removes the remaining case when CheckedSmiComparison
instruction was used. The cleanup will follow in a separate CL.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/32166
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I8114a2616becf632462d4cb3bfc255691f24e1ef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193447
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Since allocation instructions can throw OOM exceptions, they require
deopt information. In general it's rather hard to test that this deopt
information is correct.
In order to test this, this CL makes those IR instructions support lazy
deopt and add tests that exercise this deopt sequence, thereby ensuring
the deoptimization environment is correct.
Issue https://github.com/dart-lang/sdk/issues/45213
TEST=New vm/dart_2/isolates/deopt/*_test.
Change-Id: I6a02dcf5a0c47636f1f0aa4cd8cc0d2b4f032ca0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192687
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Some IR instructions (e.g. RawStoreField) can never deopt by-design. Some
instructions are calling runtime entries but those are leaf calls (e.g.
InvokeMathCFunction). Some are synthetic IR instructions that never
actually emit code (e.g. Constraint). Some IR instructions cannot deopt
due to inputs (e.g. Boxing instructions whose value is guaranteed to fit
into Smi).
Part of https://dart-review.googlesource.com/c/sdk/+/190481
TEST=Existing test suite.
Change-Id: I770b7d094ef36221d357157c736710476fd6bfb2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190482
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This CL adds code to detect situations when we hit
null error through GDT call with non-null receiver.
We achieve this by making receiver's cid part of the
GDT calling convention and checking this cid
in runtime entry responsible for throwing the null
error.
This CL is an attempt to narrow down b/179632636
and collapse various impossible crash reports
into a single native crash cluster by crashing VM
instead of throwing a null error.
TEST=tested manually
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: If2ed4646c4c0f403016266e4e83e296a7b234cb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191141
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
AssertBoolean may assume its input is nullable bool. BooleanNegate may assume its input is non-nullable bool. Use fixed alignment bits to distinguish true, false and null.
TEST=ci
Change-Id: Ife694e66c1225d943881c2325cd67f18575a47e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189240
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
When creating new instructions that inherit a token position that
represents a source location from another instruction, the inheriting
instruction must also have the same inlining ID in order for the source
position represented by the token position to be looked up in the
correct script.
Force this by wrapping both in a single InstructionSource struct which
is taken by instructions which take token positions instead of just a
token position. That way, it's more work to manually transfer the token
position separately from the inlining ID of the instruction than doing
the right thing of transfering both at once.
To ensure this information is kept consistent, we pass InstructionSource
structs through the FlowGraphCompiler all the way down to the
CodeSourceMapBuilder.
This CL also makes the following changes:
* Cache the upper bound of source positions in scripts and use it to add
a check for if a given real token position is valid for the script
without iterating over the line starts data for each token position.
* Start inlining intervals appropriately when adding descriptor and
null check information to the code source map.
Code size changes are minimal on Flutter gallery in release mode
(<0.05% decrease).
TEST=Existing tests on trybots, with manual checking with
--check-token-positions that previous errors are now removed.
Bug: https://github.com/dart-lang/sdk/issues/44436
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-ia32-try
Change-Id: I23ced262cb4e9fe9d81356f409e7e8d220d63ee0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173967
Reviewed-by: Régis Crelier <regis@google.com>
This CL adds passing structs by value in FFI trampolines.
Nested structs and inline arrays are future work.
C defines passing empty structs as undefined behavior, so that is not
supported in this CL.
Suggested review order:
1) commit message
2) ffi/marshaller (decisions for what is done in IL and what in MC)
3) frontend/kernel_to_il (IL construction)
4) backend/il (MC generation from IL)
5) rest in VM
Overall architecture is that structs are split up into word-size chunks
in IL when this is possible: 1 definition in IL per chunk, 1 Location in
IL per chunk, and 1 NativeLocation for the backend per chunk.
In some cases it is not possible or less convenient to split into
chunks. In these cases TypedDataBase objects are stored into and loaded
from directly in machine code.
The various cases:
- FFI call arguments which are not passed as pointers: pass individual
chunks to FFI call which already have the right location.
- FFI call arguments which are passed as pointers: Pass in TypedDataBase
to FFI call, allocate space on the stack, and make a copy on the stack
and pass the copies' address to the callee.
- FFI call return value: pass in TypedData to FFI call, and copy result
in machine code.
- FFI callback arguments which are not passed as pointers: IL definition
for each chunk, and populate a new TypedData with those chunks.
- FFI callback arguments which are passed as pointer: IL definition for
the pointer, and copying of contents in IL.
- FFI return value when location is pointer: Copy data to callee result
location in IL.
- FFI return value when location is not a pointer: Copy data in machine
code to the right registers.
Some other notes about the implementation:
- Due to Store/LoadIndexed loading doubles from float arrays, we use
a int32 instead and use the BitCastInstr.
- Linux ia32 uses `ret 4` when returning structs by value. This requires
special casing in the FFI callback trampolines to either use `ret` or
`ret 4` when returning.
- The 1 IL definition, 1 Location, and 1 NativeLocation approach does
not remove the need for special casing PairLocations in the machine
code generation because they are 1 Location belonging to 1 definition.
Because of the amount of corner cases in the calling conventions that
need to be covered, the tests are generated, rather than hand-written.
ABIs tested on CQ: x64 (Linux, MacOS, Windows), ia32 (Linux, Windows),
arm (Android softFP, Linux hardFP), arm64 Android.
ABIs tested locally through Flutter: ia32 Android (emulator), x64 iOS
(simulator), arm64 iOS.
ABIs not tested: arm iOS.
TEST=runtime/bin/ffi_test/ffi_test_functions_generated.cc
TEST=runtime/bin/ffi_test/ffi_test_functions.cc
TEST=tests/{ffi,ffi_2}/function_structs_by_value_generated_test.dart
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_generated_tes
TEST=tests/{ffi,ffi_2}/function_callbacks_structs_by_value_test.dart
TEST=tests/{ffi,ffi_2}/vmspecific_static_checks_test.dart
Closes https://github.com/dart-lang/sdk/issues/36730.
Change-Id: I474d3a4ee1faadbe767ddadd1b696e24d8dc364c
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140290
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL allows register allocator to mark functions as frameless
in AOT bare instructions mode if they don't contain calls (with an
exception for write barrier related calls), don't require
spills and don't use optional or named parameters.
Calls to Write Barrier helper stubs introduce some complexity
on ARM/ARM64, where we need to take into account that return
address needs to be explicitly spilled from LR register. To keep
the code size balance (frame setup vs manual LR spilling on WB
slow path) we allow one instruction with a write barrier
per frameless function.
Flutter Gallery snapshot becomes 800KB smaller (3% reduction), but
APK size for some reason increases by .1%. On a large internal
application effect is much smaller (60K - 0.16% reduction).
Flutter Gallery is disproportionally affected by the change due to
abnormal amount of small getters originating from localizations:
each localized phrase becomes a getter returning a constant on a
localization specific class.
TEST=tested through normal test suite
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-ffi-android-release-arm64-try,vm-ffi-android-release-arm-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64-try
Change-Id: I86cb45a57567ee654ead388957ac4e561634ef9c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175304
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Assembler now always knows whether LR contains return address
or can be safely cloberred.
TEST=tested through normal test suite
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-ffi-android-release-arm64-try,vm-ffi-android-release-arm-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64-try
Change-Id: I0b436509389cfd29ee413fd28930fb2d6b07616e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174468
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL enables dart:ffi on Fuchsia.
It has the necessary VM changes for x64 and arm64.
- Allocate VM callback trampoline memory pages with RX.
- Do not use x18 on arm64.
- Save x18 on arm64 in Thread to be able to restore it on exceptions.
We do not have a bot testing this
https://github.com/dart-lang/sdk/issues/38752.
The bot only tests the build, it does not run the emulator.
Manual build and test steps x64 Fuchsia emulator:
```
tools/build.py --os=fuchsia -m debug fuchsia_ffi_test_package
xvfb-run third_party/fuchsia/sdk/linux/bin/femu.sh --image qemu-x64 -N --headless
third_party/fuchsia/sdk/linux/bin/fserve.sh --bucket fuchsia-sdk --image qemu-x64 --device-name step-atom-yard-juicy
third_party/fuchsia/sdk/linux/bin/fpublish.sh out/DebugFuchsiaX64/gen/dart_ffi_test_debug/dart_ffi_test_debug.far
third_party/fuchsia/sdk/linux/bin/fssh.sh --device-name step-atom-yard-juicy run fuchsia-pkg://fuchsia.com/dart_ffi_test_debug#meta/dart.cmx --ignore-unrecognized-flags --packages=/pkg/data/.packages --disable-dart-dev /pkg/data/tests/ffi/all_positive.dart
```
Manual build and test steps arm64 Fuchsia emulator:
```
tools/build.py --no-start-goma --os=fuchsia -m debug -a arm64 fuchsia_ffi_test_package
xvfb-run third_party/fuchsia/sdk/linux/bin/femu.sh --image qemu-arm64 -N --headless --experiment-arm64
third_party/fuchsia/sdk/linux/bin/fserve.sh --bucket fuchsia-sdk --image qemu-arm64 --device-name step-atom-yard-juicy
third_party/fuchsia/sdk/linux/bin/fpublish.sh out/DebugFuchsiaARM64/gen/dart_ffi_test_debug/dart_ffi_test_debug.far
third_party/fuchsia/sdk/linux/bin/fssh.sh --device-name step-atom-yard-juicy run fuchsia-pkg://fuchsia.com/dart_ffi_test_debug#meta/dart.cmx --ignore-unrecognized-flags --packages=/pkg/data/.packages --disable-dart-dev /pkg/data/tests/ffi/all_positive.dart
```
Note that running on the arm64 emulator takes over an hour.
This has been tested on an Estelle device.
TEST=Manual test steps above.
The AOT mode has seen an initial test in g3 by rolling gen_snapshot and
the flutter runner.
Arm64 x18 treatment along the lines of
https://dart-review.googlesource.com/c/sdk/+/119481:
- FfiCallInstr, does not modify x18 at all. We do not have to do
anything special when having a C++ frame on top of a Dart frame.
- NativeEntryInstr and NativeReturnInstr, saves and restores x18 with
PushNativeCalleeSavedRegisters && PopNativeCalleeSavedRegisters. Also
save to the thread field in both places. That way the JumpToFrame stub
can load from the thread field.
Closes: https://github.com/dart-lang/sdk/issues/44434
Change-Id: Ide29369c878fed5909a0d6f7d4c3f7508e179f25
Cq-Include-Trybots: luci.dart.try:vm-fuchsia-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173273
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This is a reland of 42c76fd910
Main issue was due to the parameter order of the invoke field dispatcher
not matching the argument order described by its saved arguments
descriptor. There's no reason for it not to, so now it does.
Also fixes some issues with stack trace tests that failed due to
increased deduplication of closures by forbidding deduplication for
those tests.
TEST=Run on trybots of all architectures as well as flutter engine
trybot, new test added for downstream issues seen after initial landing.
Original change's description:
> [vm/compiler] Move AssertAssignables out of closure bodies.
>
> This CL moves the final set of checks out of closure bodies and into
> dynamic closure call dispatchers. It also adds stubs for checking top
> types and null assignability for types only known at runtime.
>
> Fixes https://github.com/dart-lang/sdk/issues/40813 .
>
> Changes in Flutter gallery in release mode:
>
> * arm7: -3.05% total, +0.99% vmisolate, -0.89% isolate,
> -1.20% readonly, -4.43% instructions
> * arm8: -3.20% total, +0.99% vmisolate, -0.88% isolate,
> -1.18% readonly, -5.05% instructions
>
> TEST=Run on trybots of all architectures, includes test adjustments where needed.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
> Change-Id: Ifb136c64339be76a642ecbb4fda26b6ce8f871f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166622
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
Change-Id: Ic5ec59cf355f7779bb82db798d97d762ba1e5556
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-product-x64-try,flutter-engine-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172644
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit 42c76fd910.
Reason for revert: Failures in flutter/google3. Initial hypothesis is related to product mode and instruction deduplication.
Original change's description:
> [vm/compiler] Move AssertAssignables out of closure bodies.
>
> This CL moves the final set of checks out of closure bodies and into
> dynamic closure call dispatchers. It also adds stubs for checking top
> types and null assignability for types only known at runtime.
>
> Fixes https://github.com/dart-lang/sdk/issues/40813 .
>
> Changes in Flutter gallery in release mode:
>
> * arm7: -3.05% total, +0.99% vmisolate, -0.89% isolate,
> -1.20% readonly, -4.43% instructions
> * arm8: -3.20% total, +0.99% vmisolate, -0.88% isolate,
> -1.18% readonly, -5.05% instructions
>
> TEST=Run on trybots of all architectures, includes test adjustments where needed.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
> Change-Id: Ifb136c64339be76a642ecbb4fda26b6ce8f871f9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166622
> Commit-Queue: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>
TBR=kustermann@google.com,regis@google.com,sstrickl@google.com
Change-Id: Iaf79acddcf18fb3699a894950b31515d6f759349
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172643
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
This CL moves the final set of checks out of closure bodies and into
dynamic closure call dispatchers. It also adds stubs for checking top
types and null assignability for types only known at runtime.
Fixes https://github.com/dart-lang/sdk/issues/40813 .
Changes in Flutter gallery in release mode:
* arm7: -3.05% total, +0.99% vmisolate, -0.89% isolate,
-1.20% readonly, -4.43% instructions
* arm8: -3.20% total, +0.99% vmisolate, -0.88% isolate,
-1.18% readonly, -5.05% instructions
TEST=Run on trybots of all architectures, includes test adjustments where needed.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ifb136c64339be76a642ecbb4fda26b6ce8f871f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166622
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
The only architecture-specific part of type testing stubs (TTS) is which
registers get saved/restored across SubtypeTestCache stub calls in the
slow TTS implementation, which is now a register mask constant in the
TypeTestABI struct.
This CL also:
* Creates a single definition of OperandSize in assembler_base.h. Moves
OperandSize into the compiler namespace. assembler_base.h. Renames
constants common to all architectures to neutral names (e.g., kWord =>
kFourBytes), since "word" has different meanings across architectures.
* Creates a new JumpDistance enum for kNearJump/kFarJump in
AssemblerBase and replace existing bool uses with the constants.
* Adds JumpDistance arguments to some of the generalized Assembler
methods so they aren't forced to use far jumps on IA32/X64. These
added JumpDistance arguments are ignored on ARM architectures.
* Adds a new generalized Assembler::LoadIndexedPayload method for
loading indexed objects out of post-header variable-length payloads.
* Moved the OperandSize argument for LoadForOffset and similar methods
on 32-bit ARM from being the first argument to being the second to
last argument just to match other architectures, and to allow it to
have a default value of kFourBytes.
TEST=Run on trybots of all architectures.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ia930613e96db4d1ab324c5a355dca8994d2a77f0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172160
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
When loading a late field without initializer, we check if it was not
initialized and throw an exception. This change adds slow path (shared
if supported) for throwing LateInitializationError instead of calling
an initializer which would throw an exception. This improves register
allocation around accesses to such late fields.
In addition, this CL removes duplicated code for generating shared
stubs.
Size of Flutter gallery snapshot on armv8 -0.45%
(gzip -0.34%, brotli -0.41%)
Null-safe version of ParticleSystemUpdate benchmark on x64
JIT +9-17%, AOT +3-4%.
Change-Id: Ia12c1efec0871a2fd7acc6db75f255853af52d0f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168993
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
These checks are now performed in the dynamic closure call dispatcher.
To avoid having to create type argument vectors (TAVs) for default type
arguments at runtime, we cache a compile-time created TAV in the
ClosureData for the closure function which is retrieved by the
dispatcher when needed.
We also keep an associated packed field of information that can also be
determined at compile time:
* Whether the cached TAV needs instantiation or can share its
instantiator or function type arguments.
* The number of parent type parameters.
The former allows the generated IL to keep the invariant that the
InstantiateTypeArguments instruction (and the runtime entry it calls) is
only used for uninstantiated TAVs.
Also changes the destination name to an Value input for the
AssertSubtype instruction and adds handling for non-constant types and
names in that instruction's backend.
Additional changes:
* Adds new slots for ClosureData, Function, and TypeArguments.
* Adds a new kUnboxedUint8 representation (needed for
TypeParameterLayout::flags_, which is of type uint8_t).
* Extends LoadField to handle uint8_t unboxed native fields.
* Adds BoxUint8 for boxing unboxed uint8_t values.
Code size impact on Flutter gallery in release mode:
* arm7: total +0.08%, vmisolate: +0.56%, isolate: +0.42%,
readonly: -0.04%, instructions: -0.03%
* arm8: total +0.12%, vmisolate: +0.56%, isolate: +0.42%,
readonly: -0.002%, instructions: +0.03%
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Bug: https://github.com/dart-lang/sdk/issues/40813
Change-Id: I5a7de27a17e3119e27752bd0d10e1c6bc1b52a16
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158844
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Current users of InstantiateTypeArguments have the uninstantiated type
arguments available at compile time. After upcoming work on #40813,
there are cases where the generated code needs to instantiate type
arguments that are only available at runtime.
This work changes the uninstantiated type arguments for the
InstantiateTypeArguments instruction to a runtime Value pointer instead
of a compile time TypeArguments reference. The changes beyond that are
limited to only what is needed to get current uses working, with ASSERTs
checking that current invariants still hold. The rest of the changes
needed for dynamic uninstantiated type arguments will come later.
Bug: https://github.com/dart-lang/sdk/issues/40813
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try
Change-Id: Iafe836cfd8744d3e835e60112a4f099e631f4782
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162183
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Unify the order of checks in both LoadFieldInstr::MakeLocationSummary()
and LoadFieldInstr::EmitNativeCode() so that all architectures first
check the slot representation, then for unboxed fields when optimizing,
then for potential unboxed fields.
Add kUnboxedInt64 handling to IA32, so that all architectures support
the same set of unboxed slot representations.
For 32-bit architectures, use RepresentationUtils::ValueSize to
determine when to use single registers vs. pairs of registers for
unboxed integers.
Move (and dedup) any asserts in LoadFieldInstr::EmitNativeCode() into
LoadFieldInstr::MakeLocationSummary(), since the latter is always called
first, making the ASSERTs fail as soon as possible, and there's no need
to re-check the duplicated ASSERTS.
Change-Id: Ia0f8fddd5a3cda036f9e4c51e3e9a374be121e7f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159667
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Previously, all boxed native field slots were marked not nullable.
However, several boxed native fields are in fact nullable, which means
that even generated IL that appropriately checks these fields for null
can break in optimized code when the null checks are removed.
Also add some helper utilities for working with representations.
Change-Id: I22703d14cfbadbabeae74179476dea47ace12161
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159280
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
This CL performs the following checks in the invoke field dispatcher for
dynamic closure calls when lazy dispatchers are enabled:
* The provided function type arguments vector (if any) has the correct
length.
* No function type arguments should be provided if the closure has
delayed type arguments.
* All required arguments (positional in all modes, named in appropriate
null safety modes) have been provided by the caller.
* If there are optional positional arguments, an appropriate number
has been provided.
* If there are optional named arguments, their names are valid.
Since the runtime already handles checking the argument shapes when lazy
dispatchers are disabled, these checks are now completely removed from
closure bodies in all cases. Thus, the only remaining checks in closure
bodies are the type checks performed by AssertSubtype and
AssertAssignable when lazy dispatchers are enabled.
Changes in the Flutter Gallery:
* ARM7, release: -3.61% instructions, -2.19% total
* ARM7, sizeopt: -3.62% instructions, -2.55% total
* ARM8, release: -3.66% instructions, -1.98% total
* ARM8, sizeopt: -3.65% instructions, -2.37% total
Most of these changes are already exercised by existing tests such as
(but not limited to):
* corelib{,_2}/dynamic_nosuchmethod_test
* language{,_2}/call/call_test
* language{,_2}/closure/tearoff_dynamic_test
* language{,_2}/generic/function_bounds_test
* language{,_2}/parameter/named_with_conversions_test
* language{,_2}/vm/no_such_args_error_message_vm_test
I've added one test to specifically check the interaction between
dynamic calls and required named parameters. There is some coverage in
other NNBD tests, but those are not directly focused on testing this
specifically.
Other changes:
* Adds initial cached ranges for certain BinarySmiOp and ShiftIntegerOp
instructions when the RHS is a constant, to avoid false negatives for
deoptimization and throw checks prior to range analysis.
* Adds new slots for various Function fields.
* Adds the ability to define unboxed native slots, which are always
unboxed after retrieval even in unoptimized code. In the first
iteration, the backend only handles loads from Uint32 unboxed native
slots. Part of https://github.com/dart-lang/sdk/issues/42793.
* Removed the special handling for loading from non-nullable int fields
in AOT compilation. Instead, their treatment is unified with the
treatment of the new unboxed native fields, since the source field is
always unboxed and the result of the load is also always unboxed, as
code involving them is always optimized.
Bug: https://github.com/dart-lang/sdk/issues/40813
Change-Id: Ia02aa3e872c1fefd906fd67b55021ea1797556e4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155604
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Don't use PC-relative calls when calling between loading units.
- Sort the contents of the code cluster by loading unit, then by text offset.
- Handle binding PC-relatives calls and inserting trampolines per loading unit.
- Create one code order table per loading unit to implement PC -> Code lookup.
- Read code order tables directly, instead of copying into malloc'd memory.
--use_table_dispatch still not yet supported.
This slightly shrinks non-split binaries (~2% clustered part, 0.4% total snapshot) due to the new delta encoding when Code references Instructions.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I51052ebc7b4968b554897d1710135a6c41821302
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157820
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
If inline bump-pointer allocation fails, allocation stubs must check whether the allocation result from the runtime is old and ensure it is in the remembered set if so to make write barrier elimination safe. CreateArrayInstr would directly call the runtime entry if inline bump-pointer allocation failed without performing the remembered check. This CL changes the slow path to call the AllocateArray stub instead, matching every other allocation slow path.
Bug: https://github.com/dart-lang/sdk/issues/42489
Change-Id: Ic502e43683607e1d0f85154f0e3a8c3ea11deefa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154623
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Right now we assign class ids to top-level classes, abstract classes as
well as concrete classes. All of them have allocated from a 16-bit pool
of ids. The VM FATAL()s once it hits that limit.
Customers who run very large programs (significant amount of generated
code) on the Dart VM have started to hit this 16-bit class limit.
Concrete classes can have instances in the heap. Our current heap layout
only allows 16-bit class ids to be encoded in the header word. To avoid
increasing the size of heap objects or shrinking the size of the identity
hash code to 16-bit we keep class ids in object headers to be 16-bit.
Abstract classes cannot have instances in the heap. Though their class
ids are encoded in type objects. Furthermore we sort classes in
AOT/AppJIT mode to perform fast class-id range checks. To avoid impacting
this optimization we treat abstract classes the same way as concrete
classes.
Top-level classes cannot have instances in the heap. Their class ids are
only used in the runtime code, for example for hot-reload as well as
part of the service protocol.
=> We can allocate class ids outside the 16-bit range for top-level
classes, thereby freeing a significant amount of space in the 16-bit
range.
This CL does exactly that: We change classid_t to be int32_t. The
ClassLayout::id_ can now be assigned ids outside 16-bit range for
top-level classes. To do this we keep dart classes and top level classes
as separate arrays in the ClassTable.
Issue https://github.com/dart-lang/sdk/issues/42533
See also b/160229360
Change-Id: I6710a644e7b0ab2d4f4c792bef8e1f91cb117421
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153607
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Previously, NullError runtime entry was used both to throw
NoSuchMethodError and _CastError. _CastError was used if selector is
null. However, with --dwarf-stack-traces CodeSourceMap is not preserved
and selector is not available, so wrong exception type was thrown.
This is fixed by introducing separate NullCastError runtime entry along
with corresponding shared stubs.
NullErrorSlowPath is extended to support 3 exception types
(NoSuchMethodError, ArgumentError and CastError).
NullArgErrorSlowPath is removed as it duplicates NullErrorSlowPath.
Fixes vm/dart/null_checks_with_dwarf_stack_traces_test
Fixes https://github.com/dart-lang/sdk/issues/42149
Change-Id: I4f2b34ad2b161459491de60d8737bb4f87117336
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/153040
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This includes support for calling Dart_PropagateError in native code
when doing FFI calls, and catching uncaught exceptions with Dart_IsError
when doing FFI callbacks.
The support for Dart_PropagateError adds a catch entry to the FFI
trampoline, which prevents inlining these trampolines in AOT. This
regresses the FfiCall benchmarks by 1-2% in AOT.
In addition, Dart_PropagateError requires maintaining a bit whether we
entered native/VM code from generated code through FFI or not. That way
we can do the proper transition on the exception path. When entering
generated code, we store this bit on the stack, right after the entry
frame.
Design: http://go/dart-ffi-handles
Issue: https://github.com/dart-lang/sdk/issues/36858
Issue: https://github.com/dart-lang/sdk/issues/41319
Change-Id: Idfd7ff69132fb29cc730931a4113d914d4437396
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145591
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Load*Field and Init*Field IL instructions are fused into load
instructions with 'calls_initializer' flag.
A new, more powerful elimination of lazy initializer calls uses
data flow analysis and load forwarding (place numbering).
In addition to improved elimination of initializer calls,
fusing instructions avoids extra load after field is already
loaded for lazy initialization check.
Fixes https://github.com/dart-lang/sdk/issues/41417
Change-Id: Ibb2a63f84b9b0c970db67b3d6684ec15384a64e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148283
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Ivan Inozemtsev