This CL optimizes Pointer operations in hot loops for Pointer<NativeInteger/NativeDouble/Pointer> (not for structs).
Design: go/dart-ffi-pointers-il
It provides roughly a 100x speedup for the FfiMemory benchmark. The next 5x speedup is to get rid of allocations due to `load` and `store` not being inlined.
FFI API is changed to enable optimizations:
* Disable dynamic invocations of Pointer.load / Pointer.store.
* Disallow implicit downcast of argument passed to Pointer.store.
* Stop zeroing out Pointer.address on Pointer.free().
Issue: https://github.com/dart-lang/sdk/issues/38172
Related issues:
Closes: https://github.com/dart-lang/sdk/issues/35902 (Disallowing dynamic invocations of Pointer ops.)
Closes: https://github.com/dart-lang/sdk/issues/37385 (Function variance checking.)
Change-Id: I96058d8b5b49052eb6999f084372e6f08b4f6f17
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-dartkb-linux-debug-simarm64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-dartkb-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-android-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117547
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The bytecode encoding allows for 8-bits of context id, but this limit is exceeded in Flutter test code. Context ids are as yet unused, but will be used to disambiguate variables at the same position in different contexts.
Change-Id: I64130c11ddf04495b2719aa6f4809fa64ff135c1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119728
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Like the constant pool, this pool is populated when the flag
`--populate-llvm-constant-pool` is used. This pool contains the `Function`
objects that were compiled, and each `FlowGraph` node in the serialized flow
graphs will contain extra information labeled by `llvm-index` which is the
index of its corresponding `Function` object in the pool.
Also fix up printing of the constant and function pools when the flag
`--print-llvm-constant-pool` is used so that the output is buffered to avoid
overlapping output from concurrently running isolates.
Bug: https://github.com/dart-lang/sdk/issues/38297
Change-Id: I67739500c665e509ed16becc2e6bca7fe4928eb5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119543
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The code from these precompiled constructors is thrown away,
and if we are adding constants to the LLVM constant pool, then
we may add constants that will not actually appear in the final
program which may cause issues for tree shaking.
If these constructors are actually used, then they will be
recompiled normally.
In addition, we also do not add unreferenced constants to the
constant pool. That is, these constant definitions will not have
an `llvm_index` value in their serialized form when the LLVM
constant pool is populated.
Bug: https://github.com/dart-lang/sdk/issues/38661
Change-Id: I2a3702fd3f174504a5f01458064306cd687d7949
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119339
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Fuchsia is about to turn on ShadowCallStack for ARM64. Once this is enabled, we need to treat R18 like a preserved register. Generated Dart has not accessed this register because it is reserved on iOS, and in the absence of Dart exceptions this would be sufficient for us to be ShadowCallStack compatible. However, our exception handling mechanism jumps past all the C++ frames between the Dart exit frame and Exceptions::JumpToFrame, skipping code that would pop from R18.
Add save/restore of R18 in the invocation stubs, and restore of R18 in the jump stub. The latter prevents the ShadowCallStack from overflowing for code that has lots of exceptions without a native call.
Bug: https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=37449
Change-Id: I2ce6e46624c8d72507e7afa7a44839b1f0def556
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119481
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Instead of storing the PC offset inside the StackMap object, store it instead
in the `Array` containing `StackMap`s in `Code` objects. That is, the `Array`
provided by `Code::stackmaps()` no longer contains just `StackMap` objects, but
instead contains `Smi`s and `StackMap` objects in alternating fashion. Each
`Smi` is the PC offset for the `StackMap` object that follows.
This ends up changing very little code outside of `StackMap`,
`Code::GetStackMap`, and `StackMapTableBuilder`, as there are only two types of
`StackMap` users:
* Users that call `Code::GetStackMap` already have the PC offset.
* Users that call Code::stackmaps() can just fetch the PC offset from the
returned `Array` instead.
On 64-bit architectures, we will use more space to represent the PC offset
as a Smi in the Array than the old uint32_t field. However, the drop in total
number of StackMap objects due to an increased ability to canonicalize
them should offset this. On 32-bit architectures, we can only represent
30 bit PC offsets, not 32 bit PC offsets, but that shouldn't be a problem
in practice except for pathological cases.
_Numbers from building the Flutter gallery in android_release mode_
Since PC offsets are no longer in the `StackMap` objects, this enables
a lot more canonicalization than before. Previously, we generated
49379 `StackMap`s, but now we only generate 16139 `StackMap`s, just under
a third of the original number. This is because there were a lot of
`StackMap`s that differed only in their PC offset, and now they can
be canonicalized into the same `StackMap` object.
When building the Flutter gallery with android_release, the app.so size
drops from 11276896 bytes to 10908256 bytes, a difference of 368640
bytes, or 3.27%.
Using the AOT snapshot profiling support, we see the following drops:
Heap snapshot size drops from 10.7 MB to 10.4 MB.
`Code` | Before | After | Difference
---------------------------------------------------------
Shallow size | 352363 | 352363 | 0
Retained size | 9114801 | 8796064 | -318737
Percent of snapshot | 81.2% | 80.7% | -0.5%
`(RO)StackMap` | Before | After | Difference
---------------------------------------------------------
Shallow size | 49381 | 16141 | -33240
Retained size | 893965 | 286613 | -607352
Percent of snapshot | 7.97% | 2.63% | -5.34%
`StackMap` | Before | After | Difference
---------------------------------------------------------
Shallow size | 844584 | 270472 | -574112
Retained size | 844584 | 270472 | -574112
Percent of snapshot | 7.53% | 2.48% | -5.05%
As we'd expect from the `StackMap` numbers above, we end up using a little
under a third of the space for `StackMap`s. We actually use even less space
(32.0% of the original) than the drop in `StackMap` numbers (32.7%), because
each `RawStackMap` instance is 32 bits smaller due to dropping the `pc_offset_`
field. Note that even though these PC offsets now show up in the
`Code::stackmaps()` `Array`, we can see this is still a net drop in space used
by looking at the retained size of `Code` objects.
Bug: https://github.com/dart-lang/sdk/issues/35274
Change-Id: I0910a43e7a5a7e2e721676209196be1884c5a71c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119147
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
AOT transformations add vm.direct-call.metadata with devirtualization
information to AST nodes.
Bytecode generator should take this information into account when
generating bytecode in case of AOT. Bytecode format is extended with
CheckReceiverForNull and UncheckedDirectCall instructions, and
DirectCallViaDynamicForwarder constant pool entry in order to represent
devirtualized calls.
Change-Id: I697432ddd0b58d2d0413715132ba5e90eb606ec1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119201
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
* Clear bytecode for all functions including closures and field
initializers, so their object pools are not preventing constant
objects from being reclaimed by GC.
* Bytecode reader should use dynamic invocation forwarders in AOT too.
* Fix assertion in RegisterTypeArgumentsUse (type arguments in
generic closure functions could come from _prependTypeArguments call).
Change-Id: I2154c87f30720607288ec12012a6d0ec1db618dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118365
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Consider the following situation:
class A { foo() {...} }
class B extends A { foo() {...} }
During AOT compilation (in case FLAG_lazy_dispatchers == false)
Resolver::ResolveDynamicAnyArgs was not looking for possible methods
when getter is requested. As a result, if method extractor A.get:foo
has been created, but B.get:foo hasn't, then lookup for B.get:foo
incorrectly returns A.get:foo (instead of null).
Similar situation was previously fixed for a particular call site of
ResolveForReceiverClass (which uses ResolveDynamicAnyArgs):
https://dart-review.googlesource.com/c/sdk/+/30520
However, there are more calls to ResolveForReceiverClass in the compiler
where its incorrect result may be used.
Change-Id: I4ebd6b91157cc048b94e5a5ce182c42dc808f4f7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118260
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Printing out the flow graph before and after round trip serialization
revealed slight differences between the flow graphs that might lead
to either bugs or different results in compilation. Remove these
differences.
Also in this change:
* Remove cases where printing or serializing a flow graph could have
the side effect of creating new `CompileType`s or `AbstractType`s
not previously in the graph.
* Ensure reaching types for `Value`s are cloned if the binding
definition changes and check that reaching types either have no
owner or are owned by the `Value`'s definition with the
FlowGraphChecker. (Before this change, we had `CompileType`s in
the graph whose owner had long since been removed from it.)
* Adds `ASSERT`s to check that unexpected non-canonical `Instances`
are not allowed as the value of `ConstantInstr`s and adds some
canonicalization that did not previously occur.
Bug: https://github.com/dart-lang/sdk/issues/36882
Change-Id: I334ffa2a6383291a7cb318343c71bd55a41269a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117143
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Aart Bik <ajcbik@google.com>
This change fixes the following problems:
* In precompiler, all PcDescriptor entries which do not have
try_index are ignored, so try_index should be always set when building
PcDescriptors for bytecode try-blocks.
* Precompiler should use more general binary_declaration_offset()
instead of kernel_offset() as a hashcode for a Field.
* Querying of PrecodueAttributesMetadata should not try to use
kernel AST if function/field is declared in bytecode.
Also, added printing of operands for LoadUntagged IL instruction to
simplify debugging, and excluded precompiler.{h,cc} from
DART_PRECOMPILED_RUNTIME.
Change-Id: I563b81a0b31b5b268709b1b7b0c3bea4e76813d8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118120
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Aart Bik <ajcbik@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Rationale:
Some processors may support just popcnt and, in theory,
some just lzcnt. This change makes the test more
specific. The bsf/bsr tests are never skipped anymore.
Change-Id: I0e7b8772789473ce238f0c5c93378c06c39c695b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117940
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Aart Bik <ajcbik@google.com>
In bytecode we have enough information to "finalize" type parameters
immediately while reading bytecode. Finalization of a type parameter is
just calculating its index in type arguments vector.
Change-Id: I6f47821b6ddb44441b3ca8c682262a38c454eeb7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117773
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Fixes the changes to assembly snapshot writer in SIMARM_X64 mode. This unfortunately can't be tested until the ELF loader is available.
Take 2 is in patchset 1.
Change-Id: Ib8b067dd4f09dcba3b142705e9cb4dfbf4e3eb53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117726
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
When installing new code, we need to ensure the mutator is stopped for a
variety of reasons.
Right now we only need to distuinguish mutator and background compiler
when installing code: The mutator can install code without any
synchronization whereas the bg compiler needs to get the mutator to a safepoint.
Yet once all isolates within one isolate group share a heap, a mutator
might need to stop all other mutators before installing code (since they
operate on pages on which we flip page protection bits)
This CL adds IsolateGroup::RunWithStoppedMutators which will get other
mutators to a safepoint (if there are multiple or we are on a bg
compiler thread).
Along with this we add a read-write lock and use it for the protection
of the IsolateGroup::isolate_: While we make assumptions about the
number of isolates in a group we force all pending additions of new
isolates to wait. Later on this will also be used to allow iterating the
list of isolates during GC and prevent new isolates from being added at
the same time.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I6e761fa51d36b2f2b4b67995cac954898ce7fd69
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-android-release-arm-try,vm-dartkb-linux-release-x64-abi-try,vm-kernel-precomp-bare-linux-release-x64-try,vm-kernel-precomp-mac-debug-simarm_x64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116767
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This change fixes assertion in Environment::DropArguments which was checking
if locations_ was set by register allocator. However, register allocator
doesn't call set_locations() if Environment is empty (see
FlowGraphAllocator::ProcessEnvironmentUses).
Fixes several tests with bytecode in dartk-mac-debug-simdbc64 configuration,
including language_2/await_future_test, language_2/deferred_optimized_test etc.
Change-Id: I6509987e91906e53615b5e693d6ac78f383646b6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117583
Reviewed-by: Aart Bik <ajcbik@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Lazy lookup of source and line starts for scripts loaded from bytecode
(implemented in https://dart-review.googlesource.com/c/sdk/+/112680)
relied on source == Symbols::Empty() as an indication that lazy lookup is
needed. However, when Script object is written into app-jit snapshot,
its source is set to null which disables lazy lookup.
This change introduces 'LazyLookupSourceAndLineStarts' flag in Script
objects to avoid relying on value of 'source'.
Fixes service/valid_source_locations_test with bytecode in app-jit mode.
Change-Id: I38174a13d745a1ad9fecad466df745b1df527878
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117214
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Introduced a new code path to load declaration of a referenced class when
loading bytecode, if referenced class is from the same binary. In such case
it's possible to reuse current BytecodeReaderHelper.
Also, AlternativeReadingScope is simplified to save/restore only reader
offset (which is needed in most cases), while saving/restoring data is
still available in AlternativeReadingScopeWithNewData.
This change fixes stack overflow while reading deep class hierarchies in the
tests co19_2/Language/Types/Interface_Types/subtype_t27 and
co19_2/Language/Types/Interface_Types/subtype_t28 with bytecode in
vm-kernel-linux-debug-ia32 configuration.
Change-Id: Iffb1678286c6926926fab465f70251992d451241
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117204
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This is a reland of https://dart-review.googlesource.com/c/sdk/+/113123
after stack overflow in the interpreter was fixed in https://dart-review.googlesource.com/c/sdk/+/117061
Original change's description:
Temporary setting of FLAG_enable_interpreter is avoided in the VM unit tests as
this flag is global and used by kernel service isolate while running tests.
Flipping this flag causes assertion failures in kernel isolate's background compiler
or infinite loop between LazyCompile stub and CompileFunction as AttachBytecode
doesn't set entry point to InterpretCall.
The less intrusive way to ensure compilation of functions in unit tests is to set
FLAG_compilation_counter_threshold to 0.
Change-Id: Ibbcf4b5bb0df558851c80cfa40a9a54f949dd187
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117122
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Ryan Macnak