Original change in patchset 1.
The reland contains a fix for handles. The objects passed in handles
must live on the stack (or in the heap) so that that memory location
can be used as a handle.
This CL introduces a new allocation policy `RequiresRegister` which
forces constants to be spilled to the stack during register
allocation.
Note: Parameters to FfiCall instructions are currently not used in a
way that can make the parameters assigned to registers. So only
constants are treated currently. If we have a way to reproduce params
being assigned to registers, then we can force spilling for
non-consts as well in the register allocator.
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
TEST=tests/ffi/vmspecific_ffi_native_handles_test.dart
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
CoreLibraryReviewExempt: Internal FFI implementation changes
Change-Id: Ia1401b335524dcbf50c663a8770d9a02802923df
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333841
Reviewed-by: Slava Egorov <vegorov@google.com>
This reverts commit e16bb210d2.
Reason for revert: Indication this caused engine test
failures of the kind:
```
[0;32m[ RUN ] [mEmbedderA11yTest.A11yTreeIsConsistentUsingV1Callbacks
../../flutter/shell/platform/embedder/tests/embedder_a11y_unittests.cc:639: Failure
Expected equality of these values:
std::strncmp(kTooltip, node->tooltip, sizeof(kTooltip) - 1)
Which is: 116
0
```
Original change's description:
> [vm/ffi] Optimize `@Native` calls
>
> This CL removes static fields for storing the `@Native`'s function
> addresses. Instead, the function addresses are stored in the object
> pool for all archs except for ia32. ia32 has no AOT and no AppJit
> snapshots, so the addresses are directly embedded in the code.
>
> This CL removes the closure wrapping for `@Native`s. Instead of
> `pointer.asFunctionInternal()()` where `asFunction` returns a closure
> which contains the trampoline, the function is compiled to a body
> which contains the trampoline `Native()`. This is possible for
> `@Native`s because the dylib and symbol names are known statically.
>
> Doing the compilation in kernel_to_il instead of a CFE transform
> enables supporting static linking later. (The alternative would have
> been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
> instead of constructing the IL in kernel_to_il.
>
> To enable running resolution in ia32 in kernel_to_il.cc, the
> resolution function has been made available via
> `runtime/lib/ffi_dynamic_library.h`.
>
> Because the new calls are simply static calls, the TFA can figure
> out const arguments flowing to these calls. This leads to constant
> locations in the parameters to FfiCalls. So, this CL also introduces
> logic to move constants into `NativeLocation`s.
>
> TEST=runtime/vm/compiler/backend/il_test.cc
> TEST=tests/ffi/function_*_native_(leaf_)test.dart
> TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
>
> Closes: https://github.com/dart-lang/sdk/issues/47625
> Closes: https://github.com/dart-lang/sdk/issues/51618
> Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
> CoreLibraryReviewExempt: Internal FFI implementation changes
> Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Icc87a6ca33bffecabb15c6b168a06ccc38c2fe5b
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333840
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL removes static fields for storing the `@Native`'s function
addresses. Instead, the function addresses are stored in the object
pool for all archs except for ia32. ia32 has no AOT and no AppJit
snapshots, so the addresses are directly embedded in the code.
This CL removes the closure wrapping for `@Native`s. Instead of
`pointer.asFunctionInternal()()` where `asFunction` returns a closure
which contains the trampoline, the function is compiled to a body
which contains the trampoline `Native()`. This is possible for
`@Native`s because the dylib and symbol names are known statically.
Doing the compilation in kernel_to_il instead of a CFE transform
enables supporting static linking later. (The alternative would have
been to transform in the cfe to a `@pragma('vm:cachable-idempotent')`
instead of constructing the IL in kernel_to_il.
To enable running resolution in ia32 in kernel_to_il.cc, the
resolution function has been made available via
`runtime/lib/ffi_dynamic_library.h`.
Because the new calls are simply static calls, the TFA can figure
out const arguments flowing to these calls. This leads to constant
locations in the parameters to FfiCalls. So, this CL also introduces
logic to move constants into `NativeLocation`s.
TEST=runtime/vm/compiler/backend/il_test.cc
TEST=tests/ffi/function_*_native_(leaf_)test.dart
TEST=pkg/vm/testcases/transformations/ffi/ffinative_compound_return.dart
Closes: https://github.com/dart-lang/sdk/issues/47625
Closes: https://github.com/dart-lang/sdk/issues/51618
Change-Id: Ic5d017005dedcedea40c455c4d24dbe774f91603
CoreLibraryReviewExempt: Internal FFI implementation changes
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64c-try,vm-aot-win-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284300
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
a) Remove 2 fields in our object representation:
* PatchClass::library_data_
* Library::kernel_data_
=> This saves O(#libraries + #patch-classes) which can amount up to
10+ MB for big apps, as we save not just the slots but the
[ExternalTypedData] objects they used to reference.
Instead we'll compute the KernelLibraryData when we need it by making a
[TypedDataView] of sub parts of the component.
b) We make a kernel binary be represented by a single [ExternalTypedData].
Whenever we need a sub-part of a kernel binary (e.g. one for each
component for concatinated kernels, or one for a library inside a
component) we use proper [TypedDataView]s for that.
c) As we sometimes need to create a view of only a particular library
within a component we need to find start/end of a library based on
index.
=> We store the library index instead of the library offset on
Library/PatchClass.
=> We can easily derive the start/end of a library from it's index by
looking at the kernel component encoding.
d) We make the [Reader] object work purely based on a pointer - instead
of making it have if/else when reading bytes (either from pointer or
from a view).
e) We make the [KernelProgramInfo] store the kernel_component and
various TD views into it.
TEST=ci
Change-Id: Ibe160881ff48635e834c3d647a977a144b5d0565
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313561
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
** KernelProgramInfo
The VM can load multiple kernel files into an isolate. Each kernel file
is represented as a [KernelProgramInfo] object. Loading a kernel blob
will result in a tree of objects
Library
- TopLevelClass
+ Field
+ Function
+ Class
+ Field
+ Function
+ ... closure functions ...
The entire tree belongs to one [KernelProgramInfo] object. There are
two exceptions
* after a hot-reload we inject [PatchClass]es (as owners of
old [Field]/[Function] objects) that identify the old kernel data
* expression evaluation functions will get a special data array with
information about the expression evaluation kernel data
=> We attach the [KernelProgramInfo] to the root of the tree, namely
the [Library]
=> We introduce `{Field,Function,Class,Library}::KernelProgramInfo()`
that will get that object - by walking up the chain.
=> We introduce `UntaggedPatchClass::kernel_program_info_` field that
allows saving the old [KernelProgramInfo] (on which the old
[Field]/[Function] objects are based upon) in the [PatchClass].
=> We include the [KernelProgramInfo] in the expression evaluation
data array.
** Scripts
A [Script] object should represent a .dart source file. It contains the
source, line offsets, etc. of that dart source file. The contents of a
dart source file may be used by many classes/libraries or even many
kernel blobs (e.g. via our copying mixin application).
When we build a tree like above, we'll attach a reference to a
[Script] object to [Class] objects. But members of a class may come from
a different script, in which case we inject an intermediary [PatchClass]
that references that other script.
When the compiler frontend loads kernel, builds flow graph, etc we walk
down the kernel binary (the tree above) - all belonging to the same
[KernelProgramInfo] object, but sub-parts may belong to different
[Script]s.
=> We use the existing [ActiveClass] class that keeps track of the
current class & member inside the class we're operating on.
=> Whenever we need the script we get it from the active member
function/class.
** Misc
We make kernel related classes (e.g. [KernelReaderHelper]) independent
of [Script].
We make the [Script] objects hold on to the [KernelProgramInfo] they are
based upon (e.g. for lazy source reading). But since a given [Script]
doesn't belong to a particular kernel blob (multiple kernel blobs may
contain information about the same script) we don't expose this in the
public API.
=> In the future we could have a unique [Script] object per url (instead
of having several as we do now)
TEST=ci
Change-Id: I7bd698f497b0bd8ab8e546540bb1dad5b5a6dd48
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/313381
Reviewed-by: Jens Johansen <jensj@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Decorate Zone, Future, Completer, Timer and Stream with newly-introduced pragma to make sure that message verification stops earlier, produces shorter retaining path for the user.
BUG=https://github.com/dart-lang/sdk/issues/51722
TEST=send_unsupported_objects_test,isolate_exit_unsendable_test.dart
CoreLibraryReviewExempt: vm-specific pragmas
Change-Id: I499eea542d228ac9cf0797a682664f93f360dc80
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/289027
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead of peeking into constant table and then delaying scanning of
constants by putting it in an array, which is walked again in some
future point, we simply read the annotation constants entirely without
requiring const evaluation. This works fine for pragma annotations the
VM is interested in - as there's no user-defined classes involved.
-> Loading kernel will no longer require constant evaluation to work.
Motivation for this is that [0] wants to make this delayed annotation
scanning logic even more complicated, so I prefer to remove it entirely.
[0] https://dart-review.googlesource.com/c/sdk/+/289027
TEST=ci
Change-Id: Ib859480107b6cf119d66035e66ec161ed11ddb32
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/290502
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The pragma can be used to annotate functions which should not appear in
stack traces unless --show-invisible-frames is passed to the VM.
TEST=vm/dart{,_2}/invisible_function_pragma_test
Change-Id: I807359146f16b67913697f07416fbe8ce9aaa6e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233381
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
As part of deprecating support for native extensions we are also
migrating away from legacy VM-specific `native 'name'` syntax
towards metadata based encoding which does not require any special
syntax.
This CL is a step 1 in migration:
- introduces support for `@pragma('vm:external-name', 'name')`
which serves as a direct replacement for `native 'name'`;
- all core libraries and tests are migrated to use the annotation;
Once this CL lands and rolls we will edit internal and external embedders
to eliminate uses of the native keyword (step 2) and finally remove
support for native keyword across our parsers (step 3).
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/28791
Change-Id: Id6dea878db82dd4fd81149243c425b5c5dc6df86
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212461
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now that support for native extensions (aka dart-ext:*) has been
removed, the kernel loader is guaranteed not to call out to embedder
anymore (it did so before for loading shared libraries).
This allos us to clean up some code as well as extend the scope of our
NoActiveIsolateScope.
TEST=ci
Change-Id: I5c8c9ee47298fdd316583ba3aed74772c1c7783f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212296
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
In the codebase, we have several different interfaces for instance
methods that return a hash:
- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;
This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.
TEST=Existing test suites, as this is an internal refactoring.
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Since loading kernel will affect all isolates within a group, there
should be nothing isolate-specific inside the kernel loader. To ensure
we don't have (or introduce) any accidental uses, we add a
NoActiveIsolateScope to the main parts of kernel loading.
Only loading of native extensions - done by the kernel loader -
currently requires an active isolate. The reason for this is that
loading native extensions happen by calling out to embedder, which calls
back into the VM using the our embedding API (which currently requires
an active isolate)
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing implementation.
Change-Id: I96e64dbfe7148b76b8fa006fe5dbde8c1f904504
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184269
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
When handling hot reload via library tag handler's Kernel tag
VM must guarantee that external typed data it receives from the
embedder stays alive for as long as VM has any structures
referencing this kernel binary. This is achieved by attaching
original typed data to KernelProgramInfo objects containing
the views into it.
Unfortunately kernel::Reader APIs were somewhat unsafe: allowing
to construct reader object from Program's raw buffer and forget to
set the link betwen the reader and original typed data.
This created a situation where reloading using a multicomponent
Kernel binary would result in KernelProgramInfo objects without
link to the original typed data, which in turn leads to premature
finalization of external typed data and subsequent crashes
when trying to use delete kernel binary.
This CL reworks kernel::Reader API (by introducing kernel::ProgramBinary
wrapper) to make it safer and make sure that connection is preserved.
TEST=vm/cc/IsolateReload_RegressB179030011
Bug: b/179030011
Change-Id: I05f8c31c3cb8e67de6e94a20d9501a5f476b7e27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182280
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
As part of making lightweight isolates work in JIT, we will need to
ensure that whenever a new static field is loaded, it will get a new
slot on every isolate and the possibly initial static value is set in
all isolates's field state.
In order to achieve this we first move field registration out of
Field::New*() - to the place where the field is created and the initial
static field value is known.
In a future CL we will make a field registration be performed on all
isolates in a group.
TEST=CL is internal refactoring, existing test coverage.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I802ae55b841be173733dbd0ad3dbffeebd5c1889
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173260
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Loading unit metadata is expanded to create an object representing each loading unit, and each library to reference the loading unit it is a part of.
To be used at compile-time to help assign objects to snapshots.
To be used at runtime to track which units have been loading and the base objects for cross-snapshot references.
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icd3fa95ffd5949bd8f7f84a6f750c4c515de0dba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151865
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Static methods and constructors may receive unboxed parameters.
This change adds a bitmap to the RawFunction object indicating
which parameters are unboxed and their types. Also, the stack frame
bitmap was updated to include new bits for those parameters.
The ArgumentsDescriptor object was updated to store not only the number
of arguments, but also their size, since 32-bit platforms would use two
words for each int64 and double value.
In addition, static methods that return non-nullable integers
and doubles may return these values unboxed.
Dart AOT x64:
TypedData.Uint8ClampedListViewVarBench +11.73%
DeltaBlue +9.32%
Meteor (Intel Xeon) +4.64%
Flutter release android-armv8:
velocity_tracker_bench -2.32% (less is better)
flutter_gallery_readonlydata_size +0.18%
flutter_gallery_total_size +0.13%
flutter_gallery_instructions_size +0.12%
Issue https://github.com/dart-lang/sdk/issues/33549
Change-Id: I97e310f8f30213920af2333fbdedf193d03db5c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/135313
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This paves the way for removing the old constant evaluator from the VM,
by no longer supporting versions without front-end constant evaluation.
For the front-end test that tests old kernel versions on the newest VM,
the dill files for versions before 29 are deleted.
Version 29 is from 2019-08-16: eae781c73e
Change-Id: Icb472bffc764d6e37ed1349a0cb4ff281e0e1b21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123739
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Aske Simon Christensen <askesc@google.com>
This is the beginning of allowing us to - wherever we process kernel
data - make decisions based on the kernel version. Said another way,
we can actually start to change the kernel format without breaking
backwards compatibility in the VM.
Change-Id: Ied7b3d9fe72976c4f3c0895d02b6ab70e0f5dba9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/111642
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Rationale:
Previously all constants were read from the constants table,
whether they were needed (at that point or at all) or not. This
CL replaces that eager reading with a lazy reading that makes
the raw bytes of the constants table available through
KernelProgramInfo and attaches a hashmap to KernelProgramInfo,
which maps offsets into the constant table to evaluated constants.
The maps starts empty and on a miss, the constant is evaluated,
but no sooner than when it it needed.
https://github.com/dart-lang/sdk/issues/36220
Change-Id: Ief4df3d70c950c4beb61d896632ce06cfd12d525
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/99088
Commit-Queue: Aart Bik <ajcbik@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Size of a large application:
Before: 26628600
After: 21480120 (-19.3%)
Size of snapshots:
isolate_snapshot_framework.bin
Before: 9322496
After: 6782976 (-27.2%)
isolate_snapshot_product_framework.bin
Before: 9166848
After: 6602752 (-27.9%)
Regressions in tests:
1) Test language_2/type_alias_equality_test/04 fails similarly to default mode, as VM
does not implement comparison of function types according to the specification.
Previously this test was passing as function types were canonicalized in bytecode,
which was not always correct. This CL fixes the problem with canonicalization of
function types in bytecode and the test starts failing again.
2) Tests standalone_2/entrypoints_verification_test, standalone_2/io/test_extension_test,
standalone_2/io/test_extension_fail_test fail as native extensions are not supported
in bytecode yet. These tests start passing after df5e7aac17,
which switched bytecode tests to kernel service (on x64), because kernel service doesn't
drop ASTs. This CL switches from reading AST library declarations to bytecode even if
AST is not removed, so tests fail again.
Change-Id: I8b7ba44bfa49d0b1599b2509553ff7c831a4e244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104700
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This reverts commit ab6aeaa106.
Revert "[vm/compiler] Speed up the compiler part which deals with kernel reading up in DEBUG mode"
This reverts commit b316210d94.
Reason for revert: regression of snapshot sizes (DNO-599).
Original change's description:
> [vm/kernel] Use GC-tracked ExternalTypedData/TypedDataView for kernel buffers
>
> Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
> objects were referencing ExternalTypedData objects pointing into the middle of
> c-allocated memory. This made it impossible for the GC to determine when the last
> reference is gone.
>
> This CL ensures that the actual buffers are *always* made available via
> ExternalTypedData and any inner pointers into it are created via TypedDataViews.
>
> The embedder guarantees to the free kernel buffers it has provided to:
> - Dart_CreateIsolateFromKernel
> - Dart_LoadScriptFromKernel
> - Dart_LoadLibraryFromKernel
> - Dart_SetDartLibrarySourcesKernel
> on isolate shutdown.
>
> All other kernel buffers will get a finalizer attached, which ensures the
> kernel buffers get freed by the GC once they are no longer referenced:
> - Kernel blobs for expression evaluation
> - Kernel blobs for Hot-Reload
> - Kernel blobs for cc tests
>
> Fixes https://github.com/dart-lang/sdk/issues/33973
> Fixes https://github.com/dart-lang/sdk/issues/36857
> Issue https://github.com/dart-lang/sdk/issues/37030
>
> Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,rmacnak@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I49715d2400f4a5c8806b7d6a2912b7258f671a0a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104343
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Until now we often leaked kernel buffers (e.g. hot reload buffers) because various
objects were referencing ExternalTypedData objects pointing into the middle of
c-allocated memory. This made it impossible for the GC to determine when the last
reference is gone.
This CL ensures that the actual buffers are *always* made available via
ExternalTypedData and any inner pointers into it are created via TypedDataViews.
The embedder guarantees to the free kernel buffers it has provided to:
- Dart_CreateIsolateFromKernel
- Dart_LoadScriptFromKernel
- Dart_LoadLibraryFromKernel
- Dart_SetDartLibrarySourcesKernel
on isolate shutdown.
All other kernel buffers will get a finalizer attached, which ensures the
kernel buffers get freed by the GC once they are no longer referenced:
- Kernel blobs for expression evaluation
- Kernel blobs for Hot-Reload
- Kernel blobs for cc tests
Fixes https://github.com/dart-lang/sdk/issues/33973
Fixes https://github.com/dart-lang/sdk/issues/36857
Issue https://github.com/dart-lang/sdk/issues/37030
Change-Id: I1cc410c94c0f4b229413e793728a261afcb10aaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103130
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This for instance allows the VM to include all patches
(e.g. the previously missing `runtime/lib/date_patch.dart`)
in what's returned from `LoadedScripts()`.
Over time this also allows for `LoadedScripts()` to be simplified and
become faster (it's currently something like O(m + m*n) where m is the
number of results in DictionaryIterator(*this) and n is the number of
returned scripts).
Change-Id: I4b5a6a0fe666b774fc0987d099ed02e81ac97b43
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98660
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
By indexing all sources in a concatenated dill file and then using that
index to find the sources and line starts we can fix missing information
(leading to crashes when collecting coverage) in for instance circular
instances.
This reverts commit 57321c1590 and adds a
fix.
Change-Id: I7a91801fb318cad0218cf3101d6a15f1ec929175
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98006
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This reverts commit a7ab99dd84.
Reason for revert: <INSERT REASONING HERE>
Original change's description:
> [vm/kernel] Index all source and line starts before using them
>
> By indexing all sources in a concatenated dill file and then using that
> index to find the sources and line starts we can fix missing information
> (leading to crashes when collecting coverage) in for instance circular
> instances.
>
> Change-Id: I454cff57f84f21de72f6486e1cecf15d1389a498
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96901
> Commit-Queue: Jens Johansen <jensj@google.com>
> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
TBR=vegorov@google.com,jensj@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Iea75c9dd56e74fa0e98c90ea6841ed66b39d40cc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97941
Reviewed-by: Keerti Parthasarathy <keertip@google.com>
Commit-Queue: Keerti Parthasarathy <keertip@google.com>
This change replaces kernel AST declarations of fields and functions
with bytecode declarations.
Size of dilp files is reduced by 11-12%.
Startup latency:
Time to the first full frame: 1.945s -> 1.687s
FinalizeClass: 554ms -> 277ms
FinishClassLoading: 296ms -> 156ms
There are following regressions in bytecode mode, which will be fixed
in future:
* dart:mirrors are not supported yet (implementation of mirrors relies
on reading kernel AST in certain cases).
As the result, lib_2/mirrors/* tests fail.
* native extensions are not supported yet (annotations on libraries
and classes in AST are cleaned up as they could reference members
which are now removed from AST).
As the result, standalone_2/entrypoints_verification_test test fails.
* language_2/spread_collections/const_error_test/* tests fail
due to https://github.com/dart-lang/sdk/issues/36286.
Change-Id: I5130f401fd7b84038b136136e7ccc1a6e51b6cea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97561
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Daco Harkes