The new implementation is based on suspend/resume stubs and doesn't
use desugaring of async functions on kernel AST.
Previously, new implementation of async/async* was only supported in
AOT mode. This change adds all necessary bits for the JIT mode:
* Suspending variable-length frames (for unoptimized code).
* Handling of Code and pool pointers in Dart stack frames.
* OSR.
* Deoptimization.
* Hot reload.
* Debugger.
The new implementation is not enabled in JIT mode yet.
Design doc: go/compact-async-await.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: I477d6684bdce7cbc1edb179ae2271ff598b7dcc5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/246081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Original CL in patchset 1.
Split-off https://dart-review.googlesource.com/c/sdk/+/238341
And pulled in fix https://dart-review.googlesource.com/c/sdk/+/238582
(Should merge cleanly when this lands later.)
This CL implements the `Finalizer` in the GC.
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: Ibdfeadc16d5d69ade50aae5b9f794284c4c4dbab
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238086
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This reverts commit 7dca34c235.
Reason for revert: b/226085355 dart_vm_test crashing. Unclear what
the cause is. Reverting so we can triage the issue.
TEST=This is a revert.
Original change's description:
> [vm] Implement `Finalizer`
>
> This CL implements the `Finalizer` in the GC.
>
> (This CL does not yet implement `NativeFinalizer`.)
>
> The GC is specially aware of two types of objects for the purposes of
> running finalizers.
>
> 1) `FinalizerEntry`
> 2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
>
> A `FinalizerEntry` contains the `value`, the optional `detach` key, and
> the `token`, and a reference to the `finalizer`.
> An entry only holds on weakly to the value, detach key, and finalizer.
> (Similar to how `WeakReference` only holds on weakly to target).
>
> A `Finalizer` contains all entries, a list of entries of which the value
> is collected, and a reference to the isolate.
>
> When a the value of an entry is GCed, the enry is added over to the
> collected list.
> If any entry is moved to the collected list, a message is sent that
> invokes the finalizer to call the callback on all entries in that list.
>
> When a finalizer is detached by the user, the entry token is set to the
> entry itself and is removed from the all entries set.
> This ensures that if the entry was already moved to the collected list,
> the finalizer is not executed.
>
> To speed up detaching, we use a weak map from detach keys to list of
> entries. This ensures entries can be GCed.
>
> Both the scavenger and marker tasks process finalizer entries in
> parallel.
> Parallel tasks use an atomic exchange on the head of the collected
> entries list, ensuring no entries get lost.
> The mutator thread is guaranteed to be stopped when processing entries.
> This ensures that we do not need barriers for moving entries into the
> finalizers collected list.
> Dart reads and replaces the collected entries list also with an atomic
> exchange, ensuring the GC doesn't run in between a load/store.
>
> When a finalizer gets posted a message to process finalized objects, it
> is being kept alive by the message.
> An alternative design would be to pre-allocate a `WeakReference` in the
> finalizer pointing to the finalizer, and send that itself.
> This would be at the cost of an extra object.
>
> Send and exit is not supported in this CL, support will be added in a
> follow up CL. Trying to send will throw.
>
> Bug: https://github.com/dart-lang/sdk/issues/47777
>
> TEST=runtime/tests/vm/dart/finalizer/*
> TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
> TEST=runtime/vm/object_test.cc
>
> Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,rmacnak@google.com,dacoharkes@google.com
Change-Id: I991f6e49896d18a8d70210cf315d858b462d66c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/47777
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238080
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL implements the `Finalizer` in the GC.
(This CL does not yet implement `NativeFinalizer`.)
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The pragma can be used to annotate functions which should not appear in
stack traces unless --show-invisible-frames is passed to the VM.
TEST=vm/dart{,_2}/invisible_function_pragma_test
Change-Id: I807359146f16b67913697f07416fbe8ce9aaa6e1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233381
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
When loading and compiling a function for expression evaluation
we might reference synthetic class of that function in types
(e.g. receiver type). However, that class is not registered
(so it could be garbage collected) and it has kIllegalCid.
Types constructed using such class are not very useful and
may crash if class is queried (for example if type is finalized).
This change attempts to replace such synthetic classes with
real classes in types constructed during loading.
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48024
Change-Id: I6c5c1b1d20ac2ce4c43e5a7e456d5a8756a43727
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/226603
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
This CL adds support for users defining integers which are mapped to
differing sizes and signedness based on the application binary interface
the Dart VM is running on.
Notable implementation design decisions:
- ABIs are open world, so that adding an ABI to the Dart VM does not
break existing definitions. Thus, we only figure out in the VM that
we're missing a mapping. We throw compile-time errors.
- In AOT, these show up in the precompilation step.
- In JIT, these show up as `_CompileTimeError` at runtime. Note that
these can be caught. So in subsequent compilation steps we need to
ensure that we also throw the same compile-time error.
- We match on the call-sites (streaming_flowgraph_builder) rather than
method bodies (kernel_to_il) of AbiSpecific loads and stores so that
we can compile for the int-size of the call site.
API design decisions:
https://github.com/dart-lang/sdk/issues/42563#issuecomment-981774001
Closes: https://github.com/dart-lang/sdk/issues/42563
TEST=tests/ffi_2/abi_*_test.dart
TEST=tests/ffi/function_*_generated_test.dart
TEST=tests/ffi/vmspecific_static_checks_test.dart
Change-Id: I8c8df36fab939b6fb614c5f1ee8e1bf46b6e9521
Cq-Include-Trybots: luci.dart.try:analyzer-linux-release-try,analyzer-nnbd-linux-release-try,app-kernel-linux-debug-x64-try,benchmark-linux-try,dart-sdk-linux-try,front-end-linux-release-x64-try,front-end-nnbd-linux-release-x64-try,pkg-linux-debug-try,vm-canary-linux-debug-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-fuchsia-release-x64-try,vm-kernel-checked-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-linux-debug-x64c-try,vm-kernel-mac-debug-x64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/221501
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Redirecting factories are never hit in source coverage report because
front-end replaces calls to redirecting factories with calls to their
targets. This change excludes redirecting factories from source
coverage reports.
TEST=vm/cc/SourceReport_Regress95008_RedirectingFactory
Fixes https://github.com/flutter/flutter/issues/95008
Change-Id: I0f6af291f7ee0c042521c92063092f990426b995
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/223600
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
When loading a value of a static final field with trivial initializer
generate a Constant instruction instead of LoadStaticField.
Initializer of a static field is considered trivial if it is null, int,
double, String or bool literal.
TEST=vm/cc/StreamingFlowGraphBuilder_StaticGetFinalFieldWithTrivialInitializer
Fixes https://github.com/dart-lang/sdk/issues/47120
Change-Id: Id2bfc3da8c14376f7f1ef829265cca29a018bad1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212873
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
As part of deprecating support for native extensions we are also
migrating away from legacy VM-specific `native 'name'` syntax
towards metadata based encoding which does not require any special
syntax.
This CL is a step 1 in migration:
- introduces support for `@pragma('vm:external-name', 'name')`
which serves as a direct replacement for `native 'name'`;
- all core libraries and tests are migrated to use the annotation;
Once this CL lands and rolls we will edit internal and external embedders
to eliminate uses of the native keyword (step 2) and finally remove
support for native keyword across our parsers (step 3).
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/28791
Change-Id: Id6dea878db82dd4fd81149243c425b5c5dc6df86
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212461
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Now that support for native extensions (aka dart-ext:*) has been
removed, the kernel loader is guaranteed not to call out to embedder
anymore (it did so before for loading shared libraries).
This allos us to clean up some code as well as extend the scope of our
NoActiveIsolateScope.
TEST=ci
Change-Id: I5c8c9ee47298fdd316583ba3aed74772c1c7783f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212296
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The flag isn't used anywhere in our tests or in embedder code. Turning
it on will result in a VM startup error.
We should therefore remove all uses of the flag and the flag itself.
This is a unmodified reland of
https://dart-review.googlesource.com/c/sdk/+/204500
after some remaining g3 usages have been fixed (the flutter
roll didn't port the GN changes to BUILD changes in g3)
TEST=Existing test suite.
Change-Id: Ic28c9b334a0b04524ee57e2554cc8d713a83fbfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204785
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Previously there were several pieces of information shared between
both FunctionType and Function, mostly in the packed fields, but
named argument names were also kept in both places.
Now the FunctionType is the primary source for this information, with
the Function only keeping the names of positional arguments, which are
discarded in AOT snapshots.
This does mean extra work to access this information via the function
object, but for the most part, this information is only accessed in the
compiler or during dynamic lookups or checks in the runtime.
After adding the count of type parameters to the packed information
in FunctionType, the packed information has been split into two pieces:
one for parameter counts, another for type parameter counts. This
split does not increase the size of UntaggedFunctionType, as there
were 2 bytes available in the existing padding.
Changes on flutter gallery in release mode:
* ARM7 code size: total -0.91%, readonly -0.22%, isolate -4.32%
* ARM7 heap size: total -2.00%
* ARM8 code size: total -0.93%, readonly -0.22%, isolate -4.32%
* ARM8 heap size: total -2.12%
Changes on flutter gallery in release-sizeopt mode:
* ARM7 code size: total -0.24%, readonly -0.08%, isolate -1.49%
* ARM7 heap size: total -0.88%
* ARM8 code size: total -0.26%, readonly -0.11%, isolate -1.49%
* ARM8 heap size: total -1.01%
TEST=Refactoring, so existing tests.
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Change-Id: Ic4d59a7b4acca039a5647f9163e716f6019163f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203241
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
This reverts commit b1f1aee94d.
Reason for revert: Some left-over uses in g3 need to removed first
(some were removed in b/380758599 but apparently there's some
usages left).
Original change's description:
> [vm] Remove --causal-async-stacks flag
>
> The flag isn't used anywhere in our tests or in embedder code. Turning
> it on will result in a VM startup error.
>
> We should therefore remove all uses of the flag and the flag itself.
>
> TEST=Existing test suite.
>
> Change-Id: I19dfba052df7948dfdb379c0610dab67ebbcd12d
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204500
> Reviewed-by: Clement Skau <cskau@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
TBR=kustermann@google.com,cskau@google.com
Change-Id: I03aad46f46153d5ea4ac2fcdd5685d0ef2a0d9af
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204723
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The flag isn't used anywhere in our tests or in embedder code. Turning
it on will result in a VM startup error.
We should therefore remove all uses of the flag and the flag itself.
TEST=Existing test suite.
Change-Id: I19dfba052df7948dfdb379c0610dab67ebbcd12d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204500
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL replaces the KernelConstantsMap (backed by an Array that got
replaced on growing) by an Array that is allocated to be the required
size.
Does not cache the start of the constants table in the constructor
as proposed in [1], because the constants table is null when the
ConstantReader constructor is called in the KernelLoader constructor.
[1] https://dart-review.googlesource.com/c/sdk/+/196925/6..10/runtime/vm/compiler/frontend/constant_reader.cc#b116
Closes: https://github.com/dart-lang/sdk/issues/45903
TEST=vm test suite, including GC tests.
Change-Id: Ie111bedc7eb7fcd627ab700a9906245bc7dd14ce
Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-linux-debug-x64c-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try
Fixed: 45903
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/200871
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Previously constants were referenced via there relative binary offset,
i.e. a constant reference saying 42 meant at byte offset 42 relative to
the start of the constant table in the binary. This was done to be able
to (on the VM side) read the needed constants lazily. It meant, though,
that constants had to be stored in a map, mapping from the byte position
to the constant.
This change adds a level of indirection when needing the lazy reading,
but lets the constant references reference the constant number instead
so that a constant reference saying 42 means constant number 42,
i.e. constants can be stored in a list instead of in a map.
This is done on the dart side, but the VM still stores it in a map.
The level of indirection is a tabel next to the constant table where
each entry has constant size (4 bytes) from which one can read the
relative byte offset into the constant table from the constant number,
thus still being able to read needed constants lazily.
This CL also cleans up a leftover where for instance double constants
had their textual representation saved as a string in the string indexer
(and thus the output dill) even though they were never referenced.
File size changes:
* Platform: increses 7,816 bytes.
* Compile of dart2js (including platform): decreases 71,424 bytes.
Speed changes:
* Adding `UserTag`s to the code and looking at observatories cpu profile
on a `pkg/kernel/test/binary_bench.dart --golem AstFromBinaryLazy`
run of a compile of dart2js reading the constant table has gone
from ~10% to ~5%.
* Doing statistics on
`pkg/kernel/test/binary_bench.dart --raw AstFromBinaryLazy` run of a
compile of dart2js says -6.28169% +/- 4.97269%.
* Golem runs has nothing above the noise level.
It does say "AstFromBinaryLazy (Intel Xeon) -4.006% (0.4 noise)" and
"AstFromBinaryEagerP50 (Intel Core i5) -8.929% (0.6 noise)" though.
TEST=All tests running the VM tests this.
Change-Id: I07ead457527a4477de803ee55ba742f5557413d3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196925
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
This is a reland of 2852408881
Original change's description:
> [vm] Reduce size of Class objects in AOT and PRODUCT modes
>
> This change conditionally removes certain fields from Class objects
> in AOT and PRODUCT modes.
>
> Flutter gallery in release-sizeopt mode:
> Snapshot size -0.47% (arm64), -0.44% (arm)
> Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
>
> On a large Flutter application compiled in --dwarf-stack-traces mode:
> Heap size of Class objects -26%
> Heap size of all snapshot objects -3.6%
>
> TEST=ci
> Issue: https://github.com/dart-lang/sdk/issues/44020
> Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TEST=ci
Change-Id: Ib473a959a2c610b7a0e6f4000b2101e5256c6119
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196103
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Cleans up unused object_store->ffi_struct_class.
Reland 1: Allow importing `dart:ffi` from `dart:core` with
`--enable-ffi=false`.
Reland 2: Allow the new `_Compound` class to extend `NativeType`. (This
got triggered in this CL on the build because of the import
from `dart:core`.)
Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072
TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart
Change-Id: I133278e16bd75cd2bb6234e7ddf042ffa0a54fd6
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195079
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.
Cleans up unused object_store->ffi_struct_class.
Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072
TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart
Change-Id: I9af6d0173db60614091068c218391f73756c135f
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195061
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Introduces the class `_Compound extends NativeType`, for sharing between
`Struct` and `Union`.
Does minimal changes to CFE and IL construction. Follow up refactor CLs
rename everything.
Bug: https://github.com/dart-lang/sdk/issues/38491
tools/test.py ffi ffi_2
TEST=tests/ffi(_2)/(.*)by_value_(*.)_test.dart
Change-Id: I276ceb9249c20bd331c2f8b6ef64e35acb525e9c
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194765
Reviewed-by: Aske Simon Christensen <askesc@google.com>
This CL enables tree shaking of `Struct` sub classes by simulating the
native behavior in Dart code.
We call the struct constructors in let expressions where FFI trampolines
are created which allocate these struct objects in native code. This way
TFA is instructed about the native behavior. The VM recognizes these
constructor calls as dead code and removes them.
For more info see go/dart-ffi-struct-treeshaking.
Closes: https://github.com/dart-lang/sdk/issues/38721
This CL fixes the types on the generated #fromTypedDataBase constructors.
Also, this CL also cleans up the struct naming:
* _addressOf -> _typedDataBase
* _fromPointer -> _fromTypedDataBase
* #pointer -> #typedDataBase
These cleanups are not split into a separate CL to prevent updating
the .expect files multiple times.
Finally, this CL enables running a single transformer test through:
`dart pkg/vm/test/transformations/type_flow/transformer_test.dart name`
TEST=pkg/vm/testcases/transformations/type_flow/transformer/ffi_struct_constructors.dart
TEST=tests/ffi(_2)/function_callbacks_structs_by_value_generated_test.dart
TEST=tests/ffi(_2)/function_structs_by_value_generated_test.dart
Change-Id: I418d0d73bc86b234dfe5b7b04ae726c33d2b8aeb
Cq-Include-Trybots: luci.dart.try:vm-precomp-ffi-qemu-linux-release-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,dart-sdk-linux-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193661
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The kernel loader works without an active isolate. It no longer causes
execution of Dart code. Constants are evaluated in CFE and are simply
"deserialized" by the kernel loader.
Since kernel loader doesn't execute Dart code and also doesn't process
OOB messages at any point, there's no reason to have a [NoReloadScope]
in it.
Formatting language errors seems to be done purely in the VM and should
therefore also require no [NoReloadScope]
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing code.
Change-Id: I80b2888b60a4d7d179988b8606cd7ca0893a2fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187400
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Whether a given isolate allows reloading is really a property of the
thread not the isolate (where it was before) or isolate group (where
it was moved to), so this CL moves it to Thread.
As the thread's stack is going down and up, NoReloadScope's might be
entered or exited, just as with our many other scopes, such as
NoOOBMessageScope/...
Longer term we might need to find a better mechanism to deal with
different OOB messages. For example: Reload as well as OOB messages
are disabled during static field initialization. Though that means *all*
OOB messages cannot reach the isolate during initialization of a static
field (which can do arbitrary amount of work).
This is part of making hot-reload work with isolate groups.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing code.
Change-Id: I9e80c6d4a184c54c0373fef13af3264cc3f27ece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187002
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
As part of implementing hot-reload for isolate groups, we have to ensure
reloads on a group are performed using the same state of incremental
compiler.
This means we cannot use a specific isolate's information (such as it's
main port) for reloading purposes. Instead we use the unqiue isolate
group id when communicating with the kernel service.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Existing test coverage, future test when hot-reload is fully
implemented with isolate groups.
Change-Id: Ifab39cd2ba689c08507dfab4091cd26951ed54e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185320
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Since loading kernel will affect all isolates within a group, there
should be nothing isolate-specific inside the kernel loader. To ensure
we don't have (or introduce) any accidental uses, we add a
NoActiveIsolateScope to the main parts of kernel loading.
Only loading of native extensions - done by the kernel loader -
currently requires an active isolate. The reason for this is that
loading native extensions happen by calling out to embedder, which calls
back into the VM using the our embedding API (which currently requires
an active isolate)
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Refactoring of existing implementation.
Change-Id: I96e64dbfe7148b76b8fa006fe5dbde8c1f904504
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184269
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
When handling hot reload via library tag handler's Kernel tag
VM must guarantee that external typed data it receives from the
embedder stays alive for as long as VM has any structures
referencing this kernel binary. This is achieved by attaching
original typed data to KernelProgramInfo objects containing
the views into it.
Unfortunately kernel::Reader APIs were somewhat unsafe: allowing
to construct reader object from Program's raw buffer and forget to
set the link betwen the reader and original typed data.
This created a situation where reloading using a multicomponent
Kernel binary would result in KernelProgramInfo objects without
link to the original typed data, which in turn leads to premature
finalization of external typed data and subsequent crashes
when trying to use delete kernel binary.
This CL reworks kernel::Reader API (by introducing kernel::ProgramBinary
wrapper) to make it safer and make sure that connection is preserved.
TEST=vm/cc/IsolateReload_RegressB179030011
Bug: b/179030011
Change-Id: I05f8c31c3cb8e67de6e94a20d9501a5f476b7e27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182280
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Alexander Markov