The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This fixes an issue where VM would run the async* generator after a
`yield` / `yield*` even though the subscription may be paused or
cancelled.
Furthermore this fixes an issue where `StackTrace.current` used
in async* generator crashes VM and/or produces truncated stack
trace.
This fixes the following existing tests that were failing before:
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t08
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t09
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t10
* language/async_star/async_star_cancel_test
* language/async_star/pause_test
New in reland: Allow the generator to to cause cancelling it's own consumer.
This addresses the issue of original revert
-> see https://github.com/flutter/flutter/issues/101514
Issue https://github.com/flutter/flutter/issues/100441
Issue https://github.com/dart-lang/sdk/issues/48695
Issue https://github.com/dart-lang/sdk/issues/34775
TEST=vm/dart{,_2}/causal_stacks/flutter_regress_100441_test
Change-Id: I091b7159d59ea15fc31162b4b6b17260d523d7cb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242400
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This reverts commit 837ee17b43.
Reason for revert: Please see https://github.com/flutter/flutter/issues/101514
Original change's description:
> [vm] Fix some async* semantics issues: Only run generator if there's active subscription (not paused/cancelled)
>
> This fixes an issue where VM would run the async* generator after a
> `yield` / `yield*` even though the subscription may be paused or
> cancelled.
>
> Furthermore this fixes an issue where `StackTrace.current` used
> in async* generator crashes VM and/or produces truncated stack
> trace.
>
> This fixes the following existing tests that were failing before:
>
> * co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t08
> * co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t09
> * co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t10
> * language/async_star/async_star_cancel_test
> * language/async_star/pause_test
>
> Issue https://github.com/flutter/flutter/issues/100441
> Issue https://github.com/dart-lang/sdk/issues/48695
> Issue https://github.com/dart-lang/sdk/issues/34775
>
> TEST=vm/dart{,_2}/causal_stacks/flutter_regress_100441_test
>
> Change-Id: I73f7d0b70937a3e3766b992740fa6fe6e6d57cec
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239421
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Ic3d9c0508310a33a2aaee67860c0bb2ec83bab4a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/240506
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This fixes an issue where VM would run the async* generator after a
`yield` / `yield*` even though the subscription may be paused or
cancelled.
Furthermore this fixes an issue where `StackTrace.current` used
in async* generator crashes VM and/or produces truncated stack
trace.
This fixes the following existing tests that were failing before:
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t08
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t09
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t10
* language/async_star/async_star_cancel_test
* language/async_star/pause_test
Issue https://github.com/flutter/flutter/issues/100441
Issue https://github.com/dart-lang/sdk/issues/48695
Issue https://github.com/dart-lang/sdk/issues/34775
TEST=vm/dart{,_2}/causal_stacks/flutter_regress_100441_test
Change-Id: I73f7d0b70937a3e3766b992740fa6fe6e6d57cec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239421
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL
* Makes :async_op only have two parameters: We take advantage
of the fact that errors not only have exception != null but
also stacktrace != null.
* Makes :async_op have no optional parameters. This reduces
code size significantly.
* Removes unused parameter in _awaitHelper calls
* Wrap the then callback instead of the error callback. (needed
to make optional parameters required)
This results in 2% code savings on a big g3 app.
The size of :async_op shrinks on average by 11%
TEST=ci
Change-Id: I38d5fba4ebebc780b48dac5aa6a250d2c7952bfd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233362
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
If the _FutureListener.handleValue function gets deoptimized at a point
when the `this` is no longer needed/live the deoptimization procedure
may override the `this` slot in the caller frame with "<optimized out>".
This has been triggered on the "iso-stress" builder due to using VM
testing flags that cause many deoptimizations in many places.
The stack walking code should take this possibility into consideration
and be therefore conservative.
Alternatives considered:
* Use "copy parameters" into our own frame approach - like we do
with methods that have optional parameters
=> This will make all unoptimized code slower.
* Add a reachability fence to the _FutureListener.handleValue function
=> The reachabilityFench is only available in dart:_internal for VM
=> If the function could be debugged after fence but before return
one might be able to trigger the issue nonetheless.
Closes https://github.com/dart-lang/sdk/issues/46823
TEST=Should remove flaky crashes on "iso-stress" builder.
Change-Id: I9d953706b292e02d2ba75fd794c872865faf45d0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/227401
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Previously VM had a hardcoded constant values for context indices of
_future variables in _Future.timeout and Future.wait (used while
collecting stack traces). As those variables are allocated in
the context after captured parameters, their indices depend on
whether the parameters are captured.
I'd like to improve tree shaking so that it will remove
part of Future.wait method and make one of the parameters
not captured, which changes the context index (and it may vary
between modes and for different programs).
So, instead of hardcoding those indices, they are now calculated
at compile time and stored in ObjectStore. This is more flexible
and accommodates for future changes in tree shaker.
TEST=existing tests
Change-Id: I47629b41e497843fe3e998e813e793cf77c18bdd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/207202
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Creates HasParent() as a predicate for Function objects with a non-null
parent function and makes IsLocalFunction() specific to local closures
(i.e. explicit closures with non-null parent functions).
Creates HasSavedArgumentsDescriptor() as a predicate to determine when
saved_args_desc() can be used, to avoid the caller having to do explicit
predicate checks for each dispatcher type that contains one.
Also simplifies Function::PrintName, creates different suffixes for a
SyncGenClosureMaker and its associateed SyncGenClosure, adds suffixes
for skipping more than one generated body when printing the parent name,
and changes ArgumentsDescriptor::PrintTo to match the compact syntax
Function::PrintName used for signifying the arguments descriptor for
certain dispatchers.
TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test
Cq-Include-Trybots: luci.dart.try:app-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,pkg-linux-release-try
Change-Id: I0ac1226d1ffe1d81743b5c6788da170d5a4010c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191560
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This is a reland of b6dc4dad4d
TEST=ci
Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
> fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
> now avoids using Code objects.
> In order to find CompressedStackMaps it now calls
> ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
> a replacement for Code objects in stack traces. It has
> PayloadStart() == 0 and Size() == kUwordMax so it includes
> arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
> frames are replaced with StubCode::UnknownDartCode().
> This simulates future behavior of ReversePc::Lookup when Code objects
> will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Ia2fc4672a085cd963b7fc103369851df3603590c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186202
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This reverts commit b6dc4dad4d.
Reason for revert: broke package:vm_snapshot_analysis in Flutter
(https://github.com/flutter/flutter/issues/76313).
Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
> fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
> now avoids using Code objects.
> In order to find CompressedStackMaps it now calls
> ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
> a replacement for Code objects in stack traces. It has
> PayloadStart() == 0 and Size() == kUwordMax so it includes
> arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
> frames are replaced with StubCode::UnknownDartCode().
> This simulates future behavior of ReversePc::Lookup when Code objects
> will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I6f66171eecf1133363a7ce56193e782e43a20baf
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185488
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The following changes are done in preparation for the removal of Code
objects in AOT with --dwarf-stack-traces:
* Stack trace objects are extended to hold uword PCs (which may not
fit into Smi range).
* Scanning stack frames in GC (StackFrame::VisitObjectPointers)
now avoids using Code objects.
In order to find CompressedStackMaps it now calls
ReversePc::FindCompressedStackMaps.
* Singleton Code object (StubCode::UnknownDartCode()) is prepared as
a replacement for Code objects in stack traces. It has
PayloadStart() == 0 and Size() == kUwordMax so it includes
arbitrary PCs.
* In --dwarf-stack-traces mode, most Code objects obtained from stack
frames are replaced with StubCode::UnknownDartCode().
This simulates future behavior of ReversePc::Lookup when Code objects
will be removed.
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Currently we have things called XPtr which are not what you get from ptr().
Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)
After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*
New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*
TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
All existing embedders have been opted into --lazy-async-stacks, the
VM also uses it as it's default in all configurations.
After this CL, any user of --causal-async-stacks will get an error
message when trying to use it.
=> In any such case, please simply remove the flag.
TEST=Exhaustive CQ.
Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ia440afcf2dba464aa8b8cf381b93bbac8eb9f8dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172564
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This adds a debugger mechanism to replace part of
--causal-async-stack's _asyncStackTraceHelper runtime entry.
It recognises async and async* functions and adds a synthetic
breakpoint on entry into the wrapped async_op, making sure we
don't get "synthetic" frames as we're stepping in and out of
the async code.
Change-Id: I1df6e6874de2fa9185f27a1a8873ad0071ad9fb6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157480
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
- Makes Future.wait a recognised function, and asserts its chained
future, _future is allocated at a known index in the context.
- Adds logic to locate, extract the chained future during lazy async
stack unwinding.
- Adds tests for the Future.wait async case.
- Minor consistency nits, comments.
This change is similar to a previous CL, adding Future.timeout support:
https://dart-review.googlesource.com/c/sdk/+/152328
Change-Id: I7439750968595d25d7bbac0068ad64fcc891e176
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155420
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Symbolic and non-symbolic stack traces had separate ToCString
implementations with the same general structure, but those
implementations diverged enough to cause issues in certain async stack
modes. They are now merged, with the --dwarf-stack-traces-mode flag
checked where appropriate within the merged method.
Also, now non-symbolic stack traces do not include frames for Code with
invisible Function owners unless the flag --show-invisible-frames is
enabled. With this change, pkg/native_stack_traces no longer needs to
guess at which frames corresponding to Dart code should be treated as
internal.
Tested by adding --dwarf-stack-traces version of the tests
in runtime/tests/vm/dart/causal_stacks.
Fixes https://github.com/dart-lang/sdk/issues/41578
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Change-Id: I41a887129616c88acd7729492addf7364d95df33
Bug: 41578
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143816
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Make them form its own source set (libdart_compiler) and completely exclude
them from AOT runtime targets.
Previously we had some inconsistencies, with some files were using
DART_PRECOMPILED_RUNTIME to fully or partially exclude either contents
from their headers or from implementation, while other files did nothing
and relied on linker to throw their contents away.
This change tries to address this inconsistency.
A follow up change would include a check in most compiler headers which
would prohibit to use them while building AOT runtime.
Change-Id: Ief11b11cbc518b301d3e93fce80580a31bbad151
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142993
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
In particular, don't serialize a function if its code is used by the
dispatch table, but there's no need for the function object itself
at runtime like dynamic lookup or dynamic invocation forwarders.
However, just because the owning function is not needed at runtime,
it may be used during serialization for things like assembly label
creation. Thus, instead of just clearing out the Code's owner field,
we add a new WeakSerializationReference (WSR) object whose target is
the original contents of the owner field. The WSR allows the
serializer to access the original owner, but also signals to the
serializer that the target should not be serialized in AOT snapshots
unless there are additional strong (non-WSR) references.
If no strong references are found, then the references to the WSR
in the snapshot are replaced with a reference to a WSR that only
contains the class ID of the original target. The serializer creates
only one WSR per class ID.
If strong references are found, then the target is still serialized.
In this case, the WSR is dropped entirely and any reference to it is
replaced with a direct reference to the serialized target. Thus, WSRs
only exist in the precompiled runtime for targets with no strong
references.
Changes on the Flutter gallery in release mode:
arm7: total size -1.10%, isolate size -6.08%
arm8: total size -1.16%, isolate size -6.09%
Bug: https://github.com/dart-lang/sdk/issues/41052
Change-Id: I5b435ca71ef85f50fe3484789087471a91aa4fe2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137104
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
- Handle bytecode for e.g. dartkb-simarm64 does not have source position.
- Use existing GetCallerSp() instead of working across frames.
- Nit: Adds clarifying comments to test.
- Nit: Updates name of non-async-stack test to clarify flags used.
Tested:
- CQ with --lazy-async-stacks on by default.
- vm/dart/causal_stacks and language_2/vm/causal_async_exception_stack_test with current flags.
Bug: https://github.com/dart-lang/sdk/issues/39525
Change-Id: Ie6581a734cdcafbd4fb641bd86bffc03ed241532
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/133063
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
For `foo() async* {}` frames we find the "caller" by finding out
what closure is registered as listener on the _AsyncStreamController.
There can be two cases:
a) The caller does a regular `foo().listen((_) {})`:
The stack trace will have the closure as the caller and unwinding stops.
b) The caller uses 'await for (... foo())':
In this case the listener will be a StreamIterator.
This CL changes our unwinding code to get the awaiter of `await it.moveNext()`.
Bug: https://github.com/dart-lang/sdk/issues/39525
Change-Id: I13f76025a15682aaf55fd968088fc2059982d842
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132841
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Running all existing tests with/without lazy async stacks ensures that
the existing functionality for --lazy-async-stacks does not regress.
There are still a few smaller things to be done in the debugger for lazy
async stacks.
This CL also ensures:
* The inner closure, i.e. AsyncClosure/AsyncGenClosure, is not
inlined if FLAG_lazy_async_stacks is on.
* Fixes a crash in dartkb mode, when the function of the Bytecode
object is null.
* Does a simple integration of the lazy async stacks in debugger.cc -
to ensure a stack is returned via vm-service.
Issue https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ibc1e887a457e2c456ae65d9ed5fa92434f122a32
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131825
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
This is done by collecting all frames on the current stack until an async/async* frame is hit
which has yielded before (i.e. is not in sync-async case).
From there on it finds the closure of the async/async* frame and starts traversing the listeners:
while (closure != null) {
yield_index = closure.context[Context::kAsyncJumpVarIndex]
pc = closure.function.code.pc_descriptors.LookupPcFromYieldIndex(yield_index);
<emit pc in frame>
closure = closure.context[Context::kAsyncCompleterVarIndex]._future._resultOrListeners.callback;
}
Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: I97030d22e529bf8a74ecd30a8a2589cfe57ad330
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122644
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This fixes service/pause_on_unhandled_async_exceptions_test in compiled bytecode
mode, but not yet in interpreted bytecode mode.
Improve debugger verbosity.
Change-Id: I89e32179b0a3f043716d5117f958bf6a0de3ccac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116483
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Alexander Markov