This CL adds @pragma('vm:awaiter-link') in various places in
Stream implementation to facilitate unwinding and expands
async unwinding logic with more information about
Stream internals.
At the same time be more conservative when checking if an
exception thrown from async method handled: failing to unwind
the stack fully creates situations when we incorrectly report
caught exceptions as uncaught, which frustrates users.
To distinguish stream subscriptions with and without error
handlers we add a state bit. Otherwise, it looks like all
subscriptions have error handlers because if no error
handler is installed we eagerly install error handler forwarding
the error to `Zone.handleUncaughtError`.
Fixes https://github.com/dart-lang/sdk/issues/53334
Fixes https://github.com/dart-lang/sdk/issues/54788
Fixes https://github.com/dart-lang/sdk/issues/47985
TEST=runtime/vm/dart/awaiter_stacks/stream_methods_test.dart,pkg/vm_service/test/pause_on_unhandled_async_exceptions6_test.dart,pkg/vm_service/test/pause_on_unhandled_async_exceptions7_test.dart
CoreLibraryReviewExempt: No behavioral change. Async changes reviewed by lrhn@
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-dwarf-linux-product-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try
Change-Id: Ic51f926867092dd0adbe801b753f57c357c7ace2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322720
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Our async unwinding code used to only respect `catchError`
which occured between async frames and ignored trailing
`catchError` like in the code below
Future(...).catchError((_) { /* handle error */ });
This CL also simplifies how we communicate the presence of the
exception handler to the debugger: the code in the debugger did
not actually care about which frame catches the error (for async
handlers), so we don't need to precisely mark async gaps with
`has_catch_error` flag. Instead we have a single boolean
produced by unwinding which signals whether there we encountered
an asynchronous error handler or not.
Fixes https://github.com/flutter/flutter/issues/141882
TEST=service/pause_on_unhandled_async_exceptions5
Change-Id: Id6f6a97ee5444c197b2c621f68d1e47082fc8997
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350320
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
The main contribution of this CL is unification of disparate
handling of various functions like `Future.timeout`,
`Future.wait`, `_SuspendState.createAsyncCallbacks` and
`_SuspendState._createAsyncStarCallback` into a single
`@pragma('vm:awaiter-link')` which allows Dart developers
to specify where awaiter unwinder should look for the next
awaiter.
For example this allows unwinding to succeed for the code like this:
Future<int> outer(Future<int> inner) {
@pragma('vm:awaiter-link')
final completer = Completer<int>();
inner.then((v) => completer.complete(v));
return completer.future;
}
This refactoring also ensures that we preserve information
(including Function & Code objects) required for awaiter
unwinding across all modes (JIT, AOT and AOT with DWARF stack
traces). This guarantees users will get the same information
no matter which mode they are running in. Previously
we have been disabling awaiter_stacks tests in some AOT
modes - which led to regressions in the quality of produced
stacks.
This CL also cleans up relationship between debugger and awaiter
stack returned by StackTrace.current - which makes stack trace
displayed by debugger (used for stepping out and determinining
whether exception is caught or not) and `StackTrace.current`
consistent.
Finally we make one user visible change to the stack trace:
awaiter stack will no always include intermediate listeners
created through `Future.then`. Previously we would sometimes
include these listeners at the tail of the stack trace,
which was inconsistent.
Ultimately this means that code like this:
Future<int> inner() async {
await null; // asynchronous gap
print(StackTrace.current); // (*)
return 0;
}
Future<int> outer() async {
int process(int v) {
return v + 1;
}
return await inner().then(process);
}
void main() async {
await outer();
}
Produces stack trace like this:
inner
<asynchronous suspension>
outer.process
<asynchronous suspension>
outer
<asynchronous suspension>
main
<asynchronous suspension>
And when stepping out of `inner` execution will stop at `outer.process`
first and the next step out will bring execution to `outer` next.
Fixes https://github.com/dart-lang/sdk/issues/52797
Fixes https://github.com/dart-lang/sdk/issues/52203
Issue https://github.com/dart-lang/sdk/issues/47985
TEST=ci
Bug: b/279929839
CoreLibraryReviewExempt: CL just adds @pragma to facilitate unwinding
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-dwarf-linux-product-x64-try
Change-Id: If377d5329d6a11c86effb9369dc603a7ae616fe7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311680
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
During 'await', when registering 'then' and 'error' callbacks in
a custom Zone, the stack trace should be collected synchronously
as there are no awaiters yet, but the suspended function and its
callers are still on the stack.
This change corrects the new implementation of async to collect
these stack traces synchronously.
Class StackZoneSpecification from package:stack_trace relies on the
stack traces during callback registration in order to provide more
detailed chain of async stack traces via Chain.capture.
Without this change package:stack_trace captured truncated Chains.
Issue: https://github.com/dart-lang/sdk/issues/48378
Issue: https://github.com/flutter/flutter/issues/105340
TEST=vm/dart/causal_stacks/zone_callback_stack_traces_test
Change-Id: Iee348aa5c9abb4126f6c5d8215a6dc2cee57e124
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/247620
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The new implementation moves away from desugaring of async
functions on kernel AST, state machine generated in the flow graph and
capturing all local variables in the context.
Instead, async/await is implemented using a few stubs
(InitSuspendableFunction, Suspend, Resume, Return and
AsyncExceptionHandler). The stubs are implemented in a
platform-independent way using (macro-)assembler helpers.
When suspending a function, its frame is copied into a SuspendState
object, and when resuming a function it is copied back onto the stack.
No extra code is generated for accessing local variables.
Callback closures are created lazily on the first await.
Design doc: go/compact-async-await.
Part 1 (kernel): https://dart-review.googlesource.com/c/sdk/+/241842
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/48378
Change-Id: Ibad757035b7cc438ebdff80b460728b1d3eff1f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242000
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
This fixes an issue where VM would run the async* generator after a
`yield` / `yield*` even though the subscription may be paused or
cancelled.
Furthermore this fixes an issue where `StackTrace.current` used
in async* generator crashes VM and/or produces truncated stack
trace.
This fixes the following existing tests that were failing before:
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t08
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t09
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t10
* language/async_star/async_star_cancel_test
* language/async_star/pause_test
New in reland: Allow the generator to to cause cancelling it's own consumer.
This addresses the issue of original revert
-> see https://github.com/flutter/flutter/issues/101514
Issue https://github.com/flutter/flutter/issues/100441
Issue https://github.com/dart-lang/sdk/issues/48695
Issue https://github.com/dart-lang/sdk/issues/34775
TEST=vm/dart{,_2}/causal_stacks/flutter_regress_100441_test
Change-Id: I091b7159d59ea15fc31162b4b6b17260d523d7cb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242400
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This reverts commit 837ee17b43.
Reason for revert: Please see https://github.com/flutter/flutter/issues/101514
Original change's description:
> [vm] Fix some async* semantics issues: Only run generator if there's active subscription (not paused/cancelled)
>
> This fixes an issue where VM would run the async* generator after a
> `yield` / `yield*` even though the subscription may be paused or
> cancelled.
>
> Furthermore this fixes an issue where `StackTrace.current` used
> in async* generator crashes VM and/or produces truncated stack
> trace.
>
> This fixes the following existing tests that were failing before:
>
> * co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t08
> * co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t09
> * co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t10
> * language/async_star/async_star_cancel_test
> * language/async_star/pause_test
>
> Issue https://github.com/flutter/flutter/issues/100441
> Issue https://github.com/dart-lang/sdk/issues/48695
> Issue https://github.com/dart-lang/sdk/issues/34775
>
> TEST=vm/dart{,_2}/causal_stacks/flutter_regress_100441_test
>
> Change-Id: I73f7d0b70937a3e3766b992740fa6fe6e6d57cec
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239421
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Commit-Queue: Martin Kustermann <kustermann@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Ic3d9c0508310a33a2aaee67860c0bb2ec83bab4a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/240506
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This fixes an issue where VM would run the async* generator after a
`yield` / `yield*` even though the subscription may be paused or
cancelled.
Furthermore this fixes an issue where `StackTrace.current` used
in async* generator crashes VM and/or produces truncated stack
trace.
This fixes the following existing tests that were failing before:
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t08
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t09
* co19/Language/Statements/Yield_and_Yield_Each/Yield_Each/execution_async_t10
* language/async_star/async_star_cancel_test
* language/async_star/pause_test
Issue https://github.com/flutter/flutter/issues/100441
Issue https://github.com/dart-lang/sdk/issues/48695
Issue https://github.com/dart-lang/sdk/issues/34775
TEST=vm/dart{,_2}/causal_stacks/flutter_regress_100441_test
Change-Id: I73f7d0b70937a3e3766b992740fa6fe6e6d57cec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239421
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This CL
* Makes :async_op only have two parameters: We take advantage
of the fact that errors not only have exception != null but
also stacktrace != null.
* Makes :async_op have no optional parameters. This reduces
code size significantly.
* Removes unused parameter in _awaitHelper calls
* Wrap the then callback instead of the error callback. (needed
to make optional parameters required)
This results in 2% code savings on a big g3 app.
The size of :async_op shrinks on average by 11%
TEST=ci
Change-Id: I38d5fba4ebebc780b48dac5aa6a250d2c7952bfd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233362
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
This is a reland of b6dc4dad4d
TEST=ci
Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
> fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
> now avoids using Code objects.
> In order to find CompressedStackMaps it now calls
> ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
> a replacement for Code objects in stack traces. It has
> PayloadStart() == 0 and Size() == kUwordMax so it includes
> arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
> frames are replaced with StubCode::UnknownDartCode().
> This simulates future behavior of ReversePc::Lookup when Code objects
> will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: Ia2fc4672a085cd963b7fc103369851df3603590c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186202
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This reverts commit b6dc4dad4d.
Reason for revert: broke package:vm_snapshot_analysis in Flutter
(https://github.com/flutter/flutter/issues/76313).
Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
> fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
> now avoids using Code objects.
> In order to find CompressedStackMaps it now calls
> ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
> a replacement for Code objects in stack traces. It has
> PayloadStart() == 0 and Size() == kUwordMax so it includes
> arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
> frames are replaced with StubCode::UnknownDartCode().
> This simulates future behavior of ReversePc::Lookup when Code objects
> will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I6f66171eecf1133363a7ce56193e782e43a20baf
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185488
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The following changes are done in preparation for the removal of Code
objects in AOT with --dwarf-stack-traces:
* Stack trace objects are extended to hold uword PCs (which may not
fit into Smi range).
* Scanning stack frames in GC (StackFrame::VisitObjectPointers)
now avoids using Code objects.
In order to find CompressedStackMaps it now calls
ReversePc::FindCompressedStackMaps.
* Singleton Code object (StubCode::UnknownDartCode()) is prepared as
a replacement for Code objects in stack traces. It has
PayloadStart() == 0 and Size() == kUwordMax so it includes
arbitrary PCs.
* In --dwarf-stack-traces mode, most Code objects obtained from stack
frames are replaced with StubCode::UnknownDartCode().
This simulates future behavior of ReversePc::Lookup when Code objects
will be removed.
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
All existing embedders have been opted into --lazy-async-stacks, the
VM also uses it as it's default in all configurations.
After this CL, any user of --causal-async-stacks will get an error
message when trying to use it.
=> In any such case, please simply remove the flag.
TEST=Exhaustive CQ.
Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ia440afcf2dba464aa8b8cf381b93bbac8eb9f8dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172564
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
In particular, don't serialize a function if its code is used by the
dispatch table, but there's no need for the function object itself
at runtime like dynamic lookup or dynamic invocation forwarders.
However, just because the owning function is not needed at runtime,
it may be used during serialization for things like assembly label
creation. Thus, instead of just clearing out the Code's owner field,
we add a new WeakSerializationReference (WSR) object whose target is
the original contents of the owner field. The WSR allows the
serializer to access the original owner, but also signals to the
serializer that the target should not be serialized in AOT snapshots
unless there are additional strong (non-WSR) references.
If no strong references are found, then the references to the WSR
in the snapshot are replaced with a reference to a WSR that only
contains the class ID of the original target. The serializer creates
only one WSR per class ID.
If strong references are found, then the target is still serialized.
In this case, the WSR is dropped entirely and any reference to it is
replaced with a direct reference to the serialized target. Thus, WSRs
only exist in the precompiled runtime for targets with no strong
references.
Changes on the Flutter gallery in release mode:
arm7: total size -1.10%, isolate size -6.08%
arm8: total size -1.16%, isolate size -6.09%
Bug: https://github.com/dart-lang/sdk/issues/41052
Change-Id: I5b435ca71ef85f50fe3484789087471a91aa4fe2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137104
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Running all existing tests with/without lazy async stacks ensures that
the existing functionality for --lazy-async-stacks does not regress.
There are still a few smaller things to be done in the debugger for lazy
async stacks.
This CL also ensures:
* The inner closure, i.e. AsyncClosure/AsyncGenClosure, is not
inlined if FLAG_lazy_async_stacks is on.
* Fixes a crash in dartkb mode, when the function of the Bytecode
object is null.
* Does a simple integration of the lazy async stacks in debugger.cc -
to ensure a stack is returned via vm-service.
Issue https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ibc1e887a457e2c456ae65d9ed5fa92434f122a32
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131825
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
This is done by collecting all frames on the current stack until an async/async* frame is hit
which has yielded before (i.e. is not in sync-async case).
From there on it finds the closure of the async/async* frame and starts traversing the listeners:
while (closure != null) {
yield_index = closure.context[Context::kAsyncJumpVarIndex]
pc = closure.function.code.pc_descriptors.LookupPcFromYieldIndex(yield_index);
<emit pc in frame>
closure = closure.context[Context::kAsyncCompleterVarIndex]._future._resultOrListeners.callback;
}
Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: I97030d22e529bf8a74ecd30a8a2589cfe57ad330
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122644
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Also fix stack trace collection to always include invisible frames. It can happen that a visible function is inlined into an invisible function, and we don't expand inlined frames until we print a stack trace.
dart2js product x64:
compile time: 12.459s
VMIsolate(CodeSize): 152292
Isolate(CodeSize): 3343117
ReadOnlyData(CodeSize): 3728928
Instructions(CodeSize): 8677600
Total(CodeSize): 15901937
->
compile time: 14.195s (+13%)
VMIsolate(CodeSize): 174034
Isolate(CodeSize): 3892418 (+16%)
ReadOnlyData(CodeSize): 5036320 (+35%)
Instructions(CodeSize): 8682624
Total(CodeSize): 17785396 (+12%)
R=asiva@google.com
Review-Url: https://codereview.chromium.org/2687143005 .
This CL improves the stack traces that accompany exceptions. Whenever an
async function is entered, we remember how we got there. This is similar
in spirit to package:stack_trace but the implementation is more efficient
and memory usage can be more easily reasoned about.
Tracking causal stack traces:
- [x] Upon entry to an async function, capture the synchronous stack trace prefix and store it into the closure.
- [x] Upon entry to an async* function, capture the synchronous stack trace prefix and store it into the closure.
- [x] Before returning from an async function, clear the Thread's asynchronous stack trace.
- [x] After resuming an async function, load the sychronous stack trace prefix into the Thread.
- [x] Filter stack traces to remove async machinery.
Service protocol changes:
- [x] Send causal async stack trace.
Observatory changes:
- [x] Display causal async stack trace below async functions.
Fixes https://github.com/dart-lang/sdk/issues/27661R=asiva@google.com, rmacnak@google.com
Comparisons: https://docs.google.com/a/google.com/document/d/10r6jEqr8OCiDZ4y9SYU_uOimcHiOGAZMly2ghTErALI/edit?usp=sharing
Review-Url: https://codereview.chromium.org/2646443005 .
Alexander Markov