TEST=Checked that Observatory's timeline view was still able to load CPU
samples correctly, and ensured that samples retrived using
getPerfettoCpuSamples were the same as the ones shown in Observatory.
Change-Id: I9b58cd32bc9a1c08848718f25f10db9fa6d4d241
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297760
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
TEST=Checked that the timeline in DevTools still looked correct, and
checked that no assertions failed when fetching the timeline
Change-Id: Ia7ed4b8faf4e7c9e8871ecca3111a82e7f298318
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/300420
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Instead of writing errno into stderr via perror include it into
FATAL message. This will help us to dignoze the problem in
environments where FATAL messages are preserved as part of the
crash report.
TEST=tested manually by changing write() parameters to be invalid
Bug: b/279184589
Change-Id: I45424af6064f9eec258a3bff1f96488b8a60f095
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/298862
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
This CL also adds the `dart_support_perfetto` GN setting and enables it
by default, and adds a test to ensure that TSAN checks for races related
to reporting async events.
TEST=TSAN, ASAN
Change-Id: I5f43ceca0185287ca4d8e295a509088ea50fd9c3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299740
Reviewed-by: Ben Konyi <bkonyi@google.com>
output unless there is an assertion or Fatal error.
(This is needed to ensure that we are able to run the flutter tests with
all these verifications turned on and not disturbing the output from the
tests with these extraneous trace messages)
TEST=ci
Change-Id: Ifb7211e340f1cf29f6d2bff412eb4107f9cb64a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299440
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
After recursive types are gone, finalization of types can be
simplified: types can be finalized with a simple recursive traversal
and they no longer need to track "being finalized" state.
Also, finalization doesn't need to write back finalized types
because type finalization without canonicalization is always performed
in place and doesn't result in a new type (or type arguments vector).
TEST=ci
Change-Id: Ifc0a4e12aa410eb2be18c4a475c2f486cfa9ebf2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299300
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
For class type parameters 'parameterized_class_id' duplicates
a recently added 'owner' and can be removed.
'owner' is changed to hold class reference as a class id (Smi).
TEST=ci
Change-Id: I93fc11e6bcfcc00058a5281b497f59c5c5847ea6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/299160
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
TypeRef type wraps around another type and it was used to represent
a graph of recursive types. After [0], the only use of TypeRef is
for TypeParameter.bound which may indirectly reference the same
TypeParameter.
This change replaces TypeParameter.bound with TypeParameter.owner and
removes TypeRef entirely. Various parts of the VM no longer need to
handle and support TypeRefs.
TypeParameter.owner can reference a FunctionType, Class,
or, as an optimization, it can be set to null in order to share
class type parameters among different classes.
With the exception of the 'TypeParameter.owner' back pointer,
VM types are now not recursive and can be visited without
additional tracking.
Caveats:
* Generic FunctionType cannot be cloned in a shallow way:
when copying a FunctionType, type parameters should be cloned too
and their owners should be updated. For that reason,
a mapping between 'from' and 'to' function types
(FunctionTypeMapping) is maintained during type transformations
such as InstantiateFrom.
FunctionType::Clone is used instead of Object::Clone where
appropriate.
* When testing types for subtyping and equivalence, mapping
between function types is passed to make sure
type parameters belong to the equivalent function types.
* IL serializer needs to serialize function types as a whole before
serializing any types potentially pointing into the middle of a
function type (such as return type 'List<Y0>' pointing into
the middle of a function type 'List<Y0> Function<Y0>()').
[0] https://dart-review.googlesource.com/c/sdk/+/296300
TEST=ci
Change-Id: I67c2fd0117c6183a45e183919a7847fd1af70b3e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294165
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Moving the class name of the allocated object to the allocation callback
gives more flexibility to the embedder, which may decide to track all
sampled allocations, not just those that are still alive.
TEST=Updated DartAPI_HeapSampling_*
Change-Id: Iaae290e15b67ca3e47d72e1896fa6e32b2d4b081
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297761
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
This is being done to allow us to access the utils from
profiler_service.cc.
TEST=Requested a trace using the getPerfettoVMTimeline RPC and confirmed
that it still looked correct.
Change-Id: Icfb7a5b41da0fc987a72098c4345e7e108b6566e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297740
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
This CL replaces https://dart-review.googlesource.com/c/sdk/+/296900
The `List` constructor is removed in Dart 3.0.
Some of the `@patch` implementations were not removed.
This is *high priority*. It seems the left-over `@patch factory List` constructor did not cause any errors, instead it *added* a constructor to `List` that can be used in web compiled code. Even if `List` doesn't have such a constructor in the SDK code proper.
The VM and analyzer will say the invocation is an error, but dart2js happily compiles it and runs.
(It used to be that patches couldn't add public members, that security seems to have been removed.)
Also removes code which tries to detect "the unnamed List constructor",
which is no longer a thing, and a number of invocations of the constructor, where it's not clear that the test is aware that the constructor no longer exists, and is not marked as `@dart=2.x` with x < 12.
TEST=ci
Change-Id: I4ffaf3ae2c4e75ca06e7ba0bf19187b6376f3888
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297100
Reviewed-by: Brian Quinlan <bquinlan@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Stephen Adams <sra@google.com>
Before the calling convention change during the introduction of AOT, this was used to get code metadata associated with a frame. Its remaining non-test use is forming service IDs for code object, which can use the regular visitors.
TEST=ci
Change-Id: Ica8eafe988d19eba8d905d9dc5907afbfd559118
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296705
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
The current hot-reload implementation [0] will perform a reload by
first sending OOB messages to all isolates and waiting until those OOB
messages are being handled. The handler of the OOB message will block
the thread (and unschedule isolate) and notify the thread performing
reload it's ready.
This requires that all isolates within a group can actually run & block.
This is the case for the VM implementation of isolates (as they are
run an unlimited size thread pool).
Though flutter seems to multiplex several engine isolates on the same OS
thread. Reloading can then result in one engine isolate performing
reload waiting for another to act on the OOB message (which it will not
do as it's multiplexed on the same thread as the former).
Now that we have a more flexible safepointing mechanism (introduced in
[1]) we can utilize for hot reloading by introducing a new "reloading"
safepoint level.
Reload safepoints
-----------------------
We introduce a new safepoint level (SafepointLevel::kGCAndDeoptAndReload).
Being at a "reload safepoint" implies being at a "deopt safepoint"
which implies being at a "gc safepoint".
Code has to explicitly opt-into making safepoint checks participate /
check into "reload safepoints" using [ReloadParticipationScope]. We do
that at certain well-defined places where reload is possible (e.g. event
loop boundaries, descheduling of isolates, OOM message processing, ...).
While running under [NoReloadScope] we disable checking into "reload
safepoints".
Initiator of hot-reload
-----------------------
When a mutator initiates a reload operation (e.g. as part of a
`ReloadSources` `vm-service` API call) it will use a
[ReloadSafepointOperationScope] to get all other mutators to a
safepoint.
For mutators that aren't already at a "reload safepoint", we'll
notify them via an OOB message (instead of scheduling kVMInterrupt).
While waiting for all mutators to check into a "reload safepoint", the
thread is itself at a safepoint (as other mutators may perform lower
level safepoint operations - e.g. GC, Deopt, ...)
Once all mutators are at a "reload safepoint" the thread will take
ownership of all safepoint levels.
Other mutators
-----------------------
Mutators can be at a "reload safepoint" already (e.g. isolate is not
scheduled). If they try to exit safepoint they will block until the
reload operation is finished.
Mutators that are not at a "reload safepoint" (e.g. executing Dart or VM
code) will be sent an OOB message indicating it should check into a
"reload safepoint". We assume mutators make progress until they can
process OOB message.
Mutators may run under a [NoReloadScope] when handling the OOM message.
In that case they will not check into the "reload safepoint" and simply
ignore the message. To ensure the thread will eventually check-in,
we'll make the destructor of [~NoReloadScope] check & send itself a new OOB
message indicating reload should happen. Eventually getting the mutator
to process the OOM message (which is a well-defined place where we can
check into the reload safepoint).
Non-isolate mutators such as the background compiler do not react to OOB
messages. This means that either those mutators have to be stopped (e.g.
bg compiler) before initiating a reload safepoint operation, the
threads have to explicitly opt-into participating in reload safepoints
or the threads have to deschedule themselves eventually.
Misc
----
Owning a reload safepoint operation implies also owning the deopt &
gc safepoint operation. Yet some code would like to ensure it actually
runs under a [DeoptSafepointOperatoinScope]/[GCSafepointOperationScope].
=> The `Thread::OwnsGCSafepoint()` handles that.
While performing hot-reload we may exercise common code (e.g. kernel
loader, ...) that acquires safepoint locks. Normally it's disallows to
acquire safepoint locks while holding a safepoint operation (since
mutators may be stopped at places where they hold locks, creating
deadlock scenarios).
=> We explicitly opt code into participating in reload safepointing
requests. Those well-defined places aren't holding safepoint locks.
=> The `Thread::CanAcquireSafepointLocks()` will return `true` despite
owning a reload operation. (But if one also holds deopt/gc safepoint
operation it will return false)
Example where this matters: As part of hot-reload, we load kernel which
may create new symbols. The symbol creation code may acquire the symbol
lock and `InsertNewOrGet()` a symbol. This is safe as other mutators
don't hold the symbol lock at reload safepoints. The same cannot be said
for Deopt/GC safepoint operations - as they can interrupt code at many
more places where there's no guarantee that no locks are held.
[0] https://dart-review.googlesource.com/c/sdk/+/187461
[1] https://dart-review.googlesource.com/c/sdk/+/196927
Issue https://github.com/flutter/flutter/issues/124546
TEST=Newly added Reload_* tests.
Change-Id: I6842d7d2b284d043cc047fd702b7c5c7dd1fa3c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296183
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
The sweeper threads are bypassing safepoints and have no need for normal
mutator state (e.g. storebuffer, marking stacks, .,..) on the [Thread]
object.
This fixes a TSAN report where sweeper would clear reusable handles
(which it didn't actually modify) and scavenger is reading those
handles (which arguably it doesn't have to either).
Issue https://github.com/dart-lang/sdk/issues/52125
TEST=ci
Change-Id: I03a36e8518b6c00eb7f3b57f65fd469dddba23e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296860
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
An embedder (or the VM) can exit an isolate via `Thread::ExitIsolate()`
at a point where there's still active state (e.g. dart frames).
Because of this the VM has so far conservatively retained the [Thread]
object of dart mutators throughout the isolate's lifetime. After which
is was manually `delete`ed. We'd never re-use those [Thread] objects (we
do re-use [Thread] objects of non-dart-mutator threads).
When exiting via `Thread::ExitIsolate()` with active state, the mutator
was assumed to be at-safepoint at all levels. It was removed from the
thread registry's active threads. This also means that when e.g. GC runs
it can't use the thread registry to find all active threads it may
need to scan, instead it uses [Isolate::mutator_thread_] of all isolates.
This causes a variety of subtle issues, but the main one that motivated
this change is the following:
If a thread obtains a safepoint operation it means all other mutators
are parked. The thread owning the safepoint can do whatever it likes.
When introducing reload operation safepoints, a thread may want to
ReloadSafepointOperation reload(thread);
...
// Compile sources.
{
TransitionVMToNative transition(thread);
// Will temporarily exit & re-enter current isolate.
response_port = Dart_NewNativePort();
Dart_PostCObject(kernel_isolate_port, ...);
// Wait on [response_port] for response.
}
This will cause the reloading thread to own the reload safepoint
operation but still transition states and even exit/re-enter the
isolate. Though this is currently not possible in the way enter/exit is
implemented.
So we'll refactor this fragile code in the following way:
* Move thread enter/exit logic entirely to the [Thread] object.
* Keep used threads in the thread registry's active list.
=> This allows us to keep various state on the [Thread] and thereby
avoids clearing it when suspending & re-initialing it when resuming
=> It makes nested `Thread::ExitIsolate()` faster as we mainly have
to enter safepoint (avoid acquiring threads lock, avoid releasing
storebuffers, ...)
=> It makes nested `Thread::EnterIsolate()` faster as we mainly have
to leave the safepoint (avoid acquiring threads lock, avoid acquiring
storebuffers, ...).
=> A mutator can now own a safepoint operation (e.g. reload safepoint
operation) and still `ExitSafepoint()` / `EnterSafepoint()` safely -
as those are based on the normal `EnterSafepoint()` and
`LeaveSafepoint()` APIs.
* We separate
- Suspend & Resume of a dart mutator (possibly with active stack)
- Setup & Reset of state only relevant for dart mutators
- Setup & Reset of state relevant for any mutator
* We unify how the [Thread] objects are freed between dart mutator and
non-dart mutators: [Thread] objects without state can be given back to
the [ThreadRegistry] and re-used (instead of being deleted in
`Isolate::~Isolate`)
* We have capability to free [Thread] objects if a dart mutator has an
empty stack & re-use for another isolate of the same group.
(In future we may have N Thread objects for N cores and the threads
would even maintain their TLABs when switching between isolates)
* Since we allow reusing of [Thread] objects also for dart mutators now,
we have extensive asserts to ensure they are "clean" when they get
into the free list and come out "clean" again.
TEST=ci
Change-Id: Id85e8e484efd98d28e323b33795716420e619986
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296585
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Some cleanups factored out of a larger CL (which refactors enter/exit of threads):
* remove unused `#include "vm/thread_registry.h"`
* remove unused/unnecessary fields from [Thread] object
* rename IsMutator() -> IsDartMutator()
* make tests using setjmp() drain the sticky error
=> to ensure there's no sticky error on isolate shutdown
TEST=ci
Change-Id: I53935e8bd0628ab3768627d6d5e01c3f0e3a57ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296582
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This CL removes the packet_ member in TimelineEventPerfettoFileRecorder
and adds a packet_ member to TimelineEventRecorder. This member will be
used when sending Perfetto traces through the VM Service.
TEST=manually verified that traces recorded with the Perfetto file
recorder still contained correct information, CI
Change-Id: I796369ea0832c265a04e37d91bc09f93967979a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/288061
Reviewed-by: Ben Konyi <bkonyi@google.com>
This will allow us to reuse the shared logic when returning Perfetto
traces through the service. This CL also changes some pointer parameters
to const reference parameters.
TEST=manually verified that traces recorded with the JSON file
recorder still contained correct information, CI
Change-Id: I70ac5183959dd8c6efeb55d4891a8e00d3550ca5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/288060
Reviewed-by: Ben Konyi <bkonyi@google.com>
TEST=manually verified that traces recorded with the Perfetto file
recorder still contained correct information, CI
Change-Id: I4bd132dd9e703274ffc262eb7e000e6ee6df4bf8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/287923
Reviewed-by: Ben Konyi <bkonyi@google.com>
This supports flow events reported through dart:developer, but does not
support ones reported with Dart_TimelineEvent in dart_tools_api.h. That
will have to be supported in a future CL.
TEST=CI, manually verifying that flow arrows appeared on flow events in
the trace viewer for traces recorded with our Perfetto recorder
Change-Id: Ic75a8ce277b436a609c027c2c0d4e94b51031aa7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/280740
Reviewed-by: Stephen Adams <sra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Leaf Petersen <leafp@google.com>
Johnni Winther