This reland commit 5a32d8bc7c with a fix
for thread leak (Issue #56717): when `ThreadPool` is shutting down
asynchronously the last worker should detach itself to prevent
leaking associated low-level data structures, because no thread will
join it.
A hang in service isolate shutdown (caused by an existing bug) was fixed by commit 157a0dc7f9.
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
Closes https://github.com/dart-lang/sdk/issues/56717
TEST=ci
Change-Id: Ic68bfb60757685afd75c80a70cdec66cc13c149b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/385000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This reverts commit 5a32d8bc7c.
Reason for revert: DartIsolateTest.CanCreateServiceIsolate flutter engine unit test hangs on Windows.
Original change's description:
> [vm] Simplify implementation of native ports
>
> This CL turns native ports into a thin abstraction over underlying
> thread pool instead of building them as full fledged MessageHandler.
>
> This allows to easily implement a variation of native ports which can
> handle messages concurrently with the given degree of concurrency.
> This type of port can be used to greatly simplify implementation of
> IOService - which previously had to do its own concurrency management
> on top of "single threaded" native ports. This capability is exposed
> as `Dart_NewConcurrentNativePort` API.
>
> The new implementation is in general much cleaner then the old one
> with one exception: `Dart_CloseNativePort` API has unfortunate design
> where underlying message handler is destroyed asynchronously and
> `Dart_CloseNativePort` returns immediately without waiting for pending
> tasks to complete. Implementing this on top of `ThreadPool` requires
> some changes to thread pool implementation.
>
> Issue https://github.com/dart-lang/sdk/issues/55844
>
> TEST=ci
>
> Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: I7de27793a54072e974bf1a9f17a07c12159a202d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384481
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
TEST=ci
Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The main purpose of the low-level [PortMap] is to coordinate between
ports being opened & closed and concurrent message senders. That's the
only thing it should do.
Each isolate owns [ReceivePort]s. Only the isolate mutator can create
ports, delete them or change their "keeps-isolate-alive" state. Right
now it requires going via [PortMap] (which acquires lock) and
[MessageHandler] (which acquires lock) to change the
"keeps-isolate-alive" state of a port.
We'll move information whether a dart [ReceivePort] is closed and
whether it keeps the isolate alive into the [ReceivePort] object itself.
=> Changing the "keeps-isolate-alive" state of the port no longer
requires any locks. We could even avoid the runtime call itself in a
future CL.
Isolates are kept alive if there's any open receive ports (that have not
been marked as "does not keep isolate alive"). This is a property of an
isolate not of the message handler. For native message handlers we do
have a 1<->1 correspondence between port and handler (i.e. there's no
"number of open ports" tracking needed).
=> We'll move the logic of counting open receive ports and ports that
keep the isolate alive to the [Isolate].
=> We'll also remove locking around incrementing/decrementing or
accessing the counts.
=> The [IsolateMessageHandler] will ask the [Isolate] whether there's
any open ports for determining whether to shut down.
=> For native ports, the `Dart_NewNativePort()` & `Dart_CloseNativePort()`
functions will manage the lifetime (as their name also suggests).
Overall this makes the [Isolate] responsible for creation of dart
[ReceivePort]s and tracking whether the isolate should be kept alive:
* Isolate::CreateReceivePort()
* Isolate::SetReceivePortKeepAliveState()
* Isolate::CloseReceivePort()
TEST=ci
Change-Id: I847ae357c26254d3810cc277962e05deca18a1de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/317960
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
The current hot-reload implementation [0] will perform a reload by
first sending OOB messages to all isolates and waiting until those OOB
messages are being handled. The handler of the OOB message will block
the thread (and unschedule isolate) and notify the thread performing
reload it's ready.
This requires that all isolates within a group can actually run & block.
This is the case for the VM implementation of isolates (as they are
run an unlimited size thread pool).
Though flutter seems to multiplex several engine isolates on the same OS
thread. Reloading can then result in one engine isolate performing
reload waiting for another to act on the OOB message (which it will not
do as it's multiplexed on the same thread as the former).
Now that we have a more flexible safepointing mechanism (introduced in
[1]) we can utilize for hot reloading by introducing a new "reloading"
safepoint level.
Reload safepoints
-----------------------
We introduce a new safepoint level (SafepointLevel::kGCAndDeoptAndReload).
Being at a "reload safepoint" implies being at a "deopt safepoint"
which implies being at a "gc safepoint".
Code has to explicitly opt-into making safepoint checks participate /
check into "reload safepoints" using [ReloadParticipationScope]. We do
that at certain well-defined places where reload is possible (e.g. event
loop boundaries, descheduling of isolates, OOM message processing, ...).
While running under [NoReloadScope] we disable checking into "reload
safepoints".
Initiator of hot-reload
-----------------------
When a mutator initiates a reload operation (e.g. as part of a
`ReloadSources` `vm-service` API call) it will use a
[ReloadSafepointOperationScope] to get all other mutators to a
safepoint.
For mutators that aren't already at a "reload safepoint", we'll
notify them via an OOB message (instead of scheduling kVMInterrupt).
While waiting for all mutators to check into a "reload safepoint", the
thread is itself at a safepoint (as other mutators may perform lower
level safepoint operations - e.g. GC, Deopt, ...)
Once all mutators are at a "reload safepoint" the thread will take
ownership of all safepoint levels.
Other mutators
-----------------------
Mutators can be at a "reload safepoint" already (e.g. isolate is not
scheduled). If they try to exit safepoint they will block until the
reload operation is finished.
Mutators that are not at a "reload safepoint" (e.g. executing Dart or VM
code) will be sent an OOB message indicating it should check into a
"reload safepoint". We assume mutators make progress until they can
process OOB message.
Mutators may run under a [NoReloadScope] when handling the OOM message.
In that case they will not check into the "reload safepoint" and simply
ignore the message. To ensure the thread will eventually check-in,
we'll make the destructor of [~NoReloadScope] check & send itself a new OOB
message indicating reload should happen. Eventually getting the mutator
to process the OOM message (which is a well-defined place where we can
check into the reload safepoint).
Non-isolate mutators such as the background compiler do not react to OOB
messages. This means that either those mutators have to be stopped (e.g.
bg compiler) before initiating a reload safepoint operation, the
threads have to explicitly opt-into participating in reload safepoints
or the threads have to deschedule themselves eventually.
Misc
----
Owning a reload safepoint operation implies also owning the deopt &
gc safepoint operation. Yet some code would like to ensure it actually
runs under a [DeoptSafepointOperatoinScope]/[GCSafepointOperationScope].
=> The `Thread::OwnsGCSafepoint()` handles that.
While performing hot-reload we may exercise common code (e.g. kernel
loader, ...) that acquires safepoint locks. Normally it's disallows to
acquire safepoint locks while holding a safepoint operation (since
mutators may be stopped at places where they hold locks, creating
deadlock scenarios).
=> We explicitly opt code into participating in reload safepointing
requests. Those well-defined places aren't holding safepoint locks.
=> The `Thread::CanAcquireSafepointLocks()` will return `true` despite
owning a reload operation. (But if one also holds deopt/gc safepoint
operation it will return false)
Example where this matters: As part of hot-reload, we load kernel which
may create new symbols. The symbol creation code may acquire the symbol
lock and `InsertNewOrGet()` a symbol. This is safe as other mutators
don't hold the symbol lock at reload safepoints. The same cannot be said
for Deopt/GC safepoint operations - as they can interrupt code at many
more places where there's no guarantee that no locks are held.
[0] https://dart-review.googlesource.com/c/sdk/+/187461
[1] https://dart-review.googlesource.com/c/sdk/+/196927
Issue https://github.com/flutter/flutter/issues/124546
TEST=Newly added Reload_* tests.
Change-Id: I6842d7d2b284d043cc047fd702b7c5c7dd1fa3c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/296183
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Original CL in patchset 1.
Split-off https://dart-review.googlesource.com/c/sdk/+/238341
And pulled in fix https://dart-review.googlesource.com/c/sdk/+/238582
(Should merge cleanly when this lands later.)
This CL implements the `Finalizer` in the GC.
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: Ibdfeadc16d5d69ade50aae5b9f794284c4c4dbab
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238086
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This reverts commit 7dca34c235.
Reason for revert: b/226085355 dart_vm_test crashing. Unclear what
the cause is. Reverting so we can triage the issue.
TEST=This is a revert.
Original change's description:
> [vm] Implement `Finalizer`
>
> This CL implements the `Finalizer` in the GC.
>
> (This CL does not yet implement `NativeFinalizer`.)
>
> The GC is specially aware of two types of objects for the purposes of
> running finalizers.
>
> 1) `FinalizerEntry`
> 2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
>
> A `FinalizerEntry` contains the `value`, the optional `detach` key, and
> the `token`, and a reference to the `finalizer`.
> An entry only holds on weakly to the value, detach key, and finalizer.
> (Similar to how `WeakReference` only holds on weakly to target).
>
> A `Finalizer` contains all entries, a list of entries of which the value
> is collected, and a reference to the isolate.
>
> When a the value of an entry is GCed, the enry is added over to the
> collected list.
> If any entry is moved to the collected list, a message is sent that
> invokes the finalizer to call the callback on all entries in that list.
>
> When a finalizer is detached by the user, the entry token is set to the
> entry itself and is removed from the all entries set.
> This ensures that if the entry was already moved to the collected list,
> the finalizer is not executed.
>
> To speed up detaching, we use a weak map from detach keys to list of
> entries. This ensures entries can be GCed.
>
> Both the scavenger and marker tasks process finalizer entries in
> parallel.
> Parallel tasks use an atomic exchange on the head of the collected
> entries list, ensuring no entries get lost.
> The mutator thread is guaranteed to be stopped when processing entries.
> This ensures that we do not need barriers for moving entries into the
> finalizers collected list.
> Dart reads and replaces the collected entries list also with an atomic
> exchange, ensuring the GC doesn't run in between a load/store.
>
> When a finalizer gets posted a message to process finalized objects, it
> is being kept alive by the message.
> An alternative design would be to pre-allocate a `WeakReference` in the
> finalizer pointing to the finalizer, and send that itself.
> This would be at the cost of an extra object.
>
> Send and exit is not supported in this CL, support will be added in a
> follow up CL. Trying to send will throw.
>
> Bug: https://github.com/dart-lang/sdk/issues/47777
>
> TEST=runtime/tests/vm/dart/finalizer/*
> TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
> TEST=runtime/vm/object_test.cc
>
> Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
> Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
> Reviewed-by: Lasse Nielsen <lrn@google.com>
> Reviewed-by: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,rmacnak@google.com,dacoharkes@google.com
Change-Id: I991f6e49896d18a8d70210cf315d858b462d66c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/47777
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/238080
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL implements the `Finalizer` in the GC.
(This CL does not yet implement `NativeFinalizer`.)
The GC is specially aware of two types of objects for the purposes of
running finalizers.
1) `FinalizerEntry`
2) `Finalizer` (`FinalizerBase`, `_FinalizerImpl`)
A `FinalizerEntry` contains the `value`, the optional `detach` key, and
the `token`, and a reference to the `finalizer`.
An entry only holds on weakly to the value, detach key, and finalizer.
(Similar to how `WeakReference` only holds on weakly to target).
A `Finalizer` contains all entries, a list of entries of which the value
is collected, and a reference to the isolate.
When a the value of an entry is GCed, the enry is added over to the
collected list.
If any entry is moved to the collected list, a message is sent that
invokes the finalizer to call the callback on all entries in that list.
When a finalizer is detached by the user, the entry token is set to the
entry itself and is removed from the all entries set.
This ensures that if the entry was already moved to the collected list,
the finalizer is not executed.
To speed up detaching, we use a weak map from detach keys to list of
entries. This ensures entries can be GCed.
Both the scavenger and marker tasks process finalizer entries in
parallel.
Parallel tasks use an atomic exchange on the head of the collected
entries list, ensuring no entries get lost.
The mutator thread is guaranteed to be stopped when processing entries.
This ensures that we do not need barriers for moving entries into the
finalizers collected list.
Dart reads and replaces the collected entries list also with an atomic
exchange, ensuring the GC doesn't run in between a load/store.
When a finalizer gets posted a message to process finalized objects, it
is being kept alive by the message.
An alternative design would be to pre-allocate a `WeakReference` in the
finalizer pointing to the finalizer, and send that itself.
This would be at the cost of an extra object.
Send and exit is not supported in this CL, support will be added in a
follow up CL. Trying to send will throw.
Bug: https://github.com/dart-lang/sdk/issues/47777
TEST=runtime/tests/vm/dart/finalizer/*
TEST=runtime/tests/vm/dart_2/isolates/fast_object_copy_test.dart
TEST=runtime/vm/object_test.cc
Change-Id: I03e6b4a46212316254bf46ba3f2df333abaa686c
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-ffi-android-debug-arm64c-try,dart-sdk-mac-arm64-try,vm-kernel-mac-release-arm64-try,pkg-mac-release-arm64-try,vm-kernel-precomp-nnbd-mac-release-arm64-try,vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-nnbd-win-release-ia32-try,vm-ffi-android-debug-arm-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-mac-debug-x64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,benchmark-linux-try,flutter-analyze-try,flutter-frontend-try,pkg-linux-debug-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-gcc-linux-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/229544
Reviewed-by: Lasse Nielsen <lrn@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
When an isolate dies it has to close all it's open ports. Right now this
is done by walking all ports in the system.
=> O(#num-isolates x #num-ports) complexity
Instead we let each isolate remember it's own open ports: When the
isolate dies we iterate over those ports that particular isolate has
open and close them.
=> O(#num-isolates + #num-ports) complexity
On a many-isolate test of the form:
foo(int n) {
if (n == 1) return 1;
return foo(n-1) + 1;
}
for `foo(50k)` this reduces the runtime from 90 seconds to 30 seconds.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I3ac68c5334a1d5e8cac47e89e15cbb50c26b65ac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144261
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Right now a message handler is taking care of detecting idle timeouts
and notifying the isolate (or rather it's heap) that a compaction can
start.
Once we move the heap to the isolate group, we should only notify the
heap that we're idle if *all* isolates are idle. As a preparatory step
we move the idle timer functionality into it's own class and make the
isolate own it.
When the heap gets moved from isolate to isolate group, we'll also move
the idle time handler.
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: If9b244baf0425ef343f909ee3a8a1147f966845b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/116200
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
MessageHandler doesn't actually need a reference to the
ThreadPool::Task, it just wants to know if one is running or not.
Replacing it with a simple boolean will simplify switching
ThreadPool::Task to use std::unique_ptr.
Updates #37244.
Change-Id: Ie69ec38523f009ba559678fd544efa4cc8ead7dd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/106008
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
Message is a C++ type with a simple ownership model appropriate for
std::unique_ptr. This CL applies the following changes:
1. All uses of "new Message(...)" are replaced with
"Message::New(...)", which is effectively
"std::make_unique<Message>(...)". (The latter was only added in C++14,
but Dart still compiles in C++11 mode.)
2. All owning Message* are replaced with std::unique_ptr<Message>. The
notable exception is MessageQueue, which still uses raw Message*
internally to simplify the linked list handling.
3. All "delete message;" statements are removed.
4. Uses of "NULL" replaced with "nullptr" as necessary.
Change-Id: I05b5804289f2a225bfa05d3c1631129358fed373
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101222
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
This fixes an issue where service RPCs are not answered by the
vm-service.
Currently new isolates are visible via the service protocol immediately
after creation. This means a vm-service client can start doing service
RPCs to a particular isolate. If the embedder has not-yet installed a message
handler the OOB messages will be queued up.
Once the embedder installs an isolate message notify handler, that
handler needs to be invoked if there are pending messages.
Change-Id: Ie75878daf55a3b380e2b2ec7930e15b002f3f520
Reviewed-on: https://dart-review.googlesource.com/c/83680
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Rework the idle notification mechanism to not use the global
IdleNotifier class as it was resulting in lock inversion issues
causing the monitor to be deleted while it could still be
required in another thread.
The new mechanism wires the idle notification check into the existing
message handling loop and uses the 'paused_for_messages_' mechanism
to wake up a idle time waiting handler when new messages arrive.
Bug: 34888
Change-Id: Ifd75e5cf4c0b6944d586146fe518b9e3e053a76e
Reviewed-on: https://dart-review.googlesource.com/c/82705
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
If a thread pool isolate has not processed a message in FLAG_idle_timeout_micros, run idle tasks.
Change-Id: If506d7805eb1213c3d1f9383d835226822012fff
Reviewed-on: https://dart-review.googlesource.com/26004
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
The only fix needed for relanding is adding _ensureScheduleImmediate
to the list of vm entrypoints in //runtime/vm/compiler/aot/precompiler.cc
Original commit message:
Adds a top-level call waitForEventSync to dart:io that blocks the
thread an Isolate is running on until messages are available.
Before the thread blocks, the microtask queue is drained.
Before waitForEventSync returns, all messages are handled.
Lifting this up from a comment:
This is apropos of the request that nweiz@ sent to the mailing list a
couple weeks back. I'm not sure we should land this. We certainly
shouldn't land it without some annotations that will make the analyzer
complain a lot in most configurations, but I don't know what those
annotations are.
fixes#31102
Change-Id: Id96de46cc5f10e1847045cfafb7cfed6a38bce16
Reviewed-on: https://dart-review.googlesource.com/28920
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
Adds a top-level call waitForEventSync to dart:io that blocks the
thread an Isolate is running on until messages are available.
Before the thread blocks, the microtask queue is drained.
Before waitForEventSync returns, all messages are handled.
Lifting this up from a comment:
This is apropos of the request that nweiz@ sent to the mailing list a
couple weeks back. I'm not sure we should land this. We certainly
shouldn't land it without some annotations that will make the analyzer
complain a lot in most configurations, but I don't know what those
annotations are.
Change-Id: If8286f4525994a162dd4f4563fefccb9d0984f7c
Reviewed-on: https://dart-review.googlesource.com/25281
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
i.e. #ifndef VM_WHATEVER -> #ifndef RUNTIME_VM_WHATEVER
This lets us remove a hack from the PRESUBMIT.py script that existed
for reasons that are no longer valid, and sets us up to add some
presubmit checks for the GN build.
R=asiva@google.com, rmacnak@google.com
Review URL: https://codereview.chromium.org/2450713004 .
- [x] The first caller of the tag handler blocks, recursive callers queue work and exit.
- [x] Use a NativeMessageHandler to receive I/O results from the service isolate.
- [x] Preserve load error message format.
- [x] Move packages map into service isolate.
- [x] Wire up Todd's native URI code.
R=turnidge@google.com
Review URL: https://codereview.chromium.org/1998963003 .
in these blocks will not have a safepoint operation
- changed boxed_field_list_monitor_ to boxed_field_list_mutex_ as we only
need a mutex for guarding access to boxed_field_list_ as changed the
lock to use SafepointMutexLocker as the list addition code could potentially
allocate and result in GC (safepoint operation)
- Added a SafepointMonitorLocker as we have a function Isolate::VisitIsolates
which could potentially have safepoints in the enclosed block.
- Make the lock around MegamorphicCacheTable::Lookup a SafepointMutexLocker
as the look up code seems to be allocating memory while the lock is held.
- Changed the ThreadPool and MessageHandler code to account for the new
MonitorLocker usage standard
- Fixed PortMap::PrintPortsForMessageHandler to use SafepointMutexLocker as
the code it encloses calls into Dart
- Removed profiler_ field in class Profiler as it doesn't seem to be used.
R=johnmccutchan@google.com, srdjan@google.com
Review URL: https://codereview.chromium.org/1748953003 .
- Add Dart_ShouldPauseOnStart, Dart_SetShouldPauseOnStart, Dart_IsPausedOnStart, Dart_SetPausedOnStart, Dart_ShouldPauseOnExit, Dart_SetShouldPauseOnExit, Dart_IsPausedOnExit, Dart_SetPausedOnExit to query and control pause on start / exit from outside of the default message handler.
- Add Dart_HandleMessages to process all regular messages from outside the default message handler.
- Allow per isolate overriding of pause on start / pause on exit flags.
- Set the resume_request bit when resuming from isolate start / exit.
- Clear the resume_request bit in the default message handler.
- Rename MessageHandler fields and accessors.
- Rename Isolate::Resume to Isolate::SetResumeRequest.
R=turnidge@google.com
Review URL: https://codereview.chromium.org/1665773004 .
This change add the ability to restart the vm through the service
protocol. All isolates are killed, and then the main isolate is
restarted cooperatively by the embedder.
This change also fixes the message handler to prevent it from
accidentally ignoring vm shutdown messages.
Previously, we would stop handling messages whenever we hit an error
(such as a compile error or an unhandled exception). This would leave
shutdown requests sitting the oob queue, neglected.
We now process *all* oob requests, up to the first shutdown request.
When we hit a shutdown request, we clear the oob queue and process no
more messages.
To make all of this work, we had to change the return value of
HandleMessage from bool to a new enum type, allowing the message
handler to distinguish *normal* error cases from the more rarified
shutdown and restart cases.
R=johnmccutchan@google.com, zra@google.com
Review URL: https://codereview.chromium.org/1371193005 .
deprecated isolate based API.)
- Update all code impacted by this change. E.g. DARTSCOPE
- TEST_CASE now passes the current thread as a parameter to the unit test.
BUG=
R=asiva@google.com
Review URL: https://codereview.chromium.org//1310463005 .
- Display paused time as tooltip over "paused" word.
- Send a timestamp with every ServiceEvent.
- Unit tests verifying that paused on start, exit, and breakpoint timestamps are stable.
- Remember the pause time in DebuggerEvent.
- Remember the pause time in MessageHandler.
- Updated service protocol documentation.
R=turnidge@google.com
Review URL: https://codereview.chromium.org//1311503004 .
- Log the parameters to RPCs
- Display messages like stack frames
- handler function and script visible
- message preview instead of local variables
- Expanded and previewed messages do not collapse when stepping within the existing message
- Mark patch classes as finalized to avoid ASSERT on class_finalizer.cc:2358 **
- Support iterating over message queues
- Support printing the message queue as JSON
- Inhibit OOB messages from being handled if we are iterating over the message queue
- avoids dead lock when looking up port handler (done in Dart code with message handler locked) and a stack overflow trigger to handle OOB messages
- make getObject understand message ids
- Fix dartbug.com/23355
** Need to discuss with Ivan and Matthias (reviewed code differs from committed code but matches my fix):
https://codereview.chromium.org//828353002https://code.google.com/p/dart/source/detail?r=42612R=turnidge@google.com
Review URL: https://codereview.chromium.org//1122503003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@45505 260f80e4-7a28-3924-810f-c04153c831b5
--------
VM
Add a new ServiceEvent class in the vm. Rework the event type names.
Add BreakpointAdded, BreakpointResolved, and BreakpointRemoved
service events.
Record the top frame for kIsolateInterrupted and kExceptionThrown
debugger events.
Send a top frame in more Pause* events.
Send breakpoint list with the Isolate response. Remove the
getBreakpoints method from the vm service.
Move Resume events into debugger.cc.
Rework MessageHandler a bit so that we can send PauseStart and
PauseExit notifications.
--------
Observatory
Move some event handler from Application to Isolate.
Rewrite breakpoint tracking code to use the new breakpoint events.
Change run state tracking in Isolate.
Change getFromMap so that it applies updates if the map isn't a ref.
Use getFromMap instead of new ServiceObject in invokeRpc. This
fixes some duplicate ServiceObject problems.
Review URL: https://codereview.chromium.org//979823003
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@44268 260f80e4-7a28-3924-810f-c04153c831b5
Deadlock looks like:
Regular Isolate
-> paused at breakpoint in debug message loop
-> holds debug message queue lock to get message notifications
-> handles an service message
-> waits for portmap lock to send response
Service Isolate
-> receives request for paused isolate
-> holds portmap lock to send message
-> runs custom message notifier to wake debug message loop
-> waits for debug message queue lock
I've solved this by releasing the debug message queue lock while
handling service messages. This requires me to poll for new service
messages after reacquiring the debug message queue lock to make sure I
haven't dropped any notifications.
It's a little weird that the embedder (runtime/bin) needs to be aware
of the locking in the core vm (runtime/vm), but this seemed like the
simplest fix for now.
BUG=
R=johnmccutchan@google.com
Review URL: https://codereview.chromium.org//629533002
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@41326 260f80e4-7a28-3924-810f-c04153c831b5
Vyacheslav Egorov