This reland commit 5a32d8bc7c with a fix
for thread leak (Issue #56717): when `ThreadPool` is shutting down
asynchronously the last worker should detach itself to prevent
leaking associated low-level data structures, because no thread will
join it.
A hang in service isolate shutdown (caused by an existing bug) was fixed by commit 157a0dc7f9.
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
Closes https://github.com/dart-lang/sdk/issues/56717
TEST=ci
Change-Id: Ic68bfb60757685afd75c80a70cdec66cc13c149b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/385000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This reverts commit 5a32d8bc7c.
Reason for revert: DartIsolateTest.CanCreateServiceIsolate flutter engine unit test hangs on Windows.
Original change's description:
> [vm] Simplify implementation of native ports
>
> This CL turns native ports into a thin abstraction over underlying
> thread pool instead of building them as full fledged MessageHandler.
>
> This allows to easily implement a variation of native ports which can
> handle messages concurrently with the given degree of concurrency.
> This type of port can be used to greatly simplify implementation of
> IOService - which previously had to do its own concurrency management
> on top of "single threaded" native ports. This capability is exposed
> as `Dart_NewConcurrentNativePort` API.
>
> The new implementation is in general much cleaner then the old one
> with one exception: `Dart_CloseNativePort` API has unfortunate design
> where underlying message handler is destroyed asynchronously and
> `Dart_CloseNativePort` returns immediately without waiting for pending
> tasks to complete. Implementing this on top of `ThreadPool` requires
> some changes to thread pool implementation.
>
> Issue https://github.com/dart-lang/sdk/issues/55844
>
> TEST=ci
>
> Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: I7de27793a54072e974bf1a9f17a07c12159a202d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384481
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
TEST=ci
Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The main purpose of the low-level [PortMap] is to coordinate between
ports being opened & closed and concurrent message senders. That's the
only thing it should do.
Each isolate owns [ReceivePort]s. Only the isolate mutator can create
ports, delete them or change their "keeps-isolate-alive" state. Right
now it requires going via [PortMap] (which acquires lock) and
[MessageHandler] (which acquires lock) to change the
"keeps-isolate-alive" state of a port.
We'll move information whether a dart [ReceivePort] is closed and
whether it keeps the isolate alive into the [ReceivePort] object itself.
=> Changing the "keeps-isolate-alive" state of the port no longer
requires any locks. We could even avoid the runtime call itself in a
future CL.
Isolates are kept alive if there's any open receive ports (that have not
been marked as "does not keep isolate alive"). This is a property of an
isolate not of the message handler. For native message handlers we do
have a 1<->1 correspondence between port and handler (i.e. there's no
"number of open ports" tracking needed).
=> We'll move the logic of counting open receive ports and ports that
keep the isolate alive to the [Isolate].
=> We'll also remove locking around incrementing/decrementing or
accessing the counts.
=> The [IsolateMessageHandler] will ask the [Isolate] whether there's
any open ports for determining whether to shut down.
=> For native ports, the `Dart_NewNativePort()` & `Dart_CloseNativePort()`
functions will manage the lifetime (as their name also suggests).
Overall this makes the [Isolate] responsible for creation of dart
[ReceivePort]s and tracking whether the isolate should be kept alive:
* Isolate::CreateReceivePort()
* Isolate::SetReceivePortKeepAliveState()
* Isolate::CloseReceivePort()
TEST=ci
Change-Id: I847ae357c26254d3810cc277962e05deca18a1de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/317960
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.
It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.
It allows few warnings still since they are triggered in third_party sources.
Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Message is a C++ type with a simple ownership model appropriate for
std::unique_ptr. This CL applies the following changes:
1. All uses of "new Message(...)" are replaced with
"Message::New(...)", which is effectively
"std::make_unique<Message>(...)". (The latter was only added in C++14,
but Dart still compiles in C++11 mode.)
2. All owning Message* are replaced with std::unique_ptr<Message>. The
notable exception is MessageQueue, which still uses raw Message*
internally to simplify the linked list handling.
3. All "delete message;" statements are removed.
4. Uses of "NULL" replaced with "nullptr" as necessary.
Change-Id: I05b5804289f2a225bfa05d3c1631129358fed373
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/101222
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Matthew Dempsky <mdempsky@google.com>
The problem with these changes was that closed ports still continue to receive
messages and as a result we start using dead peer objects.
Fixing that would require more intrusive changes into message handler implementation - so instead we are reverting the changes and restoring manual PORT -> PEER mapping.
BUG=
R=erikcorry@google.com
Review-Url: https://codereview.chromium.org/2666063002 .
This change add the ability to restart the vm through the service
protocol. All isolates are killed, and then the main isolate is
restarted cooperatively by the embedder.
This change also fixes the message handler to prevent it from
accidentally ignoring vm shutdown messages.
Previously, we would stop handling messages whenever we hit an error
(such as a compile error or an unhandled exception). This would leave
shutdown requests sitting the oob queue, neglected.
We now process *all* oob requests, up to the first shutdown request.
When we hit a shutdown request, we clear the oob queue and process no
more messages.
To make all of this work, we had to change the return value of
HandleMessage from bool to a new enum type, allowing the message
handler to distinguish *normal* error cases from the more rarified
shutdown and restart cases.
R=johnmccutchan@google.com, zra@google.com
Review URL: https://codereview.chromium.org/1371193005 .
external length available. We now pass a length with all messages in
the vm and verify that there is no mismatch with the length from the
Snapshot.
Fixed a bug in the use of ApiMessageReader. We were always manually
adding Snapshot::kHeaderSize to the data, but neglecting to subtract
kHeaderSize from the message length.
Added FullSnapshotWriter and MessageWriter classes.
Review URL: https://chromiumcodereview.appspot.com//10829444
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@11413 260f80e4-7a28-3924-810f-c04153c831b5
Instead of passing a size in bytes to the allocation function, we now
have a templatized Alloc function:
zone->Alloc<Type>(len)
This is better for security, as we can check for integer overflow in
the size computation before performing the allocation. Before, we
often failed to check this.
Review URL: https://chromiumcodereview.appspot.com//10836061
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@10254 260f80e4-7a28-3924-810f-c04153c831b5
each isolate or native port had a dedicated thread.
Refactored the MessageHandler api...
- Added a Run function to allow a MessageHandler to run on a
ThreadPool. These functions take a start and end callback to allow for
isolate initialization and shutdown.
- Made the queue private to the MessageHandler and moved all message
processing code inside the MessageHandler (got rid of all of the
different flavors of RunLoop). This helps remove some code
duplication and hides the details of how messages are handled.
- Moved all locking and notification out of MessageQueue and moved it
up to MessageHandler. Moved OOB support out of MessageQueue and up
to MessageHandler. These changes make the MessageQueue much
simpler.
- Refactored native port and isolate MessageHandlers to share more code.
- Improved --trace_isolates output.
- Added tests for MessageHandler.
Refactored lib/isolate code...
- Use the new MessageHandler::Run api.
- Got rid of the LongJump stuff in RunIsolate. No longer needed.
- Use the new StartIsolateScope/SwitchIsolateScope to make the code
less verbose and less error-prone.
- Store top-level isolate errors in the sticky_error.
Added StartIsolateScope/SwitchIsolateScope classes.
Review URL: https://chromiumcodereview.appspot.com//9924015
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@6762 260f80e4-7a28-3924-810f-c04153c831b5
The native port callback is now passed the message as a decodes
Dart_CMessage structure. The Dart_CMessage structure is allocated
in a zone and the callback receiving it should expect the
lifetime to be controlled by the caller.
Added support for zones which do not require a current
isolate. Changed the GrowableArray to support allocating in
aprovided zone instead of the zone for the current isolate.
R=turnidge@google.com, asiva@google.com
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com//9325022
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@4068 260f80e4-7a28-3924-810f-c04153c831b5
Dart_NewNativePort creates a port associated with a C handler
function. When messages come in on this port, they are forwarded to
the C function for processing.
To support this, refactored PortMap so that it operates on a new
MessageHandler type instead of directly on Isolates.
For now, native ports have a dedicated single thread. Eventually we
will back native ports (and possibly Isolates as well) by a shared
thread pool.
Review URL: https://chromiumcodereview.appspot.com//9169063
git-svn-id: https://dart.googlecode.com/svn/branches/bleeding_edge/dart@3804 260f80e4-7a28-3924-810f-c04153c831b5
Vyacheslav Egorov