To prepare for the eventual removal of Observatory, we plan on disabling
Observatory by default while providing an escape hatch to manually serve
the tool for some period of time before completely removing Observatory
from the SDK. This change adds flags that can be used to configure
whether or not Observatory is served.
Currently, '--serve-observatory' is the default behavior, but will be
changed to '--no-serve-observatory' once tooling is ready to support the
escape hatch behavior.
Part of https://github.com/dart-lang/sdk/issues/50233
TEST=run_test.dart
Change-Id: Ib6d1e1587d9fbd3c61d4a4c75d90635052835844
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/267720
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.
It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.
It allows few warnings still since they are triggered in third_party sources.
Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This CL adds a `dart::embedder::Cleanup()` (we already have `dart::embedder::InitOnce()`).
This allows us to allocate the global state and also tear it down.
As a side-effect this will also not allocate those variables if not
needed, which should fix b/151210948
Change-Id: I3c5c619586380bf27ee863ba026bbc631f243d85
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/139640
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Work towards https://github.com/flutter/flutter/issues/46724
Background:
on the latest versions of iOS today, we have to use mdns to discover the observatory port and authentication code. For a variety of reasons this can fail, leaving us with no way to connect.
An alternative approach is to specify the observatory port and disable the authentication code. If binding to this port fails, however, we would still like to attempt connecting with mdns.
Overview:
This adds a new flag to the dart SDK, --enable-service-port-fallback. This amends the behavior of the vmservice with a specified port. After failing to bind 3 times to a non-zero port, update the port selection to 0.
Results:
Tested locally two dart VMs with the same specified port:
```
jonahwilliams@jonahwilliams0:~/Documents/engine/src/out/host_debug_unopt$ ./dart --enable-service-port-fallback --observe=8080 example.dart
Observatory server failed to start after 1 tries
Falling back to automatic port selection
vm-service: isolate(2785434094928835) 'main' has no debugger attached and is paused at exit. Connect to Observatory to debug.
Observatory listening on http://127.0.0.1:45663/62A5IHjmv9E=/
```
Change-Id: I582ab402a457330928e5b490f9718fcd52b56720
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/133434
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Jonah Williams <jonahwilliams@google.com>
This reverts commit 6ff7e9ebad.
Reason for revert: <INSERT REASONING HERE>
Original change's description:
> [vmservice] allow fallback on port bind failure
>
> Work towards https://github.com/flutter/flutter/issues/46724
>
> Background:
>
> on the latest versions of iOS today, we have to use mdns to discover the observatory port and authentication code. For a variety of reasons this can fail, leaving us with no way to connect.
>
> An alternative approach is to specify the observatory port and disable the authentication code. If binding to this port fails, however, we would still like to attempt connecting with mdns.
>
> Overview:
>
> This adds a new flag to the dart SDK, --enable-service-port-fallback. This amends the behavior of the vmservice with a specified port. After failing to bind once to a non-zero port, update the port selection to 0.
>
> Results:
>
> Tested locally two dart VMs with the same specified port:
>
> ```
> jonahwilliams@jonahwilliams0:~/Documents/engine/src/out/host_debug_unopt$ ./dart --enable-service-port-fallback --observe=8080 example.dart
> Observatory server failed to start after 1 tries
> Falling back to automatic port selection
> vm-service: isolate(2785434094928835) 'main' has no debugger attached and is paused at exit. Connect to Observatory to debug.
> Observatory listening on http://127.0.0.1:45663/62A5IHjmv9E=/
> ```
>
> Change-Id: I371583edcb603325428f1cb760992e39b9f3b6dc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130441
> Commit-Queue: Jonah Williams <jonahwilliams@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>
TBR=bkonyi@google.com,jonahwilliams@google.com
Change-Id: I337b2d549e33ef9c616c276a48ce894a14b1c317
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/133327
Reviewed-by: Jonah Williams <jonahwilliams@google.com>
Commit-Queue: Jonah Williams <jonahwilliams@google.com>
Work towards https://github.com/flutter/flutter/issues/46724
Background:
on the latest versions of iOS today, we have to use mdns to discover the observatory port and authentication code. For a variety of reasons this can fail, leaving us with no way to connect.
An alternative approach is to specify the observatory port and disable the authentication code. If binding to this port fails, however, we would still like to attempt connecting with mdns.
Overview:
This adds a new flag to the dart SDK, --enable-service-port-fallback. This amends the behavior of the vmservice with a specified port. After failing to bind once to a non-zero port, update the port selection to 0.
Results:
Tested locally two dart VMs with the same specified port:
```
jonahwilliams@jonahwilliams0:~/Documents/engine/src/out/host_debug_unopt$ ./dart --enable-service-port-fallback --observe=8080 example.dart
Observatory server failed to start after 1 tries
Falling back to automatic port selection
vm-service: isolate(2785434094928835) 'main' has no debugger attached and is paused at exit. Connect to Observatory to debug.
Observatory listening on http://127.0.0.1:45663/62A5IHjmv9E=/
```
Change-Id: I371583edcb603325428f1cb760992e39b9f3b6dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130441
Commit-Queue: Jonah Williams <jonahwilliams@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Adds FromKernel suffix to helper function when Dart VM is running in JIT mode in order to be consistent with other Dart APIs.
Change-Id: I5b3adf1b4efef523ee22601cae0c7aa8725aac50
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/130863
Reviewed-by: Martin Kustermann <kustermann@google.com>
Platform::Init (now Platform::InitOnce) was only being called directly
from the CL embedder and could not be invoked via any path in the embedding
API. Platform::InitOnce is now invoked in both dart::bin::BootstrapDartIo and
dart::embedder::InitOnce.
Fixes https://github.com/dart-lang/sdk/issues/37586
Change-Id: I594908895c19e3058f707f920e265e79ca4cecd7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117591
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
This reverts commit 3d14b75f97.
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:
* have a guarantee that all isolates within one IG can safely exchange
structured objects (currently we rely on embedder for this
guarantee)
* hot-reload all isolates together (currently we only reload one
isolate, leaving same-source isolates in inconsistent state)
* make a shared heap for all isolates from the same IG, which paves
the way for faster communication and sharing of immutable objects.
All isolates within one IG will share the same IsolateGroupSource.
**Embedder changes**
This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.
Existing embedders can be updated by performing the following renames
Dart_CreateIsolate -> Dart_CreateIsolateGroup
Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
Dart_IsolateData -> Dart_IsolateGroupData
Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
Dart_InitializeParams.create -> Dart_InitializeParams.create_group
Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
By default `Isolate.spawn` will cause the creation of a new IG.
Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.
Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097
Original review: https://dart-review.googlesource.com/c/sdk/+/105241
Difference to original review:
* Give each isolate it's own [Loader] (for now)
* Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
* Fix IsolateData memory leak if isolate startup fails
Difference to first reland(Patchset 2):
* Fix typo where memory was freed twice.
Change-Id: Ib1c83fe83b629cd50ae6af90ee99fdd44da882d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108367
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit 67ab3be10d.
Reason for revert: Causes non-deterministic failures on front-end bots on Windows with "Isolate creation failed" error. See https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8909292129328681248/+/steps/unit_tests/0/stdout
Original change's description:
> Reland "[vm/concurrency] Introduce concept of Isolate Groups"
>
> An Isolate Group (IG) is a collection of isolates which were spawned from the
> same source. This allows the VM to:
>
> * have a guarantee that all isolates within one IG can safely exchange
> structured objects (currently we rely on embedder for this
> guarantee)
>
> * hot-reload all isolates together (currently we only reload one
> isolate, leaving same-source isolates in inconsistent state)
>
> * make a shared heap for all isolates from the same IG, which paves
> the way for faster communication and sharing of immutable objects.
>
> All isolates within one IG will share the same IsolateGroupSource.
>
> **Embedder changes**
>
> This change makes breaking embedder API changes to support this new
> concept of Isolate Groups: The existing isolate lifecycle callbacks
> given to Dart_Initialize will become Isolate Group lifecycle callbacks.
> A new callback `initialize_isolate` callback will be added which can
> initialize a new isolate within an existing IG.
>
> Existing embedders can be updated by performing the following renames
>
> Dart_CreateIsolate -> Dart_CreateIsolateGroup
> Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
> Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
> Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
> Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
> Dart_IsolateData -> Dart_IsolateGroupData
> Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
> Dart_InitializeParams.create -> Dart_InitializeParams.create_group
> Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
> Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
>
> By default `Isolate.spawn` will cause the creation of a new IG.
>
> Though an embedder can opt-into supporting multiple isolates within one IG by
> providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
> The responsibility of this new callback is to initialize an existing
> isolate (which was setup by re-using source code from the spawning
> isolate - i.e. the one which used `Isolate.spawn`) by setting native
> resolvers, initializing global state, etc.
>
> Issue https://github.com/dart-lang/sdk/issues/36648
> Issue https://github.com/dart-lang/sdk/issues/36097
>
> Original review: https://dart-review.googlesource.com/c/sdk/+/105241
>
> Difference to original review:
>
> * Give each isolate it's own [Loader] (for now)
> * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
> * Fix IsolateData memory leak if isolate startup fails
>
> Change-Id: I98277d3d10fe275aa9b8a16b6bdd446bbea0b100
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107506
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=kustermann@google.com,aam@google.com,rmacnak@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: Ia4e0f4f9fc317499d3570a371c5bdf9aed799e77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108101
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:
* have a guarantee that all isolates within one IG can safely exchange
structured objects (currently we rely on embedder for this
guarantee)
* hot-reload all isolates together (currently we only reload one
isolate, leaving same-source isolates in inconsistent state)
* make a shared heap for all isolates from the same IG, which paves
the way for faster communication and sharing of immutable objects.
All isolates within one IG will share the same IsolateGroupSource.
**Embedder changes**
This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.
Existing embedders can be updated by performing the following renames
Dart_CreateIsolate -> Dart_CreateIsolateGroup
Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
Dart_IsolateData -> Dart_IsolateGroupData
Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
Dart_InitializeParams.create -> Dart_InitializeParams.create_group
Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
By default `Isolate.spawn` will cause the creation of a new IG.
Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.
Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097
Original review: https://dart-review.googlesource.com/c/sdk/+/105241
Difference to original review:
* Give each isolate it's own [Loader] (for now)
* Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
* Fix IsolateData memory leak if isolate startup fails
Change-Id: I98277d3d10fe275aa9b8a16b6bdd446bbea0b100
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107506
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit b75057b687.
Reason for revert: Caused some failures on app-jit builders
Original change's description:
> [vm/concurrency] Introduce concept of Isolate Groups
>
> An Isolate Group (IG) is a collection of isolates which were spawned from the
> same source. This allows the VM to:
>
> * have a guarantee that all isolates within one IG can safely exchange
> structured objects (currently we rely on embedder for this
> guarantee)
>
> * hot-reload all isolates together (currently we only reload one
> isolate, leaving same-source isolates in inconsistent state)
>
> * make a shared heap for all isolates from the same IG, which paves
> the way for faster communication and sharing of immutable objects.
>
> All isolates within one IG will share the same IsolateGroupSource.
>
> **Embedder changes**
>
> This change makes breaking embedder API changes to support this new
> concept of Isolate Groups: The existing isolate lifecycle callbacks
> given to Dart_Initialize will become Isolate Group lifecycle callbacks.
> A new callback `initialize_isolate` callback will be added which can
> initialize a new isolate within an existing IG.
>
> Existing embedders can be updated by performing the following renames
>
> Dart_CreateIsolate -> Dart_CreateIsolateGroup
> Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
> Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
> Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
> Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
> Dart_IsolateData -> Dart_IsolateGroupData
> Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
> Dart_InitializeParams.create -> Dart_InitializeParams.create_group
> Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
> Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
>
> By default `Isolate.spawn` will cause the creation of a new IG.
>
> Though an embedder can opt-into supporting multiple isolates within one IG by
> providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
> The responsibility of this new callback is to initialize an existing
> isolate (which was setup by re-using source code from the spawning
> isolate - i.e. the one which used `Isolate.spawn`) by setting native
> resolvers, initializing global state, etc.
>
> Issue https://github.com/dart-lang/sdk/issues/36648
> Issue https://github.com/dart-lang/sdk/issues/36097
>
> Change-Id: I82437ac017ca33018d45e02f353b0672db155f6a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105241
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>
TBR=kustermann@google.com,aam@google.com,rmacnak@google.com
Change-Id: Ibd90992a01d61188f27b445c21532e0c46f996ea
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107405
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:
* have a guarantee that all isolates within one IG can safely exchange
structured objects (currently we rely on embedder for this
guarantee)
* hot-reload all isolates together (currently we only reload one
isolate, leaving same-source isolates in inconsistent state)
* make a shared heap for all isolates from the same IG, which paves
the way for faster communication and sharing of immutable objects.
All isolates within one IG will share the same IsolateGroupSource.
**Embedder changes**
This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.
Existing embedders can be updated by performing the following renames
Dart_CreateIsolate -> Dart_CreateIsolateGroup
Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
Dart_IsolateData -> Dart_IsolateGroupData
Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
Dart_InitializeParams.create -> Dart_InitializeParams.create_group
Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
By default `Isolate.spawn` will cause the creation of a new IG.
Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.
Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097
Change-Id: I82437ac017ca33018d45e02f353b0672db155f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105241
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Previously, a valid web socket connection would use the following URI:
`ws://127.0.0.1/ws`
Now, by default, the VM service requires a connection to be made with a
URI similar to the following:
`ws://127.0.0.1:8181/Ug_U0QVsqFs=/ws`
where `Ug_U0QVsqFs` is an authentication code generated and shared by the
service.
This behavior can be disabled with the `--disable-service-auth-codes`
flag.
Change-Id: I288aac58e3ba9d35dca8071f3f7e7a073896c271
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98433
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
- Remove dead code around creation of service isolate in gen_snapshot.cc
- Create service isolate from the core snapshot
- exclude sources from platform dill file (saves about 2.5MB in size)
Change-Id: Ic4f382e23b1df84abdc94cfe0d1ea2b5df315c11
Reviewed-on: https://dart-review.googlesource.com/c/79141
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This enables other emdedders to reuse parts of the standalone embedder
implementation without depending too much on the implementations details.
As part of the change we also remove a dependency from DartUtils on
IsolateData - by removing dart:_builtin library caching. Lookup
of this library does not seem to be on hot path anymore in Kernel world.
Change-Id: Ia35b28886121828fad7a96f00bcbceff75e00ae8
Reviewed-on: https://dart-review.googlesource.com/64848
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Ryan Macnak