Commit Graph

161 Commits

Author SHA1 Message Date
Martin Kustermann 8e887672f4 [vm/concurrency] First changes for hot-reloading with isolate gropus.
This CL moves the [ProgramReloadContext] from [Isolate] to
[IsolateGroup] - since the program is shared across isolates. Amongst
other things, it also deoptimizes stacks of all isolate mutators.

There is still future work to do for hot-reloading multiple isolates
within a group, this is only the first change.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: Ic394de265e3922f84657de2da25a46179b624647
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183693
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-12 05:48:15 +00:00
Vyacheslav Egorov 24a30defcc [vm] Ensure liveness of typed data containing Kernel binary
When handling hot reload via library tag handler's Kernel tag
VM must guarantee that external typed data it receives from the
embedder stays alive for as long as VM has any structures
referencing this kernel binary. This is achieved by attaching
original typed data to KernelProgramInfo objects containing
the views into it.

Unfortunately kernel::Reader APIs were somewhat unsafe: allowing
to construct reader object from Program's raw buffer and forget to
set the link betwen the reader and original typed data.

This created a situation where reloading using a multicomponent
Kernel binary would result in KernelProgramInfo objects without
link to the original typed data, which in turn leads to premature
finalization of external typed data and subsequent crashes
when trying to use delete kernel binary.

This CL reworks kernel::Reader API (by introducing kernel::ProgramBinary
wrapper) to make it safer and make sure that connection is preserved.

TEST=vm/cc/IsolateReload_RegressB179030011

Bug: b/179030011
Change-Id: I05f8c31c3cb8e67de6e94a20d9501a5f476b7e27
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182280
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-04 09:46:17 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00
Alexander Markov 50f1f1459e [vm/nnbd] Support NNBD weak mode in vm/cc tests
Change-Id: Ia49c64bf8461b8acf019c398635e27b11fe6d25e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149666
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-06-02 20:40:03 +00:00
Ryan Macnak 6fe15f6df9 [vm] Represent tagged pointers as C++ value types instead of C++ pointer types.
This works around bugs in UndefinedBehaviorSanitizer and Clang.

Bug: b/28638298
Change-Id: I6be595f9664516019d28017d24559583a1ae3a21
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144354
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-04-25 05:21:27 +00:00
Alexander Aprelev 9c4a322b08 Reland "[vm/isolates] Introduce fast isolate spawn in AOT."
This reverts commit 922ea3e9b6 in patchset 1, fix for assertion triggered in https://ci.chromium.org/b/8883214567628884960 in patchset 2, fix for deadlock around symbols table mutex in patchset 4.

Original commit description:

Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.

To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.

Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I655e337198214c9dfacbe76f7852b941b5a7e910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143462
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-04-17 03:02:27 +00:00
Alexander Aprelev 922ea3e9b6 Revert "Reland "[vm/isolates] Introduce fast isolate spawn in AOT.""
This reverts commit fdd7a2c616 as there
seem to be sporadic dartkp crashes like https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-linux-product-x64/7454

Change-Id: I4218b8ff629630991ad1ff94f217489bb16228d3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143383
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-04-14 20:37:29 +00:00
Alexander Aprelev fdd7a2c616 Reland "[vm/isolates] Introduce fast isolate spawn in AOT."
Original revert in patchset 1, fix for deadlock issue in patchset 2: avoid reentrant calls to RunWithStoppedMutator.
Further review comments addressed in successive patchsets.

This reverts commit 8ef508ba36.

Original commit description:

Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.

To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.

Change-Id: I6a0279d9812020ad7a5c2b7851980b6a29b95b9a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143327
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-14 18:42:45 +00:00
Alexander Aprelev 8ef508ba36 Revert "[vm/isolates] Introduce fast isolate spawn in AOT."
This reverts commit 61c0960a8b as it broke
vm-kernel-win-release-ia32 buildbot test https://ci.chromium.org/b/8883214567628884960

Change-Id: I6377fd622a3c7d808f46fd9c541c64d343ecc269
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143260
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-04-12 15:59:41 +00:00
Alexander Aprelev 61c0960a8b [vm/isolates] Introduce fast isolate spawn in AOT.
Speed up is achieved by sharing most of the dart code, object store
and class table between isolates in single isolate group. So
instead of bootstrapping isolate from the snapshot, isolate is
initialized by setting pointers to existing data structures already
set up for first isolate, and only few isolate-specific structures (moved
to newly introducted isolate_object_store) are created.

To allow for safe cross-isolate switchable call site, type test cache
mutations additional synchronization via RunWithStoppedMutators(that
relies on safepoints) was added.
Besides switchable call sites, no other mutation to the dart code is
done in AOT, which allows such sharing.

Bug: https://github.com/dart-lang/sdk/issues/37835
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I64c86525f4ef9cb30567a49a106bfe700355942b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/136780
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-04-12 14:57:09 +00:00
Ryan Macnak 7679924ece [vm] Reduce frame size for the functions with the largest frame sizes.
Cf. -Wframe-larger-than.

Change-Id: I47e3144bdf7f8fd8c6f308d5552a7ce4f3f990ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128008
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-12-11 18:15:27 +00:00
Martin Kustermann aceeba7567 [vm/async] Encode the yield index -> token position in PcDescriptors.
Right now `Script::yield_positions()` is an array mapping function start
token positions to arrays. Those arrays contain token positions and are
indexed by yield index.
  => This `Script::yield_positions()` is not available in AOT mode.

The fast async stack implementation will need to be able to find out
where an async closure was suspended. It does so by looking at the
":await_jump_var", which contains the yield index. It then needs to
associate the yield index to token position of the yield.

This CL adds an entry into the PcDescriptors for every yield in a
async/async* function and removes `Script::yield_positions()`.
The entry will associate the yield index with the token position.

Flutter gallery total size impact for flutter-release:

  - armv7: +0.016%
  - armv8: +0.045%

Issue https://github.com/dart-lang/sdk/issues/37668

Change-Id: I0b2ce41e85d8f5d590201bf2fb091578d7379890
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125408
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2019-11-18 11:50:13 +00:00
Samir Jindel c885bdde1d [vm] DBC is obsolete. Remove dead code.
Change-Id: Ica33af158cca53c8e951e4b2582de83660e8a60d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121851
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-10-27 18:18:29 +00:00
Liam Appelbe d49bf6f25a [vm] Remove misleading usings.
Patchset 1 contains just the usings I deleted (<100 lines). All the
other diffs in the rest of the CL are just fixing all the places that
were broken by this.

Bug: https://github.com/dart-lang/sdk/issues/36839
Change-Id: I3bb4fa62ab4363ded81fd7c2815b857f91886dd6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108502
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2019-07-10 22:20:10 +00:00
Alexander Aprelev 0425997b31 Second attempt to reland "[vm/concurrency] Introduce concept of Isolate Groups"
This reverts commit 3d14b75f97.

An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:

  * have a guarantee that all isolates within one IG can safely exchange
    structured objects (currently we rely on embedder for this
    guarantee)

  * hot-reload all isolates together (currently we only reload one
    isolate, leaving same-source isolates in inconsistent state)

  * make a shared heap for all isolates from the same IG, which paves
    the way for faster communication and sharing of immutable objects.

All isolates within one IG will share the same IsolateGroupSource.

**Embedder changes**

This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.

Existing embedders can be updated by performing the following renames

  Dart_CreateIsolate -> Dart_CreateIsolateGroup
  Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
  Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
  Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
  Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
  Dart_IsolateData -> Dart_IsolateGroupData
  Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
  Dart_InitializeParams.create -> Dart_InitializeParams.create_group
  Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
  Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate

By default `Isolate.spawn` will cause the creation of a new IG.

Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.

Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097

Original review: https://dart-review.googlesource.com/c/sdk/+/105241

Difference to original review:

  * Give each isolate it's own [Loader] (for now)
  * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
  * Fix IsolateData memory leak if isolate startup fails

Difference to first reland(Patchset 2):

  * Fix typo where memory was freed twice.

Change-Id: Ib1c83fe83b629cd50ae6af90ee99fdd44da882d4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108367
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-07-08 23:49:05 +00:00
Vyacheslav Egorov 3d14b75f97 Revert "Reland "[vm/concurrency] Introduce concept of Isolate Groups""
This reverts commit 67ab3be10d.

Reason for revert: Causes non-deterministic failures on front-end bots on Windows with "Isolate creation failed" error. See https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8909292129328681248/+/steps/unit_tests/0/stdout

Original change's description:
> Reland "[vm/concurrency] Introduce concept of Isolate Groups"
> 
> An Isolate Group (IG) is a collection of isolates which were spawned from the
> same source. This allows the VM to:
> 
>   * have a guarantee that all isolates within one IG can safely exchange
>     structured objects (currently we rely on embedder for this
>     guarantee)
> 
>   * hot-reload all isolates together (currently we only reload one
>     isolate, leaving same-source isolates in inconsistent state)
> 
>   * make a shared heap for all isolates from the same IG, which paves
>     the way for faster communication and sharing of immutable objects.
> 
> All isolates within one IG will share the same IsolateGroupSource.
> 
> **Embedder changes**
> 
> This change makes breaking embedder API changes to support this new
> concept of Isolate Groups: The existing isolate lifecycle callbacks
> given to Dart_Initialize will become Isolate Group lifecycle callbacks.
> A new callback `initialize_isolate` callback will be added which can
> initialize a new isolate within an existing IG.
> 
> Existing embedders can be updated by performing the following renames
> 
>   Dart_CreateIsolate -> Dart_CreateIsolateGroup
>   Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
>   Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
>   Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
>   Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
>   Dart_IsolateData -> Dart_IsolateGroupData
>   Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
>   Dart_InitializeParams.create -> Dart_InitializeParams.create_group
>   Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
>   Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
> 
> By default `Isolate.spawn` will cause the creation of a new IG.
> 
> Though an embedder can opt-into supporting multiple isolates within one IG by
> providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
> The responsibility of this new callback is to initialize an existing
> isolate (which was setup by re-using source code from the spawning
> isolate - i.e. the one which used `Isolate.spawn`) by setting native
> resolvers, initializing global state, etc.
> 
> Issue https://github.com/dart-lang/sdk/issues/36648
> Issue https://github.com/dart-lang/sdk/issues/36097
> 
> Original review: https://dart-review.googlesource.com/c/sdk/+/105241
> 
> Difference to original review:
> 
>   * Give each isolate it's own [Loader] (for now)
>   * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
>   * Fix IsolateData memory leak if isolate startup fails
> 
> Change-Id: I98277d3d10fe275aa9b8a16b6bdd446bbea0b100
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107506
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ia4e0f4f9fc317499d3570a371c5bdf9aed799e77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/108101
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2019-07-03 12:19:15 +00:00
Martin Kustermann 67ab3be10d Reland "[vm/concurrency] Introduce concept of Isolate Groups"
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:

  * have a guarantee that all isolates within one IG can safely exchange
    structured objects (currently we rely on embedder for this
    guarantee)

  * hot-reload all isolates together (currently we only reload one
    isolate, leaving same-source isolates in inconsistent state)

  * make a shared heap for all isolates from the same IG, which paves
    the way for faster communication and sharing of immutable objects.

All isolates within one IG will share the same IsolateGroupSource.

**Embedder changes**

This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.

Existing embedders can be updated by performing the following renames

  Dart_CreateIsolate -> Dart_CreateIsolateGroup
  Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
  Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
  Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
  Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
  Dart_IsolateData -> Dart_IsolateGroupData
  Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
  Dart_InitializeParams.create -> Dart_InitializeParams.create_group
  Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
  Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate

By default `Isolate.spawn` will cause the creation of a new IG.

Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.

Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097

Original review: https://dart-review.googlesource.com/c/sdk/+/105241

Difference to original review:

  * Give each isolate it's own [Loader] (for now)
  * Sort classes during initialization for spawned isolates if app-jit is used (to match main isolate)
  * Fix IsolateData memory leak if isolate startup fails

Change-Id: I98277d3d10fe275aa9b8a16b6bdd446bbea0b100
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107506
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-06-29 18:45:11 +00:00
Martin Kustermann b2171f16f1 Revert "[vm/concurrency] Introduce concept of Isolate Groups"
This reverts commit b75057b687.

Reason for revert: Caused some failures on app-jit builders

Original change's description:
> [vm/concurrency] Introduce concept of Isolate Groups
> 
> An Isolate Group (IG) is a collection of isolates which were spawned from the
> same source. This allows the VM to:
> 
>   * have a guarantee that all isolates within one IG can safely exchange
>     structured objects (currently we rely on embedder for this
>     guarantee)
> 
>   * hot-reload all isolates together (currently we only reload one
>     isolate, leaving same-source isolates in inconsistent state)
> 
>   * make a shared heap for all isolates from the same IG, which paves
>     the way for faster communication and sharing of immutable objects.
> 
> All isolates within one IG will share the same IsolateGroupSource.
> 
> **Embedder changes**
> 
> This change makes breaking embedder API changes to support this new
> concept of Isolate Groups: The existing isolate lifecycle callbacks
> given to Dart_Initialize will become Isolate Group lifecycle callbacks.
> A new callback `initialize_isolate` callback will be added which can
> initialize a new isolate within an existing IG.
> 
> Existing embedders can be updated by performing the following renames
> 
>   Dart_CreateIsolate -> Dart_CreateIsolateGroup
>   Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
>   Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
>   Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
>   Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
>   Dart_IsolateData -> Dart_IsolateGroupData
>   Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
>   Dart_InitializeParams.create -> Dart_InitializeParams.create_group
>   Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
>   Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate
> 
> By default `Isolate.spawn` will cause the creation of a new IG.
> 
> Though an embedder can opt-into supporting multiple isolates within one IG by
> providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
> The responsibility of this new callback is to initialize an existing
> isolate (which was setup by re-using source code from the spawning
> isolate - i.e. the one which used `Isolate.spawn`) by setting native
> resolvers, initializing global state, etc.
> 
> Issue https://github.com/dart-lang/sdk/issues/36648
> Issue https://github.com/dart-lang/sdk/issues/36097
> 
> Change-Id: I82437ac017ca33018d45e02f353b0672db155f6a
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105241
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com

Change-Id: Ibd90992a01d61188f27b445c21532e0c46f996ea
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/107405
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-06-26 16:04:27 +00:00
Martin Kustermann b75057b687 [vm/concurrency] Introduce concept of Isolate Groups
An Isolate Group (IG) is a collection of isolates which were spawned from the
same source. This allows the VM to:

  * have a guarantee that all isolates within one IG can safely exchange
    structured objects (currently we rely on embedder for this
    guarantee)

  * hot-reload all isolates together (currently we only reload one
    isolate, leaving same-source isolates in inconsistent state)

  * make a shared heap for all isolates from the same IG, which paves
    the way for faster communication and sharing of immutable objects.

All isolates within one IG will share the same IsolateGroupSource.

**Embedder changes**

This change makes breaking embedder API changes to support this new
concept of Isolate Groups: The existing isolate lifecycle callbacks
given to Dart_Initialize will become Isolate Group lifecycle callbacks.
A new callback `initialize_isolate` callback will be added which can
initialize a new isolate within an existing IG.

Existing embedders can be updated by performing the following renames

  Dart_CreateIsolate -> Dart_CreateIsolateGroup
  Dart_IsolateCreateCallback -> Dart_IsolateGroupCreateCallback
  Dart_IsolateCleanupCallback -> Dart_IsolateGroupShutdownCallback
  Dart_CreateIsolateFromKernel -> Dart_CreateIsolateGroupFromKernel
  Dart_CurrentIsolateData -> Dart_CurrentIsolateGroupData
  Dart_IsolateData -> Dart_IsolateGroupData
  Dart_GetNativeIsolateData -> Dart_GetNativeIsolateGroupData
  Dart_InitializeParams.create -> Dart_InitializeParams.create_group
  Dart_InitializeParams.cleanup -> Dart_InitializeParams.shutdown_group
  Dart_InitializeParams.shutdown -> Dart_InitializeParams.shutdown_isolate

By default `Isolate.spawn` will cause the creation of a new IG.

Though an embedder can opt-into supporting multiple isolates within one IG by
providing a callback to the newly added `Dart_InitializeParams.initialize_isolate`.
The responsibility of this new callback is to initialize an existing
isolate (which was setup by re-using source code from the spawning
isolate - i.e. the one which used `Isolate.spawn`) by setting native
resolvers, initializing global state, etc.

Issue https://github.com/dart-lang/sdk/issues/36648
Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I82437ac017ca33018d45e02f353b0672db155f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105241
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2019-06-26 11:34:48 +00:00
Ryan Macnak 2be92b08c2 [vm] Remove more of the Dart 1 compiler.
Change-Id: Ib00e040f44275b9437e7fc622bf285607f4e4124
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/105681
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-06-12 01:00:29 +00:00
Martin Kustermann 832668ff6c Reland "[Infra] Let run_vm_tests --list output a test expectation marker"
Due to not having support for __VA_OPT__ yet the CL introduces a new
wet of macros ..._WITH_EXPECTATIONS() which can be given an expectation
marker.

Change-Id: I33812937f1b226fa89b3ab17a8a3483914abf2e4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100643
Reviewed-by: Stevie Strickland <sstrickl@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-04-26 12:15:19 +00:00
Martin Kustermann 76c6b86915 Revert "[Infra] Let run_vm_tests --list output a test expectation marker"
This reverts commit 7cb73d97d2.

Reason for revert: It caused windows build to fail (reason unknown):

c:\b\s\w\ir\cache\builder\sdk\runtime\vm\compiler\assembler\assembler_x64_test.cc(1097): error C3861: 'AssemblerTestGenerate_BitScanReverse': identifier not found
c:\b\s\w\ir\cache\builder\sdk\runtime\vm\compiler\assembler\assembler_x64_test.cc(5499): error C3861: 'AssemblerTestGenerate_bittest': identifier not found

Original change's description:
> [Infra] Let run_vm_tests --list output a test expectation marker
> 
> Due to not having support for __VA_OPT__ yet the CL introduces a new
> wet of macros ..._WITH_EXPECTATIONS() which can be given an expectation
> marker.
> 
> Change-Id: Icaac4672f04340fe4644d13a14c32704ba36daec
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96940
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Jonas Termansen <sortie@google.com>
> Reviewed-by: Régis Crelier <regis@google.com>

TBR=sortie@google.com,kustermann@google.com,regis@google.com

Change-Id: Ica07ef5e7302e3d8e4b131427ff5ab20be8c6a63
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100581
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2019-04-25 14:35:45 +00:00
Martin Kustermann 7cb73d97d2 [Infra] Let run_vm_tests --list output a test expectation marker
Due to not having support for __VA_OPT__ yet the CL introduces a new
wet of macros ..._WITH_EXPECTATIONS() which can be given an expectation
marker.

Change-Id: Icaac4672f04340fe4644d13a14c32704ba36daec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96940
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2019-04-25 12:07:01 +00:00
Martin Kustermann 8e4dcf48c4 [vm/compiler/aot] Fix unboxing of int32/uint32 in typed data specializer
Issue https://github.com/dart-lang/sdk/issues/36521

Change-Id: I206a16479f734bf757b4641ca51adc7a42040888
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98841
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2019-04-08 14:56:00 +00:00
Vyacheslav Egorov f496e538f4 [vm] Decouple assemblers from runtime.
This is the next step towards preventing compiler from directly peeking
into runtime and instead interact with runtime through a well defined
surface. The goal of the refactoring to locate all places where compiler
accesses some runtime information and partion those accesses into two
categories:

- creating objects in the host runtime (e.g. allocating strings, numbers, etc)
during compilation;
- accessing properties of the target runtime (e.g. offsets of fields) to
embed those into the generated code;

This change introduces dart::compiler and dart::compiler::target namespaces.

All code in the compiler will gradually be moved into dart::compiler namespace.
One of the motivations for this change is to be able to prevent access to
globally defined host constants like kWordSize by shadowing them in the
dart::compiler namespace.

The nested namespace dart::compiler::target hosts all information about
target runtime that compiler could access, e.g. compiler::target::kWordSize
defines word size of the target which will eventually be made different
from the host kWordSize (defined by dart::kWordSize).

The API for compiler to runtime interaction is placed into compiler_api.h.

Note that we still permit runtime to access compiler internals directly -
this is not going to be decoupled as part of this work.

Issue https://github.com/dart-lang/sdk/issues/31709

Change-Id: If4396d295879391becfa6c38d4802bbff81f5b20
Reviewed-on: https://dart-review.googlesource.com/c/90242
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2019-01-25 16:45:13 +00:00
Liam Appelbe f8a680e511 Revert "Remove kernel isolate API from the public dart_api.h"
This reverts commit 9906819fb7.

Reason for revert: Follow up CLs are more complicated than I thought, so I don't want to leave it in an inconsistent state.

Original change's description:
> Remove kernel isolate API from the public dart_api.h
> 
> This is the first step in moving the kernel isolate out
> of the VM and into the standalone embedder.
> 
> Bug: https://github.com/dart-lang/sdk/issues/33433
> Change-Id: Ie8d9ac1c27efe2661f0441b75275119966d197af
> Reviewed-on: https://dart-review.googlesource.com/c/84829
> Commit-Queue: Liam Appelbe <liama@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=rmacnak@google.com,liama@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: https://github.com/dart-lang/sdk/issues/33433
Change-Id: Iaf753c210e918df167364b50ee823b2557896628
Reviewed-on: https://dart-review.googlesource.com/c/87623
Reviewed-by: Liam Appelbe <liama@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2018-12-18 20:20:42 +00:00
Liam Appelbe 9906819fb7 Remove kernel isolate API from the public dart_api.h
This is the first step in moving the kernel isolate out
of the VM and into the standalone embedder.

Bug: https://github.com/dart-lang/sdk/issues/33433
Change-Id: Ie8d9ac1c27efe2661f0441b75275119966d197af
Reviewed-on: https://dart-review.googlesource.com/c/84829
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-11-26 22:47:20 +00:00
Ryan Macnak 7514ce941e [vm] Assert handles are only allocated when the thread is preventing safepoints.
Threads in the native or blocked states don't prevent safepoints, so they may run concurrently with a safepoint operation like GC. It is not safe for handles to be allocated while the GC is visiting them, so these threads must not allocate handles. Assert only threads in the VM or generated states, which prevent safepoints until they check in, may allocate handles. (Generated code does not allocate handles, but leaf runtime entries remain in the generated state.)

Bug: https://github.com/dart-lang/sdk/issues/34883
Change-Id: I1a211778f7ef96b53a2405f0ee9dde7871b122b6
Reviewed-on: https://dart-review.googlesource.com/c/81540
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-10-31 19:51:52 +00:00
Ryan Macnak 235a918233 [vm] Remove Dart_LoadScript, Dart_LoadLibrary, Dart_LoadSource, etc.
Change-Id: I9c19625702f1df48c03de2eb3a0f7bb581d2e0b3
Reviewed-on: https://dart-review.googlesource.com/c/79140
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-10-11 00:38:37 +00:00
Ryan Macnak b8760fed67 [vm, compiler] Remove ASTs and AST flow graph builder.
Change-Id: I4d3c6500577d5285ced327954040dab42ba92341
Reviewed-on: https://dart-review.googlesource.com/c/78823
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-10-10 18:00:21 +00:00
Ben Konyi 07a8496919 [VM] Allow for Dart_SetVMFlags to be called again after Dart_Cleanup
Change-Id: Icf07258cfb05a987e0a439277a59825c06f993b2
Reviewed-on: https://dart-review.googlesource.com/77486
Auto-Submit: Ben Konyi <bkonyi@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-10-02 00:33:37 +00:00
Ben Konyi 6f3de544ed [ VM / Hot Reload ] Fixed tests which were crashing periodically. Fixes issue #34599
Change-Id: I053701122f4a95ae3d9f27b91134433201bfe8aa
Reviewed-on: https://dart-review.googlesource.com/77004
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Auto-Submit: Ben Konyi <bkonyi@google.com>
2018-09-27 21:47:39 +00:00
asiva 76470aca47 [VM] use coresnapshot when initializing an Isolate
- use core snapshot when initializing an isolate (CorelibIsolateStartup benchmark moves from 39s to 2s)
- remove use_dart_frontend flag from the isolate specific flags
- the script snapshot API functions now return an error

Change-Id: Ia562e007fdbfd1d3ebd8e1a0c8feee238bada00b
Reviewed-on: https://dart-review.googlesource.com/76709
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-27 00:55:07 +00:00
Ben Konyi 3c4c62d79a Reland "[VM] Dart_Initialize no longer crashes after Dart_Cleanup"
This is a reland of 519ee905f9

Original change's description:
> [VM] Dart_Initialize no longer crashes after Dart_Cleanup
> 
> Change-Id: I3cfdab9553aad045f024b6f9aec0b40b08234007
> Reviewed-on: https://dart-review.googlesource.com/75786
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ben Konyi <bkonyi@google.com>

Change-Id: Icdd901bf1ab0b675cc5f1497061a2b7b8a05d956
Reviewed-on: https://dart-review.googlesource.com/76561
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-26 17:54:45 +00:00
Alexander Aprelev cf055d6b88 [vm] Fix isolate reload tests so that they can pass kernel files for isolate reload.
Change-Id: I695761a790354eacc0d617cb914d7a4ea2b652aa
Reviewed-on: https://dart-review.googlesource.com/76300
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-09-26 14:10:53 +00:00
Ben Konyi b6284b41e3 Revert "[VM] Dart_Initialize no longer crashes after Dart_Cleanup"
This reverts commit 519ee905f9.

Reason for revert: Seeing multiple crashes on Windows

Original change's description:
> [VM] Dart_Initialize no longer crashes after Dart_Cleanup
> 
> Change-Id: I3cfdab9553aad045f024b6f9aec0b40b08234007
> Reviewed-on: https://dart-review.googlesource.com/75786
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Ben Konyi <bkonyi@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,zra@google.com

Change-Id: I33ad79dbc3fcf44f93612ff63bd2d8431f6067c8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/76342
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-09-25 02:17:37 +00:00
Ben Konyi 519ee905f9 [VM] Dart_Initialize no longer crashes after Dart_Cleanup
Change-Id: I3cfdab9553aad045f024b6f9aec0b40b08234007
Reviewed-on: https://dart-review.googlesource.com/75786
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-09-25 01:20:24 +00:00
Martin Kustermann df2d187f74 [VM] Move the ObjectPoolWrapper out of the Assembler
Issue https://github.com/dart-lang/sdk/issues/33274

Change-Id: I2e295a357939bc7c966c0658c344fcc4268669c1
Reviewed-on: https://dart-review.googlesource.com/71166
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-08-23 08:00:15 +00:00
Samir Jindel 66278e3919 [vm/precomp] Change checked/unchecked terminology to monomorphic/-
Change-Id: Ib54db47d60e7b822a4c73995998aecb6780a4cce
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-release-x64-try,vm-kernel-optcounter-threshold-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/69306
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-08-10 14:32:37 +00:00
Alexander Aprelev f67cba1067 [vm] Introduce kernel_multiroot options to support multiroot compilation mode in kernel service.
This extends existing support for multiroot compilation from frontend server to kernel service.

Bug: dartbug.com/33508
Change-Id: I76e49038765793b4d683869ab812dde5e6baf6c4
Reviewed-on: https://dart-review.googlesource.com/61982
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-06-26 00:56:56 +00:00
Ryan Macnak 8da46d35f5 [vm] Move heap-related code to its own subdirectory (cf. compiler).
Remove some dead includes.

Change-Id: I31f3e739e5ee46dcbba5d6a2f091491b46402943
Reviewed-on: https://dart-review.googlesource.com/60146
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-06-20 00:39:49 +00:00
Alexander Aprelev ea837c91a0 Transition tests to EvaluateCompiledExpression.
This is follow-up to https://dart-review.googlesource.com/c/sdk/+/58660.

Change-Id: Ic0ae58eb34dabe6e0902f37c8e817f949ed4ac64
Reviewed-on: https://dart-review.googlesource.com/60705
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-06-19 02:41:00 +00:00
Alexander Markov c3b5939965 [vm] Remove --limit-ints-to-64-bits option and old _Bigint class
Closes https://github.com/dart-lang/sdk/issues/33306

Change-Id: I7088d8b7143edbe24f5cefe4be037ad2006e0625
Reviewed-on: https://dart-review.googlesource.com/58101
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-06-04 20:10:40 +00:00
Ben Konyi 6d8f473d4f [ VM / Hot Reload ] Updated kernel_service and hot reload behavior to run as long as the compiler returns valid kernel.
Change-Id: Id2b20d95488ccd374f4ca337355dde9f5057292a
Reviewed-on: https://dart-review.googlesource.com/46820
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-05-22 23:50:28 +00:00
Samir Jindel 39c93a2b19 Revert "Revert "[vm/kernel] Expression execution through Kernel."
This reverts commit b5154dd01e.

Original revision is in patchset 1.

Change-Id: Ic0776011c6fdbd64c025086535351474a9a88b6e
Reviewed-on: https://dart-review.googlesource.com/54407
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-05-15 12:03:22 +00:00
Ryan Macnak a5c11d7d03 [vm] Remove Dart_ReadKernelBinary, take 2.
Update status for simulators, which the test harness invokes with DIL files instead of source files.

Change-Id: I8444ad7e17a0a71a1ce3c0021487397baa1c3e65
Reviewed-on: https://dart-review.googlesource.com/54622
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-05-14 20:58:08 +00:00
Ryan Macnak 407872b9e5 Revert "[vm] Remove Dart_ReadKernelBinary."
This reverts commit 2998c32951.

Reason for revert: Failures with kernel + simulators

Original change's description:
> [vm] Remove Dart_ReadKernelBinary.
> 
>  - Fix const-ness of Dart_FileReadCallback and callers.
>  - Fix leak on read error in DartUtils::FileRead.
> 
> This is progress towards sharing kernel memory on Fuchsia.
> 
> Bug: https://github.com/dart-lang/sdk/issues/32618
> Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
> Reviewed-on: https://dart-review.googlesource.com/47102
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Ibb636b817de1629a062fe0a6ef9462a2fef669ef
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: https://github.com/dart-lang/sdk/issues/32618
Reviewed-on: https://dart-review.googlesource.com/54560
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-05-10 01:36:39 +00:00
Ryan Macnak 2998c32951 [vm] Remove Dart_ReadKernelBinary.
- Fix const-ness of Dart_FileReadCallback and callers.
 - Fix leak on read error in DartUtils::FileRead.

This is progress towards sharing kernel memory on Fuchsia.

Bug: https://github.com/dart-lang/sdk/issues/32618
Change-Id: I47f8b224905d6a105a5ca0ab2ee4ab6a42b5e342
Reviewed-on: https://dart-review.googlesource.com/47102
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-05-10 00:35:37 +00:00
Samir Jindel b5154dd01e Revert "[vm/kernel] Expression execution through Kernel.
This reverts commit afd0a52a5a.

Reason for revert: Breaks kernel-a32 and kernel-rbc bots

Original change's description:
> [vm/kernel] Expression execution through Kernel.
> 
> Change-Id: Id90c91cb3b71f9ce703e21fe48147b86a7764d34
> Reviewed-on: https://dart-review.googlesource.com/42562
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Samir Jindel <sjindel@google.com>

TBR=aam@google.com,rmacnak@google.com,sjindel@google.com

Change-Id: If1ecad6145aeab81d5d3581798a628ead0265c38
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/54420
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2018-05-09 12:41:00 +00:00
Samir Jindel afd0a52a5a [vm/kernel] Expression execution through Kernel.
Change-Id: Id90c91cb3b71f9ce703e21fe48147b86a7764d34
Reviewed-on: https://dart-review.googlesource.com/42562
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2018-05-09 10:27:52 +00:00