Commit Graph

1132 Commits

Author SHA1 Message Date
Daco Harkes 8a2b7bb5d5 [vm/ffi] Support Unions
Closes: https://github.com/dart-lang/sdk/issues/38491

tools/test.py ffi ffi_2
TEST=tests/ffi(_2)/(.*)by_value_(*.)_test.dart

Change-Id: I6d29f7e3c3046cda4a15c9b42063c66b064787be
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194420
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2021-04-15 10:16:27 +00:00
Daco Harkes 4558112105 Rereland "[vm/ffi] Disallow Pointers and structs in finalizers and expandos"
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.

Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.

Cleans up unused object_store->ffi_struct_class.

Reland 1: Allow importing `dart:ffi` from `dart:core` with
          `--enable-ffi=false`.
Reland 2: Allow the new `_Compound` class to extend `NativeType`. (This
          got triggered in this CL on the build because of the import
          from `dart:core`.)

Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072

TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart

Change-Id: I133278e16bd75cd2bb6234e7ddf042ffa0a54fd6
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195079
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-04-14 12:07:18 +00:00
Daco Harkes c33dad37e5 Revert "Reland "[vm/ffi] Disallow Pointers and structs in finalizers and expandos""
This reverts commit 3f0ad61daa.

Reason for revert: Conflicts with https://dart-review.googlesource.com/c/sdk/+/194765

Original change's description:
> Reland "[vm/ffi] Disallow `Pointer`s and structs in finalizers and expandos"
>
> `Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
> `dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
> as values passed in to the `object` parameter.
>
> Expandos no longer accept `Pointer`s and subtypes of `Struct`s
> as values passed in to the `object` parameter.
>
> Cleans up unused object_store->ffi_struct_class.
>
> Closes: https://github.com/dart-lang/sdk/issues/45071
> Breaking change: https://github.com/dart-lang/sdk/issues/45072
>
> TEST=vm/cc/DartAPI_FinalizableHandleErrors
> TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
> TEST=tests/ffi(_2)/expando_test.dart
>
> Change-Id: I9af6d0173db60614091068c218391f73756c135f
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195061
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Commit-Queue: Daco Harkes <dacoharkes@google.com>

TBR=kustermann@google.com,dacoharkes@google.com

Change-Id: I90064b6b73155a43f37388f987c6b29f72ce9770
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195076
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-04-13 15:05:07 +00:00
Daco Harkes 3f0ad61daa Reland "[vm/ffi] Disallow Pointers and structs in finalizers and expandos"
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.

Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.

Cleans up unused object_store->ffi_struct_class.

Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072

TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart

Change-Id: I9af6d0173db60614091068c218391f73756c135f
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195061
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-04-13 13:52:36 +00:00
Daco Harkes ac2412ca01 Revert "[vm/ffi] Disallow Pointers and structs in finalizers and expandos"
This reverts commit 2376ab5186.

Reason for revert: Importing dart:ffi from dart:core breaks the VM when using --enable-ffi=false

Original change's description:
> [vm/ffi] Disallow `Pointer`s and structs in finalizers and expandos
>
> `Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
> `dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
> as values passed in to the `object` parameter.
>
> Expandos no longer accept `Pointer`s and subtypes of `Struct`s
> as values passed in to the `object` parameter.
>
> Cleans up unused object_store->ffi_struct_class.
>
> Closes: https://github.com/dart-lang/sdk/issues/45071
> Breaking change: https://github.com/dart-lang/sdk/issues/45072
>
> TEST=vm/cc/DartAPI_FinalizableHandleErrors
> TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
> TEST=tests/ffi(_2)/expando_test.dart
>
> Change-Id: I1f11adfa073c7d2c979f3c2bb15c7444c7c767a0
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186280
> Commit-Queue: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=lrn@google.com,vegorov@google.com,kustermann@google.com,dacoharkes@google.com

Change-Id: Ib76d27e59391dc6107d3f8e8fba3d67640ea9a5c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195060
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2021-04-12 16:30:57 +00:00
Daco Harkes 2376ab5186 [vm/ffi] Disallow Pointers and structs in finalizers and expandos
`Dart_NewWeakPersistentHandle` and `Dart_NewFinalizableHandle` in
`dart_api.h` do no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.

Expandos no longer accept `Pointer`s and subtypes of `Struct`s
as values passed in to the `object` parameter.

Cleans up unused object_store->ffi_struct_class.

Closes: https://github.com/dart-lang/sdk/issues/45071
Breaking change: https://github.com/dart-lang/sdk/issues/45072

TEST=vm/cc/DartAPI_FinalizableHandleErrors
TEST=vm/cc/DartAPI_WeakPersistentHandleErrors
TEST=tests/ffi(_2)/expando_test.dart

Change-Id: I1f11adfa073c7d2c979f3c2bb15c7444c7c767a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186280
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-04-12 14:04:16 +00:00
Alexander Aprelev 9302a7427b [vm/concurrency] Assert that no SafepointRwLocks are acquired while in SafepointOperationScope.
Attempts to acquire safepoint lock while in SafepointOperationScope could result
in deadlocks if somebody else was holding that lock when they were forced to a safepoint.
Clean up few places where locks were acquired in SafepointOperationScope.

Introduce StoppedMutatorsScope and GroupDebugger::RunUnderReadLockIfNeeded that acquires
a lock only if it runs outside of StoppedMutatorsScope - to prevent such deadlocks.

Also fix tsan warning about data race around message_notify_callback by making it atomic.

TEST=tsan runs of debugger CI tests

Fixes https://github.com/dart-lang/sdk/issues/45527
Fixes https://github.com/dart-lang/sdk/issues/45549

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: Ibb53d9ce760d869e044e17075aeebf20fc0016a5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193582
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-04-03 03:36:32 +00:00
Alexander Aprelev 7b94d56d67 [observatory] Provide isolate for send data events because observatory expects it for write events.
_ConsoleStreamPrinter::onEvent expects isolate on WriteEvent. It's absence causes exceptions in Observatory. This started with https://dart-review.googlesource.com/c/sdk/+/190303

TEST=updated capture_stdio_test with isolate expectations; run observatory to ensure to exceptions are generated

Change-Id: I73efe9c1ab5b0d779d9432fd64ad5ccc5fcc14ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191661
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-03-17 23:30:47 +00:00
Alexander Markov 16beb34813 [vm/aot] Remove special cases of dynamic function lookups from precompiler
Flutter gallery AOT snapshot size
release: no change;
release-sizeopt: arm -0.24%, arm64 -0.25%.

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I847bfc311552bb4e80222e0040b50b4766a18dbe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190783
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-03-15 17:53:23 +00:00
Ben Konyi dbbd71cd75 [ VM ] Remove requirement for presence of current isolate for
Dart_ServiceSendDataEvent

Fixes https://github.com/dart-lang/sdk/issues/44720

Fixed: 44720
TEST=Existing dart:io tests
Change-Id: I003a339a91dca84dafc39c44ba804ecf3f6f84ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190303
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-03-10 00:42:37 +00:00
Alexander Aprelev c22205c14e [vm/concurrency] Ensure write program_lock is held while updating class state via set_state.
Fix tests and AllocateObject.

Hopefully addresses flaky https://github.com/dart-lang/sdk/issues/44940

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=ci and standalone_2/out_of_memory_recovery_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try

Change-Id: Ife775f7a0bcdd343b887ba1bf912e07b2a8430c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188362
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-02 16:28:12 +00:00
Martin Kustermann 92af830255 [vm/concurrency] Move background compiler related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`. This CL moves background
compiler and related state.

One background compiler will be shared for all isolates within a group.

Issue https://github.com/dart-lang/sdk/issues/36097
Fixes https://github.com/dart-lang/sdk/issues/44938

TEST=Mainly refactoring, stress tests for --enable-isolate-groups with JIT will come in the future.

Change-Id: I66e863651d0b37c431ce1864e505b5413850cbde
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183401
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-02-12 03:40:14 +00:00
Martin Kustermann 75e0837f3e [vm/concurrency] Simplify logic around background compiler, allow concurrent disabling
The logic for background compiler can be simplified due to:

  * we only have one bg compiler
  * it always optimizes
  * it is allocated at isolate startup and freed on isolate shutdown
  * an existing check in the compiler for whether the BG compiler is
    disabled is racy and therefore not helpful
  * when a mutator communicates with BG compiler it will use
    SafepointMutexLocker instead of MutexLocker

This CL also ecapsulates disabling the BG compiler via a
NoBackgroundCompilerScope object. It allows nested disabling and guards
against races.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I36ed6592c9a8b91e94181698b7de662b4ee984e9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183400
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-10 19:42:55 +00:00
Ben Konyi fbbc99e1e9 [ VM ] Ensure Dart C APIs perform type checks when invoking functions
and constructors

Dart_New, Dart_InvokeConstructor, and Dart_InvokeClosure were not
checking the types of arguments before performing invocations.

Fixes https://github.com/dart-lang/sdk/issues/44205.

TEST=DartAPI_*_Issue44205

Change-Id: I6c06e3e8a0a88b9caa7e2212c4a2c8e43d39b4f4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182962
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-08 23:30:05 +00:00
Alexander Aprelev 6b3abe0a98 [vm/concurrency] Add --experimental_enable_isolate_groups_jit flag, make --enable_isolate_groups AOT only.
Issue b/177800357.

TEST=existing test suite
Change-Id: Id6b113932ab7014b8fa6c105845c5c490b60f5e8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181320
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-27 17:11:29 +00:00
asiva ac520f984d [ VM / CLI ] Add --verbosity to VM options
The verbosity option allows for specifying the CFE output verbosity

Fixes https://github.com/dart-lang/sdk/issues/44727

TEST=existing tests compile_test.dart and run_test.dart have new tests

Change-Id: I3d4e50811f84650aacf774ddb370a6eb765b9b24
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181100
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-01-26 03:01:58 +00:00
Johnni Winther 55b34a467c Reland "[vm] Pass snapshot flag to kernel_service"
Enable reporting of null safety compilation mode when running
`dart compile aot-snapshot`, `dart compile jit-snapshot`,
and `dart compile kernel`.

Closes #44234

TEST=pkg/dartdev/test/commands/compile_test.dart

Change-Id: Id9e72751068d5f208055ab953d2c8392e84c4798
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180187
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-21 21:18:32 +00:00
Alexander Aprelev efa1e18428 [vm/concurrency] Allow class finalization done by background compiler
This CL removes asserts that prevented background compiler from doing class finalization.
It adds program_lock synchronization for when object properties are updated during class finalization.
It also encloses code instructions installation into safepoint scopes to ensure that no code is executed during installation.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=existing test suite

Change-Id: If297e44f51b187242eca2cdcfff066c8f6386d97
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178163
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-20 20:08:25 +00:00
Aske Simon Christensen 903e92a196 Revert "[vm] Pass snapshot flag to kernel_service"
This reverts commit f8b0d26cc3.

Reason for revert: Multiple test failures in app-jit and simarm(64)

Original change's description:
> [vm] Pass snapshot flag to kernel_service
>
> Enable reporting of null safety compilation mode when running
> `dart compile aot-snapshot`, `dart compile jit-snapshot`,
> and `dart compile kernel`.
>
> Closes #44234
>
> TEST=pkg/dartdev/test/commands/compile_test.dart
>
> Change-Id: I0d4b35c6ccb4167c0c7539a4eb24a5139e29cf53
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178990
> Commit-Queue: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>

TBR=bkonyi@google.com,johnniwinther@google.com

Change-Id: I83aeaa8620c640d02d5ccfd1fe8112d8209a9ad4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179773
Auto-Submit: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-18 21:00:03 +00:00
Johnni Winther f8b0d26cc3 [vm] Pass snapshot flag to kernel_service
Enable reporting of null safety compilation mode when running
`dart compile aot-snapshot`, `dart compile jit-snapshot`,
and `dart compile kernel`.

Closes #44234

TEST=pkg/dartdev/test/commands/compile_test.dart

Change-Id: I0d4b35c6ccb4167c0c7539a4eb24a5139e29cf53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178990
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-01-18 09:27:59 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00
Regis Crelier b3bca1821c [VM/runtime] Decouple representation of function signatures from Function objects.
This fixes issue https://github.com/dart-lang/sdk/issues/43088

This CL improves on https://dart-review.googlesource.com/c/sdk/+/166920 by keeping the names of type parameters.

See go/dart-vm-signatures for motivation for this change.

TEST=existing suites

Change-Id: Ia82f1d322f72d07b2fb253cf914f9b868025c71e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176981
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-07 23:45:37 +00:00
Martin Kustermann 0f76981bb1 [vm/concurrency] Change references to class_table/object_store/heap from Isolate to IsolateGroup
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.

The class_table and object_store were already moved to `IsolateGroup`.

This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 15:22:11 +00:00
Martin Kustermann 12d7707179 [vm/concurrency] Move null safety related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`.

This CL moves null safety related state.

TEST=Pure refactoring - relying on existing test coverage.

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: Ie0b19b8ef84768c9665326c4f955c956c6ba60bb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177123
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 12:20:40 +00:00
Martin Kustermann 18971e409f [vm/concurrency] Move obfuscation related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`.

This CL moves obfuscation related state.

TEST=Pure refactoring - relying on existing test coverage.

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I4b0a74bff9abc200cf8f4ce76b3e1d3d688e664a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177122
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 11:12:40 +00:00
Alexander Aprelev fce355b8b0 [vm/fields] Always clone fields when running an optimizing compiler, even when it runs on the mutator thread.
Attempts to run class finalizaton on background thread revealed the problem where during optimization(which could run on mutator thread too) we might need to finalize previously unseen class, load annotatons for it, load and set some constant fields. This could result in updating Fields guarded_cid, requires check during installation of newly produced optimized code. Because mutator thread is currently exempt from cloning and validating fields guards, that could result in invalid optimized code installation.

This CL ensures that fields are always cloned in optimizing compiler even when it runs on mutator thread.

Ability to run multiple optimizing compilers is also needed for dartbug.com/36097.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=existing test suite with ltw isolate groups enabled, https://dart-review.googlesource.com/c/sdk/+/176300 mocked up test(still to be improved).

Change-Id: I608f4abff30b468cb9d0ab270221a0ba72c32470
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175341
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-17 00:22:13 +00:00
Martin Kustermann 425780f410 [vm/concurrency] Share program structure and JITed code with --enable-isolate-groups
This removes our temporary scaffolding support for JIT isolate groups
(which was implemented by creating a new isolate group, loading the
application kernel into it and then merging the heap into the original
isolate group - maintaining a different object store)

It makes all isolates within a group share the same object store, same
libraries and JITed code. It will be conservative to start with, only
allow running unoptimized code, etc.

We will gradually remove the restrictions imposed by this CL:
  https://dart-review.googlesource.com/c/sdk/+/173970

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Tests using --enable-isolate-groups with JIT sharing.

Change-Id: I2bf69a6fe3c905067c4cec2e81613f731c52e5ee
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175302
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-12-09 09:39:59 +00:00
Ryan Macnak 1b160f79fe [vm] Gracefully handle loads not solicited by prefix.loadLibrary().
TEST=flutter
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ia0e99c5e84cdff8736c44e4e82ce40b045ed0207
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174563
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-12-08 22:14:25 +00:00
Ryan Macnak abbb51e1ff [vm] Clean up metadata tracking.
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.

Retain some sloppiness matching of declarations by name instead of identity for the sake of hot reload.

TEST=ci
Change-Id: Ifa4cc7724096c606f994b057b7838e124ceb3626
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175141
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-07 19:43:48 +00:00
Alexander Markov 4ab36120d7 [vm] Fix finalization of classes when instances are allocated
Classes should be allocate-finalized before instance can be allocated
in order to properly update class hierarchy information and invalidate
code which was speculatively optimized.

TEST=runtime/tests/vm/dart/regress_44342_test.dart

Fixes https://github.com/dart-lang/sdk/issues/44342

Change-Id: I52a415f4cf25010eb23be339266a91ae6b7d7261
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175106
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-12-04 23:54:37 +00:00
Alexander Aprelev b67a585d6c [vm/isolate] Provide more verbose information when second mutator thread attempts to enter an isolate.
This should help with troubleshooting vm crashes caused by multiple mutator threads entering an isolate.

TEST=flutter/engine runtime_unittests that exhibits this flaky problem with DartIsolateTest.ValidLoadingUnitSucceeds

Change-Id: Ib503df844cceafa58882a695a1fe7ceaa88eaed4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174900
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-12-03 18:15:13 +00:00
Clement Skau c5eb666bcf Revert "[vm] Track metadata by identity instead of name."
This reverts commit bfd382820f.

Reason for revert: Breaks several lib_2/mirrors/ tests

Original change's description:
> [vm] Track metadata by identity instead of name.
>
> Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.
>
> TEST=ci
> Change-Id: Ic2e0c4408557514507fd732e0cb19b022b2539c7
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161631
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com

Change-Id: Icf75e91872d9c85ce9be29a9a178d72ca024c98b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174920
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2020-12-03 08:03:11 +00:00
Ryan Macnak bfd382820f [vm] Track metadata by identity instead of name.
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.

TEST=ci
Change-Id: Ic2e0c4408557514507fd732e0cb19b022b2539c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161631
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-02 23:43:07 +00:00
Ryan Macnak ca5853779b [vm, gc] Route most heap reads through LoadPointer.
Provides a good point to apply a read barrier, pointer decompression, or TSAN annotations.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/44091
Change-Id: I4e6930264f8ef8399e261fd24fcab78731d6e29c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174484
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-02 21:08:58 +00:00
Martin Kustermann 33b7721c51 [vm/concurrency] Make Dart_CreateIsolateInGroup() return an error if --enable-isolate-groups is not passed
This is a safeguard to prevent accidental uses of this API by embedders
if those embedders do not pass --enable-isolate-groups.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=runtime/tests/vm/dart/isolates/dart_api_create_lightweight_isolate_test.dart

Change-Id: Iba667f0e9fbbb33de963fabb80da546df794257c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174404
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-12-01 10:35:54 +00:00
Clement Skau 2d36c85ff8 [vm] Removes support for --causal-async-stacks
All existing embedders have been opted into --lazy-async-stacks, the
VM also uses it as it's default in all configurations.

After this CL, any user of --causal-async-stacks will get an error
message when trying to use it.
=> In any such case, please simply remove the flag.

TEST=Exhaustive CQ.

Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ia440afcf2dba464aa8b8cf381b93bbac8eb9f8dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172564
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-01 07:27:14 +00:00
Martin Kustermann c8ebfcd739 [vm] Remove unused code related to RedirectionData
Since the transition from Source based to Kernel based frontend to the
Dart VM, we have not supported callable redirecting factories. This has
caused all removed code in this CL to be unused and untested!

The support for reifing redirection related information would mainly be
used by embedder / dart:mirrors. Right now its not used by anyone and
the kernel support was never implemented.

I suggest we remove the unused and untested code until a decision has
been made that we actually want to support it - in which case it can be
properly implemented, possibly by re-using some of the deleted code in
this CL.

TEST=CL removes unused code, no test result changes.

Change-Id: I7ed45c85b4efcc1e81ce44cbe08bb555e52101b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166853
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-11-17 08:51:30 +00:00
Martin Kustermann d9561e5a1d [vm/concurrency] Move Isolate.spawn()-related logic out of runtime/vm/ into runtime/lib
The Isolate.spawn() is part of our dart:isolate library implementation,
it should therefore preferrably live under runtime/lib folder.

This CL does that as well as removing some special message handling
logic by using the new `Dart_RunLoopAsync()` C API, which results in
net removal of code.

As a nice side-effect it also makes isolate spawning slightly faster due
to avoiding extra safepoint/vm<->native transitions.

Issue https://github.com/dart-lang/sdk/issues/44088

TEST=Changes existing implementation, have existing test coverage for it.

Change-Id: I00607a1436946552bbe12e8e23062e8f743d4f11
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171731
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-11-15 13:31:47 +00:00
Martin Kustermann bf4280ff75 [vm/concurrency] Add Dart_CreateLightweightIsolate/Dart_RunLoopAsync APIs
Dart_CreateLightweightIsolate API:

  This API can be used by embedders to create a lightweight isolate
  (inside an existing isolate group).

  It is the analogous to Dart_CreateIsolateGroup - though taking a
  parent isolate as parameter (inside whose IG we create a new isolate)
  instead of taking kernel/snapshot data.

  Right now this API works on AOT and returns an error in JIT, both cases
  are covered by the test.

Dart_RunLoopAsync:

  The API can be used by embedders to transfer ownership of an isolate to
  the VM, which will take care of running the message handling loop and
  shuts the isolate down once the last receive port has been closed.

  It does allow listening to error/exit events generated by the message
  loop implementation as well as allows setting errors-are-fatal,
  effectively the same event-loop related parameters from the
  `Isolate.spawn()` API, just in Dart.

  It's the embedders responsibility to first launch initial dart code
  which will take care of responding to events (i.e. directly/indirectly
  open a receive port) - otherwise the isolate cannot be talked to and
  would immediately shut down.

Since our vm/cc tests do support running in AOT, we use a hybrid
approach to test the functionality: We let a normal Dart test call a
small C wrapper using FFI, to create a lightweight isolate, run it on a
new thread and join that thread.

TEST=vm/dart{,_2}/isolates/dart_api_create_lightweight_isolate_test

Issue https://github.com/dart-lang/sdk/issues/36097
Closes https://github.com/dart-lang/sdk/issues/44088

Change-Id: Id77ba928793fdb517f6cb7e8130df98a0366ddd6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170983
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-11-12 10:22:16 +00:00
Martin Kustermann 42ac762949 [vm/concurrency] Fix inconsistency in Dart's spawn implementation
When Dart invokes Isolate.spawn the Dart VM calls out to the embedder to
create an isolate (group) or initialize it (in case of lightweight
isolates).

  * If the embedder doesn't implement this functionality or fails to
    perform the creation/initialization, it will signal this failure
    via the return value it gives to the VM in the call back
    implementation (i.e. Dart_InitializeParams.create_group and
    Dart_InitializeParams.initialize_isolate}).

  * If the embeder sucessfully completed it's work, the VM owns the
    isolate after the embedder call returns. The VM is then responsible
    for running it.

It is an undocumented invariant atm that the embedder is responsible
to make the isolate runnable if it signals the VM that isolate was
sucessfully created/initialized.
  => If it does not do that, we have effectively an isolate leak.
  => Though right now seemingly all of our embedders to that correctly.

To avoid this unintentional isolate leak we make the isolate runnable
ourselves if embedder callback was successful but did not make it
runnable itself.

  => This also avoids unnecessary ceremony code in the embedders that
     look like this:

      Dart_ExitIsolate();
      Dart_IsolateMakeRunnable(child);
      Dart_EnterIsolate(child);

The implementation of `Dart_IsolateMakeRunnable()` had an untested and
unused code path that caused running the isolate (on a thread pool).
This is not documented and can lead to bugs because often the
embedders enter the isolate right after making it runnable (e.g. above
code sequence) - which would be racing with the spawned message handler
thread.

  => Furtunately our embedders are well behaved and make the isolate
     runnable during the callback.
  => As a safeguard we'll make `Dart_IsolateMakeRunnable()` return an
     error if it's called outside the callback (which we can detect by
     the presense of a spawn state)

Issue https://github.com/dart-lang/sdk/issues/44088
Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Changes internal details of implementation which is already covered by tests.

Change-Id: Ieef316cc0807d99f2fcb1fbd56bbc9c41e904ba8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170882
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-11-12 08:49:38 +00:00
Martin Kustermann ea9a4b1a78 [vm/concurrency] Remove code duplication in isolate shutdown code
We have currently two ways to shut down isolates from the outside:

  * An embedder shuts down an embedder-owned isolate via
    `Dart_ShutdownIsolate()`.

  * An `Isolate.spawn()`ed isolate finishes the VM-owned message loop
    and VM shuts it down.

The second mechanism uses almost the same as the first, but slightly
different. This CL makes us change the second one to call our normal
`Dart_ShutdownIsolate()`.

Approved failures that got revealed by moving canonicalization check
to all isolate shutdowns, see
Issue https://github.com/dart-lang/sdk/issues/44141

Issue https://github.com/dart-lang/sdk/issues/36097
Issue https://github.com/dart-lang/sdk/issues/44088

Change-Id: I58af0e9ae73842545e21edad23c106ea59b176b3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170984
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-11-11 09:14:08 +00:00
Daco Harkes 5278383736 [vm] Native API: Make Dart_NewWeakPersistentHandle not auto delete
Changes Dart_NewWeakPersistentHandle to no longer auto delete the
weak persistent handle.

Changes the signatures of WeakPersistentHandleFinalizers to no longer
have access to the handle.

Flutter PR: https://github.com/flutter/engine/pull/19843
g3 presubmit: cl/318028238

Issue: https://github.com/dart-lang/sdk/issues/42312

TEST=runtime/vm/dart_api_impl_test.cc

Change-Id: I3f77db9954d9486759f903b78c03a494f73c68ba
Cq-Include-Trybots:dart/try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151525
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-11-03 10:27:44 +00:00
Regis Crelier 0ace625d6d [VM/finalizer] Remove checks from finalizer now performed in CFE.
Also stop passing the class 'being parsed' to finalization methods.

Change-Id: I7c2256c2a8ace468021ba0ba5be10b6bfb839480
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168621
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2020-10-21 19:36:56 +00:00
Tess Strickland fe2f9aca21 [vm] Move dynamic checks into DartEntry::InvokeCallable.
DartEntry::InvokeCallable (and DartEntry::InvokeClosure) are only called
for dynamic invocations. Instead of making it the responsibility of the
caller to check the callable function if it can't receive dynamic
invocation, move the call to DoArgumentTypesMatch into
DartEntry::InvokeCallable.

Since all current callers of these and related functions already have a
handle on the current thread, thread it through instead of retrieving it
via Thread::Current().

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I5f9260b0edace5b4017aa235a980a802405a2f1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168020
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2020-10-19 13:14:37 +00:00
Alexander Markov d77fff7307 [vm/nnbd] Add separate Snapshot::Kind for core snapshots
Core snapshots should be agnostic to the sound null safety mode
(so they can be used both in weak and strong modes), and snapshot
writer verifies that.

Snapshot::kFull was previously used both for core snapshots and
app snapshots on ia32. However, app snapshots are not guaranteed to
be agnostic, which appeared as failures on a few test on ia32.
Also, VM should be able to detect null safety mode from app snapshots,
even if they do not contain code, but null safety mode was not
written into features string of kFull snapshots.

In order to disambiguate core snapshots, a new Snapshot::Kind is
added. Snapshot::kFullCore works exactly as Snapshot::kFull, except
for verification of agnostic null safety and snapshot features string
omitting null safety mode. All snapshots except kFullCore now have
null safety mode included into their features string.

Fixes https://github.com/dart-lang/sdk/issues/43626
Issue https://github.com/dart-lang/sdk/issues/43613

Change-Id: I8cd3b049ef4e428dd5e1ce666d4c7aa3b596d70c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166308
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-10-08 19:59:15 +00:00
Tess Strickland 1b50200abd [vm] Simplify symbol naming/address retrieval.
Before, picking a symbol name for a Code object when generating
assembly required adding a placeholder entry in the Dwarf object,
because the Dwarf object separately used a SnapshotTextObjectNamer
to determine the appropriate symbol name to use in relocations.

Instead, now the ImageWriter subclasses are entirely responsible for
creating the symbol name for a given Code object, and that name is
passed when adding the Code object to the Dwarf object.

For Elf output, we eagerly create symbols for sections and Code objects
again in the appropriate symbol table, and so looking up the start of a
given section or Code object can be done simply by asking for the
corresponding symbol's address, removing special case methods and other
workarounds.

We also clean up the generated static and dynamic symbol tables by
only generating local symbols for Code objects and BSS sections (these
are local in assembly output also), and only adding global symbols to
the dynamic symbol table.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-precomp-win-release-x64-try
Change-Id: Ie40c0c836681e98f345483136bb6860e5588ef30
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166002
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-06 17:10:05 +00:00
Ryan Macnak 36e42f8c87 [vm, gc] Also merge remembered set when merging heaps.
Change-Id: I3c5e7edc807fd058ee228706b9e30774315ed142
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165940
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-05 17:56:12 +00:00
Ryan Macnak f54c0a3b9b [vm] Prepare for canonicalizing general constants on load.
- Use null instead of empty array for the initial constants set, as fields dropped during snapshot are populated with null.
 - Remove assumption that canonicalization will always be bottom-up. Deserialization clusters will cause canonicalization to be ordered by class instead of by depth.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ia3ab68281c783299e682f6c60a744c320c19e981
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165160
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2020-09-30 22:29:05 +00:00
Tess Strickland a4a714ebf9 [vm] Consolidate the *WriteStream hierarchy.
All *WriteStream classes are now part of the same hierarchy:

class BaseWriteStream : ValueObject;

  Base class for all *WriteStreams. Provides all the methods from
  the old WriteStream except for buffer() and SetPosition()
  (relegated to NonStreamingWriteStreams). Has one pure virtual
  method Realloc that must be overridden by concrete subclasses.

class NonStreamingWriteStream : BaseWriteStream;

  Base class for all *WriteStreams where the entire stream is available
  at all times (i.e., no flushing to an external sink). Extends the
  public BaseWriteStream API with buffer() (for accessing the stream
  contents) and SetPosition() (for changing the current stream pointer
  to the given absolute position in the stream).

class MallocWriteStream : NonStreamingWriteStream;

  Uses realloc to reallocate the internal buffer.  Almost the same as
  the old WriteStream, except that it only takes an initial size.  Adds
  one public method Steal() for taking ownership of the current buffer
  contents (after which the buffer is reset to an empty state). Instead
  of passing a pointer to a buffer, the internal buffer must be accessed
  via either Steal() or buffer(), which allows access to the current
  stream contents without changing ownership or resetting the stream.
  The internal buffer is freed on stream destruction.

class ZoneWriteStream: NonStreamingWriteStream;

  Takes a zone and reallocates the internal buffer in that zone. No
  additional public methods beyond those available from
  NonStreamingWriteStream.

class StreamingWriteStream : BaseWriteStream;

  Uses realloc to reallocate the internal buffer.  Generally same as
  before, where the contents of the stream are periodically flushed
  using Dart_StreamingWriteCallback. Since it extends BaseWriteStream,
  there are now more methods available for writing data to the stream
  than just Print/VPrint/WriteBytes. Since portions of the stream may be
  flushed and thus no longer in the internal buffer, does not provide
  access to the contents of the stream or a way to reposition the
  current stream pointer. Flushes any unflushed data and frees the
  internal buffer on stream destruction.

Also refactor things so that write streams are passed to appropriate
recipients, instead of the recipients taking arguments they only used to
create a WriteStream internally. Thus, recipients now can just specify
the appropriate base class for the public API used:

* BaseWriteStream for just writing to the stream, or
* NonStreamingWriteStream if re-positioning or the stream contents are
  needed.

Change-Id: I419096ecd9331483d168b079fca55b69ef397f15
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164080
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-09-23 21:05:59 +00:00
Ryan Macnak a4637cd81d [vm] Build canonical string and type tables after reading snapshots.
Prevents snapshots from including canonical strings and types with no real uses and only referenced from the canonical tables.

Prep for constants not all being in the root snapshot during snapshot splitting, which is why we fill the table during load instead of rebuilding tables during snapshot writing.

Removes the size of the tables from the snapshot.
flutter_gallery_app_so_size (Pixel 2)        -1.618% 7967664 8098736
flutter_gallery_app_so_gzip_size (Pixel 2)   -2.724% 2980344 3063788
flutter_gallery_app_so_brotli_size (Pixel 2) -3.530% 2561824 2655566

Bug: https://github.com/dart-lang/sdk/issues/41974
Bug: https://github.com/dart-lang/sdk/issues/43319
Change-Id: I4f3b62da1cea9ee17caa965e809f16856518c9a6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164102
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-09-23 19:13:29 +00:00