Commit Graph

629 Commits

Author SHA1 Message Date
Ben Konyi 4faadf1bdd [ VM / Service ] Fixed Observatory URI not being printed when service is
started via SIGQUIT

Fixes https://github.com/dart-lang/sdk/issues/45283

TEST=service/sigquit_starts_service_test.dart

Change-Id: I4e7b818d60b18e0a3bc2ce4cea670accf4bbd115
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192604
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2021-03-24 23:59:23 +00:00
asiva ac520f984d [ VM / CLI ] Add --verbosity to VM options
The verbosity option allows for specifying the CFE output verbosity

Fixes https://github.com/dart-lang/sdk/issues/44727

TEST=existing tests compile_test.dart and run_test.dart have new tests

Change-Id: I3d4e50811f84650aacf774ddb370a6eb765b9b24
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181100
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-01-26 03:01:58 +00:00
Johnni Winther 55b34a467c Reland "[vm] Pass snapshot flag to kernel_service"
Enable reporting of null safety compilation mode when running
`dart compile aot-snapshot`, `dart compile jit-snapshot`,
and `dart compile kernel`.

Closes #44234

TEST=pkg/dartdev/test/commands/compile_test.dart

Change-Id: Id9e72751068d5f208055ab953d2c8392e84c4798
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180187
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-21 21:18:32 +00:00
Aske Simon Christensen 903e92a196 Revert "[vm] Pass snapshot flag to kernel_service"
This reverts commit f8b0d26cc3.

Reason for revert: Multiple test failures in app-jit and simarm(64)

Original change's description:
> [vm] Pass snapshot flag to kernel_service
>
> Enable reporting of null safety compilation mode when running
> `dart compile aot-snapshot`, `dart compile jit-snapshot`,
> and `dart compile kernel`.
>
> Closes #44234
>
> TEST=pkg/dartdev/test/commands/compile_test.dart
>
> Change-Id: I0d4b35c6ccb4167c0c7539a4eb24a5139e29cf53
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178990
> Commit-Queue: Johnni Winther <johnniwinther@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>

TBR=bkonyi@google.com,johnniwinther@google.com

Change-Id: I83aeaa8620c640d02d5ccfd1fe8112d8209a9ad4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179773
Auto-Submit: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
2021-01-18 21:00:03 +00:00
Johnni Winther f8b0d26cc3 [vm] Pass snapshot flag to kernel_service
Enable reporting of null safety compilation mode when running
`dart compile aot-snapshot`, `dart compile jit-snapshot`,
and `dart compile kernel`.

Closes #44234

TEST=pkg/dartdev/test/commands/compile_test.dart

Change-Id: I0d4b35c6ccb4167c0c7539a4eb24a5139e29cf53
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178990
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-01-18 09:27:59 +00:00
Ben Konyi cbc46012b5 [ VM ] Address comments from https://dart-review.googlesource.com/c/sdk/+/178723
TEST=N/A

Change-Id: Ied133a59eb5562971ecca7f6106650e25676a539
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178860
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-01-12 18:53:05 +00:00
Ben Konyi fd259d4377 [ VM ] Fix invalid free introduced in c42c76f590
It's possible for loading the CLI isolate from snapshot to be skipped
without failing for SIMARM, so *error would not point to a valid error,
causing free(*error) to crash.

TEST=Existing tests

Change-Id: Ic178f7bd0f89c4440b2daf714fbb2e428867bd72
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178723
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-01-11 23:25:28 +00:00
Ben Konyi c42c76f590 [ VM / CLI ] Run DartDev isolate from snapshot when possible
This change tries to run from dartdev.dart.snapshot and falls back to
running from dartdev.dill if incompatible VM flags are provided.

Fixes https://github.com/dart-lang/sdk/issues/43969

Performance results:

dart test.dart (no CLI isolate): 0.167s
dart run test (from snapshot):   0.208s
dart run test (from kernel):     0.326s

TEST=pkg/dartdev/test/load_from_dill_test.dart

Change-Id: I3195886b86676580ef2a0221f0284328964ef061
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178300
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-01-11 19:41:50 +00:00
Sigurd Meldgaard f3fe9dc3b3 Reland "Improve handling of disable-dartdev-analytics"
This is a reland of 58860f4814

Original change's description:
> Improve handling of disable-dartdev-analytics
>
> This is second try of https://dart-review.googlesource.com/c/sdk/+/171284
> that was reverted to to faulty logic in main_options.
>
> Some other refactorings are piggy-backed along.
>
> TestProject.runSync no longer takes a 'command' argument. It was anyway
> often not an argument.
>
> Also stop the messy handling of pub arguments. It is no longer needed.
>
> BUG: https://github.com/dart-lang/sdk/issues/44135
> Change-Id: I49abf5810d9ea262409ba9d93f0471037cb8a753
> TEST=The VM change is tested via all the pkg/dartdev/test/command/* tests that invoke dart with the --no-analytics flag.
> TEST=Furthermore manual test that the --no-analytics flag is passed to dartdev.
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174261
> Reviewed-by: Jonas Jensen <jonasfj@google.com>
> Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>

Change-Id: I725662f578d061f87171ceffe9aff3de83688f58
TEST=Furthermore run the vm-kernel-precomp-obfuscate-linux-release-x64-try trybot
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174473
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Jonas Jensen <jonasfj@google.com>
2020-12-04 14:22:10 +00:00
Sigurd Meldgaard 97baa6efd1 Revert "Improve handling of disable-dartdev-analytics"
This reverts commit 58860f4814.

Reason for revert: Broke bots. Will investigate.

Original change's description:
> Improve handling of disable-dartdev-analytics
>
> This is second try of https://dart-review.googlesource.com/c/sdk/+/171284
> that was reverted to to faulty logic in main_options.
>
> Some other refactorings are piggy-backed along.
>
> TestProject.runSync no longer takes a 'command' argument. It was anyway
> often not an argument.
>
> Also stop the messy handling of pub arguments. It is no longer needed.
>
> BUG: https://github.com/dart-lang/sdk/issues/44135
> Change-Id: I49abf5810d9ea262409ba9d93f0471037cb8a753
> TEST=The VM change is tested via all the pkg/dartdev/test/command/* tests that invoke dart with the --no-analytics flag.
> TEST=Furthermore manual test that the --no-analytics flag is passed to dartdev.
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174261
> Reviewed-by: Jonas Jensen <jonasfj@google.com>
> Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>

TBR=bkonyi@google.com,sigurdm@google.com,jonasfj@google.com

Change-Id: I754bcebdcfc595158b04d431662b65bf25f5b89d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174466
Reviewed-by: Sigurd Meldgaard <sigurdm@google.com>
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
2020-12-01 13:11:16 +00:00
Sigurd Meldgaard 58860f4814 Improve handling of disable-dartdev-analytics
This is second try of https://dart-review.googlesource.com/c/sdk/+/171284
that was reverted to to faulty logic in main_options.

Some other refactorings are piggy-backed along.

TestProject.runSync no longer takes a 'command' argument. It was anyway
often not an argument.

Also stop the messy handling of pub arguments. It is no longer needed.

BUG: https://github.com/dart-lang/sdk/issues/44135
Change-Id: I49abf5810d9ea262409ba9d93f0471037cb8a753
TEST=The VM change is tested via all the pkg/dartdev/test/command/* tests that invoke dart with the --no-analytics flag.
TEST=Furthermore manual test that the --no-analytics flag is passed to dartdev.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174261
Reviewed-by: Jonas Jensen <jonasfj@google.com>
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
2020-12-01 10:25:55 +00:00
Sigurd Meldgaard ce272daae8 [dart cli]: Use the correct package_config from dart run and dart test
Before `dart run` would always use the package config found by looking up the hierarchy from
whatever we got from getExecutableForCommand.

When that happened to be a precompiled snapshot this would work, because it would be ignored.

But if it would be a dart-file in another package (that would happen if that package is not cached)
we instead should resolve it relatively to the current working directory
(where the pubspec.yaml is also found).

Bug: https://github.com/dart-lang/sdk/issues/44214
Change-Id: Ib2e929c01ddc72babbcff91560236e08429f09f7
TEST=The new package config behavior is tested by the new test in: pkg/dartdev/test/commands/run_test.dart.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172181
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Jonas Jensen <jonasfj@google.com>
2020-11-17 10:20:00 +00:00
Martin Kustermann 42ac762949 [vm/concurrency] Fix inconsistency in Dart's spawn implementation
When Dart invokes Isolate.spawn the Dart VM calls out to the embedder to
create an isolate (group) or initialize it (in case of lightweight
isolates).

  * If the embedder doesn't implement this functionality or fails to
    perform the creation/initialization, it will signal this failure
    via the return value it gives to the VM in the call back
    implementation (i.e. Dart_InitializeParams.create_group and
    Dart_InitializeParams.initialize_isolate}).

  * If the embeder sucessfully completed it's work, the VM owns the
    isolate after the embedder call returns. The VM is then responsible
    for running it.

It is an undocumented invariant atm that the embedder is responsible
to make the isolate runnable if it signals the VM that isolate was
sucessfully created/initialized.
  => If it does not do that, we have effectively an isolate leak.
  => Though right now seemingly all of our embedders to that correctly.

To avoid this unintentional isolate leak we make the isolate runnable
ourselves if embedder callback was successful but did not make it
runnable itself.

  => This also avoids unnecessary ceremony code in the embedders that
     look like this:

      Dart_ExitIsolate();
      Dart_IsolateMakeRunnable(child);
      Dart_EnterIsolate(child);

The implementation of `Dart_IsolateMakeRunnable()` had an untested and
unused code path that caused running the isolate (on a thread pool).
This is not documented and can lead to bugs because often the
embedders enter the isolate right after making it runnable (e.g. above
code sequence) - which would be racing with the spawned message handler
thread.

  => Furtunately our embedders are well behaved and make the isolate
     runnable during the callback.
  => As a safeguard we'll make `Dart_IsolateMakeRunnable()` return an
     error if it's called outside the callback (which we can detect by
     the presense of a spawn state)

Issue https://github.com/dart-lang/sdk/issues/44088
Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Changes internal details of implementation which is already covered by tests.

Change-Id: Ieef316cc0807d99f2fcb1fbd56bbc9c41e904ba8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/170882
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-11-12 08:49:38 +00:00
Ben Konyi c3da025eaa Revert "[VM] Temporarily disable DDS for 2.10 branch"
This reverts commit e498ca00dc.

Reason for revert: 2.10 has been branched.

Original change's description:
> [VM] Temporarily disable DDS for 2.10 branch
> 
> Change-Id: I32916b8e816e109d4e659e9b05326cd0fffb85d7
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161764
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Devon Carew <devoncarew@google.com>

TBR=devoncarew@google.com,bkonyi@google.com,asiva@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: If895b635fc2f66c08d8b1aa28af92c250de3e95a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162942
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-09-16 18:38:28 +00:00
Ben Konyi 19875e9ab4 [ VM / Service ] Fix issue where dart:* sources were not available via
the service

As part of the dartdev refactoring, 'dfe.Init()' was only called when
an explicit script URI was found during option parsing. However, with
--observe provided, 'script_uri' is always null as dartdev is
responsible for determining which script to run. This change ensures
that vm_platform_strong.dill is always loaded by the DFE.

Fixes https://github.com/dart-lang/sdk/issues/43207

Change-Id: I118b83390efba48e3e8d5b793444d189ba3c5ae2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162060
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-09-10 17:53:19 +00:00
Ben Konyi e498ca00dc [VM] Temporarily disable DDS for 2.10 branch
Change-Id: I32916b8e816e109d4e659e9b05326cd0fffb85d7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161764
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
2020-09-04 02:25:35 +00:00
Ben Konyi f0d88cc96d Reland "[ Service / DDS ] Advertise DDS as the VM service, bump version to 4.0"
This reverts commit ed120c3c80.

Change-Id: I445f4bb2dafaad3ce2daa3ae42efe1723f9b1abe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160660
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-09-03 15:20:14 +00:00
Ben Konyi ed120c3c80 Revert "[ Service / DDS ] Advertise DDS as the VM service, bump version to 4.0"
This reverts commit 0ea18ffb36.

Reason for revert: Tree is on fire

Original change's description:
> [ Service / DDS ] Advertise DDS as the VM service, bump version to 4.0
> 
> This change does the following:
>   - The DDS URI will be reported by the VM as the VM service URI. If DDS
>     disconnects, the VM service URI will be reported instead. This only
>     impacts the standalone VM.
>   - Updated the service protocol to 4.0 and removed the following
>     deprecated functionality:
>       - Client synchronization RPCs (these now live in DDS)
>       - getWebSocketTarget (no longer needed as the VM service no longer
>         tries to redirect web socket clients to DDS)
>   - Regenerates package:vm_service based on the new spec, prepare for
>     5.0 release
> 
> Change-Id: I8a2b401062342eb99b81ef10ef6926baa88f946e
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160141
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com

Change-Id: I6a16e9fa7ceb18555d6424a169cfa811d3833419
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160340
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-08-26 16:08:27 +00:00
Ben Konyi 0ea18ffb36 [ Service / DDS ] Advertise DDS as the VM service, bump version to 4.0
This change does the following:
  - The DDS URI will be reported by the VM as the VM service URI. If DDS
    disconnects, the VM service URI will be reported instead. This only
    impacts the standalone VM.
  - Updated the service protocol to 4.0 and removed the following
    deprecated functionality:
      - Client synchronization RPCs (these now live in DDS)
      - getWebSocketTarget (no longer needed as the VM service no longer
        tries to redirect web socket clients to DDS)
  - Regenerates package:vm_service based on the new spec, prepare for
    5.0 release

Change-Id: I8a2b401062342eb99b81ef10ef6926baa88f946e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160141
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-08-26 14:55:15 +00:00
Ben Konyi d11e89f516 [ Service ] Wait for DDS connection before outputting service connection
information

When dartdev and the VM service are enabled, the service should avoid
advertising its connection information until after DDS has connected in
order to prevent race conditions between DDS and external clients.

Fixes https://github.com/dart-lang/sdk/issues/42727

Change-Id: If6393f085e9147af628997a1f8936e84f9d8310c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157462
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-08-20 19:00:59 +00:00
Ben Konyi 9bac327e32 [ VM / dartdev ] Fix issue where hard-coded false value for 'null_safety' flag in dartdev's Dart_IsolateFlags
was overwritten by isolate initialization code

Fixes situation where the following command would fail due to the
dartdev kernel not being compiled with null-safety:

dart --strong-null-safety --enable-experiment=non-nullable run foo.dart

Change-Id: I7e7c980eb3274a51e64c031e6bfef8650b897b8b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158345
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-08-13 22:04:24 +00:00
Liam Appelbe 79cec0e864 [vm] Delete ABI infrastructure
Change-Id: I26d455d96c87010293a6dcb9f6c0eecfcaf3c816
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/157102
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2020-08-06 19:55:12 +00:00
Ryan Macnak 9cb5d8e5be [standalone] Fix Isolate.spawnUri to work with AOT snapshots.
Fixes missing initialization of dart:_builtin during AOT.

Change-Id: If9f24f3658b91f490fc8215f2e9343bd437b2744
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/156050
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-08-04 19:00:18 +00:00
Zichang Guo d6d37ed203 Prevent VM crashing from \\?\ prefixed path
"\\?\" prefixed path is not properly supported. Arguments historically
were parsed as a Uri, which converted backward slashes into slashes. It
used to work because Win API is able to handle slashes as separators.
But "\\?\" has a special meaning and should not be changed. '?' is also
invalid char in the URI. So it leads to a crash.

The best solution is to avoid using URI to parse the path. This cl is a
workaround for crashing.

Bug: https://github.com/dart-lang/sdk/issues/42779
Change-Id: I13bcefc820eb578a2707181080eac9c0f1592e4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/156364
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Zichang Guo <zichangguo@google.com>
2020-07-31 21:09:40 +00:00
Ben Konyi 49d7eba24e Reland "[ VM ] Print a meaningful error message when a user attempts to run an AOT snapshot with a JIT VM."
This reverts commit a24192e786.

Change-Id: Iea1c6cb6b509ce1493bc502d90592d2ba6fca68f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155480
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-07-27 21:51:03 +00:00
Ben Konyi a24192e786 Revert "[ VM ] Print a meaningful error message when a user attempts to run an AOT snapshot with a JIT VM."
This reverts commit d870a71b20.

Reason for revert: ASAN failures

Original change's description:
> [ VM ] Print a meaningful error message when a user attempts to run an AOT snapshot with a JIT VM.
> 
> Fixes https://github.com/dart-lang/sdk/issues/42482
> 
> Change-Id: I284a5673e30f59b127e69c712df93552cdfcde41
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154834
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com

Change-Id: I3639ede6987fbb51fb95584685c472abb789970e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/155303
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-07-21 23:49:47 +00:00
Ben Konyi d870a71b20 [ VM ] Print a meaningful error message when a user attempts to run an AOT snapshot with a JIT VM.
Fixes https://github.com/dart-lang/sdk/issues/42482

Change-Id: I284a5673e30f59b127e69c712df93552cdfcde41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154834
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-07-21 19:29:15 +00:00
Ben Konyi 157ebfe50f Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
This reverts commit 7e373831ce.

Change-Id: Ib94d793a5753ec6352cd9a65e35f726a63336368
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154830
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-07-21 16:23:18 +00:00
Ryan Macnak c5a94db091 [vm] Initial implementation of deferred loading.
(Assignment of libraries to loading units is already done in the kernel generation step.)

After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.

All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.

Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.

Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.

Embedder's guide:

At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.

At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-07-17 19:29:47 +00:00
Alexander Aprelev 7e373831ce Revert "Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process""
This reverts commit bc4cb123a8.

Reason for revert: breaks crossword simarm tests https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-linux-debug-simarm_x64/2619

Original change's description:
> Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
> 
> This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
> 
> By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
> 
> VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
> Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
> Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/42092)
> 
> This reverts commit 3849b5061c.
> 
> Change-Id: I4fd3ba33840771a9f284d733c4a25fac6cde64ca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154706
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=bkonyi@google.com,asiva@google.com

Change-Id: I649d94c668417f2edbfd7039fa5c876e10dc32fe
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154800
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-07-17 04:37:48 +00:00
Ben Konyi bc4cb123a8 Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.

By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:

VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/42092)

This reverts commit 3849b5061c.

Change-Id: I4fd3ba33840771a9f284d733c4a25fac6cde64ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154706
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-07-17 00:54:11 +00:00
Ben Konyi 3849b5061c Revert "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
This reverts commit ffe258d2d4.

Reason for revert: Failures on bots

Original change's description:
> [ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process
> 
> This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
> 
> By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
> 
> VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
> Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
> Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/41978)
> 
> Change-Id: I1c6b1425831b691ad20284716aa80f817dbaf607
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152588
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Idb1d24a4524bdc3ccfb199a82710f3c0d9db539a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154702
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-07-16 19:50:57 +00:00
Ben Konyi ffe258d2d4 [ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process
This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.

By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:

VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/41978)

Change-Id: I1c6b1425831b691ad20284716aa80f817dbaf607
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152588
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-07-16 19:03:24 +00:00
Alexander Markov 6b4fe2fb78 [vm/nnbd] Detect null safety when running service isolate from AOT snapshot
AOT snapshots can be created with or without null safety, and they
include code for vm-service isolate.
When initializing vm-service isolate from AOT snapshot we should also
set isolate's null safety mode from the snapshot.

Change-Id: I2c91e1bf7fb4491144620d49e7e52be20af8b69b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152903
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2020-07-01 20:22:24 +00:00
Alexander Aprelev d33e51df3c [build] Support --clang when building dart sdk on Windows.
It relies on flutter copy of clang distribution, same one that is used to build flutter/engine.

It addressed several deprecated warnings from clang compiler for functions like strdup, unlink, etc.

It allows few warnings still since they are triggered in third_party sources.

Change-Id: Ieb13792c011438d46dbbc0fa030e1b5e4ea14315
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142704
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-06-25 03:36:41 +00:00
Martin Kustermann 39577ea564 [vm] Remove (unused & deprecated) --package-root parameter from VM binaries and from tests (Platform.packageRoot always returns null)
Closes https://github.com/dart-lang/sdk/issues/41245

Change-Id: Idd7526199b4c2479297c58b05bc13efcb79d01e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152321
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-06-24 18:17:53 +00:00
Siva Annamalai d46a696957 Revert "Reland : [VM/Runtime] Cleanup package config initialization code"
This reverts commit 03777dba31.

Reason for revert: Failures in couple of bots

Original change's description:
> Reland : [VM/Runtime] Cleanup package config initialization code
> 
> - Use _Init function for initialization of package config
> - Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
> 
>   DartUtils::SetupPackageConfig
> - Rename Loader::InitForSnapshot to Loader::Init
> - Remove _setPackagesConfig, _setWorkingDirectory from builtin library
> - Rename _setPackagesMap to _setPackagesConfig
> Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,asiva@google.com

Change-Id: Ibea8c35fe310ac7708f00bb9ffd58715b6924cb5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151869
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 08:02:53 +00:00
asiva 03777dba31 Reland : [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,

  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 05:50:49 +00:00
asiva 4b9aa2bd7e Revert "[VM/Runtime] Cleanup package config initialization code"
This reverts commit 29e93bcdbd.

Reason for revert: Windows bots are failing with package resolution
errors. Looks like we are missing a windows path sanitization step.

Change-Id: Ib56f7e926b4f385fa3fde74c9c71947f64b8be1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151469
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-06-17 00:03:19 +00:00
asiva 29e93bcdbd [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I0b83574593fa95e1b948134fcd4a7b4858ea19c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151045
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-06-16 23:24:26 +00:00
asiva a782dd82db [VM/runtime] - detect null safety before isolate initialization
- Detect null safety when not specified before isolate
      initialization
      - for source files by having CFE parse the source file
        for @dart annotations
      - for kernel files by sniffing the kernel file for
        compilation mode
      - for appJIT files by sniffing the feature string
      - for AOT snapshots by sniffing the feature string

    - Remove workaround of returning null safety to false during
      bootstrapping

    - Add a new Dart C API call for detecting null safety

Bug: 41766
Change-Id: Ia8cf264323a2d0d58c2855ce6491456aa6f1da07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150089
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-06-11 01:22:35 +00:00
Ben Konyi c4b7097e97 [ VM ] Migrate VM tests to support null safety and return errors from Dart_GetType and Dart_NewListOf when --null-safety is enabled
These methods return legacy types, which aren't valid with null safety
enabled.

Fixes https://github.com/dart-lang/sdk/issues/41154

Change-Id: I1181f0f3b9a8df156dea4dc4c82fef8afdf97ab9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148685
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-05-26 22:40:52 +00:00
Ben Konyi 444490d640 Reland "[ VM / DartDev ] Run all debugging sessions via 'dart run'"
This is a reland of 604a366065

Original change's description:
> [ VM / DartDev ] Run all debugging sessions via 'dart run'
> 
> Prior to this change, running `dart --observe foo.dart` would result in
> the DartDev flow being bypassed. While `dart foo.dart` will continue to
> bypass DartDev and be run directly, running either `dart --observe
> <vm-flags> foo.dart` or `dart --enable-vm-service <vm-flags> foo.dart`
> will result in the command being implicitly converted to `dart run
> --observe <vm-flags> foo.dart`. This is required for all standalone VM
> instances to run against DDS instead of the VM service directly.
> 
> As usual, the DartDev flow can always be bypassed by providing the
> --disable-dart-dev flag.
> 
> Change-Id: I211cd1ec4b1ec0e75ae0a568a66f1a0fc7b3852f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147342
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

Change-Id: I917d3056322d74766bdf376a3e28871ad5b66cfa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147980
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-05-13 19:58:56 +00:00
David Morgan 964f966c0c Revert "[ VM / DartDev ] Run all debugging sessions via 'dart run'"
This reverts commit 604a366065.

Reason for revert: Breaks google3, see b/156414355

Original change's description:
> [ VM / DartDev ] Run all debugging sessions via 'dart run'
> 
> Prior to this change, running `dart --observe foo.dart` would result in
> the DartDev flow being bypassed. While `dart foo.dart` will continue to
> bypass DartDev and be run directly, running either `dart --observe
> <vm-flags> foo.dart` or `dart --enable-vm-service <vm-flags> foo.dart`
> will result in the command being implicitly converted to `dart run
> --observe <vm-flags> foo.dart`. This is required for all standalone VM
> instances to run against DDS instead of the VM service directly.
> 
> As usual, the DartDev flow can always be bypassed by providing the
> --disable-dart-dev flag.
> 
> Change-Id: I211cd1ec4b1ec0e75ae0a568a66f1a0fc7b3852f
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147342
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com

Change-Id: I688a3779e7ccaf85a28dc4d6f5ccf23596109519
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147900
Reviewed-by: David Morgan <davidmorgan@google.com>
Commit-Queue: David Morgan <davidmorgan@google.com>
2020-05-13 07:13:35 +00:00
Ben Konyi 604a366065 [ VM / DartDev ] Run all debugging sessions via 'dart run'
Prior to this change, running `dart --observe foo.dart` would result in
the DartDev flow being bypassed. While `dart foo.dart` will continue to
bypass DartDev and be run directly, running either `dart --observe
<vm-flags> foo.dart` or `dart --enable-vm-service <vm-flags> foo.dart`
will result in the command being implicitly converted to `dart run
--observe <vm-flags> foo.dart`. This is required for all standalone VM
instances to run against DDS instead of the VM service directly.

As usual, the DartDev flow can always be bypassed by providing the
--disable-dart-dev flag.

Change-Id: I211cd1ec4b1ec0e75ae0a568a66f1a0fc7b3852f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/147342
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-05-12 19:53:11 +00:00
Martin Kustermann a94b8ae45d Reland "[vm] Make VM support .dart_tool/package_config.json"
This relands the following CLs

    [vm] Make VM support .dart_tool/package_config.json
    https://dart-review.googlesource.com/c/sdk/+/145462

    [vm] Make Isolate.resolveUri() work for both `.packages` and `dart_tool/package_config.json`
    https://dart-review.googlesource.com/c/sdk/+/145803

    [vm] Skip (newly enabled) isolate tests on simarm* and hot-reload configs
    https://dart-review.googlesource.com/c/sdk/+/145861

    [vm] Remove ../0 postfix from status file entries (test framework does not recognize those multitests)
    https://dart-review.googlesource.com/c/sdk/+/145940

    [vm] - Fix hang when no package config or .packages exist
    https://dart-review.googlesource.com/c/sdk/+/145661

The only change (extra patchset) is to let VM prefer the old/legacy
".packages" format and only use the newer
".dart_tool/package_config.json" if we failed to find a ".packages".

Closes https://github.com/dart-lang/sdk/issues/41739
Closes https://github.com/dart-lang/sdk/issues/41649

Change-Id: I220f808540e6e521d985ae763968f64dbf57fe67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145942
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-05-04 14:51:14 +00:00
asiva 170d047b20 The rolls of Dart into Flutter have been failing for more than a day now.
We have accumulated almost 69 commits. Please see issue
https://github.com/dart-lang/sdk/issues/41739
for details of why these CLs had to be reverted.

Revert "Reland "[vm] Make VM support .dart_tool/package_config.json""
This reverts commit fa6cc57a32.

Revert "[VM] - Fix hang when no package config or .packages exist"
This reverts commit 31be795457.

Revert "[vm/isolate] Make Isolate.resolveUri() work for both `.packages` and `dart_tool/package_config.json`"
This reverts commit a99f7aef6e.

Revert "[vm] Skip (newly enabled) isolate tests on simarm* and hot-reload configs"
This reverts commit 7082e99c28.

Revert "[vm] Remove ../0 postfix from status file entries (test framework does not recognize those multitests)"
This reverts commit 708468684a.

Change-Id: Id6b951bde4f379fdcfac3cc23e0057a77a7e6db0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146041
Reviewed-by: Ben Konyi <bkonyi@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-05-01 22:11:55 +00:00
Martin Kustermann fa6cc57a32 Reland "[vm] Make VM support .dart_tool/package_config.json"
This CL makes the VM add support for .dart_tool/package_config.json via
  * the --packages=<file> command line parameter
  * the `packageConfig: <file>` parameter to `Isolate.spawnUri`

It also allows now `package:*` uris being used in `Isolate.spawnUri`

The CL also removes the unused `packageRoot` parameter from the Dart to
C++ interface as well as in various places in C++.

The CL also aligns the implementations between sdk and sdk_nnbd more.

Issue https://github.com/dart-lang/sdk/issues/41649
Issue https://github.com/dart-lang/sdk/issues/41245
Issue https://github.com/dart-lang/sdk/issues/41246

Change-Id: I0a7aa3040332abafa19bf80bdbd8a8f8594cc6fe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145462
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2020-04-29 19:05:26 +00:00
Alexander Aprelev 334284a69b Revert "Reland "[vm] Make VM support .dart_tool/package_config.json""
This reverts commit 30085612f4 as it
broke vm-kernel-precomp-nnbd-linux-release-x64 bot.

Seems to be the same failure as the last time it was reverted: https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-nnbd-linux-release-x64/537

Change-Id: I35a1897367bcb65f518b58659001280c5d5ed930
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145262
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-04-28 23:23:44 +00:00
Martin Kustermann 30085612f4 Reland "[vm] Make VM support .dart_tool/package_config.json"
This CL makes the VM add support for .dart_tool/package_config.json via
  * the --packages=<file> command line parameter
  * the `packageConfig: <file>` parameter to `Isolate.spawnUri`

It also allows now `package:*` uris being used in `Isolate.spawnUri`

The CL also removes the unused `packageRoot` parameter from the Dart to
C++ interface as well as in various places in C++.

The CL also aligns the implementations between sdk and sdk_nnbd more.

Issue https://github.com/dart-lang/sdk/issues/41649
Issue https://github.com/dart-lang/sdk/issues/41245
Issue https://github.com/dart-lang/sdk/issues/41246

Change-Id: Idec0990c731f34efda9c39672067eae9174319f1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/144826
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2020-04-28 16:52:02 +00:00