Commit Graph

207 Commits

Author SHA1 Message Date
Derek Xu 2877d78576 Reland "[VM/Service] Record timeline events representing completed microtasks"
This is a reland of commit 22021cc120

I showed some excerpts of Golem results to @mraleph for varying targets
and constant CPU, patch A, and patch B. Those constants being:
  CPU: Intel Xeon
  patch A: revision=115869&patch=20143
  patch B: revision=115876
(example results:
https://golem.corp.goog/Comparison?repository=dart#targetA%3Ddart%3BmachineTypeA%3Dlinux-x64%3BrevisionA%3D115869%3BpatchA%3Dderekx-Reland---VM%2FService--Record-timeline-events-representing-completed-microtasks--4%3BtargetB%3Ddart%3BmachineTypeB%3Dlinux-x64%3BrevisionB%3D115876%3BpatchB%3DNone)

He said that this change could be relanded as long as the following
"RunTime as Score" regressions of the "dart" target were noted in the
commit message:
  - StreamSingleSubTest: -3.907% (-0.8 noise)
  - ScheduleMicrotaskTest: -12.67% (-0.6 noise)

TEST=pkg/vm_service/test/timeline_events_for_completed_microtasks_test

Original change's description:
> [VM/Service] Record timeline events representing completed microtasks
>
> TEST=pkg/vm_service/test/timeline_events_for_completed_microtasks_test
>
> CoreLibraryReviewExempt: This CL does not include any core library API
> changes, it only modifies the implementation of microtasks (by
> instrumenting them).
> Change-Id: I54d886db9519c73f9e3218a9cc1c46bc9fe9acc3
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/420221
> Commit-Queue: Derek Xu <derekx@google.com>
> Reviewed-by: Ben Konyi <bkonyi@google.com>

CoreLibraryReviewExempt: This CL does not include any core library API
changes, it only modifies the implementation of microtasks (by
instrumenting them).
Change-Id: Ia7741a9bb9ab948e8d5fff1cf9abe9915e247d81
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/423921
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
2025-04-25 08:51:22 -07:00
Ryan Macnak 2a08770912 [vm] If waiting for an active mutator slot for too long, steal a slot from a thread at a native safepoint.
The thread whose slot was stolen will take the slow path attempting to exit its safepoint and wait to reacquire an active mutator slot.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/54687
Change-Id: I49c4c12b38d38d7afa4df6b346c2a464f75ef6e0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/406404
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2025-02-10 09:38:42 -08:00
Ryan Macnak 322baef3e1 [io] Exit the isolate during Process.runSync and sleep.
This prevents such an isolate from occupying one of the limited number of mutator slots and blocking other isolates in the same group from running.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/51254
Bug: https://github.com/dart-lang/sdk/issues/54687
Bug: https://github.com/dart-lang/sdk/issues/57119
Change-Id: Ic04bbaa7f482d533ad0ecf2c6da17ea9f00c264e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/398927
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2024-12-05 00:16:38 +00:00
Alexander Aprelev d2bc055651 [vm/shared] Add 'dart:concurrent' core library.
This brings Mutex and ConditionVariable classes to guard access to shared variables.

TEST=shared_test, shared_primitives_test
BUG=https://github.com/dart-lang/sdk/issues/55991
CoreLibraryReviewExempt: dart:concurrent library is experimental, restricted to main/dev channels, vm-only at this point.
Change-Id: I64ca7a0434cbe0079976133ccbf9de2dd4c0d7b1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/372181
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
2024-07-10 19:45:29 +00:00
Ben Konyi 9c78c8f14b [ CLI ] Strip --observe and --enable-vm-service from dartdev argument list
The dart run argument parser was incorrectly parsing any argument following an
option that the VM treats as having a default (e.g., --observe and
--enable-vm-service), resulting in a "File not found ''" error and the
VM failing to start the user program. The following invocations would
fail due to 'foo.dart' being treated as the value for the option to the
left:

- dart run --observe foo.dart
- dart run --enable-vm-service foo.dart

This change strips these two options from the arguments sent to dartdev
as both are handled directly in the VM and are only included in the dart
run argument parser for the help message.

Fixes https://github.com/dart-lang/sdk/issues/55185

TEST=regression test added to run_test.dart.

Fixed: 55185
Change-Id: I1c89fec3ee46296ee4ebb847eb47a7845dfdb399
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357360
Reviewed-by: Derek Xu <derekx@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2024-03-13 17:08:39 +00:00
Vyacheslav Egorov 14f7622ef4 [vm] Remove bin/*_android{.cc,.h} and use Linux implementations.
Android is based on Linux, so most of the files were identical
sans some subtle discrepancies caused by drift over time.

This discrepancies were making code base harder to maintain and in fact
were hiding bugs. For example, on Android eventhandler's implementation
of timers which relied on passing timeout to `epoll_wait` contained
a bug which was not present on Linux which used `timerfd` instead.

TEST=ci and manual testing of Flutter app on Android device

Fixes https://github.com/dart-lang/sdk/issues/54868

Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-android-product-arm-try,vm-ffi-android-product-arm64c-try,vm-ffi-android-release-arm-try,vm-ffi-android-release-arm64c-try
Bug: b/311165013
Change-Id: Ia166f69c14177ec34160805a0983eafee8ea65f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350923
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2024-02-09 14:10:49 +00:00
Daco Harkes 24eb977761 [vm] Fix Mac Arm64 AOT
The magic number has the opposite endianness on the arm64 macs.

TEST=locally verified + SDK build

Closes: https://github.com/dart-lang/sdk/issues/54796
Change-Id: I4a66f44e0a32fbc3530bda6ffca177ba14d9dca7
Cq-Include-Trybots: luci.dart.try:vm-aot-mac-release-arm64-try,vm-mac-debug-arm64-try,dart-sdk-mac-arm64-try,pkg-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/349661
Reviewed-by: Martin Kustermann <kustermann@google.com>
2024-02-01 15:30:08 +00:00
asiva eabfad1158 [VM/Runtime] Report error when an AOT snapshot is used in spawnUri while
running in the JIT VM

TEST=new test added

Bug:54528
Change-Id: Ifa0176ee453e5af4e45bb0b74cf9ac9ba23fb93e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/346060
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2024-01-31 21:43:27 +00:00
Ryan Macnak fa7cadde80 [vm, io] Don't have pending Dart_TypedDataAcquire during I/O operations.
I/O can take arbitrarily long, and open interior pointers prevent all other isolates in the group to reaching safepoints.

TEST=ci
Bug: https://github.com/flutter/flutter/issues/140763
Change-Id: I1fa732d0db31500946aee232dc31f5baf324a55e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/345746
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
2024-01-11 23:54:59 +00:00
Ryan Macnak 3c298dbca5 [vm] Update to constexpr in runtime/bin.
TEST=build
Change-Id: I18fc7cfe725dc978d4b23de6191e455ac7cd75e5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/293800
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2023-04-12 20:32:29 +00:00
Ryan Macnak b68351fbc3 [vm] Update NULL to nullptr in runtime/bin.
TEST=build
Change-Id: Ie3be570c274b0275a995a0f54b5e6ccdfc77ccd3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292287
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
2023-04-12 01:11:05 +00:00
Josh Soref 50b94ef9fc Spelling runtime bin
TEST=build VM

Closes: https://github.com/dart-lang/sdk/pull/50862

GitOrigin-RevId: ae54b37666f70f670ee3af11c984c7e9a7e9da26
Change-Id: I459fda0439a1cd368f488a70d84ee6bb915e60bb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/277761
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2023-01-06 07:48:22 +00:00
Haidar cbe458703c [vm/api] Make CObject as_string const char* instead of const
Closes https://github.com/dart-lang/sdk/pull/50884

TEST=ci

GitOrigin-RevId: f898451bb21f1e49550698fcf08ec5dfc1036dce
Change-Id: I808d37afe3e953afd632e4ffc4ab4b152b468e4f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278140
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
2023-01-04 12:51:03 +00:00
Daco Harkes f524ec74ce [vm/ffi] FfiNative process lookup
This CL makes `FfiNative`s use `DynamicLibrary.process()` lookup if
resolving with the resolver set by `Dart_SetFfiNativeResolver` fails.

Also moves the implementation over from ffi.cc to
ffi_dynamic_library.cc so the implementation can be shared with
`DynamicLibrary.process()`.

Moves the implementation behind non-simulator and non-precompiler.
However, the implementation is tested in vm/cc tests which are in
precompiler mode. So enables the implementation if TESTED is defined.

This CL massages the build files so that TESTED is properly
defined when compiling the runtime for the vm/cc tests, and links
the ole32 symbols on windows for vm/cc tests.

(And some unrelated small cleanup changes here and there.)

TEST=tests/ffi/native_assets/process_test.dart

Change-Id: I25395d381db1d9b4b7a5759171a798a1140a6140
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-debug-x64c-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,dart-sdk-win-try,vm-kernel-win-release-x64-try,vm-kernel-win-release-ia32-try,vm-kernel-precomp-win-product-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264982
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
2022-10-25 10:57:07 +00:00
asiva 3cdeb58c98 [VM/Runtime] - Use 'const' qualifier for CObject typed data
Addresses https://github.com/dart-lang/sdk/issues/49827

TEST=ci

Change-Id: I525cc27d0bf01945d4f700f48355a3f17e297007
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/256602
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2022-09-13 22:38:22 +00:00
Alexander Aprelev b29bb7763b [vm/doc] Add a note to CObjectNativePointer class.
Addresses https://github.com/dart-lang/sdk/issues/48379

TEST=not needed, only documentation update

Change-Id: I2d82c6533ef64cd9ff62428e20a2e66013a35e31
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/233642
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2022-02-21 21:54:11 +00:00
Brian Quinlan 917ae52f70 Add the ability to log TLS keys.
TEST=unit tests
Bug: https://github.com/dart-lang/sdk/issues/47838
Change-Id: I8a64e8623022215cae261eadb25b22deb9f3d910
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/231330
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Brian Quinlan <bquinlan@google.com>
2022-02-17 21:39:31 +00:00
Alexander Aprelev 935ee25f3b [io/unix_domain_socket] Add support for sending/receiving open files, sockets, std streams via sockets.
This adds [sendMessage] and [receiveMessage] methods to [RawSocket] class. They are only supported on Linux at the moment as they require connection opened as unix domain socket connection.

This introduces [SocketControlMessage] class that represents a message sent via socket and also introduces [ResourceHandle] class that wraps [RandomAccessFile], [Socket]/[RawSocket]/[RawDatagramSocket], [Stdin]/[Stdout] for
marshalling/unmarshalling purposes.

Underlying OS implementation supports various kinds of control messages that can be passed via sockets, this CL only adds support for sending/receiving opened file description handles.
When receiving a message recipient can attempt to extract handles out of it via [tryExtractHandles]. It returns [null] if message has no handles to extract.

This is continuation of the work started on https://dart-review.googlesource.com/c/sdk/+/205067.

Bug: https://github.com/dart-lang/sdk/issues/46328
TEST=unix_socket_test

Change-Id: Ic9125b51dc80b677452e454366bae4118c298081
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212036
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Lasse R.H. Nielsen <lrn@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2021-10-08 15:46:23 +00:00
Vyacheslav Egorov bda70c8489 [vm] Remove support for dart-ext: imports
Per breaking change #45451 we are removing support for dart-ext:
style native extensions from the Dart VM.

This CL removes the associated VM code, tests and samples. It also ports
a single test which used dart-ext: import to use FFI instead.

TEST=ci

Bug: https://github.com/dart-lang/sdk/issues/45451
Change-Id: Iae984bce32baf29a950b5de1323939006a217b94
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212050
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-09-02 14:14:35 +00:00
asiva bc387837c4 [VM/Runtime] Reland : Fix 'File' object leak in async file open operation
The 'File' object created in the async file open method is leaked
if the operation is not completed when the isolate shuts down
with an unhandled exception.
This change adds a finalizable state for the 'File' object so that
the message deletion that happens when ports are closed could run
the callback to clean up the 'File' object.


TEST=new tests added

Change-Id: I64c18a7905261b0fc7bf9f220086791478232d0d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211340
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-08-27 17:45:17 +00:00
William Hesse 637e14006c Revert "[VM/Runtime] Fix 'File' object leak in async file open operation"
This reverts commit b67f45f955.

Reason for revert: This seems to cause a crash on ARMv7 aot compiled code.

Original change's description:
> [VM/Runtime] Fix 'File' object leak in async file open operation
>
> The 'File' object created in the async file open method is leaked
> if the operation is not completed when the isolate shuts down
> with an unhandled exception.
> This change adds a finalizable state for the 'File' object so that
> the message deletion that happens when ports are closed could run
> the callback to clean up the 'File' object.
>
> TEST=new tests added
>
> Change-Id: I4a3cb28370d27306c795c1914aeb5c18a1d85e2b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210761
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Ie91aadd318ef19a0bb4d7f769c9e876e76d719d8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211021
Reviewed-by: William Hesse <whesse@google.com>
Commit-Queue: William Hesse <whesse@google.com>
2021-08-24 12:49:23 +00:00
asiva b67f45f955 [VM/Runtime] Fix 'File' object leak in async file open operation
The 'File' object created in the async file open method is leaked
if the operation is not completed when the isolate shuts down
with an unhandled exception.
This change adds a finalizable state for the 'File' object so that
the message deletion that happens when ports are closed could run
the callback to clean up the 'File' object.

TEST=new tests added

Change-Id: I4a3cb28370d27306c795c1914aeb5c18a1d85e2b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210761
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-08-23 20:57:48 +00:00
Martin Kustermann c3def74ffe [vm/io] Shrink buffers using realloc() if actual used data is small
Issue b/178993708

TEST=standalone{,_2}/io/large_file_read_small_file_test

Change-Id: I3c6f09f4f79f6dd1150db2918cefed3587e34012
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182268
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-02 20:04:22 +00:00
Vyacheslav Egorov aed72ae8c8 [vm/bin] Improve fallback error message for process creation
Instead of just giving up try to give as much information as possible
including error code, error returned by Dart_StringFromUtf8 and
ascii portion of the OS error message.

We are making this change because we are getting reports of
ProcessException being thrown with "OS error message was a
not a utf8 string." message. There is no way to reproduce
these situations so instead we are going to add additional
information into the fallback message to aid the investigation.

TEST=tested manually by simulating fallback to the newly added code

Bug: b/178383611
Change-Id: Idf528dc762723694145a15aa07320220e8af9bfc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181200
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-01-26 12:56:49 +00:00
Daco Harkes 5278383736 [vm] Native API: Make Dart_NewWeakPersistentHandle not auto delete
Changes Dart_NewWeakPersistentHandle to no longer auto delete the
weak persistent handle.

Changes the signatures of WeakPersistentHandleFinalizers to no longer
have access to the handle.

Flutter PR: https://github.com/flutter/engine/pull/19843
g3 presubmit: cl/318028238

Issue: https://github.com/dart-lang/sdk/issues/42312

TEST=runtime/vm/dart_api_impl_test.cc

Change-Id: I3f77db9954d9486759f903b78c03a494f73c68ba
Cq-Include-Trybots:dart/try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,app-kernel-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-android-release-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-android-release-arm_x64-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,dart-sdk-linux-try,analyzer-analysis-server-linux-try,analyzer-linux-release-try,front-end-linux-release-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-mac-debug-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try,analyzer-nnbd-linux-release-try,front-end-nnbd-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151525
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-11-03 10:27:44 +00:00
Ben Konyi 157ebfe50f Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
This reverts commit 7e373831ce.

Change-Id: Ib94d793a5753ec6352cd9a65e35f726a63336368
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154830
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-07-21 16:23:18 +00:00
Alexander Aprelev 7e373831ce Revert "Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process""
This reverts commit bc4cb123a8.

Reason for revert: breaks crossword simarm tests https://ci.chromium.org/p/dart/builders/ci.sandbox/vm-kernel-precomp-linux-debug-simarm_x64/2619

Original change's description:
> Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
> 
> This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
> 
> By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
> 
> VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
> Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
> Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/42092)
> 
> This reverts commit 3849b5061c.
> 
> Change-Id: I4fd3ba33840771a9f284d733c4a25fac6cde64ca
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154706
> Reviewed-by: Siva Annamalai <asiva@google.com>

TBR=bkonyi@google.com,asiva@google.com

Change-Id: I649d94c668417f2edbfd7039fa5c876e10dc32fe
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154800
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-07-17 04:37:48 +00:00
Ben Konyi bc4cb123a8 Reland "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.

By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:

VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/42092)

This reverts commit 3849b5061c.

Change-Id: I4fd3ba33840771a9f284d733c4a25fac6cde64ca
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154706
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-07-17 00:54:11 +00:00
Ben Konyi 3849b5061c Revert "[ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process"
This reverts commit ffe258d2d4.

Reason for revert: Failures on bots

Original change's description:
> [ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process
> 
> This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.
> 
> By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:
> 
> VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
> Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
> Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/41978)
> 
> Change-Id: I1c6b1425831b691ad20284716aa80f817dbaf607
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152588
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,asiva@google.com

Change-Id: Idb1d24a4524bdc3ccfb199a82710f3c0d9db539a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154702
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2020-07-16 19:50:57 +00:00
Ben Konyi ffe258d2d4 [ VM / DartDev ] Launch DartDev in an isolate within a single main Dart process
This CL changes how DartDev is run and how the run command handles executing a Dart program (will port additional commands in a separate CL). Rather than using DartDev to spawn a child process to run user code, the VM will instead launch a DartDev isolate after doing some VM options processing. DartDev will communicate information like exit codes and script/arg pairs with the VM via isolate ports. Once DartDev runs to completion and notifies the VM that a script should be run, the VM will move on to spawning another isolate with user code and continue executing in the same VM process.

By moving DartDev into an isolate within the same process that user code will eventually run in we're able to resolve the following issues that arose due to signal handling and IPC issues:

VM hangs when --enable-vm-service is supplied and there are compile time errors (https://github.com/dart-lang/sdk/issues/42630)
Dart daemon spinning in exit code handler / zombie Dart processes (https://github.com/dart-lang/sdk/issues/41978)
Signal handling in children of 'dartdev run' is problematic (https://github.com/dart-lang/sdk/issues/41978)

Change-Id: I1c6b1425831b691ad20284716aa80f817dbaf607
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152588
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-07-16 19:03:24 +00:00
Siva Annamalai d46a696957 Revert "Reland : [VM/Runtime] Cleanup package config initialization code"
This reverts commit 03777dba31.

Reason for revert: Failures in couple of bots

Original change's description:
> Reland : [VM/Runtime] Cleanup package config initialization code
> 
> - Use _Init function for initialization of package config
> - Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
> 
>   DartUtils::SetupPackageConfig
> - Rename Loader::InitForSnapshot to Loader::Init
> - Remove _setPackagesConfig, _setWorkingDirectory from builtin library
> - Rename _setPackagesMap to _setPackagesConfig
> Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Commit-Queue: Siva Annamalai <asiva@google.com>

TBR=aam@google.com,asiva@google.com

Change-Id: Ibea8c35fe310ac7708f00bb9ffd58715b6924cb5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151869
Reviewed-by: Siva Annamalai <asiva@google.com>
Auto-Submit: Siva Annamalai <asiva@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 08:02:53 +00:00
asiva 03777dba31 Reland : [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,

  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I1e4a25e79706d7764247c9fbfcc8e8ce3231343c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151633
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2020-06-20 05:50:49 +00:00
asiva 4b9aa2bd7e Revert "[VM/Runtime] Cleanup package config initialization code"
This reverts commit 29e93bcdbd.

Reason for revert: Windows bots are failing with package resolution
errors. Looks like we are missing a windows path sanitization step.

Change-Id: Ib56f7e926b4f385fa3fde74c9c71947f64b8be1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151469
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-06-17 00:03:19 +00:00
asiva 29e93bcdbd [VM/Runtime] Cleanup package config initialization code
- Use _Init function for initialization of package config
- Remove DartUtils::SetWorkingDirectory, SingleArgDart_Invoke,
  DartUtils::SetupPackageConfig
- Rename Loader::InitForSnapshot to Loader::Init
- Remove _setPackagesConfig, _setWorkingDirectory from builtin library
- Rename _setPackagesMap to _setPackagesConfig
Change-Id: I0b83574593fa95e1b948134fcd4a7b4858ea19c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151045
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2020-06-16 23:24:26 +00:00
Ben Konyi 5ccbe533ab [ VM / dartdev ] Ignore VM flags in dartdev VM instance
In order to reduce confusion, if a VM instance detects that it will be
running a dartdev snapshot it will ignore all VM flags and print a
warning message to stderr informing users that the flags are being
ignored and that they should be passed immediately after the 'dart run'
command. This will prevent the dartdev snapshot from being run with
incompatible flags in addition to preventing users from accidentally
setting flags for the dartdev VM when they meant to set those flags for
the target process.

As usual, passing --disable-dart-dev will allow for the user to bypass
this behavior and run their dart program directly.

Fixes https://github.com/dart-lang/sdk/issues/41590

Change-Id: I7d5dc3d4ddae8be99994620b5b2c2e4cc492ae5e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/146984
Reviewed-by: Siva Annamalai <asiva@google.com>
2020-05-08 19:22:56 +00:00
Zichang Guo 8f7b40be77 [dart:io] Throw a FormatException when filters receive bad data.
Bug: https://github.com/dart-lang/sdk/issues/41270
Change-Id: Ie4de46129d6fe6ba00fde27f6c1c96cc3f42c888
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/141914
Commit-Queue: Zichang Guo <zichangguo@google.com>
Reviewed-by: Jonas Termansen <sortie@google.com>
2020-04-08 20:38:34 +00:00
asiva b1c6d89fb0 [VM/runtime] Remove support for loading URLs through service isolate
- Removes the load port functionality from the service isolate, this was
  used to load sources in Dart 1 and is pretty much dead code in Dart 2
- Moves resolution of URIs and Package URI (based on package map) to the
  individual isolates instead of sending a request to the service isolate
- Setups the package map lazily as it is needed only if resolveURI is
  called in Dart code, it is not needed for loading/compiling sources
  (this is now done in the front end).
- Removed Dart_ServiceWaitForLoadPort() methods from the Dart C API as
  it is not used anywhere.

Change-Id: I6c3704bc431bdcd49bf074eb58bee1ed492ccccb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/129742
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2019-12-27 17:30:05 +00:00
asiva b99599bae1 More cleanup of the loader code.
Change-Id: I8ee048f2a06f8aa54cb5efb832740a53b2bd042d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/127444
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2019-12-05 23:07:55 +00:00
Ryan Macnak d6373525f7 [standalone] Remove dead Dart 1 loading code.
Change-Id: I5551a40819b1e011cf50fe0dc38c74fe45f81160
Reviewed-on: https://dart-review.googlesource.com/c/85841
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-12-04 17:17:25 +00:00
Ryan Macnak 2f25dcda16 [vm] Remove script snapshots.
Change-Id: I5bc692b754c0909de4aa5f05af81b074578b16f4
Reviewed-on: https://dart-review.googlesource.com/76706
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2018-09-26 22:49:27 +00:00
Zach Anderson 27c79f9459 [vm] Propagate more UTF8 conversion errors
Change-Id: I8d4f3a892a51bed8b922961bd74bcad727f66508
Reviewed-on: https://dart-review.googlesource.com/73280
Commit-Queue: Zach Anderson <zra@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-09-06 02:23:21 +00:00
Alexander Aprelev 41e720b486 [vm] Add support for a new source file format kernel file lists.
Given a source file that starts with '#@dill' on a new line, then if the rest of the lines are full paths to the kernel dill files, all kernel dill files will be loaded and executed as single program.

So, for example, given `mytest.dillref` with following content:
```
/projects/mytest/build/bin/main.vm.dill
/projects/mytest/build/packages/mytest/lib.vm.dill
```

invocation of `dart mytest.dillref` will result in loading two dill files and starting combined app.

Bug: dartbug.com/33952
Change-Id: I22a99ec9a018da1bc984bc0d716af94bc10e20d6

Add few tests

Change-Id: I22a99ec9a018da1bc984bc0d716af94bc10e20d6
Reviewed-on: https://dart-review.googlesource.com/67460
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2018-08-21 22:12:54 +00:00
Vyacheslav Egorov c910738825 [vm/embedder] Introduce dart_standalone_embedder_api.h
This enables other emdedders to reuse parts of the standalone embedder
implementation without depending too much on the implementations details.

As part of the change we also remove a dependency from DartUtils on
IsolateData - by removing dart:_builtin library caching. Lookup
of this library does not seem to be on hot path anymore in Kernel world.

Change-Id: Ia35b28886121828fad7a96f00bcbceff75e00ae8
Reviewed-on: https://dart-review.googlesource.com/64848
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2018-07-17 09:04:05 +00:00
Alexander Markov c3b5939965 [vm] Remove --limit-ints-to-64-bits option and old _Bigint class
Closes https://github.com/dart-lang/sdk/issues/33306

Change-Id: I7088d8b7143edbe24f5cefe4be037ad2006e0625
Reviewed-on: https://dart-review.googlesource.com/58101
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2018-06-04 20:10:40 +00:00
Ben Konyi 3ab7cf0590 Revert "[dart:io] Revert recent non-utf8 path handling"
This reverts commit 42bcdcd332.

Change-Id: I11fb7db2aac3b7dc15cb35a22fea38b31069cbc0
Reviewed-on: https://dart-review.googlesource.com/55502
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-05-23 21:01:44 +00:00
Zach Anderson 42bcdcd332 [dart:io] Revert recent non-utf8 path handling
Reason for revert: Breaking API change blocks Flutter rolls. Reverting
to reconsider, work around, etc..

Revert "[ VM ] Fixed issue where backing array was deleted from an ExternalTypedData before the ExternalTypedData was used."

This reverts commit 916b9da48d.

Revert "[ VM ] Fixed tests for non-UTF8 paths on Windows / Macos."

This reverts commit 6327fe91c2.

Revert "[ VM ] Fix failing tests on Windows bots after non-UTF-8 paths change."

This reverts commit b51f4b7956.

Revert "[ VM ] Updated non-UTF-8 tests to expect exceptions on OSX/iOS."

This reverts commit bcb36247d2.

Revert "Revert "[ VM ] Updated non-UTF-8 tests to expect exceptions on OSX/iOS.""

This reverts commit 24450c79a5.

Change-Id: I756f74f4acdde168d0984fe130f05f9252366334
Reviewed-on: https://dart-review.googlesource.com/55441
Reviewed-by: Todd Volkert <tvolkert@google.com>
Commit-Queue: Zach Anderson <zra@google.com>
2018-05-16 15:47:10 +00:00
Ben Konyi 916b9da48d [ VM ] Fixed issue where backing array was deleted from an ExternalTypedData before the ExternalTypedData was used.
Reland "Fix issues with FileSystemEntities which have non-UTF8 paths causing exceptions to be thrown when converting to/from a Dart VM String."

This reverts commit 4f549e2900.

Change-Id: Ia27c8c300beb178fa7b08ffe0bfcb2d61f65bbb5
Reviewed-on: https://dart-review.googlesource.com/55271
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-05-15 21:54:46 +00:00
Ben Konyi 4f549e2900 Revert "Fix issues with FileSystemEntities which have non-UTF8 paths causing exceptions to be thrown when converting to/from a Dart VM String."
This reverts commit ba342a94ec.

Reason for revert: Bots are red.

Original change's description:
> Fix issues with FileSystemEntities which have non-UTF8 paths causing exceptions to be thrown when converting to/from a Dart VM String.
> 
> Continuation of PR #32583 by powdercloud
> 
> Fixes #29451
> 
> Change-Id: I4bdf24f9c8ead425a4f0cad33b642908f1ec203d
> Reviewed-on: https://dart-review.googlesource.com/51580
> Commit-Queue: Ben Konyi <bkonyi@google.com>
> Reviewed-by: Zach Anderson <zra@google.com>

TBR=bkonyi@google.com,rmacnak@google.com,zra@google.com,tvolkert@google.com

Change-Id: I12850f178a13a75289d18af4e565bd73c59b14ad
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/55060
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2018-05-14 23:40:29 +00:00
Ben Konyi ba342a94ec Fix issues with FileSystemEntities which have non-UTF8 paths causing exceptions to be thrown when converting to/from a Dart VM String.
Continuation of PR #32583 by powdercloud

Fixes #29451

Change-Id: I4bdf24f9c8ead425a4f0cad33b642908f1ec203d
Reviewed-on: https://dart-review.googlesource.com/51580
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-05-14 23:06:57 +00:00
Ryan Macnak a5c11d7d03 [vm] Remove Dart_ReadKernelBinary, take 2.
Update status for simulators, which the test harness invokes with DIL files instead of source files.

Change-Id: I8444ad7e17a0a71a1ce3c0021487397baa1c3e65
Reviewed-on: https://dart-review.googlesource.com/54622
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2018-05-14 20:58:08 +00:00