This reverts commit 553887bc56.
Reason for revert: Broken android builds
Original change's description:
> [vm] Add NativeRuntime.streamTimelineTo
>
> This API allows to programmatically generate timeline information
> including profiling samples if timeline format supports it.
>
> This change also includes Perfetto timeline writer and profiler
> into non-mobile PRODUCT builds. The size of Mac OS X dartaotruntime
> has increased by 34352 bytes (0.7%).
>
> TEST=vm/dart/stream_timeline_to_test
>
> CoreLibraryReviewExempt: VM specific dart:developer API.
> Change-Id: I326f5fd29ee82a503160f5f5a2a857a7c55330ec
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/426220
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Slava Egorov <vegorov@google.com>
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I583ad74b850514fb47054149da464a07874fc2f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/457000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This API allows to programmatically generate timeline information
including profiling samples if timeline format supports it.
This change also includes Perfetto timeline writer and profiler
into non-mobile PRODUCT builds. The size of Mac OS X dartaotruntime
has increased by 34352 bytes (0.7%).
TEST=vm/dart/stream_timeline_to_test
CoreLibraryReviewExempt: VM specific dart:developer API.
Change-Id: I326f5fd29ee82a503160f5f5a2a857a7c55330ec
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/426220
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
For some reason unboxing information was not generated for unreachable
fields. However, unreachable field can be used as an interface target,
and unboxing info contains a flag indicating if register calling
convention should be used which affects how the call via this field
as an interface target is going to be performed.
So it is incorrect to omit such unboxing info and this change fixes
this bug.
TEST=vm/dart/regress_b_450678575_test
Bug: b/450678575
Change-Id: Ic1f299d9c0b09005c8bbfd95d1a409c113630263
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/454681
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Instead of requiring embedder to initialize dart:io by bits and pieces
by invoking various hooks and setting fields provide a structured C++
API for preparing dart:io for execution.
TEST=ci
CoreLibraryReviewExempt: VM specific changes only
Change-Id: I66e95c0a451d384b4f7582d1df059e506a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/454062
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This CL tries to address version skew issues which occur in internal
monorepo which we created by checking in *.pbzero.h files into
runtime/vm/protos/*. These files refer to protozero internal headers,
which in the internal monorepo come from a shared Perfetto dependency
which does not necessarily match the version we pinned in our DEPS and
used to generated these files.
This version skew caused problems for the Perfetto team, see
https://github.com/google/perfetto/pull/3195
To resolve this change layout of our repo to allow us to depend on the
shared Perfetto entirely - including its version of *.pbzero.h headers.
We change include paths in the VM source to
#include "third_party/perfetto/protos/..."
which match location of these headers in the internal monorepo.
To make the code compile in the normal SDK checkout we shift location of
Perfetto and checked in code:
* Perfetto source moves from third_party/perfetto to
third_party/perfetto/src.
* Generated code moves from runtime/vm/protos into
third_party/perfetto/protos.
Note: experiments show that *.pbzero.h files tree-shake very well and
you pay only for things that you use. That means depending on *.pbzero.h
files generated from full message definitions rather than our manually
tree-shaken versions does not actually increase the size of the VM
binaries. We could switch our external build to do the same but this
would make protoc a build time dependency, which seems excessive.
TEST=ci
Change-Id: I724b9d9304be83302a5d83c2337154b36a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/454260
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
There are two possible flags for each source position currently:
a flag that marks the source position as synthetic and a flag that
marks the source position as within a yield point.
Synthetic source positions in bytecode are treated the same as synthetic
source positions in compiled code. That is, they encode the source
position in the text that caused them to be synthesized, but denote that
the covered instructions are internal and not to be used for debugger
pause points or for call site/branch coverage information.
Adding these flags allow us to mark appropriate parts of the async
machinery as synthetic, and also allow us to mark all the bytecode
involved in yield points as having the same token position.
The latter fixes tests where the code would step over a previous
expression, thus being paused at the start of the await bytecode,
and would record the fp and token position there as the ones to
ignore. However, since a new source position wasn't emitted until the
direct call to the await method, the recorded token position would
be the token position prior to the await call, and so the change
in token position at the await call would trigger an early pause.
TEST=pkg/vm_service/test/async_single_step_exception_test
pkg/vm_service/test/async_single_step_into_test
pkg/vm_service/test/async_single_step_out_test
pkg/vm_service/test/async_star_single_step_into_test
pkg/vm_service/test/async_step_out_test
pkg/vm_service/test/positive_token_pos_test
pkg/vm_service/test/step_into_async_no_await_test
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-product-x64-try
Change-Id: Ic7642a74fb76227a473f461f360e84dd3d5a45a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/453322
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
This change leans on VMs ability to provide a awaiter stack traces (if
the code is written using async syntax sugar or properly annotated with
@pragma('vm:awaiter-link').
Currently doing
await Stream.fromIterable([]).first;
would result in an empty stack trace. However after this change
we actually get a meaningful stack trace when running on the VM.
Fixes https://github.com/dart-lang/sdk/issues/51025
TEST=vm/dart/awaiter_stacks/stream_methods
CoreLibraryReviewExempt: No functional or API changes. Just better error messages on exceptional code paths.
Change-Id: Ifc3e446c00121cc97f2f870c3e3f5bf56a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/453940
Reviewed-by: Lasse Nielsen <lrn@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
Fixes bug in commit 25df2b3f11.
The code for converting FSEventStream notification into Dart format
was checking file existence using relative path instead of absolute
path. This caused it to incorrectly classify rename events which
were creations as deletions.
This was not caught by testing because this was not well covered
by the test and the test closest to checking this was even
disabled on Mac. CL updates updates the test to cover this better.
Fixes https://github.com/dart-lang/sdk/issues/61693
TEST=standalone/io/file_system_watcher_test
Change-Id: I2a7c984dfcec0be74c974f13b6ead7d86a6a6964
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/453880
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Also the pending objects list can be added to while handling other
pending objects, so the length of the list must be re-retrieved on each
iteration.
TEST=pkg/vm_service/test/coverage_leaf_function_test.dart
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try
Change-Id: I1089515f3e967ddc41fb82ce9f0541a81b9529d2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/452784
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Watcher used pipes to fit into the "socket"-like implementation used by
other OSes. However it turns out that pipe buffer on Mac OS X is not
necessarily large enough to fit the whole FSEvent structure meaning that
we can't expect to be able to read FSEvents atomically from a pipe.
This situation happens because Mac OS X inherits BSD behavior of
limiting total number of kernel memory reserved for pipe buffers. Once
kern.ipc.maxpipekva limit is crossed pipe buffers no longer grow to 64KB
and remain 512 bytes large (which is absolute minimum allowed by POSIX
which requires writes smaller than PIPE_BUF to be atomic).
So our code needs to be prepared that only half of FSEvent structure
(which is 1032 bytes large) will fit into pipe buffer.
We could fix this by reading each FSEvent in chunks but this seems
ridiculously ineffecient. Instead we rewrite the code to pass file
system events via SendPort instead.
Fixes https://github.com/dart-lang/sdk/issues/61551
TEST=runtime/tests/vm/dart/regress_61551_test.dart
CoreLibraryReviewExempt: VM only changes.
Change-Id: I6a6a696499044832cfb61998bc6a519bbaadf8bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/451260
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Existing implementation is an entangled mess which consists of shared
code residing in the base class which in random places invokes a number
of undocumented poorly named methods overloaded in OS specific
subclasses. Some of these methods mutate static state. There are no
clear lifetime guarantees for different parts of the system (including
comments saying that some values might or might not be valid at certain
points).
The rewrite aims to clean most of this up - sharing everything that can
be shared and moving OS specific logic to clearly documented methods.
Furthermore, we change the code to ensure proper lifetime guarantees -
so we no longer find ourself in situations where we don't know whether
pathId is valid or not.
This refactoring by itself fixes a number of issues, most specifically a
bug where watcher would stop receiving events on Windows because
DirectoryWatchHandle ends up allocated at precisely the same address as
a previous destroyed one - which confuses Dart side to think that newly
created handle is the same as the old one (due to a race between event
handler thread and Dart thread).
We fix Windows lifetime issue by a) not keeping pathId based mapping in
the watcher anymore and b) keeping DirectoryWatchHandler alive until it
is stoped by the Dart side - this is achieved by retaining it after it
is created and releasing it once path is unwatched. This way Dart side
is always sure that pathId values are valid until they are explicitly
released via _unwatchPath - which makes code very uniform.
To make sure that native objects created by _watchPath are released when
surrounding isolate exists abruptly (e.g. via Isolate.exit - without
letting Dart code to shutdown and call _unwatchPath naturally) we attach
NativeFinalizer to them. This fixes the existing leak of file watchers
on Mac OS X - as Node objects it created were not freed if surrounding
isolate exited. Note that inotify descriptors did not leak in the same
way because they were wrapped into sockets.
Finally, this refactoring also make sure that the last subscriber
cancelling subscription on filesystem event stream will get a proper
cancellation future back and can wait for the watcher to shutdown.
Previously implementation used broadcast streams which simply return an
already completed future when subscriber cancels. New implementation
uses Stream.multi instead which gives a better result. Now doing
watch().listen().cancel() returns a future which will only complete once
watcher is fully disposed (e.g. inotify descriptor is closed). Bad
behavior was revealed by analysing standalone/regress_52715 - which
revealed that repeatedly watching and cancelling might flakely cause us
to hit fd limit depending on whether eventhandler thread can keep up
closing file descriptors created by the main thread or not.
Fixes https://github.com/dart-lang/sdk/issues/61378
TEST=standalone/{regress_61378,file_system_watcher_isolate_exit_leak}
CoreLibraryReviewExempt: VM only changes.
Change-Id: I6a6a69642b1f2673f2be78434bc64270846ad8c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/450921
Reviewed-by: Lasse Nielsen <lrn@google.com>
This library no longer has old school natives and uses FFI natives
instead which are configured in bootstrap instead.
TEST=ci
Change-Id: I6a6a6964e1ee692153dfeb8dbaf692b35851e893
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/452141
Reviewed-by: Alexander Aprelev <aam@google.com>
Add a flag to SourceFile which is true if the associated script
contains a list of covered const constructors, and serialize said
list if true.
Delay reading and resolving the list of covered const constructors in
the bytecode reader until code is read.
Update usage counters for interpreted functions in Entry instructions.
Fix up cases where SourceReport assumed compiled code as appropriate.
Remaining:
* Record call and assert coverage information if the current isolate
group has coverage enabled.
* Record branch coverage information if the current isolate group
has branch coverage enabled.
TEST=pkg/vm_service pkg/dart2bytecode
Cq-Include-Trybots: luci.dart.try:vm-dyn-linux-debug-x64-try,vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-mac-debug-arm64-try
Change-Id: I14b6ffba1e175993e992c0fe939473557303bfeb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/449900
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Bytecode doesn't fully support dart:mirrors and support for
dart:mirrors is already disabled in dart2bytecode.
This change disables dart:mirrors in the standalone VM running with
--interpreter flag and updates status files.
TEST=ci
Change-Id: Id9423bafb10926e8274835b77a6f50a825ad7fd7
Cq-Include-Trybots: luci.dart.try:vm-aot-dyn-linux-debug-x64-try,vm-aot-dyn-linux-product-x64-try,vm-dyn-linux-debug-x64-try,vm-dyn-mac-debug-arm64-try,vm-ffi-dyn-mac-debug-simarm64_arm64-try,vm-ffi-dyn-mac-release-simarm64_arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/451580
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Bob Nystrom <rnystrom@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Slava Egorov