This is a reland of commit 7424295ce9
The differences between this reland and the original CL are: now the
Perfetto file recorder does not get built in PRODUCT builds, and it does
not get built unless the SUPPORT_PERFETTO macro is defined.
TEST=Recorded traces with the Perfetto file recorder and explored them
in Perfetto's trace viewer, CI
Original change's description:
> [VM] Begin supporting Perfetto file recorder
>
> This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
> timeline recorder that writes a trace to file in Perfetto's proto
> format. This CL supports the recording of all types of timeline events
> except flow events. Support for flow events will be added in a future
> CL.
>
> TEST=Recorded traces with the Perfetto file recorder and explored them
> in Perfetto's trace viewer, CI
>
> Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
Change-Id: I8713f704b5fbeed5f1231012bce8a32aaf566ae4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286020
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
For applications that want to have arbitrary number of isolates call
into native code that may be blocking, we expose the API functions that
allows those native threads to exit an isolate before running
long/blocking code.
Without the ability to exit/re-enter isolate, one may experience
deadlocks as we have a fixed limit on the number of concurrently
executing isolates atm.
In the longer term we may find a way to do this automatically
with low overhead, see [0]. But since those API functions are quite
stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't
see a reason not to expose `Dart_{Enter,Exit}Isolate`.
Difference to original CL:
Do use STL synchronization primitives (as the ones in runtime/bin
are not always available in shared libraries)
[0] Issue https://github.com/dart-lang/sdk/issues/51261
Issue https://github.com/dart-lang/sdk/issues/51254
TEST=ffi{,_2}/dl_api_exit_enter_isolate_test
Change-Id: Id817e8d4edb3db35f029248d62388cbd0682001d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294980
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This reverts commit a2512819f8.
Reason for revert: FFI tests fail to link on Windows, fail to load on product-mode Android
Original change's description:
> [vm] Expose Dart_{CurrentIsolate,ExitIsolate,EnterIsolate}
>
> For applications that want to have arbitrary number of isolates call
> into native code that may be blocking, we expose the API functions that
> allows those native threads to exit an isolate before running
> long/blocking code.
>
> Without the ability to exit/re-enter isolate, one may experience
> deadlocks as we have a fixed limit on the number of concurrently
> executing isolates atm.
>
> In the longer term we may find a way to do this automatically
> with low overhead, see [0]. But since those API functions are quite
> stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't
> see a reason not to expose `Dart_{Enter,Exit}Isolate`.
>
> [0] Issue https://github.com/dart-lang/sdk/issues/51261
>
> Issue https://github.com/dart-lang/sdk/issues/51254
>
> TEST=ffi{,_2}/dl_api_exit_enter_isolate_test
>
> Change-Id: I91c772ca962fddb87919663fea07939a498fa205
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292722
> Commit-Queue: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
Change-Id: I05ad5b9ce24754a68693160e470f8eb71a812c75
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294860
Auto-Submit: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
For applications that want to have arbitrary number of isolates call
into native code that may be blocking, we expose the API functions that
allows those native threads to exit an isolate before running
long/blocking code.
Without the ability to exit/re-enter isolate, one may experience
deadlocks as we have a fixed limit on the number of concurrently
executing isolates atm.
In the longer term we may find a way to do this automatically
with low overhead, see [0]. But since those API functions are quite
stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't
see a reason not to expose `Dart_{Enter,Exit}Isolate`.
[0] Issue https://github.com/dart-lang/sdk/issues/51261
Issue https://github.com/dart-lang/sdk/issues/51254
TEST=ffi{,_2}/dl_api_exit_enter_isolate_test
Change-Id: I91c772ca962fddb87919663fea07939a498fa205
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292722
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This forwards any `--packages` flag from before a dartdev command name to dartdev, so also eg. `dart --packages=... compile` will now work as expected.
And commands that don't take a --packages flag will complain (eg `dart --packages=... devtools`.
Change-Id: I0448b97aae394cb94541cfa087a6d28908e480e5
Tested: <Tested via pkg/dartdev/test/commands/run_test.dart>
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292020
Reviewed-by: Jonas Jensen <jonasfj@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
--timeline-streams has been listed in the "options implied by --observe"
section of the dart run help messages, but it has been getting ignored.
This CL fixes that problem.
TEST=manual testing, CI
Change-Id: Ib6c2425a2681b61375df186673e0f195e3fae580
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/288581
Reviewed-by: Ben Konyi <bkonyi@google.com>
This reverts commit c67fac9cb4.
Reason for revert: Regresses `dart:io` performance and causes
failures.
https://github.com/dart-lang/sdk/issues/51639
Original change's description:
> [vm] Remove tcmalloc and malloc profiler.
>
> The standalone VM originally began statically linking tcmalloc to work around bugs in the system malloc for Fiber. Later it used tcmalloc's hooks to implement a profiler, but this is rarely used since it is only available in debug mode, misses early allocations, and often misses late allocations from an exhausted sample buffer. Removing it altogether avoids build complexity around which combinations of compiler/architecture/sysroot support tcmalloc, and reduces binary size.
>
> TEST=ci
> Change-Id: I4b259e18b82b2d12a2a60962aabf83bd8d997d19
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286120
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
Change-Id: I4395edd6f5bd7e26b4e38f4d931ad2ea67afba18
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286925
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
The standalone VM originally began statically linking tcmalloc to work around bugs in the system malloc for Fiber. Later it used tcmalloc's hooks to implement a profiler, but this is rarely used since it is only available in debug mode, misses early allocations, and often misses late allocations from an exhausted sample buffer. Removing it altogether avoids build complexity around which combinations of compiler/architecture/sysroot support tcmalloc, and reduces binary size.
TEST=ci
Change-Id: I4b259e18b82b2d12a2a60962aabf83bd8d997d19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286120
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Avoid TLS initialization checks by using inline initialization.
- Avoid global offset table indirection by reducing -fPIC to -fPIE.
out/ReleaseXARM64/exe.stripped/dart_precompiled_runtime
11137992 -> 11274776 (-1.21%)
We still need -fPIC in some places because we build a few shared libraries for FFI, so copy some of Fuchsia's GN setup to use -fPIE or -fPIC as appropriate.
Account for older gcc that does not default to -fpie.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/51602
Change-Id: I85072153cb1aef9047c1adbf36c7496fbeb11e10
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/286221
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
- Avoid TLS initialization checks by using inline initialization.
- Avoid global offset table indirection by reducing -fPIC to -fPIE.
out/ReleaseXARM64/exe.stripped/dart_precompiled_runtime
11137992 -> 11274776 (-1.21%)
We still need -fPIC in some places because we build a few shared libraries for FFI, so copy some of Fuchsia's GN setup to use -fPIE or -fPIC as appropriate.
TEST=ci
Change-Id: I6402fce3366a9c4b2741ffb4811562292b8ecb71
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285704
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This reverts commit 7424295ce9.
Reason for revert: Causes VM crash in internal tests - b/271081079
Original change's description:
> [VM] Begin supporting Perfetto file recorder
>
> This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
> timeline recorder that writes a trace to file in Perfetto's proto
> format. This CL supports the recording of all types of timeline events
> except flow events. Support for flow events will be added in a future
> CL.
>
> TEST=Recorded traces with the Perfetto file recorder and explored them
> in Perfetto's trace viewer, CI
>
> Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
> Reviewed-by: Ben Konyi <bkonyi@google.com>
> Commit-Queue: Derek Xu <derekx@google.com>
Change-Id: I1c4b0b78be3748b525b82ea66079dd95d412d82e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/285906
Auto-Submit: Oleh Prypin <oprypin@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Commit-Queue: Alexander Thomas <athom@google.com>
This CL adds the `TimelineEventPerfettoFileRecorder` class, which is a
timeline recorder that writes a trace to file in Perfetto's proto
format. This CL supports the recording of all types of timeline events
except flow events. Support for flow events will be added in a future
CL.
TEST=Recorded traces with the Perfetto file recorder and explored them
in Perfetto's trace viewer, CI
Change-Id: Iaa2051e536589a473c5e15f9de9bb9c251f13d38
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278942
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
Some fields of `OSThread` are initialized by retrieving information by
calling OS functions. For example, `trace_id_` is initialized by calling
`OSThread::GetCurrentThreadTraceId()`. Similarly, this CL adds the
`OSThread::GetCurrentThreadName()` method and uses it to initialize
`name_`.
TEST=CI
Change-Id: Iee121d71e660be01ff684a298144fa7d59e3b61d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278662
Commit-Queue: Derek Xu <derekx@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
Updated strictness on build configuration for analyze_snapshot
Will run only under Linux/Android 64 bit arm, arm64c, simarm64,
simarm64c and x64 architectures.
analyze_snapshot just has a single target that links into product precompiled programatically
Testing suite updated to comply with null-safety requirements and
fix for simulated platform failing to use gen_snapshot with assembly.
TEST=ci
Change-Id: I3d58400db2e26a441a40fe7197b22510a52732b6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/279391
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Chris Evans <cmevans@google.com>
This CL introduces `VarArgs` to `NativeFunction` signatures. The
`VarArgs` type takes a single type argument. This type argument is a
subtype of `NativeType` if there is a single variadic argument, and a
record with native types if there are multiple variadic arguments.
For example:
`NativeFunction<Void Function(Pointer<Char>, VarArgs<(Int32,Int32)>)>`
for calling refering to a `printf` binding with two `int32_t` arguments
passed as variadic arguments.
The logic of the native calling conventions are detailed in
https://dart-review.googlesource.com/c/sdk/+/278342.
Here we explain how this influences the FFI pipeline.
First, now that `VarArgs` is part of signatures, we have to unwrap
that when with the C types in the CFE transform and checking (analyzer
is in a separate CL), and also in the marshaller when looking up the
C type of arguments.
Second, we have to deal with `BothNativeLocations`. On windows x64,
floating point arguments must be passed both in FPU _and_ CPU
registers. For FFI calls, we solve this in the argument moves by just
copying to both locations. For FFI callbacks, we just take the FPU
register location (which avoids an extra bitcast).
Third, on System-V, we have to pass an upper bound of the number of
XMM registers used in AL. This means we instead RAX, we use R13 for the
target address. For variadic calls, we always pass 8 in AL as the valid
upper bound. We could consider passing the actual number of XMM
registers used.
We keep using RAX as default register for the function address on non-
variadic calls, because changing to R13 (the first free) register
creates more spilling in leaf calls. R13 is callee-saved while RAX is
not, so using R13 instead of RAX causes us to have to spill the value
from RAX on leaf calls.
Fourth, on both x64 and RISC-V, we pass floats in integer locations.
`EmitNativeMove` has been modified to deal with this, so that we do not
have to insert more `BitCastInstr`s.
The tests are generated by a test generator: `tests/ffi/generator/`.
The formatter doesn't support records yet, so the tests are not properly
formatted.
Bug: https://github.com/dart-lang/sdk/issues/50798
TEST=tests/ffi/*_varargs_*
Closes: https://github.com/dart-lang/sdk/issues/38578
Closes: https://github.com/dart-lang/sdk/issues/49460
Closes: https://github.com/dart-lang/sdk/issues/50858
Change-Id: I6a6296fe972527f8a54ac75a630131769e3cc540
Cq-Include-Trybots: luci.dart.try:vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-win-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-nnbd-win-release-ia32-try,vm-kernel-nnbd-win-debug-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-kernel-precomp-android-release-arm64c-try,vm-kernel-precomp-android-release-arm_x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-precomp-ffi-qemu-linux-release-riscv64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-precomp-asan-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,app-kernel-linux-debug-x64-try,vm-kernel-mac-release-arm64-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/276921
Reviewed-by: Devon Carew <devoncarew@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Adds support for `asset: 'asset-id'` in `FfiNative`s.
Symbols in assets are resolved by
1. the provided [asset] (the bulk of this CL),
2. the native resolver set with `Dart_SetFfiNativeResolver` in
`dart_api.h` (old behavior), and
3. the current process.
Symbols in assets are resolved by the Dart VM through the mapping
provided in the `vm:ffi:native_assets` library:
```
@pragma('vm:ffi:native-assets', {
"linux_x64": {
"<asset1>": [
"absolute",
"<absolute-path>"
],
"<asset2>": [
"relative",
"<relative-path>"
],
"<asset3>": [
"system",
"<system-path>"
],
"<asset4>": [
"process",
],
"<asset5>": [
"executable",
],
}
})
library;
```
There are 5 asset path types. Symbols resolved in these asset types are
resolved as follows.
A. Absolute paths are dlopened, after which the symbol is looked up.
B. Relative paths are resolved relative to `Platform.script.uri`, and
dlopened, after which the symbol is looked up.
C. System paths are treated as absolute paths. (But we might explicitly
try to use `PATH` in the future.)
D. Symbols looked up in Process assets are looked up in the current
process.
E. Symbols looked up in Executable assets are looked up in the current
executable.
The native assets mapping can be provided in three ways.
X. In the invocation to `gen_kernel` with the `--native-assets`
argument. This uses the gen_kernel entry point for compiling
kernel.
Y. By placing a `native_assets.yaml` next to the
`package_config.json` used by the isolate group. This works for
`dart <source file>` and `Isolate.spawnUri(<dart source file>)`.
This uses the kernel_service entry point for compiling kernel.
Z. By sending a String containing the native assets mapping when
invoking the kernel_service directly from within the VM. Currently,
only used for unit tests.
TEST=tests/ffi/native_assets/asset_*_test.dart (X)
TEST=tests/ffi/native_assets/infer_native_assets_yaml_*.dart (Y)
TEST=runtime/vm/ffi/native_assets_test.cc (Z)
The library is synthesized from yaml in pkg:vm as AST.
Design doc: go/dart-native-assets
Bug: https://github.com/dart-lang/sdk/issues/49803
Change-Id: I8bf7000bfcc03b362948efd3baf168838948e6b9
Cq-Include-Trybots: luci.dart.try:vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-mac-debug-arm64-try,vm-kernel-nnbd-mac-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-debug-x64c-try,vm-kernel-precomp-nnbd-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264842
Reviewed-by: Martin Kustermann <kustermann@google.com>
This enables running other C/C++ code before the Dart main function
without having to implement the full embedder API by including
main_lib and friends in a build but excluding `main.cc`.
TEST=ci build + flutter bot
Change-Id: I5739eb21fa6918b0a2c0f33f7d97dad73c5cf0b1
Cq-Include-Trybots: luci.dart.try:flutter-engine-linux-try,flutter-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/278518
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Derek Xu