Extends the `NativeAssetsApi` with a `dlopen` that takes an asset id
instead of an asset path. This enables the embedder (instead of the
vm) to resolve the asset id to asset path.
Additionally, the `NativeAssetsApi` gets an `available_assets`
callback to report which asset ids are available if asset resolution
failed. (Otherwise, users would lose this important part of the error
message on failed resolution.)
We postpone migrating the Dart standalone embedder to this API: https://dart-review.googlesource.com/c/sdk/+/388160. That CL verifies
this new API and it's implementation. Without that CL this new API is
hard to test. So, no further tests are added here.
Tests that verify the old behavior for the standalone embedder:
TEST=tests/ffi/native_assets/*
TEST=pkg/dartdev/test/native_assets/*
Bug: https://github.com/flutter/flutter/issues/154425
Change-Id: I04d0fb45dc5663e63d91b21a6f5764929f10aaff
Cq-Include-Trybots: dart/try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,pkg-linux-debug-try,pkg-linux-release-arm64-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-win-release-try,pkg-win-release-arm64-try,vm-aot-asan-linux-release-x64-try,vm-asan-linux-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-msan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/388161
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reland commit 5a32d8bc7c with a fix
for thread leak (Issue #56717): when `ThreadPool` is shutting down
asynchronously the last worker should detach itself to prevent
leaking associated low-level data structures, because no thread will
join it.
A hang in service isolate shutdown (caused by an existing bug) was fixed by commit 157a0dc7f9.
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
Closes https://github.com/dart-lang/sdk/issues/56717
TEST=ci
Change-Id: Ic68bfb60757685afd75c80a70cdec66cc13c149b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/385000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
This reverts commit 5a32d8bc7c.
Reason for revert: DartIsolateTest.CanCreateServiceIsolate flutter engine unit test hangs on Windows.
Original change's description:
> [vm] Simplify implementation of native ports
>
> This CL turns native ports into a thin abstraction over underlying
> thread pool instead of building them as full fledged MessageHandler.
>
> This allows to easily implement a variation of native ports which can
> handle messages concurrently with the given degree of concurrency.
> This type of port can be used to greatly simplify implementation of
> IOService - which previously had to do its own concurrency management
> on top of "single threaded" native ports. This capability is exposed
> as `Dart_NewConcurrentNativePort` API.
>
> The new implementation is in general much cleaner then the old one
> with one exception: `Dart_CloseNativePort` API has unfortunate design
> where underlying message handler is destroyed asynchronously and
> `Dart_CloseNativePort` returns immediately without waiting for pending
> tasks to complete. Implementing this on top of `ThreadPool` requires
> some changes to thread pool implementation.
>
> Issue https://github.com/dart-lang/sdk/issues/55844
>
> TEST=ci
>
> Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: I7de27793a54072e974bf1a9f17a07c12159a202d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384481
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
TEST=ci
Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This CL moves native assets resolution to the embedder.
The Dart VM looks up the asset path (for example
`['relative', 'foo.so']`) with the asset id. The embedder defines
callbacks for asset loading, and returns a handle to the dylib.
Then the VM calls the embedder again with `dlsym` to lookup the symbol.
The Dart VM & kernel compiler are responsible for the asset id to
asset path mapping. The kernel compiler compiles it into the snapshot
and the VM looks it up in the snapshot.
Relative paths are resolved relative to the isolate script uri (kernel
snapshot, jit snapshot, aot snapshot, or `dart compile exe` result).
The embedder is responsible for remembering the script uri it set when
spawning the isolate group.
This CL does not add `dlclose` or `dladdr` embedder callbacks yet.
Bug: https://github.com/dart-lang/sdk/issues/55521
Bug: https://github.com/dart-lang/sdk/issues/55966
TEST=pkg/dartdev/test/native_assets/build_test.dart
TEST=tests/ffi/native_assets/asset_relative_test.dart
Bug: https://github.com/dart-lang/sdk/issues/55410
Bug: https://github.com/dart-lang/sdk/issues/55523
Bug: https://github.com/dart-lang/sdk/issues/55207
Change-Id: I75ec4a368c5fb3d2f76b03771e796ff56bcac941
Cq-Include-Trybots: dart/try:vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,pkg-linux-debug-try,pkg-linux-release-arm64-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-win-release-try,pkg-win-release-arm64-try,vm-aot-asan-linux-release-x64-try,vm-asan-linux-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-msan-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/361881
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This will enable the standalone embedder to resolve uris in a follow
up CL:
https://dart-review.googlesource.com/c/sdk/+/361881
The implementation was using zones for allocation, this CL switches
that to using `malloc`. The caller is responsible for freeing the
memory if resolving succeeds.
This CL removes `Dart_DefaultCanonicalizeUrl`.
We don't have a run_platform_tests, so this CL keeps the tests in
run_vm_tests.
TEST=vm/cc/ParseUri
TEST=vm/cc/ResolveUri
TEST=tests/ffi/native_assets/asset_relative_test.dart
Bug: https://github.com/dart-lang/sdk/issues/55523
Change-Id: Ifb300d8164eb50506f22ce619fad0811f74ef34c
Cq-Include-Trybots: luci.dart.try:vm-aot-asan-linux-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-asan-linux-release-arm64-try,vm-asan-linux-release-x64-try,vm-msan-linux-release-arm64-try,vm-msan-linux-release-x64-try,pkg-linux-debug-try,pkg-linux-release-arm64-try,pkg-linux-release-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-win-release-arm64-try,pkg-win-release-try,vm-aot-linux-debug-x64-try,vm-linux-debug-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-aot-win-debug-x64-try,vm-win-debug-arm64-try,vm-mac-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/368423
Reviewed-by: Martin Kustermann <kustermann@google.com>
When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
This CL removes support for --no-sound-null-safety option, the VM will
only run sound null safety code.
First part of deleting support for legacy and unsound null safe code.
TEST=CI
Change-Id: I9dbed10a18ef102533d6af516ff5075297d17b74
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/357183
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
This change removes support for external strings from the VM along with
Dart_NewExternalLatin1String, Dart_NewExternalUTF16String and
Dart_IsExternalString Dart C API functions.
External strings are not used by the VM nor any known embedder, but
Dart VM was paying the maintenance and performance price for
the external string implementation classes.
TEST=ci
Change-Id: I094cd2d2b7ec0840e9f09e1ca9e5a7acd4e78c28
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/358760
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Moving the class name of the allocated object to the allocation callback
gives more flexibility to the embedder, which may decide to track all
sampled allocations, not just those that are still alive.
TEST=Updated DartAPI_HeapSampling_*
Change-Id: Iaae290e15b67ca3e47d72e1896fa6e32b2d4b081
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/297761
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
Also in this change:
- Add heap_size to Dart_HeapSamplingCreateCallback
- Remove heap_size from Dart_HeapSamplingReportCallback
TEST=Existing dart_api_impl_tests updated
Change-Id: Iaa8d340de94d95f93c94f4a3d29d6d889dad49de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294800
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This API allows embedder to append arbitrary footers to DWARF stack
traces. For example, embedder can append a link to a symbolizer
service, which could be used to symbolize the stack-trace.
TEST=manually tested, hard to create automatic tests because vm/cc tests only support JIT, not AOT.
Bug: b/255741575
Change-Id: Id034b9b3194f7b91a8405574ea771c4a06fda2c2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/275700
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
- Flip flag to make strong null safety the default
- Remove code that auto detects null safety mode from source files,
it is necessary to specify --no-strong-null-safety to opt out.
- Retains sniffing of AOT/JIT snapshots and kernel files to determine
null safety mode, the opt out has to be done when generating these
file.
TEST=ci
Change-Id: If2c9608eedb7c46d9c3cd85e261ee9640e0d28eb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/261140
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
This CL introduces new embedding APIs for supporting heap sample
profiling. A registered sampling callback is invoked approximately every
N bytes based on an exponential distribution, providing information
about the isolate group the allocation occurred in, the user visible
name of the allocated object type, a weak persistent handle to the
allocated object, and the size of the allocation.
Sampling is triggered using artificial TLAB boundaries to cause
allocations to be sampled to take the allocation slow path where the
registered callback can be invoked with the allocation information.
Only new space allocations are currently traced, with old space
allocation support to be added in a future CL.
TEST=Dart_HeapSampling
Change-Id: I22bcdeec6e823bc1ab44898d4c596fbed7169fa1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/264520
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Liam Appelbe