This reland commit 5a32d8bc7c with a fix
for thread leak (Issue #56717): when `ThreadPool` is shutting down
asynchronously the last worker should detach itself to prevent
leaking associated low-level data structures, because no thread will
join it.
A hang in service isolate shutdown (caused by an existing bug) was fixed by commit 157a0dc7f9.
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
Closes https://github.com/dart-lang/sdk/issues/56717
TEST=ci
Change-Id: Ic68bfb60757685afd75c80a70cdec66cc13c149b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/385000
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
If main function throws an error it might leave service isolate
in an inconsitent state and it does make sense to keep the isolate running
we might even dead lock trying to shut it down later (e.g. it might have failed
to create native port).
Flutter engine has a unit test which beings startup sequence for service
isolate and then quickly shutdowns VM - in this case vmservice main throws
and error trying to create native port. If we mark service isolate as
running at this state we will end up deadlocking in shutdown sequence
because main thread can't send exit message to the service isolate (as
we failed to create a port).
Fixes https://github.com/dart-lang/sdk/issues/56709
TEST=flutter engine unit test DartIsolateTest.CanCreateServiceIsolate
Change-Id: Ia0dc089cd31f93022708ff647950febd25eb3d7c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384482
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
`KernelLoader::FinishClassLoading` was leaving guarded state of this
field in an inconsistent state: it was setting guarded cid to
`kDynamicCid`, but not setting guarded length to `kNoFixedLength`.
As a result in JIT mode we would generate stale length guards
for these fields and these guards would inhibit allocation
sinking.
This CL fixes this.
Issue https://github.com/dart-lang/sdk/issues/56705
TEST=vm/cc/Ffi_StructSinking
R=dacoharkes@google.com
Change-Id: I2eae6447555a7c1dfa25591e51815936aa3dbaf3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384842
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
This reverts commit 5a32d8bc7c.
Reason for revert: DartIsolateTest.CanCreateServiceIsolate flutter engine unit test hangs on Windows.
Original change's description:
> [vm] Simplify implementation of native ports
>
> This CL turns native ports into a thin abstraction over underlying
> thread pool instead of building them as full fledged MessageHandler.
>
> This allows to easily implement a variation of native ports which can
> handle messages concurrently with the given degree of concurrency.
> This type of port can be used to greatly simplify implementation of
> IOService - which previously had to do its own concurrency management
> on top of "single threaded" native ports. This capability is exposed
> as `Dart_NewConcurrentNativePort` API.
>
> The new implementation is in general much cleaner then the old one
> with one exception: `Dart_CloseNativePort` API has unfortunate design
> where underlying message handler is destroyed asynchronously and
> `Dart_CloseNativePort` returns immediately without waiting for pending
> tasks to complete. Implementing this on top of `ThreadPool` requires
> some changes to thread pool implementation.
>
> Issue https://github.com/dart-lang/sdk/issues/55844
>
> TEST=ci
>
> Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
> Commit-Queue: Slava Egorov <vegorov@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
Change-Id: I7de27793a54072e974bf1a9f17a07c12159a202d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384481
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This CL turns native ports into a thin abstraction over underlying
thread pool instead of building them as full fledged MessageHandler.
This allows to easily implement a variation of native ports which can
handle messages concurrently with the given degree of concurrency.
This type of port can be used to greatly simplify implementation of
IOService - which previously had to do its own concurrency management
on top of "single threaded" native ports. This capability is exposed
as `Dart_NewConcurrentNativePort` API.
The new implementation is in general much cleaner then the old one
with one exception: `Dart_CloseNativePort` API has unfortunate design
where underlying message handler is destroyed asynchronously and
`Dart_CloseNativePort` returns immediately without waiting for pending
tasks to complete. Implementing this on top of `ThreadPool` requires
some changes to thread pool implementation.
Issue https://github.com/dart-lang/sdk/issues/55844
TEST=ci
Change-Id: I062040ff233e93962ae93684e9b044d8facdaffc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382163
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
The algorithm for ordering sections in `CreateProgramTable` permitted
skew between the order of sections in snapshot and corresponding
debug info. This skew can lead to inconsistencies in `.text` section
memory offset between two generated ELF files making debug info
invalid.
This skew was left unnoticed for a long time because executable
segment has a huge alignment requirement (64K) while the effect
of the skew was usually rather small and thus hidden by realignment.
This CL adds a strict verification pass (always enabled) which
guards against such skew in the future.
A better architecture would be to ditch parallel generation of
snapshot and corresponding ELF files altogether and other implement
splitting as a post processing path over generated ELF or
rely on external tooling. But this is a much bigger change
than I am willing to invest time into right now.
TEST=ci
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-release-x64-try
Change-Id: I0881783237faa6534eb3c076b6cae472bbdc7243
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384742
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Clean up and optimize some RegExps, and fix uses of `.group`.
Switch to a newer language version, to be able to use newer features.
Add a little documentation about why some RegExps are as they are.
Add (tentative) warning for multitests.
Change-Id: I59f73b87ce30caaeca1c0e0aa7954af1b97abd1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382620
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Samuel Rawlins <srawlins@google.com>
Commit-Queue: Lasse Nielsen <lrn@google.com>
* Switch UInt32 from big-endian to little-endian encoding.
* Reorder object kinds, constant tags, type tags and constant pool tags.
* Reorder field and function flags.
* Cleanup yield point marker from source positions.
TEST=ci
Change-Id: I05ee940b6393a478831af11ac93fee9ec1441c4e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384040
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Specifically apply prefer-inline to:
* All generated accessors. In general we almost always seem to
select them for inlining through small-accessor heuristic,
but sometimes it does not hit. Marking with pragma ensures
that we are not dependent on specific heuristics to produce
good code.
* Various helpers involved in creating struct instances from
typed data. These we *not* hit by normal heuristics and
consequently not inlined prior to this change. Inlining them
opens possibilities to fully eliminate allocations associated
with `Struct.create(...)` if Struct does not escape.
* Array creation helpers and accessors. They were also not inlined
which meant we were producing unnecessary temporary objects
when reading from inline arrays.
TEST=/vm/dart/ffi_structs_optimizations_il_test
CoreLibraryReviewExempt: no API or behavioral changes
Cq-Include-Trybots: luci.dart.try:vm-aot-linux-product-x64-try,vm-aot-linux-release-x64-try
Change-Id: I1650023200446bd5289118777365627ca19d3e5b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/384261
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
- `build/mac/find_sdk.py` can search for iPhone and Watch SDKs and their simulators
- `tools/build.py` supports `--os=ios` and `-os=ios_simulator` now. Treating simulator as a separate os to minimize changes and avoid an additional dimension for configs.
- `vm-mac-(release|debug)-arm64-try` tryjobs make sure a shared library builds successfully for ios.
TEST=ci
Change-Id: I76358ec8fd33752260bf0b8462da22a13cd7562e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381623
Auto-Submit: Ivan Inozemtsev <iinozemtsev@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
In Dart, we use the architecture specific logic for float signs
on clamping https://github.com/dart-lang/sdk/issues/40426.
The simulator was patched to account for this
https://dart-review.googlesource.com/c/sdk/+/244320.
However, the runtime entry was not patched leading, to inconsistent
results in unoptimized and optimized code on arm and arm64
architectures (including simulators).
This CL ports the simulator fix to the runtime entry.
RISC-V, x64, and ia32 are untouched.
TEST=runtime/tests/vm/dart/regress_56589_test.dart
Bug: https://github.com/dart-lang/sdk/issues/56589
Change-Id: Ie8a7e5e707b8529b5bf8c1d7397743d056ca5a13
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-asan-linux-release-x64-try,vm-aot-linux-debug-simarm_x64-try,vm-aot-linux-debug-simriscv64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-msan-linux-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-tsan-linux-release-x64-try,vm-aot-ubsan-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-simriscv64-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-linux-release-simarm-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-tsan-linux-release-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382600
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
It used `ObjectVisitor` to peak into constant objects. The visitor
called `AddConstObject` directly while keeping raw pointers around.
This is unsafe because `AddConstObject` can cause GC for several
different reasons - which would potentially invalidate those raw
pointers.
I have added `NoSafepointScope` around `VisitPointers` but this
does not actually easily reveal the bug because allocations are
really sporadic and often require to be running in PRODUCT mode
to trigger corresponding code path. Unfortunately we don't have
any existing infrastructure to catch "allocation from unsafe
place" which works across all build modes and catches even
situations where allocation can _potentially_ happen.
Fixes https://github.com/flutter/flutter/issues/153358
TEST=manually with a reproduction provided by internal user.
R=kustermann@google.com
Change-Id: I0c9e14a137b1f8ac749443d80f6904f1e9b20ed7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382883
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Slava Egorov <vegorov@google.com>
With sound null safety front-end guarantees that all logical
expressions have a non-nullable bool type, so legacy AssertBoolean
checks are no longer needed and can be removed.
TEST=ci
Change-Id: If952da7bd0ac83c43de3e5d98845c5e5d8d29f6a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382744
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
service/get_object_rpc_test was failing because one ID in it hadn't been
updated to the new format.
service/bad_reload_test was failing because `ServiceEvent::PrintJSON`
was attempting to populate the `reloadError` property of every `Event`
of kind `IsolateReload` with an `@Error`, but it was not possible to
allocate IDs for these `@Error`s because `ReloadSources` runs for an
isolate group, not for an individual isolate. I fixed this by removing
the `reloadError` property and adding a `reloadFailureReason` property
with type `string`.
TEST=confirmed that service/bad_reload_test and
service/get_object_rpc_test complete successfully on a local build of
the linux-debug-x64 SDK, vm-linux-debug-x64 tryjob
Issue: https://github.com/dart-lang/sdk/issues/55869
Change-Id: Ibf507aa0e475a6b9bed42b055e9d19b54aa81844
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382661
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Derek Xu <derekx@google.com>
* Make dynamic module entry point fully compatible to script main
function (allow taking optional parameters and up to 2 arguments).
* Fix reading of function types within generic members.
* Add crashing tests to status files to avoid generation of
core dumps on the bots.
TEST=ci (vm-aot-dyn-linux-debug-x64)
Change-Id: Ibe8651ca13734101f2df2c8634f70ebf421dccef
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382640
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
* Use AllocateClosure instruction for closure instantiations in order
to initialize closure entry point.
* Fix null handling in operator== to be before argument type checks.
* Add _InvocationMirror._withType to dynamic interface as it is
implicitly used by noSuchMethod forwarders.
* Fix AssertAssignable for null objects.
TEST=language tests in vm-aot-dyn-linux-debug-x64 configuration
Change-Id: I7b1a037d4fde4d22ed32969e0f099b31ea4432ab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382500
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Assembler::AddressCanHoldConstantIndex, which is used by
MakeLocationSummary to determine whether to keep the index constant or
to allocate a register for it, performs the check:
const int64_t disp =
index * index_scale +
(is_external ? 0 : target::Instance::DataOffsetFor(cid) - kHeapObjectTag);
return Utils::IsInt(32, disp);
However, in Assembler::ElementAddressForIntIndex, used by EmitNativeCode to
produce an appropriate Address for the constant index, the code performs
a slightly different check:
const int64_t disp = static_cast<int64_t>(index) * index_scale +
target::Instance::DataOffsetFor(cid);
ASSERT(Utils::IsInt(32, disp));
return FieldAddress(array, static_cast<int32_t>(disp));
Thus, if a constant index produces the displacement kMaxInt32 + 1,
MakeLocationSummary will appropriately keep the index as a constant, but
the ASSERT in ElementAddressForIntIndex will trigger.
Modify ElementAddressForIntIndex in the X64 and IA32 assemblers so that
the same displacement is checked in both places.
TEST=vm/cc/IRTest_Regress_56588
Fixes: https://github.com/dart-lang/sdk/issues/56588
Change-Id: I504d6f92230540d74409a99b64b444acdb9c85f7
Cq-Include-Trybots: luci.dart.try:vm-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382801
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Also add a SignedBitField alias so that sign extended BitFields can
be declared with default positions and/or sizes.
TEST=ci
Change-Id: I46bf7f9e378e0f20e7f2e4b4e7ed3e242946d9bb
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-win-release-arm64-try,vm-aot-win-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382404
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is a reland of commit bfc1a44527
Change how the partial specializations for AtomicBitFieldContainer
are written so that Visual Studio properly chooses them instead of
the base BitField template when appropriate.
TEST=Windows ARM64 CI trybots
Original change's description:
> [vm] Improvements in the BitField API.
>
> If no position is specified, then the bitfield starts at bit 0.
>
> The default size for bool BitFields is 1 instead of the remaining
> bits in the container.
>
> If the size of the value type is smaller than the remaining bits
> in the container, then the size of the value type is used as
> the default size instead.
>
> If a signed value is used in a non-sign-extended BitField, only
> the magnitude of the value is stored, not the sign bit. This means
> the actual size of the bitfield may be one less than the requested
> size in this case.
>
> If the requested size of the bitfield is larger than the size of the
> value type, a compile-time error is thrown. (For signed types, the
> requested size is allowed to be the size of the entire value, even if
> only the magnitude bits are stored.)
>
> Rework uses of BitFields to avoid using separate constants for
> bit positions/sizes except for macro-defined bitfields (which now
> are universally bool, and so size 1).
>
> TEST=vm/cc/BitFields_Defaults
>
> Change-Id: I40711c929d2e5165ce40823772beb49e8cfdb820
> Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/381644
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>
Change-Id: I5e3a9e3e2a80d5689a23a0603f8f81fac1576cd3
Cq-Include-Trybots: luci.dart.try:vm-aot-dwarf-linux-product-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-linux-release-x64-try,vm-aot-mac-release-arm64-try,vm-aot-linux-release-simarm_x64-try,vm-win-debug-x64c-try,vm-win-release-x64-try,vm-aot-win-release-x64-try,vm-win-release-arm64-try,vm-aot-win-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/382383
Auto-Submit: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Vyacheslav Egorov