Commit Graph

18413 Commits

Author SHA1 Message Date
asiva 9f6e64a06f [VM/runtime] Fix a warning
TEST=ci

Change-Id: I3bdfa6b1069f4f6f0f889eedf07c4c6527fe13f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189943
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-03-09 17:21:50 +00:00
Alexander Aprelev bcf06bad4a [vm/concurrency] Fix Function bit field setters.
This is follow-up to 182d0dd156.

TEST=flaky vm/dart_2/isolates/reload_many_isolates_live_and_die_test

Fixes https://github.com/dart-lang/sdk/issues/45169.

Change-Id: Ie59d26b7fc0594940742b37fe9ed0127e5871245
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190040
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2021-03-09 16:22:00 +00:00
Alexander Aprelev c0124aed87 [vm/concurrency] Fix tsan warning data race around Field guarded_cid_.
Use relaxed atomic write to match relaxed atomic read for guarded_cid_ field.

Fixes https://github.com/dart-lang/sdk/issues/45186

TEST=flaky tsan failures on  vm/dart_2/isolates/reload_many_isolates_test

Change-Id: Ib538a04187cff77ef278db4d62e551d5db9fd023
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189280
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-09 16:20:50 +00:00
Martin Kustermann 546c2d8f90 [vm] Skip --deoptimize-on-runtime-call-every in AOT
TEST=Fixes broken dartkp bots.

Change-Id: Ifb6569ffc53b89afb79bcbe2503096803f4c6501
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190020
Auto-Submit: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-03-09 14:23:40 +00:00
Martin Kustermann cd104aef6f [vm] Fix deoptimization environment of AssertAssignable
The deoptimization environment of AssertAssignable should not have it's
inputs in it, since those get poped before the call to the
TypeTestingStub.

A lazy deopt should therefore continue in unoptimized code after the TTS
call, without the inputs.

This was discovered by running many lightweight isolates concurrently on
the JIT.

To simplify a regression test we'll add new VM flags that can trigger
deoptimization on any runtime call (optionally filtered by name of
runtime call).

We can use this mechanism to possibly discover more bugs of the same
kind.

Closes https://github.com/dart-lang/sdk/issues/45207
Issue https://github.com/dart-lang/sdk/issues/36097

TEST=runtime/tests/vm/dart{,_2}/regress_45207_test

Change-Id: Ib713fa8be5914eaf4c7395d2e1d8dd88714f1ce2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189202
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-03-09 11:27:00 +00:00
Daco Harkes ffa5d16ae7 [vm/ffi] Support multi-dimensional inline arrays
This CL only changes dart:ffi API, CFE, and analyzer. No VM changes
were needed because the dimensions of inline arrays can be flattened
before passing them to the VM. The multi-dimensionality does not
impact the ABI.

Closes: https://github.com/dart-lang/sdk/issues/45023

TEST=pkg/analyzer/test/src/diagnostics/size_annotation_dimensions_test.dart
TEST=pkg/front_end/testcases/nnbd/ffi_struct_inline_array_multi_dimensional.dart
TEST=tests/ffi/function_structs_by_value_generated_test.dart
TEST=tests/ffi/inline_array_multi_dimensional_test.dart

Change-Id: Ica2c01fccbea7e513879365b34086d8968b54c5b
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188286
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
2021-03-09 10:50:17 +00:00
Johnni Winther 2d63f26ef5 Reland "[cfe] Encode field references as @getters and @setters"
This removes the @fields and @=fields canonical name
encodings that allowed for encoding of conflicting members
and didn't support field<->getter/setter conversion
between dills or between outline and full dill.

TEST=existing tests+add aot expectation tests

Change-Id: I119b0c95f90e456356146cdc2d9241de4c1b4fff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186680
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-03-09 05:47:16 +00:00
asiva 787d2659c9 [VM/runtime] - Minor cleanups.
TEST=ci

Change-Id: I2ff123b67e89ca79cba39e07fc2de191af9fc713
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189740
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-03-08 20:51:53 +00:00
Alexander Aprelev e4ab127fbc [vm/concurrency] Make Function::kind_tag_ concurrency-safe.
Fixes https://github.com/dart-lang/sdk/issues/45182.

TEST=flaky vm/dart_2/isolates/reload_many_isolates_test on tsan bot
Change-Id: Ie874e65643d1a1dc94a94446b3141a4fabb15adb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189261
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-08 19:29:21 +00:00
Vyacheslav Egorov ae37ff098a [vm] Compact serialization for canonical sets
This CL changes how canonical sets for some specific types are written
into the root snapshot: instead of writing canonical set as a separate
object we reorder objects within a canonical cluster in such a way that
the order matches order of elements in the backing store of a canonical
set and then we write canonical set layout out using differential
encoding (essentially writing gaps between elements instead of writing
absolute indices).

This significantly reduces the overhead of having canonical sets in the
snapshot while maintaining fast deserialisation: for example on build
microbenchmark this brings regression in the snapshot size from 4% to
.3%.

On sizeopt benchmarks:

  flutter_gallery_app_so_gzip_size   -1.5%
  flutter_gallery_app_so_size        -4.7%
  flutter_gallery_total_heap_size   -16.2%


TEST=ci

Change-Id: I2be7fd073668e9b52098e2acda9f11d128cfda95
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185381
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-03-08 14:21:01 +00:00
Johnni Winther 01b0947a6d [kernel] Update kernel encodings
* Add InstanceGetterInvocation for getter/field invocation in web
  backends
* Add localFunction getter to LocalFunctionInvocation
* Remove isNot from EqualsCall and EqualsNull - the encoding didn't
  carry its weight.
* Remove uses of Name.name
* Remove BottomType code from VM

TEST=existing

Change-Id: I99d05d35b9ef193d092cc151c99ad472dbd60834
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188725
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2021-03-08 11:29:51 +00:00
Martin Kustermann 2d6073bbf8 [vm] Remove assertion that doesnt always hold
Threads waiting for reload can mark themselves as checked in while
participating in multiple reloads.

Closes https://github.com/dart-lang/sdk/issues/45211

TEST=Fixes flaky assertion hits.

Change-Id: Idedf821ecc49ace7fbbebf20be7a9c02e3d71a4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189204
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-08 10:52:41 +00:00
Alexander Aprelev 182d0dd156 [vm/concurrency] Ensure updates to Isolate::isolate_flags_ are atomic and thread-safe.
Fixes https://github.com/dart-lang/sdk/issues/45169.

TEST=flaky vm/dart_2/isolates/reload_many_isolates_live_and_die_test

Change-Id: I3a378aef76a91de49611f7debe9cbc2c9335e872
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189160
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-05 15:24:49 +00:00
Ryan Macnak dce0b514d4 [vm] Defer object pools and object pool entries.
This defers literals, to effectively split binary size for libraries that are more data than code, such as localizations.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I506722037cc0247b90756959e6a6f12bb5021b5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183781
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-04 23:19:30 +00:00
Ryan Macnak 9d803a5263 [vm] Fail gracefully when unable to reserve address space for the compresssed heap.
TEST=ulimit -v 900000 && dart hello.dart
Bug: https://github.com/dart-lang/sdk/issues/45132
Change-Id: Ifcb0ad26f59ef4269b3e60dccc844d34406f4fcd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189162
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-04 22:32:20 +00:00
Ryan Macnak 8c1cb52c5a [vm, arm64] Fix encoding of immediates in some 32-bit operations.
TEST=ci
Change-Id: I53333355f04636ed49e9a05aeaf1378ea2dd35a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188601
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-03 23:54:45 +00:00
Liam Appelbe b6188939ec [vm] Align compressed pointer allocations on Fuchsia.
TEST=Local testing using Fuchsia emulator
Change-Id: I6946f07b8abab534986d7c23284e6e250c13b822
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188080
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-03-03 21:51:24 +00:00
Martin Kustermann ab5d639e36 [vm/concurrency] Lookup & set error class during bootstrapping instead of lazily
Closes https://github.com/dart-lang/sdk/issues/45176

TEST=Fixes flaky tsan error of vm/dart_2/isolates/reload_no_active_stack_test

Change-Id: I7839ec4aa8fe66acbe709e379afbc65ca2e973a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188721
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-03 18:18:14 +00:00
Martin Kustermann 187117ddf4 [vm/concurrency] Allow concurrent calculation of Class::num_type_arguments_
The lazily calculated value can be calculated concurrently and installed
in arbitratry order as long as all writers write the same value (which
they should).

There's no need to guard it with a lock.

Closes https://github.com/dart-lang/sdk/issues/45175

TEST=Refactoring of existing code, fixes flaky tsan reports.

Change-Id: Iaa45cf3b70300958acc04953e37599836436bc59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188720
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-03 17:14:48 +00:00
Johnni Winther dd1c3a3233 [vm] Make async transform resilient in context of invalid type and never type of iterables
The CFE can invoke transformations despite having compile-time errors.
The async transform was crashing the compiler if it hits for-in
iterables that have an invalid type or never type.

In case of an invalid type, the program has a compile-time error and
won't be able to run propertly. So we'll replace the entire for-in with
an invalid expression statement if the iterable is an invalid
expression.

In case of a never type, the program should compile fine, which this CL
also fixes.

This makes the newly added tests no longer result in DartkCrash but
rather in a compile-time error (invalid expression) or runtime-error
(never type).

Closes https://github.com/dart-lang/sdk/issues/45014

Change-Id: Ic50f68400b67b57dd4a2c0a125b08f0f3e0d8dd6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188463
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2021-03-03 11:58:08 +00:00
Martin Kustermann 700ba716cd [vm/concurrency] Avoid O(n) traversal of isolates to find pending deopt
To avoid a O(n) traversal of isolates in order to find pending deopts,
we'll move the pending deopt state to the Thread. It's really a property
of a thread: when the thread's stack gets unwound - via return or
exceptional flow - we remove pending deopts.

This CL also encapsulates the pending deopt logic into a [PendingDeopts]
class - thereby avoiding logic around manipulating this array being
spread around. It simplifies the already very big [Isolate] class.

TEST=Refactoring of existing code.

Issue https://github.com/dart-lang/sdk/issues/36097
Closes https://github.com/dart-lang/sdk/issues/45168

Change-Id: I5a9b18c577f21a0f2a88332b1bc26641ec39d2af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188521
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-03 09:31:53 +00:00
Martin Kustermann 197fe0c193 [vm] Make reload tests NOP in product mode
TEST=Fixes failures on product mode.

Change-Id: I2cb00bbbcd18a7835ead301946457aa773edbf99
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188700
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-03 09:24:48 +00:00
Clement Skau e1df4361b1 [ffi] Makes FFI dyn.lib. fail more verbose.
This helps you figure out which load failed when running
a bigger test suite, where multiple libraries loaded.

TEST=CQ

Change-Id: I00d89d7732cd38a06b5b6256c6086cfa232bb76f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188464
Auto-Submit: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-03-03 08:10:18 +00:00
Alexander Aprelev 5f7e027c27 [gardening] Ensure path is converted to uri in reload tests.
Follow-up to 91c628d63a

Change-Id: I2504d0b69c060b5d77002b3b0c58ec0ff6e0a4af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188640
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2021-03-03 00:03:37 +00:00
Ben Konyi 86c0d1199c [ VM / package:vm_service ] Fix issue where events were only sent over Stdout and Stderr streams when Debug stream was active
TEST=pkg/vm_service/test/capture_stdio_test.dart
Change-Id: I9029f1c42455b7d757aa5d424e05b5d45a048c4a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185483
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-03-02 21:47:27 +00:00
Martin Kustermann 91c628d63a [vm/concurrency] Final support for hot-reload of multi-isolate groups
This is the initial implementation of hot reload with multi-isolate
groups.

Implementation:

As before, when a service API call triggers a reload it will be routed
as an OOB message to a specific isolate (**). As opposed to before, that
isolate has now to coordinate with all other isolates, ensuring that it
"owns" the reload and all other isolates are waiting in a state that
allows reload.

This is implemented as a [ReloadOperationScope] which first participates
in other reloads (if there are any) and then owns the reload. It will
send a new kind of service message to all other registered isolates. All
of them have to check in before reload can proceed. If a new isolate
is about to join the group, it will participate when registering the
isolate. If an old isolate wants to die, it will participate when
unregistering the isolate.

This means that in addition to the existing StackOverFlow checks that
can process OOB messages and therefore reload, we'll have isolate
registration and unregistration as well as a new
Isolate::kCheckForReload OOB message handler where an isolate can
participate in a reload.

We consider the isolate group to be reloadable if the main isolate has
loaded the program and set the root library. Helper isolates don't need
to load any more kernel code and only initialize core libraries, so it's
fine to reload them during this time.

(**) The reason we continue to send reload service API calls to any
isolate in an isolate group is that re-loading might involve calling out
to the embedder's tag handler. Doing so currently requires an active
isolate.

If we allowed a subset of dart_api.h (the subset needed by the tag
handler) to be used only with an active IsolateGroup instead of an
active Isolate we could remove this requirement.

Edge cases:

There's various edge cases to consider: The main edge case is, we currently
maintain an upper limit to the number of isolates executing in parallel
(to ensure each can have big enough chunk of new space, i.e. TLAB).

If there are more isolates with active work they are waiting until one
of the exiting ones "yields". To ensure progress, if any such actively
running isolate gets a request to participate in a reload, it will mark
its own thread as "blocked" and therefore "yields", so another isolate
can make progress until all isolates are participating and the reload
can start.

Marking an isolate as "blocked" happens by exiting that isolate. It will
free up it's TLAB, decrease active mutator count and (if running on VM's
thread pool) also temporarily increase the thread pool size.

The side-effect of this is that it will use one pthread per isolate
during reload. In the future we can extend this first implementation, by
specially handling isolates that don't have a message handler running.
Doing so would require careful consideration to avoid races.

Testing:

In order to test this we use a small helper framework for reload tests.
The helper framework will, similar to real world reload e.g. in flutter,
will spawn a subprocess. It will use the service API to trigger reloads
in this subproces.

To synchronize between the reload driver and the application being
reloaded it allows watching for events to be printed to stdout/stderr.

The reload test itself can be written - similar to multitests - with
annotations such as `// @include-in-relload-0` in them. The testing
framework will then generate multiple application versions that all get
compiled to kernel.

For simplicity we generate the kernel using the standalone VM with
`--snapshot-kind=kernel` and avoid using the incremental compiler.

There are 4 different tests exercising different aspects of
multi-isolate reload:

  vm/dart_2/isolates/reload_active_stack_test:
    Performs a reload while a fixed number of isolates have an active
    stack, thereby ensuring e.g. that all frames of all isolate mutator
    stacks get deoptimized, ...

  vm/dart_2/isolates/reload_no_active_stack_test:
    Similar to the test above, but instead of having an active stack the
    isolates can yield to the event loop, possibly be even descheduled

  vm/dart_2/isolates/reload_many_isolates_test:
    Similar to the test above, but this test uses many more isolates.

  vm/dart_2/isolates/reload_many_isolates_live_and_die_test:
    Performs a reload where isolates get spawned and die all the time.
    There are always P isolates alive at any given point in time, each
    of them spawns children when their parent has died.
    Performing a reload catches isolates as various stages of their
    lifecycle and can therefore cover a lot of corner cases.

TEST=vm/dart_2/isolates/reload_*_test.dart

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I97039b4084de040b7f2e22f5832a40d57ba398d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 18:57:02 +00:00
Alexander Aprelev c22205c14e [vm/concurrency] Ensure write program_lock is held while updating class state via set_state.
Fix tests and AllocateObject.

Hopefully addresses flaky https://github.com/dart-lang/sdk/issues/44940

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=ci and standalone_2/out_of_memory_recovery_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try

Change-Id: Ife775f7a0bcdd343b887ba1bf912e07b2a8430c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188362
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-02 16:28:12 +00:00
Martin Kustermann 5e51ee2e01 [vm/concurrency] Remove NoReloadScope where its not needed
The kernel loader works without an active isolate. It no longer causes
execution of Dart code. Constants are evaluated in CFE and are simply
"deserialized" by the kernel loader.

Since kernel loader doesn't execute Dart code and also doesn't process
OOB messages at any point, there's no reason to have a [NoReloadScope]
in it.

Formatting language errors seems to be done purely in the VM and should
therefore also require no [NoReloadScope]

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I80b2888b60a4d7d179988b8606cd7ca0893a2fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187400
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 16:17:32 +00:00
Martin Kustermann e2be5322e6 [vm] Use custom realloc() on Windows
Seemingly Windows doesn't always implement realloc() by freeing space if
the new buffer is significantly smaller than the old one.

This CL does so manually in our code base.

Issue b/178993708
Issue https://github.com/dart-lang/sdk/issues/44846

TEST=standalone{,_2}/io/large_file_read_small_file_test

Change-Id: I370346a693f8e8e879e079d91e9de5c756949a11
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183694
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 16:02:52 +00:00
Martin Kustermann 3fac077ebc [vm/concurrency] Use relaxed atomics when accessing usage counters - for TSAN
For now our decision is to not synchronize access to usage counters and
instead let there be possible non-observable writes (if two concurrent writes
happen).

Though TSAN will report concurrent access in C++ to the usage counters
as a race, so we'll use a relaxed atomic.

(We do have non-synchronized access to the usage counters already:
If the mutator kicks of a background compilation, the BG compiler
might reset the counters while mutator is running.)

Closes https://github.com/dart-lang/sdk/issues/45134

TEST=Fixes flaky tsan errors.

Change-Id: I7919747536d43094e7201e03f464e4ac7b1a4740
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187923
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 15:53:22 +00:00
Martin Kustermann ed78971d90 [vm/concurrency] Simplify conditions for hot reload and test mode implementation
This is part of making hot-reload work with isolate groups.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: Id32e7d8f95ba0b1fb5bd11179dcd1805d42aba89
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187083
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 15:21:22 +00:00
Martin Kustermann d7a9015268 [vm/compiler] Remove BG compiler bailouts that afaik should not be possible
Many years ago those bailouts were seemingly added due to deferred
loading in [0]. Though it seems like those bailouts might've become
obsolete.

All places that clear the IC array on the function should run inside
a [NoBackgroundCompiler] scope, therefore ensuring no optimizing
compilations are done at that time. Those places also clear code,
therefore forcing re-compilation of unoptimized code (which will
re-create the ICData).

[0] https://codereview.chromium.org/1672873003/#ps60001

TEST=Removing dead code.

Change-Id: Iec4f22be500f6e874fa5286b51396bd6d978dd6c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187902
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-03-02 15:16:32 +00:00
Martin Kustermann 7fc9a1a14c [vm/compiler] Remove non-background optizable bit
The background compiler no longer has bailouts due to something that
cannot be done on BG compiler and has to be done on mutator (was
originally introduced in [0])

All optimizing compilations can happen on the BG thread. Class
finalization and other things no longer cause a bailout on BG
compiler.

If e.g. invalidated field guards cause a compilation to be discarded, it
can be retried on BG compiler.

[0] https://dart-review.googlesource.com/c/sdk/+/54886

Closes https://github.com/dart-lang/sdk/issues/45136

TEST=Fixes flaky hits of an assertion.

Change-Id: I50313dbedf96b8dec205acdb8c1ed5731d00433b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187901
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 14:20:02 +00:00
Martin Kustermann 29268ba7d2 [vm/concurrency] Move no-reload-scope counter to Thread
Whether a given isolate allows reloading is really a property of the
thread not the isolate (where it was before) or isolate group (where
it was moved to), so this CL moves it to Thread.

As the thread's stack is going down and up, NoReloadScope's might be
entered or exited, just as with our many other scopes, such as
NoOOBMessageScope/...

Longer term we might need to find a better mechanism to deal with
different OOB messages. For example: Reload as well as OOB messages
are disabled during static field initialization. Though that means *all*
OOB messages cannot reach the isolate during initialization of a static
field (which can do arbitrary amount of work).

This is part of making hot-reload work with isolate groups.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I9e80c6d4a184c54c0373fef13af3264cc3f27ece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187002
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 14:11:42 +00:00
Martin Kustermann f3740ced59 [vm/concurrency] Add SafepointMonitorUnlockScope to allow scoped unlocking of a monitor
In order to simplify code that needs to temporarily give up a monitor
lock, this CL adds a scoped object that releases the monitor on
construction and re-acquires it on destruction.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I004a04e54dcdaea009bfbef25d2a946a307e41c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187001
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 13:11:42 +00:00
Tess Strickland 3382bbc510 [vm] Print out canonical clusters in --print-snapshot-sizes-verbose.
TEST=Manual use of the flag.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I4c506848f97f6b9a3f1455f90872093497c853a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188287
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-02 08:37:01 +00:00
Ryan Macnak 9451963493 [vm, compiler] Remove intrinsics from internal double-dispatching int methods.
If the intrinsic on int.op fails, then the intrinsic on _opFromInteger will also fail, since both handle the same range of values.

Add intrinsic on int.% as previously only _moduloFromInteger was intrinsified. All the other ops already intrinsified the public method.

Remove _bitAndFromSmi; this was once useful for handling BigInt & Smi, but this case no longer exists.

TEST=ci
Change-Id: Ie33b75126e56110db3caabb9bcb6e9198a3158bf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187160
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-01 19:29:00 +00:00
Tess Strickland 960c944940 [vm/compiler] Get parameter names/counts from the closure signature.
Doing so removes some dependencies on the closure function itself,
making it easier to shake the function when possible in the future.

TEST=Existing tests that check dynamic closure invocations.

Bug: https://github.com/dart-lang/sdk/issues/41052
Bug: https://github.com/dart-lang/sdk/issues/44005

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I88c4b37efff9a9894633db7f380ecc39da717e6e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-03-01 16:25:24 +00:00
Vyacheslav Egorov 6e7c0ae227 [vm] Comment for RegressFlutter76919
Add a comment which describes what this test is testing.

TEST=not needed
TBR=kustermann@google.com

Change-Id: I88238595d1c11609130bfdde3219f58c39db57f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188284
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-01 14:35:09 +00:00
Vyacheslav Egorov afa22dd1ab [vm/aot] Disable assert strengthening optimization in AOT
This optimization does not make sense in AOT and can actually cause
AOT compiler to segfault because it expects environment to be
attached to AssertAssignable instructions which is not always
the case on AOT compiler.

Fixes https://github.com/flutter/flutter/issues/76919

TEST=vm/cc/TypePropagator_RegressFlutter76919,vm/dart{,_2}/regress_flutter76919

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I8f4314d7dac276833a9050bba835616b670a88d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188281
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-01 13:13:19 +00:00
Vyacheslav Egorov 1f828c7cb2 [vm/compiler] Fix IntConverter canonicalization
When going from Int64 through another representation back into Int64
you can discard a conversion if and only if it does not lose some
bits on the way. Previously we were checking for truncation flag, but
this is incorrect because it does not actually tell us anything about
about whether or not bits are going to be lost.

Instead, we should be checking the range of the source value.

In addition to fixing potential correctness issues, this CL actually
improves quality of code generated for Uint8 loads because it removes
a redundant chain of Int64->Uint32->Int64 conversions that follows
the load (which compiled 2 two redundant moves).

Issue https://github.com/dart-lang/sdk/issues/40308

TEST=vm/cc/IL_IntConverterCanonicalization

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Change-Id: Ifef0943a50d414bde41b2194e30b58ad3a8c106c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187921
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-01 09:55:09 +00:00
Alexander Markov 26e7106937 [vm/aot] Partially remove Code objects from AOT snapshots
Code objects which do not carry useful information are
marked as 'discarded' during precompilation. Snapshot
serializer only writes instructions, compressed stack
maps and state bits for discarded Code objects.

This is a preliminary step before deserializer would be able
to omit creating instances for discarded Code objects at
runtime.

On a large app, AOT-compiled in PRODUCT mode with
--dwarf-stack-traces on arm64:
Discarded: 77% of all Code objects in snapshot.
Uncompressed AOT snapshot size -1.9%.

AOT snapshot size of Flutter gallery in release-sizeopt mode:
arm64 uncompressed -2.1% (gzip -0.6%, brotli -0.7%)
arm32 uncompressed -1.9% (gzip -0.6%, brotli -0.7%)

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try
Change-Id: I159cfebbdac76cc60060fedb1d22ed86672a68a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187600
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-27 00:41:37 +00:00
Daco Harkes 0295dfc74e [vm] Support redirecting external const factories
This CL introduces the first combination of `external const factory`
with `@patch const factory (.*) = (.*)`. This specific combination was
never used in the core libraries, and did not work.

Half of the fix is a missing flag from the CFE:
https://dart-review.googlesource.com/c/sdk/+/187440

The other half of the fix is matching on the
kRedirectingFactoryConstructor tag, and allowing constructor names
in procedure targets (even though we never compile those expressions
because they are invalid).

Bug: https://github.com/dart-lang/sdk/issues/45101

TEST=Using redirecting const factory in patch file in core lib.
TEST=tests/ffi(_2)/*

Change-Id: I5524bde928290bf32aaea9170eda1f2d03127fa6
Cq-Include-Trybots: luci.dart.try:benchmark-linux-try,front-end-linux-release-x64-try,vm-kernel-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187003
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-02-26 11:15:26 +00:00
Vyacheslav Egorov 7ba12664ea [vm/compiler] Fix LoadStaticField::ComputeCanDeoptimize
It should only return true in JIT mode similar to other calls.

Discovered when looking at allocation sinking in FFI version of
reverse-complement microbenchmark.

TEST=ci

Issue https://github.com/dart-lang/sdk/issues/40308

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I79f432ffb88a467336fcb412a2d9fcdedc390cd8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187404
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-02-26 10:24:27 +00:00
Clement Skau f2dfe3a8be [vm/debugger] Fixes async CollectAwaiterReturn()
The async, off-stack part of DebuggerStackTrace::CollectAwaiterReturn's
stack unwinding incorrectly looked on the stack instead of following
the Closure's awaiter chain.
This caused the async stack traces to be truncated and missing any
async frames, in turn causing ShouldPauseOnException to incorrectly
conclude no handler frame existing.
This would affect any case where the exception handling was located
around any except for the inner most awaiter.

TEST=Added regression test to pause_on_unhandled_exceptions_catcherror_test

Closes: https://github.com/dart-lang/sdk/issues/37953
Bug: https://github.com/dart-lang/sdk/issues/37953
Change-Id: I34a2b5ce3c7532032b2ef3dfbc2af7294d13a4c7
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187405
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-02-26 06:46:36 +00:00
Ryan Macnak 5a385890b9 [test] Apply simarm64 status file entries to simarm64c.
Remove references to the removed simarmv6 architecture.

TEST=ci
Change-Id: I10e9fc8291c3e5b6b9a4bd71ca05de95e53f22c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187500
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-25 21:18:02 +00:00
Ryan Macnak 20bc0a0ce1 [vm] Fix annotated pool references when disassembling in JIT mode.
FLAG_use_bare_instructions is vacuously true in JIT mode, so use GetObjectPool which additionally accounts for JIT versus AOT.

TEST=examine output of --disassemble in JIT, AOT compile and AOT runtime
Change-Id: Idae17b779f16a6b823cf702bf868b232ca0e03ab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187521
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-25 21:09:57 +00:00
Liam Appelbe 807874d51a [vm] Align compressed pointer allocations on windows.
Change-Id: I95ad0a2cf679a5918d42b12bb95f1503bb219fca
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184846
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-25 20:48:07 +00:00
Martin Kustermann c78f12c550 [vm/concurrency] Allow caller of Thread::{Exit,Enter}Isolate to specify nesting
It's mainly hoisting an expression out of
IsolateGroup::{Increase,Decrease}MutatorCount to callers of it.

Right now we consider a Thread::{Exit,Enter} to be nested if there are
existing Dart frames on the stack. A future CL will have a case where
there's a nested exit&re-enter without Dart frames on the stack.

Trying to do this automatically by checking whether there's an
active api scope / zone on the thread fails due to some
asymmetry in our codebase.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: Id70eda03efb2099487c5f0c8affd25ddff926553
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186287
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-25 18:44:16 +00:00
asiva 02c1e2e56b Remove obsolete comment,
Full snapshots are generated from the clustered snapshot modules
and code in this file is used for only isolate messaging and the comment
about full snapshot does not apply here.

TEST=no tests, just a comment delete.

Change-Id: Ib2453514639aae1c99d3eab6bb120f0ec5ac9f4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186747
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-02-24 23:05:14 +00:00