Commit Graph

18204 Commits

Author SHA1 Message Date
Vyacheslav Egorov ae37ff098a [vm] Compact serialization for canonical sets
This CL changes how canonical sets for some specific types are written
into the root snapshot: instead of writing canonical set as a separate
object we reorder objects within a canonical cluster in such a way that
the order matches order of elements in the backing store of a canonical
set and then we write canonical set layout out using differential
encoding (essentially writing gaps between elements instead of writing
absolute indices).

This significantly reduces the overhead of having canonical sets in the
snapshot while maintaining fast deserialisation: for example on build
microbenchmark this brings regression in the snapshot size from 4% to
.3%.

On sizeopt benchmarks:

  flutter_gallery_app_so_gzip_size   -1.5%
  flutter_gallery_app_so_size        -4.7%
  flutter_gallery_total_heap_size   -16.2%


TEST=ci

Change-Id: I2be7fd073668e9b52098e2acda9f11d128cfda95
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185381
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-03-08 14:21:01 +00:00
Johnni Winther 01b0947a6d [kernel] Update kernel encodings
* Add InstanceGetterInvocation for getter/field invocation in web
  backends
* Add localFunction getter to LocalFunctionInvocation
* Remove isNot from EqualsCall and EqualsNull - the encoding didn't
  carry its weight.
* Remove uses of Name.name
* Remove BottomType code from VM

TEST=existing

Change-Id: I99d05d35b9ef193d092cc151c99ad472dbd60834
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188725
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2021-03-08 11:29:51 +00:00
Martin Kustermann 2d6073bbf8 [vm] Remove assertion that doesnt always hold
Threads waiting for reload can mark themselves as checked in while
participating in multiple reloads.

Closes https://github.com/dart-lang/sdk/issues/45211

TEST=Fixes flaky assertion hits.

Change-Id: Idedf821ecc49ace7fbbebf20be7a9c02e3d71a4d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189204
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-08 10:52:41 +00:00
Alexander Aprelev 182d0dd156 [vm/concurrency] Ensure updates to Isolate::isolate_flags_ are atomic and thread-safe.
Fixes https://github.com/dart-lang/sdk/issues/45169.

TEST=flaky vm/dart_2/isolates/reload_many_isolates_live_and_die_test

Change-Id: I3a378aef76a91de49611f7debe9cbc2c9335e872
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189160
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-05 15:24:49 +00:00
Ryan Macnak dce0b514d4 [vm] Defer object pools and object pool entries.
This defers literals, to effectively split binary size for libraries that are more data than code, such as localizations.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I506722037cc0247b90756959e6a6f12bb5021b5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183781
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-04 23:19:30 +00:00
Ryan Macnak 9d803a5263 [vm] Fail gracefully when unable to reserve address space for the compresssed heap.
TEST=ulimit -v 900000 && dart hello.dart
Bug: https://github.com/dart-lang/sdk/issues/45132
Change-Id: Ifcb0ad26f59ef4269b3e60dccc844d34406f4fcd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189162
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-04 22:32:20 +00:00
Ryan Macnak 8c1cb52c5a [vm, arm64] Fix encoding of immediates in some 32-bit operations.
TEST=ci
Change-Id: I53333355f04636ed49e9a05aeaf1378ea2dd35a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188601
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-03 23:54:45 +00:00
Liam Appelbe b6188939ec [vm] Align compressed pointer allocations on Fuchsia.
TEST=Local testing using Fuchsia emulator
Change-Id: I6946f07b8abab534986d7c23284e6e250c13b822
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188080
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-03-03 21:51:24 +00:00
Martin Kustermann ab5d639e36 [vm/concurrency] Lookup & set error class during bootstrapping instead of lazily
Closes https://github.com/dart-lang/sdk/issues/45176

TEST=Fixes flaky tsan error of vm/dart_2/isolates/reload_no_active_stack_test

Change-Id: I7839ec4aa8fe66acbe709e379afbc65ca2e973a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188721
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-03 18:18:14 +00:00
Martin Kustermann 187117ddf4 [vm/concurrency] Allow concurrent calculation of Class::num_type_arguments_
The lazily calculated value can be calculated concurrently and installed
in arbitratry order as long as all writers write the same value (which
they should).

There's no need to guard it with a lock.

Closes https://github.com/dart-lang/sdk/issues/45175

TEST=Refactoring of existing code, fixes flaky tsan reports.

Change-Id: Iaa45cf3b70300958acc04953e37599836436bc59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188720
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-03 17:14:48 +00:00
Johnni Winther dd1c3a3233 [vm] Make async transform resilient in context of invalid type and never type of iterables
The CFE can invoke transformations despite having compile-time errors.
The async transform was crashing the compiler if it hits for-in
iterables that have an invalid type or never type.

In case of an invalid type, the program has a compile-time error and
won't be able to run propertly. So we'll replace the entire for-in with
an invalid expression statement if the iterable is an invalid
expression.

In case of a never type, the program should compile fine, which this CL
also fixes.

This makes the newly added tests no longer result in DartkCrash but
rather in a compile-time error (invalid expression) or runtime-error
(never type).

Closes https://github.com/dart-lang/sdk/issues/45014

Change-Id: Ic50f68400b67b57dd4a2c0a125b08f0f3e0d8dd6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188463
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2021-03-03 11:58:08 +00:00
Martin Kustermann 700ba716cd [vm/concurrency] Avoid O(n) traversal of isolates to find pending deopt
To avoid a O(n) traversal of isolates in order to find pending deopts,
we'll move the pending deopt state to the Thread. It's really a property
of a thread: when the thread's stack gets unwound - via return or
exceptional flow - we remove pending deopts.

This CL also encapsulates the pending deopt logic into a [PendingDeopts]
class - thereby avoiding logic around manipulating this array being
spread around. It simplifies the already very big [Isolate] class.

TEST=Refactoring of existing code.

Issue https://github.com/dart-lang/sdk/issues/36097
Closes https://github.com/dart-lang/sdk/issues/45168

Change-Id: I5a9b18c577f21a0f2a88332b1bc26641ec39d2af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188521
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-03 09:31:53 +00:00
Martin Kustermann 197fe0c193 [vm] Make reload tests NOP in product mode
TEST=Fixes failures on product mode.

Change-Id: I2cb00bbbcd18a7835ead301946457aa773edbf99
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188700
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-03 09:24:48 +00:00
Clement Skau e1df4361b1 [ffi] Makes FFI dyn.lib. fail more verbose.
This helps you figure out which load failed when running
a bigger test suite, where multiple libraries loaded.

TEST=CQ

Change-Id: I00d89d7732cd38a06b5b6256c6086cfa232bb76f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188464
Auto-Submit: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-03-03 08:10:18 +00:00
Alexander Aprelev 5f7e027c27 [gardening] Ensure path is converted to uri in reload tests.
Follow-up to 91c628d63a

Change-Id: I2504d0b69c060b5d77002b3b0c58ec0ff6e0a4af
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188640
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2021-03-03 00:03:37 +00:00
Ben Konyi 86c0d1199c [ VM / package:vm_service ] Fix issue where events were only sent over Stdout and Stderr streams when Debug stream was active
TEST=pkg/vm_service/test/capture_stdio_test.dart
Change-Id: I9029f1c42455b7d757aa5d424e05b5d45a048c4a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185483
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-03-02 21:47:27 +00:00
Martin Kustermann 91c628d63a [vm/concurrency] Final support for hot-reload of multi-isolate groups
This is the initial implementation of hot reload with multi-isolate
groups.

Implementation:

As before, when a service API call triggers a reload it will be routed
as an OOB message to a specific isolate (**). As opposed to before, that
isolate has now to coordinate with all other isolates, ensuring that it
"owns" the reload and all other isolates are waiting in a state that
allows reload.

This is implemented as a [ReloadOperationScope] which first participates
in other reloads (if there are any) and then owns the reload. It will
send a new kind of service message to all other registered isolates. All
of them have to check in before reload can proceed. If a new isolate
is about to join the group, it will participate when registering the
isolate. If an old isolate wants to die, it will participate when
unregistering the isolate.

This means that in addition to the existing StackOverFlow checks that
can process OOB messages and therefore reload, we'll have isolate
registration and unregistration as well as a new
Isolate::kCheckForReload OOB message handler where an isolate can
participate in a reload.

We consider the isolate group to be reloadable if the main isolate has
loaded the program and set the root library. Helper isolates don't need
to load any more kernel code and only initialize core libraries, so it's
fine to reload them during this time.

(**) The reason we continue to send reload service API calls to any
isolate in an isolate group is that re-loading might involve calling out
to the embedder's tag handler. Doing so currently requires an active
isolate.

If we allowed a subset of dart_api.h (the subset needed by the tag
handler) to be used only with an active IsolateGroup instead of an
active Isolate we could remove this requirement.

Edge cases:

There's various edge cases to consider: The main edge case is, we currently
maintain an upper limit to the number of isolates executing in parallel
(to ensure each can have big enough chunk of new space, i.e. TLAB).

If there are more isolates with active work they are waiting until one
of the exiting ones "yields". To ensure progress, if any such actively
running isolate gets a request to participate in a reload, it will mark
its own thread as "blocked" and therefore "yields", so another isolate
can make progress until all isolates are participating and the reload
can start.

Marking an isolate as "blocked" happens by exiting that isolate. It will
free up it's TLAB, decrease active mutator count and (if running on VM's
thread pool) also temporarily increase the thread pool size.

The side-effect of this is that it will use one pthread per isolate
during reload. In the future we can extend this first implementation, by
specially handling isolates that don't have a message handler running.
Doing so would require careful consideration to avoid races.

Testing:

In order to test this we use a small helper framework for reload tests.
The helper framework will, similar to real world reload e.g. in flutter,
will spawn a subprocess. It will use the service API to trigger reloads
in this subproces.

To synchronize between the reload driver and the application being
reloaded it allows watching for events to be printed to stdout/stderr.

The reload test itself can be written - similar to multitests - with
annotations such as `// @include-in-relload-0` in them. The testing
framework will then generate multiple application versions that all get
compiled to kernel.

For simplicity we generate the kernel using the standalone VM with
`--snapshot-kind=kernel` and avoid using the incremental compiler.

There are 4 different tests exercising different aspects of
multi-isolate reload:

  vm/dart_2/isolates/reload_active_stack_test:
    Performs a reload while a fixed number of isolates have an active
    stack, thereby ensuring e.g. that all frames of all isolate mutator
    stacks get deoptimized, ...

  vm/dart_2/isolates/reload_no_active_stack_test:
    Similar to the test above, but instead of having an active stack the
    isolates can yield to the event loop, possibly be even descheduled

  vm/dart_2/isolates/reload_many_isolates_test:
    Similar to the test above, but this test uses many more isolates.

  vm/dart_2/isolates/reload_many_isolates_live_and_die_test:
    Performs a reload where isolates get spawned and die all the time.
    There are always P isolates alive at any given point in time, each
    of them spawns children when their parent has died.
    Performing a reload catches isolates as various stages of their
    lifecycle and can therefore cover a lot of corner cases.

TEST=vm/dart_2/isolates/reload_*_test.dart

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I97039b4084de040b7f2e22f5832a40d57ba398d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 18:57:02 +00:00
Alexander Aprelev c22205c14e [vm/concurrency] Ensure write program_lock is held while updating class state via set_state.
Fix tests and AllocateObject.

Hopefully addresses flaky https://github.com/dart-lang/sdk/issues/44940

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=ci and standalone_2/out_of_memory_recovery_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try

Change-Id: Ife775f7a0bcdd343b887ba1bf912e07b2a8430c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188362
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-02 16:28:12 +00:00
Martin Kustermann 5e51ee2e01 [vm/concurrency] Remove NoReloadScope where its not needed
The kernel loader works without an active isolate. It no longer causes
execution of Dart code. Constants are evaluated in CFE and are simply
"deserialized" by the kernel loader.

Since kernel loader doesn't execute Dart code and also doesn't process
OOB messages at any point, there's no reason to have a [NoReloadScope]
in it.

Formatting language errors seems to be done purely in the VM and should
therefore also require no [NoReloadScope]

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I80b2888b60a4d7d179988b8606cd7ca0893a2fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187400
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 16:17:32 +00:00
Martin Kustermann e2be5322e6 [vm] Use custom realloc() on Windows
Seemingly Windows doesn't always implement realloc() by freeing space if
the new buffer is significantly smaller than the old one.

This CL does so manually in our code base.

Issue b/178993708
Issue https://github.com/dart-lang/sdk/issues/44846

TEST=standalone{,_2}/io/large_file_read_small_file_test

Change-Id: I370346a693f8e8e879e079d91e9de5c756949a11
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183694
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 16:02:52 +00:00
Martin Kustermann 3fac077ebc [vm/concurrency] Use relaxed atomics when accessing usage counters - for TSAN
For now our decision is to not synchronize access to usage counters and
instead let there be possible non-observable writes (if two concurrent writes
happen).

Though TSAN will report concurrent access in C++ to the usage counters
as a race, so we'll use a relaxed atomic.

(We do have non-synchronized access to the usage counters already:
If the mutator kicks of a background compilation, the BG compiler
might reset the counters while mutator is running.)

Closes https://github.com/dart-lang/sdk/issues/45134

TEST=Fixes flaky tsan errors.

Change-Id: I7919747536d43094e7201e03f464e4ac7b1a4740
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187923
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 15:53:22 +00:00
Martin Kustermann ed78971d90 [vm/concurrency] Simplify conditions for hot reload and test mode implementation
This is part of making hot-reload work with isolate groups.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: Id32e7d8f95ba0b1fb5bd11179dcd1805d42aba89
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187083
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 15:21:22 +00:00
Martin Kustermann d7a9015268 [vm/compiler] Remove BG compiler bailouts that afaik should not be possible
Many years ago those bailouts were seemingly added due to deferred
loading in [0]. Though it seems like those bailouts might've become
obsolete.

All places that clear the IC array on the function should run inside
a [NoBackgroundCompiler] scope, therefore ensuring no optimizing
compilations are done at that time. Those places also clear code,
therefore forcing re-compilation of unoptimized code (which will
re-create the ICData).

[0] https://codereview.chromium.org/1672873003/#ps60001

TEST=Removing dead code.

Change-Id: Iec4f22be500f6e874fa5286b51396bd6d978dd6c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187902
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-03-02 15:16:32 +00:00
Martin Kustermann 7fc9a1a14c [vm/compiler] Remove non-background optizable bit
The background compiler no longer has bailouts due to something that
cannot be done on BG compiler and has to be done on mutator (was
originally introduced in [0])

All optimizing compilations can happen on the BG thread. Class
finalization and other things no longer cause a bailout on BG
compiler.

If e.g. invalidated field guards cause a compilation to be discarded, it
can be retried on BG compiler.

[0] https://dart-review.googlesource.com/c/sdk/+/54886

Closes https://github.com/dart-lang/sdk/issues/45136

TEST=Fixes flaky hits of an assertion.

Change-Id: I50313dbedf96b8dec205acdb8c1ed5731d00433b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187901
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 14:20:02 +00:00
Martin Kustermann 29268ba7d2 [vm/concurrency] Move no-reload-scope counter to Thread
Whether a given isolate allows reloading is really a property of the
thread not the isolate (where it was before) or isolate group (where
it was moved to), so this CL moves it to Thread.

As the thread's stack is going down and up, NoReloadScope's might be
entered or exited, just as with our many other scopes, such as
NoOOBMessageScope/...

Longer term we might need to find a better mechanism to deal with
different OOB messages. For example: Reload as well as OOB messages
are disabled during static field initialization. Though that means *all*
OOB messages cannot reach the isolate during initialization of a static
field (which can do arbitrary amount of work).

This is part of making hot-reload work with isolate groups.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I9e80c6d4a184c54c0373fef13af3264cc3f27ece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187002
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 14:11:42 +00:00
Martin Kustermann f3740ced59 [vm/concurrency] Add SafepointMonitorUnlockScope to allow scoped unlocking of a monitor
In order to simplify code that needs to temporarily give up a monitor
lock, this CL adds a scoped object that releases the monitor on
construction and re-acquires it on destruction.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I004a04e54dcdaea009bfbef25d2a946a307e41c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187001
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 13:11:42 +00:00
Tess Strickland 3382bbc510 [vm] Print out canonical clusters in --print-snapshot-sizes-verbose.
TEST=Manual use of the flag.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I4c506848f97f6b9a3f1455f90872093497c853a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188287
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-02 08:37:01 +00:00
Ryan Macnak 9451963493 [vm, compiler] Remove intrinsics from internal double-dispatching int methods.
If the intrinsic on int.op fails, then the intrinsic on _opFromInteger will also fail, since both handle the same range of values.

Add intrinsic on int.% as previously only _moduloFromInteger was intrinsified. All the other ops already intrinsified the public method.

Remove _bitAndFromSmi; this was once useful for handling BigInt & Smi, but this case no longer exists.

TEST=ci
Change-Id: Ie33b75126e56110db3caabb9bcb6e9198a3158bf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187160
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-01 19:29:00 +00:00
Tess Strickland 960c944940 [vm/compiler] Get parameter names/counts from the closure signature.
Doing so removes some dependencies on the closure function itself,
making it easier to shake the function when possible in the future.

TEST=Existing tests that check dynamic closure invocations.

Bug: https://github.com/dart-lang/sdk/issues/41052
Bug: https://github.com/dart-lang/sdk/issues/44005

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I88c4b37efff9a9894633db7f380ecc39da717e6e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187081
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-03-01 16:25:24 +00:00
Vyacheslav Egorov 6e7c0ae227 [vm] Comment for RegressFlutter76919
Add a comment which describes what this test is testing.

TEST=not needed
TBR=kustermann@google.com

Change-Id: I88238595d1c11609130bfdde3219f58c39db57f6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188284
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-01 14:35:09 +00:00
Vyacheslav Egorov afa22dd1ab [vm/aot] Disable assert strengthening optimization in AOT
This optimization does not make sense in AOT and can actually cause
AOT compiler to segfault because it expects environment to be
attached to AssertAssignable instructions which is not always
the case on AOT compiler.

Fixes https://github.com/flutter/flutter/issues/76919

TEST=vm/cc/TypePropagator_RegressFlutter76919,vm/dart{,_2}/regress_flutter76919

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I8f4314d7dac276833a9050bba835616b670a88d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188281
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-01 13:13:19 +00:00
Vyacheslav Egorov 1f828c7cb2 [vm/compiler] Fix IntConverter canonicalization
When going from Int64 through another representation back into Int64
you can discard a conversion if and only if it does not lose some
bits on the way. Previously we were checking for truncation flag, but
this is incorrect because it does not actually tell us anything about
about whether or not bits are going to be lost.

Instead, we should be checking the range of the source value.

In addition to fixing potential correctness issues, this CL actually
improves quality of code generated for Uint8 loads because it removes
a redundant chain of Int64->Uint32->Int64 conversions that follows
the load (which compiled 2 two redundant moves).

Issue https://github.com/dart-lang/sdk/issues/40308

TEST=vm/cc/IL_IntConverterCanonicalization

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try
Change-Id: Ifef0943a50d414bde41b2194e30b58ad3a8c106c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187921
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-01 09:55:09 +00:00
Alexander Markov 26e7106937 [vm/aot] Partially remove Code objects from AOT snapshots
Code objects which do not carry useful information are
marked as 'discarded' during precompilation. Snapshot
serializer only writes instructions, compressed stack
maps and state bits for discarded Code objects.

This is a preliminary step before deserializer would be able
to omit creating instances for discarded Code objects at
runtime.

On a large app, AOT-compiled in PRODUCT mode with
--dwarf-stack-traces on arm64:
Discarded: 77% of all Code objects in snapshot.
Uncompressed AOT snapshot size -1.9%.

AOT snapshot size of Flutter gallery in release-sizeopt mode:
arm64 uncompressed -2.1% (gzip -0.6%, brotli -0.7%)
arm32 uncompressed -1.9% (gzip -0.6%, brotli -0.7%)

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try
Change-Id: I159cfebbdac76cc60060fedb1d22ed86672a68a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187600
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-27 00:41:37 +00:00
Daco Harkes 0295dfc74e [vm] Support redirecting external const factories
This CL introduces the first combination of `external const factory`
with `@patch const factory (.*) = (.*)`. This specific combination was
never used in the core libraries, and did not work.

Half of the fix is a missing flag from the CFE:
https://dart-review.googlesource.com/c/sdk/+/187440

The other half of the fix is matching on the
kRedirectingFactoryConstructor tag, and allowing constructor names
in procedure targets (even though we never compile those expressions
because they are invalid).

Bug: https://github.com/dart-lang/sdk/issues/45101

TEST=Using redirecting const factory in patch file in core lib.
TEST=tests/ffi(_2)/*

Change-Id: I5524bde928290bf32aaea9170eda1f2d03127fa6
Cq-Include-Trybots: luci.dart.try:benchmark-linux-try,front-end-linux-release-x64-try,vm-kernel-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187003
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-02-26 11:15:26 +00:00
Vyacheslav Egorov 7ba12664ea [vm/compiler] Fix LoadStaticField::ComputeCanDeoptimize
It should only return true in JIT mode similar to other calls.

Discovered when looking at allocation sinking in FFI version of
reverse-complement microbenchmark.

TEST=ci

Issue https://github.com/dart-lang/sdk/issues/40308

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try
Change-Id: I79f432ffb88a467336fcb412a2d9fcdedc390cd8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187404
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-02-26 10:24:27 +00:00
Clement Skau f2dfe3a8be [vm/debugger] Fixes async CollectAwaiterReturn()
The async, off-stack part of DebuggerStackTrace::CollectAwaiterReturn's
stack unwinding incorrectly looked on the stack instead of following
the Closure's awaiter chain.
This caused the async stack traces to be truncated and missing any
async frames, in turn causing ShouldPauseOnException to incorrectly
conclude no handler frame existing.
This would affect any case where the exception handling was located
around any except for the inner most awaiter.

TEST=Added regression test to pause_on_unhandled_exceptions_catcherror_test

Closes: https://github.com/dart-lang/sdk/issues/37953
Bug: https://github.com/dart-lang/sdk/issues/37953
Change-Id: I34a2b5ce3c7532032b2ef3dfbc2af7294d13a4c7
Cq-Do-Not-Cancel-Tryjobs: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187405
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-02-26 06:46:36 +00:00
Ryan Macnak 5a385890b9 [test] Apply simarm64 status file entries to simarm64c.
Remove references to the removed simarmv6 architecture.

TEST=ci
Change-Id: I10e9fc8291c3e5b6b9a4bd71ca05de95e53f22c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187500
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-25 21:18:02 +00:00
Ryan Macnak 20bc0a0ce1 [vm] Fix annotated pool references when disassembling in JIT mode.
FLAG_use_bare_instructions is vacuously true in JIT mode, so use GetObjectPool which additionally accounts for JIT versus AOT.

TEST=examine output of --disassemble in JIT, AOT compile and AOT runtime
Change-Id: Idae17b779f16a6b823cf702bf868b232ca0e03ab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187521
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-25 21:09:57 +00:00
Liam Appelbe 807874d51a [vm] Align compressed pointer allocations on windows.
Change-Id: I95ad0a2cf679a5918d42b12bb95f1503bb219fca
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184846
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-25 20:48:07 +00:00
Martin Kustermann c78f12c550 [vm/concurrency] Allow caller of Thread::{Exit,Enter}Isolate to specify nesting
It's mainly hoisting an expression out of
IsolateGroup::{Increase,Decrease}MutatorCount to callers of it.

Right now we consider a Thread::{Exit,Enter} to be nested if there are
existing Dart frames on the stack. A future CL will have a case where
there's a nested exit&re-enter without Dart frames on the stack.

Trying to do this automatically by checking whether there's an
active api scope / zone on the thread fails due to some
asymmetry in our codebase.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: Id70eda03efb2099487c5f0c8affd25ddff926553
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186287
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-25 18:44:16 +00:00
asiva 02c1e2e56b Remove obsolete comment,
Full snapshots are generated from the clustered snapshot modules
and code in this file is used for only isolate messaging and the comment
about full snapshot does not apply here.

TEST=no tests, just a comment delete.

Change-Id: Ib2453514639aae1c99d3eab6bb120f0ec5ac9f4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186747
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-02-24 23:05:14 +00:00
Ryan Macnak 9876de6838 [vm, x64] Cleanup BoxInteger32Instr to remove effects when not using compressed pointers.
TEST=ci
Change-Id: I28dd873a27d6ecc96855b810db55f568c71645ce
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186763
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-24 21:23:54 +00:00
Alexander Aprelev 203532a647 [vm/concurrency] Use SafepointMonitorLocker when enabling background compiler.
The way how BackgroundCompiler::Enable manipulates done_monitor_ lock should match how BackgroundCompiler::Disable manipulates it - by getting to a safepoint if needed.

TEST=Fixes flaky timeouts

Fixes https://github.com/dart-lang/sdk/issues/45108

Change-Id: Iac3e4b864951ccbe904a1031bc7f5a557faf3b41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187061
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-02-24 20:43:54 +00:00
Ryan Macnak 1a83a454dd [vm, arm64] Fix simulation of 32-bit clz.
TEST=ci
Change-Id: Ie9cd4d39ed95ec3c770b94c40362c5827ceb4a76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186460
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-24 20:18:44 +00:00
Dmitry Stefantsov c191551fac [cfe] Remove BottomType
TEST=Covered by existing tests.

Change-Id: Ied8be1874164e68e0a3a48e1b2f9a33310071381
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183009
Commit-Queue: Dmitry Stefantsov <dmitryas@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2021-02-24 13:21:55 +00:00
Daco Harkes d45fd0f9d5 [vm/ffi] Support inline arrays in Structs
Adds support for single dimension inline arrays in structs. Multi-
dimensional arrays will be supported in a future CL.

This CL adds:
- CFE static error checks for inline arrays.
- CFE transformations for inline arrays.
- VM consumption of inline array fields for NativeType.
- Test generator support for inline arrays + generated tests.

Previous CLs added support for inline arrays in:
- analyzer https://dart-review.googlesource.com/c/sdk/+/183684
  - updated in this CL to new API.
- ABI calculation https://dart-review.googlesource.com/c/sdk/+/183682

Closes: https://github.com/dart-lang/sdk/issues/35763

Open issue: https://github.com/dart-lang/sdk/issues/45101

CFE transformations are tested with expectation files:
TEST=pkg/front_end/testcases/(.*)/ffi_struct_inline_array.dart

Trampolines and CArray API are tested with end-to-end Dart tests:
TEST=tests/ffi(_2)/(.*)by_value(.*)test.dart
TEST=tests/ffi(_2)/inline_array_test.dart

Compile-time errors (both CFE and analyzer) are tested in:
TEST=tests/ffi(_2)/vmspecific_static_checks_test.dart

Change-Id: I014c0e4153f1b885638adce80de6ab3cac8e6bb2
Cq-Include-Trybots: luci.dart.try:dart-sdk-linux-try,dart-sdk-mac-try,dart-sdk-win-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64-try,vm-kernel-asan-linux-release-x64-try,vm-kernel-mac-debug-x64-try,vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-mac-release-x64-try,vm-kernel-nnbd-win-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-win-release-x64-try,vm-kernel-reload-linux-debug-x64-try,vm-kernel-reload-rollback-linux-debug-x64-try,vm-kernel-win-debug-x64-try,vm-kernel-win-debug-ia32-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-precomp-obfuscate-linux-release-x64-try,vm-kernel-msan-linux-release-x64-try,vm-kernel-precomp-msan-linux-release-x64-try,vm-kernel-precomp-android-release-arm_x64-try,analyzer-analysis-server-linux-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183640
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-02-24 13:02:45 +00:00
Jens Johansen 3f8cbc96d9 [observatory] Make Multi-line evaluation work
Before this CL, if enabling multi-line expression evaluation, clicking
`Evaluate` would crash because of - essentially - a null pointer
exception.

TEST=Existing test suites.

Change-Id: Ibe517870dee33ff9d3edd427601d724b151e9699
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186293
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-02-24 10:41:50 +00:00
Jens Johansen bbb24d3364 [observatory] Properly handle expression compile time errors
In observatory there are two different ways to do expression evaluation.
One calls `evaluateInFrame` and catches the ServerRpcException thrown
when there's a compilation error. Another calls `evaluate` and does not
catch the ServerRpcException thrown when there's a compilation error.

This means, for instance, if evaluting an expression on an instance
and makes a typo there's suddenly a big box with scrambled javascript
on the top-right of the screen.

This CL:
* Handles the ServerRpcException thrown when there's a compilation error
  for `evaluate` as well.
* Makes both `evaluate` and `evaluateInFrame` unpack the `details` sent
  by the VM instead of printing out the textual representation of a map.
  This means that you get an error message like

```
org-dartlang-debug:synthetic_debug_expression:1:1: Error: Getter not found: 'test'.
test
^^^^
```

instead of

```
{details: org-dartlang-debug:synthetic_debug_expression:1:1: Error: Getter not found: 'test'.
test
^^^^}
```

TEST=Existing test suites.

Change-Id: I2dc3f19f024e918e8d0e9b0f4166a4ef2c6cbd9b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186292
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-02-24 10:34:35 +00:00
Regis Crelier ba611b720b [VM/runtime] Dereference top-level TypeRefs type arguments during finalization and hashing.
This fixes https://github.com/dart-lang/sdk/issues/45060

TEST= added regression tests

Change-Id: Ie6e3fd8514ee5b42af1ab5a5c72883c2510da31d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186421
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2021-02-24 02:39:27 +00:00
Alexander Aprelev ea3dfc9ac4 [vm/io] Fix an exception when attempting to immediately cancel socket connection.
Fixes dartbug.com/45047

Change-Id: Ie4630684722e8c17cbc3766f91111b2a76bce7c1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186641
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-02-24 00:32:07 +00:00