Commit Graph

2907 Commits

Author SHA1 Message Date
Regis Crelier e35ba5b9e3 [VM/runtime] Consider trail when comparing type parameter and bounds of function types.
Fixes https://github.com/dart-lang/sdk/issues/45443

TEST=added regression test

Change-Id: I82c581d9627d9eb1cf31d108a534fd4f5d6bb0a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192947
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2021-03-29 16:49:47 +00:00
Tess Strickland bf74cad477 [vm/compiler] Change when parents of local functions are retained.
Now they are only retained if one of the following is true:

* The parent is Future.wait or Future.timeout.
* The local function is a generated body of an async/async*/sync*
  function.
* Symbolic stack traces are enabled.

When a local closure's parent is dropped, the parent name will
be printed as `<optimized out>` in places like exceptions.

Changes in Flutter gallery in release-sizeopt mode:

* ARM7: ROData -1.41%, Isolate snapshot -2.35%, Total snapshot -0.54%,
        Isolate heap -1.38%, Total heap -1.38%
* ARM8: ROData -1.55%, Isolate snapshot -2.34%, Total snapshot -0.65%,
        Isolate heap -1.41%, Total heap -1.41%

TEST=Existing tests on CI.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try, vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: I9d05f9e0e30e9f428eff16a15b0f1eeb974419c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190023
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-25 13:47:03 +00:00
Tess Strickland e4ea8e8d97 [vm] Remove dead Function:IsInFactoryScope method.
TEST=None needed.

Change-Id: I1fa6a0647d4d54f425f1209acbcee81cf7f7af47
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/192923
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Auto-Submit: Tess Strickland <sstrickl@google.com>
2021-03-24 16:58:41 +00:00
Tess Strickland e7d3a6027c [vm] Create more Function helper methods for code clarity.
Creates HasParent() as a predicate for Function objects with a non-null
parent function and makes IsLocalFunction() specific to local closures
(i.e. explicit closures with non-null parent functions).

Creates HasSavedArgumentsDescriptor() as a predicate to determine when
saved_args_desc() can be used, to avoid the caller having to do explicit
predicate checks for each dispatcher type that contains one.

Also simplifies Function::PrintName, creates different suffixes for a
SyncGenClosureMaker and its associateed SyncGenClosure, adds suffixes
for skipping more than one generated body when printing the parent name,
and changes ArgumentsDescriptor::PrintTo to match the compact syntax
Function::PrintName used for signifying the arguments descriptor for
certain dispatchers.

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test

Cq-Include-Trybots: luci.dart.try:app-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,pkg-linux-release-try
Change-Id: I0ac1226d1ffe1d81743b5c6788da170d5a4010c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191560
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-03-23 12:21:26 +00:00
Tess Strickland d504084f42 [vm] Fix uses of NullIsAssignableTo at runtime.
NullIsAssignableTo only takes the type to check for nullability.
However, while a type parameter may itself be non-nullable, its
instantiation with a nullable type is nullable.

Thus, this CL adds type instantiation before runtime uses of
NullIsAssignableTo in Function::DoArgumentTypesMatch.

Fixes https://github.com/dart-lang/sdk/issues/45270

TEST=vm/dart/regress_45270

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm_x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-nnbd-linux-release-ia32-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try
Change-Id: Iaefffe9ab6d4d3ad889c4962489b2153d2bbaee1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190443
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-03-19 12:29:27 +00:00
Ryan Macnak 19e5899a02 [vm] Switch most non-Instances to compressed pointers.
The following are not compressed: Code, ObjectPool, Context, CallSiteData and subclasses.

Code::instructions_ is uncompressible because the VM doesn't control the address of Instructions when they are loaded via dynamic library.

Loads from the ObjectPool are a frequent occurrence in generated code, and shrinking ObjectPool entries would be offset by longer code sequences to decompress them. Also, some ObjectPool entries contain uncompressible pointers to C functions.

Context is not compressed based on reports from earlier work on compressed pointers that compressing them is disproporately bad for performance. The earlier work used a different, more-expensive compression scheme, so we should reinvestigate this.

CallSiteData et al. are on the hot path for unoptimized code but make up a comparatively small portion of the heap size.

TEST=ci
Change-Id: I9a425fa4e12113c9fd0b51999525d32a7e589a90
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190600
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-03-18 22:52:04 +00:00
Ryan Macnak 08295a8cb3 [vm, compiler] Consider T{_IntegerImplementation} as equivalent to T{int} for optimization purposes.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/45297
Change-Id: Ic6a8956ad6344058c480e20018e876f5aae4c610
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191662
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-03-18 17:59:03 +00:00
Alexander Aprelev 4baf7e5e3d [vm/concurrency] Introduce GroupDebugger that takes CodeBreakpoints management away from isolate Debugger.
CodeBreakpoint class tracks patching of the call targets for the purpose of routing the call to the debugger runtime.
With this CL CodeBreakpoint instance is managed by GroupDebugger, debugger class owned by IsolateGroup, and can be associated with multiple BreakpointLocation, one location per isolate that set a breakpoint at particular code location.

TEST=observatory/tests/service/break_on_function_child_isolate_test

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I3b3d0974a0fb9668742e5c1e2a2ce13b84b1d630
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190732
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-17 18:03:14 +00:00
Regis Crelier 33cabf4313 [VM/runtime] Allow generic function types as type arguments in the VM.
Type parameter scopes were not set properly when loading bounds with generic function types from kernel file.
The trail to avoid infinite recursion while comparing type parameter bounds was not used properly.

Fixes https://github.com/dart-lang/sdk/issues/45313
Fixes https://github.com/dart-lang/sdk/issues/45322
Fixes https://github.com/dart-lang/sdk/issues/44930

TEST=regression tests added

Change-Id: I096b7d8c22aa5b3a3a9e12a59b1417388ff5171f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191465
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-17 00:16:41 +00:00
Ryan Macnak c9ba99ae6d [vm, compiler] Make the difference between the CompileTypes for int and _IntegerImplementation visible when printing IL.
TEST=ci
Change-Id: I17f6a79e3642a809a57f7d240c3894a093429a81
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191460
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-03-16 21:57:01 +00:00
Tess Strickland 901fed103b [vm] Use the cached number of parent type args in FunctionType.
Also ensure the number of parent type arguments is appropriately
adjusted in the returned FunctionType when partially instantiating
a FunctionType, and that free parent type arguments are _not_
included in types built while reading constants.

Since the number of parent type arguments is now cached there, we
no longer need to also cache them in ClosureData. We can also remove
the parent walking in Function::NumParentTypeArguments().

Previously, a FunctionType where the component types did not use
parent type parameters was considered instantiated. This CL changes
it so that FunctionType with free parent type parameters are never
considered instantiated. This is necessary because otherwise,
when instantiating the parent type parameters, a FunctionType that
does not use its parent type parameters will be used unchanged
instead of creating a copy with fewer parent type parameters.

Because of this, places where IsInstantiated was used to simply
check for generic parent functions (namely, for implicit closure
creation) has been appropriately weakened to check
!HasGenericParent() instead.

TEST=Existing tests on CI.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Ifb4a0a1273d8d01908cdf4ffc3c4c28a1c33ffa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190021
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-03-16 15:04:51 +00:00
Ryan Macnak d1605bc5c5 [vm, compiler] Don't touch memory in AssertBoolean and BooleanNegate.
AssertBoolean may assume its input is nullable bool. BooleanNegate may assume its input is non-nullable bool. Use fixed alignment bits to distinguish true, false and null.

TEST=ci
Change-Id: Ife694e66c1225d943881c2325cd67f18575a47e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189240
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-09 20:27:00 +00:00
asiva 9f6e64a06f [VM/runtime] Fix a warning
TEST=ci

Change-Id: I3bdfa6b1069f4f6f0f889eedf07c4c6527fe13f5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189943
Reviewed-by: Ben Konyi <bkonyi@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-03-09 17:21:50 +00:00
Alexander Aprelev c0124aed87 [vm/concurrency] Fix tsan warning data race around Field guarded_cid_.
Use relaxed atomic write to match relaxed atomic read for guarded_cid_ field.

Fixes https://github.com/dart-lang/sdk/issues/45186

TEST=flaky tsan failures on  vm/dart_2/isolates/reload_many_isolates_test

Change-Id: Ib538a04187cff77ef278db4d62e551d5db9fd023
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189280
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-09 16:20:50 +00:00
Alexander Aprelev e4ab127fbc [vm/concurrency] Make Function::kind_tag_ concurrency-safe.
Fixes https://github.com/dart-lang/sdk/issues/45182.

TEST=flaky vm/dart_2/isolates/reload_many_isolates_test on tsan bot
Change-Id: Ie874e65643d1a1dc94a94446b3141a4fabb15adb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189261
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-08 19:29:21 +00:00
Vyacheslav Egorov ae37ff098a [vm] Compact serialization for canonical sets
This CL changes how canonical sets for some specific types are written
into the root snapshot: instead of writing canonical set as a separate
object we reorder objects within a canonical cluster in such a way that
the order matches order of elements in the backing store of a canonical
set and then we write canonical set layout out using differential
encoding (essentially writing gaps between elements instead of writing
absolute indices).

This significantly reduces the overhead of having canonical sets in the
snapshot while maintaining fast deserialisation: for example on build
microbenchmark this brings regression in the snapshot size from 4% to
.3%.

On sizeopt benchmarks:

  flutter_gallery_app_so_gzip_size   -1.5%
  flutter_gallery_app_so_size        -4.7%
  flutter_gallery_total_heap_size   -16.2%


TEST=ci

Change-Id: I2be7fd073668e9b52098e2acda9f11d128cfda95
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185381
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-03-08 14:21:01 +00:00
Ryan Macnak dce0b514d4 [vm] Defer object pools and object pool entries.
This defers literals, to effectively split binary size for libraries that are more data than code, such as localizations.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I506722037cc0247b90756959e6a6f12bb5021b5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183781
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-04 23:19:30 +00:00
Martin Kustermann 187117ddf4 [vm/concurrency] Allow concurrent calculation of Class::num_type_arguments_
The lazily calculated value can be calculated concurrently and installed
in arbitratry order as long as all writers write the same value (which
they should).

There's no need to guard it with a lock.

Closes https://github.com/dart-lang/sdk/issues/45175

TEST=Refactoring of existing code, fixes flaky tsan reports.

Change-Id: Iaa45cf3b70300958acc04953e37599836436bc59
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188720
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-03 17:14:48 +00:00
Martin Kustermann 91c628d63a [vm/concurrency] Final support for hot-reload of multi-isolate groups
This is the initial implementation of hot reload with multi-isolate
groups.

Implementation:

As before, when a service API call triggers a reload it will be routed
as an OOB message to a specific isolate (**). As opposed to before, that
isolate has now to coordinate with all other isolates, ensuring that it
"owns" the reload and all other isolates are waiting in a state that
allows reload.

This is implemented as a [ReloadOperationScope] which first participates
in other reloads (if there are any) and then owns the reload. It will
send a new kind of service message to all other registered isolates. All
of them have to check in before reload can proceed. If a new isolate
is about to join the group, it will participate when registering the
isolate. If an old isolate wants to die, it will participate when
unregistering the isolate.

This means that in addition to the existing StackOverFlow checks that
can process OOB messages and therefore reload, we'll have isolate
registration and unregistration as well as a new
Isolate::kCheckForReload OOB message handler where an isolate can
participate in a reload.

We consider the isolate group to be reloadable if the main isolate has
loaded the program and set the root library. Helper isolates don't need
to load any more kernel code and only initialize core libraries, so it's
fine to reload them during this time.

(**) The reason we continue to send reload service API calls to any
isolate in an isolate group is that re-loading might involve calling out
to the embedder's tag handler. Doing so currently requires an active
isolate.

If we allowed a subset of dart_api.h (the subset needed by the tag
handler) to be used only with an active IsolateGroup instead of an
active Isolate we could remove this requirement.

Edge cases:

There's various edge cases to consider: The main edge case is, we currently
maintain an upper limit to the number of isolates executing in parallel
(to ensure each can have big enough chunk of new space, i.e. TLAB).

If there are more isolates with active work they are waiting until one
of the exiting ones "yields". To ensure progress, if any such actively
running isolate gets a request to participate in a reload, it will mark
its own thread as "blocked" and therefore "yields", so another isolate
can make progress until all isolates are participating and the reload
can start.

Marking an isolate as "blocked" happens by exiting that isolate. It will
free up it's TLAB, decrease active mutator count and (if running on VM's
thread pool) also temporarily increase the thread pool size.

The side-effect of this is that it will use one pthread per isolate
during reload. In the future we can extend this first implementation, by
specially handling isolates that don't have a message handler running.
Doing so would require careful consideration to avoid races.

Testing:

In order to test this we use a small helper framework for reload tests.
The helper framework will, similar to real world reload e.g. in flutter,
will spawn a subprocess. It will use the service API to trigger reloads
in this subproces.

To synchronize between the reload driver and the application being
reloaded it allows watching for events to be printed to stdout/stderr.

The reload test itself can be written - similar to multitests - with
annotations such as `// @include-in-relload-0` in them. The testing
framework will then generate multiple application versions that all get
compiled to kernel.

For simplicity we generate the kernel using the standalone VM with
`--snapshot-kind=kernel` and avoid using the incremental compiler.

There are 4 different tests exercising different aspects of
multi-isolate reload:

  vm/dart_2/isolates/reload_active_stack_test:
    Performs a reload while a fixed number of isolates have an active
    stack, thereby ensuring e.g. that all frames of all isolate mutator
    stacks get deoptimized, ...

  vm/dart_2/isolates/reload_no_active_stack_test:
    Similar to the test above, but instead of having an active stack the
    isolates can yield to the event loop, possibly be even descheduled

  vm/dart_2/isolates/reload_many_isolates_test:
    Similar to the test above, but this test uses many more isolates.

  vm/dart_2/isolates/reload_many_isolates_live_and_die_test:
    Performs a reload where isolates get spawned and die all the time.
    There are always P isolates alive at any given point in time, each
    of them spawns children when their parent has died.
    Performing a reload catches isolates as various stages of their
    lifecycle and can therefore cover a lot of corner cases.

TEST=vm/dart_2/isolates/reload_*_test.dart

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I97039b4084de040b7f2e22f5832a40d57ba398d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187461
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 18:57:02 +00:00
Alexander Aprelev c22205c14e [vm/concurrency] Ensure write program_lock is held while updating class state via set_state.
Fix tests and AllocateObject.

Hopefully addresses flaky https://github.com/dart-lang/sdk/issues/44940

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=ci and standalone_2/out_of_memory_recovery_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try

Change-Id: Ife775f7a0bcdd343b887ba1bf912e07b2a8430c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188362
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-02 16:28:12 +00:00
Martin Kustermann 5e51ee2e01 [vm/concurrency] Remove NoReloadScope where its not needed
The kernel loader works without an active isolate. It no longer causes
execution of Dart code. Constants are evaluated in CFE and are simply
"deserialized" by the kernel loader.

Since kernel loader doesn't execute Dart code and also doesn't process
OOB messages at any point, there's no reason to have a [NoReloadScope]
in it.

Formatting language errors seems to be done purely in the VM and should
therefore also require no [NoReloadScope]

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I80b2888b60a4d7d179988b8606cd7ca0893a2fa4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187400
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-03-02 16:17:32 +00:00
Martin Kustermann d7a9015268 [vm/compiler] Remove BG compiler bailouts that afaik should not be possible
Many years ago those bailouts were seemingly added due to deferred
loading in [0]. Though it seems like those bailouts might've become
obsolete.

All places that clear the IC array on the function should run inside
a [NoBackgroundCompiler] scope, therefore ensuring no optimizing
compilations are done at that time. Those places also clear code,
therefore forcing re-compilation of unoptimized code (which will
re-create the ICData).

[0] https://codereview.chromium.org/1672873003/#ps60001

TEST=Removing dead code.

Change-Id: Iec4f22be500f6e874fa5286b51396bd6d978dd6c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187902
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-03-02 15:16:32 +00:00
Martin Kustermann 7fc9a1a14c [vm/compiler] Remove non-background optizable bit
The background compiler no longer has bailouts due to something that
cannot be done on BG compiler and has to be done on mutator (was
originally introduced in [0])

All optimizing compilations can happen on the BG thread. Class
finalization and other things no longer cause a bailout on BG
compiler.

If e.g. invalidated field guards cause a compilation to be discarded, it
can be retried on BG compiler.

[0] https://dart-review.googlesource.com/c/sdk/+/54886

Closes https://github.com/dart-lang/sdk/issues/45136

TEST=Fixes flaky hits of an assertion.

Change-Id: I50313dbedf96b8dec205acdb8c1ed5731d00433b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187901
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 14:20:02 +00:00
Martin Kustermann 29268ba7d2 [vm/concurrency] Move no-reload-scope counter to Thread
Whether a given isolate allows reloading is really a property of the
thread not the isolate (where it was before) or isolate group (where
it was moved to), so this CL moves it to Thread.

As the thread's stack is going down and up, NoReloadScope's might be
entered or exited, just as with our many other scopes, such as
NoOOBMessageScope/...

Longer term we might need to find a better mechanism to deal with
different OOB messages. For example: Reload as well as OOB messages
are disabled during static field initialization. Though that means *all*
OOB messages cannot reach the isolate during initialization of a static
field (which can do arbitrary amount of work).

This is part of making hot-reload work with isolate groups.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I9e80c6d4a184c54c0373fef13af3264cc3f27ece
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187002
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-03-02 14:11:42 +00:00
Alexander Markov 26e7106937 [vm/aot] Partially remove Code objects from AOT snapshots
Code objects which do not carry useful information are
marked as 'discarded' during precompilation. Snapshot
serializer only writes instructions, compressed stack
maps and state bits for discarded Code objects.

This is a preliminary step before deserializer would be able
to omit creating instances for discarded Code objects at
runtime.

On a large app, AOT-compiled in PRODUCT mode with
--dwarf-stack-traces on arm64:
Discarded: 77% of all Code objects in snapshot.
Uncompressed AOT snapshot size -1.9%.

AOT snapshot size of Flutter gallery in release-sizeopt mode:
arm64 uncompressed -2.1% (gzip -0.6%, brotli -0.7%)
arm32 uncompressed -1.9% (gzip -0.6%, brotli -0.7%)

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try
Change-Id: I159cfebbdac76cc60060fedb1d22ed86672a68a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187600
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-27 00:41:37 +00:00
Regis Crelier ba611b720b [VM/runtime] Dereference top-level TypeRefs type arguments during finalization and hashing.
This fixes https://github.com/dart-lang/sdk/issues/45060

TEST= added regression tests

Change-Id: Ie6e3fd8514ee5b42af1ab5a5c72883c2510da31d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186421
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2021-02-24 02:39:27 +00:00
Alexander Markov 7f8001218b [vm] Support unsigned shift operator int.>>>
TEST=runtime/tests/vm/dart/unsigned_shift_right_test.dart and other
tests for >>>.

Issue https://github.com/dart-lang/sdk/issues/30892
Issue https://github.com/dart-lang/sdk/issues/44914

Change-Id: I7f0a9e1e486be06576c5d9bff3659b0aa6d68c23
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184921
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-02-24 00:14:57 +00:00
Liam Appelbe be0978cbde Reland "[vm] Align heap allocations to 2GB regions in comp ptr mode"
Fixed the mac build.

This reverts commit 0ed3fb3dee.

Bug: https://github.com/dart-lang/sdk/issues/45059
Change-Id: I0cd6710a641b195403dda717108bdbe3395a21fa
TEST=CI
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186340
Commit-Queue: Liam Appelbe <liama@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-22 19:58:49 +00:00
Alexander Markov ddfc98337b Reland "[vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces"
This is a reland of b6dc4dad4d

TEST=ci

Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
>   fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
>   now avoids using Code objects.
>   In order to find CompressedStackMaps it now calls
>   ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
>   a replacement for Code objects in stack traces. It has
>   PayloadStart() == 0 and Size() == kUwordMax so it includes
>   arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
>   frames are replaced with StubCode::UnknownDartCode().
>   This simulates future behavior of ReversePc::Lookup when Code objects
>   will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

Change-Id: Ia2fc4672a085cd963b7fc103369851df3603590c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186202
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-22 10:46:28 +00:00
Vyacheslav Egorov 0ed3fb3dee Revert "[vm] Align heap allocations to 2GB regions in comp ptr mode"
This reverts commit d10d17cca2.

Reason for revert: Broke Mac builds

Original change's description:
> [vm] Align heap allocations to 2GB regions in comp ptr mode
>
> This only handles the POSIX case. The Windows case will work exactly the
> same as this, so I want to get this reviewed before I start porting it
> to windows.
>
> TEST=Manual testing. I'll update the unit tests once it's ported.
>
> Bug: https://github.com/dart-lang/sdk/issues/45059
> Change-Id: I0bc0eddd95101fc2fddfe5668488670e741af586
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183142
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Liam Appelbe <liama@google.com>

Bug: https://github.com/dart-lang/sdk/issues/45059
Change-Id: I840c31ead015f60bca32b551c8f8d31fa75ad645
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/186101
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-21 15:25:03 +00:00
Liam Appelbe d10d17cca2 [vm] Align heap allocations to 2GB regions in comp ptr mode
This only handles the POSIX case. The Windows case will work exactly the
same as this, so I want to get this reviewed before I start porting it
to windows.

TEST=Manual testing. I'll update the unit tests once it's ported.

Bug: https://github.com/dart-lang/sdk/issues/45059
Change-Id: I0bc0eddd95101fc2fddfe5668488670e741af586
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183142
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Liam Appelbe <liama@google.com>
2021-02-20 01:29:35 +00:00
Alexander Markov fde0ed2e93 Revert "[vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces"
This reverts commit b6dc4dad4d.

Reason for revert: broke package:vm_snapshot_analysis in Flutter
(https://github.com/flutter/flutter/issues/76313).

Original change's description:
> [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
>
> The following changes are done in preparation for the removal of Code
> objects in AOT with --dwarf-stack-traces:
>
> * Stack trace objects are extended to hold uword PCs (which may not
>   fit into Smi range).
>
> * Scanning stack frames in GC (StackFrame::VisitObjectPointers)
>   now avoids using Code objects.
>   In order to find CompressedStackMaps it now calls
>   ReversePc::FindCompressedStackMaps.
>
> * Singleton Code object (StubCode::UnknownDartCode()) is prepared as
>   a replacement for Code objects in stack traces. It has
>   PayloadStart() == 0 and Size() == kUwordMax so it includes
>   arbitrary PCs.
>
> * In --dwarf-stack-traces mode, most Code objects obtained from stack
>   frames are replaced with StubCode::UnknownDartCode().
>   This simulates future behavior of ReversePc::Lookup when Code objects
>   will be removed.
>
> Issue: https://github.com/dart-lang/sdk/issues/44852
> Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
> TEST=ci
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Tess Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I6f66171eecf1133363a7ce56193e782e43a20baf
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185488
Reviewed-by: Zach Anderson <zra@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-02-18 18:04:29 +00:00
Alexander Markov b6dc4dad4d [vm/aot] Avoid using most Code objects in stack traces with --dwarf-stack-traces
The following changes are done in preparation for the removal of Code
objects in AOT with --dwarf-stack-traces:

* Stack trace objects are extended to hold uword PCs (which may not
  fit into Smi range).

* Scanning stack frames in GC (StackFrame::VisitObjectPointers)
  now avoids using Code objects.
  In order to find CompressedStackMaps it now calls
  ReversePc::FindCompressedStackMaps.

* Singleton Code object (StubCode::UnknownDartCode()) is prepared as
  a replacement for Code objects in stack traces. It has
  PayloadStart() == 0 and Size() == kUwordMax so it includes
  arbitrary PCs.

* In --dwarf-stack-traces mode, most Code objects obtained from stack
  frames are replaced with StubCode::UnknownDartCode().
  This simulates future behavior of ReversePc::Lookup when Code objects
  will be removed.

Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I7cec7b8b9396c9cfeca3c256a412ba4e82a7e0c4
TEST=ci
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182720
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-02-17 20:12:32 +00:00
Martin Kustermann 5b4349de54 [vm/concurrency] Make main hot-reload implementation independent of current isolate
Since hot-reloading will affect all isolates within a group, there
should be nothing isolate-specific inside it. We therefore add a
NoActiveIsolateScope to the main part of the reloading
implementation.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: I1c8db00e6a016c77c68c2c04448de056a76d42c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184782
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-02-17 09:38:29 +00:00
Regis Crelier 4d5f027ccb [VM/runtime] Reland: Avoid comparing unfinalized type parameters.
This is done by simplifying the code looking for a type argument vector overlap between class and super class.

This is a reland of reverted https://dart-review.googlesource.com/c/sdk/+/184202
First patchset is the reverted CL.
A check for nullability was missing.

TEST=added regression test for the reland.

Change-Id: I77de3eded631d5223e9335ee9743b95878fe5bd7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185202
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
2021-02-16 22:17:00 +00:00
Ryan Macnak dd83f15826 [vm] Account for compressed pointers in pointer visitors.
TEST=ci
Change-Id: I033bb30e86dfc66ddeb7316a625e3f8925528a19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183784
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-02-16 18:27:39 +00:00
Ben Konyi 3e83400e0f [ VM / Service ] Added getAllocationTraces and setTraceClassAllocation RPCs and identity hash support
This change adds two new RPCs and various new properties:
  - getAllocationTraces
  - setTraceClassAllocation
  - classId and identityHashCode properties in CpuSample
  - traceAllocations property in Class

TEST=get_allocation_traces_test.dart

package:vm_service has been regenerated for 3.43 of the service protocol
and is ready for a 6.1.0 release.
Change-Id: Ia8ed055423798d7d17fe9f5fd74efb4239b875fc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182666
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-12 21:15:35 +00:00
Martin Kustermann b80ab44a53 [vm/concurrency] Move class registration from Isolate to IsolateGroup
We share the program across all isolates within a group. So hot-reload
should not depend on a particular isolate. This CL moves class
registration (which hot reload triggers) from Isolate to IsolateGroup.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: Ibdb4e67a095c7f2f3e3c1a8f8194bd0ca936b140
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184266
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-12 07:53:46 +00:00
Martin Kustermann 8e887672f4 [vm/concurrency] First changes for hot-reloading with isolate gropus.
This CL moves the [ProgramReloadContext] from [Isolate] to
[IsolateGroup] - since the program is shared across isolates. Amongst
other things, it also deoptimizes stacks of all isolate mutators.

There is still future work to do for hot-reloading multiple isolates
within a group, this is only the first change.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing implementation.

Change-Id: Ic394de265e3922f84657de2da25a46179b624647
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183693
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-12 05:48:15 +00:00
Martin Kustermann 92af830255 [vm/concurrency] Move background compiler related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`. This CL moves background
compiler and related state.

One background compiler will be shared for all isolates within a group.

Issue https://github.com/dart-lang/sdk/issues/36097
Fixes https://github.com/dart-lang/sdk/issues/44938

TEST=Mainly refactoring, stress tests for --enable-isolate-groups with JIT will come in the future.

Change-Id: I66e863651d0b37c431ce1864e505b5413850cbde
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183401
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-02-12 03:40:14 +00:00
Daco Harkes fca959e5f6 [vm/ffi] Add class to vm:ffi:struct-fields pragma
This CL changes `@pragma('vm:ffi:struct-fields', [...])` to
`@pragma('vm:ffi:struct-fields', _FfiStructLayout([...]))` which makes
it easier to add more data in subsequent CLs.

Extends `FindPragma` to allow returning multiple matched pragma's, so
that we can filter them. (In this case to avoid matching user-defined
pragma's that do not have an instance of the private class.)

Separated out from https://dart-review.googlesource.com/c/sdk/+/183640
because of the extra constant in existing expectation files.

Bug: https://github.com/dart-lang/sdk/issues/35763
Bug: https://github.com/dart-lang/sdk/issues/38158

TEST=tests/ffi(_2)/*_by_value_*_test.dart

Change-Id: Idef9f82e9b53c2a32dffabcec19669eae550fe2f
Cq-Include-Trybots: luci.dart.try:front-end-nnbd-mac-release-x64-try,front-end-linux-release-x64-try,vm-precomp-ffi-qemu-linux-release-arm-try,vm-kernel-nnbd-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184181
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
2021-02-11 18:09:05 +00:00
Alexander Aprelev 26c0de0acf [vm/deferred] Keep deferred library prefix status on isolate.
This is to ensure that deferred library status is per isolate, rather than per isolate-group.

TEST=deferred_isolate_test.dart

Issue dartbug.com/36097

Change-Id: I1fb8b8f1830aac688a078fe0a82b06c7daa856fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184040
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-11 18:00:59 +00:00
Michal Terepeta f0a4b8a6d7 Revert "[VM/runtime] Avoid comparing unfinalized type parameters."
This reverts commit 18f0599224.

Reason for revert: Breaks internal tests: http://b/179942377

Original change's description:
> [VM/runtime] Avoid comparing unfinalized type parameters.
>
> This is done by simplifying the code looking for a type argument vector overlap between class and super class.
>
> TEST=all existing ones
>
> Change-Id: I96b11deb3450de380280c693d05e9b70a01e3383
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184202
> Commit-Queue: Régis Crelier <regis@google.com>
> Reviewed-by: Alexander Markov <alexmarkov@google.com>

TBR=alexmarkov@google.com,asiva@google.com,regis@google.com

Change-Id: I2fbebbb9f429a9ee389ab891cb3775c6d88eca9f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184424
Reviewed-by: Michal Terepeta <michalt@google.com>
Commit-Queue: Michal Terepeta <michalt@google.com>
2021-02-11 12:35:46 +00:00
Regis Crelier 18f0599224 [VM/runtime] Avoid comparing unfinalized type parameters.
This is done by simplifying the code looking for a type argument vector overlap between class and super class.

TEST=all existing ones

Change-Id: I96b11deb3450de380280c693d05e9b70a01e3383
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184202
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-10 23:12:16 +00:00
Ryan Macnak 25fd0200ef [vm] Replace recanonicalization in primary snapshots with weak constant tables.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icc4a3ebf861dca5172a0cfa2cd2eea266e814d0c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181480
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-10 01:21:07 +00:00
Alexander Aprelev 6ac90525a0 [vm/concurrency] Allow for background compiler to disable field unboxing.
Issue dartbug.com/36097

TEST=existing test suite
Change-Id: Idd0c26fe9c81a733685105aa944b96ed58bc2765
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182669
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-02-09 16:27:13 +00:00
Ben Konyi fbbc99e1e9 [ VM ] Ensure Dart C APIs perform type checks when invoking functions
and constructors

Dart_New, Dart_InvokeConstructor, and Dart_InvokeClosure were not
checking the types of arguments before performing invocations.

Fixes https://github.com/dart-lang/sdk/issues/44205.

TEST=DartAPI_*_Issue44205

Change-Id: I6c06e3e8a0a88b9caa7e2212c4a2c8e43d39b4f4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182962
Commit-Queue: Ben Konyi <bkonyi@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-08 23:30:05 +00:00
Martin Kustermann 75a4fa6e23 [vm/concurrency] Move AllClassesFinalized/RemappingCids bits from Isolate to IsolateGroup
Those bits are used by parts of the VM that should be independent of the
current isolate (e.g. class finalizer, compiler).

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I25c8340304d21fdfa77fa5ab4a5f77a4884d0bba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182620
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-04 18:09:37 +00:00
Martin Kustermann 376bf1826a [vm/concurrency] Move HasAttemptedReload() bool from Isolate to IsolateGroup
As part of making the compiler as well as hot-reload independent of
a particular isolate we move this boolean to `IsolateGroup`.

For hot-reloading isolate groups there are more changes needed that
come in the future. This CL is preparation for making the compiler
independent of current isolate.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I434cf1180bc963462e2901dfa83e915df9dc1712
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182561
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-04 17:45:57 +00:00
Martin Kustermann 3020f7dca2 [vm/concurrency] Remove the unused Isolate::deoptimized_code_array_ and related
Supposedly the intention was to use this in runtime/vm/profiler.cc but
it has been seemingly unused for a while without any issues or test
failures, so this CL removes it.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Removing of code.

Change-Id: I53df5e92e146169b6d8d21c763dd7e8b8f50289a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182510
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-04 10:54:12 +00:00