Commit Graph

267 Commits

Author SHA1 Message Date
Ryan Macnak f54bb55244 Reland "[vm, service, observatory] Add ability to retrieve the set of all instances of a class, implementation hierarchy or interface hierarchy as an array."
Filter Class::direct_subclasses/implementors_ to remove tree-shaken classes.

TEST=ci, manual UI interaction
Bug: https://github.com/dart-lang/sdk/issues/45564
Change-Id: Ia2a46e9701745ce88dffa95a86b7756f3ab0921d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194110
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>
2021-04-14 19:40:20 +00:00
Tess Strickland 0f8861e3ba [vm] Standardize hashing instance methods in C++.
In the codebase, we have several different interfaces for instance
methods that return a hash:

- uword Hash() const;
- intptr_t Hash() const;
- uint32_t Hash() const;
- intptr_t Hashcode() const;

This CL standardizes on `uword Hash() const` and adjusts any related
functions to match.

TEST=Existing test suites, as this is an internal refactoring.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: If2cbce57f3fae0f0d24031b6e324f0323c965f41
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195067
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-04-14 10:04:48 +00:00
Alexander Markov 5129cff930 [vm/aot] Discard Code objects from the heap
While deserializing AOT snapshot, Code objects which do not contain
valuable information besides entry point and stack maps are discarded
and not allocated on the heap (they are replaced with
StubCode::UnknownDartCode()).

PC -> Code/CompressedStackMaps lookup is implemented using a separate
table (InstructionsTable).

Flutter gallery in release-sizeopt mode:
Heap size of snapshot objects: arm -26.89%, arm64 -27.68%

Large Flutter application in release mode with --dwarf-stack-traces:
Heap size of snapshot objects: -24.3%.
Discarded Code objects: 72.5% of all Code objects.

Issue: https://github.com/dart-lang/sdk/issues/44852.

TEST=existing tests; "--dwarf_stack_traces --no-retain_function_objects
--no-retain_code_objects" mode is enabled for a few tests.

Change-Id: I5fe3e283630c8e8f4442319d5dcae38d174dd0d8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189560
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-13 02:36:16 +00:00
Ben Konyi 2a62fc49a3 [ VM / gen_snapshot ] Include resolved script URLs in non-PRODUCT AOT snapshots
Required for https://github.com/dart-lang/sdk/issues/37553

TEST=runtime/tests/vm/dart/gen_snapshot_include_resolved_urls_test.dart

Change-Id: I0a2f6293566e5e5436df92d21e6bd38bb58c7654
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193520
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Ben Konyi <bkonyi@google.com>
2021-04-06 22:30:48 +00:00
Alexander Markov a38a2dd343 [vm/aot] Fix retaining of targets of FFI callbacks in precompiler
Account for the fact that a Function could be only reachable via
FfiTrampolineData in the FFI trampoline function.

TEST=ffi_2/function_callbacks_test
Fixes https://github.com/dart-lang/sdk/issues/45510

Change-Id: I2db81c7730ae48d1f1355aa236339e54562ee6c4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193893
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-04-06 18:33:17 +00:00
Alexander Aprelev abdca81fa0 Revert "[vm, service, observatory] Add ability to retrieve the set of all instances of a class, implementation hierarchy or interface hierarchy as an array."
This reverts commit 170bda2b74 as it breaks ffi test.

See https://github.com/dart-lang/sdk/issues/45564

TEST=rollback

Change-Id: Ie27cfa7c3bc093c1e8d46722942d1f2e5e3cb77e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193882
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2021-04-02 16:46:11 +00:00
Ryan Macnak 170bda2b74 [vm, service, observatory] Add ability to retrieve the set of all instances of a class, implementation hierarchy or interface hierarchy as an array.
Allows developers to perform arbitrary filtering or analysis of instances, such as finding the largest strings, degreeses of duplication, histograms of various properties, etc.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/44479
Change-Id: I0b4005b5778038945e5f1b2d7858806c8e0dbbff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176381
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
2021-04-01 20:53:22 +00:00
Tess Strickland bf74cad477 [vm/compiler] Change when parents of local functions are retained.
Now they are only retained if one of the following is true:

* The parent is Future.wait or Future.timeout.
* The local function is a generated body of an async/async*/sync*
  function.
* Symbolic stack traces are enabled.

When a local closure's parent is dropped, the parent name will
be printed as `<optimized out>` in places like exceptions.

Changes in Flutter gallery in release-sizeopt mode:

* ARM7: ROData -1.41%, Isolate snapshot -2.35%, Total snapshot -0.54%,
        Isolate heap -1.38%, Total heap -1.38%
* ARM8: ROData -1.55%, Isolate snapshot -2.34%, Total snapshot -0.65%,
        Isolate heap -1.41%, Total heap -1.41%

TEST=Existing tests on CI.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try, vm-kernel-precomp-dwarf-linux-product-x64-try
Change-Id: I9d05f9e0e30e9f428eff16a15b0f1eeb974419c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190023
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-25 13:47:03 +00:00
Alexander Markov 16beb34813 [vm/aot] Remove special cases of dynamic function lookups from precompiler
Flutter gallery AOT snapshot size
release: no change;
release-sizeopt: arm -0.24%, arm64 -0.25%.

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: I847bfc311552bb4e80222e0040b50b4766a18dbe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190783
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-03-15 17:53:23 +00:00
Tess Strickland dc62c9c3c2 [vm/compiler] Write retained reasons JSON to a file.
Instead of being part of the --trace-precompiler output, there is
is a new flag --write-retained-reasons-to=<path>.

To allow for non-functions to have reasons for retention, the
JSON objects in the array now have one additional field:

* type: a string, the C++ type of the Dart object.

This CL also wraps all the machinery in a RetainedReasonsWriter
class instead of putting it in the Precompiler interface.

TEST=vm/dart{,2}/use_trace_precompiler_flag

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Ifa40ac64fdcbdad5114499425c7f8b08e327fa76
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190525
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-15 08:37:40 +00:00
Alexander Markov 11d6203130 [vm/aot] Fix retaining of exception types and auxiliary functions
There are two fixes:

1) If a function was compiled but not retained, precompiler
should still trace through types used in exception handlers.

2) Certain auxiliary functions such as implicit closure functions
and field initializers were added to 'possibly_retained_functions_'
set but were not traced through AddTypesOf and were not added to
'functions_to_retain_' set.

Flutter gallery AOT snapshot size
release - no changes
release-sizeopt arm +0.13%, arm64 +0.14%

TEST=ci
Fixes https://github.com/dart-lang/sdk/issues/45179
Issue https://github.com/dart-lang/sdk/issues/44852

Change-Id: Ide9a7e82a7bb9098dc90bc41a76e527e7a5cb603
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190781
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-03-13 00:30:38 +00:00
Tess Strickland 5b8b55dfdd [vm/compiler] Fix size regression with symbolic stack traces enabled.
TEST=Manually checked via Golem.

Change-Id: I4986d99afd9dbcc5f63292baa89ec9053b0f37a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190524
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-03-11 11:16:41 +00:00
Vyacheslav Egorov a942c10bff [vm] Remove IL serializer.
Unfortunately, we never capitalized on having it (e.g. original plan was
to use it for writing tests, among other things) and continuing to
maintain it does not make sense.

TEST=ci

Change-Id: I02b2c35149298a295aa0de75916d48434d4a3ea9
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190220
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-10 10:02:58 +00:00
Tess Strickland 7ba7d65e5b [vm/compiler] Print retained functions when tracing precompiler.
To get this information, enable --trace-precompiler.

There are two version of the output: a human readable version
and a JSON version (preceded by 'JSON for function decisions:')
for easier tabulation. The JSON version is an array of objects,
where the fields of the object are as follows:

* name: a string, the function name
* kind: a string, the function kind
* retained: a boolean, whether the function was retained
* reasons: an array of strings, reasons for retaining
  (only for retained objects)

TEST=vm/dart{,_2}/use_trace_precompiler_flag_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: I9b006a49e1410a832950a1bd29a0abaabe9431f9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190000
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-10 08:21:58 +00:00
Alexander Aprelev c22205c14e [vm/concurrency] Ensure write program_lock is held while updating class state via set_state.
Fix tests and AllocateObject.

Hopefully addresses flaky https://github.com/dart-lang/sdk/issues/44940

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=ci and standalone_2/out_of_memory_recovery_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try

Change-Id: Ife775f7a0bcdd343b887ba1bf912e07b2a8430c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188362
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-03-02 16:28:12 +00:00
Alexander Markov 26e7106937 [vm/aot] Partially remove Code objects from AOT snapshots
Code objects which do not carry useful information are
marked as 'discarded' during precompilation. Snapshot
serializer only writes instructions, compressed stack
maps and state bits for discarded Code objects.

This is a preliminary step before deserializer would be able
to omit creating instances for discarded Code objects at
runtime.

On a large app, AOT-compiled in PRODUCT mode with
--dwarf-stack-traces on arm64:
Discarded: 77% of all Code objects in snapshot.
Uncompressed AOT snapshot size -1.9%.

AOT snapshot size of Flutter gallery in release-sizeopt mode:
arm64 uncompressed -2.1% (gzip -0.6%, brotli -0.7%)
arm32 uncompressed -1.9% (gzip -0.6%, brotli -0.7%)

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try
Change-Id: I159cfebbdac76cc60060fedb1d22ed86672a68a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187600
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-27 00:41:37 +00:00
Ryan Macnak dd83f15826 [vm] Account for compressed pointers in pointer visitors.
TEST=ci
Change-Id: I033bb30e86dfc66ddeb7316a625e3f8925528a19
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183784
Reviewed-by: Liam Appelbe <liama@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-02-16 18:27:39 +00:00
Martin Kustermann c95714d069 [vm/concurrency] Add NoActiveIsolateScope to precompiler & bg compiler
With --enable-isolate-groups we will start sharing JITed code. The JITed
code - which will be executed by all isolates - should therefore not
depend on a particular isolate's state, flags or any other
isolate-specific information.

To avoid accidental bugs where the compiler starts using
isolate-specific information, we ensure the compiler runs without a
current isolate.

This CL doesn't add NoActiveIsolateScope because on unoptimized compiles
of natives, the compiler calls out to embedder for eager native
resolution of native functions. This API forces embedder to callback
into VM atm.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Adds more checking.

Change-Id: I6c86674c010b74c15855652415706dbaa8749d33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183692
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-12 05:59:05 +00:00
Ryan Macnak 25fd0200ef [vm] Replace recanonicalization in primary snapshots with weak constant tables.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icc4a3ebf861dca5172a0cfa2cd2eea266e814d0c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181480
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-10 01:21:07 +00:00
Martin Kustermann 75a4fa6e23 [vm/concurrency] Move AllClassesFinalized/RemappingCids bits from Isolate to IsolateGroup
Those bits are used by parts of the VM that should be independent of the
current isolate (e.g. class finalizer, compiler).

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Refactoring of existing code.

Change-Id: I25c8340304d21fdfa77fa5ab4a5f77a4884d0bba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/182620
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-04 18:09:37 +00:00
Alexander Markov 97e2f3ddc5 [vm/aot] Avoid retaining inlined functions with --dwarf_stack_traces
When --dwarf-stack-trace option is used, we don't use nor write inlined
functions information to the snapshot. However, we were retaining
Function objects corresponding to inlined functions.

On a large app, AOT-compiled in PRODUCT mode with
--dwarf-stack-traces on arm64:

Function objects in AOT snapshot -19.4%
Heap size used by objects in snapshot -4.45%
Total AOT snapshot size -2.35%

Flutter gallery AOT snapshot size
(release mode with --dwarf-stack-traces):
arm64 -1.1% (gzip -1.3%, brotli -1.4%)
arm -0.95% (gzip -1.2%, brotli -1.4%)

TEST=ci

Change-Id: Id2d7a2d6b6bff1a5929be97477b0e17a9afae475
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181700
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-02-01 18:26:29 +00:00
Alexander Aprelev efa1e18428 [vm/concurrency] Allow class finalization done by background compiler
This CL removes asserts that prevented background compiler from doing class finalization.
It adds program_lock synchronization for when object properties are updated during class finalization.
It also encloses code instructions installation into safepoint scopes to ensure that no code is executed during installation.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=existing test suite

Change-Id: If297e44f51b187242eca2cdcfff066c8f6386d97
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178163
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-20 20:08:25 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00
Ryan Macnak b02cd95263 [vm, compiler] Fix per-Code ObjectPools to track all the objects used by a Code in bare instructions mode.
Fix disassembly in bare instructions mode to query the global object pool.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I3ad66a8cd7d9ee19c848b68ed584d58dca6bf583
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179300
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-01-14 23:06:56 +00:00
Regis Crelier b3bca1821c [VM/runtime] Decouple representation of function signatures from Function objects.
This fixes issue https://github.com/dart-lang/sdk/issues/43088

This CL improves on https://dart-review.googlesource.com/c/sdk/+/166920 by keeping the names of type parameters.

See go/dart-vm-signatures for motivation for this change.

TEST=existing suites

Change-Id: Ia82f1d322f72d07b2fb253cf914f9b868025c71e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176981
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-07 23:45:37 +00:00
Martin Kustermann 51674dea10 [vm/concurrency] Make static-const fields that are purely used by runtime operate on initial field table
For normal Dart code a usage of a constant (e.g. from a static const
field) is "inlined": It is represented as a `ConstantExpression` AST
node with a reference to the "evaluated" `Constant` node.
=> Static const fields are never accessed by generated code.

We would therefore not need to represent static const fields in
Kernel/VM anymore. Though there are a few exceptions where the VM's
runtime code needs access to them, which is why they are kept.

The VM runtime code doesn't need an isolate-specific slot for such
static const fields (it's a waste).

=> This CL changes the runtime code that accesses such fields to operate
   on the `IsolateGroup`s initial field table instead.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I5994811a1c1016b2f523b318b698c9e052a98c29
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177709
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-07 11:44:58 +00:00
Martin Kustermann 5cd0770c9c [vm/concurrency] Move closure function related state from Isolate to ClosureFunctionsCache
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup` (or to another place).

The access of `ObjectStore::closure_functions()` was spread out over our
codebase, including from `Isolate`. This CL moves it to a centralized
place - ClosureFunctionsCache.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I3b282623f07614cdc2c2a863bd5b9d6c7748d36f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177130
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-07 09:41:27 +00:00
Martin Kustermann 0f76981bb1 [vm/concurrency] Change references to class_table/object_store/heap from Isolate to IsolateGroup
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.

The class_table and object_store were already moved to `IsolateGroup`.

This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 15:22:11 +00:00
Martin Kustermann 18971e409f [vm/concurrency] Move obfuscation related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`.

This CL moves obfuscation related state.

TEST=Pure refactoring - relying on existing test coverage.

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: I4b0a74bff9abc200cf8f4ce76b3e1d3d688e664a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177122
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 11:12:40 +00:00
Tess Strickland dadf3f56e8 [vm/compiler] Add flag for checking token positions.
When this flag is enabled, then the validity of real token positions are
checked in three places:

* FlowGraphChecker::VisitInstruction
* CodeSourceMapBuilder::BufferChangeLocation
* DescriptorList::AddDescriptor

The validity check ensures that, if the flow graph has an inlining
ID, that the instruction has an inlining ID and that any real token
position is a valid source offset for the appropriate script (found
via the inlining ID) by attempting to look up line and column.

This flag is disabled by default until existing issues with invalid
inlining IDs and token positions are resolved.

TEST=Enabled the flag manually to check that it caught invalid token
positions.

Bug: https://github.com/dart-lang/sdk/issues/44436

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try
Change-Id: I86439db3661d976ac4ea608d132e3277425615d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175641
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-17 12:19:43 +00:00
Alexander Aprelev fce355b8b0 [vm/fields] Always clone fields when running an optimizing compiler, even when it runs on the mutator thread.
Attempts to run class finalizaton on background thread revealed the problem where during optimization(which could run on mutator thread too) we might need to finalize previously unseen class, load annotatons for it, load and set some constant fields. This could result in updating Fields guarded_cid, requires check during installation of newly produced optimized code. Because mutator thread is currently exempt from cloning and validating fields guards, that could result in invalid optimized code installation.

This CL ensures that fields are always cloned in optimizing compiler even when it runs on mutator thread.

Ability to run multiple optimizing compilers is also needed for dartbug.com/36097.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=existing test suite with ltw isolate groups enabled, https://dart-review.googlesource.com/c/sdk/+/176300 mocked up test(still to be improved).

Change-Id: I608f4abff30b468cb9d0ab270221a0ba72c32470
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175341
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-17 00:22:13 +00:00
Martin Kustermann a97b43f4ec [vm/concurrency] Allow concurrent evaluation (and caching) of metadata
If multiple threads want to obtain metadata they might both evaluate and
possibly cache the result of evaluation. Since the kernel loader
initialy adds metadata mappings (the value being kernel offsets) we use
the existing `IG->program_lock()` which the kernel loader already holds.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Tests using --enable-isolate-groups with JIT sharing.

Change-Id: I3b618f196abe9f7b584e4135a72a0017399ff2d5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175301
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-12-08 14:56:04 +00:00
Ryan Macnak abbb51e1ff [vm] Clean up metadata tracking.
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.

Retain some sloppiness matching of declarations by name instead of identity for the sake of hot reload.

TEST=ci
Change-Id: Ifa4cc7724096c606f994b057b7838e124ceb3626
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175141
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-07 19:43:48 +00:00
Alexander Markov 4ab36120d7 [vm] Fix finalization of classes when instances are allocated
Classes should be allocate-finalized before instance can be allocated
in order to properly update class hierarchy information and invalidate
code which was speculatively optimized.

TEST=runtime/tests/vm/dart/regress_44342_test.dart

Fixes https://github.com/dart-lang/sdk/issues/44342

Change-Id: I52a415f4cf25010eb23be339266a91ae6b7d7261
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175106
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-12-04 23:54:37 +00:00
Clement Skau c5eb666bcf Revert "[vm] Track metadata by identity instead of name."
This reverts commit bfd382820f.

Reason for revert: Breaks several lib_2/mirrors/ tests

Original change's description:
> [vm] Track metadata by identity instead of name.
>
> Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.
>
> TEST=ci
> Change-Id: Ic2e0c4408557514507fd732e0cb19b022b2539c7
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161631
> Commit-Queue: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,aam@google.com,rmacnak@google.com

Change-Id: Icf75e91872d9c85ce9be29a9a178d72ca024c98b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/174920
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2020-12-03 08:03:11 +00:00
Ryan Macnak bfd382820f [vm] Track metadata by identity instead of name.
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.

TEST=ci
Change-Id: Ic2e0c4408557514507fd732e0cb19b022b2539c7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161631
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-02 23:43:07 +00:00
Clement Skau 2d36c85ff8 [vm] Removes support for --causal-async-stacks
All existing embedders have been opted into --lazy-async-stacks, the
VM also uses it as it's default in all configurations.

After this CL, any user of --causal-async-stacks will get an error
message when trying to use it.
=> In any such case, please simply remove the flag.

TEST=Exhaustive CQ.

Bug: https://github.com/dart-lang/sdk/issues/37668
Change-Id: Ia440afcf2dba464aa8b8cf381b93bbac8eb9f8dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172564
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-01 07:27:14 +00:00
Martin Kustermann c8ebfcd739 [vm] Remove unused code related to RedirectionData
Since the transition from Source based to Kernel based frontend to the
Dart VM, we have not supported callable redirecting factories. This has
caused all removed code in this CL to be unused and untested!

The support for reifing redirection related information would mainly be
used by embedder / dart:mirrors. Right now its not used by anyone and
the kernel support was never implemented.

I suggest we remove the unused and untested code until a decision has
been made that we actually want to support it - in which case it can be
properly implemented, possibly by re-using some of the deleted code in
this CL.

TEST=CL removes unused code, no test result changes.

Change-Id: I7ed45c85b4efcc1e81ce44cbe08bb555e52101b9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166853
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-11-17 08:51:30 +00:00
Alexander Markov 7588ed86de Remove bytecode mode from the VM
Change-Id: Ief167b7ffc128105a03cc225ab750234c9a6a7a0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/169147
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2020-10-28 17:42:35 +00:00
Alexander Aprelev 20b8b1c0cf [vm/concurrency] Introduce a read lock acquisition of program_lock.
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I36e661cd448327e7391ed854af4173c31e523430
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159645
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-27 23:51:57 +00:00
Alexander Aprelev af8413d8e9 [vm/concurrency] Add write lock acquisiton when updating Class direct_subclasses.
Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I8cdac8cf755e8944bb2991ef35d96913da6f511e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/169161
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2020-10-27 18:35:35 +00:00
Tess Strickland d9685d1c4c [vm/compiler] Move type argument type checks out of closure bodies.
These checks are now performed in the dynamic closure call dispatcher.
To avoid having to create type argument vectors (TAVs) for default type
arguments at runtime, we cache a compile-time created TAV in the
ClosureData for the closure function which is retrieved by the
dispatcher when needed.

We also keep an associated packed field of information that can also be
determined at compile time:

* Whether the cached TAV needs instantiation or can share its
  instantiator or function type arguments.

* The number of parent type parameters.

The former allows the generated IL to keep the invariant that the
InstantiateTypeArguments instruction (and the runtime entry it calls) is
only used for uninstantiated TAVs.

Also changes the destination name to an Value input for the
AssertSubtype instruction and adds handling for non-constant types and
names in that instruction's backend.

Additional changes:

* Adds new slots for ClosureData, Function, and TypeArguments.

* Adds a new kUnboxedUint8 representation (needed for
  TypeParameterLayout::flags_, which is of type uint8_t).

  * Extends LoadField to handle uint8_t unboxed native fields.

  * Adds BoxUint8 for boxing unboxed uint8_t values.

Code size impact on Flutter gallery in release mode:

* arm7: total +0.08%, vmisolate: +0.56%, isolate: +0.42%,
        readonly: -0.04%, instructions: -0.03%

* arm8: total +0.12%, vmisolate: +0.56%, isolate: +0.42%,
        readonly: -0.002%, instructions: +0.03%

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-ia32-try,vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-ia32-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-simarm_x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-simarm64-try,vm-kernel-nnbd-linux-release-simarm-try,vm-kernel-nnbd-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-nnbd-linux-release-simarm64-try
Bug: https://github.com/dart-lang/sdk/issues/40813
Change-Id: I5a7de27a17e3119e27752bd0d10e1c6bc1b52a16
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158844
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2020-10-26 18:48:18 +00:00
Alexander Aprelev feeb7ad461 [vm/concurrency] Add write-lock assertion to Class::SetFields protecting Class fields.
This is follow-up to https://dart-review.googlesource.com/c/sdk/+/168140 which added write-lock to protect Class functions.

Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: Idb2070d005c44509fc6701f2eb297b90e7ded581
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168772
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
2020-10-22 19:07:20 +00:00
Alexander Aprelev 37abbb2e3d [vm/concurrency] Introduce program_lock to guard program structure changes.
This CL adds acquisiton of write lock and check for whether write lock is held when updating class functions, primarily populated during class finalization. Read locks will be added in successive CLs, so is extending of the locks coverage to all aspects of program structure changes.

Bug: https://github.com/dart-lang/sdk/issues/36097
Change-Id: I3dba6bc23db4e45599a20717226210f12e7fd2b3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168140
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-10-22 13:08:10 +00:00
Ryan Macnak f54c0a3b9b [vm] Prepare for canonicalizing general constants on load.
- Use null instead of empty array for the initial constants set, as fields dropped during snapshot are populated with null.
 - Remove assumption that canonicalization will always be bottom-up. Deserialization clusters will cause canonicalization to be ordered by class instead of by depth.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ia3ab68281c783299e682f6c60a744c320c19e981
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165160
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2020-09-30 22:29:05 +00:00
Ryan Macnak a4637cd81d [vm] Build canonical string and type tables after reading snapshots.
Prevents snapshots from including canonical strings and types with no real uses and only referenced from the canonical tables.

Prep for constants not all being in the root snapshot during snapshot splitting, which is why we fill the table during load instead of rebuilding tables during snapshot writing.

Removes the size of the tables from the snapshot.
flutter_gallery_app_so_size (Pixel 2)        -1.618% 7967664 8098736
flutter_gallery_app_so_gzip_size (Pixel 2)   -2.724% 2980344 3063788
flutter_gallery_app_so_brotli_size (Pixel 2) -3.530% 2561824 2655566

Bug: https://github.com/dart-lang/sdk/issues/41974
Bug: https://github.com/dart-lang/sdk/issues/43319
Change-Id: I4f3b62da1cea9ee17caa965e809f16856518c9a6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164102
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-09-23 19:13:29 +00:00
Ryan Macnak d8950da048 [vm] Fix --no-retain-function-objects when there are calls through Code.
With --no-use-bare-instructions or with splitting, not all calls are pc-relative. We must replace the Function with its Code in the static call table before detaching the Function's code as part of removing Function objects.

Also handle WeakSerializationReferences in AssignLoadingUnitsCodeVisitor.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I29f9258c240f5a2b1b8dca52f74146dfe44d6401
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158165
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2020-08-13 17:01:24 +00:00
Ryan Macnak c5a94db091 [vm] Initial implementation of deferred loading.
(Assignment of libraries to loading units is already done in the kernel generation step.)

After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.

All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.

Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.

Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.

Embedder's guide:

At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.

At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-07-17 19:29:47 +00:00
Martin Kustermann f955b82a92 [vm/compiler] Support unboxing of return values of getters
This CL adds support for unboxing of return values of getters and
therefore completes the work on unboxing support for normal members
(closures are still missing).

As with existing unboxing support for methods and setters, we utilize
TFA information to proof a getter will always return non-nullable
int/double.

We then make such a getter return unboxed int/double. If there are
dynamic calls to the getter a dyn:get:* forwarder will be created
which performs boxing of the return value.

Overall this reduces RX by eliminating BoxInt64 instructions. It
sometimes increases metadata due to more dyn:get:* function objects.

=> On our main size benchmark targets show no significant change.

Issue https://github.com/dart-lang/sdk/issues/40876

Change-Id: If7450ef7e5e3fc9c2e0eaa6b86ffa817699a7e17
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154329
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2020-07-16 20:26:04 +00:00
Martin Kustermann 06183779d7 [vm/compiler] Make the compiler use dyn:* selectors when calling getters
We already make the callsites use dyn:* selectors for normal method
calls as well as setters, this is doing the same thing for getters.

Once callsites use dyn:get:* various pieces in runtime need to be
adjusted to accomodate for that (e.g. NSM handling, etc).

A follow-up CL will then start actually generating dyn:* getters in
certain situations.

Issue https://github.com/dart-lang/sdk/issues/40876

Change-Id: If219603bc0b8eb119edd08b211a8897d21ec0fb6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/154320
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2020-07-15 12:38:04 +00:00