Commit Graph

449 Commits

Author SHA1 Message Date
Alexander Markov f49a95fec9 [vm] Reduce Class object size in JIT mode
The following fields in Class objects

  target_instance_size_in_words_
  target_type_arguments_field_offset_in_words_
  target_next_field_offset_in_words_

are only useful when cross-compiling.

This change removes these fields if DART_PRECOMPILER is not defined.

TEST=ci
Change-Id: I45e6e60ad6d0a738e2ac3afd45328209167393ff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196222
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-04-22 22:10:06 +00:00
Alexander Markov 9af507e0f5 [vm] Auto-generate InstanceSize definitions for array objects in runtime_api
Previously, compiler::target::Foo::InstanceSize(intptr_t length)
definitions were hand-written and compiler::target::Foo::InstanceSize()
were often defined using the size of UntaggedFoo for array-like objects.
This results in an unnecessary code duplication and discrepancy
between Foo::InstanceSize() which returns 0 and
compiler::target::Foo::InstanceSize() which returns size of the header.

This change adds ARRAY_SIZEOF variant to runtime_offsets_list.h
and switches array-like objects to auto-generated InstanceSize methods.

Also, this change strengthens assertions around host and target
values of InstanceSize() and NextFieldOffset() being the same
unless DART_PRECOMPILER is defined (which may do cross-compilation).

Stronger assertions flushed out a few discrepancies between
runtime_api.h and object.h, which are also fixed in this change.

TEST=ci

Change-Id: I9f67c6a71b9300af5e656fdc70f6a3e341c1d029
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196345
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
2021-04-22 18:21:16 +00:00
Tess Strickland 6616bb4696 [vm] Mark canonical-ness in written cluster.
Instead of depending on all canonical clusters being written first,
followed by all non-canonical clusters, mark whether a cluster is
canonical or not in its written form. This allows future changes to
interleave canonical and non-canonical clusters as needed.

TEST=Existing tests, as this is purely a refactoring.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I0ab835c262e061ffd3dd93be0c8f5358c9b51fc2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195994
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-22 15:27:46 +00:00
Ryan Macnak 7e1d1963e5 [vm] Switch TypeArguments to compressed pointers.
TEST=ci
Change-Id: Ic3162ef1558fb803ab68749f1f1a4d820645ec7e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194017
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-04-21 00:06:58 +00:00
Alexander Markov 17c33d8d52 Reland "[vm] Reduce size of Class objects in AOT and PRODUCT modes"
This is a reland of 2852408881

Original change's description:
> [vm] Reduce size of Class objects in AOT and PRODUCT modes
>
> This change conditionally removes certain fields from Class objects
> in AOT and PRODUCT modes.
>
> Flutter gallery in release-sizeopt mode:
> Snapshot size -0.47% (arm64), -0.44% (arm)
> Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
>
> On a large Flutter application compiled in --dwarf-stack-traces mode:
> Heap size of Class objects -26%
> Heap size of all snapshot objects -3.6%
>
> TEST=ci
> Issue: https://github.com/dart-lang/sdk/issues/44020
> Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TEST=ci
Change-Id: Ib473a959a2c610b7a0e6f4000b2101e5256c6119
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196103
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-20 21:11:58 +00:00
Alexander Markov 867a95c50e Revert "[vm] Reduce size of Class objects in AOT and PRODUCT modes"
This reverts commit 2852408881.

Reason for revert: failures on Flutter engine tests.
Issue: https://github.com/flutter/flutter/issues/80801

Original change's description:
> [vm] Reduce size of Class objects in AOT and PRODUCT modes
>
> This change conditionally removes certain fields from Class objects
> in AOT and PRODUCT modes.
>
> Flutter gallery in release-sizeopt mode:
> Snapshot size -0.47% (arm64), -0.44% (arm)
> Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
>
> On a large Flutter application compiled in --dwarf-stack-traces mode:
> Heap size of Class objects -26%
> Heap size of all snapshot objects -3.6%
>
> TEST=ci
> Issue: https://github.com/dart-lang/sdk/issues/44020
> Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>

TBR=vegorov@google.com,rmacnak@google.com,alexmarkov@google.com,sstrickl@google.com

Change-Id: I0dbf1d5fc139f3654ce52f185b799c09bba66de7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Issue: https://github.com/dart-lang/sdk/issues/44020
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196102
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-04-20 16:54:29 +00:00
Tess Strickland 6c8f4aa190 Reland "[vm] Fix V8 snapshot profile handling of the dispatch table."
This is a reland of 5909fd111d

Does a large refactoring on the V8 snapshot profile writer
to clean things up, add more debugging support, and to fix
the problems that surfaced during the original landing.

Other changes:

Changes Serializer::CreateArtificialNodeIfNeeded() to create
artificial nodes for Code objects and immutable arrays.

Fixes CodeSerializationCluster::Trace() to only push needed parts of
discarded code objects, instead of tracing them like full code objects.

Adds test cases to v8_snapshot_profile_writer_test that exercise
the following situations (both separately and together):

* Non-symbolic stack traces are enabled and code and function objects
  are dropped when not needed at runtime.

* Creation of the dispatch table is disabled.

TEST=vm/dart{,_2}/v8_snapshot_profile_writer_test

Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

Change-Id: I8e7030267fe190079a8f68d00fe20bf7170e5719
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195513
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-20 11:17:36 +00:00
Alexander Markov 2852408881 [vm] Reduce size of Class objects in AOT and PRODUCT modes
This change conditionally removes certain fields from Class objects
in AOT and PRODUCT modes.

Flutter gallery in release-sizeopt mode:
Snapshot size -0.47% (arm64), -0.44% (arm)
Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)

On a large Flutter application compiled in --dwarf-stack-traces mode:
Heap size of Class objects -26%
Heap size of all snapshot objects -3.6%

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44020
Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-20 00:45:38 +00:00
Tess Strickland 3fdecfc169 Revert "[vm] Fix V8 snapshot profile handling of the dispatch table."
This reverts commit 5909fd111d.

Reason for revert: Ended up breaking tests on non-x64c as well, investigating.

Original change's description:
> [vm] Fix V8 snapshot profile handling of the dispatch table.
>
> Fixes https://github.com/dart-lang/sdk/issues/45702.
>
> TEST=Tests listed in the issue above.
>
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
> Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
> Reviewed-by: Daco Harkes <dacoharkes@google.com>
> Commit-Queue: Tess Strickland <sstrickl@google.com>

TBR=rmacnak@google.com,dacoharkes@google.com,sstrickl@google.com

Change-Id: I61d0eb217e7be04a0841fb9e7dddd180e31bed2a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195512
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-16 09:48:57 +00:00
Tess Strickland 5909fd111d [vm] Fix V8 snapshot profile handling of the dispatch table.
Fixes https://github.com/dart-lang/sdk/issues/45702.

TEST=Tests listed in the issue above.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try
Change-Id: Ibf5e3ccf3828c01f9dda47de360314dabe8cb8a9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195272
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-16 09:08:13 +00:00
Tess Strickland dc88076767 [vm] Handle WSRs more generally in v8 snapshot profile writing.
Instead of special-casing the current fields that may have
WeakSerializationReferences, handle WSRs appearing as elements
or properties of objects more generally. This removes existing
special casing and avoids the need for it in case of new
future uses of WSRs.

For artificial nodes being added for dropped
WeakSerializationReference targets, add them as kArtificial nodes
(not kSnapshot) that has the original offset (element) or
name (property). The replacement is added as a kSnapshot node
that has a negative offset with the same magnitude as the
artificial node (element) or ":real_<property name>" (property).
This simplifies the work done in pkg/vm_snapshot_analysis to
use the artificial nodes instead of replacement ones for
reassembling hierarchies and the like.

This CL also cleans up the old SerializerWritingObjectScope
class, both moving it to Serializer::WritingObjectScope and
allowing nesting of WritingObjectScopes with the correct
semantics.

Thanks to this, not only can we recur when under a WritingObjectScope
instead of lifting recursion outside of those scopes, but we can
also create artificial nodes for non-empty per-code object pools and
static call target tables instead of attributing their contents
as supposed elements of the Code object being written.

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test

Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: Ib945c5afcd89b1458b8be3559b6eae24048aba2f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194243
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-04-14 08:44:58 +00:00
Tess Strickland c4f0229d56 [vm] Remove no-longer-used definitions.
TEST=SDK builds.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try

Change-Id: Ibf157c08090d6e150532003806e522a15c3526d2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195064
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-13 09:52:20 +00:00
Alexander Markov 5129cff930 [vm/aot] Discard Code objects from the heap
While deserializing AOT snapshot, Code objects which do not contain
valuable information besides entry point and stack maps are discarded
and not allocated on the heap (they are replaced with
StubCode::UnknownDartCode()).

PC -> Code/CompressedStackMaps lookup is implemented using a separate
table (InstructionsTable).

Flutter gallery in release-sizeopt mode:
Heap size of snapshot objects: arm -26.89%, arm64 -27.68%

Large Flutter application in release mode with --dwarf-stack-traces:
Heap size of snapshot objects: -24.3%.
Discarded Code objects: 72.5% of all Code objects.

Issue: https://github.com/dart-lang/sdk/issues/44852.

TEST=existing tests; "--dwarf_stack_traces --no-retain_function_objects
--no-retain_code_objects" mode is enabled for a few tests.

Change-Id: I5fe3e283630c8e8f4442319d5dcae38d174dd0d8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189560
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-04-13 02:36:16 +00:00
Ryan Macnak 37616dae79 [vm] Account for the conditional nature of WeakSerializtionReference and WeakProperty references in clustered snapshots.
Fixes missing ref for WeakSerializationReference::target; fixes incorrect treatment of WeakProperties as strong references. Note isolate message snapshots still incorrectly treat WeakProperties as strong references.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/45609
Change-Id: I6dba93bc6f2e94886efde021fdf5c90a0dcb09b4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194300
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-04-08 00:54:08 +00:00
Tess Strickland bc924667f4 [vm/compiler] Add artificial nodes for dropped parent functions.
Fixes https://github.com/dart-lang/sdk/issues/45483

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test

Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-mac-release-try,pkg-win-release-try
Change-Id: I9e347bd38ec339b22d2ecefcce1b276d8ce3e3e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193303
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-04-07 04:48:46 +00:00
Ryan Macnak a6ffc74a4a [vm] Switch most Instances whose layout is determined in C++ to compressed pointers.
Instances whose layout must agree with a layout determined by Dart code are not switched: Array, GrowableObjectArray, Closure, LinkedHashMap.

TEST=ci
Change-Id: I19baf6473a29dd14f84aa1e087eee16b0e6e9d77
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193442
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-04-05 17:43:16 +00:00
Alexander Markov 2a14259fd0 [vm/compiler] Cleanup CheckedSmi* instructions
This is a final cleanup of CheckedSmiOpInstr and
CheckedSmiComparisonInstr instructions.

Their uses were removed in the following changes:

  https://dart-review.googlesource.com/c/sdk/+/191761
  https://dart-review.googlesource.com/c/sdk/+/191760
  https://dart-review.googlesource.com/c/sdk/+/193447

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852
Change-Id: Ica52ab9e2fcabab1913228818642a27a8d842f92
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/193449
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-03-31 22:10:42 +00:00
Tess Strickland e7d3a6027c [vm] Create more Function helper methods for code clarity.
Creates HasParent() as a predicate for Function objects with a non-null
parent function and makes IsLocalFunction() specific to local closures
(i.e. explicit closures with non-null parent functions).

Creates HasSavedArgumentsDescriptor() as a predicate to determine when
saved_args_desc() can be used, to avoid the caller having to do explicit
predicate checks for each dispatcher type that contains one.

Also simplifies Function::PrintName, creates different suffixes for a
SyncGenClosureMaker and its associateed SyncGenClosure, adds suffixes
for skipping more than one generated body when printing the parent name,
and changes ArgumentsDescriptor::PrintTo to match the compact syntax
Function::PrintName used for signifying the arguments descriptor for
certain dispatchers.

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test

Cq-Include-Trybots: luci.dart.try:app-kernel-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,pkg-linux-release-try
Change-Id: I0ac1226d1ffe1d81743b5c6788da170d5a4010c5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/191560
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
2021-03-23 12:21:26 +00:00
Ryan Macnak 19e5899a02 [vm] Switch most non-Instances to compressed pointers.
The following are not compressed: Code, ObjectPool, Context, CallSiteData and subclasses.

Code::instructions_ is uncompressible because the VM doesn't control the address of Instructions when they are loaded via dynamic library.

Loads from the ObjectPool are a frequent occurrence in generated code, and shrinking ObjectPool entries would be offset by longer code sequences to decompress them. Also, some ObjectPool entries contain uncompressible pointers to C functions.

Context is not compressed based on reports from earlier work on compressed pointers that compressing them is disproporately bad for performance. The earlier work used a different, more-expensive compression scheme, so we should reinvestigate this.

CallSiteData et al. are on the hot path for unoptimized code but make up a comparatively small portion of the heap size.

TEST=ci
Change-Id: I9a425fa4e12113c9fd0b51999525d32a7e589a90
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190600
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-03-18 22:52:04 +00:00
Tess Strickland 901fed103b [vm] Use the cached number of parent type args in FunctionType.
Also ensure the number of parent type arguments is appropriately
adjusted in the returned FunctionType when partially instantiating
a FunctionType, and that free parent type arguments are _not_
included in types built while reading constants.

Since the number of parent type arguments is now cached there, we
no longer need to also cache them in ClosureData. We can also remove
the parent walking in Function::NumParentTypeArguments().

Previously, a FunctionType where the component types did not use
parent type parameters was considered instantiated. This CL changes
it so that FunctionType with free parent type parameters are never
considered instantiated. This is necessary because otherwise,
when instantiating the parent type parameters, a FunctionType that
does not use its parent type parameters will be used unchanged
instead of creating a copy with fewer parent type parameters.

Because of this, places where IsInstantiated was used to simply
check for generic parent functions (namely, for implicit closure
creation) has been appropriately weakened to check
!HasGenericParent() instead.

TEST=Existing tests on CI.

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try
Change-Id: Ifb4a0a1273d8d01908cdf4ffc3c4c28a1c33ffa0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190021
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
2021-03-16 15:04:51 +00:00
Alexander Markov c73ab38596 [vm] Count heap size for canonical tables
This is a follow-up to https://dart-review.googlesource.com/c/sdk/+/185381.
That change introduced a more compact serialization of canonical sets,
but it didn't include correct counting of heap size of tables
allocated during deserialization of canonical sets.

This change adds counting of heap size of canonical tables.
The heap size benchmark numbers will regress with this change,
but this is only due to a more accurate counting, not due to a real
regression.

TEST=ci

Change-Id: I42ee3bd91f0420bf41c660d2eaa51e9901d9b198
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190580
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-03-11 17:47:41 +00:00
Vyacheslav Egorov 3ed83c398d [vm] Fix compressed pointers build
Only use unified kStringCid cluster in non-compressed variant (when
we write strings into rodata section)

TEST=x64c bots

Change-Id: Id1781801eae40f570228503fefac43a90d033c7b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-debug-x64c-try,vm-kernel-linux-debug-x64c-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190221
Auto-Submit: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
2021-03-10 09:50:18 +00:00
asiva 787d2659c9 [VM/runtime] - Minor cleanups.
TEST=ci

Change-Id: I2ff123b67e89ca79cba39e07fc2de191af9fc713
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189740
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Siva Annamalai <asiva@google.com>
2021-03-08 20:51:53 +00:00
Vyacheslav Egorov ae37ff098a [vm] Compact serialization for canonical sets
This CL changes how canonical sets for some specific types are written
into the root snapshot: instead of writing canonical set as a separate
object we reorder objects within a canonical cluster in such a way that
the order matches order of elements in the backing store of a canonical
set and then we write canonical set layout out using differential
encoding (essentially writing gaps between elements instead of writing
absolute indices).

This significantly reduces the overhead of having canonical sets in the
snapshot while maintaining fast deserialisation: for example on build
microbenchmark this brings regression in the snapshot size from 4% to
.3%.

On sizeopt benchmarks:

  flutter_gallery_app_so_gzip_size   -1.5%
  flutter_gallery_app_so_size        -4.7%
  flutter_gallery_total_heap_size   -16.2%


TEST=ci

Change-Id: I2be7fd073668e9b52098e2acda9f11d128cfda95
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-dwarf-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-linux-release-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185381
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-03-08 14:21:01 +00:00
Ryan Macnak dce0b514d4 [vm] Defer object pools and object pool entries.
This defers literals, to effectively split binary size for libraries that are more data than code, such as localizations.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I506722037cc0247b90756959e6a6f12bb5021b5c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183781
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-03-04 23:19:30 +00:00
Tess Strickland 3382bbc510 [vm] Print out canonical clusters in --print-snapshot-sizes-verbose.
TEST=Manual use of the flag.

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-nnbd-linux-debug-x64-try
Change-Id: I4c506848f97f6b9a3f1455f90872093497c853a8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188287
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-03-02 08:37:01 +00:00
Alexander Markov 26e7106937 [vm/aot] Partially remove Code objects from AOT snapshots
Code objects which do not carry useful information are
marked as 'discarded' during precompilation. Snapshot
serializer only writes instructions, compressed stack
maps and state bits for discarded Code objects.

This is a preliminary step before deserializer would be able
to omit creating instances for discarded Code objects at
runtime.

On a large app, AOT-compiled in PRODUCT mode with
--dwarf-stack-traces on arm64:
Discarded: 77% of all Code objects in snapshot.
Uncompressed AOT snapshot size -1.9%.

AOT snapshot size of Flutter gallery in release-sizeopt mode:
arm64 uncompressed -2.1% (gzip -0.6%, brotli -0.7%)
arm32 uncompressed -1.9% (gzip -0.6%, brotli -0.7%)

TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44852

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,pkg-mac-release-try
Change-Id: I159cfebbdac76cc60060fedb1d22ed86672a68a1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/187600
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-02-27 00:41:37 +00:00
Tess Strickland 348c704b25 [vm] Add heap sizes to --print-snapshot-sizes output.
TEST=Tested manually with the flag.

Bug: https://github.com/dart-lang/sdk/issues/44005
Change-Id: I553f88b2552b693801793203cffc33e096143023
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185801
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2021-02-22 09:59:30 +00:00
Vyacheslav Egorov d499f68df5 [vm] Fix vm_snapshot_analysis logic and tests
* When looking for Code node owner simply always check for synthetic
  owner (referenced through ':owner_' property) first. If it is present
  then it means 'owner_' property is invalid anyway. 25fd0200ef changed
  how WSRs are written into snapshot (instead of writing WSRs we are
  simply writing SMI CIDs) which broke previous logic.
* When building ProgramInfoNode for a Class node accommodate for the
  existence of classes which don't have 'library_' property (e.g. base
  objects like void and dynamic). Additionally, adjust how these base
  objects are written into the snapshot profile so that we preserve
  their names.
* Tweak tests diff and diff-collapse tests to pass, they were failing
  because prologue removal compiler change made some of the diffs so
  tiny that they are ignored by threshold filtering.
* Add two more tests: one that verifies that tracings flags don't have
  any impact on the snapshot size and one that checks we correct
  attribute instructions size to the function node from which these
  instructions originated. This last test is added in anticipation of
  Code object removal to make sure that vm_snapshot_analysis continues
  to work correctly.
* Adjust sorting rules for Code cluster to make it more stable between
  slightly different programs to stabilize output of
  --print-instructions-sizes-to (depends on the sorting algorithm used
  in libc, the test was failing on Mac but not on Linux).

These fixes should unblock relanding of b6dc4dad4d

Fixes #44507
Fixes flutter/flutter#76313

TEST=pkg/vm_snapshot_analysis/test/instruction_sizes_test.dart

Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-win-release-try,pkg-mac-release-try
Change-Id: Iee0221dd3a507cb4f36d530404693ebfab3b7bf2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/185826
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-21 15:48:27 +00:00
Martin Kustermann 02f65fa501 [vm/concurrency] Make clustered snapshot serializer independent of current isolate
Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Mainly refactoring, stress tests for --enable-isolate-groups with JIT will come in the future.

Change-Id: Iebb2fb8bd144cb9abd21916856a04f49f85541c0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183680
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
2021-02-12 04:32:48 +00:00
Martin Kustermann 92af830255 [vm/concurrency] Move background compiler related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`. This CL moves background
compiler and related state.

One background compiler will be shared for all isolates within a group.

Issue https://github.com/dart-lang/sdk/issues/36097
Fixes https://github.com/dart-lang/sdk/issues/44938

TEST=Mainly refactoring, stress tests for --enable-isolate-groups with JIT will come in the future.

Change-Id: I66e863651d0b37c431ce1864e505b5413850cbde
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183401
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
2021-02-12 03:40:14 +00:00
Alexander Aprelev 26c0de0acf [vm/deferred] Keep deferred library prefix status on isolate.
This is to ensure that deferred library status is per isolate, rather than per isolate-group.

TEST=deferred_isolate_test.dart

Issue dartbug.com/36097

Change-Id: I1fb8b8f1830aac688a078fe0a82b06c7daa856fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/184040
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-02-11 18:00:59 +00:00
Ryan Macnak 4b1098577e [vm] Don't use ROData serialization when using compressed pointers.
TEST=ci
Change-Id: I7ddc7783ac57d82403681b8fbc0e3004a757aabe
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/183220
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Liam Appelbe <liama@google.com>
2021-02-10 17:44:49 +00:00
Ryan Macnak 25fd0200ef [vm] Replace recanonicalization in primary snapshots with weak constant tables.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Icc4a3ebf861dca5172a0cfa2cd2eea266e814d0c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181480
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-02-10 01:21:07 +00:00
Alexander Markov 7f65bb475a [vm/snapshot] Add statistics for heap size of objects in snapshot
This statistics is printed with --print_snapshot_sizes_verbose.
It is useful in order to figure out how much memory is consumed by
program structure objects without running an application.

TEST=manual testing with --print_snapshot_sizes_verbose

Change-Id: I6eeae7b303ecaf44806deb0f532058492c1d5808
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181444
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2021-01-28 21:48:05 +00:00
asiva 27deca1457 Revert "[vm] Defer object pools." and
"[vm] Defer object pool entries in instructions bare mode."

This reverts commit 0bf8d38b3d and
             commit 63d17b0ad8.

Reason for revert : Performance regression, please see https://github.com/flutter/flutter/issues/73794

TEST=(none it is a revert).

Change-Id: I828734f00c55d5095f632b154d7f4ab27a184baf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181542
Commit-Queue: Siva Annamalai <asiva@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
2021-01-28 21:13:35 +00:00
Alexander Aprelev efa1e18428 [vm/concurrency] Allow class finalization done by background compiler
This CL removes asserts that prevented background compiler from doing class finalization.
It adds program_lock synchronization for when object properties are updated during class finalization.
It also encloses code instructions installation into safepoint scopes to ensure that no code is executed during installation.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=existing test suite

Change-Id: If297e44f51b187242eca2cdcfff066c8f6386d97
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178163
Commit-Queue: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-20 20:08:25 +00:00
Clement Skau 6313c1c47c [VM] Fixes wrong static_type_exactness_state_
This fixes an internal test crash where snapshots are compiled
with an x64 toolchain and run on ia32:

`runtime/vm/compiler/backend/il_ia32.cc: 1859: error: unimplemented code`

TEST=tools/test.py -n dartk-linux-debug-ia32 vm/dart_2/causal_stacks/

Change-Id: I9ac652e428e5463926052e0c94c430c86eadd4bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179762
Commit-Queue: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-18 12:03:29 +00:00
Ryan Macnak b1c09ecd8f [vm] Make naming more consistent when converting between handles, tagged and untagged pointers.
Currently we have things called XPtr which are not what you get from ptr().

Old world:
handle->raw() returns RawObject* (tagged)
raw_obj->ptr() returns RawObject* (untagged)

After 6fe15f6df9:
handle->raw() returns ObjectPtr
obj_ptr->ptr() returns ObjectLayout*

New world:
handle->ptr() returns ObjectPtr
obj_ptr->untag() returns UntaggedObject*

TEST=ci
Change-Id: I6c7f34014cf20737607caaf84979838300d12df2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149367
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2021-01-15 23:32:02 +00:00
Ryan Macnak 63d17b0ad8 [vm] Defer object pool entries in instructions bare mode.
dart2js+analyzer product X64 bare:
app.so            9289320 ->  8491864 (-8.58%)
app.so-2.part.so 12010936 -> 12315880 (2.53%)
app.so-3.part.so  7579704 ->  7847768 (3.53%)

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I8e695966bc11c3912a01cd525ca69be6ef8e5ea2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/171102
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-01-15 03:32:44 +00:00
Regis Crelier 16f2a1edc5 [VM/runtime] Implement constant canonical type erasure when reading from kernel.
Modify the constant verification when writing a full snapshot in agnostic mode.
The implemented changed are specified in https://github.com/dart-lang/language/pull/1389
This (partially) fixes issue https://github.com/dart-lang/sdk/issues/44643
More work may be required for type parameter names of type literals.

TEST=new tests added in https://dart-review.googlesource.com/c/sdk/+/178929

Change-Id: Ie42bbe482763f8e737c3f5bd26e7752511e7398e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178927
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2021-01-13 20:26:18 +00:00
Ryan Macnak 784e69a6ad [vm, gc] Ensure exclusive access to the heap during deserialization.
TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I37d2b83802a11b9cf5ffc2d7d4852daa18941006
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/178720
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2021-01-12 17:42:41 +00:00
Ryan Macnak 0bf8d38b3d [vm] Defer object pools.
This is a step toward deferring literals, to effectively split binary size for libraries that are more data than code, such as localizations. Bare instructions mode still has all constants in the root snapshot because it has one global ObjectPool.

dart2js+analyzer product X64 non-bare:
app.so           10072728 ->  8479824 (-15.8%)
app.so-2.part.so 14074512 -> 15016976 (6.69%)
app.so-3.part.so  9225936 ->  9888984 (7.18%)

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: If61f6156e3fc2dd539331335b3bf0458102b7c75
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168991
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2021-01-08 22:11:03 +00:00
Regis Crelier b3bca1821c [VM/runtime] Decouple representation of function signatures from Function objects.
This fixes issue https://github.com/dart-lang/sdk/issues/43088

This CL improves on https://dart-review.googlesource.com/c/sdk/+/166920 by keeping the names of type parameters.

See go/dart-vm-signatures for motivation for this change.

TEST=existing suites

Change-Id: Ia82f1d322f72d07b2fb253cf914f9b868025c71e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/176981
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-07 23:45:37 +00:00
Martin Kustermann 0f76981bb1 [vm/concurrency] Change references to class_table/object_store/heap from Isolate to IsolateGroup
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.

The class_table and object_store were already moved to `IsolateGroup`.

This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.

Issue https://github.com/dart-lang/sdk/issues/36097

TEST=Pure refactoring - relying on existing test coverage.

Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 15:22:11 +00:00
Martin Kustermann 897b1a2121 [vm/concurrency] Move osr/field-guard related state from Isolate to IsolateGroup
As part of making the compiler independent of `Isolate` we have to move
various state from `Isolate` to `IsolateGroup`.

This CL moves osr/field-guard related state.

TEST=Pure refactoring - relying on existing test coverage.

Issue https://github.com/dart-lang/sdk/issues/36097

Change-Id: Id943891476f892f64f2a74952681270d012a4693
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177125
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2021-01-06 13:32:10 +00:00
Tess Strickland 100e12b6cd [vm/compiler] Copy inlined IDs with token positions.
When creating new instructions that inherit a token position that
represents a source location from another instruction, the inheriting
instruction must also have the same inlining ID in order for the source
position represented by the token position to be looked up in the
correct script.

Force this by wrapping both in a single InstructionSource struct which
is taken by instructions which take token positions instead of just a
token position. That way, it's more work to manually transfer the token
position separately from the inlining ID of the instruction than doing
the right thing of transfering both at once.

To ensure this information is kept consistent, we pass InstructionSource
structs through the FlowGraphCompiler all the way down to the
CodeSourceMapBuilder.

This CL also makes the following changes:

* Cache the upper bound of source positions in scripts and use it to add
  a check for if a given real token position is valid for the script
  without iterating over the line starts data for each token position.

* Start inlining intervals appropriately when adding descriptor and
  null check information to the code source map.

Code size changes are minimal on Flutter gallery in release mode
(<0.05% decrease).

TEST=Existing tests on trybots, with manual checking with
--check-token-positions that previous errors are now removed.

Bug: https://github.com/dart-lang/sdk/issues/44436

Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-nnbd-linux-debug-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-nnbd-linux-debug-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-linux-release-x64-try,vm-kernel-nnbd-linux-release-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-nnbd-linux-release-x64-try,vm-kernel-linux-product-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-linux-release-simarm64-try,vm-kernel-linux-release-simarm-try,vm-kernel-linux-release-ia32-try
Change-Id: I23ced262cb4e9fe9d81356f409e7e8d220d63ee0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173967
Reviewed-by: Régis Crelier <regis@google.com>
2020-12-18 00:42:14 +00:00
Ryan Macnak 947c318ec7 [vm] Add timeline events for deserialization phases.
TEST=n/a
Change-Id: I5f534784db98038ed6abe55873b9c56c434c462c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175420
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2020-12-10 18:30:48 +00:00
Ryan Macnak abbb51e1ff [vm] Clean up metadata tracking.
Remove the use of Fields, which have since the metadata mechanism was introduced come to require global registration.

Retain some sloppiness matching of declarations by name instead of identity for the sake of hot reload.

TEST=ci
Change-Id: Ifa4cc7724096c606f994b057b7838e124ceb3626
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175141
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-12-07 19:43:48 +00:00
Ryan Macnak 28fe36c637 [vm, compiler] Recognize WeakProperty's accessors.
The layout of WeakProperty is defined in C++ instead of Dart. Add missing recognition of its accessors so the compiler can use direct field access instead of always going through natives. Compare LinkedHashMap.

Change the sentinel for WeakProperty lists to object null so that no special constructor is needed.

TEST=ci
Bug: https://github.com/dart-lang/sdk/issues/44333
Change-Id: I8686383053c2c345c972b6615b514a1381e79fda
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/175001
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-12-04 22:25:12 +00:00