Commit Graph

58 Commits

Author SHA1 Message Date
Johnni Winther 1aa6f00107 [kernel] Refactor CanonicalName/Reference integration
The CL is a step towards have a more restricted and wellstructured
handled of references and canonical names.

The CL moves Reference to canonical_name.dart and makes
CanonicalName.reference private, and replaces CanonicalName.getReference
with a 'reference' getter.

It also removes NamedNode.canonicalName, Field.getterCanonicalName and
Field.setterCanonicalName so that these can only be accessed through the
corresponding reference. This is to reduce the reliance on the
canonical names which, ideally, should only be part of serialization and
deserialization.

TEST=existing

Change-Id: I955fb7d52d4e112d8741f7c12dcf38b74ae0c91a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190442
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-03-10 15:39:28 +00:00
Johnni Winther 7d64d528ce [kernel] Migrate ast_to_binary.dart
TEST=existing

Change-Id: If080190b80776a7613f983b28c2e800727599328
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/189201
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-03-09 09:29:56 +00:00
Johnni Winther af4757dd0b [kernel] Migrate ast_from_binary.dart
TEST=existing tests

Change-Id: Ie6b3baa9233e2f02179d0eaf31224e628436495e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/188282
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2021-03-02 12:11:12 +00:00
Johnni Winther 79f36c9d49 [kernel] Rename readUInt to readUInt30 to match corresponding write-method.
TEST=Refactoring

Change-Id: I455c498065785909fb175a0b5fa82bd9d5b38f1f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/173524
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2020-11-24 08:50:38 +00:00
Jens Johansen 9ec87c2da0 [kernel] Field has 2 references
Change-Id: I9c946119e55256b9fbcbf3076ea008aaa1312e86
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168949
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-11-03 16:02:16 +00:00
Alexander Markov 061848e97e Cleanup misc remaining references to bytecode
Change-Id: I37c2dfb29a5bc4faf1478fb31623749cb8f67d67
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/169502
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2020-10-28 21:51:20 +00:00
Alexander Markov 1615c04c63 Remove bytecode generator
Change-Id: Ic5fd2ab67c10ae642ee277a5acd9b4084e059861
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167762
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
2020-10-23 19:30:43 +00:00
Ryan Macnak 6cc943957b [vm] Compute a partitioning of the program into loading units.
The partitioning has the property that for every reference from loading unit S to unit T, either
 - the reference is through a deferred prefix, or
 - T is guarenteed to be loaded before S

The partitioning is constructed by cutting the dominator tree of the library import graph at deferred imports. It is suboptimal if there are deferred imports to multiple libraries in a connected component, but the expected use case involves loading units with a single deferred entry.

Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: I3c49044ae19112525f197b077b36e4ce874b81ac
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/151470
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-06-19 21:35:47 +00:00
Johnni Winther 7fce125300 [cfe,kernel] Add AstPrinter
This CL adds a Node.toText method together with an AstPrinter. These
facility and better toString implementation on AST nodes while allowing
for toString independent printing of AST to use in testing. This also
add support for an integrated toString of custom/internal nodes.

Some work is still needed in bringing the toString implementation on
all nodes to the old quality, and not all internal nodes have
customized textual representations yet. This work is left for future
CLs.

Change-Id: Ib0bf8a0bc02f489dfacdc8aa5f96da9c52f26058
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150923
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2020-06-16 17:15:10 +00:00
Aske Simon Christensen 5b78cb8fb5 [vm/aot] Handle tearoffs in dispatch table.
Adds another interface target to instance calls, pointing to the torn
off method for tearoff getter calls. This is then used to direct the
call to the method extractor for the method.

Increases the fraction of dynamically dispatched instance calls in
Flutter Gallery that are handled by the dispatch table from 95.8% to
98.4%. The remaining 92 IC calls are all on static receiver type
dynamic or Function (34 function calls, 22 method calls, 21 operator
calls, 14 getter calls and 1 setter call).

Change-Id: I0bde1a8271c8563f4631534bd39d1254cd779380
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142155
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-04-20 12:42:46 +00:00
Martin Kustermann c83c9b40c5 [vm] Implement toString() for unboxing metadata, only write unboxing metadata if not fully boxed (which is default)
Change-Id: Idf9911551609e063b6ad22f58cc3e3d18ef24ed9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138806
Auto-Submit: Martin Kustermann <kustermann@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-03-10 13:51:51 +00:00
Victor Lima 8fd8fec185 [vm/aot] Add support to instance calls with unboxed parameters
In order to have instance calls with unboxed parameters, all
the possible targets need to be able to those parameters unboxed.
Then, the methods were partitioned according to the class hierarchy,
and the inferred types from TFA are used to determine the parameters
that could be unboxed.

This is done for non-nullable integer and double parameters, and
return value.

dart-aot linux x64:

Bench2D                8.028%
DeltaBlue              4.688%
ParticleSystemPaint    27.79%

Flutter release android-armv8:

velocity_tracker_bench -2.026% (less is better)

flutter_gallery_total_size -0.1460%
flutter_gallery_instructions_size -0.2715%
flutter_gallery_readonlydata_size 0.0539%

Issue https://github.com/dart-lang/sdk/issues/33549

Change-Id: Iedd9b4321c3d78b406ea1708e15c747754433027
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138501
Commit-Queue: Victor Agnez Lima <victoragnez@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2020-03-06 19:47:23 +00:00
Aske Simon Christensen 6a7d4e22b5 [vm/aot] Include entries for null in the dispatch table to avoid check.
NullCheck pc descriptors are added to all dispatch table calls where the
receiver may be null (and the selector is not one implemented by null).

All null entries in the table go to the NullError runtime entry, which
reads the NullCheck pc descriptor to get the name of the called member
for the error message.

Change-Id: I9d2847d0ccdfdb735b06e879916920ec299f39bc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134294
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-02-13 15:32:48 +00:00
Aske Simon Christensen 1c2d378068 [vm/aot] Use TFA to eliminate unused selectors in dispatch table.
For each selector, count the number of calls to the selector that are
both reachable and polymorphic according to the TFA. Only include
selectors in the table with non-zero counts.

This reduces the dispatch table size by 30% in dart2js and by 49% in
Flutter Gallery (312k memory use reduction on ARM64).

Call counts are transferred in a dedicated metadata block with
per-selector information. This mechanism also prepares for transferring
other per-selector information in the future.

Change-Id: Iba15aa4d6c50e67e53c3fd8e542123d3fc98bd07
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132603
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-02-13 15:32:48 +00:00
Aske Simon Christensen 76c6282de6 [vm/aot] Use a global dispatch table for instance calls in AOT.
This is a Dart-tailored implementation of the "row-displacement
dispatch table" technique for closed-world instance calls:

All interface targets in the program are grouped into selectors such
that all targets that could potentially be called from the same call
site have the same selector (currently just grouped by name).

Each selector is assigned a selector offset such that offset + classid
is unique for all selector/classid combinations where the class
implements the selector.

At every instance call site that has an interface target (i.e. where
the static type of the receiver is not dynamic), the selector offset +
receiver classid is computed and used as index into a global table of
entry points.

If the receiver can be null (as determined by the front-end TFA and the
VM type propagation), a null check is inserted before the call.

An arguments descriptor is provided (only) for selectors that need it
(those which have type parameters or optional/named parameters).

The dispatch table calls don't need the monomorphic entry code, so for
functions that are only called via dispatch table calls (i.e. never
called dynamically), the monomorphic entry code is left out.


Some future improvements to the table dispatch implementation are
mentioned in https://github.com/dart-lang/sdk/issues/40188


The table dispatch flag is disabled by default in this commit. A
separate commit enables the flag.

Change-Id: Ic2911742b4a2c9a8d3bc7df60605454cbe4c0714
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/126648
Commit-Queue: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2020-02-03 11:04:15 +00:00
Samir Jindel d0909e068a Reland "[vm/aot] Only patch call-sites to monomorphic entry-points when the receiver is proven to be a heap object."
This reverts commit 5f81efb4a3.

Reason for revert: 3xH bot is green again.

Original change's description:
> Revert "[vm/aot] Only patch call-sites to monomorphic entry-points when the receiver is proven to be a heap object."
> 
> This reverts commit ab2026af45.
> 
> Reason for revert: Flutter 3xH is broken on Golem.
> 
> Original change's description:
> > [vm/aot] Only patch call-sites to monomorphic entry-points when the receiver is proven to be a heap object.
> > 
> > Flutter Gallery ARMv8: -0.8% total, -1.3% instructions
> > Flutter Gallery ARMv7: -0.7% total, -1.2% instructions
> > 
> > There are some performance regressions, in four categories:
> > 
> > 1. Arithmetic on num types (e.g. MinLib, MaxLib):
> >    We believe arithmetic on num types is rare in practice, and can be optimized later
> >    by specializing the call-sites.
> > 
> > 2. Lack of types (e.g. MeshDecompression, ImagingGaussianBlur, Crypto*):
> >    These (and similar benchmarks) are written in a very untyped style, which is not
> >    representative of Dart 2 style. We've confirmed that the regressions disappear if
> >    the benchmarks are annotated with appropriate types.
> > 
> > 3. Megamorphic calls which are specialized in the benchmark (e.g. DeltaBlue):
> >    DeltaBlue uses string interpolation in one place, so the toString() method in the
> >    string interpolation helper is monomorphic for _Smis. However, in a realistic programs,
> >    string interpolation is used much more frequently and with different types, so it's very
> >    unlikely that this call would be monomorphic. When string interpolation is removed, there's
> >    a remaining 1.3% regression on DeltaBlue which is not yet accounted for.
> > 
> > 4. Noisy benchmarks
> >    The remaining regressions are on benchmarks which show frequent jitter historically.
> > 
> > There are also 1-4% improvements on many benchmarks (Golem's "noise" analysis is hiding
> > many of them, look at the graphs).
> > 
> > For the reasons above, we believe the regressions justified by the code size improvement.
> > 
> > Change-Id: Ic9b281d4383a6111de9d6f44347976ffa61a6ca6
> > Cq-Include-Trybots:luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-bare-linux-release-simarm64-try
> > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128667
> > Commit-Queue: Samir Jindel <sjindel@google.com>
> > Reviewed-by: Ryan Macnak <rmacnak@google.com>
> > Reviewed-by: Martin Kustermann <kustermann@google.com>
> 
> TBR=kustermann@google.com,rmacnak@google.com,alexmarkov@google.com,sjindel@google.com
> 
> Change-Id: I411aae4b230f2a08146ad3bf3e7a10aa6592db0e
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-bare-linux-release-simarm64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131660
> Reviewed-by: Samir Jindel <sjindel@google.com>
> Commit-Queue: Samir Jindel <sjindel@google.com>

TBR=kustermann@google.com,rmacnak@google.com,alexmarkov@google.com,sjindel@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: I36e4e9d598033fcc934c2cdf5b061aa11255d3b2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-bare-linux-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131837
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2020-01-16 14:21:16 +00:00
Samir Jindel 5f81efb4a3 Revert "[vm/aot] Only patch call-sites to monomorphic entry-points when the receiver is proven to be a heap object."
This reverts commit ab2026af45.

Reason for revert: Flutter 3xH is broken on Golem.

Original change's description:
> [vm/aot] Only patch call-sites to monomorphic entry-points when the receiver is proven to be a heap object.
> 
> Flutter Gallery ARMv8: -0.8% total, -1.3% instructions
> Flutter Gallery ARMv7: -0.7% total, -1.2% instructions
> 
> There are some performance regressions, in four categories:
> 
> 1. Arithmetic on num types (e.g. MinLib, MaxLib):
>    We believe arithmetic on num types is rare in practice, and can be optimized later
>    by specializing the call-sites.
> 
> 2. Lack of types (e.g. MeshDecompression, ImagingGaussianBlur, Crypto*):
>    These (and similar benchmarks) are written in a very untyped style, which is not
>    representative of Dart 2 style. We've confirmed that the regressions disappear if
>    the benchmarks are annotated with appropriate types.
> 
> 3. Megamorphic calls which are specialized in the benchmark (e.g. DeltaBlue):
>    DeltaBlue uses string interpolation in one place, so the toString() method in the
>    string interpolation helper is monomorphic for _Smis. However, in a realistic programs,
>    string interpolation is used much more frequently and with different types, so it's very
>    unlikely that this call would be monomorphic. When string interpolation is removed, there's
>    a remaining 1.3% regression on DeltaBlue which is not yet accounted for.
> 
> 4. Noisy benchmarks
>    The remaining regressions are on benchmarks which show frequent jitter historically.
> 
> There are also 1-4% improvements on many benchmarks (Golem's "noise" analysis is hiding
> many of them, look at the graphs).
> 
> For the reasons above, we believe the regressions justified by the code size improvement.
> 
> Change-Id: Ic9b281d4383a6111de9d6f44347976ffa61a6ca6
> Cq-Include-Trybots:luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-bare-linux-release-simarm64-try
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128667
> Commit-Queue: Samir Jindel <sjindel@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>

TBR=kustermann@google.com,rmacnak@google.com,alexmarkov@google.com,sjindel@google.com

Change-Id: I411aae4b230f2a08146ad3bf3e7a10aa6592db0e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-bare-linux-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/131660
Reviewed-by: Samir Jindel <sjindel@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
2020-01-14 14:58:45 +00:00
Samir Jindel ab2026af45 [vm/aot] Only patch call-sites to monomorphic entry-points when the receiver is proven to be a heap object.
Flutter Gallery ARMv8: -0.8% total, -1.3% instructions
Flutter Gallery ARMv7: -0.7% total, -1.2% instructions

There are some performance regressions, in four categories:

1. Arithmetic on num types (e.g. MinLib, MaxLib):
   We believe arithmetic on num types is rare in practice, and can be optimized later
   by specializing the call-sites.

2. Lack of types (e.g. MeshDecompression, ImagingGaussianBlur, Crypto*):
   These (and similar benchmarks) are written in a very untyped style, which is not
   representative of Dart 2 style. We've confirmed that the regressions disappear if
   the benchmarks are annotated with appropriate types.

3. Megamorphic calls which are specialized in the benchmark (e.g. DeltaBlue):
   DeltaBlue uses string interpolation in one place, so the toString() method in the
   string interpolation helper is monomorphic for _Smis. However, in a realistic programs,
   string interpolation is used much more frequently and with different types, so it's very
   unlikely that this call would be monomorphic. When string interpolation is removed, there's
   a remaining 1.3% regression on DeltaBlue which is not yet accounted for.

4. Noisy benchmarks
   The remaining regressions are on benchmarks which show frequent jitter historically.

There are also 1-4% improvements on many benchmarks (Golem's "noise" analysis is hiding
many of them, look at the graphs).

For the reasons above, we believe the regressions justified by the code size improvement.

Change-Id: Ic9b281d4383a6111de9d6f44347976ffa61a6ca6
Cq-Include-Trybots:luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-bare-linux-release-simarm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/128667
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2020-01-14 13:18:05 +00:00
Alexander Markov f56b0f6907 [vm/aot/tfa] Whole-program constant propagation
Size:
flutter_gallery total size -2.48% (arm), -2.3% (arm64)
flutter_gallery instructions size -2.91% (arm), -2.77% (arm64)
velocity_tracker_bench total size -7.5% (arm), -7.1% (arm64)

Performance:
SkeletalAnimation +46.02% (Intel Core i5), +37.75% (Intel Xeon), +24.86% (arm), +39.75% (arm64).
FfiMemory.Pointer* +44-64% (x64)
FfiMemory.PointerPointer +436-465% (x64), +443% (arm64).

Issue: https://github.com/dart-lang/sdk/issues/37710

Change-Id: I6221bfa02b165ccc17d4ee8b857bb89212febaff
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/125936
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-12-04 19:19:29 +00:00
Clement Skau a196d68145 Revert "[vm, front-end server] Cache generated bytecode using separate file"
This reverts commit 86641d07ae.

Reason for revert: Appears to break Flutter 3H:

- https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8897037674557388176/+/steps/flutter_test_framework_coverage/0/stdout

- https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8897037674557388176/+/steps/flutter_test_framework_tests/0/stdout

- https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8897037674557388176/+/steps/flutter_test_hostonly_devicelab_tests/0/stdout

Original change's description:
> [vm, front-end server] Cache generated bytecode using separate file
> 
> Previously, generated bytecode was cached using metadata in kernel AST.
> When serializaing kernel AST (for subsequent compilations) incremental
> serialization was not used, so AST serialization was considerably
> slower in bytecode mode compared to non-bytecode mode.
> 
> Apparently, metadata is not compatible with incremental serialization
> (https://github.com/dart-lang/sdk/issues/39302).
> 
> This change reimplements caching of generated bytecode using a separate
> 'foo.dill.cache' file near the generated 'foo.dill' and 'foo.dill.ast'.
> AST is now written using incremental serialization both in bytecode
> and non-bytecode modes.
> 
> flutter_test_performance benchmark:
> 
> AST mode:
> 
>   "without_change_elapsed_time_ms": 2615-2652,
>   "implementation_change_elapsed_time_ms": 6292-6394,
>   "interface_change_elapsed_time_ms": 6183-6484,
>   "with_coverage_time_ms": 2723-2834
> 
> Bytecode mode, before this change:
> 
>   "without_change_elapsed_time_ms": 3246-3295,
>   "implementation_change_elapsed_time_ms": 7998-8068,
>   "interface_change_elapsed_time_ms": 7899-8029,
>   "with_coverage_time_ms": 3316-3378
> 
> Bytecode mode, after this change:
> 
>   "without_change_elapsed_time_ms": 2689-2737,
>   "implementation_change_elapsed_time_ms": 7630-7677,
>   "interface_change_elapsed_time_ms": 7724-7917,
>   "with_coverage_time_ms": 2841-2949
> 
> Change-Id: I603d33f49949146cf6620226d810f2f3cb9853a9
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124592
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
> Commit-Queue: Alexander Markov <alexmarkov@google.com>

TBR=rmacnak@google.com,alexmarkov@google.com

Change-Id: Ib433b68b7de3b08179c9c080a10f62703cd2546d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124983
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Clement Skau <cskau@google.com>
2019-11-12 11:11:10 +00:00
Alexander Markov 86641d07ae [vm, front-end server] Cache generated bytecode using separate file
Previously, generated bytecode was cached using metadata in kernel AST.
When serializaing kernel AST (for subsequent compilations) incremental
serialization was not used, so AST serialization was considerably
slower in bytecode mode compared to non-bytecode mode.

Apparently, metadata is not compatible with incremental serialization
(https://github.com/dart-lang/sdk/issues/39302).

This change reimplements caching of generated bytecode using a separate
'foo.dill.cache' file near the generated 'foo.dill' and 'foo.dill.ast'.
AST is now written using incremental serialization both in bytecode
and non-bytecode modes.

flutter_test_performance benchmark:

AST mode:

  "without_change_elapsed_time_ms": 2615-2652,
  "implementation_change_elapsed_time_ms": 6292-6394,
  "interface_change_elapsed_time_ms": 6183-6484,
  "with_coverage_time_ms": 2723-2834

Bytecode mode, before this change:

  "without_change_elapsed_time_ms": 3246-3295,
  "implementation_change_elapsed_time_ms": 7998-8068,
  "interface_change_elapsed_time_ms": 7899-8029,
  "with_coverage_time_ms": 3316-3378

Bytecode mode, after this change:

  "without_change_elapsed_time_ms": 2689-2737,
  "implementation_change_elapsed_time_ms": 7630-7677,
  "interface_change_elapsed_time_ms": 7724-7917,
  "with_coverage_time_ms": 2841-2949

Change-Id: I603d33f49949146cf6620226d810f2f3cb9853a9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/124592
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-11-11 19:32:53 +00:00
Alexander Markov 8a09d7ab5a [vm/bytecode] Remove alignment of sections in bytecode
Previously, before switching to compact encoding of bytecode
instructions, we had to align bytecode instructions by 4.
Since that time sections of bytecode binary were aligned.

However, this alignment doesn't work well in AOT mode if dill file
contains both bytecode and AST, as other metadata (such as obfuscation
prohibitions) may precede bytecode metadata and may make the whole
bytecode binary misaligned.

This results in incorrect calculation of section offsets,
failed assertion

  assert(writer.offset - start == section.offset);

and VM may fail to load bytecode binary with error

  bytecode_reader.cc: 1368: error: unreachable code

In order to fix these problems, extra alignment of bytecode
sections is removed.

Change-Id: Iaa3b7c46999615aaa446781a7018db409261ead8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123686
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-10-31 21:37:44 +00:00
Alexander Markov 6432d26a52 [vm/bytecode] Avoid excessive memory copying when generating bytecode
Change-Id: Id20cfd19621027d829bdf4c3a14f3c9699b90ab3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/122681
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-10-23 23:31:07 +00:00
Ryan Macnak 8e05cd278c [vm, bytecode] Emit bytecode without ASTs by default.
After d65a60db00, we don't need AST with the bytecode to initialize a kernel compiler since the kernel compiler will initialize from a separate file.

Remove duplicated BinaryCacheMetadataRepository.

Fix generating bytecode in non-incremental mode.

Change-Id: Ica7bfb5a2d86dcc224849a9934a7a4676b01f075
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119702
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2019-10-07 22:12:49 +00:00
Alexander Markov c77c2728dd [vm,aot,bytecode] Handle inferred-type metadata when generating bytecode
Change-Id: Ifadb25c782db5c95025e7275daf0676abd73cf49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120341
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-10-06 16:51:43 +00:00
Alexander Markov 07a63a17a4 [vm,aot,bytecode] Handle procedure-attributes metadata when generating bytecode
Change-Id: I26e1a15d979ce13ca60298de28ced49b6882cf52
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119765
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2019-10-03 16:27:33 +00:00
Ryan Macnak d65a60db00 [vm, bytecode] Populate bytecode cache when initializing an incremental compiler from a kernel file.
AST
"without_change_elapsed_time_ms": 2863,
"implementation_change_elapsed_time_ms": 6009,
"interface_change_elapsed_time_ms": 5888,
"with_coverage_time_ms": 2884

BYTECODE BEFORE
"without_change_elapsed_time_ms": 5216,
"implementation_change_elapsed_time_ms": 8517,
"interface_change_elapsed_time_ms": 8895,
"with_coverage_time_ms": 5462

BYTECODE AFTER
"without_change_elapsed_time_ms": 3384,
"implementation_change_elapsed_time_ms": 8139,
"interface_change_elapsed_time_ms": 8073,
"with_coverage_time_ms": 3473

Change-Id: Ic8e1c183070cb6019ea48f16c04ea5363df41a97
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119620
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2019-10-02 23:53:42 +00:00
Alexander Markov 5920048fa2 [vm,aot,bytecode] Handle direct-call metadata when generating bytecode
AOT transformations add vm.direct-call.metadata with devirtualization
information to AST nodes.

Bytecode generator should take this information into account when
generating bytecode in case of AOT. Bytecode format is extended with
CheckReceiverForNull and UncheckedDirectCall instructions, and
DirectCallViaDynamicForwarder constant pool entry in order to represent
devirtualized calls.

Change-Id: I697432ddd0b58d2d0413715132ba5e90eb606ec1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119201
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-10-01 00:20:48 +00:00
Ryan Macnak 14983a2941 [vm/bytecode] Add timeline events for major compilation phases.
Change-Id: Ie2a1f8a5979bb58133535b7698c5137d4ab2b046
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118648
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
2019-09-30 16:36:09 +00:00
Alexander Markov da8cb470cc [vm/bytecode] Represent classes, libraries, scripts and recursive types in bytecode
Size of a large application:
Before: 26628600
After: 21480120 (-19.3%)

Size of snapshots:

isolate_snapshot_framework.bin
Before: 9322496
After: 6782976 (-27.2%)

isolate_snapshot_product_framework.bin
Before: 9166848
After: 6602752 (-27.9%)

Regressions in tests:

1) Test language_2/type_alias_equality_test/04 fails similarly to default mode, as VM
does not implement comparison of function types according to the specification.
Previously this test was passing as function types were canonicalized in bytecode,
which was not always correct. This CL fixes the problem with canonicalization of
function types in bytecode and the test starts failing again.

2) Tests standalone_2/entrypoints_verification_test, standalone_2/io/test_extension_test,
standalone_2/io/test_extension_fail_test fail as native extensions are not supported
in bytecode yet. These tests start passing after df5e7aac17,
which switched bytecode tests to kernel service (on x64), because kernel service doesn't
drop ASTs. This CL switches from reading AST library declarations to bytecode even if
AST is not removed, so tests fail again.

Change-Id: I8b7ba44bfa49d0b1599b2509553ff7c831a4e244
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/104700
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2019-06-20 18:27:21 +00:00
Alexander Markov 61f0f5bc43 [vm/bytecode] Declare members in bytecode
This change replaces kernel AST declarations of fields and functions
with bytecode declarations.

Size of dilp files is reduced by 11-12%.

Startup latency:
Time to the first full frame: 1.945s -> 1.687s
FinalizeClass: 554ms -> 277ms
FinishClassLoading: 296ms -> 156ms

There are following regressions in bytecode mode, which will be fixed
in future:

* dart:mirrors are not supported yet (implementation of mirrors relies
  on reading kernel AST in certain cases).

  As the result, lib_2/mirrors/* tests fail.

* native extensions are not supported yet (annotations on libraries
  and classes in AST are cleaned up as they could reference members
  which are now removed from AST).

  As the result, standalone_2/entrypoints_verification_test test fails.

* language_2/spread_collections/const_error_test/* tests fail
  due to https://github.com/dart-lang/sdk/issues/36286.

Change-Id: I5130f401fd7b84038b136136e7ccc1a6e51b6cea
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/97561
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
2019-03-26 21:40:10 +00:00
Alexander Markov f7be27e6c0 [vm/bytecode] Add an option to print bytecode size statistics
This CL adds '--show-bytecode-size-stat' option to gen_kernel and
Fuchsia kernel compiler to print breakdown of size of generated bytecode.

Change-Id: I0637738b0a1e9c635cb687bbfd482a9bdae32183
Reviewed-on: https://dart-review.googlesource.com/c/90322
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2019-01-23 22:03:05 +00:00
Jens Johansen 6c909a0efa Be explicit about whether a reference is allowed to be null or not
Before this change we allowed to serialize some references as null,
and would only complain when trying to read the dill back in.
This CL changes that so it complains up front.

It is unknown if any of this would ever occur in practise.

Change-Id: Id5e1af89abfb88a2d4249bd439b53b062d7aeffa
Reviewed-on: https://dart-review.googlesource.com/c/85392
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
2019-01-04 08:39:36 +00:00
Samir Jindel 921e62f1bf [vm] Introduce multiple entrypoints to AOT.
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-release-x64-try, vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-win-release-x64-try
Change-Id: I8643033febf9bdc57c8298dca5f43346fb8bf6d1
Reviewed-on: https://dart-review.googlesource.com/c/80304
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
2018-12-07 13:19:24 +00:00
Alexander Markov 9f49b47823 [vm/bytecode] Revise representation of objects and strings in bytecode
This is a major revamp of bytecode metadata format. Now bytecode has
its own serialization mechanisms.

This CL adds 'bytecode component' metadata, which contains bytecode
object table and string table. All references from bytecode (constant
pools) to libraries, classes, members, types and strings now have a new
format. References to frequently used objects are represented as indices
in object table, while rarely used objects are written inline.
This allows VM to cache frequently used objects while reading bytecode.

Representation of strings is aligned with VM - string characters are
stored in separate pools of one-byte and two-byte strings. This allows
VM to avoid UTF-8 decoding and extra copying.

Closure declarations are now explicit. Type parameters no longer require
enslosing scopes when reading/writing them.

Benchmarks:
GenKernelKernelReadAllBytecode (Intel Core i5) +29.84%
GenKernelKernelReadAllBytecode (Intel Xeon) +28.74%
Change-Id: I4b80009733a8f8c038264af74f97c4e094b9e311
Reviewed-on: https://dart-review.googlesource.com/c/85469
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-12-03 19:08:42 +00:00
Alexander Markov 8f7a0f400b [vm/bytecode] Emit source positions for bytecode instructions
Change-Id: I83963771db301089317823f6bf14eb4a2728dbfb
Reviewed-on: https://dart-review.googlesource.com/c/81740
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Régis Crelier <regis@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2018-11-14 00:27:45 +00:00
Alexander Markov a4dc8cd22e [vm/kernel/bytecode] Make sure bytecode instructions are aligned in kernel
Change-Id: I109123e88b8352b198d5ef9aad9eedc1df7141c8
Reviewed-on: https://dart-review.googlesource.com/c/83543
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-11-09 17:08:46 +00:00
Samir Jindel e23acaaf90 [vm/aot] Fix obfuscation.
Test Plan:

Make vm-kernel-precomp-obfuscate-linux-release-x64-try green, ensuring status file changes
reflect design gaps and not bugs.

Tested locally that obfuscated Flutter Gallery runs.

Change-Id: Ifcdc334de58f43c310e15b58dbcf6fe1597206f2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-obfuscate-linux-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/81009
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-10-26 13:00:33 +00:00
Alexander Markov 9406b7b44e [vm/bytecode] Add versioning to bytecode format
Change-Id: I35d86aec17aa0f6894a6380e4bb5ac7ca3092fac
Reviewed-on: https://dart-review.googlesource.com/c/80522
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-10-22 18:03:20 +00:00
Samir Jindel 3e7ce992cf [vm/tfa] Implementation of type arguments tracking in TFA.
Change-Id: I9398186e27ae7a040e249df010ae16fb6ab6da89
Cq-Include-Trybots: luci.dart.try:vm-kernel-win-release-x64-try, vm-kernel-optcounter-threshold-linux-release-x64-try, vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-release-simarm-try, vm-kernel-precomp-linux-release-simarm64-try, vm-kernel-precomp-linux-release-x64-try, vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/74962
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Dmitry Stefantsov <dmitryas@google.com>
2018-10-11 19:09:42 +00:00
Alexander Markov 46983c2de9 [vm/aot/tfa] Detect if a method/setter is called from this
This information could useful in AOT to omit unchecked entry points.

Change-Id: I1c54720e46d578dc64f67306abdf4619a34ae122
Reviewed-on: https://dart-review.googlesource.com/c/77921
Auto-Submit: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Samir Jindel <sjindel@google.com>
Reviewed-by: Samir Jindel <sjindel@google.com>
2018-10-04 11:26:44 +00:00
Alexander Markov 62d154f6a3 [vm/bytecode] Record null-initialized fields in bytecode
If a field is initialized with null (either explicitly or implicitly),
field store can be omitted in bytecode. In such case, bytecode should
still convey the information about this initialization to VM for
field guards to work correctly.

Change-Id: I1fd45b858c3c521b97fa5dbffe0e15b1ea75d92f
Reviewed-on: https://dart-review.googlesource.com/76060
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
2018-09-24 17:56:35 +00:00
Vyacheslav Egorov aa8145a03d [vm/kernel] Add a transformation that annotates invocations with receiver type.
Currently we only annotate those call-sites that would result
in generic covariant checks performed on the callee side.

Bug: https://github.com/dart-lang/sdk/issues/31798
Change-Id: Ifcf60032036575f615015d716276484a7c1236b3
Reviewed-on: https://dart-review.googlesource.com/69580
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-08-16 12:30:56 +00:00
Alexander Markov a8155f25fc [vm/aot/tfa] Infer int type, devirtualize dynamic calls
This change improves type flow analysis to infer int types, in addition
to concrete classes (int type is abstract and has 2 concrete subclasses,
_Smi and _Mint).

Also, this CL enables devirtualization of dynamic calls (previously, only
calls with known interface target were devirtualized).

DartMicroBench.PrimeNumber:
x64: +221%
armv8: +461%
armv7h: +47.98%

No measurable impact on Flutter gallery build time in release mode.

Issue: https://github.com/flutter/flutter/issues/19677

Change-Id: I34db195f52f2a434218ee11eee7f308d4661738b
Reviewed-on: https://dart-review.googlesource.com/69520
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
2018-08-13 20:48:33 +00:00
Alexander Markov 1babc13b93 [vm/kernel/bytecode] Report compile-time errors from constant evaluation while generating bytecode
Change-Id: I931622f5e442a439f5bfa241aade686b1bc91afd
Reviewed-on: https://dart-review.googlesource.com/68222
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
2018-08-03 19:21:17 +00:00
Alexander Markov 714697efbb [vm/kernel/bytecode] Write constant pool before bytecode and exceptions table
This simplifies reading, as exceptions table refers to constant pool.

Change-Id: Id20d2a315ad87aa4e5247fcd98d7e5996c188eec
Reviewed-on: https://dart-review.googlesource.com/56032
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-05-21 23:51:41 +00:00
Alexander Markov 6202749429 [vm/kernel/bytecode] Support try-catch-finally in bytecode generator
Change-Id: I2882546741b1dd8f6e03a21d85b5487fb754b742
Reviewed-on: https://dart-review.googlesource.com/55324
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-05-20 19:52:07 +00:00
Alexander Markov 6b65dcbcf6 [vm/kernel/bytecode] Support closures in bytecode generator
Change-Id: I038ba92c7bc210568d817ebaa109ca0ad02550e1
Reviewed-on: https://dart-review.googlesource.com/52988
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
2018-05-15 16:51:06 +00:00
Alexander Markov 344c68e54c [kernel, vm] Revise how metadata is written in kernel binaries
Metadata is no longer written ahead of all nodes. Instead, metadata for
each node is written in the same context as the node itself (into a separate
buffer). This allows metadata to contain (serialize) arbitrary nodes
(for example, arbitrary DartTypes) and use serialization context of parent
nodes (such as declared type parameters).

However, with this change metadata looses the ability to reference
arbitrary AST nodes. This ability was overly restricted and had no
practical uses. (It was not possible to reference nodes which are not
reachable from root Component. As a consequence, it was not possible to
write references to arbitrary DartTypes.)

This change aligns the serialization capabilities of metadata with
how kernel AST nodes are serialized.

Change-Id: I027299a33b599b62572eccd4aa7083ad1dd2b3b3
Reviewed-on: https://dart-review.googlesource.com/54481
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
2018-05-15 00:41:08 +00:00
Alexander Markov ac7ca17652 [vm/kernel] Experimental: generate DBC-like bytecode in kernel binaries
To enable bytecode generation, toggle kEnableKernelBytecode flag at
the beginning of pkg/vm/lib/bytecode/gen_bytecode.dart.
This will also enable generation of bytecode in platform dill files.

Also, bytecode generation can be enabled using --gen-bytecode option of
pkg/vm/tool/gen_kernel.

In kernel binaries, the generated bytecode and constant pool are
placed into 'vm.bytecode' metadata attached to members.
pkg/vm/tool/dump_kernel tool can be used to disassemble
bytecode and print constant pool.

Differences between generated bytecode and original DBC are
described in pkg/vm/lib/bytecode/dbc.dart.

Format of constant pool is described in
pkg/vm/lib/bytecode/constant_pool.dart.

Currently, only a small subset of Dart language is supported.
Bytecode generator will not emit vm.bytecode metadata for a
member if its body has an unsupported operation.


Change-Id: I27d9a361dc779ea115e5676508ee757c1754e05d
Reviewed-on: https://dart-review.googlesource.com/49600
Reviewed-by: Samir Jindel <sjindel@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Zach Anderson <zra@google.com>
Reviewed-by: Régis Crelier <regis@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
2018-04-23 03:42:52 +00:00