When uploading CLs, the presubmit checks verify that the lines in the
diff are formatted correctly according to `git cl format runtime`.
However, when `buildtools/<os>-<arch>/clang/bin/clang-format` is
updated, it does not force reformatting of files that would be
reformatted.
This leads to two issues:
* Inconsistent style within the code base and within a single file.
* Spurious reformatting in CLs when (1) clang-format is used on the
whole file, or (2) the diff lines overlap.
`clang-format` doesn't change that frequently, so in general this is
not a large issue, but I've seen a bit too many "spurious formatting,
please revert" comments on CLs recently.
This CL formats the runtime to be in line with the current pinned
`clang-format`:
```
$ find runtime/ -iname *.h -o -iname *.cc | xargs buildtools/mac-arm64/clang/bin/clang-format -i
```
`git cl format` (which only formats changed lines, and does so with
`clang-format`) seems to not agree with itself, or clang-format, or
cpplint in a handful of places. This CL adds `// clang-format off`
for these. (See previous patchsets for the specific instances.)
TEST=A variety of bots including GCC, MacOS and Windows.
Change-Id: I470892e898971899fda14bb3b8f2c8efefd67686
Cq-Include-Trybots: luci.dart.try:vm-gcc-linux-try,vm-ffi-qemu-linux-release-riscv64-try,vm-ffi-qemu-linux-release-arm-try,vm-aot-win-debug-x64-try,vm-win-debug-x64c-try,vm-mac-debug-x64-try,vm-mac-debug-arm64-try,vm-aot-linux-debug-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/362780
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Use more zones and handle scopes to prevent temporary allocations from having unnecessarily long lifespans.
/usr/bin/time -v gen_snapshot --snapshot-kind=app-aot-elf ... dart2js.dill
Maximum resident set size (kbytes): 821344 -> 443688 (-46.0%)
TEST=ci
Bug: b/196510517
Change-Id: I41cdcac69f485b2f33adf2c94bb11f466e6a54fb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210944
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
This is a reland of 2852408881
Original change's description:
> [vm] Reduce size of Class objects in AOT and PRODUCT modes
>
> This change conditionally removes certain fields from Class objects
> in AOT and PRODUCT modes.
>
> Flutter gallery in release-sizeopt mode:
> Snapshot size -0.47% (arm64), -0.44% (arm)
> Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
>
> On a large Flutter application compiled in --dwarf-stack-traces mode:
> Heap size of Class objects -26%
> Heap size of all snapshot objects -3.6%
>
> TEST=ci
> Issue: https://github.com/dart-lang/sdk/issues/44020
> Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
> Commit-Queue: Alexander Markov <alexmarkov@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TEST=ci
Change-Id: Ib473a959a2c610b7a0e6f4000b2101e5256c6119
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/196103
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This change conditionally removes certain fields from Class objects
in AOT and PRODUCT modes.
Flutter gallery in release-sizeopt mode:
Snapshot size -0.47% (arm64), -0.44% (arm)
Heap size of snapshot objects -4.1% (arm64), -4.3% (arm)
On a large Flutter application compiled in --dwarf-stack-traces mode:
Heap size of Class objects -26%
Heap size of all snapshot objects -3.6%
TEST=ci
Issue: https://github.com/dart-lang/sdk/issues/44020
Change-Id: I795c625b71558cd2f336f92fc326c36fd339cd4b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/195700
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
As part of making the compiler and other subsystems independent
of `Isolate` we have to move various state from `Isolate` to
`IsolateGroup`.
The class_table and object_store were already moved to `IsolateGroup`.
This CL only replaces usages of `Isolate::{object_store,class_table}`
with the equivalent in `IsolateGroup`.
Issue https://github.com/dart-lang/sdk/issues/36097
TEST=Pure refactoring - relying on existing test coverage.
Change-Id: I34a0682d715b054d6c5faff077a513980f59a348
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/177126
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
(Assignment of libraries to loading units is already done in the kernel generation step.)
After compiling and before serializing, we walk the program and for each Code we assign its Instructions, CodeSourceMap and CompressedStackMap to the loading unit of that Code's defining library. Deduplication may cause Instructions, CodeSourceMaps and CompressedStackMaps to belong to more than one loading unit; in this case the objects are assigned to the root loading unit. Later they can be more precisely assigned to the dominating loading unit.
All objects except some Instructions, CodeSourceMaps and CompressedStackMaps belong to the root loading unit's snapshot. This snapshot is written like an unsplit snapshot, except that when serializing Code, we will write a reference to a stub or null when the Code's Instructions, CodeSourceMap or CompressedStackMap belongs to a non-root loading unit. The snapshots of non-root loading units contain these deferred objects and references to the corresponding Code objects to patch. The types of objects we defer (Instructions, CodeSourceMaps and CompressedStackMaps) usually represent 70+% of the snapshot size.
Bare instructions mode must be disabled when splitting because we cannot have PC-relative calls between loading units. Later we can re-enable this for calls within loading units.
Broken: Compactor probably crashes we can now have an unbounded number of image pages and the compactor assumes a fixed number.
Embedder's guide:
At compile-time, gen_snapshot should be passed --loading_unit_manifest with a path, which will enable splitting and output a mapping from loading unit ids to snapshot output paths.
At runtime, sometime during isolate startup, an embedder should call Dart_SetDeferredLoadHandler, probably near an existing call to Dart_SetLibraryTagHandler. The callback is given a loading unit id, and should eventually call Dart_DeferredLoadComplete[Error].
Bug: https://github.com/dart-lang/sdk/issues/41974
Change-Id: Ib597eb87c8cd634416d5ee1f00629c5550aebb00
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152427
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Before, use of a function in the static calls table of a Code object
would cause it to be retained. Now, like dispatch table use, we only
retain such functions if there are other uses like dynamic function
lookup that needs the function object or if the
--retain-function-objects flag is enabled.
In most modes, --retain-function-objects is enabled by default.
It is only disabled by default in product mode when
--dwarf-stack-traces is enabled, since otherwise the removed
function objects may be needed for debugging platforms like the
Observatory or when creating symbolic stack traces.
Changes on flutter gallery in release mode:
Default:
arm7: isolate: +6.65%, total: +1.30%
arm8: isolate: +6.67%, total: +1.28%
(The increase is due to changing back to all function objects
being retained as the default when using symbolic stack traces.)
With --no-retain-function-objects:
arm7: isolate: -9.95%, total: -1.73%
arm8: isolate: -10.04%, total: -1.77%
(This measures how much dropping static call function objects
when possible affects the size compared to just dropping
function objects for dispatch table entries when possible.)
Bug: https://github.com/dart-lang/sdk/issues/41052
Change-Id: I3c834b14b0c58ccfdbaca3f154df536df29c13ef
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142146
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Instead of each Visitor being a distinct type, now FunctionVisitors
are subclasses of ClassVisitors (since we visit each class to visit
its methods) and CodeVisitors a subclass of FunctionVisitors (since
we visit each function to visit its code). Thus, we replace the
different ProgramVisitor::Visit<type> methods with a single
ProgramVisitor::WalkProgram method that walks the program, looking
for applicable objects.
Change-Id: If93638ec2eccebb8b726fad9b8ea46f94f8c099b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/141604
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Commit-Queue: Tess Strickland <sstrickl@google.com>
In particular, don't serialize a function if its code is used by the
dispatch table, but there's no need for the function object itself
at runtime like dynamic lookup or dynamic invocation forwarders.
However, just because the owning function is not needed at runtime,
it may be used during serialization for things like assembly label
creation. Thus, instead of just clearing out the Code's owner field,
we add a new WeakSerializationReference (WSR) object whose target is
the original contents of the owner field. The WSR allows the
serializer to access the original owner, but also signals to the
serializer that the target should not be serialized in AOT snapshots
unless there are additional strong (non-WSR) references.
If no strong references are found, then the references to the WSR
in the snapshot are replaced with a reference to a WSR that only
contains the class ID of the original target. The serializer creates
only one WSR per class ID.
If strong references are found, then the target is still serialized.
In this case, the WSR is dropped entirely and any reference to it is
replaced with a direct reference to the serialized target. Thus, WSRs
only exist in the precompiled runtime for targets with no strong
references.
Changes on the Flutter gallery in release mode:
arm7: total size -1.10%, isolate size -6.08%
arm8: total size -1.16%, isolate size -6.09%
Bug: https://github.com/dart-lang/sdk/issues/41052
Change-Id: I5b435ca71ef85f50fe3484789087471a91aa4fe2
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-release-x64-try,vm-kernel-precomp-linux-product-x64-try,vm-kernel-precomp-linux-release-simarm-try,vm-kernel-precomp-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm_x64-try,vm-kernel-precomp-mac-release-simarm64-try,vm-kernel-precomp-win-release-x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/137104
Commit-Queue: Tess Strickland <sstrickl@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>
When we are writing a snapshot, canonicalize all entries within
CompressedStackMaps payloads. We do this by creating a global table
of stack map information (bit counts + bit payload) that we store
in the isolate object store, and then for each existing CSM, we
replace it with an alternate version that refers to the global table.
This gets back some of the canonicalization we lost when we moved from
individual StackMap objects to per-Code CompressedStackMaps objects.
Here, we also represent the global table as a CompressedStackMaps
object. This means that there are three types of CompressedStackMaps:
* The original version that directly contains all entry information.
* The version where each entry is just the PC offset delta and an
offset into the global table payload.
* A version representing the global table where the entries are like
the original version except they don't contain a PC offset delta.
-----
The impact on AOT snapshot size when compiling the Flutter Gallery
in release mode:
armv7: Total size -0.88% (Isolate RO: -5.05%, Isolate snapshot: +0.00%)
armv8: Total size -1.04% (Isolate RO: -5.28%, Isolate snapshot: +0.00%)
-----
Bug: https://github.com/dart-lang/sdk/issues/35274
Change-Id: I1ce0f8b3cc58e2f11584f3c218e0fdf8984b799b
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/120667
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Fixes an assumption that CompressedStackMapsIterator::Find() is never
passed a PC offset of 0. Adds back an ASSERT that was dropped which checks
for specific cases where a given PC offset does not have a stack map entry.
Original commit message:
Lifting the PC offset in a2bb730 was a small, lightweight change that
gave us big gains, at least on 32-bit architectures. Here, we make
much more invasive changes that will improve the amount of memory used
by the information previously stored in StackMap objects.
Instead of allocating separate objects for StackMaps, we instead compress
all StackMap information for a given Code object into a single object
(CompressedStackMaps, or CSM for short). This replaces the Array used to
store PC offsets (as Smis) and the individual StackMap objects.
While we lose all canonicalization for individual StackMap entries, the
drop in space required to store stack map information more than offsets that.
-----
The impact on AOT snapshot size when compiling the Flutter Gallery
in release mode:
armv7: Total size -2.58% (Isolate RO: +14.46%, Isolate snapshot: -22.93%)
armv8: Total size -1.85% (Isolate RO: +15.69%, Isolate snapshot: -22.97%)
The impact on in-memory, not on-disk, size for the Flutter Gallery as seen
in the Observatory while running a profile (not release) build:
armv7: Drops from 7.1 MB to 6.2MB (-0.9 MB)
armv8: Drops from 13.5MB to 11.7MB (-1.8 MB)
-----
Bug: https://github.com/dart-lang/sdk/issues/35274, https://github.com/dart-lang/sdk/issues/38873
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try,vm-kernel-mac-debug-simdbc64-try
Change-Id: I111b129b0ed64f03184370bceb7cda69d5d4b3c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121700
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
This reverts commit d77f4913a1.
Reason for revert: large number of flaky crashes on flakiness dashboard (https://github.com/dart-lang/sdk/issues/38873); crashes on vm-kernel-mac-debug-simdbc64 bot.
Original change's description:
> [vm/compiler] Further compress the information previously in StackMaps.
>
> Lifting the PC offset in a2bb730 was a small, lightweight change that
> gave us big gains, at least on 32-bit architectures. Here, we make
> much more invasive changes that will improve the amount of memory used
> by the information previously stored in StackMap objects.
>
> Instead of allocating separate objects for StackMaps, we instead compress
> all StackMap information for a given Code object into a single object
> (CompressedStackMaps, or CSM for short). This replaces the Array used to
> store PC offsets (as Smis) and the individual StackMap objects.
>
> While we lose all canonicalization for individual StackMap entries, the
> drop in space required to store stack map information more than offsets that.
>
> -----
>
> The impact on AOT snapshot size when compiling the Flutter Gallery
> in release mode:
>
> armv7: Total size -2.58% (Isolate RO: +14.46%, Isolate snapshot: -22.93%)
> armv8: Total size -1.85% (Isolate RO: +15.69%, Isolate snapshot: -22.97%)
>
> The impact on in-memory, not on-disk, size for the Flutter Gallery as seen
> in the Observatory while running a profile (not release) build:
>
> armv7: Drops from 7.1 MB to 6.2MB (-0.9 MB)
> armv8: Drops from 13.5MB to 11.7MB (-1.8 MB)
>
> -----
>
> Bug: https://github.com/dart-lang/sdk/issues/35274
> Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try
> Change-Id: Ie3bb898d557215146260a560423f5fa27bdff512
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119640
> Commit-Queue: Teagan Strickland <sstrickl@google.com>
> Reviewed-by: Martin Kustermann <kustermann@google.com>
> Reviewed-by: Ryan Macnak <rmacnak@google.com>
TBR=kustermann@google.com,rmacnak@google.com,sstrickl@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: https://github.com/dart-lang/sdk/issues/35274
Change-Id: Ib844b4d52e3aa4aca62a896cfe349c680bda7461
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try, vm-kernel-precomp-linux-debug-simarm_x64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/121642
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Lifting the PC offset in a2bb730 was a small, lightweight change that
gave us big gains, at least on 32-bit architectures. Here, we make
much more invasive changes that will improve the amount of memory used
by the information previously stored in StackMap objects.
Instead of allocating separate objects for StackMaps, we instead compress
all StackMap information for a given Code object into a single object
(CompressedStackMaps, or CSM for short). This replaces the Array used to
store PC offsets (as Smis) and the individual StackMap objects.
While we lose all canonicalization for individual StackMap entries, the
drop in space required to store stack map information more than offsets that.
-----
The impact on AOT snapshot size when compiling the Flutter Gallery
in release mode:
armv7: Total size -2.58% (Isolate RO: +14.46%, Isolate snapshot: -22.93%)
armv8: Total size -1.85% (Isolate RO: +15.69%, Isolate snapshot: -22.97%)
The impact on in-memory, not on-disk, size for the Flutter Gallery as seen
in the Observatory while running a profile (not release) build:
armv7: Drops from 7.1 MB to 6.2MB (-0.9 MB)
armv8: Drops from 13.5MB to 11.7MB (-1.8 MB)
-----
Bug: https://github.com/dart-lang/sdk/issues/35274
Cq-Include-Trybots: luci.dart.try:vm-kernel-precomp-linux-debug-x64-try,vm-kernel-precomp-linux-debug-simarm_x64-try
Change-Id: Ie3bb898d557215146260a560423f5fa27bdff512
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119640
Commit-Queue: Teagan Strickland <sstrickl@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
This is the final CL which adds a new --use-bare-instructions flag to
the VM.
If this flag is set during AOT compilation, we will:
* Build one global object pool (abbr: GOP) which all code objects
share. This gop will be stored in the object store. The PP register
is populated in the enter dart stub and it is restored when
returning from native calls.
* Gets rid of the CODE_REG/PP slots from the dart frames. Instead the
compiled code uses the global object pool, which is always in PP.
* Starts emitting pc-relative calls for calls between two dart
functions or when invoking a stub.
Limitation: We only emit pc-relative calls between two code objects
in the same isolate (this is because the image writer is writing
instruction objects for vm-isolate/main-isolate seperately)
* We do compile-time relocation of those static calls after the
precompiler has finished its work, but before writing the snapshot.
This patches all the instruction objects with pc-relative calls to
have the right .text distance.
* We emit a sorted list of code objects in ObjectStore::reverse_code_table,
which will be used by the AOT runtime to go back from PC to Code
objects (where all metadata, e.g. stack maps, catch entry moves, pc
descriptors are available).
Issue https://github.com/dart-lang/sdk/issues/33274
Change-Id: I6c5dd2b1571e3a889b27e804a24c2986c71e03b6
Reviewed-on: https://dart-review.googlesource.com/c/85769
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Previously we tried to rely on the assumption that all variables would be
boxed - so the machinery for setting correct catch-entry state only
supported tagged values and constants. However this both leads to worse code
and is not entirely correct assumption.
This also:
- renames various confusingly named classes: we move away from talking
about "catch entry state" to "catch entry moves" - because we only
record a subset of moves that needs to be performed and that does
not describe the whole state;
- refactors a bunch of associated code to be more readable and maintainable;
- adds documentation about catch implementation in optimized code
to runtime/docs/compiler;
Fixes https://github.com/flutter/flutter/issues/21685.
Change-Id: I03ae361a1bb7710acbd9f661ae014e663a163c59
Reviewed-on: https://dart-review.googlesource.com/74860
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
- Adjust fingerprints to be independent of the library's private key, which varies with load order.
- Use usage_counter in AOT inlining decisions if JIT feedback is available.
- Reduce inlining in cold functions.
dart2js product aot snapshot 15841351 -> 14436273 (-8.86%)
R=fschneider@google.com
Review-Url: https://codereview.chromium.org/2562693003 .
Daco Harkes