Kenzie Schmoll
dabc54edb8
Change-Id: Ibaa017ffc80d674f439af1f2097729df50755fe7 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158368 Commit-Queue: Kenzie Schmoll <kenzieschmoll@google.com> Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
286 lines
12 KiB
Markdown
286 lines
12 KiB
Markdown
# vm_snapshot_analysis
|
|
|
|
This package provides libraries and a utility for analysing the size and
|
|
contents of Dart VM AOT snapshots based on the output of
|
|
`--print-instructions-sizes-to`, `--write-v8-snapshot-profile-to` and
|
|
`--trace-precompiler-to` VM flags.
|
|
|
|
## AOT Snapshot Basics
|
|
|
|
Dart VM AOT snapshot is simply a serialized representation of the Dart VM
|
|
heap graph. It consists of two parts: data (e.g. strings, `const` instances,
|
|
objects representing classes, libraries, functions and runtime metadata) and
|
|
executable code (machine code generated from Dart sources). Some nodes in this
|
|
graph have clean and direct relationship to the original program (e.g. objects
|
|
representing libraries, classes, functions), while other nodes don't. Bitwise
|
|
equivalent objects can be deduplicated and shared (e.g. two functions with the
|
|
same body will end up using the same machine code object). This makes
|
|
impossible to attribute of every single byte from the snapshot to a particular
|
|
place in the program with a 100% accuracy.
|
|
|
|
* `--print-instructions-sizes-to` attributes _executable code_ from the snapshot
|
|
to a particular Dart function (or internal stub) from which this code
|
|
originated (ignoring deduplication). Executable code usually constitutes around
|
|
half of the snapshot, those this varies depending on the application.
|
|
* `--write-v8-snapshot-profile-to` is a graph representation of the snapshot,
|
|
it attributes bytes written into a snapshot to a node in the heap graph. This
|
|
format covers both data and code sections of the snapshot.
|
|
* `--trace-precompiler-to` gives information about dependencies between
|
|
compiled functions, allowing to determine why certain function was pulled into
|
|
the snapshot.
|
|
|
|
### Passing flags to the AOT compiler
|
|
|
|
Both in `dart2native` and Flutter you can use `--extra-gen-snapshot-options` to
|
|
pass flags to the AOT compiler:
|
|
|
|
```console
|
|
$ flutter build aot --release --extra-gen-snapshot-options=--write-v8-snapshot-profile-to=profile.json
|
|
|
|
$ dart2native --extra-gen-snapshot-options=--write-v8-snapshot-profile-to=profile.json -o binary input.dart
|
|
```
|
|
|
|
Similarly with `--print-instructions-sizes-to`.
|
|
|
|
If you are working on the Dart SDK you can use the `pkg/vm/tool/precompiler2`
|
|
script, in which case you can just pass these flags directly:
|
|
|
|
```console
|
|
$ pkg/vm/tool/precompiler2 --write-v8-snapshot-profile-to=profile.json input.dart binary
|
|
```
|
|
|
|
## CLI
|
|
|
|
The command line interface to the tools in this package is provided by a single
|
|
entry point `bin/analyse.dart`. It consumes output of
|
|
`--print-instructions-sizes-to` and `--write-v8-snapshot-profile-to` flags and
|
|
presents it in different human readable ways.
|
|
|
|
This script can be installed globally as `snapshot_analysis` using
|
|
|
|
```console
|
|
$ pub global activate vm_snapshot_analysis
|
|
```
|
|
|
|
`snapshot_analysis` supports the following subcommands:
|
|
|
|
### `summary`
|
|
|
|
```console
|
|
$ snapshot_analysis summary [-b granularity] [-w filter] <input-profile.json>
|
|
```
|
|
|
|
This command shows breakdown of snapshot bytes at the given `granularity` (e.g.
|
|
`method`, `class`, `library` or `package`), filtered by the given substring
|
|
`filter`.
|
|
|
|
For example, here is a output showing how many bytes from a snapshot
|
|
can be attributed to classes in the `dart:core` library:
|
|
|
|
```console
|
|
$ pkg/vm/bin/snapshot_analysis.dart summary -b class -w dart:core profile.json
|
|
+-----------+------------------------+--------------+---------+----------+
|
|
| Library | Class | Size (Bytes) | Percent | Of total |
|
|
+-----------+------------------------+--------------+---------+----------+
|
|
| dart:core | _Uri | 43563 | 15.53% | 5.70% |
|
|
| dart:core | _StringBase | 28831 | 10.28% | 3.77% |
|
|
| dart:core | :: | 27559 | 9.83% | 3.60% |
|
|
| @other | | 25467 | 9.08% | 3.33% |
|
|
| dart:core | Uri | 14936 | 5.33% | 1.95% |
|
|
| dart:core | int | 12276 | 4.38% | 1.61% |
|
|
| dart:core | NoSuchMethodError | 12222 | 4.36% | 1.60% |
|
|
...
|
|
```
|
|
|
|
Here objects which can be attributed to `_Uri` take `5.7%` of the snapshot,
|
|
at the same time objects which can be attributed to `dart:core` library
|
|
but not to any specific class within this library take `3.33%` of the snapshot.
|
|
|
|
This command also supports _estimating_ cumulative impact of a library or a
|
|
package together with its dependencies - which can be computed from
|
|
a precompiler trace (generated by adding
|
|
`--trace-precompiler-to=path/to/trace.json` to `--extra-gen-snapshot-options=`).
|
|
|
|
```console
|
|
$ snapshot_analysis summary [-b granularity] [-w filter] [-s dep-tree-start-depth] [-d dep-tree-display-depth] --precompiler-trace=<input-trace.json> <input-profile.json>
|
|
```
|
|
For example:
|
|
|
|
```console
|
|
$ snapshot_analysis summary -b package /tmp/profile.json
|
|
+-----------------------------+--------------+---------+
|
|
| Package | Size (Bytes) | Percent |
|
|
+-----------------------------+--------------+---------+
|
|
| package:compiler | 5369933 | 38.93% |
|
|
| package:front_end | 2644942 | 19.18% |
|
|
| package:kernel | 1443568 | 10.47% |
|
|
| package:_fe_analyzer_shared | 944555 | 6.85% |
|
|
...
|
|
$ snapshot_analysis summary -b package -s 1 -d 3 --precompiler-trace=/tmp/trace.json /tmp/profile.json
|
|
+------------------------------+--------------+---------+
|
|
| Package | Size (Bytes) | Percent |
|
|
+------------------------------+--------------+---------+
|
|
| package:compiler (+ 8 deps) | 5762761 | 41.78% |
|
|
| package:front_end (+ 1 deps) | 2708981 | 19.64% |
|
|
| package:kernel | 1443568 | 10.47% |
|
|
| package:_fe_analyzer_shared | 944555 | 6.85% |
|
|
...
|
|
Dependency trees:
|
|
|
|
package:compiler (total 5762761 bytes)
|
|
├── package:js_ast (total 242490 bytes)
|
|
├── package:dart2js_info (total 101280 bytes)
|
|
├── package:crypto (total 27434 bytes)
|
|
│ ├── package:typed_data (total 11850 bytes)
|
|
│ └── package:convert (total 5185 bytes)
|
|
├── package:collection (total 15182 bytes)
|
|
├── package:_js_interop_checks (total 4627 bytes)
|
|
└── package:js_runtime (total 1815 bytes)
|
|
|
|
package:front_end (total 2708981 bytes)
|
|
└── package:package_config (total 64039 bytes)
|
|
```
|
|
|
|
|
|
### `compare`
|
|
|
|
```console
|
|
$ snapshot_analysis compare [-b granularity] <old.json> <new.json>
|
|
```
|
|
|
|
This command shows comparison between two size profiles, allowing to understand
|
|
changes to which part of the program contributed most to the change in the
|
|
overall snapshot size.
|
|
|
|
```console
|
|
$ pkg/vm/bin/snapshot_analysis.dart compare -b class old.json new.json
|
|
+------------------------+--------------------------+--------------+---------+
|
|
| Library | Class | Diff (Bytes) | Percent |
|
|
+------------------------+--------------------------+--------------+---------+
|
|
| dart:core | _SimpleUri | 11519 | 22.34% |
|
|
| dart:core | _Uri | 6563 | 12.73% |
|
|
| dart:io | _RandomAccessFile | 5337 | 10.35% |
|
|
| @other | | 4009 | 7.78% |
|
|
...
|
|
```
|
|
|
|
In this example `11519` more bytes can be attributed to `_SimpleUri` class in
|
|
`new.json` compared to `old.json`.
|
|
|
|
### `treemap`
|
|
|
|
```console
|
|
$ snapshot_analysis treemap [--format <format>] <input.json> <output-dir>
|
|
$ google-chrome <output-dir>/index.html
|
|
```
|
|
|
|
This command generates treemap representation of the information from the
|
|
profile `input.json` and stores it in `output-dir` directory. Treemap can
|
|
later be viewed by opening `<output-dir>/index.html` in the browser of your
|
|
choice.
|
|
|
|
`--format` flag allows to control granularity of the output when `input.json`
|
|
is a V8 snapshot profile, available options are:
|
|
* `collapsed` essentially renders `ProgramInfo` as a treemap, individual
|
|
snapshot nodes are ignored.
|
|
* `simplified` same as `collapsed`, but also folds size information from
|
|
nested functions into outermost function (e.g. top level function or a
|
|
method) producing easy to consume output.
|
|
* `data-and-code` collapses snapshot nodes based on whether they represent
|
|
data or executable code.
|
|
* `object-type` (default) collapses snapshot nodes based on their type only.
|
|
|
|
### `explain`
|
|
|
|
#### `explain dynamic-calls`
|
|
|
|
```console
|
|
$ snapshot_analysis explain dynamic-calls <profile.json> <trace.json>
|
|
```
|
|
|
|
This command generates a report listing dynamically dispatched selectors
|
|
and their approximate impact on the code size.
|
|
|
|
```console
|
|
snapshot_analysis explain dynamic-calls /tmp/profile.json /tmp/trace.json
|
|
+------------------------------+--------------+---------+----------+
|
|
| Selector | Size (Bytes) | Percent | Of total |
|
|
+------------------------------+--------------+---------+----------+
|
|
| set:requestHeader | 10054 | 28.00% | 0.03% |
|
|
| get:scale | 3630 | 10.11% | 0.01% |
|
|
...
|
|
Dynamic call to set:requestHeader (retaining ~10054 bytes) occurs in:
|
|
package:my-super-app/src/injector.dart::Injector.handle{body}
|
|
|
|
Dynamic call to get:scale (retaining ~3630 bytes) occurs in:
|
|
package:some-dependency/src/image.dart::Image.==
|
|
```
|
|
|
|
## API
|
|
|
|
This package can also be used as a building block for other packages which
|
|
want to analyse VM AOT snapshots.
|
|
|
|
* `package:vm_snapshot_analysis/instruction_sizes.dart` provides helpers to
|
|
read output of `--print-instructions-sizes-to=...`
|
|
* `package:vm_snapshot_analysis/v8_profile.dart` provides helpers to read
|
|
output of `--write-v8-snapshot-profile-to=...`
|
|
|
|
Both formats can be converted into a `ProgramInfo` structure which attempts
|
|
to breakdown snapshot size into hierarchical representation of the program
|
|
structure which can be understood by a Dart developer, attributing bytes
|
|
to packages, libraries, classes and functions.
|
|
|
|
* `package:vm_snapshot_analysis/utils.dart` contains helper method
|
|
`loadProgramInfo` which automatically detects format of the input JSON file
|
|
and creates `ProgramInfo` in an appropriate way, allowing to write code
|
|
which works in the same way with both formats.
|
|
|
|
## Precompiler Trace Format (`--write-precompiler-trace-to=...`)
|
|
|
|
AOT compiler can produce a JSON file containing information about compiled
|
|
functions and dependencies between them. This file has the following structure:
|
|
|
|
```json
|
|
{
|
|
"trace": traceArray,
|
|
"entities": entitiesArray,
|
|
"strings": stringsArray,
|
|
}
|
|
```
|
|
|
|
- `stringsArray` is an array of strings referenced by other parts of the trace
|
|
by their index in this array.
|
|
- `entitiesArray` is an flattened array of entities:
|
|
|
|
- `"C", <library-uri-idx>, <class-name-idx>, 0` - class record;
|
|
- `"V", <class-idx>, <name-idx>, 0` - static field record;
|
|
- `"F"|"S", <class-idx>, <name-idx>, <selector-id>` - function record (`F` for dynamic functions and `S` for static functions);
|
|
|
|
Note that all records in this array occupy the same amount of elements (`4`)
|
|
to make random access by index possible.
|
|
|
|
- `traceArray` is an flattened array of precompilation events:
|
|
|
|
- `"R"` - root event (always the first element)
|
|
- `"E"` - end event (always the last element)
|
|
- `"C", <function-idx>` - function compilation event
|
|
|
|
Root and function compilation events can additionally be followed by a
|
|
sequence of references which enumerate outgoing dependencies discovered
|
|
by the AOT compiler:
|
|
|
|
- `<entity-idx>` - a reference to a function or a static field;
|
|
- `"S", <selector-idx>` - a dynamic call with the given selector;
|
|
- `"T", <selector-id>` - dispatch table call with the given selector id;
|
|
|
|
*Flattened array* is an array of records formed by consecutive elements:
|
|
`[R0_0, R0_1, R0_2, R1_0, R1_1, R1_2, ...]` here `R0_*` is the first record
|
|
and `R1_*` is the second record and so on.
|
|
|
|
## Features and bugs
|
|
|
|
Please file feature requests and bugs at the [issue tracker][tracker].
|
|
|
|
[tracker]: https://github.com/dart-lang/sdk/issues
|