Today dump info generates the entire JSON string in memory before writing it out to disk. The JSON encoder supports chunked conversion so we can do partial writes to disk and keep less of the output in memory. When converting chunks the encoder writes to the sink a lot so we also need a buffering layer to limit the number of writes.
For large programs the dump info output can be almost 2GB. After some testing a buffer size of 500MB seems to consistently reduce memory consumption by ~3.5GB.
Note: This stacks with the changes in https://dart-review.googlesource.com/c/sdk/+/322843 to reduce memory consumption by 4.8GB on a local build (18.3GB -> 13.5GB).
Change-Id: Id8f359af0724c3925f2ff767caee80d949c46da9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323040
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Some customers use tools to preprocess this set of load IDs in order to optimize how code is loaded in their app. This flag allows us to emit this information as early as possible in the compiler without having to completely compile to JS.
Change-Id: I572d4f06ffd49a672767adc344819388a741e20c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323340
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Calls to methods annotated with `@pragma('dart2js:resource-identifier')` are tracked, with their primitive constant arguments, through to the `.js` file which contains the call.
- JavaScript annotations are attached to the JavaScript AST node for
the call.
- At the time of `.js` file printing, the JavaScript annotations are
collected and attributed to the file. This allows the construction
of a map from `.js` files to the 'resource identifiers' contained in
the file.
- Alongside the `main.js` file the resource identifiers are emitted in
a file called `main.js.resources.json`. This is controlled by the
`--write-resources` command line option.
- Serialization of JavaScript ASTs now serializes the attached
JavaScript annotations.
- The internal method used to implement deferred library loading is
annotated, to allow analysis of which deferred library parts load
other libraries.
pkg/js_ast was tweaked to make the pkg/js_ast was tweaked to make the
- pkg/js_ast was tweaked to make propagating the JavaScript annotations through the async transforms easier.
TODO:
- Annotate const constructors
- Add golden-style tests
Change-Id: Iea77550e22ee98f81dca61dfd713c09f734583d2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/284492
Reviewed-by: Nate Biggs <natebiggs@google.com>
Commit-Queue: Stephen Adams <sra@google.com>
This change addresses a technical debt issue in dart2js, where
errors sometimes get reported as a binary offset location, instead
of a line/column position with contextual data from the source file.
The reason for this behavior is that dartj2s reports binary offsets
when it can't recover line/column data and show the contents of the
file. This typically happens if the file cannot be read directly
from disk.
The fix consits of adding an API to record the contents of files
that are not read directly by the compiler. We then use this API to
provide all sources collected by the CFE when parsing components
from .dill files.
Note: this CL also deletes the `autoread` feature in
source-file-provider. That feature was added for the same purpose.
The way it worked was that instead of registering source content, it
tried to read the sources from disk on-demand as errors got
reported. This doesn't always work for three reasons:
* First, we often use custom schemes (like the multi-root scheme) in
dill files to make the .dill deterministic in distributed build
systems, the autoload feature had no understanding of how to
translate those custom URIs to file URIs (that translation is only
define at the time the .dill is being built).
* Second, sometimes the files are simply not available, for example,
in hermetic build systems like bazel we have no access to those
files.
* Third, when files were found, there was no guarantee that the
contents were consistent. That is, the current version of the file
on disk could have been modified and have different contents than
those used when the .dill file was built.
I tested this manually by adding a crash in SSA and observing the
location in multiple scenarios, including running from source,
running from a .dill with file URIs, running from a .dill with
multi-root URIs, crashing in SDK locations in regular files and
patch files.
Change-Id: Ief09be577f4c9c4b345b4e2641918cafbe93c3fc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/251700
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
`diagnostic_listener.dart` is the last unmigrated import of
`common.dart`, which is imported in ~100 files.
`compiler_api.dart` is split to move the `compile` method with
unmigrated dependencies to another file to allow `compiler_api.dart` to
become migrated. Luckily there is only one reference to `compile`.
We no longer allow null `SourceSpan`s and use `SourceSpan.unknown()`
instead.
Added `SpannableWithEntity` as a migrated interface that `HInstruction`
can implement to break the dependency on a large unmigrated ssa library.
Change-Id: I0a5ff6e6c36a34ff55d98f634c671bb8f1fb9e1c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/242161
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Stephen Adams <sra@google.com>
Always import 'lib/compiler_api.dart' with prefix 'api'.
Remove import of 'lib/src/compiler.dart' with prefix 'api'.
Some of the test helper files under 'compiler/test' are a little tedious
with lots of 'api.' prefixes and I could be convinced to change them
back but I think we should stick with always using a prefix under
'compiler/lib/'.
Change-Id: I3f82d7d31f1c90a860c9811fce031a64cea04ddb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/241867
Reviewed-by: Joshua Litt <joshualitt@google.com>
Commit-Queue: Stephen Adams <sra@google.com>
Mayank Patke