Files
sdk/pkg/wasm_builder/lib/source_map.dart
T
Martin Kustermann 6184c447ab [dart2wasm] Introduce unresolved accesses & link phase
This reduces ACX Gallery's main module from 8.4 MB to 2.75 MB.

This CL adds new infrastructure to our compiler: We allow codegen to
emit unresolved instructions which will get patched up later on in a
final link phase.

We use this for constants: Generating the body of a function (or
initializer of a global) may need to access a constant. Though in
deferred loading mode we may not have decided (yet) into which module
to place the constant.

* We emit an unresolved constant access (kind of dummy instructions
  which still maintain stack machine) as a patchable region in the
  instruction stream and remember that the constant (and any constant it
  refers to, directly or indirectly) was used by the corresponding module.

* After code generation we have collected all constant uses (and the
  modules they are used in) and have therefore all knowledge to decide
  where to place constants. (See below on placement logic)

* In a final link phase that will walk over any instructions with
  unresolved constant accesses (patchable regions) and patch them up
  with the actual instructions to access the constants.

Part 1) Determination of global order

So far the creation order of globals determined the order in the global
section. But now we emit unresolved global uses and later on have to
define (or import) the globals in modules.

=> To allow this we now determine the order of globals when we build the
   globals section instead.

=> This would also allow other things: Choose ordering of globals based
   on usage count, etc.

Part 2) Separation of concerns in `constants.ensureConstant()`

So far the `constants.ensureConstant()` has done several things:

  * performed constant lowering
  * analyzing whether the constant should be lazy or eager
  * determine the type of the global of the constant
  * actual creation of global & initializer function (if needed)
  * doing the above for all transitive constants

=> The result was the `ConstantInfo` object.

We now separate these things:

The first part will lower constants, determine lazy or not, determine
type. This will recursively walk the constant DAGs and create
`Constantinfo` as needed for all of them.

=> Each `ConstantInfo` (representing information about a `Constant`)
   will now also remember the child constants (in the form of
   `List<ConstantInfo> children`) it will use when defining the constant.

=> When code generation uses a constant we will remember that that
   module-use of the constant and all it's child constants.

=> Representing this as `constantInfo.children` avoids recursive AST
   visiting, avoids re-lowering the constants and ensures we don't have
   to keep two AST visitors in sync.

Part 3) Tracking constant uses

When the code generation uses a constant, we remember it being used in
the module being currently compiled. We use this usage information in
the final stage to determine where to place constants.

Special situation: If we have constant uses across modules where
deferred loading is involved. For example here:
    ```
    import 'foo.d.dart' deferred as foo;

    main() {
      ...
      print(foo.topLevelConstant);
    }
    ```
which gets lowered to something like this
    ```
      StaticInvocation(target=print, args=[
        let
          _ = StaticInvocation(target=checkLibraryIsLoaded, args=[StringLiteral('foo')])
        in
          ConstantExpression(topLevelConstant)
      )
    ```

Even though the main module is using the `topLevelConstant` it does so
under what I call a deferred loading "load guard": The code accessing
the constant will never be executed unless the `foo` deferred library
was successfully loaded.

=> We make our usage tracking consider such uses not a usage of the main
   module but rather the module containing deferred library of the
   "load guard".
=> The `CodeGenerator` will track the active "load guard" when it goes
   down the tree.
=> This allows pushing constants to deferred modules even if they are
   used in main module code.

Part 4) Defining of constants

During code generation we (generally speaking) emit a patchable region &
record the constant use of the constant DAG (see above).

During the linking phase we then have global knowldge of constant uses
and start defining them.

Theoretically we want to define a constant in a wasm module in the
loading graph where all using modules have it as direct or indirect
dependency but the dependency being the closest one to the uses.

=> As simplification for now: If two different modules use a constant we
   place it in the main module. We can later on make this more precise if
   complexity is warrented.

To avoid emitting many patchable regions that we later on have to fix
up we add an optimization during code generation:

=> As soon as a use is in the main module, we define the constant DAG
   in the main module.
=> As soon as there's 2 uses in different modules, we define the
   constant DAG in the main module.

Misc

* We separate constant definition from importing / exporting them.

* The new architecture changes constant visiting slightly so the names
  of constants in expectation files change as a side-effect of this.

Issue https://github.com/dart-lang/sdk/issues/61727

Change-Id: Ib44dee4c2514fb4af871e7078f5bfe43077922fd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/458240
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
2025-10-30 16:30:28 -07:00

207 lines
5.8 KiB
Dart

// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
/// Represents a mapping from a range of generated instructions to some source
/// code.
class SourceMapping {
/// Start offset of mapped instructions.
final int instructionOffset;
/// Source info for the mapped instructions starting at [instructionOffset].
///
/// When `null`, the mapping effectively makes the code unmapped. This is
/// useful for compiler-generated code that doesn't correstpond to any lines
/// in the source.
final SourceInfo? sourceInfo;
SourceMapping._(this.instructionOffset, this.sourceInfo);
SourceMapping(
this.instructionOffset, Uri fileUri, int line, int col, String? name)
: sourceInfo = SourceInfo(fileUri, line, col, name);
SourceMapping.unmapped(this.instructionOffset) : sourceInfo = null;
SourceMapping shiftBy(int shift) {
if (shift == 0) return this;
return SourceMapping._(shift + instructionOffset, sourceInfo);
}
@override
String toString() => '$instructionOffset -> $sourceInfo';
}
class SourceInfo {
/// URI of the compiled code's file.
final Uri fileUri;
/// 0-based line number of the compiled code.
final int line;
/// 0-based column number of the compiled code.
final int col;
/// Name of the mapped code. This is usually the name of the function that
/// contains the code.
final String? name;
SourceInfo(this.fileUri, this.line, this.col, this.name);
@override
String toString() => '$fileUri:$line:$col ($name)';
@override
bool operator ==(Object other) {
if (identical(this, other)) {
return true;
}
if (other is! SourceInfo) {
return false;
}
return fileUri == other.fileUri &&
line == other.line &&
col == other.col &&
name == other.name;
}
@override
int get hashCode => Object.hash(fileUri, line, col, name);
}
class SourceMapSerializer {
final List<SourceMapping> mappings = [];
void addMapping(int instructionOffset, SourceInfo? sourceInfo) {
final mapping = SourceMapping._(instructionOffset, sourceInfo);
mappings.add(mapping);
}
void copyMappings(SourceMapSerializer other, int offset) {
for (final mapping in other.mappings) {
mappings.add(SourceMapping._(
mapping.instructionOffset + offset,
mapping.sourceInfo,
));
}
}
String serialize() => _serializeSourceMap(mappings);
}
String _serializeSourceMap(List<SourceMapping> mappings) {
final Set<Uri> sourcesSet = {};
for (final mapping in mappings) {
if (mapping.sourceInfo?.fileUri != null) {
sourcesSet.add(mapping.sourceInfo!.fileUri);
}
}
final List<Uri> sourcesList = sourcesSet.toList();
// Maps sources to their indices in the 'sources' list.
final Map<Uri, int> sourceIndices = {};
for (Uri source in sourcesList) {
sourceIndices[source] = sourceIndices.length;
}
final Set<String> namesSet = {};
for (final mapping in mappings) {
if (mapping.sourceInfo?.name != null) {
namesSet.add(mapping.sourceInfo!.name!);
}
}
final List<String> namesList = namesSet.toList();
// Maps names to their index in the 'names' list.
final Map<String, int> nameIndices = {};
for (String name in namesList) {
nameIndices[name] = nameIndices.length;
}
// Generate the 'mappings' field.
final StringBuffer mappingsStr = StringBuffer();
int lastTargetColumn = 0;
int lastSourceIndex = 0;
int lastSourceLine = 0;
int lastSourceColumn = 0;
int lastNameIndex = 0;
bool first = true;
for (int i = 0; i < mappings.length; ++i) {
final mapping = mappings[i];
final sourceInfo = mapping.sourceInfo;
if (sourceInfo == null && first) {
// Initial parts of the code will be unmapped my default, we don't need to
// explicitly unmap them. More importantly, current version of binaryen
// cannot handle single-segment mappings at the beginning of the mappings.
// We can remove this block of code after switching to a version with
// https://github.com/WebAssembly/binaryen/pull/6794.
continue;
}
first = false;
lastTargetColumn =
_encodeVLQ(mappingsStr, mapping.instructionOffset, lastTargetColumn);
if (sourceInfo != null) {
final sourceIndex = sourceIndices[sourceInfo.fileUri]!;
lastSourceIndex = _encodeVLQ(mappingsStr, sourceIndex, lastSourceIndex);
lastSourceLine = _encodeVLQ(mappingsStr, sourceInfo.line, lastSourceLine);
lastSourceColumn =
_encodeVLQ(mappingsStr, sourceInfo.col, lastSourceColumn);
if (sourceInfo.name != null) {
final nameIndex = nameIndices[sourceInfo.name!]!;
lastNameIndex = _encodeVLQ(mappingsStr, nameIndex, lastNameIndex);
}
}
if (i != mappings.length - 1) {
mappingsStr.write(',');
}
}
return """{
"version": 3,
"sources": [${sourcesList.map((source) => '"$source"').join(",")}],
"names": [${namesList.map((name) => '"$name"').join(",")}],
"mappings": "$mappingsStr"
}""";
}
/// Writes the VLQ of delta between [value] and [offset] into [output] and
/// return [value].
int _encodeVLQ(StringSink output, int value, int offset) {
int delta = value - offset;
int signBit = 0;
if (delta < 0) {
signBit = 1;
delta = -delta;
}
delta = (delta << 1) | signBit;
do {
int digit = delta & _vlqBaseMask;
delta >>= _vlqBaseShift;
if (delta > 0) {
digit |= _vlqContinuationBit;
}
output.write(_base64Digits[digit]);
} while (delta > 0);
return value;
}
const int _vlqBaseShift = 5;
const int _vlqBaseMask = (1 << 5) - 1;
const int _vlqContinuationBit = 1 << 5;
const String _base64Digits = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmn'
'opqrstuvwxyz0123456789+/';