The current tracer does a separate visit of the entire JS AST using metadata from the JS printer in order to generate source maps. This new strategy uses a shadow tree tracking position info to generate source map metadata while the JS AST is bring printed. This saves us an entire tree walk and the memory overhead of storing metadata to pass to the separate source map tracer.
In local invocations of Dart2JS I see a memory improvement of >100MB in the emitter phase and a small time improvement of ~4%.
Change-Id: I622e7015e2eddc7835551f970386e13019394a79
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336760
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
- Removes @staticInterop JS types declarations in favor of
typedefs (the function types' reified types and the typed data's
reified types have changed to make the type hierarchy work in the
JS backends).
- Adds extension types to dart:js_interop
- Adds a fromInteropObject helper to JSObject to avoid casting to
an extension type
- Deletes now stale tests
- Refactors some dart2wasm @staticInterop declarations since they
can no longer implement JS types
- Updates extension types tests to use the prefix extension_type
instead of inline_class
- Updates comments
CoreLibraryReviewExempt: Backend-specific library.
Change-Id: Ibe04afac9585ddb99fcf6dbaa7f12050d8b876dc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332860
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Srujan Gaddam <srujzs@google.com>
Recognize larger regions of no-op control flow. Ideally we would be able to remove these regions from the CFG. Until then, pattern matching larger regions leaves fewer ghost conditions in the generated code.
With pattern matching, we are seeing more examples where this is necessary.
Bug: #29475
Change-Id: Ib0981400883631f56940288f8a32fbc5f3985dfb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/340067
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Stephen Adams <sra@google.com>
This is a reland of commit b9e25afcb4
Original change's description:
> [dart2js] Stop emitting a dill with closed world.
>
> All transformations to the dill are now performed with the CFE linker phase (phase0b). This means we no longer need to emit a dill as part of the closed world computation. This saves both time and memory resources associated with the cost of serialization during that phase.
>
> Change-Id: I28dde8a1eecd71e823880027505f62fc804fcca6
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332821
> Reviewed-by: Sigmund Cherem <sigmund@google.com>
> Reviewed-by: Mayank Patke <fishythefish@google.com>
> Commit-Queue: Nate Biggs <natebiggs@google.com>
Change-Id: Ia693213dcdc949a8ef4b4204cc033130274d2381
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336982
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Simplify the js_ast printer - it should not transform the program
while printing.
js_ast:
- Don't skip blocks containing a single statement.
- Don't rewrite arrow functions.
dart2js:
- Fix rewrite_async.dart to avoid a few singleton blocks.
- Static initializer thunks are transformed in arrow functions with expression bodies.
Change-Id: I4ab75c6ca7f580835a6c9d4eb45240d76df9d895
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336820
Commit-Queue: Stephen Adams <sra@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
This is a reland of commit bebd08746b
Original change's description:
> [dart2js] Evaluate CFE consts as part of phase 0b (CFE linker).
>
> Constants are current evaluated in a few places during closed world generation, primarily as part of the ScopeModelBuilder. The scope visitor was modifying the AST which meant we had to emit a new dill with these evaluated constants along with the closed world results.
>
> This change instead evaluates the constants directly after linking the Kernel as part of the global transformations. This means we can update the ScopeModelBuilder to not mutate the AST at all as all constants are already simplified.
>
> A potential follow up here is to simplify the ScopeModelBuilder since all nodes should already be simplified if they can be, we should be able to avoid visiting some children.
>
> After this change we only directly create a single ConstantEvaluator, the one in `load_kernel`. The const simplifier also creates one and a follow up CL moves this to to run right after this new transformation.
>
> Note: Alternate versions of this CL tried to make the global transformation simpler by either:
> 1) Running the const evaluator indiscriminately on all expressions. This didn't work because it lead to exponential computation on constants set up as a DAG (see tests/language/const/constant_dag_test).
> 2) Only evaluating ConstantExpression nodes to update UnevaluatedConstants. This does not cover all the cases where the ScopeModelBuilder is modifying the tree and lead to a different compiler output.
>
> Change-Id: I746d889b37feddc9ab6c386c6252016dec745e6e
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332601
> Reviewed-by: Mayank Patke <fishythefish@google.com>
Change-Id: I53871a57144a3a1bd363af141d8f31c8ffa5ca6d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/336221
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
All transformations to the dill are now performed with the CFE linker phase (phase0b). This means we no longer need to emit a dill as part of the closed world computation. This saves both time and memory resources associated with the cost of serialization during that phase.
Change-Id: I28dde8a1eecd71e823880027505f62fc804fcca6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332821
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Constants are current evaluated in a few places during closed world generation, primarily as part of the ScopeModelBuilder. The scope visitor was modifying the AST which meant we had to emit a new dill with these evaluated constants along with the closed world results.
This change instead evaluates the constants directly after linking the Kernel as part of the global transformations. This means we can update the ScopeModelBuilder to not mutate the AST at all as all constants are already simplified.
A potential follow up here is to simplify the ScopeModelBuilder since all nodes should already be simplified if they can be, we should be able to avoid visiting some children.
After this change we only directly create a single ConstantEvaluator, the one in `load_kernel`. The const simplifier also creates one and a follow up CL moves this to to run right after this new transformation.
Note: Alternate versions of this CL tried to make the global transformation simpler by either:
1) Running the const evaluator indiscriminately on all expressions. This didn't work because it lead to exponential computation on constants set up as a DAG (see tests/language/const/constant_dag_test).
2) Only evaluating ConstantExpression nodes to update UnevaluatedConstants. This does not cover all the cases where the ScopeModelBuilder is modifying the tree and lead to a different compiler output.
Change-Id: I746d889b37feddc9ab6c386c6252016dec745e6e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332601
Reviewed-by: Mayank Patke <fishythefish@google.com>
I suspect that when the list of type preserving selectors was created the first/last setters either didn't exist or were just overlooked. Any dynamic List could have its inferred type changed by these operations.
Due to Dart2js's type representation this bug affects more than just dynamic lists. We track some simple values as part of the type system. So an operation that modifies a value can technically modify the type, as Dart2js represents it, even if the "real" type is preserved.
In the added test we would represent the list literal's type as "List(length: 2, elementType: Bool(true))". Notice the type states the elementType is specifically true, not just bool. Thus the first/last operations are modifying that type. But since we aren't registering this type change, SSA optimizes away the call/index and inlines the element itself.
Interestingly, this primarily manifested for bools due to a check in SSA:
https://github.com/dart-lang/sdk/blob/main/pkg/compiler/lib/src/ssa/optimize.dart#L475
In that code we are inlining constant-like expressions for the arguments of static invocations (such as the argument to a Expect.isTrue call). However, a few lines earlier you will see we only inline bool constants:
https://github.com/dart-lang/sdk/blob/main/pkg/compiler/lib/src/ssa/optimize.dart#L467
So when the list element type is anything other than a bool, this inlining does not trigger thus avoiding the bug.
Bug: https://github.com/dart-lang/sdk/issues/53944
Change-Id: I4e893903f335fc99b13cf526736c27bb066a4bad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/334420
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
The Dart2JS team has agreed that this mode of compilation is no longer worth investing in at this time and its existence adds some overhead to other feature work so it is worth fully removing. In the future we may revisit this mode of compilation. Below is some more context for any future exploration in this space.
What didn't work with modular analysis:
- current modular analysis was computing impacts, which were dense (50% of the size of kernel)
- using it moved work to a modular phase, and cut Phase1 in half however end-to-end time was not better
- data overhead was very high
- it made it much harder to maintain invariants throughout the pipeline: the data is tightly coupled with the kernel AST, making it hard to make late modifications to the AST.
How to potentially make it better:
- make the data much more sparse
- make the data more independent from the kernel AST so that transformations are not breaking
- reduce the critical path in a more substantial way.
Note: We retain and ignore the commandline flags used for modular analysis in order to avoid breaking build pipelines that were passing them. We may remove these at a later date.
Change-Id: If574ce2358280ab5fedd89c62665328601e72e22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/333360
Reviewed-by: Mayank Patke <fishythefish@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
The only constructor invocation of `NonConstantValue` is in the deserializer. Which means we're never creating any of these objects to serialize.
Also remove unused `handleClosureCall` method in `pkg/compiler/lib/src/inferrer/builder.dart`
Change-Id: I04e492bc4824795dd8fc109ee7464be56b27cb93
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/332500
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
We recently added the ability to pass a `--write-dump-info-data=` flag to Dart2JS's emitter phase which will serialize the necessary metadata to run dump info as a standalone phase of the compiler.
This extra metadata was incurring an overhead since we were maintaining JS AST nodes longer in order to extract metadata out of them. With this change we get rid of that overhead. The nodes are only retained until they printed and then any references to them are freed.
After this change the memory usage of the emitter phase is the same with or without the `--write-dump-info-data=` flag. There is still however some additional overhead in runtime. This is of course unavoidable as we are doing extra work to serialize the data.
Change-Id: Ib45a0e1de96d41503cbd76217b841ff013ac0655
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331000
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
The element map indexing system adds complexity to our element model with little benefit. Serialization already has its own indexing logic that we now reuse for entities.
Some things to note:
1) Libraries are still assigned an index. We only use that index in one place in the namer to disambiguate members. I now assign these indices as a late final field when adding the libraries to the element map.
2) The sizes of serialized files get slightly bigger since serialization indexing uses "addresses" rather than incrementing numbers for the indices. This effect is very minimal (<4MB+ for the largest programs).
3) "Late" members (constructor and generator bodies) which are created during codegen must be tracked and registered separately. Before the index was used to determine which entities were "late".
I see a very small memory improvement across all phases because of this change. There is also a small regression in runtime because the entity map queries are hash map lookups instead of list index lookups now. I think this very small regression is worth the reduced complexity in the compiler.
If we care about getting that time regression back we can add an "IndexedEntityMixin" with a late final index member that the registry would set/manage internally and would allow it to go back to tracking entities in a List. The field should be private so it would only be used for this purpose. This would only be necessary if we find the added time overhead of the map lookups to be too much.
Change-Id: Idd6d22bfc6ab61943a07feb58b35287d50e7d72e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326461
Reviewed-by: Mayank Patke <fishythefish@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Simplifies indices management for serialization by reusing the same indexing readers/writers for each phase. This way the caches are shared across all the phases and we can decouple the indices with any specific data read.
This fixes an issue that prevented the indices from being used in certain read/write patterns. None of our current usages of these indices exhibited this pattern but later changes I have planned do run into this issue.
This new pattern is also more resilient to sharding/parallel data files. Previously there could be collisions if two files were written in "parallel" (such as the codegen shards) and later their address spaces had to be merged. Now we do the address space merging at read time when we have full knowledge of the parallel files and can therefore avoid collisions.
Change-Id: Iff4c1461e734fc00f251d81f9fff1b9db83484d6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/326460
Reviewed-by: Stephen Adams <sra@google.com>
Unfortuantely this golden test is set up to test two configurations, the
default dump-info and a experimental kernel-first dump-info. The latter
wants to reuse the same expectations to ensure the output is the same,
but it's configuration confuses how `dump_info_test.dart -g` works. If
it sees a discrepancy it will duplicate the expectation and add a new
one instead of updating the old one.
I haven't investigated the best way to fix that, but for now, this CL
remerges the expectation so that we continue to only have one copy of
the expectations.
Change-Id: I5d2781b85ae92e8e83b6a4c395a7353bcbcca17a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/331321
Commit-Queue: Stephen Adams <sra@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Auto-Submit: Sigmund Cherem <sigmund@google.com>
This renames ExtensionType.typeErasure and adds it to DartType. This also fixes the extension type erasure for when extension types are used in the arguments of an extension type.
DartType.extensionTypeErasure can be used by backends to easily
access the type without extension types for any type.
TEST=pkg/kernel/test/extension_type_erasure_test.dart
Change-Id: Ia49d273ed85111e3ae822720860a3e0be5ea0252
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/329960
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Reviewed-by: Srujan Gaddam <srujzs@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Dart2JS uses the file offset on these nodes to create JS source maps. We want to maintain these file offsets across the serialization layer when we compile from dill files rather than directly from sources.
Dart2JS tests were serializing via `--test-mode` flag and thus had incorrect source locations for some stack traces.
TEST=Updating tests for all of these in fasta offsets tests.
Change-Id: I33862462fbff84d88f8c51bdeb1efc5771cfb8b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/327160
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
This CL introduces StructuralParameter and StructuralParameterType
classes. They are intended to replace TypeParameter and
TypeParameterType respectively where those were used as type
parameters defined by FunctionTypes. Previously, type parameters of
FunctionTypes were represented by TypeParameter objects with the
‘parent’ field set to null. By introducing StructuralParameter and
StructuralParameterType this CL unambiguously separates the two
notions of type parameters.
TEST=existing
Change-Id: Ida3feb7ad96a7b2acef55840eacba9e36bf2a3e7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312264
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Aske Simon Christensen <askesc@google.com>
Commit-Queue: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Mayank Patke <fishythefish@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
When serializing JS, we can sometimes end up changing the order we visit ModularExpressions. This can mean their indices are not stable against serialization and so the emitted code (and as a result hashes for that code) can change if the compiler did any serializing during the compilation.
We can make the order stable by sorting the holders before we process them.
This removes diff in hashes when running this (`--test-mode` forces serialization):
```
out/ReleaseX64/dart-sdk/bin/dart compile js benchmarks/OmnibusDeferred/dart/OmnibusDeferred.dart --out=om1a/m.js
out/ReleaseX64/dart-sdk/bin/dart compile js benchmarks/OmnibusDeferred/dart/OmnibusDeferred.dart --test-mode --out=om1b/m.js
```
Bug: https://github.com/dart-lang/sdk/issues/53466
Change-Id: Ibec88ff6acb71f1e96cbdb1d5ef4200d370cc3c6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/325100
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Some customers use tools to preprocess this set of load IDs in order to optimize how code is loaded in their app. This flag allows us to emit this information as early as possible in the compiler without having to completely compile to JS.
Change-Id: I572d4f06ffd49a672767adc344819388a741e20c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/323340
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Isolates data needed from emitter phase for dump info task. The new DumpInfoProgramData captures only the necessary data and allows it to be serialized so that dump info can be run independently of the emitter task.
This change also ends up improving the combined emitter and dump info task. By processing the JS AST and extracting out the relevant info up front, the AST is able to be GCed before the dump info task runs. In local runs on a big app this resulted in ~1.4GB less ram being.
Runtime stat comparison:
Without --dump-info: 117s 5.4GB
Before with --dump-info: 205s 18.3GB
Before with --dump-info=binary: 148s 9.9GB
After with --dump-info: 205s 16.9GB
After with --dump-info=binary: 150s 7.5GB
After with --dump-info-write: 132s 6.2GB
After with --dump-info-read: 118s 14.8GB
After with --dump-info-read --dump-info=binary: 67.9s 5.5GB
By providing the new "--dump-info-write-uri=URI" a binary file will be written with the necessary dump info data. Then if Dart2JS is invoked with the same parameters and "--dump-info-read-uri=URI" then the compiler will skip generating/emitting JS and use the output file to generate the info dump. The existing invocation of just adding "--dump-info" works as before.
I also had the goal of removing DumpInfoTask dependencies on other parts of the compiler and vice versa.
Change-Id: I3f40b0136eb77a709c49737607a2ad6127127e49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322843
Commit-Queue: Nate Biggs <natebiggs@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Treat the loop update marker more like an ordinary symbolic value. The old symbolic marker attempted to do widening 'on the fly', which could lead to incorrect results when the update could reset the value to a constant lower than the starting value. The new version moves the widening to a single place, at the loop update.
Fix#53355 by caching intermediate results so that long chains of diamond control flow are not explored exponentially.
There are very few changes in apps. There is one change in a Flutter app that is like the changed codegen/value_range_test where the bounds check can be eliminated because the loop index may be decremented, but not more than the increment, so is still weakly monotonic. There is one change in a large ACX app where a lower bounds check is no longer removed but I *think* it was previously removed incorrectly, though it is hard to tell since it is in a huge function.
Issue: #48465
Issue: #53078
Issue: #53355
Change-Id: Ib125cd6bb30cef52f8dfcd53eaa13e439f26316c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322594
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Stephen Adams <sra@google.com>
We've been using the Unordered version (which support deferred serialization) of these for a while now. Removing the ordered version (and associated flags) since they're essentially dead code at this point.
Change-Id: I522682b8e0688deda6784628b65d41389cdb9791
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/322161
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Adds JSSymbol and JSBigInt and erases to either their respective
interceptor types in the JS backends or JSValue in dart2wasm.
CoreLibraryReviewExempt: Backend-specific library.
Change-Id: Ib2c70d22a70c6308733cd170b91eafa8ec3b3aeb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/321749
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Several changes are made:
- createDartExport now does not export methods that define type parameters.
- createStaticInteropMock adds conformance checks to make sure implementing
members can handle all possible values of a type parameter in an interop
member. An error is added to reduce confusion around this.
- Export creator now uses dart:js_interop_unsafe for a lot of its lowering
as the dart:js_util equivalents are buggy when it comes to calling exported
functions in JS with JS types.
- Small code changes are added to backends to handle the above changes.
Change-Id: Ie3b6b157930537267f270b60373b2b17e0a14344
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316141
Reviewed-by: Joshua Litt <joshualitt@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Srujan Gaddam <srujzs@google.com>
This renames InlineClass to ExtensionTypeDeclaration, and InlineType
to ExtensionType. Members of extension type declarations are called
extension type members instead of extension type declaration members
for "brevity".
TEST=existing
Change-Id: I91ed62533ddd345644492f04dc3310d007460288
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316780
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
This removes ExtensionType, ExtensionTypeShowHideClause and
CallSiteAccessKind from package:kernel which where only used by the now
removed 'extension-types' experiment.
A follow-up CL will rename InlineClass/InlineType to
ExtensionTypeDeclaration/ExtensionType to match the names of the
Extension Type feature currently being implemented.
TEST=existing
Change-Id: I58d2e8b0a92ac61329ee161cc6884a2c0e6f87ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316420
Reviewed-by: Chloe Stefantsova <cstefantsova@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Alexander Markov <alexmarkov@google.com>
The non-sharded tests takes 10 minutes and each of the sharded tests
takes 14 minutes. It's faster to shard immediately (costing one bot
more) and concurrently run the local tests.
Fix end to end dart2js test that times out when sharded and run outside
a directory called sdk.
Bug: b/290617138
Change-Id: If71f0d301edf565c9f15847098320106ca383635
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/312983
Reviewed-by: William Hesse <whesse@google.com>
- Only include the script tag src if it differs from the previous log entry.
- Use single letters for each event field's key.
- Fix some typos.
- Remove 'uri' from download event as it can be inferred from script tag src and part file name.
Change-Id: Ib7916ea5e92720a10c7e1e1cab2e535fae2bbf49
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/311180
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nate Biggs <natebiggs@google.com>
Nate Biggs