Even when non-nullable is enabled by default, enabling the experiment
explicitly should result in the experiment release version (and not
the experiment enabled version) to be used for opting in.
For this change, the semantics of parseExperimentalFlags was change
to _not_ normalize the flags to a full mapping including default values.
For this reason all uses of such maps are renamed to
'explicitExperimentalFlags'.
Closes#43879
Change-Id: I0d0262e68ec1403549abcfd305ae3a4404fe93e3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/168654
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Johnni Winther <johnniwinther@google.com>
Front-end server prints all dependencies after compilation, which could
result in a lot of output when AOT compiling a large application.
This change adds --no-print-incremental-dependencies option which
suppresses extra output. This skips printing dependencies which
takes time and avoids I/O which may be blocked.
Issue: https://github.com/dart-lang/sdk/issues/43299
Change-Id: I7779d3b5f1b513c2370978a5384a71cff371f017
b/154155290
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167860
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Most of the time a hot restart will bail out due to the changed libraries,
but since the operation is pointless anyway it is better to short-circuit.
Change-Id: I86d3266f1502f612b0dd7cc6b7409c05bd7456c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/165381
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Jonah Williams <jonahwilliams@google.com>
- Move expression evaluation to ddc in preparation for google3
- Added server to ddc to handle update and compileExpression requests
- Added tests
- Added 'experimental-output-compiled-kernel' option to ddc to generate
full kernel files only for compiled libraries, and store with
'.full.dill' extension
- Added AssetFileSystem to communicate to the asset server in the
debugger
- Made expression_compiler_worker work with full kernel files,
so removed invalidation of current file to improve performance
- Made expression_compiler_worker reuse already loaded imports
to avoid reading them from source in the incremental compiler
- Updated tests to work with DDC (for simulating webdev)
- Disabled tests that work with bazel kernel worker for now
as it does not generate full dill files yet
- Addressed code review comments from the prototype version:
https://dart-review.googlesource.com/c/sdk/+/157005
Details:
Currently, in flutter tools, expression evaluation is supported via
expression compilation, which is done by the incremental compiler in
the frontend server. The same incremental compiler is used for initial
application compilation, incremental code compilation for hot reload,
and any number of expression compilation requests.
In google3, the apps are typically too large to be compiled as a whole
in memory by the frontend server. Build in google3 is currently done by
blaze, as a distributed build using a task dependency graph. Build tasks
output kernel outline files as an interface between components produced
by individual tasks.
We are proposing an implementation of the expression compilation in
google3 that is taking advantage of full kernel files produced by the
build (supporting build changes to follow). This change introduces a
small server based on dev_compiler, which can handle following requests:
- update: load full kernel for given modules (done on app start)
- compileExpression: compile expression in a given library and module
(done when paused on a breakpoint)
Expression compilation uses previously loaded kernel files for the
application component and its dependencies to compile an expression.
Change-Id: Icf73868069faf3a2eb6d43ba78e459f8457e9e35
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160944
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
This change introduces handling of protobufs while doing type flow
analysis. Metadata in protobuf message classes is updated dynamically
according to the set of called accessors, invalidating and rebuilding
TFA summaries as needed.
Previously, protobuf-aware tree shaker required the 2nd run of TFA
in order to do the actual tree-shaking after protobuf messages are
pruned. This significantly increases compilation time.
AOT compilation time of a large app (--from-dill): 274s -> 152s
New tree shaker is available in kernel compilers under the flag
--protobuf-tree-shaker-v2.
Issue https://github.com/dart-lang/sdk/issues/42442
Fixes https://github.com/dart-lang/sdk/issues/40785
Change-Id: I4347896737b9b0f7407b845e614dda9ba7621921
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/152100
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Clement Skau <cskau@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
This option can be used to split AOT kernel generation into two rounds:
* The 1st round uses CFE to parse source files and generate intermediate
dill file in *non-AOT* mode.
* The 2nd round loads intermediate dill file and applies AOT global
transformations, then generates final dill file.
Each round is faster than one-shot generation of a dill file in AOT mode.
In order to use this 2-round compilation, command
gen_kernel <other-options> --aot -o output.dill main.dart
should be replaced with the following 2 commands:
gen_kernel <other-options> --no-aot -o intermediate.dill main.dart
gen_kernel <other-options> --aot --from-dill intermediate.dill -o output.dill main.dart
Bug: b/158101962
Change-Id: I2d4deef45b50eab11df47f4a372598e6cf1d0edf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/150272
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Commit-Queue: Alexander Markov <alexmarkov@google.com>
The first compilation result is leaked in two ways:
1) Directly by saving the component in a variable; and
2) Via an unfortunate context thing, probably a variation of
http://dartbug.com/36983. I will update that bug with a reproduction
example later.
The reason this creates a (big) leak is illustrated with an example:
* Say the first component (A) has 10 libraries in it. Each of these
libraries has parent pointers and points (currently) to A, which again
points to all of the 10 libraries.
* We then do a recompilation, say 5 libraries are reused and 5 are new.
They are put into a component (B). We really should have 10 libraries,
the 5 old ones and the 5 new ones (and for simplicity lets say these are
the ones in B). Notice that the 5 old ones will have their parent
pointers updated and also still be in the list of libraries in A.
We keep 15 libraries alive because we have the 10 original ones saved via
A and the 10 (where 5 is new) saved via B.
* We then do a recompilation, say 2 of the same libraries as was also
recompiled before, these end up in compnent C which has 5 libraries from A,
3 libraries from B and the 2 new ones. All of these libraries will have
their parent pointers updated to point to C.
* Because we saved A we keep all the 10 libraries in A though.
Because we saved A and some of the libraries in A had parent pointers
updated to point to B we also keep B and all libraries in B.
Because we keep B and some of the libraries in B had parent pointers
updated to point to C we also keep C and all libraries in C.
So instead of only having the 10 "live" libraries, we have 10 + 5 + 2 = 17
libraries, a leak of 7. With more compilations this keeps happening and
the leak keeps growing.
This CL stops the leak by not holding on to A (which, in turn, stops holding
on to B etc.)
Change-Id: If4f8b1e240b7c39f084df9cb2690570ff26fa9b3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/149280
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
During expression compilation, if the compiled expression is using
a type that was not used in the current function, the incremental
compiler and DDC produce code that refers to undefined types.
To fix that, discharge types created during expression compilation
and add their definitions at the beginning of the generated
function.
Added tests to validate the behavior.
Fixes: https://github.com/dart-lang/sdk/issues/41443
Change-Id: If3fdce85a50ab2590d107f3b9e8c19b4768fd482
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/148468
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
- Remove collecting private fields from scope calculation
We only need to collect the fields from the evaluated
expression and they are already collected separately.
- Correct private fields calculation
Currently, library names are assumed to be the same as the module,
which creates redefinitions of library variable in produced Javascript
code, and exceptions in chrome for modules that contain more than one
library.
The actual fix will come from making DDC incremental by storing more
information, for now make a best effort to compute correct names for
library variables from the module data that expression compiler
receives from chrome. The following code will be produced to calculate
private symbols _pf1 (from lib1, module1) and _pf2
(from lib2 module1):
let lib1 = require('module1.dart').lib1;
let lib2 = require('module1.dart').lib2;
let _pf1 = dart.privateNames(lib1, '_pf1');
let _pf2 = dart.privateNames(lib2, '_pf2');
Note that this seems to work even if the current breakpoint is inside
the define statement for lib1, and the lib1 symbol is not exported
yet.
Closes: https://github.com/dart-lang/sdk/issues/40272
Closes: https://github.com/dart-lang/sdk/issues/41585
Change-Id: I141719d2d5d5c08dd3c0ef5f0406756dce5575ab
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/145307
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
Currently expression evaluation in frontend server throws exception
when it cannot find a ProgramCompiler (DDC) to compile the module
where the expression is located. Given that IDEs use expression
evaluation for many information displaying cases, such as watch window,
hover over, and more, this results in unexplained exceptions thrown
during debugging.
Instead of throwing exception, return an error for the debugger
to report.
Fixes: https://github.com/flutter/flutter/issues/54520
Change-Id: Ie87c67d715a76f70439702181114ce422fad7ce1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143660
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
This relands commit ff34fd8110,
but puts tree shaking of write-only fields under the flag which is
disabled by default.
So far tree shaking was removing fields which are not used at all.
This change improves tree shaking of fields so fields
which are only written or used as interface targets can be removed.
The following limitations apply:
* Field is not removed if there is a constant object with that field, as
it may impact identity of constant objects which is an observable
behavior.
* Instance field is not removed if it has a non-trivial initializer as
it may have side-effects when executed by constructors.
* Late final fields are not removed, as writing such fields may have
side-effect.
* When field is removed, we may need to introduce an abstract getter
or abstract setter if field is used as a target of an interface call.
If a field was written, then setter would be non-abstract (but empty).
Issue https://github.com/dart-lang/sdk/issues/35310
Change-Id: Iec75b8301892664f4f955a01e5960b17e6620531
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/143286
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
ExpressionCompiler currently translates offsets of scopes to line
numbers during the binary search in the library in order to find
the inner-most scope containing the line where debugger has stopped.
This results in many offset-to-line translations per evaluation request,
each translation also performing a binary search in Source.lineStarts
table.
To make scope lookup more efficient, this change uses Source.lineStarts
table to find the offset for the debugger stop once, and uses integer
offset comparisons instead in the scope binary search.
Closes: https://github.com/dart-lang/sdk/issues/40281
Change-Id: I72052b144d15acef684a5f8d29b40f6c5774b723
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142461
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
New option can be used instead of --aot to produce a tree-shaken
minimal .dill file which can run on the VM in JIT mode.
Can be used along with --no-link-platform and --gen-bytecode to further
decrease size of the .dill file.
Change-Id: I35e8d81263a5e223decf160d79322c06cfd27321
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/140740
Commit-Queue: Alexander Markov <alexmarkov@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
This allows us to start experimenting with the legacy module format in flutter web/internal.
This also updates the DDC module name parsing to throw an ArgumentError on invalid formats, and exposes the parsing method.
Change-Id: I4ce85ecd4078a9827aba046e34425a4f1d1e9abd
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142263
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
ExpressionCompiler crashes while detecting current scope
if there are any scopes in the library with valid fileOffset
and invalid fileEndOffset.
Make ExpressionCompiler more tolerant to incomplete offset
information by just skipping scopes where fileEndScope is not set.
Fixes: https://github.com/dart-lang/sdk/issues/41241
Change-Id: I357017be1cb641053f23144edd5b74f30a9240ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/142080
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
Module names and import module names generated by the frontend server
are currently breaking debugger's assumptions on module names. This
change fixes it to make debugger works with various compiler
configurations, such as build_runner, frontend server with dwds tests,
and unblocks flutter tools to continue debugger and frontend server
integration.
JavaScriptBundle
- Change module import names to module names instead of file paths,
omit leading '/' as it cannot be in a module name in requireJS
- Module name changes are hidden under "debugger-module-names" flag
to prevent breaking flutter tools.
FrontendServer
- Always call exit after listening to commands is finished,
otherwise process running frontend server does not exit on 'quit'
command.
- Call computeCanonicalNames after initial compilation to support
subsequent module importing in compileExpressioToJs calls
- Add error handling to usage of kernelToJs compilers
ExpressionCompiler
- Separate library variables from library fields for loading modules,
as they can be different, for example
```main = require('web/main.dart').web__main```
Closes https://github.com/dart-lang/sdk/issues/40832
Change-Id: Ic103c22932c7181ee325f239193d32361d057ed0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/138010
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Jonah Williams <jonahwilliams@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
This is a reland of 344c584ec4
First attempt failed due to lack of implementation of the new
API from SDK in Flutter/engine that this change introduces.
Fixed by creating dummy implementation in flutter/engine:
https://github.com/flutter/engine/pull/16309
Original change's description:
> Implementation of evaluation to JS in frontend server
>
> - made ProgramCompiler somewhat incremental by letting the user
> set the current library and class
> - fixed a bug in procedure_builder where extension method wrappers
> did not have correct fileEndOffset set, which prevented finding
> dart scope at a line when extension methods are used
> - added new compileExpressionToJs API to frontend server and
> frontend compiler interface
> - added ExpressionCompier class that combines IncrementalCompiler
> and ProgramCompiler to compile expression at given dart location
> to JavaScript
> - in JavascriptBundle, set the module name for library tracking in
> JavaScript and saved program compilers to a table for incremental
> reuse in expression compiler
> - Exposed generator from IncrementalCompiler for use in
> ExpressionCompiler
> - added tests for expression compiler
> - added test for compileExpressionToJs API in frontend server tests
>
>
> Change-Id: Ic5e7829e07030b8ad044da7d35bcf27e9fba81c8
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132701
> Commit-Queue: Anna Gringauze <annagrin@google.com>
> Reviewed-by: Alexander Aprelev <aam@google.com>
> Reviewed-by: Sigmund Cherem <sigmund@google.com>
> Reviewed-by: Gary Roumanis <grouma@google.com>
> Reviewed-by: Jake Macdonald <jakemac@google.com>
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
Change-Id: I56426d2b6d831b2f8d8c57d468d0b54833a0062e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/134561
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
- made ProgramCompiler somewhat incremental by letting the user set
the current library and class
- fixed a bug in procedure_builder where extension method wrappers
did not have correct fileEndOffset set, which prevented finding
dart scope at a line when extension methods are used
- added new compileExpressionToJs API to frontend server and
frontend compiler interface
- added ExpressionCompier class that combines IncrementalCompiler
and ProgramCompiler to compile expression at given dart location
to JavaScript
- in JavascriptBundle, set the module name for library tracking in
JavaScript and saved program compilers to a table for incremental
reuse in expression compiler
- Exposed generator from IncrementalCompiler for use in
ExpressionCompiler
- added tests for expression compiler
- added test for compileExpressionToJs API in frontend server tests
Change-Id: Ic5e7829e07030b8ad044da7d35bcf27e9fba81c8
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132701
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Also updates StrongComponents to prefer the main component as the root if it is in a cycle. This makes bootstrapping more predictable.
This enables a clean integration of build_runner or ddr (internal) and the frontend_server by allowing it to work with any dev server.
All that is needed is a custom .packages file with `http` uris pointing at a dev server which does all code generation, etc.
Bug: https://github.com/dart-lang/webdev/issues/865
Change-Id: I14533d71e5f7ddac58d98073cf016c2589165e9f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132962
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Converts `package:` import uris into `/packages/` modules.
Also renames the output modules to append `.lib.js` instead of just `.js`. This allows us to distinguish between modules and applications based on extension.
Updates DDC source map code to be able to convert absolute file uris in sources so that they are relative to the source map.
Bug: https://github.com/dart-lang/webdev/issues/865
Change-Id: I55d70aa3761f10cc8bd7e92f5b567478040660de
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/132300
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Jonah Williams <jonahwilliams@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
Alexander Markov