This is a reland of commit 244f8cee87
with fixes for the file URIs passed on the command line.
Original change's description:
> [ddc] Fix running d8 with test.py
>
> - Add build targets to compile .dill outlines for common test
> dependencies using kernel_worker.dart.
> - Build ddc modules for common test dependencies and move amd versions
> to a separate subdirectory.
>
> Change-Id: I126b5821f6aea00b753d83560c76f00c6c5edc61
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350081
> Reviewed-by: Sigmund Cherem <sigmund@google.com>
> Commit-Queue: Nicholas Shahan <nshahan@google.com>
Change-Id: I5a77512984fda1d91c812f1e240e9c10609914f1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/351622
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
- Add build targets to compile .dill outlines for common test
dependencies using kernel_worker.dart.
- Build ddc modules for common test dependencies and move amd versions
to a separate subdirectory.
Change-Id: I126b5821f6aea00b753d83560c76f00c6c5edc61
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350081
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
This reverts commit 659fdae898.
Reason for revert: Broke the HHH bots.
The fix according to zra@ should be:
"The new dependencies should be added as dependency overrides to flutter_frontend_server's pubspec.yaml: https://github.com/flutter/engine/blob/master/flutter_frontend_server/pubspec.yaml"
Original change's description:
> Move most of the code from u/b/kernel_worker.dart to p/frontend_server
>
> This should allow us to import the code in cases when we want to
> implement the Bazel worker loop ourselves. This opens some possibilities
> of extra optimizations for internal use cases.
>
> I'm leaving some of the code (e.g., `main`) in the old location, so that
> we do not break anything that depends on it.
>
> The only thing that I've done is to pass the input-digest map to
> `computeKernel` instead of using `Input` objects directly, this way
> we can avoid the dependency on `package:bazel_worker` from
> `frontend_server`. So now, if we're using the worker loop, the map will
> be computed by `kernel_worker.dart` (in non-worker mode, we rely on
> the default empty map).
>
> Bug: http://b/187910394
> Change-Id: I408407ebdebc17c7898552c053a8e1c269cf2df5
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201187
> Commit-Queue: Michal Terepeta <michalt@google.com>
> Reviewed-by: Johnni Winther <johnniwinther@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: http://b/187910394
Change-Id: Ieb44d61b668a7d4629a27eef5a6ef61d0db9deaf
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201880
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Phil Quitslund <pquitslund@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
Commit-Queue: Jake Macdonald <jakemac@google.com>
Commit-Queue: Alexander Aprelev <aam@google.com>
Commit-Queue: Phil Quitslund <pquitslund@google.com>
This should allow us to import the code in cases when we want to
implement the Bazel worker loop ourselves. This opens some possibilities
of extra optimizations for internal use cases.
I'm leaving some of the code (e.g., `main`) in the old location, so that
we do not break anything that depends on it.
The only thing that I've done is to pass the input-digest map to
`computeKernel` instead of using `Input` objects directly, this way
we can avoid the dependency on `package:bazel_worker` from
`frontend_server`. So now, if we're using the worker loop, the map will
be computed by `kernel_worker.dart` (in non-worker mode, we rely on
the default empty map).
Bug: http://b/187910394
Change-Id: I408407ebdebc17c7898552c053a8e1c269cf2df5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/201187
Commit-Queue: Michal Terepeta <michalt@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Opt an initial batch of files under samples, samples-dev,
utils, and runtime/tools/dartfuzz out of null safety in preparation
for switching the flag on by default.
Change-Id: Icdfd52a5a969e678a7205903332f73fe3841c223
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/166960
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Leaf Petersen <leafp@google.com>
Problem:
We have been having a lot of reports of failures when initializing the incremental compiler
(see https://github.com/dart-lang/sdk/issues/38102).
These appear to be the result of empty kernel files but we have added other logic to check
for that when kernel files are written, which has had zero reports of throwing from users
(https://github.com/dart-lang/build/pull/2387).
We also use the same mechanism for temp directory management as we did with analyzer, but only
see this problem with the switch to kernel, and specifically the incremental compiler.
Solution:
Add retry logic and see if that fixes the problem. Any time there is a retry log a warning
so that we don't just silently do retries all the time.
This is a general defensive mechanism to cover a broad spectrum of potential failures with the
incremental compiler in the wild. I don't intend to remove it as it isn't harmful, and the
warning logs should be enough to encourage issues to be filed if it is happening often.
I have no direct reason to believe this will actually solve the particular linked problem other
than that we only see this issue when the incremental compiler is enabled.
Bug: https://github.com/dart-lang/sdk/issues/39122, https://github.com/dart-lang/sdk/issues/38102
Change-Id: Iaabb4497d6da69684692c1d7b9c030c59ebc6072
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/123001
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
This CL introduces 'tags' as a way to distinguish different setups and
be able to throw away previous state when it cannot be used.
These tags are - for now - basically filled up with the roots used for
the multi root filesystem, the idea being, that if they have changed we
cannot reuse the old state.
Change-Id: I19e069513ce3836f5bc6abf047e4359836fc7e09
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114945
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Before this CL one could ask to use the incremental compiler, but not
actually get the incremental compiler because one supplied linked inputs
(and not just summary inputs).
This is an artificial limitation, and inside the compiler there are no
real difference between "linked" and "summary" inputs. "Summaries" are
marked as "external" but that's a historical hack to mark it as something
we don't want to serialize --- something that is not used in this package.
Generally, the distinction and hack should go away entirely.
Considering there is no difference in this package, a good start would be
to remove the distinction in this package, but as this package is used
both internally and externally, this CL won't do that, but will just treat
the two 'different' input types the same.
Change-Id: I11a3962d22424387eca83212d08f535da350bd1b
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/113022
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Before this CL, if using the incremental compiler in bazels
kernel_worker, did not do summaries and did ask to "exclude-non-sources"
one would still get the entire thing (including loaded summaries etc)
(i.e. it would behave as if "exclude-non-sources" wasn't set).
This CL fixes it and makes it serialize as in the non-incremental case.
Change-Id: I396b53664a3d1ac32a243e903c8ff5c049fca453
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/113021
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
This change basically consists of these steps:
* Enable the incremental compiler to trace used inputs.
* Translate used libraries into used dill inputs.
* Output the list of used dills.
Bug: #37788
Change-Id: I08cffe299166cf10e990c9e261f190afd25da8b1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112384
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
When invoking the kernel worker in worker mode, reusing previous state,
but not using the incremental compiler, we will currently leak,
see dartbug.com/36983.
This was previously fixed in the incremental compiler case, but this CL
moves the fix outside the if to fix it in every configuration.
Change-Id: I993fecbcc053d2bca64af9e65d270f75411f2e68
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103626
Reviewed-by: Kevin Millikin <kmillikin@google.com>
This includes:
* accepting null values like sdk-summary or .packages (which are not used when building kernel for the sdk itself)
* allow enabling language experiments
* only exclude-non-sources on the non-incremental code path if that's requested
Change-Id: I08eeb643676f1f1406f0f3030c341d68179d42a2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103574
Reviewed-by: Nate Bosch <nbosch@google.com>
This CL fixes a number of issues:
* Semi-leaking old components when starting over
(i.e. when loading a new sdk)
* Setting new filesystem not really working because the old one was
cached in the ProcessedOptions
* Leaking old state because of an apparent VM bug (http://dartbug.com/36983)
Change-Id: Ib124257c456b019941e253409c1ef56eedcf66b2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103121
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Had to do a few changes to make this happen:
1. Restored some of the code removed in the transformer, making class
hierarchy optional.
2. Split the "compile" method in bazel_worker.dart into two, since we
need the intermediate Component object in one of the case.
Change-Id: I0ee004467e76dff0025f7979834c62b9db328ecc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100480
Commit-Queue: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jacob Richman <jacobr@google.com>
TL;DR: Unbind canonical names doesn't do what you think it does and
probably shouldn't (ever) be used. This CL stops using it in a few places.
Longer version:
When loading a dill file it:
- First loads the table of canonical names. These have no references yet.
- When a canonical name is asked for its reference it creates one if it
doesn't yet have one.
- When loading, for instance, a library, it asks for the reference.
When unbinding a canonical name:
- It removes itself (the canonical name) from the reference
- It removes the reference in itself
- Note: Whatever has a pointer to the reference keeps it, and the
reference points to whatever node it already pointed to.
This also means, that if we have a dill file that's split in two and:
- Load #1
- Load #2
that works fine, but if we
- Load #1
- Unbind canonical names
- Binds canonical names
- Load #2
stuff is not bound correctly (and an error is thrown).
And - the cause of this bug:
- Load #1
- Load #2
everything is fine
- Unbind canonical names
- Binds canonical names
- Load #2'
stuff is not bound correctly --- references points to stuff loaded as #2,
not as #2'. On top of being weird, wrong and confusing it also caused wrong
things to be but into the class hierarchy which ultimatly caused the crash.
This CL fixes it by not calling unbind and force loading of dill files
(at specific call sites) to create new libraries
(and in the process overwriting references ".node").
Revert "[dartdevc] Retry ddc incremental compile on crash"
This reverts commit ecdbdf00b8.
Revert "[kernel_worker] retry on failure"
This reverts commit 43eebea5a3.
Fixes#36644
Bug: #36644
Change-Id: Id8f548179e6a409b01f2ebfa3219f94cb64b1c05
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/100380
Reviewed-by: Kevin Millikin <kmillikin@google.com>
Reviewed-by: Vijay Menon <vsm@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jenny Messerly <jmesserly@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>
Without this, if an error occurs during serialization you get this message:
```
Internal problem: Compiler cannot run without a compiler context.
```
Instead of the real error.
Change-Id: I016ec0923a2e01b68f850988724be9d30179f78d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/98740
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Auto-Submit: Jake Macdonald <jakemac@google.com>
This allows dartdevc and the bazel worker to user
the incremental compiler with kernel.
Use --reuse-compiler-result --use-incremental-compiler to run in this mode.
Change-Id: I9189ce5f1a51320d9d96e071f8c95c80fe6fca84
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/95386
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Reviewed-by: Jenny Messerly <jmesserly@google.com>
This includes a `--target` flag (that in the future we can use for other targets
too) and a flag to accept the libraries.json file (currently needed to properly
compute constants of the form `bool.fromEnvironment('dart.library.io')`.
Change-Id: I8beea48610f0e1e721b7c76e545c9f662e3255ad
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/96849
Commit-Queue: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
This flag was always true so we can remove it and specialize the
remaining code that used it. The only code that used it was in the
Dart VM's async transformation.
Specializing that code led to some unused fields in the async helper
class which were removed. This triggered me to look at core types
which seems to have a lot of unused stuff.
Change-Id: I7da800ca17ac05478149f4c90337a38d2c2b1459
Reviewed-on: https://dart-review.googlesource.com/c/87960
Reviewed-by: Peter von der Ahé <ahe@google.com>
Commit-Queue: Kevin Millikin <kmillikin@google.com>
Nicholas Shahan