There were `pubspec.yaml` files in `tools/` that were not
included in the global package config.
That means that their imports were irrelevant, which could
be misleading. And if any tool would look at the
`pubspec.yaml` file, it might be inconsistent with
the actual package resolution.
Makes every `pubspec.yaml` file either be included by the
`tools/generate_package_config.dart` script, or deletes them
if they seem to be stale and unused.
(Compare vs. `git ls-files '**/pubspec.yaml'`.)
Excepted `tools/dart2js/sourceMapViewer/pubspec.yaml`.
The entire directory might be stale. The dependencies of
that pubspec are not SDK dependencies otherwise,
the pubspec has no SDK min-version, which is now a requirement,
and the README refers to a *packages directory*.
Keeping as-is and filing issue to have owners take a look.
Added `lib/` directory to `tools/` to avoid `tools/bots/`
being inside the package URI root of the `tools/` package,
which would cause its `../../pkg/...` import to fail.
Makes every `pubspec.yaml` file use a `^...` SDK constraint instead
of the longer `>= ... < ...` format.
Tested: No new tests, goal is to keep running the same way
Change-Id: I688e463fe985fc4de43550a1f4c7ff350536cffc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/414020
Commit-Queue: Lasse Nielsen <lrn@google.com>
Reviewed-by: Martin Kustermann <kustermann@google.com>
Reviewed-by: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Brian Quinlan <bquinlan@google.com>
This is a reland of commit b9b77058a9
Original change's description:
> Migrate to use pub workspace
>
> Use `pub get` to generate `.dart_tool/package_config.json` on gclient sync.
>
> All pkg/ (and a few third_party) packages that are developed inside the sdk repo are included in the workspace from the root `pubspec.yaml`.
>
> All dependencies that are pulled in via DEPS are added as path dependencies via `dependency_overrides` in the root `pubspec.yaml`.
>
> Bug: https://github.com/dart-lang/sdk/issues/56220
> Change-Id: I38c12b608c68da54c57821116cf9aa6696936746
> Tested: relies on CQ of existing tests. Should have no effect on functionality
> CoreLibraryReviewExempt: only core library change is adding a `// ignore:` comment. Should have no influence on functionality
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/397164
> Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
> Reviewed-by: Alexander Thomas <athom@google.com>
Bug: https://github.com/dart-lang/sdk/issues/56220
Change-Id: I29afabade2d2447dea05121cb87ff50bb21a4b76
Cq-Include-Trybots: luci.dart.try:flutter-linux-try,flutter-web-try
Tested: relies on CQ of existing tests. Should have no effect on functionality
CoreLibraryReviewExempt: only core library change is adding a `//
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415561
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
When parsing `--define` or `-D` arguments don't split the the value by
commas.
This is consistent with how dart2js handles `-D`, but inconsistent with
how VM handles it.
Example:
void main() {
print(const String.fromEnvironment("FOO"));
}
When compiled with `dart compile js -DFOO="a, b"` and run, dart2js
prints
a, b
VM prints (when compiled to exe)
a
Between these two, I think dart2js' behavior is more common, so we
follow dart2js.
Also update compile_benchmark to avoid splitting a single argument "a b"
into "a" and "b" when parsing the arguments and then splicing them back
before calling `dart2wasm`.
Also update the test runner and ddc batch mode argument parser to handle
splitting quoted arguments in `// dart2jsOption = ...` and the same
options for ddc and dart2wasm, by moving dart2js's `splitLine` to a new
library and reusing it in the test runner and ddc.
Fixes https://github.com/flutter/flutter/issues/164873.
See also https://github.com/dart-lang/sdk/issues/60341 for relevant
future work.
Change-Id: Idbdf69072fa212c8e4a390990577eb5a57b49e8a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415280
Reviewed-by: Martin Kustermann <kustermann@google.com>
Commit-Queue: Ömer Ağacan <omersa@google.com>
This reverts commit b9b77058a9.
Revert "Add missing sample pubspec to workspace"
This reverts commit 892ea15ac7.
These seem to break the engine.
Change-Id: Ieee26deb7928c3869a1b6265326c3ce568ffe731
Tested: this is a revert.
CoreLibraryReviewExempt: this is a revert
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/415582
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Use `pub get` to generate `.dart_tool/package_config.json` on gclient sync.
All pkg/ (and a few third_party) packages that are developed inside the sdk repo are included in the workspace from the root `pubspec.yaml`.
All dependencies that are pulled in via DEPS are added as path dependencies via `dependency_overrides` in the root `pubspec.yaml`.
Bug: https://github.com/dart-lang/sdk/issues/56220
Change-Id: I38c12b608c68da54c57821116cf9aa6696936746
Tested: relies on CQ of existing tests. Should have no effect on functionality
CoreLibraryReviewExempt: only core library change is adding a `// ignore:` comment. Should have no influence on functionality
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/397164
Commit-Queue: Sigurd Meldgaard <sigurdm@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Preparation for the bump to 3.7 and reformatting all the source code
in pkg/dev_compiler. This makes it obvious that the next bump will
only contain changes needed for 3.7.
Change-Id: I997358aea4fcf3d899490611353df8b067bde929
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/401387
Reviewed-by: Mayank Patke <fishythefish@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
- Copy members from `SharedCompiler` into `ProgramCompiler`.
- Delete `@override` annotations from the `ProgramCompiler`
members that are now the only implementation.
- Delete abstract members that were part of `SharedCompiler` and
copy the doc comments to the actual implementation.
- Delete unnecessary getters and just access the private field:
- `_symbolContainer`
- `_incrementalMode`
- Copy body of `super.setIncrementalMode()` into the call site.
- Copy all other members from shared_compiler.dart into
js_utils.dart.
- Fix imports and namespaces accordingly.
Change-Id: Ia94434a6f01a58bac4e1eb05a4414243dade6de4
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/376280
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nate Biggs <natebiggs@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
The hot reload runner currently only supports d8, but I plan to add support for Chrome and VM execution.
Notable changes:
* Creates `package:reload_test` with helpers for running this suite.
* Updates the module loader with D8-specific branches and hooks for hot reload/restart.
* Exposes DDC runtime variables via a `HotReloadTestRuntime` API.
* Ports constant equality hot restart tests from webdev/dwds (validated to fail if either cache-clearing mechanism fails).
* Partially rolls DDC's d8 preamble forward (towards dart2js's).
* Wraps D8's timer implementation with custom timeout logic to better match Chrome's timing semantics when executing with native JS async.
Tests for the framework and matrix updates will be added in an upcoming change.
Change-Id: I2773b29f464cfd0330e4c653c05e117ae150b4a6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/350021
Reviewed-by: Sigmund Cherem <sigmund@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Mark Zhou <markzipan@google.com>
- Allow running the tests with asserts enabled
- Allow running the tests in canary mode
- Unify common test code and remove duplication
- merge expression_compiler/setup_compiler_options.dart
into share_compiler_options.dart
- Merge TestCompiler code from expression evaluation and
expression compilation tests and move into
expression_compiler/test_compiler.dart
- Rename various TestDrivers so they have more descriptive names
- Remove 'golden' JS comparison tests from expression_compiler_test.dart
- replace by evaluation tests where needed
Closes: https://github.com/dart-lang/sdk/issues/53145
Change-Id: Ic797fa4ee9bfa6b858b924be9f9a53fd10ae1448
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/318080
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
It should not be necessary to ever run `pub get` for a package which is
not published. All packages used in the SDK are controlled by a single
package config, so it's not necessary to declare versions or paths for
any packages.
Remove all dependency overrides.
R=devoncarew@google.com
Change-Id: Icb328813b471f35ee4c99995f4e90aac4d8ed438
Tested: Covered by existing static analysis.
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/244767
Commit-Queue: Nate Bosch <nbosch@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
This reverts commit 243ac04dbf.
Reason for revert: Breaks roll to flutter engine still.
Original change's description:
> Sync packages from shelf mono repo
>
> Keep the DEPS entries and directories for the old locations for these
> packages, but ignore them in `generate_package_config`. Removing the
> hashes and directories would invalidate the DEPS file in the flutter
> engine repository.
>
> Add the `shelf` repository to the specially handled directories with
> nested packages in `generate_package_config`.
>
> Update path dependencies in pubspecs to the new location.
>
> Reland of https://dart-review.googlesource.com/c/sdk/+/243929
> without the removal of the old directory locations.
>
> R=devoncarew@google.com
>
> Change-Id: I3d3b3eb0722f3eba518a6a1034ed9c24f83c70f0
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/244300
> Reviewed-by: Devon Carew <devoncarew@google.com>
> Commit-Queue: Nate Bosch <nbosch@google.com>
TBR=devoncarew@google.com,nbosch@google.com
Change-Id: Ied6d9aa685208eddea6d82d04ca8876937651051
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/244302
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Keep the DEPS entries and directories for the old locations for these
packages, but ignore them in `generate_package_config`. Removing the
hashes and directories would invalidate the DEPS file in the flutter
engine repository.
Add the `shelf` repository to the specially handled directories with
nested packages in `generate_package_config`.
Update path dependencies in pubspecs to the new location.
Reland of https://dart-review.googlesource.com/c/sdk/+/243929
without the removal of the old directory locations.
R=devoncarew@google.com
Change-Id: I3d3b3eb0722f3eba518a6a1034ed9c24f83c70f0
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/244300
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
Drop the DEPS entries for the repositories which will be archived. Each
of the packages has been merged into the `shelf` repository.
Add the `shelf` repository to the specially handled directories with
nested packages in `generate_package_config`.
Update path dependencies in pubspecs to the new location.
R=devoncarew@google.com
Change-Id: Iefca4fdb2abb6bafa273b9b6b1b4f25d6c91b005
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/243929
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
Auto-Submit: Nate Bosch <nbosch@google.com>
Since the null-safety test migration we've been generating both a .packages file
and a package_config.json file. This deletes the use of the old .packages file
and updates all references to use the new file.
Change-Id: Iecef64ac8ed8579338795ad5327765118d643236
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/240650
Reviewed-by: Joshua Litt <joshualitt@google.com>
These are now enforced in pkg/js_ast so we don't need to worry about
introducing unwanted diffs between the forked versions.
Change-Id: I89efc97f53a433f98e2a06747690ac7a635c388d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/239560
Reviewed-by: Stephen Adams <sra@google.com>
Commit-Queue: Nicholas Shahan <nshahan@google.com>
Zero runtime affect to these unpublished packages
Allows using standard tools such as dart test, etc
Also allows better checking of dependency constraints
TEST=no effect on SDK build process
Change-Id: Ic6fbe6ce8f57724a87c058eccad64ce8e7473af1
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/218180
Auto-Submit: Kevin Moore <kevmoo@google.com>
Reviewed-by: Devon Carew <devoncarew@google.com>
Recent flakes in webdev CI show failures to load dill files
due to broken sockets (most likely caused by connection timeouts).
This change adds retries and timeouts to the asset server file
system to make it tolerant to breaking connections.
- Add RetryTimeoutClient wrapper for HttpClient that performs
retries with delays and connection/response timeouts, based
on the idea in
https://github.com/dart-lang/http/blob/master/lib/retry.dart
- Make AssetFileSystem use the RetryTimeoutClient.
- Add tests for reading multiple files concurrently.
- Add tests for ExpressionCompilerWorker reading assets using
standard, multiroot, and asset file systems.
- Simplify uri that AssetFileSystem uses to query the asset
server, since we no longer need to load package uris.
This will eliminate a step in uri translation in dwds.
- Validated that changes do not break google3 manually.
Change-Id: I2ff163b8d99cd0b7551f3f33a71e6b306540e3c9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/204880
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
We are seeing CI flakes in webdev when using expression_compiler_worker
in tests involving expression evaluation, for example, see linux tests
for dwds in the following PR:
https://github.com/dart-lang/webdev/pull/1343
The logs show that HttpFileSystemEntity.exists fails due to a broken
pipe, which we suspect is due to http client closed too early.
This change is an attempt to fix the problem by
- logging exceptions thrown in expression compiler worker
- awaiting for the request to be read before closing http client in
HttpFileSystemEntity.
TEST=pkg/dev_compiler/test/expression_compiler/asset_file_system_test.dart
See investigation of the CI issue in:
https://github.com/dart-lang/webdev/issues/1345
Closes: https://github.com/dart-lang/sdk/issues/46388
Change-Id: Iaf98f08e8ebb618bf8365bd497f01ae685f0d3f2
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203841
Commit-Queue: Anna Gringauze <annagrin@google.com>
Reviewed-by: Alexander Aprelev <aam@google.com>
Reviewed-by: Nate Bosch <nbosch@google.com>
Reviewed-by: Sigmund Cherem <sigmund@google.com>
The full workflow for an end-to-end test:
1) Perform test/compiler options pre-initialization.
2) Start an instance of Chrome (with 1 tab) and save a persisted debug connection for use in all tests.
3) For each separate Dart program to be tested:
1) Create inside a temp directory the HTML bootstrapper, DDC-compiled JS sources, and auxiliary JS files (dart_library.js or require.js).
2) Initialize a DDC ProgramCompiler/IncrementalCompiler and perform a full compile of the program, saving the compiler's state for subsequent incremental compiles.
3) Enable the debugger and set a URL breakpoint at the program's main JS entrypoint.
4) For each test for this Dart program:
1) Navigate to the HTML bootstrapper, pausing when the target module is loaded (at the breakpoint set above).
2) Set a breakpoint at a JS location translated from a Dart breakpoint ID via source maps.
3) Resume the debugger, pausing at that breakpoint. Remove the breakpoint.
4) Reconstruct the local JS scope, then evaluate the Dart expression.
5) Evaluate the JS expression inline, then compare results with expectations.
5) Disable the debugger to clear its state.
4) Shut down Chrome and clean up any temporary directories.
Other notes:
* Supports bootstrapping/compilation on AMD and DDC module systems only (though expression eval is only supported on AMD)
* Requires that Dart breakpoint IDs be set declaratively (no support for columns)
The test structure/flow is subject to change. While I port existing tests, Dart programs will consist of inline source file strings with simple breakpoints. In the future, we want to allow more complicated Dart programs to be specified (like in the modular test framework) and more complicated expression evaluation test patterns (rather than allowing only a single evaluation per test).
Change-Id: I5de7cfa66991fd14b24fa16edd1615f55c041ae3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190962
Commit-Queue: Mark Zhou <markzipan@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
- Move expression evaluation to ddc in preparation for google3
- Added server to ddc to handle update and compileExpression requests
- Added tests
- Added 'experimental-output-compiled-kernel' option to ddc to generate
full kernel files only for compiled libraries, and store with
'.full.dill' extension
- Added AssetFileSystem to communicate to the asset server in the
debugger
- Made expression_compiler_worker work with full kernel files,
so removed invalidation of current file to improve performance
- Made expression_compiler_worker reuse already loaded imports
to avoid reading them from source in the incremental compiler
- Updated tests to work with DDC (for simulating webdev)
- Disabled tests that work with bazel kernel worker for now
as it does not generate full dill files yet
- Addressed code review comments from the prototype version:
https://dart-review.googlesource.com/c/sdk/+/157005
Details:
Currently, in flutter tools, expression evaluation is supported via
expression compilation, which is done by the incremental compiler in
the frontend server. The same incremental compiler is used for initial
application compilation, incremental code compilation for hot reload,
and any number of expression compilation requests.
In google3, the apps are typically too large to be compiled as a whole
in memory by the frontend server. Build in google3 is currently done by
blaze, as a distributed build using a task dependency graph. Build tasks
output kernel outline files as an interface between components produced
by individual tasks.
We are proposing an implementation of the expression compilation in
google3 that is taking advantage of full kernel files produced by the
build (supporting build changes to follow). This change introduces a
small server based on dev_compiler, which can handle following requests:
- update: load full kernel for given modules (done on app start)
- compileExpression: compile expression in a given library and module
(done when paused on a breakpoint)
Expression compilation uses previously loaded kernel files for the
application component and its dependencies to compile an expression.
Change-Id: Icf73868069faf3a2eb6d43ba78e459f8457e9e35
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160944
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
Only ~29/100 calls for runtimeCall and ~2/31 calls for runtimeStatement
were using the single value option. Change to take a `List<Object>`
always and add list literals to those calls which were previously only
passing one value.
Update pubspec SDk constraint so that the analyzer will allow the `..?`
syntax.
Change-Id: Ib8eb8eb854a8aa04e36dc5cc713016a43b8e9325
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103361
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Commit-Queue: Nate Bosch <nbosch@google.com>
Lasse R.H. Nielsen