Reviewed in https://dart-review.googlesource.com/c/sdk/+/497583
[dartpad] Part 1: Protocol and Scaffolding
This sets up folders for `package:dartpad` and `package:dartpad_worker`,
R=athom@google.com, sigurdm@google.com
including:
* `OWNERS` files,
* Protocol definition,
* Exceptions, and,
* Virtual file system utilities.
[dartpad] Part 2: HotReloadCompiler
Adds the hot reload compiler logic to dartpad_worker.
This is models on dartdevc with `--reload-last-accepted-kernel` and
`--reload-delta-kernel` options. I don't think this does an incremental
compilation, instead it does a modular compilation. Meaning, we can have
a huge collection of precompiled DDC modules and the compiler just gets
a DDC outline/summary dill files for these modules. But on the other hand
the actual code that is compiled will be recompiled on every compile()
call.
This is very important for flutter_web.js / flutter_web.dill where the
precompiled DDC modules is around 60 MB in total. The fact that
package:flutter is precompiled into flutter_web.js is critical for
compilation performance for small dartpad-style example apps.
The downside is that unlike incremental compilation with _frontendserver_,
if the user were to install package:http and use it in the app they are
compiling then every `compile()` call will recompile `package:http`.
Where as with _frontendserver_ we'd be able to invalidate specific files,
and only those would be recompiled.
This leaves future work to either:
* (A) Add support for modular compilation to _frontendserver_, such that
we can do incremental compilation without having to first compile all
of `package:flutter`.
* (B) Refactor compilation strategy implemented here, to leverage
modular further. We could for example compile all dependencies from
`PUB_CACHE` into a `pub_cache_outline.dill` inside the dartpad worker.
Then recompilation would only touch the user-code. Granted this is
still less than ideal.
At the moment the lack of incremental compilation is not a blocker,
current dartpad doesn't have incremental compilation, but granted it also
only supported a single file. This dartpad environment will support
multiple files, but adding many files will result in poor performance
until we refactor and figure out how to do incremental compilation.
Hence, why we shouldn't block progress waiting for incremental compilation
to work.
R=nshahan@google.com
[dartpad] Part 3: Pub execution wrapper
Adds the internal wrapper for executing pub commands inside the
worker's virtual file system.
R=sigurdm@google.com
[dartpad] Part 4: Language Server wrapper
Adds the internal wrapper for spawning and communicating with the
Dart LSP inside the DartPad worker.
R=scheglov@google.com
[dartpad] Part 5: DartPad Worker
Implements the main worker entrypoint, RPC message handling. The
`bin/worker.dart` is what will be compiled to WASM and will run as a
_Web Worker_.
R=sigurdm@google.com
[dartpad] Part 6: DartPad Client library
The public-facing `dartpad` library that developers use to embed
the compilation environment in their web applications.
This finishes `package:dartpad`featuring:
* `DartPad.create()`, which creates a _Web Worker_ running the compiled
`pkg/dartpad_worker/bin/worker.dart`, and returns a `DartPad` instance
wrapping RPC calls into the _Web Worker_.
* `Sandbox.createIFrame()` which creates a _sandboxed iframe_ containing
precompiled DDC modules from the SDK assets and `sandbox.js` for
wrapping `ddc_module_loader.js` with an RPC interface. This returns a
`Sandbox` instance wrapping RPC calls into the _sandboxed iframe_,
making it easy to load modules, run library entrypoints (main()),
initiate hot-reload, or launch a flutter app.
This is an initial API design that works. There are minor inconsistencies
and improvements that we should do. But I propose that we do so in
follow-up PRs.
R=sigurdm@google.com
[dartpad] Part 7: Build Targets
Adds a `dartpad` target to the Dart SDK, which produces:
```
out/ReleaseX64/dartpad/
├── dart
│ ├── dart_sdk.js.map
│ ├── sdk.js
│ └── sdk.tar
├── ddc_module_loader.js
├── sandbox.js
├── worker.loader.js
├── worker.mjs
├── worker.support.js
├── worker.wasm
└── worker.wasm.map
```
The `dartpad/` output folder is intended to be used as `assetBaseUrl` in
client libraries provided in `package:dartpad`. And the `dartpad/dart/`
folder is intended to be used as `sdkLocation`. These files are not
intended to be distributed along side the normal Dart SDK, instead we'll
publish them on a CDN and let people use `package:dartpad` to point at
these files. If they want to self-host everything, they can copy from our
CDN or rebuild the files using a local Dart SDK checkout.
These files are also necessary for testing, to ensure that we have proper
integration tests for `pkg/dartpad_worker/`. As we will want to publish
these files on a CDN the actual compiled size in release-mode matters,
thus, we have enabled wasm optimizations steps in this mode.
R=rmacnak@google.com
[dartpad] Part 8: Tests for `package:dartpad` and `pkg/dartpad_worker/`.
We have 3 kinds of tests:
* Unit tests (vm, browser or both),
* Worker tests (vm and browser),
* Integration tests (browser-only).
As the Dart SDK test runner simply executes `*_test.dart` files with
`dart` and doesn't support compiling tests to the browser, launching and
running tests in the browser, we employ `dart test` to run tests. We do
this by having `pkg/dartpad_worker/dart_test.yaml` specify that
`dart test` should look for files names `test_*.dart`, and we then create
`pkg/dartpad_worker/test/ci_test.dart´ to be a single test that simply
runs `dart test` as a subprocess. Thus, the only test file triggered by
Dart SDK test runner is `ci_test.dart`, and if you locally run `dart test`
this will run the individual `test_*.dart` files.
While this hack to use `dart test` isn't ideal it gives everyone a decent
work flow. And saves us from having to invent complex test harness for
`pkg/dartpad_worker/`.
**Worker tests** defined in `test/dart/worker/` and `test/flutter/worker/`
are all imported into `test_dart_worker.dart` and
`test_flutter_worker.dart` to reduce test compilation time. These have a
non-trivial compilation time because the worker is running in the test
process. This allows testing on VM, which can be easier to debug.
**Integration tests** unlike _worker tests_ launch the worker compiled by
the `dartpad` build target in a _Web Worker_. This has less compilation
overhead for the individual tests, but involves more setup, and can be
a bit harder to debug.
Both worker and integration tests employ `test/asset_server/` which serves
assets built by the `dartpad` build target, and serves as a pub-server for
tests. It also serves assets built by `tool/setup_local_flutter.dart`
which creates flutter assets, though this is not intended to run as part
of CI, since we do not have a Flutter checkout available. Thus, for the
time being flutter testing is local only. Maybe, later we'll move this
script to another repository.
R=sigurdm@google.com
Cq-Include-Trybots: luci.dart.try:pkg-linux-release-try,pkg-win-release-try
Change-Id: I3a99939ec5217b9f3a855fc5b9ad9699047d02cc
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/507321
Commit-Queue: Jonas Jensen <jonasfj@google.com>
Reviewed-by: Alexander Thomas <athom@google.com>
Jonas Finnemann Jensen
8c42f67d35