Mark Zhou
ce473df469
The full workflow for an end-to-end test:
1) Perform test/compiler options pre-initialization.
2) Start an instance of Chrome (with 1 tab) and save a persisted debug connection for use in all tests.
3) For each separate Dart program to be tested:
1) Create inside a temp directory the HTML bootstrapper, DDC-compiled JS sources, and auxiliary JS files (dart_library.js or require.js).
2) Initialize a DDC ProgramCompiler/IncrementalCompiler and perform a full compile of the program, saving the compiler's state for subsequent incremental compiles.
3) Enable the debugger and set a URL breakpoint at the program's main JS entrypoint.
4) For each test for this Dart program:
1) Navigate to the HTML bootstrapper, pausing when the target module is loaded (at the breakpoint set above).
2) Set a breakpoint at a JS location translated from a Dart breakpoint ID via source maps.
3) Resume the debugger, pausing at that breakpoint. Remove the breakpoint.
4) Reconstruct the local JS scope, then evaluate the Dart expression.
5) Evaluate the JS expression inline, then compare results with expectations.
5) Disable the debugger to clear its state.
4) Shut down Chrome and clean up any temporary directories.
Other notes:
* Supports bootstrapping/compilation on AMD and DDC module systems only (though expression eval is only supported on AMD)
* Requires that Dart breakpoint IDs be set declaratively (no support for columns)
The test structure/flow is subject to change. While I port existing tests, Dart programs will consist of inline source file strings with simple breakpoints. In the future, we want to allow more complicated Dart programs to be specified (like in the modular test framework) and more complicated expression evaluation test patterns (rather than allowing only a single evaluation per test).
Change-Id: I5de7cfa66991fd14b24fa16edd1615f55c041ae3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/190962
Commit-Queue: Mark Zhou <markzipan@google.com>
Reviewed-by: Anna Gringauze <annagrin@google.com>
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Package validation
The packages in pkg/ are automatically validated on the LUCI CI bots. The
validation is largely done by the tools/package_deps package; it can be tested
locally via:
dart tools/package_deps/bin/package_deps.dart
Packages which are published
There are several packages developed in pkg/ which are published to pub.
Validation of these packages is particularly important because the pub tools are
not used for these packages during development; we get our dependency versions
from the DEPS file. Its very easy for the dependencies specified in a package's
pubspec file to get out of date wrt the packages and versions actually used.
In order to better ensure we're publishing correct packages, we validate some properties of the pubspec files on our CI system. These validations include:
- that the dependencies listed in the pubspec are used in the package
- that all the packages used by the source are listed in the pubspec
- that we don't use relative path deps to pkg/ or third_party/ packages
Packages which are not published
For packages in pkg/ which we do not intend to be published, we put the following comment in the pubspec.yaml file:
# This package is not intended for consumption on pub.dev. DO NOT publish.
publish_to: none
These pubspecs are still validated by the package validation tool. The contents are more informational as the pubspecs for these packages are not consumed by the pub tool or ecosystem.
We validate:
- that the dependencies listed in the pubspec are used in the package
- that all the packages used by the source are listed in the pubspec
- that a reference to a pkg/ package is done via a relative path dependency