8072ed9ad7
* docs: integration tests design proposal Adds docs/integration_tests.md proposing a desktop-only integration suite that drives the Dart `ShorebirdUpdater` API against the real Rust core via FFI, with a fake HTTP server and per-test tempdir state. Describes a new `library_test_hooks` cdylib that reaches into `updater` via a `test-hooks` Cargo feature, so production C API stays clean. Status: design exploration, not a committed plan. * feat: stage 1 integration test harness for shorebird_code_push Lands the test_hooks crate, the Dart-side loader, and one trivial end-to-end test, per docs/integration_tests.md stage 1. Pieces: - New `library_test_hooks` workspace crate (cdylib). Depends on `updater` with a new `test-hooks` Cargo feature that widens the visibility of internal items (currently `testing_reset_config`) for sibling-crate access only — production builds do not enable it. The crate also re-exports `updater::c_api::dart::*` and `updater::c_api::engine::*`, which keeps the rlib's `#[no_mangle]` symbols out of DCE so the resulting cdylib carries both the production C API and the new `shorebird_test_*` hooks. - Workspace `default-members` excludes `library_test_hooks` so plain `cargo build` / `cargo test` invocations don't unify the `test-hooks` feature into production builds. - `shorebird_code_push/test/integration/all_test.dart` (single file, with a header comment explaining why) loads the cdylib via `DynamicLibrary.open`, reassigns the existing `@visibleForTesting` `Updater.bindings` setter, and exercises both surfaces. The build helper shells out to `cargo build -p library_test_hooks`; if it fails, `markTestSkipped` keeps the suite green on machines without a working Rust toolchain. - ffigen config (`ffigen_test_hooks.yaml`) generates the test-only Dart bindings under `test/integration/generated/`, not `lib/`. - CI: `library_test_hooks` is added to the rust_crate matrix, and the shorebird_code_push job triggers on `library/**` and `library_test_hooks/**` so cdylib changes can't break the Dart-side integration suite without CI noticing. Verified locally: 232 Rust unit tests + 42 Dart tests (41 existing + 1 integration) green; clippy/fmt/cspell clean; production cdylib does not contain `shorebird_test_reset` (`nm` confirms feature isolation). Stages 2 (FakePatchServer + golden path) and 3 (adversarial scenarios) land separately. * fix(integration test): early-return on skip and bump per-test timeout Two issues caught by Shorebird CI on PR #353: 1. `markTestSkipped` does not abort test execution — it only flags the test as skipped on its way out. The body kept running and crashed on the `late testHooks` field when the cdylib build had failed in setUpAll. Move the skip check into the test body itself with an early return; drop the (no-op) skip in `setUp`. 2. Default per-test timeout (30s) also covers `setUpAll`. A cold `cargo build -p library_test_hooks` compiles `updater` and ~100 transitive deps, which can run minutes on CI. Bump to 10 minutes via `@Timeout` on the library. Verified locally: passes when cargo is on PATH (1 passed), reports a clean skip and exit 0 when cargo is removed from PATH (1 skipped). * refactor(integration test): drop ! by promoting testHooks to late final `testHooks` was nullable so accessing it after the skipReason check required `!`, and `markTestSkipped(skipReason!)` had the same smell. Make `testHooks` `late final` (non-nullable, throws if read before setUpAll assigns) and pull `skipReason` into a non-null local before use. Same control flow, no bang operators.
86 lines
3.4 KiB
Dart
86 lines
3.4 KiB
Dart
// All updater integration tests live in this single file. This is
|
|
// deliberate: `package:test` parallelizes tests across files via
|
|
// isolates, but `dlopen` loads the test_hooks cdylib exactly once per
|
|
// process and the updater's `OnceCell<UpdateConfig>` is shared across
|
|
// every isolate. Splitting these tests across multiple files would
|
|
// require either a `dart_test.yaml` concurrency override scoped to
|
|
// `test/integration/`, or a subprocess-per-test runner.
|
|
//
|
|
// Same file = same isolate = serial = no contention. If the suite
|
|
// outgrows one file, see `docs/integration_tests.md` for the
|
|
// concurrency-override path.
|
|
//
|
|
// The unit tests under `test/src/` are unaffected: they only use
|
|
// `_MockUpdaterBindings` and never load the real cdylib.
|
|
|
|
// `setUpAll` shells out to `cargo build -p library_test_hooks`. On a
|
|
// cold checkout that compiles `updater` and its dependencies and can
|
|
// take a couple of minutes — well past `package:test`'s default 30s
|
|
// per-test timeout, which also covers `setUpAll`.
|
|
@Timeout(Duration(minutes: 10))
|
|
library;
|
|
|
|
import 'dart:ffi';
|
|
|
|
import 'package:shorebird_code_push/src/generated/updater_bindings.g.dart';
|
|
import 'package:shorebird_code_push/src/updater.dart';
|
|
import 'package:test/test.dart';
|
|
|
|
import 'generated/test_hooks_bindings.g.dart';
|
|
import 'helpers/build.dart';
|
|
|
|
void main() {
|
|
// Set in setUpAll exactly when the cdylib build/load failed. The
|
|
// pair (`skipReason`, `testHooks`) is contractually mutually
|
|
// exclusive: a null `skipReason` means `testHooks` is initialized,
|
|
// and tests early-return on a non-null `skipReason` before touching
|
|
// `testHooks`.
|
|
String? skipReason;
|
|
late final TestHooksBindings testHooks;
|
|
|
|
setUpAll(() async {
|
|
try {
|
|
final path = await buildTestHooksCdylib();
|
|
final lib = DynamicLibrary.open(path);
|
|
// `Updater.bindings` is `@visibleForTesting` and the package's own
|
|
// unit tests reassign it. We do the same here, pointing the
|
|
// production code path at our test_hooks cdylib (which re-exports
|
|
// the production C API alongside the `shorebird_test_*` hooks).
|
|
Updater.bindings = UpdaterBindings(lib);
|
|
testHooks = TestHooksBindings(lib);
|
|
} on Object catch (e, st) {
|
|
skipReason = 'Could not build/load library_test_hooks cdylib.\n$e\n$st';
|
|
}
|
|
});
|
|
|
|
group('library_test_hooks', () {
|
|
test('exposes both production and test-hook symbols', () {
|
|
// `markTestSkipped` only flags the test as skipped — it does not
|
|
// abort execution. Early-return after marking, otherwise the body
|
|
// below would touch the uninitialized `testHooks` when the
|
|
// setUpAll build couldn't run (e.g., no Rust toolchain on the
|
|
// host).
|
|
final reason = skipReason;
|
|
if (reason != null) {
|
|
markTestSkipped(reason);
|
|
return;
|
|
}
|
|
|
|
// Test-only symbol layered on top of the production C API.
|
|
// Calling on a process with no prior init clears already-empty
|
|
// globals — should be a no-op, not a crash.
|
|
expect(testHooks.shorebird_test_reset, returnsNormally);
|
|
|
|
// Production symbols flow through the same library. With no
|
|
// init, `shorebird_current_boot_patch_number` returns the
|
|
// `log_on_error` default (0).
|
|
const updater = Updater();
|
|
expect(updater.currentPatchNumber(), 0);
|
|
expect(updater.nextPatchNumber(), 0);
|
|
|
|
// Reset is still callable after exercising production symbols.
|
|
expect(testHooks.shorebird_test_reset, returnsNormally);
|
|
});
|
|
});
|
|
}
|