Files
shorebird-updater/shorebird_code_push/test/integration/all_test.dart
T
Eric Seidel 8072ed9ad7 feat: stage 1 integration test harness (#353)
* docs: integration tests design proposal

Adds docs/integration_tests.md proposing a desktop-only integration suite
that drives the Dart `ShorebirdUpdater` API against the real Rust core via
FFI, with a fake HTTP server and per-test tempdir state. Describes a new
`library_test_hooks` cdylib that reaches into `updater` via a `test-hooks`
Cargo feature, so production C API stays clean. Status: design exploration,
not a committed plan.

* feat: stage 1 integration test harness for shorebird_code_push

Lands the test_hooks crate, the Dart-side loader, and one trivial
end-to-end test, per docs/integration_tests.md stage 1.

Pieces:

- New `library_test_hooks` workspace crate (cdylib). Depends on `updater`
  with a new `test-hooks` Cargo feature that widens the visibility of
  internal items (currently `testing_reset_config`) for sibling-crate
  access only — production builds do not enable it. The crate also
  re-exports `updater::c_api::dart::*` and `updater::c_api::engine::*`,
  which keeps the rlib's `#[no_mangle]` symbols out of DCE so the
  resulting cdylib carries both the production C API and the new
  `shorebird_test_*` hooks.
- Workspace `default-members` excludes `library_test_hooks` so plain
  `cargo build` / `cargo test` invocations don't unify the `test-hooks`
  feature into production builds.
- `shorebird_code_push/test/integration/all_test.dart` (single file,
  with a header comment explaining why) loads the cdylib via
  `DynamicLibrary.open`, reassigns the existing `@visibleForTesting`
  `Updater.bindings` setter, and exercises both surfaces. The build
  helper shells out to `cargo build -p library_test_hooks`; if it
  fails, `markTestSkipped` keeps the suite green on machines without
  a working Rust toolchain.
- ffigen config (`ffigen_test_hooks.yaml`) generates the test-only
  Dart bindings under `test/integration/generated/`, not `lib/`.
- CI: `library_test_hooks` is added to the rust_crate matrix, and the
  shorebird_code_push job triggers on `library/**` and
  `library_test_hooks/**` so cdylib changes can't break the Dart-side
  integration suite without CI noticing.

Verified locally: 232 Rust unit tests + 42 Dart tests (41 existing + 1
integration) green; clippy/fmt/cspell clean; production cdylib does
not contain `shorebird_test_reset` (`nm` confirms feature isolation).

Stages 2 (FakePatchServer + golden path) and 3 (adversarial scenarios)
land separately.

* fix(integration test): early-return on skip and bump per-test timeout

Two issues caught by Shorebird CI on PR #353:

1. `markTestSkipped` does not abort test execution — it only flags the
   test as skipped on its way out. The body kept running and crashed on
   the `late testHooks` field when the cdylib build had failed in
   setUpAll. Move the skip check into the test body itself with an
   early return; drop the (no-op) skip in `setUp`.

2. Default per-test timeout (30s) also covers `setUpAll`. A cold
   `cargo build -p library_test_hooks` compiles `updater` and ~100
   transitive deps, which can run minutes on CI. Bump to 10 minutes
   via `@Timeout` on the library.

Verified locally: passes when cargo is on PATH (1 passed), reports a
clean skip and exit 0 when cargo is removed from PATH (1 skipped).

* refactor(integration test): drop ! by promoting testHooks to late final

`testHooks` was nullable so accessing it after the skipReason check
required `!`, and `markTestSkipped(skipReason!)` had the same smell.
Make `testHooks` `late final` (non-nullable, throws if read before
setUpAll assigns) and pull `skipReason` into a non-null local before
use. Same control flow, no bang operators.
2026-05-05 16:32:57 -07:00

86 lines
3.4 KiB
Dart

// All updater integration tests live in this single file. This is
// deliberate: `package:test` parallelizes tests across files via
// isolates, but `dlopen` loads the test_hooks cdylib exactly once per
// process and the updater's `OnceCell<UpdateConfig>` is shared across
// every isolate. Splitting these tests across multiple files would
// require either a `dart_test.yaml` concurrency override scoped to
// `test/integration/`, or a subprocess-per-test runner.
//
// Same file = same isolate = serial = no contention. If the suite
// outgrows one file, see `docs/integration_tests.md` for the
// concurrency-override path.
//
// The unit tests under `test/src/` are unaffected: they only use
// `_MockUpdaterBindings` and never load the real cdylib.
// `setUpAll` shells out to `cargo build -p library_test_hooks`. On a
// cold checkout that compiles `updater` and its dependencies and can
// take a couple of minutes — well past `package:test`'s default 30s
// per-test timeout, which also covers `setUpAll`.
@Timeout(Duration(minutes: 10))
library;
import 'dart:ffi';
import 'package:shorebird_code_push/src/generated/updater_bindings.g.dart';
import 'package:shorebird_code_push/src/updater.dart';
import 'package:test/test.dart';
import 'generated/test_hooks_bindings.g.dart';
import 'helpers/build.dart';
void main() {
// Set in setUpAll exactly when the cdylib build/load failed. The
// pair (`skipReason`, `testHooks`) is contractually mutually
// exclusive: a null `skipReason` means `testHooks` is initialized,
// and tests early-return on a non-null `skipReason` before touching
// `testHooks`.
String? skipReason;
late final TestHooksBindings testHooks;
setUpAll(() async {
try {
final path = await buildTestHooksCdylib();
final lib = DynamicLibrary.open(path);
// `Updater.bindings` is `@visibleForTesting` and the package's own
// unit tests reassign it. We do the same here, pointing the
// production code path at our test_hooks cdylib (which re-exports
// the production C API alongside the `shorebird_test_*` hooks).
Updater.bindings = UpdaterBindings(lib);
testHooks = TestHooksBindings(lib);
} on Object catch (e, st) {
skipReason = 'Could not build/load library_test_hooks cdylib.\n$e\n$st';
}
});
group('library_test_hooks', () {
test('exposes both production and test-hook symbols', () {
// `markTestSkipped` only flags the test as skipped — it does not
// abort execution. Early-return after marking, otherwise the body
// below would touch the uninitialized `testHooks` when the
// setUpAll build couldn't run (e.g., no Rust toolchain on the
// host).
final reason = skipReason;
if (reason != null) {
markTestSkipped(reason);
return;
}
// Test-only symbol layered on top of the production C API.
// Calling on a process with no prior init clears already-empty
// globals — should be a no-op, not a crash.
expect(testHooks.shorebird_test_reset, returnsNormally);
// Production symbols flow through the same library. With no
// init, `shorebird_current_boot_patch_number` returns the
// `log_on_error` default (0).
const updater = Updater();
expect(updater.currentPatchNumber(), 0);
expect(updater.nextPatchNumber(), 0);
// Reset is still callable after exercising production symbols.
expect(testHooks.shorebird_test_reset, returnsNormally);
});
});
}