shorebird-updater

flutter/shorebird-updater

Fork 0

Commit Graph

Author	SHA1	Message	Date
Eric Seidel	96fd32796e	feat: stage 2 integration tests — fake patch server + golden path + rollback regression (#355 ) Lands the fake HTTP server, the patch fixture pipeline, and the first two scenarios that exercise the real FFI path end-to-end. Plus a test that would have caught the patch-to-release rollback bug (shorebirdtech/shorebird#3728). Architecture follows the principles surfaced in review: - Dart tests call the public `ShorebirdUpdater` API only — never the raw `Updater` FFI wrapper, never the engine API. - Engine API stays inside the `library_test_hooks` Rust crate. `shorebird_test_init` constructs `AppParameters` + stub `FileCallbacks` internally so Dart never sees those types. `shorebird_test_simulate_successful_launch` wraps the start/success protocol so the Dart layer never knows there's a protocol — it just knows "the engine reported a successful boot." - ffigen scans only the test_hooks header. The engine header (updater_engine.h) does not appear in the Dart bindings. - A `TestEngine` Dart helper concentrates engine-side simulation in one place; the test bodies stay focused on `ShorebirdUpdater`. Implementation choices worth flagging: - FFI calls run via `Isolate.run`. Synchronous run blocks the main isolate, deadlocking against the in-isolate shelf server. The test's IsolateRun callback re-opens the cdylib (cheap: dlopen is ref-counted) and resets `Updater.bindings` in the sub-isolate because Dart isolates do not share static fields. - `libapp_path` must be a real file on the desktop integration build: the non-Android non-iOS non-test `patch_base` reads it directly from disk. Tests that install a patch write the fixture's `base` bytes to `libapp.so` before init. - Fake server kept minimal: shelf, no Range support, no auth, no concurrency knobs. Stage 3+ scenarios (download cutoff, hash mismatch loop, etc.) extend it as needed. Three scenarios cover three reasons we wanted this suite: 1. `checkForUpdate returns upToDate when server has no patch` — baseline: confirms the harness boots cleanly and returns the expected enum. 2. `install a patch and boot from it` — golden path: check → update → simulateSuccessfulLaunch, then assert `readCurrentPatch` / `readNextPatch` / `checkForUpdate` transitions match the public API contract. 3. `checkForUpdate returns restartRequired after patch-to-release rollback` — regression for shorebirdtech/shorebird#3728. Pre-fix this returned `upToDate` and left no signal to prompt a restart. Verified locally: 232 Rust unit tests + 44 Dart tests (41 existing unit + 3 new integration) green; clippy/fmt/cspell clean.	2026-05-06 08:11:47 -07:00
Eric Seidel	8072ed9ad7	feat: stage 1 integration test harness (#353 ) * docs: integration tests design proposal Adds docs/integration_tests.md proposing a desktop-only integration suite that drives the Dart `ShorebirdUpdater` API against the real Rust core via FFI, with a fake HTTP server and per-test tempdir state. Describes a new `library_test_hooks` cdylib that reaches into `updater` via a `test-hooks` Cargo feature, so production C API stays clean. Status: design exploration, not a committed plan. * feat: stage 1 integration test harness for shorebird_code_push Lands the test_hooks crate, the Dart-side loader, and one trivial end-to-end test, per docs/integration_tests.md stage 1. Pieces: - New `library_test_hooks` workspace crate (cdylib). Depends on `updater` with a new `test-hooks` Cargo feature that widens the visibility of internal items (currently `testing_reset_config`) for sibling-crate access only — production builds do not enable it. The crate also re-exports `updater::c_api::dart::` and `updater::c_api::engine::`, which keeps the rlib's `#[no_mangle]` symbols out of DCE so the resulting cdylib carries both the production C API and the new `shorebird_test_` hooks. - Workspace `default-members` excludes `library_test_hooks` so plain `cargo build` / `cargo test` invocations don't unify the `test-hooks` feature into production builds. - `shorebird_code_push/test/integration/all_test.dart` (single file, with a header comment explaining why) loads the cdylib via `DynamicLibrary.open`, reassigns the existing `@visibleForTesting` `Updater.bindings` setter, and exercises both surfaces. The build helper shells out to `cargo build -p library_test_hooks`; if it fails, `markTestSkipped` keeps the suite green on machines without a working Rust toolchain. - ffigen config (`ffigen_test_hooks.yaml`) generates the test-only Dart bindings under `test/integration/generated/`, not `lib/`. - CI: `library_test_hooks` is added to the rust_crate matrix, and the shorebird_code_push job triggers on `library/` and `library_test_hooks/` so cdylib changes can't break the Dart-side integration suite without CI noticing. Verified locally: 232 Rust unit tests + 42 Dart tests (41 existing + 1 integration) green; clippy/fmt/cspell clean; production cdylib does not contain `shorebird_test_reset` (`nm` confirms feature isolation). Stages 2 (FakePatchServer + golden path) and 3 (adversarial scenarios) land separately. fix(integration test): early-return on skip and bump per-test timeout Two issues caught by Shorebird CI on PR #353: 1. `markTestSkipped` does not abort test execution — it only flags the test as skipped on its way out. The body kept running and crashed on the `late testHooks` field when the cdylib build had failed in setUpAll. Move the skip check into the test body itself with an early return; drop the (no-op) skip in `setUp`. 2. Default per-test timeout (30s) also covers `setUpAll`. A cold `cargo build -p library_test_hooks` compiles `updater` and ~100 transitive deps, which can run minutes on CI. Bump to 10 minutes via `@Timeout` on the library. Verified locally: passes when cargo is on PATH (1 passed), reports a clean skip and exit 0 when cargo is removed from PATH (1 skipped). * refactor(integration test): drop ! by promoting testHooks to late final `testHooks` was nullable so accessing it after the skipReason check required `!`, and `markTestSkipped(skipReason!)` had the same smell. Make `testHooks` `late final` (non-nullable, throws if read before setUpAll assigns) and pull `skipReason` into a non-null local before use. Same control flow, no bang operators.	2026-05-05 16:32:57 -07:00

Author

SHA1

Message

Date

Eric Seidel

96fd32796e

feat: stage 2 integration tests — fake patch server + golden path + rollback regression (#355 )

Lands the fake HTTP server, the patch fixture pipeline, and the first
two scenarios that exercise the real FFI path end-to-end. Plus a test
that would have caught the patch-to-release rollback bug
(shorebirdtech/shorebird#3728).

Architecture follows the principles surfaced in review:

- Dart tests call the public `ShorebirdUpdater` API only — never the
  raw `Updater` FFI wrapper, never the engine API.
- Engine API stays inside the `library_test_hooks` Rust crate.
  `shorebird_test_init` constructs `AppParameters` + stub
  `FileCallbacks` internally so Dart never sees those types.
  `shorebird_test_simulate_successful_launch` wraps the
  start/success protocol so the Dart layer never knows there's a
  protocol — it just knows "the engine reported a successful boot."
- ffigen scans only the test_hooks header. The engine header
  (updater_engine.h) does not appear in the Dart bindings.
- A `TestEngine` Dart helper concentrates engine-side simulation in
  one place; the test bodies stay focused on `ShorebirdUpdater`.

Implementation choices worth flagging:

- FFI calls run via `Isolate.run`. Synchronous run blocks the main
  isolate, deadlocking against the in-isolate shelf server. The
  test's IsolateRun callback re-opens the cdylib (cheap: dlopen is
  ref-counted) and resets `Updater.bindings` in the sub-isolate
  because Dart isolates do not share static fields.
- `libapp_path` must be a real file on the desktop integration build:
  the non-Android non-iOS non-test `patch_base` reads it directly
  from disk. Tests that install a patch write the fixture's `base`
  bytes to `libapp.so` before init.
- Fake server kept minimal: shelf, no Range support, no auth, no
  concurrency knobs. Stage 3+ scenarios (download cutoff, hash
  mismatch loop, etc.) extend it as needed.

Three scenarios cover three reasons we wanted this suite:

1. `checkForUpdate returns upToDate when server has no patch` —
   baseline: confirms the harness boots cleanly and returns the
   expected enum.
2. `install a patch and boot from it` — golden path: check →
   update → simulateSuccessfulLaunch, then assert
   `readCurrentPatch` / `readNextPatch` / `checkForUpdate`
   transitions match the public API contract.
3. `checkForUpdate returns restartRequired after patch-to-release
   rollback` — regression for shorebirdtech/shorebird#3728. Pre-fix
   this returned `upToDate` and left no signal to prompt a restart.

Verified locally: 232 Rust unit tests + 44 Dart tests (41 existing
unit + 3 new integration) green; clippy/fmt/cspell clean.

2026-05-06 08:11:47 -07:00

Eric Seidel

8072ed9ad7

feat: stage 1 integration test harness (#353 )

* docs: integration tests design proposal

Adds docs/integration_tests.md proposing a desktop-only integration suite
that drives the Dart `ShorebirdUpdater` API against the real Rust core via
FFI, with a fake HTTP server and per-test tempdir state. Describes a new
`library_test_hooks` cdylib that reaches into `updater` via a `test-hooks`
Cargo feature, so production C API stays clean. Status: design exploration,
not a committed plan.

* feat: stage 1 integration test harness for shorebird_code_push

Lands the test_hooks crate, the Dart-side loader, and one trivial
end-to-end test, per docs/integration_tests.md stage 1.

Pieces:

- New `library_test_hooks` workspace crate (cdylib). Depends on `updater`
  with a new `test-hooks` Cargo feature that widens the visibility of
  internal items (currently `testing_reset_config`) for sibling-crate
  access only — production builds do not enable it. The crate also
  re-exports `updater::c_api::dart::*` and `updater::c_api::engine::*`,
  which keeps the rlib's `#[no_mangle]` symbols out of DCE so the
  resulting cdylib carries both the production C API and the new
  `shorebird_test_*` hooks.
- Workspace `default-members` excludes `library_test_hooks` so plain
  `cargo build` / `cargo test` invocations don't unify the `test-hooks`
  feature into production builds.
- `shorebird_code_push/test/integration/all_test.dart` (single file,
  with a header comment explaining why) loads the cdylib via
  `DynamicLibrary.open`, reassigns the existing `@visibleForTesting`
  `Updater.bindings` setter, and exercises both surfaces. The build
  helper shells out to `cargo build -p library_test_hooks`; if it
  fails, `markTestSkipped` keeps the suite green on machines without
  a working Rust toolchain.
- ffigen config (`ffigen_test_hooks.yaml`) generates the test-only
  Dart bindings under `test/integration/generated/`, not `lib/`.
- CI: `library_test_hooks` is added to the rust_crate matrix, and the
  shorebird_code_push job triggers on `library/**` and
  `library_test_hooks/**` so cdylib changes can't break the Dart-side
  integration suite without CI noticing.

Verified locally: 232 Rust unit tests + 42 Dart tests (41 existing + 1
integration) green; clippy/fmt/cspell clean; production cdylib does
not contain `shorebird_test_reset` (`nm` confirms feature isolation).

Stages 2 (FakePatchServer + golden path) and 3 (adversarial scenarios)
land separately.

* fix(integration test): early-return on skip and bump per-test timeout

Two issues caught by Shorebird CI on PR #353:

1. `markTestSkipped` does not abort test execution — it only flags the
   test as skipped on its way out. The body kept running and crashed on
   the `late testHooks` field when the cdylib build had failed in
   setUpAll. Move the skip check into the test body itself with an
   early return; drop the (no-op) skip in `setUp`.

2. Default per-test timeout (30s) also covers `setUpAll`. A cold
   `cargo build -p library_test_hooks` compiles `updater` and ~100
   transitive deps, which can run minutes on CI. Bump to 10 minutes
   via `@Timeout` on the library.

Verified locally: passes when cargo is on PATH (1 passed), reports a
clean skip and exit 0 when cargo is removed from PATH (1 skipped).

* refactor(integration test): drop ! by promoting testHooks to late final

`testHooks` was nullable so accessing it after the skipReason check
required `!`, and `markTestSkipped(skipReason!)` had the same smell.
Make `testHooks` `late final` (non-nullable, throws if read before
setUpAll assigns) and pull `skipReason` into a non-null local before
use. Same control flow, no bang operators.

2026-05-05 16:32:57 -07:00

2 Commits