feat(harness): agent-drivable dev harness and CI that gates
A coding agent could develop this repo's Go packages and could not develop the application: every path to running YellowJacket ended in a blocking GTK window, so 265 bound methods, 46 events, 33 component directories and 13 stores had exactly one form of verification available — `tsc --noEmit`. The unlock is that `wails dev`'s dev server on :34115 serves the real frontend with the real generated bindings against the same Go backend a desktop window attaches to, so a plain Chromium under Xvfb gets a fully functional app. Four test tiers now exist, cheapest first: - `make ui-test` — 313 Vitest tests in a real browser in ~2 s, no app, no backend, no display. Works because `frontend/wailsjs/` is a pure passthrough to `window.go`/`window.runtime`, so faking just those two globals runs the real bindings and the real store code. - `make test` — services in-process, asserting on the payload the frontend would receive, via a new `events.Emit` wrapper. - `make dev-headless` + `playwright-cli` — the real app, driven interactively, with an event bridge on `window.__yjEvents` and a dev-only control surface at `/__test/`. - `make e2e` — 19 of those flows frozen as Playwright specs. `events.Emit(ctx, …)` replaces all 35 direct `runtime.EventsEmit` call sites: wails' `getEvents` `log.Fatalf`s on any context without its runtime, so those paths could not run under test and a background worker could take the app down. Four packages had each hand-rolled the same guard; nine more guarded on `ctx != nil`, which does not help. `TestNoDirectRuntimeEmits` fails the build on a new one. Fixtures are generated, not committed (`make testdata`), and seeds are built by *running the app* — never by hand-writing config and DB rows, which would be a second description of a valid YJ_HOME. `.gitea/workflows/ci.yml` is the first workflow here that tests anything; the other three only package, so `gitea_ci` reported only packaging jobs and misled anyone asking whether a push was healthy. Both jobs were prototyped to green in a bare ubuntu:24.04 container before the YAML was written, which immediately caught `make lint` linting three configurations that nothing builds: all three passes omitted `webkit2_41`, so wails resolved webkit2gtk-4.0 — which Arch still ships and Ubuntu 24.04 dropped. Operational instructions live in `.pi/skills/yellowjacket-dev/`, measured discoveries in `.planning/NOTES.md`, and architecture in `CLAUDE.md` — split by tense, not by topic, because a topical split gives every new fact two plausible homes. `make skill-check` fails a commit if the skill cites a make target that does not exist.
This commit is contained in:
@@ -22,11 +22,23 @@ Numbering is sequential and stable across status moves (a plan keeps its `NNN-`
|
||||
```bash
|
||||
make dev # Hot-reload development (installs deps, generates code, cleans frontend)
|
||||
make dev-debug # Same as dev but with YJ_LOG_LEVEL=debug
|
||||
make dev-headless # Start headless in the background and return (SEED=<name> to seed)
|
||||
make dev-stop # Stop it (SIGTERM, so shutdown hooks run)
|
||||
make dev-logs # Tail .dev/app.log
|
||||
make testdata # Generate the deterministic fixture music library
|
||||
make sandbox-seed NAME=<n> # Build a seeded YJ_HOME by *running* the app
|
||||
make build-dev # Debug build with symbols
|
||||
make build-prod # Production build (stripped, UPX-compressed)
|
||||
make generate # Run code generators (sqlc + templ via go generate)
|
||||
make lint # golangci-lint v2 (strict), both build configurations
|
||||
make test # All tests with race detector, both build configurations
|
||||
make e2e # Playwright smoke suite against a running dev-headless app
|
||||
make e2e-setup # Install the e2e runner + its browser (once)
|
||||
make ui-test # Vitest component/store suite in a real browser (no app)
|
||||
make ui-visual # Same, including toMatchScreenshot comparisons
|
||||
make ui-setup # Install the Vitest provider's own Chromium (once)
|
||||
make bindings-check # Fail if frontend/wailsjs is stale vs the Go bindings
|
||||
make skill-check # Fail if .pi/ documents a make target that doesn't exist
|
||||
make lint # golangci-lint v2 (strict), all three build configurations
|
||||
make test # All tests with race detector, all three build configurations
|
||||
make vulncheck # govulncheck for CVEs
|
||||
make setup # Install go tools, frontend deps, git hooks (lefthook)
|
||||
```
|
||||
@@ -49,8 +61,75 @@ needs it spelled out:
|
||||
go test -tags "webkit2_41 indexbuild" ./backend/explore/... ./cmd/...
|
||||
```
|
||||
|
||||
`backend/testctl` is behind a third tag and needs its own pass too
|
||||
(`make test` runs all three):
|
||||
|
||||
```bash
|
||||
go test -tags "webkit2_41 dev" ./backend/testctl/...
|
||||
```
|
||||
|
||||
Audio playback integration tests require `YELLOWJACKET_INTEGRATION=1`.
|
||||
|
||||
### Fixtures and the headless harness
|
||||
|
||||
`test_data/music_library_test/` is **generated, not committed**: run
|
||||
`make testdata` (~1 s) before anything that needs audio. Tests reach it
|
||||
through `internal/testfixtures`, selecting files by *case*
|
||||
(`CaseCoverDedup`, `CaseUnicode`, `CaseDuplicates`, …) rather than by
|
||||
path, and skip themselves when it has not been generated.
|
||||
|
||||
The app itself can be run without a blocking window — `make
|
||||
dev-headless` — and driven with `playwright-cli` against the dev server
|
||||
on `:34115`, which is the real app with real bindings on `window.go`,
|
||||
bridged to the same Go backend a desktop window would use.
|
||||
|
||||
**The operational half of all this lives in the
|
||||
`yellowjacket-dev` skill** (`.pi/skills/yellowjacket-dev/`): which tier
|
||||
to reach for, the exact command sequences, seed lifecycle, and the
|
||||
failure modes worth knowing before you meet them. It is deliberately
|
||||
not repeated here — this section describes what exists, the skill says
|
||||
what to run.
|
||||
|
||||
Two things ride on top of the headless launch, both from plan 005
|
||||
phase 3:
|
||||
|
||||
- **The event bridge.** `.playwright/cli.config.json` loads
|
||||
`.playwright/init-events.js` as an `initScript`, which records every
|
||||
backend event on `window.__yjEvents`. Half this app is push-driven,
|
||||
so assertions **await an event, not a timeout**:
|
||||
`await window.__yjEvents.wait('LibraryScanComplete', {timeoutMs: 60000})`.
|
||||
It also provides `ready()` and `call('queue.Queue.GetState', [])`,
|
||||
which times out instead of hanging.
|
||||
- **The dev-only control surface**, `backend/testctl`, mounted at
|
||||
`/__test/` on the same port: `health`, `db/snapshot`, `db/restore`,
|
||||
`emit` (force any backend event, which renders push-driven views
|
||||
without staging the work that would produce them) and `sql`. It is
|
||||
compiled out of non-dev builds and additionally requires
|
||||
`YJ_TESTCTL=1`, which `dev-headless.sh` sets and `make dev` does not.
|
||||
|
||||
Frozen regression specs live in `e2e/` (its own npm package, so the
|
||||
Vitest browser mode does not share a package with the Playwright
|
||||
runner): `make e2e` against an already-running app.
|
||||
|
||||
**The cheapest tier needs none of that.** `make ui-test` runs 313
|
||||
Vitest tests in a real Chromium in ~2 s with no Wails, no backend, no
|
||||
seeded library and no virtual display, because `frontend/wailsjs/` is a
|
||||
pure passthrough to `window.go` / `window.runtime` and
|
||||
`frontend/test/support/wails-fake.ts` replaces just those two globals —
|
||||
so the tests exercise the real generated bindings and the real store
|
||||
code.
|
||||
|
||||
**`frontend/wailsjs/` is generated by `wails`, not `go generate`**, so
|
||||
the pre-commit codegen check does not cover it. `make bindings-check`
|
||||
(~1.5 s, also a pre-commit hook) regenerates it and fails on a dirty
|
||||
tree; `make bindings` regenerates it for real.
|
||||
|
||||
**Seeds are produced by running the app**, never by hand-writing a
|
||||
`config.toml` and DB rows — the same discipline `sql/schemas/` gets,
|
||||
for the same reason.
|
||||
|
||||
See `.planning/plans/active/005-agent-development-harness.md`.
|
||||
|
||||
## Architecture
|
||||
|
||||
**Wails app lifecycle** (`main.go` → `backend/app.go`): `YellowJacketApp` is the root struct bound to Wails. Its methods are callable from the frontend. Lifecycle hooks: `OnStartup` (init audio), `OnDomReady` (start library scan), `OnBeforeClose` (save window state), `OnShutdown` (persist player/queue state).
|
||||
@@ -140,6 +219,19 @@ work happens **once, centrally**, and users download the result:
|
||||
|
||||
**Event-driven communication**: Backend emits events via Wails runtime; frontend stores subscribe to them. Event names are constants in `backend/events/`.
|
||||
|
||||
Emit through **`events.Emit(ctx, name, data...)`**, never
|
||||
`runtime.EventsEmit` — wails `log.Fatalf`s (unrecoverably) on any
|
||||
context that does not carry its runtime, which includes every
|
||||
`context.Background()`, so a direct call cannot run under test and can
|
||||
kill the app from a background worker. `TestNoDirectRuntimeEmits` fails
|
||||
the build on a direct call anywhere outside `backend/events`.
|
||||
|
||||
That wrapper is what makes services testable in-process: install a
|
||||
recorder with `events.WithSink(ctx, rec)` and assert on the payload the
|
||||
frontend would receive (`backend/queue/emit_test.go` is the model).
|
||||
`events.Deliver` is the same call returning an error instead of
|
||||
dropping, and has one legitimate caller — `/__test/emit`.
|
||||
|
||||
## Code Generation
|
||||
|
||||
Two generators run via `go generate ./...` (or `make generate`):
|
||||
@@ -162,3 +254,36 @@ Tests use `database.NewTestDB(t)` for in-memory SQLite, built by the same
|
||||
## Git Workflow
|
||||
|
||||
Feature branches and PRs are the norm, but direct pushes to `main` are allowed. Pre-commit runs vet, lint, codegen check, and frontend typecheck in parallel. Pre-push runs the full test suite.
|
||||
|
||||
## CI
|
||||
|
||||
Four workflows in `.gitea/workflows/`. Three of them package and
|
||||
publish (`arch-package`, `homebrew-formula`, `index-artifact`); only
|
||||
`ci.yml` gates, and it is the one to look at when deciding whether a
|
||||
push was healthy.
|
||||
|
||||
Two jobs, both in an `ubuntu:24.04` container:
|
||||
|
||||
- **`check`** — no display: `make lint` and `make test` (three build
|
||||
configurations each), `tsc --noEmit`, `make ui-test`,
|
||||
`make bindings-check`, `make skill-check`.
|
||||
- **`e2e`** — under Xvfb and a private D-Bus: fixtures, a seed built by
|
||||
running the app, `make dev-headless`, then the Playwright suite
|
||||
against **both** Chromium and WebKit. Playwright's Linux WebKit links
|
||||
Ubuntu 24.04 libraries that Arch does not provide, so CI is the only
|
||||
place it can run, and it is the closest available approximation of
|
||||
the WebKit2GTK renderer that ships.
|
||||
|
||||
Two things the container needs that a developer machine does not. It
|
||||
has no PulseAudio socket, so `/etc/asound.conf` makes ALSA's `null`
|
||||
plugin the default device — that plugin advances its pointer on a
|
||||
timer, so playback is consumed at real-time rate and the elapsed clock
|
||||
moves, which `e2e/specs/playback.spec.ts` asserts. And
|
||||
`YJ_CORE_INDEX_URL` points at a dead address so no run fetches the real
|
||||
explore artifact, matching what `scripts/seed-sandbox.sh` already does.
|
||||
|
||||
**`make lint`'s tag sets must stay identical to `make test`'s.**
|
||||
Without `webkit2_41` wails resolves `webkit2gtk-4.0`, which Arch still
|
||||
ships and Ubuntu 24.04 does not — so a mismatch lints a configuration
|
||||
that only builds on one developer's distro, and says nothing about what
|
||||
ships.
|
||||
|
||||
Reference in New Issue
Block a user