feat(harness): agent-drivable dev harness and CI that gates
A coding agent could develop this repo's Go packages and could not develop the application: every path to running YellowJacket ended in a blocking GTK window, so 265 bound methods, 46 events, 33 component directories and 13 stores had exactly one form of verification available — `tsc --noEmit`. The unlock is that `wails dev`'s dev server on :34115 serves the real frontend with the real generated bindings against the same Go backend a desktop window attaches to, so a plain Chromium under Xvfb gets a fully functional app. Four test tiers now exist, cheapest first: - `make ui-test` — 313 Vitest tests in a real browser in ~2 s, no app, no backend, no display. Works because `frontend/wailsjs/` is a pure passthrough to `window.go`/`window.runtime`, so faking just those two globals runs the real bindings and the real store code. - `make test` — services in-process, asserting on the payload the frontend would receive, via a new `events.Emit` wrapper. - `make dev-headless` + `playwright-cli` — the real app, driven interactively, with an event bridge on `window.__yjEvents` and a dev-only control surface at `/__test/`. - `make e2e` — 19 of those flows frozen as Playwright specs. `events.Emit(ctx, …)` replaces all 35 direct `runtime.EventsEmit` call sites: wails' `getEvents` `log.Fatalf`s on any context without its runtime, so those paths could not run under test and a background worker could take the app down. Four packages had each hand-rolled the same guard; nine more guarded on `ctx != nil`, which does not help. `TestNoDirectRuntimeEmits` fails the build on a new one. Fixtures are generated, not committed (`make testdata`), and seeds are built by *running the app* — never by hand-writing config and DB rows, which would be a second description of a valid YJ_HOME. `.gitea/workflows/ci.yml` is the first workflow here that tests anything; the other three only package, so `gitea_ci` reported only packaging jobs and misled anyone asking whether a push was healthy. Both jobs were prototyped to green in a bare ubuntu:24.04 container before the YAML was written, which immediately caught `make lint` linting three configurations that nothing builds: all three passes omitted `webkit2_41`, so wails resolved webkit2gtk-4.0 — which Arch still ships and Ubuntu 24.04 dropped. Operational instructions live in `.pi/skills/yellowjacket-dev/`, measured discoveries in `.planning/NOTES.md`, and architecture in `CLAUDE.md` — split by tense, not by topic, because a topical split gives every new fact two plausible homes. `make skill-check` fails a commit if the skill cites a make target that does not exist.
This commit is contained in:
@@ -13,6 +13,93 @@ dev: setup generate clean
|
||||
dev-debug: setup generate clean
|
||||
if [ -f .env ]; then set -a; . ./.env; set +a; fi; : "$${YJ_HOME:=$(DEV_YJ_HOME)}"; export YJ_HOME; YJ_LOG_LEVEL=debug go tool wails dev -tags webkit2_41 -loglevel Debug -v 2
|
||||
|
||||
# ── Headless harness (plan 005) ──────────────────────────────────────
|
||||
# The same dev server `make dev` runs, minus the blocking GTK window:
|
||||
# Xvfb gives it the display it insists on, and the script returns once
|
||||
# :34115 answers. This is the only entry point an agent can use, since
|
||||
# every other one blocks the terminal forever.
|
||||
dev-headless: ## Start the app headless in the background (SEED=<name> to seed)
|
||||
@./scripts/dev-headless.sh $(if $(SEED),--seed $(SEED),) $(HEADLESS_ARGS)
|
||||
|
||||
dev-headless-fresh: ## Same, but on an empty YJ_HOME (first-run wizard)
|
||||
@./scripts/dev-headless.sh --fresh $(HEADLESS_ARGS)
|
||||
|
||||
dev-stop: ## Stop the headless app (SIGTERM, so shutdown hooks run)
|
||||
@./scripts/dev-stop.sh
|
||||
|
||||
dev-logs: ## Tail the headless app log
|
||||
@tail -f .dev/app.log
|
||||
|
||||
# Seeds are produced by *running the app* — driving the real AddLibrary
|
||||
# binding and waiting for the real scan — never by hand-writing a
|
||||
# config.toml and DB rows. A hand-built seed is a second description
|
||||
# of a valid YJ_HOME and would drift from the real one.
|
||||
sandbox-seed: testdata ## Build a seeded YJ_HOME snapshot: make sandbox-seed NAME=<n>
|
||||
@./scripts/seed-sandbox.sh $(if $(NAME),--name $(NAME),)
|
||||
|
||||
sandbox-seeds: ## List built seeds
|
||||
@ls -1 .dev/seeds/*.tar 2>/dev/null | sed 's|.*/||; s|\.tar$$||' \
|
||||
|| echo " (none; build one with: make sandbox-seed NAME=default)"
|
||||
|
||||
# The specs drive the app that is *already* running: `make dev-headless`
|
||||
# daemonises, which is the opposite of what Playwright's `webServer`
|
||||
# supervises, and starting one per run would rebuild the frontend every
|
||||
# time. globalSetup fails with the exact commands to run if it is down.
|
||||
e2e: ## Run the Playwright smoke suite against a running dev-headless app
|
||||
@cd e2e && pnpm install --silent && npx playwright test $(E2E_ARGS)
|
||||
|
||||
e2e-setup: ## Install the e2e runner and its browser (once)
|
||||
@cd e2e && pnpm install && npx playwright install chromium
|
||||
|
||||
e2e-report: ## Open the HTML report from the last e2e run
|
||||
@cd e2e && npx playwright show-report
|
||||
|
||||
# The cheapest tier: components and stores in a real browser, with no
|
||||
# Wails, no backend, no seeded library and no virtual display. Lives in
|
||||
# frontend/ rather than e2e/ so the Vitest browser provider and the
|
||||
# Playwright runner cannot fight over versions or globs.
|
||||
ui-test: ## Run the Vitest component and store suite (frontend/)
|
||||
@cd frontend && pnpm install --silent && npx vitest run $(UI_ARGS)
|
||||
|
||||
ui-watch: ## Same suite, in watch mode
|
||||
@cd frontend && npx vitest
|
||||
|
||||
# Visual regression is opt-in: toMatchScreenshot baselines depend on
|
||||
# font hinting and compositing, so they only mean anything on the
|
||||
# machine (or container) that took them.
|
||||
ui-visual: ## Run the suite including screenshot comparisons
|
||||
@cd frontend && YJ_VISUAL=1 npx vitest run $(UI_ARGS)
|
||||
|
||||
ui-visual-update: ## Re-record the screenshot baselines
|
||||
@cd frontend && YJ_VISUAL=1 npx vitest run --update $(UI_ARGS)
|
||||
|
||||
ui-setup: ## Install the Vitest browser provider's own Chromium (once)
|
||||
@cd frontend && pnpm install && npx playwright install chromium
|
||||
|
||||
# frontend/wailsjs is generated by `wails`, NOT by `go generate`, so the
|
||||
# pre-commit codegen check does not cover it: a renamed Go struct field
|
||||
# currently surfaces at runtime, in a window. File modes are ignored
|
||||
# because `wails generate module` rewrites the runtime files as 755.
|
||||
bindings-check: ## Fail if frontend/wailsjs is stale against the Go bindings
|
||||
@./scripts/bindings-check.sh
|
||||
|
||||
# .pi/ documents commands, and a skill that documents a command wrongly
|
||||
# is worse than no skill: an agent runs it confidently. Every command
|
||||
# in there is a make target on purpose, so this is checkable.
|
||||
skill-check: ## Fail if .pi/ documents a make target that does not exist
|
||||
@./scripts/skill-check.sh
|
||||
|
||||
bindings: ## Regenerate frontend/wailsjs from the bound Go structs
|
||||
go tool wails generate module -tags webkit2_41
|
||||
@chmod 644 frontend/wailsjs/runtime/runtime.js \
|
||||
frontend/wailsjs/runtime/runtime.d.ts \
|
||||
frontend/wailsjs/runtime/package.json
|
||||
|
||||
.PHONY: dev-headless dev-headless-fresh dev-stop dev-logs \
|
||||
sandbox-seed sandbox-seeds e2e e2e-setup e2e-report \
|
||||
ui-test ui-watch ui-visual ui-visual-update ui-setup \
|
||||
bindings bindings-check skill-check
|
||||
|
||||
# Base directory for fresh-install sandboxes. Deliberately NOT $TMPDIR:
|
||||
# on most Linux distros /tmp is tmpfs (RAM-backed) and only a few GB, so
|
||||
# the search index dump import — which wants 6GB free before it will even
|
||||
@@ -119,17 +206,45 @@ clean:
|
||||
generate:
|
||||
go generate ./...
|
||||
|
||||
lint:
|
||||
go tool golangci-lint run
|
||||
go tool golangci-lint run --build-tags indexbuild
|
||||
# The fixture library is generated, not committed: deterministic audio
|
||||
# across all four supported formats, tagged by backend/tagwriter so the
|
||||
# fixtures and the reader under test cannot drift. Regenerates only
|
||||
# when the spec's manifest hash has changed, so it is cheap to depend on.
|
||||
testdata: ## Generate the deterministic fixture music library
|
||||
go run ./cmd/gentestdata
|
||||
|
||||
# Two passes: the app build, then the `indexbuild` build that adds the
|
||||
# CI-only dump importer. Without the second pass nothing would compile
|
||||
# or exercise backend/explore/dump*.go or cmd/indexbuild at all.
|
||||
test:
|
||||
testdata-force: ## Regenerate the fixture library unconditionally
|
||||
go run ./cmd/gentestdata -force
|
||||
|
||||
testdata-clean: ## Delete the generated fixture library
|
||||
rm -rf test_data/music_library_test test_data/music_library_broken \
|
||||
test_data/music_library_test.manifest.json
|
||||
|
||||
.PHONY: testdata testdata-force testdata-clean
|
||||
|
||||
# The tag sets must match `make test` exactly, or lint is checking three
|
||||
# configurations that nothing builds. webkit2_41 is not optional: without
|
||||
# it wails resolves webkit2gtk-4.0, which Ubuntu 24.04 no longer ships, so
|
||||
# the `dev` pass (wails' own app_dev.go is dev-tagged and pulls in the 4.0
|
||||
# assetserver) fails to typecheck anywhere but Arch.
|
||||
lint:
|
||||
go tool golangci-lint run --build-tags webkit2_41
|
||||
go tool golangci-lint run --build-tags "webkit2_41 indexbuild"
|
||||
go tool golangci-lint run --build-tags "webkit2_41 dev"
|
||||
|
||||
# Three passes: the app build, the `indexbuild` build that adds the
|
||||
# CI-only dump importer, and the `dev` build that adds profiling and
|
||||
# backend/testctl. Without the extra passes nothing would compile or
|
||||
# exercise backend/explore/dump*.go, cmd/indexbuild or the harness
|
||||
# control surface at all.
|
||||
test: testdata
|
||||
go test -tags webkit2_41 -race -count=1 -timeout 120s ./...
|
||||
go test -tags "webkit2_41 indexbuild" -race -count=1 -timeout 300s \
|
||||
./backend/explore/... ./cmd/...
|
||||
# backend/testctl only exists under the `dev` tag, so the pass above
|
||||
# does not compile it, let alone run it.
|
||||
go test -tags "webkit2_41 dev" -race -count=1 -timeout 120s \
|
||||
./backend/testctl/...
|
||||
|
||||
vulncheck:
|
||||
go tool govulncheck ./...
|
||||
|
||||
Reference in New Issue
Block a user