feat(harness): agent-drivable dev harness and CI that gates
Build & publish Arch package / arch-package (push) Successful in 2m8s
CI / check (push) Failing after 1m56s
CI / e2e (push) Skipped
Search index maintenance / maintain-index (push) Successful in 13s

A coding agent could develop this repo's Go packages and could not
develop the application: every path to running YellowJacket ended in a
blocking GTK window, so 265 bound methods, 46 events, 33 component
directories and 13 stores had exactly one form of verification
available — `tsc --noEmit`.

The unlock is that `wails dev`'s dev server on :34115 serves the real
frontend with the real generated bindings against the same Go backend a
desktop window attaches to, so a plain Chromium under Xvfb gets a fully
functional app. Four test tiers now exist, cheapest first:

- `make ui-test` — 313 Vitest tests in a real browser in ~2 s, no app,
  no backend, no display. Works because `frontend/wailsjs/` is a pure
  passthrough to `window.go`/`window.runtime`, so faking just those two
  globals runs the real bindings and the real store code.
- `make test` — services in-process, asserting on the payload the
  frontend would receive, via a new `events.Emit` wrapper.
- `make dev-headless` + `playwright-cli` — the real app, driven
  interactively, with an event bridge on `window.__yjEvents` and a
  dev-only control surface at `/__test/`.
- `make e2e` — 19 of those flows frozen as Playwright specs.

`events.Emit(ctx, …)` replaces all 35 direct `runtime.EventsEmit` call
sites: wails' `getEvents` `log.Fatalf`s on any context without its
runtime, so those paths could not run under test and a background
worker could take the app down. Four packages had each hand-rolled the
same guard; nine more guarded on `ctx != nil`, which does not help.
`TestNoDirectRuntimeEmits` fails the build on a new one.

Fixtures are generated, not committed (`make testdata`), and seeds are
built by *running the app* — never by hand-writing config and DB rows,
which would be a second description of a valid YJ_HOME.

`.gitea/workflows/ci.yml` is the first workflow here that tests
anything; the other three only package, so `gitea_ci` reported only
packaging jobs and misled anyone asking whether a push was healthy.
Both jobs were prototyped to green in a bare ubuntu:24.04 container
before the YAML was written, which immediately caught `make lint`
linting three configurations that nothing builds: all three passes
omitted `webkit2_41`, so wails resolved webkit2gtk-4.0 — which Arch
still ships and Ubuntu 24.04 dropped.

Operational instructions live in `.pi/skills/yellowjacket-dev/`,
measured discoveries in `.planning/NOTES.md`, and architecture in
`CLAUDE.md` — split by tense, not by topic, because a topical split
gives every new fact two plausible homes. `make skill-check` fails a
commit if the skill cites a make target that does not exist.
This commit is contained in:
2026-08-10 23:20:42 -04:00
parent 65333857e2
commit 5ca6cad45a
117 changed files with 14585 additions and 262 deletions
+172
View File
@@ -0,0 +1,172 @@
import { fileURLToPath } from 'node:url';
import { dirname, resolve } from 'node:path';
import { test as base, expect, type Page } from '@playwright/test';
const here = dirname(fileURLToPath(import.meta.url));
/** The same bridge `playwright-cli` loads, so an exploratory session and
* a committed spec see an identical page. */
const INIT_SCRIPT = resolve(here, '../../.playwright/init-events.js');
/**
* The 90-second fixture track (`cmd/gentestdata`, case `edge-lengths`).
*
* Every other fixture is 26 seconds, which is shorter than the time a
* spec takes to click something — a "pause it" test against one of
* those races the track finishing and fails on a UI that is correct.
*/
export const LONG_TRACK = 'Long Player';
/** Shape of the recorder installed by .playwright/init-events.js. */
export type YjEvent = {
seq: number;
name: string;
data: unknown[];
dir: 'in' | 'out';
t: number;
};
/**
* Await a backend event instead of a timeout.
*
* Half of this app is push-driven, and the events that matter
* (scan progress, job updates, playback state) arrive whenever the
* backend gets to them. `waitForEvent` resolves against events already
* buffered as well as future ones, so there is no race between doing
* the thing and starting to listen.
*/
export async function waitForEvent(
page: Page,
name: string,
opts: { timeoutMs?: number; since?: number } = {},
): Promise<YjEvent> {
return page.evaluate(
([n, o]) => window.__yjEvents.wait(n as string, o as object),
[name, { timeoutMs: 10_000, ...opts }] as const,
) as Promise<YjEvent>;
}
/** Drop the event buffer. Never re-register a recorder: listeners
* survive across evaluate calls and a second recorder double-counts. */
export async function resetEvents(page: Page): Promise<void> {
await page.evaluate(() => void window.__yjEvents.reset());
}
/** name -> count, for asserting on (or debugging) what actually fired. */
export async function eventNames(
page: Page,
): Promise<Record<string, number>> {
return page.evaluate(() => window.__yjEvents.names());
}
/**
* Call a bound Go method with a timeout.
*
* Wrong argument types make the backend log "error parsing arguments"
* and never fire the callback, so an unguarded call hangs until the
* whole spec times out with no clue why. This fails in seconds and
* says where to look.
*/
export async function callBinding<T = unknown>(
page: Page,
path: string,
args: unknown[] = [],
timeoutMs = 10_000,
): Promise<T> {
return page.evaluate(
([p, a, t]) =>
window.__yjEvents.call(p as string, a as unknown[], t as number),
[path, args, timeoutMs] as const,
) as Promise<T>;
}
/** Thin client for the dev-only /__test/ surface (backend/testctl). */
export class TestCtl {
constructor(private readonly baseURL: string) {}
private async req(path: string, init?: RequestInit) {
const res = await fetch(`${this.baseURL}${path}`, {
signal: AbortSignal.timeout(120_000),
...init,
});
const body = await res.json();
if (!res.ok) {
throw new Error(`testctl ${path}: ${body.error ?? res.status}`);
}
return body;
}
health() {
return this.req('/__test/health');
}
snapshot(name: string) {
return this.req(`/__test/db/snapshot?name=${name}`, { method: 'POST' });
}
restore(name: string) {
return this.req(`/__test/db/restore?name=${name}`, { method: 'POST' });
}
emit(name: string, ...data: unknown[]) {
return this.req('/__test/emit', {
method: 'POST',
body: JSON.stringify({ name, data }),
});
}
sql(sql: string, args: unknown[] = []) {
return this.req('/__test/sql', {
method: 'POST',
body: JSON.stringify({ sql, args }),
});
}
}
export const test = base.extend<{ app: Page; testctl: TestCtl }>({
/** A page with the event bridge installed and the app loaded and
* actually talking to the backend — not merely DOM-ready, which is
* earlier and lies. */
app: async ({ page, baseURL }, use) => {
await page.addInitScript({ path: INIT_SCRIPT });
await page.goto(baseURL!);
await page.evaluate(() => window.__yjEvents.ready(20_000));
await use(page);
},
testctl: async ({ baseURL }, use) => {
await use(new TestCtl(baseURL!));
},
});
export { expect };
declare global {
interface Window {
__yjEvents: {
version: number;
seq: number;
log: YjEvent[];
reset(): number;
all(name?: string): YjEvent[];
count(name?: string): number;
last(name?: string): YjEvent | null;
since(seq: number): YjEvent[];
names(): Record<string, number>;
wait(
name: string,
opts?: {
timeoutMs?: number;
since?: number;
match?: (data: unknown[], entry: YjEvent) => boolean;
},
): Promise<YjEvent>;
ready(timeoutMs?: number): Promise<boolean>;
call(path: string, args?: unknown[], timeoutMs?: number): Promise<any>;
};
go: Record<string, Record<string, Record<string, (...a: any[]) => any>>>;
}
}
+62
View File
@@ -0,0 +1,62 @@
import type { FullConfig } from '@playwright/test';
/**
* Fail fast, and fail legibly.
*
* Without this the first spec dies on a connection refused deep inside
* a `page.goto`, which reads like a Playwright problem rather than "you
* forgot to start the app". Checking /__test/health also verifies the
* two things every spec below assumes: that the control surface is
* mounted (dev build + YJ_TESTCTL=1) and that the app is pointed at the
* seeded fixture library rather than someone's real collection.
*/
const HELP = `
The app is not running, or is not a seeded dev build. Start it with:
make testdata # once — generates the fixtures
make sandbox-seed NAME=default # once — builds a seed by running the app
make dev-headless SEED=default # starts in the background and returns
and stop it afterwards with 'make dev-stop'.
`;
export default async function globalSetup(config: FullConfig) {
const baseURL =
config.projects[0]?.use?.baseURL ?? 'http://localhost:34115';
let health: any;
try {
const res = await fetch(`${baseURL}/__test/health`, {
signal: AbortSignal.timeout(5_000),
});
if (!res.ok) throw new Error(`health returned ${res.status}`);
health = await res.json();
} catch (err) {
throw new Error(
`e2e: no healthy backend at ${baseURL} (${String(err)})\n${HELP}`,
);
}
if (!health.libraries?.length) {
throw new Error(
`e2e: backend has no library configured — specs would land on ` +
`the first-run wizard, which intercepts every pointer event.\n${HELP}`,
);
}
const tracks = health.counts?.tracks ?? 0;
if (tracks < 1) {
throw new Error(
`e2e: backend library is empty (${tracks} tracks).\n${HELP}`,
);
}
console.log(
`e2e: backend up — ${tracks} tracks in ` +
`${health.libraries.map((l: any) => l.name).join(', ')}`,
);
}