feat(harness): agent-drivable dev harness and CI that gates
Build & publish Arch package / arch-package (push) Successful in 2m8s
CI / check (push) Failing after 1m56s
CI / e2e (push) Skipped
Search index maintenance / maintain-index (push) Successful in 13s

A coding agent could develop this repo's Go packages and could not
develop the application: every path to running YellowJacket ended in a
blocking GTK window, so 265 bound methods, 46 events, 33 component
directories and 13 stores had exactly one form of verification
available — `tsc --noEmit`.

The unlock is that `wails dev`'s dev server on :34115 serves the real
frontend with the real generated bindings against the same Go backend a
desktop window attaches to, so a plain Chromium under Xvfb gets a fully
functional app. Four test tiers now exist, cheapest first:

- `make ui-test` — 313 Vitest tests in a real browser in ~2 s, no app,
  no backend, no display. Works because `frontend/wailsjs/` is a pure
  passthrough to `window.go`/`window.runtime`, so faking just those two
  globals runs the real bindings and the real store code.
- `make test` — services in-process, asserting on the payload the
  frontend would receive, via a new `events.Emit` wrapper.
- `make dev-headless` + `playwright-cli` — the real app, driven
  interactively, with an event bridge on `window.__yjEvents` and a
  dev-only control surface at `/__test/`.
- `make e2e` — 19 of those flows frozen as Playwright specs.

`events.Emit(ctx, …)` replaces all 35 direct `runtime.EventsEmit` call
sites: wails' `getEvents` `log.Fatalf`s on any context without its
runtime, so those paths could not run under test and a background
worker could take the app down. Four packages had each hand-rolled the
same guard; nine more guarded on `ctx != nil`, which does not help.
`TestNoDirectRuntimeEmits` fails the build on a new one.

Fixtures are generated, not committed (`make testdata`), and seeds are
built by *running the app* — never by hand-writing config and DB rows,
which would be a second description of a valid YJ_HOME.

`.gitea/workflows/ci.yml` is the first workflow here that tests
anything; the other three only package, so `gitea_ci` reported only
packaging jobs and misled anyone asking whether a push was healthy.
Both jobs were prototyped to green in a bare ubuntu:24.04 container
before the YAML was written, which immediately caught `make lint`
linting three configurations that nothing builds: all three passes
omitted `webkit2_41`, so wails resolved webkit2gtk-4.0 — which Arch
still ships and Ubuntu 24.04 dropped.

Operational instructions live in `.pi/skills/yellowjacket-dev/`,
measured discoveries in `.planning/NOTES.md`, and architecture in
`CLAUDE.md` — split by tense, not by topic, because a topical split
gives every new fact two plausible homes. `make skill-check` fails a
commit if the skill cites a make target that does not exist.
This commit is contained in:
2026-08-10 23:20:42 -04:00
parent 65333857e2
commit 5ca6cad45a
117 changed files with 14585 additions and 262 deletions
+89
View File
@@ -0,0 +1,89 @@
import {
test,
expect,
callBinding,
resetEvents,
waitForEvent,
} from '../support/fixtures.js';
/**
* The harness testing itself.
*
* If these fail, every other spec's failure is uninterpretable — a
* missing event could mean a broken feature or a broken recorder, and
* telling those apart afterwards is expensive.
*/
test.describe('harness', () => {
test('the app is the real app, not a mock', async ({ app }) => {
// All 11 bound services land on window.go through the dev server.
const services = await app.evaluate(() => Object.keys(window.go));
expect(services).toEqual(
expect.arrayContaining(['queue', 'player', 'library', 'explore']),
);
const state = await callBinding<{ tracks: unknown[] }>(
app,
'queue.Queue.GetState',
);
expect(state).toHaveProperty('tracks');
});
test('backend events are recorded, in order, with payloads', async ({
app,
}) => {
await resetEvents(app);
await callBinding(app, 'player.Player.SetVolume', [37]);
const ev = await waitForEvent(app, 'VolumeChanged');
expect(ev.data).toEqual([37]);
expect(ev.dir).toBe('in');
});
test('exactly one recorder is installed', async ({ app }) => {
// Listeners registered by one evaluate survive into the next, so a
// recorder that re-registers counts every event twice. This is the
// regression test for that.
await resetEvents(app);
await callBinding(app, 'player.Player.SetVolume', [41]);
await waitForEvent(app, 'VolumeChanged');
const count = await app.evaluate(() =>
window.__yjEvents.count('VolumeChanged'),
);
expect(count).toBe(1);
});
test('a binding called with wrong types fails fast', async ({ app }) => {
// player.UserVolume is an int. Passing a float makes the backend
// log "error parsing arguments" and never fire the callback; without
// a timeout the promise never settles and the spec hangs until the
// suite gives up.
const failure = await app.evaluate(async () => {
try {
await window.__yjEvents.call(
'player.Player.SetVolume',
[0.42],
2_000,
);
return 'settled';
} catch (err) {
return (err as Error).message;
}
});
expect(failure).toContain('did not settle');
});
test('the control surface is mounted and seeded', async ({ testctl }) => {
const health = await testctl.health();
expect(health.ok).toBe(true);
expect(health.libraries.length).toBeGreaterThan(0);
expect(health.counts.tracks).toBeGreaterThan(0);
});
});
+68
View File
@@ -0,0 +1,68 @@
import { test, expect } from '../support/fixtures.js';
/**
* The library views, against the generated fixture library
* (`make testdata`): 31 tracks chosen to cover the cases the app has
* code for — unicode and RTL titles, missing tags, a deliberately
* absurd artist name for truncation, duplicates.
*/
test.describe('library views', () => {
test('lands in the app, not the first-run wizard', async ({ app }) => {
// A fresh YJ_HOME puts <first-run-wizard> over everything and it
// intercepts every pointer event, so "the click did nothing" is the
// symptom of an unseeded sandbox rather than a broken control.
//
// Asserted by clicking rather than by inspecting the wizard element:
// the element is always in the DOM and merely renders nothing once a
// library exists, so its presence proves nothing. Playwright's own
// actionability check fails a covered click with "intercepts pointer
// events", which is exactly the condition worth catching.
await expect(app.getByTestId('track-row').first()).toBeVisible();
await app.getByTestId('nav-artists').click({ timeout: 5_000 });
await expect(app.getByTestId('main-content')).toHaveAttribute(
'data-active-view',
'artists',
);
});
test('renders every fixture track, unicode included', async ({
app,
testctl,
}) => {
const health = await testctl.health();
const rows = app.getByTestId('track-row');
await expect(rows).toHaveCount(health.counts.tracks);
// Non-Latin scripts survive the tag reader, the database and the
// renderer. These titles exist in the fixtures for this reason.
await expect(app.getByText('Привет мир')).toBeVisible();
await expect(app.getByText('さくら')).toBeVisible();
await expect(app.getByText('مرحبا بالعالم')).toBeVisible();
});
test('the sidebar navigates between primary views', async ({ app }) => {
const main = app.getByTestId('main-content');
for (const view of ['artists', 'genres', 'albums', 'playlists', 'tracks']) {
await app.getByTestId(`nav-${view}`).click();
await expect(main).toHaveAttribute('data-active-view', view);
await expect(app.getByTestId(`nav-${view}`)).toHaveAttribute(
'aria-current',
'page',
);
}
});
test('the artists view shows the fixture artists', async ({ app }) => {
await app.getByTestId('nav-artists').click();
await expect(app.getByTestId('main-content')).toHaveAttribute(
'data-active-view',
'artists',
);
await expect(app.getByText('Aurora Fields').first()).toBeVisible();
await expect(app.getByText('Pale Circuit').first()).toBeVisible();
});
});
+114
View File
@@ -0,0 +1,114 @@
import {
test,
expect,
callBinding,
resetEvents,
waitForEvent,
LONG_TRACK,
} from '../support/fixtures.js';
/** The one fixture long enough to still be playing on the next line. */
const longRow = (app: import('@playwright/test').Page) =>
app.getByTestId('track-row').filter({ hasText: LONG_TRACK }).first();
/**
* Playback and the queue, driven through the UI and asserted on the
* events the backend actually emits.
*
* Audio really is initialised here: under `dbus-run-session` + Xvfb the
* PulseAudio socket in /run/user is untouched, so InitSpeaker succeeds
* and these tracks genuinely play. A CI container without /run/user
* needs a null sink; everything except the audio itself still works
* without one.
*/
test.describe('playback', () => {
test.beforeEach(async ({ app }) => {
await callBinding(app, 'queue.Queue.Clear').catch(() => {
/* older builds may not expose Clear; the specs below do not need it */
});
await resetEvents(app);
});
test('double-clicking a track plays it', async ({ app }) => {
await longRow(app).dblclick();
const changed = await waitForEvent(app, 'TrackChanged');
expect(changed.data[0]).toBeTruthy();
// The transport flips to Pause, which is the only place the UI
// states "we are playing" in a way a user can see. `exact` is not
// optional: "Add queue to playlist" also matches /play/i.
await expect(
app.getByRole('button', { name: 'Pause', exact: true }),
).toBeVisible();
await expect(app.getByTestId('now-playing-title')).toContainText(
LONG_TRACK,
);
});
test('the elapsed time advances', async ({ app }) => {
await longRow(app).dblclick();
await waitForEvent(app, 'TrackChanged');
// Not a fixed sleep on a fixed value: assert the observable
// outcome, which is that the clock is no longer at zero.
await expect(app.getByTestId('elapsed-time')).not.toHaveText('--:--');
await expect(app.getByTestId('elapsed-time')).not.toHaveText('00:00', {
timeout: 15_000,
});
});
test('pause and play round-trip through the backend', async ({ app }) => {
await longRow(app).dblclick();
await waitForEvent(app, 'TrackChanged');
await resetEvents(app);
await app.getByRole('button', { name: 'Pause', exact: true }).click();
await waitForEvent(app, 'PlaybackStateChanged');
await expect(
app.getByRole('button', { name: 'Play', exact: true }),
).toBeVisible();
});
test('volume changes are pushed back from Go', async ({ app }) => {
await resetEvents(app);
await callBinding(app, 'player.Player.SetVolume', [55]);
const ev = await waitForEvent(app, 'VolumeChanged');
expect(ev.data).toEqual([55]);
});
});
test.describe('queue', () => {
test('playing a track populates the queue panel', async ({ app }) => {
await resetEvents(app);
await longRow(app).dblclick();
await waitForEvent(app, 'QueueChanged');
await expect(app.getByTestId('queue-row')).toHaveCount(1);
const state = await callBinding<{ tracks: unknown[] }>(
app,
'queue.Queue.GetState',
);
expect(state.tracks).toHaveLength(1);
});
test('shuffle and repeat toggles report their state', async ({ app }) => {
const shuffle = app.getByRole('button', { name: 'Shuffle' });
await resetEvents(app);
await shuffle.click();
await waitForEvent(app, 'QueueModeChanged');
await expect(shuffle).toHaveAttribute('aria-pressed', 'true');
await shuffle.click();
await expect(shuffle).toHaveAttribute('aria-pressed', 'false');
});
});
+68
View File
@@ -0,0 +1,68 @@
import { test, expect, resetEvents, waitForEvent } from '../support/fixtures.js';
/**
* The dev-only control surface (backend/testctl), which exists for the
* things a browser genuinely cannot do.
*/
test.describe('control surface', () => {
test('database snapshot and restore round-trip', async ({ testctl }) => {
// VACUUM INTO copies the whole file and the restore copies every
// row back; on a database carrying an explore catalog that is tens
// of seconds, not the default 30s budget for a whole test.
test.setTimeout(180_000);
const before = (await testctl.health()).counts.tracks;
await testctl.snapshot('e2e-pristine');
await testctl.sql('DELETE FROM audio_files');
expect((await testctl.health()).counts.tracks).toBe(0);
// Restore copies rows rather than files, because the app holds the
// database open across two connection pools and cannot be made to
// reopen it from here.
await testctl.restore('e2e-pristine');
expect((await testctl.health()).counts.tracks).toBe(before);
});
test('a forced backend event reaches the browser', async ({
app,
testctl,
}) => {
// LibraryScanProgress normally only arrives during a real scan.
// Emitting it directly is how a push-driven view gets exercised
// without staging the work that would produce it.
await resetEvents(app);
await testctl.emit('LibraryScanProgress', {
current: 7,
total: 31,
currentFile: 'probe.mp3',
});
const ev = await waitForEvent(app, 'LibraryScanProgress');
expect(ev.data[0]).toMatchObject({ current: 7, total: 31 });
});
test('sql reads return rows, writes return a count', async ({ testctl }) => {
const read = await testctl.sql(
'SELECT COUNT(*) AS n FROM audio_files',
);
expect(read.rows[0].n).toBeGreaterThan(0);
const write = await testctl.sql(
'UPDATE player_state SET volume = volume',
);
expect(write).toHaveProperty('rowsAffected');
});
test('bad input is rejected with a reason, not a bare status', async ({
testctl,
}) => {
await expect(testctl.snapshot('../escape')).rejects.toThrow(/name must/);
await expect(testctl.restore('nope')).rejects.toThrow(/no such snapshot/);
});
});