test(ui): refresh two stale baselines and settle whether they gate #205

Merged
logan merged 2 commits from test/196-visual-tier-gates into main 2026-08-25 16:39:33 +00:00
Collaborator

The issue. make ui-visual had been failing on main since #27
and nothing runs it, so four stored screenshots had drifted across
three unrelated pieces of work. Two of the four were refreshed with
#186 (2100f00); this is the other two, plus the question the issue
holds open — whether this tier should gate.

Commits

commit issue what
7708bc7 #196 refresh the two remaining baselines, and record why the tier cannot gate

What changed, and why

The refresh, having read each diff. Both stale references recorded
two changes, not one:

  • app-sidebar lost Jobs (#27, shipped) and moved its
    highlight from Home to Tracks.
  • now-playing gained the source line (shipped) and was
    playing from "a dynamic mix".

Neither highlight nor source is set by the case taking the shot: the
stores are singletons, so it photographs whatever the case above it
left behind. Blessing that would have pinned the file's own test
ordering into a PNG. Both cases state their world now — setView('home')
and a real album source — and only then were the references
re-recorded.

Whether the tier gates: measured, not preferred. The issue lists
three options and asks for an argument. The whole question turns on
whether an Arch-recorded baseline survives CI's container, so I ran the
suite in a bare ubuntu:24.04 (node 22, the same pnpm install and
playwright install --with-deps chromium CI's check job does) against
the committed baselines:

baseline Arch ubuntu:24.04
page-header filtered-by-search passes ratio 0.03 differ, against a 0.02 allowance
track-info passes ratio 0.03 differ
seek-bar 1152×18 1152×17
app-sidebar (stale) 180×335 180×335
now-playing (stale) 1152×65 1152×64

Three of ten fail on font metrics and compositing alone, and the two
genuinely stale ones do not even agree about their new height. So
option 1 is not a one-line CI change — it needs a second,
container-recorded baseline set that every local run would then fail
against, which is the same trap with the machines swapped, and a
pre-push hook (option 2's stronger form) is that fault again.

Option 3 is declined because this tier has caught one thing no
other could: swapping library-status-indicator's <button> for a
<span> lost the UA stylesheet's box-sizing and grew the badge
36→38px (.planning/NOTES.md: "Nothing but the stored screenshot would
have noticed").

So option 2: the tier stays local and opt-in, and what replaces the
gate is a rule written where a person meets it —
.pi/skills/yellowjacket-dev/references/ui-tier.md (the table, the
reasons, and what the tier is and is not worth),
SKILL.md's "rules about climbing" (the one-line obligation), and
CLAUDE.md (the constraint). The rule: a change that moves a
component's geometry refreshes that component's baseline in the same
commit, having read the image, and never one it did not cause
— plus
a visual case states the world it photographs, which is the fault
found above.

Verification

tier result
make ui-visual 1092 passed, 99 files — first green run of this tier since before #27; both refreshed files also pass in isolation
make ui-test 1092 passed
npx tsc --noEmit (frontend) clean
make skill-check clean, 47 documented targets
make css-check clean
ubuntu:24.04 container replay the table above — the evidence for the decision half

make lint, make test and make e2e were not run: no Go and no
shipped frontend code changed. The only source files touched are two
test cases and two PNGs.

Deliberately not done

  • The Makefile trap I hit on the way is filed as #204, not
    fixed here: make ui-visual-update UI_ARGS=<path> re-records every
    baseline, because vitest's --update swallows the following
    positional as its value. It is #196's own hazard living in the tool
    meant to fix it, and it is one character — but it is a different
    issue.
  • No container-recorded second baseline set. That is the door left
    open if this should ever gate; the measurement above is what it would
    have to pay for.

Closes #196

**The issue.** `make ui-visual` had been failing on `main` since #27 and nothing runs it, so four stored screenshots had drifted across three unrelated pieces of work. Two of the four were refreshed with #186 (`2100f00`); this is the other two, plus the question the issue holds open — whether this tier should gate. ## Commits | commit | issue | what | |---|---|---| | `7708bc7` | #196 | refresh the two remaining baselines, and record why the tier cannot gate | ## What changed, and why **The refresh, having read each diff.** Both stale references recorded *two* changes, not one: - `app-sidebar` lost **Jobs** (#27, shipped) **and** moved its highlight from Home to Tracks. - `now-playing` gained the **source line** (shipped) **and** was playing from "a dynamic mix". Neither highlight nor source is set by the case taking the shot: the stores are singletons, so it photographs whatever the case above it left behind. Blessing that would have pinned the file's own test ordering into a PNG. Both cases state their world now — `setView('home')` and a real album source — and only then were the references re-recorded. **Whether the tier gates: measured, not preferred.** The issue lists three options and asks for an argument. The whole question turns on whether an Arch-recorded baseline survives CI's container, so I ran the suite in a bare `ubuntu:24.04` (node 22, the same `pnpm install` and `playwright install --with-deps chromium` CI's `check` job does) against the committed baselines: | baseline | Arch | ubuntu:24.04 | |---|---|---| | `page-header` filtered-by-search | passes | ratio 0.03 differ, against a 0.02 allowance | | `track-info` | passes | ratio 0.03 differ | | `seek-bar` | 1152×18 | 1152×17 | | `app-sidebar` (stale) | 180×335 | 180×335 | | `now-playing` (stale) | 1152×**65** | 1152×**64** | Three of ten fail on font metrics and compositing alone, and the two genuinely stale ones do not even agree about their *new* height. So **option 1 is not a one-line CI change** — it needs a second, container-recorded baseline set that every local run would then fail against, which is the same trap with the machines swapped, and a pre-push hook (option 2's stronger form) is that fault again. **Option 3 is declined** because this tier has caught one thing no other could: swapping `library-status-indicator`'s `<button>` for a `<span>` lost the UA stylesheet's `box-sizing` and grew the badge 36→38px (`.planning/NOTES.md`: "Nothing but the stored screenshot would have noticed"). So **option 2**: the tier stays local and opt-in, and what replaces the gate is a rule written where a person meets it — `.pi/skills/yellowjacket-dev/references/ui-tier.md` (the table, the reasons, and what the tier is and is not worth), `SKILL.md`'s "rules about climbing" (the one-line obligation), and `CLAUDE.md` (the constraint). The rule: *a change that moves a component's geometry refreshes that component's baseline in the same commit, having read the image, and never one it did not cause* — plus *a visual case states the world it photographs*, which is the fault found above. ## Verification | tier | result | |---|---| | `make ui-visual` | **1092 passed, 99 files** — first green run of this tier since before #27; both refreshed files also pass in isolation | | `make ui-test` | 1092 passed | | `npx tsc --noEmit` (frontend) | clean | | `make skill-check` | clean, 47 documented targets | | `make css-check` | clean | | `ubuntu:24.04` container replay | the table above — the evidence for the decision half | `make lint`, `make test` and `make e2e` were **not** run: no Go and no shipped frontend code changed. The only source files touched are two test cases and two PNGs. ## Deliberately not done - **The Makefile trap I hit on the way** is filed as **#204**, not fixed here: `make ui-visual-update UI_ARGS=<path>` re-records *every* baseline, because vitest's `--update` swallows the following positional as its value. It is #196's own hazard living in the tool meant to fix it, and it is one character — but it is a different issue. - **No container-recorded second baseline set.** That is the door left open if this should ever gate; the measurement above is what it would have to pay for. Closes #196
Author
Collaborator

CI is green on 7708bc7 (run 544): check success, e2e success —
both Chromium and WebKit.

Worth one note against the check job, since this PR is about that
distinction: it runs make ui-test, which is this same suite with the
comparisons off, so a green tick here says nothing about the ten
screenshots. That is the property being written down rather than a gap
introduced by this change — the container measurement in the
description is why it stays that way.

**CI is green** on `7708bc7` (run 544): `check` success, `e2e` success — both Chromium and WebKit. Worth one note against the `check` job, since this PR is about that distinction: it runs `make ui-test`, which is this same suite with the comparisons off, so a green tick here says nothing about the ten screenshots. That is the property being written down rather than a gap introduced by this change — the container measurement in the description is why it stays that way.
logan added 2 commits 2026-08-25 16:39:28 +00:00
`make ui-visual` had been red on main since #27, and nothing runs it,
so four references had drifted across three unrelated merges. Two were
refreshed with #186; these are the other two.

Each recorded two changes, not one. `app-sidebar` lost Jobs (#27,
shipped) and moved its highlight from Home to Tracks; `now-playing`
gained the source line (shipped) and was playing from "a dynamic mix".
Both are singleton stores read by a case that sets nothing, so the shot
photographs whatever the case above it left behind — blessing that
would have pinned the file's own ordering into a PNG. Both cases state
their world now, and only then are the references re-recorded.

The second half of the issue asks whether this tier should gate, and
the answer is measured rather than preferred: replayed in a bare
ubuntu:24.04 container — CI's `check` image — three of the ten
baselines fail on rendering alone (`track-info` and one `page-header`
shot at ratio 0.03 against a 0.02 allowance, `seek-bar` one pixel
shorter), and the two stale ones disagree about their new height
between the machines. So CI cannot run this suite without a second,
container-recorded baseline set that every local run would then fail
against, and a pre-push hook is the same fault with the machines
swapped. It stays local and opt-in; what replaces the gate is the rule
that a change moving a component's geometry refreshes that component's
baseline in the same commit, having read the image, and never one it
did not cause. Written where a person meets it: the skill's tier doc
has the table, SKILL.md has the obligation, CLAUDE.md has the
constraint.

Deleting the baselines was the third option and is declined: this tier
has caught one thing no other could, the `<span>` that lost the UA
stylesheet's `box-sizing` and grew a badge 36→38px.

Closes #196
docs(agent): drop #204's workaround from the baseline rule
CI / e2e (push) Skipped
CI / check (push) Skipped
CI / check (pull_request) Successful in 2m43s
CI / e2e (pull_request) Successful in 10m1s
a82d29abd7
#204 landed first, so the ui-tier rule can name `make ui-visual-update
UI_ARGS=<path>` rather than the raw vitest invocation it needed while
the recipe swallowed its filter.
logan force-pushed test/196-visual-tier-gates from 7708bc7454 to a82d29abd7 2026-08-25 16:39:28 +00:00 Compare
logan merged commit 025ed59480 into main 2026-08-25 16:39:33 +00:00
Sign in to join this conversation.