test(ui): refresh two stale baselines and settle whether they gate
`make ui-visual` had been red on main since #27, and nothing runs it, so four references had drifted across three unrelated merges. Two were refreshed with #186; these are the other two. Each recorded two changes, not one. `app-sidebar` lost Jobs (#27, shipped) and moved its highlight from Home to Tracks; `now-playing` gained the source line (shipped) and was playing from "a dynamic mix". Both are singleton stores read by a case that sets nothing, so the shot photographs whatever the case above it left behind — blessing that would have pinned the file's own ordering into a PNG. Both cases state their world now, and only then are the references re-recorded. The second half of the issue asks whether this tier should gate, and the answer is measured rather than preferred: replayed in a bare ubuntu:24.04 container — CI's `check` image — three of the ten baselines fail on rendering alone (`track-info` and one `page-header` shot at ratio 0.03 against a 0.02 allowance, `seek-bar` one pixel shorter), and the two stale ones disagree about their new height between the machines. So CI cannot run this suite without a second, container-recorded baseline set that every local run would then fail against, and a pre-push hook is the same fault with the machines swapped. It stays local and opt-in; what replaces the gate is the rule that a change moving a component's geometry refreshes that component's baseline in the same commit, having read the image, and never one it did not cause. Written where a person meets it: the skill's tier doc has the table, SKILL.md has the obligation, CLAUDE.md has the constraint. Deleting the baselines was the third option and is declined: this tier has caught one thing no other could, the `<span>` that lost the UA stylesheet's `box-sizing` and grew a badge 36→38px. Closes #196
This commit is contained in:
@@ -158,7 +158,7 @@ only climb when it cannot.
|
||||
| You changed | Run | Cost |
|
||||
|---|---|---|
|
||||
| A Lit component, a store, the shortcut service | `make ui-test` | ~2 s, no app |
|
||||
| …and it renders differently | `make ui-visual` | + 6 baselines, opt-in |
|
||||
| …and it renders differently | `make ui-visual` | + 10 baselines, opt-in, never gates |
|
||||
| Any Go code | `make test` | 3 passes, ~2 min |
|
||||
| A service that emits events | `make test` — assert on the payload, see `backend/queue/emit_test.go` | in-process, no app |
|
||||
| A bound method or a bound struct field | `make bindings` then `make ui-test` | ~1.5 s + 2 s |
|
||||
@@ -180,6 +180,13 @@ less than it looks.)
|
||||
|
||||
Two rules about climbing:
|
||||
|
||||
- **If you moved a component's geometry, run `make ui-visual` and
|
||||
refresh that component's baseline in the same commit.** Nothing else
|
||||
will: it is the one tier in this repo no hook and no CI job runs, and
|
||||
it cannot be one — its references are machine-specific, measured in
|
||||
[references/ui-tier.md](references/ui-tier.md). Four of them drifted
|
||||
across three merges before anyone noticed (#196). Read the image;
|
||||
never bless a reference you did not cause.
|
||||
- **A component test passing is not the app rendering.** If you touched
|
||||
anything in `frontend/src`, verify it in the real app too — start it
|
||||
headless, `screenshot --filename=/tmp/shot.png`, and *read the PNG*.
|
||||
|
||||
@@ -78,9 +78,58 @@ synchronously.
|
||||
Microtasks and not a timer, deliberately: a timer hangs forever under
|
||||
the suites that install fake ones.
|
||||
|
||||
Visual baselines are font-hinting and compositing sensitive, which is
|
||||
why they are opt-in: they only mean anything on the machine that
|
||||
recorded them.
|
||||
## The visual tier does not gate, and that is measured (#196)
|
||||
|
||||
`make ui-visual` is the same suite with nine `toMatchScreenshot`
|
||||
baselines switched on. **Nothing runs it but a person**, deliberately,
|
||||
and the reason is a number rather than a preference: the committed
|
||||
baselines were recorded on Arch, and replayed in a bare `ubuntu:24.04`
|
||||
container — CI's `check` image — three of them fail for reasons that
|
||||
have nothing to do with any component.
|
||||
|
||||
| baseline | Arch | ubuntu:24.04 |
|
||||
|---|---|---|
|
||||
| `page-header` filtered-by-search | passes | ratio 0.03 differ, against a 0.02 allowance |
|
||||
| `track-info` | passes | ratio 0.03 differ |
|
||||
| `seek-bar` | 1152×18 | 1152×17 |
|
||||
|
||||
The two references that were genuinely stale did not even agree about
|
||||
their *new* size — `now-playing` renders 1152×65 on Arch and 1152×64 in
|
||||
the container. So moving CI's `check` job from `make ui-test` to
|
||||
`make ui-visual` is not a one-line change: it needs a second,
|
||||
container-recorded baseline set, which every local run would then fail
|
||||
against. That is the same trap the other way round, and a pre-push hook
|
||||
is the same fault again — one machine's baselines against everybody
|
||||
else's renderer.
|
||||
|
||||
So the tier stays local and opt-in, and the rule that replaces the gate
|
||||
is:
|
||||
|
||||
- **A change that moves a component's geometry refreshes that
|
||||
component's reference in the same commit, having read the image.**
|
||||
Look at the PNG; the dimensions in the failure message are the cheap
|
||||
half of the answer.
|
||||
- **Never refresh a reference you did not cause.** #196 exists because
|
||||
four of them drifted across three unrelated merges, and every red run
|
||||
made the next person likelier to stop running the tier than to read
|
||||
it.
|
||||
- **State the world the shot is taken in.** The stores are singletons,
|
||||
so a visual case that sets nothing photographs whatever the previous
|
||||
case left behind — which is how the sidebar's baseline came to have
|
||||
Tracks lit and `now-playing`'s to be playing from a dynamic mix.
|
||||
- **`make ui-visual-update UI_ARGS=<path>` does not filter** and
|
||||
re-records *every* baseline, blessing any stale one in silence:
|
||||
vitest's `--update` takes the following positional as its value. Until
|
||||
#204 lands, record one file with
|
||||
`cd frontend && YJ_VISUAL=1 npx vitest run --update=true <path>`, and
|
||||
check `git status` before committing either way.
|
||||
|
||||
What the tier is worth, for the record: it is a *layout* check, blind to
|
||||
colour (the component tier has no `:root`, so it renders the fallbacks —
|
||||
`make ui-visual` passed unchanged through a whole palette rewrite,
|
||||
twice), and it has caught one thing nothing else could — swapping
|
||||
`library-status-indicator`'s `<button>` for a `<span>` lost the UA
|
||||
stylesheet's `box-sizing` and grew the badge 36→38px.
|
||||
|
||||
## Bindings
|
||||
|
||||
|
||||
Reference in New Issue
Block a user