ci(skill-check): scan the docs a contributor reads #229

Merged
logan merged 3 commits from docs/220-skill-check-scope into main 2026-09-03 17:35:40 +00:00
Collaborator

What the issue was

scripts/skill-check.sh asserts that every make <target> named in a
doc exists. Its scanned set was .pi/ plus CLAUDE.md — so
CONTRIBUTING.md, which #50 made the document a human contributor
reads a build command out of, named 21 targets that nothing verified.
README.md names none today and is in the set for the same reason.

#220 also names a smaller half: the lefthook pre-commit glob was
{Makefile,.pi/**/*.md} while the script already read CLAUDE.md, so a
commit touching only CLAUDE.md did not fire the hook at all.

What changed

Commit Issue
ci(skill-check): scan the docs a contributor reads #220
  • scripts/skill-check.sh — README.md and CONTRIBUTING.md join
    the scanned set. The file list is one docs variable used twice, once
    to read the mentions out and once to say which file a missing target
    came from, because the failure message carried its own copy and a
    second list is a second thing to forget. [ -d .pi ] || exit 0 went
    with it: gating the whole run on .pi/ would make the human-facing
    half conditional on the agent-facing one, and [ -n "$docs" ] is the
    same "nothing to scan" exit without the coupling. ls carries
    || true because it exits non-zero when any argument is missing
    while still printing the others, and under set -e that sinks the
    assignment rather than scanning what is there.
  • lefthook.yml — the glob is the script's scanned set, AGENTS.md
    included, since the script asserts that one is a symlink.
  • Makefile, CLAUDE.md — the two one-line descriptions of the
    target said .pi/ / "the agent docs" and no longer do.

Verification

A widened file list that catches nothing looks identical to one that
works, so every claim below was planted rather than read.

  • The new surfaces catch. A bad target planted in README.md and in
    CONTRIBUTING.md, in both matched forms (backticked inline, and
    line-start inside a fence) — 4/4 fail, each naming the right file.
  • The hole, measured. The same two plants against the pre-change
    script: skill-check: 47 documented make targets, all present,
    exit 0.
  • No coverage lost. CLAUDE.md and .pi/skills/…/SKILL.md
    re-planted: both still caught, both still named correctly.
  • No new false positives. Unfenced prose (a renamed make no-such-prose-target is just English here.) still does not match,
    which is the rule that stops the check failing on reflow. #220 asked
    for a bare non-target make mention to be checked for while here:
    there is none in either file, which is what the clean run says.
  • The hook. A planted CONTRIBUTING.md staged, then
    lefthook run pre-commit --commands skill-check: fails under the new
    glob, (skip) no matching staged files under the old one. Same for a
    CLAUDE.md-only stage. This PR's own commit fired it, on a staged
    CLAUDE.md.
  • Degenerate trees (/tmp, script + Makefile only): only
    CONTRIBUTING.md and no .pi/ → catches; no docs at all → exit 0;
    clean docs → passes.
  • make skill-check on the branch: 47 documented make targets, all
    present
    — unchanged, because the set is a union and everything
    CONTRIBUTING.md names was already named elsewhere. Coverage moved;
    the count could not.

Tiers not run, and why. No Go, no .sql, no .templ, no bound
signature, no frontend/: make lint, make test, make generate,
make bindings, make ui-test and make e2e do not apply and were not
run. Nothing renders differently, so no screenshot tier applies. CI's
check job runs make skill-check itself.

Deliberately not done

Filed #228 rather than fixed here: a `make … ` span the file
hard-wraps across two lines is matched by nothing, and
CONTRIBUTING.md:80 already holds one (make dev-headless, wrapped).
Nothing is broken today — that target is named on one line elsewhere —
but a target named only inside a wrapped span would be missed, and
these two files are the ones that invite the shape. Joining lines before
matching re-opens the failure the fence rule exists for, where a
sentence wrapped onto a line beginning with the word make and a single
letter, and the check failed on a target of that name. That needs a rule
for which joins are safe, which is a different change from widening a
file list. (It bit this PR's journal entry, which said so with
backticks on the first draft and failed the check.)

Closes #220

### What the issue was `scripts/skill-check.sh` asserts that every `make <target>` named in a doc exists. Its scanned set was `.pi/` plus `CLAUDE.md` — so `CONTRIBUTING.md`, which #50 made the document a *human* contributor reads a build command out of, named 21 targets that nothing verified. `README.md` names none today and is in the set for the same reason. #220 also names a smaller half: the lefthook `pre-commit` glob was `{Makefile,.pi/**/*.md}` while the script already read `CLAUDE.md`, so a commit touching only `CLAUDE.md` did not fire the hook at all. ### What changed | Commit | Issue | | --- | --- | | `ci(skill-check): scan the docs a contributor reads` | #220 | - **`scripts/skill-check.sh`** — `README.md` and `CONTRIBUTING.md` join the scanned set. The file list is one `docs` variable used twice, once to read the mentions out and once to say which file a missing target came from, because the failure message carried its own copy and a second list is a second thing to forget. `[ -d .pi ] || exit 0` went with it: gating the whole run on `.pi/` would make the human-facing half conditional on the agent-facing one, and `[ -n "$docs" ]` is the same "nothing to scan" exit without the coupling. `ls` carries `|| true` because it exits non-zero when *any* argument is missing while still printing the others, and under `set -e` that sinks the assignment rather than scanning what is there. - **`lefthook.yml`** — the glob is the script's scanned set, `AGENTS.md` included, since the script asserts that one is a symlink. - **`Makefile`, `CLAUDE.md`** — the two one-line descriptions of the target said `.pi/` / "the agent docs" and no longer do. ### Verification A widened file list that catches nothing looks identical to one that works, so every claim below was planted rather than read. - **The new surfaces catch.** A bad target planted in `README.md` and in `CONTRIBUTING.md`, in both matched forms (backticked inline, and line-start inside a fence) — 4/4 fail, each naming the right file. - **The hole, measured.** The same two plants against the *pre-change* script: `skill-check: 47 documented make targets, all present`, exit 0. - **No coverage lost.** `CLAUDE.md` and `.pi/skills/…/SKILL.md` re-planted: both still caught, both still named correctly. - **No new false positives.** Unfenced prose (`a renamed make no-such-prose-target is just English here.`) still does not match, which is the rule that stops the check failing on reflow. #220 asked for a bare non-target `make` mention to be checked for while here: there is none in either file, which is what the clean run says. - **The hook.** A planted `CONTRIBUTING.md` staged, then `lefthook run pre-commit --commands skill-check`: fails under the new glob, `(skip) no matching staged files` under the old one. Same for a `CLAUDE.md`-only stage. This PR's own commit fired it, on a staged `CLAUDE.md`. - **Degenerate trees** (`/tmp`, script + Makefile only): only `CONTRIBUTING.md` and no `.pi/` → catches; no docs at all → exit 0; clean docs → passes. - `make skill-check` on the branch: **47 documented make targets, all present** — unchanged, because the set is a union and everything `CONTRIBUTING.md` names was already named elsewhere. Coverage moved; the count could not. **Tiers not run, and why.** No Go, no `.sql`, no `.templ`, no bound signature, no `frontend/`: `make lint`, `make test`, `make generate`, `make bindings`, `make ui-test` and `make e2e` do not apply and were not run. Nothing renders differently, so no screenshot tier applies. CI's `check` job runs `make skill-check` itself. ### Deliberately not done **Filed #228** rather than fixed here: a `` `make … ` `` span the file hard-wraps across two lines is matched by nothing, and `CONTRIBUTING.md:80` already holds one (`make dev-headless`, wrapped). Nothing is broken today — that target is named on one line elsewhere — but a target named *only* inside a wrapped span would be missed, and these two files are the ones that invite the shape. Joining lines before matching re-opens the failure the fence rule exists for, where a sentence wrapped onto a line beginning with the word make and a single letter, and the check failed on a target of that name. That needs a rule for which joins are safe, which is a different change from widening a file list. (It bit this PR's journal entry, which said so with backticks on the first draft and failed the check.) Closes #220
logan added 1 commit 2026-08-28 07:39:00 +00:00
ci(skill-check): scan the docs a contributor reads
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m0s
CI / e2e (pull_request) Successful in 10m18s
26251badda
The check asserts that every make target named in a doc exists, and its
scanned set was .pi/ plus CLAUDE.md.  Since #50, CONTRIBUTING.md is the
document a *human* goes to for a build command, and it names 21 targets
that nothing verified; README.md names none today and is in for the same
reason.  The script's own header sentence is the argument — a renamed
target sends a person off the same cliff it sends an agent off.

The file list is now one `docs` variable used twice, because the failure
message carried a second copy of it and a second list is a second thing
to forget.  The `[ -d .pi ]` guard went with it: gating the whole run on
.pi/ would make the human-facing half conditional on the agent-facing
one, and an empty list is the same "nothing to scan" exit without the
coupling.

The lefthook glob is that scanned set now rather than
{Makefile,.pi/**/*.md} — #220's smaller half, and it did not fire on
CLAUDE.md either, which the script had read for months.

Verified by planting a bad target rather than by reading the diff: both
matched forms in each of the four scanned surfaces, each naming the
right file; the same two plants pass on the pre-change script; unfenced
prose still does not match; and the hook fires on a staged
CONTRIBUTING.md under the new glob where the old one skipped it.  The
count is unchanged at 47 — the set is a union — so coverage is the only
thing that moved.

Closes #220
Author
Collaborator

CI: green on run 18244 (re-run). check success, e2e success,
commit 26251ba.

The first attempt of that run failed check at the Test step, on
TestYtDlpSearchSkipsUnparseableLines — which is #146, and unrelated to
this branch by construction: the diff contains no Go. The re-run passed
with nothing changed. That run did produce the message #146 asked for
and it is now on that issue:

--- FAIL: TestYtDlpSearchSkipsUnparseableLines (0.00s)
    provider_ytdlp_test.go:178: Search: yt-dlp failed: search:
        fork/exec /tmp/TestYtDlpSearchSkipsUnparseableLines.../001/yt-dlp:
        text file busy
**CI: green on run 18244 (re-run).** `check` success, `e2e` success, commit `26251ba`. The **first** attempt of that run failed `check` at the Test step, on `TestYtDlpSearchSkipsUnparseableLines` — which is #146, and unrelated to this branch by construction: the diff contains no Go. The re-run passed with nothing changed. That run did produce the message #146 asked for and it is now on that issue: ``` --- FAIL: TestYtDlpSearchSkipsUnparseableLines (0.00s) provider_ytdlp_test.go:178: Search: yt-dlp failed: search: fork/exec /tmp/TestYtDlpSearchSkipsUnparseableLines.../001/yt-dlp: text file busy ```
logan added 1 commit 2026-09-03 17:04:19 +00:00
Merge remote-tracking branch 'origin/main' into docs/220-skill-check-scope
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m11s
CI / e2e (pull_request) Successful in 10m44s
9432f68c8b
logan added 1 commit 2026-09-03 17:20:25 +00:00
Merge remote-tracking branch 'origin/main' into docs/220-skill-check-scope
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m10s
CI / e2e (pull_request) Successful in 11m6s
b1368bbc7e
logan merged commit 5fa68fcf43 into main 2026-09-03 17:35:40 +00:00
Sign in to join this conversation.