ci(skill-check): scan the docs a contributor reads #229

2026-08-28T07:39:00Z

logan commented

2026-08-28 07:39:00 +00:00

What the issue was

scripts/skill-check.sh asserts that every make <target> named in a
doc exists. Its scanned set was .pi/ plus CLAUDE.md — so
CONTRIBUTING.md, which #50 made the document a human contributor
reads a build command out of, named 21 targets that nothing verified.
README.md names none today and is in the set for the same reason.

#220 also names a smaller half: the lefthook pre-commit glob was
{Makefile,.pi/**/*.md} while the script already read CLAUDE.md, so a
commit touching only CLAUDE.md did not fire the hook at all.

What changed

Commit	Issue
`ci(skill-check): scan the docs a contributor reads`	#220

scripts/skill-check.sh — README.md and CONTRIBUTING.md join
the scanned set. The file list is one docs variable used twice, once
to read the mentions out and once to say which file a missing target
came from, because the failure message carried its own copy and a
second list is a second thing to forget. [ -d .pi ] || exit 0 went
with it: gating the whole run on .pi/ would make the human-facing
half conditional on the agent-facing one, and [ -n "$docs" ] is the
same "nothing to scan" exit without the coupling. ls carries
|| true because it exits non-zero when any argument is missing
while still printing the others, and under set -e that sinks the
assignment rather than scanning what is there.
lefthook.yml — the glob is the script's scanned set, AGENTS.md
included, since the script asserts that one is a symlink.
Makefile, CLAUDE.md — the two one-line descriptions of the
target said .pi/ / "the agent docs" and no longer do.

Verification

A widened file list that catches nothing looks identical to one that
works, so every claim below was planted rather than read.

The new surfaces catch. A bad target planted in README.md and in
CONTRIBUTING.md, in both matched forms (backticked inline, and
line-start inside a fence) — 4/4 fail, each naming the right file.
The hole, measured. The same two plants against the pre-change
script: skill-check: 47 documented make targets, all present,
exit 0.
No coverage lost. CLAUDE.md and .pi/skills/…/SKILL.md
re-planted: both still caught, both still named correctly.
No new false positives. Unfenced prose (a renamed make no-such-prose-target is just English here.) still does not match,
which is the rule that stops the check failing on reflow. #220 asked
for a bare non-target make mention to be checked for while here:
there is none in either file, which is what the clean run says.
The hook. A planted CONTRIBUTING.md staged, then
lefthook run pre-commit --commands skill-check: fails under the new
glob, (skip) no matching staged files under the old one. Same for a
CLAUDE.md-only stage. This PR's own commit fired it, on a staged
CLAUDE.md.
Degenerate trees (/tmp, script + Makefile only): only
CONTRIBUTING.md and no .pi/ → catches; no docs at all → exit 0;
clean docs → passes.
make skill-check on the branch: 47 documented make targets, all
present — unchanged, because the set is a union and everything
CONTRIBUTING.md names was already named elsewhere. Coverage moved;
the count could not.

Tiers not run, and why. No Go, no .sql, no .templ, no bound
signature, no frontend/: make lint, make test, make generate,
make bindings, make ui-test and make e2e do not apply and were not
run. Nothing renders differently, so no screenshot tier applies. CI's
check job runs make skill-check itself.

Deliberately not done

Filed #228 rather than fixed here: a `make … ` span the file
hard-wraps across two lines is matched by nothing, and
CONTRIBUTING.md:80 already holds one (make dev-headless, wrapped).
Nothing is broken today — that target is named on one line elsewhere —
but a target named only inside a wrapped span would be missed, and
these two files are the ones that invite the shape. Joining lines before
matching re-opens the failure the fence rule exists for, where a
sentence wrapped onto a line beginning with the word make and a single
letter, and the check failed on a target of that name. That needs a rule
for which joins are safe, which is a different change from widening a
file list. (It bit this PR's journal entry, which said so with
backticks on the first draft and failed the check.)

Closes #220

### What the issue was `scripts/skill-check.sh` asserts that every `make <target>` named in a doc exists. Its scanned set was `.pi/` plus `CLAUDE.md` — so `CONTRIBUTING.md`, which #50 made the document a *human* contributor reads a build command out of, named 21 targets that nothing verified. `README.md` names none today and is in the set for the same reason. #220 also names a smaller half: the lefthook `pre-commit` glob was `{Makefile,.pi/**/*.md}` while the script already read `CLAUDE.md`, so a commit touching only `CLAUDE.md` did not fire the hook at all. ### What changed | Commit | Issue | | --- | --- | | `ci(skill-check): scan the docs a contributor reads` | #220 | - **`scripts/skill-check.sh`** — `README.md` and `CONTRIBUTING.md` join the scanned set. The file list is one `docs` variable used twice, once to read the mentions out and once to say which file a missing target came from, because the failure message carried its own copy and a second list is a second thing to forget. `[ -d .pi ] || exit 0` went with it: gating the whole run on `.pi/` would make the human-facing half conditional on the agent-facing one, and `[ -n "$docs" ]` is the same "nothing to scan" exit without the coupling. `ls` carries `|| true` because it exits non-zero when *any* argument is missing while still printing the others, and under `set -e` that sinks the assignment rather than scanning what is there. - **`lefthook.yml`** — the glob is the script's scanned set, `AGENTS.md` included, since the script asserts that one is a symlink. - **`Makefile`, `CLAUDE.md`** — the two one-line descriptions of the target said `.pi/` / "the agent docs" and no longer do. ### Verification A widened file list that catches nothing looks identical to one that works, so every claim below was planted rather than read. - **The new surfaces catch.** A bad target planted in `README.md` and in `CONTRIBUTING.md`, in both matched forms (backticked inline, and line-start inside a fence) — 4/4 fail, each naming the right file. - **The hole, measured.** The same two plants against the *pre-change* script: `skill-check: 47 documented make targets, all present`, exit 0. - **No coverage lost.** `CLAUDE.md` and `.pi/skills/…/SKILL.md` re-planted: both still caught, both still named correctly. - **No new false positives.** Unfenced prose (`a renamed make no-such-prose-target is just English here.`) still does not match, which is the rule that stops the check failing on reflow. #220 asked for a bare non-target `make` mention to be checked for while here: there is none in either file, which is what the clean run says. - **The hook.** A planted `CONTRIBUTING.md` staged, then `lefthook run pre-commit --commands skill-check`: fails under the new glob, `(skip) no matching staged files` under the old one. Same for a `CLAUDE.md`-only stage. This PR's own commit fired it, on a staged `CLAUDE.md`. - **Degenerate trees** (`/tmp`, script + Makefile only): only `CONTRIBUTING.md` and no `.pi/` → catches; no docs at all → exit 0; clean docs → passes. - `make skill-check` on the branch: **47 documented make targets, all present** — unchanged, because the set is a union and everything `CONTRIBUTING.md` names was already named elsewhere. Coverage moved; the count could not. **Tiers not run, and why.** No Go, no `.sql`, no `.templ`, no bound signature, no `frontend/`: `make lint`, `make test`, `make generate`, `make bindings`, `make ui-test` and `make e2e` do not apply and were not run. Nothing renders differently, so no screenshot tier applies. CI's `check` job runs `make skill-check` itself. ### Deliberately not done **Filed #228** rather than fixed here: a `` `make … ` `` span the file hard-wraps across two lines is matched by nothing, and `CONTRIBUTING.md:80` already holds one (`make dev-headless`, wrapped). Nothing is broken today — that target is named on one line elsewhere — but a target named *only* inside a wrapped span would be missed, and these two files are the ones that invite the shape. Joining lines before matching re-opens the failure the fence rule exists for, where a sentence wrapped onto a line beginning with the word make and a single letter, and the check failed on a target of that name. That needs a rule for which joins are safe, which is a different change from widening a file list. (It bit this PR's journal entry, which said so with backticks on the first draft and failed the check.) Closes #220

logan added 1 commit 2026-08-28 07:39:00 +00:00

ci(skill-check): scan the docs a contributor reads

CI / check (push) Skipped

CI / e2e (push) Skipped

CI / check (pull_request) Successful in 3m0s

Details

CI / e2e (pull_request) Successful in 10m18s

Details

26251badda

The check asserts that every make target named in a doc exists, and its
scanned set was .pi/ plus CLAUDE.md.  Since #50, CONTRIBUTING.md is the
document a *human* goes to for a build command, and it names 21 targets
that nothing verified; README.md names none today and is in for the same
reason.  The script's own header sentence is the argument — a renamed
target sends a person off the same cliff it sends an agent off.

The file list is now one `docs` variable used twice, because the failure
message carried a second copy of it and a second list is a second thing
to forget.  The `[ -d .pi ]` guard went with it: gating the whole run on
.pi/ would make the human-facing half conditional on the agent-facing
one, and an empty list is the same "nothing to scan" exit without the
coupling.

The lefthook glob is that scanned set now rather than
{Makefile,.pi/**/*.md} — #220's smaller half, and it did not fire on
CLAUDE.md either, which the script had read for months.

Verified by planting a bad target rather than by reading the diff: both
matched forms in each of the four scanned surfaces, each naming the
right file; the same two plants pass on the pre-change script; unfenced
prose still does not match; and the hook fires on a staged
CONTRIBUTING.md under the new glob where the old one skipped it.  The
count is unchanged at 47 — the set is a union — so coverage is the only
thing that moved.

Closes #220

logan commented

2026-08-28 07:56:02 +00:00

CI: green on run 18244 (re-run). check success, e2e success,
commit 26251ba.

The first attempt of that run failed check at the Test step, on
TestYtDlpSearchSkipsUnparseableLines — which is #146, and unrelated to
this branch by construction: the diff contains no Go. The re-run passed
with nothing changed. That run did produce the message #146 asked for
and it is now on that issue:

--- FAIL: TestYtDlpSearchSkipsUnparseableLines (0.00s)
    provider_ytdlp_test.go:178: Search: yt-dlp failed: search:
        fork/exec /tmp/TestYtDlpSearchSkipsUnparseableLines.../001/yt-dlp:
        text file busy

**CI: green on run 18244 (re-run).** `check` success, `e2e` success, commit `26251ba`. The **first** attempt of that run failed `check` at the Test step, on `TestYtDlpSearchSkipsUnparseableLines` — which is #146, and unrelated to this branch by construction: the diff contains no Go. The re-run passed with nothing changed. That run did produce the message #146 asked for and it is now on that issue: ``` --- FAIL: TestYtDlpSearchSkipsUnparseableLines (0.00s) provider_ytdlp_test.go:178: Search: yt-dlp failed: search: fork/exec /tmp/TestYtDlpSearchSkipsUnparseableLines.../001/yt-dlp: text file busy ```

logan referenced this pull request

2026-08-28 07:56:03 +00:00

skill-check does not scan README.md or CONTRIBUTING.md #220

All checks were successful

CI / check (push) Skipped

Required

CI / e2e (push) Skipped

Required

CI / check (pull_request) Successful in 3m0s

Required

Details

CI / e2e (pull_request) Successful in 10m18s

Required

Details

You are not authorized to merge this pull request.

This pull request can be merged automatically.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin docs/220-skill-check-scope:docs/220-skill-check-scope

git checkout docs/220-skill-check-scope

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: yonlu/yellowjacket#229