yonluandClaude Opus 5
e51cb13662
Two guardrails for the 2026-08-17 incident, and one is not about CI. index-artifact.yml's `push` trigger was commented out that day with a note to restore it once the rebuild completed. Restoring it is the bug. A refresh is individually cheap, which is what made the trigger look free; what it actually did was put an unattended job that mutates the only copy of a ~205 GB catalog on the same trigger as an ordinary code change, on a runner with capacity 1. The rule the file now states is the general one -- a job that mutates state which cannot be rebuilt in ten minutes is triggered deliberately -- so the next such job has somewhere to look. The cron and workflow_dispatch lose nothing: indexbuild resumes from its checkpoint either way. Note what no branching or PR gate would have caught here. That change was green on its branch, green on the merge and green on main; the fault existed only against the persistent /cache database, which no fixture reproduces. Code is gated by CI, irreplaceable state by refusing to touch it and by docs/index-cache.md's restore. The other half is the mismatch that started this: two harnesses reading two files. AGENTS.md is a symlink to CLAUDE.md and skill-check asserts the symlink rather than comparing contents, because a copy would satisfy every other check in this repo while silently drifting -- which is the failure being prevented. The same check now scans CLAUDE.md for make targets, which it never did: 27 targets named in the file agents trust most, none of them verified. Coverage goes 19 -> 46. Scanning prose meant the line-start rule needed a fence. "Two green branches do not / make a green merge" wrapped onto a line beginning `make a` and duly failed on a target called `a`. Inside a fence it is code; outside one it is a sentence that broke there, and a check that fails on reflow gets disabled rather than fixed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
106 lines
3.9 KiB
Bash
Executable File
106 lines
3.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Every command in the agent-facing docs is a `make` target on purpose:
|
|
# the Makefile is the source of truth for *how* to invoke something, and
|
|
# the docs only decide *which* and *in what order*. This check keeps
|
|
# that honest — a renamed or deleted target turns into a failing commit
|
|
# rather than into an agent confidently running a command that no longer
|
|
# exists.
|
|
#
|
|
# It checks two things. Usage: scripts/skill-check.sh
|
|
#
|
|
# **Every `make <target>` named in an agent-facing doc exists.** The
|
|
# scanned set is `.pi/` *and* CLAUDE.md, which is the half that was
|
|
# missing: CLAUDE.md names 27 targets and nothing verified one of them,
|
|
# so the file the agents trust most was the file least checked.
|
|
#
|
|
# **AGENTS.md is a symlink to CLAUDE.md.** This repo is worked on by
|
|
# two agent harnesses that read different files by convention — Claude
|
|
# Code reads CLAUDE.md, others read AGENTS.md — and two harnesses
|
|
# reading two descriptions of one project is how they come to hold
|
|
# different beliefs about it. A symlink makes that impossible by
|
|
# construction; a *copy* would pass every other check in this repo while
|
|
# silently drifting, which is exactly the failure being prevented, so
|
|
# the symlink itself is asserted rather than its contents compared.
|
|
set -euo pipefail
|
|
|
|
cd "$(dirname "$0")/.."
|
|
|
|
# The symlink half runs even without .pi/, since it is not about .pi/.
|
|
if [ -e AGENTS.md ] || [ -L AGENTS.md ]; then
|
|
if [ ! -L AGENTS.md ]; then
|
|
echo "skill-check: AGENTS.md is a regular file, not a symlink to CLAUDE.md." >&2
|
|
echo " Two harnesses would read two descriptions of one project." >&2
|
|
echo " Fix: rm AGENTS.md && ln -s CLAUDE.md AGENTS.md" >&2
|
|
exit 1
|
|
fi
|
|
|
|
target="$(readlink AGENTS.md)"
|
|
|
|
if [ "$target" != "CLAUDE.md" ]; then
|
|
echo "skill-check: AGENTS.md points at '$target', expected CLAUDE.md." >&2
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
[ -d .pi ] || exit 0
|
|
|
|
# `make -pq` prints the database including every rule, without running
|
|
# anything. It exits non-zero when a target is out of date, and under
|
|
# `pipefail` that would sink the whole assignment, so swallow it.
|
|
targets="$({ make -pqRr 2>/dev/null || true; } |
|
|
awk '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ {sub(/:.*/, "", $0); print}' |
|
|
sort -u)"
|
|
|
|
# A mention counts only when it is code: backticked (`make ui-test`)
|
|
# anywhere, or at the start of a line **inside a fenced block**. Bare
|
|
# prose is not scanned, because English says things like "a renamed make
|
|
# target".
|
|
#
|
|
# The fence is why this is awk rather than one grep. Line-start alone is
|
|
# not evidence of code in a file that is mostly hard-wrapped prose: the
|
|
# sentence "Two green branches do not / make a green merge" wrapped onto
|
|
# a line beginning `make a`, and the check duly failed on a target called
|
|
# `a`. Inside a fence it is code; outside one it is a sentence that
|
|
# happened to break there, and a check that fails on reflow gets
|
|
# disabled rather than fixed.
|
|
#
|
|
# AGENTS.md is deliberately not in this list: it is a symlink to
|
|
# CLAUDE.md, asserted above, so scanning it would report every failure
|
|
# twice under two names.
|
|
mentioned="$({ find .pi -name '*.md' 2>/dev/null; echo CLAUDE.md; } |
|
|
xargs awk '
|
|
FNR == 1 { fence = 0 }
|
|
/^```/ { fence = !fence; next }
|
|
{
|
|
rest = $0
|
|
while (match(rest, /`make [a-z][a-z0-9-]*/)) {
|
|
print substr(rest, RSTART + 6, RLENGTH - 6)
|
|
rest = substr(rest, RSTART + RLENGTH)
|
|
}
|
|
if (fence && match($0, /^make [a-z][a-z0-9-]*/)) {
|
|
print substr($0, 6, RLENGTH - 5)
|
|
}
|
|
}
|
|
' | sort -u)"
|
|
|
|
missing=""
|
|
|
|
for t in $mentioned; do
|
|
if ! printf '%s\n' "$targets" | grep -qx -- "$t"; then
|
|
missing="$missing $t"
|
|
fi
|
|
done
|
|
|
|
if [ -n "$missing" ]; then
|
|
echo "skill-check: the agent docs name make targets that do not exist:" >&2
|
|
for t in $missing; do
|
|
echo " make $t" >&2
|
|
grep -rln "make $t" .pi CLAUDE.md --include='*.md' | sed 's/^/ /' >&2
|
|
done
|
|
echo "Fix the docs, or restore the target." >&2
|
|
exit 1
|
|
fi
|
|
|
|
echo "skill-check: $(printf '%s\n' "$mentioned" | wc -w) documented make targets, all present"
|