From e51cb13662079b61e1541c678ef69f4930b9bf98 Mon Sep 17 00:00:00 2001 From: Caleb Allen Date: Mon, 17 Aug 2026 15:30:38 -0400 Subject: [PATCH] ci: trigger the catalog job deliberately, pin agent docs to one file Two guardrails for the 2026-08-17 incident, and one is not about CI. index-artifact.yml's `push` trigger was commented out that day with a note to restore it once the rebuild completed. Restoring it is the bug. A refresh is individually cheap, which is what made the trigger look free; what it actually did was put an unattended job that mutates the only copy of a ~205 GB catalog on the same trigger as an ordinary code change, on a runner with capacity 1. The rule the file now states is the general one -- a job that mutates state which cannot be rebuilt in ten minutes is triggered deliberately -- so the next such job has somewhere to look. The cron and workflow_dispatch lose nothing: indexbuild resumes from its checkpoint either way. Note what no branching or PR gate would have caught here. That change was green on its branch, green on the merge and green on main; the fault existed only against the persistent /cache database, which no fixture reproduces. Code is gated by CI, irreplaceable state by refusing to touch it and by docs/index-cache.md's restore. The other half is the mismatch that started this: two harnesses reading two files. AGENTS.md is a symlink to CLAUDE.md and skill-check asserts the symlink rather than comparing contents, because a copy would satisfy every other check in this repo while silently drifting -- which is the failure being prevented. The same check now scans CLAUDE.md for make targets, which it never did: 27 targets named in the file agents trust most, none of them verified. Coverage goes 19 -> 46. Scanning prose meant the line-start rule needed a fence. "Two green branches do not / make a green merge" wrapped onto a line beginning `make a` and duly failed on a target called `a`. Inside a fence it is code; outside one it is a sentence that broke there, and a check that fails on reflow gets disabled rather than fixed. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh --- .gitea/workflows/index-artifact.yml | 42 +++++++------- AGENTS.md | 1 + Makefile | 10 ++-- scripts/skill-check.sh | 85 ++++++++++++++++++++++++----- 4 files changed, 98 insertions(+), 40 deletions(-) create mode 120000 AGENTS.md diff --git a/.gitea/workflows/index-artifact.yml b/.gitea/workflows/index-artifact.yml index 77e0c91..4ae894c 100644 --- a/.gitea/workflows/index-artifact.yml +++ b/.gitea/workflows/index-artifact.yml @@ -7,31 +7,29 @@ name: Search index maintenance # import older than 6mo -> rebuild (re-import from the newest dump) # otherwise -> refresh (fold in new incremental listens) # -# A refresh is cheap and no-ops when nothing new has been published, so -# running it on every push to main is safe. +# **There is deliberately no `push` trigger, and restoring one is a +# decision rather than a cleanup.** A refresh is individually cheap, so +# running it on every push to main looked free; what it actually does is +# put an unattended job that mutates the only copy of a ~205 GB catalog +# on the same trigger as an ordinary code change, on a runner with +# capacity 1. # -# --------------------------------------------------------------------- -# TEMPORARY (2026-08-17): the `push` trigger is off while the catalog -# rebuilds. +# That is not hypothetical. On 2026-08-17 `fix(database): retire a table +# whose shape the schema moved past` landed on main, green — the CI +# database is deliberately in the older encoding, so the stale-shape +# repair judged its `explore_index` stale and dropped it, and this job +# fell back to a full import from the dumps. `fix(database): never +# retire the catalog the index build derives` stops that specific repair +# and cannot undo it. Every push to main then booked another `budget` +# (3h) of the one runner while ordinary CI queued behind it. # -# `fix(database): retire a table whose shape the schema moved past` -# dropped this job's `explore_index` on its first run -- the CI database -# is deliberately in the older encoding, so the repair judged it stale -- -# and the job fell back to a full ~205 GB import from the dumps. -# `fix(database): never retire the catalog the index build derives` -# stops it happening again but cannot undo it. -# -# Until that import reports complete, every push to main books another -# `budget` (3h) of a runner with capacity 1, and ordinary CI queues -# behind it. The weekly cron and workflow_dispatch still resume the -# build, which is all it needs: indexbuild picks up from its checkpoint. -# -# RESTORE the two `push` lines below once a run reports -# `complete=true`. Nothing else here changed. -# --------------------------------------------------------------------- +# So the rule this file is an instance of: **a job that mutates state +# which cannot be rebuilt in ten minutes is triggered deliberately, not +# by a push.** The weekly cron keeps the catalog current, and +# workflow_dispatch resumes or forces a build — indexbuild picks up from +# its checkpoint either way, so nothing is lost by not running on every +# merge. See docs/index-cache.md for the snapshot and the restore. on: - # push: - # branches: [main] schedule: # Weekly update pass. The 6-month rebuild is triggered by the same # command when it notices the import has aged out. diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 0000000..681311e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +CLAUDE.md \ No newline at end of file diff --git a/Makefile b/Makefile index 1b4defc..cd9c499 100644 --- a/Makefile +++ b/Makefile @@ -179,10 +179,12 @@ bindings-check: ## Fail if the generated bindings are stale css-check: ## Fail if a css`` literal was ended early by a backtick in a comment @cd frontend && node scripts/check-css-literals.mjs -# .pi/ documents commands, and a skill that documents a command wrongly -# is worse than no skill: an agent runs it confidently. Every command -# in there is a make target on purpose, so this is checkable. -skill-check: ## Fail if .pi/ documents a make target that does not exist +# .pi/ and CLAUDE.md document commands, and a doc that documents a +# command wrongly is worse than no doc: an agent runs it confidently. +# Every command in them is a make target on purpose, so this is +# checkable. It also asserts AGENTS.md is a symlink to CLAUDE.md, so the +# two harnesses cannot drift onto two descriptions of one project. +skill-check: ## Fail if the agent docs name a missing make target, or AGENTS.md is not a symlink @./scripts/skill-check.sh # Conventional Commits, which CLAUDE.md claimed CI enforced for a long diff --git a/scripts/skill-check.sh b/scripts/skill-check.sh index ccf3c4e..56930a9 100755 --- a/scripts/skill-check.sh +++ b/scripts/skill-check.sh @@ -1,17 +1,48 @@ #!/usr/bin/env bash # -# Every command in .pi/ is a `make` target on purpose: the Makefile is -# the source of truth for *how* to invoke something, and the skill only -# decides *which* and *in what order*. This check keeps that honest — -# a renamed or deleted target turns into a failing commit rather than -# into an agent confidently running a command that no longer exists. +# Every command in the agent-facing docs is a `make` target on purpose: +# the Makefile is the source of truth for *how* to invoke something, and +# the docs only decide *which* and *in what order*. This check keeps +# that honest — a renamed or deleted target turns into a failing commit +# rather than into an agent confidently running a command that no longer +# exists. # -# It extracts every `make ` mentioned under .pi/ and asserts the -# target exists. Usage: scripts/skill-check.sh +# It checks two things. Usage: scripts/skill-check.sh +# +# **Every `make ` named in an agent-facing doc exists.** The +# scanned set is `.pi/` *and* CLAUDE.md, which is the half that was +# missing: CLAUDE.md names 27 targets and nothing verified one of them, +# so the file the agents trust most was the file least checked. +# +# **AGENTS.md is a symlink to CLAUDE.md.** This repo is worked on by +# two agent harnesses that read different files by convention — Claude +# Code reads CLAUDE.md, others read AGENTS.md — and two harnesses +# reading two descriptions of one project is how they come to hold +# different beliefs about it. A symlink makes that impossible by +# construction; a *copy* would pass every other check in this repo while +# silently drifting, which is exactly the failure being prevented, so +# the symlink itself is asserted rather than its contents compared. set -euo pipefail cd "$(dirname "$0")/.." +# The symlink half runs even without .pi/, since it is not about .pi/. +if [ -e AGENTS.md ] || [ -L AGENTS.md ]; then + if [ ! -L AGENTS.md ]; then + echo "skill-check: AGENTS.md is a regular file, not a symlink to CLAUDE.md." >&2 + echo " Two harnesses would read two descriptions of one project." >&2 + echo " Fix: rm AGENTS.md && ln -s CLAUDE.md AGENTS.md" >&2 + exit 1 + fi + + target="$(readlink AGENTS.md)" + + if [ "$target" != "CLAUDE.md" ]; then + echo "skill-check: AGENTS.md points at '$target', expected CLAUDE.md." >&2 + exit 1 + fi +fi + [ -d .pi ] || exit 0 # `make -pq` prints the database including every rule, without running @@ -21,11 +52,37 @@ targets="$({ make -pqRr 2>/dev/null || true; } | awk '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ {sub(/:.*/, "", $0); print}' | sort -u)" -# A mention counts only when it is code: backticked (`make ui-test`) or -# the first thing on a line, as in a fenced block. Bare prose is not -# scanned, because English says things like "a renamed make target". -mentioned="$(grep -rhoE '(`|^)make [a-z][a-z0-9-]*' .pi --include='*.md' | - sed 's/^`//' | awk '{print $2}' | sort -u)" +# A mention counts only when it is code: backticked (`make ui-test`) +# anywhere, or at the start of a line **inside a fenced block**. Bare +# prose is not scanned, because English says things like "a renamed make +# target". +# +# The fence is why this is awk rather than one grep. Line-start alone is +# not evidence of code in a file that is mostly hard-wrapped prose: the +# sentence "Two green branches do not / make a green merge" wrapped onto +# a line beginning `make a`, and the check duly failed on a target called +# `a`. Inside a fence it is code; outside one it is a sentence that +# happened to break there, and a check that fails on reflow gets +# disabled rather than fixed. +# +# AGENTS.md is deliberately not in this list: it is a symlink to +# CLAUDE.md, asserted above, so scanning it would report every failure +# twice under two names. +mentioned="$({ find .pi -name '*.md' 2>/dev/null; echo CLAUDE.md; } | + xargs awk ' + FNR == 1 { fence = 0 } + /^```/ { fence = !fence; next } + { + rest = $0 + while (match(rest, /`make [a-z][a-z0-9-]*/)) { + print substr(rest, RSTART + 6, RLENGTH - 6) + rest = substr(rest, RSTART + RLENGTH) + } + if (fence && match($0, /^make [a-z][a-z0-9-]*/)) { + print substr($0, 6, RLENGTH - 5) + } + } + ' | sort -u)" missing="" @@ -36,10 +93,10 @@ for t in $mentioned; do done if [ -n "$missing" ]; then - echo "skill-check: .pi/ documents make targets that do not exist:" >&2 + echo "skill-check: the agent docs name make targets that do not exist:" >&2 for t in $missing; do echo " make $t" >&2 - grep -rln "make $t" .pi --include='*.md' | sed 's/^/ /' >&2 + grep -rln "make $t" .pi CLAUDE.md --include='*.md' | sed 's/^/ /' >&2 done echo "Fix the docs, or restore the target." >&2 exit 1