ci(index-artifact): import the exported artifact before publishing it
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m59s
CI / e2e (pull_request) Successful in 12m51s

Nothing should be published until the code that imports it on a user's
machine has imported it here. The exporter and the importer are two
descriptions of one storage format, and every other tier tests the
importer against a *fixture* rather than against the file being shipped —
a second description free to be wrong in the same direction as the code
reading it.

That is how #258 reached everyone: the importer positioned its batch walk
with a Go `string` cursor against this file's 16-byte `mbid` column, and
SQLite neither coerces between TEXT and BLOB nor complains about the
comparison, so the walk merged no rows and never advanced. The fixture
guarding that walk writes the old text encoding, and the only compact
fixture is one row, below the batch size, so the bound query never ran.
Both were green throughout, and no install could finish a first index
build.

`TestImportPublishedArtifact` takes the published file and runs the
client's own path over it — checksum, decompress, merge — and asserts
that what the artifact holds is what the client ends up with: the row
count, the rows carrying a listen count, an FTS index in step with the
table, and one row read back through the app's own MBID conversions. It
skips without `YJ_CORE_INDEX_ARTIFACT`, so an ordinary run pays nothing.

It needs no Wails, which is why the step runs it under the indexbuild
tag: that container has no GTK. Measured on the current artifact, 64.8 MB
compressed: 39 seconds including the decompress.

Verified against the pre-fix comparison behaviour, the step goes red in
about 3 seconds — the strictly-advancing guard fails the import with a
named reason rather than the day-long spin it used to produce.

Refs #258
This commit is contained in:
2026-09-25 11:03:52 -04:00
parent 1e3a490c12
commit 5d9c677cf7
2 changed files with 310 additions and 1 deletions
+44 -1
View File
@@ -68,7 +68,7 @@ jobs:
# claim with a test behind it now (cmd/indexbuild/deps_test.go),
# because the v3 migration quietly broke it and this job was where
# that surfaced.
image: golang:1.25
image: golang:1.26
# This host path must exist on the runner and be listed verbatim in
# act_runner's container.valid_volumes. It holds explore-staging/
# (counts.bin + state.json) and yj.db — the checkpoint that makes
@@ -148,6 +148,49 @@ jobs:
sha256sum /tmp/core-index.db.zst | tee /tmp/core-index.db.zst.sha256
ls -lh /tmp/core-index.db.zst
# Nothing is published until it has been imported by the code that
# imports it on a user's machine. The exporter and the importer are
# two descriptions of one storage format, and every other tier tests
# the importer against a *fixture* rather than against the file being
# shipped — a second description free to be wrong in the same
# direction as the code reading it.
#
# That is how #258 reached everyone: the importer positioned its batch
# walk with a Go `string` cursor against this file's 16-byte `mbid`
# column, and SQLite neither coerces between TEXT and BLOB nor
# complains about the comparison — so the walk merged no rows and
# never advanced, and no install could finish its first index build.
# The fixture guarding that walk writes the old text encoding, and the
# only compact fixture is one row, below the batch size, so the bound
# query never ran. Both were green throughout.
#
# Running it here is also what keeps the failure cheap: the previous
# artifact stays published while this runs, so a failure costs one
# stale catalog rather than an empty one for every install.
#
# `-tags indexbuild` because this container has no GTK and the default
# tag set links the app through Wails. The `--- PASS` grep is not
# decoration — the test skips without the path, and a skip is
# indistinguishable from a pass in a summary line.
- name: Import the exported artifact as a client does
if: steps.maintain.outputs.complete == 'true' && steps.maintain.outputs.changed == 'true'
working-directory: /src
env:
YJ_CORE_INDEX_ARTIFACT: /tmp/core-index.db.zst
run: |
set -eu
log=/tmp/import-check.log
if ! go test -tags indexbuild -count=1 -timeout 30m -v \
-run TestImportPublishedArtifact ./backend/explore/ > "$log" 2>&1;
then
tail -60 "$log"
echo "::error::The artifact does not import; not publishing it."
exit 1
fi
cat "$log"
grep -qF -- 'PASS: TestImportPublishedArtifact' "$log"
echo "::notice::The artifact imports as a client would merge it."
- name: Publish to the Gitea package registry
if: steps.maintain.outputs.complete == 'true' && steps.maintain.outputs.changed == 'true'
run: |