`applySchema` is CREATE ... IF NOT EXISTS and there is no migration
chain, so a *changed* table never migrates: the statement silently
no-ops against the old shape. Two plans had already landed on that, and
neither showed up in a test because a fresh install is perfectly
healthy.
- 014 added `total_tracks` to explore_index and to `indexRowFields`,
the projection every explore read uses, so every search, browse,
artist page and album page failed with "no such column: total_tracks"
on any database that already had a catalog.
- 013 reshaped audio_files, so applySchema could not run at all and the
app did not open.
staleshape.go runs before applySchema and drops what disagrees, so the
create is a create. It parses sql/schemas/ for the expectation rather
than writing the column list down a second time, and it notices a
changed *type* as well as a missing column — 013 moved mbid TEXT to
BLOB, which no ALTER could express and which SQLite will not coerce, so
a query against 16 raw bytes returns no rows rather than an error.
Only Authored tables are exempt. Cache is rebuildable by definition,
Owned is what a rescan rebuilds (plan 013's stated "delete and
rescan"), and a table the schema no longer describes at all goes too --
013 left seven behind plus schema_migrations.
Three things in it are load-bearing, and each was a bug first:
- The parser read `UNIQUE(mbid)` as a column, which made a healthy
catalog look stale. That would have retired it on every launch and
cost every user an artifact download per start.
- The drops are one transaction with defer_foreign_keys. Those legacy
tables reference each other, so any order fails on whichever goes
first; turning foreign keys off instead would suppress
playlist_tracks.audio_file_id's ON DELETE SET NULL and leave entries
pointing at ids a rescan reissues to *different songs*. Nulled
entries are empty; stale ones are wrong, and wrong quietly.
- The order is sorted, so a failure reproduces. Map order is random,
and the foreign-key bug passed its own regression test on two runs in
three until the order was fixed.
Verified against a real pre-013 install: it opens, its 22 playlists
survive, 1,887 linked playlist entries become 0 rather than dangling,
and the legacy tables are swept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
A track credited to more than one artist has exactly one navigable
artist in this app and the rest are punctuation. `primaryArtist()`
string-parses the credit, strips a " feat. " clause and discards the
guest; it deliberately does not split on "&", "with" or "," because
those live inside real artist names.
Measured on a real 26,069-file library plus an 80+80 MusicBrainz
sample: 13% of recordings are multi-artist upstream, while only 0.86%
of files carry any structured multi-artist tag — mp3 carries zero
files with multiple MUSICBRAINZ_ARTISTID across 19,840. Of 1,286 files
saying "feat.", 90% have nothing structured behind it, and a sample of
80 such files was multi-artist in MB 80 times out of 80.
CLAUDE.md justified plan 013's removal of the credit tables with "3
credits of 2,823 listed more than one artist". That measured our own
*writer* — cachedLinkArtist was called once per credit, so a
collaboration could never have been recorded. Dropping the join table
was still right on cost; the evidence for "multi-artist is rare" was
not.
A credit is ordered parts and the credit string is derived from them,
so join phrases are assembly instructions, not disassembly ones.
Nothing here reconstructs a credit by searching a name inside a credit
string: the stored text may come from tags while the parts come from
the catalog, and those disagree for ~1 in 3 multi-artist credits.
Where it comes from, after two dead ends: the canonical dump CI
already streams has no join phrases and no as-credited names, and the
JSON dumps cover 153,691 recordings of ~35M with *zero* overlap
against a real library. So mbdump.tar.bz2 — 7.1 GB, ~13.7 min in
pure-Go bzip2, whose members are alphabetical, which is what lets one
pass resolve an entity's credit without buffering 35M recordings.
- artist_credit_part / artist_credit_ref, multi-artist credits only:
a single-artist credit is already explore_index's own artist_name.
- Column layouts verified against the real 20260815 export;
ErrDumpShape makes a wrong guess a failed build, not a wrong catalog.
- The pass runs on every mode, not just a build. The job picks its mode
from the index's own state, and a complete import means "refresh",
which never enters the importer — so credits could otherwise only
arrive via a rebuild that re-downloads ~205 GB. It reports whether it
populated anything, which is what flips `changed` and republishes.
- The importer asks whether an artifact carries the tables, on the
writer where `core` is attached, so the artifact already published
still imports.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
The index job's /cache volume is a real YJ_HOME that outlives every
run, so plan 013's reshaped audio_files met a database still in the
old shape: `CREATE INDEX ... album_id` against a table without that
column, on every launch. "Delete and rescan" is the squash's answer
and is free everywhere except here, where half the file is the catalog
and deleting it costs ~205GB of downloading.
indexbuild now drops every table datamap does not classify as Cache
before the schema is applied. Nothing scans, plays or authors in that
database, so its non-catalog half is empty by construction and a shape
the schema stopped describing is pure liability; the catalog is never
touched.
TestRetireLibraryTables reproduces the failure symptom-first: build
the real schema, put audio_files back the way the volume had it,
assert the open fails, then assert the repair makes it open with the
catalog row still there.
The v3 migration put application.Get() in backend/events and a
ServiceStartup hook in backend/explore, both of which cmd/indexbuild
reaches. v3's application package is GTK/WebKit bindings on Linux, so
the index-artifact job — a plain golang container with CGO_ENABLED=0,
on the stated grounds that neither command imports the app — stopped
compiling with "undefined: pointer". That job owns the ~205 GB dump
checkpoint, so it is the worst place to learn this.
Both are behind the indexbuild tag now: the one app.Event.Emit lives in
runtime_wails.go, runtime_indexbuild.go answers ErrNoRuntime (what the
app itself returns before Run, so Deliver's callers need no second
path), and explore's ServiceStartup moves to its own tagged file.
TestIndexToolsDoNotImportWails walks `go list -deps -tags indexbuild`
so the claim the workflow makes is checked rather than assumed.
Ships the fresh-start schema cleanup: rebuilt explore catalog index
pipeline (dump import, artifact fetch/build, incremental listen-count
refresh), a new download subsystem (Lidarr/Prowlarr/qBittorrent/SABnzbd/
slskd/yt-dlp providers, staging, reconciliation, wanted list), and the
supporting schema/query/store changes across backend and frontend.
Also includes two smaller follow-ups: bump the central index's
rebuild-after cadence from 90 to 180 days, and remove the Explore
"library only" online/offline toggle entirely (frontend-only, no
backend counterpart) rather than carry unused UI/state.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2Agd9af5hE7qzti2ackiS
Add a central job registry that library scans and search index builds
report into, so background work is visible instead of buried in the
settings page.
- backend/jobs: registry with per-job ring-buffer logs, capability-driven
controls, and one coalesced JobsChanged snapshot at 4Hz
- pause survives restart via a job_state table; a paused scan is adopted
back on launch and skipped by the soft scan
- top-bar indicator, popover, details drawer and a Jobs page replacing
the config page's scan UI; per-library start/stop retained
- scan timing breakdown moves into the job log, Full rescan to the Jobs
page; delete the orphaned library-manager component
Also add cmd/indexbuild and cmd/indexexport so the explore index can be
built once centrally rather than by every install, which today streams
~205GB from the ListenBrainz spark dump on first run. indexbuild picks
build/refresh/rebuild from index state; the Gitea workflow runs it on
push, weekly, or manually and publishes only when content changed.
fresh-install no longer defaults YJ_HOME under /tmp: it is tmpfs on most
distros, and the import needs ~6GB of real disk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
yonluandClaude Opus 5
logan