feat: data lifecycle rewrite, download clients, wanted list, and central catalog index

Ships the fresh-start schema cleanup: rebuilt explore catalog index
pipeline (dump import, artifact fetch/build, incremental listen-count
refresh), a new download subsystem (Lidarr/Prowlarr/qBittorrent/SABnzbd/
slskd/yt-dlp providers, staging, reconciliation, wanted list), and the
supporting schema/query/store changes across backend and frontend.

Also includes two smaller follow-ups: bump the central index's
rebuild-after cadence from 90 to 180 days, and remove the Explore
"library only" online/offline toggle entirely (frontend-only, no
backend counterpart) rather than carry unused UI/state.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2Agd9af5hE7qzti2ackiS

.planning/NOTES.md

@@ -0,0 +1,140 @@
+# Notes
+
+Gotchas, measured facts, and things already considered and rejected.
+Measurements carry the date they were taken — several of these are
+properties of someone else's server and can change.
+
+## MetaBrainz caps a client at ~2 MB/s (measured 2026-07-29)
+
+`data.metabrainz.org` serves a single client at roughly 2.1 MB/s, and
+**concurrency does not help**: one Range stream and four concurrent
+lanes delivered 32 MB at 2,111,195 B/s and 2,209,000 B/s respectively,
+while the same machine pulled 66.9 MB/s from a CDN. One of the four
+lanes starved to 0.5 MB/s. The lanes divide a fixed cap; they do not
+raise it.
+
+Consequences:
+
+- No client-side concurrency change will speed up a dump download.
+  Pushing harder earns 503s (the reason `dumpLanes` is 4).
+- Stage 1 of a full import costs ~11.8 h at best (89 GB after column
+  projection). Before projection it was 205 GB — about 27 h.
+
+This is the entire reason the catalog is built centrally and shipped as
+an artifact rather than derived per install.
+
+## Further stage-1 reductions, not yet taken
+
+Both are CI-side options; neither is safe as a silent client default
+because each changes *what gets counted*.
+
+- **Project `recording_mbid` only** (24.1% of row-group bytes instead of
+  43.4%): ~49 GB, ~6.5 h. `canonical_musicbrainz_data.csv` already
+  carries `recording_mbid`, `release_mbid`, `artist_mbids` and
+  `release_group_mbid`, so release/RG/artist counts can be rolled up
+  locally. Cost: listens with no recording MBID are dropped, and artist
+  totals become "sum of their recordings" rather than direct attribution.
+- **Stride-sample members** (1-in-4): ~12 GB, ~1.6 h. The dump is flat
+  numbered members (`0.parquet`, …). Sampling is viable because the
+  counts only feed a *ranking* for a top-N cut. Must be a stride, never
+  a prefix — if members are time-ordered a prefix biases hard toward one
+  era.
+
+## Incremental dump retention is 30 days (measured 2026-07-29)
+
+The incremental directory held 30 dumps (series 2579–2610), and full
+dumps land roughly monthly. An artifact older than ~30 days cannot be
+topped up: the dailies bridging the gap are gone. That is a permanent
+undercount of that window, not corruption — but it pins the artifact
+republish cadence at monthly.
+
+## Anonymous package download is UNVERIFIED
+
+The client fetches the artifact from a fixed `latest` URL because Gitea's
+package *listing* API requires a token while a plain file GET appears not
+to — a probe of the not-yet-published artifact returned 404 rather than
+401. **That is suggestive, not proof.** No artifact has been published
+yet to test against. Confirm before relying on it.
+
+Also worth deciding deliberately: every install pulling from a personal
+Gitea makes its bandwidth and uptime a user-facing dependency.
+
+## No migration chain
+
+`applySchema` creates the whole schema from `sql/schemas/*.sql` on every
+open; all DDL is `IF NOT EXISTS`. A database written by an older build is
+not supported and there is no upgrade path by design.
+
+Two things this replaced, worth not reintroducing:
+
+- The 48-step chain was ~3,700 of `database.go`'s 4,061 lines, plus
+  helpers that existed only to serve it (`backupDatabase`,
+  `readLibraryDirFromTOML`, `isDuplicateColumnErr`, …).
+- `sql/schemas/` had drifted badly from the real schema — it still
+  described a `genre_recordings` table that migrations had renamed, and
+  omitted `explore_index`, `http_cache`, `artist_images`,
+  `similar_artist_map`, `release_to_rg`, `lyrics_index` and
+  `artist_metadata` entirely. sqlc reads that directory, so it had been
+  generating against a stale schema and silently missed columns such as
+  `audio_files.modified_at`.
+
+**When regenerating schema files from a live database, remember the seed
+rows.** `file_types` (the four supported extensions), `player_state` and
+`queue` each carry `INSERT OR IGNORE` rows that `sqlite_master` does not
+contain. Dropping them breaks every audio-file foreign key.
+
+## ANALYZE runs after the catalog merge, not at schema creation
+
+The old migration 45 ran `ANALYZE` once. With the migration chain gone
+there is no equivalent moment — an empty database has nothing to measure
+— so it runs at the end of the artifact import instead
+(`SearchIndex.analyzeIndex`). Without current statistics the planner
+mis-estimates the partial expression indexes on `explore_index`
+(`idx_explore_title_lower`, `idx_explore_artist_lower`) and scans a
+million rows for queries that should seek.
+
+If another path ever populates the catalog, it needs the same call.
+
+## Writers, not readers, are responsible for name quality
+
+`resolveArtistName` falls back to returning the artist MBID when it
+cannot find a name. That is fine for a one-off render but must never be
+persisted — an MBID stored as a title is unsearchable and shows as a
+UUID in the UI.
+
+This used to be defended at every read (`title != mbid` predicates) and
+in the upsert's conflict rules. Those defenses are gone; `AddFromCache`
+now refuses to write a name equal to the MBID and lets the upsert's
+"non-empty wins" rule fill it in when a real name arrives.
+`TestAddFromCacheNeverStoresMBIDAsName` guards this.
+
+## Attached databases are invisible to the read pool
+
+`database.DB` holds two handles: a single-writer connection and a
+separate query-only pool. `ATTACH` binds to one connection, so anything
+touching an attached database must use `ExecContext`/`QueryRowWriter`
+(the writer) — `QueryContext` routes to the pool, where the attachment
+does not exist and the query fails with "no such table".
+
+## FTS triggers are defined in Go, not in the schema
+
+`explore_index`'s three FTS sync triggers live in
+`exploreIndexFTSTriggers` in `database.go` rather than in
+`sql/schemas/explore_index.sql`, because the bulk-load path drops and
+recreates them (`SuspendExploreIndexFTS`). Defining them in both places
+would be two copies free to drift.
+
+Bulk loads must suspend them: measured on a real import, assembly runs at
+~31 rows/s with the triggers attached and ~4,700 rows/s without.
+
+## Explore "library only" toggle was removed (2026-08-06)
+
+The Explore UI used to have a "library only" mode toggle
+(`frontend/src/store/explore-settings.ts`, `explore:libraryOnly` in
+localStorage) that filtered the Explore UI to owned content only. It was
+removed outright — the app now always shows full (network-enriched)
+Explore data. If offline/library-only mode is wanted again, it should be
+built from scratch rather than restored; the old implementation gated
+several component code paths in ad hoc ways. A separate
+`deep_catalog_enabled` backend flag briefly existed for the same idea and
+was removed earlier, when the dump importer left the app binary.

.planning/plans/completed/001-ship-core-index.md

@@ -1,8 +1,27 @@
 # 001 — Ship a prebuilt "core" explore index
 
-**Status:** pending
-**Branch:** wip
+**Status:** complete
+**Branch:** cleanup/fresh-start-schema
 **Created:** 2026-07-25
+**Completed:** 2026-07-30
+
+## Outcome
+
+A fresh install downloads a 70.6 MB artifact and merges 1,076,133 rows
+in ~43 s, instead of streaming 205 GB over ~27 h. The dump importer that
+produces the artifact left the app binary entirely — it is behind the
+`indexbuild` build tag and runs only in CI.
+
+Phase 5 landed differently than planned: rather than a user-facing
+setting gating the deep import, the deep import is simply not in the
+app. `deep_catalog_enabled` existed briefly and was removed with it.
+
+Two things remain unverified or undone, both recorded in
+`.planning/NOTES.md`: anonymous package download on git.ljones.me has
+not been confirmed against a real published artifact, and installs whose
+index was built by older code (no `listens_applied_series`) have no
+rescue path — though with no migration chain, those databases are now
+unsupported anyway.
 
 ## Problem
 
@@ -250,17 +269,96 @@ egress if it is self-hosted on a home connection.
    columns dropped, metadata stamped, vacuumed. Verified against a
    synthetic index: no personal columns leak, no orphaned rows, caps
    respected.
-3. ⬜ **One real build.** Run `indexbuild` against a persistent volume
-   until it converges. This yields the first genuine dump-built index and
-   with it true row counts, on-disk size, real FTS share, and
-   `release_to_rg` size. **Every tier number above is still an
-   extrapolation from synthetic rows until this exists.**
-4. ⬜ **Import path.** First-run download + attach + upsert, with checksum
-   verification, resumability, and clean degradation on failure. Report
-   it as a job in the Jobs panel — the plumbing for that already exists.
-5. ⬜ **Gate the dump build.** Add the setting that makes the full import
-   opt-in, so a shipped core index isn't immediately followed by the
-   multi-GB download it was meant to replace.
+3. ✅ **One real build.** Superseded by a real dump-built index that
+   already existed on the dev machine (`dump_import_done` 2026-07-17).
+   Measured 2026-07-29 — these replace every extrapolation above:
+
+   | | rows | on disk |
+   |---|---|---|
+   | `explore_index` | 2,052,168 (227,359 artists / 400,675 RGs / 1,424,134 recordings) | 383 MB |
+   | its indexes | | 395 MB |
+   | FTS | | 80 MB |
+
+   187 B/row for the shippable table, 418 B/row all-in — so the ~900 MB
+   full-budget estimate was right. Two real exports:
+
+   | tier | rows | artifact | zstd -19 |
+   |---|---|---|---|
+   | 50K artists (default) | 1,076,133 | 191.5 MB | **70.6 MB** |
+   | 25K artists / 10 RG / 20 rec | 620,973 | 110.6 MB | **37.8 MB** |
+
+   `release_to_rg` was empty in that index — it predates the code that
+   persists it — so its size is still unmeasured.
+4. ✅ **Import path.** `backend/explore/artifactfetch.go` (download,
+   Range-resume, sha256, zstd) and `artifactimport.go` (validate, ATTACH,
+   batched merge, FTS rebuild, meta stamping). Reported in the Jobs panel
+   under its own two stages. Measured end to end on the real 50K-artist
+   artifact against a disk-backed DB: **1,076,133 rows merged in 43.2s**
+   (24,900 rows/s), yielding a 455 MB `yj.db`, FTS populated and
+   searchable. In-memory the same merge runs in 28.3s.
+5. ✅ **Gate the dump build.** `deep_catalog_enabled` in
+   `explore_index_meta` (beside `index_build_paused` — it is build state,
+   read at one decision point). Off by default; exposed as
+   `DeepCatalogEnabled` / `SetDeepCatalogEnabled` on the explore Service.
+   An interrupted dump import resumes regardless of the setting, so the
+   gate never discards a checkpoint that already cost hours.
+
+## Measured 2026-07-29: why the client cannot fix this itself
+
+`data.metabrainz.org` caps a client at ~2.1 MB/s. One Range stream and
+four concurrent Range lanes both delivered 32 MB at the same aggregate
+rate (2,111,195 B/s vs 2,209,000 B/s) while the same machine pulled
+66.9 MB/s from a CDN. **Parallelism buys nothing** — the four lanes just
+divide the same cap, and one of them starved to 0.5 MB/s.
+
+So stage 1 costs, unavoidably:
+
+| | bytes | wall clock |
+|---|---|---|
+| Whole tar (what shipped before column projection) | 205 GB | ~27 h |
+| Column projection, 3 columns (43.4%) | 89 GB | ~11.8 h |
+| `recording_mbid` only (24.1%), rolled up via canonical | 49 GB | ~6.5 h |
+| + 1-in-4 member stride sample | 12 GB | ~1.6 h |
+
+The last two are CI-side options, not client defaults: recording-only
+drops listens carrying no recording MBID and re-derives artist totals as
+a sum over recordings, and sampling trades exact counts for a ranking.
+Both are only safe because the selection they feed is a top-N cut.
+
+## Distribution: the "latest" version trick
+
+The client cannot enumerate package versions — Gitea's package listing
+API requires a token, while an anonymous file GET does not (a probe of a
+non-existent artifact returns 404, not 401). So `index-artifact.yml`
+publishes each artifact twice: under a dated version for history, and
+under a fixed `latest` version that the client fetches from a
+predictable URL. Generic packages reject overwriting an existing
+filename, so `latest` is DELETEd before each rewrite.
+
+⚠️ **Unverified:** that anonymous package *download* is actually enabled
+on git.ljones.me. The 404-vs-401 probe is suggestive, not proof — no
+artifact has been published yet to test against. Confirm before relying
+on it, and note that every install pulling from a personal Gitea makes
+its bandwidth and uptime a user-facing dependency.
+
+## Incremental retention bounds artifact staleness
+
+The incremental dump directory holds 30 dumps (series 2579–2610 as of
+2026-07-29) and full dumps land roughly monthly. An artifact older than
+~30 days therefore cannot be topped up: the dailies bridging the gap are
+gone. That is a permanent undercount of that window's listens, not
+corruption — but it pins the republish cadence at monthly.
+
+## Upgrade path for indexes built by older code
+
+The dev machine's index has `dump_import_done` set but **no**
+`listens_applied_series` and an empty `release_to_rg`, because it was
+built before the code that writes them. That combination is a dead end:
+`RefreshListenCounts` bails with "no baseline series recorded", and
+`runDumpBuild` short-circuits on the done marker, so popularity can
+never update again. Current code writes both, so this affects only
+pre-existing installs — but the artifact import is the natural place to
+rescue them, since merging one stamps a fresh baseline series.
 6. ✅ **CI wiring.** `.gitea/workflows/index-artifact.yml` — push +
    weekly cron + manual, concurrency-guarded, publishes only when
    `complete && changed` so identical artifacts don't accumulate.

.planning/plans/completed/002-data-lifecycle-architecture.md

@@ -0,0 +1,155 @@
+# 002 — Data lifecycle architecture
+
+**Status:** completed (first tranche); follow-ups tracked below
+**Branch:** main
+**Created:** 2026-07-26
+
+## Problem
+
+An audit of asset and row cleanup found five leaks, four of which shared
+one root cause: **deletion logic was hand-written per call site and lived
+far from the thing being deleted.** `RemoveLibrary` knew about ten tables
+because someone enumerated them once; migration 32 added an eleventh and
+nothing noticed. Files written by `explore` had no cleanup counterpart
+anywhere. A function that evicted expired cache rows was written and
+never called.
+
+Findings, in severity order:
+
+1. **`RemoveLibrary` was broken for any scanned library.** `tagging_items`
+   holds `FOREIGN KEY(library_id) REFERENCES libraries(id)` with no
+   `ON DELETE` clause and was never cleared, so `DELETE FROM libraries`
+   failed with `FOREIGN KEY constraint failed (787)` and rolled back the
+   whole removal. Every scanned library has `tagging_items` rows (the
+   scan upserts one per album folder), so this fired on essentially every
+   real removal. `RemoveLibrary` had zero test coverage.
+2. **Artist images were never deleted by anything.** No `os.Remove` in
+   `explore`, no `DELETE FROM artist_images` in the codebase. Unbounded
+   in the number of artists ever browsed in Explore, most of whom are not
+   in the library.
+3. **Cover art size variants leaked on removal.** Only the base
+   `cover_art.file_path` was unlinked; the `_sm/_md/_lg` files beside it
+   are derived filenames, not rows, so three files per cover survived.
+4. **`http_cache` was never pruned.** `Cache.Evict()` existed with no
+   callers. Reads filter on `expires_at`, so expired rows were inert but
+   accumulated for the life of the install.
+5. **Cover-art proxy cache was never pruned.** No eviction, no size cap.
+
+## Approach
+
+Rather than patch five holes, classify the data so the *class* of bug
+becomes hard to write. Everything persisted falls on two axes —
+regenerability and cost of regeneration — which collapse to four kinds:
+
+| Kind | Regenerable? | Deletion policy |
+|---|---|---|
+| **Owned** — projection of the user's files | Yes, by rescan | Follows the files |
+| **Authored** — user-created, no other copy | **No** | Explicit user action only |
+| **Derived** — computed from owned | Yes, cheaply | Free; must never block owned deletion |
+| **Cache** — network or dump sourced | Yes, expensively | TTL/age eviction, never cascade |
+
+The classification is not just vocabulary — it produces the right fix for
+each finding. Finding 1 is derived data acting as a referential parent of
+owned data, which the taxonomy makes categorically illegal. Finding 2 is
+cache data that never needed owner-linked cleanup at all; it wants age
+eviction. Finding 3 is derived data that must be swept against a live set
+rather than tracked individually.
+
+A Go interface was considered and rejected: the only polymorphic consumer
+is the janitor, the substrates have nothing in common (SQL rows, an FTS
+virtual table, a view, three directories of JPEGs, a 900 MB index), and
+provenance is a static fact better enforced by package boundaries than by
+methods an implementation may lie about. A declarative catalog gets the
+same benefit for a tenth of the cost.
+
+## What shipped
+
+**`backend/datamap`** — the catalog. Every table, view, and asset
+directory declared with its `Kind`, its `Lifetime` (`cascade`, `set-null`,
+`swept`, `retained`), and a note explaining the classification. Plain data
+with no service dependencies, so tests can assert it against a live
+schema. FTS5 shadow tables resolve to their parent.
+
+Tests that give it teeth (`backend/datamap/datamap_test.go`):
+
+- `TestCatalogCoversSchema` — every table in `sqlite_master` is claimed by
+  exactly one entry. **A new table fails the build until somebody states
+  what it is and how it dies.**
+- `TestCatalogHasNoStaleEntries` — the reverse, catching drift.
+- `TestNoActionForeignKeysAreDeclaredSwept` — a `NO ACTION` foreign key
+  blocks its parent's deletion, so its table must declare `swept`. This is
+  the exact shape of finding 1, now caught at CI time.
+- `TestLifetimesMatchSchema` — declared cascade/set-null must match what
+  SQLite actually enforces.
+- `TestAuthoredCascadesAreDeliberate` — authored data is unrecoverable, so
+  a cascade onto it needs an explicit exemption.
+
+**`backend/maintenance`** — the janitor. A registry of named jobs with
+per-job minimum intervals, run at startup-idle and on a 6h tick. Policies
+follow the taxonomy: derived data sweeps against a live set, cache data
+ages out. Registered in one place (`app.go: startJanitor`) so the full set
+of janitorial work is a single visible list.
+
+Jobs: `http-cache-evict` (6h), `covers-sweep` (24h, live set from
+`cover_art` expanded via `CoverArtFileSet`), `artist-images-sweep` (24h,
+keeps art for library artists indefinitely, evicts browsed-artist art
+after 90d), `cover-art-proxy-sweep` (24h, 30d age eviction).
+
+The covers sweep refuses to act on an empty live set — that means the
+query failed to see the table, not that every cover is garbage.
+
+**Leak tests** (`backend/library/leak_test.go`) — driven by the catalog
+rather than a hardcoded list, so new tables are covered the moment they
+are catalogued:
+
+- `TestRemoveLibraryLeavesNoOwnedOrDerivedRows` — removing the only
+  library leaves no owned or derived rows, except those in
+  `staleTolerated` with a written reason.
+- `TestRemoveLibraryPreservesAuthoredData` — authored data survives.
+- `TestSweptTablesAreActuallySwept` — a table declaring `swept` that
+  nothing sweeps is caught.
+
+All three were verified to fail when the finding-1 fix is reverted.
+
+**Fixes** — `tagging_items` cleared inside the removal transaction
+(`crud.go` step 17); `CoverArtFileSet` expands originals to variants and
+the legacy `_thumb` name; `Cache.Evict` logic moved into a registered job.
+
+**Incidental:** `Library.emit` — `runtime.EventsEmit` calls `log.Fatalf`
+on a context without a Wails runtime, which killed the test binary and
+made the whole package untestable. All ten emits in the package now route
+through a nil-safe helper. This also removes a real crash risk for
+background workers that outlive their context.
+
+## Follow-ups
+
+**`audio_files` is a mixed-kind table.** `play_count`, `last_played`, and
+`tag_status` are *authored* data living in an *owned* table. Orphan
+cleanup treats the whole row as regenerable, which is why renaming a file
+destroys its play count — the row is deleted and re-imported fresh. This
+is the strongest argument for splitting authored per-track state into its
+own table keyed by something more stable than a path. Related: an
+audio-stream content hash (excluding tag blocks, so it survives
+retagging) would let a rename be recognised as the same file. Deliberately
+out of scope here; it is a schema change plus a rename-detection pass, not
+a cleanup fix.
+
+**Cascade adoption.** Fourteen of nineteen foreign keys are `NO ACTION`.
+Converting them to `CASCADE` would delete a lot of hand-written orphan
+sweeps, but SQLite cannot add `ON DELETE` via `ALTER TABLE` — each needs
+the 12-step table rebuild. Note the ordering constraint: cascades delete
+rows silently, so any code that collects file paths *before* deleting rows
+(as `RemoveLibrary` does for cover art) breaks under cascade. Mark-and-
+sweep must land first; the two compose, cascade plus path-collection does
+not.
+
+**Consolidate the ten orphan sweeps.** `DELETE ... WHERE id NOT IN (...)`
+appears ten times across `crud.go`, `dbsync.go`, `smartplaylist.go`, and
+`database.go`. One shared `sweepOrphans(tx)` would shrink the surface where
+a new table can be forgotten. Worth doing opportunistically rather than as
+a big-bang refactor.
+
+**Storage settings pane.** The catalog knows every table and directory and
+its kind; the janitor already computes bytes freed. A settings pane showing
+per-kind disk usage with "clear cache" and "rebuild derived data" buttons
+is now mostly a UI job.

.planning/plans/completed/003-download-clients.md

@@ -0,0 +1,279 @@
+# 003 — Download clients
+
+**Status:** implemented (v1); follow-ups tracked below
+**Branch:** main
+**Created:** 2026-07-27
+
+## Problem
+
+YellowJacket can find music (`explore`), identify it (`autotag`), and
+manage it (`library`) — but it can't acquire it. The one gap between
+"you're missing this album" and "you own this album" is filled today by
+the user alt-tabbing to some other tool.
+
+The naive fix is an HTTP client for Soulseek and a shell-out to yt-dlp.
+That produces two bespoke code paths with duplicated queueing, retry,
+staging and import logic, and a third service means a third copy. The
+services users want to connect are also not the same *kind* of thing —
+some search, some transfer bytes, some are entire automation systems we
+delegate to — so a single `DownloadClient` interface would be a lie that
+every adapter partially implements.
+
+## The role decomposition
+
+Every candidate integration fills one or two of three roles:
+
+| Service | Searches | Transports | Delegates |
+|---|---|---|---|
+| slskd (Soulseek) | ✅ | ✅ | |
+| yt-dlp | ✅ | ✅ | |
+| Lidarr | | | ✅ |
+| Prowlarr | ✅ | | |
+| qBittorrent / Transmission | | ✅ | |
+| SABnzbd / NZBGet | | ✅ | |
+
+So: three small interfaces, not one big one. A provider implements
+whichever it supports and declares that in a capability struct, the same
+way `jobs.Caps` lets the frontend render controls without switching on
+`Kind`.
+
+```go
+// Searcher turns a request into ranked candidates.
+type Searcher interface {
+    Search(ctx context.Context, req Request) ([]Candidate, error)
+}
+
+// Transporter moves a candidate's bytes to a local staging directory.
+type Transporter interface {
+    Grab(ctx context.Context, c Candidate, dst string, p ProgressFunc) (Result, error)
+    Cancel(ctx context.Context, grabID string) error
+}
+
+// Delegator hands the whole request to an external manager and
+// reports back when files land.
+type Delegator interface {
+    Request(ctx context.Context, req Request) (string, error)
+    Poll(ctx context.Context, externalID string) (DelegateStatus, error)
+}
+```
+
+A `Provider` is the registry entry: identity, config, health check, caps,
+plus whichever of the three it satisfies. Search-only providers
+(Prowlarr) are paired with a transport at grab time by protocol match
+(`torrent` → qBittorrent, `usenet` → SABnzbd); providers that do both
+are self-pairing.
+
+## v1 decisions (settled)
+
+- **On-demand only.** User-initiated "find this album" from an Explore
+  artist/album page or a missing-album row. No wanted list, no artist
+  monitoring, no quality-cutoff upgrades. The queue and pipeline built
+  here are exactly what monitoring would later sit on top of — see
+  Deferred.
+- **Soulseek via slskd's REST API**, not a native protocol client. Same
+  adapter shape as everything else, no wire protocol, no credentials in
+  our process, fully testable against an `httptest` server. A native
+  provider can slot in behind `Searcher`/`Transporter` later with no
+  pipeline changes.
+- **Stage → autotag → import.** Downloads land in a staging directory,
+  are matched against the intended release with the existing `autotag`
+  scorer, tagged, then moved into the library and scanned. Never write
+  into the library root directly.
+- **All four provider families in v1**, sequenced so each phase proves a
+  different role shape (see Phases).
+
+## Pipeline
+
+```
+Request (MBID-anchored where possible)
+  └─> fan-out Search across enabled providers (per-provider timeout)
+       └─> merge + rank Candidates
+            └─> user picks (or auto-pick above confidence threshold)
+                 └─> Grab into staging/<request-id>/
+                      └─> verify (audio decodes, expected track count)
+                           └─> autotag against the intended release
+                                └─> tagwriter writes tags
+                                     └─> move into library layout
+                                          └─> targeted incremental scan
+```
+
+The `Request` should carry a release-group or release MBID whenever the
+user started from an Explore page, because that anchor is what makes the
+autotag step reliable instead of a second guess. Free-text requests are
+supported but flagged lower-confidence, and never auto-pick.
+
+Staging lives under the user data dir, not the library. Partial grabs are
+resumable where the provider supports it and swept on startup where it
+doesn't.
+
+## Candidate ranking
+
+Two independent scores, kept separate:
+
+1. **Match confidence** — does this candidate contain the release the
+   user asked for? Reuse `autotag`'s distance/alignment machinery on the
+   candidate's *filenames* (Soulseek gives paths, not tags), against the
+   expected tracklist from the explore index.
+2. **Source quality** — format (FLAC > V0 > 320 > lower), bitrate,
+   completeness (file count vs. expected track count), source health
+   (slskd queue length and upload slots; seeders for torrents), and a
+   user-set per-provider priority.
+
+Ranking presents both, because they trade off — a perfectly-matched
+128kbps rip should lose to a well-matched FLAC, and the user should be
+able to see why. Reusing `autotag.ScoreBreakdown`'s "explain the ranking"
+pattern here is deliberate.
+
+## Persistence
+
+New tables (migration TBD, next free number):
+
+- `download_providers` — id, kind, name, enabled, priority, config blob
+  (JSON), `created_at`. Non-secret config only.
+- `download_requests` — id, source (`explore-album`, `explore-artist`,
+  `manual`), release_mbid / release_group_mbid, free-text query,
+  requested_at, state, resolved_download_id.
+- `download_items` — one row per grab attempt: request_id, provider_id,
+  candidate JSON, state, bytes/total, staging path, error, timestamps.
+
+**Secrets** (slskd API key, Lidarr/Prowlarr API keys, qBittorrent
+password) do not go in the TOML config or the DB in plaintext. Use the OS
+keyring where available with a clearly-labelled encrypted-file fallback,
+and never log a config value from a provider's secret field. Open
+question below on the exact library.
+
+## Jobs integration
+
+Add `jobs.KindDownload`. One job per request (not per file), with
+`Stages` for search → grab → import so the existing detail panel renders
+the pipeline for free. `Caps{Cancellable: true}`; pausable only for
+providers that can resume. Per-provider concurrency caps and a global
+cap, both configurable — hammering a Soulseek peer with eight parallel
+transfers gets you queued or banned.
+
+## Frontend
+
+- New `download-providers` section in `config-page` (HTMX + templ, same
+  as existing settings) for provider CRUD, test-connection, priority.
+- New `download-picker` Lit component: the ranked-candidate dialog,
+  invoked from Explore album/artist pages and from a missing-album row.
+- `download-store.ts` subscribing to the existing `JobsChanged` event —
+  no new event channel needed for progress.
+
+## Phases
+
+Each phase is independently shippable and proves a distinct role shape.
+
+1. **Core.** Interfaces, registry, `Request`/`Candidate`/`Result` types,
+   staging dir, ranking, the stage→autotag→import tail, jobs wiring,
+   schema, secret storage. Ships with a fake provider and full test
+   coverage of the pipeline. No real network.
+2. **yt-dlp.** Subprocess provider: search + transport, no server for the
+   user to run, so it's the fastest path to an end-to-end working
+   feature. Proves the local-subprocess shape (binary discovery,
+   version checks, stdout progress parsing, sandboxing the arg list).
+3. **slskd.** Remote search + transport over REST. Proves the remote
+   HTTP shape and is the highest-value source. This is where filename-
+   based match confidence earns its keep.
+4. **Lidarr.** Delegate. Proves the fire-and-poll shape, where we don't
+   own the transfer and the "import" step is really "detect what Lidarr
+   already imported and reconcile".
+5. **Prowlarr + qBittorrent/SABnzbd.** Proves split search/transport
+   pairing — the one case where two providers cooperate on a single
+   request.
+
+## Risks and constraints
+
+- **No bundled credentials, no default-on providers, no preconfigured
+  indexers.** Every provider is off until the user configures it. The
+  app ships the ability to connect to services the user already runs.
+- **yt-dlp is a moving target.** Pin a minimum version, check it at
+  provider-enable time, and fail with a clear message rather than
+  parsing garbage output.
+- **Filename-only matching is genuinely hard.** Soulseek results are
+  `\Music\Album (1997) [FLAC]\01 - Track.flac` at best. Budget real
+  effort for the path-parsing heuristics; `autotag/normalize.go` is the
+  starting point.
+- **Partial and failed grabs must never reach the library.** The import
+  step is the only writer into library paths, and it runs after
+  verification. Staging sweep on startup.
+- **Tests must not hit the network.** `httptest` servers for slskd/
+  Lidarr/Prowlarr, a stub binary for yt-dlp.
+
+## Deferred
+
+- Wanted list with background retry (the natural next plan).
+- Artist monitoring + auto-grab of new releases — cheap once the wanted
+  list exists, because `explore`'s dump index already knows the full
+  discography and `library` already knows what's owned.
+- Quality profiles and upgrade-if-better.
+- Native Soulseek protocol client.
+- Transmission/Deluge/NZBGet (same shape as their shipped siblings —
+  add on demand).
+- Internet Archive / Bandcamp-collection providers: cheap REST adapters,
+  worth adding once the core is proven.
+
+## Resolved questions
+
+1. **Secret storage.** No keyring dependency was added. Credentials go
+   in a 0600 JSON file in the user data directory (`download-secrets.json`),
+   keyed by provider row ID. This is deliberately *not* encryption — a
+   key stored beside the data it unlocks protects nothing, and claiming
+   otherwise would be worse than being clear about it. What the file
+   mode buys is protection from other local users and from the config
+   file being pasted into a bug report. `SecretStore` is an interface so
+   an OS keyring backend can be added later without touching any
+   provider.
+2. **Auto-pick.** Implemented behind `Downloads.AutoPick`, default off.
+   It requires an MBID-anchored request, match ≥ 0.85, quality ≥ 0.5,
+   and ≥ 0.08 of daylight over second place. Free-text requests can
+   never auto-pick, because there is no tracklist to be right about.
+3. **Library layout.** Configurable path template, default
+   `{albumartist}/{album}/{track} {title}`. Segments are sanitized for
+   Windows-reserved characters and trailing dots/spaces so a library
+   synced between platforms does not produce unopenable files. Existing
+   files are never overwritten — a collision gets a numbered variant,
+   because the file already there may be a better copy the user owns.
+4. **Entry point.** "Find this album" on the Explore album page, shown
+   only when a client is connected and the album is not already owned.
+   The artist-discography right-click is not wired up yet.
+
+## What shipped
+
+All five phases, ~4,500 lines with tests, `make lint` clean and the full
+backend suite green (including under `-race`).
+
+**Core** (`backend/download/`): `Searcher`/`Transporter`/`Delegator`
+interfaces with capability-driven composition; `Request`/`Candidate`/
+`Result` types; provider registry with self-registering adapters;
+two-axis ranking; staging area with escape-guards and startup sweep;
+verify → tag → import tail; jobs integration under `KindDownload`;
+three tables catalogued in `datamap`.
+
+**Providers**: yt-dlp (subprocess; assembles albums from per-track
+searches, since a "full album" video cannot be imported as tracks),
+slskd (remote search + transport, peer-health scoring, collects from the
+daemon's own downloads folder), Lidarr (delegate; reconciles in place
+rather than moving files out from under a system still managing them),
+Prowlarr (search-only) paired at grab time with qBittorrent or SABnzbd.
+
+**Frontend**: `download-store.ts`, `download-picker` + `candidate-row`
+(two meters, not one blended score), `download-clients` settings section
+rendering its forms from backend descriptors so a new adapter needs no
+frontend change.
+
+## Follow-ups
+
+- **Resume across restart.** Live transfers are currently marked failed
+  on startup and their staging swept, because the transports do not
+  survive the process. slskd and qBittorrent can both resume in
+  principle; the item rows already carry what would be needed.
+- ~~**Per-provider concurrency caps.**~~ Done in 004: per-kind defaults
+  (slskd 1, yt-dlp 2, torrent/usenet 4) with a per-provider override,
+  and the provider's slot is taken before the global one.
+- **Prowlarr candidates score blind.** Indexer results carry no file
+  list, so match scoring has only the release title. Fetching the
+  torrent metadata before ranking would fix this and is the single
+  biggest ranking improvement available.
+- ~~Wanted list, artist monitoring~~ — done in 004. Quality profiles
+  and upgrade-if-better remain deferred.

.planning/plans/completed/004-wanted-list.md

@@ -0,0 +1,163 @@
+# 004 — Wanted list
+
+**Status:** implemented
+**Branch:** main
+**Created:** 2026-07-29
+**Follows:** 003-download-clients
+
+## Problem
+
+Plan 003 shipped a request as a heavyweight row: library, anchors,
+cached tracklist, state machine, error text, cascading items. That is
+the right shape for *one attempt to acquire something* and the wrong
+shape for *the user wanting something*, and 003 used it for both.
+
+The consequences showed up immediately. A request that found nothing was
+marked `failed`, which is a lie — the album exists, no source had it
+today. Retrying meant the user remembering to press a button. Wanting an
+artist's future releases was not expressible at all. And a user who
+acquired an album by other means kept a failed row about it forever.
+
+## The model
+
+A **want** is an MBID, what that MBID names, and retry bookkeeping.
+That is all.
+
+```
+download_wants(mbid, entity, library_id, scope, secondary, state,
+               parent_id, attempts, last_error, next_try_at,
+               external_ids)
+```
+
+`entity` is the only type distinction, and it carries all the policy:
+
+| entity | meaning |
+|---|---|
+| `artist` | a subscription. Never satisfied; each pass expands the discography into child wants |
+| `release-group` | an album in the abstract — any release satisfies it |
+| `release` | one specific edition |
+| `recording` | one track |
+
+`UNIQUE(mbid, library_id)` is load-bearing: it is what makes artist
+expansion idempotent, so a subscription can re-run every pass and add
+only what is genuinely new.
+
+Requests did not go away — they became what they always were, the
+ephemeral record of one attempt, with a nullable `want_id` back-link.
+The lifetimes are now opposite and explicit: **a request is history, a
+want is intent.**
+
+### Nothing here fails
+
+There is no `failed` want state. An attempt can fail; a want cannot. A
+want that found nothing gets `attempts + 1`, a reason the user can read,
+and a longer backoff — 6h doubling to a 7-day ceiling, jittered so a
+list added in one sitting does not come due in one burst.
+
+### Satisfaction is ownership, not download
+
+A want retires when the *library* owns what it names, however it got
+there — bought, ripped, copied in. Inferring satisfaction from our own
+completed downloads would keep hunting for music already on disk.
+
+### Artist scope defaults to `future`
+
+Following an artist takes new releases only, and skips compilations,
+live albums and remixes. `all` backfills the discography, and the user
+can widen it from the wanted list. Subscribing should not silently queue
+forty albums.
+
+## The reconciler
+
+A 6-hourly loop (plus on-demand, plus a 3-minute startup delay so the
+explore index has loaded). Four steps, in this order:
+
+1. **Expand** artist subscriptions into album wants — first, so step 2
+   sees them this pass rather than next.
+2. **Retire** wants the library already owns.
+3. **Sync** to clients that keep their own list.
+4. **Attempt** a bounded batch (25) of due wants.
+
+Everything the loop needs about music comes through a four-method
+`CatalogPort`, adapted to the explore index in `backend/downloadcatalog.go`
+— the composition root, so neither package learns about the other.
+
+### Unattended grabs, and what stops them
+
+`Manager.Attempt` is `Start` without the parking: it searches, and grabs
+only if `AutoPickable` clears. When it does not, **nothing is
+persisted** — no request row. A want retried weekly for a year would
+otherwise leave fifty identical failed rows, none of them anything the
+user can act on.
+
+`AutoPickable` gained one condition: an anchored request with an empty
+`Expected` is refused. An anchor with no tracklist behind it is an
+anchor in name only, and match then rests on album/artist text — exactly
+the evidence a wrong-album candidate also has. Nobody is watching a
+reconcile pass.
+
+## Per-provider concurrency
+
+`Downloads.MaxConcurrent` was the only limit, and was never actually
+applied (`SetMaxConcurrent` did not exist). Now:
+
+- **slskd defaults to 1.** A Soulseek peer serves one file at a time
+  from one person's upload slot; asking for more gets you queued behind
+  everyone else at best. One is both the polite number and usually the
+  fastest.
+- yt-dlp 2, torrent/usenet clients 4, overridable per provider via a
+  `maxConcurrent` field that `Register` appends automatically to any
+  descriptor declaring `CanTransport`.
+- A grab takes its **provider's** slot before the global one, so a queue
+  on a busy slskd cannot sit on a global slot a usenet transfer could
+  have used. The transport is resolved before either slot is taken;
+  delegates take neither, since the transfer is happening inside another
+  system that is doing its own limiting.
+
+## The Lister role
+
+The fourth role, alongside Searcher/Transporter/Delegator. Lidarr
+already models a want — a monitored artist or album — and it is always
+on, where a desktop player is not. A subscription mirrored there keeps
+working while the app is closed.
+
+- `artist` → Lidarr artist, `monitor: future|missing` per scope
+- `release-group`/`release` → monitored album
+- `recording` → not pushed. Lidarr cannot say "one track", and
+  monitoring the album to get it downloads far more than was asked.
+
+Sync is push-only in the loop; pulling happens only when the user
+explicitly imports ("adopt the artists Lidarr already monitors", which
+arrive at `future` scope). Removal **unmonitors**, never deletes — the
+user's Lidarr may predate this app.
+
+## Frontend
+
+- `Wanted` view in the sidebar: Following / Looking for / Paused /
+  Found, with pause, remove, scope toggle and "Check now".
+- "Want this" on the album page, "Follow for new releases" on the artist
+  page. The want button shows whether or not a client is connected —
+  wanting is durable and stays queued until one exists.
+- `WantedListChanged` event, since a background pass changes the list
+  without the UI doing anything.
+
+## Files
+
+`backend/download/want.go`, `wantstore.go`, `reconcile.go`,
+`provider_lidarr_list.go`; `backend/downloadcatalog.go`;
+schema `download_wants.sql` + migration 48 for the two new
+`download_requests` columns; `frontend/src/components/wanted-view/`.
+
+## Deferred
+
+- **Release-group wants are not retired by ownership of a specific
+  release.** The library indexes release groups and recordings, not
+  editions, so a `release` want is only satisfied by its own download
+  completing.
+- **No recording lookup on the explore index**, so a track want relies
+  on the title the UI passed in. A want added as a bare recording MBID
+  has no tracklist and waits.
+- Quality profiles and upgrade-if-better (from 003).
+- Resume across restart (from 003) — still the largest gap, and it now
+  matters more: an unattended grab that dies on restart is retried by
+  the reconciler, but from zero bytes.