docs: make the issue tracker the source of truth
Work has been starting from a chat message and a plan file, so two people could pick up the same thing and neither could see the other. The tracker is where that is visible. Search before starting, claim before the first edit -- not before the commit, since the point is that the other person can see the work is taken while it is being done. If no issue covers it, open one first: that is what makes the tracker a description of the project rather than a description of the past. The conventions were already right and are written down rather than reinvented -- the Kind/Area/Priority/Platform/Reviewed/Status taxonomy, its exclusive scopes, #73 as the roadmap, real Gitea dependencies for hard blockers, and PR #83's body shape. What #83 also demonstrated is that a Closes list closes nothing reliably: it listed ten and five of them sat open in main for a fortnight. So closing is a step you take and verify, not a keyword you trust. .planning/ stops being a queue and keeps design documents and measured history -- NOTES.md, the audits, the completed plans and the arguments in them. plans/pending/ is gone, because a plan nobody is executing is an issue; everything unimplemented in it is now #85-#91, and each completed plan says which issue carries its remainder. autotag.md is kept as a historical record, marked stale where the scoring overhaul overtook it. The commit grammar is unchanged and is load-bearing for a different reason, so the issue number lives in the branch name and the PR body rather than the commit subject. Refs #92
This commit is contained in:
+2
@@ -1,5 +1,7 @@
|
||||
# 012 — What we ask the network for, and what we already had
|
||||
|
||||
> **Completed.** Findings 1, 2 and 4 shipped. Finding 3 — the bound-but-uncalled methods — is now **#86**.
|
||||
|
||||
**Status:** all four findings fixed. Lint (3 configs), Go tests (3
|
||||
configs), `tsc` and 752 Vitest tests pass; **not driven against the
|
||||
real app**, so the numbers below are read off the code, not measured.
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
# 015 — Android release pipeline
|
||||
|
||||
> **Completed.** The pipeline ships a signed APK from CI on every `v*` tag; `docs/android-release.md` is its operating document.
|
||||
|
||||
Ship an Android APK from CI on every version tag, published to the Gitea
|
||||
generic package registry so Obtainium can poll a plain URL.
|
||||
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
# 015 — Multi-artist credits, navigable
|
||||
|
||||
> **Completed.** Phases 1, 2 and 4 shipped. Running the ingest against the real dump and publishing an artifact that carries credits is **#88**; Phase 3 (`file_artists`) is **#89**, blocked on it.
|
||||
|
||||
## The problem
|
||||
|
||||
A track credited to more than one artist has exactly one navigable
|
||||
+2
@@ -1,5 +1,7 @@
|
||||
# 016 — What Android parity would actually take
|
||||
|
||||
> **Completed.** Sections A, B1, B2 and B4 shipped. B3, writing tags on the device, is now **#87**; the device-found UI faults are #51–#72, sequenced by #73.
|
||||
|
||||
> **Status: all of section A is done.** A1–A3 landed with "let the app
|
||||
> reach the user's music"; A4 (MediaSession, transport notification,
|
||||
> audio focus) landed with "survive the screen locking". The direction
|
||||
@@ -0,0 +1,164 @@
|
||||
# Autotag (v1.3) — MusicBrainz Autotagger
|
||||
|
||||
> **Historical record.** Phases 008–010 shipped, and the scoring engine was subsequently overhauled (`recommend.go`, `rank.go`, `mixedbag.go`), which makes the 011/012 sections below stale in their details. What is actually left is **#90** (auto-accept and entry points) and **#91** (settings, and a way back from the dismissed file-write warning).
|
||||
|
||||
The MusicBrainz autotagger, collectively **v1.3**. Builds on the explore-browser API client + cache foundation. Five sequential phases (008–012), each depending on the prior one.
|
||||
|
||||
| Phase | Title | Status |
|
||||
|-------|-------|--------|
|
||||
| 008 | Schema & Grouping Foundation | shipped 2026-04-20 |
|
||||
| 009 | Scoring Engine & MB Orchestration | shipped 2026-04-21 |
|
||||
| 010 | Review UI & Apply Pipeline | **active** |
|
||||
| 011 | Auto-Accept & Entry Points | pending |
|
||||
| 012 | Settings & Polish | pending |
|
||||
|
||||
---
|
||||
|
||||
## 008 — Schema & Grouping Foundation · shipped 2026-04-20
|
||||
|
||||
> First v1.3 phase. Lays down the schema + bookkeeping — no scoring, no UI, no MB calls yet.
|
||||
|
||||
### What landed
|
||||
|
||||
- **008.1 — Migration 31: `audio_files.tag_status`.** Text column with inline `CHECK(tag_status IN (...))` constraint (SQLite `ALTER TABLE ADD COLUMN` supports column constraints, so both fresh and upgraded DBs enforce it). Partial index `idx_audio_files_tag_status_untagged ON audio_files(library_id) WHERE tag_status = 'untagged'` powers the pending badge. Backfill sets `user_confirmed` where the joined recording already has an MBID; everything else defaults to `untagged`.
|
||||
- **008.2 — Migration 32: `tagging_items` + `group_key`.** New `tagging_items` table (PK `group_key`, `status CHECK IN ('pending','matched','confirmed','skipped')`, two indexes including the partial `WHERE status = 'pending'` for the badge). `audio_files.group_key TEXT NOT NULL DEFAULT ''` with partial index `WHERE group_key != ''`. Column-referencing indexes live in the migration, not the schema file, because `CREATE TABLE IF NOT EXISTS` is a no-op on existing tables and the partial-index predicate would reference a column that hasn't been added yet.
|
||||
- **008.3 — `backend/autotag.GroupKey` + scan-path integration.** Lower-case hex SHA-1 over `libraryID || 0 || parentDirLower || 0 || albumTrimmed || 0 || discNumber` — intentionally shallow, deeper normalization is scoring territory (009). `saveAudioFile` switched to `CreateAudioFileWithGroupKey` + `UpsertTaggingItemOnTrackAdd`. `updateAudioFileMetadata` rebinds on key change: decrement old group → delete if empty → upsert new group → write new key onto the audio_files row. Migration 32 streams existing rows in batches of 500 and aggregates `tagging_items` at the end, defaulting status to `confirmed` when every track in the group is `user_confirmed`.
|
||||
- **008.4 — Pending-queue sqlc queries.** `CountPendingTaggingItems`, three `ListPendingTaggingItems...` variants (alphabetical, by-score nulls-last, by-recent) — sqlc has no dynamic ORDER BY so each sort has its own query. `CAST(@param AS INTEGER|TEXT)` hints give typed params (otherwise sqlc emits `interface{}`). `GetTaggingItem` and `ListAudioFilesInTaggingGroup` round out the queue API. `EXPLAIN QUERY PLAN` test asserts the badge query uses `idx_tagging_items_status_pending` so a future schema change that breaks the partial index fails loudly.
|
||||
|
||||
### Key decisions retained
|
||||
|
||||
- **Hash algorithm in Go, not SQL.** `autotag.GroupKey` is the single source of truth for the key format so we can evolve it without coupling to SQLite functions.
|
||||
- **SHA-1 over alternatives.** Matches the codebase's existing non-crypto deterministic-key convention. Collision risk at album-group cardinality (millions) is irrelevant.
|
||||
- **Null-byte separators between hash inputs.** Prevents `a|b` vs `ab|` ambiguity.
|
||||
- **Per-track integration inside `commitBatch`'s transaction**, not a post-scan callback — keeps `tagging_items` coherent after partial scans.
|
||||
- **`best_match_release_mbid`, `score`, `last_checked_at` shapes fixed now** even though they stay NULL until 009-010. Avoids schema churn later.
|
||||
|
||||
---
|
||||
|
||||
## 009 — Scoring Engine & MB Orchestration · shipped 2026-04-21
|
||||
|
||||
> Second v1.3 phase. Given an album-group, produce a ranked list of candidate releases with per-track alignment data, using as few MusicBrainz calls as the API-minimization playbook allows. No UI yet — 010 surfaces this to the user.
|
||||
|
||||
### What landed
|
||||
|
||||
- **`backend/autotag/` domain types** — `Candidate`, `TrackAlignment`, `GroupScore`, `LocalTrack`, `CandidateSource` (`local` / `musicbrainz`), `AlignmentStatus` (`matched` / `missing` / `extra` / `mismatched`).
|
||||
- **`Normalize(s)`** — Unicode NFC → qualifier-suffix strip (`(Remastered 2009)`, `[Bonus Track]`, `(feat. X)`, etc.) → case fold → punctuation drop → whitespace collapse. Comparison-only, not human-readable.
|
||||
- **Per-track distance** — weighted 60% title similarity (1.0 − Levenshtein/max-len), 30% length delta (linear: ≤1 s = 1.0, ≥30 s = 0.0, neutral 0.5 when either side unknown), 10% track-number match.
|
||||
- **Greedy alignment** — `AlignTracks` picks the highest-scoring (local, cand) pair iteratively; not Hungarian-optimal but fine at album cardinalities. Emits `matched` / `missing` / `extra` / `mismatched` rows so the review UI can render the diff.
|
||||
- **Local-first resolver** — `ListLocalReleaseGroupCandidates` sqlc query pre-filters on `rg.mbid != '' AND r.mbid != '' AND rg.name = ? COLLATE NOCASE`, then Go applies full normalization. Candidate track lists only include recordings that themselves have MBIDs (avoids untagged dupes polluting the "canonical" view when the same `(name, artist)` release group is shared across libraries).
|
||||
- **MB orchestration** — `MBClient` interface (`SearchReleaseGroups`, `BrowseReleases`, `LookupArtist`) hides `explore.MusicBrainzClient`; `backend/explore/autotagclient.go` adapts one to the other. `buildMBQuery` assembles `release:"X" AND arid:<mbid> AND tracks:N` — `arid:` makes the search cache key deterministic, `tracks:N` filters out box-set-style releases. One search per album + one `BrowseReleases` per candidate RG.
|
||||
- **Release-level ranker** — aggregate track score (70%) + track-count match (15%, zero at ≥50% delta) + meta (15%, avg of year/Official/country bonuses). `Scorer.ScoreGroup` hits local first, runs MB only when no local candidate scores ≥ 0.90.
|
||||
- **Persistence** — `SetTaggingItemBestMatch` writes `best_match_release_mbid`, `score`, `last_checked_at = CURRENT_TIMESTAMP`, `status = 'matched'`.
|
||||
|
||||
### Key decisions retained
|
||||
|
||||
- **SHA-1-grade normalization vs. full MB-equivalent.** Qualifier regex handles the common cases (remaster, deluxe, explicit, feat., bonus, etc.) without dragging in a full MB title-parsing library. Edges that bite in real libraries will show up in 010 review UX and can be patched then.
|
||||
- **`*sqlcgen.Queries` as the DB boundary for autotag**, not `*database.DB`. The `database` package already depends on `autotag.GroupKey` (from 008.3), so reversing the dependency via `database` would create a cycle. Using `sqlcgen` directly is acyclic and keeps `autotag` swappable.
|
||||
- **Scorer constructor takes `MBClient` as interface, not `*explore.MusicBrainzClient`.** Lets tests inject a stub without spinning up the HTTP + cache layer. The concrete adapter lives in `explore/autotagclient.go`.
|
||||
- **`localSufficient = 0.90` threshold for skipping MB.** Empirical guess — will get retuned in 011 auto-accept phase when we observe real corpus scores.
|
||||
- **Weights `60/30/10` for title/length/track-number.** Cribbed from beets' broad intuition; tuned to emphasize title-matching since length data can be unreliable from Vorbis Comments. Tests document the expected floors (e.g. "exact match should score ≥ 0.99") so nudging weights won't silently regress.
|
||||
- **`release_groups` and `recordings` must both carry MBIDs** for a local candidate. Otherwise untagged dupes of the same album (across libraries) falsely expand the "canonical" track list.
|
||||
- **UTF-8 em-dashes in SQL comments broke sqlc's string-literal emitter**, truncating generated query strings mid-word. All autotag SQL comments use ASCII punctuation.
|
||||
|
||||
### Known follow-ups into 010+
|
||||
|
||||
- **`guessArtistMBID` is a stub** returning `""` because `LocalTrack` doesn't currently carry artist MBIDs. 010 should thread artist MBIDs through `ListAudioFilesInTaggingGroup` so the MB resolver can use `arid:` filters.
|
||||
- **`yearBonus` uses `time.Now().Year()`** as a placeholder target. Should become the earliest release-date hint from the group's tracks once 010 provides it.
|
||||
- **VA compilation detection threshold** is still open. The scorer doesn't special-case per-track artist credits differing from album-artist.
|
||||
|
||||
---
|
||||
|
||||
## 010 — Review UI & Apply Pipeline · ACTIVE
|
||||
|
||||
> Third v1.3 phase. User reviews one album at a time, sees the diff clearly, and applies or skips — file tags get written, DB gets synced, cover art follows the never-replace-existing rule.
|
||||
|
||||
**Requirements:** REVIEW-01..07 · **Depends on:** 009 (needs candidates + scores)
|
||||
|
||||
### Success criteria
|
||||
|
||||
1. `/autotag` shows the next pending album with its top candidate as a field-by-field diff. Missing-from-local and extra-in-local tracks are shown explicitly.
|
||||
2. Keyboard shortcuts work without the mouse: `A` apply, `M` more candidates, `S` skip, `L` leave-as-is, `U` paste URL, `→`/`←` navigate.
|
||||
3. Apply writes tags to every track in the group via the existing format-specific writers + atomic write + DB sync + FTS5 sync. Whitelisted fields only: title, artist, album, album-artist, year, track#, track-total, disc#, disc-total, all MBIDs.
|
||||
4. Cover art rule: embed only when the file has no existing art **and** CAA returns ≥500 px on the shortest side. Never replace existing embedded art (auto or manual).
|
||||
5. First-ever apply per library shows an irreversibility warning. "Don't show again" sets a flag on the `libraries` row; never shows again for that library.
|
||||
6. While the user reviews album N, candidates for album N+1 are prefetched into `http_cache` so advancing feels instant.
|
||||
7. "Paste MB URL" dialog accepts a release URL, extracts the MBID, runs one `LookupRelease`, renders the diff against the current album.
|
||||
|
||||
### Sub-plans
|
||||
|
||||
- Wails bindings — `StartAutotagQueue`, `GetCurrentCandidate`, `GetCandidates(groupKey)`, `Apply(groupKey, releaseMBID)`, `Skip`, `LeaveAsIs(groupKey)`, `RetagGroup(groupKey)`.
|
||||
- `/autotag` page layout — focused album header, diff table, candidate sidebar, missing/extra panel.
|
||||
- Keyboard shortcut wiring through the existing scope-aware dispatch.
|
||||
- Apply pipeline integration with existing tag writers + DB sync.
|
||||
- Cover art apply rule + CAA fetch + 500 px minimum check.
|
||||
- File-write warning dialog with per-library persistence.
|
||||
- Prefetch-next-album goroutine, rate-limiter aware.
|
||||
- Paste-MB-URL escape hatch.
|
||||
|
||||
### Risk callout
|
||||
|
||||
Every apply rewrites a file. The `AtomicWrite` pipeline mitigates corruption risk; the per-library warning mitigates surprise. Dry-run mode (from 009) lets developers validate scoring changes without file writes.
|
||||
|
||||
---
|
||||
|
||||
## 011 — Auto-Accept & Entry Points · pending
|
||||
|
||||
> Fourth v1.3 phase. The strict all-match auto-accept path runs as a background job; the tool is reachable from every place a user expects.
|
||||
|
||||
**Requirements:** AUTO-01..06 · **Depends on:** 010 (needs the apply pipeline)
|
||||
|
||||
### Success criteria
|
||||
|
||||
1. An album-group qualifies for auto-accept iff: exact track-count match, every track's normalized title matches, every track's length within ±2s, no cover-art replacement required, no existing-MBID conflicts. Decision uses already-cached candidate data — **no additional MB calls**.
|
||||
2. Auto-accept job processes all qualifying groups in the queue, emits progress events, is cancellable at any point, honors the shared rate limiter.
|
||||
3. Right-click on a track / album / artist exposes "Autotag this album" (queues + jumps to review) and "Retag" (flips status to `untagged` and requeues).
|
||||
4. After a library scan finishes with N new untagged albums, a non-blocking toast appears linking to `/autotag`.
|
||||
5. Sidebar has an "Autotag" nav entry with a pending-count badge, updates reactively.
|
||||
6. Pasting a MB release URL into the Paste-URL dialog renders a full diff against the current album with one `LookupRelease` call.
|
||||
|
||||
### Sub-plans
|
||||
|
||||
- Strict all-match rule + unit tests.
|
||||
- Auto-accept background job — progress events, cancellation, queue traversal.
|
||||
- Context menu integrations on track/album/artist views.
|
||||
- Post-scan toast wiring via the existing scan-complete event.
|
||||
- Sidebar nav entry + pending-count badge store integration.
|
||||
|
||||
### Risk callout
|
||||
|
||||
Release selection can pick the wrong edition. The exact-track-count gate prevents most silent misbehavior, but bonus-track editions and remaster reissues with matching track counts are genuine ambiguity. Manual review handles the edge cases — that's why auto-accept is strict by design, and the slider for fuzzy auto-accept is explicitly out of scope.
|
||||
|
||||
---
|
||||
|
||||
## 012 — Settings & Polish · pending
|
||||
|
||||
> Fifth and final v1.3 phase. Configuration surfaces in the existing settings system; the known sharp edges (rate-limit contention, VA compilations, singleton files) get sanded; the fingerprinting seam is in place for the future.
|
||||
|
||||
**Requirements:** CFG-01..06 · **Depends on:** 011
|
||||
|
||||
### Success criteria
|
||||
|
||||
1. Autotag settings panel accessible via the existing templ/HTMX settings UI. Exposes: enable/disable auto-accept, per-library file-write warning reset, default review filter, default sort order.
|
||||
2. Shared rate limiter distinguishes interactive from background requests. User-initiated MB calls (paste-URL, opening a review, explore browsing) are never blocked behind a running auto-accept job.
|
||||
3. VA compilation albums (per-track artist credits differ from album-artist) are detected. The auto-accept artist-match rule relaxes for them; the ranker prefers MB releases credited to "Various Artists".
|
||||
4. Singleton files (`track_count = 1`, no sibling context) use a recording-level match path (`SearchRecordings` with title + artist + length filters). Lower confidence ceiling — never eligible for auto-accept regardless of confidence.
|
||||
5. `type Identifier interface { Identify(path) ([]Candidate, error) }` exists with `MetadataIdentifier` as the v1 implementation. No fpcalc integration, but the seam is in place for a future `AcoustIDIdentifier`.
|
||||
6. User-facing quickstart docs exist; CLAUDE.md gets a `backend/autotag/` package description; scoring-function dev notes are committed.
|
||||
|
||||
### Sub-plans
|
||||
|
||||
- Autotag settings panel (templ + HTMX).
|
||||
- Rate-limiter priority support.
|
||||
- VA compilation detection and scoring adjustments.
|
||||
- Singleton-file match path.
|
||||
- `Identifier` interface seam with `MetadataIdentifier`.
|
||||
- Docs — user quickstart + dev notes + CLAUDE.md update.
|
||||
|
||||
---
|
||||
|
||||
## Ship criteria for v1.3 overall
|
||||
|
||||
- All 29 SCHEMA/MATCH/REVIEW/AUTO/CFG requirements complete.
|
||||
- All five phases' success criteria verified end-to-end on a real library (10k+ tracks, mixed match quality).
|
||||
- Auto-accept run against a well-tagged subset produces zero incorrect matches.
|
||||
- Manual review workflow can process 100 albums in under 30 minutes without mouse use.
|
||||
@@ -1,195 +0,0 @@
|
||||
# 010 — Owned albums, offline
|
||||
|
||||
**Status:** not started — and **much smaller than when it was written**
|
||||
**Branch:** none yet
|
||||
**Created:** 2026-08-13
|
||||
**Depends on:** nothing
|
||||
**Related:** the `AlbumReleasesFailed` fix that prompted it, and the
|
||||
tag-derived completeness that landed after it (same session)
|
||||
|
||||
---
|
||||
|
||||
## What already shipped, and what it leaves
|
||||
|
||||
The common case is solved without this plan. `GetAlbumCompleteness`
|
||||
reads the "5/12" denominator off the files' own tags — persisted to
|
||||
`release_group_recordings.total_tracks`, having been extracted at every
|
||||
scan since forever and discarded — and an album that is **MBID-matched
|
||||
and complete** now opens with **no catalog call at all**. Identity from
|
||||
the MBID, tracklist from the tags; those were the two things the browse
|
||||
was being spent on.
|
||||
|
||||
So the set this plan still has to serve is not "albums you own a track
|
||||
of". It is:
|
||||
|
||||
- albums that are genuinely **incomplete** (the catalog is the only way
|
||||
to say *which* tracks are missing — tags give the count, not the
|
||||
names), and
|
||||
- albums whose tags **never declared a total**, where completeness is
|
||||
unknowable locally and the catalog is the only source.
|
||||
|
||||
On a well-tagged library that is a small minority, which changes the
|
||||
economics below considerably: the run is shorter, and the rate limiter
|
||||
contention that dominates this design is proportionally less severe.
|
||||
Re-measure before building — the answer may now be "the prefetch is
|
||||
enough".
|
||||
|
||||
---
|
||||
|
||||
## The problem
|
||||
|
||||
Opening an album detail page for an album **you already own** hits
|
||||
MusicBrainz. Every time it is not in the response cache, which for most
|
||||
of a library is every time, because nothing warms that cache except a
|
||||
capped prefetch on the artist page.
|
||||
|
||||
The user's framing: *this is a classic example of an album we should
|
||||
have had locally.*
|
||||
|
||||
## Why we do not have it, despite the discography backfill
|
||||
|
||||
`BackfillLibraryDiscographies` / `EnsureArtistDiscography`
|
||||
(`backend/explore/searchindex.go:301`, `:397`) do less than the name
|
||||
suggests. Per artist, `indexOneArtist` fetches:
|
||||
|
||||
- `fetchTopReleaseGroups` — capped at `indexMaxRGs` (50)
|
||||
- `fetchTopRecordings` — capped at `indexMaxRecs` (200)
|
||||
|
||||
and writes them as **flat `explore_index` rows**. There is no release
|
||||
group → tracklist relation anywhere in the index, and no release-level
|
||||
rows at all. `explore_index` recordings carry `caa_release_mbid` and
|
||||
`release_name`, which name the release used for cover art — not a
|
||||
tracklist.
|
||||
|
||||
So "we have full discographies for library artists" means *we know
|
||||
which albums the artist made, offline*. It has never meant we know
|
||||
what is on any of them.
|
||||
|
||||
The only store of release-level catalog data in the app is `http_cache`
|
||||
under `mb:browse:releases:<rg>` (90-day TTL, `musicbrainz.go:27`),
|
||||
populated **only** by a live `BrowseReleases` with
|
||||
`Includes: ["recordings", "media"]` at `MaxLimit` — the most expensive
|
||||
call the app makes to MusicBrainz. It is warmed by exactly one thing:
|
||||
`PrefetchReleases` (`explore.go:746`), capped at 8, called only when an
|
||||
artist page renders.
|
||||
|
||||
An album opened from the library grid therefore always browses live.
|
||||
|
||||
## What to build
|
||||
|
||||
**A post-scan backfill that warms the release cache for release groups
|
||||
that are owned but not known-complete** — bounded, resumable, and
|
||||
shaped exactly like `BackfillLibraryDiscographies`, which is the proven
|
||||
pattern for this in the codebase.
|
||||
|
||||
The scoping rule is the user's and it is the right one: not "every
|
||||
album by every artist in the library" (50 release groups per artist,
|
||||
mostly never opened) but albums with owned tracks — narrowed further,
|
||||
now, to the ones a local answer cannot already cover. The query gains
|
||||
one clause: skip release groups whose `GetAlbumCompleteness` reports
|
||||
`complete`.
|
||||
|
||||
Sketch:
|
||||
|
||||
1. A query for release groups with ≥1 owned track and no warm release
|
||||
cache entry. `release_groups.mbid` is the key; the owned-track join
|
||||
is `audio_files → recordings → release_group_recordings`, the same
|
||||
shape `unenrichedLibraryArtistMBIDs` already uses one table over.
|
||||
2. Order by owned-track count descending, so the albums the user has
|
||||
most of are warmed first — same reasoning as the discography
|
||||
backfill's ordering, same benefit if a run is cut short.
|
||||
3. Run through `releasesSF`, so it never double-fetches a release group
|
||||
an interactive open is already handling.
|
||||
4. Bound a run (`discogBackfillMaxPerRun` has a value to copy) and make
|
||||
it resumable: the resume marker is the response cache itself —
|
||||
`BrowseReleasesCached` already answers "is this one done", so unlike
|
||||
the discography path this needs **no new flag column**.
|
||||
5. Trigger it where `BackfillLibraryDiscographies` is triggered, and
|
||||
register it with `jobs` so it has progress, pause and cancel like
|
||||
every other long-running operation.
|
||||
|
||||
### The rate limiter is the whole design constraint
|
||||
|
||||
> **Update (2026-08-13): the priority half is built, and the sentence
|
||||
> below is wrong on a detail.** `e.mb` runs on `mbSearchLimiter`
|
||||
> (`NewRateLimiterBurst(3, 1)`); the 1 req/s `NewRateLimiter()` cited
|
||||
> here is the *artist image* limiter. Both are shared and both were
|
||||
> FIFO. `RateLimiter.WithBackgroundLane` + `WithBackgroundPriority(ctx)`
|
||||
> now make a marked caller yield to interactive work and pace at 1/s,
|
||||
> and `jobs.KindCatalogEnrich` + `startBackfillJob` give the existing
|
||||
> backfills progress and cancel. **"Do not start until the priority
|
||||
> question has an answer" is satisfied** — mark this backfill's context
|
||||
> and register it the way `BackfillLibraryDiscographies` now is.
|
||||
> `PrefetchReleases`' cap of 8 is still unrevisited.
|
||||
|
||||
One shared `NewRateLimiter()` at 1 req/s (`explore.go:84`) serves this,
|
||||
`PrefetchReleases`, and every interactive browse. A backfill over a
|
||||
few thousand owned albums is *hours* of wall clock at that rate — which
|
||||
is fine for a background job, and not fine if it starves the album page
|
||||
the user is looking at right now.
|
||||
|
||||
That is the real work in this plan, and it is not the query:
|
||||
|
||||
- Interactive browses need to **jump the queue**. Today they cannot;
|
||||
there is one limiter and it is FIFO.
|
||||
- `PrefetchReleases`' cap of 8 was sized when nothing else competed for
|
||||
the limiter. Revisit it in the same change.
|
||||
- The 60 s fallback the `AlbumReleasesFailed` fix installed is sized
|
||||
for today's contention. If a backfill can queue behind it, that
|
||||
number is wrong again — which is an argument for priority, not for a
|
||||
bigger number.
|
||||
|
||||
Do not start the query until the priority question has an answer.
|
||||
|
||||
## The alternative that was considered and rejected
|
||||
|
||||
**Project release-group tracklists in the dump build and ship them in
|
||||
the artifact.** The data is there: `canonical_musicbrainz_data.csv`
|
||||
carries `release_mbid` *and* `recording_mbid`
|
||||
(`dumpcatalog.go:520`), and `release_to_rg` already maps release →
|
||||
release group. It is derivable from bytes the index build already
|
||||
streams, with no new API surface at all, and it would work offline on
|
||||
first launch with no per-user backfill.
|
||||
|
||||
It is rejected **for this plan** because the artifact is built
|
||||
centrally and is byte-identical for every user, so "albums the user
|
||||
owns a track of" cannot be a filter on it. Shipping tracklists for the
|
||||
whole catalog means per-recording rows against a ~900 MB artifact
|
||||
budget (~426 B/row measured), and gating on a popularity floor means it
|
||||
is absent for exactly the obscure albums a local backfill would have
|
||||
covered.
|
||||
|
||||
Worse than absent, in fact — and this is the argument that actually
|
||||
kills it. The floor is not one number over artists; it is a **per
|
||||
artist track budget** (`dumpcatalog.go:58-89`): 50 tracks for a tier-A
|
||||
artist, 25 for tier B, 12 for tier C. A projected tracklist would
|
||||
therefore be *whichever* of an album's tracks survived that budget,
|
||||
with nothing marking the rest as absent — so the album page would count
|
||||
owned against a truncated denominator and render "Play 7 of 9" for a
|
||||
twelve-track album. That is a confident lie, where the honest states
|
||||
this plan's alternative produces (complete / incomplete / unknown) are
|
||||
at worst silent.
|
||||
|
||||
Note that `markLibraryArtists` (`dumpcatalog.go:246`) already grants
|
||||
every library artist full coverage — 500 tracks, 100 release groups —
|
||||
by reading the local library, so the per-user tailoring this option
|
||||
supposedly cannot have does exist in code. It is a no-op in the CI
|
||||
build (empty library), and reaching it means a **local** dump build:
|
||||
the ~205 GB, half-a-day download the entire artifact design exists to
|
||||
avoid. Whoever finds that function next should read this paragraph
|
||||
before getting excited about it.
|
||||
|
||||
Worth revisiting if the artifact ever gains per-user tailoring, or if a
|
||||
measurement shows the row count is smaller than feared. Note it also
|
||||
yields the *canonical* tracklist rather than MusicBrainz's full version
|
||||
list, so the versions dropdown would still browse live when opened.
|
||||
|
||||
## Done when
|
||||
|
||||
- Opening an owned album that has never been opened before renders its
|
||||
catalog tracklist with no network call, after one backfill run.
|
||||
- An interactive browse issued while the backfill is running is not
|
||||
delayed by it.
|
||||
- The backfill appears in the jobs indicator, and can be paused and
|
||||
cancelled there.
|
||||
- A second run after a completed one does approximately nothing.
|
||||
Reference in New Issue
Block a user