yonluandClaude Opus 5
edb13a6f39
Three things on the Explore surfaces, all about not asking twice. A portrait already on disk costs no network call. explore-view seeded only from the library store — owned artists, which on a catalog search is nearly none of the results — and sent everything else to GetArtistImageURL, the resolving entry point, one await at a time. GetArtistImagesCachedPaths asks the disk about every unresolved artist in one call, and only what it does not answer reaches the resolver, in parallel. The artist page's two sections both wanted PrefetchReleases and each called it, so the most expensive call the app makes was issued twice for an overlapping set on a 1 req/s limiter. They are collected and sent once on a microtask, and prefetchRequested stops the cold-artist refetch re-asking for what it already asked for. The release cards — most of the artist page — had no context menu at all. They have one now on both release shapes, normalised to a ReleaseMenuTarget when the menu opens so the union does not reach the action handlers. It is a discriminated union rather than one nullable field per kind because the panel is shared with the track menu: that is what keeps aria-label moving with the target, which is the fault cover-grid shipped. Which items appear is three different questions — playback is gated on a local album id, not on "owned", and the request needs a catalog MBID, so it is absent for a library-only release. Note on the docs: the CLAUDE.md and NOTES.md prose here was reconstructed after a mishandled `git stash --keep-index` destroyed the uncommitted originals. One NOTES.md section is marked as incomplete where its text could not be recovered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UDCbcCZQepnpSQYJ6SxxZm
196 lines
9.0 KiB
Markdown
196 lines
9.0 KiB
Markdown
# 010 — Owned albums, offline
|
|
|
|
**Status:** not started — and **much smaller than when it was written**
|
|
**Branch:** none yet
|
|
**Created:** 2026-08-13
|
|
**Depends on:** nothing
|
|
**Related:** the `AlbumReleasesFailed` fix that prompted it, and the
|
|
tag-derived completeness that landed after it (same session)
|
|
|
|
---
|
|
|
|
## What already shipped, and what it leaves
|
|
|
|
The common case is solved without this plan. `GetAlbumCompleteness`
|
|
reads the "5/12" denominator off the files' own tags — persisted to
|
|
`release_group_recordings.total_tracks`, having been extracted at every
|
|
scan since forever and discarded — and an album that is **MBID-matched
|
|
and complete** now opens with **no catalog call at all**. Identity from
|
|
the MBID, tracklist from the tags; those were the two things the browse
|
|
was being spent on.
|
|
|
|
So the set this plan still has to serve is not "albums you own a track
|
|
of". It is:
|
|
|
|
- albums that are genuinely **incomplete** (the catalog is the only way
|
|
to say *which* tracks are missing — tags give the count, not the
|
|
names), and
|
|
- albums whose tags **never declared a total**, where completeness is
|
|
unknowable locally and the catalog is the only source.
|
|
|
|
On a well-tagged library that is a small minority, which changes the
|
|
economics below considerably: the run is shorter, and the rate limiter
|
|
contention that dominates this design is proportionally less severe.
|
|
Re-measure before building — the answer may now be "the prefetch is
|
|
enough".
|
|
|
|
---
|
|
|
|
## The problem
|
|
|
|
Opening an album detail page for an album **you already own** hits
|
|
MusicBrainz. Every time it is not in the response cache, which for most
|
|
of a library is every time, because nothing warms that cache except a
|
|
capped prefetch on the artist page.
|
|
|
|
The user's framing: *this is a classic example of an album we should
|
|
have had locally.*
|
|
|
|
## Why we do not have it, despite the discography backfill
|
|
|
|
`BackfillLibraryDiscographies` / `EnsureArtistDiscography`
|
|
(`backend/explore/searchindex.go:301`, `:397`) do less than the name
|
|
suggests. Per artist, `indexOneArtist` fetches:
|
|
|
|
- `fetchTopReleaseGroups` — capped at `indexMaxRGs` (50)
|
|
- `fetchTopRecordings` — capped at `indexMaxRecs` (200)
|
|
|
|
and writes them as **flat `explore_index` rows**. There is no release
|
|
group → tracklist relation anywhere in the index, and no release-level
|
|
rows at all. `explore_index` recordings carry `caa_release_mbid` and
|
|
`release_name`, which name the release used for cover art — not a
|
|
tracklist.
|
|
|
|
So "we have full discographies for library artists" means *we know
|
|
which albums the artist made, offline*. It has never meant we know
|
|
what is on any of them.
|
|
|
|
The only store of release-level catalog data in the app is `http_cache`
|
|
under `mb:browse:releases:<rg>` (90-day TTL, `musicbrainz.go:27`),
|
|
populated **only** by a live `BrowseReleases` with
|
|
`Includes: ["recordings", "media"]` at `MaxLimit` — the most expensive
|
|
call the app makes to MusicBrainz. It is warmed by exactly one thing:
|
|
`PrefetchReleases` (`explore.go:746`), capped at 8, called only when an
|
|
artist page renders.
|
|
|
|
An album opened from the library grid therefore always browses live.
|
|
|
|
## What to build
|
|
|
|
**A post-scan backfill that warms the release cache for release groups
|
|
that are owned but not known-complete** — bounded, resumable, and
|
|
shaped exactly like `BackfillLibraryDiscographies`, which is the proven
|
|
pattern for this in the codebase.
|
|
|
|
The scoping rule is the user's and it is the right one: not "every
|
|
album by every artist in the library" (50 release groups per artist,
|
|
mostly never opened) but albums with owned tracks — narrowed further,
|
|
now, to the ones a local answer cannot already cover. The query gains
|
|
one clause: skip release groups whose `GetAlbumCompleteness` reports
|
|
`complete`.
|
|
|
|
Sketch:
|
|
|
|
1. A query for release groups with ≥1 owned track and no warm release
|
|
cache entry. `release_groups.mbid` is the key; the owned-track join
|
|
is `audio_files → recordings → release_group_recordings`, the same
|
|
shape `unenrichedLibraryArtistMBIDs` already uses one table over.
|
|
2. Order by owned-track count descending, so the albums the user has
|
|
most of are warmed first — same reasoning as the discography
|
|
backfill's ordering, same benefit if a run is cut short.
|
|
3. Run through `releasesSF`, so it never double-fetches a release group
|
|
an interactive open is already handling.
|
|
4. Bound a run (`discogBackfillMaxPerRun` has a value to copy) and make
|
|
it resumable: the resume marker is the response cache itself —
|
|
`BrowseReleasesCached` already answers "is this one done", so unlike
|
|
the discography path this needs **no new flag column**.
|
|
5. Trigger it where `BackfillLibraryDiscographies` is triggered, and
|
|
register it with `jobs` so it has progress, pause and cancel like
|
|
every other long-running operation.
|
|
|
|
### The rate limiter is the whole design constraint
|
|
|
|
> **Update (2026-08-13): the priority half is built, and the sentence
|
|
> below is wrong on a detail.** `e.mb` runs on `mbSearchLimiter`
|
|
> (`NewRateLimiterBurst(3, 1)`); the 1 req/s `NewRateLimiter()` cited
|
|
> here is the *artist image* limiter. Both are shared and both were
|
|
> FIFO. `RateLimiter.WithBackgroundLane` + `WithBackgroundPriority(ctx)`
|
|
> now make a marked caller yield to interactive work and pace at 1/s,
|
|
> and `jobs.KindCatalogEnrich` + `startBackfillJob` give the existing
|
|
> backfills progress and cancel. **"Do not start until the priority
|
|
> question has an answer" is satisfied** — mark this backfill's context
|
|
> and register it the way `BackfillLibraryDiscographies` now is.
|
|
> `PrefetchReleases`' cap of 8 is still unrevisited.
|
|
|
|
One shared `NewRateLimiter()` at 1 req/s (`explore.go:84`) serves this,
|
|
`PrefetchReleases`, and every interactive browse. A backfill over a
|
|
few thousand owned albums is *hours* of wall clock at that rate — which
|
|
is fine for a background job, and not fine if it starves the album page
|
|
the user is looking at right now.
|
|
|
|
That is the real work in this plan, and it is not the query:
|
|
|
|
- Interactive browses need to **jump the queue**. Today they cannot;
|
|
there is one limiter and it is FIFO.
|
|
- `PrefetchReleases`' cap of 8 was sized when nothing else competed for
|
|
the limiter. Revisit it in the same change.
|
|
- The 60 s fallback the `AlbumReleasesFailed` fix installed is sized
|
|
for today's contention. If a backfill can queue behind it, that
|
|
number is wrong again — which is an argument for priority, not for a
|
|
bigger number.
|
|
|
|
Do not start the query until the priority question has an answer.
|
|
|
|
## The alternative that was considered and rejected
|
|
|
|
**Project release-group tracklists in the dump build and ship them in
|
|
the artifact.** The data is there: `canonical_musicbrainz_data.csv`
|
|
carries `release_mbid` *and* `recording_mbid`
|
|
(`dumpcatalog.go:520`), and `release_to_rg` already maps release →
|
|
release group. It is derivable from bytes the index build already
|
|
streams, with no new API surface at all, and it would work offline on
|
|
first launch with no per-user backfill.
|
|
|
|
It is rejected **for this plan** because the artifact is built
|
|
centrally and is byte-identical for every user, so "albums the user
|
|
owns a track of" cannot be a filter on it. Shipping tracklists for the
|
|
whole catalog means per-recording rows against a ~900 MB artifact
|
|
budget (~426 B/row measured), and gating on a popularity floor means it
|
|
is absent for exactly the obscure albums a local backfill would have
|
|
covered.
|
|
|
|
Worse than absent, in fact — and this is the argument that actually
|
|
kills it. The floor is not one number over artists; it is a **per
|
|
artist track budget** (`dumpcatalog.go:58-89`): 50 tracks for a tier-A
|
|
artist, 25 for tier B, 12 for tier C. A projected tracklist would
|
|
therefore be *whichever* of an album's tracks survived that budget,
|
|
with nothing marking the rest as absent — so the album page would count
|
|
owned against a truncated denominator and render "Play 7 of 9" for a
|
|
twelve-track album. That is a confident lie, where the honest states
|
|
this plan's alternative produces (complete / incomplete / unknown) are
|
|
at worst silent.
|
|
|
|
Note that `markLibraryArtists` (`dumpcatalog.go:246`) already grants
|
|
every library artist full coverage — 500 tracks, 100 release groups —
|
|
by reading the local library, so the per-user tailoring this option
|
|
supposedly cannot have does exist in code. It is a no-op in the CI
|
|
build (empty library), and reaching it means a **local** dump build:
|
|
the ~205 GB, half-a-day download the entire artifact design exists to
|
|
avoid. Whoever finds that function next should read this paragraph
|
|
before getting excited about it.
|
|
|
|
Worth revisiting if the artifact ever gains per-user tailoring, or if a
|
|
measurement shows the row count is smaller than feared. Note it also
|
|
yields the *canonical* tracklist rather than MusicBrainz's full version
|
|
list, so the versions dropdown would still browse live when opened.
|
|
|
|
## Done when
|
|
|
|
- Opening an owned album that has never been opened before renders its
|
|
catalog tracklist with no network call, after one backfill run.
|
|
- An interactive browse issued while the backfill is running is not
|
|
delayed by it.
|
|
- The backfill appears in the jobs indicator, and can be paused and
|
|
cancelled there.
|
|
- A second run after a completed one does approximately nothing.
|