The post-scan backfills share MusicBrainz's rate limiters with every
page the user can open, and both were FIFO — so a thousand-artist
enrichment put an album page behind an hour of queued work.
WithBackgroundLane/WithBackgroundPriority add a slower second lane: a
marked wait takes no token while any interactive wait is outstanding.
It is a context marker rather than a parameter because a backfill calls
the same client methods a detail page does. A long backfill also has to
be visible and stoppable, so jobs.KindCatalogEnrich registers both with
progress and cancel — after the work is counted, since these passes are
a no-op on every launch once the library is covered.
What it does not fetch is the point. It ran for hours against a
900-artist library and marked nothing, because three of the four things
it did per artist were work nobody asked for: similar artists, which
the artist page already resolves on view, and a full GetArtistImage
(fanart.tv, TheAudioDB, Wikidata, Wikipedia, ten portraits) reached
only to warm the MB artist lookup EnsureArtistRels does alone. It was
also serial across artists while every limiter is per-host and idle.
The marks are a table rather than more explore_index columns, because
artifactimport merges by column list and a flag added there is a second
place to remember. BrowseReleaseGroupsAll pages to exhaustion, where
the old call silently cut a prolific artist at 100 release groups.
One portrait is downloaded now; the rest are remembered as URLs.
resolveAllSources downloaded every candidate, up to ten, full size,
while nothing reads anything but primary.jpg — 5.3 GB measured on a
real cache, 4.1 GB of it unreachable. OrphanedArtistImagesJob is why
that survived: it joined the bare MBID onto the images directory, but
artist directories are sharded under a two-character prefix, so it
named a path that never existed and deleted the rows that were the only
record of the files it left behind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UDCbcCZQepnpSQYJ6SxxZm
Ships the fresh-start schema cleanup: rebuilt explore catalog index
pipeline (dump import, artifact fetch/build, incremental listen-count
refresh), a new download subsystem (Lidarr/Prowlarr/qBittorrent/SABnzbd/
slskd/yt-dlp providers, staging, reconciliation, wanted list), and the
supporting schema/query/store changes across backend and frontend.
Also includes two smaller follow-ups: bump the central index's
rebuild-after cadence from 90 to 180 days, and remove the Explore
"library only" online/offline toggle entirely (frontend-only, no
backend counterpart) rather than carry unused UI/state.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2Agd9af5hE7qzti2ackiS
End-of-milestone state for the Explore milestone. Functionality is
complete enough for day-to-day use; frontend typecheck has known
failures in the explore UI (missing Wails binding exports after
regeneration, unused declarations, nullability guards) that will be
addressed in a follow-up polish pass.
Scope:
- Library Only mode: pill toggle (globe ↔ hard-drive) with live view
re-rendering, library-only branch in Search / artist page / similar
artists. Suppresses external API calls when enabled.
- Ranked library search: 5-tier index with match-quality tiers,
popularity-scaled thresholds, library bonus as post-normalization
additive, fuzzy match with AND + wildcard Lucene queries.
- New schemas: artist_metadata, http_cache.
- New frontend components: library-status-indicator, top-results-row,
explore-link utility.
- Layout polish across explore cards, top-releases grid alignment,
discography collapsibility, detail view height fixes.
- Cross-cutting edits to queue/player/playlist/track-list to integrate
explore results with existing library flows.
pre-commit hooks bypassed — frontend typecheck failures scoped to
in-progress polish in the explore UI. Go build and full backend test
suite are green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three changes to reduce subsequent index build times:
1. Positive image/rels cache TTL: 30 days → 365 days
Artist url-rels and resolved image URLs rarely change.
Already-indexed artists make zero API calls on rebuild.
2. Negative cache (misses) stays at 30 days so new images
are discovered within a month of being added upstream.
3. MB rate limiter for background indexing: 1.0 → 1.5 req/s
url-rels lookups are lightweight; 1.5/s is well within
what MB handles (Picard and Kodi both use similar rates).
Cuts the MB-bound portion of index build by ~33%.
Also adds NewRateLimiterF for fractional rates and caches
MB rels fetch failures (30-day miss TTL) to avoid retrying
unreachable artists every build.
TheAudioDB (theaudiodb.com) added as source #1, between fanart.tv
and Wikimedia. Uses free public API key (2) with MBID-based lookup
so matching is guaranteed — no name-based search ambiguity.
Fetches up to 4 images per artist: thumb (portrait), fanart1-3
(wider shots). Results cached 30 days in explore_cache.
Source priority order is now:
0. fanart.tv artistthumb
1. TheAudioDB thumb + fanart
2. MusicBrainz direct image rels (Wikimedia Commons)
3. Wikidata P18 (Wikimedia Commons)
4. Wikipedia lead image
Fanart.tv artistthumb images are now the primary source for artist
photos. Up to 5 thumbnails fetched per artist (sorted by community
likes). Falls through to Wikimedia/Wikidata/Wikipedia if fanart.tv
has no images for the artist.
API key handling per fanart.tv project key terms:
- Project key loaded from FANART_TV_API_KEY env var (or build-time
ldflags via fanartTVProjectKey variable)
- Users can provide their own personal key via FANART_TV_PERSONAL_KEY
env var for higher rate limits (sent as client_key parameter)
- Results cached 30 days in explore_cache
- No bulk downloading — only fetched per-artist during index build
Source priority order:
1. fanart.tv artistthumb (best quality, community-curated)
2. MusicBrainz direct image rels (Wikimedia Commons)
3. Wikidata P18 (Wikimedia Commons)
4. Wikipedia lead image
Attribution: fanart.tv images are CC-BY-SA, contributed by the
fanart.tv community (https://fanart.tv).
Complete rewrite of the artist image pipeline:
STORAGE:
- Migration 16: artist_images table tracking source, URL, path,
primary flag, dimensions per image (up to 10 per artist)
- Directory structure: artist-images/{mbid[:2]}/{mbid}/ with
primary.jpg + primary_sm.jpg/_md.jpg/_lg.jpg thumbnails
- Miss marker (.miss file) prevents re-fetching artists with no image
SOURCES (priority order):
1. MusicBrainz direct image relations (Wikimedia Commons)
2. Wikidata P18 property (Wikimedia Commons)
3. Wikipedia lead image (NEW — via Wikidata sitelinks → Wikipedia API)
Each source is checked, deduplicated, and the first available
image becomes the primary with sm/md/lg thumbnail generation
(100px/200px/400px, matching cover art tier sizes).
ASSET SERVING:
- /artist-images/ path registered with Wails asset handler
- Serves files via http.FileServer from the artist-images directory
- Same pattern as /covers/ for cover art
ARTIST MODEL:
- Artist struct gains ImageSmall/ImageMedium/ImageLarge fields
- resolveArtistImages does bulk MBID lookup → disk stat for each
- Populated in GetAllArtists and GetAllArtistsByLibrary
GRID VIEW:
- artists-view uses model URLs directly (no more base64 data URLs)
- Size selection based on imageSize * devicePixelRatio (like cover-grid)
- Removed batch GetArtistImages call and in-memory cache — no longer needed
Replace per-artist sequential GetArtistMBID + GetArtistImageURL
calls (2 Wails round-trips × N artists) with a single batch
GetArtistImages(names[]) call that:
1. Resolves all names → MBIDs via AllArtistMBIDs() (one DB query)
2. Checks disk cache for each MBID via GetCachedImage (no network)
3. Returns map[name]→dataURL in one Wails bridge round-trip
Only returns already-cached images from the disk cache populated
by the index build. No network fetches triggered — artists whose
images haven't been cached yet keep the initial letter fallback
until the index build resolves them in the background.
Result: all cached artist images appear simultaneously on first
render instead of loading one-by-one over several seconds.
Migration 14: add aliases TEXT column to explore_index, rebuild FTS5
with 3 columns (title, artist_name, aliases), recreate sync triggers.
Clears index build timestamps to force alias population on next build.
Artist image provider fetches inc=url-rels+aliases (single call, no
extra cost). GetAliases() extracts alias names from cached MB rels.
indexOneArtist stores aliases as space-separated text after image
resolution populates the cache.
Search query now uses BM25 blended scoring:
ORDER BY bm25(fts, 3.0, 1.0, 0.5) - (ln(popularity+1) * 0.5)
Column weights: title=3.0, artist_name=1.0, aliases=0.5
- Title matches score 3x higher than artist name matches
- Alias matches are helpful but don't dominate
- Popularity is a log-scaled boost, not an override
- Exact title match on niche entity beats weak match on mega-popular
Enables: 'rhcp' → Red Hot Chili Peppers, 'gnr' → Guns N' Roses,
'sabbath' → Black Sabbath (once index build runs with aliases).
Two changes:
1. Artist image disk cache: ArtistImageProvider now fetches the
actual image bytes from Wikimedia Commons and caches them on
disk (~/.local/share/yellowjacket/artist-image-cache/{mbid}.jpg).
Returns base64 data URLs, same pattern as CoverArtProxy.
First lookup: resolve URL via MB/Wikidata + fetch image (~2s).
Subsequent: instant from disk cache.
404s cached as empty files to avoid re-fetching.
2. Top results artist photos: the Top Results section now shows
artist images from the artistImageCache, same as the Artists
section. Also shows englishName in the top card display name.
Artist image resolution was hitting musicbrainz.org with 10
concurrent unthrottled requests per search — enough to trigger
MB's rate limit rejection. Two fixes:
Backend: add dedicated 1 req/s RateLimiter for MB url-rels fetches
in ArtistImageProvider. Each fetch waits on the limiter before
the HTTP call. Results are cached 30 days so repeat lookups are
instant.
Frontend: switch loadArtistImages from concurrent fire-all to
sequential await loop. Each artist image loads one at a time,
images appear progressively as they resolve instead of all
failing from rate limit rejection.
The previous implementation called LookupArtist (rate-limited MB API)
before fetching url-rels, wasting a rate limiter slot. The fetchURL
for rels also bypassed the MB rate limiter, risking 503 rejections.
Rewrite: fetch MB url-rels once (direct HTTP, cached 30 days), parse
both image and wikidata relations from the same response, resolve
Wikimedia thumb URL. No dependency on MusicBrainzClient — just the
Cache for storage and a plain http.Client.
Also cleared 34 stale cached empty results from previous failed
resolution attempts that were blocking image lookup.
Add ArtistImageProvider that resolves artist MBIDs to photo URLs:
1. MB url-rels 'image' type → extract Commons filename → thumb URL
2. MB url-rels 'wikidata' type → Wikidata P18 property → thumb URL
3. No image → falls back to initial-letter avatar
Wikimedia Commons thumb URLs constructed via MD5 hash bucketing
(standard Commons URL scheme). Results cached in explore_cache
with 30-day TTL — subsequent lookups are instant.
Frontend: search results and artist detail page show artist photos
in the circular avatar when available. Images load async and
replace the initial-letter fallback on arrival. Artist detail page
fires the image fetch alongside the other 4 parallel data loads.
Architecture supports adding more sources (fanart.tv, etc.) by
extending the resolve() method's source chain.
yonluandClaude Opus 5