FullRescan scans the first library directly, then queues the rest.
The PostScan hook was restarting the index build after the FIRST
library, which starved the queued libraries for DB access — they
never scanned, leaving the library with only 6 tracks.
Fix: move StartIndexBuild to the OnAllScansComplete hook, which
fires when drainQueue finds no more libraries to scan. This ensures
ALL libraries finish scanning before the index build starts.
For startup soft scans: if no scans were queued (library unchanged),
start the index build directly. If scans WERE queued, the hook
handles it.
Added OnAllScansComplete callback to ScanHooks. Called from
drainQueue when the scan pipeline goes idle.
Track details dialog now shows:
1. Green checkmark badge next to the track title when the recording
has a MusicBrainz ID (hover: 'Metadata verified by MusicBrainz')
2. MusicBrainz section at the bottom with clickable MBID links for:
- Recording (track) → musicbrainz.org/recording/{mbid}
- Release Group (album) → musicbrainz.org/release-group/{mbid}
- Artist → musicbrainz.org/artist/{mbid}
Links open in the system browser. Only shown for entities that
have MBIDs from audio file tags.
Backend: GetTrackMBIDs(filePath) Wails binding queries recording,
release_group, and artist mbid columns via a single JOIN query.
Frontend: loaded async when the dialog opens, non-blocking.
Migration 14: add aliases TEXT column to explore_index, rebuild FTS5
with 3 columns (title, artist_name, aliases), recreate sync triggers.
Clears index build timestamps to force alias population on next build.
Artist image provider fetches inc=url-rels+aliases (single call, no
extra cost). GetAliases() extracts alias names from cached MB rels.
indexOneArtist stores aliases as space-separated text after image
resolution populates the cache.
Search query now uses BM25 blended scoring:
ORDER BY bm25(fts, 3.0, 1.0, 0.5) - (ln(popularity+1) * 0.5)
Column weights: title=3.0, artist_name=1.0, aliases=0.5
- Title matches score 3x higher than artist name matches
- Alias matches are helpful but don't dominate
- Popularity is a log-scaled boost, not an override
- Exact title match on niche entity beats weak match on mega-popular
Enables: 'rhcp' → Red Hot Chili Peppers, 'gnr' → Guns N' Roses,
'sabbath' → Black Sabbath (once index build runs with aliases).
The search index build and library scan both write to the same
single-connection SQLite DB. The index build runs continuous batch
transactions that can starve the scan's clearLibraryTables call,
causing the scan to silently hang without logging.
Fix: decouple index build from SetContext. The build now starts
AFTER the soft scan completes on startup. For full rescans, the
PreClear hook stops the index build, and PostScan restarts it.
Also made StartBuild/StopBuild safe for multiple calls:
- StartBuild is a no-op if already running
- StopBuild is a no-op if not running (no deadlock on done channel)
- done channel created per-build, not in constructor
The dhowden/tag library's Comm.Text and UFID.Identifier fields
include trailing null bytes from the C-style strings in the ID3v2
binary format. strings.TrimSpace doesn't strip \x00, so MBIDs
stored from MP3 files had invisible null bytes appended.
This caused WHERE mbid = ? queries to fail — the stored value
'uuid\x00' didn't match the clean 'uuid' from search results.
FLAC files (Vorbis comments with plain strings) were unaffected.
Fix: use strings.TrimRight with explicit \x00 in the cutset.
Requires a full rescan to fix existing corrupted MBIDs.
Add CheckLibraryMBIDs call after artist data loads. Checks all
release group MBIDs from the discography against the local library.
Matching albums show a green 'In Library' badge in the album meta
section alongside the year.
Fires after Promise.allSettled completes (same timing as artist
image fetch). Non-blocking — badge appears on re-render when the
check completes.
The dhowden/tag library returns ID3v2 TXXX frames as *tag.Comm
structs (key='TXXX_N', Description='MusicBrainz Artist Id',
Text='uuid'), not plain strings. Vorbis comments are plain strings
(key='musicbrainz_artistid', value='uuid').
Previous code only handled the string case — all MP3 files silently
got empty MBIDs. Now handles three value types:
- string: Vorbis comments (FLAC/OGG) — key is the tag name
- *tag.Comm: ID3v2 TXXX frames (MP3) — Description is the tag name
- *tag.UFID: ID3v2 UFID frame (MP3) — MusicBrainz recording ID
Requires a full rescan to backfill MBIDs for MP3 files.
updateMBIDs was calling l.db.ExecContext (main connection) while
inside a transaction that held the write lock. With SQLite's
SetMaxOpenConns(1), this deadlocked — the UPDATE waited for the
transaction to release the lock, but the transaction waited for
the UPDATE to complete.
Fix: pass *sql.Tx through processMetadata to updateMBIDs and use
tx.ExecContext instead. All MBID writes now happen within the same
transaction as the entity upserts.
Three features wired together:
1. 'In Library' badges on explore search results:
CheckLibraryMBIDs Wails binding batch-checks which search result
MBIDs exist in the local library. Green badges render on matching
artist cards and album cards.
2. Artist images on local artist-details page:
Local artist pages now call GetArtistMBID(name) to resolve the
MBID from tags, then GetArtistImageURL(mbid) to fetch the cached
Wikimedia photo. Falls back to initial-letter avatar.
3. Tier 3 search index uses direct MBIDs from tags:
buildTier3Library now reads artists.mbid column (from audio tags)
for direct MBID matching, falling back to name matching for
untagged artists. Eliminates false matches and catches artists
that name matching misses.
Migration 13 adds nullable mbid TEXT columns to artists,
release_groups, and recordings with partial indexes.
Metadata extraction (tags.go) now reads MusicBrainz IDs from Raw()
tags — handles both Vorbis (musicbrainz_artistid) and ID3v2
(MusicBrainz Artist Id) key formats.
Scan pipeline (library.go) updates MBIDs after entity upsert via
raw SQL UPDATE. Only sets mbid if currently NULL (preserves existing
values on rescan).
LibraryMBIDIndex (librarymbid.go) provides:
- CheckMBIDs: batch lookup for 'In Library' badges
- GetArtistMBID: single artist name→MBID lookup
- AllArtistMBIDs: full dump for search index Tier 3
MBIDs will be populated on next library rescan. Existing files
need a rescan to backfill.
Add nullable TEXT mbid column to artists, release_groups, and
recordings tables. Partial indexes on each (WHERE mbid IS NOT NULL)
for fast MBID lookups without bloating the index for rows without
MBIDs.
Enables linking local library entities to MusicBrainz/ListenBrainz
explore data, artist image sharing, and 'In Library' badges.
Restructure indexOneArtist to run LB discography fetches and MB
artist image resolution concurrently. They use different rate
limiters (LB: 3 req/s, MB: 1 req/s) so they overlap without
contention.
Per artist, the indexer now runs two parallel pipelines:
LB pipeline: top-release-groups + top-recordings
MB pipeline: url-rels → Wikidata P18 → Wikimedia image fetch
All artist images are pre-cached during the index build instead
of being resolved on-demand during search. Total build time
drops from ~105 min (sequential) to ~63 min (parallel, MB-bound).
SearchIndex now takes ArtistImageProvider as a dependency. The
Service constructor creates artistImg before the index so both
can share it.
Two changes:
1. Artist image disk cache: ArtistImageProvider now fetches the
actual image bytes from Wikimedia Commons and caches them on
disk (~/.local/share/yellowjacket/artist-image-cache/{mbid}.jpg).
Returns base64 data URLs, same pattern as CoverArtProxy.
First lookup: resolve URL via MB/Wikidata + fetch image (~2s).
Subsequent: instant from disk cache.
404s cached as empty files to avoid re-fetching.
2. Top results artist photos: the Top Results section now shows
artist images from the artistImageCache, same as the Artists
section. Also shows englishName in the top card display name.
Artist image resolution was hitting musicbrainz.org with 10
concurrent unthrottled requests per search — enough to trigger
MB's rate limit rejection. Two fixes:
Backend: add dedicated 1 req/s RateLimiter for MB url-rels fetches
in ArtistImageProvider. Each fetch waits on the limiter before
the HTTP call. Results are cached 30 days so repeat lookups are
instant.
Frontend: switch loadArtistImages from concurrent fire-all to
sequential await loop. Each artist image loads one at a time,
images appear progressively as they resolve instead of all
failing from rate limit rejection.
The previous implementation called LookupArtist (rate-limited MB API)
before fetching url-rels, wasting a rate limiter slot. The fetchURL
for rels also bypassed the MB rate limiter, risking 503 rejections.
Rewrite: fetch MB url-rels once (direct HTTP, cached 30 days), parse
both image and wikidata relations from the same response, resolve
Wikimedia thumb URL. No dependency on MusicBrainzClient — just the
Cache for storage and a plain http.Client.
Also cleared 34 stale cached empty results from previous failed
resolution attempts that were blocking image lookup.
Add ArtistImageProvider that resolves artist MBIDs to photo URLs:
1. MB url-rels 'image' type → extract Commons filename → thumb URL
2. MB url-rels 'wikidata' type → Wikidata P18 property → thumb URL
3. No image → falls back to initial-letter avatar
Wikimedia Commons thumb URLs constructed via MD5 hash bucketing
(standard Commons URL scheme). Results cached in explore_cache
with 30-day TTL — subsequent lookups are instant.
Frontend: search results and artist detail page show artist photos
in the circular avatar when available. Images load async and
replace the initial-letter fallback on arrival. Artist detail page
fires the image fetch alongside the other 4 parallel data loads.
Architecture supports adding more sources (fanart.tv, etc.) by
extending the resolve() method's source chain.
Replace per-card GetThumbnail calls (10 round-trips) with a single
GetThumbnails batch call that fetches all visible album thumbnails
in one Wails bridge round-trip.
Backend GetThumbnails accepts []ThumbnailRequest and returns
map[mbid]→dataURL. Each request still checks library → disk cache
→ CAA in order, but the bridge overhead is 1 call instead of 10.
Frontend fires loadThumbnails() once after search results arrive.
Album cards render immediately with CAA URL fallback, then re-render
once the batch resolves with cached/local data URLs.
CoverArtProxy now checks three sources in order:
1. Local library (instant) — matches by album+artist name against
the release_groups/cover_art tables. Albums the user already
owns show their local cover art immediately.
2. Disk cache (instant) — previously fetched CAA thumbnails.
3. Cover Art Archive (network) — fetches and caches to disk.
Library index is built once on first access (sync.Once) from a
single SQL query joining release_groups → cover_art → artists.
Keyed by lowercased 'album\x00artist' for exact name matching.
GetThumbnail now takes (mbid, albumName, artistName) so the proxy
can check the library before falling back to CAA. Frontend passes
the album title and artist credit from the search result.
The CAA proxy was caching all fetch failures (including 503s and
timeouts) as empty files, treating them as permanent 'no art' misses.
During Internet Archive outages, this meant every album got cached
as having no cover art, and the cache persisted after IA recovered.
Now only 404 responses (no cover art exists) are cached as permanent
misses. 503, timeouts, and other transient errors are not cached,
so the next request retries the fetch.
Also cleared 33 incorrectly cached 0-byte miss files from a
concurrent IA outage.
Add CoverArtProxy that fetches cover art from CAA, caches the image
bytes on disk (~/.local/share/yellowjacket/cover-art-cache/), and
returns base64 data URLs via the GetThumbnail Wails binding.
First load: fetches from CAA (rate-limited), caches to disk.
Subsequent loads: instant from disk cache, no network.
404s: cached as empty files to avoid re-fetching.
Frontend explore-view loads thumbnails async via GetThumbnail()
calls that fire during render. Cached thumbnails appear as data
URLs directly in img src, bypassing the browser's HTTP stack.
Uncached thumbnails fall back to the CAA URL while the proxy
fetches in the background, then re-render with the cached version.
Also stores caa_id and caa_release_mbid in the search index's
extra_json for future direct Internet Archive URL construction.
Phases 2 (3 LB popularity POST calls) and 3 (3 MB discography
browse calls) were adding ~3-6 seconds to every search through
rate-limited API calls. Now they only run as a fallback during
first launch before the search index is built.
Once the index is ready (after Tier 1, <5 seconds from startup):
Phase 0: local FTS5 index query (instant)
Phase 1: MB search (3 concurrent calls, ~1s)
Phase 4: merge index hits (instant)
Phase 5: filter and cap (instant)
Search drops from ~4-7s to ~1s. The index already carries
popularity data and covers discography cross-referencing,
making the live API calls redundant.
Replace the single 7-day full rebuild with per-tier scheduling:
- Tier 1 (sitewide top lists): weekly refresh, 12 API calls
- Tiers 2-4 (discographies): monthly refresh, incremental —
only fetches discographies for artists not already indexed
On subsequent runs:
- If T1 is fresh, load cached artists from the index (~0 calls)
- If discographies are fresh, skip Tiers 2-4 entirely (~0 calls)
- If discographies are stale, diff against indexed set and only
fetch new artists that appeared in the sitewide lists
Add helpers: isMetaFresh (per-key freshness check),
loadCachedSitewideArtists (read artists from existing index),
filterUnindexed (diff artist list against indexed set).
After first build: typical startup is <5s (T1 cache load).
Monthly incremental: ~50-100 calls for newly appeared artists.
Instead of fixed 20 RGs + 100 recordings for every artist, scale
the budget by popularity using a power curve (exponent 0.3):
Radiohead (2.5M listens): 20 RGs, 100 recordings
Hans Zimmer (715K): 15 RGs, 71 recordings
Clutch (178K): 11 RGs, 50 recordings
Similar (~10K): 7 RGs, 27 recordings
Organic (unknown): 5 RGs, 10 recordings
Saves ~53% index size (~29 MB vs ~62 MB) with identical API calls.
The savings come from T4 similar artists (long tail) where full
discographies were wasteful. Top artists still get full coverage.
Rewrite SearchIndex with tiered background build:
Tier 1 — Sitewide instant (<5s, 12 calls): top artists, recordings,
and release groups across 4 time ranges. Searchable immediately.
Tier 2 — Sitewide full discog (~16min, 2881 calls): top 20 RGs +
top 100 recordings per sitewide artist (~1440 unique artists from
all_time/this_year/this_month/this_week union).
Tier 3 — Library artists (~4min, 664 calls): match local library
artist names against known MBIDs, index their full discographies.
Catches the user's personal taste that sitewide misses.
Tier 4 — Similar artists (~24min, ~4300 calls): fetch similar
artists from LB labs for each library artist, index their
discographies. Fans out into the user's taste neighborhood.
Tier 5 — Organic growth (0 calls): BrowseReleaseGroups now writes
to the search index in a background goroutine. Every artist page
view adds that artist's discography to the index for free.
Other changes:
- indexRGsPerArtist bumped 10→20 (96% vs 88% coverage)
- indexRecsPerArtist bumped 10→100 (track-name searchability)
- indexMinPopularity = 50 (cuts noise from long tails)
- Dedicated 3 req/s rate limiter for indexer
- Labs similar-artists endpoint at labs.api.listenbrainz.org
- Each tier marks index as ready on completion so search improves
progressively during the ~44min total build
Create SearchIndex in NewExploreService, start background build in
SetContext (on app startup). Search() now has 6 phases:
Phase 0: query local FTS5 index (instant, no API calls)
Phase 1: concurrent MB search
Phase 2: LB popularity boost
Phase 3: cross-reference artist discographies
Phase 4: merge index hits (prepend new entries, dedup by MBID)
Phase 5: filter and cap
Index hits for release groups/recordings not already in MB results
are prepended so popular albums surface even when MB search can't
find them. scalePopularity() maps raw listen counts to 0-100 scores
via log scaling for compatibility with the blended score system.
New SearchIndex struct in searchindex.go:
- Background build fetches top 1000 LB artists, then their top 10
release groups and top 10 recordings (2001 API calls total)
- Dedicated 3 req/s rate limiter for indexer (LB allows 30/10s)
- Bounded concurrency (3 goroutines) with progress logging
- Batch INSERTs in transactions of 100 rows
- FTS5 query with prefix matching ('for you' → 'for* you*')
- Results sorted by popularity descending
- Skips rebuild if index is < 7 days old
- Marks index ready from existing rows if build fails
- Context cancellation for clean shutdown
Add explore_index table (entity_type, mbid, title, artist_name,
artist_mbid, popularity, extra_json) with a unique index on
(entity_type, mbid). FTS5 virtual table explore_index_fts backed
by the content table with auto-sync triggers for insert/update/delete.
explore_index_meta table tracks build timestamps.
After MB search + popularity reranking, browse the discographies of
the top 3 artists and fuzzy-match the full query against album titles.
Matching albums not already in results are injected at the front.
Fuzzy matching uses substring containment with word-level ratio
(handles 'for you tatsuro' → 'FOR YOU' at 0.667) and word overlap
as fallback. Threshold: 0.4 ratio.
Example: 'for you tatsuro' now finds FOR YOU by 山下達郎 even though
MB text search treats 'for' and 'you' as stop words and never
returns it. The album is found via Yamashita's cached discography.
Extract primary English alias from MusicBrainz artist data when the
canonical name uses non-Latin script (CJK, Cyrillic, etc.). Display
it as the primary name in search results and artist detail header,
with the native script name as a subtitle beneath.
Example: 山下達郎 now shows 'Tatsuro Yamashita' prominently with
'山下達郎' as a subtitle. Artists with Latin names are unchanged.
Drop artists and recordings with blended score < 25 after popularity
reranking. Cap each entity slice to 15 server-side. Request 20 from
MB to allow filtering headroom. Reduces payload size and noise.
Don't fire search for single-character queries — show 'Keep typing…'
instead. Cap rendered results at 10 per section (artists, albums,
tracks) to reduce noise. Top results already capped at 3.
After MB search returns text-relevance-scored results, fetch bulk
popularity data from ListenBrainz (POST /1/popularity/{artist,
recording,release-group}) for all result MBIDs. Blend scores:
final = 0.6 * mb_relevance + 0.4 * log10_popularity
Log-scale normalization ensures massive artists don't drown out
everything, but popular results rise above obscure exact matches.
Release groups (no MB score) sort by raw popularity.
Three LB POST calls run concurrently — each hits a different
endpoint. All are rate-limited and cached (24h TTL).
Example: searching 'tatsuro' now ranks Tatsuro Yamashita (2.5M LB
listens, score 97) above 'tatsuro' vocaloid producer (4 listens,
score 64) despite the latter being an exact name match on MB.
Use MusicBrainz secondaryTypes to distinguish studio albums from
compilations, soundtracks, live albums, remixes, etc. Albums with
no non-studio secondary types show under 'Albums'; everything else
under 'Other Albums'. EPs and Singles remain their own sections.
Section order: Albums → EP → Single → Other Albums → ...rest.
The ListenBrainz popularity API returns snake_case JSON fields
(recording_name, artist_name, total_listen_count, recording_mbid)
but LBTopRecording used camelCase JSON tags for Wails serialization.
All fields silently deserialized as zero values — empty strings and
zero counts — producing ~8000 blank rows in the top tracks section.
Fix: add lbTopRecordingWire with snake_case tags for API unmarshal,
convert to LBTopRecording (camelCase) for Wails. Cap results at 10
to avoid rendering thousands of rows for prolific artists.
The mapTrackRow signature was extended with playCount and lastPlayed
fields in the play history feature, but the scan_test.go callers
were not updated, breaking go vet and golangci-lint.
When seeking, the underlying decoder position was updated but the
BufferedStreamer's ring buffer still contained up to 2 seconds of
pre-seek audio data. The speaker would drain this stale buffer
before playing audio from the new position, causing an audible
delay where the old position's audio continued playing.
Add a Flush() method to BufferedStreamer that resets the ring buffer
pointers, and call it in seekLocked() immediately after a successful
seek. This ensures the speaker starts playing from the seeked
position without any stale audio artifact.
yonlu