Migration 13 adds nullable mbid TEXT columns to artists,
release_groups, and recordings with partial indexes.
Metadata extraction (tags.go) now reads MusicBrainz IDs from Raw()
tags — handles both Vorbis (musicbrainz_artistid) and ID3v2
(MusicBrainz Artist Id) key formats.
Scan pipeline (library.go) updates MBIDs after entity upsert via
raw SQL UPDATE. Only sets mbid if currently NULL (preserves existing
values on rescan).
LibraryMBIDIndex (librarymbid.go) provides:
- CheckMBIDs: batch lookup for 'In Library' badges
- GetArtistMBID: single artist name→MBID lookup
- AllArtistMBIDs: full dump for search index Tier 3
MBIDs will be populated on next library rescan. Existing files
need a rescan to backfill.
Add nullable TEXT mbid column to artists, release_groups, and
recordings tables. Partial indexes on each (WHERE mbid IS NOT NULL)
for fast MBID lookups without bloating the index for rows without
MBIDs.
Enables linking local library entities to MusicBrainz/ListenBrainz
explore data, artist image sharing, and 'In Library' badges.
Restructure indexOneArtist to run LB discography fetches and MB
artist image resolution concurrently. They use different rate
limiters (LB: 3 req/s, MB: 1 req/s) so they overlap without
contention.
Per artist, the indexer now runs two parallel pipelines:
LB pipeline: top-release-groups + top-recordings
MB pipeline: url-rels → Wikidata P18 → Wikimedia image fetch
All artist images are pre-cached during the index build instead
of being resolved on-demand during search. Total build time
drops from ~105 min (sequential) to ~63 min (parallel, MB-bound).
SearchIndex now takes ArtistImageProvider as a dependency. The
Service constructor creates artistImg before the index so both
can share it.
Two changes:
1. Artist image disk cache: ArtistImageProvider now fetches the
actual image bytes from Wikimedia Commons and caches them on
disk (~/.local/share/yellowjacket/artist-image-cache/{mbid}.jpg).
Returns base64 data URLs, same pattern as CoverArtProxy.
First lookup: resolve URL via MB/Wikidata + fetch image (~2s).
Subsequent: instant from disk cache.
404s cached as empty files to avoid re-fetching.
2. Top results artist photos: the Top Results section now shows
artist images from the artistImageCache, same as the Artists
section. Also shows englishName in the top card display name.
Artist image resolution was hitting musicbrainz.org with 10
concurrent unthrottled requests per search — enough to trigger
MB's rate limit rejection. Two fixes:
Backend: add dedicated 1 req/s RateLimiter for MB url-rels fetches
in ArtistImageProvider. Each fetch waits on the limiter before
the HTTP call. Results are cached 30 days so repeat lookups are
instant.
Frontend: switch loadArtistImages from concurrent fire-all to
sequential await loop. Each artist image loads one at a time,
images appear progressively as they resolve instead of all
failing from rate limit rejection.
The previous implementation called LookupArtist (rate-limited MB API)
before fetching url-rels, wasting a rate limiter slot. The fetchURL
for rels also bypassed the MB rate limiter, risking 503 rejections.
Rewrite: fetch MB url-rels once (direct HTTP, cached 30 days), parse
both image and wikidata relations from the same response, resolve
Wikimedia thumb URL. No dependency on MusicBrainzClient — just the
Cache for storage and a plain http.Client.
Also cleared 34 stale cached empty results from previous failed
resolution attempts that were blocking image lookup.
Add ArtistImageProvider that resolves artist MBIDs to photo URLs:
1. MB url-rels 'image' type → extract Commons filename → thumb URL
2. MB url-rels 'wikidata' type → Wikidata P18 property → thumb URL
3. No image → falls back to initial-letter avatar
Wikimedia Commons thumb URLs constructed via MD5 hash bucketing
(standard Commons URL scheme). Results cached in explore_cache
with 30-day TTL — subsequent lookups are instant.
Frontend: search results and artist detail page show artist photos
in the circular avatar when available. Images load async and
replace the initial-letter fallback on arrival. Artist detail page
fires the image fetch alongside the other 4 parallel data loads.
Architecture supports adding more sources (fanart.tv, etc.) by
extending the resolve() method's source chain.
Replace per-card GetThumbnail calls (10 round-trips) with a single
GetThumbnails batch call that fetches all visible album thumbnails
in one Wails bridge round-trip.
Backend GetThumbnails accepts []ThumbnailRequest and returns
map[mbid]→dataURL. Each request still checks library → disk cache
→ CAA in order, but the bridge overhead is 1 call instead of 10.
Frontend fires loadThumbnails() once after search results arrive.
Album cards render immediately with CAA URL fallback, then re-render
once the batch resolves with cached/local data URLs.
CoverArtProxy now checks three sources in order:
1. Local library (instant) — matches by album+artist name against
the release_groups/cover_art tables. Albums the user already
owns show their local cover art immediately.
2. Disk cache (instant) — previously fetched CAA thumbnails.
3. Cover Art Archive (network) — fetches and caches to disk.
Library index is built once on first access (sync.Once) from a
single SQL query joining release_groups → cover_art → artists.
Keyed by lowercased 'album\x00artist' for exact name matching.
GetThumbnail now takes (mbid, albumName, artistName) so the proxy
can check the library before falling back to CAA. Frontend passes
the album title and artist credit from the search result.
The CAA proxy was caching all fetch failures (including 503s and
timeouts) as empty files, treating them as permanent 'no art' misses.
During Internet Archive outages, this meant every album got cached
as having no cover art, and the cache persisted after IA recovered.
Now only 404 responses (no cover art exists) are cached as permanent
misses. 503, timeouts, and other transient errors are not cached,
so the next request retries the fetch.
Also cleared 33 incorrectly cached 0-byte miss files from a
concurrent IA outage.
Add CoverArtProxy that fetches cover art from CAA, caches the image
bytes on disk (~/.local/share/yellowjacket/cover-art-cache/), and
returns base64 data URLs via the GetThumbnail Wails binding.
First load: fetches from CAA (rate-limited), caches to disk.
Subsequent loads: instant from disk cache, no network.
404s: cached as empty files to avoid re-fetching.
Frontend explore-view loads thumbnails async via GetThumbnail()
calls that fire during render. Cached thumbnails appear as data
URLs directly in img src, bypassing the browser's HTTP stack.
Uncached thumbnails fall back to the CAA URL while the proxy
fetches in the background, then re-render with the cached version.
Also stores caa_id and caa_release_mbid in the search index's
extra_json for future direct Internet Archive URL construction.
Phases 2 (3 LB popularity POST calls) and 3 (3 MB discography
browse calls) were adding ~3-6 seconds to every search through
rate-limited API calls. Now they only run as a fallback during
first launch before the search index is built.
Once the index is ready (after Tier 1, <5 seconds from startup):
Phase 0: local FTS5 index query (instant)
Phase 1: MB search (3 concurrent calls, ~1s)
Phase 4: merge index hits (instant)
Phase 5: filter and cap (instant)
Search drops from ~4-7s to ~1s. The index already carries
popularity data and covers discography cross-referencing,
making the live API calls redundant.
Replace the single 7-day full rebuild with per-tier scheduling:
- Tier 1 (sitewide top lists): weekly refresh, 12 API calls
- Tiers 2-4 (discographies): monthly refresh, incremental —
only fetches discographies for artists not already indexed
On subsequent runs:
- If T1 is fresh, load cached artists from the index (~0 calls)
- If discographies are fresh, skip Tiers 2-4 entirely (~0 calls)
- If discographies are stale, diff against indexed set and only
fetch new artists that appeared in the sitewide lists
Add helpers: isMetaFresh (per-key freshness check),
loadCachedSitewideArtists (read artists from existing index),
filterUnindexed (diff artist list against indexed set).
After first build: typical startup is <5s (T1 cache load).
Monthly incremental: ~50-100 calls for newly appeared artists.
Instead of fixed 20 RGs + 100 recordings for every artist, scale
the budget by popularity using a power curve (exponent 0.3):
Radiohead (2.5M listens): 20 RGs, 100 recordings
Hans Zimmer (715K): 15 RGs, 71 recordings
Clutch (178K): 11 RGs, 50 recordings
Similar (~10K): 7 RGs, 27 recordings
Organic (unknown): 5 RGs, 10 recordings
Saves ~53% index size (~29 MB vs ~62 MB) with identical API calls.
The savings come from T4 similar artists (long tail) where full
discographies were wasteful. Top artists still get full coverage.
Rewrite SearchIndex with tiered background build:
Tier 1 — Sitewide instant (<5s, 12 calls): top artists, recordings,
and release groups across 4 time ranges. Searchable immediately.
Tier 2 — Sitewide full discog (~16min, 2881 calls): top 20 RGs +
top 100 recordings per sitewide artist (~1440 unique artists from
all_time/this_year/this_month/this_week union).
Tier 3 — Library artists (~4min, 664 calls): match local library
artist names against known MBIDs, index their full discographies.
Catches the user's personal taste that sitewide misses.
Tier 4 — Similar artists (~24min, ~4300 calls): fetch similar
artists from LB labs for each library artist, index their
discographies. Fans out into the user's taste neighborhood.
Tier 5 — Organic growth (0 calls): BrowseReleaseGroups now writes
to the search index in a background goroutine. Every artist page
view adds that artist's discography to the index for free.
Other changes:
- indexRGsPerArtist bumped 10→20 (96% vs 88% coverage)
- indexRecsPerArtist bumped 10→100 (track-name searchability)
- indexMinPopularity = 50 (cuts noise from long tails)
- Dedicated 3 req/s rate limiter for indexer
- Labs similar-artists endpoint at labs.api.listenbrainz.org
- Each tier marks index as ready on completion so search improves
progressively during the ~44min total build
Create SearchIndex in NewExploreService, start background build in
SetContext (on app startup). Search() now has 6 phases:
Phase 0: query local FTS5 index (instant, no API calls)
Phase 1: concurrent MB search
Phase 2: LB popularity boost
Phase 3: cross-reference artist discographies
Phase 4: merge index hits (prepend new entries, dedup by MBID)
Phase 5: filter and cap
Index hits for release groups/recordings not already in MB results
are prepended so popular albums surface even when MB search can't
find them. scalePopularity() maps raw listen counts to 0-100 scores
via log scaling for compatibility with the blended score system.
New SearchIndex struct in searchindex.go:
- Background build fetches top 1000 LB artists, then their top 10
release groups and top 10 recordings (2001 API calls total)
- Dedicated 3 req/s rate limiter for indexer (LB allows 30/10s)
- Bounded concurrency (3 goroutines) with progress logging
- Batch INSERTs in transactions of 100 rows
- FTS5 query with prefix matching ('for you' → 'for* you*')
- Results sorted by popularity descending
- Skips rebuild if index is < 7 days old
- Marks index ready from existing rows if build fails
- Context cancellation for clean shutdown
Add explore_index table (entity_type, mbid, title, artist_name,
artist_mbid, popularity, extra_json) with a unique index on
(entity_type, mbid). FTS5 virtual table explore_index_fts backed
by the content table with auto-sync triggers for insert/update/delete.
explore_index_meta table tracks build timestamps.
After MB search + popularity reranking, browse the discographies of
the top 3 artists and fuzzy-match the full query against album titles.
Matching albums not already in results are injected at the front.
Fuzzy matching uses substring containment with word-level ratio
(handles 'for you tatsuro' → 'FOR YOU' at 0.667) and word overlap
as fallback. Threshold: 0.4 ratio.
Example: 'for you tatsuro' now finds FOR YOU by 山下達郎 even though
MB text search treats 'for' and 'you' as stop words and never
returns it. The album is found via Yamashita's cached discography.
Extract primary English alias from MusicBrainz artist data when the
canonical name uses non-Latin script (CJK, Cyrillic, etc.). Display
it as the primary name in search results and artist detail header,
with the native script name as a subtitle beneath.
Example: 山下達郎 now shows 'Tatsuro Yamashita' prominently with
'山下達郎' as a subtitle. Artists with Latin names are unchanged.
Drop artists and recordings with blended score < 25 after popularity
reranking. Cap each entity slice to 15 server-side. Request 20 from
MB to allow filtering headroom. Reduces payload size and noise.
Don't fire search for single-character queries — show 'Keep typing…'
instead. Cap rendered results at 10 per section (artists, albums,
tracks) to reduce noise. Top results already capped at 3.
After MB search returns text-relevance-scored results, fetch bulk
popularity data from ListenBrainz (POST /1/popularity/{artist,
recording,release-group}) for all result MBIDs. Blend scores:
final = 0.6 * mb_relevance + 0.4 * log10_popularity
Log-scale normalization ensures massive artists don't drown out
everything, but popular results rise above obscure exact matches.
Release groups (no MB score) sort by raw popularity.
Three LB POST calls run concurrently — each hits a different
endpoint. All are rate-limited and cached (24h TTL).
Example: searching 'tatsuro' now ranks Tatsuro Yamashita (2.5M LB
listens, score 97) above 'tatsuro' vocaloid producer (4 listens,
score 64) despite the latter being an exact name match on MB.
Use MusicBrainz secondaryTypes to distinguish studio albums from
compilations, soundtracks, live albums, remixes, etc. Albums with
no non-studio secondary types show under 'Albums'; everything else
under 'Other Albums'. EPs and Singles remain their own sections.
Section order: Albums → EP → Single → Other Albums → ...rest.
The ListenBrainz popularity API returns snake_case JSON fields
(recording_name, artist_name, total_listen_count, recording_mbid)
but LBTopRecording used camelCase JSON tags for Wails serialization.
All fields silently deserialized as zero values — empty strings and
zero counts — producing ~8000 blank rows in the top tracks section.
Fix: add lbTopRecordingWire with snake_case tags for API unmarshal,
convert to LBTopRecording (camelCase) for Wails. Cap results at 10
to avoid rendering thousands of rows for prolific artists.
The mapTrackRow signature was extended with playCount and lastPlayed
fields in the play history feature, but the scan_test.go callers
were not updated, breaking go vet and golangci-lint.
When seeking, the underlying decoder position was updated but the
BufferedStreamer's ring buffer still contained up to 2 seconds of
pre-seek audio data. The speaker would drain this stale buffer
before playing audio from the new position, causing an audible
delay where the old position's audio continued playing.
Add a Flush() method to BufferedStreamer that resets the ring buffer
pointers, and call it in seekLocked() immediately after a successful
seek. This ensures the speaker starts playing from the seeked
position without any stale audio artifact.
- Column accessor shows '0' instead of empty string for unplayed tracks
- recordPlay emits TrackMetadataChanged event after updating DB
- Frontend library store invalidates on that event, refreshing play counts
Backend validation rejected 'playCount' as unknown column ID, silently
preventing it from being enabled or persisted. Added ColPlayCount to
the tracklist package's constant list and AllColumnIDs slice.
COALESCE(last_played, '') returned empty string which can't scan into
time.Time. Removed COALESCE, use sql.NullTime instead. Format to string
only when Valid.
Backend:
- GetAllTracksWithFullMetadata queries now select play_count and last_played
- mapTrackRow accepts and passes through PlayCount/LastPlayed
- PlayCount + LastPlayed added to library.Track struct
- sqlcgen Row types updated with new fields
Frontend:
- PlayCount + LastPlayed added to library.Track TypeScript model
- 'Plays' column added to track-list column definitions (60px, right-aligned, sortable)
Queries without play data (search, genre, album) pass 0/empty defaults.
Backend:
- Added play_count and days_since_played to rule engine field whitelist
- days_since_played uses julianday() expression with COALESCE for NULL handling
- Never-played tracks (NULL last_played) match 'greater_than' but not 'less_than'
- Added PlayCount + LastPlayed to library.Track struct
- Evaluate query selects play_count and last_played from track_metadata
- Added play_count to sort field options
Frontend:
- Added play_count and days_since_played to field and numeric field lists
- Added play_count to sort options
All 49 rule engine + 15 service tests pass unchanged.
Migration 10:
- play_history table (audio_file_id FK, played_at DATETIME, CASCADE delete)
- play_count + last_played columns on audio_files (denormalized)
- Recreated track_metadata VIEW with play_count and last_played columns
Play recording:
- queue.recordPlay() inserts play_history row + updates denormalized columns
- Called from OnPlaybackFinished after queue advance completes
- Mutex released before DB write to avoid MaxOpenConns(1) deadlock
- Natural finish only — skip/stop does not count
Tests:
- TestMigration10PlayHistory: schema, columns, VIEW, round-trip verification
- All 49 smart playlist + 15 service + existing DB tests still pass
- Grid rows use align-items: stretch so all cells share the select's height
- Combobox host, wrapper, and input all set height: 100% to fill the cell
- Remove button uses align-self: center to stay centered instead of stretching
Details view:
- Skip evaluation on new playlists (was returning all 25k tracks)
- Go straight to editor on auto-edit instead of awaiting loadTracks
Editor:
- Default sort to Random instead of None, remove None option
- Hide sort direction button when sort is Random
- Fixed-width field (160px) and operator (140px) columns, value fills remaining space
- Preview fills remaining vertical space (flex layout) instead of fixed 200px max-height
- Preview scrolls independently while rules/options stay pinned
Combobox:
- Accept free-form text on blur and Enter — no longer requires selection from dropdown
- Enables typing values like 'indie' that may not be exact DB entries
Backend:
- Case-insensitive text matching: COLLATE NOCASE on is/is_not/is_any_of operators
- Applies to both regular fields and genre subqueries
- LIKE operators were already case-insensitive (SQLite default)
- Add blank lines before cuddled expressions in wav_test.go
- Fix cuddled copy expression in wav.go writeRIFF
- All WAV lint issues resolved (remaining are pre-existing in other files)
- createTestWAV: builds minimal valid WAV with optional ID3v2 tag
- readWavID3Tags: extracts ID3v2 from RIFF via parseRIFF + dhowden/tag
- createTestWAVWithExtraChunks: adds LIST INFO and bext chunks for preservation tests
- makePCMFmtData: generates 16-byte PCM format data
Archive v1.2 milestone: ROADMAP + REQUIREMENTS + phases to milestones/.
Evolve PROJECT.md with v1.2 validated requirements and key decisions.
Update RETROSPECTIVE.md with v1.2 lessons and cross-milestone trends.
Clean STATE.md for next milestone.
- track-list: branch on selection count, resolve tracks from this.tracks
- cover-grid: branch on selection count, resolve from expandedTracks
- queue-panel: branch on indices count, resolve via libraryStore.getCachedTracks
- playlist-details: branch on selection count, resolve via libraryStore
- Each view determines coverArt/coverArtMixed state and calls showBatch()
- showBatch() API for multi-track entry with merged field values
- Three-state field model: keep/set/clear via dirty tracking in editValues
- Confirmation dialog showing field changes before batch save
- Progress bar with live counter wired to BatchWriteProgress events
- Cancel button calling CancelBatchWrite during batch write
- Results view with success/failure counts and expandable failure details
- Batch cover art: pick, preview, or clear for all selected tracks
- Post-save data refresh returning to updated summary view
- Single-track show() path unchanged (batchMode = false)
- Add BatchFailure and BatchResult types for structured batch outcomes
- Add cancelBatch channel and suppressEvents flag to TagWriter struct
- Add CancelBatchWrite method for mid-batch cancellation from frontend
- Add BatchWriteTrackTags method processing tracks sequentially
- Emit BatchWriteProgress event per-track with current/total/succeeded/failed
- Suppress per-track TrackMetadataChanged; emit single event after batch
- Wails bindings auto-generated for BatchWriteTrackTags and CancelBatchWrite
- tagwriter namespace with BatchResult/BatchFailure in models.ts
After a successful save, re-fetch albums alongside tracks and re-resolve
cover art URLs from the updated album data. Previously the dialog only
refreshed this.track but kept stale this.coverArt URLs pointing to the
old content-hash files, causing the image to revert until reopen.
Three issues fixed:
1. asBytes() helper for []interface{} → []byte conversion — same
float64 deserialization issue as numeric fields. Cover art data
from the frontend arrives as []interface{} of float64, not []byte.
2. DB sync for cover art — was a placeholder no-op. Now saves image
to covers cache dir (content-hash dedup + thumbnail generation),
upserts cover_art row, and updates release_groups.cover_art_id.
Clear sets cover_art_id to NULL on linked release groups.
3. Frontend ReadFile returns base64 string (Go []byte JSON encoding),
not number[]. Decode with atob() before creating Uint8Array for
preview blob URL.
Wails deserializes JSON numbers from JavaScript as float64, not int.
All .(int) type assertions on year, track_number, and disc_number
silently failed. Add asInt() helper, replace all assertions.
After saveEdit() succeeds, re-fetch tracks from library store and update
this.track with the fresh data so the dialog shows updated values instead
of the stale snapshot passed via show().
- Wire saveEdit() to WriteTrackTagsByPath with diff-only TagChanges map
- Add cover art selection via ImageFilePicker with instant blob preview
- Add cover art removal (× button) with clearCoverArt state
- Show saving indicator and disable buttons during save
- Display errors inline in action bar; edit mode stays active on error
- Add ReadFile Go method on FrontendUtil to read cover art bytes
- Add Wails binding for ReadFile
- Clean up all edit state (editValues, pendingCoverArt, errorMessage) on close/cancel
- Created 17-01-SUMMARY.md
- Updated STATE.md with position, decisions, metrics
- Updated ROADMAP.md with plan progress
- Marked EDIT-01 and EDIT-04 complete in REQUIREMENTS.md
yonlu