yonlu/yellowjacket
1
1
Fork 0
Code Issues 48 Pull Requests Actions Packages Projects Releases 13 Wiki Activity

feat(database): shape the library like files, and shrink the catalog
CI / check (push) Successful in 3m7s
Details
CI / e2e (push) Canceled after 1m45s
Details

Browse Source 
Plans 013 and 014, the album page that prompted them, and the smaller
fixes they turned up. Changelog, largest first.

## The local library is shaped like files, not like MusicBrainz

`audio_files` carries its own tags and points at `albums` and
`artists`; `file_genres` is the one real many-to-many. `recordings`,
`release_group_recordings`, `artist_credit`, `artist_credit_artist`,
`recording_genres`, `release_groups` and `release_to_rg` are gone from
the local side, and with them a six-way join in every read, a
`MIN(release_group_id)` subquery in eleven queries and a
first-credited-artist subquery in nine. Measured on a real 25,966-file
library, every many-to-many that model expressed was 1:1 in the data.

- Ownership is a file. `GetFilePathsByRecordingMBIDs`,
  `LibraryMBIDIndex.CheckMBIDs`, `collectLibraryEntities` and
  `pruneStaleLocalCrossReferences` all join `audio_files`, so the 812
  orphaned recordings, 216 release groups and 260 artists that library
  carried are now structurally impossible.
- One projection: every track query selects from the `track_metadata`
  view, one row type, one mapper. Nine hand-rolled copies had drifted
  far enough to report different years on different screens.
- `library_id = 0` means every library, so each list query exists once
  instead of scoped and unscoped with a branch at every call site.
- No migration chain. `sql/schemas/` is the one description of the
  shape; `sql/migrations/`, `applyMigrations` and `schema_migrations`
  are squashed away, along with the drift between them that had sqlc
  generating against a stale schema.
- `database.InsertTestTrack` is the one test seeder; twenty test files
  had been assembling the old FK chain each in its own order.

## The catalog stores its ids as bytes

`explore_index`'s three 36-char MBID columns and its entity-type text
are 16 raw bytes and a small integer. The table and its six indexes go
780 MB to 405 MB on a real 2,052,200-row catalog, which is why a fresh
install is ~0.6 GB rather than ~1.0 GB.

- `backend/explore/mbid.go` is the only place the encoding is known;
  everything above it speaks dashed strings.
- `CHECK(length(mbid) = 16)` makes a stringly write fail at the insert
  rather than silently returning no rows, since SQLite does not coerce
  between TEXT and BLOB.
- The importer asks the artifact what encoding it carries and converts
  on the way in, so the artifact already published keeps working and no
  format bump is needed.
- `indexRowColumns`/`scanIndexRow` replace four copies of a 22-column
  list, and `TestStoredEncodingRoundTrips` sweeps every read path.

## An album page that says how much of the album is yours

- One question, asked once: is there a file. `filePaths` is filled by a
  single batched lookup when the tracklist settles, and the badge, the
  Play count, the dimmed rows and every menu item read it — replacing
  four claims of decreasing confidence that could show a green tick on
  an album whose every action did nothing.
- Play, Play 7 of 12, or no play button at all.
- `total_tracks` on `explore_index` (~2 bytes over 400,677 release
  groups) and on `audio_files` from tags that have always carried it:
  a complete MBID-matched album now makes no catalog call at all, where
  it used to spend the most expensive request the app makes.
- A merged cluster shows the running order the most releases agree on,
  and the version list marks the release you own rather than standing a
  synthetic entry in for it.
- `AlbumReleasesFailed`: a slow fetch is no longer reported as a failed
  one by a 12-second timer.
- Rows not in the library are dimmed in place (with `aria-disabled`)
  instead of the owned ones wearing a green tick and a legend.

## Caches and cover art get ceilings

- Only the three tiers of a cover are stored; the full-resolution copy
  nothing rendered was 1,134 MB of a 1.4 GB covers directory.
- One artist portrait is downloaded and the rest are remembered as
  URLs — 4.1 GB of a 5.3 GB cache was candidates no code path reads.
- `browsedArtBudget` and `httpCacheBudget` bound what an age cannot:
  the same install held art for 5,770 artists in a 1,301-artist
  library.
- `OrphanedArtistImagesJob` joined a bare MBID onto a sharded
  directory, so it deleted the rows that were the only record of the
  files it left behind. `explore.ArtistImageDir` is that layout's one
  definition now.

## The autotag queue asks whether there is work

`tagging_items` was a row per album folder, not a queue, and no query
read the `tag_status` column that held the answer. The four queue
queries ask the files, which matters most where it is least visible:
`startPrefetch` was scoring every album in a tagged library against
MusicBrainz.

## Phantom playlist tracks resolve in place

An M3U8 imported before its files leaves phantom rows; they now match
by path and fall back to position, keep their place in the playlist
when resolved, and pair best-first so two phantoms cannot claim the
same file.

## Playing a track plays the list it is in

Double-click, and Play on a single row's menu, queue the list as
displayed with `startIndex` on that row — the album page and the track
list used to queue one track and discard the album around it. A
multi-row selection still plays exactly itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
This commit is contained in:
yonlu
2026-08-16 13:58:15 -04:00
co-authored by Claude Opus 5
parent 1128881e8d
commit e7748f1fd5
208 changed files with 10944 additions and 12104 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/playlist/playlist.go
+307 -148
View File
@@ -2,6 +2,7 @@
package playlist
import (
"cmp"
"context"
"database/sql"
"errors"
@@ -154,6 +155,8 @@ func NewService(
// SetFavoritesConfig sets the provider used to read and write
// the default-playlist configuration.
//
//wails:ignore // internal wiring, not part of the app's IPC surface.
func (s *Service) SetFavoritesConfig(
provider FavoritesConfigProvider,
) {
@@ -1845,6 +1848,142 @@ type phantomTrackRow struct {
phantomFilePath string
}
// phantomRowSet is a playlist's unresolved rows plus the ones a
// resolution pass has already consumed, so two matches cannot land on
// the same row.
type phantomRowSet struct {
rows []phantomTrackRow
taken map[int64]struct{}
}
// loadPhantomRows reads the playlist's phantom rows — the tracks whose
// file was missing when the M3U8 was imported.
func (s *Service) loadPhantomRows(playlistID int64) *phantomRowSet {
set := &phantomRowSet{taken: map[int64]struct{}{}}
// SAFETY: Hand-crafted SELECT for phantom tracks with position and
// phantom_file_path. Parameterized by playlist ID.
rows, err := s.db.QueryContext(
`SELECT id, position, COALESCE(phantom_file_path, '')
FROM playlist_tracks
WHERE playlist_id = ? AND audio_file_id IS NULL`,
playlistID,
)
if err != nil {
s.logger.Warn(
"could not query phantom tracks",
"playlistId", playlistID,
"err", err,
)
return set
}
defer func() {
if closeErr := rows.Close(); closeErr != nil {
s.logger.Warn(
"could not close phantom track rows",
"err", closeErr,
)
}
}()
for rows.Next() {
var pt phantomTrackRow
if err := rows.Scan(
&pt.id, &pt.position, &pt.phantomFilePath,
); err != nil {
continue
}
set.rows = append(set.rows, pt)
}
return set
}
// takePhantomRow finds the phantom row standing for phantomAbs and
// marks it consumed. It matches on phantom_file_path first and falls
// back to the row sitting at that entry's index in the M3U8, which is
// the same two-step resolvePlaylistPhantoms uses: rows imported before
// migration 7 carry no phantom_file_path at all.
func takePhantomRow(
set *phantomRowSet,
phantomAbs string,
entries []m3uEntry,
libraryRoots []string,
) (int64, bool) {
_, idx := findM3UEntry(entries, phantomAbs, libraryRoots)
return takePhantomRowAt(set, phantomAbs, idx)
}
// takePhantomRowAt is takePhantomRow for a caller that already knows
// the entry's index. A negative index means "no positional fallback":
// positions are non-negative, so it simply never matches.
func takePhantomRowAt(
set *phantomRowSet, phantomAbs string, index int,
) (int64, bool) {
for _, pt := range set.rows {
if _, done := set.taken[pt.id]; done {
continue
}
if pt.phantomFilePath != "" &&
pt.phantomFilePath == phantomAbs {
set.taken[pt.id] = struct{}{}
return pt.id, true
}
}
for _, pt := range set.rows {
if _, done := set.taken[pt.id]; done {
continue
}
if pt.position == int64(index) {
set.taken[pt.id] = struct{}{}
return pt.id, true
}
}
return 0, false
}
// fillPhantomRow points a phantom playlist_tracks row at a real audio
// file and drops the placeholder metadata it was displaying. The row
// keeps its position, which is why resolving a phantom does not move
// the track to the end of the playlist.
func (s *Service) fillPhantomRow(
playlistTrackID, audioFileID int64,
) error {
// SAFETY: Hand-crafted UPDATE to resolve a phantom playlist track.
// Sets audio_file_id and clears all phantom metadata columns.
// Parameterized by ID.
if _, err := s.db.ExecContext(
`UPDATE playlist_tracks SET
audio_file_id = ?,
phantom_title = NULL,
phantom_artist = NULL,
phantom_album = NULL,
phantom_duration_ms = NULL,
phantom_genre = NULL,
phantom_cover_art_path = NULL,
phantom_file_path = NULL
WHERE id = ?`,
audioFileID, playlistTrackID,
); err != nil {
return fmt.Errorf(
"could not resolve phantom track %d: %w",
playlistTrackID, err,
)
}
return nil
}
// resolvePlaylistPhantoms resolves phantom tracks for a single
// playlist by reading its M3U8 file and matching entries against
// the audio_files table. Returns the number of resolved tracks.
@@ -1873,52 +2012,12 @@ func (s *Service) resolvePlaylistPhantoms(
}
// Load phantom tracks for this playlist.
// SAFETY: Hand-crafted SELECT for phantom tracks with
// position and phantom_file_path. No user input.
ptRows, err := s.db.QueryContext(
`SELECT id, position, COALESCE(phantom_file_path, '')
FROM playlist_tracks
WHERE playlist_id = ? AND audio_file_id IS NULL`,
playlistID,
)
if err != nil {
s.logger.Warn(
"could not query phantom tracks",
"playlistId", playlistID,
"err", err,
)
phantoms := s.loadPhantomRows(playlistID)
if len(phantoms.rows) == 0 {
return 0
}
var phantoms []phantomTrackRow
for ptRows.Next() {
var pt phantomTrackRow
if err := ptRows.Scan(
&pt.id, &pt.position, &pt.phantomFilePath,
); err != nil {
continue
}
phantoms = append(phantoms, pt)
}
if err := ptRows.Close(); err != nil {
s.logger.Warn(
"could not close phantom track rows",
"err", err,
)
}
if len(phantoms) == 0 {
return 0
}
// Build a set of already-resolved phantom IDs to avoid
// double-matching.
resolvedIDs := make(map[int64]struct{})
var resolved int
// For each M3U8 entry, resolve its path and try to match
@@ -1933,63 +2032,17 @@ func (s *Service) resolvePlaylistPhantoms(
continue
}
// Find the phantom that corresponds to this entry.
// Priority 1: match by phantom_file_path (exact).
// Priority 2: match by position (M3U8 index).
matchIdx := -1
for j, pt := range phantoms {
if _, done := resolvedIDs[pt.id]; done {
continue
}
if pt.phantomFilePath != "" &&
pt.phantomFilePath == absPath {
matchIdx = j
break
}
}
if matchIdx == -1 {
for j, pt := range phantoms {
if _, done := resolvedIDs[pt.id]; done {
continue
}
if pt.position == int64(i) {
matchIdx = j
break
}
}
}
if matchIdx == -1 {
// Find the phantom that corresponds to this entry:
// phantom_file_path first, then this entry's index.
ptID, found := takePhantomRowAt(phantoms, absPath, i)
if !found {
continue
}
pt := phantoms[matchIdx]
// SAFETY: Hand-crafted UPDATE to resolve a phantom
// playlist track. Sets audio_file_id and clears all
// phantom metadata columns. Parameterized by ID.
if _, err := s.db.ExecContext(
`UPDATE playlist_tracks SET
audio_file_id = ?,
phantom_title = NULL,
phantom_artist = NULL,
phantom_album = NULL,
phantom_duration_ms = NULL,
phantom_genre = NULL,
phantom_cover_art_path = NULL,
phantom_file_path = NULL
WHERE id = ?`,
audioFileID, pt.id,
); err != nil {
if err := s.fillPhantomRow(ptID, audioFileID); err != nil {
s.logger.Warn(
"could not resolve phantom track",
"playlistTrackId", pt.id,
"playlistTrackId", ptID,
"audioFileId", audioFileID,
"err", err,
)
@@ -1997,7 +2050,6 @@ func (s *Service) resolvePlaylistPhantoms(
continue
}
resolvedIDs[pt.id] = struct{}{}
resolved++
}
@@ -2053,52 +2105,110 @@ func (s *Service) FindPhantomMatches(
entryByPath[absPath] = e
}
// Track which candidates have been claimed by auto-match
// so we don't assign the same candidate to two phantoms.
claimed := make(map[string]struct{})
var result PhantomSearchResult
// Every pairing confident enough to apply without asking.
var offers []phantomOffer
for _, phantomPath := range phantomPaths {
entry := entryByPath[phantomPath]
candidates := s.searchCandidates(
phantomPath, entry,
)
matched := false
for _, c := range candidates {
if _, taken := claimed[c.FilePath]; taken {
continue
}
if c.Score >= autoMatchMinimum {
result.AutoMatched = append(
result.AutoMatched,
PhantomMatch{
PhantomPath: phantomPath,
PhantomTitle: entry.DisplayTitle,
Candidate: c,
},
)
claimed[c.FilePath] = struct{}{}
matched = true
for _, c := range s.searchCandidates(phantomPath, entry) {
if c.Score < autoMatchMinimum {
// searchCandidates sorts by score, so nothing
// below this one qualifies either.
break
}
}
if !matched {
result.Unmatched = append(
result.Unmatched, phantomPath,
)
offers = append(offers, phantomOffer{
phantomPath: phantomPath,
phantomTitle: entry.DisplayTitle,
candidate: c,
})
}
}
var result PhantomSearchResult
result.AutoMatched = assignBestFirst(offers)
matched := make(map[string]struct{}, len(result.AutoMatched))
for _, m := range result.AutoMatched {
matched[m.PhantomPath] = struct{}{}
}
// Reported in the order the playlist has them, not the order they
// were matched in.
for _, phantomPath := range phantomPaths {
if _, done := matched[phantomPath]; done {
continue
}
result.Unmatched = append(
result.Unmatched, phantomPath,
)
}
return result, nil
}
// phantomOffer is one candidate a phantom track could be resolved to,
// carrying enough to become a PhantomMatch.
type phantomOffer struct {
phantomPath string
phantomTitle string
candidate CandidateTrack
}
// assignBestFirst pairs phantoms with candidates highest score first,
// at most one candidate per phantom and one phantom per candidate.
//
// A candidate can only stand in for one phantom — two would put that
// file in the playlist twice. Taking the offers in *phantom* order,
// which is what this replaced, let an early phantom claim a candidate
// that was a better answer for a later one, so which of two similar
// tracks got the good match depended on the order they happen to sit in
// the playlist.
func assignBestFirst(offers []phantomOffer) []PhantomMatch {
if len(offers) == 0 {
return nil
}
ordered := make([]phantomOffer, len(offers))
copy(ordered, offers)
slices.SortStableFunc(
ordered,
func(a, b phantomOffer) int {
return cmp.Compare(b.candidate.Score, a.candidate.Score)
},
)
var matches []PhantomMatch
claimedCandidates := make(map[string]struct{}, len(ordered))
matchedPhantoms := make(map[string]struct{}, len(ordered))
for _, o := range ordered {
if _, taken := claimedCandidates[o.candidate.FilePath]; taken {
continue
}
if _, done := matchedPhantoms[o.phantomPath]; done {
continue
}
matches = append(matches, PhantomMatch{
PhantomPath: o.phantomPath,
PhantomTitle: o.phantomTitle,
Candidate: o.candidate,
})
claimedCandidates[o.candidate.FilePath] = struct{}{}
matchedPhantoms[o.phantomPath] = struct{}{}
}
return matches
}
// GetPhantomCandidates returns scored candidate matches for a
// single phantom track.
func (s *Service) GetPhantomCandidates(
@@ -2219,14 +2329,38 @@ func (s *Service) ResolvePhantomTracks(
)
}
// The phantom rows this is about to fill in, so a resolution
// updates the row the user pointed at rather than appending a
// second one beside it.
phantoms := s.loadPhantomRows(playlistID)
// Build M3U path replacements and insert DB rows.
pathReplacements := make(
map[string]string, len(matches),
)
var resolved int
// Two phantoms resolving to one file would put that file in the
// playlist twice, which is what FindPhantomMatches' claimed set
// already refuses to do on the auto-match path.
claimed := make(map[string]struct{}, len(matches))
var (
resolved int
appended int
)
for phantomAbs, resolvedAbs := range matches {
if _, taken := claimed[resolvedAbs]; taken {
s.logger.Warn(
"Two phantom tracks resolved to the same file",
"playlistId", playlistID,
"phantomPath", phantomAbs,
"resolvedPath", resolvedAbs,
)
continue
}
audioFile, lookupErr := s.db.Queries.GetAudioFileByPath(
s.db.Ctx, resolvedAbs,
)
@@ -2241,29 +2375,54 @@ func (s *Service) ResolvePhantomTracks(
continue
}
_, addErr := s.db.Queries.AddPlaylistTrack(
s.db.Ctx,
sqlcgen.AddPlaylistTrackParams{
PlaylistID: playlistID,
AudioFileID: sql.NullInt64{Int64: audioFile.ID, Valid: true},
Position: nextPos + int64(resolved),
},
ptID, found := takePhantomRow(
phantoms, phantomAbs, parsed.Entries, libraryRoots,
)
if addErr != nil {
s.logger.Warn(
"Could not add resolved track",
"playlistId", playlistID,
"path", resolvedAbs,
"err", addErr,
)
continue
switch {
case found:
if updateErr := s.fillPhantomRow(
ptID, audioFile.ID,
); updateErr != nil {
s.logger.Warn(
"Could not resolve phantom row",
"playlistId", playlistID,
"playlistTrackId", ptID,
"path", resolvedAbs,
"err", updateErr,
)
continue
}
default:
// No phantom row for this path — the M3U8 and the
// database disagree, so fall back to appending.
if _, addErr := s.db.Queries.AddPlaylistTrack(
s.db.Ctx,
sqlcgen.AddPlaylistTrackParams{
PlaylistID: playlistID,
AudioFileID: sql.NullInt64{Int64: audioFile.ID, Valid: true},
Position: nextPos + int64(appended),
},
); addErr != nil {
s.logger.Warn(
"Could not add resolved track",
"playlistId", playlistID,
"path", resolvedAbs,
"err", addErr,
)
continue
}
appended++
}
newRel := toRelativePathMultiRoot(
resolvedAbs, libraryRoots,
)
pathReplacements[phantomAbs] = newRel
claimed[resolvedAbs] = struct{}{}
resolved++
}
@@ -2390,11 +2549,11 @@ func (s *Service) searchCandidates(
var combined []database.SearchRow
// 1. Exact basename match via indexed column.
bnRows, err := s.db.Queries.SearchAudioFilesByBasename(
bnRows, err := s.db.Queries.SearchTracksByBasename(
s.db.Ctx,
sqlcgen.SearchAudioFilesByBasenameParams{
sqlcgen.SearchTracksByBasenameParams{
Basename: basename,
Limit: int64(maxCandidates),
Lim: int64(maxCandidates),
},
)
if err != nil {
@@ -2416,7 +2575,7 @@ func (s *Service) searchCandidates(
FilePath: r.FilePath,
LengthMilliseconds: r.LengthMilliseconds,
Title: r.Title,
Artist: r.Artist,
Artist: r.ArtistName,
Album: r.Album,
})
}