yonlu/yellowjacket
1
1
Fork 0
Code Issues 52 Pull Requests 4 Actions Packages Projects Releases 12 Wiki Activity

feat(database): shape the library like files, and shrink the catalog
CI / check (push) Successful in 3m7s
Details
CI / e2e (push) Canceled after 1m45s
Details

Browse Source 
Plans 013 and 014, the album page that prompted them, and the smaller
fixes they turned up. Changelog, largest first.

## The local library is shaped like files, not like MusicBrainz

`audio_files` carries its own tags and points at `albums` and
`artists`; `file_genres` is the one real many-to-many. `recordings`,
`release_group_recordings`, `artist_credit`, `artist_credit_artist`,
`recording_genres`, `release_groups` and `release_to_rg` are gone from
the local side, and with them a six-way join in every read, a
`MIN(release_group_id)` subquery in eleven queries and a
first-credited-artist subquery in nine. Measured on a real 25,966-file
library, every many-to-many that model expressed was 1:1 in the data.

- Ownership is a file. `GetFilePathsByRecordingMBIDs`,
  `LibraryMBIDIndex.CheckMBIDs`, `collectLibraryEntities` and
  `pruneStaleLocalCrossReferences` all join `audio_files`, so the 812
  orphaned recordings, 216 release groups and 260 artists that library
  carried are now structurally impossible.
- One projection: every track query selects from the `track_metadata`
  view, one row type, one mapper. Nine hand-rolled copies had drifted
  far enough to report different years on different screens.
- `library_id = 0` means every library, so each list query exists once
  instead of scoped and unscoped with a branch at every call site.
- No migration chain. `sql/schemas/` is the one description of the
  shape; `sql/migrations/`, `applyMigrations` and `schema_migrations`
  are squashed away, along with the drift between them that had sqlc
  generating against a stale schema.
- `database.InsertTestTrack` is the one test seeder; twenty test files
  had been assembling the old FK chain each in its own order.

## The catalog stores its ids as bytes

`explore_index`'s three 36-char MBID columns and its entity-type text
are 16 raw bytes and a small integer. The table and its six indexes go
780 MB to 405 MB on a real 2,052,200-row catalog, which is why a fresh
install is ~0.6 GB rather than ~1.0 GB.

- `backend/explore/mbid.go` is the only place the encoding is known;
  everything above it speaks dashed strings.
- `CHECK(length(mbid) = 16)` makes a stringly write fail at the insert
  rather than silently returning no rows, since SQLite does not coerce
  between TEXT and BLOB.
- The importer asks the artifact what encoding it carries and converts
  on the way in, so the artifact already published keeps working and no
  format bump is needed.
- `indexRowColumns`/`scanIndexRow` replace four copies of a 22-column
  list, and `TestStoredEncodingRoundTrips` sweeps every read path.

## An album page that says how much of the album is yours

- One question, asked once: is there a file. `filePaths` is filled by a
  single batched lookup when the tracklist settles, and the badge, the
  Play count, the dimmed rows and every menu item read it — replacing
  four claims of decreasing confidence that could show a green tick on
  an album whose every action did nothing.
- Play, Play 7 of 12, or no play button at all.
- `total_tracks` on `explore_index` (~2 bytes over 400,677 release
  groups) and on `audio_files` from tags that have always carried it:
  a complete MBID-matched album now makes no catalog call at all, where
  it used to spend the most expensive request the app makes.
- A merged cluster shows the running order the most releases agree on,
  and the version list marks the release you own rather than standing a
  synthetic entry in for it.
- `AlbumReleasesFailed`: a slow fetch is no longer reported as a failed
  one by a 12-second timer.
- Rows not in the library are dimmed in place (with `aria-disabled`)
  instead of the owned ones wearing a green tick and a legend.

## Caches and cover art get ceilings

- Only the three tiers of a cover are stored; the full-resolution copy
  nothing rendered was 1,134 MB of a 1.4 GB covers directory.
- One artist portrait is downloaded and the rest are remembered as
  URLs — 4.1 GB of a 5.3 GB cache was candidates no code path reads.
- `browsedArtBudget` and `httpCacheBudget` bound what an age cannot:
  the same install held art for 5,770 artists in a 1,301-artist
  library.
- `OrphanedArtistImagesJob` joined a bare MBID onto a sharded
  directory, so it deleted the rows that were the only record of the
  files it left behind. `explore.ArtistImageDir` is that layout's one
  definition now.

## The autotag queue asks whether there is work

`tagging_items` was a row per album folder, not a queue, and no query
read the `tag_status` column that held the answer. The four queue
queries ask the files, which matters most where it is least visible:
`startPrefetch` was scoring every album in a tagged library against
MusicBrainz.

## Phantom playlist tracks resolve in place

An M3U8 imported before its files leaves phantom rows; they now match
by path and fall back to position, keep their place in the playlist
when resolved, and pair best-first so two phantoms cannot claim the
same file.

## Playing a track plays the list it is in

Double-click, and Play on a single row's menu, queue the list as
displayed with `startIndex` on that row — the album page and the track
list used to queue one track and discard the album around it. A
multi-row selection still plays exactly itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
This commit is contained in:
yonlu
2026-08-16 13:58:15 -04:00
co-authored by Claude Opus 5
parent 1128881e8d
commit e7748f1fd5
208 changed files with 10944 additions and 12104 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/library/completeness_test.go
+32 -50
View File
@@ -1,12 +1,15 @@
package library
import (
"fmt"
"testing"
"yellowjacket/backend/database"
)
// track is one row of release_group_recordings as the scan would write
// it: a position on a disc, and whatever total the file's tag declared
// (0 meaning the tag did not say).
// track is one file as the scan would write it: a position on a disc,
// and whatever total the file's tag declared (0 meaning the tag did not
// say).
type track struct {
recordingID int
disc int
@@ -14,58 +17,37 @@ type track struct {
total int
}
// stageAlbum writes an album's tracks straight into
// release_group_recordings. The completeness query reads only that
// table, so this exercises the arithmetic without standing up a scan.
// stageAlbum writes an album's files straight in. The completeness
// query reads only audio_files now - the totals used to live on a join
// table - so this exercises the arithmetic without standing up a scan.
func stageAlbum(t *testing.T, lib *Library, albumID int, tracks []track) {
t.Helper()
// The foreign keys are enforced, so the album and its recordings
// have to exist before they can be linked.
if _, err := lib.db.ExecContext(
`INSERT INTO artist_credit (id, text) VALUES (1, 'Test Artist')`,
); err != nil {
t.Fatalf("staging artist credit: %v", err)
}
if _, err := lib.db.ExecContext(
`INSERT INTO release_groups (id, name, album_artist_credit_id)
VALUES (?, ?, 1)`,
albumID, "Test Album",
); err != nil {
t.Fatalf("staging album: %v", err)
}
for _, tr := range tracks {
if _, err := lib.db.ExecContext(
`INSERT INTO recordings (id, name, artist_credit_id) VALUES (?, ?, 1)`,
tr.recordingID, "Test Track",
); err != nil {
t.Fatalf("staging recording %d: %v", tr.recordingID, err)
}
database.InsertTestTrack(t, lib.db, database.TestTrack{
FilePath: fmt.Sprintf("/music/album%d/%d.mp3", albumID, tr.recordingID),
Title: "Test Track",
Artist: "Test Artist",
Album: fmt.Sprintf("Test Album %d", albumID),
TrackNumber: int64(tr.number),
DiscNumber: int64(tr.disc),
TotalTracks: int64(tr.total),
})
}
}
// albumIDFor reads back the id stageAlbum's files were filed under.
func albumIDFor(t *testing.T, lib *Library, albumID int) int64 {
t.Helper()
var id int64
if err := lib.db.QueryRowWriter(
"SELECT id FROM albums WHERE name = ?", fmt.Sprintf("Test Album %d", albumID),
).Scan(&id); err != nil {
t.Fatalf("read album id: %v", err)
}
for _, tr := range tracks {
var total any
if tr.total > 0 {
total = tr.total
}
var number any
if tr.number > 0 {
number = tr.number
}
_, err := lib.db.ExecContext(
`INSERT INTO release_group_recordings
(release_group_id, recording_id, track_number, disc_number, total_tracks)
VALUES (?, ?, ?, ?, ?)`,
albumID, tr.recordingID, number, tr.disc, total,
)
if err != nil {
t.Fatalf("staging track %d: %v", tr.recordingID, err)
}
}
return id
}
// disc builds a run of tracks on one disc, each declaring the same
@@ -210,7 +192,7 @@ func TestGetAlbumCompleteness(t *testing.T) {
stageAlbum(t, lib, albumID, tc.tracks)
got, err := lib.GetAlbumCompleteness(int64(albumID))
got, err := lib.GetAlbumCompleteness(albumIDFor(t, lib, albumID))
if err != nil {
t.Fatalf("GetAlbumCompleteness: %v", err)
}
backend/library/coverart.go
+35 -312
View File
@@ -10,7 +10,6 @@ import (
_ "image/png" // Register PNG decoder.
"os"
"path/filepath"
"strings"
"time"
"golang.org/x/image/draw"
@@ -44,24 +43,22 @@ var thumbnailTiers = []thumbnailTier{
{Suffix: "_lg", MaxSize: 400, Quality: 85},
}
// legacyThumbSuffix is the old single-thumbnail suffix used before the
// multi-tier system. Kept for migration purposes only.
const legacyThumbSuffix = "_thumb"
// largestTier is the tier stored as the cover's canonical file.
func largestTier() thumbnailTier {
return thumbnailTiers[len(thumbnailTiers)-1]
}
// CoverArtFileSet returns every file on disk belonging to one cover art
// entry: the original plus each generated size variant, plus the legacy
// _thumb file for databases that predate the multi-tier thumbnails.
// entry: each generated size variant.
//
// Only the original is recorded in cover_art.file_path — the variants
// are derived filenames — so any code deleting cover art has to expand
// the set or the thumbnails are orphaned.
func CoverArtFileSet(originalPath string) []string {
dir := filepath.Dir(originalPath)
base := filepath.Base(originalPath)
// Only one of them is recorded in cover_art.file_path — the others are
// derived filenames — so any code deleting cover art has to expand the
// set or the rest are orphaned.
func CoverArtFileSet(coverPath string) []string {
dir := filepath.Dir(coverPath)
base := filepath.Base(coverPath)
paths := make([]string, 0, len(thumbnailTiers)+2) //nolint:mnd
paths = append(paths, originalPath)
paths := make([]string, 0, len(thumbnailTiers))
for _, tier := range thumbnailTiers {
paths = append(paths, filepath.Join(
@@ -69,25 +66,7 @@ func CoverArtFileSet(originalPath string) []string {
))
}
return append(paths, filepath.Join(
dir, coverart.SizedFilename(base, legacyThumbSuffix),
))
}
// isSizedVariant reports whether a filename contains any known size suffix
// (current tiers or legacy).
func isSizedVariant(name string) bool {
if strings.Contains(name, legacyThumbSuffix) {
return true
}
for _, tier := range thumbnailTiers {
if strings.Contains(name, tier.Suffix) {
return true
}
}
return false
return paths
}
// saveCoverArt saves embedded cover art to the cache directory.
@@ -121,47 +100,30 @@ func (l *Library) saveCoverArt(
)
}
// Generate filename from content hash (deduplication).
// The content hash identifies the cover and dedupes it; the largest
// tier is what gets stored under that name.
//
// The full-resolution image used to be written here too, and it was
// 1,134 MB of a 1.4 GB covers directory on a real library - against
// 110 MB for all three tiers together - with nothing rendering it.
// The bytes are still in the audio file if a bigger one is ever
// needed, which is where these came from.
hash := sha256.Sum256(pic.Data)
hashStr := hex.EncodeToString(hash[:8]) // First 8 bytes = 16 hex chars.
ext := pic.Ext
if ext == "" {
ext = extensionFromMIME(pic.MIMEType)
}
filePath := filepath.Join(
coverDir, coverart.SizedFilename(hashStr, largestTier().Suffix),
)
filename := fmt.Sprintf("%s.%s", hashStr, ext)
filePath := filepath.Join(coverDir, filename)
// Skip if already exists (same content hash).
// Missing sized variants are handled by
// generateMissingSizedVariants() at the end of a scan.
// Skip if this cover has already been stored (same content hash).
if _, err := os.Stat(filePath); err == nil {
l.logger.Debug(
"cover art already exists", "path", filePath,
)
l.logger.Debug("cover art already stored", "path", filePath)
return filePath, nil
}
// Write file.
if err := os.WriteFile(
filePath, pic.Data, 0o644,
); err != nil {
return "", fmt.Errorf(
"could not write cover art: %w", err,
)
}
metrics.addCoverArtSave(time.Since(saveStart))
l.logger.Debug(
"saved cover art",
"path", filePath, "size", len(pic.Data),
)
// Dispatch thumbnail generation to the async worker pool
// if available, otherwise generate inline.
// Dispatch thumbnail generation to the async worker pool if
// available, otherwise generate inline.
if thumbChan != nil {
thumbChan <- thumbnailWork{
imgData: pic.Data,
@@ -169,38 +131,18 @@ func (l *Library) saveCoverArt(
hashStr: hashStr,
metrics: metrics,
}
} else {
if err := l.generateSizedVariantsWithMetrics(
pic.Data, coverDir, hashStr, metrics,
); err != nil {
l.logger.Warn(
"could not generate sized variants",
"path", filePath, "err", err,
)
}
} else if err := l.generateSizedVariantsWithMetrics(
pic.Data, coverDir, hashStr, metrics,
); err != nil {
l.logger.Warn("could not generate sized variants",
"path", filePath, "err", err)
}
metrics.addCoverArtSave(time.Since(saveStart))
return filePath, nil
}
// generateSizedVariants creates all thumbnail tiers for the given image data.
// Each tier is saved as {hashStr}{suffix}.jpg in the given directory.
func (l *Library) generateSizedVariants(
imgData []byte,
dir, hashStr string,
) error {
src, _, err := image.Decode(bytes.NewReader(imgData))
if err != nil {
return fmt.Errorf(
"could not decode image for thumbnails: %w", err,
)
}
l.generateTiersFromImage(src, dir, hashStr)
return nil
}
// generateSizedVariantsWithMetrics is like generateSizedVariants
// but records per-tier timing in the provided metrics.
func (l *Library) generateSizedVariantsWithMetrics(
@@ -257,46 +199,6 @@ func (l *Library) generateSizedVariantsWithMetrics(
return nil
}
// generateTiersFromImage creates all thumbnail tiers from an
// already-decoded image.
func (l *Library) generateTiersFromImage(
src image.Image,
dir, hashStr string,
) {
bounds := src.Bounds()
srcW := bounds.Dx()
srcH := bounds.Dy()
for _, tier := range thumbnailTiers {
tierPath := filepath.Join(
dir,
fmt.Sprintf("%s%s.jpg", hashStr, tier.Suffix),
)
w, h := fitDimensions(srcW, srcH, tier.MaxSize)
if err := encodeAndSaveImage(
src, tierPath, w, h, tier.Quality,
); err != nil {
l.logger.Warn(
"could not generate sized variant",
"tier", tier.Suffix,
"path", tierPath,
"err", err,
)
continue
}
l.logger.Debug(
"saved sized variant",
"tier", tier.Suffix,
"path", tierPath,
"dimensions", fmt.Sprintf("%dx%d", w, h),
)
}
}
// fitDimensions calculates the output dimensions that fit within maxSize
// while preserving the aspect ratio. If the source is already smaller
// than maxSize, the original dimensions are returned unchanged.
@@ -343,182 +245,3 @@ func encodeAndSaveImage(
return nil
}
// generateMissingSizedVariants scans the covers directory, migrates legacy
// _thumb files to _md, and generates any missing sized variants for each
// original cover art file.
func (l *Library) generateMissingSizedVariants() error {
coverDir, err := coverart.CoversDir()
if err != nil {
return fmt.Errorf(
"could not resolve covers directory: %w", err,
)
}
entries, err := os.ReadDir(coverDir)
if err != nil {
return fmt.Errorf(
"could not read covers directory: %w", err,
)
}
// Build a set of existing filenames for quick lookup.
existing := make(map[string]struct{}, len(entries))
for _, entry := range entries {
if !entry.IsDir() {
existing[entry.Name()] = struct{}{}
}
}
// First pass: migrate legacy _thumb files to _md.
migrated := l.migrateLegacyThumbs(
coverDir, existing,
)
// Second pass: generate missing sized variants.
var generated, skipped int
for _, entry := range entries {
name := entry.Name()
// Skip directories and any sized variants.
if entry.IsDir() || isSizedVariant(name) {
continue
}
hashStr := strings.SplitN(name, ".", 2)[0]
// Check which tiers are missing.
allPresent := true
for _, tier := range thumbnailTiers {
tierName := fmt.Sprintf(
"%s%s.jpg", hashStr, tier.Suffix,
)
if _, exists := existing[tierName]; !exists {
allPresent = false
break
}
}
if allPresent {
skipped++
continue
}
// Read the original and generate missing tiers.
imgData, err := os.ReadFile(
filepath.Join(coverDir, name),
)
if err != nil {
l.logger.Warn(
"could not read cover art for variant generation",
"file", name, "err", err,
)
continue
}
if err := l.generateSizedVariants(
imgData, coverDir, hashStr,
); err != nil {
l.logger.Warn(
"could not generate sized variants",
"file", name, "err", err,
)
continue
}
generated++
}
l.logger.Info(
"sized variant generation complete",
"generated", generated,
"skipped", skipped,
"migrated", migrated,
)
return nil
}
// migrateLegacyThumbs renames _thumb.jpg files to _md.jpg.
// Returns the number of files migrated.
func (l *Library) migrateLegacyThumbs(
coverDir string,
existing map[string]struct{},
) int {
var migrated int
for name := range existing {
if !strings.Contains(name, legacyThumbSuffix) {
continue
}
// Derive the _md name from the legacy name.
mdName := strings.Replace(
name, legacyThumbSuffix, "_md", 1,
)
oldPath := filepath.Join(coverDir, name)
newPath := filepath.Join(coverDir, mdName)
// Only rename if _md doesn't already exist.
if _, exists := existing[mdName]; exists {
// Both exist; remove the legacy file.
if err := os.Remove(oldPath); err != nil {
l.logger.Warn(
"could not remove legacy thumbnail",
"file", name, "err", err,
)
}
continue
}
if err := os.Rename(oldPath, newPath); err != nil {
l.logger.Warn(
"could not migrate legacy thumbnail",
"from", name, "to", mdName, "err", err,
)
continue
}
// Update the existing set so subsequent lookups
// see the new name.
delete(existing, name)
existing[mdName] = struct{}{}
migrated++
l.logger.Debug(
"migrated legacy thumbnail",
"from", name, "to", mdName,
)
}
return migrated
}
// extensionFromMIME returns a file extension for common image MIME types.
func extensionFromMIME(mimeType string) string {
switch mimeType {
case "image/jpeg":
return "jpg"
case "image/png":
return "png"
case "image/gif":
return "gif"
case "image/webp":
return "webp"
case "image/bmp":
return "bmp"
default:
return "jpg" // Default to jpg.
}
}
backend/library/coverart_storage_test.go
+90
View File
@@ -0,0 +1,90 @@
package library
import (
"os"
"path/filepath"
"strings"
"testing"
"yellowjacket/backend/coverart"
"yellowjacket/backend/database/sql/sqlcgen"
)
// TestScan_StoresOnlyCoverTiers pins the size decision: a scan writes
// the three rendered tiers and nothing else.
//
// The full-resolution image used to be written beside them, and on a
// real 2,057-album library that was 1,134 MB of a 1.4 GB covers
// directory against 110 MB for all three tiers together - with nothing
// rendering it, since the grid caps at 350 px and the largest tier is
// 400. The bytes are still in the audio file if a bigger one is ever
// wanted, which is where these came from.
func TestScan_StoresOnlyCoverTiers(t *testing.T) {
// Not parallel: YJ_HOME is process-wide, and this test needs the
// covers directory to itself.
t.Setenv("YJ_HOME", t.TempDir())
lib, db := setupTestLibrary(t)
root, err := filepath.Abs("../../test_data/music_library_test")
if err != nil {
t.Fatalf("resolve fixture path: %v", err)
}
library, err := db.Queries.CreateLibrary(lib.ctx, sqlcgen.CreateLibraryParams{
Name: "Fixtures",
Path: root,
})
if err != nil {
t.Fatalf("create library: %v", err)
}
lib.scanInternal(library.ID, library.Name, library.Path)
coversDir, err := coverart.CoversDir()
if err != nil {
t.Fatalf("covers dir: %v", err)
}
entries, err := os.ReadDir(coversDir)
if err != nil {
t.Fatalf("read covers dir: %v", err)
}
if len(entries) == 0 {
t.Skip("fixture library produced no cover art; run make testdata")
}
perTier := map[string]int{}
for _, entry := range entries {
name := entry.Name()
base := coverart.BaseName(name)
if base+filepath.Ext(name) == name {
t.Errorf("full-size cover written: %s", name)
continue
}
perTier[strings.TrimSuffix(strings.TrimPrefix(name, base), ".jpg")]++
}
for _, suffix := range coverart.Suffixes {
if perTier[suffix] == 0 {
t.Errorf("no %s tier written", suffix)
}
}
// And what the database points at is a file that exists.
var stored string
if err := db.QueryRowWriter(
"SELECT file_path FROM cover_art LIMIT 1",
).Scan(&stored); err != nil {
t.Fatalf("read cover_art path: %v", err)
}
if _, err := os.Stat(stored); err != nil {
t.Errorf("cover_art.file_path names a file that is not there: %v", err)
}
}
backend/library/crud.go
+26 -102
View File
@@ -42,6 +42,8 @@ type RemovalHooks struct {
// SetRemovalHooks provides optional hooks for cross-cutting
// orchestration during RemoveLibrary.
//
//wails:ignore // internal wiring, not part of the app's IPC surface.
func (l *Library) SetRemovalHooks(h RemovalHooks) {
l.mu.Lock()
defer l.mu.Unlock()
@@ -168,9 +170,7 @@ func (l *Library) RenameLibrary(id int64, newName string) error {
// GetRemovalImpact returns pre-removal counts for the confirmation
// dialog. All queries are read-only.
func (l *Library) GetRemovalImpact(libraryID int64) (*RemovalImpact, error) {
// SAFETY: Hand-crafted SQL for track count. sqlc query CountAudioFilesByLibrary
// exists but we inline the remaining two for consistency. Parameterized.
trackCount, err := l.db.Queries.CountAudioFilesByLibrary(l.ctx, libraryID)
trackCount, err := l.db.Queries.CountAudioFiles(l.ctx, libraryID)
if err != nil {
return nil, fmt.Errorf("could not count tracks: %w", err)
}
@@ -251,40 +251,17 @@ func (l *Library) RemoveLibrary(id int64) (*RemovalSummary, error) {
if _, err := tx.ExecContext(l.ctx, `
UPDATE playlist_tracks SET
phantom_title = sub.title,
phantom_artist = sub.artist,
phantom_artist = sub.artist_name,
phantom_album = sub.album,
phantom_duration_ms = sub.duration,
phantom_duration_ms = sub.length_milliseconds,
phantom_genre = sub.genre,
phantom_cover_art_path = sub.cover_art_path,
phantom_file_path = sub.file_path
FROM (
SELECT
pt.id AS pt_id,
af.file_path AS file_path,
COALESCE(r.name, '') AS title,
COALESCE(ac.text, '') AS artist,
COALESCE(rg.name, '') AS album,
af.length_milliseconds AS duration,
CAST(COALESCE(
(SELECT GROUP_CONCAT(g.name, '||')
FROM recording_genres rg_sub
JOIN genres g ON rg_sub.genre_id = g.id
WHERE rg_sub.recording_id = r.id),
''
) AS TEXT) AS genre,
COALESCE(ca.file_path, '') AS cover_art_path
SELECT pt.id AS pt_id, tm.*
FROM playlist_tracks pt
JOIN audio_files af ON pt.audio_file_id = af.id
LEFT JOIN recordings r ON af.recording_id = r.id
LEFT JOIN artist_credit ac ON r.artist_credit_id = ac.id
LEFT JOIN (
SELECT recording_id, MIN(release_group_id) AS release_group_id
FROM release_group_recordings
GROUP BY recording_id
) rgr ON r.id = rgr.recording_id
LEFT JOIN release_groups rg ON rgr.release_group_id = rg.id
LEFT JOIN cover_art ca ON rg.cover_art_id = ca.id
WHERE af.library_id = ?
JOIN track_metadata tm ON tm.id = pt.audio_file_id
WHERE tm.library_id = ?
) sub
WHERE playlist_tracks.id = sub.pt_id`, id); err != nil {
return nil, fmt.Errorf("could not populate phantom metadata: %w", err)
@@ -301,97 +278,44 @@ func (l *Library) RemoveLibrary(id int64) (*RemovalSummary, error) {
tracksDeleted, _ := result.RowsAffected()
// 7. Delete orphaned recording_genres (must run BEFORE recordings
// because recording_genres.recording_id references recordings.id).
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM recording_genres WHERE recording_id NOT IN (
SELECT DISTINCT recording_id FROM audio_files
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned recording_genres: %w", err)
}
// 8. Delete orphaned release_group_recordings (must run BEFORE
// recordings because release_group_recordings.recording_id
// references recordings.id).
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM release_group_recordings WHERE recording_id NOT IN (
SELECT DISTINCT recording_id FROM audio_files
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned release_group_recordings: %w", err)
}
// 9. Delete orphaned recordings (safe now that child tables are cleaned).
// SAFETY: Hand-crafted orphan cleanup SQL. Reference-counting delete
// with NOT IN subquery unsupported by sqlc. No user input.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM recordings WHERE id NOT IN (
SELECT DISTINCT recording_id FROM audio_files
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned recordings: %w", err)
}
// 10. Delete orphaned release_groups.
// 7. Sweep what the files left behind. This used to be eight
// statements in dependency order, because deleting a file cascaded
// to none of the five metadata tables it had created. file_genres
// cascades now, so what is left is the two tables that genuinely
// outlive a file and the genres nothing references.
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
result, err = tx.ExecContext(l.ctx,
`DELETE FROM release_groups WHERE id NOT IN (
SELECT DISTINCT release_group_id FROM release_group_recordings
`DELETE FROM albums WHERE id NOT IN (
SELECT DISTINCT album_id FROM audio_files WHERE album_id IS NOT NULL
)`)
if err != nil {
return nil, fmt.Errorf("could not delete orphaned release_groups: %w", err)
return nil, fmt.Errorf("could not delete empty albums: %w", err)
}
albumsRemoved, _ := result.RowsAffected()
// 11. Delete orphaned artist_credit_artists (must run BEFORE
// artist_credit because artist_credit_artist.credit_id references
// artist_credit.id).
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM artist_credit_artist WHERE credit_id NOT IN (
SELECT DISTINCT artist_credit_id FROM recordings
) AND credit_id NOT IN (
SELECT DISTINCT album_artist_credit_id FROM release_groups
WHERE album_artist_credit_id IS NOT NULL
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned artist_credit_artists: %w", err)
}
// 12. Delete orphaned artist_credits (safe now that child table is cleaned).
// SAFETY: Hand-crafted orphan cleanup SQL. Dual-FK reference counting
// (recordings.artist_credit_id + release_groups.album_artist_credit_id)
// unsupported by sqlc. Parameterless.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM artist_credit WHERE id NOT IN (
SELECT DISTINCT artist_credit_id FROM recordings
) AND id NOT IN (
SELECT DISTINCT album_artist_credit_id FROM release_groups
WHERE album_artist_credit_id IS NOT NULL
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned artist_credits: %w", err)
}
// 13. Delete orphaned artists.
// Artists after albums: an artist is unreferenced only once the
// albums pointing at it are gone.
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
result, err = tx.ExecContext(l.ctx,
`DELETE FROM artists WHERE id NOT IN (
SELECT DISTINCT artist_id FROM artist_credit_artist
SELECT DISTINCT artist_id FROM audio_files WHERE artist_id IS NOT NULL
) AND id NOT IN (
SELECT DISTINCT artist_id FROM albums WHERE artist_id IS NOT NULL
)`)
if err != nil {
return nil, fmt.Errorf("could not delete orphaned artists: %w", err)
return nil, fmt.Errorf("could not delete unreferenced artists: %w", err)
}
artistsRemoved, _ := result.RowsAffected()
// 14. Delete orphaned genres.
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
result, err = tx.ExecContext(l.ctx,
`DELETE FROM genres WHERE id NOT IN (
SELECT DISTINCT genre_id FROM recording_genres
SELECT DISTINCT genre_id FROM file_genres
)`)
if err != nil {
return nil, fmt.Errorf("could not delete orphaned genres: %w", err)
return nil, fmt.Errorf("could not delete unused genres: %w", err)
}
genresRemoved, _ := result.RowsAffected()
@@ -401,7 +325,7 @@ func (l *Library) RemoveLibrary(id int64) (*RemovalSummary, error) {
// Parameterless.
rows, err := tx.QueryContext(l.ctx,
`SELECT file_path FROM cover_art WHERE id NOT IN (
SELECT DISTINCT cover_art_id FROM release_groups
SELECT DISTINCT cover_art_id FROM albums
WHERE cover_art_id IS NOT NULL
)`)
if err != nil {
@@ -429,7 +353,7 @@ func (l *Library) RemoveLibrary(id int64) (*RemovalSummary, error) {
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM cover_art WHERE id NOT IN (
SELECT DISTINCT cover_art_id FROM release_groups
SELECT DISTINCT cover_art_id FROM albums
WHERE cover_art_id IS NOT NULL
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned cover_art: %w", err)
backend/library/filepaths_test.go
+26 -86
View File
@@ -1,9 +1,9 @@
package library
import (
"database/sql"
"testing"
"yellowjacket/backend/database"
"yellowjacket/backend/database/sql/sqlcgen"
)
@@ -32,22 +32,6 @@ func seedAlbumsAndGenres(t *testing.T, lib *Library) (albumIDs []int64, libraryI
t.Fatalf("create other library: %v", err)
}
ac, err := q.UpsertArtistCredit(ctx, "Test Artist")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
genreIDs := map[string]int64{}
for _, name := range []string{"Ambient", "Baroque"} {
g, err := q.UpsertGenre(ctx, name)
if err != nil {
t.Fatalf("upsert genre %s: %v", name, err)
}
genreIDs[name] = g.ID
}
// Two albums; the second lives in the other library so the
// library-scoped variants have something to exclude.
type spec struct {
@@ -66,63 +50,32 @@ func seedAlbumsAndGenres(t *testing.T, lib *Library) (albumIDs []int64, libraryI
{"Second", "B1", "/other/b1.mp3", other.ID, 1, 1, []string{"Baroque"}},
}
byAlbum := map[string]int64{}
seen := map[string]bool{}
for _, s := range specs {
rec, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: s.track,
ArtistCreditID: ac.ID,
database.InsertTestTrack(t, lib.db, database.TestTrack{
FilePath: s.path,
Title: s.track,
Artist: "Test Artist",
Album: s.album,
Genres: s.genres,
TrackNumber: s.number,
DiscNumber: s.disc,
LibraryID: s.library,
LengthMs: 1000,
})
if err != nil {
t.Fatalf("create recording: %v", err)
}
rgID, ok := byAlbum[s.album]
if !seen[s.album] {
seen[s.album] = true
if !ok {
rg, err := q.UpsertReleaseGroup(ctx, sqlcgen.UpsertReleaseGroupParams{
Name: s.album,
AlbumArtistCreditID: sql.NullInt64{Int64: ac.ID, Valid: true},
})
if err != nil {
t.Fatalf("upsert release group: %v", err)
var id int64
if err := lib.db.QueryRowWriter(
"SELECT id FROM albums WHERE name = ?", s.album,
).Scan(&id); err != nil {
t.Fatalf("album id for %q: %v", s.album, err)
}
rgID = rg.ID
byAlbum[s.album] = rgID
albumIDs = append(albumIDs, rgID)
}
if _, err := q.CreateReleaseGroupRecording(
ctx, sqlcgen.CreateReleaseGroupRecordingParams{
ReleaseGroupID: rgID,
RecordingID: rec.ID,
TrackNumber: sql.NullInt64{Int64: s.number, Valid: true},
DiscNumber: sql.NullInt64{Int64: s.disc, Valid: true},
},
); err != nil {
t.Fatalf("link recording: %v", err)
}
if _, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: s.path,
LengthMilliseconds: 1000,
RecordingID: rec.ID,
LibraryID: s.library,
Basename: s.track + ".mp3",
}); err != nil {
t.Fatalf("create audio file: %v", err)
}
for _, g := range s.genres {
if err := q.CreateRecordingGenre(
ctx, sqlcgen.CreateRecordingGenreParams{
RecordingID: rec.ID,
GenreID: genreIDs[g],
},
); err != nil {
t.Fatalf("link genre: %v", err)
}
albumIDs = append(albumIDs, id)
}
}
@@ -237,9 +190,6 @@ func TestGetFilePathsByGenres_Empty(t *testing.T) {
func seedRecordingMBIDs(t *testing.T, lib *Library) (tagged, shared string) {
t.Helper()
ctx := lib.ctx
q := lib.db.Queries
tagged = "11111111-1111-1111-1111-111111111111"
shared = "22222222-2222-2222-2222-222222222222"
@@ -249,22 +199,12 @@ func seedRecordingMBIDs(t *testing.T, lib *Library) (tagged, shared string) {
"/other/b1.mp3": shared,
}
files, err := q.GetAllAudioFiles(ctx)
if err != nil {
t.Fatalf("get audio files: %v", err)
}
for _, f := range files {
mbid, ok := byPath[f.FilePath]
if !ok {
continue
}
if err := q.SetRecordingMBID(ctx, sqlcgen.SetRecordingMBIDParams{
Mbid: sql.NullString{String: mbid, Valid: true},
ID: f.RecordingID,
}); err != nil {
t.Fatalf("set recording mbid: %v", err)
for path, mbid := range byPath {
if _, err := lib.db.ExecContext(
"UPDATE audio_files SET recording_mbid = ? WHERE file_path = ?",
mbid, path,
); err != nil {
t.Fatalf("set recording mbid for %s: %v", path, err)
}
}
backend/library/library.go
+370 -545
View File
File diff suppressed because it is too large Load Diff
backend/library/metrics.go
-2
View File
@@ -20,7 +20,6 @@ type ScanMetrics struct {
ExtractionWallClock time.Duration `json:"extractionWallClock"`
DBWritesWallClock time.Duration `json:"dbWritesWallClock"`
OrphanCleanup time.Duration `json:"orphanCleanup"`
PostScanVariants time.Duration `json:"postScanVariants"`
// Per-format extraction (cumulative across workers).
FormatExtraction map[string]int64 `json:"formatExtraction"`
@@ -137,7 +136,6 @@ func (m *ScanMetrics) timingBreakdown() string {
line(" Medium", m.ThumbnailMedium)
line(" Large", m.ThumbnailLarge)
line(" Orphan cleanup", m.OrphanCleanup)
line(" Post-scan variants", m.PostScanVariants)
return b.String()
}
backend/library/query.go
+301 -1064
View File
File diff suppressed because it is too large Load Diff
backend/library/removal_test.go
+18 -35
View File
@@ -6,6 +6,7 @@ import (
"testing"
"yellowjacket/backend/coverart"
"yellowjacket/backend/database"
"yellowjacket/backend/database/sql/sqlcgen"
)
@@ -30,19 +31,6 @@ func seedRemovableLibrary(
t.Fatalf("create library: %v", err)
}
ac, err := q.UpsertArtistCredit(ctx, "Test Artist")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
rec, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: "Test Song",
ArtistCreditID: ac.ID,
})
if err != nil {
t.Fatalf("create recording: %v", err)
}
if _, err := lib.db.ExecContext(
`INSERT INTO cover_art (is_embedded, file_path, mime_type)
VALUES (0, ?, 'image/jpeg')`, coverPath,
@@ -50,15 +38,14 @@ func seedRemovableLibrary(
t.Fatalf("insert cover art: %v", err)
}
if _, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/song.mp3",
LengthMilliseconds: 180000,
RecordingID: rec.ID,
LibraryID: library.ID,
Basename: "song.mp3",
}); err != nil {
t.Fatalf("create audio file: %v", err)
}
database.InsertTestTrack(t, lib.db, database.TestTrack{
FilePath: "/music/song.mp3",
Title: "Test Song",
Artist: "Test Artist",
Album: "Test Album",
LengthMs: 180000,
LibraryID: library.ID,
})
// Every scanned library gets tagging_items rows, one per album
// folder. These FK-reference libraries.
@@ -137,9 +124,9 @@ func TestRemoveLibrary_WithTaggingItems(t *testing.T) {
for _, table := range []string{
"audio_files",
"recordings",
"artist_credit",
"albums",
"artists",
"file_genres",
"tagging_items",
"tagging_candidates",
"cover_art",
@@ -158,18 +145,16 @@ func TestRemoveLibrary_DeletesCoverArtVariants(t *testing.T) {
lib, _ := setupTestLibrary(t)
dir := t.TempDir()
original := filepath.Join(dir, "abc123.jpg")
paths := []string{original}
var paths []string
for _, tier := range thumbnailTiers {
paths = append(paths, filepath.Join(
dir, coverart.SizedFilename("abc123.jpg", tier.Suffix),
))
}
paths = append(paths, filepath.Join(
dir, coverart.SizedFilename("abc123.jpg", legacyThumbSuffix),
))
// The largest tier is what cover_art.file_path names.
cover := filepath.Join(dir, coverart.SizedFilename("abc123.jpg", "_lg"))
for _, p := range paths {
if err := os.WriteFile(p, []byte("img"), 0o600); err != nil {
@@ -177,7 +162,7 @@ func TestRemoveLibrary_DeletesCoverArtVariants(t *testing.T) {
}
}
library := seedRemovableLibrary(t, lib, original)
library := seedRemovableLibrary(t, lib, cover)
if _, err := lib.RemoveLibrary(library.ID); err != nil {
t.Fatalf("RemoveLibrary: %v", err)
@@ -190,19 +175,17 @@ func TestRemoveLibrary_DeletesCoverArtVariants(t *testing.T) {
}
}
// CoverArtFileSet must cover the original, every generated tier, and
// the legacy _thumb name.
// CoverArtFileSet must cover every generated tier, from any of them:
// cover_art.file_path names the largest, and the others are derived.
func TestCoverArtFileSet(t *testing.T) {
t.Parallel()
got := CoverArtFileSet("/covers/abc123.jpg")
got := CoverArtFileSet("/covers/abc123_lg.jpg")
want := []string{
"/covers/abc123.jpg",
"/covers/abc123_sm.jpg",
"/covers/abc123_md.jpg",
"/covers/abc123_lg.jpg",
"/covers/abc123_thumb.jpg",
}
if len(got) != len(want) {
backend/library/remove_tracks.go
+1 -1
View File
@@ -142,7 +142,7 @@ func (l *Library) RemoveFromLibrary(filePaths []string) (*RemovalResult, error)
// with nothing behind it, and the album list selects from
// release_groups rather than from audio_files — so it would keep
// rendering an album the user has no tracks of.
l.pruneOrphanedMetadata()
l.pruneEmptyEntities()
l.emit(events.TracksRemovedFromLibrary, map[string]any{
"filePaths": filePaths,
backend/library/remove_tracks_test.go
+1 -1
View File
@@ -169,7 +169,7 @@ func TestRemoveFromLibrary_SoftScanSeesNoChange(t *testing.T) {
t.Fatalf("RemoveFromLibrary: %v", err)
}
dbCount, err := lib.db.Queries.CountAudioFilesByLibrary(t.Context(), libID)
dbCount, err := lib.db.Queries.CountAudioFiles(t.Context(), libID)
if err != nil {
t.Fatalf("count rows: %v", err)
}
backend/library/rescan.go
+9 -46
View File
@@ -141,23 +141,16 @@ func (l *Library) clearLibraryTables() error {
UPDATE playlist_tracks
SET
phantom_title = COALESCE(phantom_title, (
SELECT r.name FROM audio_files af
JOIN recordings r ON af.recording_id = r.id
WHERE af.id = playlist_tracks.audio_file_id
SELECT tm.title FROM track_metadata tm
WHERE tm.id = playlist_tracks.audio_file_id
)),
phantom_artist = COALESCE(phantom_artist, (
SELECT ac.text FROM audio_files af
JOIN recordings r ON af.recording_id = r.id
JOIN artist_credit ac ON r.artist_credit_id = ac.id
WHERE af.id = playlist_tracks.audio_file_id
SELECT tm.artist_name FROM track_metadata tm
WHERE tm.id = playlist_tracks.audio_file_id
)),
phantom_album = COALESCE(phantom_album, (
SELECT rg.name FROM audio_files af
JOIN recordings r ON af.recording_id = r.id
LEFT JOIN release_group_recordings rgr ON r.id = rgr.recording_id
LEFT JOIN release_groups rg ON rgr.release_group_id = rg.id
WHERE af.id = playlist_tracks.audio_file_id
LIMIT 1
SELECT tm.album FROM track_metadata tm
WHERE tm.id = playlist_tracks.audio_file_id
)),
phantom_duration_ms = COALESCE(phantom_duration_ms, (
SELECT af.length_milliseconds FROM audio_files af
@@ -174,40 +167,16 @@ func (l *Library) clearLibraryTables() error {
)
}
if err := txq.DeleteAllRecordingGenres(l.ctx); err != nil {
return fmt.Errorf(
"could not clear recording genres: %w", err,
)
}
if err := txq.DeleteAllReleaseGroupRecordings(l.ctx); err != nil {
return fmt.Errorf(
"could not clear release group recordings: %w", err,
)
}
if err := txq.DeleteAllArtistCreditArtists(l.ctx); err != nil {
return fmt.Errorf(
"could not clear artist credit artists: %w", err,
)
}
// Phase 2: mid-level tables.
// Phase 2: the files. file_genres cascades with them.
if err := txq.DeleteAllAudioFiles(l.ctx); err != nil {
return fmt.Errorf(
"could not clear audio files: %w", err,
)
}
if err := txq.DeleteAllReleaseGroups(l.ctx); err != nil {
if err := txq.DeleteAllAlbums(l.ctx); err != nil {
return fmt.Errorf(
"could not clear release groups: %w", err,
)
}
if err := txq.DeleteAllRecordings(l.ctx); err != nil {
return fmt.Errorf(
"could not clear recordings: %w", err,
"could not clear albums: %w", err,
)
}
@@ -218,12 +187,6 @@ func (l *Library) clearLibraryTables() error {
)
}
if err := txq.DeleteAllArtistCredits(l.ctx); err != nil {
return fmt.Errorf(
"could not clear artist credits: %w", err,
)
}
if err := txq.DeleteAllArtists(l.ctx); err != nil {
return fmt.Errorf(
"could not clear artists: %w", err,
backend/library/scan_dirdisc_test.go
+1 -1
View File
@@ -81,7 +81,7 @@ func scanTestGroupKeys(t *testing.T, lib *Library, root string) map[string]strin
t.Fatal("scanInternal returned nil metrics")
}
rows, err := lib.db.Queries.GetAudioFilesByLibrary(lib.ctx, library.ID)
rows, err := lib.db.Queries.GetAudioFilesInLibrary(lib.ctx, library.ID)
if err != nil {
t.Fatalf("list audio files: %v", err)
}
backend/library/scan_fixtures_test.go
+103
View File
@@ -0,0 +1,103 @@
package library
import (
"path/filepath"
"testing"
"yellowjacket/backend/database/sql/sqlcgen"
)
// TestScan_FixtureLibraryLeavesNothingBehind runs a real scan over the
// generated fixture library and asserts the invariant the file-shaped
// schema exists for: every row is a file's, and nothing outlives one.
//
// The old schema could not state this. A scan wrote a recording, an
// artist credit, a credit-artist link and a release-group link per
// file, all of which survived the file's deletion, and a real library
// accumulated 812 recordings, 216 release groups and 260 artists with
// nothing behind them - which is what made "do I own this" unanswerable.
func TestScan_FixtureLibraryLeavesNothingBehind(t *testing.T) {
t.Parallel()
lib, db := setupTestLibrary(t)
root, err := filepath.Abs("../../test_data/music_library_test")
if err != nil {
t.Fatalf("resolve fixture path: %v", err)
}
if _, err := filepath.Glob(filepath.Join(root, "*")); err != nil {
t.Skipf("fixture library not generated (make testdata): %v", err)
}
library, err := db.Queries.CreateLibrary(lib.ctx, sqlcgen.CreateLibraryParams{
Name: "Fixtures",
Path: root,
})
if err != nil {
t.Fatalf("create library: %v", err)
}
if metrics := lib.scanInternal(library.ID, library.Name, library.Path); metrics == nil {
t.Fatal("scanInternal returned nil metrics")
}
count := func(query string) int64 {
t.Helper()
var n int64
if err := db.QueryRowWriter(query).Scan(&n); err != nil {
t.Fatalf("%s: %v", query, err)
}
return n
}
files := count("SELECT COUNT(*) FROM audio_files")
if files == 0 {
t.Skip("fixture library is empty; run make testdata")
}
tracks, err := lib.GetTracks(0)
if err != nil {
t.Fatalf("GetTracks: %v", err)
}
// One track per file: the projection cannot multiply rows, because
// there is no join table left to multiply them.
if int64(len(tracks)) != files {
t.Errorf("GetTracks returned %d rows for %d files", len(tracks), files)
}
// Nothing shared outlives what refers to it.
for _, c := range []struct {
what string
query string
}{
{"albums with no file", `SELECT COUNT(*) FROM albums al
WHERE NOT EXISTS (SELECT 1 FROM audio_files af WHERE af.album_id = al.id)`},
{"artists nothing refers to", `SELECT COUNT(*) FROM artists a
WHERE NOT EXISTS (SELECT 1 FROM audio_files af WHERE af.artist_id = a.id)
AND NOT EXISTS (SELECT 1 FROM albums al WHERE al.artist_id = a.id)`},
{"genre links with no file", `SELECT COUNT(*) FROM file_genres fg
WHERE NOT EXISTS (SELECT 1 FROM audio_files af WHERE af.id = fg.audio_file_id)`},
} {
if n := count(c.query); n != 0 {
t.Errorf("%s = %d, want 0", c.what, n)
}
}
// And the scan actually filed things: albums, artists and genres
// all resolved, with the tags on the files that named them.
if albums, err := lib.GetAlbums(0); err != nil || len(albums) == 0 {
t.Errorf("GetAlbums = %d albums, err %v; want some", len(albums), err)
}
if artists, err := lib.GetArtists(0); err != nil || len(artists) == 0 {
t.Errorf("GetArtists = %d artists, err %v; want some", len(artists), err)
}
if genres, err := lib.GetGenres(0); err != nil || len(genres) == 0 {
t.Errorf("GetGenres = %d genres, err %v; want some", len(genres), err)
}
}
backend/library/scan_jobs.go
+2
View File
@@ -21,6 +21,8 @@ var scanPhaseLabels = map[string]string{
// SetJobRegistry wires the background job registry so scans report
// progress, logs, and pause/cancel controls to the frontend.
//
//wails:ignore // internal wiring, not part of the app's IPC surface.
func (l *Library) SetJobRegistry(reg *jobs.Registry) {
l.mu.Lock()
l.jobs = reg
backend/library/scan_queue.go
+1 -1
View File
@@ -149,7 +149,7 @@ func (l *Library) SoftScanAllLibraries() error {
continue
}
dbCount, countErr := l.db.Queries.CountAudioFilesByLibrary(
dbCount, countErr := l.db.Queries.CountAudioFiles(
l.ctx, lib.ID,
)
if countErr != nil {
backend/library/scan_test.go
+262 -585
View File
@@ -1,9 +1,7 @@
package library
import (
"context"
"database/sql"
"fmt"
"log/slog"
"sync/atomic"
"testing"
@@ -188,35 +186,28 @@ func TestSplitGenres(t *testing.T) {
}
}
func TestMapTrackRow(t *testing.T) {
func TestTrackFromRow(t *testing.T) {
t.Parallel()
track := mapTrackRow(
"/music/queen/bohemian.flac", // filePath
180000, // lengthMs
"Bohemian Rhapsody", // title
"Queen", // artistName
sql.NullInt64{Int64: 1, Valid: true}, // trackNumber
sql.NullInt64{Int64: 1, Valid: true}, // discNumber
"A Night at the Opera", // album
"Rock||Progressive Rock", // genre
1975, // year
"Freddie Mercury", // composer
".flac", // fileType
44100, // sampleRate
16, // bitDepth
2, // channels
1411, // bitrate
35000000, // fileSize
0, // playCount
sql.NullTime{}, // lastPlayed
"", // coverArtPath
"", // artistMBID
"", // releaseGroupMBID
"", // recordingMBID
)
track := trackFromRow(sqlcgen.TrackMetadatum{
FilePath: "/music/queen/bohemian.flac",
LengthMilliseconds: 180000,
Title: "Bohemian Rhapsody",
ArtistName: "Queen",
TrackNumber: sql.NullInt64{Int64: 1, Valid: true},
DiscNumber: sql.NullInt64{Int64: 1, Valid: true},
Album: "A Night at the Opera",
Genre: "Rock||Progressive Rock",
Year: 1975,
Composer: "Freddie Mercury",
FileType: ".flac",
SampleRate: 44100,
BitDepth: 16,
Channels: 2,
Bitrate: 1411,
FileSize: 35000000,
})
// Verify all 16 fields.
if track.TrackName != "Bohemian Rhapsody" {
t.Errorf("TrackName = %q, want %q", track.TrackName, "Bohemian Rhapsody")
}
@@ -225,50 +216,19 @@ func TestMapTrackRow(t *testing.T) {
t.Errorf("ArtistName = %q, want %q", track.ArtistName, "Queen")
}
// TrackLength is string-formatted milliseconds.
if track.TrackLength != "180000" {
t.Errorf("TrackLength = %q, want %q", track.TrackLength, "180000")
}
if track.FilePath != "/music/queen/bohemian.flac" {
t.Errorf("FilePath = %q, want %q", track.FilePath, "/music/queen/bohemian.flac")
}
if track.TrackNumber != 1 {
t.Errorf("TrackNumber = %d, want %d", track.TrackNumber, 1)
}
if track.DiscNumber != 1 {
t.Errorf("DiscNumber = %d, want %d", track.DiscNumber, 1)
}
if track.Album != "A Night at the Opera" {
t.Errorf("Album = %q, want %q", track.Album, "A Night at the Opera")
}
wantGenres := []string{"Rock", "Progressive Rock"}
if len(track.Genre) != len(wantGenres) {
t.Fatalf("Genre length = %d, want %d", len(track.Genre), len(wantGenres))
}
for i, g := range track.Genre {
if g != wantGenres[i] {
t.Errorf("Genre[%d] = %q, want %q", i, g, wantGenres[i])
}
if len(track.Genre) != 2 || track.Genre[0] != "Rock" ||
track.Genre[1] != "Progressive Rock" {
t.Errorf("Genre = %v, want [Rock, Progressive Rock]", track.Genre)
}
if track.Year != 1975 {
t.Errorf("Year = %d, want %d", track.Year, 1975)
}
if track.Composer != "Freddie Mercury" {
t.Errorf("Composer = %q, want %q", track.Composer, "Freddie Mercury")
}
if track.FileType != ".flac" {
t.Errorf("FileType = %q, want %q", track.FileType, ".flac")
}
if track.SampleRate != 44100 {
t.Errorf("SampleRate = %d, want %d", track.SampleRate, 44100)
}
@@ -289,16 +249,12 @@ func TestMapTrackRow(t *testing.T) {
t.Errorf("FileSize = %d, want %d", track.FileSize, 35000000)
}
// Verify NullInt64 with Valid=false yields 0.
trackNull := mapTrackRow(
"/music/unknown.mp3", 0, "Test", "Artist",
sql.NullInt64{}, sql.NullInt64{}, // invalid (null)
"", "", 0, "", "", 0, 0, 0, 0, 0,
0, // playCount
sql.NullTime{}, // lastPlayed
"", // coverArtPath
"", "", "", // artistMBID, releaseGroupMBID, recordingMBID
)
// A NULL track/disc number yields 0, not a panic.
trackNull := trackFromRow(sqlcgen.TrackMetadatum{
FilePath: "/music/unknown.mp3",
Title: "Test",
ArtistName: "Artist",
})
if trackNull.TrackNumber != 0 {
t.Errorf("null TrackNumber = %d, want 0", trackNull.TrackNumber)
@@ -307,11 +263,11 @@ func TestMapTrackRow(t *testing.T) {
if trackNull.DiscNumber != 0 {
t.Errorf("null DiscNumber = %d, want 0", trackNull.DiscNumber)
}
}
// ---------------------------------------------------------------------------
// Test helper — constructs a Library backed by an in-memory test DB
// ---------------------------------------------------------------------------
if trackNull.Genre != nil {
t.Errorf("empty Genre = %v, want nil", trackNull.Genre)
}
}
func setupTestLibrary(t *testing.T) (*Library, *database.DB) {
t.Helper()
@@ -335,129 +291,111 @@ func setupTestLibrary(t *testing.T) (*Library, *database.DB) {
// Entity cache tests — DB-backed
// ---------------------------------------------------------------------------
func TestCachedUpsertArtistCredit(t *testing.T) {
func TestCachedUpsertArtist(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
cache := newEntityCache()
q := lib.db.Queries
// First call — hits DB.
ac1, err := lib.cachedUpsertArtistCredit(q, cache, "Queen")
if err != nil {
t.Fatalf("first cachedUpsertArtistCredit: %v", err)
first := lib.cachedUpsertArtist(q, cache, "Queen", "")
if first.ID == 0 {
t.Fatal("expected non-zero artist ID")
}
if ac1.ID == 0 {
t.Fatal("expected non-zero ArtistCredit ID")
// Second call is a cache hit and returns the same row.
if second := lib.cachedUpsertArtist(q, cache, "Queen", ""); second.ID != first.ID {
t.Errorf("cache miss: got ID %d, want %d", second.ID, first.ID)
}
// Second call — cache hit, same ID.
ac2, err := lib.cachedUpsertArtistCredit(q, cache, "Queen")
if err != nil {
t.Fatalf("second cachedUpsertArtistCredit: %v", err)
if other := lib.cachedUpsertArtist(q, cache, "Beyonce", ""); other.ID == first.ID {
t.Errorf("different name returned same ID %d", other.ID)
}
if ac2.ID != ac1.ID {
t.Errorf("cache miss: got ID %d, want %d", ac2.ID, ac1.ID)
if len(cache.artists) != 2 {
t.Errorf("cache entries = %d, want 2", len(cache.artists))
}
// Different name — different ID.
ac3, err := lib.cachedUpsertArtistCredit(q, cache, "Beyoncé")
if err != nil {
t.Fatalf("cachedUpsertArtistCredit(Beyoncé): %v", err)
// An MBID arriving on a later file is written to the cached row -
// the first file of an album often has no MBID and a later one does.
withMBID := lib.cachedUpsertArtist(q, cache, "Queen", "mbid-queen")
if !withMBID.Mbid.Valid || withMBID.Mbid.String != "mbid-queen" {
t.Errorf("artist mbid = %v, want mbid-queen", withMBID.Mbid)
}
if ac3.ID == ac1.ID {
t.Errorf("different name returned same ID %d", ac3.ID)
}
// Cache should have 2 entries.
if len(cache.artistCredits) != 2 {
t.Errorf("cache entries = %d, want 2", len(cache.artistCredits))
// An empty name is not a missing row: it becomes "Unknown Artist",
// because a file with no artist tag still has to belong somewhere.
unknown := lib.cachedUpsertArtist(q, cache, "", "")
if unknown.Name != "Unknown Artist" {
t.Errorf("empty artist name = %q, want %q", unknown.Name, "Unknown Artist")
}
}
func TestCachedLinkArtist(t *testing.T) {
func TestCachedUpsertAlbum(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
cache := newEntityCache()
q := lib.db.Queries
metrics := newScanMetrics()
// Create an artist credit first.
ac, err := lib.cachedUpsertArtistCredit(q, cache, "Queen")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
first := lib.cachedUpsertAlbum(q, cache, albumParams{
name: "A Night at the Opera",
credit: "Queen",
})
if first.ID == 0 {
t.Fatal("expected non-zero album ID")
}
// First link — creates artist + artist-credit-artist link.
lib.cachedLinkArtist(q, cache, metrics, "Queen", ac.ID)
if len(cache.artists) != 1 {
t.Errorf("artists cache = %d, want 1", len(cache.artists))
same := lib.cachedUpsertAlbum(q, cache, albumParams{
name: "A Night at the Opera",
credit: "Queen",
})
if same.ID != first.ID {
t.Errorf("cache miss: got ID %d, want %d", same.ID, first.ID)
}
if len(cache.linkedCredits) != 1 {
t.Errorf("linkedCredits cache = %d, want 1", len(cache.linkedCredits))
}
// Second call with same args — should skip (cache hit).
lib.cachedLinkArtist(q, cache, metrics, "Queen", ac.ID)
if len(cache.linkedCredits) != 1 {
t.Errorf(
"linkedCredits after duplicate = %d, want 1 (should skip)",
len(cache.linkedCredits),
)
// Album identity is (name, credit), so the same title by someone
// else is a different album.
other := lib.cachedUpsertAlbum(q, cache, albumParams{
name: "A Night at the Opera",
credit: "Blind Guardian",
})
if other.ID == first.ID {
t.Error("same album name by a different artist collapsed into one album")
}
}
func TestCachedLinkArtist_MultiCredit(t *testing.T) {
func TestCachedUpsertAlbum_FillsCoverArtLater(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
cache := newEntityCache()
q := lib.db.Queries
metrics := newScanMetrics()
// Two different artist credits referencing the same artist name.
ac1, err := lib.cachedUpsertArtistCredit(q, cache, "Queen")
album := lib.cachedUpsertAlbum(q, cache, albumParams{name: "Art", credit: "A"})
ca, err := q.UpsertCoverArt(lib.ctx, sqlcgen.UpsertCoverArtParams{
FilePath: "/covers/art.jpg",
MimeType: "image/jpeg",
})
if err != nil {
t.Fatalf("upsert credit 1: %v", err)
t.Fatalf("upsert cover art: %v", err)
}
ac2, err := lib.cachedUpsertArtistCredit(q, cache, "Queen feat. David Bowie")
if err != nil {
t.Fatalf("upsert credit 2: %v", err)
// The first file of an album often carries no embedded art and a
// later one does; the album has to pick it up.
withArt := lib.cachedUpsertAlbum(q, cache, albumParams{
name: "Art",
credit: "A",
coverArtID: sql.NullInt64{Int64: ca.ID, Valid: true},
})
if withArt.ID != album.ID {
t.Fatalf("album ID changed: got %d, want %d", withArt.ID, album.ID)
}
// Link "Queen" artist to both credits.
lib.cachedLinkArtist(q, cache, metrics, "Queen", ac1.ID)
lib.cachedLinkArtist(q, cache, metrics, "Queen", ac2.ID)
// Artist cached once.
if len(cache.artists) != 1 {
t.Errorf("artists cache = %d, want 1 (same artist name)", len(cache.artists))
}
// Two distinct linked-credit entries.
if len(cache.linkedCredits) != 2 {
t.Errorf("linkedCredits = %d, want 2", len(cache.linkedCredits))
}
// Verify link keys are correct format.
queenArtist := cache.artists["Queen"]
key1 := fmt.Sprintf("%d:%d", queenArtist.ID, ac1.ID)
key2 := fmt.Sprintf("%d:%d", queenArtist.ID, ac2.ID)
if _, ok := cache.linkedCredits[key1]; !ok {
t.Errorf("missing linked credit key %q", key1)
}
if _, ok := cache.linkedCredits[key2]; !ok {
t.Errorf("missing linked credit key %q", key2)
if !withArt.CoverArtID.Valid || withArt.CoverArtID.Int64 != ca.ID {
t.Errorf("cover art = %v, want %d", withArt.CoverArtID, ca.ID)
}
}
@@ -468,460 +406,83 @@ func TestCachedUpsertGenre(t *testing.T) {
cache := newEntityCache()
q := lib.db.Queries
// First call — creates genre.
g1, err := lib.cachedUpsertGenre(q, cache, "Rock")
first, err := lib.cachedUpsertGenre(q, cache, "Rock")
if err != nil {
t.Fatalf("first cachedUpsertGenre: %v", err)
t.Fatalf("cachedUpsertGenre: %v", err)
}
if g1.ID == 0 {
t.Fatal("expected non-zero Genre ID")
}
// Second call — cache hit.
g2, err := lib.cachedUpsertGenre(q, cache, "Rock")
second, err := lib.cachedUpsertGenre(q, cache, "Rock")
if err != nil {
t.Fatalf("second cachedUpsertGenre: %v", err)
t.Fatalf("cachedUpsertGenre (cached): %v", err)
}
if g2.ID != g1.ID {
t.Errorf("cache miss: got ID %d, want %d", g2.ID, g1.ID)
}
if len(cache.genres) != 1 {
t.Errorf("genre cache entries = %d, want 1", len(cache.genres))
if second.ID != first.ID {
t.Errorf("cache miss: got ID %d, want %d", second.ID, first.ID)
}
}
func TestResolveReleaseGroup(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
cache := newEntityCache()
q := lib.db.Queries
// Need an album artist credit for the release group.
ac, err := lib.cachedUpsertArtistCredit(q, cache, "Queen")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
albumArtistCreditID := sql.NullInt64{Int64: ac.ID, Valid: true}
// First call — no cover art.
tags := &metadata.TrackMetadata{
Album: "A Night at the Opera",
Year: 1975,
}
rgID := lib.resolveReleaseGroup(q, cache, tags, albumArtistCreditID, sql.NullInt64{})
if !rgID.Valid {
t.Fatal("expected valid release group ID")
}
if rgID.Int64 == 0 {
t.Fatal("expected non-zero release group ID")
}
// Verify cached.
if len(cache.releaseGroups) != 1 {
t.Errorf("releaseGroups cache = %d, want 1", len(cache.releaseGroups))
}
// Second call — same album with cover art → should update cover art on cached entry.
// First, create a cover art record in the DB.
coverArt, err := q.UpsertCoverArt(lib.ctx, sqlcgen.UpsertCoverArtParams{
IsEmbedded: true,
FilePath: "/covers/opera.jpg",
MimeType: "image/jpeg",
})
if err != nil {
t.Fatalf("create cover art: %v", err)
}
coverArtID := sql.NullInt64{Int64: coverArt.ID, Valid: true}
rgID2 := lib.resolveReleaseGroup(q, cache, tags, albumArtistCreditID, coverArtID)
if rgID2.Int64 != rgID.Int64 {
t.Errorf("cache miss: got ID %d, want %d", rgID2.Int64, rgID.Int64)
}
// Cover art should be updated on the cached release group.
// Cache key is composite: "albumName\x00artistCreditID".
cacheKey := fmt.Sprintf("%s\x00%d", "A Night at the Opera", ac.ID)
cachedRG := cache.releaseGroups[cacheKey]
if !cachedRG.CoverArtID.Valid {
t.Error("expected CoverArtID to be set after update")
}
if cachedRG.CoverArtID.Int64 != coverArt.ID {
t.Errorf("CoverArtID = %d, want %d", cachedRG.CoverArtID.Int64, coverArt.ID)
}
// Empty album → invalid NullInt64.
emptyTags := &metadata.TrackMetadata{Album: ""}
rgEmpty := lib.resolveReleaseGroup(q, cache, emptyTags, albumArtistCreditID, sql.NullInt64{})
if rgEmpty.Valid {
t.Errorf("empty album should return invalid NullInt64, got valid with ID %d", rgEmpty.Int64)
}
}
func TestResolveReleaseGroup_CacheHit(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
cache := newEntityCache()
q := lib.db.Queries
// Pre-populate cache with a known release group.
// Cache key is composite: "albumName\x00artistCreditID" (use -1 for no artist).
cache.releaseGroups[fmt.Sprintf("%s\x00%d", "Cached Album", int64(-1))] = sqlcgen.ReleaseGroup{
ID: 42,
Name: "Cached Album",
}
tags := &metadata.TrackMetadata{Album: "Cached Album"}
rgID := lib.resolveReleaseGroup(q, cache, tags, sql.NullInt64{}, sql.NullInt64{})
if !rgID.Valid {
t.Fatal("expected valid release group ID from cache")
}
if rgID.Int64 != 42 {
t.Errorf("resolveReleaseGroup() = %d, want 42 (cached)", rgID.Int64)
}
}
// ---------------------------------------------------------------------------
// Orphan cleanup test — DB-level
// ---------------------------------------------------------------------------
func TestOrphanDeletion(t *testing.T) {
t.Parallel()
_, db := setupTestLibrary(t)
ctx := context.Background()
q := db.Queries
// Seed an artist credit → recording → audio file chain.
ac, err := q.UpsertArtistCredit(ctx, "Test Artist")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
rec, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: "Test Song",
ArtistCreditID: ac.ID,
})
if err != nil {
t.Fatalf("create recording: %v", err)
}
af, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/test.mp3",
LengthMilliseconds: 180000,
FileTypeID: 0,
RecordingID: rec.ID,
Basename: "test.mp3",
})
if err != nil {
t.Fatalf("create audio file: %v", err)
}
// Add FTS search index entry.
if err := db.InsertSearchIndex(
af.ID, "/music/test.mp3", "Test Song", "Test Artist", "",
); err != nil {
t.Fatalf("insert search index: %v", err)
}
// Verify the search index entry exists before deletion.
results, err := db.SearchFTS("Test Song", 10)
if err != nil {
t.Fatalf("search before delete: %v", err)
}
if len(results) != 1 {
t.Fatalf("search results before delete = %d, want 1", len(results))
}
// Delete audio file — this is the primary orphan cleanup step.
if err := q.DeleteAudioFile(ctx, af.ID); err != nil {
t.Fatalf("delete audio file: %v", err)
}
// Verify audio file is gone by attempting to query all audio files.
allFiles, err := q.GetAllAudioFiles(ctx)
if err != nil {
t.Fatalf("get all audio files: %v", err)
}
if len(allFiles) != 0 {
t.Errorf("audio files after delete = %d, want 0", len(allFiles))
}
// DeleteSearchIndex on contentless FTS5 table (content='') is
// expected to error. The production orphan cleanup code in
// library.go logs this as a warning — the search index entries
// become stale but harmless (they reference a non-existent
// audio_file ID, so JOINs return no results).
// ClearSearchIndex (used during full rescan) handles bulk cleanup.
// DeleteSearchIndex on contentless FTS5 is expected to error.
// Not a fatal error — documents the contentless FTS5 limitation.
err = db.DeleteSearchIndex(af.ID)
if err == nil {
t.Log("DeleteSearchIndex succeeded (unexpected for contentless FTS5)")
}
}
func TestPruneOrphanedMetadata(t *testing.T) {
// TestPruneEmptyEntities is what is left of four orphan-sweep tests.
//
// Three of the tables they covered are gone, and with them the bug they
// were guarding: a file used to create a recording, a credit, a
// credit-artist link and a release-group link, none of which were
// deleted when the file was, so a real library accumulated 812
// recordings, 216 release groups and 260 artists with nothing behind
// them. Two tables can still be left empty by a removal, and this is
// that.
func TestPruneEmptyEntities(t *testing.T) {
t.Parallel()
lib, db := setupTestLibrary(t)
ctx := context.Background()
q := db.Queries
// Seed a full chain: artist -> artist_credit -> recording -> audio_file,
// plus a release group crediting the same artist.
artist, err := q.UpsertArtist(ctx, "Orphaned Artist")
if err != nil {
t.Fatalf("upsert artist: %v", err)
}
ac, err := q.UpsertArtistCredit(ctx, "Orphaned Artist")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
if _, err := q.CreateArtistCreditArtist(ctx, sqlcgen.CreateArtistCreditArtistParams{
ArtistID: artist.ID,
CreditID: ac.ID,
}); err != nil {
t.Fatalf("link artist credit artist: %v", err)
}
rec, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: "Orphaned Song",
ArtistCreditID: ac.ID,
kept := database.InsertTestTrack(t, db, database.TestTrack{
FilePath: "/music/kept.mp3",
Title: "Kept",
Artist: "Kept Artist",
Album: "Kept Album",
Genres: []string{"Kept Genre"},
})
if err != nil {
t.Fatalf("create recording: %v", err)
}
rg, err := q.CreateReleaseGroupFull(ctx, sqlcgen.CreateReleaseGroupFullParams{
Name: "Orphaned Album",
AlbumArtistCreditID: sql.NullInt64{Int64: ac.ID, Valid: true},
gone := database.InsertTestTrack(t, db, database.TestTrack{
FilePath: "/music/gone.mp3",
Title: "Gone",
Artist: "Gone Artist",
Album: "Gone Album",
Genres: []string{"Gone Genre"},
})
if err != nil {
t.Fatalf("create release group: %v", err)
}
if _, err := q.CreateReleaseGroupRecording(ctx, sqlcgen.CreateReleaseGroupRecordingParams{
ReleaseGroupID: rg.ID,
RecordingID: rec.ID,
}); err != nil {
t.Fatalf("link release group recording: %v", err)
}
_ = kept
af, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/orphaned.mp3",
LengthMilliseconds: 180000,
FileTypeID: 0,
RecordingID: rec.ID,
Basename: "orphaned.mp3",
})
if err != nil {
t.Fatalf("create audio file: %v", err)
}
// Simulate a rescan removing the file: delete the audio_files row
// (what the existing Phase 5 orphan cleanup does), then run the new
// metadata cleanup that's supposed to cascade the rest.
if err := q.DeleteAudioFile(ctx, af.ID); err != nil {
if err := db.Queries.DeleteAudioFile(lib.ctx, gone); err != nil {
t.Fatalf("delete audio file: %v", err)
}
lib.pruneOrphanedMetadata()
lib.pruneEmptyEntities()
if _, err := q.GetRecording(ctx, rec.ID); err == nil {
t.Error("expected orphaned recording to be deleted")
}
for _, c := range []struct {
table string
name string
want int
}{
{"albums", "Gone Album", 0},
{"albums", "Kept Album", 1},
{"artists", "Gone Artist", 0},
{"artists", "Kept Artist", 1},
{"genres", "Gone Genre", 0},
{"genres", "Kept Genre", 1},
} {
var n int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM "+c.table+" WHERE name = ?", c.name,
).Scan(&n); err != nil {
t.Fatalf("count %s %q: %v", c.table, c.name, err)
}
if _, err := q.GetReleaseGroup(ctx, rg.ID); err == nil {
t.Error("expected orphaned release group to be deleted")
}
if _, err := q.GetArtistCredit(ctx, ac.ID); err == nil {
t.Error("expected orphaned artist credit to be deleted")
}
if _, err := q.GetArtist(ctx, artist.ID); err == nil {
t.Error("expected orphaned artist to be deleted")
if n != c.want {
t.Errorf("%s %q rows = %d, want %d", c.table, c.name, n, c.want)
}
}
}
// TestPruneOrphanedMetadata_KeepsStillOwnedEntities verifies that pruning
// only removes rows with no remaining audio_files, leaving an artist who
// still owns other tracks untouched.
func TestPruneOrphanedMetadata_KeepsStillOwnedEntities(t *testing.T) {
t.Parallel()
lib, db := setupTestLibrary(t)
ctx := context.Background()
q := db.Queries
artist, err := q.UpsertArtist(ctx, "Still Owned Artist")
if err != nil {
t.Fatalf("upsert artist: %v", err)
}
ac, err := q.UpsertArtistCredit(ctx, "Still Owned Artist")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
if _, err := q.CreateArtistCreditArtist(ctx, sqlcgen.CreateArtistCreditArtistParams{
ArtistID: artist.ID,
CreditID: ac.ID,
}); err != nil {
t.Fatalf("link artist credit artist: %v", err)
}
// Two recordings under the same artist credit; only one loses its file.
recGone, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: "Removed Song",
ArtistCreditID: ac.ID,
})
if err != nil {
t.Fatalf("create recording (removed): %v", err)
}
recKept, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: "Kept Song",
ArtistCreditID: ac.ID,
})
if err != nil {
t.Fatalf("create recording (kept): %v", err)
}
afGone, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/gone.mp3",
LengthMilliseconds: 180000,
FileTypeID: 0,
RecordingID: recGone.ID,
Basename: "gone.mp3",
})
if err != nil {
t.Fatalf("create audio file (gone): %v", err)
}
if _, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/kept.mp3",
LengthMilliseconds: 180000,
FileTypeID: 0,
RecordingID: recKept.ID,
Basename: "kept.mp3",
}); err != nil {
t.Fatalf("create audio file (kept): %v", err)
}
if err := q.DeleteAudioFile(ctx, afGone.ID); err != nil {
t.Fatalf("delete audio file: %v", err)
}
lib.pruneOrphanedMetadata()
if _, err := q.GetRecording(ctx, recGone.ID); err == nil {
t.Error("expected orphaned recording to be deleted")
}
if _, err := q.GetRecording(ctx, recKept.ID); err != nil {
t.Errorf("expected still-owned recording to survive, got: %v", err)
}
if _, err := q.GetArtistCredit(ctx, ac.ID); err != nil {
t.Errorf("expected still-referenced artist credit to survive, got: %v", err)
}
if _, err := q.GetArtist(ctx, artist.ID); err != nil {
t.Errorf("expected still-referenced artist to survive, got: %v", err)
}
}
// ---------------------------------------------------------------------------
// Empty/missing metadata tests
// ---------------------------------------------------------------------------
func TestEntityCache_EmptyFields(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
cache := newEntityCache()
q := lib.db.Queries
metrics := newScanMetrics()
// Empty artist credit name — documents behavior (creates "" credit).
ac, err := lib.cachedUpsertArtistCredit(q, cache, "")
if err != nil {
t.Fatalf("cachedUpsertArtistCredit with empty name: %v", err)
}
if ac.ID == 0 {
t.Error("expected non-zero ID even for empty artist credit name")
}
// Empty album → resolveReleaseGroup returns invalid NullInt64.
tags := &metadata.TrackMetadata{Album: ""}
rgID := lib.resolveReleaseGroup(q, cache, tags, sql.NullInt64{}, sql.NullInt64{})
if rgID.Valid {
t.Errorf("empty album should return invalid NullInt64, got valid ID %d", rgID.Int64)
}
// resolveAlbumArtistCredit with empty AlbumArtist reuses track artist credit.
trackTags := &metadata.TrackMetadata{
Artist: "Queen",
AlbumArtist: "",
}
trackAC, err := lib.cachedUpsertArtistCredit(q, cache, "Queen")
if err != nil {
t.Fatalf("upsert track artist credit: %v", err)
}
albumACID := lib.resolveAlbumArtistCredit(q, cache, metrics, trackTags, trackAC.ID)
if !albumACID.Valid {
t.Fatal("expected valid album artist credit ID when AlbumArtist is empty")
}
if albumACID.Int64 != trackAC.ID {
t.Errorf(
"empty AlbumArtist should reuse track credit: got %d, want %d",
albumACID.Int64, trackAC.ID,
)
}
// resolveAlbumArtistCredit when AlbumArtist matches Artist also reuses.
sameTags := &metadata.TrackMetadata{
Artist: "Queen",
AlbumArtist: "Queen",
}
sameACID := lib.resolveAlbumArtistCredit(q, cache, metrics, sameTags, trackAC.ID)
if sameACID.Int64 != trackAC.ID {
t.Errorf(
"matching AlbumArtist should reuse track credit: got %d, want %d",
sameACID.Int64, trackAC.ID,
)
}
}
// ---------------------------------------------------------------------------
// commitBatch + tagging_items bookkeeping (phase 008.3)
// ---------------------------------------------------------------------------
func TestCommitBatch_TaggingItemsBookkeeping(t *testing.T) {
t.Parallel()
@@ -1146,6 +707,122 @@ func TestCommitBatch_AlbumTagChangeKeepsGroup(t *testing.T) {
}
}
func TestCommitBatch_RescanPromotesTagStatus(t *testing.T) {
t.Parallel()
// Only the insert path stamps tag_status, so a file another
// tagger stamped with MBIDs after import used to keep 'untagged'
// for ever — and its folder kept asking to be tagged, since that
// column is what the autotag queue reads.
lib, db := setupTestLibrary(t)
cache := newEntityCache()
metrics := newScanMetrics()
var added, updated, skipped atomic.Int64
const path = "/music/Artist/Album Folder/01.mp3"
initial := []importResult{
{
absolutePath: path,
fileType: metadata.MP3,
lengthMillis: 200000,
tags: &metadata.TrackMetadata{
Title: "Track", Artist: "Artist", AlbumArtist: "Artist",
Album: "Album",
},
libraryID: 0,
},
}
if err := lib.commitBatch(
initial, cache, metrics, &added, &updated, &skipped, nil,
); err != nil {
t.Fatalf("initial commitBatch: %v", err)
}
fileID := queryInt(t, db,
`SELECT id FROM audio_files WHERE file_path = ?`, path,
)
if got := queryString(t, db,
`SELECT tag_status FROM audio_files WHERE id = ?`, fileID,
); got != "untagged" {
t.Fatalf("tag_status after import = %q, want %q", got, "untagged")
}
update := []importResult{
{
absolutePath: path,
fileType: metadata.MP3,
lengthMillis: 200000,
existingFileID: fileID,
needsUpdate: true,
tags: &metadata.TrackMetadata{
Title: "Track", Artist: "Artist", AlbumArtist: "Artist",
Album: "Album",
RecordingMBID: "11111111-2222-3333-4444-555555555555",
},
libraryID: 0,
},
}
if err := lib.commitBatch(
update, cache, metrics, &added, &updated, &skipped, nil,
); err != nil {
t.Fatalf("update commitBatch: %v", err)
}
if got := queryString(t, db,
`SELECT tag_status FROM audio_files WHERE id = ?`, fileID,
); got != "user_confirmed" {
t.Errorf("tag_status after rescan = %q, want %q", got, "user_confirmed")
}
// A deliberate "never ask me about this file again" outranks the
// promotion: the guard is on 'untagged', not on the MBID.
if _, err := db.ExecContext(
`UPDATE audio_files SET tag_status = 'user_skipped_permanent' WHERE id = ?`,
fileID,
); err != nil {
t.Fatalf("mark skipped: %v", err)
}
if err := lib.commitBatch(
update, cache, metrics, &added, &updated, &skipped, nil,
); err != nil {
t.Fatalf("second update commitBatch: %v", err)
}
if got := queryString(t, db,
`SELECT tag_status FROM audio_files WHERE id = ?`, fileID,
); got != "user_skipped_permanent" {
t.Errorf("tag_status = %q, want the skip to survive a rescan", got)
}
}
// queryString is queryInt's text counterpart.
func queryString(t *testing.T, db *database.DB, query string, args ...any) string {
t.Helper()
rows, err := db.QueryContext(query, args...)
if err != nil {
t.Fatalf("query %q: %v", query, err)
}
defer func() { _ = rows.Close() }()
var out string
if rows.Next() {
if scanErr := rows.Scan(&out); scanErr != nil {
t.Fatalf("scan: %v", scanErr)
}
}
return out
}
// queryInt runs a single-column scalar query and returns the first
// int64 result; fails the test on any error.
func queryInt(t *testing.T, db *database.DB, query string, args ...any) int64 {
backend/library/staleness_test.go
+21 -34
View File
@@ -6,6 +6,7 @@ import (
"testing"
"time"
"yellowjacket/backend/database"
"yellowjacket/backend/database/sql/sqlcgen"
)
@@ -179,50 +180,36 @@ func TestFlushStatBackfill(t *testing.T) {
ctx := lib.ctx
q := db.Queries
ac, err := q.UpsertArtistCredit(ctx, "Test Artist")
if err != nil {
t.Fatalf("upsert artist credit: %v", err)
}
rec, err := q.CreateRecordingFull(ctx, sqlcgen.CreateRecordingFullParams{
Name: "Test Song",
ArtistCreditID: ac.ID,
// Seed two rows with no staleness baseline, as an older install
// leaves them.
first := database.InsertTestTrack(t, db, database.TestTrack{
FilePath: "/music/first.mp3",
Title: "Test Song",
Artist: "Test Artist",
LengthMs: 180000,
})
if err != nil {
t.Fatalf("create recording: %v", err)
}
// Seed two rows with no baseline, as migration 47 leaves them.
first, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/first.mp3",
LengthMilliseconds: 180000,
RecordingID: rec.ID,
Basename: "first.mp3",
second := database.InsertTestTrack(t, db, database.TestTrack{
FilePath: "/music/second.mp3",
Title: "Test Song",
Artist: "Test Artist",
LengthMs: 200000,
})
seeded, err := q.GetAudioFile(ctx, first)
if err != nil {
t.Fatalf("create first audio file: %v", err)
t.Fatalf("get seeded file: %v", err)
}
second, err := q.CreateAudioFile(ctx, sqlcgen.CreateAudioFileParams{
FilePath: "/music/second.mp3",
LengthMilliseconds: 200000,
RecordingID: rec.ID,
Basename: "second.mp3",
})
if err != nil {
t.Fatalf("create second audio file: %v", err)
}
if first.ModifiedAt != 0 {
t.Fatalf("seeded ModifiedAt = %d, want 0", first.ModifiedAt)
if seeded.ModifiedAt != 0 {
t.Fatalf("seeded ModifiedAt = %d, want 0", seeded.ModifiedAt)
}
lib.flushStatBackfill([]sqlcgen.UpdateAudioFileStatParams{
{ModifiedAt: 1700000000, FileSize: 4096, ID: first.ID},
{ModifiedAt: 1700000500, FileSize: 8192, ID: second.ID},
{ModifiedAt: 1700000000, FileSize: 4096, ID: first},
{ModifiedAt: 1700000500, FileSize: 8192, ID: second},
})
got, err := q.GetAudioFile(ctx, first.ID)
got, err := q.GetAudioFile(ctx, first)
if err != nil {
t.Fatalf("get first audio file: %v", err)
}