perf(explore): make the owned-artist backfill yield, mark, and stop

The post-scan backfills share MusicBrainz's rate limiters with every
page the user can open, and both were FIFO — so a thousand-artist
enrichment put an album page behind an hour of queued work.
WithBackgroundLane/WithBackgroundPriority add a slower second lane: a
marked wait takes no token while any interactive wait is outstanding.
It is a context marker rather than a parameter because a backfill calls
the same client methods a detail page does. A long backfill also has to
be visible and stoppable, so jobs.KindCatalogEnrich registers both with
progress and cancel — after the work is counted, since these passes are
a no-op on every launch once the library is covered.

What it does not fetch is the point. It ran for hours against a
900-artist library and marked nothing, because three of the four things
it did per artist were work nobody asked for: similar artists, which
the artist page already resolves on view, and a full GetArtistImage
(fanart.tv, TheAudioDB, Wikidata, Wikipedia, ten portraits) reached
only to warm the MB artist lookup EnsureArtistRels does alone. It was
also serial across artists while every limiter is per-host and idle.

The marks are a table rather than more explore_index columns, because
artifactimport merges by column list and a flag added there is a second
place to remember. BrowseReleaseGroupsAll pages to exhaustion, where
the old call silently cut a prolific artist at 100 release groups.

One portrait is downloaded now; the rest are remembered as URLs.
resolveAllSources downloaded every candidate, up to ten, full size,
while nothing reads anything but primary.jpg — 5.3 GB measured on a
real cache, 4.1 GB of it unreachable. OrphanedArtistImagesJob is why
that survived: it joined the bare MBID onto the images directory, but
artist directories are sharded under a two-character prefix, so it
named a path that never existed and deleted the rows that were the only
record of the files it left behind.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UDCbcCZQepnpSQYJ6SxxZm
This commit is contained in:
2026-08-14 13:33:54 -04:00
co-authored by Claude Opus 5
parent 878cf4b561
commit 20fbf28f2a
22 changed files with 2058 additions and 236 deletions
+17 -1
View File
@@ -182,6 +182,16 @@ func NewYellowJacketApp(
yjApp.library.SetJobRegistry(yjApp.jobs)
yjApp.explore.SetJobRegistry(yjApp.jobs)
// Let the release prefetch skip albums the user already owns in
// full — those open with no catalog call at all, so warming their
// tracklists spends the most expensive request in the app on
// nothing. Injected because neither package imports the other.
yjApp.explore.SetAlbumComplete(func(albumID int64) bool {
c, err := yjApp.library.GetAlbumCompleteness(albumID)
return err == nil && c.Known && c.Complete
})
// create autotag service (depends on explore + tagWriter)
yjApp.autotag = autotagservice.NewService(
yjApp.logger.WithGroup("autotag"),
@@ -697,7 +707,13 @@ func (yj *YellowJacketApp) startJanitor() {
yj.database, coversDir, library.CoverArtFileSet,
))
yj.janitor.Register(maintenance.OrphanedArtistImagesJob(
yj.database, filepath.Join(dataDir, explore.ArtistImageDirName),
yj.database,
filepath.Join(dataDir, explore.ArtistImageDirName),
explore.ArtistImageDir,
))
yj.janitor.Register(maintenance.StrayArtistImageFilesJob(
filepath.Join(dataDir, explore.ArtistImageDirName),
explore.ArtistImageKeepNames(),
))
yj.janitor.Register(maintenance.ExpiredProxyCacheJob(
filepath.Join(dataDir, explore.CoverArtCacheDirName),