feat(explore): carry multi-artist credits in the catalog
A track credited to more than one artist has exactly one navigable artist in this app and the rest are punctuation. `primaryArtist()` string-parses the credit, strips a " feat. " clause and discards the guest; it deliberately does not split on "&", "with" or "," because those live inside real artist names. Measured on a real 26,069-file library plus an 80+80 MusicBrainz sample: 13% of recordings are multi-artist upstream, while only 0.86% of files carry any structured multi-artist tag — mp3 carries zero files with multiple MUSICBRAINZ_ARTISTID across 19,840. Of 1,286 files saying "feat.", 90% have nothing structured behind it, and a sample of 80 such files was multi-artist in MB 80 times out of 80. CLAUDE.md justified plan 013's removal of the credit tables with "3 credits of 2,823 listed more than one artist". That measured our own *writer* — cachedLinkArtist was called once per credit, so a collaboration could never have been recorded. Dropping the join table was still right on cost; the evidence for "multi-artist is rare" was not. A credit is ordered parts and the credit string is derived from them, so join phrases are assembly instructions, not disassembly ones. Nothing here reconstructs a credit by searching a name inside a credit string: the stored text may come from tags while the parts come from the catalog, and those disagree for ~1 in 3 multi-artist credits. Where it comes from, after two dead ends: the canonical dump CI already streams has no join phrases and no as-credited names, and the JSON dumps cover 153,691 recordings of ~35M with *zero* overlap against a real library. So mbdump.tar.bz2 — 7.1 GB, ~13.7 min in pure-Go bzip2, whose members are alphabetical, which is what lets one pass resolve an entity's credit without buffering 35M recordings. - artist_credit_part / artist_credit_ref, multi-artist credits only: a single-artist credit is already explore_index's own artist_name. - Column layouts verified against the real 20260815 export; ErrDumpShape makes a wrong guess a failed build, not a wrong catalog. - The pass runs on every mode, not just a build. The job picks its mode from the index's own state, and a complete import means "refresh", which never enters the importer — so credits could otherwise only arrive via a rebuild that re-downloads ~205 GB. It reports whether it populated anything, which is what flips `changed` and republishes. - The importer asks whether an artifact carries the tables, on the writer where `core` is attached, so the artifact already published still imports. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
This commit is contained in:
+14
-1
@@ -177,12 +177,25 @@ func run(o opts) error {
|
||||
|
||||
complete := svc.IndexImportComplete() && !errors.Is(err, errIncomplete)
|
||||
|
||||
// Credits are maintenance, not part of any one mode. They come from
|
||||
// a different dump, they are keyed on entities the catalog already
|
||||
// holds, and a catalog built before the pass existed would otherwise
|
||||
// only gain them from a rebuild — which re-downloads ~205 GB to
|
||||
// re-derive rows it already has. Skipped when the import is not
|
||||
// complete, because there is nothing to key them against yet.
|
||||
creditsAdded := false
|
||||
if complete {
|
||||
creditsAdded = svc.EnsureArtistCredits(context.Background())
|
||||
}
|
||||
|
||||
// "Changed" means there is something new worth publishing, so it is
|
||||
// only ever true for a finished import: a build stamps the listens
|
||||
// series early, long before its rows are assembled, and reporting a
|
||||
// change off that would be a lie about a half-built index.
|
||||
changed := complete &&
|
||||
(svc.IndexBaselineSeries() != seriesBefore || chosen != modeRefresh)
|
||||
(svc.IndexBaselineSeries() != seriesBefore ||
|
||||
chosen != modeRefresh ||
|
||||
creditsAdded)
|
||||
|
||||
report(logger, svc, chosen, complete, changed)
|
||||
|
||||
|
||||
@@ -171,6 +171,26 @@ func createSchema(db *sql.DB) error {
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
)`,
|
||||
// Multi-artist credits. Shipped as their own tables rather than
|
||||
// as an explore_index column because a credit is a variable
|
||||
// number of ordered parts, and because credits are *shared* --
|
||||
// an album's tracks by one artist reference one credit, which is
|
||||
// what keeps this to a few hundred thousand rows.
|
||||
//
|
||||
// An importer that predates these reads an artifact without
|
||||
// them; artifactHasCredits is what asks.
|
||||
`CREATE TABLE core.artist_credit_part (
|
||||
credit_id INTEGER NOT NULL,
|
||||
position INTEGER NOT NULL,
|
||||
artist_mbid BLOB NOT NULL,
|
||||
credited_name TEXT NOT NULL,
|
||||
join_phrase TEXT NOT NULL DEFAULT '',
|
||||
PRIMARY KEY (credit_id, position)
|
||||
) WITHOUT ROWID`,
|
||||
`CREATE TABLE core.artist_credit_ref (
|
||||
mbid BLOB NOT NULL PRIMARY KEY,
|
||||
credit_id INTEGER NOT NULL
|
||||
) WITHOUT ROWID`,
|
||||
}
|
||||
|
||||
for _, stmt := range stmts {
|
||||
@@ -243,6 +263,63 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
|
||||
fmt.Printf(" %-15s %d\n", sel.label+":", n)
|
||||
}
|
||||
|
||||
return copyCredits(db)
|
||||
}
|
||||
|
||||
// copyCredits ships the credit decomposition for the entities that made
|
||||
// it into the artifact, and only those.
|
||||
//
|
||||
// The refs go first and the parts follow *from* the refs, so a credit is
|
||||
// carried only if something in the artifact points at it. The source
|
||||
// index holds credits for every catalog entity, while the artifact is a
|
||||
// windowed subset -- copying all of them would carry a large table most
|
||||
// of which nothing in the artifact can reach.
|
||||
//
|
||||
// A source index built before the credit pass simply has no rows here,
|
||||
// which is not an error: the artifact then carries the tables empty, and
|
||||
// every credit falls back to its single artist exactly as before.
|
||||
func copyCredits(db *sql.DB) error {
|
||||
// Asked, not assumed. A source index built before the credit pass
|
||||
// has no such table, and "no such table" would fail an export whose
|
||||
// catalog is otherwise complete.
|
||||
for _, table := range []string{"artist_credit_ref", "artist_credit_part"} {
|
||||
var n int
|
||||
|
||||
if err := db.QueryRow(
|
||||
`SELECT COUNT(*) FROM main.sqlite_master
|
||||
WHERE type = 'table' AND name = ?`, table,
|
||||
).Scan(&n); err != nil {
|
||||
return fmt.Errorf("probe %s: %w", table, err)
|
||||
}
|
||||
|
||||
if n == 0 {
|
||||
fmt.Printf(" %-15s none in source\n", "credits:")
|
||||
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
refs, err := insertSelect(db, `
|
||||
INSERT INTO core.artist_credit_ref (mbid, credit_id)
|
||||
SELECT r.mbid, r.credit_id
|
||||
FROM main.artist_credit_ref r
|
||||
WHERE r.mbid IN (SELECT mbid FROM core.explore_index)`)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
parts, err := insertSelect(db, `
|
||||
INSERT INTO core.artist_credit_part
|
||||
(credit_id, position, artist_mbid, credited_name, join_phrase)
|
||||
SELECT p.credit_id, p.position, p.artist_mbid, p.credited_name, p.join_phrase
|
||||
FROM main.artist_credit_part p
|
||||
WHERE p.credit_id IN (SELECT credit_id FROM core.artist_credit_ref)`)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fmt.Printf(" %-15s %d refs, %d parts\n", "credits:", refs, parts)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user