feat: 5-tier search index with library + similar artist expansion

Browse Source

Rewrite SearchIndex with tiered background build:

Tier 1 — Sitewide instant (<5s, 12 calls): top artists, recordings,
  and release groups across 4 time ranges. Searchable immediately.

Tier 2 — Sitewide full discog (~16min, 2881 calls): top 20 RGs +
  top 100 recordings per sitewide artist (~1440 unique artists from
  all_time/this_year/this_month/this_week union).

Tier 3 — Library artists (~4min, 664 calls): match local library
  artist names against known MBIDs, index their full discographies.
  Catches the user's personal taste that sitewide misses.

Tier 4 — Similar artists (~24min, ~4300 calls): fetch similar
  artists from LB labs for each library artist, index their
  discographies. Fans out into the user's taste neighborhood.

Tier 5 — Organic growth (0 calls): BrowseReleaseGroups now writes
  to the search index in a background goroutine. Every artist page
  view adds that artist's discography to the index for free.

Other changes:
- indexRGsPerArtist bumped 10→20 (96% vs 88% coverage)
- indexRecsPerArtist bumped 10→100 (track-name searchability)
- indexMinPopularity = 50 (cuts noise from long tails)
- Dedicated 3 req/s rate limiter for indexer
- Labs similar-artists endpoint at labs.api.listenbrainz.org
- Each tier marks index as ready on completion so search improves
  progressively during the ~44min total build

...

This commit is contained in:

yonlu

2026-03-25 12:49:20 -04:00

parent cf40909313

commit 48acede1bc

2 changed files with 695 additions and 166 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

backend/explore/searchindex.go

+676 -165

File diff suppressed because it is too large Load Diff