Compare commits

..
Author SHA1 Message Date
yonlu 655434dcaa test(e2e): sweep the binding names a spec calls
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 7m25s
CI / e2e (pull_request) Successful in 12m52s
Four specs still called `library.Library.GetTracks` after #281 replaced
it, and the suite reported twenty failures across transport, bottom-bar
and reduced-motion specs — none of which mention the library list. A
binding call carries only a method id, so a stale name fails at runtime
in whichever spec happens to call it.

`binding-names.spec.ts` reads every spec and harness file, extracts the
names they pass as strings, and checks them against the map derived
from the generated bindings tree. It asserts first that it read
something: a sweep over an empty glob passes and proves nothing.

It immediately found a second thing: `play-count.spec.ts` asserted that
a play refetched no collection by matching a `GetAll` prefix, which
matches exactly one real name (`GetAllLibrariesWithTrackCounts`) — so
the assertion held whatever the app refetched. It names the four
collection bindings now.
2026-10-06 02:36:28 -04:00
yonlu 2ed9e725e3 docs(planning): what the library payloads cost, measured
Plan 023 (#279–#284): the before/after table on 50 000 tracks, and the
six things worth keeping — including two measurements that pointed away
from where the work had been (mmap is a bound, and the remaining second
is Wails encoding every result twice).
2026-10-06 01:59:57 -04:00
yonlu 5b236ca5af perf(library): patch what changed instead of reloading the list
A retag named the one file it rewrote and the store threw the whole
list away anyway: 20.5 MB across the IPC at 26 138 tracks to pick up one
new title. A soft rescan of an unchanged library did the same, though
its metrics say zero added, zero updated and zero removed.

- `TrackMetadataChanged` with a `filePath` re-reads that one row
  through `trackCache` and splices it in, replacing the array so the
  memoized filter and sort caches notice. The summaries are still
  refetched: a retag can move a track between albums. A failed patch
  falls back to the full reload, because the answer that cannot be
  wrong is the one to keep.
- A batch write names no paths (it can be thousands) and still reloads.
- `LibraryScanComplete` reads its own metrics and reloads only when the
  scan changed something; a payload of an unknown shape is treated as a
  change, so nothing can leave a stale list on screen.

Closes #282
2026-10-06 01:59:17 -04:00
yonlu bce8d27370 fix(explore): stop asking ListenBrainz after it refuses this client
Every popularity request answered 401 on 2026-10-05: 399 in one minute
of a real library's discography backfill, one rate-limited request per
artist to be told the same thing, each with its own log line.

A token is a property of the installation, not of the artist, so the
first 401 is the answer for the rest of that client's life. The refusal
is latched, logged once at warning level, and returned as
ErrListenBrainzUnauthorized — separate from the generic HTTP error
because it is the one failure no retry can fix. A 500 still does not
latch, so a transient failure leaves the artist unmarked and the next
run asks again.

The backfill also stops feeding artists once the client is refused: the
rest of the pass would be the same 401, and the artists stay unmarked
for a run that has a token.

Closes #284
2026-10-06 01:55:15 -04:00
yonlu c9b3048a20 perf(database): bound the read pool's mapping and cache by measurement
#283 read as "64 MB of mmap and up to 32 MB of page cache per pool",
and it is — but on a 50 000-track library neither is what the
process's memory is: the RSS of the database mapping is 46 MB whether
the bound is 64 MB or 16 MB, because a mapping costs what the working
set touches. Quartering it and halving the page cache moved no query
either: 1 172 → 1 197 ms for the whole track list, 106 → 124 ms for the
album list, 6 → 8 ms for an FTS search, all inside the run-to-run
spread.

Kept, because a bound that costs nothing measurable is worth having on
a phone: five read connections' worth of mapping is 320 MB of address
space at 64 MB each against 80 MB here, and #52 was a low-memory kill.

Closes #283
2026-10-06 01:51:56 -04:00
yonlu 620151aa41 perf(library): load a collection when a view needs it, not at startup
All four collections were fetched at DOMContentLoaded and refetched on
every invalidation, whichever view was showing. On a 26 138-track
library the track list was 20.5 MB of that, and encoding it cost the
backend ~170 MB of transient allocation — paid by someone looking at
Home, which draws none of it.

- The store warms only albums, artists and genres, on idle after first
  paint: 1.6 MB together, and what made those views instant.
- A nav item prefetches on hover and on keyboard focus, which is the
  ~100 ms before the click that a cold open would otherwise wait.
- An invalidation refetches what something had loaded, and nothing
  else — a scan no longer loads the track list of a library whose
  Tracks view nobody has opened.
- `index.html`'s first-paint `<track-list>` is `view-hidden`, and
  `index.ts` no longer activates it: it is markup, not a decision about
  which view the launch lands on, and activating it was what fetched
  the whole list for a landing on Home. The track list itself loads on
  view activation, which the shell drives.

Measured on 50 000 tracks: backend RSS at rest 543 → 296 MB, peak 571 →
296 MB, Go heap held 361 → 125 MB, JS heap 31.8 → 18 MB, binding bytes
at rest 35.9 → 12.1 MB, heap after a browse 36.5 → 22.7 MB. Tracks
first open 26 ms; slowest view open 57 ms.

Closes #280
2026-10-06 01:43:00 -04:00
yonlu 3d9828b847 perf(library): hand the batch dialog the rows the view already has
Fetching whole tracks back for a dialog that reads only fields the
rows in hand already carry cost 3 s and 700 ms of blocked main thread
on a "select all" over 50 000 tracks — the regression #281 introduced
by dropping four fields from the list.

`showBatchTrackDetails` takes the rows; the Tracks view passes its
own, and the views that hold no library rows (Explore, the queue,
playlists) keep the path form over `trackCache`. The batch view's
merged-fields extractors read only list fields already, and a cover
tier the list does not carry falls back to the one it does.

Measured on 50 000 tracks: 50.4 ms, from 87.9 ms before #281.

Refs #281
2026-10-05 23:55:54 -04:00
yonlu 7a1412383f perf(library): send the track list as a dictionary-encoded column table
GetTracks answered with one JSON object per track: 20.5 MB at 26k
tracks, ~350 bytes a row of repeated key names, four cover URLs that
are identical across an album, and the album, artist and genre strings
repeated on every track. Encoding it was ~170 MB of transient Go
allocation, and parsing it the WebView's memory peak.

GetTrackTable replaces it: one array per column, every repeated
string stored once and sent as an index, genre lists interned, and only
the fields the Tracks view reads. LastPlayed and the three larger cover
tiers are left to the details dialog, which now fetches whole tracks by
path. Each row still goes through trackFromRow, so this is an encoding
of the one projection, not a second one. 167 bytes a track against 929
in the test library; a Go test holds it under a quarter of the object
encoding, and a Vitest test reads the Go columns and the generated
Track interface and fails if they drift.

An empty library is now an empty table rather than the "no tracks in
library" error the old binding returned.

Closes #281
2026-10-05 23:30:37 -04:00
yonlu 4a9fe1d207 fix(library): open track details by path, not from the whole library
Track details from the queue, a playlist or a smart playlist looked
the track up in libraryStore's whole-library array, and returned
silently when it had not loaded, so the menu item did nothing. That
dependency is also why every track was fetched eagerly at startup
(20.5 MB at 26k tracks).

GetTracksByPaths answers for the paths in hand, and trackCache holds
the answers: bounded, coalesced into one call per task, and forgetting
what the retag, removal and scan events name. Every opener, and
track-details' own refresh after a save, go through it. A source sweep
pins the whole-library array to the store and the Tracks view.

Closes #279
2026-10-05 23:01:37 -04:00
yonlu 3cddf70c4d perf(smart-playlist): suggest rule values for what was typed
The rule editor's value box built its suggestions from libraryStore's
whole-library arrays, which made it one more reason to load every
track at startup, and left it empty when they had not loaded.

SuggestSmartPlaylistValues answers with up to 50 distinct values of a
field that contain the typed text, from the same joined row the rules
match against, so a suggestion is always something a rule can match.
yj-combobox announces its filter text so the editor can ask, debounced
and cached per field and text.

Refs #279
2026-10-05 22:59:24 -04:00
yonlu 3f23bb4396 Merge pull request 'Batch: explore catalog + UI, Go 1.26, Soulseek downloads, CI fixes' (#273) from batch/258-272 into main
CI / check (push) Successful in 4m8s
CI / e2e (push) Successful in 14m37s
Reviewed-on: #273
2026-09-27 01:45:18 +00:00
yonlu af2ff17342 Merge remote-tracking branch 'origin/fix/263-slskd-transfer-lifecycle' into batch/258-272
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 4m53s
CI / e2e (pull_request) Successful in 13m34s
2026-09-26 20:58:54 -04:00
yonlu fc19ca54b7 Merge remote-tracking branch 'origin/build/265-go-1.26' into batch/258-272 2026-09-26 20:58:54 -04:00
yonlu 613901847a Merge remote-tracking branch 'origin/feat/264-explore-cards' into batch/258-272 2026-09-26 20:58:54 -04:00
yonlu ecf0109331 Merge remote-tracking branch 'origin/fix/258-artifact-blob-cursor' into batch/258-272 2026-09-26 20:58:53 -04:00
yonluandClaude Opus 5.5 9710c11476 feat(download): one grab per Soulseek peer, several peers at once
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Failing after 2m51s
CI / e2e (pull_request) Skipped
slskd was capped at one transfer per daemon, on the grounds that
Soulseek peers punish clients that ask for too much. That politeness is
per peer: two different users do not compete for anyone's upload slot.
So one slow peer serialised every other Soulseek download behind it.

The manager now takes a per-(provider, peer) lock before any slot, so a
grab waiting on a busy peer does not hold a provider slot another peer
could use, and the slskd default rises to 3, which now counts peers.
The help text says so.

Running grabs at once exposed the folder collision: slskd names a
download's directory after the remote leaf folder, so two peers'
"Greatest Hits" (or any two rips' "CD1") share one directory, and
collect finds files by name there. Grabs whose local folders overlap
now take a package-level lock per folder, in sorted order, keyed on the
full path because two clients can share one daemon.

Closes #272

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 17:31:35 -04:00
yonluandClaude Opus 5.5 5e3ac8fb1b fix(download): search Soulseek more than once, and read file lengths
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Canceled after 0s
CI / e2e (pull_request) Canceled after 0s
The slskd search asked one question and ignored part of the answer.

Two queries.  Soulseek matches every term against a file's full path,
so every extra word is a filter, and several filter wrongly: an edition
qualifier from the catalog title that no one puts in a folder name, a
term with a leading "-", which Soulseek reads as an exclusion, and
"Various Artists", which is in no one's path.  When a normalised form of
the request differs, it runs alongside the original and the candidates
are merged by peer and folder.  Concurrently, not as a fallback: the
manager gives a provider one search budget, and a Soulseek search spends
most of it waiting.  A query the user typed is searched as written.

Stated options.  The search carried only its id and text, so slskd's
own defaults for its timeout and response limits applied.  Its timeout
is now set inside our wait, the limits are well above a popular album,
and slskd drops folders below the file floor and peers with a queue we
would not reach today.

A state-only poll.  Every one-second poll re-sent every response; the
responses are now fetched once at the end, falling back to the old
includeResponses form for a daemon without that endpoint.

Durations.  slskd reports each file's length and it was discarded.  It
is now carried as CandidateFile.LengthMillis and scored against the
expected tracks as DurationFit, which takes 0.15 of title fit's weight
when at least half the aligned pairs are timed: a title says which song
a file claims to be, a length says whether it is that recording.
Without lengths the score is exactly the previous formula.

freeUploadSlots is removed from the response type; slskd sends
hasFreeUploadSlot and nothing by that name.

Closes #271

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 17:11:23 -04:00
yonluandClaude Opus 5.5 f81a950916 fix(download): score a multi-disc rip as one album, and count tracks
CI / check (push) Skipped
CI / e2e (push) Skipped
Three faults in how candidates are shaped and scored, one commit because
they meet in the same completeness number.

Multi-disc albums were split in two.  Soulseek shares them as
Album/CD1 and Album/CD2, and candidates were grouped by the immediate
parent, so each disc became its own candidate titled "CD1": about half
complete, with an album title that could not match.  Such a release
essentially never cleared auto-pick.  AlbumDir groups a disc folder
under its parent, ParsePath takes the disc number from the folder (a
disc in the filename still wins), and collect keeps the disc folders in
staging, where flattened, disc 2's "01 Intro.flac" overwrote disc 1's.

A single-track request could never be served from Soulseek.  A track
search matches one file per folder, and the two-file floor that screens
out noise for an album screened out every result.  A recording request
takes one.

Completeness counted files.  Ten files against a ten-track album scored
full marks whether or not they were its tracks, and title fit is the
mean over the files that did align, so a folder where three titles
matched read as near-perfect on both.  Coverage is now counted in
aligned tracks, with the file count still setting the penalty for
extras.

Closes #270

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 17:07:28 -04:00
yonluandClaude Opus 5.5 7fbfd9c105 test(library): skip the cover-tier scan test when fixtures are absent
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 4m23s
CI / e2e (pull_request) Canceled after 0s
TestScan_StoresOnlyCoverTiers built the fixture path by hand, so in a
tree where make testdata had not run it failed on a missing covers
directory, where every other fixture test skips via testfixtures.Load.
In a fresh worktree that failure blocked the pre-push hook for every
branch.

Closes #266

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 17:02:56 -04:00
yonluandClaude Opus 5.5 792c2d9fbc build: require Go 1.26 everywhere at once
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 8m21s
CI / e2e (pull_request) Successful in 15m56s
The newest go-json-experiment/json, which wails/v3's application
package imports, declares go 1.26, so taking it raises our go
directive with it.  Every other place that names a Go version moves in
the same commit: GO_VERSION in ci, desktop-assets and android-apk, the
Arch makedepends, and CONTRIBUTING's table.

index-artifact's container image moves too, and it is the one that
matters most.  The official golang images set GOTOOLCHAIN=local, so a
golang:1.25 container refuses a go 1.26 module outright rather than
fetching a newer toolchain, and that job owns the ~205 GB checkpoint.

Closes #265

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 17:00:47 -04:00
yonluandClaude Opus 5.5 bb26d5f289 feat(explore): arrows on every sideways row, and one album card size
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 4m25s
CI / e2e (pull_request) Successful in 13m43s
The shelves, the top results and the artist page's discography and
similar-artists rows scrolled sideways only by a horizontal wheel or a
trackpad, so a plain mouse could not reach anything past the fold.
<scroll-row> wraps each of them with previous/next arrows that are
hidden at the end they cannot move from, revealed on hover where there
is hover, and always shown where there is not.

The album card is defined once, in albumCardStyles, instead of twice.
explore-view clamped its cards to 130-150px, and with square artwork
that made cards in one row different heights as well as widths.  The
width is fixed now and each line under the art reserves its own space.
Covers are inset rather than cropped (contain, not cover).

The ownership badge sits over the artwork for owned and unowned alike,
and catalog cards no longer dim: a grid of dimmed covers read as a page
that had failed to load.  The album page's tracklist still dims unowned
rows, which says something different about something different.

Every MusicBrainz link now goes through openMusicBrainz, which pins the
origin, including the one on the album page that still called
window.open directly.  solid/chevron-left is bundled for the left arrow;
without it the arrow drew the missing-icon fallback.

Closes #264

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 16:59:28 -04:00
yonluandClaude Opus 5.5 7b42b9ce56 feat(explore): sample the one-album-owned shelf instead of ranking it
"More from artists you own one album by" drew its artists and their
albums most-popular first, so it was a second leaderboard: the same
handful of big names every time the page opened, which is not what the
shelf is saying.  The pool is still bounded, but which artists and
which albums fill it is RANDOM(), so each visit is a different sample.
The test asserts the set rather than the order.

Refs #264

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 16:58:57 -04:00
yonluandClaude Opus 5.5 0a33b9d653 fix(download): try the next acceptable copy when a transfer fails
A failed transfer failed the whole download.  On Soulseek the usual
failure is one peer being offline or refusing, and a popular album has
several other peers offering the same folder; the ranked list that
names them was already held in m.results and nothing walked it.

grab now loops: when a candidate's transfer fails, or delivers too
little of the album to import, the next candidate is tried in its
place, up to three in all.  Three rules keep that honest:

- Only a candidate auto-pick would itself have accepted is offered, so
  a second choice clears the same match, quality and guardrail gates
  as the first.
- On Soulseek the failure is the peer's, so every folder that peer
  offered is skipped with it; elsewhere only the failed release is.
- A candidate the user picked by hand does not fall back.  They chose
  that copy, and quietly substituting another is a decision they did
  not make.

The same change fixes auto-pick grabbing the wrong candidate.
AutoPickVeto judges the best candidate inside the user's guardrails,
but Start and Attempt then grabbed ranked[0] -- so when the overall best
was over the size ceiling, the veto passed on the strength of the
second and the first was downloaded anyway: the one copy the user had
said not to take unattended.  autoPick returns the candidate the veto
actually judged.

Closes #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 16:12:06 -04:00
yonluandClaude Opus 5.5 fc0121228e fix(download): give up on a stalled slskd peer and cancel its transfers
awaitTransfers waited on every requested file reaching a terminal state
with no bound but the caller's six-hour context.  slskd's transfer
limit is one, so a peer that queued us and never sent a byte held every
other Soulseek download behind it for the whole six hours.  A grab now
gives up after ten minutes with no bytes moving; a folder that stalls
on its last tracks goes forward with what arrived, as a partial failure
always has.

Three smaller faults on the same path:

- A file slskd never lists (refused at enqueue) could never reach a
  terminal state, so the wait could not end.  It counts as failed after
  a short grace period.
- A terminal record left by an earlier attempt at the same file from the
  same peer was read as this attempt's answer on the first poll.  The
  ids already terminal before enqueue are ignored.
- Giving up, for any reason, left slskd downloading for a request nobody
  was waiting on.  The live transfers are cancelled and removed there,
  on a context of their own so a cancelled caller still sends it.

Usernames are now path-escaped; they may carry spaces and slashes.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HJiuc3ZZhxsPXz3ozTirT
2026-09-26 16:11:52 -04:00
yonlu 924097b246 Merge branch 'docs/256-claude-md-split' into fix/258-artifact-blob-cursor
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m57s
CI / e2e (pull_request) Successful in 14m49s
Both branches rewrite CLAUDE.md. Mine appended a note about #258 to the
"the catalog stores ids as bytes" section; theirs cuts the file from
4,046 lines to 376 and deletes those write-ups, on the grounds that each
one already exists as a comment beside the code it describes.

Resolved in theirs. Nothing is lost by it: the refactor already states
both halves of the note as broad engineering rules, and cites #258 —
"Encodings are converted at one boundary … a Go `string` cursor against
a BLOB column made the catalog merge loop forever while merging nothing"
and "A loop that must make progress checks that it did … a merge that
lands fewer rows than it read". The detail stays in `artifactKey`'s doc
comment and in `publishedartifact_test.go`'s, which is where the new
file says this class belongs.
2026-09-26 15:56:44 -04:00
yonluandClaude Opus 5.5 1997276def docs: cut CLAUDE.md to the rules it is for
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 4m22s
CI / e2e (pull_request) Successful in 14m24s
CLAUDE.md had grown to 4,046 lines, ~3,300 of them per-component
write-ups: why a breakpoint is 500px, why a cap is a quarter, what a
spec once missed. Every session loaded all of it, and the rules that
apply to every change were buried among decisions that apply to one.

Those write-ups were already duplicated as comments beside the code
they describe -- every issue number they cite also appears in a code
comment -- so they are deleted rather than moved. What is left is the
tracker workflow, the commands, the verification tiers and a set of
broad engineering rules distilled from them, each pointing at where
its example lives. Three declined decisions recorded nowhere else go
to NOTES.md, and three code comments that cited CLAUDE.md sections by
name no longer do.

Closes #256

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 15:47:13 -04:00
yonlu 5d9c677cf7 ci(index-artifact): import the exported artifact before publishing it
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m59s
CI / e2e (pull_request) Successful in 12m51s
Nothing should be published until the code that imports it on a user's
machine has imported it here. The exporter and the importer are two
descriptions of one storage format, and every other tier tests the
importer against a *fixture* rather than against the file being shipped —
a second description free to be wrong in the same direction as the code
reading it.

That is how #258 reached everyone: the importer positioned its batch walk
with a Go `string` cursor against this file's 16-byte `mbid` column, and
SQLite neither coerces between TEXT and BLOB nor complains about the
comparison, so the walk merged no rows and never advanced. The fixture
guarding that walk writes the old text encoding, and the only compact
fixture is one row, below the batch size, so the bound query never ran.
Both were green throughout, and no install could finish a first index
build.

`TestImportPublishedArtifact` takes the published file and runs the
client's own path over it — checksum, decompress, merge — and asserts
that what the artifact holds is what the client ends up with: the row
count, the rows carrying a listen count, an FTS index in step with the
table, and one row read back through the app's own MBID conversions. It
skips without `YJ_CORE_INDEX_ARTIFACT`, so an ordinary run pays nothing.

It needs no Wails, which is why the step runs it under the indexbuild
tag: that container has no GTK. Measured on the current artifact, 64.8 MB
compressed: 39 seconds including the decompress.

Verified against the pre-fix comparison behaviour, the step goes red in
about 3 seconds — the strictly-advancing guard fails the import with a
named reason rather than the day-long spin it used to produce.

Refs #258
2026-09-25 11:03:52 -04:00
yonlu 1e3a490c12 fix(explore): refuse a catalog merge that does not land every row
The walk's predicates partition the artifact's key space, so a merge that
ends with fewer rows than the artifact declares does not mean the artifact
was smaller than it said — it means a predicate filtered rows out, and
the catalog is quietly partial while reporting complete.

Equality rather than a lower bound: RowsAffected counts an upsert that
changes nothing, and a row already merged locally is counted again here.

One reachable case, so this is not merely a tripwire. A row whose mbid is
empty is excluded by `mbid > ?` in both encodings, so an artifact
carrying one imports as a success with a row missing — which is the shape
#258 had, one cause over. The test covers exactly that artifact.

Refs #258
2026-09-25 11:03:33 -04:00
yonlu d4ea14ca5c fix(explore): merge the catalog artifact in its own mbid encoding
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 4m4s
CI / e2e (pull_request) Successful in 14m14s
The prebuilt catalog never merged. `mergeArtifactRows` positions itself
with `WHERE mbid > ? ORDER BY mbid LIMIT 1 OFFSET ?` against the
attached artifact, and it bound that cursor as a Go `string` while
`cmd/indexexport` publishes `explore_index.mbid` as 16 raw bytes — the
storage change that took the table from 677 MB to 389 MB.

SQLite does not coerce between TEXT and BLOB and orders every blob after
every text value, so against a byte column the predicate was not wrong
but unconditional: `mbid > <text>` matched the whole artifact, so the
bound the walk looked up was the same row every time and the cursor
never advanced, and `mbid <= <text>` matched nothing, so no batch
merged. No error, no rows, no state change — a fresh install sat at "0
of 1,077,893 rows" burning a core indefinitely, which is what it did
here for a day, while Explore showed only the rows the library scan and
the lazy artist enrichment had produced and popularity for none of the
catalog.

The cursor is now an `artifactKey`, typed to the encoding
`artifactStoresText` reports for the file it is attached to, so the
comparison is made in the same type as the column it is made against.
Two things guard the class rather than the instance: a nil key binds as
an empty value instead of SQL NULL, because `mbid > NULL` agrees with
nothing and would import nothing just as silently; and the walk returns
an error when its bound does not strictly advance, because the failure
here is silence and the next one should be a failed job with a reason.

It was never caught because the fixture that guards the walk writes the
old text encoding, and the only compact fixture is a single row — below
`artifactMergeBatch`, so the bound query never ran at all. The walk is
now covered on both encodings, across several batch boundaries.

Closes #258
2026-09-25 10:18:51 -04:00
yonlu 62c1a95ead test(e2e): give the job specs their own state back
Both specs stage a job through `/__test/emit` and neither cleared it.
Nothing resets those stores, so the spec that staged it is the one that
should put it back, and `JobsChanged` with `[]` is the whole cleanup --
`JobStore` replaces its list from every snapshot, so `testctl` needs no
special case.

**The leak as reported did not reproduce, and that is worth recording
rather than quietly fixing.**  Measured with a temporary probe: a
positive control confirmed a staged job really does move the shell at a
phone width (`job-band` renders a row, `.main-panel`'s top goes 0 to
55), and the very next page had no job at all.  The reason is that
every test gets a fresh page and `JobStore.init()` refetches `GetJobs()`
from a backend registry `/__test/emit` never writes to -- it calls
`events.Deliver`, which touches frontends and no state.  So the state
cannot cross a spec boundary as described, and the 55px offset the
draft assertion saw in that suite run has another cause that is not in
evidence.

The cleanup stays, because it costs a line and the leak would need only
one spec that keeps a page alive, and the comments say what was
measured rather than asserting the mechanism.

The durable half is the rule, now in the harness reference: measure
against the element next to you, not an absolute coordinate.  An
absolute number in a shell measurement is also a claim about everything
above it -- `contentTop === 0` asserts "and no background job is
running", which that spec could not arrange.

Closes #168
2026-09-23 07:51:07 -04:00
yonlu e67462ab53 ci: lint every commit a PR would merge, not just its tip
Gitea leaves `github.event.before` empty on a `pull_request`, so the
Commit messages step fell through to bare `make commit-check`, which
lints `git log -1` -- the tip alone.  Every other commit the branch
would bring was first examined by *main's* post-merge run, so a green
PR stopped being true after the merge, and it happened twice: PR #245
merged a 75-char subject its own CI never saw.

The PR's base is the stand-in.  `base.sha..head` lints the PR's own
commits because base advances on main, so the commits the branch shares
with it stay reachable from it and drop out of the range.

Both payload fields are handed to the shell rather than chosen in an
expression: `github.event.issue.number` in unclaim.yml is this repo's
proof that payload fields resolve, and `github.event` is the webhook
body unmarshalled into a map, so `pull_request.base.sha` comes from
Gitea's own `PRBranchInfo.Sha`.  The shell then falls back to today's
behaviour for a dispatch run, an all-zeros push, or a base commit the
clone does not have -- so the worst case is the fix not taking effect
rather than a broken job.

Verified locally against the report's own evidence: at 68e7edb8 the old
invocation passes ("HEAD is well-formed") while the range catches
a3b5b437 at 75 chars, which is what main's post-merge run did.  All
four event shapes were exercised against the new snippet.  The
end-to-end proof is the next PR with an over-length commit that is not
its tip.

Closes #254
2026-09-23 07:50:59 -04:00
yonlu e5dc54d0ec ci(skill-check): find a make target inside a hard-wrapped span
`scripts/skill-check.sh` matched one regex against one line, so a
mention the file hard-wraps -- `` `make `` at the end of one line and
the target at the start of the next -- was invisible to it.  These docs
are mostly hard-wrapped prose, so the wrap is what the author does not
think about, and `CONTRIBUTING.md:80` is already that shape.

Lines are now joined while the inline span is still open, which an odd
number of backticks means.  The fence and line-start halves are
untouched: a fenced command is already whole, and joining inside one
would break the rule that made this awk rather than a grep.  Joining is
bounded three ways -- a fence, a blank line (CommonMark allows no blank
line inside a code span) and a file boundary -- so a stray backtick
costs one paragraph of over-matching rather than the rest of the file.

The reporting loop needed the other half of the same fix: it named the
offending file with `grep -ln "make $t"`, which cannot see a wrapped
mention either, so a target the new parser found reported no file at
all and `set -o pipefail` turned the empty grep into exit 123 before
the line telling the author what to do.  It falls back to the bare
name.

Verified by planting the report's own wrapped `make
no-such-wrapped-target` into `CONTRIBUTING.md`: the old script reports
48 targets and exits 0, the new one names the target and the file and
exits 1.  Plant removed afterwards.

Closes #228
2026-09-23 07:50:51 -04:00
yonlu a5c3990d12 Merge pull request 'Small-fix batch: artist_metadata sweep, and the three unbounded surfaces (#248, #249)' (#255) from batch/248-249 into main
CI / check (push) Skipped
CI / e2e (push) Skipped
Build & publish the Android APK / apk (push) Successful in 2m22s
Build & publish Arch package / arch-package (push) Successful in 3m23s
Attach the desktop build to the release / linux (push) Successful in 3m23s
Sync Homebrew formula / sync-formula (push) Successful in 9s
Closes #248
Closes #249
2026-09-21 02:01:54 +00:00
yonlu 8dbdb7ad75 Merge branch 'fix/249-unbounded-growth' into batch/248-249
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m42s
CI / e2e (pull_request) Successful in 11m12s
Both branches add to the same two registries, so the conflicts are
between the two fixes rather than with main:

- backend/app.go: both register a janitor job.  Both are registered.
- backend/maintenance/sweeps.go: both append a job at the end of the
  file.  Both are kept, each with its own closing tail.
- backend/maintenance/maintenance_test.go: both append a test.  Both are
  kept as separate functions.
- backend/library/library.go: #249's orphan-path lyrics delete was
  written against the loop variable before #250 renamed it, so its
  `audioFile.ID` no longer exists in that function.  Adapted to `f.ID`.

Closes #248
Closes #249
2026-09-20 21:40:49 -04:00
yonlu a205224a26 Merge branch 'fix/248-artist-metadata-sweep' into batch/248-249 2026-09-20 21:34:50 -04:00
yonlu a96cc9be1f Merge remote-tracking branch 'origin/main' into fix/248-artist-metadata-sweep
CI / e2e (push) Skipped
CI / check (push) Skipped
CI / check (pull_request) Successful in 3m22s
CI / e2e (pull_request) Successful in 11m43s
2026-09-20 21:15:14 -04:00
yonlu 8db19622b2 Merge pull request 'fix(library): sweep orphaned cover art when an album empties' (#251) from fix/247-cover-art-orphans into main
CI / check (push) Successful in 3m20s
CI / e2e (push) Successful in 12m5s
default
2026-09-21 01:14:58 +00:00
yonlu 53f480980f Merge remote-tracking branch 'origin/main' into fix/247-cover-art-orphans
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m35s
CI / e2e (pull_request) Successful in 11m41s
2026-09-20 20:59:14 -04:00
yonlu 1335572f0a Merge pull request 'fix(library): preserve playlist phantoms on incremental scan and removal' (#250) from fix/246-incremental-scan-phantoms into main
CI / check (push) Successful in 3m28s
CI / e2e (push) Successful in 12m53s
default
2026-09-21 00:41:05 +00:00
yonlu cf90030463 chore(bindings): regenerate for the queue's DropSourceForPlaylist
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 6m50s
CI / e2e (pull_request) Successful in 11m9s
frontend/bindings is generated by wails3 and is not covered by the
codegen pre-commit hook, so the new bound method went out without it and
make bindings-check failed on the PR.
2026-09-20 20:22:16 -04:00
yonlu 88f5524aa2 fix(maintenance): bound lyrics search and clicks, clear queue source
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Failing after 3m44s
CI / e2e (pull_request) Skipped
Three unbounded or stale surfaces, each small on its own:

- lyrics_index rows were never pruned on track removal, so the FTS index
  grew forever. Delete the entry where the library search FTS entry is
  already deleted, on the orphan and RemoveFromLibrary paths.
- search_clicks had no ceiling; age out ranking rows after a retention
  window via a daily janitor job.
- queue.source_* kept a "Playing from X" label after its playlist was
  deleted. Drop the source when the queue's own playlist goes, wired
  through a playlist-service hook like Library.SetRemovalHooks.

Closes #249
2026-09-11 17:14:41 -04:00
yonlu e745acf88a fix(maintenance): sweep artist_metadata rows nothing references
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m6s
CI / e2e (pull_request) Successful in 11m22s
artist_metadata was classified Cache/Swept but had no sweep and no
DELETE anywhere, so long-lived entity data (no TTL by design) grew for
the life of the install.  Sweep rows whose MBID is neither a library
artist nor holding cached artwork, and register the job with the
janitor.

Closes #248
2026-09-09 10:16:32 -04:00
yonlu 32bb64918c fix(library): sweep orphaned cover art when an album empties
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m19s
CI / e2e (pull_request) Successful in 10m53s
pruneEmptyEntities deleted empty albums but never the cover_art rows
they referenced, so removing the last track of an album leaked the row
and its files forever — the janitor's covers sweep computes its live set
from cover_art.file_path, which keeps the orphaned row's files exempt.

Extract sweepOrphanedCoverArt/removeCoverArtFiles as one implementation
and run it from pruneEmptyEntities (scan orphan path and
RemoveFromLibrary) and RemoveLibrary alike.

Closes #247
2026-09-09 10:14:07 -04:00
143 changed files with 8870 additions and 5719 deletions

No files matched your search

+1 -1
View File
@@ -68,7 +68,7 @@ jobs:
SHA: ${{ github.sha }} SHA: ${{ github.sha }}
REF_NAME: ${{ github.ref_name }} REF_NAME: ${{ github.ref_name }}
DEBIAN_FRONTEND: noninteractive DEBIAN_FRONTEND: noninteractive
GO_VERSION: '1.25.0' GO_VERSION: '1.26.0'
npm_config_store_dir: /cache/pnpm-store npm_config_store_dir: /cache/pnpm-store
# The Go half wants the NDK; the Gradle half wants a platform. # The Go half wants the NDK; the Gradle half wants a platform.
ANDROID_HOME: /cache/android-sdk ANDROID_HOME: /cache/android-sdk
+23 -6
View File
@@ -36,7 +36,7 @@ concurrency:
cancel-in-progress: true cancel-in-progress: true
env: env:
GO_VERSION: '1.25.0' GO_VERSION: '1.26.0'
# Shared by all three Playwright consumers (@playwright/cli, e2e/'s # Shared by all three Playwright consumers (@playwright/cli, e2e/'s
# @playwright/test, frontend/'s Vitest provider). See the browsers # @playwright/test, frontend/'s Vitest provider). See the browsers
# step in job 2 for why that is not the whole story. # step in job 2 for why that is not the whole story.
@@ -53,7 +53,7 @@ jobs:
check: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: container:
# Not golang:1.25 — this job runs `make ui-test`, which is Vitest # Not golang:1.26 — this job runs `make ui-test`, which is Vitest
# *browser* mode and needs a Chromium and its system libraries # *browser* mode and needs a Chromium and its system libraries
# anyway, so the "fast job needs no browser" split does not hold. # anyway, so the "fast job needs no browser" split does not hold.
# Not the Playwright image either: e2e/ pins @playwright/test # Not the Playwright image either: e2e/ pins @playwright/test
@@ -107,15 +107,32 @@ jobs:
# Conventional Commits. `.releaserc.yml` has always derived the # Conventional Commits. `.releaserc.yml` has always derived the
# version from the commit type; until now nothing checked that the # version from the commit type; until now nothing checked that the
# type was one it recognises, so a malformed subject silently meant # type was one it recognises, so a malformed subject silently meant
# "no release". BEFORE is the push's previous tip and is absent or # "no release".
# all-zeros for a new branch, in which case only the tip is linted. #
# **On a `pull_request` there is no `before`.** Gitea leaves
# `github.event.before` empty for one, so this step fell through to
# bare `make commit-check`, which lints `git log -1` — the tip
# alone. Every other commit the branch would bring was first
# examined by *main's* post-merge run, which is a green PR that
# stops being true after the merge, and which happened twice (#254).
# The PR's base is the stand-in: the range below already excludes
# what the base shares with the branch, because base advances on
# main and those commits stay reachable from it.
#
# Both are handed to the shell rather than chosen in an expression:
# `github.event.issue.number` in unclaim.yml is this repo's proof
# that payload fields resolve, and the shell then falls back to
# today's behaviour for a dispatch run or a missing field instead of
# depending on how `&&`/`||` treat an absent context.
- name: Commit messages - name: Commit messages
working-directory: /src working-directory: /src
env: env:
BEFORE: ${{ github.event.before }} PR_BASE: ${{ github.event.pull_request.base.sha }}
PUSH_BEFORE: ${{ github.event.before }}
run: | run: |
set -eu set -eu
if [ -n "${BEFORE:-}" ] && [ "${BEFORE#0000000}" = "$BEFORE" ] \ BEFORE="${PR_BASE:-${PUSH_BEFORE:-}}"
if [ -n "$BEFORE" ] && [ "${BEFORE#0000000}" = "$BEFORE" ] \
&& git cat-file -e "$BEFORE^{commit}" 2>/dev/null; then && git cat-file -e "$BEFORE^{commit}" 2>/dev/null; then
make commit-check RANGE="$BEFORE..$SHA" make commit-check RANGE="$BEFORE..$SHA"
else else
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
SHA: ${{ github.sha }} SHA: ${{ github.sha }}
REF_NAME: ${{ github.ref_name }} REF_NAME: ${{ github.ref_name }}
DEBIAN_FRONTEND: noninteractive DEBIAN_FRONTEND: noninteractive
GO_VERSION: '1.25.0' GO_VERSION: '1.26.0'
npm_config_store_dir: /cache/pnpm-store npm_config_store_dir: /cache/pnpm-store
steps: steps:
# The same set ci.yml's check job installs: the app is cgo, and # The same set ci.yml's check job installs: the app is cgo, and
+44 -1
View File
@@ -68,7 +68,7 @@ jobs:
# claim with a test behind it now (cmd/indexbuild/deps_test.go), # claim with a test behind it now (cmd/indexbuild/deps_test.go),
# because the v3 migration quietly broke it and this job was where # because the v3 migration quietly broke it and this job was where
# that surfaced. # that surfaced.
image: golang:1.25 image: golang:1.26
# This host path must exist on the runner and be listed verbatim in # This host path must exist on the runner and be listed verbatim in
# act_runner's container.valid_volumes. It holds explore-staging/ # act_runner's container.valid_volumes. It holds explore-staging/
# (counts.bin + state.json) and yj.db — the checkpoint that makes # (counts.bin + state.json) and yj.db — the checkpoint that makes
@@ -148,6 +148,49 @@ jobs:
sha256sum /tmp/core-index.db.zst | tee /tmp/core-index.db.zst.sha256 sha256sum /tmp/core-index.db.zst | tee /tmp/core-index.db.zst.sha256
ls -lh /tmp/core-index.db.zst ls -lh /tmp/core-index.db.zst
# Nothing is published until it has been imported by the code that
# imports it on a user's machine. The exporter and the importer are
# two descriptions of one storage format, and every other tier tests
# the importer against a *fixture* rather than against the file being
# shipped — a second description free to be wrong in the same
# direction as the code reading it.
#
# That is how #258 reached everyone: the importer positioned its batch
# walk with a Go `string` cursor against this file's 16-byte `mbid`
# column, and SQLite neither coerces between TEXT and BLOB nor
# complains about the comparison — so the walk merged no rows and
# never advanced, and no install could finish its first index build.
# The fixture guarding that walk writes the old text encoding, and the
# only compact fixture is one row, below the batch size, so the bound
# query never ran. Both were green throughout.
#
# Running it here is also what keeps the failure cheap: the previous
# artifact stays published while this runs, so a failure costs one
# stale catalog rather than an empty one for every install.
#
# `-tags indexbuild` because this container has no GTK and the default
# tag set links the app through Wails. The `--- PASS` grep is not
# decoration — the test skips without the path, and a skip is
# indistinguishable from a pass in a summary line.
- name: Import the exported artifact as a client does
if: steps.maintain.outputs.complete == 'true' && steps.maintain.outputs.changed == 'true'
working-directory: /src
env:
YJ_CORE_INDEX_ARTIFACT: /tmp/core-index.db.zst
run: |
set -eu
log=/tmp/import-check.log
if ! go test -tags indexbuild -count=1 -timeout 30m -v \
-run TestImportPublishedArtifact ./backend/explore/ > "$log" 2>&1;
then
tail -60 "$log"
echo "::error::The artifact does not import; not publishing it."
exit 1
fi
cat "$log"
grep -qF -- 'PASS: TestImportPublishedArtifact' "$log"
echo "::notice::The artifact imports as a client would merge it."
- name: Publish to the Gitea package registry - name: Publish to the Gitea package registry
if: steps.maintain.outputs.complete == 'true' && steps.maintain.outputs.changed == 'true' if: steps.maintain.outputs.complete == 'true' && steps.maintain.outputs.changed == 'true'
run: | run: |
@@ -647,7 +647,7 @@ window.__yj = { call(name, args) {
That turns the device into a tier that can be *driven* rather than only That turns the device into a tier that can be *driven* rather than only
looked at — `__yj.call("player.Player.LoadFile", [path])` and looked at — `__yj.call("player.Player.LoadFile", [path])` and
`__yj.call("library.Library.AddLibrary", ["/sdcard/Music/..."])` are how `__yj.call("library.Library.AddLibrary", ["/sdcard/Music/..."])` are how
#53 was measured. Names are the Go ones (`GetTracks`, not #53 was measured. Names are the Go ones (`GetTrackTable`, not
`GetAllTracks`); an unknown one comes back as a plain `GetAllTracks`); an unknown one comes back as a plain
`unknown bound method name`, so a wrong guess is loud. `unknown bound method name`, so a wrong guess is loud.
@@ -57,6 +57,21 @@ behind `YJ_TESTCTL=1`, which `scripts/dev-headless.sh` sets and
staging the work that would produce it — job progress, download staging the work that would produce it — job progress, download
progress, scan progress. It calls `events.Deliver`, which *errors* progress, scan progress. It calls `events.Deliver`, which *errors*
when the event reaches nobody, so a `200` means it really arrived. when the event reaches nobody, so a `200` means it really arrived.
- **State you stage, you own** (#168). Nothing resets those stores, so
clear yours in `test.afterEach` with the same event that staged it
(`emit('JobsChanged', [])`) — the store replaces its list from every
snapshot, so `testctl` needs no special case. **Measured: this does
not currently cross a spec boundary**, because every test gets a fresh
page and `JobStore.init()` refetches `GetJobs()` from a backend
registry that `/__test/emit` never writes to. Stated anyway, because
it costs one line and the leak needs only one spec that keeps a page
alive — but do not cite #168 for a symptom you have not reproduced.
- **Measure against the thing next to you, not an absolute
coordinate.** An absolute number in a shell measurement is also a
claim about everything above it — `contentTop === 0` quietly asserts
"and no background job is running", which is not what that spec was
about or could arrange, while `contentTop === jobBandBottom` is true
either way. This is the half of #168 that stands on its own.
- **`restore` is slow** (~40 s in the suite) because it copies every - **`restore` is slow** (~40 s in the suite) because it copies every
table. Prefer snapshotting once and restoring only when a spec table. Prefer snapshotting once and restoring only when a spec
genuinely mutates state. genuinely mutates state.
+103
View File
@@ -5078,3 +5078,106 @@ last rendered card.
("ask the virtualizer for a larger overscan") is therefore not ("ask the virtualizer for a larger overscan") is therefore not
available without patching a private, which is why the request is available without patching a private, which is why the request is
issued ahead of the element instead. issued ahead of the element instead.
## Declined, and recorded nowhere else (2026-09-25, #256)
When CLAUDE.md was cut down to rules, most of its "considered and
declined" paragraphs already had a home in a code or config comment
beside what they explain. These three did not:
- **`touch-action: manipulation` was declined** (#54). The 300ms tap
delay it is offered for is already absent on a `width=device-width`
viewport; what it would actually change is the gesture stack #63 tuned
by measurement on the reference device.
- **There is no "Go to Genre"** in the phone row menu (#67). That menu
replaces name links a phone cannot use, and there has never been a
genre link to replace — it would be new navigation, which wants its
own issue.
- **The overlaid queue has no tap-outside gutter on a phone** (#171).
The drawer-style gutter would buy the affordance by taking width off a
full-screen surface on a 424px viewport; back and a 44px close button
answer it instead.
## What the library payloads actually cost (measured 2026-10-05, desktop dev build)
Plan 023 (#279–#284). Measured against `make sandbox-seed-bulk`
(50 000 tracks, 371 MB `yj.db` in the smaller copy) with the desktop
dev build and `e2e/perf/measure.mjs`, which grew a `memory` section for
it: backend RSS and peak from `/proc`, the Go heap from pprof, the
page's JS heap and DOM counters from CDP, and the bytes every binding
returned, before and after the first open of Tracks.
| | before | after #281 | after #280 |
|---|---|---|---|
| Backend RSS at rest | 543 MB | 296 MB | 189 MB |
| Backend peak RSS | 571 MB | 337 MB | 281 MB |
| Go heap held | 361 MB | 125 MB | 7 MB |
| JS heap at rest | 31.8 MB | 18.0 MB | 5.2 MB |
| Binding bytes at rest | 35.9 MB | 12.1 MB | 1.7 MB |
| JS heap after a browse | 36.5 MB | 22.7 MB | 23.1 MB |
| Tracks first open → first row | 38 ms | 26 ms | 1 255 ms |
| Slowest first view open | 44 ms | 57 ms | 76 ms |
Six things worth keeping:
- **The number that fell was not the number being optimised.** The
binding payload fell 20.5 → 10.45 MB (dictionary-encoded columns) and
35.9 → 12.1 MB, but what made the backend's RSS fall by 354 MB was
the *fetch not happening*: `GetTracks` was the only thing in the app
that allocated 170 MB transiently (`sqlcgen.GetTracks` 80 MB, `json/v2`
128 MB, `slices.Grow` 77 MB, `bytes.Clone` 47 MB of 484 MB total
`alloc_space`). Encoding smaller would not have touched that.
- **A `dictionary` is what makes dropping fields the wrong trade.** The
first version of the column table left out `LastPlayed` and the three
larger cover tiers, and then a "select all → edit tags" over 50 000
tracks had to fetch every track back to open the dialog: 3 071 ms
against 88 ms before. Interning means the repeated strings are nearly
free, so the fields belong in the table; what fixed it in the end was
neither — the Tracks view hands the dialog the rows it already has.
**A payload optimisation that removes a field is a new fetch waiting
to be written.**
- **`mmap_size` is a bound, not an allocation.** #283 read as "64 MB of
mapping per connection", and the RSS of the database mapping is 46 MB
whether the bound is 64 MB or 16 MB, because a mapping costs what the
working set touches. Quartering it moved no query either (1 172 →
1 197 ms for the whole track list, 106 → 124 ms for the album list, 6
→ 8 ms for an FTS search — all inside the run-to-run spread). Kept for
the phone, where the bound is the address space the low-memory killer
reads.
- **A view that is first paint is a view that is active.** `index.html`
renders a `<track-list>`, so the track list was connected at launch
and fetched 12 MB whoever was looking at — and the fix was not in the
component but in the shell: markup is not a decision about which view
the launch lands on, so the seed is `view-hidden` and the first
navigation is what activates it. Hover and keyboard focus on a nav
item prefetch, which is the ~100 ms before the click.
- **The remaining second is the transport, not the data.** A cold open
of Tracks on 50 000 tracks is 1 255 ms, and a *raw* binding call for
the same table is 1 118–1 382 ms: none of it is the TypeScript decode
or the render. It is the Go-side query, the column encode and Wails
encoding every result twice — once for a debug log that is off
(#286). Splitting that number is what says where to go next, and the
answer was not where the payload work had been.
- **A test named for the behaviour caught the thing the design missed.**
The source sweep that pins "the whole-library track array has one
reader" was written to catch the four call sites #279 had already
converted; it found `smart-playlist-editor`, which built its value
suggestions from the same array and would have gone silently empty
once nothing loaded it. The suggestion box is a backend query now
(`SuggestSmartPlaylistValues`).
And two things about this machine, since they cost a cycle each:
- **`make ui-test` is not reliable at load average 14.** Under the
workstation's own background services, the browser provider's module
fetches fail in a different handful of files each run ("Failed to
import test file", "Cannot connect to the iframe") while every test
that runs passes. `--maxWorkers=1 --retry=2` reduces it; individual
files always pass. A failure list that changes between runs is the
environment, not the branch.
- **A measurement run inherits the machine's mood.** The first
after-#281 numbers said view opens had doubled (albums 27 → 78 ms,
settings 44 → 179 ms, Tracks first row 38 → 105 ms). Re-running
unchanged gave 26 ms and 57 ms. The tell was the same one
`NOTES.md` already records twice: before and after suspiciously
equal, or suspiciously worse, across *unrelated* measurements.
+331 -4001
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -13,7 +13,7 @@ frontend, bridged by [Wails v3](https://wails.io/).
| Tool | Version | | Tool | Version |
|------|---------| |------|---------|
| Go | 1.25+ | | Go | 1.26+ |
| Node.js | 22+ | | Node.js | 22+ |
| pnpm | 10+ | | pnpm | 10+ |
| Wails CLI | v3 — vendored, no install needed (`go tool wails3`) | | Wails CLI | v3 — vendored, no install needed (`go tool wails3`) |
+6
View File
@@ -499,6 +499,10 @@ func (yj *YellowJacketApp) OnStartup(ctx context.Context) {
PostRemove: yj.explore.InvalidateLibrarySync, PostRemove: yj.explore.InvalidateLibrarySync,
}) })
// A deleted playlist must not leave the queue's "Playing from"
// label pointing at it.
yj.playlist.SetOnPlaylistDeleted(yj.queue.DropSourceForPlaylist)
// Register playback finished handler to drive queue auto-advance. // Register playback finished handler to drive queue auto-advance.
yj.player.SetPlaybackFinishedHandler(yj.queue.OnPlaybackFinished) yj.player.SetPlaybackFinishedHandler(yj.queue.OnPlaybackFinished)
@@ -775,6 +779,8 @@ func (yj *YellowJacketApp) startJanitor() {
} }
yj.janitor.Register(maintenance.ExpiredHTTPCacheJob(yj.database)) yj.janitor.Register(maintenance.ExpiredHTTPCacheJob(yj.database))
yj.janitor.Register(maintenance.StaleArtistMetadataJob(yj.database))
yj.janitor.Register(maintenance.StaleSearchClicksJob(yj.database))
yj.janitor.Register(maintenance.OrphanedCoverFilesJob( yj.janitor.Register(maintenance.OrphanedCoverFilesJob(
yj.database, coversDir, library.CoverArtFileSet, yj.database, coversDir, library.CoverArtFileSet,
)) ))
+14 -4
View File
@@ -54,9 +54,19 @@ type DB struct {
// form is silently ignored, which is why WAL was never actually on. // form is silently ignored, which is why WAL was never actually on.
const ( const (
writeDSNParams = "?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)" writeDSNParams = "?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)"
readDSNParams = "?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)" + // The read pool's own page cache and mapping bound, sized by
// measurement rather than by appetite (#283). On a 50 000-track
// library, halving the cache and quartering the mapping moved no
// query — 1 172 → 1 197 ms for the whole track list, 106 → 124 ms
// for the album list, 6 → 8 ms for an FTS search, all inside the
// run-to-run spread — and the RSS of the database mapping was 46 MB
// under both settings, because a mapping's cost is what the working
// set touches, not the bound. The bound is what matters on a
// phone, where five connections' worth of address space is the
// thing the low-memory killer reads (cf. #52).
readDSNParams = "?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)" +
"&_pragma=query_only(true)&_pragma=synchronous(NORMAL)" + "&_pragma=query_only(true)&_pragma=synchronous(NORMAL)" +
"&_pragma=cache_size(-8000)&_pragma=mmap_size(67108864)" "&_pragma=cache_size(-2000)&_pragma=mmap_size(16777216)"
// readPoolConns bounds concurrent read connections. A handful is // readPoolConns bounds concurrent read connections. A handful is
// plenty for interactive search + art/lookup fan-out and keeps WAL // plenty for interactive search + art/lookup fan-out and keeps WAL
// reader overhead small. // reader overhead small.
@@ -354,8 +364,8 @@ func applyPRAGMAs(ctx context.Context, db *sql.DB) error {
pragmas := []string{ pragmas := []string{
"PRAGMA foreign_keys = ON", "PRAGMA foreign_keys = ON",
"PRAGMA synchronous = NORMAL", "PRAGMA synchronous = NORMAL",
"PRAGMA cache_size = -8000", "PRAGMA cache_size = -2000",
"PRAGMA mmap_size = 67108864", "PRAGMA mmap_size = 16777216",
} }
for _, pragma := range pragmas { for _, pragma := range pragmas {
+16 -4
View File
@@ -151,16 +151,28 @@ func (d *DB) SetLyrics(audioFileID int64, lyrics, source, recordingMBID string)
return d.upsertLyricsIndex(audioFileID, lyrics) return d.upsertLyricsIndex(audioFileID, lyrics)
} }
// upsertLyricsIndex refreshes a single file's entry in the contentless // DeleteLyricsIndex removes one file's entry from the contentless
// lyrics_index. contentless_delete=1 makes the DELETE valid; an empty // lyrics_index. It is called wherever a file row is deleted — the
// lyrics string leaves the row deleted. // `lyrics` table cascades with its file, but the FTS entry does not and
func (d *DB) upsertLyricsIndex(audioFileID int64, lyrics string) error { // would otherwise accumulate for the life of the install (#249).
func (d *DB) DeleteLyricsIndex(audioFileID int64) error {
if _, err := d.db.ExecContext(d.Ctx, if _, err := d.db.ExecContext(d.Ctx,
"DELETE FROM lyrics_index WHERE rowid = ?", audioFileID, "DELETE FROM lyrics_index WHERE rowid = ?", audioFileID,
); err != nil { ); err != nil {
return fmt.Errorf("could not delete lyrics_index row: %w", err) return fmt.Errorf("could not delete lyrics_index row: %w", err)
} }
return nil
}
// upsertLyricsIndex refreshes a single file's entry in the contentless
// lyrics_index. contentless_delete=1 makes the DELETE valid; an empty
// lyrics string leaves the row deleted.
func (d *DB) upsertLyricsIndex(audioFileID int64, lyrics string) error {
if err := d.DeleteLyricsIndex(audioFileID); err != nil {
return err
}
if strings.TrimSpace(lyrics) == "" { if strings.TrimSpace(lyrics) == "" {
return nil return nil
} }
@@ -117,6 +117,9 @@ WHERE library_id = COALESCE(NULLIF(CAST(sqlc.arg(library_id) AS INTEGER), 0), li
-- name: GetTrackByPath :one -- name: GetTrackByPath :one
SELECT * FROM track_metadata WHERE file_path = ? LIMIT 1; SELECT * FROM track_metadata WHERE file_path = ? LIMIT 1;
-- name: GetTracksByPaths :many
SELECT * FROM track_metadata WHERE file_path IN (sqlc.slice('paths'));
-- name: GetTracksByAlbum :many -- name: GetTracksByAlbum :many
SELECT * FROM track_metadata SELECT * FROM track_metadata
WHERE album_id = sqlc.arg(album_id) WHERE album_id = sqlc.arg(album_id)
@@ -830,6 +830,71 @@ func (q *Queries) GetTracksByGenre(ctx context.Context, arg GetTracksByGenrePara
return items, nil return items, nil
} }
const getTracksByPaths = `-- name: GetTracksByPaths :many
SELECT id, file_path, length_milliseconds, title, artist_name, track_number, disc_number, album, genre, year, release_year, composer, file_type, sample_rate, bit_depth, channels, bitrate, file_size, library_id, play_count, last_played, cover_art_path, artist_mbid, release_group_mbid, recording_mbid, album_id, artist_id FROM track_metadata WHERE file_path IN (/*SLICE:paths*/?)
`
func (q *Queries) GetTracksByPaths(ctx context.Context, paths []string) ([]TrackMetadatum, error) {
query := getTracksByPaths
var queryParams []interface{}
if len(paths) > 0 {
for _, v := range paths {
queryParams = append(queryParams, v)
}
query = strings.Replace(query, "/*SLICE:paths*/?", strings.Repeat(",?", len(paths))[1:], 1)
} else {
query = strings.Replace(query, "/*SLICE:paths*/?", "NULL", 1)
}
rows, err := q.db.QueryContext(ctx, query, queryParams...)
if err != nil {
return nil, err
}
defer rows.Close()
var items []TrackMetadatum
for rows.Next() {
var i TrackMetadatum
if err := rows.Scan(
&i.ID,
&i.FilePath,
&i.LengthMilliseconds,
&i.Title,
&i.ArtistName,
&i.TrackNumber,
&i.DiscNumber,
&i.Album,
&i.Genre,
&i.Year,
&i.ReleaseYear,
&i.Composer,
&i.FileType,
&i.SampleRate,
&i.BitDepth,
&i.Channels,
&i.Bitrate,
&i.FileSize,
&i.LibraryID,
&i.PlayCount,
&i.LastPlayed,
&i.CoverArtPath,
&i.ArtistMbid,
&i.ReleaseGroupMbid,
&i.RecordingMbid,
&i.AlbumID,
&i.ArtistID,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Close(); err != nil {
return nil, err
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const lookupTrackMetaByPaths = `-- name: LookupTrackMetaByPaths :many const lookupTrackMetaByPaths = `-- name: LookupTrackMetaByPaths :many
SELECT id, file_path, title, artist_name, album, cover_art_path, SELECT id, file_path, title, artist_name, album, cover_art_path,
artist_mbid, release_group_mbid, recording_mbid artist_mbid, release_group_mbid, recording_mbid
+248
View File
@@ -0,0 +1,248 @@
package download
import (
"context"
"os"
"path/filepath"
"testing"
)
// Multi-disc rips, single-track results and coverage counted in tracks
// rather than files (#270).
func TestParsePathReadsTheDiscFromItsFolder(t *testing.T) {
t.Parallel()
cases := []struct {
path string
disc int
track int
folder string
}{
{`\share\Pink Floyd - The Wall (1979)\CD2\03 Hey You.flac`, 2, 3, "The Wall"},
{`\share\The Wall\Disc 1\01 In The Flesh.flac`, 1, 1, "The Wall"},
{`\share\The Wall\[Disk-2]\01 Hey You.flac`, 2, 1, "The Wall"},
{`\share\The Wall\CD1 - Live\04 Mother.flac`, 1, 4, "The Wall"},
// The filename's own disc number is more specific than the folder.
{`\share\The Wall\CD1\2-05 Comfortably Numb.flac`, 2, 5, "The Wall"},
// Not a disc folder: a number is required.
{`\share\CDs\The Wall\01 In The Flesh.flac`, 0, 1, "The Wall"},
}
for _, tc := range cases {
t.Run(tc.path, func(t *testing.T) {
t.Parallel()
got := ParsePath(tc.path)
if got.Disc != tc.disc || got.Track != tc.track || got.Folder != tc.folder {
t.Errorf(
"ParsePath = disc %d track %d folder %q, want %d %d %q",
got.Disc, got.Track, got.Folder, tc.disc, tc.track, tc.folder,
)
}
})
}
}
func TestAlbumDir(t *testing.T) {
t.Parallel()
cases := map[string]string{
`\share\Album\CD1\01 A.flac`: "/share/Album",
`\share\Album\01 A.flac`: "/share/Album",
`CD1\01 A.flac`: "CD1",
`\share\CD Collection\01.mp3`: "/share/CD Collection",
}
for in, want := range cases {
if got := AlbumDir(in); got != want {
t.Errorf("AlbumDir(%q) = %q, want %q", in, got, want)
}
}
}
// One album shared as CD1/CD2 is one candidate, named after the album.
func TestSlskdGroupsDiscFoldersIntoOneCandidate(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.responses = []slskdResponse{{
Username: "peer",
Files: []slskdFile{
{Filename: `\m\The Wall\CD1\01 In The Flesh.flac`, Size: 1},
{Filename: `\m\The Wall\CD1\02 The Thin Ice.flac`, Size: 1},
{Filename: `\m\The Wall\CD2\01 Hey You.flac`, Size: 1},
{Filename: `\m\The Wall\CD2\02 Is There Anybody Out There.flac`, Size: 1},
},
}}
s, _ := newStubSlskd(t, stub)
got, err := s.Search(context.Background(), Download{Query: "the wall"})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(got) != 1 {
t.Fatalf("got %d candidates, want the two discs as one", len(got))
}
if got[0].Title != "The Wall" || len(got[0].Files) != 4 {
t.Errorf(
"candidate = %q with %d files, want \"The Wall\" with 4",
got[0].Title, len(got[0].Files),
)
}
}
// A track search matches one file per folder, so a single-track request
// must accept a one-file folder that an album request rightly drops.
func TestSlskdKeepsASingleFileForATrackRequest(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.responses = []slskdResponse{{
Username: "peer",
Files: []slskdFile{
{Filename: `\m\OK Computer\02 Paranoid Android.flac`, Size: 1},
},
}}
s, _ := newStubSlskd(t, stub)
track, err := s.Search(context.Background(), Download{
RecordingMBID: "rec-1", Artist: "Radiohead", Album: "Paranoid Android",
})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(track) != 1 {
t.Errorf("track request: got %d candidates, want 1", len(track))
}
album, err := s.Search(context.Background(), Download{
ReleaseMBID: "rel-1", Artist: "Radiohead", Album: "OK Computer",
})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(album) != 0 {
t.Errorf("album request: got %d candidates, want the one-file folder dropped", len(album))
}
}
// Two discs with a file of the same name both reach staging, each under
// its disc folder, where the importer reads the disc number from.
func TestSlskdCollectKeepsDiscFolders(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
s, downloads := newStubSlskd(t, stub)
for _, disc := range []string{"CD1", "CD2"} {
dir := filepath.Join(downloads, disc)
if err := os.MkdirAll(dir, 0o750); err != nil {
t.Fatalf("mkdir: %v", err)
}
if err := os.WriteFile(
filepath.Join(dir, "01 Intro.flac"), []byte(disc), 0o600,
); err != nil {
t.Fatalf("write: %v", err)
}
}
dst := t.TempDir()
got, err := s.collect(Candidate{Files: []CandidateFile{
{Path: `\m\Album\CD1\01 Intro.flac`, IsAudio: true},
{Path: `\m\Album\CD2\01 Intro.flac`, IsAudio: true},
}}, dst)
if err != nil {
t.Fatalf("collect: %v", err)
}
if len(got.Files) != 2 {
t.Fatalf("collected %d files, want 2", len(got.Files))
}
for _, disc := range []string{"CD1", "CD2"} {
data, err := os.ReadFile(filepath.Join(dst, disc, "01 Intro.flac"))
if err != nil || string(data) != disc {
t.Errorf("%s's file missing or overwritten: %q, %v", disc, data, err)
}
if hint := ParsePath(filepath.Join(dst, disc, "01 Intro.flac")); hint.Disc == 0 {
t.Errorf("staged %s file lost its disc number", disc)
}
}
}
// A two-disc release whose discs both number from 01 aligns completely
// once the disc comes from the folder; before, disc 2's 01 collided with
// disc 1's.
func TestMultiDiscCandidateAlignsEveryTrack(t *testing.T) {
t.Parallel()
dl := Download{
ReleaseMBID: "the-wall",
Artist: "Pink Floyd",
Album: "The Wall",
Expected: []ExpectedTrack{
{DiscNumber: 1, Position: 1, Title: "In the Flesh?"},
{DiscNumber: 1, Position: 2, Title: "The Thin Ice"},
{DiscNumber: 2, Position: 1, Title: "Hey You"},
{DiscNumber: 2, Position: 2, Title: "Is There Anybody Out There?"},
},
}
c := Candidate{
Title: "The Wall",
Files: []CandidateFile{
{Path: `\m\Pink Floyd - The Wall\CD1\01 In the Flesh.flac`, Size: 1},
{Path: `\m\Pink Floyd - The Wall\CD1\02 The Thin Ice.flac`, Size: 1},
{Path: `\m\Pink Floyd - The Wall\CD2\01 Hey You.flac`, Size: 1},
{Path: `\m\Pink Floyd - The Wall\CD2\02 Is There Anybody Out There.flac`, Size: 1},
},
}
got := Score(dl, c, 50, AutoDownloadPrefs{})
if got.Match.Completeness != 1 {
t.Errorf("completeness = %f, want 1", got.Match.Completeness)
}
if got.Match.AlbumFit < 0.99 {
t.Errorf("album fit = %f, want the album's own name to match", got.Match.AlbumFit)
}
if got.Match.Overall < minMatch {
t.Errorf("match = %f, want it to clear the auto-pick bar %f", got.Match.Overall, minMatch)
}
}
// Ten files against a ten-track album is not a complete album when only
// three of them are its tracks.
func TestCompletenessCountsTracksNotFiles(t *testing.T) {
t.Parallel()
dl := okComputer()
c := Candidate{Title: "OK Computer", Files: []CandidateFile{
{Path: `\m\Radiohead - OK Computer\Airbag.flac`, Size: 1},
{Path: `\m\Radiohead - OK Computer\Paranoid Android.flac`, Size: 1},
{Path: `\m\Radiohead - OK Computer\Exit Music (For a Film).flac`, Size: 1},
{Path: `\m\Radiohead - OK Computer\Creep.flac`, Size: 1},
}}
got := Score(dl, c, 50, AutoDownloadPrefs{})
if got.Match.Completeness > 0.76 {
t.Errorf(
"completeness = %f with 3 of 4 tracks present, want at most 0.75",
got.Match.Completeness,
)
}
}
+13 -12
View File
@@ -47,7 +47,7 @@ func grabAll(
go func() { go func() {
defer wg.Done() defer wg.Done()
f.manager.grab(ctx, dl, candidate, nil) f.manager.grab(ctx, dl, candidate, nil, false)
}() }()
} }
@@ -79,9 +79,9 @@ func TestConcurrencyForPrefersOverrideThenKind(t *testing.T) {
want int want int
}{ }{
{ {
name: "slskd defaults to one", name: "slskd defaults to a few peers",
cfg: Config{Kind: KindSlskd}, cfg: Config{Kind: KindSlskd},
want: 1, want: 3,
}, },
{ {
name: "usenet defaults higher", name: "usenet defaults higher",
@@ -92,9 +92,9 @@ func TestConcurrencyForPrefersOverrideThenKind(t *testing.T) {
name: "explicit override wins", name: "explicit override wins",
cfg: Config{ cfg: Config{
Kind: KindSlskd, Kind: KindSlskd,
Settings: map[string]string{concurrencyKey: "3"}, Settings: map[string]string{concurrencyKey: "1"},
}, },
want: 3, want: 1,
}, },
{ {
name: "nonsense override falls back", name: "nonsense override falls back",
@@ -102,7 +102,7 @@ func TestConcurrencyForPrefersOverrideThenKind(t *testing.T) {
Kind: KindSlskd, Kind: KindSlskd,
Settings: map[string]string{concurrencyKey: "not a number"}, Settings: map[string]string{concurrencyKey: "not a number"},
}, },
want: 1, want: 3,
}, },
{ {
name: "zero override falls back", name: "zero override falls back",
@@ -110,7 +110,7 @@ func TestConcurrencyForPrefersOverrideThenKind(t *testing.T) {
Kind: KindSlskd, Kind: KindSlskd,
Settings: map[string]string{concurrencyKey: "0"}, Settings: map[string]string{concurrencyKey: "0"},
}, },
want: 1, want: 3,
}, },
{ {
name: "unknown kind falls back to the global default", name: "unknown kind falls back to the global default",
@@ -126,9 +126,9 @@ func TestConcurrencyForPrefersOverrideThenKind(t *testing.T) {
} }
} }
// The reason the per-provider cap exists: a Soulseek daemon capped at // The reason the per-provider cap exists: a daemon capped at one
// one transfer must serialize, even when the global cap would allow // transfer must serialize, even when the global cap would allow more and
// more and the user has queued several albums at once. // the user has queued several albums at once.
func TestPerProviderCapSerializesTransfers(t *testing.T) { func TestPerProviderCapSerializesTransfers(t *testing.T) {
t.Parallel() t.Parallel()
@@ -142,6 +142,7 @@ func TestPerProviderCapSerializesTransfers(t *testing.T) {
ID: 1, ID: 1,
Kind: KindSlskd, Kind: KindSlskd,
Priority: 50, Priority: 50,
Settings: map[string]string{concurrencyKey: "1"},
}, slow) }, slow)
// Three requests against the same one-at-a-time provider. // Three requests against the same one-at-a-time provider.
@@ -210,8 +211,8 @@ func TestSyncSemaphoresReplacesChangedLimits(t *testing.T) {
f.manager.installProvider(Config{ID: 1, Kind: KindSlskd}, nil) f.manager.installProvider(Config{ID: 1, Kind: KindSlskd}, nil)
first := f.manager.semaphoreFor(1) first := f.manager.semaphoreFor(1)
if cap(first) != 1 { if want := kindConcurrency[KindSlskd]; cap(first) != want {
t.Fatalf("slskd semaphore cap = %d, want 1", cap(first)) t.Fatalf("slskd semaphore cap = %d, want %d", cap(first), want)
} }
// Same limit: the semaphore is kept, so in-flight accounting is not // Same limit: the semaphore is kept, so in-flight accounting is not
+261
View File
@@ -0,0 +1,261 @@
package download
import (
"context"
"errors"
"os"
"testing"
)
// A transfer that fails on one copy of an album is not a failed
// download while another acceptable copy exists. On Soulseek the usual
// failure is one peer being offline, with several others offering the
// same folder.
var errPeerOffline = errors.New("peer went offline")
func TestManagerFallsBackToTheNextCandidate(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
// The failing source ranks first on priority, so the fallback is
// what reaches the one that works.
bad := fakeWithAlbum(1, "offline-peer", ".flac")
bad.GrabErr = errPeerOffline
good := fakeWithAlbum(2, "online-peer", ".flac")
f.manager.installProvider(Config{ID: 1, Priority: 90}, bad)
f.manager.installProvider(Config{ID: 2, Priority: 10}, good)
dl := fourTrackDownload()
if _, err := f.manager.Start(context.Background(), dl); err != nil {
t.Fatalf("Start: %v", err)
}
waitForDownloadState(t, f.store, dl.ID, StateComplete)
if bad.GrabCalls != 1 || good.GrabCalls != 1 {
t.Errorf(
"grabs: failing=%d working=%d, want 1 and 1",
bad.GrabCalls, good.GrabCalls,
)
}
// The abandoned attempt's staging goes with it; only a request that
// fails outright keeps its staging for inspection.
waitFor(t, func() bool {
entries, err := os.ReadDir(f.staging.Root())
return err == nil && len(entries) == 0
}, "the failed attempt's staging was never released")
}
// Falling back must not lower the bar. A second choice outside the
// user's guardrails is not a choice auto-pick may make, first or second.
func TestManagerFallbackRespectsTheGuardrails(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
f.manager.SetPreferences(AutoDownloadPrefs{MaxSizeMB: 50})
bad := fakeWithAlbum(1, "offline-peer", ".flac")
bad.GrabErr = errPeerOffline
bad.Candidates[0].TotalSize = 40 << 20
huge := fakeWithAlbum(2, "oversized", ".flac")
huge.Candidates[0].TotalSize = 900 << 20
f.manager.installProvider(Config{ID: 1, Priority: 90}, bad)
f.manager.installProvider(Config{ID: 2, Priority: 10}, huge)
dl := fourTrackDownload()
if _, err := f.manager.Start(context.Background(), dl); err != nil {
t.Fatalf("Start: %v", err)
}
waitForDownloadState(t, f.store, dl.ID, StateFailed)
if huge.GrabCalls != 0 {
t.Errorf("fell back to a candidate over the size ceiling")
}
}
// A copy the user picked by hand is the copy they asked for. Quietly
// substituting another is a decision they did not make.
func TestManagerPickDoesNotFallBack(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
bad := fakeWithAlbum(1, "offline-peer", ".flac")
bad.GrabErr = errPeerOffline
good := fakeWithAlbum(2, "online-peer", ".flac")
f.manager.installProvider(Config{ID: 1, Priority: 90}, bad)
f.manager.installProvider(Config{ID: 2, Priority: 10}, good)
// A ceiling below both copies parks the result set for the user.
f.manager.SetPreferences(AutoDownloadPrefs{MaxSizeMB: 1})
bad.Candidates[0].TotalSize = 30 << 20
good.Candidates[0].TotalSize = 30 << 20
dl := fourTrackDownload()
if _, err := f.manager.Start(context.Background(), dl); err != nil {
t.Fatalf("Start: %v", err)
}
if err := f.manager.Pick(
context.Background(), dl.ID, "offline-peer-cand",
); err != nil {
t.Fatalf("Pick: %v", err)
}
waitForDownloadState(t, f.store, dl.ID, StateFailed)
if good.GrabCalls != 0 {
t.Errorf("a hand-picked grab fell back to another candidate")
}
}
// Fallback is for surviving an offline peer or two, not for walking a
// forty-peer list for six hours.
func TestManagerFallbackIsBounded(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
var providers []*FakeProvider
for i := int64(1); i <= maxGrabAttempts+2; i++ {
p := fakeWithAlbum(i, "peer-"+itoa(int(i)), ".flac")
p.GrabErr = errPeerOffline
f.manager.installProvider(Config{ID: i, Priority: 50}, p)
providers = append(providers, p)
}
dl := fourTrackDownload()
if _, err := f.manager.Start(context.Background(), dl); err != nil {
t.Fatalf("Start: %v", err)
}
waitForDownloadState(t, f.store, dl.ID, StateFailed)
grabs := 0
for _, p := range providers {
grabs += p.GrabCalls
}
if grabs != maxGrabAttempts {
t.Errorf("grabs = %d, want %d", grabs, maxGrabAttempts)
}
}
// The veto judges the best candidate *inside* the guardrails, so the
// grab has to take that one — not the overall best, which may be the
// very copy the user said not to take unattended.
func TestManagerAutoPickTakesTheBestEligibleCandidate(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
f.manager.SetPreferences(AutoDownloadPrefs{MaxSizeMB: 50})
huge := fakeWithAlbum(1, "oversized", ".flac")
huge.Candidates[0].TotalSize = 900 << 20
fits := fakeWithAlbum(2, "fits", ".flac")
fits.Candidates[0].TotalSize = 40 << 20
f.manager.installProvider(Config{ID: 1, Priority: 90}, huge)
f.manager.installProvider(Config{ID: 2, Priority: 10}, fits)
dl := fourTrackDownload()
ranked, err := f.manager.Start(context.Background(), dl)
if err != nil {
t.Fatalf("Start: %v", err)
}
if ranked[0].ID != "oversized-cand" {
t.Fatalf("fixture: best overall is %s, want the oversized copy", ranked[0].ID)
}
waitForDownloadState(t, f.store, dl.ID, StateComplete)
if huge.GrabCalls != 0 || fits.GrabCalls != 1 {
t.Errorf(
"grabs: oversized=%d fits=%d, want 0 and 1",
huge.GrabCalls, fits.GrabCalls,
)
}
}
// On Soulseek a failure is the peer's, so every folder that peer offered
// goes with it. Elsewhere a failure is the release's, and one indexer's
// other releases are still worth trying.
func TestRuledOutBy(t *testing.T) {
t.Parallel()
failed := []Candidate{
{ID: "slskd:alice:Album", Kind: KindSlskd, ProviderID: 1, Origin: "alice"},
{ID: "tracker-1", Kind: KindProwlarr, ProviderID: 2, Origin: "indexer"},
}
cases := []struct {
name string
c Candidate
want bool
}{
{
name: "the same candidate",
c: Candidate{ID: "tracker-1", Kind: KindProwlarr, ProviderID: 2, Origin: "indexer"},
want: true,
},
{
name: "another folder from a failed peer",
c: Candidate{
ID: "slskd:alice:Album (2)",
Kind: KindSlskd,
ProviderID: 1,
Origin: "alice",
},
want: true,
},
{
name: "another peer",
c: Candidate{ID: "slskd:bob:Album", Kind: KindSlskd, ProviderID: 1, Origin: "bob"},
want: false,
},
{
name: "another release from the same indexer",
c: Candidate{ID: "tracker-2", Kind: KindProwlarr, ProviderID: 2, Origin: "indexer"},
want: false,
},
{
name: "a peer of the same name on a different daemon",
c: Candidate{
ID: "slskd:alice:Album",
Kind: KindSlskd,
ProviderID: 3,
Origin: "alice",
},
want: false,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
if got := ruledOutBy(tc.c, failed); got != tc.want {
t.Errorf("ruledOutBy = %v, want %v", got, tc.want)
}
})
}
}
+77
View File
@@ -0,0 +1,77 @@
package download
import (
"context"
"sync"
)
// keyedLock is a set of mutexes created on demand, one per key, that
// honour a context while waiting. An entry lives only while someone
// holds or waits on it, so a key per Soulseek peer or per folder name
// does not accumulate for the life of the process.
type keyedLock[K comparable] struct {
mu sync.Mutex
held map[K]*keyedEntry
}
type keyedEntry struct {
ch chan struct{}
// refs counts holders and waiters; the entry is dropped at zero.
refs int
}
// acquire blocks until k is free or ctx ends, and returns the function
// that frees it.
func (l *keyedLock[K]) acquire(ctx context.Context, k K) (func(), error) {
l.mu.Lock()
if l.held == nil {
l.held = map[K]*keyedEntry{}
}
e, ok := l.held[k]
if !ok {
e = &keyedEntry{ch: make(chan struct{}, 1)}
l.held[k] = e
}
e.refs++
l.mu.Unlock()
select {
case e.ch <- struct{}{}:
case <-ctx.Done():
l.drop(k, e)
return nil, ctx.Err()
}
var once sync.Once
return func() {
once.Do(func() {
<-e.ch
l.drop(k, e)
})
}, nil
}
func (l *keyedLock[K]) drop(k K, e *keyedEntry) {
l.mu.Lock()
defer l.mu.Unlock()
e.refs--
if e.refs == 0 {
delete(l.held, k)
}
}
// size reports how many keys are held or awaited, for tests.
func (l *keyedLock[K]) size() int {
l.mu.Lock()
defer l.mu.Unlock()
return len(l.held)
}
+245 -65
View File
@@ -59,13 +59,15 @@ const concurrencyKey = "maxConcurrent"
// A single global cap is the wrong shape here: usenet and torrent // A single global cap is the wrong shape here: usenet and torrent
// clients are built to run many transfers at once and are throttled by // clients are built to run many transfers at once and are throttled by
// bandwidth, while Soulseek transfers come from one person's home // bandwidth, while Soulseek transfers come from one person's home
// upload slot. Hitting the same peer with parallel requests gets you // upload slot. Politeness there is per *peer* — asking one user for two
// queued behind everyone else at best and banned at worst, so slskd is // folders at once gets you queued behind everyone else at best and
// capped at one — the polite number, and the one that actually // banned at worst — and the manager holds that line separately, one
// completes fastest, because a Soulseek peer serves one file at a time // grab per peer (peerLocks). Two different users do not compete for
// regardless of how many you ask for. // anyone's slot, so the daemon-wide number only bounds how many peers
// are asked at once, and one slow peer no longer serialises every other
// Soulseek download behind it.
var kindConcurrency = map[Kind]int{ var kindConcurrency = map[Kind]int{
KindSlskd: 1, KindSlskd: 3,
KindYtDlp: 2, KindYtDlp: 2,
KindQBittorrent: 4, KindQBittorrent: 4,
KindSABnzbd: 4, KindSABnzbd: 4,
@@ -155,6 +157,11 @@ type Manager struct {
semMu sync.Mutex semMu sync.Mutex
provSem map[int64]chan struct{} provSem map[int64]chan struct{}
// peerLocks holds one grab per Soulseek peer, taken before any
// slot: a grab waiting for a busy peer must not sit on a provider
// slot another peer could be using.
peerLocks keyedLock[peerKey]
// delegatePoll is how often delegating managers are asked for // delegatePoll is how often delegating managers are asked for
// status. A field rather than the constant so tests can drive the // status. A field rather than the constant so tests can drive the
// full delegate flow without sleeping through it. // full delegate flow without sleeping through it.
@@ -577,12 +584,12 @@ func (m *Manager) Start(
)) ))
} }
if m.AutoPickable(dl, ranked) { if pick, ok := autoPick(dl, ranked, m.preferences()); ok {
if job != nil { if job != nil {
job.Logf(jobs.LevelInfo, "Auto-selected best candidate") job.Logf(jobs.LevelInfo, "Auto-selected best candidate")
} }
go m.grab(context.WithoutCancel(ctx), dl, ranked[0], job) go m.grab(context.WithoutCancel(ctx), dl, pick, job, true)
return ranked, nil return ranked, nil
} }
@@ -622,6 +629,13 @@ func (m *Manager) Attempt(
return false, veto, nil return false, veto, nil
} }
pick, ok := autoPick(dl, ranked, m.preferences())
if !ok {
// Unreachable while autoPick and AutoPickVeto agree; kept so a
// future divergence refuses rather than grabbing blind.
return false, "no candidate clears the auto-download bar", nil
}
if err := m.store.CreateDownload(ctx, dl); err != nil { if err := m.store.CreateDownload(ctx, dl); err != nil {
return false, "", err return false, "", err
} }
@@ -643,7 +657,7 @@ func (m *Manager) Attempt(
)) ))
} }
go m.grab(context.WithoutCancel(ctx), dl, ranked[0], job) go m.grab(context.WithoutCancel(ctx), dl, pick, job, true)
return true, "", nil return true, "", nil
} }
@@ -678,7 +692,7 @@ func (m *Manager) Pick(
job := m.startJob(dl) job := m.startJob(dl)
go m.grab(context.WithoutCancel(ctx), dl, *chosen, job) go m.grab(context.WithoutCancel(ctx), dl, *chosen, job, false)
return nil return nil
} }
@@ -702,13 +716,20 @@ func (m *Manager) Cancel(ctx context.Context, downloadID string) error {
return nil return nil
} }
// grab drives one candidate all the way to the library. It runs on its // grab drives one request all the way to the library. It runs on its
// own goroutine and owns the job from here on. // own goroutine and owns the job from here on.
//
// When fallback is set and a candidate's transfer fails, the next
// candidate that auto-pick would itself have accepted is tried in its
// place (see nextCandidate). It is set for the two unattended routes
// and not for a candidate the user picked by hand: they chose that copy,
// and quietly substituting another is a decision they did not make.
func (m *Manager) grab( func (m *Manager) grab(
ctx context.Context, ctx context.Context,
dl Download, dl Download,
c Candidate, c Candidate,
job *jobs.Handle, job *jobs.Handle,
fallback bool,
) { ) {
ctx, cancel := context.WithTimeout(ctx, grabTimeout) ctx, cancel := context.WithTimeout(ctx, grabTimeout)
defer cancel() defer cancel()
@@ -723,6 +744,99 @@ func (m *Manager) grab(
m.actMu.Unlock() m.actMu.Unlock()
}() }()
var failed []Candidate
for {
out := m.attemptGrab(ctx, dl, c, job)
if out.err == nil {
m.finishGrab(ctx, dl, out.item, out.imported, job)
return
}
failed = append(failed, c)
next, ok := m.nextCandidate(ctx, dl, failed, out, fallback)
if !ok {
m.failDownload(ctx, job, dl.ID, out.err)
return
}
m.logger.Info(
"download candidate failed; trying the next",
"download", dl.ID,
"failed", c.ID,
"next", next.ID,
"error", out.err,
)
if job != nil {
job.Logf(jobs.LevelWarn, fmt.Sprintf(
"%s failed (%v); trying %s instead",
describeCandidate(c), out.err, describeCandidate(next),
))
}
// The failed attempt's staging holds at most a partial folder
// nobody is going to import, and the next attempt reserves its
// own. Only the final failure keeps its staging for inspection.
if out.item.StagingDir != "" {
if err := m.staging.Release(out.item.StagingDir); err != nil {
m.logger.Warn("could not release staging dir", "error", err)
}
}
c = next
}
}
// maxGrabAttempts bounds how many candidates one request will try. A
// popular album can have dozens of peers; the point of falling back is
// to survive the ordinary one or two that are offline, not to walk the
// whole list for six hours.
const maxGrabAttempts = 3
// peerKey names one Soulseek user on one daemon. The same username on
// two daemons is two logins and two queues.
type peerKey struct {
provider int64
peer string
}
// peerKeyFor returns the peer a candidate is fetched from, when the
// source is one where asking a peer for two things at once is rude.
func peerKeyFor(c Candidate) (peerKey, bool) {
if c.Kind != KindSlskd || c.Origin == "" {
return peerKey{}, false
}
return peerKey{provider: c.ProviderID, peer: c.Origin}, true
}
// grabOutcome is how one candidate's attempt ended.
type grabOutcome struct {
item DownloadItem
imported ImportResult
err error
// retryable reports whether another candidate might succeed where
// this one failed: the transfer failed, or delivered too little of
// the album. Anything else — no staging space, no library root, a
// tag write failing — would fail the next candidate identically.
retryable bool
}
// attemptGrab takes one candidate through transfer and import. It
// records the item's own failure, but not the download's: whether the
// download has failed is the caller's decision, since another candidate
// may yet succeed.
func (m *Manager) attemptGrab(
ctx context.Context,
dl Download,
c Candidate,
job *jobs.Handle,
) grabOutcome {
// Who will move the bytes is decided before any slot is taken, so // Who will move the bytes is decided before any slot is taken, so
// the transfer waits in its own provider's queue rather than in a // the transfer waits in its own provider's queue rather than in a
// global one. A delegate takes no slot at all: the transfer is // global one. A delegate takes no slot at all: the transfer is
@@ -731,21 +845,26 @@ func (m *Manager) grab(
// work against our budget. // work against our budget.
plan, err := m.planTransfer(dl, c) plan, err := m.planTransfer(dl, c)
if err != nil { if err != nil {
m.failDownload(ctx, job, dl.ID, err) return grabOutcome{err: err}
return
} }
if !plan.delegated() { if !plan.delegated() {
if key, ok := peerKeyFor(c); ok {
release, err := m.peerLocks.acquire(ctx, key)
if err != nil {
return grabOutcome{err: err}
}
defer release()
}
provSem := m.semaphoreFor(plan.transportID) provSem := m.semaphoreFor(plan.transportID)
select { select {
case provSem <- struct{}{}: case provSem <- struct{}{}:
defer func() { <-provSem }() defer func() { <-provSem }()
case <-ctx.Done(): case <-ctx.Done():
m.failDownload(ctx, job, dl.ID, ctx.Err()) return grabOutcome{err: ctx.Err()}
return
} }
globalSem := m.globalSem() globalSem := m.globalSem()
@@ -754,9 +873,7 @@ func (m *Manager) grab(
case globalSem <- struct{}{}: case globalSem <- struct{}{}:
defer func() { <-globalSem }() defer func() { <-globalSem }()
case <-ctx.Done(): case <-ctx.Done():
m.failDownload(ctx, job, dl.ID, ctx.Err()) return grabOutcome{err: ctx.Err()}
return
} }
} }
@@ -771,24 +888,30 @@ func (m *Manager) grab(
dir, err := m.staging.Reserve(item.ID) dir, err := m.staging.Reserve(item.ID)
if err != nil { if err != nil {
m.failDownload(ctx, job, dl.ID, err) return grabOutcome{err: err}
return
} }
item.StagingDir = dir item.StagingDir = dir
if err := m.store.CreateItem(ctx, item); err != nil { if err := m.store.CreateItem(ctx, item); err != nil {
m.failDownload(ctx, job, dl.ID, err) return grabOutcome{item: item, err: err}
}
return fail := func(err error, retryable bool) grabOutcome {
if serr := m.store.SetItemState(
ctx, item.ID, StateFailed, err.Error(),
); serr != nil {
m.logger.Warn("could not record item failure", "error", serr)
}
return grabOutcome{item: item, err: err, retryable: retryable}
} }
result, err := m.transfer(ctx, dl, item, plan, job) result, err := m.transfer(ctx, dl, item, plan, job)
if err != nil { if err != nil {
m.failItem(ctx, job, item, dl.ID, err) // A delegate's failure is the external manager's verdict on the
// whole request, not on one copy of it.
return return fail(err, !plan.delegated())
} }
m.setStates(ctx, dl.ID, item.ID, StateImporting) m.setStates(ctx, dl.ID, item.ID, StateImporting)
@@ -798,42 +921,116 @@ func (m *Manager) grab(
job.SetStages(importStages(2)) job.SetStages(importStages(2))
} }
var imported ImportResult
if result.Delegated { if result.Delegated {
// The external manager already placed and tagged these files in // The external manager already placed and tagged these files in
// its own library. Moving them out from under a system that is // its own library. Moving them out from under a system that is
// still managing them would be worse than useless, so the files // still managing them would be worse than useless, so the files
// are recorded where they are and the library scan picks them // are recorded where they are and the library scan picks them
// up in place. // up in place.
imported = ImportResult{Paths: result.Files}
if job != nil { if job != nil {
job.Logf(jobs.LevelInfo, fmt.Sprintf( job.Logf(jobs.LevelInfo, fmt.Sprintf(
"External manager imported %d files; recording them in place", "External manager imported %d files; recording them in place",
len(result.Files), len(result.Files),
)) ))
} }
} else {
opts := m.importOptions()
opts.WriteTags = true
opts.LibraryRoot, err = m.library.LibraryPath(dl.LibraryID) return grabOutcome{
if err != nil { item: item,
m.failItem(ctx, job, item, dl.ID, imported: ImportResult{Paths: result.Files},
fmt.Errorf("resolve library root: %w", err))
return
}
imported, err = m.importer.Import(ctx, dl, result, opts)
if err != nil {
m.failItem(ctx, job, item, dl.ID, err)
return
} }
} }
opts := m.importOptions()
opts.WriteTags = true
opts.LibraryRoot, err = m.library.LibraryPath(dl.LibraryID)
if err != nil {
return fail(fmt.Errorf("resolve library root: %w", err), false)
}
imported, err := m.importer.Import(ctx, dl, result, opts)
if err != nil {
return fail(err, errors.Is(err, ErrTooIncomplete))
}
return grabOutcome{item: item, imported: imported}
}
// nextCandidate picks the candidate to try after the ones in failed.
//
// It only ever offers a candidate auto-pick would have taken on its own
// (autoAcceptable), so falling back cannot lower the bar an unattended
// download is held to: the second choice has to clear the same gates
// the first did.
//
// On Soulseek a failure belongs to the *peer* — offline, refusing, or
// holding us in a queue — so every folder that peer offered is skipped
// with it. Elsewhere a failure belongs to the release, and only that
// candidate is.
func (m *Manager) nextCandidate(
ctx context.Context,
dl Download,
failed []Candidate,
out grabOutcome,
fallback bool,
) (Candidate, bool) {
if !fallback || !out.retryable || ctx.Err() != nil ||
len(failed) >= maxGrabAttempts {
return Candidate{}, false
}
m.resMu.RLock()
ranked := m.results[dl.ID]
m.resMu.RUnlock()
prefs := m.preferences()
for _, c := range ranked {
if ruledOutBy(c, failed) || !autoAcceptable(dl, c, prefs) {
continue
}
return c, true
}
return Candidate{}, false
}
// ruledOutBy reports whether a failure among failed also rules out c.
func ruledOutBy(c Candidate, failed []Candidate) bool {
for _, f := range failed {
if c.ID == f.ID && c.ProviderID == f.ProviderID {
return true
}
if c.Kind == KindSlskd && f.Kind == KindSlskd &&
c.ProviderID == f.ProviderID && c.Origin != "" &&
c.Origin == f.Origin {
return true
}
}
return false
}
// describeCandidate names a candidate for the job log.
func describeCandidate(c Candidate) string {
if c.Origin != "" {
return fmt.Sprintf("%q from %s", c.Title, c.Origin)
}
return fmt.Sprintf("%q", c.Title)
}
// finishGrab records a successful import and retires what the request
// was holding.
func (m *Manager) finishGrab(
ctx context.Context,
dl Download,
item DownloadItem,
imported ImportResult,
job *jobs.Handle,
) {
if err := m.store.SetItemImported( if err := m.store.SetItemImported(
ctx, item.ID, imported.Paths, ctx, item.ID, imported.Paths,
); err != nil { ); err != nil {
@@ -1177,23 +1374,6 @@ func (m *Manager) failDownload(
} }
} }
// failItem records an item-level failure and fails its download.
func (m *Manager) failItem(
ctx context.Context,
job *jobs.Handle,
item DownloadItem,
downloadID string,
err error,
) {
if serr := m.store.SetItemState(
ctx, item.ID, StateFailed, err.Error(),
); serr != nil {
m.logger.Warn("could not record item failure", "error", serr)
}
m.failDownload(ctx, job, downloadID, err)
}
// startJob registers the request in the background jobs panel. // startJob registers the request in the background jobs panel.
func (m *Manager) startJob(dl Download) *jobs.Handle { func (m *Manager) startJob(dl Download) *jobs.Handle {
if m.jobsReg == nil { if m.jobsReg == nil {
+140 -8
View File
@@ -66,6 +66,14 @@ var (
// separatorPattern splits "Artist - Album" style folder names. // separatorPattern splits "Artist - Album" style folder names.
separatorPattern = regexp.MustCompile(`\s+[-–—]\s+`) separatorPattern = regexp.MustCompile(`\s+[-–—]\s+`)
// discFolderPattern matches a directory that holds one disc of an
// album rather than the album: "CD1", "CD 2", "Disc 3", "Disk-1",
// "[Disc 2]", "CD1 - The Early Years". A number is required, so a
// folder merely called "CDs" is not one.
discFolderPattern = regexp.MustCompile(
`(?i)^\s*[\[(]?\s*(?:cd|disc|disk)\s*[-_.#]?\s*(\d{1,2})\b`,
)
) )
// FormatForPath returns the audio format implied by a path's extension, // FormatForPath returns the audio format implied by a path's extension,
@@ -94,16 +102,63 @@ type TrackHint struct {
Folder string Folder string
} }
// discFolder reports whether a directory name is one disc of an album,
// and which.
func discFolder(name string) (int, bool) {
m := discFolderPattern.FindStringSubmatch(name)
if m == nil {
return 0, false
}
n, err := strconv.Atoi(m[1])
if err != nil || n == 0 {
return 0, false
}
return n, true
}
// AlbumDir is the directory that holds a file's *album*: its parent,
// or its grandparent when the parent is a disc folder.
//
// Multi-disc rips are shared as `Album/CD1/…` and `Album/CD2/…`, and
// grouping candidates by the immediate parent split one album into two
// half-albums, each titled "CD1". Neither could clear the completeness
// or album-title bars, so a multi-disc release could not be auto-picked
// at all. A disc folder at the root has no album above it and is
// returned as it is.
func AlbumDir(p string) string {
dir := path.Dir(strings.ReplaceAll(p, `\`, "/"))
if _, ok := discFolder(path.Base(dir)); !ok {
return dir
}
parent := path.Dir(dir)
if parent == "." || parent == "/" || parent == "" {
return dir
}
return parent
}
// ParsePath extracts what it can from one candidate file path. // ParsePath extracts what it can from one candidate file path.
func ParsePath(p string) TrackHint { func ParsePath(p string) TrackHint {
// Soulseek paths are Windows-style; normalize before splitting. // Soulseek paths are Windows-style; normalize before splitting.
norm := strings.ReplaceAll(p, `\`, "/") norm := strings.ReplaceAll(p, `\`, "/")
base := path.Base(norm) base := path.Base(norm)
folder := path.Base(path.Dir(norm))
name := strings.TrimSuffix(base, path.Ext(base)) name := strings.TrimSuffix(base, path.Ext(base))
hint := TrackHint{Folder: cleanAlbumName(folder)} // The album's name is the album directory's, not a disc folder's,
// and the disc folder is where a multi-disc rip says which disc a
// file is on. A disc number in the filename ("2-01 …") is more
// specific and overrides it below.
hint := TrackHint{Folder: cleanAlbumName(path.Base(AlbumDir(norm)))}
if disc, ok := discFolder(path.Base(path.Dir(norm))); ok {
hint.Disc = disc
}
if m := trackNumPattern.FindStringSubmatch(name); m != nil { if m := trackNumPattern.FindStringSubmatch(name); m != nil {
if m[1] != "" { if m[1] != "" {
@@ -203,21 +258,72 @@ func AnnotateFiles(files []CandidateFile) []CandidateFile {
// matchFiles aligns a candidate's audio files to the expected tracklist // matchFiles aligns a candidate's audio files to the expected tracklist
// and returns the per-file assignment plus the mean title similarity of // and returns the per-file assignment plus the mean title similarity of
// the aligned pairs. // the aligned pairs. alignFiles is the same alignment with the
// duration evidence as well.
func matchFiles(
files []CandidateFile,
expected []ExpectedTrack,
) ([]CandidateFile, float64) {
a := alignFiles(files, expected)
return a.files, a.titleFit
}
// alignment is what aligning a candidate to a tracklist found.
type alignment struct {
files []CandidateFile
// titleFit is the mean title similarity over aligned pairs.
titleFit float64
// durationFit is the mean duration agreement over aligned pairs
// where both sides state a length, and timedPairs is how many such
// pairs there were.
durationFit float64
timedPairs int
aligned int
}
// durationAgreement scores how well a file's length matches the
// expected track's, in 0..1. Rips of the same master differ by a
// second or two of silence; a different edit, a live take or a
// truncated file differs by tens of seconds.
func durationAgreement(got, want int64) float64 {
const (
exactMillis = 3_000
wrongMillis = 30_000
)
d := got - want
if d < 0 {
d = -d
}
switch {
case d <= exactMillis:
return 1
case d >= wrongMillis:
return 0
default:
return 1 - float64(d-exactMillis)/float64(wrongMillis-exactMillis)
}
}
// alignFiles aligns a candidate's audio files to the expected tracklist.
// //
// Alignment is greedy by score rather than optimal: candidate folders // Alignment is greedy by score rather than optimal: candidate folders
// are small (a few dozen files at most) and the common cases — correct // are small (a few dozen files at most) and the common cases — correct
// track numbers, or clean "NN Title" names — are unambiguous, so the // track numbers, or clean "NN Title" names — are unambiguous, so the
// extra machinery of Hungarian assignment buys nothing here. // extra machinery of Hungarian assignment buys nothing here.
func matchFiles( func alignFiles(
files []CandidateFile, files []CandidateFile,
expected []ExpectedTrack, expected []ExpectedTrack,
) ([]CandidateFile, float64) { ) alignment {
annotated := make([]CandidateFile, len(files)) annotated := make([]CandidateFile, len(files))
copy(annotated, files) copy(annotated, files)
if len(expected) == 0 { if len(expected) == 0 {
return annotated, 0 return alignment{files: annotated}
} }
hints := make([]TrackHint, len(annotated)) hints := make([]TrackHint, len(annotated))
@@ -230,8 +336,19 @@ func matchFiles(
var ( var (
total float64 total float64
matched int matched int
durTotal float64
timed int
) )
// timing adds a pair's duration evidence when both sides state one.
timing := func(f CandidateFile, e ExpectedTrack) {
if f.LengthMillis > 0 && e.LengthMillis > 0 {
durTotal += durationAgreement(f.LengthMillis, e.LengthMillis)
timed++
}
}
// Pass 1: trust explicit track numbers when they are unique and in // Pass 1: trust explicit track numbers when they are unique and in
// range. A folder that numbers its files correctly is the strong // range. A folder that numbers its files correctly is the strong
// case, and title comparison only adds noise there. // case, and title comparison only adds noise there.
@@ -250,6 +367,8 @@ func matchFiles(
total += autotag.TitleSimilarity(hints[i].Title, expected[idx].Title) total += autotag.TitleSimilarity(hints[i].Title, expected[idx].Title)
matched++ matched++
timing(annotated[i], expected[idx])
} }
// Pass 2: title similarity for whatever is left. // Pass 2: title similarity for whatever is left.
@@ -284,13 +403,26 @@ func matchFiles(
total += bestSim total += bestSim
matched++ matched++
timing(annotated[i], expected[bestIdx])
} }
if matched == 0 { if matched == 0 {
return annotated, 0 return alignment{files: annotated}
} }
return annotated, total / float64(matched) a := alignment{
files: annotated,
titleFit: total / float64(matched),
timedPairs: timed,
aligned: matched,
}
if timed > 0 {
a.durationFit = durTotal / float64(timed)
}
return a
} }
// indexForPosition finds the expected track at a disc/track position. // indexForPosition finds the expected track at a disc/track position.
+224
View File
@@ -0,0 +1,224 @@
package download
import (
"context"
"errors"
"path/filepath"
"sync"
"testing"
"time"
)
// Soulseek politeness is per peer, not per daemon (#272).
func TestKeyedLockSerialisesOneKeyOnly(t *testing.T) {
t.Parallel()
var l keyedLock[string]
ctx := context.Background()
releaseA, err := l.acquire(ctx, "a")
if err != nil {
t.Fatalf("acquire a: %v", err)
}
// Another key is free while "a" is held.
releaseB, err := l.acquire(ctx, "b")
if err != nil {
t.Fatalf("acquire b: %v", err)
}
releaseB()
// The same key waits, and gives up with its context.
short, cancel := context.WithTimeout(ctx, 20*time.Millisecond)
defer cancel()
if _, err := l.acquire(short, "a"); !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("second acquire of a held key = %v, want the deadline", err)
}
releaseA()
releaseA() // Idempotent: a second call must not free someone else's hold.
if n := l.size(); n != 0 {
t.Errorf("%d keys left behind, want none once nobody holds or waits", n)
}
}
// grabEach runs one grab per candidate and returns a function that waits
// for all of them; grabAll's reasons for waiting apply.
func grabEach(t *testing.T, f managerFixture, cands []Candidate) func() {
t.Helper()
ctx := context.Background()
var wg sync.WaitGroup
for i, c := range cands {
dl := fourTrackDownload()
dl.ID = "dl-" + string(rune('a'+i))
if err := f.store.CreateDownload(ctx, dl); err != nil {
t.Fatalf("CreateDownload: %v", err)
}
wg.Add(1)
go func() {
defer wg.Done()
f.manager.grab(ctx, dl, c, nil, false)
}()
}
return func() {
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Error("transfers did not finish")
}
}
}
func slskdCandidates(p *FakeProvider, peers ...string) []Candidate {
out := make([]Candidate, 0, len(peers))
for i, peer := range peers {
c := p.Candidates[0]
c.ID = c.ID + "-" + itoa(i)
c.Kind = KindSlskd
c.ProviderID = 1
c.Origin = peer
out = append(out, c)
}
return out
}
// Three albums from one user are asked for one at a time, even though
// the daemon would allow three transfers.
func TestOnePeerIsAskedForOneThingAtATime(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
f.manager.SetMaxConcurrent(4)
p := fakeWithAlbum(1, "slskd", ".flac")
p.GrabGate = make(chan struct{})
f.manager.installProvider(Config{ID: 1, Kind: KindSlskd, Priority: 50}, p)
wait := grabEach(t, f, slskdCandidates(p, "alice", "alice", "alice"))
waitFor(t, func() bool { return p.GrabCallCount() >= 1 }, "no grab started")
time.Sleep(150 * time.Millisecond)
if got := p.MaxParallelGrabs(); got != 1 {
t.Errorf("%d simultaneous grabs from one peer, want 1", got)
}
close(p.GrabGate)
waitFor(t, func() bool { return p.GrabCallCount() == 3 }, "queued grabs never ran")
wait()
if n := f.manager.peerLocks.size(); n != 0 {
t.Errorf("%d peer locks left behind", n)
}
}
// Different users run at once, up to the daemon's cap — the point of
// the change: one slow peer no longer holds up every other.
func TestDifferentPeersRunTogether(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
f.manager.SetMaxConcurrent(8)
p := fakeWithAlbum(1, "slskd", ".flac")
p.GrabGate = make(chan struct{})
f.manager.installProvider(Config{ID: 1, Kind: KindSlskd, Priority: 50}, p)
wait := grabEach(t, f, slskdCandidates(p, "alice", "bob", "carol", "dave"))
waitFor(
t,
func() bool { return p.MaxParallelGrabs() >= kindConcurrency[KindSlskd] },
"different peers were serialised",
)
time.Sleep(100 * time.Millisecond)
if got := p.MaxParallelGrabs(); got != kindConcurrency[KindSlskd] {
t.Errorf("%d simultaneous grabs, want the daemon cap %d", got, kindConcurrency[KindSlskd])
}
close(p.GrabGate)
wait()
}
func TestSlskdLocalFolders(t *testing.T) {
t.Parallel()
s := &slskd{downloadsPath: "/dl"}
got := s.localFolders(Candidate{Files: []CandidateFile{
{Path: `\m\The Wall\CD2\01 Hey You.flac`},
{Path: `\m\The Wall\CD1\01 In The Flesh.flac`},
{Path: `\m\The Wall\CD1\02 The Thin Ice.flac`},
}})
want := []string{filepath.Join("/dl", "CD1"), filepath.Join("/dl", "CD2")}
if len(got) != len(want) || got[0] != want[0] || got[1] != want[1] {
t.Errorf("localFolders = %q, want %q", got, want)
}
}
// Two peers' "Greatest Hits" land in one slskd directory, so the second
// grab does not enqueue until the first has collected its files.
func TestSlskdSameFolderNameWaits(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
s, downloads := newStubSlskd(t, stub)
c := Candidate{
Payload: map[string]string{"username": "bob"},
Files: []CandidateFile{
{Path: `\music\Greatest Hits\01 Intro.flac`, Size: 1, IsAudio: true},
},
}
release, err := lockSlskdFolders(
context.Background(), []string{filepath.Join(downloads, "Greatest Hits")},
)
if err != nil {
t.Fatalf("lock: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if _, err := s.Grab(ctx, c, t.TempDir(), nil); !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("Grab = %v, want it to wait on the held folder", err)
}
release()
stub.mu.Lock()
posted := stub.posted
stub.mu.Unlock()
if posted {
t.Error("enqueued transfers into a folder another grab held")
}
}
+8 -7
View File
@@ -236,17 +236,18 @@ func Register(d Descriptor, c Constructor) {
} }
// concurrencyField describes the per-provider transfer limit, with help // concurrencyField describes the per-provider transfer limit, with help
// text explaining why the default is what it is — a user who raises // text explaining what the number means where it means something
// slskd from 1 to 8 and gets themselves queued behind every other // unusual: on slskd it counts peers, since each peer is only ever asked
// Soulseek user deserves to have been warned. // for one folder at a time whatever it is set to.
func concurrencyField(k Kind) Field { func concurrencyField(k Kind) Field {
help := "Maximum simultaneous transfers from this client." help := "Maximum simultaneous transfers from this client."
if k == KindSlskd { if k == KindSlskd {
help = "Maximum simultaneous transfers. Soulseek peers serve " + help = "How many Soulseek users to download from at once. " +
"one file at a time and queue or ban clients that ask for " + "Each user is only ever asked for one album at a time, " +
"more, so 1 is both the polite setting and usually the " + "since peers queue or ban clients that ask for more; " +
"fastest." "this bounds how many different users are asked in " +
"parallel."
} }
return Field{ return Field{
+597 -85
View File
@@ -5,9 +5,12 @@ import (
"errors" "errors"
"fmt" "fmt"
"log/slog" "log/slog"
"net/url"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
"regexp"
"slices"
"strings" "strings"
"time" "time"
@@ -69,12 +72,34 @@ const (
slskdTransferPoll = 3 * time.Second slskdTransferPoll = 3 * time.Second
// slskdMinFiles is the fewest audio files a folder needs before it // slskdMinFiles is the fewest audio files a folder needs before it
// is offered as a candidate. Soulseek returns a lot of one-file // is offered as a candidate for an album. Soulseek returns a lot of
// noise for common queries. // one-file noise for common queries. A single-track request takes
// one (see minFilesFor).
slskdMinFiles = 2 slskdMinFiles = 2
// slskdHTTPTimeout bounds one API call. // slskdHTTPTimeout bounds one API call.
slskdHTTPTimeout = 20 * time.Second slskdHTTPTimeout = 20 * time.Second
// millisPerSecond converts slskd's whole-second file lengths.
millisPerSecond = 1000
// slskdStallAfter is how long a grab may go without a byte arriving
// before the peer is given up on. It is measured from enqueue, so
// it covers a peer that queues us and never starts as well as one
// that starts and stops. Ten minutes is long enough for a short
// queue ahead of us to clear and short enough that one unresponsive
// peer does not hold slskd's single transfer slot for an evening.
slskdStallAfter = 10 * time.Minute
// slskdAbsentGrace is how long a requested file may be missing from
// slskd's transfer list before it is counted as failed. slskd lists
// a transfer as soon as it accepts it, so a file still absent after
// a few polls was refused.
slskdAbsentGrace = 30 * time.Second
// slskdCancelTimeout bounds the cleanup that cancels abandoned
// transfers.
slskdCancelTimeout = 15 * time.Second
) )
func init() { func init() {
@@ -134,6 +159,8 @@ type slskd struct {
searchPoll time.Duration searchPoll time.Duration
searchWait time.Duration searchWait time.Duration
transferPoll time.Duration transferPoll time.Duration
stallAfter time.Duration
absentGrace time.Duration
} }
// newSlskd builds the provider from config. // newSlskd builds the provider from config.
@@ -188,6 +215,8 @@ func newSlskd(
searchPoll: slskdSearchPoll, searchPoll: slskdSearchPoll,
searchWait: slskdSearchWait, searchWait: slskdSearchWait,
transferPoll: slskdTransferPoll, transferPoll: slskdTransferPoll,
stallAfter: slskdStallAfter,
absentGrace: slskdAbsentGrace,
}, nil }, nil
} }
@@ -231,14 +260,13 @@ type slskdSearch struct {
// slskdResponse is one peer's answer to a search. // slskdResponse is one peer's answer to a search.
type slskdResponse struct { type slskdResponse struct {
Username string `json:"username"` Username string `json:"username"`
HasFreeUploadSlot bool `json:"hasFreeUploadSlot"` HasFreeUploadSlot bool `json:"hasFreeUploadSlot"`
QueueLength int `json:"queueLength"` QueueLength int `json:"queueLength"`
UploadSpeed int64 `json:"uploadSpeed"` UploadSpeed int64 `json:"uploadSpeed"`
Files []slskdFile `json:"files"` Files []slskdFile `json:"files"`
LockedFileCount int `json:"lockedFileCount"` LockedFileCount int `json:"lockedFileCount"`
FileCount int `json:"fileCount"` FileCount int `json:"fileCount"`
FreeUploadSlotFlag bool `json:"freeUploadSlots"`
} }
// slskdFile is one file a peer is offering. // slskdFile is one file a peer is offering.
@@ -246,7 +274,9 @@ type slskdFile struct {
Filename string `json:"filename"` Filename string `json:"filename"`
Size int64 `json:"size"` Size int64 `json:"size"`
BitRate int `json:"bitRate"` BitRate int `json:"bitRate"`
Length int `json:"length"`
// Length is the duration in whole seconds.
Length int `json:"length"`
} }
// slskdTransfer is one download's state. // slskdTransfer is one download's state.
@@ -278,24 +308,89 @@ func (t slskdTransfer) done() (finished, ok bool) {
// per-folder candidates. A folder from one peer is the unit a user // per-folder candidates. A folder from one peer is the unit a user
// actually wants: Soulseek has no album concept, but people organise // actually wants: Soulseek has no album concept, but people organise
// their shares by album directory. // their shares by album directory.
//
// Up to two queries run at once — the request as written and a
// normalised form of it (see slskdQueries) — and their candidates are
// merged. They run concurrently rather than as a fallback because the
// manager gives a provider one search budget, and a Soulseek search
// spends most of it waiting for peers to answer; a second query after
// the first would not fit.
func (s *slskd) Search(ctx context.Context, dl Download) ([]Candidate, error) { func (s *slskd) Search(ctx context.Context, dl Download) ([]Candidate, error) {
queries := slskdQueries(dl)
if len(queries) == 0 {
return nil, nil
}
type found struct {
candidates []Candidate
err error
}
results := make(chan found, len(queries))
for _, q := range queries {
go func(q string) {
c, err := s.searchOnce(ctx, q, minFilesFor(dl))
results <- found{candidates: c, err: err}
}(q)
}
var (
out []Candidate
seen = map[string]bool{}
firstErr error
answered int
)
for range queries {
r := <-results
if r.err != nil {
s.logger.Debug("slskd search failed", "error", r.err)
if firstErr == nil {
firstErr = r.err
}
continue
}
answered++
// The same peer's folder turns up under both queries; the ID is
// peer and folder, so it is the same candidate.
for _, c := range r.candidates {
if seen[c.ID] {
continue
}
seen[c.ID] = true
out = append(out, c)
}
}
if answered == 0 {
return nil, firstErr
}
return out, nil
}
// searchOnce runs one query to completion and returns its candidates.
func (s *slskd) searchOnce(
ctx context.Context,
text string,
minFiles int,
) ([]Candidate, error) {
// slskd's search endpoint deserializes id as a .NET Guid server-side, // slskd's search endpoint deserializes id as a .NET Guid server-side,
// so it must be a dashed UUID — the app's own newID() (a plain hex // so it must be a dashed UUID — the app's own newID() (a plain hex
// string, used for request/item IDs elsewhere) is rejected with an // string, used for request/item IDs elsewhere) is rejected with an
// HTTP 400 before any search happens. // HTTP 400 before any search happens.
searchID := uuid.NewString() searchID := uuid.NewString()
body := map[string]any{ if err := s.client.post(
"id": searchID, ctx, "/api/v0/searches", s.searchRequest(searchID, text, minFiles), nil,
"searchText": dl.SearchText(), ); err != nil {
}
if err := s.client.post(ctx, "/api/v0/searches", body, nil); err != nil {
return nil, err
}
search, err := s.awaitSearch(ctx, searchID)
if err != nil {
return nil, err return nil, err
} }
@@ -307,52 +402,222 @@ func (s *slskd) Search(ctx context.Context, dl Download) ([]Candidate, error) {
) )
}() }()
return s.candidatesFrom(search), nil if err := s.awaitSearch(ctx, searchID); err != nil {
return nil, err
}
responses, err := s.searchResponses(ctx, searchID)
if err != nil {
return nil, err
}
return s.candidatesFrom(responses, minFiles), nil
}
// searchRequest is the body that starts a search.
//
// Every option is stated rather than left to the daemon, because
// slskd's defaults are its own and not ours. Its search timeout in
// particular has to finish inside our wait: a search that slskd is still
// running when we stop polling is results we asked for and discarded.
// The response and file limits are raised well above what a popular
// album produces, and the peer filters let slskd drop answers this
// provider would only score down to nothing — a folder too small to be
// a candidate, a peer with a queue it will not reach today.
func (s *slskd) searchRequest(id, text string, minFiles int) map[string]any {
const (
responseLimit = 500
fileLimit = 20_000
maximumPeerQueueLength = 100
)
// A tenth of the wait is left for the last poll and the responses
// fetch.
timeout := s.searchWait - s.searchWait/10
return map[string]any{
"id": id,
"searchText": text,
"searchTimeout": timeout.Milliseconds(),
"responseLimit": responseLimit,
"fileLimit": fileLimit,
"filterResponses": true,
"minimumResponseFileCount": minFiles,
"maximumPeerQueueLength": maximumPeerQueueLength,
}
}
// slskdQueries is what is searched for a request: the request's own
// search text, and a normalised form of it when that differs.
//
// Soulseek matches every term against the file's full path, so each
// extra word is a filter, and some words filter wrongly:
//
// - edition qualifiers — "(Deluxe Edition)", "[2011 Remaster]" — are
// in the catalog's title and rarely in anyone's folder name;
// - punctuation splits a term oddly, and a term that starts with "-"
// is an *exclusion*, so an album called "-ism" searches for
// everything without it;
// - "Various Artists" is in no one's path for a compilation.
//
// A query the user typed is theirs and is searched exactly as written.
func slskdQueries(dl Download) []string {
primary := strings.TrimSpace(dl.SearchText())
if primary == "" {
return nil
}
out := []string{primary}
if dl.Query != "" {
return out
}
artist := dl.Artist
if isVariousArtists(artist) {
artist = ""
}
normal := Download{
Artist: normalizeSearchTerms(artist),
Album: normalizeSearchTerms(editionPattern.ReplaceAllString(dl.Album, " ")),
}
if alt := strings.TrimSpace(normal.SearchText()); alt != "" &&
!strings.EqualFold(alt, primary) {
out = append(out, alt)
}
return out
}
var (
// editionPattern finds an edition qualifier: a bracketed group that
// names an edition, or a trailing " - 2011 Remaster".
editionPattern = regexp.MustCompile(
`(?i)\s*[(\[][^)\]]*\b(?:deluxe|edition|remaster(?:ed)?|expanded|` +
`anniversary|bonus|explicit|reissue|special|collector'?s?|` +
`version|mono|stereo)\b[^)\]]*[)\]]` +
`|\s+-\s+(?:\d{4}\s+)?remaster(?:ed)?\b.*$`,
)
// nonWordPattern is everything that is not a letter or a digit.
nonWordPattern = regexp.MustCompile(`[^\p{L}\p{N}]+`)
)
// normalizeSearchTerms reduces text to plain words.
func normalizeSearchTerms(s string) string {
return strings.Join(strings.Fields(nonWordPattern.ReplaceAllString(s, " ")), " ")
}
// isVariousArtists reports whether an artist credit is a compilation's
// placeholder rather than an artist.
func isVariousArtists(artist string) bool {
switch strings.ToLower(strings.TrimSpace(artist)) {
case "various artists", "various", "va":
return true
default:
return false
}
}
// minFilesFor is the fewest audio files a folder must offer to be a
// candidate for this request.
//
// Soulseek answers a search with the files that match it, not with the
// folders they sit in. An album query matches every file in the album's
// folder, because the folder name carries the terms; a *track* query
// usually matches one file per folder. The two-file floor that filters
// out one-file noise for an album therefore filtered out every result
// for a track, and a single-track request could never be served here.
func minFilesFor(dl Download) int {
if dl.RecordingMBID != "" {
return 1
}
return slskdMinFiles
} }
// awaitSearch polls until the search completes or the budget runs out. // awaitSearch polls until the search completes or the budget runs out.
// A timeout is not an error: partial Soulseek results are normal and // A timeout is not an error: partial Soulseek results are normal and
// often good enough. // often good enough.
func (s *slskd) awaitSearch( //
ctx context.Context, // The poll asks for the search's state only. It used to ask for every
searchID string, // response on every one-second tick, which for a popular album is the
) (slskdSearch, error) { // same few thousand file entries serialised twenty times to be read
// once; searchResponses fetches them once at the end.
func (s *slskd) awaitSearch(ctx context.Context, searchID string) error {
deadline := time.Now().Add(s.searchWait) deadline := time.Now().Add(s.searchWait)
var last slskdSearch
for time.Now().Before(deadline) { for time.Now().Before(deadline) {
select { select {
case <-ctx.Done(): case <-ctx.Done():
return last, fmt.Errorf("%w: search cancelled", ErrSlskdTimeout) return fmt.Errorf("%w: search cancelled", ErrSlskdTimeout)
case <-time.After(s.searchPoll): case <-time.After(s.searchPoll):
} }
var search slskdSearch var search slskdSearch
if err := s.client.get( if err := s.client.get(
ctx, ctx, "/api/v0/searches/"+searchID, &search,
"/api/v0/searches/"+searchID+"?includeResponses=true",
&search,
); err != nil { ); err != nil {
return last, err return err
} }
last = search
if search.IsComplete { if search.IsComplete {
return search, nil return nil
} }
} }
return last, nil return nil
} }
// candidatesFrom groups a search's responses into candidates. // searchResponses fetches a search's responses once.
func (s *slskd) candidatesFrom(search slskdSearch) []Candidate { //
out := make([]Candidate, 0, len(search.Responses)) // `/searches/{id}/responses` is the endpoint for that; a daemon that
// does not answer it is asked the older way, with the search itself
// carrying its responses, so an older slskd degrades to the previous
// behaviour rather than to no results at all.
func (s *slskd) searchResponses(
ctx context.Context,
searchID string,
) ([]slskdResponse, error) {
var responses []slskdResponse
for _, resp := range search.Responses { err := s.client.get(
ctx, "/api/v0/searches/"+searchID+"/responses", &responses,
)
if err == nil {
return responses, nil
}
s.logger.Debug(
"slskd responses endpoint failed; asking with the search",
"error", err,
)
var search slskdSearch
if err := s.client.get(
ctx,
"/api/v0/searches/"+searchID+"?includeResponses=true",
&search,
); err != nil {
return nil, err
}
return search.Responses, nil
}
// candidatesFrom groups a search's responses into candidates, dropping
// folders with fewer than minFiles audio files.
func (s *slskd) candidatesFrom(
responses []slskdResponse,
minFiles int,
) []Candidate {
out := make([]Candidate, 0, len(responses))
for _, resp := range responses {
for folder, files := range groupByFolder(resp.Files) { for folder, files := range groupByFolder(resp.Files) {
audio := 0 audio := 0
@@ -372,12 +637,14 @@ func (s *slskd) candidatesFrom(search slskdSearch) []Candidate {
Format: format, Format: format,
Bitrate: f.BitRate, Bitrate: f.BitRate,
IsAudio: isAudio, IsAudio: isAudio,
LengthMillis: int64(f.Length) * millisPerSecond,
}) })
total += f.Size total += f.Size
} }
if audio < slskdMinFiles { if audio < minFiles {
continue continue
} }
@@ -398,13 +665,15 @@ func (s *slskd) candidatesFrom(search slskdSearch) []Candidate {
return out return out
} }
// groupByFolder buckets a peer's files by their containing directory. // groupByFolder buckets a peer's files by the album directory they sit
// in — the containing directory, or the one above it for a disc folder
// (see AlbumDir), so a multi-disc rip is one candidate and not two.
func groupByFolder(files []slskdFile) map[string][]slskdFile { func groupByFolder(files []slskdFile) map[string][]slskdFile {
out := map[string][]slskdFile{} out := map[string][]slskdFile{}
for _, f := range files { for _, f := range files {
norm := strings.ReplaceAll(f.Filename, `\`, "/") dir := AlbumDir(f.Filename)
out[path.Dir(norm)] = append(out[path.Dir(norm)], f) out[dir] = append(out[dir], f)
} }
return out return out
@@ -418,7 +687,7 @@ func groupByFolder(files []slskdFile) map[string][]slskdFile {
func peerHealth(r slskdResponse) float64 { func peerHealth(r slskdResponse) float64 {
score := 0.35 score := 0.35
if r.HasFreeUploadSlot || r.FreeUploadSlotFlag { if r.HasFreeUploadSlot {
score += 0.4 score += 0.4
} }
@@ -467,6 +736,21 @@ func (s *slskd) Grab(
) )
} }
// slskd keeps finished transfers listed until someone removes them,
// and a transfer is matched to the request by filename. A record
// left by an earlier attempt at the same file from the same peer
// would otherwise be read as this attempt's answer the moment the
// first poll came back — an old failure failing a transfer that has
// not started. So what is already terminal is noted before enqueueing
// and ignored after.
release, err := lockSlskdFolders(ctx, s.localFolders(c))
if err != nil {
return Result{}, err
}
defer release()
stale := s.terminalTransferIDs(ctx, username)
wanted := make([]map[string]any, 0, len(c.Files)) wanted := make([]map[string]any, 0, len(c.Files))
for _, f := range c.Files { for _, f := range c.Files {
wanted = append(wanted, map[string]any{ wanted = append(wanted, map[string]any{
@@ -476,24 +760,122 @@ func (s *slskd) Grab(
} }
if err := s.client.post( if err := s.client.post(
ctx, "/api/v0/transfers/downloads/"+username, wanted, nil, ctx, slskdDownloadsPath(username), wanted, nil,
); err != nil { ); err != nil {
return Result{}, err return Result{}, err
} }
if err := s.awaitTransfers(ctx, username, c, onProgress); err != nil { if err := s.awaitTransfers(
ctx, username, stale, c, onProgress,
); err != nil {
return Result{}, err return Result{}, err
} }
return s.collect(c, dst) return s.collect(c, dst)
} }
// slskdFolders serialises grabs that land in the same local folder.
//
// slskd names a download's directory after the remote *leaf* folder, so
// two different albums both shared as "Greatest Hits" — or any two
// multi-disc rips, whose leaves are "CD1" and "CD2" — are written into
// one directory, and collect finds files by name there. Run at once,
// a file one peer never sent is filled by the other peer's file of the
// same name. One grab per peer made that impossible; several peers at
// once makes it likely. It is package-level and keyed on the full
// path because two configured clients can share one daemon.
var slskdFolders keyedLock[string]
// localFolders returns the directories under downloadsPath a candidate's
// files will be written to, sorted so every grab takes them in the same
// order and two cannot each hold what the other waits for.
func (s *slskd) localFolders(c Candidate) []string {
var out []string
for _, f := range c.Files {
norm := strings.ReplaceAll(f.Path, `\`, "/")
out = append(out, filepath.Join(s.downloadsPath, path.Base(path.Dir(norm))))
}
slices.Sort(out)
return slices.Compact(out)
}
// lockSlskdFolders takes every folder in order, releasing what it holds
// if the context ends part way.
func lockSlskdFolders(ctx context.Context, folders []string) (func(), error) {
releases := make([]func(), 0, len(folders))
releaseAll := func() {
for _, r := range slices.Backward(releases) {
r()
}
}
for _, f := range folders {
r, err := slskdFolders.acquire(ctx, f)
if err != nil {
releaseAll()
return nil, err
}
releases = append(releases, r)
}
return releaseAll, nil
}
// slskdDownloadsPath is the transfers endpoint for one peer. Soulseek
// usernames may contain spaces and punctuation, so the name is escaped
// rather than spliced into the path.
func slskdDownloadsPath(username string) string {
return "/api/v0/transfers/downloads/" + url.PathEscape(username)
}
// terminalTransferIDs returns the ids of this peer's transfers that are
// already finished. Best effort: slskd answers 404 for a peer it has no
// transfers with, and any failure here means only that there is nothing
// to ignore.
func (s *slskd) terminalTransferIDs(
ctx context.Context,
username string,
) map[string]bool {
transfers, err := s.transfersFor(ctx, username)
if err != nil {
return nil
}
out := make(map[string]bool, len(transfers))
for _, t := range transfers {
if finished, _ := t.done(); finished && t.ID != "" {
out[t.ID] = true
}
}
return out
}
// awaitTransfers polls until every requested file reaches a terminal // awaitTransfers polls until every requested file reaches a terminal
// state. Soulseek queues are measured in hours, so the only deadline // state, the transfer stalls, or the caller gives up.
// is the caller's context. //
// Soulseek queues are measured in hours, so there is no deadline on the
// transfer as a whole — but there is one on *progress*. slskd's
// transfer limit is one, so a peer that holds us in its queue without
// sending a byte is not only failing this download, it is holding every
// other Soulseek download behind it. After stallAfter with nothing
// moving the peer is given up on, and the manager tries another.
//
// Whatever way this ends short of every file finishing, the transfers
// still live in slskd are cancelled there. Returning without doing so
// leaves the daemon downloading into its own folder for a request
// nobody is waiting on any more.
func (s *slskd) awaitTransfers( func (s *slskd) awaitTransfers(
ctx context.Context, ctx context.Context,
username string, username string,
stale map[string]bool,
c Candidate, c Candidate,
onProgress ProgressFunc, onProgress ProgressFunc,
) error { ) error {
@@ -502,9 +884,18 @@ func (s *slskd) awaitTransfers(
wanted[f.Path] = true wanted[f.Path] = true
} }
var (
started = time.Now()
lastProgress = started
lastBytes int64
live []slskdTransfer
)
for { for {
select { select {
case <-ctx.Done(): case <-ctx.Done():
s.cancelTransfers(username, live)
return fmt.Errorf("%w: transfer cancelled", ErrSlskdTimeout) return fmt.Errorf("%w: transfer cancelled", ErrSlskdTimeout)
case <-time.After(s.transferPoll): case <-time.After(s.transferPoll):
} }
@@ -512,60 +903,177 @@ func (s *slskd) awaitTransfers(
transfers, err := s.transfersFor(ctx, username) transfers, err := s.transfersFor(ctx, username)
if err != nil { if err != nil {
// A blip talking to the daemon should not abandon a // A blip talking to the daemon should not abandon a
// transfer that may be hours in. // transfer that may be hours in — but a daemon that stays
// away is a stall like any other.
s.logger.Debug("slskd transfer poll failed", "error", err) s.logger.Debug("slskd transfer poll failed", "error", err)
if time.Since(lastProgress) >= s.stallAfter {
s.cancelTransfers(username, live)
return fmt.Errorf(
"%w: slskd has not answered for %s: %w",
ErrSlskdTimeout, s.stallAfter, err,
)
}
continue continue
} }
var ( tally := tallyTransfers(
done, failed int transfers, wanted, stale,
current int64 time.Since(started) >= s.absentGrace,
) )
live = tally.live
for _, t := range transfers { if tally.bytes > lastBytes {
if !wanted[t.Filename] { lastBytes = tally.bytes
continue lastProgress = time.Now()
}
current += t.BytesTransferred
finished, ok := t.done()
if !finished {
continue
}
if ok {
done++
} else {
failed++
}
} }
if onProgress != nil { if onProgress != nil {
onProgress(Progress{ onProgress(Progress{
Current: current, Current: tally.bytes,
Total: c.TotalSize, Total: c.TotalSize,
Phase: fmt.Sprintf( Phase: fmt.Sprintf(
"Transferring from %s (%d/%d)", username, done, len(wanted), "Transferring from %s (%d/%d)",
username, tally.done, len(wanted),
), ),
}) })
} }
if done+failed < len(wanted) { if tally.done+tally.failed >= len(wanted) {
// Some files failing is normal — a peer goes offline
// mid-folder. Let the importer's completeness check decide
// whether what arrived is enough, rather than discarding it
// here.
if tally.done == 0 {
return fmt.Errorf(
"%w: all %d files failed",
ErrSlskdTransferFailed, tally.failed,
)
}
return nil
}
if time.Since(lastProgress) < s.stallAfter {
continue continue
} }
// Some files failing is normal — a peer goes offline mid-folder. s.cancelTransfers(username, live)
// Let the importer's completeness check decide whether what
// arrived is enough, rather than discarding it here. // A folder that stalls on its last track is the same shape as
if done == 0 { // one whose last track failed, and goes forward the same way.
return fmt.Errorf( if tally.done > 0 {
"%w: all %d files failed", ErrSlskdTransferFailed, failed, s.logger.Info(
"slskd transfer stalled; keeping what arrived",
"peer", username,
"done", tally.done,
"wanted", len(wanted),
) )
return nil
} }
return nil return fmt.Errorf(
"%w: %s sent nothing in %s",
ErrSlskdTimeout, username, s.stallAfter,
)
}
}
// transferTally is one poll's reading of the files a grab asked for.
type transferTally struct {
done, failed int
bytes int64
// live are the requested transfers slskd is still working on,
// which are what has to be cancelled if the grab is abandoned.
live []slskdTransfer
}
// tallyTransfers reads a peer's transfer list against the files a grab
// asked for.
//
// A requested file slskd does not list at all is one it never accepted
// — refused at enqueue, or dropped — and it will never reach a terminal
// state to be counted by. Once absentExpired, such a file counts as
// failed, or the grab would wait on it until the six-hour ceiling.
func tallyTransfers(
transfers []slskdTransfer,
wanted map[string]bool,
stale map[string]bool,
absentExpired bool,
) transferTally {
seen := make(map[string]slskdTransfer, len(wanted))
for _, t := range transfers {
if !wanted[t.Filename] || stale[t.ID] {
continue
}
seen[t.Filename] = t
}
var out transferTally
for name := range wanted {
t, ok := seen[name]
if !ok {
if absentExpired {
out.failed++
}
continue
}
out.bytes += t.BytesTransferred
finished, succeeded := t.done()
switch {
case !finished:
out.live = append(out.live, t)
case succeeded:
out.done++
default:
out.failed++
}
}
return out
}
// cancelTransfers asks slskd to cancel and forget transfers this grab
// is abandoning. It runs on a context of its own: the usual reason to
// be here is that the caller's context has just been cancelled, and a
// cleanup that inherited it would never be sent.
func (s *slskd) cancelTransfers(username string, live []slskdTransfer) {
if len(live) == 0 {
return
}
ctx, cancel := context.WithTimeout(
context.Background(), slskdCancelTimeout,
)
defer cancel()
for _, t := range live {
if t.ID == "" {
continue
}
endpoint := slskdDownloadsPath(username) + "/" +
url.PathEscape(t.ID) + "?remove=true"
if err := s.client.delete(ctx, endpoint); err != nil {
s.logger.Warn(
"could not cancel slskd transfer",
"peer", username,
"file", t.Filename,
"error", err,
)
}
} }
} }
@@ -581,9 +1089,7 @@ func (s *slskd) transfersFor(
} `json:"directories"` } `json:"directories"`
} }
if err := s.client.get( if err := s.client.get(ctx, slskdDownloadsPath(username), &raw); err != nil {
ctx, "/api/v0/transfers/downloads/"+username, &raw,
); err != nil {
return nil, err return nil, err
} }
@@ -616,7 +1122,13 @@ func (s *slskd) collect(c Candidate, dst string) (Result, error) {
continue continue
} }
// A multi-disc candidate keeps its disc folders in staging.
// Flattened, disc 2's "01 Intro.flac" overwrites disc 1's, and
// the importer loses the folder it reads the disc number from.
target := filepath.Join(dst, base) target := filepath.Join(dst, base)
if _, ok := discFolder(folder); ok {
target = filepath.Join(dst, folder, base)
}
if err := movePath(src, target); err != nil { if err := movePath(src, target); err != nil {
return Result{}, fmt.Errorf("collect %s: %w", base, err) return Result{}, fmt.Errorf("collect %s: %w", base, err)
+368 -1
View File
@@ -31,11 +31,30 @@ type slskdStub struct {
transfers [][]slskdTransfer transfers [][]slskdTransfer
pollCount int pollCount int
// before is what the downloads endpoint reports until something is
// enqueued: records slskd already held from earlier attempts.
before []slskdTransfer
// enqueued records what was requested for download. // enqueued records what was requested for download.
enqueued []map[string]any enqueued []map[string]any
posted bool
// paths records the escaped path of every transfers call, and
// cancelled the escaped request URI of every DELETE.
paths []string
cancelled []string
// unauthorized makes every call return 401. // unauthorized makes every call return 401.
unauthorized bool unauthorized bool
// searches records every search request body, and searchGets the
// request URI of every search GET.
searches []map[string]any
searchGets []string
// noResponsesEndpoint makes /searches/{id}/responses 404, as an
// older daemon would.
noResponsesEndpoint bool
} }
func newSlskdStub(t *testing.T) *slskdStub { func newSlskdStub(t *testing.T) *slskdStub {
@@ -57,6 +76,16 @@ func newSlskdStub(t *testing.T) *slskdStub {
return return
} }
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Errorf("decode search body: %v", err)
}
s.mu.Lock()
s.searches = append(s.searches, body)
s.mu.Unlock()
w.WriteHeader(http.StatusCreated) w.WriteHeader(http.StatusCreated)
}) })
@@ -73,8 +102,26 @@ func newSlskdStub(t *testing.T) *slskdStub {
s.mu.Lock() s.mu.Lock()
responses := s.responses responses := s.responses
noEndpoint := s.noResponsesEndpoint
s.searchGets = append(s.searchGets, r.URL.RequestURI())
s.mu.Unlock() s.mu.Unlock()
if strings.HasSuffix(r.URL.Path, "/responses") {
if noEndpoint {
w.WriteHeader(http.StatusNotFound)
return
}
writeJSON(t, w, responses)
return
}
if r.URL.Query().Get("includeResponses") != "true" {
responses = nil
}
writeJSON(t, w, slskdSearch{ writeJSON(t, w, slskdSearch{
ID: "search-1", ID: "search-1",
IsComplete: true, IsComplete: true,
@@ -87,7 +134,12 @@ func newSlskdStub(t *testing.T) *slskdStub {
return return
} }
if r.Method == http.MethodPost { s.mu.Lock()
s.paths = append(s.paths, r.URL.EscapedPath())
s.mu.Unlock()
switch r.Method {
case http.MethodPost:
var body []map[string]any var body []map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil { if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
@@ -96,15 +148,35 @@ func newSlskdStub(t *testing.T) *slskdStub {
s.mu.Lock() s.mu.Lock()
s.enqueued = body s.enqueued = body
s.posted = true
s.mu.Unlock() s.mu.Unlock()
w.WriteHeader(http.StatusCreated) w.WriteHeader(http.StatusCreated)
return
case http.MethodDelete:
s.mu.Lock()
s.cancelled = append(s.cancelled, r.URL.RequestURI())
s.mu.Unlock()
w.WriteHeader(http.StatusNoContent)
return return
} }
s.mu.Lock() s.mu.Lock()
if !s.posted {
before := s.before
s.mu.Unlock()
writeJSON(t, w, map[string]any{
"directories": []map[string]any{{"files": before}},
})
return
}
idx := s.pollCount idx := s.pollCount
if idx >= len(s.transfers) { if idx >= len(s.transfers) {
idx = len(s.transfers) - 1 idx = len(s.transfers) - 1
@@ -191,6 +263,11 @@ func newStubSlskd(t *testing.T, stub *slskdStub) (*slskd, string) {
s.searchWait = 200 * time.Millisecond s.searchWait = 200 * time.Millisecond
s.transferPoll = time.Millisecond s.transferPoll = time.Millisecond
// Long enough that no existing test trips them by accident; the
// tests about stalls and absences set their own.
s.stallAfter = time.Minute
s.absentGrace = time.Minute
return s, downloads return s, downloads
} }
@@ -565,3 +642,293 @@ func TestSlskdRequiresConfiguration(t *testing.T) {
}) })
} }
} }
// slskdAlbum is a two-file candidate from peer, with the files slskd
// would have written already in place under downloads.
func slskdAlbum(t *testing.T, downloads, peer string, arrived ...string) Candidate {
t.Helper()
folder := filepath.Join(downloads, "Album")
if err := os.MkdirAll(folder, 0o750); err != nil {
t.Fatalf("mkdir: %v", err)
}
for _, name := range arrived {
if err := os.WriteFile(
filepath.Join(folder, name), []byte("audio"), 0o600,
); err != nil {
t.Fatalf("write: %v", err)
}
}
return Candidate{
Files: []CandidateFile{
{Path: `\s\Album\01 A.flac`, Size: 500, IsAudio: true},
{Path: `\s\Album\02 B.flac`, Size: 500, IsAudio: true},
},
TotalSize: 1000,
Payload: map[string]string{"username": peer},
}
}
// cancelledURIs returns what the stub was asked to cancel.
func (s *slskdStub) cancelledURIs() []string {
s.mu.Lock()
defer s.mu.Unlock()
return append([]string(nil), s.cancelled...)
}
// A peer that queues us and never sends a byte is given up on, and the
// queued transfers are cancelled in slskd rather than left to start
// hours later for a request nobody is waiting on.
func TestSlskdGrabGivesUpOnAStalledPeer(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.transfers = [][]slskdTransfer{{
{ID: "t1", Filename: `\s\Album\01 A.flac`, State: "Queued, Remotely"},
{ID: "t2", Filename: `\s\Album\02 B.flac`, State: "Queued, Remotely"},
}}
s, downloads := newStubSlskd(t, stub)
s.stallAfter = 30 * time.Millisecond
_, err := s.Grab(
context.Background(), slskdAlbum(t, downloads, "peer"), t.TempDir(), nil,
)
if !errors.Is(err, ErrSlskdTimeout) {
t.Fatalf("error = %v, want ErrSlskdTimeout", err)
}
got := stub.cancelledURIs()
if len(got) != 2 {
t.Fatalf("cancelled %v, want both queued transfers", got)
}
for _, uri := range got {
if !strings.HasSuffix(uri, "?remove=true") {
t.Errorf("cancel %s does not remove the record", uri)
}
}
}
// A folder that stalls on its last track goes forward with what
// arrived, the same as one whose last track failed; the importer's
// completeness check decides whether that is enough.
func TestSlskdGrabKeepsWhatArrivedBeforeAStall(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.transfers = [][]slskdTransfer{{
{
ID: "t1", Filename: `\s\Album\01 A.flac`,
State: "Completed, Succeeded", BytesTransferred: 500,
},
{ID: "t2", Filename: `\s\Album\02 B.flac`, State: "Queued, Remotely"},
}}
s, downloads := newStubSlskd(t, stub)
s.stallAfter = 30 * time.Millisecond
got, err := s.Grab(
context.Background(),
slskdAlbum(t, downloads, "peer", "01 A.flac"),
t.TempDir(), nil,
)
if err != nil {
t.Fatalf("Grab: %v", err)
}
if len(got.Files) != 1 {
t.Errorf("collected %d files, want the 1 that arrived", len(got.Files))
}
if cancelled := stub.cancelledURIs(); len(cancelled) != 1 ||
!strings.Contains(cancelled[0], "/t2") {
t.Errorf("cancelled %v, want only the stalled t2", cancelled)
}
}
// Progress is what holds the stall timer off. A transfer that keeps
// moving bytes is never abandoned, however long it takes.
func TestSlskdGrabWaitsOnATransferThatIsMoving(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
for b := int64(1); b <= 100; b++ {
stub.transfers = append(stub.transfers, []slskdTransfer{
{ID: "t1", Filename: `\s\Album\01 A.flac`, State: "InProgress", BytesTransferred: b},
{ID: "t2", Filename: `\s\Album\02 B.flac`, State: "Queued, Remotely"},
})
}
stub.transfers = append(stub.transfers, []slskdTransfer{
{
ID: "t1",
Filename: `\s\Album\01 A.flac`,
State: "Completed, Succeeded",
BytesTransferred: 500,
},
{
ID: "t2",
Filename: `\s\Album\02 B.flac`,
State: "Completed, Succeeded",
BytesTransferred: 500,
},
})
s, downloads := newStubSlskd(t, stub)
// A hundred polls take several times the stall window; each one
// moves a byte. The window is kept well above one poll so a
// descheduled test runner does not read as a stall.
s.transferPoll = 5 * time.Millisecond
s.stallAfter = 150 * time.Millisecond
got, err := s.Grab(
context.Background(),
slskdAlbum(t, downloads, "peer", "01 A.flac", "02 B.flac"),
t.TempDir(), nil,
)
if err != nil {
t.Fatalf("Grab: %v", err)
}
if len(got.Files) != 2 {
t.Errorf("collected %d files, want 2", len(got.Files))
}
}
// A file slskd never lists was refused at enqueue and will never reach
// a terminal state. It counts as failed once the grace period is up,
// rather than being waited on until the six-hour ceiling.
func TestSlskdGrabCountsAnUnlistedFileAsFailed(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.transfers = [][]slskdTransfer{{
{
ID: "t1", Filename: `\s\Album\01 A.flac`,
State: "Completed, Succeeded", BytesTransferred: 500,
},
}}
s, downloads := newStubSlskd(t, stub)
s.absentGrace = 20 * time.Millisecond
got, err := s.Grab(
context.Background(),
slskdAlbum(t, downloads, "peer", "01 A.flac"),
t.TempDir(), nil,
)
if err != nil {
t.Fatalf("Grab: %v", err)
}
if len(got.Files) != 1 {
t.Errorf("collected %d files, want 1", len(got.Files))
}
}
// A finished record left by an earlier attempt at the same file is not
// this attempt's answer. Without the snapshot it would fail the grab on
// the first poll, before the new transfer had started.
func TestSlskdGrabIgnoresAnEarlierAttemptsRecord(t *testing.T) {
t.Parallel()
stale := slskdTransfer{
ID: "old", Filename: `\s\Album\01 A.flac`, State: "Completed, Errored",
}
stub := newSlskdStub(t)
stub.before = []slskdTransfer{stale}
stub.transfers = [][]slskdTransfer{
{stale},
{
stale,
{
ID: "new", Filename: `\s\Album\01 A.flac`,
State: "Completed, Succeeded", BytesTransferred: 500,
},
},
}
s, downloads := newStubSlskd(t, stub)
c := slskdAlbum(t, downloads, "peer", "01 A.flac")
c.Files = c.Files[:1]
got, err := s.Grab(context.Background(), c, t.TempDir(), nil)
if err != nil {
t.Fatalf("Grab: %v", err)
}
if len(got.Files) != 1 {
t.Errorf("collected %d files, want 1", len(got.Files))
}
}
// Cancelling the download cancels the transfer in slskd too. The
// cleanup must not inherit the cancelled context, or it is never sent.
func TestSlskdGrabCancelsTransfersWhenTheCallerGivesUp(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.transfers = [][]slskdTransfer{{
{ID: "t1", Filename: `\s\Album\01 A.flac`, State: "InProgress", BytesTransferred: 10},
{ID: "t2", Filename: `\s\Album\02 B.flac`, State: "Queued, Remotely"},
}}
s, downloads := newStubSlskd(t, stub)
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
_, err := s.Grab(ctx, slskdAlbum(t, downloads, "peer"), t.TempDir(), nil)
if !errors.Is(err, ErrSlskdTimeout) {
t.Fatalf("error = %v, want ErrSlskdTimeout", err)
}
if got := stub.cancelledURIs(); len(got) != 2 {
t.Errorf("cancelled %v, want both live transfers", got)
}
}
// Soulseek usernames carry spaces and punctuation; spliced raw into the
// path, a name with a slash addresses a different endpoint entirely.
func TestSlskdEscapesTheUsername(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.transfers = [][]slskdTransfer{{
{
ID: "t1", Filename: `\s\Album\01 A.flac`,
State: "Completed, Succeeded", BytesTransferred: 500,
},
{
ID: "t2", Filename: `\s\Album\02 B.flac`,
State: "Completed, Succeeded", BytesTransferred: 500,
},
}}
s, downloads := newStubSlskd(t, stub)
if _, err := s.Grab(
context.Background(),
slskdAlbum(t, downloads, "dj a/b", "01 A.flac", "02 B.flac"),
t.TempDir(), nil,
); err != nil {
t.Fatalf("Grab: %v", err)
}
stub.mu.Lock()
paths := append([]string(nil), stub.paths...)
stub.mu.Unlock()
for _, p := range paths {
if p != "/api/v0/transfers/downloads/dj%20a%2Fb" {
t.Errorf("transfers call went to %s", p)
}
}
}
+104 -18
View File
@@ -35,6 +35,15 @@ const (
weightArtistFit = 0.12 weightArtistFit = 0.12
) )
// Match sub-weights when the candidate's durations are known. Duration
// takes its weight from title fit, the signal it corroborates: a title
// says which song a file claims to be, a length says whether it is that
// recording — the right edit, the whole file, not the live take.
const (
timedWeightTitleFit = 0.25
timedWeightDurationFit = 0.15
)
// Quality sub-weights. Each set sums to 1.0. // Quality sub-weights. Each set sums to 1.0.
// //
// There are two of them because a stated preference changes what the // There are two of them because a stated preference changes what the
@@ -319,13 +328,13 @@ func Score(dl Download, c Candidate, priority int, prefs AutoDownloadPrefs) Cand
audio := c.AudioFiles() audio := c.AudioFiles()
matched, titleFit := matchFiles(audio, dl.Expected) a := alignFiles(audio, dl.Expected)
// Write the alignment back so the picker can show which file maps // Write the alignment back so the picker can show which file maps
// to which track. // to which track.
c.Files = mergeMatched(c.Files, matched) c.Files = mergeMatched(c.Files, a.files)
c.Match = scoreMatch(dl, c, audio, titleFit) c.Match = scoreMatch(dl, c, audio, a)
c.Quality = scoreQuality( c.Quality = scoreQuality(
c, audio, priority, prefs, dl.runtimeMillis(), c, audio, priority, prefs, dl.runtimeMillis(),
) )
@@ -340,14 +349,21 @@ func scoreMatch(
dl Download, dl Download,
c Candidate, c Candidate,
audio []CandidateFile, audio []CandidateFile,
titleFit float64, a alignment,
) MatchScore { ) MatchScore {
m := MatchScore{ m := MatchScore{
Anchored: dl.Anchored(), Anchored: dl.Anchored(),
TitleFit: titleFit, TitleFit: a.titleFit,
DurationFit: a.durationFit,
// Durations count once at least half the aligned pairs state
// one; a single timed pair would be a coin toss carrying 15%.
DurationKnown: a.timedPairs > 0 && a.timedPairs*2 >= a.aligned,
} }
m.Completeness = completeness(len(audio), len(dl.Expected)) m.Completeness = completeness(
alignedCount(c.Files), len(audio), len(dl.Expected),
)
// The candidate's own title, and the folder its files sit in, are // The candidate's own title, and the folder its files sit in, are
// two independent guesses at the album name. Take the better one: // two independent guesses at the album name. Take the better one:
@@ -367,9 +383,16 @@ func scoreMatch(
// With no expected tracklist there is no title signal at all, so // With no expected tracklist there is no title signal at all, so
// redistribute its weight onto the album/artist evidence rather // redistribute its weight onto the album/artist evidence rather
// than scoring every free-text result as half-wrong. // than scoring every free-text result as half-wrong.
if len(dl.Expected) == 0 { switch {
case len(dl.Expected) == 0:
m.Overall = 0.55*m.AlbumFit + 0.45*m.ArtistFit m.Overall = 0.55*m.AlbumFit + 0.45*m.ArtistFit
} else { case m.DurationKnown:
m.Overall = timedWeightTitleFit*m.TitleFit +
timedWeightDurationFit*m.DurationFit +
weightCompleteness*m.Completeness +
weightAlbumFit*m.AlbumFit +
weightArtistFit*m.ArtistFit
default:
m.Overall = weightTitleFit*m.TitleFit + m.Overall = weightTitleFit*m.TitleFit +
weightCompleteness*m.Completeness + weightCompleteness*m.Completeness +
weightAlbumFit*m.AlbumFit + weightAlbumFit*m.AlbumFit +
@@ -415,30 +438,54 @@ func artistFit(want string, c Candidate) float64 {
return best return best
} }
// completeness scores audio file count against the expected track // completeness scores how much of the expected tracklist a candidate
// count. Extra files are penalized far more gently than missing ones: // covers. Extra files are penalized far more gently than missing ones:
// a folder with bonus tracks or a stray intro is still the album, while // a folder with bonus tracks or a stray intro is still the album, while
// a folder missing half the tracks is not. // a folder missing half the tracks is not.
func completeness(got, want int) float64 { //
// **Coverage is counted in aligned tracks, not in files.** It used to
// be the audio file count, so any ten files scored full marks against
// a ten-track album whether or not they were its tracks — and since
// title fit is the mean over the files that *did* align, a folder where
// three titles matched read as a near-perfect candidate on both counts.
// `aligned` is how many files matchFiles assigned to an expected track;
// `audio` still sets the penalty for extras, because a folder of thirty
// files holding the ten wanted is a worse copy than one holding ten.
func completeness(aligned, audio, want int) float64 {
if want == 0 { if want == 0 {
if got > 0 { if audio > 0 {
return 0.5 return 0.5
} }
return 0 return 0
} }
if got == 0 { if aligned == 0 {
return 0 return 0
} }
if got >= want { cover := float64(min(aligned, want)) / float64(want)
extra := float64(got-want) / float64(want)
return math.Max(0.75, 1.0-0.25*extra) if audio > want {
extra := float64(audio-want) / float64(want)
cover *= math.Max(0.75, 1.0-0.25*extra)
} }
return float64(got) / float64(want) return cover
}
// alignedCount is how many audio files were assigned to an expected
// track.
func alignedCount(files []CandidateFile) int {
n := 0
for _, f := range files {
if f.IsAudio && f.MatchedTo != 0 {
n++
}
}
return n
} }
// scoreQuality answers whether this is a good copy. // scoreQuality answers whether this is a good copy.
@@ -735,6 +782,45 @@ func AutoPickVeto(
return "" return ""
} }
// autoAcceptable reports whether auto-pick may take this one candidate
// without asking: the request is anchored to a tracklist, and the
// candidate is inside the user's guardrails and clears the match and
// quality bars. It is AutoPickVeto's test applied to a single
// candidate, which is what falling back to a second choice needs.
func autoAcceptable(dl Download, c Candidate, prefs AutoDownloadPrefs) bool {
return dl.Anchored() &&
len(dl.Expected) > 0 &&
prefs.eligible(c, dl.runtimeMillis()) &&
c.Match.Overall >= minMatch &&
c.Quality.Overall >= minQuality
}
// autoPick returns the candidate auto-pick takes: the best-ranked one
// it may take at all.
//
// That is not `ranked[0]`. AutoPickVeto judges the best candidate
// *inside* the guardrails, so when the overall best is outside them —
// over the size ceiling, say — the veto passes on the strength of the
// second, and grabbing the first would download exactly the copy the
// user said not to take unattended.
func autoPick(
dl Download,
ranked []Candidate,
prefs AutoDownloadPrefs,
) (Candidate, bool) {
if AutoPickVeto(dl, ranked, prefs) != "" {
return Candidate{}, false
}
for _, c := range ranked {
if autoAcceptable(dl, c, prefs) {
return c, true
}
}
return Candidate{}, false
}
// mergeMatched copies MatchedTo assignments from the audio-only slice // mergeMatched copies MatchedTo assignments from the audio-only slice
// back onto the full file list. // back onto the full file list.
func mergeMatched(all, matched []CandidateFile) []CandidateFile { func mergeMatched(all, matched []CandidateFile) []CandidateFile {
+14 -10
View File
@@ -282,28 +282,32 @@ func TestCompleteness(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
got int aligned int
audio int
want int want int
minScore float64 minScore float64
maxScore float64 maxScore float64
}{ }{
{"exact", 10, 10, 1.0, 1.0}, {"exact", 10, 10, 10, 1.0, 1.0},
{"half missing", 5, 10, 0.49, 0.51}, {"half missing", 5, 5, 10, 0.49, 0.51},
{"one bonus track", 11, 10, 0.95, 1.0}, {"one bonus track", 10, 11, 10, 0.95, 1.0},
{"double", 20, 10, 0.74, 0.76}, {"double", 10, 20, 10, 0.74, 0.76},
{"nothing", 0, 10, 0, 0}, {"nothing", 0, 0, 10, 0, 0},
{"no expectation", 5, 0, 0.5, 0.5}, {"no expectation", 0, 5, 0, 0.5, 0.5},
// Ten files are not ten tracks: three that align are three.
{"right count, wrong tracks", 3, 10, 10, 0.29, 0.31},
{"files that align to nothing", 0, 10, 10, 0, 0},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
got := completeness(tt.got, tt.want) got := completeness(tt.aligned, tt.audio, tt.want)
if got < tt.minScore || got > tt.maxScore { if got < tt.minScore || got > tt.maxScore {
t.Errorf( t.Errorf(
"completeness(%d, %d) = %f, want in [%f, %f]", "completeness(%d, %d, %d) = %f, want in [%f, %f]",
tt.got, tt.want, got, tt.minScore, tt.maxScore, tt.aligned, tt.audio, tt.want, got, tt.minScore, tt.maxScore,
) )
} }
}) })
+238
View File
@@ -0,0 +1,238 @@
package download
import (
"context"
"strings"
"testing"
)
// What Soulseek is asked, how, and what is kept from the answer (#271).
func TestSlskdQueries(t *testing.T) {
t.Parallel()
cases := []struct {
name string
dl Download
want []string
}{
{
name: "a plain request is searched once",
dl: Download{Artist: "Radiohead", Album: "OK Computer"},
want: []string{"Radiohead OK Computer"},
},
{
name: "an edition qualifier gets a second query without it",
dl: Download{Artist: "Radiohead", Album: "OK Computer (Collector's Edition)"},
want: []string{
"Radiohead OK Computer (Collector's Edition)",
"Radiohead OK Computer",
},
},
{
name: "a trailing remaster note",
dl: Download{Artist: "Pink Floyd", Album: "Animals - 2018 Remaster"},
want: []string{
"Pink Floyd Animals - 2018 Remaster",
"Pink Floyd Animals",
},
},
{
name: "a leading dash would be an exclusion",
dl: Download{Artist: "Mocky", Album: "-ism"},
want: []string{"Mocky -ism", "Mocky ism"},
},
{
name: "a compilation is not searched by its placeholder artist",
dl: Download{Artist: "Various Artists", Album: "Pulp Fiction"},
want: []string{"Various Artists Pulp Fiction", "Pulp Fiction"},
},
{
name: "what the user typed is searched as written",
dl: Download{Query: "ok computer (deluxe)", Album: "OK Computer (Deluxe)"},
want: []string{"ok computer (deluxe)"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
got := slskdQueries(tc.dl)
if strings.Join(got, "|") != strings.Join(tc.want, "|") {
t.Errorf("slskdQueries = %q, want %q", got, tc.want)
}
})
}
}
// Both queries run, the options are stated rather than left to the
// daemon's defaults, and a folder both queries found is one candidate.
func TestSlskdSearchRunsBothQueriesAndMerges(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.responses = []slskdResponse{{
Username: "peer",
Files: []slskdFile{
{Filename: `\m\Radiohead - OK Computer\01 Airbag.flac`, Size: 1, Length: 284},
{Filename: `\m\Radiohead - OK Computer\02 Paranoid Android.flac`, Size: 1, Length: 383},
},
}}
s, _ := newStubSlskd(t, stub)
got, err := s.Search(context.Background(), Download{
ReleaseMBID: "rel", Artist: "Radiohead", Album: "OK Computer (Deluxe Edition)",
})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(got) != 1 {
t.Fatalf("got %d candidates, want the one folder once", len(got))
}
if got[0].Files[0].LengthMillis != 284_000 {
t.Errorf("length = %d ms, want 284000 from slskd's seconds", got[0].Files[0].LengthMillis)
}
stub.mu.Lock()
searches := append([]map[string]any(nil), stub.searches...)
gets := append([]string(nil), stub.searchGets...)
stub.mu.Unlock()
if len(searches) != 2 {
t.Fatalf("ran %d searches, want 2", len(searches))
}
for _, body := range searches {
for _, key := range []string{
"searchTimeout", "responseLimit", "fileLimit",
"minimumResponseFileCount", "maximumPeerQueueLength",
} {
if _, ok := body[key]; !ok {
t.Errorf("search %q does not state %s", body["searchText"], key)
}
}
}
// The responses are fetched once at the end, not with every poll.
for _, uri := range gets {
if strings.Contains(uri, "includeResponses") {
t.Errorf("poll %s asked for every response", uri)
}
}
}
// A daemon without the responses endpoint still returns results.
func TestSlskdSearchFallsBackForAnOlderDaemon(t *testing.T) {
t.Parallel()
stub := newSlskdStub(t)
stub.noResponsesEndpoint = true
stub.responses = []slskdResponse{{
Username: "peer",
Files: []slskdFile{
{Filename: `\m\Album\01 A.flac`, Size: 1},
{Filename: `\m\Album\02 B.flac`, Size: 1},
},
}}
s, _ := newStubSlskd(t, stub)
got, err := s.Search(context.Background(), Download{Query: "album"})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(got) != 1 {
t.Errorf("got %d candidates, want 1 through the fallback", len(got))
}
}
func TestDurationAgreement(t *testing.T) {
t.Parallel()
cases := []struct {
got, want int64
score float64
}{
{300_000, 300_000, 1},
{301_500, 300_000, 1}, // a second of silence
{300_000, 316_500, 0.5},
{300_000, 345_000, 0}, // a different edit
}
for _, tc := range cases {
if got := durationAgreement(tc.got, tc.want); got < tc.score-0.01 || got > tc.score+0.01 {
t.Errorf("durationAgreement(%d, %d) = %f, want %f", tc.got, tc.want, got, tc.score)
}
}
}
// Two folders with the same track names are told apart by their
// lengths: one is the album, the other a live record of the same songs.
func TestDurationsSeparateTheRightRecording(t *testing.T) {
t.Parallel()
dl := okComputer()
timed := func(id string, lengths ...int64) Candidate {
c := candidateFor(id, allTitles(), ".flac", 30_000_000)
for i := range c.Files {
c.Files[i].LengthMillis = lengths[i]
}
return c
}
studio := timed("studio", trackMillis, trackMillis+1_000, trackMillis, trackMillis-500)
live := timed(
"live",
trackMillis+60_000,
trackMillis+75_000,
trackMillis+50_000,
trackMillis+90_000,
)
ranked := Rank(dl, []Candidate{live, studio}, nil, AutoDownloadPrefs{})
if ranked[0].ID != "studio" {
t.Fatalf("winner = %s, want the recording whose lengths match", ranked[0].ID)
}
if !ranked[0].Match.DurationKnown || ranked[0].Match.DurationFit < 0.99 {
t.Errorf(
"studio duration fit = %f known=%v",
ranked[0].Match.DurationFit,
ranked[0].Match.DurationKnown,
)
}
if ranked[1].Match.DurationFit != 0 {
t.Errorf("live duration fit = %f, want 0", ranked[1].Match.DurationFit)
}
}
// Without lengths the score is exactly what it was before durations
// were read, so a provider that reports none is not penalised.
func TestUnknownDurationsLeaveTheScoreAlone(t *testing.T) {
t.Parallel()
dl := okComputer()
c := Score(dl, candidateFor("c", allTitles(), ".flac", 30_000_000), 50, AutoDownloadPrefs{})
if c.Match.DurationKnown {
t.Fatal("no file states a length, yet durations are known")
}
want := weightTitleFit*c.Match.TitleFit +
weightCompleteness*c.Match.Completeness +
weightAlbumFit*c.Match.AlbumFit +
weightArtistFit*c.Match.ArtistFit
if c.Match.Overall != want {
t.Errorf("match = %f, want the untimed formula's %f", c.Match.Overall, want)
}
}
+19 -7
View File
@@ -232,12 +232,17 @@ type Candidate struct {
// results give paths and sizes but no tags, so Format and duration are // results give paths and sizes but no tags, so Format and duration are
// inferred from the path and size where possible. // inferred from the path and size where possible.
type CandidateFile struct { type CandidateFile struct {
Path string `json:"path"` Path string `json:"path"`
Size int64 `json:"size"` Size int64 `json:"size"`
Format Format `json:"format"` Format Format `json:"format"`
Bitrate int `json:"bitrate,omitempty"` // kbps, 0 when unknown Bitrate int `json:"bitrate,omitempty"` // kbps, 0 when unknown
IsAudio bool `json:"isAudio"` IsAudio bool `json:"isAudio"`
MatchedTo int `json:"matchedTo,omitempty"` // expected track position
// LengthMillis is the file's duration as the source reports it, or
// 0 when it does not. Soulseek reports it for most audio files.
LengthMillis int64 `json:"lengthMillis,omitempty"`
MatchedTo int `json:"matchedTo,omitempty"` // expected track position
} }
// Format is a normalized audio container/codec name. // Format is a normalized audio container/codec name.
@@ -286,7 +291,14 @@ type MatchScore struct {
TitleFit float64 `json:"titleFit"` // filenames vs expected titles TitleFit float64 `json:"titleFit"` // filenames vs expected titles
ArtistFit float64 `json:"artistFit"` // path/origin vs expected artist ArtistFit float64 `json:"artistFit"` // path/origin vs expected artist
AlbumFit float64 `json:"albumFit"` // folder name vs album title AlbumFit float64 `json:"albumFit"` // folder name vs album title
Completeness float64 `json:"completeness"` // audio files vs expected count Completeness float64 `json:"completeness"` // aligned tracks vs expected count
// DurationFit is how well the aligned files' lengths agree with the
// expected tracks', and DurationKnown whether enough of them stated
// a length for that to count. When it does not, the score is the
// four text signals alone, exactly as before durations were read.
DurationFit float64 `json:"durationFit"`
DurationKnown bool `json:"durationKnown"`
// Anchored records whether an MBID drove this score. Unanchored // Anchored records whether an MBID drove this score. Unanchored
// matches are capped, because there is nothing to be right about. // matches are capped, because there is nothing to be right about.
+88 -11
View File
@@ -1,8 +1,10 @@
package explore package explore
import ( import (
"bytes"
"context" "context"
"database/sql" "database/sql"
"database/sql/driver"
"errors" "errors"
"fmt" "fmt"
"os" "os"
@@ -283,6 +285,25 @@ func (si *SearchIndex) importCoreArtifact(ctx context.Context, path string) erro
} }
merged, mergeErr := si.mergeArtifactRows(ctx, info.rows) merged, mergeErr := si.mergeArtifactRows(ctx, info.rows)
// Every row the artifact declares has to land. The walk partitions
// the artifact's key space, so a total short of info.rows does not
// mean the artifact was smaller than it said -- it means a predicate
// filtered rows out, and the catalog is quietly partial. Equality
// rather than a lower bound because RowsAffected counts an upsert
// that changes nothing, and a row already merged locally is counted
// again here.
//
// One reachable case, so this is not merely a tripwire: a row whose
// mbid is empty is excluded by `mbid > ?` in both encodings, and an
// artifact carrying one would otherwise import as complete.
if mergeErr == nil && merged != info.rows {
mergeErr = fmt.Errorf(
"%w: merged %d of %d rows — a row the artifact holds was not selected",
ErrArtifactUnusable, merged, info.rows,
)
}
if mergeErr == nil { if mergeErr == nil {
si.mergeArtifactCredits(ctx) si.mergeArtifactCredits(ctx)
} }
@@ -348,8 +369,12 @@ func (si *SearchIndex) analyzeIndex() {
// is an index range scan and a cancelled import leaves committed work // is an index range scan and a cancelled import leaves committed work
// behind rather than rolling it all back. // behind rather than rolling it all back.
func (si *SearchIndex) mergeArtifactRows(ctx context.Context, total int) (int, error) { func (si *SearchIndex) mergeArtifactRows(ctx context.Context, total int) (int, error) {
// Asked once, because it is a property of the file and it decides
// how the walk's own comparisons are typed. See artifactKey.
storesText := si.artifactStoresText()
selectColumns := artifactSelectColumns( selectColumns := artifactSelectColumns(
si.artifactStoresText(), si.artifactHasTotals(), storesText, si.artifactHasTotals(),
) )
insertSQL := ` insertSQL := `
@@ -367,7 +392,7 @@ func (si *SearchIndex) mergeArtifactRows(ctx context.Context, total int) (int, e
WHERE mbid > ? AND mbid <= ?` + upsertIndexConflictSQL WHERE mbid > ? AND mbid <= ?` + upsertIndexConflictSQL
var ( var (
cursor string cursor artifactKey
merged int merged int
) )
@@ -376,17 +401,30 @@ func (si *SearchIndex) mergeArtifactRows(ctx context.Context, total int) (int, e
return merged, err return merged, err
} }
upper, hasUpper, err := si.artifactBatchBound(cursor) upper, hasUpper, err := si.artifactBatchBound(storesText, cursor)
if err != nil { if err != nil {
return merged, err return merged, err
} }
if hasUpper && bytes.Compare(upper, cursor) <= 0 {
// The predicate matched the cursor itself, so the walk can
// never advance. SQLite says nothing when a comparison is
// made between types it will not coerce - the query simply
// answers wrongly - so a mismatch here would otherwise spin
// forever behind an unmoving progress bar. Fail instead.
return merged, fmt.Errorf(
"%w: artifact walk did not advance past %x",
ErrArtifactUnusable, []byte(cursor),
)
}
var res sql.Result var res sql.Result
if hasUpper { if hasUpper {
res, err = si.db.ExecContext(insertRangeSQL, cursor, upper) res, err = si.db.ExecContext(insertRangeSQL,
cursor.bind(storesText), upper.bind(storesText))
} else { } else {
res, err = si.db.ExecContext(insertSQL, cursor) res, err = si.db.ExecContext(insertSQL, cursor.bind(storesText))
} }
if err != nil { if err != nil {
@@ -413,26 +451,65 @@ func (si *SearchIndex) mergeArtifactRows(ctx context.Context, total int) (int, e
} }
} }
// artifactKey is one MBID as the attached artifact stores it: 16 raw
// bytes in a compact artifact, the dashed 36-character form in one
// published before that storage change.
//
// It is a type with a bind method rather than a string because the
// comparison it feeds is typed, and the wrong type is silent. SQLite
// does not coerce between TEXT and BLOB and orders every blob after
// every text value, so a cursor bound as text against a byte column
// makes `mbid > ?` true of the whole table - the walk rediscovers the
// same batch bound forever, and `mbid <= ?` false of the whole table,
// so no batch merges at all. Nothing errors; the import simply never
// finishes. bind is the one place that knows which form the column is
// in, decided by artifactStoresText, which asks the artifact rather than
// trusting a version number.
type artifactKey []byte
// bind renders the key as a statement argument in the artifact's own
// encoding.
func (k artifactKey) bind(storesText bool) driver.Value {
if storesText {
return string(k)
}
// Never nil. database/sql converts a nil []byte to SQL NULL, and
// `mbid > NULL` is NULL for every row - so an unset cursor would
// agree with nothing and import nothing, which is the same silently
// empty merge this type exists to prevent, one type over.
if k == nil {
return []byte{}
}
return []byte(k)
}
// artifactBatchBound returns the MBID that ends the next batch, and // artifactBatchBound returns the MBID that ends the next batch, and
// whether one exists — no bound means the remainder is the last batch. // whether one exists — no bound means the remainder is the last batch.
func (si *SearchIndex) artifactBatchBound(cursor string) (string, bool, error) { //
var bound string // The bound is read out of the artifact and handed back as an
// artifactKey, because it becomes the next comparison the walk makes.
func (si *SearchIndex) artifactBatchBound(
storesText bool, cursor artifactKey,
) (artifactKey, bool, error) {
var bound []byte
err := si.db.QueryRowWriter( err := si.db.QueryRowWriter(
`SELECT mbid FROM core.explore_index `SELECT mbid FROM core.explore_index
WHERE mbid > ? ORDER BY mbid LIMIT 1 OFFSET ?`, WHERE mbid > ? ORDER BY mbid LIMIT 1 OFFSET ?`,
cursor, artifactMergeBatch-1, cursor.bind(storesText), artifactMergeBatch-1,
).Scan(&bound) ).Scan(&bound)
if errors.Is(err, sql.ErrNoRows) { if errors.Is(err, sql.ErrNoRows) {
return "", false, nil return nil, false, nil
} }
if err != nil { if err != nil {
return "", false, fmt.Errorf("%w: batch bound: %w", ErrArtifactUnusable, err) return nil, false, fmt.Errorf("%w: batch bound: %w", ErrArtifactUnusable, err)
} }
return bound, true, nil return artifactKey(bound), true, nil
} }
// stampArtifactMeta records what the merge established: the catalog half // stampArtifactMeta records what the merge established: the catalog half
+250 -62
View File
@@ -1,6 +1,7 @@
package explore package explore
import ( import (
"bytes"
"context" "context"
"database/sql" "database/sql"
"encoding/hex" "encoding/hex"
@@ -71,13 +72,7 @@ func writeTestArtifact(
} }
} }
for k, v := range meta { stampArtifactMeta(t, db, meta)
if _, err := db.Exec(
`INSERT INTO artifact_meta (key, value) VALUES (?, ?)`, k, v,
); err != nil {
t.Fatalf("stamp artifact meta: %v", err)
}
}
for _, r := range rows { for _, r := range rows {
if _, err := db.Exec(` if _, err := db.Exec(`
@@ -93,6 +88,101 @@ func writeTestArtifact(
return path return path
} }
// compactArtifactSchema is the artifact cmd/indexexport publishes: the
// catalog's ids as 16 raw bytes, its entity types as codes, and the
// per-release-group total_tracks the exporter added after the first
// artifact was shipped.
//
// It matters that a fixture carries this encoding and not the older
// text one, because SQLite does not coerce between TEXT and BLOB and
// every comparison the importer makes against an mbid is therefore
// encoding-sensitive. writeTestArtifact above is the *other* fixture:
// it still writes the text form, which is what the first published
// artifact carries and what the importer must keep reading.
var compactArtifactSchema = []string{
`CREATE TABLE explore_index (
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid BLOB NOT NULL,
aliases TEXT NOT NULL DEFAULT '',
popularity INTEGER NOT NULL DEFAULT 0,
listener_count INTEGER NOT NULL DEFAULT 0,
duration INTEGER NOT NULL DEFAULT 0,
caa_release_mbid BLOB NOT NULL DEFAULT x'',
release_name TEXT NOT NULL DEFAULT '',
primary_type TEXT NOT NULL DEFAULT '',
secondary_types TEXT NOT NULL DEFAULT '',
release_date TEXT NOT NULL DEFAULT '',
total_tracks INTEGER NOT NULL DEFAULT 0,
artist_type TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
disambiguation TEXT NOT NULL DEFAULT '',
sort_name TEXT NOT NULL DEFAULT '',
discog_fetched INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (mbid)
) WITHOUT ROWID`,
`CREATE TABLE artifact_meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
)`,
}
// writeCompactTestArtifact builds the artifact the exporter publishes
// today, in its own encoding, so the importer is exercised against what
// a client actually downloads rather than against what it was written
// for.
func writeCompactTestArtifact(
t *testing.T, meta map[string]string, rows []artifactRow,
) string {
t.Helper()
path := filepath.Join(t.TempDir(), "core-index.db")
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatalf("open artifact: %v", err)
}
defer func() { _ = db.Close() }()
for _, stmt := range compactArtifactSchema {
if _, err := db.Exec(stmt); err != nil {
t.Fatalf("create artifact schema: %v", err)
}
}
stampArtifactMeta(t, db, meta)
for _, r := range rows {
if _, err := db.Exec(`
INSERT INTO explore_index
(entity_type, mbid, title, artist_name, artist_mbid, popularity)
VALUES (?, ?, ?, ?, ?, ?)`,
entityCode(r.entityType), mbidBytes(r.mbid), r.title,
r.artistName, mbidBytes(r.artistMBID), r.popularity,
); err != nil {
t.Fatalf("insert artifact row: %v", err)
}
}
return path
}
// stampArtifactMeta writes the artifact_meta rows a fixture declares.
func stampArtifactMeta(t *testing.T, db *sql.DB, meta map[string]string) {
t.Helper()
for k, v := range meta {
if _, err := db.Exec(
`INSERT INTO artifact_meta (key, value) VALUES (?, ?)`, k, v,
); err != nil {
t.Fatalf("stamp artifact meta: %v", err)
}
}
}
// validMeta is the artifact_meta a well-formed artifact carries. // validMeta is the artifact_meta a well-formed artifact carries.
func validMeta() map[string]string { func validMeta() map[string]string {
return map[string]string{ return map[string]string{
@@ -270,6 +360,71 @@ func TestImportCoreArtifactBatchWalkCoversAllRows(t *testing.T) {
} }
} }
// TestImportCoreArtifactBatchWalkCoversAllRowsCompact is the batch walk
// on the encoding the exporter actually publishes.
//
// The walk positions itself by comparing the artifact's own mbid column
// against the last id it reached, and that column holds 16 raw bytes.
// SQLite does not coerce between TEXT and BLOB, and a blob sorts after
// every text value, so a cursor bound as text is a predicate that either
// matches every row or none: `mbid > ?` with an empty text key is true
// of the whole table, so
// the 100th row is always the 100th row and the bound never advances,
// while `mbid <= <text>` is false of the whole table, so no batch ever
// merges. The result is not a wrong import but an unbounded loop that
// merges nothing and never fails.
//
// Both encodings are covered on purpose. The walk was only ever tested
// against the text fixture above, which is why it shipped broken on the
// one the clients download.
func TestImportCoreArtifactBatchWalkCoversAllRowsCompact(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
original := artifactMergeBatch
artifactMergeBatch = 100
t.Cleanup(func() { artifactMergeBatch = original })
const total = 337
rows := make([]artifactRow, 0, total)
for i := range total {
rows = append(rows, artifactRow{
entityType: EntityRecording,
mbid: syntheticMBID(i),
title: "Song",
artistName: "Artist",
artistMBID: artA,
popularity: i,
})
}
path := writeCompactTestArtifact(t, validMeta(), rows)
if err := si.importCoreArtifact(context.Background(), path); err != nil {
t.Fatalf("importCoreArtifact: %v", err)
}
var got, top int
if err := db.QueryRowWriter(
"SELECT COUNT(*), MAX(popularity) FROM explore_index",
).Scan(&got, &top); err != nil {
t.Fatalf("count rows: %v", err)
}
if got != total {
t.Errorf("merged %d rows, want %d", got, total)
}
// A count alone would pass if the walk re-merged the same first
// batch forever, so the far end of the artifact is checked too.
if top != total-1 {
t.Errorf("highest popularity = %d, want %d", top, total-1)
}
}
func TestImportCoreArtifactRejectsBadArtifacts(t *testing.T) { func TestImportCoreArtifactRejectsBadArtifacts(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -454,61 +609,9 @@ func TestArtifactColumnsMatchExporter(t *testing.T) {
// the importer decides by asking the artifact, not by trusting a // the importer decides by asking the artifact, not by trusting a
// version number, and both must land identically. // version number, and both must land identically.
func TestImportCoreArtifactAcceptsBothEncodings(t *testing.T) { func TestImportCoreArtifactAcceptsBothEncodings(t *testing.T) {
compact := filepath.Join(t.TempDir(), "core-index.db") compact := writeCompactTestArtifact(t, validMeta(), []artifactRow{
{EntityArtist, artA, "Artist A", "Artist A", artA, 5000},
db, err := sql.Open("sqlite", "file:"+compact) })
if err != nil {
t.Fatalf("open artifact: %v", err)
}
if _, err := db.Exec(`CREATE TABLE explore_index (
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid BLOB NOT NULL,
aliases TEXT NOT NULL DEFAULT '',
popularity INTEGER NOT NULL DEFAULT 0,
listener_count INTEGER NOT NULL DEFAULT 0,
duration INTEGER NOT NULL DEFAULT 0,
caa_release_mbid BLOB NOT NULL DEFAULT x'',
release_name TEXT NOT NULL DEFAULT '',
primary_type TEXT NOT NULL DEFAULT '',
secondary_types TEXT NOT NULL DEFAULT '',
release_date TEXT NOT NULL DEFAULT '',
artist_type TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
disambiguation TEXT NOT NULL DEFAULT '',
sort_name TEXT NOT NULL DEFAULT '',
discog_fetched INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (mbid)
)`); err != nil {
t.Fatalf("create artifact table: %v", err)
}
if _, err := db.Exec(
`CREATE TABLE artifact_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL)`,
); err != nil {
t.Fatalf("create artifact meta: %v", err)
}
for k, v := range validMeta() {
if _, err := db.Exec(
"INSERT INTO artifact_meta (key, value) VALUES (?, ?)", k, v,
); err != nil {
t.Fatalf("write artifact meta: %v", err)
}
}
if _, err := db.Exec(`
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid, popularity)
VALUES (1, ?, 'Artist A', 'Artist A', ?, 5000)`,
mbidBytes(artA), mbidBytes(artA),
); err != nil {
t.Fatalf("write artifact row: %v", err)
}
_ = db.Close()
live := database.NewTestDB(t) live := database.NewTestDB(t)
si := NewSearchIndex(live, nil, nil, testLogger()) si := NewSearchIndex(live, nil, nil, testLogger())
@@ -748,3 +851,88 @@ func TestImportCoreArtifactWithoutCredits(t *testing.T) {
t.Errorf("credit refs = %d, want 0", refs) t.Errorf("credit refs = %d, want 0", refs)
} }
} }
// TestImportCoreArtifactRefusesAMergeThatLosesRows is the count guard's
// positive case.
//
// The walk's predicates partition the artifact's key space, so a merge
// that lands fewer rows than the artifact declares means a predicate
// dropped some — and the failure is a catalog that looks populated and
// is missing things nobody can name. An empty mbid is the reachable
// way to get there: `mbid > ?` is false of it in both encodings, so it
// is never selected, and nothing else in the import would notice.
func TestImportCoreArtifactRefusesAMergeThatLosesRows(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
path := writeCompactTestArtifact(t, validMeta(), []artifactRow{
{EntityArtist, artA, "Artist A", "Artist A", artA, 5000},
{EntityArtist, "", "Nameless", "Artist A", artA, 4000},
})
err := si.importCoreArtifact(context.Background(), path)
if err == nil {
t.Fatal("a merge that lost a row was reported as a complete import")
}
if !strings.Contains(err.Error(), "merged 1 of 2 rows") {
t.Errorf("error = %v, want it to name the shortfall", err)
}
// And the same rule as every other rejection: a failed merge must not
// leave the index claiming it has a catalog, or the real build would
// never run again.
if si.hasMeta(dumpImportDoneKey) {
t.Error("a failed import still stamped dump_import_done")
}
}
// TestArtifactKeyBindsInTheArtifactsOwnEncoding pins the one place the
// batch walk's comparison type is decided.
//
// Every wrong answer is silent, which is why it is worth pinning all
// four. SQLite does not coerce TEXT to BLOB and orders every blob after
// every text value, so a text key against a byte column makes
// `mbid > ?` true of the whole artifact - the cursor never advances and
// the walk spins forever without merging a row - while a byte key
// against a text column makes it false of the whole artifact, so every
// batch merges nothing and the import "succeeds" empty. An unset cursor
// is the same fault once more: database/sql converts a nil []byte to
// SQL NULL, and `mbid > NULL` matches no row at all.
func TestArtifactKeyBindsInTheArtifactsOwnEncoding(t *testing.T) {
raw := mbidBytes(artA)
for _, tt := range []struct {
name string
key artifactKey
want []byte
}{
{"unset", nil, []byte{}},
{"set", artifactKey(raw), raw},
} {
t.Run("bytes/"+tt.name, func(t *testing.T) {
got, ok := tt.key.bind(false).([]byte)
if !ok {
t.Fatalf("bind(false) = %T, want []byte", tt.key.bind(false))
}
if got == nil {
t.Fatal("bound to SQL NULL, which matches no row")
}
if !bytes.Equal(got, tt.want) {
t.Errorf("bind(false) = %x, want %x", got, tt.want)
}
})
}
// The dashed form is what an artifact published before the storage
// change carries, and it has to compare as text against text.
if got := artifactKey(nil).bind(true); got != "" {
t.Errorf("bind(true) on an unset cursor = %#v, want an empty string", got)
}
if got := artifactKey(artA).bind(true); got != artA {
t.Errorf("bind(true) = %#v, want %q", got, artA)
}
}
+49
View File
@@ -13,6 +13,7 @@ import (
"net/http" "net/http"
"slices" "slices"
"strings" "strings"
"sync/atomic"
"time" "time"
) )
@@ -25,6 +26,19 @@ const (
// responds with a non-2xx status code. // responds with a non-2xx status code.
var ErrListenBrainzHTTP = errors.New("listenbrainz HTTP error") var ErrListenBrainzHTTP = errors.New("listenbrainz HTTP error")
// ErrListenBrainzUnauthorized is a 401: the endpoint wants a token, and
// no retry will change that.
//
// It is separate from ErrListenBrainzHTTP because it is the one
// failure that is about *this client* rather than about the thing being
// asked for — which is what makes it the one worth latching. The
// popularity endpoints answered 401 to every request on 2026-10-05, so
// a discography backfill spent one rate-limited request per artist to
// be told the same thing: 399 of them in a minute of a real library's
// backfill, each one a log line and a wasted slot in the shared
// limiter.
var ErrListenBrainzUnauthorized = errors.New("listenbrainz requires a token")
// ListenBrainzClient is a thin HTTP client for the ListenBrainz // ListenBrainzClient is a thin HTTP client for the ListenBrainz
// popularity and labs APIs. All requests are rate-limited via the // popularity and labs APIs. All requests are rate-limited via the
// shared RateLimiter and cached via the shared Cache. // shared RateLimiter and cached via the shared Cache.
@@ -39,6 +53,13 @@ type ListenBrainzClient struct {
// SetBaseURL shape — so a test that points one client at an // SetBaseURL shape — so a test that points one client at an
// httptest server does not stop being parallel-safe. // httptest server does not stop being parallel-safe.
baseURL string baseURL string
// refused latches the first 401. A token is a property of the
// installation, not of the artist being asked about, so the answer
// is the same for every later request and asking again is pure
// cost. Per client rather than global so a test can have one that
// is refused and one that is not.
refused atomic.Bool
} }
// NewListenBrainzClient creates a ListenBrainz API client. // NewListenBrainzClient creates a ListenBrainz API client.
@@ -56,6 +77,14 @@ func NewListenBrainzClient(
} }
} }
// Unauthorized reports whether this client has been refused with a 401
// during its life. A caller that is about to do a long pass of
// requests should ask before starting it: the answer will not change
// mid-pass.
func (c *ListenBrainzClient) Unauthorized() bool {
return c.refused.Load()
}
// SetBaseURL redirects this client at another host. Tests only. // SetBaseURL redirects this client at another host. Tests only.
func (c *ListenBrainzClient) SetBaseURL(url string) { func (c *ListenBrainzClient) SetBaseURL(url string) {
c.baseURL = strings.TrimSuffix(url, "/") c.baseURL = strings.TrimSuffix(url, "/")
@@ -489,6 +518,11 @@ func (c *ListenBrainzClient) doPost(
func (c *ListenBrainzClient) doRequest( func (c *ListenBrainzClient) doRequest(
ctx context.Context, method string, url string, body []byte, ctx context.Context, method string, url string, body []byte,
) ([]byte, error) { ) ([]byte, error) {
// Asked and answered, for the rest of this client's life.
if c.refused.Load() {
return nil, fmt.Errorf("%w: %s", ErrListenBrainzUnauthorized, url)
}
c.logger.Debug("listenbrainz rate limiter wait", "url", url) c.logger.Debug("listenbrainz rate limiter wait", "url", url)
if err := c.limiter.Wait(ctx); err != nil { if err := c.limiter.Wait(ctx); err != nil {
@@ -533,6 +567,21 @@ func (c *ListenBrainzClient) doRequest(
"status", resp.StatusCode, "status", resp.StatusCode,
) )
if resp.StatusCode == http.StatusUnauthorized {
// Recorded once, at warning level, because the next thing this
// client does is stop asking: a log line per artist is the
// symptom this latch exists to remove.
if c.refused.CompareAndSwap(false, true) {
c.logger.Warn("listenbrainz refused this client: "+
"popularity data needs a token, so the rest of this "+
"run will not ask for it",
"url", url,
)
}
return nil, fmt.Errorf("%w: %s", ErrListenBrainzUnauthorized, url)
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 { if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf( return nil, fmt.Errorf(
"%w: %d %s", ErrListenBrainzHTTP, resp.StatusCode, truncateBody(respBody), "%w: %d %s", ErrListenBrainzHTTP, resp.StatusCode, truncateBody(respBody),
@@ -0,0 +1,90 @@
package explore
import (
"context"
"errors"
"log/slog"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"yellowjacket/backend/database"
)
// #284: the popularity endpoints answered 401 to every request, so a
// backfill spent one rate-limited request per artist to be told the same
// thing — 399 of them in a minute against a real library. A token is a
// property of the installation, not of the artist, so the first refusal
// is the answer for the whole client.
func TestListenBrainzLatchesARefusal(t *testing.T) {
t.Parallel()
var requests atomic.Int64
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
requests.Add(1)
w.WriteHeader(http.StatusUnauthorized)
},
))
t.Cleanup(srv.Close)
c := NewListenBrainzClient(
NewRateLimiter(), NewCache(database.NewTestDB(t), slog.Default()), slog.Default(),
)
c.SetBaseURL(srv.URL)
for i := range 5 {
_, err := c.TopRecordingsForArtist(context.Background(), "an-mbid")
if !errors.Is(err, ErrListenBrainzUnauthorized) {
t.Fatalf("call %d: err = %v, want ErrListenBrainzUnauthorized", i, err)
}
}
if got := requests.Load(); got != 1 {
t.Errorf("requests = %d, want 1: the rest of the calls are the same answer", got)
}
if !c.Unauthorized() {
t.Error("Unauthorized() = false after a 401")
}
}
// A failure that a retry could fix must not latch: the artist stays
// unmarked and the next run asks again.
func TestListenBrainzDoesNotLatchATransientFailure(t *testing.T) {
t.Parallel()
var requests atomic.Int64
srv := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) {
requests.Add(1)
w.WriteHeader(http.StatusInternalServerError)
},
))
t.Cleanup(srv.Close)
c := NewListenBrainzClient(
NewRateLimiter(), NewCache(database.NewTestDB(t), slog.Default()), slog.Default(),
)
c.SetBaseURL(srv.URL)
for range 3 {
_, err := c.TopRecordingsForArtist(context.Background(), "an-mbid")
if !errors.Is(err, ErrListenBrainzHTTP) {
t.Fatalf("err = %v, want ErrListenBrainzHTTP", err)
}
}
if got := requests.Load(); got != 3 {
t.Errorf("requests = %d, want 3", got)
}
if c.Unauthorized() {
t.Error("Unauthorized() = true after a 500")
}
}
+266
View File
@@ -0,0 +1,266 @@
package explore
import (
"context"
"database/sql"
"io"
"os"
"path/filepath"
"strings"
"testing"
"yellowjacket/backend/database"
)
// Import of the artifact we actually publish, as a client imports it.
//
// Every other test here builds a fixture, and a fixture is a second
// description of the storage format that can be wrong in the same
// direction as the code reading it. That is how #258 shipped: the
// importer positioned its batch walk with a Go `string` cursor against
// the artifact's 16-byte `mbid` column, and SQLite neither coerces
// between TEXT and BLOB nor complains about the comparison — so the walk
// merged nothing and never advanced, and no install could finish its
// first index build. The fixture that guards the walk writes the old
// text encoding; the only compact fixture is one row, below the batch
// size, so the bound query never ran. Both passed throughout.
//
// So this one takes the published file and runs the client's own path
// over it — checksum, decompress, merge — and asserts that what the
// artifact holds is what the client ends up with.
//
// It skips without the path, so an ordinary test run pays nothing for
// it, and the publish job is where it is meant to run:
//
// YJ_CORE_INDEX_ARTIFACT=/tmp/core-index.db.zst \
// go test -tags indexbuild -run TestImportPublishedArtifact \
// ./backend/explore/
//
// The indexbuild tag is not incidental: that job's container has no GTK,
// and the default tag set links the app through Wails.
// publishedArtifactEnv points at the published artifact: the compressed
// core-index.db.zst, or the unpacked core-index.db.
const publishedArtifactEnv = "YJ_CORE_INDEX_ARTIFACT"
// artifactTotals is the pair this test compares across the boundary.
//
// Rows is the whole point — a merge that lands fewer of them than the
// artifact declares is a catalog that looks populated and is missing
// things nobody can name — and popularity is the half whose absence was
// reported when it happened, because it arrives only through the merge.
type artifactTotals struct {
rows int
withListen int
}
func TestImportPublishedArtifact(t *testing.T) {
published := strings.TrimSpace(os.Getenv(publishedArtifactEnv))
if published == "" {
t.Skipf("set %s=<core-index.db.zst> to import the published artifact",
publishedArtifactEnv)
}
if _, err := os.Stat(published); err != nil {
t.Fatalf("%s: %v", publishedArtifactEnv, err)
}
// A file-backed database rather than NewTestDB's in-memory one: the
// artifact is ~135MB and a million rows, which is not a thing to hold
// in RAM inside a test. YJ_HOME is how NewDB is pointed somewhere
// disposable, and going through NewDB means this is the constructor,
// the schema and the read pool the app itself opens.
//
// Nothing closes it, because nothing can: `DB` has no Close and the
// app's handles are process-lifetime by design. The directory is
// unlinked at cleanup and the file goes with it.
t.Setenv("YJ_HOME", t.TempDir())
db, err := database.NewDB(testLogger())
if err != nil {
t.Fatalf("open database: %v", err)
}
si := NewSearchIndex(db, nil, nil, testLogger())
// The checksum the publisher shipped, if it shipped one. Every
// client verifies it and refuses the artifact when it does not
// match, so a wrong one breaks Explore for everyone who has not
// already imported — and nothing else would see it, because the
// comparison is between two files only the publisher has.
if want, ok := publishedChecksum(published); ok {
got, err := fileSHA256(published)
if err != nil {
t.Fatalf("checksum the artifact: %v", err)
}
if got != want {
t.Errorf("published artifact hashes to %s, but its .sha256 says %s",
got, want)
}
}
unpacked := unpackPublishedArtifact(t, si, published)
want, err := artifactTotalsOf(unpacked)
if err != nil {
t.Fatalf("count the artifact's rows: %v", err)
}
if err := si.importCoreArtifact(context.Background(), unpacked); err != nil {
t.Fatalf("importCoreArtifact: %v", err)
}
got, err := indexTotalsOf(db)
if err != nil {
t.Fatalf("count the index's rows: %v", err)
}
if got.rows != want.rows {
t.Errorf("merged %d rows, but the artifact holds %d",
got.rows, want.rows)
}
if got.withListen != want.withListen {
t.Errorf("%d rows carry a listen count, but the artifact holds %d of them",
got.withListen, want.withListen)
}
// The FTS index is rebuilt from the table once the merge is done, and
// it is what search actually reads: a merge that lands without it
// leaves Explore silently matching nothing, which is the state #258
// produced by a different route.
var indexed int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM explore_index_fts",
).Scan(&indexed); err != nil {
t.Fatalf("count the FTS index: %v", err)
}
if indexed != got.rows {
t.Errorf("FTS index holds %d rows against the table's %d",
indexed, got.rows)
}
// And one row read back through the app's own path, which is the
// other direction of every conversion the merge makes: a byte MBID
// out of the table, the app's dashed form, and back in as a lookup.
var raw []byte
if err := db.QueryRowWriter(`
SELECT mbid FROM explore_index
WHERE entity_type = 1 /* artist */ AND popularity > 0
ORDER BY popularity DESC LIMIT 1`).Scan(&raw); err != nil {
t.Fatalf("read a stored mbid: %v", err)
}
dashed, err := mbidFromBytes(raw)
if err != nil {
t.Fatalf("the stored mbid is not one: %v", err)
}
artist := si.LookupArtistByMBID(dashed)
if artist == nil {
t.Fatalf("the artifact's most popular artist %s does not look up", dashed)
}
if artist.Popularity == 0 {
t.Errorf("artist %s came back with no popularity", dashed)
}
}
// publishedChecksum reads the sha256 the publisher wrote beside the
// artifact, in `sha256sum` output form. A missing file is not a
// failure: it is only there when the artifact came from the publish job.
func publishedChecksum(path string) (string, bool) {
body, err := os.ReadFile(path + ".sha256")
if err != nil {
return "", false
}
sum := strings.TrimSpace(string(body))
if i := strings.IndexAny(sum, " \t"); i > 0 {
sum = sum[:i]
}
if len(sum) != 64 {
return "", false
}
return strings.ToLower(sum), true
}
// unpackPublishedArtifact returns a path to the unpacked database,
// going through the client's own decompression when it is handed the
// compressed file that is actually published.
func unpackPublishedArtifact(t *testing.T, si *SearchIndex, path string) string {
t.Helper()
if strings.HasSuffix(path, ".db") {
return path
}
// Copied into the test's own directory first: decompress writes
// beside the compressed file, and the publisher's directory is not
// this test's to write in.
staging := t.TempDir()
dst := filepath.Join(staging, coreArtifactFile)
src, err := os.Open(path)
if err != nil {
t.Fatalf("open the published artifact: %v", err)
}
defer func() { _ = src.Close() }()
out, err := os.Create(dst)
if err != nil {
t.Fatalf("create a staging copy: %v", err)
}
if _, err := io.Copy(out, src); err != nil {
t.Fatalf("copy the published artifact: %v", err)
}
if err := out.Close(); err != nil {
t.Fatalf("close the staging copy: %v", err)
}
fetcher := &artifactFetcher{si: si, stagingDir: staging}
if err := fetcher.decompress(context.Background()); err != nil {
t.Fatalf("decompress the published artifact: %v", err)
}
return fetcher.unpackedPath()
}
// artifactTotalsOf counts what an artifact file holds, read directly so
// the numbers do not depend on anything the client does.
func artifactTotalsOf(path string) (artifactTotals, error) {
db, err := sql.Open("sqlite", "file:"+path+"?mode=ro")
if err != nil {
return artifactTotals{}, err
}
defer func() { _ = db.Close() }()
var totals artifactTotals
err = db.QueryRow(`SELECT COUNT(*), COALESCE(SUM(popularity > 0), 0)
FROM explore_index`).Scan(&totals.rows, &totals.withListen)
if err != nil {
return artifactTotals{}, err
}
return totals, nil
}
// indexTotalsOf counts what the client ended up with.
func indexTotalsOf(db *database.DB) (artifactTotals, error) {
var totals artifactTotals
err := db.QueryRowWriter(`SELECT COUNT(*), COALESCE(SUM(popularity > 0), 0)
FROM explore_index`).Scan(&totals.rows, &totals.withListen)
return totals, err
}
+18
View File
@@ -518,6 +518,13 @@ func (si *SearchIndex) BackfillLibraryDiscographies(ctx context.Context) {
break break
} }
// A refused client is refused for every artist: the rest of
// this pass would be the same 401, once per artist (#284). The
// artists stay unmarked, so a run with a token picks them up.
if indexLB.Unauthorized() {
break
}
work <- mbid work <- mbid
} }
@@ -534,6 +541,17 @@ func (si *SearchIndex) BackfillLibraryDiscographies(ctx context.Context) {
return return
} }
if indexLB.Unauthorized() {
si.logger.Warn("discography backfill stopped early: "+
"listenbrainz refused this client, and a token is what it wants",
"artists", total, "of", len(mbids),
)
job.logf(jobs.LevelWarn, "Stopped early: ListenBrainz needs a token")
return
}
job.logf(jobs.LevelInfo, "Filled in "+strconv.Itoa(total)+" artists") job.logf(jobs.LevelInfo, "Filled in "+strconv.Itoa(total)+" artists")
si.logger.Info("discography backfill complete", "artists", total) si.logger.Info("discography backfill complete", "artists", total)
+17 -11
View File
@@ -383,9 +383,13 @@ func (si *SearchIndex) topByPopularity(
// MusicBrainz IDs, so this never touches the library tables and asks // MusicBrainz IDs, so this never touches the library tables and asks
// one query rather than one per artist. // one query rather than one per artist.
// //
// The artists are drawn most-popular-owned-album first, so a large // **The row is drawn at random, and that is the whole point of it.**
// library's pool is the part of it the user is likeliest to recognise // Ordered by popularity it was a second leaderboard: the same handful of
// rather than whichever artists sort first. // big names appeared every time the page opened, which is not what "you
// own one album by these artists" is saying. The pool is still bounded
// to `pool` artists — a 4 000-artist library does not need all of them
// ranked — but which of them, and which of their albums, is `RANDOM()`,
// so the shelf is a different sample each visit.
func (si *SearchIndex) unownedAlbumsBySinglyOwnedArtists( func (si *SearchIndex) unownedAlbumsBySinglyOwnedArtists(
ctx context.Context, ctx context.Context,
pool, limit int, pool, limit int,
@@ -396,15 +400,17 @@ func (si *SearchIndex) unownedAlbumsBySinglyOwnedArtists(
WHERE entity_type = 2 /* release_group */ WHERE entity_type = 2 /* release_group */
AND in_library = 0 AND in_library = 0
AND artist_mbid IN ( AND artist_mbid IN (
SELECT artist_mbid FROM explore_index SELECT artist_mbid FROM (
WHERE entity_type = 2 /* release_group */ SELECT artist_mbid FROM explore_index
AND in_library = 1 WHERE entity_type = 2 /* release_group */
AND artist_mbid != x'' AND in_library = 1
GROUP BY artist_mbid AND artist_mbid != x''
HAVING COUNT(*) = 1 GROUP BY artist_mbid
ORDER BY MAX(popularity) DESC HAVING COUNT(*) = 1
)
ORDER BY RANDOM()
LIMIT ?) LIMIT ?)
ORDER BY popularity DESC ORDER BY RANDOM()
LIMIT ?`, LIMIT ?`,
pool, limit, pool, limit,
)) ))
+6
View File
@@ -3,6 +3,7 @@ package explore
import ( import (
"context" "context"
"log/slog" "log/slog"
"sort"
"testing" "testing"
"yellowjacket/backend/database" "yellowjacket/backend/database"
@@ -202,6 +203,11 @@ func TestShelves_MoreFromOwnedNeedsExactlyOneOwnedAlbum(t *testing.T) {
titles = append(titles, album.Title) titles = append(titles, album.Title)
} }
// The row is a random sample, so the *set* is what is asserted and
// not the order — see `unownedAlbumsBySinglyOwnedArtists` for why
// the ordering was given up.
sort.Strings(titles)
if len(titles) != 2 || titles[0] != "Second" || titles[1] != "Third" { if len(titles) != 2 || titles[0] != "Second" || titles[1] != "Third" {
t.Fatalf("albums = %v, want [Second Third]", titles) t.Fatalf("albums = %v, want [Second Third]", titles)
} }
+72
View File
@@ -3,6 +3,7 @@ package library
import ( import (
"bytes" "bytes"
"crypto/sha256" "crypto/sha256"
"database/sql"
"encoding/hex" "encoding/hex"
"fmt" "fmt"
"image" "image"
@@ -69,6 +70,77 @@ func CoverArtFileSet(coverPath string) []string {
return paths return paths
} }
// sweepOrphanedCoverArt deletes the cover_art rows no album references
// and returns their file paths, for the caller to remove from disk
// after the transaction commits. Cover art is referenced only by
// albums.cover_art_id, so an orphan is a cover whose album is gone —
// which is every album the caller just swept.
//
// One implementation because the scan path, RemoveFromLibrary and
// RemoveLibrary all reach this state, and the scan side used to skip it
// entirely while RemoveLibrary did it inline (#247).
func (l *Library) sweepOrphanedCoverArt(tx *sql.Tx) ([]string, error) {
const orphanSQL = `
SELECT file_path FROM cover_art WHERE id NOT IN (
SELECT DISTINCT cover_art_id FROM albums
WHERE cover_art_id IS NOT NULL
)`
rows, err := tx.QueryContext(l.ctx, orphanSQL)
if err != nil {
return nil, fmt.Errorf("could not query orphaned cover art: %w", err)
}
var paths []string
for rows.Next() {
var filePath string
if err := rows.Scan(&filePath); err != nil {
l.logger.Warn("could not scan cover art path", "err", err)
continue
}
paths = append(paths, filePath)
}
// Close before the DELETE: the two run on the one writer connection.
if err := rows.Close(); err != nil {
l.logger.Warn("could not close cover art rows", "err", err)
}
if len(paths) > 0 {
if _, err := tx.ExecContext(l.ctx, `
DELETE FROM cover_art WHERE id NOT IN (
SELECT DISTINCT cover_art_id FROM albums
WHERE cover_art_id IS NOT NULL
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned cover_art: %w", err)
}
}
return paths, nil
}
// removeCoverArtFiles removes a cover original and its derived size
// variants. Only the original is stored in cover_art.file_path; the
// _sm/_md/_lg tiers are derived filenames beside it, so they have to be
// removed by name or they accumulate forever.
func (l *Library) removeCoverArtFiles(coverPaths []string) {
for _, coverPath := range coverPaths {
for _, path := range CoverArtFileSet(coverPath) {
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
l.logger.Warn(
"could not remove orphaned cover art file",
"path", path,
"err", err,
)
}
}
}
}
// saveCoverArt saves embedded cover art to the cache directory. // saveCoverArt saves embedded cover art to the cache directory.
// Returns the file path where the art was saved, or empty string // Returns the file path where the art was saved, or empty string
// if no picture data. Timing is recorded in the provided metrics. // if no picture data. Timing is recorded in the provided metrics.
+4 -4
View File
@@ -8,6 +8,7 @@ import (
"yellowjacket/backend/coverart" "yellowjacket/backend/coverart"
"yellowjacket/backend/database/sql/sqlcgen" "yellowjacket/backend/database/sql/sqlcgen"
"yellowjacket/internal/testfixtures"
) )
// TestScan_StoresOnlyCoverTiers pins the size decision: a scan writes // TestScan_StoresOnlyCoverTiers pins the size decision: a scan writes
@@ -26,10 +27,9 @@ func TestScan_StoresOnlyCoverTiers(t *testing.T) {
lib, db := setupTestLibrary(t) lib, db := setupTestLibrary(t)
root, err := filepath.Abs("../../test_data/music_library_test") // Load skips when the fixture library has not been generated, as
if err != nil { // every other fixture test does.
t.Fatalf("resolve fixture path: %v", err) root := testfixtures.Load(t).Root()
}
library, err := db.Queries.CreateLibrary(lib.ctx, sqlcgen.CreateLibraryParams{ library, err := db.Queries.CreateLibrary(lib.ctx, sqlcgen.CreateLibraryParams{
Name: "Fixtures", Name: "Fixtures",
+8 -50
View File
@@ -320,43 +320,12 @@ func (l *Library) RemoveLibrary(id int64) (*RemovalSummary, error) {
genresRemoved, _ := result.RowsAffected() genresRemoved, _ := result.RowsAffected()
// 15. Collect orphaned cover_art file paths for post-commit cleanup. // Collect and delete orphaned cover_art rows before the commit. The
// SAFETY: Hand-crafted SELECT for orphaned cover art identification. // shared helper is the one place this sweep lives, so the scan path,
// Parameterless. // RemoveFromLibrary and this removal cannot drift (#247).
rows, err := tx.QueryContext(l.ctx, orphanedCoverArtPaths, err := l.sweepOrphanedCoverArt(tx)
`SELECT file_path FROM cover_art WHERE id NOT IN (
SELECT DISTINCT cover_art_id FROM albums
WHERE cover_art_id IS NOT NULL
)`)
if err != nil { if err != nil {
return nil, fmt.Errorf("could not query orphaned cover art: %w", err) return nil, err
}
var orphanedCoverArtPaths []string
for rows.Next() {
var filePath string
if err := rows.Scan(&filePath); err != nil {
l.logger.Warn("could not scan cover art path", "err", err)
continue
}
orphanedCoverArtPaths = append(orphanedCoverArtPaths, filePath)
}
if err := rows.Close(); err != nil {
l.logger.Warn("could not close cover art rows", "err", err)
}
// 16. Delete orphaned cover_art rows.
// SAFETY: Hand-crafted orphan cleanup SQL. Parameterless.
if _, err := tx.ExecContext(l.ctx,
`DELETE FROM cover_art WHERE id NOT IN (
SELECT DISTINCT cover_art_id FROM albums
WHERE cover_art_id IS NOT NULL
)`); err != nil {
return nil, fmt.Errorf("could not delete orphaned cover_art: %w", err)
} }
// 17. Delete the library's tagging queue. tagging_items holds a // 17. Delete the library's tagging queue. tagging_items holds a
@@ -392,20 +361,9 @@ func (l *Library) RemoveLibrary(id int64) (*RemovalSummary, error) {
// avoids a costly full re-index of all remaining tracks (~10s for // avoids a costly full re-index of all remaining tracks (~10s for
// 25K tracks). // 25K tracks).
// 21. Post-commit: Delete orphaned cover art files and their sized // Post-commit: remove the orphaned cover art files and their sized
// variants. Only the original is stored in cover_art.file_path; the // variants.
// _sm/_md/_lg thumbnails are derived filenames beside it, so they l.removeCoverArtFiles(orphanedCoverArtPaths)
// have to be removed by name or they accumulate forever.
for _, coverPath := range orphanedCoverArtPaths {
for _, path := range CoverArtFileSet(coverPath) {
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
l.logger.Warn("could not remove orphaned cover art file",
"path", path,
"err", err,
)
}
}
}
// 22. Post-commit: Compact queue. // 22. Post-commit: Compact queue.
if l.removalHooks.CompactQueue != nil { if l.removalHooks.CompactQueue != nil {
+27 -2
View File
@@ -1034,7 +1034,7 @@ func (l *Library) scanInternal(
} }
} }
// Remove from FTS5 search index. // Remove from FTS5 search index and the lyrics index.
if err := l.db.DeleteSearchIndex(f.ID); err != nil { if err := l.db.DeleteSearchIndex(f.ID); err != nil {
l.logger.Warn( l.logger.Warn(
"failed to delete FTS entry for orphan", "failed to delete FTS entry for orphan",
@@ -1045,6 +1045,16 @@ func (l *Library) scanInternal(
metrics.addWarning(path, "orphan", err) metrics.addWarning(path, "orphan", err)
} }
if err := l.db.DeleteLyricsIndex(f.ID); err != nil {
l.logger.Warn(
"failed to delete lyrics index entry for orphan",
"id", f.ID,
"err", err,
)
metrics.addWarning(path, "orphan", err)
}
removed.Add(1) removed.Add(1)
} }
@@ -1255,17 +1265,32 @@ func (l *Library) pruneEmptyEntities() {
} }
} }
// Cover art after albums: a cover whose album just went is
// unreferenced, and leaving the row behind keeps its files exempt
// from the janitor's covers sweep forever (#247).
orphanedCovers, err := l.sweepOrphanedCoverArt(tx)
if err != nil {
l.logger.Warn("could not sweep orphaned cover art", "err", err)
return
}
if err := tx.Commit(); err != nil { if err := tx.Commit(); err != nil {
l.logger.Warn("could not commit entity cleanup", "err", err) l.logger.Warn("could not commit entity cleanup", "err", err)
return return
} }
if len(albumIDs) > 0 || len(artistIDs) > 0 || len(genreIDs) > 0 { // Post-commit: the rows are gone, so their files can go too.
l.removeCoverArtFiles(orphanedCovers)
if len(albumIDs) > 0 || len(artistIDs) > 0 || len(genreIDs) > 0 ||
len(orphanedCovers) > 0 {
l.logger.Info("pruned empty library entities", l.logger.Info("pruned empty library entities",
"albums", len(albumIDs), "albums", len(albumIDs),
"artists", len(artistIDs), "artists", len(artistIDs),
"genres", len(genreIDs), "genres", len(genreIDs),
"covers", len(orphanedCovers),
) )
} }
} }
+33 -18
View File
@@ -2,7 +2,6 @@ package library
import ( import (
"database/sql" "database/sql"
"errors"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
@@ -18,8 +17,6 @@ import (
// searchTrackLimit bounds an FTS search's result set. // searchTrackLimit bounds an FTS search's result set.
const searchTrackLimit = 500 const searchTrackLimit = 500
var errNoTracksInLibrary = errors.New("no tracks in library")
// Track is one audio file with everything a list needs to draw it. // Track is one audio file with everything a list needs to draw it.
type Track struct { type Track struct {
TrackName string TrackName string
@@ -169,28 +166,46 @@ func (l *Library) GetTrackMBIDs(filePath string) TrackMBIDs {
} }
} }
// GetTracks returns every track in a library, or in all of them when // pathLookupChunk bounds the paths bound into one IN (...) query, well
// libraryID is 0. // under SQLite's bind-variable limit.
const pathLookupChunk = 500
// GetTracksByPaths returns whole tracks for the given file paths, in
// the order asked, dropping any path that is not in the library.
// //
// The library id is a parameter rather than a second method because the // It is how the frontend resolves the tracks a surface is actually
// two used to be separate queries, separate bindings and a branch at // showing (#279). Track details from the queue, a playlist or a smart
// every call site - and the scoped form costs nothing (measured: 23 ms // playlist used to look the path up in the whole library's track
// against 21 ms over 26k rows). // array, which had to be loaded first — so it was fetched eagerly at
func (l *Library) GetTracks(libraryID int64) ([]Track, error) { // startup, 20.5 MB at 26k tracks, to answer questions about a handful
rows, err := l.db.ReadQueries.GetTracks(l.ctx, libraryID) // of rows.
if err != nil { func (l *Library) GetTracksByPaths(paths []string) ([]Track, error) {
l.logger.Error("could not retrieve audio files", "error", err) byPath := make(map[string]Track, len(paths))
return nil, fmt.Errorf("could not get tracks: %w", err) for start := 0; start < len(paths); start += pathLookupChunk {
chunk := paths[start:min(start+pathLookupChunk, len(paths))]
rows, err := l.db.ReadQueries.GetTracksByPaths(l.ctx, chunk)
if err != nil {
return nil, fmt.Errorf("could not get tracks by path: %w", err)
}
for _, row := range rows {
byPath[row.FilePath] = trackFromRow(row)
}
} }
l.logger.Info("audio file list", "count", len(rows), "libraryID", libraryID) tracks := make([]Track, 0, len(byPath))
if len(rows) == 0 { for _, path := range paths {
return nil, errNoTracksInLibrary if t, ok := byPath[path]; ok {
tracks = append(tracks, t)
// A path asked twice is answered once.
delete(byPath, path)
}
} }
return tracksFromRows(rows), nil return tracks, nil
} }
// SearchTracks runs the library's FTS index and returns whole tracks. // SearchTracks runs the library's FTS index and returns whole tracks.
+50
View File
@@ -249,3 +249,53 @@ func TestCoverArtFileSet(t *testing.T) {
} }
} }
} }
// Removing the last track of an album must take the album's cover art
// with it — both the row and every derived file — or the row keeps its
// files exempt from the janitor's covers sweep forever (#247).
func TestRemoveFromLibrary_DeletesOrphanedCoverArt(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
dir := t.TempDir()
// The largest tier is what cover_art.file_path names; write every
// variant so the sweep has a real set to remove.
for _, tier := range thumbnailTiers {
p := filepath.Join(dir, coverart.SizedFilename("abc123.jpg", tier.Suffix))
if err := os.WriteFile(p, []byte("img"), 0o600); err != nil {
t.Fatalf("write %s: %v", p, err)
}
}
cover := filepath.Join(dir, coverart.SizedFilename("abc123.jpg", "_lg"))
seedRemovableLibrary(t, lib, cover)
// Link the album to the cover so it is not orphaned until the track
// (and with it the album) goes.
if _, err := lib.db.ExecContext(
`UPDATE albums SET cover_art_id =
(SELECT id FROM cover_art WHERE file_path = ?)
WHERE name = 'Test Album'`,
cover,
); err != nil {
t.Fatalf("link cover art: %v", err)
}
if _, err := lib.RemoveFromLibrary([]string{"/music/song.mp3"}); err != nil {
t.Fatalf("RemoveFromLibrary: %v", err)
}
if n := countRows(t, lib, "cover_art"); n != 0 {
t.Errorf("cover_art has %d rows after removal, want 0", n)
}
for _, tier := range thumbnailTiers {
p := filepath.Join(dir, coverart.SizedFilename("abc123.jpg", tier.Suffix))
if _, err := os.Stat(p); !os.IsNotExist(err) {
t.Errorf("cover art file still present: %s", filepath.Base(p))
}
}
}
+5
View File
@@ -147,6 +147,11 @@ func (l *Library) RemoveFromLibrary(filePaths []string) (*RemovalResult, error)
l.logger.Warn("could not delete FTS entry for removed track", l.logger.Warn("could not delete FTS entry for removed track",
"path", row.FilePath, "id", row.ID, "err", err) "path", row.FilePath, "id", row.ID, "err", err)
} }
if err := l.db.DeleteLyricsIndex(row.ID); err != nil {
l.logger.Warn("could not delete lyrics index entry for removed track",
"path", row.FilePath, "id", row.ID, "err", err)
}
} }
// Deleting an audio_files row cascades to queue_tracks, so the // Deleting an audio_files row cascades to queue_tracks, so the
+4 -4
View File
@@ -58,15 +58,15 @@ func TestScan_FixtureLibraryLeavesNothingBehind(t *testing.T) {
t.Skip("fixture library is empty; run make testdata") t.Skip("fixture library is empty; run make testdata")
} }
tracks, err := lib.GetTracks(0) table, err := lib.GetTrackTable(0)
if err != nil { if err != nil {
t.Fatalf("GetTracks: %v", err) t.Fatalf("GetTrackTable: %v", err)
} }
// One track per file: the projection cannot multiply rows, because // One track per file: the projection cannot multiply rows, because
// there is no join table left to multiply them. // there is no join table left to multiply them.
if int64(len(tracks)) != files { if int64(len(table.FilePath)) != files {
t.Errorf("GetTracks returned %d rows for %d files", len(tracks), files) t.Errorf("GetTrackTable returned %d rows for %d files", len(table.FilePath), files)
} }
// Nothing shared outlives what refers to it. // Nothing shared outlives what refers to it.
+64
View File
@@ -0,0 +1,64 @@
package library
import (
"fmt"
"testing"
)
// #279: the track-details openers resolve the rows a surface is showing
// by path, instead of finding them in the whole library's array. The
// answer must keep the caller's order (a batch dialog lists them as
// selected), drop what is not in the library rather than invent it, and
// survive more paths than one IN (...) can bind.
func TestGetTracksByPaths(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
seedAlbumsAndGenres(t, lib)
got, err := lib.GetTracksByPaths([]string{
"/other/b1.mp3", "/music/missing.mp3", "/music/a1.mp3", "/other/b1.mp3",
})
if err != nil {
t.Fatalf("GetTracksByPaths: %v", err)
}
paths := make([]string, 0, len(got))
for _, tr := range got {
paths = append(paths, tr.FilePath)
}
if want := "[/other/b1.mp3 /music/a1.mp3]"; fmt.Sprint(paths) != want {
t.Fatalf("paths = %v, want %s (caller order, missing dropped, duplicate once)", paths, want)
}
// Whole tracks, not just the key: details render every field.
if got[1].TrackName != "A1" || len(got[1].Genre) != 2 {
t.Errorf("a1 = %+v, want title A1 with two genres", got[1])
}
}
func TestGetTracksByPathsSpansChunks(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
seedAlbumsAndGenres(t, lib)
// The real path last, behind more misses than one chunk binds, so a
// loop that only ran the first chunk would return nothing.
paths := make([]string, 0, pathLookupChunk+2)
for i := range pathLookupChunk + 1 {
paths = append(paths, fmt.Sprintf("/nowhere/%d.mp3", i))
}
paths = append(paths, "/music/a2.mp3")
got, err := lib.GetTracksByPaths(paths)
if err != nil {
t.Fatalf("GetTracksByPaths: %v", err)
}
if len(got) != 1 || got[0].FilePath != "/music/a2.mp3" {
t.Fatalf("got %d tracks (%v), want only /music/a2.mp3", len(got), got)
}
}
+190
View File
@@ -0,0 +1,190 @@
package library
import (
"fmt"
"strconv"
"strings"
)
// TrackTable is every track in a library as the Tracks view uses it:
// one array per column, and every repeated string stored once (#281).
//
// GetTracks used to answer with one object per track, which at 26 138
// tracks was 20.5 MB of JSON — ~350 bytes a row of key names, four
// cover URLs identical across an album, and artist, album and genre
// strings repeated on every track of the album. Encoding it cost the
// backend ~170 MB of transient allocation and parsing it was the
// WebView's peak. Here the keys appear once, a repeated string is a
// small integer, and the columns the Tracks view does not read are not
// sent at all: LastPlayed and the three larger cover tiers belong to
// the details dialog, which fetches whole tracks by path.
//
// The projection is still trackFromRow's — each row goes through it —
// so this is an encoding of a Track, never a second description of
// one. frontend/src/utils/track-table.ts is the only decoder.
type TrackTable struct {
// Strings holds every distinct string value; a string column holds
// indexes into it. Index 0 is always "".
Strings []string `json:"strings"`
// GenreSets holds every distinct genre list, as indexes into
// Strings; Genre holds an index into it per track.
GenreSets [][]uint32 `json:"genreSets"`
FilePath []string `json:"filePath"`
TrackName []uint32 `json:"trackName"`
ArtistName []uint32 `json:"artistName"`
Album []uint32 `json:"album"`
Composer []uint32 `json:"composer"`
FileType []uint32 `json:"fileType"`
Genre []uint32 `json:"genre"`
ArtistMBID []uint32 `json:"artistMbid"`
ReleaseGroupMBID []uint32 `json:"releaseGroupMbid"`
RecordingMBID []uint32 `json:"recordingMbid"`
CoverArtSmall []uint32 `json:"coverArtSmall"`
// LengthMs is Track.TrackLength as the number it encodes.
LengthMs []int64 `json:"lengthMs"`
TrackNumber []int64 `json:"trackNumber"`
DiscNumber []int64 `json:"discNumber"`
Year []int64 `json:"year"`
SampleRate []int64 `json:"sampleRate"`
BitDepth []int64 `json:"bitDepth"`
Channels []int64 `json:"channels"`
Bitrate []int64 `json:"bitrate"`
FileSize []int64 `json:"fileSize"`
PlayCount []int64 `json:"playCount"`
}
// trackTableBuilder interns strings and genre lists while rows are
// appended.
type trackTableBuilder struct {
table TrackTable
strings map[string]uint32
genres map[string]uint32
}
func newTrackTableBuilder(capacity int) *trackTableBuilder {
b := &trackTableBuilder{
strings: map[string]uint32{"": 0},
genres: map[string]uint32{},
}
t := &b.table
t.Strings = []string{""}
t.FilePath = make([]string, 0, capacity)
for _, col := range b.stringColumns() {
*col = make([]uint32, 0, capacity)
}
for _, col := range b.intColumns() {
*col = make([]int64, 0, capacity)
}
t.Genre = make([]uint32, 0, capacity)
return b
}
// stringColumns are the interned columns, in one place so the builder
// cannot allocate one and forget to fill it.
func (b *trackTableBuilder) stringColumns() []*[]uint32 {
t := &b.table
return []*[]uint32{
&t.TrackName, &t.ArtistName, &t.Album, &t.Composer, &t.FileType,
&t.ArtistMBID, &t.ReleaseGroupMBID, &t.RecordingMBID, &t.CoverArtSmall,
}
}
func (b *trackTableBuilder) intColumns() []*[]int64 {
t := &b.table
return []*[]int64{
&t.LengthMs, &t.TrackNumber, &t.DiscNumber, &t.Year, &t.SampleRate,
&t.BitDepth, &t.Channels, &t.Bitrate, &t.FileSize, &t.PlayCount,
}
}
func (b *trackTableBuilder) intern(s string) uint32 {
if i, ok := b.strings[s]; ok {
return i
}
i := uint32(len(b.table.Strings)) //nolint:gosec // bounded by the row count
b.table.Strings = append(b.table.Strings, s)
b.strings[s] = i
return i
}
func (b *trackTableBuilder) internGenres(genres []string) uint32 {
key := strings.Join(genres, genreDelimiter)
if i, ok := b.genres[key]; ok {
return i
}
set := make([]uint32, len(genres))
for j, g := range genres {
set[j] = b.intern(g)
}
i := uint32(len(b.table.GenreSets)) //nolint:gosec // bounded by the row count
b.table.GenreSets = append(b.table.GenreSets, set)
b.genres[key] = i
return i
}
func (b *trackTableBuilder) add(tr Track) error {
lengthMs, err := strconv.ParseInt(tr.TrackLength, 10, 64)
if err != nil {
return fmt.Errorf("track %q length %q: %w", tr.FilePath, tr.TrackLength, err)
}
t := &b.table
t.FilePath = append(t.FilePath, tr.FilePath)
strs := []string{
tr.TrackName, tr.ArtistName, tr.Album, tr.Composer, tr.FileType,
tr.ArtistMBID, tr.ReleaseGroupMBID, tr.RecordingMBID, tr.CoverArtSmall,
}
for i, col := range b.stringColumns() {
*col = append(*col, b.intern(strs[i]))
}
ints := []int64{
lengthMs, tr.TrackNumber, tr.DiscNumber, tr.Year, tr.SampleRate,
tr.BitDepth, tr.Channels, tr.Bitrate, tr.FileSize, tr.PlayCount,
}
for i, col := range b.intColumns() {
*col = append(*col, ints[i])
}
t.Genre = append(t.Genre, b.internGenres(tr.Genre))
return nil
}
// GetTrackTable returns every track in a library, or in all of them
// when libraryID is 0, as a TrackTable. An empty library is an empty
// table, not an error.
func (l *Library) GetTrackTable(libraryID int64) (TrackTable, error) {
rows, err := l.db.ReadQueries.GetTracks(l.ctx, libraryID)
if err != nil {
return TrackTable{}, fmt.Errorf("could not get tracks: %w", err)
}
b := newTrackTableBuilder(len(rows))
for i := range rows {
if err := b.add(trackFromRow(rows[i])); err != nil {
return TrackTable{}, err
}
}
return b.table, nil
}
+198
View File
@@ -0,0 +1,198 @@
package library
import (
"encoding/json"
"fmt"
"strconv"
"testing"
"yellowjacket/backend/database"
)
// seedTableLibrary seeds n albums of perAlbum tracks each, with a cover
// on every album, so the table has the repetition it exists to remove.
func seedTableLibrary(t *testing.T, lib *Library, albums, perAlbum int) {
t.Helper()
for a := range albums {
for n := range perAlbum {
database.InsertTestTrack(t, lib.db, database.TestTrack{
FilePath: fmt.Sprintf(
"/music/artist-%d/album-%d/%02d - Some Track Title.flac", a%7, a, n+1,
),
Title: fmt.Sprintf("Some Track Title %d", n+1),
Artist: fmt.Sprintf("Artist %d", a%7),
ArtistMBID: fmt.Sprintf("0b7a8d2e-0000-4000-8000-%012d", a%7),
Album: fmt.Sprintf("Album Name %d", a),
AlbumMBID: fmt.Sprintf("1c7a8d2e-0000-4000-8000-%012d", a),
RecordingMBID: fmt.Sprintf("2d7a8d2e-0000-4000-8000-%012d", a*100+n),
Genres: []string{"Ambient", fmt.Sprintf("Genre %d", a%3)},
TrackNumber: int64(n + 1),
DiscNumber: 1,
Year: 2000 + int64(a%20),
LengthMs: 200_000 + int64(n),
PlayCount: int64(n),
})
}
res, err := lib.db.ExecContext(
"INSERT INTO cover_art (file_path, mime_type) VALUES (?, 'image/jpeg')",
fmt.Sprintf("/data/covers/%064d.jpg", a),
)
if err != nil {
t.Fatalf("seed cover: %v", err)
}
coverID, _ := res.LastInsertId()
if _, err := lib.db.ExecContext(
"UPDATE albums SET cover_art_id = ? WHERE name = ?",
coverID, fmt.Sprintf("Album Name %d", a),
); err != nil {
t.Fatalf("seed album cover: %v", err)
}
}
}
// decodeTable is the Go mirror of frontend/src/utils/track-table.ts,
// for asserting the encoding loses nothing it claims to carry.
func decodeTable(t *testing.T, tbl TrackTable) []Track {
t.Helper()
str := func(col []uint32, i int) string { return tbl.Strings[col[i]] }
tracks := make([]Track, len(tbl.FilePath))
for i := range tracks {
var genres []string
for _, g := range tbl.GenreSets[tbl.Genre[i]] {
genres = append(genres, tbl.Strings[g])
}
tracks[i] = Track{
FilePath: tbl.FilePath[i],
TrackName: str(tbl.TrackName, i),
ArtistName: str(tbl.ArtistName, i),
Album: str(tbl.Album, i),
Composer: str(tbl.Composer, i),
FileType: str(tbl.FileType, i),
ArtistMBID: str(tbl.ArtistMBID, i),
ReleaseGroupMBID: str(tbl.ReleaseGroupMBID, i),
RecordingMBID: str(tbl.RecordingMBID, i),
CoverArtSmall: str(tbl.CoverArtSmall, i),
Genre: genres,
TrackLength: strconv.FormatInt(tbl.LengthMs[i], 10),
TrackNumber: tbl.TrackNumber[i],
DiscNumber: tbl.DiscNumber[i],
Year: tbl.Year[i],
SampleRate: tbl.SampleRate[i],
BitDepth: tbl.BitDepth[i],
Channels: tbl.Channels[i],
Bitrate: tbl.Bitrate[i],
FileSize: tbl.FileSize[i],
PlayCount: tbl.PlayCount[i],
}
}
return tracks
}
// #281: the table is an encoding of trackFromRow's Track, minus the
// fields the Tracks view does not read. Decoding it must give back
// exactly that, row for row.
func TestTrackTableRoundTrip(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
seedTableLibrary(t, lib, 6, 4)
tbl, err := lib.GetTrackTable(0)
if err != nil {
t.Fatalf("GetTrackTable: %v", err)
}
rows, err := lib.db.ReadQueries.GetTracks(lib.ctx, 0)
if err != nil {
t.Fatalf("GetTracks: %v", err)
}
if len(rows) != 24 || len(tbl.FilePath) != len(rows) {
t.Fatalf("table has %d rows, query %d, want 24", len(tbl.FilePath), len(rows))
}
got := decodeTable(t, tbl)
for i, row := range rows {
want := trackFromRow(row)
// Not carried: the details dialog fetches these by path.
want.LastPlayed, want.CoverArtPath, want.CoverArtMedium, want.CoverArtLarge = "", "", "", ""
if fmt.Sprintf("%+v", got[i]) != fmt.Sprintf("%+v", want) {
t.Fatalf("row %d:\n got %+v\nwant %+v", i, got[i], want)
}
}
// It was the cover, genre and name repetition that was paid for; a
// table that interned nothing would still round-trip.
if len(tbl.GenreSets) != 3 {
t.Errorf("genre sets = %d, want 3 distinct lists", len(tbl.GenreSets))
}
if a, b := tbl.CoverArtSmall[0], tbl.CoverArtSmall[1]; a != b || tbl.Strings[a] == "" {
t.Errorf("two tracks of one album hold cover indexes %d and %d", a, b)
}
}
// The point of the table, pinned: the same rows encode to a fraction of
// the object-per-track JSON they replace. A new column has to fit
// under this or raise it on purpose.
func TestTrackTableSizeBudget(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
seedTableLibrary(t, lib, 40, 10)
tbl, err := lib.GetTrackTable(0)
if err != nil {
t.Fatalf("GetTrackTable: %v", err)
}
rows, err := lib.db.ReadQueries.GetTracks(lib.ctx, 0)
if err != nil {
t.Fatalf("GetTracks: %v", err)
}
asTable, err := json.Marshal(tbl)
if err != nil {
t.Fatal(err)
}
asObjects, err := json.Marshal(tracksFromRows(rows))
if err != nil {
t.Fatal(err)
}
perTrack := len(asTable) / len(rows)
ratio := float64(len(asTable)) / float64(len(asObjects))
t.Logf("table %d B (%d B/track), objects %d B, ratio %.2f",
len(asTable), perTrack, len(asObjects), ratio)
if ratio > 0.25 {
t.Errorf("table is %.0f%% of the object encoding, budget 25%%", ratio*100)
}
}
func TestTrackTableEmptyLibraryIsAnAnswer(t *testing.T) {
t.Parallel()
lib, _ := setupTestLibrary(t)
tbl, err := lib.GetTrackTable(0)
if err != nil {
t.Fatalf("an empty library is an empty table, not an error: %v", err)
}
if len(tbl.FilePath) != 0 || len(tbl.Strings) != 1 {
t.Errorf("empty table = %+v", tbl)
}
}
+117
View File
@@ -666,3 +666,120 @@ func TestExpiredHTTPCacheJob_TrimsToBudget(t *testing.T) {
t.Errorf("kept %q, want the longest-lived row", kept) t.Errorf("kept %q, want the longest-lived row", kept)
} }
} }
// TestStaleArtistMetadataJob pins the sweep's two keep rules: an owned
// artist's metadata survives, a browsed artist's survives while it still
// holds cached artwork, and everything else goes (#248).
func TestStaleArtistMetadataJob(t *testing.T) {
t.Parallel()
db := database.NewTestDB(t)
const (
ownedMBID = "11111111-1111-1111-1111-111111111111"
browsedMBID = "22222222-2222-2222-2222-222222222222"
staleMBID = "33333333-3333-3333-3333-333333333333"
)
// The owned artist is in the library - which means a *file* says
// so. An artists row on its own is not ownership.
database.InsertTestTrack(t, db, database.TestTrack{
FilePath: "/music/owned.mp3",
Artist: "Owned",
ArtistMBID: ownedMBID,
})
for _, mbid := range []string{ownedMBID, browsedMBID, staleMBID} {
if _, err := db.ExecContext(
`INSERT INTO artist_metadata (mbid, source, data, fetched_at)
VALUES (?, 'wikidata-p18', x'00', CURRENT_TIMESTAMP)`,
mbid,
); err != nil {
t.Fatalf("seed artist_metadata for %s: %v", mbid, err)
}
}
// The browsed artist holds cached artwork, so its metadata is still
// referenced and must survive.
if _, err := db.ExecContext(
`INSERT INTO artist_images
(artist_mbid, source, source_url, file_path)
VALUES (?, 'test', 'http://x', '/art/primary.jpg')`,
browsedMBID,
); err != nil {
t.Fatalf("seed artist_images: %v", err)
}
if _, err := StaleArtistMetadataJob(db).Run(context.Background()); err != nil {
t.Fatalf("run job: %v", err)
}
for _, tc := range []struct {
mbid string
want int
}{
{ownedMBID, 1},
{browsedMBID, 1},
{staleMBID, 0},
} {
var n int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_metadata WHERE mbid = ?", tc.mbid,
).Scan(&n); err != nil {
t.Fatalf("count %s: %v", tc.mbid, err)
}
if n != tc.want {
t.Errorf("artist_metadata rows for %s = %d, want %d", tc.mbid, n, tc.want)
}
}
}
// TestStaleSearchClicksJob deletes only the clicks old enough to have
// left the retention window (#249).
func TestStaleSearchClicksJob(t *testing.T) {
t.Parallel()
db := database.NewTestDB(t)
count := func(mbid string) int {
t.Helper()
var n int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM search_clicks WHERE entity_mbid = ?", mbid,
).Scan(&n); err != nil {
t.Fatalf("count %s: %v", mbid, err)
}
return n
}
seed := func(query, mbid, lastClicked string) {
t.Helper()
if _, err := db.ExecContext(
`INSERT INTO search_clicks
(query, entity_mbid, entity_type, click_count, last_clicked)
VALUES (?, ?, 'recording', 1, ?)`,
query, mbid, lastClicked,
); err != nil {
t.Fatalf("seed search_clicks: %v", err)
}
}
seed("tide", "aaaa", "2024-01-01 00:00:00") // stale
seed("tide", "bbbb", "2999-01-01 00:00:00") // recent
if _, err := StaleSearchClicksJob(db).Run(context.Background()); err != nil {
t.Fatalf("run job: %v", err)
}
if n := count("bbbb"); n != 1 {
t.Errorf("recent click was deleted: %d rows, want 1", n)
}
if n := count("aaaa"); n != 0 {
t.Errorf("stale click survived: %d rows, want 0", n)
}
}
+66
View File
@@ -628,3 +628,69 @@ func dirSize(dir string) (bytes, files int64) {
return bytes, files return bytes, files
} }
// StaleArtistMetadataJob evicts long-lived artist metadata (bios, wiki
// leads, relationships) for artists the user no longer has any reason
// to keep around: not owned and holding no cached artwork.
//
// artist_metadata has no TTL by design — entity data changes rarely and
// re-fetching spends someone else's rate limit — so without a sweep it
// grows for the life of the install. This is the "swept when the
// artist is no longer referenced" contract the datamap always declared
// for it and nothing ever performed (#248).
func StaleArtistMetadataJob(db *database.DB) Job {
return Job{
Name: "artist-metadata-sweep",
MinInterval: dailyInterval,
Run: func(_ context.Context) (Result, error) {
res, err := db.ExecContext(
`DELETE FROM artist_metadata
WHERE mbid NOT IN (` + ownedArtistMBIDs + `)
AND mbid NOT IN (
SELECT artist_mbid FROM artist_images
)`,
)
if err != nil {
return Result{}, fmt.Errorf(
"delete stale artist_metadata rows: %w", err,
)
}
rows, _ := res.RowsAffected()
return Result{RowsDeleted: rows}, nil
},
}
}
// searchClicksRetention is how long a search-click ranking signal stays
// useful. search_clicks is authored behavioural data — nothing that
// owns a row ever drops it — so age is the ceiling that keeps the table
// from growing without bound for the life of the install (#249).
const searchClicksRetention = "-180 days"
// StaleSearchClicksJob deletes search-click ranking rows older than the
// retention window. Rows are small and the table grows slowly, so this
// runs daily and does almost nothing most runs.
func StaleSearchClicksJob(db *database.DB) Job {
return Job{
Name: "search-clicks-sweep",
MinInterval: dailyInterval,
Run: func(_ context.Context) (Result, error) {
res, err := db.ExecContext(
`DELETE FROM search_clicks
WHERE last_clicked < datetime('now', ?)`,
searchClicksRetention,
)
if err != nil {
return Result{}, fmt.Errorf(
"delete stale search_clicks rows: %w", err,
)
}
rows, _ := res.RowsAffected()
return Result{RowsDeleted: rows}, nil
},
}
}
+41
View File
@@ -134,6 +134,12 @@ type Service struct {
libraryDir LibraryDirProvider libraryDir LibraryDirProvider
favoritesConf FavoritesConfigProvider favoritesConf FavoritesConfigProvider
// onDeleted, when set, is called after a playlist is deleted so
// cross-cutting state that points at it (the queue's "Playing
// from" label) can stop pointing at a playlist that no longer
// exists. Wired from app.go, like Library.SetRemovalHooks.
onDeleted func(playlistID int64)
// dataDirOverride, when non-empty, replaces the OS user data // dataDirOverride, when non-empty, replaces the OS user data
// directory as the base for the playlists folder. Set by tests to // directory as the base for the playlists folder. Set by tests to
// keep M3U writes out of the real user data directory. // keep M3U writes out of the real user data directory.
@@ -166,6 +172,17 @@ func (s *Service) SetFavoritesConfig(
s.favoritesConf = provider s.favoritesConf = provider
} }
// SetOnPlaylistDeleted registers a callback invoked after a playlist is
// deleted, for cross-cutting invalidation.
//
//wails:ignore // internal wiring, not part of the app's IPC surface.
func (s *Service) SetOnPlaylistDeleted(onDeleted func(playlistID int64)) {
s.mu.Lock()
defer s.mu.Unlock()
s.onDeleted = onDeleted
}
// ServiceStartup is v3's service lifecycle hook: it runs once the // ServiceStartup is v3's service lifecycle hook: it runs once the
// runtime exists, and ctx is cancelled when the app shuts down. It // runtime exists, and ctx is cancelled when the app shuts down. It
// replaces v2's SetContext, which had to be called by hand from // replaces v2's SetContext, which had to be called by hand from
@@ -766,6 +783,17 @@ func (s *Service) DeletePlaylist(playlistID int64) error {
s.emitEvent(events.PlaylistDeleted, playlistID) s.emitEvent(events.PlaylistDeleted, playlistID)
// Cross-cutting invalidation: the queue's "Playing from" label may
// point at this playlist, and a link to a playlist that no longer
// exists is worse than none.
s.mu.Lock()
onDeleted := s.onDeleted
s.mu.Unlock()
if onDeleted != nil {
onDeleted(playlistID)
}
// Recreate the default playlist if we just deleted it. // Recreate the default playlist if we just deleted it.
if s.defaultPlaylistID() == playlistID { if s.defaultPlaylistID() == playlistID {
s.EnsureDefaultPlaylist() s.EnsureDefaultPlaylist()
@@ -3097,6 +3125,19 @@ func (s *Service) EvaluateSmartPlaylist(
return tracks, nil return tracks, nil
} }
// SuggestSmartPlaylistValues returns values of field present in the
// library that contain needle, for the rule editor's value box.
func (s *Service) SuggestSmartPlaylistValues(
field, needle string,
) ([]string, error) {
values, err := smartplaylist.SuggestValues(s.db, field, needle)
if err != nil {
return nil, fmt.Errorf("suggest smart playlist values: %w", err)
}
return values, nil
}
// PreviewSmartPlaylist evaluates a rule set from raw JSON without // PreviewSmartPlaylist evaluates a rule set from raw JSON without
// requiring a saved playlist. This powers live preview in the rule // requiring a saved playlist. This powers live preview in the rule
// editor — the frontend sends rules as they are being edited and // editor — the frontend sends rules as they are being edited and
+15
View File
@@ -1581,6 +1581,21 @@ func (q *Queue) dropSource() {
q.source = Source{} q.source = Source{}
} }
// DropSourceForPlaylist clears the queue's "Playing from" label when
// its source playlist is deleted. A link back to a playlist that no
// longer exists is worse than none, and the label otherwise survives
// the deletion until the next SetQueue (#249).
func (q *Queue) DropSourceForPlaylist(playlistID int64) {
q.mu.Lock()
defer q.mu.Unlock()
if (q.source.Type == "playlist" || q.source.Type == "smartPlaylist") &&
q.source.ID == playlistID {
q.dropSource()
q.persistState()
}
}
// commitMutation persists the current queue state after a mutation. // commitMutation persists the current queue state after a mutation.
// When reindex is true, track positions are renumbered first. // When reindex is true, track positions are renumbered first.
// The caller must hold q.mu. // The caller must hold q.mu.
+32
View File
@@ -535,3 +535,35 @@ func TestCycleRepeat_CyclesThroughModes(t *testing.T) {
t.Errorf("after third cycle: got %q, want %q", state.RepeatMode, RepeatOff) t.Errorf("after third cycle: got %q, want %q", state.RepeatMode, RepeatOff)
} }
} }
// TestDropSourceForPlaylist clears the "Playing from" label when the
// queue's source playlist is deleted, and leaves it alone otherwise
// (#249).
func TestDropSourceForPlaylist(t *testing.T) {
t.Parallel()
q, db := setupTestQueue(t)
paths := seedAudioFiles(t, db, 2)
q.SetQueue(paths, 0, false, Source{Type: "playlist", ID: 42, Label: "Road Trip"})
q.DropSourceForPlaylist(42)
if got := q.GetState().Source; got != (Source{}) {
t.Errorf("source = %+v, want empty after playlist 42 deleted", got)
}
}
func TestDropSourceForPlaylistIgnoresOtherPlaylists(t *testing.T) {
t.Parallel()
q, db := setupTestQueue(t)
paths := seedAudioFiles(t, db, 2)
source := Source{Type: "smartPlaylist", ID: 42, Label: "Road Trip"}
q.SetQueue(paths, 0, false, source)
q.DropSourceForPlaylist(7)
if got := q.GetState().Source; got != source {
t.Errorf("source = %+v, want %+v unchanged for a different playlist", got, source)
}
}
+82 -1
View File
@@ -614,7 +614,12 @@ const leanTrackQuery = `SELECT
af.file_size, af.file_size,
af.play_count, af.play_count,
COALESCE(af.last_played, '') AS last_played COALESCE(af.last_played, '') AS last_played
FROM ( FROM ` + leanTrackSource
// leanTrackSource is the joined row every rule's column resolves
// against, aliased af. Shared by Evaluate and SuggestValues so a
// suggestion is always a value a rule on the same field can match.
const leanTrackSource = `(
SELECT SELECT
af.id, af.id,
af.file_path, af.file_path,
@@ -1117,3 +1122,79 @@ func splitGenres(concatenated string) []string {
return strings.Split(concatenated, genreDelimiter) return strings.Split(concatenated, genreDelimiter)
} }
// suggestLimit bounds one suggestion answer: a combobox shows a
// screenful, and the user narrows by typing.
const suggestLimit = 50
// errNoSuggestions is a field the editor does not offer values for.
var errNoSuggestions = errors.New("field has no suggestions")
// suggestFields are the fields whose values the rule editor suggests.
// Every one is in fieldMap; numeric fields other than the years are
// ranges, where a list of every value present is no help.
var suggestFields = map[string]bool{
"title": true, "artist": true, "album": true, "genre": true,
"composer": true, "file_type": true, "year": true, "release_year": true,
}
// SuggestValues returns up to suggestLimit distinct values of field
// present in the library that contain needle (case-insensitively),
// sorted.
//
// The rule editor used to build these lists from libraryStore's
// whole-library arrays, which made the editor one more reason to fetch
// every track at startup (#279) — and gave it an empty list when the
// arrays had not landed. Asking for the values that match what has
// been typed is a few hundred bytes, whatever the library's size.
func SuggestValues(db *database.DB, field, needle string) ([]string, error) {
col, ok := fieldMap[field]
if !ok || !suggestFields[field] {
return nil, fmt.Errorf("%w: %q", errNoSuggestions, field)
}
pattern := "%" + escapeLike(needle) + "%"
// SAFETY: col comes from fieldMap, never from the caller; the
// needle is a bound parameter.
query := `SELECT DISTINCT CAST(af.` + col + ` AS TEXT) AS v FROM ` +
leanTrackSource + `
WHERE v != '' AND v != '0' AND v LIKE ? ESCAPE '\'
ORDER BY v COLLATE NOCASE LIMIT ?`
if field == "genre" {
query = `SELECT name FROM genres
WHERE name != '' AND name LIKE ? ESCAPE '\'
ORDER BY name COLLATE NOCASE LIMIT ?`
}
rows, err := db.QueryContext(query, pattern, suggestLimit)
if err != nil {
return nil, fmt.Errorf("suggest %s: %w", field, err)
}
defer func() { _ = rows.Close() }()
values := make([]string, 0, suggestLimit)
for rows.Next() {
var v string
if err := rows.Scan(&v); err != nil {
return nil, fmt.Errorf("suggest %s: %w", field, err)
}
values = append(values, v)
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("suggest %s: %w", field, err)
}
return values, nil
}
// escapeLike makes needle match itself literally inside a LIKE pattern
// with ESCAPE '\'.
func escapeLike(needle string) string {
return strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(needle)
}
+61
View File
@@ -0,0 +1,61 @@
package smartplaylist
import (
"errors"
"slices"
"testing"
"yellowjacket/backend/database"
)
// The rule editor's value box asks the backend for what matches the
// text typed (#279), rather than building every list from the whole
// library loaded into the frontend.
func TestSuggestValues(t *testing.T) {
t.Parallel()
db := database.NewTestDB(t)
seedSmartPlaylistData(t, db)
cases := []struct {
field, needle string
want []string
}{
// Case-insensitive substring, distinct, sorted.
{"artist", "q", []string{"QOTSA", "Queen"}},
{"album", "OPERA", []string{"A Night at the Opera"}},
// Genres come from the genre table, one value per genre, not
// from a concatenated per-track column.
{"genre", "rock", []string{
"Funk Rock", "Hard Rock", "Progressive Rock", "Rock", "Stoner Rock",
}},
// Years are numbers and are suggested as their text.
{"year", "199", []string{"1990", "1991"}},
// The needle is literal: a LIKE wildcard in it matches only itself.
{"title", "%", []string{}},
{"composer", "_", []string{}},
}
for _, tc := range cases {
got, err := SuggestValues(db, tc.field, tc.needle)
if err != nil {
t.Fatalf("SuggestValues(%q, %q): %v", tc.field, tc.needle, err)
}
if !slices.Equal(got, tc.want) {
t.Errorf("SuggestValues(%q, %q) = %q, want %q", tc.field, tc.needle, got, tc.want)
}
}
}
func TestSuggestValuesRefusesFieldsItDoesNotSuggest(t *testing.T) {
t.Parallel()
db := database.NewTestDB(t)
for _, field := range []string{"duration", "file_path", "title; DROP TABLE x"} {
if _, err := SuggestValues(db, field, ""); !errors.Is(err, errNoSuggestions) {
t.Errorf("SuggestValues(%q) err = %v, want errNoSuggestions", field, err)
}
}
}
+27
View File
@@ -0,0 +1,27 @@
import { chromium } from '@playwright/test';
const b = await chromium.launch({ executablePath: '/usr/bin/chromium' });
const p = await b.newPage();
await p.addInitScript({ path: '/mnt/vault/dev/golang/yellowjacket/.playwright/init-events.js' });
await p.goto('http://localhost:34115', { waitUntil: 'load' });
await p.evaluate(() => window.__yjEvents.ready(30000));
const out = await p.evaluate(async () => {
const time = async (label, path, args, n = 5) => {
const ms = [];
for (let i = 0; i < n; i++) {
const t0 = performance.now();
await window.__yjEvents.call(path, args, 60000);
ms.push(performance.now() - t0);
}
ms.sort((a, b) => a - b);
return { label, medianMs: Math.round(ms[Math.floor(n / 2)]), all: ms.map((m) => Math.round(m)) };
};
return [
await time('GetTrackTable(0)', 'library.Library.GetTrackTable', [0], 4),
await time('GetAlbums(0)', 'library.Library.GetAlbums', [0], 4),
await time('SearchLocal(tide)', 'explore.Service.SearchLocal', ['tide'], 4),
await time('SearchLocal(rock)', 'explore.Service.SearchLocal', ['rock'], 4),
await time('SearchTracks(tide)', 'library.Library.SearchTracks', ['tide', 0], 4),
];
});
console.log(JSON.stringify(out, null, 1));
await b.close();
+182 -11
View File
@@ -42,6 +42,13 @@
* browse number below could not see this, because it * browse number below could not see this, because it
* visits Explore without ever typing in it. * visits Explore without ever typing in it.
* heap JS heap after a scripted browse, post-GC. m3. * heap JS heap after a scripted browse, post-GC. m3.
* memory what the app holds at rest after launch, before any
* view but the landing one is opened: the backend
* process's RSS and peak, its Go heap, the page's JS
* heap, and the bytes each binding returned on the way.
* Then the same after the first open of Tracks, and how
* long that open took to its first *row* — the number
* eager loading was buying (#280/#281).
* *
* Usage: * Usage:
* node e2e/perf/measure.mjs --label before * node e2e/perf/measure.mjs --label before
@@ -51,7 +58,7 @@
* Requires a running app: `make dev-headless SEED=bulk`. * Requires a running app: `make dev-headless SEED=bulk`.
*/ */
import { readFileSync, writeFileSync, mkdirSync, existsSync } from 'node:fs'; import { readFileSync, writeFileSync, mkdirSync, existsSync, readdirSync } from 'node:fs';
import { dirname, resolve } from 'node:path'; import { dirname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url'; import { fileURLToPath } from 'node:url';
import { chromium } from '@playwright/test'; import { chromium } from '@playwright/test';
@@ -190,6 +197,135 @@ const SEARCH_DEBOUNCE_MS = 150;
/* -------------------------------------------------------------------- */ /* -------------------------------------------------------------------- */
/**
* The backend process's memory, from /proc and its own pprof endpoint.
*
* Linux and a dev build only (pprof is mounted by `-tags dev`); either
* missing is a null, not a failure — this is a measurement harness, and
* a number it cannot take is reported as absent rather than as zero.
*/
async function backendMemory() {
const out = { rssMB: null, peakMB: null, anonMB: null, goHeapInuseMB: null, goHeapHeldMB: null };
const pid = findBackendPid();
if (pid) {
try {
const status = readFileSync(`/proc/${pid}/status`, 'utf8');
const kb = (k) => Number(status.match(new RegExp(`^${k}:\\s+(\\d+)`, 'm'))?.[1] ?? NaN);
out.rssMB = round(kb('VmRSS') / 1024, 1);
out.peakMB = round(kb('VmHWM') / 1024, 1);
out.anonMB = round(kb('RssAnon') / 1024, 1);
} catch {
// The process went away between finding it and reading it.
}
}
try {
const text = await (await fetch('http://localhost:6060/debug/pprof/heap?debug=1')).text();
const field = (k) => Number(text.match(new RegExp(`^# ${k} = (\\d+)`, 'm'))?.[1] ?? NaN);
out.goHeapInuseMB = round(field('HeapInuse') / 1048576, 1);
// What the runtime holds from the OS: the RSS the heap accounts for.
out.goHeapHeldMB = round((field('HeapSys') - field('HeapReleased')) / 1048576, 1);
} catch {
// Not a dev build, or pprof is not listening.
}
return out;
}
/** The pid of the running `yj-dev` binary, found by name under /proc. */
function findBackendPid() {
try {
for (const entry of readdirSync('/proc')) {
if (!/^\d+$/.test(entry)) continue;
try {
if (readFileSync(`/proc/${entry}/comm`, 'utf8').trim() === 'yj-dev') return entry;
} catch {
// Raced with an exiting process.
}
}
} catch {
// No /proc: not Linux.
}
return null;
}
/** Bytes returned per binding name, largest first. */
function bytesByBinding(calls) {
const by = {};
for (const c of calls) by[c.path] = (by[c.path] ?? 0) + (c.bytes ?? 0);
return Object.fromEntries(
Object.entries(by)
.filter(([, b]) => b > 0)
.sort((a, b) => b[1] - a[1])
.map(([k, b]) => [k, round(b / 1048576, 2)]),
);
}
async function pageMemory(page, client) {
await client.send('HeapProfiler.collectGarbage');
await page.waitForTimeout(300);
const heap = await client.send('Runtime.getHeapUsage');
const dom = await client.send('Memory.getDOMCounters');
return { jsHeapMB: round(heap.usedSize / 1048576, 1), domNodes: dom.nodes };
}
async function measureMemory(page, client) {
// Settled: the landing view has loaded and any idle warming has run.
await page.waitForTimeout(5000);
const startupCalls = await page.evaluate(() => window.__yjPerf.calls);
const atRest = {
backend: await backendMemory(),
page: await pageMemory(page, client),
bytesByBindingMB: bytesByBinding(startupCalls),
totalBindingMB: round(startupCalls.reduce((n, c) => n + (c.bytes ?? 0), 0) / 1048576, 2),
};
// First open of Tracks, to its first row: what a user waits for when
// nothing was loaded ahead of them.
const since = await page.evaluate(() => performance.now());
const firstRowMs = await page.evaluate(async () => {
const t0 = performance.now();
document.dispatchEvent(new CustomEvent('navigate', { detail: { view: 'tracks' } }));
for (;;) {
const list = document.querySelector('#main-content > track-list:not(.view-hidden)');
if (list?.shadowRoot?.querySelector('[data-testid="track-row"]')) {
return Math.round(performance.now() - t0);
}
if (performance.now() - t0 > 60000) return null;
await new Promise((r) => requestAnimationFrame(r));
}
});
await page.waitForTimeout(2000);
const tracksCalls = await page.evaluate((t) => window.__yjPerf.since(t), since);
return {
atRest,
tracksOpen: {
firstRowMs,
bytesByBindingMB: bytesByBinding(tracksCalls),
backend: await backendMemory(),
page: await pageMemory(page, client),
},
};
}
/* -------------------------------------------------------------------- */
async function measureStartup(page) { async function measureStartup(page) {
const nav = await page.evaluate(() => { const nav = await page.evaluate(() => {
const n = performance.getEntriesByType('navigation')[0]; const n = performance.getEntriesByType('navigation')[0];
@@ -224,20 +360,33 @@ async function measureStartup(page) {
scriptBytesBeforePaint, scriptBytesBeforePaint,
requests: res.length, requests: res.length,
crossOriginRequests: crossOrigin.length, crossOriginRequests: crossOrigin.length,
crossOriginHosts: [...new Set(crossOrigin.map((u) => new URL(u).host))], crossOriginHosts: [...new Set(crossOrigin.map((u) => {
try {
return new URL(u).host;
} catch {
return u;
}
}))],
}; };
}); });
// "First row on screen" is the number a user experiences as startup; // "First row on screen" is the number a user experiences as startup
// FCP fires on the chrome around an empty list. // *when the app lands on Tracks*; FCP fires on the chrome around an
// empty list.
//
// The deadline is short because since #280 a landing on any other
// view leaves the track list unloaded, and this would otherwise
// spend a minute waiting for a row that is not coming. The number
// that means something either way is `memory.tracksOpen.firstRowMs`,
// which opens Tracks and waits for its first row deliberately.
const firstRowMs = await page.evaluate(async () => { const firstRowMs = await page.evaluate(async () => {
const t0 = performance.now(); const t0 = performance.now();
const deadline = t0 + 60000; const deadline = t0 + 2000;
for (;;) { for (;;) {
const list = document.querySelector('track-list'); const list = document.querySelector('track-list');
const row = list?.shadowRoot?.querySelector('[role="row"], .track-row'); const row = list?.shadowRoot?.querySelector('[role="row"], .track-row');
if (row) return Math.round(performance.now() - t0 + (performance.timeOrigin ? 0 : 0)); if (row) return Math.round(performance.now() - t0);
if (performance.now() > deadline) return null; if (performance.now() > deadline) return null;
await new Promise((r) => setTimeout(r, 16)); await new Promise((r) => setTimeout(r, 16));
} }
@@ -337,7 +486,8 @@ async function measureTrackChange(page) {
await ev.call('queue.Queue.Clear', [], 10000).catch(() => {}); await ev.call('queue.Queue.Clear', [], 10000).catch(() => {});
const tracks = await ev.call('library.Library.GetAllTracks', [], 60000); // The columnar list (#281): only the paths are needed here.
const tracks = ((await ev.call('library.Library.GetTrackTable', [0], 60000))?.filePath ?? []).map((FilePath) => ({ FilePath }));
const paths = (tracks ?? []).slice(0, 4).map((t) => t.FilePath); const paths = (tracks ?? []).slice(0, 4).map((t) => t.FilePath);
if (paths.length < 2) return { error: 'library too small to measure' }; if (paths.length < 2) return { error: 'library too small to measure' };
@@ -435,7 +585,8 @@ async function measureFavouriteToggle(page) {
const ev = window.__yjEvents; const ev = window.__yjEvents;
const perf = window.__yjPerf; const perf = window.__yjPerf;
const tracks = await ev.call('library.Library.GetAllTracks', [], 60000); // The columnar list (#281): only the paths are needed here.
const tracks = ((await ev.call('library.Library.GetTrackTable', [0], 60000))?.filePath ?? []).map((FilePath) => ({ FilePath }));
const paths = (tracks ?? []).map((t) => t.FilePath); const paths = (tracks ?? []).map((t) => t.FilePath);
if (paths.length < per * count) { if (paths.length < per * count) {
return { error: `library too small: ${paths.length} tracks` }; return { error: `library too small: ${paths.length} tracks` };
@@ -671,7 +822,8 @@ async function measurePlaylistOpen(page, client) {
let pl = (existing ?? []).find((p) => (p.Name ?? p.name) === name); let pl = (existing ?? []).find((p) => (p.Name ?? p.name) === name);
if (!pl) { if (!pl) {
const tracks = await ev.call('library.Library.GetAllTracks', [], 60000); // The columnar list (#281): only the paths are needed here.
const tracks = ((await ev.call('library.Library.GetTrackTable', [0], 60000))?.filePath ?? []).map((FilePath) => ({ FilePath }));
const paths = (tracks ?? []).map((t) => t.FilePath); const paths = (tracks ?? []).map((t) => t.FilePath);
if (paths.length < n) return { error: `library too small: ${paths.length}` }; if (paths.length < n) return { error: `library too small: ${paths.length}` };
@@ -1600,7 +1752,8 @@ async function measurePlayerBarPass(page) {
// Stage a loaded track. Deliberately the same first tracks the // Stage a loaded track. Deliberately the same first tracks the
// track-change measurement already played, so this warms no cover // track-change measurement already played, so this warms no cover
// art that a later measurement counts requests for. // art that a later measurement counts requests for.
const tracks = await ev.call('library.Library.GetAllTracks', [], 60000); // The columnar list (#281): only the paths are needed here.
const tracks = ((await ev.call('library.Library.GetTrackTable', [0], 60000))?.filePath ?? []).map((FilePath) => ({ FilePath }));
const paths = (tracks ?? []).slice(0, 4).map((t) => t.FilePath); const paths = (tracks ?? []).slice(0, 4).map((t) => t.FilePath);
if (paths.length < 2) return { error: 'library too small to measure' }; if (paths.length < 2) return { error: 'library too small to measure' };
@@ -1835,6 +1988,12 @@ async function run(label) {
loadWallMs: Date.now() - t0, loadWallMs: Date.now() - t0,
}; };
// First of all: at rest means before any measurement opens a view.
console.log(' memory at rest, then Tracks first open…');
report.memory = await measureMemory(page, client);
await page.evaluate(() => document.dispatchEvent(
new CustomEvent('navigate', { detail: { view: 'home' } }),
));
console.log(' startup…'); console.log(' startup…');
report.startup = await measureStartup(page); report.startup = await measureStartup(page);
// Before anything else navigates: every view's first open has to be // Before anything else navigates: every view's first open has to be
@@ -1897,6 +2056,14 @@ async function run(label) {
const ROWS = [ const ROWS = [
['First contentful paint', (r) => fmt(r.startup.firstContentfulPaintMs, 'ms')], ['First contentful paint', (r) => fmt(r.startup.firstContentfulPaintMs, 'ms')],
['At rest: backend RSS', (r) => fmt(r.memory?.atRest.backend.rssMB, 'MB')],
['At rest: backend peak RSS', (r) => fmt(r.memory?.atRest.backend.peakMB, 'MB')],
['At rest: Go heap held', (r) => fmt(r.memory?.atRest.backend.goHeapHeldMB, 'MB')],
['At rest: JS heap', (r) => fmt(r.memory?.atRest.page.jsHeapMB, 'MB')],
['At rest: binding bytes', (r) => fmt(r.memory?.atRest.totalBindingMB, 'MB')],
['Tracks first open: first row', (r) => fmt(r.memory?.tracksOpen.firstRowMs, 'ms')],
['Tracks open: backend peak RSS', (r) => fmt(r.memory?.tracksOpen.backend.peakMB, 'MB')],
['Tracks open: JS heap', (r) => fmt(r.memory?.tracksOpen.page.jsHeapMB, 'MB')],
['First track row', (r) => fmt(r.startup.firstRowAfterLoadMs, 'ms')], ['First track row', (r) => fmt(r.startup.firstRowAfterLoadMs, 'ms')],
['JS transferred', (r) => fmt(round(r.startup.scriptBytes / 1024), 'kB')], ['JS transferred', (r) => fmt(round(r.startup.scriptBytes / 1024), 'kB')],
['JS evaluated before first paint', (r) => fmt(round((r.startup.scriptBytesBeforePaint ?? 0) / 1024), 'kB')], ['JS evaluated before first paint', (r) => fmt(round((r.startup.scriptBytesBeforePaint ?? 0) / 1024), 'kB')],
@@ -2000,7 +2167,11 @@ function compare(a, b) {
const p = resolve(OUT_DIR, `${l}.json`); const p = resolve(OUT_DIR, `${l}.json`);
if (!existsSync(p)) throw new Error(`no measurement labelled '${l}' at ${p}`); if (!existsSync(p)) throw new Error(`no measurement labelled '${l}' at ${p}`);
return JSON.parse(readFileSync(p, 'utf8')); try {
return JSON.parse(readFileSync(p, 'utf8'));
} catch (err) {
throw new Error(`measurement '${l}' at ${p} is not valid JSON: ${err.message}`);
}
}; };
console.log(table([load(a), load(b)])); console.log(table([load(a), load(b)]));
+80
View File
@@ -0,0 +1,80 @@
/**
* Every bound method the suite names by hand must exist.
*
* A binding call carries only a method id, so a spec that names a method
* the Go side no longer has fails at *runtime*, in whichever spec
* happens to call it, with `unknown bound method name` — and nothing
* before that. `library.Library.GetTracks` was replaced by
* `GetTrackTable` (#281) and four specs kept calling the old name: the
* suite reported twenty failures across transport, bottom-bar and
* reduced-motion specs, none of which mention the library list.
*
* The known names come from `support/method-ids.mjs`, which derives them
* from the generated bindings tree that `make bindings-check` keeps
* current — so this cannot disagree with what the app can answer.
*
* It asserts first that it read something: a glob that matched nothing
* would pass over an empty list.
*/
import { readFileSync, readdirSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
import { test, expect } from '../support/fixtures.js';
import { methodIDs } from '../support/method-ids.mjs';
const here = dirname(fileURLToPath(import.meta.url));
/** The specs, and the harness that calls bindings on their behalf. */
const DIRS = [here, join(here, '..', 'support')];
function sources(): Array<[string, string]> {
const out: Array<[string, string]> = [];
for (const dir of DIRS) {
for (const entry of readdirSync(dir)) {
if (!entry.endsWith('.ts')) continue;
if (entry.endsWith('.spec.ts') && entry === 'binding-names.spec.ts') continue;
const path = join(dir, entry);
out.push([path, readFileSync(path, 'utf8')]);
}
}
return out;
}
/** `'library.Library.GetTrackTable'` — package, service, method. */
const NAMED_BINDING = /'([a-z][A-Za-z0-9]*\.[A-Z]\w*\.[A-Z]\w*)'/g;
/**
* Names that are deliberately not bindings.
*
* `harness.spec.ts` calls an unknown method on purpose, to assert that
* the bridge rejects with a ReferenceError naming it rather than
* hanging — that spec is the reason a bad call is loud.
*/
const DELIBERATE = new Set(['queue.Queue.Nope']);
test('every binding named by a spec exists', () => {
// `methodIDs()` is the id -> name map the recorder names calls with.
const known = new Set(methodIDs().values());
const files = sources();
// A sweep over an empty glob passes and proves nothing.
expect(files.length, 'specs and support files read').toBeGreaterThan(20);
expect(known.size, 'bound method names derived').toBeGreaterThan(100);
const unknown: string[] = [];
for (const [path, source] of files) {
for (const [, name] of source.matchAll(NAMED_BINDING)) {
if (known.has(name!) || DELIBERATE.has(name!)) continue;
unknown.push(`${name} (${path.split('/').pop()})`);
}
}
expect([...new Set(unknown)]).toEqual([]);
});
+8 -10
View File
@@ -1,4 +1,10 @@
import { test, expect, callBinding, NO_QUEUE_SOURCE } from '../support/fixtures.js'; import {
test,
expect,
callBinding,
libraryTracks,
NO_QUEUE_SOURCE,
} from '../support/fixtures.js';
import type { Page } from '@playwright/test'; import type { Page } from '@playwright/test';
/** /**
@@ -39,15 +45,7 @@ const geometry = (app: Page) =>
/** Something has to be playing before the transport draws a seek bar. */ /** Something has to be playing before the transport draws a seek bar. */
async function play(app: Page): Promise<void> { async function play(app: Page): Promise<void> {
const paths = await app.evaluate(async () => { const paths = (await libraryTracks(app)).slice(0, 3).map((t) => t.FilePath);
const tracks = (await window.__yjEvents.call(
'library.Library.GetTracks',
[0],
10_000,
)) as { FilePath: string }[];
return tracks.slice(0, 3).map((t) => t.FilePath);
});
await callBinding(app, 'queue.Queue.SetQueue', [ await callBinding(app, 'queue.Queue.SetQueue', [
paths, paths,
+17
View File
@@ -60,6 +60,23 @@ const PHONE = { width: 424, height: 439 };
const DESKTOP = { width: 1100, height: 800 }; const DESKTOP = { width: 1100, height: 800 };
test.describe('background jobs on a phone', () => { test.describe('background jobs on a phone', () => {
/**
* **State a spec stages is the spec's to clear.** `/__test/emit` writes
* to a store nothing resets, so the event that staged a job is the
* event that clears it — `JobStore` replaces its whole list from every
* snapshot, so `testctl` needs no special case.
*
* **Measured on #168: this does not currently outlive the page.** Every
* test gets a fresh page, and `JobStore.init()` refetches `GetJobs()`
* from a backend registry that `/__test/emit` never writes to, so the
* staged job is gone before the next spec starts. Ownership is stated
* rather than a live leak repaired — the leak needs a page that
* survives its own spec, and there is none today.
*/
test.afterEach(async ({ testctl }) => {
await testctl.emit('JobsChanged', []);
});
test('are shown in the band, without opening anything', async ({ test('are shown in the band, without opening anything', async ({
app, app,
testctl, testctl,
+58
View File
@@ -0,0 +1,58 @@
/**
* #280: launching the app must not load the track list.
*
* Every collection used to be fetched at `DOMContentLoaded` and
* refetched on every invalidation, whichever view was showing. On a
* 26 138-track library the track list was 20.5 MB of that, and encoding
* it cost the backend ~170 MB of transient allocation — for someone
* looking at Home, which draws none of it. Measured on 50 000 tracks:
* 543 MB of backend RSS at rest before, 296 MB after.
*
* The negative assertion is the point, and it needs its complement: "no
* `GetTrackTable`" also holds on a build that fetches nothing at all,
* so the same spec opens Tracks and watches it arrive.
*/
import { test, expect, bindingCalls, navigateTo } from '../support/fixtures.js';
const TRACKS = 'library.Library.GetTrackTable';
/** The seed's default page, which is what a launch lands on. */
const LANDING_VIEW = 'home';
test('launching on Home does not load the track list', async ({ app }) => {
// Past the store's idle warm-up: "nothing was fetched" has to mean
// nothing, not "nothing has happened yet".
await app.waitForTimeout(3_500);
const calls = await bindingCalls(app);
expect(calls, 'the warm-up for the small collections ran').toContain(
'library.Library.GetAlbums',
);
expect(calls, `the track list was fetched at launch (${LANDING_VIEW})`).not.toContain(
TRACKS,
);
});
test('opening Tracks loads it, and only then', async ({ app }) => {
await app.waitForTimeout(1_000);
expect(await bindingCalls(app)).not.toContain(TRACKS);
await navigateTo(app, 'tracks');
await expect(app.getByTestId('track-row').first()).toBeVisible();
expect(await bindingCalls(app)).toContain(TRACKS);
});
test('a hover on the nav item starts the load before the click', async ({ app }) => {
await app.waitForTimeout(1_000);
const nav = app.getByTestId('nav-tracks');
// The pointer entering the item is the whole mechanism; nothing is
// clicked, so the data must arrive because of the hover alone.
await nav.hover();
await expect
.poll(async () => (await bindingCalls(app)).includes(TRACKS))
.toBe(true);
});
+6 -12
View File
@@ -1,6 +1,7 @@
import { import {
test, test,
expect, expect,
libraryTracks,
callBinding, callBinding,
openTheQueue, openTheQueue,
NO_QUEUE_SOURCE, NO_QUEUE_SOURCE,
@@ -56,18 +57,11 @@ async function menuLabels(app: Page): Promise<string[]> {
* the wrong "no link". * the wrong "no link".
*/ */
async function queueThree(app: Page): Promise<void> { async function queueThree(app: Page): Promise<void> {
const paths = await app.evaluate(async () => { const tracks = await libraryTracks(app);
const tracks = (await window.__yjEvents.call( const paths = tracks
'library.Library.GetTracks', .filter((t) => t.Album !== '' && t.ArtistName !== '')
[0], .slice(0, 3)
10_000, .map((t) => t.FilePath);
)) as { FilePath: string; Album: string; ArtistName: string }[];
return tracks
.filter((t) => t.Album !== '' && t.ArtistName !== '')
.slice(0, 3)
.map((t) => t.FilePath);
});
await callBinding(app, 'queue.Queue.SetQueue', [ await callBinding(app, 'queue.Queue.SetQueue', [
paths, paths,
+5 -9
View File
@@ -1,4 +1,4 @@
import { test, expect } from '../support/fixtures.js'; import { test, expect, libraryTracks } from '../support/fixtures.js';
/** /**
* Now Playing on a short screen (#51). * Now Playing on a short screen (#51).
@@ -57,19 +57,15 @@ const REFLOW_AT = 500;
/** Put a track in the player, so the view has art and names to lay out. */ /** Put a track in the player, so the view has art and names to lay out. */
async function stageATrack(page: Page): Promise<void> { async function stageATrack(page: Page): Promise<void> {
await page.evaluate(async () => { const paths = (await libraryTracks(page)).slice(0, 4).map((t) => t.FilePath);
const tracks = (await window.__yjEvents.call(
'library.Library.GetTracks',
[0],
10_000,
)) as { FilePath: string }[];
await page.evaluate(async (paths) => {
await window.__yjEvents.call( await window.__yjEvents.call(
'queue.Queue.SetQueue', 'queue.Queue.SetQueue',
[tracks.slice(0, 4).map((t) => t.FilePath), 0, false, { type: '', id: 0, label: '' }], [paths, 0, false, { type: '', id: 0, label: '' }],
10_000, 10_000,
); );
}); }, paths);
} }
/** Open the full-screen view and wait for the shell to say so. */ /** Open the full-screen view and wait for the shell to say so. */
+2 -5
View File
@@ -1,4 +1,5 @@
import { import {
libraryTracks,
test, test,
expect, expect,
callBinding, callBinding,
@@ -49,11 +50,7 @@ async function rectOf(app: Page, selector: string): Promise<Rect | null> {
* three rectangles. * three rectangles.
*/ */
async function play(app: Page): Promise<void> { async function play(app: Page): Promise<void> {
const tracks = await callBinding<{ FilePath: string; TrackName: string }[]>( const tracks = await libraryTracks(app);
app,
'library.Library.GetTracks',
[0],
);
// `TrackName`, not `Title`: that is what the library model calls it. // `TrackName`, not `Title`: that is what the library model calls it.
const long = tracks.find((t) => t.TrackName === LONG_TRACK); const long = tracks.find((t) => t.TrackName === LONG_TRACK);
+2 -2
View File
@@ -171,8 +171,8 @@ test.describe('search on a phone', () => {
// Attached, not visible: `wa-dialog`'s host is `display: contents`, // Attached, not visible: `wa-dialog`'s host is `display: contents`,
// so the element carrying the testid always reports hidden — what // so the element carrying the testid always reports hidden — what
// is visible is the native `<dialog>` inside it. That awkwardness // is visible is the native `<dialog>` inside it. That awkwardness
// is written down in CLAUDE.md and is why the assertion that this // is why the assertion that this is really up is the role query
// is really up is the role query below. // below.
await expect(dialog).toBeAttached(); await expect(dialog).toBeAttached();
// Named, which `getByRole` can answer and the a11y snapshot cannot // Named, which `getByRole` can answer and the a11y snapshot cannot
+4 -11
View File
@@ -1,4 +1,4 @@
import { test, expect, LONG_TRACK } from '../support/fixtures.js'; import { test, expect, LONG_TRACK, libraryTracks } from '../support/fixtures.js';
/** /**
* The phone shell (plan 016 B2, phase 1). * The phone shell (plan 016 B2, phase 1).
@@ -235,15 +235,8 @@ test.describe('the shell on a phone', () => {
// nothing in particular and picked a 2-second track, which had // nothing in particular and picked a 2-second track, which had
// finished before the assertions ran. The placeholder check below // finished before the assertions ran. The placeholder check below
// is what actually holds the property this test needs. // is what actually holds the property this test needs.
const started = await app.evaluate(async (longTitle) => { const bare = (await libraryTracks(app)).find((t) => t.TrackName === LONG_TRACK);
const tracks = (await window.__yjEvents.call( const started = await app.evaluate(async (bare) => {
'library.Library.GetTracks',
[0],
10_000,
)) as { FilePath: string; TrackName: string }[];
const bare = tracks.find((t) => t.TrackName === longTitle);
if (!bare) return null; if (!bare) return null;
await window.__yjEvents.call( await window.__yjEvents.call(
@@ -254,7 +247,7 @@ test.describe('the shell on a phone', () => {
await window.__yjEvents.call('queue.Queue.Play', [], 5_000); await window.__yjEvents.call('queue.Queue.Play', [], 5_000);
return bare.TrackName; return bare.TrackName;
}, LONG_TRACK); }, bare);
expect(started).toBe(LONG_TRACK); expect(started).toBe(LONG_TRACK);
+5 -9
View File
@@ -1,4 +1,4 @@
import { test, expect } from '../support/fixtures.js'; import { test, expect, libraryTracks } from '../support/fixtures.js';
/** /**
* The phone's transport (#59, #56). * The phone's transport (#59, #56).
@@ -64,19 +64,15 @@ async function sizeOf(
/** Put something in the queue, so the transport has a track to act on. */ /** Put something in the queue, so the transport has a track to act on. */
async function stageATrack(page: Page): Promise<void> { async function stageATrack(page: Page): Promise<void> {
await page.evaluate(async () => { const paths = (await libraryTracks(page)).slice(0, 4).map((t) => t.FilePath);
const tracks = (await window.__yjEvents.call(
'library.Library.GetTracks',
[0],
10_000,
)) as { FilePath: string }[];
await page.evaluate(async (paths) => {
await window.__yjEvents.call( await window.__yjEvents.call(
'queue.Queue.SetQueue', 'queue.Queue.SetQueue',
[tracks.slice(0, 4).map((t) => t.FilePath), 0, false, { type: '', id: 0, label: '' }], [paths, 0, false, { type: '', id: 0, label: '' }],
10_000, 10_000,
); );
}); }, paths);
} }
test.describe('the phone bar carries three controls', () => { test.describe('the phone bar carries three controls', () => {
+16 -1
View File
@@ -40,6 +40,21 @@ const FINISH_TIMEOUT = 60_000;
const libraryCalls = async (app: Page): Promise<string[]> => const libraryCalls = async (app: Page): Promise<string[]> =>
(await bindingCalls(app)).filter((c) => c.startsWith('library.Library.')); (await bindingCalls(app)).filter((c) => c.startsWith('library.Library.'));
/**
* The bindings that fetch a collection.
*
* A prefix test (GetAll*) used to stand in for this, and matched
* exactly one name — GetAllLibrariesWithTrackCounts — so the assertion
* below held whatever the app refetched. The sweep in
* binding-names.spec.ts is what found it.
*/
const COLLECTION_FETCHES = [
'library.Library.GetTrackTable',
'library.Library.GetAlbums',
'library.Library.GetArtists',
'library.Library.GetGenres',
];
/** /**
* Select rows by dispatching on the row rather than clicking it. * Select rows by dispatching on the row rather than clicking it.
* *
@@ -116,7 +131,7 @@ test.describe('a finished track', () => {
const refetched = await libraryCalls(app); const refetched = await libraryCalls(app);
expect( expect(
refetched.filter((c) => c.startsWith('library.Library.GetAll')), refetched.filter((c) => COLLECTION_FETCHES.includes(c)),
'a play refetched a collection', 'a play refetched a collection',
).toEqual([]); ).toEqual([]);
+2 -12
View File
@@ -1,4 +1,5 @@
import { import {
libraryTracks,
test, test,
expect, expect,
callBinding, callBinding,
@@ -30,18 +31,7 @@ async function order(app: Page): Promise<string[]> {
} }
async function queueFourAndOpen(app: Page): Promise<string[]> { async function queueFourAndOpen(app: Page): Promise<string[]> {
const paths: string[] = await app.evaluate(async () => { const paths = (await libraryTracks(app)).slice(0, 4).map((t) => t.FilePath);
// One argument, and 0 means every library: the scoped and
// unscoped list queries collapsed into one when the schema did
// (plan 013 R3), so `GetTracks()` no longer exists to call.
const tracks = await window.__yjEvents.call(
'library.Library.GetTracks',
[0],
10_000,
);
return (tracks as { FilePath: string }[]).slice(0, 4).map((t) => t.FilePath);
});
await callBinding(app, 'queue.Queue.SetQueue', [paths, 0, false, NO_QUEUE_SOURCE]); await callBinding(app, 'queue.Queue.SetQueue', [paths, 0, false, NO_QUEUE_SOURCE]);
+7 -10
View File
@@ -1,4 +1,5 @@
import { import {
libraryTracks,
test, test,
expect, expect,
callBinding, callBinding,
@@ -86,15 +87,11 @@ const selected = (app: Page) =>
* is read back. * is read back.
*/ */
async function queueSixAndOpen(app: Page): Promise<void> { async function queueSixAndOpen(app: Page): Promise<void> {
const paths = await app.evaluate(async (longTitle) => { // `TrackName`, not `Title`: the library model names it after the
// `TrackName`, not `Title`: the library model names it after the // tag, and the *queue* is what calls it `title`.
// tag, and the *queue* is what calls it `title`. const tracks = await libraryTracks(app);
const tracks = (await window.__yjEvents.call( const longTitle = LONG_TRACK;
'library.Library.GetTracks', const paths = (() => {
[0],
10_000,
)) as { FilePath: string; TrackName: string; Album: string }[];
const long = tracks.find((t) => t.TrackName === longTitle); const long = tracks.find((t) => t.TrackName === longTitle);
/** /**
@@ -131,7 +128,7 @@ async function queueSixAndOpen(app: Page): Promise<void> {
long!.FilePath, long!.FilePath,
...rest.slice(3).map((t) => t.FilePath), ...rest.slice(3).map((t) => t.FilePath),
]; ];
}, LONG_TRACK); })();
await callBinding(app, 'queue.Queue.SetQueue', [ await callBinding(app, 'queue.Queue.SetQueue', [
paths, paths,
+4 -14
View File
@@ -2,6 +2,7 @@ import {
test, test,
expect, expect,
callBinding, callBinding,
libraryTracks,
waitForEvent, waitForEvent,
NO_QUEUE_SOURCE, NO_QUEUE_SOURCE,
} from '../support/fixtures.js'; } from '../support/fixtures.js';
@@ -56,20 +57,9 @@ async function playTheLongOne(app: Page): Promise<void> {
window.dispatchEvent(new CustomEvent('yj-scroll-mode-changed')); window.dispatchEvent(new CustomEvent('yj-scroll-mode-changed'));
}); });
const paths: string[] = await app.evaluate(async (needle) => { const paths: string[] = (await libraryTracks(app))
// One argument, and 0 means every library: the scoped and .filter((t) => t.TrackName.startsWith(LONG_TITLE))
// unscoped list queries collapsed into one when the schema did .map((t) => t.FilePath);
// (plan 013 R3), so `GetTracks()` no longer exists to call.
const tracks = await window.__yjEvents.call(
'library.Library.GetTracks',
[0],
10_000,
);
return (tracks as { TrackName: string; FilePath: string }[])
.filter((t) => t.TrackName.startsWith(needle))
.map((t) => t.FilePath);
}, LONG_TITLE);
expect(paths.length).toBeGreaterThan(0); expect(paths.length).toBeGreaterThan(0);
+16
View File
@@ -102,6 +102,22 @@ const collapsed = (page: Page) =>
})); }));
test.describe('the top bar fits the window', () => { test.describe('the top bar fits the window', () => {
/**
* **State a spec stages is the spec's to clear** (#168). `/__test/emit`
* writes to a store nothing resets, and this file stages the widest job
* in the app, so it puts it back — with the same event, since the store
* replaces its whole list from every snapshot.
*
* **Measured: it does not currently outlive the page.** Every test gets
* a fresh page and `JobStore.init()` refetches `GetJobs()` from a
* backend registry `/__test/emit` never writes to, so nothing is being
* repaired here; the rule is stated because it costs one line and the
* leak would need only one spec that keeps a page alive.
*/
test.afterEach(async ({ testctl }) => {
await testctl.emit('JobsChanged', []);
});
/** /**
* The phone's answer, which is not "it fits" (#57). * The phone's answer, which is not "it fits" (#57).
* *
+33
View File
@@ -96,6 +96,39 @@ export async function callBinding<T = unknown>(
) as Promise<T>; ) as Promise<T>;
} }
/** A library track as the specs use it: the path and its names. */
export interface LibraryTrack {
FilePath: string;
TrackName: string;
ArtistName: string;
Album: string;
}
/**
* Every track in the library, in the order the backend lists them.
*
* The list arrives as a columnar `TrackTable` since #281 (repeated
* strings sent once, as indexes into `strings`), so it cannot be read
* as an array of tracks any more. This reads the three columns the
* specs use; `frontend/src/utils/track-table.ts` is the real decoder.
*/
export async function libraryTracks(page: Page): Promise<LibraryTrack[]> {
const t = await callBinding<{
strings: string[];
filePath: string[];
trackName: number[];
artistName: number[];
album: number[];
}>(page, 'library.Library.GetTrackTable', [0]);
return (t.filePath ?? []).map((FilePath, i) => ({
FilePath,
TrackName: t.strings[t.trackName[i]!] ?? '',
ArtistName: t.strings[t.artistName[i]!] ?? '',
Album: t.strings[t.album[i]!] ?? '',
}));
}
/** /**
* The binding calls the *app* made, newest last, as `pkg.Type.Method`. * The binding calls the *app* made, newest last, as `pkg.Type.Method`.
* *
@@ -121,6 +121,12 @@ export interface CandidateFile {
"bitrate"?: number; "bitrate"?: number;
"isAudio": boolean; "isAudio": boolean;
/**
* LengthMillis is the file's duration as the source reports it, or
* 0 when it does not. Soulseek reports it for most audio files.
*/
"lengthMillis"?: number;
/** /**
* expected track position * expected track position
*/ */
@@ -453,10 +459,19 @@ export interface MatchScore {
"albumFit": number; "albumFit": number;
/** /**
* audio files vs expected count * aligned tracks vs expected count
*/ */
"completeness": number; "completeness": number;
/**
* DurationFit is how well the aligned files' lengths agree with the
* expected tracks', and DurationKnown whether enough of them stated
* a length for that to count. When it does not, the score is the
* four text signals alone, exactly as before durations were read.
*/
"durationFit": number;
"durationKnown": boolean;
/** /**
* Anchored records whether an MBID drove this score. Unanchored * Anchored records whether an MBID drove this score. Unanchored
* matches are capped, because there is nothing to be right about. * matches are capped, because there is nothing to be right about.
@@ -18,5 +18,6 @@ export type {
ScanMetrics, ScanMetrics,
ScanWarning, ScanWarning,
Track, Track,
TrackMBIDs TrackMBIDs,
TrackTable
} from "./models.js"; } from "./models.js";
@@ -203,16 +203,12 @@ export function GetTrackMBIDs(filePath: string): $CancellablePromise<$models.Tra
} }
/** /**
* GetTracks returns every track in a library, or in all of them when * GetTrackTable returns every track in a library, or in all of them
* libraryID is 0. * when libraryID is 0, as a TrackTable. An empty library is an empty
* * table, not an error.
* The library id is a parameter rather than a second method because the
* two used to be separate queries, separate bindings and a branch at
* every call site - and the scoped form costs nothing (measured: 23 ms
* against 21 ms over 26k rows).
*/ */
export function GetTracks(libraryID: number): $CancellablePromise<$models.Track[] | null> { export function GetTrackTable(libraryID: number): $CancellablePromise<$models.TrackTable> {
return $Call.ByID(933082923, libraryID); return $Call.ByID(1179690926, libraryID);
} }
/** /**
@@ -222,6 +218,21 @@ export function GetTracksByGenre(genre: string, libraryID: number): $Cancellable
return $Call.ByID(1674220245, genre, libraryID); return $Call.ByID(1674220245, genre, libraryID);
} }
/**
* GetTracksByPaths returns whole tracks for the given file paths, in
* the order asked, dropping any path that is not in the library.
*
* It is how the frontend resolves the tracks a surface is actually
* showing (#279). Track details from the queue, a playlist or a smart
* playlist used to look the path up in the whole library's track
* array, which had to be loaded first — so it was fetched eagerly at
* startup, 20.5 MB at 26k tracks, to answer questions about a handful
* of rows.
*/
export function GetTracksByPaths(paths: string[] | null): $CancellablePromise<$models.Track[] | null> {
return $Call.ByID(3966945290, paths);
}
/** /**
* IsScanActive returns whether a scan is currently running. * IsScanActive returns whether a scan is currently running.
*/ */
@@ -239,3 +239,60 @@ export interface TrackMBIDs {
"releaseGroupMbid": string; "releaseGroupMbid": string;
"artistMbid": string; "artistMbid": string;
} }
/**
* TrackTable is every track in a library as the Tracks view uses it:
* one array per column, and every repeated string stored once (#281).
*
* GetTracks used to answer with one object per track, which at 26 138
* tracks was 20.5 MB of JSON — ~350 bytes a row of key names, four
* cover URLs identical across an album, and artist, album and genre
* strings repeated on every track of the album. Encoding it cost the
* backend ~170 MB of transient allocation and parsing it was the
* WebView's peak. Here the keys appear once, a repeated string is a
* small integer, and the columns the Tracks view does not read are not
* sent at all: LastPlayed and the three larger cover tiers belong to
* the details dialog, which fetches whole tracks by path.
*
* The projection is still trackFromRow's — each row goes through it —
* so this is an encoding of a Track, never a second description of
* one. frontend/src/utils/track-table.ts is the only decoder.
*/
export interface TrackTable {
/**
* Strings holds every distinct string value; a string column holds
* indexes into it. Index 0 is always "".
*/
"strings": string[] | null;
/**
* GenreSets holds every distinct genre list, as indexes into
* Strings; Genre holds an index into it per track.
*/
"genreSets": (number[] | null)[] | null;
"filePath": string[] | null;
"trackName": number[] | null;
"artistName": number[] | null;
"album": number[] | null;
"composer": number[] | null;
"fileType": number[] | null;
"genre": number[] | null;
"artistMbid": number[] | null;
"releaseGroupMbid": number[] | null;
"recordingMbid": number[] | null;
"coverArtSmall": number[] | null;
/**
* LengthMs is Track.TrackLength as the number it encodes.
*/
"lengthMs": number[] | null;
"trackNumber": number[] | null;
"discNumber": number[] | null;
"year": number[] | null;
"sampleRate": number[] | null;
"bitDepth": number[] | null;
"channels": number[] | null;
"bitrate": number[] | null;
"fileSize": number[] | null;
"playCount": number[] | null;
}
@@ -309,6 +309,14 @@ export function SearchLibrary(query: string): $CancellablePromise<$models.Candid
return $Call.ByID(3912116995, query); return $Call.ByID(3912116995, query);
} }
/**
* SuggestSmartPlaylistValues returns values of field present in the
* library that contain needle, for the rule editor's value box.
*/
export function SuggestSmartPlaylistValues(field: string, needle: string): $CancellablePromise<string[] | null> {
return $Call.ByID(3718198115, field, needle);
}
/** /**
* ToggleDefaultPlaylistTrack adds or removes a single track * ToggleDefaultPlaylistTrack adds or removes a single track
* from the default playlist. Returns true if the track is now * from the default playlist. Returns true if the track is now
@@ -56,6 +56,16 @@ export function CycleRepeat(): $CancellablePromise<void> {
return $Call.ByID(3510519482); return $Call.ByID(3510519482);
} }
/**
* DropSourceForPlaylist clears the queue's "Playing from" label when
* its source playlist is deleted. A link back to a playlist that no
* longer exists is worse than none, and the label otherwise survives
* the deletion until the next SetQueue (#249).
*/
export function DropSourceForPlaylist(playlistID: number): $CancellablePromise<void> {
return $Call.ByID(1435106374, playlistID);
}
/** /**
* EmitCurrentState emits the current queue state to the frontend. * EmitCurrentState emits the current queue state to the frontend.
* This is called after the frontend DOM is ready. * This is called after the frontend DOM is ready.
+9 -1
View File
@@ -51,7 +51,15 @@
<div class="content-area"> <div class="content-area">
<main class="main-panel" data-active-view="tracks" data-testid="main-content" id="main-content" <main class="main-panel" data-active-view="tracks" data-testid="main-content" id="main-content"
tabindex="-1"> tabindex="-1">
<track-list></track-list> <!-- The first-paint content of the main panel. It is
`view-hidden` because whether Tracks is the view to
show is not known until `GetDefaultPage` answers, and a
seed that is visible is a seed that is *active* -- which
is what made the whole track list load at launch for a
landing on Home (#280). The first navigation decides:
it unhides and activates this element if the landing
view is Tracks, and otherwise replaces it. -->
<track-list class="view-hidden"></track-list>
</main> </main>
<queue-panel id="queue-panel"></queue-panel> <queue-panel id="queue-panel"></queue-panel>
</div> </div>
+15 -2
View File
@@ -200,13 +200,18 @@ const mainContent = document.getElementById('main-content');
// tracked as currentViewEl, is never hidden: two visible primary views // tracked as currentViewEl, is never hidden: two visible primary views
// splitting the main panel between them regardless of which is // splitting the main panel between them regardless of which is
// selected. // selected.
//
// It is deliberately not activated here. It is markup, not a decision:
// the shell does not yet know which view the launch lands on, and
// activating it starts the Tracks view's work — its data fetch, above
// all (#280) — for a launch that is about to land on Home. The first
// navigation is what activates whichever view it lands on.
if (mainContent) { if (mainContent) {
const initialTrackList = mainContent.querySelector('track-list'); const initialTrackList = mainContent.querySelector('track-list');
if (initialTrackList) { if (initialTrackList) {
viewCache.set('tracks', initialTrackList as HTMLElement); viewCache.set('tracks', initialTrackList as HTMLElement);
currentViewEl = initialTrackList as HTMLElement; currentViewEl = initialTrackList as HTMLElement;
activateView(currentViewEl);
} }
} }
@@ -580,10 +585,14 @@ async function handleNavigate(
} }
default: { default: {
const fallback = document.createElement('div'); const fallback = document.createElement('div');
const message = document.createElement('p');
fallback.style.padding = '1em'; fallback.style.padding = '1em';
fallback.style.color = 'var(--yj-text-secondary, #b3b3b3)'; fallback.style.color = 'var(--yj-text-secondary, #b3b3b3)';
fallback.innerHTML = `<p>Coming soon: ${view}</p>`; // textContent, not innerHTML: `view` comes from a navigation
// detail, which is app-supplied but not app-owned.
message.textContent = `Coming soon: ${view}`;
fallback.append(message);
mainContent.appendChild(fallback); mainContent.appendChild(fallback);
currentDetailEl = fallback; currentDetailEl = fallback;
} }
@@ -620,6 +629,10 @@ function warmViewChunks(): void {
/** requestIdleCallback where it exists; WebKit2GTK does not have it. */ /** requestIdleCallback where it exists; WebKit2GTK does not have it. */
function schedule(fn: () => void): void { function schedule(fn: () => void): void {
// SAFETY: `requestIdleCallback` is not in this project's DOM lib
// types, so it is read through an optional-property shape; the
// property is either absent (undefined) or the browser's own
// function, which is what the guard below checks.
const ric = ( const ric = (
window as unknown as { window as unknown as {
requestIdleCallback?: (cb: () => void) => number; requestIdleCallback?: (cb: () => void) => number;
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M9.4 233.4c-12.5 12.5-12.5 32.8 0 45.3l192 192c12.5 12.5 32.8 12.5 45.3 0s12.5-32.8 0-45.3L77.3 256 246.6 86.6c12.5-12.5 12.5-32.8 0-45.3s-32.8-12.5-45.3 0l-192 192z"/></svg>

After

Width:  |  Height:  |  Size: 476 B

@@ -7,6 +7,11 @@ import { designTokens } from '../../styles/tokens.css';
* keyboard navigation. Accepts a flat `options` string array, filters as * keyboard navigation. Accepts a flat `options` string array, filters as
* the user types, and emits `combobox-change` when a value is selected. * the user types, and emits `combobox-change` when a value is selected.
* *
* It also emits `combobox-input` with `{ text }` whenever the text it
* filters by changes (typing, and the reset to empty on focus), so a
* host whose options are too many to hand over at once can fetch the
* ones matching what was typed instead (#279).
*
* Key implementation detail: option `<li>` elements use `@mousedown` with * Key implementation detail: option `<li>` elements use `@mousedown` with
* `e.preventDefault()` so that the input's `blur` event does not close the * `e.preventDefault()` so that the input's `blur` event does not close the
* dropdown before the click registers. * dropdown before the click registers.
@@ -196,6 +201,7 @@ export class YjCombobox extends LitElement {
this.filterText = input.value; this.filterText = input.value;
this.open = true; this.open = true;
this.highlightedIndex = -1; this.highlightedIndex = -1;
this.announceInput();
} }
private handleFocus() { private handleFocus() {
@@ -203,6 +209,17 @@ export class YjCombobox extends LitElement {
this.filterText = ''; this.filterText = '';
this.open = true; this.open = true;
this.highlightedIndex = -1; this.highlightedIndex = -1;
this.announceInput();
}
private announceInput() {
this.dispatchEvent(
new CustomEvent('combobox-input', {
detail: { text: this.filterText },
bubbles: true,
composed: true,
}),
);
} }
private handleBlur() { private handleBlur() {
@@ -60,6 +60,7 @@ import '@awesome.me/webawesome/dist/components/dropdown-item/dropdown-item.js';
import { dictByName } from '@utils/binding'; import { dictByName } from '@utils/binding';
import type { TrackDetails } from '@components/track-details/track-details.js'; import type { TrackDetails } from '@components/track-details/track-details.js';
import { showTrackDetailsForPath } from '@utils/track-details-opener.js'; import { showTrackDetailsForPath } from '@utils/track-details-opener.js';
import { openMusicBrainz } from '@utils/external-link';
import '@components/playlist-picker/playlist-picker.js'; import '@components/playlist-picker/playlist-picker.js';
import { import {
ICON_CAN_REQUEST, ICON_CAN_REQUEST,
@@ -2942,7 +2943,7 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
if (!track?.mbid) return; if (!track?.mbid) return;
window.open(`https://musicbrainz.org/recording/${track.mbid}`, '_blank', 'noopener'); openMusicBrainz(`/recording/${track.mbid}`);
} }
/** /**
@@ -4,6 +4,8 @@ import { customElement, property, state, query } from 'lit/decorators.js';
import { classMap } from 'lit/directives/class-map.js'; import { classMap } from 'lit/directives/class-map.js';
import { designTokens } from '../../styles/tokens.css'; import { designTokens } from '../../styles/tokens.css';
import { backButton } from '../../styles/back-button.css'; import { backButton } from '../../styles/back-button.css';
import { albumCardStyles } from '../../styles/album-card.css';
import '../scroll-row/scroll-row.js';
import { import {
LookupArtist, LookupArtist,
BrowseReleaseGroups, BrowseReleaseGroups,
@@ -46,11 +48,8 @@ import {
libraryStatusFor, libraryStatusFor,
toggleRequest, toggleRequest,
} from '@utils/library-status'; } from '@utils/library-status';
import { import { isOwned, ownershipLabel } from '@utils/ownership';
isOwned, import { openMusicBrainz } from '@utils/external-link';
ownershipLabel,
unownedStyles,
} from '@utils/ownership';
import { completenessStore } from '@store/completeness-store'; import { completenessStore } from '@store/completeness-store';
import '../catalog-scope-notice/catalog-scope-notice.js'; import '../catalog-scope-notice/catalog-scope-notice.js';
import type { CatalogScope } from '../catalog-scope-notice/catalog-scope-notice.js'; import type { CatalogScope } from '../catalog-scope-notice/catalog-scope-notice.js';
@@ -62,6 +61,7 @@ import {
ContextMenuController, ContextMenuController,
contextMenuStyles, contextMenuStyles,
isContextMenuKey, isContextMenuKey,
MenuKeyboard,
} from '@utils/context-menu-controller.js'; } from '@utils/context-menu-controller.js';
import type { ContextMenuHost, MenuTarget } from '@utils/context-menu-controller.js'; import type { ContextMenuHost, MenuTarget } from '@utils/context-menu-controller.js';
import '@awesome.me/webawesome/dist/components/popup/popup.js'; import '@awesome.me/webawesome/dist/components/popup/popup.js';
@@ -187,11 +187,11 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
@state() private topReleasesExpanded = false; @state() private topReleasesExpanded = false;
private topSectionStacked = false; private topSectionStacked = false;
private topSectionObserver?: ResizeObserver; private topSectionObserver?: ResizeObserver;
@state() private expandedDiscoGroups = new Set<string>();
/** Number of album cards that fit in one row of the discography grid. */ /** Whether the Play button's Shuffle dropdown is up. */
@state() private discoRowSize = 5; @state() private playMenuOpen = false;
private discoObserver?: ResizeObserver; private playMenuKeyboard = new MenuKeyboard(() => this.closePlayMenu());
@state() private similarExpanded = false; private playOutsideAttached = false;
/* ── Release prefetch ── */ /* ── Release prefetch ── */
@@ -221,6 +221,12 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
@query('#context-menu') @query('#context-menu')
private contextMenuPopup!: MenuSurface; private contextMenuPopup!: MenuSurface;
@query('.play-menu-button')
private playMenuButton?: HTMLButtonElement;
@query('#artist-play-menu')
private playMenuPanel?: HTMLElement;
@query('#playlist-submenu') @query('#playlist-submenu')
private playlistSubmenuPopup?: WaPopup; private playlistSubmenuPopup?: WaPopup;
@@ -271,7 +277,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
backButton, backButton,
exploreLinkStyles, exploreLinkStyles,
contextMenuStyles, contextMenuStyles,
unownedStyles, albumCardStyles,
css` css`
:host { :host {
display: flex; display: flex;
@@ -319,10 +325,45 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
object-fit: cover; object-fit: cover;
} }
.artist-follow { .artist-actions {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
margin-top: 10px; margin-top: 10px;
} }
/* The Play button and its caret are one control, so they
are one box: no gap between them, and the caret carries
the same filled appearance as the button it extends. */
.play-split {
display: inline-flex;
align-items: stretch;
}
.play-menu-button {
display: inline-flex;
align-items: center;
justify-content: center;
width: 28px;
padding: 0;
border: none;
border-left: 1px solid rgba(0, 0, 0, 0.25);
border-radius: 0 6px 6px 0;
background: var(--yj-accent, #ffd43b);
color: var(--yj-accent-fg, #000);
cursor: pointer;
}
.play-menu-button:hover {
filter: brightness(1.1);
}
.play-menu-button:focus-visible {
outline: 2px solid var(--yj-accent-text, #ffd43b);
outline-offset: 2px;
}
.artist-info { .artist-info {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
@@ -331,7 +372,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
} }
.artist-title { .artist-title {
font-size: 24px; font-size: 28px;
font-weight: 700; font-weight: 700;
color: var(--yj-text-primary, #fff); color: var(--yj-text-primary, #fff);
white-space: nowrap; white-space: nowrap;
@@ -359,6 +400,13 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
flex-wrap: wrap; flex-wrap: wrap;
} }
/* The listen count is a headline number, not metadata, so
it sits a size above the type/country line. */
.artist-listens {
font-size: var(--yj-text-lg);
color: var(--yj-text-secondary, #b3b3b3);
}
.meta-separator { .meta-separator {
opacity: 0.4; opacity: 0.4;
} }
@@ -446,23 +494,6 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
outline-offset: -2px; outline-offset: -2px;
} }
.artist-play-actions {
margin-top: 10px;
display: flex;
gap: 8px;
align-items: center;
flex-wrap: wrap;
}
.track-rank {
width: 24px;
text-align: right;
color: var(--yj-text-tertiary, #888);
font-size: var(--yj-text-md);
font-variant-numeric: tabular-nums;
flex-shrink: 0;
}
.track-art { .track-art {
width: 32px; width: 32px;
height: 32px; height: 32px;
@@ -489,6 +520,52 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
opacity: 0.5; opacity: 0.5;
} }
/* Play where you own the track, the request badge where you
do not — over the artwork rather than at the end of the
row, where it was a badge beside a row you can already
double-click. */
.track-art-overlay {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
border-radius: 4px;
background: rgba(0, 0, 0, 0.55);
visibility: hidden;
opacity: 0;
transition: opacity 0.15s ease, visibility 0.15s ease;
}
.track-art-play {
display: flex;
align-items: center;
justify-content: center;
padding: 0;
border: none;
background: none;
color: #fff;
font-size: 14px;
cursor: pointer;
}
@media (hover: hover) and (pointer: fine) {
.track-item:hover .track-art-overlay,
.track-item:focus-within .track-art-overlay {
visibility: visible;
opacity: 1;
}
}
/* No hover means no double-click either, so the overlay is
the only route to playing a top track and must be there. */
@media not all and (hover: hover) {
.track-art-overlay {
visibility: visible;
opacity: 1;
}
}
.track-info { .track-info {
flex: 1; flex: 1;
min-width: 0; min-width: 0;
@@ -522,7 +599,6 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
.track-item library-status-indicator { .track-item library-status-indicator {
flex-shrink: 0; flex-shrink: 0;
} }
/* ── Top section (tracks + releases side-by-side) ── */ /* ── Top section (tracks + releases side-by-side) ── */
.top-section-wrapper { .top-section-wrapper {
container-type: inline-size; container-type: inline-size;
@@ -754,8 +830,30 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
white-space: nowrap; white-space: nowrap;
} }
.top-release-meta library-status-indicator { .top-release-art .album-card-badge {
flex-shrink: 0; position: absolute;
top: 4px;
left: 4px;
z-index: 1;
display: flex;
visibility: hidden;
opacity: 0;
transition: opacity 0.15s ease, visibility 0.15s ease;
}
@media (hover: hover) and (pointer: fine) {
.top-release-card:hover .album-card-badge,
.top-release-card:focus-within .album-card-badge {
visibility: visible;
opacity: 1;
}
}
@media not all and (hover: hover) {
.top-release-art .album-card-badge {
visibility: visible;
opacity: 1;
}
} }
@@ -773,150 +871,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
margin: 0; margin: 0;
} }
.album-grid {
display: grid;
grid-template-columns: repeat(auto-fill, 140px);
gap: 16px;
}
.album-grid.collapsed {
grid-template-rows: 1fr;
overflow: hidden;
}
.disco-toggle {
display: flex;
align-items: center;
justify-content: center;
gap: 6px;
padding: 4px 10px;
margin-top: 4px;
border: none;
border-radius: 6px;
background: var(--yj-bg-overlay, rgba(255, 255, 255, 0.06));
color: var(--yj-text-secondary, #b3b3b3);
font-size: var(--yj-text-xs);
cursor: pointer;
transition: background 0.15s ease, color 0.15s ease;
width: 100%;
}
.disco-toggle:hover {
background: var(--yj-bg-hover, rgba(255, 255, 255, 0.1));
color: var(--yj-text-primary, #fff);
}
.disco-toggle wa-icon {
font-size: 11px;
transition: transform 0.2s ease;
}
.disco-toggle[aria-expanded='true'] wa-icon {
transform: rotate(180deg);
}
.album-card {
display: flex;
flex-direction: column;
gap: 6px;
padding: 8px;
border-radius: 8px;
cursor: pointer;
transition: background 0.15s ease;
}
.album-card:hover {
background: var(
--yj-bg-overlay,
rgba(255, 255, 255, 0.06)
);
}
.album-card:active {
transform: scale(0.97);
}
.album-art-container {
width: 100%;
aspect-ratio: 1;
border-radius: 4px;
overflow: hidden;
flex-shrink: 0;
position: relative;
background: var(--yj-bg-overlay, rgba(255, 255, 255, 0.06));
}
.album-art-container img {
width: 100%;
height: 100%;
object-fit: cover;
display: block;
border-radius: 4px;
}
.album-art-fallback {
display: flex;
align-items: center;
justify-content: center;
width: 100%;
height: 100%;
position: absolute;
inset: 0;
}
.album-art-fallback wa-icon {
color: var(--yj-text-tertiary, #888);
font-size: 24px;
opacity: 0.5;
}
.album-title {
font-weight: 500;
color: var(--yj-text-primary, #fff);
font-size: var(--yj-text-sm);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.album-meta {
display: flex;
align-items: center;
justify-content: space-between;
gap: 6px;
color: var(--yj-text-tertiary, #888);
font-size: var(--yj-text-xs);
min-height: 20px;
}
.album-meta-text {
display: flex;
align-items: center;
gap: 6px;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.album-meta library-status-indicator {
flex-shrink: 0;
margin-left: auto;
}
/* ── Similar artists ── */ /* ── Similar artists ── */
.similar-row {
display: grid;
grid-template-columns: repeat(auto-fill, 140px);
gap: 16px;
overflow: hidden;
}
.similar-row.collapsed {
grid-template-rows: 1fr;
overflow: hidden;
}
.similar-artist-card { .similar-artist-card {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
@@ -926,6 +881,9 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
border-radius: 8px; border-radius: 8px;
cursor: pointer; cursor: pointer;
text-align: center; text-align: center;
width: 120px;
box-sizing: border-box;
flex-shrink: 0;
transition: background 0.15s ease; transition: background 0.15s ease;
} }
@@ -1056,7 +1014,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
this.unsubSimilarReady?.(); this.unsubSimilarReady?.();
if (this.discogFallbackTimer) clearTimeout(this.discogFallbackTimer); if (this.discogFallbackTimer) clearTimeout(this.discogFallbackTimer);
this.topSectionObserver?.disconnect(); this.topSectionObserver?.disconnect();
this.discoObserver?.disconnect(); this.detachPlayOutsideClose();
} }
/** /**
@@ -1083,17 +1041,14 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
protected override firstUpdated() { protected override firstUpdated() {
this.observeTopSectionWidth(); this.observeTopSectionWidth();
this.observeDiscoWidth();
} }
protected override updated() { protected override updated() {
// Re-attach observers if elements appeared after initial render. // Re-attach the observer if the section appeared after initial
// render.
if (!this.topSectionObserver) { if (!this.topSectionObserver) {
this.observeTopSectionWidth(); this.observeTopSectionWidth();
} }
if (!this.discoObserver) {
this.observeDiscoWidth();
}
} }
/** /**
@@ -1126,32 +1081,6 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
this.topSectionObserver.observe(wrapper); this.topSectionObserver.observe(wrapper);
} }
/**
* Watch the .content width and compute how many album cards
* fit in one row of the discography grid.
* Grid uses: repeat(auto-fill, minmax(140px, 1fr)) with 16px gap
* and album-card has 8px padding on each side.
*/
private observeDiscoWidth() {
const content = this.renderRoot.querySelector('.content');
if (!content) return;
const CARD_MIN = 140;
const GAP = 16;
this.discoObserver = new ResizeObserver((entries) => {
for (const entry of entries) {
const width = entry.contentBoxSize?.[0]?.inlineSize ?? entry.contentRect.width;
const cols = Math.max(1, Math.floor((width + GAP) / (CARD_MIN + GAP)));
if (cols !== this.discoRowSize) {
this.discoRowSize = cols;
}
}
});
this.discoObserver.observe(content);
}
/* ── Data Loading ── */ /* ── Data Loading ── */
private async loadAllData() { private async loadAllData() {
@@ -1862,6 +1791,8 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
} catch { } catch {
// No image — letter avatar stays. // No image — letter avatar stays.
} }
return undefined;
}), }),
); );
} }
@@ -1999,6 +1930,65 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
} }
} }
/* ── Play / Shuffle split button ── */
/**
* Open the Play button's Shuffle dropdown.
*
* `page-header`'s overflow menu one control over: the same
* `MenuKeyboard`, the same document-level outside-close, and the
* same `menu-surface`, so the phone gets the bottom sheet rather
* than a popup that Chrome 113 clips.
*/
private togglePlayMenu = (): void => {
if (this.playMenuOpen) {
this.closePlayMenu();
return;
}
this.playMenuOpen = true;
void this.updateComplete.then(() => {
if (!this.playMenuOpen) return;
this.playMenuKeyboard.open(
this.playMenuPanel ?? null,
this.playMenuButton ?? null,
);
this.attachPlayOutsideClose();
});
};
private closePlayMenu = (): void => {
if (!this.playMenuOpen) return;
this.detachPlayOutsideClose();
this.playMenuKeyboard.close();
this.playMenuOpen = false;
};
private onPlayOutsideDown = (e: Event): void => {
if (e.composedPath().includes(this.playMenuPanel as EventTarget)) return;
if (e.composedPath().includes(this.playMenuButton as EventTarget)) return;
this.closePlayMenu();
};
private attachPlayOutsideClose(): void {
if (this.playOutsideAttached) return;
this.playOutsideAttached = true;
document.addEventListener('mousedown', this.onPlayOutsideDown, true);
}
private detachPlayOutsideClose(): void {
if (!this.playOutsideAttached) return;
this.playOutsideAttached = false;
document.removeEventListener('mousedown', this.onPlayOutsideDown, true);
}
/** /**
* File path for one top track, resolved by recording MBID — the * File path for one top track, resolved by recording MBID — the
* same key `localId` was set from. Works whether or not the * same key `localId` was set from. Works whether or not the
@@ -2240,11 +2230,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
if (!release?.mbid) return; if (!release?.mbid) return;
window.open( openMusicBrainz(`/release-group/${release.mbid}`);
`https://musicbrainz.org/release-group/${release.mbid}`,
'_blank',
'noopener',
);
} }
private onContextMenuAction( private onContextMenuAction(
@@ -2358,7 +2344,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
if (!track?.recordingMbid) return; if (!track?.recordingMbid) return;
window.open(`https://musicbrainz.org/recording/${track.recordingMbid}`, '_blank', 'noopener'); openMusicBrainz(`/recording/${track.recordingMbid}`);
} }
/* ── Navigation ── */ /* ── Navigation ── */
@@ -2538,10 +2524,12 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
: nothing} : nothing}
${this.renderArtistMeta()} ${this.renderArtistMeta()}
${this.artist?.popularity && this.artist.popularity > 0 ${this.artist?.popularity && this.artist.popularity > 0
? html`<span class="artist-meta">${formatListenCount(this.artist.popularity)} plays on ListenBrainz</span>` ? html`<span class="artist-listens">${formatListenCount(this.artist.popularity)} plays on ListenBrainz</span>`
: nothing} : nothing}
${this.renderPlayLibraryAction()} <div class="artist-actions">
${this.renderFollowAction()} ${this.renderPlayLibraryAction()}
${this.renderFollowAction()}
</div>
</div> </div>
</div> </div>
<div class="content"> <div class="content">
@@ -2572,25 +2560,53 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
if (this.ownedLocalAlbumIds().length === 0) return nothing; if (this.ownedLocalAlbumIds().length === 0) return nothing;
return html` return html`
<div class="artist-play-actions"> <div class="play-split">
<wa-button <wa-button
size="small" size="small"
appearance="filled" appearance="filled"
data-testid="artist-play-library" data-testid="artist-play-library"
title="Play library tracks"
@click=${() => void this.playLibraryTracks(false)} @click=${() => void this.playLibraryTracks(false)}
> >
<wa-icon slot="start" name="play"></wa-icon> <wa-icon slot="start" name="play"></wa-icon>
Play library tracks Play
</wa-button> </wa-button>
<wa-button <menu-surface
size="small" placement="bottom-start"
appearance="outlined" .active=${this.playMenuOpen}
data-testid="artist-shuffle-library" @menu-dismiss=${this.closePlayMenu}
@click=${() => void this.playLibraryTracks(true)}
> >
<wa-icon slot="start" name="shuffle"></wa-icon> <button
Shuffle slot="anchor"
</wa-button> class="play-menu-button"
type="button"
data-testid="artist-play-menu"
aria-label="More play options"
aria-haspopup="menu"
aria-expanded=${this.playMenuOpen ? 'true' : 'false'}
aria-controls="artist-play-menu"
@click=${this.togglePlayMenu}
>
<wa-icon name="chevron-down"></wa-icon>
</button>
<div
id="artist-play-menu"
class="context-menu-panel"
role="menu"
aria-label="Play options"
>
<wa-dropdown-item
data-testid="artist-shuffle-library"
@click=${() => {
this.closePlayMenu();
void this.playLibraryTracks(true);
}}
>
<wa-icon slot="icon" name="shuffle"></wa-icon>
Shuffle
</wa-dropdown-item>
</div>
</menu-surface>
</div> </div>
`; `;
} }
@@ -2786,25 +2802,27 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
const request = downloadStore.requestFor(this.artistMBID); const request = downloadStore.requestFor(this.artistMBID);
return html` return html`
<div class="artist-follow"> <wa-button
<wa-button size="small"
size="small" appearance=${request ? 'filled' : 'outlined'}
appearance=${request ? 'filled' : 'outlined'} data-testid="artist-follow"
@click=${() => void this.toggleFollow(request?.id)} title=${request
> ? 'Following this artist'
<!-- This was bookmark-check, which is not in : 'Follow this artist for new releases'}
names.txt and so has rendered the missing-icon @click=${() => void this.toggleFollow(request?.id)}
fallback — a circled question mark — on every >
followed artist since it was written. A <!-- This was bookmark-check, which is not in
backtick around that name would end this names.txt and so has rendered the missing-icon
template literal, which is why there is none. --> fallback — a circled question mark — on every
<wa-icon followed artist since it was written. A
slot="start" backtick around that name would end this
name=${request ? ICON_REQUESTED : ICON_CAN_REQUEST} template literal, which is why there is none. -->
></wa-icon> <wa-icon
${request ? 'Following' : 'Follow for new releases'} slot="start"
</wa-button> name=${request ? ICON_REQUESTED : ICON_CAN_REQUEST}
</div> ></wa-icon>
${request ? 'Following' : 'Follow'}
</wa-button>
`; `;
} }
@@ -2888,16 +2906,6 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
this.topReleasesExpanded = !this.topReleasesExpanded; this.topReleasesExpanded = !this.topReleasesExpanded;
} }
private toggleDiscoGroup(type: string) {
const next = new Set(this.expandedDiscoGroups);
if (next.has(type)) {
next.delete(type);
} else {
next.add(type);
}
this.expandedDiscoGroups = next;
}
private renderTopSection() { private renderTopSection() {
const hasTracks = !this.loadingTracks && this.topTracks.length > 0; const hasTracks = !this.loadingTracks && this.topTracks.length > 0;
const hasReleases = !this.loadingTopReleases && this.topReleaseGroups.length > 0; const hasReleases = !this.loadingTopReleases && this.topReleaseGroups.length > 0;
@@ -2971,6 +2979,31 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
}} />` }} />`
: html`<wa-icon name="compact-disc"></wa-icon>`; : html`<wa-icon name="compact-disc"></wa-icon>`;
})()} })()}
<!-- Over the artwork, not beside the
row: play where you own it, the
request badge where you do not. -->
<div class="track-art-overlay">
${owned
? html`<button
class="track-art-play"
type="button"
aria-label=${`Play ${t.trackName}`}
@click=${(e: Event) => {
e.stopPropagation();
void this.playTrack(t);
}}
>
<wa-icon name="play"></wa-icon>
</button>`
: html`<library-status-indicator
status=${libraryStatusFor(false, t.recordingMbid)}
entity-type="track"
label=${t.trackName}
request-mbid=${t.recordingMbid}
request-artist=${t.artistName ?? ''}
size="18"
></library-status-indicator>`}
</div>
</div> </div>
<div class="track-info"> <div class="track-info">
<div class="track-title">${trackLink(t.trackName, t.releaseName, t.releaseGroupMbid ?? '', t.recordingMbid)}</div> <div class="track-title">${trackLink(t.trackName, t.releaseName, t.releaseGroupMbid ?? '', t.recordingMbid)}</div>
@@ -2979,15 +3012,6 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
<span class="track-listens"> <span class="track-listens">
${formatListenCount(t.totalListenCount)} plays ${formatListenCount(t.totalListenCount)} plays
</span> </span>
${owned
? nothing
: html`<library-status-indicator
status=${libraryStatusFor(false, t.recordingMbid)}
entity-type="track"
label=${t.trackName}
request-mbid=${t.recordingMbid}
request-artist=${t.artistName ?? ''}
></library-status-indicator>`}
</div> </div>
`; `;
})} })}
@@ -3093,6 +3117,18 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
<div class="album-art-fallback" style="${artURL ? 'display: none' : ''}"> <div class="album-art-fallback" style="${artURL ? 'display: none' : ''}">
<wa-icon name="compact-disc"></wa-icon> <wa-icon name="compact-disc"></wa-icon>
</div> </div>
<div class="album-card-badge">
<library-status-indicator
status=${badge.status}
owned=${badge.owned}
expected=${badge.expected}
entity-type="album"
label=${rg.title}
request-mbid=${rg.releaseGroupMbid}
request-artist=${this.artist?.name ?? ''}
size="21"
></library-status-indicator>
</div>
</div> </div>
<div class="top-release-text"> <div class="top-release-text">
<div class="top-release-title" title="${rg.title}"> <div class="top-release-title" title="${rg.title}">
@@ -3102,18 +3138,6 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
<div class="top-release-meta-text"> <div class="top-release-meta-text">
${rg.date ? html`<span>${extractYear(rg.date)}</span>` : nothing} ${rg.date ? html`<span>${extractYear(rg.date)}</span>` : nothing}
</div> </div>
${badge.status === 'in-library'
? nothing
: html`<library-status-indicator
status=${badge.status}
owned=${badge.owned}
expected=${badge.expected}
entity-type="album"
label=${rg.title}
request-mbid=${rg.releaseGroupMbid}
request-artist=${this.artist?.name ?? ''}
size="18"
></library-status-indicator>`}
</div> </div>
</div> </div>
</div> </div>
@@ -3164,37 +3188,16 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
<section> <section>
<h3 class="section-header">Discography</h3> <h3 class="section-header">Discography</h3>
${groups.map( ${groups.map(
(g) => { (g) => html`
const isExpanded = this.expandedDiscoGroups.has(g.type); <div class="disco-group">
const rowSize = this.discoRowSize; <h4 class="disco-type-header">
const showToggle = g.items.length > rowSize; ${g.type === 'Other' ? 'Other Releases' : g.type.endsWith('s') ? g.type : `${g.type}s`}
const visibleItems = isExpanded ? g.items : g.items.slice(0, rowSize); </h4>
<scroll-row>
return html` ${g.items.map((rg) => this.renderAlbumCard(rg))}
<div class="disco-group"> </scroll-row>
<h4 class="disco-type-header"> </div>
${g.type === 'Other' ? 'Other Releases' : g.type.endsWith('s') ? g.type : `${g.type}s`} `,
</h4>
<div class="album-grid">
${visibleItems.map((rg) => this.renderAlbumCard(rg))}
</div>
${showToggle
? html`
<button
class="disco-toggle"
aria-expanded="${isExpanded}"
@click=${() => this.toggleDiscoGroup(g.type)}
>
${isExpanded
? 'Show less'
: `Show all ${g.items.length}`}
<wa-icon name="chevron-down"></wa-icon>
</button>
`
: nothing}
</div>
`;
},
)} )}
</section> </section>
`; `;
@@ -3234,23 +3237,25 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
<div class="album-art-fallback" style="${artURL ? 'display: none' : ''}"> <div class="album-art-fallback" style="${artURL ? 'display: none' : ''}">
<wa-icon name="compact-disc"></wa-icon> <wa-icon name="compact-disc"></wa-icon>
</div> </div>
<div class="album-card-badge">
<library-status-indicator
status=${badge.status}
owned=${badge.owned}
expected=${badge.expected}
entity-type="album"
label=${rg.title}
request-mbid=${rg.mbid}
request-artist=${this.artist?.name ?? ''}
size="23"
></library-status-indicator>
</div>
</div> </div>
<div class="album-title" title="${rg.title}">${rg.title}</div> <div class="album-title" title="${rg.title}">${rg.title}</div>
<div class="album-artist">${rg.artistCredit ?? ''}</div>
<div class="album-meta"> <div class="album-meta">
<div class="album-meta-text"> <div class="album-meta-text">
${year ? html`<span>${year}</span>` : nothing} ${year ? html`<span>${year}</span>` : nothing}
</div> </div>
${badge.status === 'in-library'
? nothing
: html`<library-status-indicator
status=${badge.status}
owned=${badge.owned}
expected=${badge.expected}
entity-type="album"
label=${rg.title}
request-mbid=${rg.mbid}
request-artist=${this.artist?.name ?? ''}
></library-status-indicator>`}
</div> </div>
</div> </div>
`; `;
@@ -3266,15 +3271,12 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
// Cap the similar-artists list at 10 to avoid a very long list. // Cap the similar-artists list at 10 to avoid a very long list.
const maxSimilar = 10; const maxSimilar = 10;
const artists = this.similarArtists.slice(0, maxSimilar); const artists = this.similarArtists.slice(0, maxSimilar);
const showToggle = artists.length > this.discoRowSize;
const collapsed = !this.similarExpanded && showToggle;
const visible = collapsed ? artists.slice(0, this.discoRowSize) : artists;
return html` return html`
<section> <section>
<h3 class="section-header">Similar Artists</h3> <h3 class="section-header">Similar Artists</h3>
<div class="similar-row ${collapsed ? 'collapsed' : ''}"> <scroll-row>
${visible.map((a) => { ${artists.map((a) => {
const imgURL = this.similarImageURLs.get(a.artistMbid); const imgURL = this.similarImageURLs.get(a.artistMbid);
return html` return html`
<div <div
@@ -3313,21 +3315,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
</div> </div>
`; `;
})} })}
</div> </scroll-row>
${showToggle
? html`
<button
class="disco-toggle"
aria-expanded="${this.similarExpanded}"
@click=${() => { this.similarExpanded = !this.similarExpanded; }}
>
${this.similarExpanded
? 'Show less'
: `Show all ${artists.length}`}
<wa-icon name="chevron-down"></wa-icon>
</button>
`
: nothing}
</section> </section>
`; `;
} }
@@ -1,10 +1,7 @@
import { avatarBackground } from '@utils/avatar-color'; import { avatarBackground } from '@utils/avatar-color';
import { albumBadgeFor, libraryStatusFor } from '@utils/library-status'; import { albumBadgeFor, libraryStatusFor } from '@utils/library-status';
import { import { isOwned, ownershipLabel } from '@utils/ownership';
isOwned, import { openMusicBrainz } from '@utils/external-link';
ownershipLabel,
unownedStyles,
} from '@utils/ownership';
import { completenessStore } from '@store/completeness-store'; import { completenessStore } from '@store/completeness-store';
import { downloadStore } from '@store/download-store'; import { downloadStore } from '@store/download-store';
import { LitElement, html, css, nothing } from 'lit'; import { LitElement, html, css, nothing } from 'lit';
@@ -13,6 +10,8 @@ import { classMap } from 'lit/directives/class-map.js';
import '@components/page-header/page-header'; import '@components/page-header/page-header';
import { designTokens } from '../../styles/tokens.css'; import { designTokens } from '../../styles/tokens.css';
import { srOnly } from '../../styles/sr-only.css'; import { srOnly } from '../../styles/sr-only.css';
import { albumCardStyles } from '../../styles/album-card.css';
import '../scroll-row/scroll-row.js';
import { SearchLocal, SearchLyrics, GetThumbnail, GetThumbnails, GetArtistImageURL, GetArtistImagesCachedPaths, GetExploreShelves, RecordSearchClick } from '@go/explore/service.js'; import { SearchLocal, SearchLyrics, GetThumbnail, GetThumbnails, GetArtistImageURL, GetArtistImagesCachedPaths, GetExploreShelves, RecordSearchClick } from '@go/explore/service.js';
import { GetFilePathsByAlbums, GetFilePathsByRecordingMBIDs } from '@go/library/library.js'; import { GetFilePathsByAlbums, GetFilePathsByRecordingMBIDs } from '@go/library/library.js';
import { EventsOn } from '@runtime/runtime'; import { EventsOn } from '@runtime/runtime';
@@ -253,7 +252,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
srOnly, srOnly,
exploreLinkStyles, exploreLinkStyles,
contextMenuStyles, contextMenuStyles,
unownedStyles, albumCardStyles,
css` css`
:host { :host {
display: block; display: block;
@@ -529,21 +528,9 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
line-height: 1.5; line-height: 1.5;
} }
/* ── Horizontal scroll rows ── */
.horizontal-row {
display: flex;
gap: 12px;
overflow-x: auto;
padding-bottom: 4px;
scrollbar-width: none;
}
.horizontal-row::-webkit-scrollbar {
display: none;
}
/* ── Top result cards ── */
/* ── Artist cards ── */ /* ── Artist cards ── */
/* Fixed width, for the reason the album card is: a range
means two cards in one row are different sizes. */
.artist-card { .artist-card {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
@@ -552,8 +539,8 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
padding: 10px; padding: 10px;
border-radius: 8px; border-radius: 8px;
cursor: pointer; cursor: pointer;
min-width: 100px; width: 120px;
max-width: 120px; box-sizing: border-box;
flex-shrink: 0; flex-shrink: 0;
text-align: center; text-align: center;
transition: background 0.15s ease; transition: background 0.15s ease;
@@ -624,115 +611,6 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
font-size: var(--yj-text-xs); font-size: var(--yj-text-xs);
} }
/* ── Album cards ── */
.album-card {
display: flex;
flex-direction: column;
gap: 6px;
padding: 8px;
border-radius: 8px;
cursor: pointer;
min-width: 130px;
max-width: 150px;
flex-shrink: 0;
transition: background 0.15s ease;
}
.album-card:hover {
background: var(--yj-bg-overlay, rgba(255, 255, 255, 0.06));
}
.album-card:active {
transform: scale(0.97);
}
.album-art-container {
width: 100%;
aspect-ratio: 1;
border-radius: 4px;
overflow: hidden;
background: linear-gradient(
135deg,
var(--yj-bg-overlay, #404040) 0%,
var(--yj-bg-surface, #282828) 100%
);
display: flex;
align-items: center;
justify-content: center;
position: relative;
}
.album-art-container img {
width: 100%;
height: 100%;
object-fit: cover;
display: block;
}
.album-art-fallback {
display: flex;
align-items: center;
justify-content: center;
width: 100%;
height: 100%;
position: absolute;
inset: 0;
}
.album-art-fallback wa-icon {
color: var(--yj-text-tertiary, #888);
font-size: 24px;
opacity: 0.5;
}
.album-title {
font-weight: 500;
color: var(--yj-text-primary, #fff);
font-size: var(--yj-text-sm);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.album-artist {
color: var(--yj-text-tertiary, #888);
font-size: var(--yj-text-xs);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.album-meta {
display: flex;
align-items: center;
justify-content: space-between;
gap: 6px;
color: var(--yj-text-tertiary, #888);
font-size: var(--yj-text-xs);
min-height: 20px;
}
.album-meta-text {
display: flex;
align-items: center;
gap: 6px;
min-width: 0;
overflow: hidden;
}
.album-meta library-status-indicator {
flex-shrink: 0;
margin-left: auto;
}
.type-badge {
background: var(--yj-bg-overlay, rgba(255, 255, 255, 0.08));
padding: 1px 6px;
border-radius: 3px;
font-size: 10px;
white-space: nowrap;
}
/* ── Track list ── */ /* ── Track list ── */
.track-list { .track-list {
display: flex; display: flex;
@@ -753,7 +631,6 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
cursor: pointer; cursor: pointer;
} }
.album-card:focus-visible,
.track-item:focus-visible { .track-item:focus-visible {
outline: 2px solid var(--yj-accent-text, #ffd43b); outline: 2px solid var(--yj-accent-text, #ffd43b);
outline-offset: -2px; outline-offset: -2px;
@@ -1371,7 +1248,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
const entity = target.kind === 'album' ? 'release-group' : 'recording'; const entity = target.kind === 'album' ? 'release-group' : 'recording';
window.open(`https://musicbrainz.org/${entity}/${target.mbid}`, '_blank', 'noopener'); openMusicBrainz(`/${entity}/${target.mbid}`);
} }
private renderExploreContextMenu() { private renderExploreContextMenu() {
@@ -1662,6 +1539,8 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
} catch { } catch {
// No image — leave empty string. // No image — leave empty string.
} }
return undefined;
}), }),
); );
@@ -2122,7 +2001,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
${subtitle ${subtitle
? html`<p class="section-reason">${subtitle}</p>` ? html`<p class="section-reason">${subtitle}</p>`
: nothing} : nothing}
<div class="horizontal-row"> <scroll-row>
${artists.map((a) => { ${artists.map((a) => {
const owned = isOwned(a); const owned = isOwned(a);
const name = a.englishName || a.name; const name = a.englishName || a.name;
@@ -2171,7 +2050,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
</div> </div>
`; `;
})} })}
</div> </scroll-row>
</section> </section>
`; `;
} }
@@ -2187,7 +2066,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
${subtitle ${subtitle
? html`<p class="section-reason">${subtitle}</p>` ? html`<p class="section-reason">${subtitle}</p>`
: nothing} : nothing}
<div class="horizontal-row"> <scroll-row>
${releaseGroups.map((rg) => { ${releaseGroups.map((rg) => {
const artURL = this.thumbnailCache.get(rg.mbid) || ''; const artURL = this.thumbnailCache.get(rg.mbid) || '';
const year = extractYear(rg.firstReleaseDate); const year = extractYear(rg.firstReleaseDate);
@@ -2249,6 +2128,18 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
> >
<wa-icon name="compact-disc"></wa-icon> <wa-icon name="compact-disc"></wa-icon>
</div> </div>
<div class="album-card-badge">
<library-status-indicator
status=${badge.status}
owned=${badge.owned}
expected=${badge.expected}
entity-type="album"
label=${rg.title}
request-mbid=${rg.mbid}
request-artist=${rg.artistCredit ?? ''}
size="23"
></library-status-indicator>
</div>
</div> </div>
<div class="album-title" title="${rg.title}"> <div class="album-title" title="${rg.title}">
${rg.title} ${rg.title}
@@ -2256,29 +2147,18 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
<div class="album-artist">${creditLink(creditStore.credits(rg.mbid), rg.artistCredit, rg.artistMbid ?? '')}</div> <div class="album-artist">${creditLink(creditStore.credits(rg.mbid), rg.artistCredit, rg.artistMbid ?? '')}</div>
<div class="album-meta"> <div class="album-meta">
<div class="album-meta-text"> <div class="album-meta-text">
${year ? html`<span>${year}</span>` : nothing}
${rg.primaryType ${rg.primaryType
? html`<span class="type-badge" ? html`<span class="type-badge"
>${rg.primaryType}</span >${rg.primaryType}</span
>` >`
: nothing} : nothing}
${year ? html`<span>${year}</span>` : nothing}
</div> </div>
${badge.status === 'in-library'
? nothing
: html`<library-status-indicator
status=${badge.status}
owned=${badge.owned}
expected=${badge.expected}
entity-type="album"
label=${rg.title}
request-mbid=${rg.mbid}
request-artist=${rg.artistCredit ?? ''}
></library-status-indicator>`}
</div> </div>
</div> </div>
`; `;
})} })}
</div> </scroll-row>
</section> </section>
`; `;
} }
+5 -12
View File
@@ -12,6 +12,7 @@ import { libraryStore } from '@store/library-store';
import { EventsOn } from '@runtime/runtime'; import { EventsOn } from '@runtime/runtime';
import { Events } from '../../events'; import { Events } from '../../events';
import '@components/page-header/page-header'; import '@components/page-header/page-header';
import '../scroll-row/scroll-row.js';
import { designTokens } from '../../styles/tokens.css'; import { designTokens } from '../../styles/tokens.css';
import { ViewLifecycleMixin } from '../../utils/view-lifecycle'; import { ViewLifecycleMixin } from '../../utils/view-lifecycle';
@@ -99,16 +100,6 @@ export class HomeView extends ViewLifecycleMixin(LitElement) {
color: var(--yj-text-tertiary, #888); color: var(--yj-text-tertiary, #888);
} }
.row {
display: grid;
grid-auto-flow: column;
grid-auto-columns: 160px;
gap: 14px;
overflow-x: auto;
padding-bottom: 6px;
scrollbar-width: thin;
}
.card { .card {
background: none; background: none;
border: none; border: none;
@@ -117,6 +108,8 @@ export class HomeView extends ViewLifecycleMixin(LitElement) {
cursor: pointer; cursor: pointer;
color: inherit; color: inherit;
display: block; display: block;
width: 160px;
flex-shrink: 0;
} }
.art { .art {
@@ -336,9 +329,9 @@ export class HomeView extends ViewLifecycleMixin(LitElement) {
<span class="shelf-title">${shelf.title}</span> <span class="shelf-title">${shelf.title}</span>
</div> </div>
<p class="shelf-sub">${shelf.subtitle}</p> <p class="shelf-sub">${shelf.subtitle}</p>
<div class="row"> <scroll-row>
${(shelf.albums ?? []).map((album) => this.renderCard(album))} ${(shelf.albums ?? []).map((album) => this.renderCard(album))}
</div> </scroll-row>
</section> </section>
`; `;
} }
@@ -56,12 +56,9 @@ import {
createTrackCardDragImage, createTrackCardDragImage,
removeDragImage, removeDragImage,
} from '@utils/drag-image'; } from '@utils/drag-image';
import { libraryStore } from '@store/library-store';
import '@components/playlist-picker/playlist-picker.js'; import '@components/playlist-picker/playlist-picker.js';
import { loadTrackDetails } from '@utils/lazy-track-details.js'; import { showTrackDetailsForPath, showBatchTrackDetailsForPaths } from '@utils/track-details-opener.js';
import { tracksByFilePath, tracksForPaths } from '@utils/track-index.js';
import type { TrackDetails } from '@components/track-details/track-details.js'; import type { TrackDetails } from '@components/track-details/track-details.js';
import type { CoverArtUrls } from '@components/track-details/track-details.js';
import '@components/phantom-resolver/phantom-resolver.js'; import '@components/phantom-resolver/phantom-resolver.js';
import type { PhantomResolver } from '@components/phantom-resolver/phantom-resolver.js'; import type { PhantomResolver } from '@components/phantom-resolver/phantom-resolver.js';
import '@components/duplicate-tracks-dialog/duplicate-tracks-dialog.js'; import '@components/duplicate-tracks-dialog/duplicate-tracks-dialog.js';
@@ -758,76 +755,21 @@ export class PlaylistDetails
} }
private async openTrackDetails(filePath: string) { private async openTrackDetails(filePath: string) {
const tracks = libraryStore.getCachedTracks(); await showTrackDetailsForPath(
const track = tracks () => this.trackDetailsDialog,
? tracksByFilePath(tracks).get(filePath) filePath,
: undefined;
if (!track) return;
const ready = await loadTrackDetails(
() => void this.openTrackDetails(filePath), () => void this.openTrackDetails(filePath),
); );
if (!ready) return;
const coverArt = track.CoverArtPath
? {
coverArtPath: track.CoverArtPath,
coverArtSmall: track.CoverArtSmall,
coverArtMedium: track.CoverArtMedium,
coverArtLarge: track.CoverArtLarge,
}
: undefined;
this.trackDetailsDialog?.show(
track,
coverArt,
);
} }
private async openBatchTrackDetails( private async openBatchTrackDetails(
filePaths: string[], filePaths: string[],
) { ) {
const cachedTracks = await showBatchTrackDetailsForPaths(
libraryStore.getCachedTracks(); () => this.trackDetailsDialog,
if (!cachedTracks) return;
const tracks = tracksForPaths(
cachedTracks,
filePaths, filePaths,
);
if (tracks.length === 0) return;
const ready = await loadTrackDetails(
() => void this.openBatchTrackDetails(filePaths), () => void this.openBatchTrackDetails(filePaths),
); );
if (!ready) return;
const first = tracks[0]!;
const albumNames = new Set(tracks.map((t) => t.Album));
let coverArt: CoverArtUrls | null = null;
let coverArtMixed = false;
if (albumNames.size === 1 && first.CoverArtPath) {
coverArt = {
coverArtPath: first.CoverArtPath,
coverArtSmall: first.CoverArtSmall,
coverArtMedium: first.CoverArtMedium,
coverArtLarge: first.CoverArtLarge,
};
} else if (albumNames.size > 1) {
coverArtMixed = true;
}
this.trackDetailsDialog?.showBatch(
tracks,
coverArt,
coverArtMixed,
);
} }
/** /**
@@ -51,12 +51,8 @@ import {
createTrackCardDragImage, createTrackCardDragImage,
removeDragImage, removeDragImage,
} from '@utils/drag-image'; } from '@utils/drag-image';
import { libraryStore } from '@store/library-store'; import { showTrackDetailsForPath, showBatchTrackDetailsForPaths } from '@utils/track-details-opener.js';
import type * as library from '@go/library/models.js';
import { loadTrackDetails } from '@utils/lazy-track-details.js';
import { tracksByFilePath } from '@utils/track-index.js';
import type { TrackDetails } from '@components/track-details/track-details.js'; import type { TrackDetails } from '@components/track-details/track-details.js';
import type { CoverArtUrls } from '@components/track-details/track-details.js';
import { import {
creditLink, creditLink,
trackLink, trackLink,
@@ -1542,89 +1538,33 @@ export class QueuePanel
} }
private async openTrackDetails(index: number) { private async openTrackDetails(index: number) {
const queueTrack = const queueTrack = this.queue.tracks[index];
this.queue.tracks[index];
if (!queueTrack) return; if (!queueTrack) return;
const tracks = await showTrackDetailsForPath(
libraryStore.getCachedTracks(); () => this.trackDetailsDialog,
const track = tracks queueTrack.filePath,
? tracksByFilePath(tracks).get(
queueTrack.filePath,
)
: undefined;
if (!track) return;
const ready = await loadTrackDetails(
() => void this.openTrackDetails(index), () => void this.openTrackDetails(index),
); );
if (!ready) return;
const coverArt = track.CoverArtPath
? {
coverArtPath: track.CoverArtPath,
coverArtSmall: track.CoverArtSmall,
coverArtMedium: track.CoverArtMedium,
coverArtLarge: track.CoverArtLarge,
}
: undefined;
this.trackDetailsDialog?.show(
track,
coverArt,
);
} }
private async openBatchTrackDetails( private async openBatchTrackDetails(
indices: number[], indices: number[],
) { ) {
const queueTracks = this.queue.tracks; const queueTracks = this.queue.tracks;
const cachedTracks = const filePaths: string[] = [];
libraryStore.getCachedTracks();
if (!cachedTracks) return; for (const i of indices) {
const path = queueTracks[i]?.filePath;
const byPath = tracksByFilePath(cachedTracks); if (path != null) filePaths.push(path);
const tracks = indices
.map((i) => queueTracks[i])
.filter((qt) => qt != null)
.map((qt) => byPath.get(qt.filePath))
.filter(
(t): t is library.Track =>
t != null,
);
if (tracks.length === 0) return;
const ready = await loadTrackDetails(
() => void this.openBatchTrackDetails(indices),
);
if (!ready) return;
const first = tracks[0]!;
const albumNames = new Set(tracks.map((t) => t.Album));
let coverArt: CoverArtUrls | null = null;
let coverArtMixed = false;
if (albumNames.size === 1 && first.CoverArtPath) {
coverArt = {
coverArtPath: first.CoverArtPath,
coverArtSmall: first.CoverArtSmall,
coverArtMedium: first.CoverArtMedium,
coverArtLarge: first.CoverArtLarge,
};
} else if (albumNames.size > 1) {
coverArtMixed = true;
} }
this.trackDetailsDialog?.showBatch( await showBatchTrackDetailsForPaths(
tracks, () => this.trackDetailsDialog,
coverArt, filePaths,
coverArtMixed, () => void this.openBatchTrackDetails(indices),
); );
} }
@@ -0,0 +1,213 @@
import { LitElement, css, html } from 'lit';
import { customElement, query, state } from 'lit/decorators.js';
import '@awesome.me/webawesome/dist/components/icon/icon.js';
/** How far one press moves the row — most of a screenful, not all of
* it, so the card that was at the edge stays as an anchor. */
const SCROLL_FRACTION = 0.8;
/**
* A horizontally scrolling row with arrow buttons.
*
* The shelves, the search results and (now) the artist page's
* discography and similar-artists rows are all "more than fits, scroll
* sideways". Until this existed the only way to see the rest was a
* mousewheel or a trackpad gesture, which is not an affordance — a
* mouse with no horizontal wheel simply could not reach the cards past
* the fold.
*
* It is a component rather than a rule on `.horizontal-row` for two
* reasons. The arrows are *state* — which way the row can still move —
* and that state has to be recomputed when the viewport resizes or a
* card arrives with its cover art; a stylesheet cannot do that. And
* every caller then gets the same arrows, the same reveal and the same
* keyboard labels without writing them again.
*
* **The arrows are `hidden`, not merely transparent, at the end they
* cannot move from** — a control that cannot act is worse than none,
* and an invisible one still holds a hit area and a tab stop. On a
* pointer device the pair fades in with the row's hover; where there is
* no hover they are always visible, because there is no other route to
* them there (a swipe is not an affordance a mouse-less keyboard user
* has either).
*
* The cards are light DOM children and stay in the *host's* shadow
* root, so the host's own `.album-card` / `.artist-card` styles apply
* unchanged — this component only owns the box they scroll inside.
*/
@customElement('scroll-row')
export class ScrollRow extends LitElement {
@query('.viewport') private viewport?: HTMLElement;
@state() private atStart = true;
@state() private atEnd = true;
@state() private overflowing = false;
private observer?: ResizeObserver;
static override styles = css`
:host {
display: block;
position: relative;
}
.viewport {
overflow-x: auto;
overflow-y: hidden;
scrollbar-width: none;
/* A swipe that reaches the row's end should not drag the
whole page sideways with it. */
overscroll-behavior-x: contain;
}
.viewport::-webkit-scrollbar {
display: none;
}
.track {
display: flex;
gap: 12px;
}
.arrow {
position: absolute;
top: 50%;
transform: translateY(-50%);
z-index: 2;
display: flex;
align-items: center;
justify-content: center;
width: 36px;
height: 36px;
padding: 0;
border-radius: 50%;
border: 1px solid var(--yj-border-subtle, rgba(255, 255, 255, 0.1));
background: var(--yj-bg-elevated, #343a40);
color: var(--yj-text-primary, #fff);
cursor: pointer;
opacity: 0;
transition: opacity 0.15s ease;
}
.arrow[hidden] {
display: none;
}
.arrow.prev {
left: 4px;
}
.arrow.next {
right: 4px;
}
.arrow:hover {
background: var(--yj-bg-overlay, rgba(255, 255, 255, 0.12));
}
.arrow:focus-visible {
outline: 2px solid var(--yj-accent, #ffd43b);
outline-offset: 2px;
}
@media (hover: hover) and (pointer: fine) {
:host(:hover) .arrow,
.arrow:focus-visible {
opacity: 1;
}
}
@media not all and (hover: hover) {
.arrow {
opacity: 1;
}
}
`;
override firstUpdated(): void {
const viewport = this.viewport;
if (!viewport) return;
this.observer = new ResizeObserver(() => this.measure());
this.observer.observe(viewport);
// The track's own size is what changes when a card arrives with
// its cover art, and a ResizeObserver on the viewport alone
// never fires for that.
const track = viewport.firstElementChild;
if (track) this.observer.observe(track);
this.measure();
}
override disconnectedCallback(): void {
super.disconnectedCallback();
this.observer?.disconnect();
this.observer = undefined;
}
private measure(): void {
const viewport = this.viewport;
if (!viewport) return;
this.overflowing = viewport.scrollWidth > viewport.clientWidth + 1;
this.atStart = viewport.scrollLeft <= 1;
this.atEnd =
viewport.scrollLeft + viewport.clientWidth >=
viewport.scrollWidth - 1;
}
private onScroll = (): void => this.measure();
private scrollStep(direction: -1 | 1): void {
const viewport = this.viewport;
if (!viewport) return;
viewport.scrollBy({
left: direction * viewport.clientWidth * SCROLL_FRACTION,
behavior: 'smooth',
});
}
override render() {
const showPrev = this.overflowing && !this.atStart;
const showNext = this.overflowing && !this.atEnd;
return html`
<button
class="arrow prev"
type="button"
aria-label="Scroll left"
?hidden=${!showPrev}
@click=${() => this.scrollStep(-1)}
>
<wa-icon name="chevron-left"></wa-icon>
</button>
<div class="viewport" @scroll=${this.onScroll}>
<div class="track"><slot></slot></div>
</div>
<button
class="arrow next"
type="button"
aria-label="Scroll right"
?hidden=${!showNext}
@click=${() => this.scrollStep(1)}
>
<wa-icon name="chevron-right"></wa-icon>
</button>
`;
}
}
declare global {
interface HTMLElementTagNameMap {
'scroll-row': ScrollRow;
}
}
@@ -6,6 +6,7 @@ import { designTokens } from '../../styles/tokens.css';
import type { DragActiveDetail } from '@utils/drag-controller'; import type { DragActiveDetail } from '@utils/drag-controller';
import { ActiveViewController } from '@store/controllers/active-view-controller'; import { ActiveViewController } from '@store/controllers/active-view-controller';
import { ViewVisibilityController } from '@store/controllers/view-visibility-controller'; import { ViewVisibilityController } from '@store/controllers/view-visibility-controller';
import { libraryStore } from '@store/library-store';
import { VIEW_META } from '../../services/view-meta'; import { VIEW_META } from '../../services/view-meta';
import type { View } from '../../services/view-meta'; import type { View } from '../../services/view-meta';
@@ -352,6 +353,10 @@ export class AppSidebar extends LitElement {
: 'false'} : 'false'}
@click=${() => @click=${() =>
this.navigate(item.id)} this.navigate(item.id)}
@mouseenter=${() =>
this.prefetch(item.id)}
@focus=${() =>
this.prefetch(item.id)}
@dragover=${(e: DragEvent) => @dragover=${(e: DragEvent) =>
this.onNavDragOver( this.onNavDragOver(
e, e,
@@ -503,6 +508,19 @@ export class AppSidebar extends LitElement {
} }
} }
/**
* Start loading what this view draws, on hover or keyboard focus.
*
* The ~100 ms before the click is the whole point: #280 stopped
* fetching every collection at startup, and this is what keeps the
* view that *is* opened from paying the whole payload after the
* click. Nothing is awaited and nothing is reported here — the view
* itself reports a failure, and this is the same request.
*/
private prefetch(view: View) {
libraryStore.prefetch(view);
}
private navigate(view: View) { private navigate(view: View) {
// No optimistic highlight: the shell answers, and it answers // No optimistic highlight: the shell answers, and it answers
// synchronously in `handleNavigate` before it awaits anything. // synchronously in `handleNavigate` before it awaits anything.
@@ -52,11 +52,8 @@ import '@lit-labs/virtualizer';
import type { LitVirtualizer } from '@lit-labs/virtualizer'; import type { LitVirtualizer } from '@lit-labs/virtualizer';
import { flow } from '@lit-labs/virtualizer/layouts/flow.js'; import { flow } from '@lit-labs/virtualizer/layouts/flow.js';
import '@components/playlist-picker/playlist-picker.js'; import '@components/playlist-picker/playlist-picker.js';
import { loadTrackDetails } from '@utils/lazy-track-details.js'; import { showTrackDetailsForPath, showBatchTrackDetailsForPaths } from '@utils/track-details-opener.js';
import { tracksByFilePath, tracksForPaths } from '@utils/track-index.js';
import type { TrackDetails } from '@components/track-details/track-details.js'; import type { TrackDetails } from '@components/track-details/track-details.js';
import type { CoverArtUrls } from '@components/track-details/track-details.js';
import { libraryStore } from '@store/library-store';
import { formatMilliseconds } from '@utils/time'; import { formatMilliseconds } from '@utils/time';
import { import {
creditLink, creditLink,
@@ -1145,76 +1142,21 @@ export class SmartPlaylistDetails
} }
private async openTrackDetails(filePath: string) { private async openTrackDetails(filePath: string) {
const tracks = libraryStore.getCachedTracks(); await showTrackDetailsForPath(
const track = tracks () => this.trackDetailsDialog,
? tracksByFilePath(tracks).get(filePath) filePath,
: undefined;
if (!track) return;
const ready = await loadTrackDetails(
() => void this.openTrackDetails(filePath), () => void this.openTrackDetails(filePath),
); );
if (!ready) return;
const coverArt = track.CoverArtPath
? {
coverArtPath: track.CoverArtPath,
coverArtSmall: track.CoverArtSmall,
coverArtMedium: track.CoverArtMedium,
coverArtLarge: track.CoverArtLarge,
}
: undefined;
this.trackDetailsDialog?.show(
track,
coverArt,
);
} }
private async openBatchTrackDetails( private async openBatchTrackDetails(
filePaths: string[], filePaths: string[],
) { ) {
const cachedTracks = await showBatchTrackDetailsForPaths(
libraryStore.getCachedTracks(); () => this.trackDetailsDialog,
if (!cachedTracks) return;
const tracks = tracksForPaths(
cachedTracks,
filePaths, filePaths,
);
if (tracks.length === 0) return;
const ready = await loadTrackDetails(
() => void this.openBatchTrackDetails(filePaths), () => void this.openBatchTrackDetails(filePaths),
); );
if (!ready) return;
const first = tracks[0]!;
const albumNames = new Set(tracks.map((t) => t.Album));
let coverArt: CoverArtUrls | null = null;
let coverArtMixed = false;
if (albumNames.size === 1 && first.CoverArtPath) {
coverArt = {
coverArtPath: first.CoverArtPath,
coverArtSmall: first.CoverArtSmall,
coverArtMedium: first.CoverArtMedium,
coverArtLarge: first.CoverArtLarge,
};
} else if (albumNames.size > 1) {
coverArtMixed = true;
}
this.trackDetailsDialog?.showBatch(
tracks,
coverArt,
coverArtMixed,
);
} }
// ================================================================= // =================================================================
@@ -1,8 +1,12 @@
import { LitElement, html, css, nothing } from 'lit'; import { LitElement, html, css, nothing } from 'lit';
import { customElement, property, state } from 'lit/decorators.js'; import { customElement, property, state } from 'lit/decorators.js';
import * as library from '@go/library/models.js'; import * as library from '@go/library/models.js';
import { PreviewSmartPlaylist } from '@go/playlist/service.js'; import {
import { libraryStore } from '@store/library-store'; PreviewSmartPlaylist,
SuggestSmartPlaylistValues,
} from '@go/playlist/service.js';
import { list } from '@utils/binding';
import { LRUMap } from '@utils/lru-map';
import { describeError } from '@utils/describe-error'; import { describeError } from '@utils/describe-error';
import { designTokens } from '../../styles/tokens.css'; import { designTokens } from '../../styles/tokens.css';
import '@components/combobox/combobox.ts'; import '@components/combobox/combobox.ts';
@@ -91,51 +95,23 @@ function formatOperatorLabel(op: string): string {
return op.replace(/_/g, ' '); return op.replace(/_/g, ' ');
} }
/** Returns autocomplete suggestions for a given field from libraryStore. */ /**
function getAutocompleteOptions(field: string): string[] { * Fields whose value box suggests values present in the library. Must
switch (field) { * match `suggestFields` in backend/smartplaylist; any other field gets
case 'artist': * a plain box.
return libraryStore.getCachedArtists()?.map((a) => a.Name) ?? []; *
case 'genre': * Suggestions are asked for as the user types (#279): they used to be
return libraryStore.getCachedGenres()?.map((g) => g.Name) ?? []; * built from libraryStore's whole-library arrays, which made this
case 'album': * editor one more reason to load every track at startup, and gave it
return libraryStore.getCachedAlbums()?.map((a) => a.Name) ?? []; * empty lists whenever those arrays had not landed.
case 'title': { */
const tracks = libraryStore.getCachedTracks(); const SUGGEST_FIELDS = new Set([
if (!tracks) return []; 'title', 'artist', 'album', 'genre', 'composer', 'file_type',
return [...new Set(tracks.map((t) => t.TrackName).filter(Boolean))]; 'year', 'release_year',
} ]);
case 'composer': {
const tracks = libraryStore.getCachedTracks(); /** How long typing must pause before suggestions are asked for. */
if (!tracks) return []; const SUGGEST_DEBOUNCE_MS = 120;
return [...new Set(tracks.map((t) => t.Composer).filter(Boolean))];
}
case 'file_type': {
const tracks = libraryStore.getCachedTracks();
if (!tracks) return [];
return [...new Set(tracks.map((t) => t.FileType).filter(Boolean))];
}
case 'year':
case 'release_year': {
// Both year fields draw suggestions from the set of years
// present in the library. The cached Track only carries the
// display (original) year, so it seeds both datalists — the
// list is just a hint, and the real filter runs server-side.
const tracks = libraryStore.getCachedTracks();
if (!tracks) return [];
return [
...new Set(
tracks
.map((t) => t.Year)
.filter((y) => y > 0)
.map(String),
),
].sort();
}
default:
return [];
}
}
/** /**
* Overrides for fields whose title-cased name would be ambiguous. The * Overrides for fields whose title-cased name would be ambiguous. The
@@ -206,6 +182,17 @@ export class SmartPlaylistEditor extends LitElement {
private previewTimer: ReturnType<typeof setTimeout> | null = null; private previewTimer: ReturnType<typeof setTimeout> | null = null;
/** The value suggestions each rule row is showing, by row index. */
@state() private suggestions = new Map<number, readonly string[]>();
private suggestTimer: ReturnType<typeof setTimeout> | null = null;
/** Answers already fetched this session, by field and typed text. */
private suggestCache = new LRUMap<string, readonly string[]>(100);
/** The latest request per row, so a slow answer cannot overwrite a newer one. */
private suggestWanted = new Map<number, string>();
// ── Styles ────────────────────────────────────────────────────── // ── Styles ──────────────────────────────────────────────────────
static override styles = [ static override styles = [
@@ -599,6 +586,7 @@ export class SmartPlaylistEditor extends LitElement {
// Reset value when field changes to avoid stale autocomplete data // Reset value when field changes to avoid stale autocomplete data
row.value = ''; row.value = '';
row.value2 = ''; row.value2 = '';
this.dropSuggestions();
this.ruleRows = [...this.ruleRows]; this.ruleRows = [...this.ruleRows];
this.onRulesChanged(); this.onRulesChanged();
@@ -628,6 +616,56 @@ export class SmartPlaylistEditor extends LitElement {
this.onRulesChanged(); this.onRulesChanged();
} }
/** Ask for the values of row `index`'s field that contain `text`. */
private requestSuggestions(index: number, text: string) {
const field = this.ruleRows[index]?.field ?? '';
if (!SUGGEST_FIELDS.has(field)) return;
const key = `${field}\u0000${text}`;
this.suggestWanted.set(index, key);
const cached = this.suggestCache.get(key);
if (cached) {
this.setSuggestions(index, cached);
return;
}
if (this.suggestTimer !== null) clearTimeout(this.suggestTimer);
this.suggestTimer = setTimeout(() => {
this.suggestTimer = null;
void list(SuggestSmartPlaylistValues(field, text))
.then((values) => {
this.suggestCache.set(key, values);
if (this.suggestWanted.get(index) === key) {
this.setSuggestions(index, values);
}
})
.catch((err: unknown) => {
// A suggestion is a hint; the box still takes any text.
console.error('smart playlist: suggestions failed', err);
});
}, SUGGEST_DEBOUNCE_MS);
}
private setSuggestions(index: number, values: readonly string[]) {
const next = new Map(this.suggestions);
next.set(index, values);
this.suggestions = next;
}
/** Row indexes and fields changed: what was suggested no longer applies. */
private dropSuggestions() {
this.suggestions = new Map();
this.suggestWanted.clear();
}
private updateValue2(index: number, newValue: string) { private updateValue2(index: number, newValue: string) {
const row = this.ruleRows[index]; const row = this.ruleRows[index];
if (!row) return; if (!row) return;
@@ -644,6 +682,7 @@ export class SmartPlaylistEditor extends LitElement {
private removeRule(index: number) { private removeRule(index: number) {
if (this.ruleRows.length <= 1) return; if (this.ruleRows.length <= 1) return;
this.ruleRows = this.ruleRows.filter((_, i) => i !== index); this.ruleRows = this.ruleRows.filter((_, i) => i !== index);
this.dropSuggestions();
this.onRulesChanged(); this.onRulesChanged();
} }
@@ -892,7 +931,9 @@ export class SmartPlaylistEditor extends LitElement {
` `
: html` : html`
<yj-combobox <yj-combobox
.options=${getAutocompleteOptions(row.field)} .options=${this.suggestions.get(index) ?? []}
@combobox-input=${(e: CustomEvent<{ text: string }>) =>
this.requestSuggestions(index, e.detail.text)}
.value=${row.value} .value=${row.value}
placeholder=${isAnyOf placeholder=${isAnyOf
? 'Comma-separated values' ? 'Comma-separated values'
@@ -1,6 +1,7 @@
import { LitElement, html, css, nothing } from 'lit'; import { LitElement, html, css, nothing } from 'lit';
import { customElement, property } from 'lit/decorators.js'; import { customElement, property } from 'lit/decorators.js';
import { designTokens } from '../../styles/tokens.css'; import { designTokens } from '../../styles/tokens.css';
import '../scroll-row/scroll-row.js';
import type * as explore from '@go/explore/models.js'; import type * as explore from '@go/explore/models.js';
import { import {
GetArtistImageURL, GetArtistImageURL,
@@ -15,7 +16,6 @@ import { albumBadgeFor, libraryStatusFor } from '../../utils/library-status';
import { import {
isOwned, isOwned,
ownershipLabel, ownershipLabel,
unownedStyles,
type OwnableKind, type OwnableKind,
} from '../../utils/ownership'; } from '../../utils/ownership';
import { completenessStore } from '../../store/completeness-store'; import { completenessStore } from '../../store/completeness-store';
@@ -103,20 +103,12 @@ export class TopResultsRow extends LitElement {
static override styles = [ static override styles = [
designTokens, designTokens,
exploreLinkStyles, exploreLinkStyles,
unownedStyles,
css` css`
:host { :host {
display: block; display: block;
margin-bottom: 16px; margin-bottom: 16px;
} }
.row {
display: flex;
gap: 12px;
overflow-x: auto;
padding-bottom: 4px;
}
.card { .card {
flex: 0 0 auto; flex: 0 0 auto;
width: 200px; width: 200px;
@@ -285,9 +277,9 @@ export class TopResultsRow extends LitElement {
return html` return html`
<div class="section-label">Top Results</div> <div class="section-label">Top Results</div>
<div class="row"> <scroll-row>
${this.results.map((r) => this.renderCard(r))} ${this.results.map((r) => this.renderCard(r))}
</div> </scroll-row>
`; `;
} }
@@ -22,7 +22,9 @@ import {
import { GetTrackMBIDs } from '@go/library/library.js'; import { GetTrackMBIDs } from '@go/library/library.js';
type TrackMBIDs = library.TrackMBIDs; type TrackMBIDs = library.TrackMBIDs;
import { ImageFilePicker, ReadFile } from '@go/frontendutil/frontendutil.js'; import { ImageFilePicker, ReadFile } from '@go/frontendutil/frontendutil.js';
import { libraryStore } from '../../store/library-store'; import { trackCache } from '../../store/track-cache';
import { batchCoverArt } from '@utils/track-details-opener.js';
import type { ListTrack } from '@utils/track-table';
import { EventsOn } from '@runtime/runtime'; import { EventsOn } from '@runtime/runtime';
import { Events } from '../../events'; import { Events } from '../../events';
@@ -87,7 +89,11 @@ export class TrackDetails extends LitElement {
// -- Batch-specific state -- // -- Batch-specific state --
@state() private batchMode = false; @state() private batchMode = false;
@state() private batchTracks: library.Track[] = []; // `ListTrack`, not `Track`: every field the merged-values view reads
// is on the list's rows, so a caller that is already showing them
// can open this dialog without fetching anything (#281). The
// openers that hold no rows ask `trackCache` instead.
@state() private batchTracks: ListTrack[] = [];
@state() private batchFilePaths: string[] = []; @state() private batchFilePaths: string[] = [];
@state() private batchCoverArtMixed = false; @state() private batchCoverArtMixed = false;
@state() private batchProgress: { @state() private batchProgress: {
@@ -135,12 +141,12 @@ export class TrackDetails extends LitElement {
/** Open the dialog for batch editing multiple tracks. */ /** Open the dialog for batch editing multiple tracks. */
showBatch( showBatch(
tracks: library.Track[], tracks: readonly ListTrack[],
coverArt: CoverArtUrls | null, coverArt: CoverArtUrls | null,
coverArtMixed: boolean, coverArtMixed: boolean,
): void { ): void {
this.batchMode = true; this.batchMode = true;
this.batchTracks = tracks; this.batchTracks = [...tracks];
this.batchFilePaths = tracks.map( this.batchFilePaths = tracks.map(
(t) => t.FilePath, (t) => t.FilePath,
); );
@@ -1317,10 +1323,13 @@ export class TrackDetails extends LitElement {
const container = img.parentElement; const container = img.parentElement;
if (container) { if (container) {
container.innerHTML = const placeholder = document.createElement('div');
'<div class="cover-placeholder">' + const icon = document.createElement('wa-icon');
'<wa-icon name="music"></wa-icon>' +
'</div>'; placeholder.className = 'cover-placeholder';
icon.setAttribute('name', 'music');
placeholder.append(icon);
container.replaceChildren(placeholder);
} }
}; };
@@ -1690,20 +1699,11 @@ export class TrackDetails extends LitElement {
// invalidation, which refreshes all other views. // invalidation, which refreshes all other views.
this.exitEditMode(); this.exitEditMode();
// Re-fetch track and album data so the dialog // Re-read this one track so the dialog shows the
// shows updated values and cover art. The store // values and cover art just written. Not the whole
// invalidation is already in-flight from the event; // library (#279): the views that hold it refetch on
// these calls await the pending fetch or start one. // the event, and only if something is showing them.
// Awaited for the side effect of refreshing the const [updated] = await trackCache.refresh([filePath]);
// store; the album list is consumed elsewhere.
const [tracks] = await Promise.all([
libraryStore.getTracks(),
libraryStore.getAlbums(),
]);
const updated = tracks.find(
(t) => t.FilePath === filePath,
);
if (updated) { if (updated) {
this.track = updated; this.track = updated;
@@ -1828,39 +1828,18 @@ export class TrackDetails extends LitElement {
this.errorMessage = ''; this.errorMessage = '';
this.cleanupPendingCoverArt(); this.cleanupPendingCoverArt();
// Refresh data from library store; album list is // Re-read the tracks just written, in the order the batch
// refreshed for side effects only. // holds them (#279) — not the whole library to filter.
const [tracks] = await Promise.all([ // `refresh`, because the write's event may not have reached
libraryStore.getTracks(), // the cache before its own result did.
libraryStore.getAlbums(), const refreshed = await trackCache.refresh(this.batchFilePaths);
]);
// Re-resolve batch tracks.
const pathSet = new Set(this.batchFilePaths);
const refreshed = tracks.filter((t) =>
pathSet.has(t.FilePath),
);
this.batchTracks = refreshed; this.batchTracks = refreshed;
// Re-resolve cover art state. const { coverArt, mixed } = batchCoverArt(refreshed);
const first = refreshed[0];
const albumNames = new Set(refreshed.map((t) => t.Album));
if (albumNames.size === 1 && first?.CoverArtPath) { this.coverArt = coverArt;
this.coverArt = { this.batchCoverArtMixed = mixed;
coverArtPath: first.CoverArtPath,
coverArtSmall: first.CoverArtSmall,
coverArtMedium: first.CoverArtMedium,
coverArtLarge: first.CoverArtLarge,
};
this.batchCoverArtMixed = false;
} else if (albumNames.size > 1) {
this.coverArt = null;
this.batchCoverArtMixed = true;
} else {
this.coverArt = null;
this.batchCoverArtMixed = false;
}
}; };
// -- Shared edit logic -- // -- Shared edit logic --
@@ -2103,6 +2082,8 @@ export class TrackDetails extends LitElement {
// as a base64-encoded string (standard encoding/json // as a base64-encoded string (standard encoding/json
// behaviour for []byte). // behaviour for []byte).
const result = await ReadFile(filePath); const result = await ReadFile(filePath);
// SAFETY: the binding is typed as the Go []byte, but the
// wire value is the base64 string encoding/json made of it.
const b64 = result as unknown as string; const b64 = result as unknown as string;
const binary = atob(b64); const binary = atob(b64);
const bytes = new Uint8Array(binary.length); const bytes = new Uint8Array(binary.length);
@@ -2157,7 +2138,7 @@ export class TrackDetails extends LitElement {
key: string; key: string;
label: string; label: string;
type: 'text' | 'number'; type: 'text' | 'number';
extract: (t: library.Track) => string; extract: (t: ListTrack) => string;
}> = [ }> = [
{ {
key: 'title', key: 'title',
@@ -2241,7 +2222,7 @@ export class TrackDetails extends LitElement {
private countDistinctValues(key: string): number { private countDistinctValues(key: string): number {
const extractMap: Record< const extractMap: Record<
string, string,
(t: library.Track) => string (t: ListTrack) => string
> = { > = {
title: (t) => t.TrackName ?? '', title: (t) => t.TrackName ?? '',
artist: (t) => t.ArtistName ?? '', artist: (t) => t.ArtistName ?? '',
@@ -2265,7 +2246,7 @@ export class TrackDetails extends LitElement {
if (!extract) return 0; if (!extract) return 0;
const unique = new Set( const unique = new Set(
this.batchTracks.map(extract), this.batchTracks.map((t) => extract(t)),
); );
return unique.size; return unique.size;
+10 -10
View File
@@ -1,4 +1,4 @@
import type * as library from '@go/library/models.js'; import type { ListTrack } from '@utils/track-table';
import { import {
formatSampleRate, formatSampleRate,
formatBitDepth, formatBitDepth,
@@ -45,7 +45,7 @@ export interface ColumnDef {
*/ */
configurable?: boolean; configurable?: boolean;
/** Extracts the display value from a track. */ /** Extracts the display value from a track. */
accessor: (track: library.Track) => string; accessor: (track: ListTrack) => string;
/** Default CSS width (used when no saved width exists). */ /** Default CSS width (used when no saved width exists). */
defaultWidth: string; defaultWidth: string;
/** Text alignment. Defaults to left. */ /** Text alignment. Defaults to left. */
@@ -60,15 +60,15 @@ export interface ColumnDef {
* needs it, which is why it is optional rather than a second * needs it, which is why it is optional rather than a second
* required parameter on all of them. * required parameter on all of them.
*/ */
renderCell?: (track: library.Track, term?: string) => unknown; renderCell?: (track: ListTrack, term?: string) => unknown;
/** /**
* Comparison function for sorting two tracks by this column. * Comparison function for sorting two tracks by this column.
* Returns negative if a < b, positive if a > b, zero if equal. * Returns negative if a < b, positive if a > b, zero if equal.
* If omitted the column is not sortable. * If omitted the column is not sortable.
*/ */
comparator?: ( comparator?: (
a: library.Track, a: ListTrack,
b: library.Track, b: ListTrack,
) => number; ) => number;
} }
@@ -79,7 +79,7 @@ export const COLUMN_DEFS: Record<string, ColumnDef> = {
label: 'Art', label: 'Art',
accessor: () => '', accessor: () => '',
defaultWidth: '36px', defaultWidth: '36px',
renderCell: (track: library.Track) => { renderCell: (track: ListTrack) => {
// `perf.M3`. This rendered `CoverArtPath` — the *original* // `perf.M3`. This rendered `CoverArtPath` — the *original*
// embedded artwork, commonly 1500×1500 and several hundred // embedded artwork, commonly 1500×1500 and several hundred
// kB — scaled by CSS into a 24 px box, while the 100 px // kB — scaled by CSS into a 24 px box, while the 100 px
@@ -90,10 +90,10 @@ export const COLUMN_DEFS: Record<string, ColumnDef> = {
// `cover-grid.getCoverUrl()` has picked the right tier all // `cover-grid.getCoverUrl()` has picked the right tier all
// along; this is the same rule for a much smaller box, with // along; this is the same rule for a much smaller box, with
// the two attributes that keep the decode off the scroll // the two attributes that keep the decode off the scroll
// path. // path. The list carries only this tier (#281): every tier
const src = track.CoverArtSmall // is derived from the same file, so it is set whenever any
|| track.CoverArtMedium // of them would be.
|| track.CoverArtPath; const src = track.CoverArtSmall;
if (!src) return nothing; if (!src) return nothing;
@@ -1,4 +1,4 @@
import type * as library from '@go/library/models.js'; import type { ListTrack } from '@utils/track-table';
import { html } from 'lit'; import { html } from 'lit';
import type { TemplateResult } from 'lit'; import type { TemplateResult } from 'lit';
@@ -99,7 +99,7 @@ function matchQuality(
* fields are always included on top of these. * fields are always included on top of these.
*/ */
function scoreTrack( function scoreTrack(
track: library.Track, track: ListTrack,
termLower: string, termLower: string,
columns: ColumnDef[], columns: ColumnDef[],
): number { ): number {
@@ -147,7 +147,7 @@ function scoreTrack(
/** A track paired with its relevance score. */ /** A track paired with its relevance score. */
export interface RankedTrack { export interface RankedTrack {
track: library.Track; track: ListTrack;
score: number; score: number;
} }
@@ -166,10 +166,10 @@ export interface RankedTrack {
* `scores` (Map of FilePath → relevance score). * `scores` (Map of FilePath → relevance score).
*/ */
export function rankTracks( export function rankTracks(
tracks: library.Track[], tracks: ListTrack[],
term: string, term: string,
activeColumns: ColumnDef[], activeColumns: ColumnDef[],
): { tracks: library.Track[]; scores: Map<string, number> } { ): { tracks: ListTrack[]; scores: Map<string, number> } {
const termLower = term.toLowerCase(); const termLower = term.toLowerCase();
const ranked: RankedTrack[] = []; const ranked: RankedTrack[] = [];
@@ -188,7 +188,7 @@ export function rankTracks(
// Sort descending by score (highest relevance first). // Sort descending by score (highest relevance first).
ranked.sort((a, b) => b.score - a.score); ranked.sort((a, b) => b.score - a.score);
const result: library.Track[] = []; const result: ListTrack[] = [];
const scores = new Map<string, number>(); const scores = new Map<string, number>();
for (const r of ranked) { for (const r of ranked) {
@@ -1,5 +1,5 @@
import * as library from '@go/library/models.js'; import type { ListTrack } from '@utils/track-table';
import { LitElement, html, svg, css, nothing } from 'lit'; import { LitElement, html, svg, css, nothing, type TemplateResult } from 'lit';
import { designTokens } from '../../styles/tokens.css'; import { designTokens } from '../../styles/tokens.css';
import { srOnly } from '../../styles/sr-only.css'; import { srOnly } from '../../styles/sr-only.css';
import { import {
@@ -80,11 +80,10 @@ import { describeError } from '@utils/describe-error';
import { notificationStore } from '@store/notification-store'; import { notificationStore } from '@store/notification-store';
import { confirmAction } from '@components/confirm-dialog/confirm-dialog'; import { confirmAction } from '@components/confirm-dialog/confirm-dialog';
import { RemoveFromLibrary } from '@go/library/library.js'; import { RemoveFromLibrary } from '@go/library/library.js';
import { loadTrackDetails } from '@utils/lazy-track-details.js'; import { showTrackDetailsForPath, showBatchTrackDetails } from '@utils/track-details-opener.js';
import { tracksByFilePath, tracksForPaths } from '@utils/track-index.js'; import { tracksByFilePath, tracksForPaths } from '@utils/track-index.js';
import '@components/playlist-picker/playlist-picker.js'; import '@components/playlist-picker/playlist-picker.js';
import type { TrackDetails } from '@components/track-details/track-details.js'; import type { TrackDetails } from '@components/track-details/track-details.js';
import type { CoverArtUrls } from '@components/track-details/track-details.js';
import { import {
ICON_PLAY, ICON_PLAY,
ICON_PLAYLIST, ICON_PLAYLIST,
@@ -155,7 +154,7 @@ export class TrackList
* parent is responsible for reloading when data changes. * parent is responsible for reloading when data changes.
*/ */
@property({ type: Array, attribute: false }) @property({ type: Array, attribute: false })
externalTracks?: library.Track[]; externalTracks?: ListTrack[];
/** /**
* What a host embedding this list (e.g. `genre-details`) should say * What a host embedding this list (e.g. `genre-details`) should say
@@ -199,7 +198,7 @@ export class TrackList
private lastSearchTerm = ''; private lastSearchTerm = '';
/** Tracks the store's cached array reference to detect refreshes. */ /** Tracks the store's cached array reference to detect refreshes. */
private lastTracksRef: library.Track[] | null = private lastTracksRef: ListTrack[] | null =
null; null;
/** /**
@@ -242,7 +241,7 @@ export class TrackList
} }
@state() @state()
private tracks: library.Track[] = []; private tracks: ListTrack[] = [];
@query('#context-menu') @query('#context-menu')
private contextMenuPopup!: MenuSurface; private contextMenuPopup!: MenuSurface;
@@ -269,16 +268,16 @@ export class TrackList
private lastActiveTrackPath: string | null = null; private lastActiveTrackPath: string | null = null;
// -- Memoisation caches for filtered / sorted tracks -- // -- Memoisation caches for filtered / sorted tracks --
private cachedFilteredTracks: library.Track[] = []; private cachedFilteredTracks: ListTrack[] = [];
private cachedSortedTracks: library.Track[] = []; private cachedSortedTracks: ListTrack[] = [];
private cachedRelevanceScores = new Map< private cachedRelevanceScores = new Map<
string, string,
number number
>(); >();
private prevFilterTracks: library.Track[] = []; private prevFilterTracks: ListTrack[] = [];
private prevFilterTerm = ''; private prevFilterTerm = '';
private prevFilterColIds = ''; private prevFilterColIds = '';
private prevSortFiltered: library.Track[] = []; private prevSortFiltered: ListTrack[] = [];
private prevSortField: string | null = null; private prevSortField: string | null = null;
private prevSortDir: SortDirection = 'asc'; private prevSortDir: SortDirection = 'asc';
@@ -523,7 +522,7 @@ export class TrackList
} }
} }
private computeFilteredTracks(): library.Track[] { private computeFilteredTracks(): ListTrack[] {
const term = this.searchCtrl.term; const term = this.searchCtrl.term;
if (!term) { if (!term) {
@@ -543,7 +542,7 @@ export class TrackList
return result.tracks; return result.tracks;
} }
private computeSortedTracks(): library.Track[] { private computeSortedTracks(): ListTrack[] {
const tracks = this.cachedFilteredTracks; const tracks = this.cachedFilteredTracks;
const hasSearch = const hasSearch =
this.cachedRelevanceScores.size > 0; this.cachedRelevanceScores.size > 0;
@@ -1337,11 +1336,6 @@ export class TrackList
super.connectedCallback(); super.connectedCallback();
this.restoreSortPreferences(); this.restoreSortPreferences();
if (this.externalTracks) {
this.tracks = this.externalTracks;
} else {
this.loadTracks();
}
this.resizeObserver = new ResizeObserver( this.resizeObserver = new ResizeObserver(
() => { () => {
this.onHostResize(); this.onHostResize();
@@ -1391,10 +1385,33 @@ export class TrackList
this.resizeObserver = null; this.resizeObserver = null;
} }
/**
* Fetch the list when this becomes the view on screen (#280).
*
* Not on connection: `index.html` renders a `<track-list>` as the
* main panel's first-paint content, so a connection-time fetch was
* the whole library loaded at launch for a landing on Home — 12 MB
* at 50 000 tracks, and the backend's peak RSS with it. The shell
* activates this element only when a navigation lands on Tracks.
*
* Called on *every* activation, not just the first: the store may
* have refetched while this view was off screen, and `getTracks()`
* answers from its cache when nothing changed.
*/
protected override onViewActivate(): void {
if (this.externalTracks) {
this.tracks = this.externalTracks;
} else {
void this.loadTracks();
}
this.attachListListeners();
}
/** Document-level listeners belong to the *visible* list. A cached /** Document-level listeners belong to the *visible* list. A cached
* list is never disconnected, so this is the only place they can be * list is never disconnected, so this is the only place they can be
* taken down again. */ * taken down again. */
protected override onViewActivate(): void { private attachListListeners(): void {
this.listenWhileActive(document, 'click', this.clearSelectionHandler); this.listenWhileActive(document, 'click', this.clearSelectionHandler);
this.listenWhileActive( this.listenWhileActive(
document, document,
@@ -1561,9 +1578,10 @@ export class TrackList
this.selection.clear(); this.selection.clear();
} }
// Re-fetch when the store delivers fresh // Re-fetch when the store delivers fresh data after an
// data after eager refetch on invalidation. // invalidation. Off screen the list does nothing with it, and
if (!this.externalTracks) { // the next activation re-reads the store (#280).
if (!this.externalTracks && this.viewActive) {
const cached = const cached =
this.libraryCtrl.cachedTracks; this.libraryCtrl.cachedTracks;
@@ -1734,7 +1752,7 @@ export class TrackList
*/ */
private resolveTrackFromEvent( private resolveTrackFromEvent(
e: Event, e: Event,
): { track: library.Track; index: number } | null { ): { track: ListTrack; index: number } | null {
const row = (e.target as HTMLElement).closest( const row = (e.target as HTMLElement).closest(
'.track-row', '.track-row',
) as HTMLElement | null; ) as HTMLElement | null;
@@ -1875,7 +1893,7 @@ export class TrackList
private onTrackRowClick( private onTrackRowClick(
e: MouseEvent, e: MouseEvent,
track: library.Track, track: ListTrack,
index: number, index: number,
) { ) {
// Clicking is also how the keyboard's starting point is chosen: // Clicking is also how the keyboard's starting point is chosen:
@@ -1884,7 +1902,7 @@ export class TrackList
this.selection.handleItemClick(e, track.FilePath, index); this.selection.handleItemClick(e, track.FilePath, index);
} }
private onTrackRowDblClick(_track: library.Track, index: number) { private onTrackRowDblClick(_track: ListTrack, index: number) {
this.selection.clear(); this.selection.clear();
this.playFromRow(index); this.playFromRow(index);
} }
@@ -1925,7 +1943,7 @@ export class TrackList
); );
} }
private onTrackContextMenu(e: MouseEvent, track: library.Track) { private onTrackContextMenu(e: MouseEvent, track: ListTrack) {
e.preventDefault(); e.preventDefault();
e.stopPropagation(); e.stopPropagation();
@@ -1939,7 +1957,7 @@ export class TrackList
private onTrackDragStart = ( private onTrackDragStart = (
e: DragEvent, e: DragEvent,
track: library.Track, track: ListTrack,
) => { ) => {
// Gather file paths: all selected if this track is selected, // Gather file paths: all selected if this track is selected,
// otherwise just the dragged track. // otherwise just the dragged track.
@@ -2129,74 +2147,26 @@ export class TrackList
this.ctxMenu.close(); this.ctxMenu.close();
} }
// The list's rows do not carry what the dialog shows (#281), so
// the dialog gets whole tracks by path, like every other opener.
private async openTrackDetails(filePath: string) { private async openTrackDetails(filePath: string) {
const track = tracksByFilePath(this.tracks).get( await showTrackDetailsForPath(
() => this.trackDetailsDialog,
filePath, filePath,
);
if (!track) return;
const ready = await loadTrackDetails(
() => void this.openTrackDetails(filePath), () => void this.openTrackDetails(filePath),
); );
if (!ready) return;
const coverArt = track.CoverArtPath
? {
coverArtPath: track.CoverArtPath,
coverArtSmall: track.CoverArtSmall,
coverArtMedium: track.CoverArtMedium,
coverArtLarge: track.CoverArtLarge,
}
: undefined;
this.trackDetailsDialog?.show(
track,
coverArt,
);
} }
private async openBatchTrackDetails( private async openBatchTrackDetails(
filePaths: string[], filePaths: string[],
) { ) {
const tracks = tracksForPaths( // The rows in hand are what the dialog reads, so a "select all"
this.tracks, // on 50 000 tracks opens it without asking the backend for them.
filePaths, await showBatchTrackDetails(
); () => this.trackDetailsDialog,
tracksForPaths(this.tracks, filePaths),
if (tracks.length === 0) return;
const ready = await loadTrackDetails(
() => void this.openBatchTrackDetails(filePaths), () => void this.openBatchTrackDetails(filePaths),
); );
if (!ready) return;
// Use cover art from the first track. If all tracks share
// the same album, they share the same art.
const first = tracks[0]!;
let coverArt: CoverArtUrls | null = null;
let coverArtMixed = false;
const albumNames = new Set(tracks.map((t) => t.Album));
if (albumNames.size === 1 && first.CoverArtPath) {
coverArt = {
coverArtPath: first.CoverArtPath,
coverArtSmall: first.CoverArtSmall,
coverArtMedium: first.CoverArtMedium,
coverArtLarge: first.CoverArtLarge,
};
} else if (albumNames.size > 1) {
coverArtMixed = true;
}
this.trackDetailsDialog?.showBatch(
tracks,
coverArt,
coverArtMixed,
);
} }
// ================================================================= // =================================================================
@@ -2274,7 +2244,7 @@ export class TrackList
this.saveSortPreferences(); this.saveSortPreferences();
} }
private isActiveTrack(track: library.Track): boolean { private isActiveTrack(track: ListTrack): boolean {
const currentTrack = this.player.currentTrack; const currentTrack = this.player.currentTrack;
if (!currentTrack) return false; if (!currentTrack) return false;
@@ -2283,9 +2253,9 @@ export class TrackList
} }
private renderTrackRow = ( private renderTrackRow = (
track: library.Track, track: ListTrack,
index: number, index: number,
): unknown => { ): TemplateResult => {
const active = this.isActiveTrack(track); const active = this.isActiveTrack(track);
const selected = this.selection.isSelected( const selected = this.selection.isSelected(
track.FilePath, track.FilePath,
@@ -2568,7 +2538,7 @@ export class TrackList
scroller scroller
.items=${visibleTracks} .items=${visibleTracks}
.renderItem=${this.renderTrackRow} .renderItem=${this.renderTrackRow}
.keyFunction=${(track: library.Track) => track.FilePath} .keyFunction=${(track: ListTrack) => track.FilePath}
.layout=${this.rowLayout} .layout=${this.rowLayout}
></lit-virtualizer> ></lit-virtualizer>
`} `}
+1
View File
@@ -31,6 +31,7 @@ solid/bookmark
solid/box-open solid/box-open
solid/check solid/check
solid/chevron-down solid/chevron-down
solid/chevron-left
solid/chevron-right solid/chevron-right
solid/circle-check solid/circle-check
solid/circle-exclamation solid/circle-exclamation
@@ -1,6 +1,7 @@
import type { ReactiveController, ReactiveControllerHost } from 'lit'; import type { ReactiveController, ReactiveControllerHost } from 'lit';
import type * as library from '@go/library/models.js'; import type * as library from '@go/library/models.js';
import { libraryStore } from '../library-store'; import { libraryStore } from '../library-store';
import type { ListTrack } from '@utils/track-table';
type ViewName = 'tracks' | 'albums' | 'artists' | 'genres'; type ViewName = 'tracks' | 'albums' | 'artists' | 'genres';
@@ -55,7 +56,7 @@ export class LibraryController implements ReactiveController {
// DATA ACCESS // DATA ACCESS
// =================================================================== // ===================================================================
async getTracks(): Promise<library.Track[]> { async getTracks(): Promise<ListTrack[]> {
return libraryStore.getTracks(); return libraryStore.getTracks();
} }
@@ -85,7 +86,7 @@ export class LibraryController implements ReactiveController {
); );
} }
get cachedTracks(): library.Track[] | null { get cachedTracks(): ListTrack[] | null {
return libraryStore.getCachedTracks(); return libraryStore.getCachedTracks();
} }
+245 -65
View File
@@ -1,6 +1,6 @@
import { EventsOn } from '@runtime/runtime'; import { EventsOn } from '@runtime/runtime';
import { import {
GetTracks, GetTrackTable,
GetAlbums, GetAlbums,
GetArtists, GetArtists,
GetGenres, GetGenres,
@@ -9,6 +9,8 @@ import {
} from '@go/library/library.js'; } from '@go/library/library.js';
import type * as library from '@go/library/models.js'; import type * as library from '@go/library/models.js';
import { list } from '@utils/binding'; import { list } from '@utils/binding';
import { trackCache } from './track-cache';
import { decodeTrackTable, type ListTrack } from '@utils/track-table';
import { Events } from '../events'; import { Events } from '../events';
type ViewName = 'tracks' | 'albums' | 'artists' | 'genres'; type ViewName = 'tracks' | 'albums' | 'artists' | 'genres';
@@ -27,8 +29,16 @@ const COVER_SIZE_DEFAULT = 176;
/** localStorage key for persisted cover size. */ /** localStorage key for persisted cover size. */
const COVER_SIZE_KEY = 'cover-grid-size'; const COVER_SIZE_KEY = 'cover-grid-size';
/**
* How long the small-collection warm-up will wait for idle before
* running anyway. It is speculative, but a busy main thread must not
* mean Albums is slow to open — the timeout is the promise that it is
* only ever deferred, never skipped.
*/
const WARM_IDLE_TIMEOUT_MS = 3_000;
class LibraryStore { class LibraryStore {
private tracks: library.Track[] | null = null; private tracks: ListTrack[] | null = null;
private albums: library.Album[] | null = null; private albums: library.Album[] | null = null;
private artists: library.Artist[] | null = null; private artists: library.Artist[] | null = null;
private genres: library.GenreWithCount[] | null = null; private genres: library.GenreWithCount[] | null = null;
@@ -81,8 +91,8 @@ class LibraryStore {
private cacheGen = 0; private cacheGen = 0;
constructor() { constructor() {
EventsOn(Events.LibraryScanComplete, () => { EventsOn(Events.LibraryScanComplete, (payload: unknown) => {
this.invalidate(); this.applyScanComplete(payload);
}); });
EventsOn(Events.LibraryRemoved, () => { EventsOn(Events.LibraryRemoved, () => {
this.libraries = null; this.libraries = null;
@@ -98,8 +108,8 @@ class LibraryStore {
this.changeGen++; this.changeGen++;
this.notify(); this.notify();
}); });
EventsOn(Events.TrackMetadataChanged, () => { EventsOn(Events.TrackMetadataChanged, (payload: unknown) => {
this.invalidate(); this.applyMetadataChanged(payload);
}); });
EventsOn(Events.TrackPlayCountChanged, (payload: unknown) => { EventsOn(Events.TrackPlayCountChanged, (payload: unknown) => {
this.applyPlayCount(payload); this.applyPlayCount(payload);
@@ -109,34 +119,82 @@ class LibraryStore {
}); });
this.loadCoverSize(); this.loadCoverSize();
this.deferEagerFetch(); this.warmSmallCollectionsOnIdle();
} }
/** /**
* Schedules eagerFetch() to run after the DOM is ready. * Warm the three small collections once the app is idle after first
* The LibraryStore singleton is instantiated during ES module * paint, and deliberately not the tracks (#280).
* evaluation (import time), so calling eagerFetch() in the *
* constructor would fire 4 backend roundtrips before the app * All four used to be fetched together at `DOMContentLoaded`,
* shell has rendered. Deferring to the 'DOMContentLoaded' * whichever view was showing. On a 26 138-track library the track
* event (or calling immediately if the DOM is already parsed) * list was 20.5 MB of that, and encoding it cost the backend ~170 MB
* lets the shell paint first, then begins data loading. * of transient allocation — paid by someone looking at Home, which
* needs none of it. Measured on 50 000 tracks: 543 MB of backend RSS
* at rest before, 296 MB after, and 12.1 MB of binding bytes instead
* of 35.9.
*
* Albums, artists and genres are 1.6 MB together, so they are still
* fetched ahead of the click — that is what made those views
* instant. Tracks are 12 MB at 50 000 and are fetched by the view
* that draws them, or by `prefetch` from a hover.
*
* Idle rather than immediate: this is speculative, so it must not
* compete with the first paint.
*/ */
private deferEagerFetch(): void { private warmSmallCollectionsOnIdle(): void {
if (document.readyState === 'loading') { const warm = () => {
window.addEventListener( const logged = this.failureReporter();
'DOMContentLoaded',
() => { void this.getAlbums().catch(logged('albums'));
this.eagerFetch(); void this.getArtists().catch(logged('artists'));
}, void this.getGenres().catch(logged('genres'));
{ once: true }, };
);
if (typeof requestIdleCallback === 'function') {
requestIdleCallback(warm, { timeout: WARM_IDLE_TIMEOUT_MS });
} else { } else {
// DOM already parsed (shouldn't happen during module setTimeout(warm, 0);
// eval, but handles dynamic instantiation safely).
this.eagerFetch();
} }
} }
/**
* Start loading what a view will need, without waiting for it.
*
* A nav item calls this on hover or focus: that is the ~100 ms
* before the click, and it is what #280 trades for not paying for
* every collection at startup whether or not anyone goes there.
*
* A view this store holds nothing for is ignored rather than an
* error — it is a hint, and a hint about Home is not a mistake.
* The failure is swallowed here because the view that wanted the
* data reports it: this is the same request, already deduplicated
* by `inFlight`, and it has no caller to reject to.
*/
prefetch(view: string): void {
const started = (() => {
switch (view) {
case 'tracks':
return this.getTracks();
case 'albums':
return this.getAlbums();
case 'artists':
return this.getArtists();
case 'genres':
return this.getGenres();
default:
return null;
}
})();
void started?.catch(() => undefined);
}
private failureReporter(): (what: string) => (err: unknown) => void {
return (what) => (err) =>
console.error(`library: could not load ${what}`, err);
}
// =================================================================== // ===================================================================
// DATA ACCESS // DATA ACCESS
// Returns cached data or fetches from backend on first access. // Returns cached data or fetches from backend on first access.
@@ -214,18 +272,18 @@ class LibraryStore {
} }
} }
async getTracks(): Promise<library.Track[]> { async getTracks(): Promise<ListTrack[]> {
if (this.tracks !== null) { if (this.tracks !== null) {
return this.tracks; return this.tracks;
} }
const pending = this.pending<library.Track[]>('tracks'); const pending = this.pending<ListTrack[]>('tracks');
if (pending) return pending; if (pending) return pending;
return this.track( return this.track(
'tracks', 'tracks',
list(GetTracks(this.libraryFilter())), GetTrackTable(this.libraryFilter()).then(decodeTrackTable),
(tracks) => { (tracks) => {
this.tracks = tracks; this.tracks = tracks;
}, },
@@ -341,7 +399,7 @@ class LibraryStore {
// Synchronous access for controllers that need current cached values. // Synchronous access for controllers that need current cached values.
// =================================================================== // ===================================================================
getCachedTracks(): library.Track[] | null { getCachedTracks(): ListTrack[] | null {
return this.tracks; return this.tracks;
} }
@@ -497,6 +555,116 @@ class LibraryStore {
// INVALIDATION // INVALIDATION
// =================================================================== // ===================================================================
/**
* A scan finished. Reload only if it changed something (#282).
*
* `ScanMetrics` says how many files were added, updated and removed,
* and a soft rescan of an unchanged library reports zero of each —
* yet every loaded collection was thrown away and refetched, which
* at 26 138 tracks is 20.5 MB across the IPC for a scan that found
* nothing. Anything non-zero still reloads everything: a scan can
* change a tag, an album name or a genre on any file it touched, and
* it reports counts rather than paths.
*
* A payload that says nothing at all is treated as a change, not as
* a no-op: an unknown shape must not be able to leave a stale list
* on screen.
*/
private applyScanComplete(payload: unknown): void {
const m = payload as {
added?: number;
updated?: number;
removed?: number;
} | null;
if (!m) {
this.invalidate();
return;
}
const changed = (m.added ?? 0) + (m.updated ?? 0) + (m.removed ?? 0);
if (changed === 0) return;
this.invalidate();
}
/**
* Tags were rewritten on disk. A single file is patched; a batch is
* a reload (#282).
*
* The event names the one file it rewrote, and that file's row is
* the only row that changed — so re-reading the whole list to pick
* up one new title is 20.5 MB at 26 138 tracks. A batch write
* carries no paths (it can be thousands of files), and the summaries
* really do change with it, so that one still reloads.
*/
private applyMetadataChanged(payload: unknown): void {
const p = payload as { filePath?: string; batch?: boolean } | null;
if (!p?.filePath || p.batch) {
this.invalidate();
return;
}
this.patchOneTrack(p.filePath);
}
/**
* Re-read one file's row and splice it in.
*
* The summaries are dropped and refetched, because a retag can move
* a track between albums and change a genre — they are the small
* collections, and they are what makes the album and artist views
* agree with the row that was just patched.
*/
private patchOneTrack(filePath: string): void {
const loaded = this.loadedCollections();
void trackCache
.refresh([filePath])
.then(([fresh]) => {
if (!fresh || this.tracks === null) return;
const idx = this.tracks.findIndex(
(t) => t.FilePath === filePath,
);
// Not in this view's list (another library's file, or
// gone): the list is right as it stands.
if (idx === -1) return;
this.tracks = [
...this.tracks.slice(0, idx),
fresh,
...this.tracks.slice(idx + 1),
];
this.changeGen++;
this.notify();
})
.catch((err: unknown) => {
// The patch failed, so the list may now be stale. Fall
// back to the answer that cannot be wrong.
console.error('library: could not re-read a retagged track', err);
this.invalidate();
});
this.albums = null;
this.artists = null;
this.genres = null;
this.cacheGen++;
this.inFlight.delete('albums');
this.inFlight.delete('artists');
this.inFlight.delete('genres');
this.refetchLoaded({
albums: loaded.albums,
artists: loaded.artists,
genres: loaded.genres,
});
}
/** /**
* Patch one track's play statistics in place. * Patch one track's play statistics in place.
* *
@@ -520,10 +688,11 @@ class LibraryStore {
private applyPlayCount(payload: unknown): void { private applyPlayCount(payload: unknown): void {
if (this.tracks === null) return; if (this.tracks === null) return;
// The list does not carry LastPlayed (#281); trackCache patches
// it on the whole tracks the details dialog reads.
const p = payload as { const p = payload as {
filePath?: string; filePath?: string;
playCount?: number; playCount?: number;
lastPlayed?: string;
} | null; } | null;
if (!p?.filePath) return; if (!p?.filePath) return;
@@ -536,14 +705,10 @@ class LibraryStore {
if (existing === undefined) return; if (existing === undefined) return;
const patched = Object.assign( const patched: ListTrack = {
Object.create(Object.getPrototypeOf(existing) as object), ...existing,
existing, PlayCount: p.playCount ?? existing.PlayCount,
{ };
PlayCount: p.playCount ?? existing.PlayCount,
LastPlayed: p.lastPlayed ?? existing.LastPlayed,
},
) as library.Track;
this.tracks = [ this.tracks = [
...this.tracks.slice(0, idx), ...this.tracks.slice(0, idx),
@@ -597,6 +762,8 @@ class LibraryStore {
} }
} }
const loaded = this.loadedCollections();
this.albums = null; this.albums = null;
this.artists = null; this.artists = null;
this.genres = null; this.genres = null;
@@ -612,50 +779,63 @@ class LibraryStore {
this.changeGen++; this.changeGen++;
this.notify(); this.notify();
const logged = (what: string) => (err: unknown) => // Only the summaries something was showing (#280): a removal
console.error(`library: could not reload ${what}`, err); // nobody was looking at does not load a collection to correct it.
this.refetchLoaded({
void this.getAlbums().catch(logged('albums')); albums: loaded.albums,
void this.getArtists().catch(logged('artists')); artists: loaded.artists,
void this.getGenres().catch(logged('genres')); genres: loaded.genres,
});
} }
private invalidate(): void { private invalidate(): void {
// Which collections something has actually loaded, before they
// are dropped. Refetching all four here would undo #280: a scan
// finishing would fetch the track list of a library nobody has
// opened the Tracks view on.
const loaded = this.loadedCollections();
this.tracks = null; this.tracks = null;
this.albums = null; this.albums = null;
this.artists = null; this.artists = null;
this.genres = null; this.genres = null;
// Anything still in flight was asked for on behalf of a // Anything still in flight was asked for on behalf of a
// selection that no longer applies: forget it, so the eager // selection that no longer applies: forget it, so the refetch
// refetch below starts a request for the current one rather // below starts a request for the current one rather than
// than adopting the old one's answer. // adopting the old one's answer.
this.inFlight.clear(); this.inFlight.clear();
this.cacheGen++; this.cacheGen++;
this.changeGen++; this.changeGen++;
this.scrollPositions = { tracks: 0, albums: 0, artists: 0, genres: 0 }; this.scrollPositions = { tracks: 0, albums: 0, artists: 0, genres: 0 };
this.notify(); this.notify();
this.eagerFetch(); this.refetchLoaded(loaded);
}
/** The collections currently held, keyed by the view that draws them. */
private loadedCollections(): Partial<Record<ViewName, boolean>> {
return {
tracks: this.tracks !== null,
albums: this.albums !== null,
artists: this.artists !== null,
genres: this.genres !== null,
};
} }
/** /**
* Fetches all library data. Called after DOM ready * Refetch exactly the collections named, and nothing else.
* (initial load, via deferEagerFetch) and after cache *
* invalidation so that controller subscribers receive * A failed fetch is reported by whichever view asked for the data
* fresh data on the next requestUpdate() cycle without * (it is that panel's failure, not the app's), but this refetch has
* needing their own LibraryScanComplete listener. * no caller to reject to — without a catch it is an unhandled
* rejection.
*/ */
private eagerFetch(): void { private refetchLoaded(loaded: Partial<Record<ViewName, boolean>>): void {
// A failed fetch is reported by whichever view asked for the const logged = this.failureReporter();
// data (it is that panel's failure, not the app's), but the
// eager refetch has no caller to reject to — without a catch it
// is an unhandled rejection.
const logged = (what: string) => (err: unknown) =>
console.error(`library: could not load ${what}`, err);
void this.getTracks().catch(logged('tracks')); if (loaded.tracks) void this.getTracks().catch(logged('tracks'));
void this.getAlbums().catch(logged('albums')); if (loaded.albums) void this.getAlbums().catch(logged('albums'));
void this.getArtists().catch(logged('artists')); if (loaded.artists) void this.getArtists().catch(logged('artists'));
void this.getGenres().catch(logged('genres')); if (loaded.genres) void this.getGenres().catch(logged('genres'));
} }
// =================================================================== // ===================================================================
Loaded 100 of 143 files, more files were not shown because too many files have changed in this diff. Show more