Compare commits

..
75 Commits
Author SHA1 Message Date
yonlu bf0a53e64c Merge pull request 'Give the unclaim step a CA bundle' (#110) from fix/unclaim-ca-certs into main
Release / release (push) Successful in 31s
CI / e2e (push) Successful in 6m3s
CI / check (push) Successful in 2m22s
Build & publish the Android APK / apk (push) Successful in 1m24s
Build & publish Arch package / arch-package (push) Successful in 2m29s
Attach the desktop build to the release / linux (push) Successful in 52s
Sync Homebrew formula / sync-formula (push) Successful in 6s
Reviewed-on: #110
2026-08-18 23:30:31 +00:00
yonlu 7be4a02e31 fix(ci): give the unclaim step a CA bundle
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m27s
CI / e2e (pull_request) Successful in 6m14s
Second defect in the same workflow. The shell fix took -- the step ran
under `bash --noprofile --norc -e -o pipefail` -- and got one layer
further before failing:

  curl: (77) error setting certificate file: /etc/ssl/certs/ca-certificates.crt

ubuntu:24.04 ships no CA bundle, and --no-install-recommends skips the
ca-certificates that curl recommends, so curl came up unable to verify
TLS against our own Gitea.

This was avoidable by reading the repo rather than reasoning about it:
ci.yml (twice), desktop-assets.yml, android-apk.yml and release.yml all
spell out `ca-certificates curl ... jq` for exactly this reason. The
convention was written down five times already.

Validated in the real image this time rather than by extracting the
script and running it on the host, which is what missed this: the step
now succeeds inside `docker run ubuntu:24.04` against a scratch issue --
label present, 204, label gone -- and the previous version reproduces
`curl: (77)` in the same image. Both checked, then the scratch issue was
deleted.

The DELETE also keeps its response body now and prints it on a non-204.
Whether the automatic token carries issue-write scope is still unproven,
because both failures happened before the API call, and "403" without
Gitea's own sentence would cost another merge to interpret.

Closes #102
2026-08-18 19:08:50 -04:00
yonlu ad9c25a5a2 Merge pull request 'Run the unclaim step under bash' (#108) from fix/unclaim-shell into main
Release / release (push) Successful in 32s
CI / e2e (push) Successful in 6m10s
CI / check (push) Successful in 2m23s
Build & publish the Android APK / apk (push) Successful in 1m23s
Build & publish Arch package / arch-package (push) Successful in 2m37s
Attach the desktop build to the release / linux (push) Successful in 52s
Sync Homebrew formula / sync-formula (push) Successful in 7s
Reviewed-on: #108
2026-08-18 23:05:38 +00:00
yonlu a83a127e31 fix(ci): run the unclaim step under bash
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m28s
CI / e2e (pull_request) Successful in 6m14s
The workflow shipped in #103 and failed on every close, on its second
line, before reaching the API:

  shell: sh -e {0}
  /var/run/act/workflow/0.sh: 2: set: Illegal option -o pipefail

Inside `container:` the act runner selects sh, not bash, and
`set -o pipefail` is a bashism. homebrew-formula.yml carries the same
line without trouble because it runs with no container, on the host
image where bash is the default -- so "another workflow does it" was
not the evidence it looked like, and the comment now says so where the
next person will read it.

pipefail is kept rather than dropped for POSIX's sake: the lookup is
`curl -sSf ... | jq`, so without it an API error yields empty output,
an empty label id, and a cheerful "nothing to do" on every close. A
silent no-op is the one outcome worse than a failing job here.

Validated end to end against scratch issues rather than by reading it:
with the label present the step returns 204 and the label is gone, and
against an issue that never carried it the step also returns 204 and
exits 0 -- which is what makes it safe to run on every close rather
than only claimed ones.

Still untested: whether secrets.GITEA_TOKEN carries issue-write scope.
The old run never got far enough to find out. If it 403s, the fix is
one line -- secrets.PACKAGE_TOKEN, which is a user PAT.

Closes #102
2026-08-18 18:54:39 -04:00
yonlu e049a71458 Merge pull request 'Drop the claim label when an issue closes' (#103) from ci/unclaim-on-close into main
CI / check (push) Successful in 2m30s
Release / release (push) Successful in 31s
CI / e2e (push) Successful in 6m6s
Reviewed-on: #103
2026-08-18 21:57:11 +00:00
yonlu 0c944f2382 ci: drop the claim label when an issue closes
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m27s
CI / e2e (pull_request) Successful in 6m17s
A `Closes #N` footer closes the issue on merge and leaves
`Status/In Progress` on it, because Gitea's auto-close touches state
and nothing else. #100 was closed and simultaneously marked as being
actively worked on. `scripts/issue.sh close` does drop the label, and
is exactly the call the footer exists to avoid making.

This hooks the close rather than the merge. Stripping the label in the
PR would work and would be a per-PR habit, which is what the footer
removed in the first place; `issues: [closed]` covers the footer,
issue.sh close and a click in the web UI alike, and asks nothing of
anyone at any of them.

Reopening deliberately does not restore the label: reopening says the
work was not finished, not that somebody is at a keyboard now.

Two costs, both stated in the file rather than discovered later. The
runner has capacity 1 and is shared with an index build that can hold
it for three hours, so this is not instant -- stale for an afternoon
beats stale forever, which is what it was. And it is an eighth
workflow, so CLAUDE.md's count moves with it.

The audit stays, because a workflow that silently stops firing is the
failure mode this area has already produced once:

  ./scripts/issue.sh list --state closed --label "Status/In Progress"

Closes #102
2026-08-18 17:36:05 -04:00
yonlu 75525b67e4 Merge pull request 'Put the closing keyword where Gitea will actually read it' (#101) from docs/closing-keyword into main
CI / check (push) Successful in 2m27s
Release / release (push) Successful in 31s
CI / e2e (push) Successful in 6m15s
Reviewed-on: #101
2026-08-18 21:30:35 +00:00
yonlu 85768dc489 docs: put the closing keyword where Gitea will actually read it
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m27s
CI / e2e (pull_request) Successful in 6m7s
CLAUDE.md said the Closes list was unreliable and to close by hand. It
is unreliable for a specific reason, and the rule can say what works.

Gitea parses commit messages that reach main. It does not parse the PR
body, which closes something only if the merge happens to copy it into
the merge commit message. Both halves were measured here: #83's merge
commit carried "Closes #9, #13, #14, ..." and closed five of the ten,
because a comma list is only partially matched; #93's merge commit body
was a lone Reviewed-on: trailer, so #92 stayed open behind a perfectly
correct Closes line in the PR description.

So the keyword goes in the commit body as a footer, one issue per line.
That costs nothing elsewhere -- Conventional Commits allows a footer,
commit-check only regexes the subject, and semantic-release reads the
type from the subject, so no release decision changes. The existing
rule that the issue number stays out of the subject is untouched and
was never about the body.

The verification step stays, because a squash or a hand-edited merge
message still drops the footer.

This commit is the experiment: if #98 and #100 close when this branch
merges without anyone touching them, the mechanism is confirmed.

Closes #98
Closes #100
2026-08-18 17:15:28 -04:00
yonlu 1a221a40d3 Merge pull request 'Delete four documents that contradict the code' (#99) from docs/retire-stale-planning-docs into main
CI / check (push) Successful in 2m26s
Release / release (push) Successful in 29s
CI / e2e (push) Successful in 6m14s
Reviewed-on: #99
2026-08-18 21:06:59 +00:00
yonlu 0821deb877 Merge branch 'main' into docs/retire-stale-planning-docs
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m26s
CI / e2e (pull_request) Successful in 6m17s
2026-08-18 20:55:52 +00:00
yonlu 31ada14111 docs: delete four documents that contradict the code
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m27s
CI / e2e (pull_request) Successful in 6m9s
They do not merely lag it, they contradict it, which is worse than
absent: the only way to find out one is wrong is to trust it.

docs/dev/roadmap.md, 1648 lines. Its "Current State" describes an app
with basic playback, a track list and an album grid. Phases 1, 2, 3 and
5 have shipped entire -- playlists, shuffle and repeat, shortcuts,
virtualised lists, search, custom columns, smart playlists, the
MusicBrainz client, autotag. Its Decision Log records "Testing:
Deferred" against 836 Vitest tests, a Playwright suite on two engines
and race-detector passes in three build configurations.

.pi/journal.md, "what happened and what's next", frozen at plan 005 and
still saying everything from phase 1 onward is uncommitted. That is the
tracker's job now, and a work log that is wrong about what is committed
is a hazard rather than a stale file.

docs/dev/overview.md, a shallower and partly incorrect CLAUDE.md
architecture section. README pointed at it and points at CLAUDE.md now.

docs/dev/config-suggestions.md, eight suggestions of which one survived
-- the rest were overtaken by rewrites: applyDefaults exists, scan
concurrency is configurable with SSD/HDD detection, and httphandler.go
and the WriteHeader-after-render bug it described are gone entirely.

Nothing is lost: the genuinely unbuilt work was filed first, as #94
device sync, #95 layout customisation, #96 AcoustID and #97 the config
setters that take no lock.

Refs #98
2026-08-18 16:39:21 -04:00
yonlu 20139394f3 Merge pull request 'Make the issue tracker the source of truth' (#93) from docs/issue-driven-workflow into main
CI / check (push) Successful in 2m27s
CI / e2e (push) Successful in 6m14s
Release / release (push) Successful in 30s
Reviewed-on: #93
2026-08-18 20:35:44 +00:00
yonlu eb139cf872 docs: make the issue tracker the source of truth
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m28s
CI / e2e (pull_request) Successful in 6m9s
Work has been starting from a chat message and a plan file, so two
people could pick up the same thing and neither could see the other.
The tracker is where that is visible.

Search before starting, claim before the first edit -- not before the
commit, since the point is that the other person can see the work is
taken while it is being done. If no issue covers it, open one first:
that is what makes the tracker a description of the project rather than
a description of the past.

The conventions were already right and are written down rather than
reinvented -- the Kind/Area/Priority/Platform/Reviewed/Status taxonomy,
its exclusive scopes, #73 as the roadmap, real Gitea dependencies for
hard blockers, and PR #83's body shape.

What #83 also demonstrated is that a Closes list closes nothing
reliably: it listed ten and five of them sat open in main for a
fortnight. So closing is a step you take and verify, not a keyword you
trust.

.planning/ stops being a queue and keeps design documents and measured
history -- NOTES.md, the audits, the completed plans and the arguments
in them. plans/pending/ is gone, because a plan nobody is executing is
an issue; everything unimplemented in it is now #85-#91, and each
completed plan says which issue carries its remainder. autotag.md is
kept as a historical record, marked stale where the scoring overhaul
overtook it.

The commit grammar is unchanged and is load-bearing for a different
reason, so the issue number lives in the branch name and the PR body
rather than the commit subject.

Refs #92
2026-08-18 16:23:52 -04:00
yonlu ae82fd2233 chore(scripts): reach the issue tracker from the command line
Issues become this project's source of truth for what is wanted and what
is already being worked on, which puts "search the tracker" at the top of
every task rather than occasionally. Fifty-odd open issues make that a
real lookup, and a lookup nobody can remember the shape of is a lookup
that gets skipped -- the same way the CI log endpoint cost two sessions
to a tool that 404s.

Text reaches the API as JSON and never as shell, which is why the
formatting half is its own Python file: an issue body is arbitrary prose
carrying backticks, quotes and $, and every attempt to build that JSON
inside the shell ends in nested quoting nobody can verify. Same reasoning
that keeps release notes out of gitea-release.sh's argument list.

Claiming is an assignment, a label and a comment together, because any
one alone is a claim somebody has to go looking for. It resolves the
comment before it mutates anything -- reading it afterwards is how a
claim ends up half-made, with the issue saying it is taken without saying
by what work -- and refuses outright if somebody else holds it.

Three API shapes are pinned here because each fails quietly:

- Labels are resolved to ids rather than posted as names. Gitea accepts
  a list of unknown names with 200 and applies none of them, so a typo
  reports success and does nothing.
- The dependency endpoint takes a whole IssueMeta, not an index. A body
  of {"index": 88} answers 404, which reads exactly like a Gitea build
  without the feature.
- close drops Status/In Progress, or a claim outlives the work.

Refs #92
2026-08-18 16:23:39 -04:00
logan 3c3197df4b Small-fix batch: ten issues from the desktop backlog (#83)
Release / release (push) Successful in 33s
CI / e2e (push) Successful in 6m10s
CI / check (push) Successful in 2m26s
Build & publish the Android APK / apk (push) Successful in 1m26s
Build & publish Arch package / arch-package (push) Successful in 2m36s
Attach the desktop build to the release / linux (push) Successful in 56s
Sync Homebrew formula / sync-formula (push) Successful in 6s
Rolls up #82 (@yonlu) and #74, #75, #76, #77, #78, #79, #81 as one push, so the batch cuts one release rather than eight.

Closes #9, #13, #14, #19, #26, #29, #33, #35, #37, #41.
2026-08-18 16:18:36 +00:00
logan e16bd245bd Merge remote-tracking branch 'origin/fix/queue-toggle-state' into integration/small-fixes
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m28s
CI / e2e (pull_request) Successful in 6m14s
2026-08-18 11:37:33 -04:00
logan 887a9324b4 Merge remote-tracking branch 'origin/fix/drag-count-badge' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan fcb484ead5 Merge remote-tracking branch 'origin/fix/album-card-year' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan 48de41cd69 Merge remote-tracking branch 'origin/fix/album-tracklist-heading' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan 66a6ee63ab Merge remote-tracking branch 'origin/fix/seek-bar-clock-width' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan 10660c8168 Merge remote-tracking branch 'origin/fix/wanted-without-client' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan 441b67daaa Merge remote-tracking branch 'origin/fix/album-track-request-badge' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan 026f26bdf6 Merge remote-tracking branch 'origin/fix/small-issue-batch' into integration/small-fixes 2026-08-18 11:37:33 -04:00
logan 73dc80bdc9 fix(explore): stop hiding the request badge until the row is hovered
CI / e2e (push) Skipped
CI / check (push) Skipped
CI / check (pull_request) Successful in 2m29s
CI / e2e (pull_request) Canceled after 0s
The badge on a row you do not own was transparent until the row was
hovered or focused. That rule was inherited from the green ticks it
replaced, and it does not survive the reason those went: a tick marked
the *common* case, while this marks the rows that are not here. A mark
on the exception is the information on this page, and one that appears
only under the pointer cannot be seen, counted, or reached by anyone
driving the app with a finger.

The repaint half of #33 is fixed in #82; this is only the visibility,
rebased to leave that alone.

Refs #33
2026-08-18 11:31:49 -04:00
logan 760021ea5a fix(downloads): stop searching a list there is nothing to search with
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m35s
CI / e2e (pull_request) Canceled after 0s
Every pass attempted every request, each came back "no download clients
are enabled", and RecordAttempt wrote that down as an attempt and put a
retry on the clock -- so a wanted list built deliberately without a
client accrued failures and announced "next check in 6 hours" about a
check that cannot happen.

Wanting something with no way to fetch it is supported. Being told it
is being looked for is a lie, and the row says what is true instead.

Everything above the attempt still runs: an artist subscription still
expands, and a request satisfied by some other route -- ripped, bought,
copied in -- is still retired. Neither needs a provider.

TestReconcileRespectsBatchSize now installs a client that finds
nothing, because a batch size is about how many requests one pass
searches for and that only means something when there is something to
search with.

Refs #37
2026-08-18 11:26:35 -04:00
yonlu 63ec068add Merge branch 'main' into fix/small-issue-batch
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m31s
CI / e2e (pull_request) Canceled after 0s
2026-08-18 15:22:56 +00:00
logan a2ff0aed4c fix(ui): make the queue button say whether the queue is open
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m27s
CI / e2e (pull_request) Canceled after 5m49s
It looked identical in both states, so the only way to tell what
pressing it would do was to look at the other side of the window and
infer it -- and for anyone not looking there was nothing to infer from:
no aria-expanded, no aria-controls, no drawn state.

The state is reflected *from the panel* rather than kept beside the
click. This button is not the only thing that opens the queue --
now-playing-view sets the same attribute, because it hides the bar the
button lives in -- so a flag maintained by the click handler would be
right until something else opened the panel and then quietly wrong.
The panel's `open` attribute stays the one fact; a MutationObserver
reflects it.

Refs #26
2026-08-18 11:15:32 -04:00
logan 12e75ee24c feat(ui): badge an album drag with how many tracks it carries
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m25s
CI / e2e (pull_request) Successful in 6m5s
Dragging an album to the queue put its cover under the cursor and said
nothing about how much that was -- an album is 1 track or 30 and the
thumbnail is the same picture either way, so the one number the drop is
about was the one thing the drag did not show. Every other drag in the
app already says it; this was the exception, because it had a picture
to show instead.

A count of 1 draws no badge: "1" over a single cover is noise, and the
absence reads clearly beside a badge that only appears above one.

The badge sits inside the cover's box rather than overhanging it,
because setDragImage snapshots the element and anything outside it
risks being clipped -- while padding the box instead would move the
cover away from the cursor.

Refs #19
2026-08-18 11:10:09 -04:00
logan 792e87298b fix(ui): stop the album grid eating the year it was sorted by
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m30s
CI / e2e (pull_request) Successful in 6m5s
The year sat inside the same ellipsis box as the title, so it was the
first thing truncation took: a card wide enough for a long album name
never showed its year, and browsing the grid *by year* showed years
only for the albums with short names. The sort said one thing and the
cards showed another.

Title and year are now a flex row where only the title gives way. A
row rather than a second line, because the card's height is what the
virtualizer measures rows by.

Refs #29
2026-08-18 11:08:37 -04:00
logan 266e7032dd fix(explore): stop labelling the album tracklist "TRACKLIST"
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m31s
CI / e2e (pull_request) Successful in 6m9s
A list of numbered titles with durations, under the album's cover, was
the one thing on the page carrying a word above it saying what it is.

What goes is the ink and not the element: the section is a landmark and
the page's heading structure runs through it, so the h3 stays and is
clipped the way sr-only clips -- never display:none, which would take
it out of the accessibility tree along with the layout.

Refs #9
2026-08-18 11:06:14 -04:00
logan d6b48fb3ac fix(player): stop the seek bar resizing as its clocks count
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m28s
CI / e2e (pull_request) Successful in 6m16s
Two different things moved it and they need different answers. Digits
in a proportional font are different widths, so 1:11 is narrower than
4:08 and the bar breathed once a second -- tabular figures fix that.
The character *count* changes too, at the hundredth minute and whenever
the right-hand clock is toggled to remaining and grows a minus sign,
which a figure width cannot fix -- so each clock reserves the widest
string this track can put in it.

The budget is per track rather than a constant: reserving six
characters on every track would push the slider in by a character at
each end to buy nothing.

Measured in the component tier: 4.5px of drift across three positions
before, none after.

Refs #13
2026-08-18 11:04:21 -04:00
yonlu 3bf27e3fd5 Merge pull request 'Fix/explore art scanner requests' (#21) from fix/explore-art-scanner-requests into main
CI / check (push) Skipped
CI / e2e (push) Skipped
Release / release (push) Successful in 32s
Build & publish the Android APK / apk (push) Successful in 1m26s
Build & publish Arch package / arch-package (push) Successful in 2m35s
Attach the desktop build to the release / linux (push) Successful in 1m12s
Sync Homebrew formula / sync-formula (push) Successful in 9s
CI / check (pull_request) Canceled after 0s
CI / e2e (pull_request) Canceled after 0s
Reviewed-on: #21
2026-08-18 13:48:37 +00:00
yonluandClaude Opus 5 185eb1b125 feat(smartplaylist): let a rule set match any rule, not only all of them
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Canceled after 0s
CI / e2e (pull_request) Canceled after 0s
The conditions were joined with " AND " and nothing else, so a smart
playlist could only ever narrow: "jazz released after 1960" was
expressible and "jazz or blues" was not, which is most of what anyone
reaches for a second rule to say.

`RuleSet.Match` is "all" or "any", and an empty match is "all" — which
is what every playlist saved before the field existed carries, so an
upgrade cannot silently widen one. ParseRuleSet rejects anything else
rather than falling through to AND, since a playlist quietly returning
the wrong tracks is worse than one that refuses to be saved.

Under OR each condition is parenthesised and under AND it is not: AND
is the tighter operator, so an OR-join has to protect a condition
carrying a top-level AND of its own — `days_since_played less_than` is
two predicates belonging to one rule.

The editor shows the choice as a sentence with the control in the
middle, and hides it while there is one rule: with nothing to combine,
all and any are the same query.

Closes #35

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 08:06:16 -04:00
yonluandClaude Opus 5 b3556d825c fix(mediacontrols): always send an art URL, even when there is no art
Every other key in the MPRIS metadata map can be omitted safely,
because a client reading it renders a track with no title as a track
with no title. Art is different: KDE's applet treats an absent
mpris:artUrl as no news about the art and keeps drawing whatever the
last track had, so playing something without a cover left the previous
album's sleeve on screen — which reads as the wrong track playing
rather than as missing artwork.

The map's construction moves out of UpdateMetadata into a pure
metadataMap so it can be asserted on at all: everything else in this
file needs a live session bus, which is the same reason the Android
contract lives in an untagged file.

Closes #41

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 08:06:01 -04:00
yonluandClaude Opus 5 bf4f352117 fix(queue): stop claiming a queue came from somewhere it no longer does
`q.source` was written by SetQueue and cleared in exactly one place,
Clear, so no append path touched it: adding a track to a queue built
from an album left the page still offering "Playing from <that album>",
and since the source is persisted alongside the queue state the wrong
label outlived the session that earned it.

Every add and insert path drops it now. Removing and reordering
deliberately do not — a queue with a track taken out of it is still
that album, and the link still goes somewhere true. Only the arrival of
a track from elsewhere makes the claim false.

The delta event carries the source for the same reason it carries the
current index: an append emits nothing else, so the frontend would keep
the label it was last given until something forced a full state.

Closes #14

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 08:05:43 -04:00
yonluandClaude Opus 5 1062b7c0bc fix(explore): tell Lit that a track request changed something
The album page's tracklist badges read `libraryStatusFor(false,
track.mbid)` at render time, which is a dependency on `downloadStore`
that Lit cannot see. The page did subscribe to that store, but its
callback only assigned `canDownload` and `isRequested` — neither of
which a *track* request changes — so no reactive field moved and the
component never re-rendered. The request was filed, the plus stayed a
plus, and clicking again cancelled it.

The other three hosts rendering these badges have always asked for the
repaint in the same place, which is what made this one look correct on
inspection.

Closes #33

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 08:04:39 -04:00
yonlu 48abecb830 Merge remote-tracking branch 'origin/main' into fix/explore-art-scanner-requests
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m26s
CI / e2e (pull_request) Successful in 6m13s
2026-08-18 07:43:25 -04:00
logan e1c07438e9 docs: record what shipping the release pipeline taught us (#4)
CI / check (push) Successful in 2m21s
Release / release (push) Successful in 31s
CI / e2e (push) Successful in 6m4s
2026-08-18 03:49:00 +00:00
logan 6e563f3846 docs: record what shipping the release pipeline taught us
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m27s
CI / e2e (pull_request) Successful in 6m8s
Moves plan 017 to completed with a recap, and lifts the three findings
that generalise into NOTES.md: a preset major that renders empty notes
with everything green, a 403 that looks like branch protection and is a
token scope, and tag-triggered workflows running the tagged commit's
own definitions.
2026-08-17 23:35:59 -04:00
logan 186f6a5839 fix(release): seed the version floor on the parent, not on HEAD (#3)
Release / release (push) Successful in 32s
CI / e2e (push) Successful in 6m7s
CI / check (push) Successful in 2m29s
Build & publish the Android APK / apk (push) Successful in 1m24s
Build & publish Arch package / arch-package (push) Successful in 2m26s
Attach the desktop build to the release / linux (push) Successful in 2m29s
Sync Homebrew formula / sync-formula (push) Successful in 6s
2026-08-18 03:10:31 +00:00
yonluandClaude Opus 5 590a0d86dd perf(library): size the scan to the drive, and prefetch what it reads
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m47s
CI / e2e (pull_request) Successful in 5m59s
Every parser in `backend/metadata` is header-only -- a few hundred
bytes and return -- so on a spinning disk a scan is not waiting on CPU
or on bytes, it is waiting on the head to arrive. Two things follow,
and the drive says which.

**How many reads should be in flight.** This was a flat 2 for anything
rotational, which is a pre-NCQ assumption: a modern SATA disk reports a
queue depth of 32 and reorders outstanding reads into the order its
head passes over them, and was being handed a quarter of what it can
use. It gets 4 now. A drive that reports 1 -- a USB bridge, a pre-2004
disk -- services one command at a time in the order given, where every
extra worker is one more seek competing for one head and the scan gets
*slower* the harder it is pushed; that keeps 2.

**And that the next seek should already be queued.** A prefetch stage
between the walk and the workers issues `POSIX_FADV_WILLNEED` over the
first 512 KB of each file -- enough for an ID3v2 tag carrying cover
art, or FLAC's STREAMINFO and PICTURE blocks. The buffered channel *is*
the lookahead: the goroutine runs 16 files ahead of the workers,
hinting as it goes, so the read a worker needs has been in flight for
sixteen files' worth of parsing by the time it asks. Rotational only;
an SSD gets the channel back unwrapped and pays nothing, since it has
no seek to hide and already has one worker per core.

`workersForProfile` is the policy on its own so it can be tested
against drives this machine does not have, and the scan logs the
device, its rotational flag and its queue depth, so the decision is
inspectable rather than inferred.

Also: `ScanConcurrency` has been a validated three-value config field
with exactly one caller, passing the constant `auto` -- so choosing
`ssd` or `hdd` by hand did nothing at all. It reads the config now.
The two modes overrule detection about the *disk* and not about its
queue, since a user who picks `hdd` on a queueing drive still wants
that drive's queue used.

What is not here is inode-ordered dispatch. It needs the streaming walk
restructured to buffer per directory, and with queueing the drive is
already reordering what the hints put in front of it; that wants a
measurement on real hardware before the complexity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:12:15 -04:00
yonluandClaude Opus 5 36af7090d9 fix(system): resolve a path to its own disk, not the first on its major
`deviceForPath` scanned `/sys/block` comparing device numbers and, when
no entry matched exactly, took the first one whose *major* agreed. Every
SATA disk is major 8. A filesystem's `st_dev` is its **partition**, so
the exact match never hits for anything on one, and the fallback then
resolved `/dev/sdb3` to whatever `/sys/block` listed first -- which is
alphabetical, which is `sda`.

On the machine this was found on that is a Samsung SSD sitting next to
the 6 TB spinning disk the library is actually on, so
`IsRotationalDisk` answered false and the scanner ran one worker per
core across a drive with one head. Matching on major alone cannot be
right on any machine with two disks, which is the case this exists for.

It goes through `/sys/dev/block/<major>:<minor>` instead -- a symlink
the kernel maintains to the device's own sysfs directory -- and climbs
to the parent when that turns out to be a partition. One readlink, no
scan, no ambiguity. The dev_t decode goes with it: Linux packs 12 bits
of major and 20 of minor split across the word, and masking the low
byte of each is right only for the first 256 of either.

`ProfileForPath` returns what the scanner needs to ask next, and the
new half is `queue_depth`: how many commands the drive will accept and
reorder at once. A SATA disk with NCQ enabled reports 31 or 32 and one
without reports 1, which is the difference between concurrency helping
and hurting. An absent file is read as "queues", because everything
that does not publish it -- NVMe, virtio, device-mapper -- is a device
where concurrency is fine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:11:51 -04:00
yonluandClaude Opus 5 3e142f8c35 test(downloads): guard the service fixture on something the fake sets
`newServiceFixture` stops auto-pick from starting a grab, because none
of its tests is about the download and a detached `go m.grab(...)`
racing `t.TempDir()`'s cleanup is how they fail. It did that with
`MaxSizeMB: 1` -- and the size gates read `Candidate.TotalSize`, which
real providers fill and the fake leaves at zero. Zero is under every
ceiling, so the guard never fired and the race it was written to
prevent kept happening, roughly one run in fifteen:

    TempDir RemoveAll cleanup: unlinkat ... : directory not empty

The guard is a format the fake never produces. Thirty consecutive
whole-package runs, none.

`TestManualDownloadSatisfiesRequestOnSuccess` was relying on the guard
being broken -- it is the one test here that wants the download -- so
it now clears the preferences itself rather than depending on a bug.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:11:10 -04:00
yonluandClaude Opus 5 3d375adab1 feat(downloads): bound auto-pick by bitrate, and take a good copy
Three faults, one subsystem, and the middle one is why a request that
looked obviously satisfiable came back refused.

**The guardrails were in megabytes, which cannot mean anything.** 300 MB
is a generous FLAC single and a suspiciously small boxset, and whoever
fills the field in has no idea which release the pipeline will apply it
to. `MinKbps`/`MaxKbps`/`PreferredKbps` are the same statement divided
by how long the music is, so one number holds across a nine-minute EP
and a three-hour opera. The runtime comes from `Download.Expected`,
which every anchored request already carries, so this costs no lookup;
the rate is audio bytes over that, falling back to the mean stated
per-file bitrate when the runtime is unknown. Artwork is excluded from
the numerator, or a folder with 30 MB of scans reads as a better rip.

An unknown runtime *passes* the window rather than failing it: the
window is a statement about quality, and refusing everything the moment
MusicBrainz is missing a track length would be a silent embargo.
`MaxFileSizeMB` survives as a separate ceiling, still in megabytes on
purpose -- it is a question about disk space, and it has to apply to a
candidate whose bitrate cannot be worked out at all.

**Auto-pick required daylight over the runner-up**, 0.08 on the
combined score, and so fired hardest in the case it was never written
for: a popular album turns up five *correct* copies, all matching the
tracklist at 95%+ and differing only in format and seeders, their
scores land within a point of each other, and it refused forever on the
grounds that the choice was the user's. It was not. There was no
question about what to fetch, only about which copy -- and abundance is
the condition under which that matters least. A candidate no longer has
to beat the field, only clear the bars on its own terms; where several
do, ranking puts the one closest to the preferred bitrate first.

That tie-break needed the preference to carry weight or it would have
been decorative in a new unit: `BitrateFit` was 0.05 against format's
0.42, so asking for 320 and being handed a FLAC every time was the
designed behaviour. When a preference is set the weights shift to fit
0.40 / format 0.20 / bitrate 0.10, taking it off the two heuristics
that exist as stand-ins for the preference the user has now given.
Health and priority are untouched. And the fit spans 0.5 to 1.0 rather
than 0 to 1, so a preference can promote the copy that matches it and
can never push the others under `minQuality` -- turning "I like 320"
into "never take anything else" silently is what `MinKbps`/`MaxKbps`
are for, out loud.

**And a refusal quoted numbers that passed.** The request list built its
message from `ranked[0]` -- the best candidate *before* the guardrails
and before the lead check -- so a request killed by the size window, or
by having too many good copies, reported "best of 12 found is not a
confident enough match (match 96%, quality 88%)". `AutoPickVeto` names
the gate that actually refused, and `AutoPickable` is that returning
empty.

Existing configs: the old `MinFileSizeMB`/`PreferredFileSizeMB` are not
migrated. A number meaning "300 MB" cannot be reinterpreted as a rate
without knowing the album it was aimed at, so carrying it over would be
inventing an intent nobody expressed. Those two fall back to no window,
which is the permissive default and what a fresh install gets;
`MaxFileSizeMB` carries over unchanged, because a ceiling on bytes
still means exactly what it did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:10:51 -04:00
yonluandClaude Opus 5 e3d492e130 fix(downloads): call a request a request, and mark it with a bookmark
The feature was renamed to requests and the copy was not. The badge on
every Explore card and track row still offered "Want track X", the
album page's button read "Want this" / "Wanted", the artist page's
release menu said "Want This", and the Downloads empty state told the
user to look for a control by a name nothing rendered.

The `queued` badge is a bookmark rather than an hourglass. An hourglass
says "wait, this is under way", which overstates what a request is:
nothing may be downloading, nothing may ever be found, and the list is
somewhere a user can leave one indefinitely. A bookmark says the honest
thing -- it is on your list -- and reads as the opposite of the plus
that put it there, which is what a toggle's two states have to do.

The backend's `'wanted'` request state is deliberately untouched: it is
a stored enum, not copy.

Also removes a dead duplicate branch in the badge's `render()`. The
first `if (this.actionable)` returned before the ring was built, so a
partly-held album that could still be requested drew a plus instead of
its progress arc.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:10:23 -04:00
yonluandClaude Opus 5 e6f30b6e43 fix(a11y): draw an unfavourited track as an outline, not a dimmer fill
`favCtrl.iconName` returned the solid glyph in both states, so "not a
favourite" was a filled heart in a duller colour and the only thing
separating the two states was hue. That fails outright for anyone who
cannot tell the two colours apart (WCAG 1.4.1), and reads as
"everything is a favourite" to everyone else.

`iconFor(favorited)` returns the outline or the fill, and the nine
`<wa-icon>` call sites split into the two cases they always were. The
three that show a *state* -- the mini player, the phone's now-playing
view, and the sidebar's marker for the favourites playlist itself --
pass it. The rest are context-menu items, which are actions rather than
states and take the outline `iconName` still returns.

`track-list` and `album-dropdown` already had this right, from inline
SVG paths of their own; this is the same rule for the call sites that
go through the icon library. `regular/star` is vendored to go with
`regular/heart`, which was already there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:10:06 -04:00
yonluandClaude Opus 5 351798fd66 fix(ui): spend a row's leftover space on the gaps, not the margins
The three card grids -- albums, artists, genres -- laid out with
`justify: 'center'` and a fixed 8px gap and padding, which gives the
row a fixed width and pushes everything left over to the two margins.
Measured on a 1440px window: cards 16px apart inside 78px of nothing
down each side. The outside was five times the inside.

`utils/grid-spacing.ts` computes one number instead, from what the row
could not spend on another card: the same value between two cards,
between two rows, and down each edge. That window now reads 30px
outside against 34px between, and it holds at any width.

The virtualizer has a word for this -- `justify: 'space-evenly'` with
`gap: 'auto'` -- and it cannot be used. It fits `floor(width /
cardWidth)` columns without reserving the gap it is about to need, so a
width one card short of exact leaves seven cards a pixel apart. On the
window above it would fit 7 columns with 1px between them. Deciding the
column count here is what puts a floor under the spacing.

Two consequences. The layout is rebuilt when the container width
changes the spacing rather than only when the cover size changes, so
each grid observes its own scroller -- keyed on the spacing, or every
pixel of a drag rebuilds a layout that comes out the same. And
`cover-grid`'s ScrollManager took `GRID_GAP`/`GRID_PADDING` as
constants, which stopped describing anything the moment the spacing
became elastic: it asks the host for the geometry now, since a scroll
position rebuilt from a stale 8px lands in the wrong row.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:09:50 -04:00
yonluandClaude Opus 5 40984f6086 fix(explore): let a slow archive node finish, and read the 404 back
Explore's album art was almost entirely missing: 5 of 24 cards on the
shelves had a cover, and those five were the ones already on disk.

The Cover Art Archive answers `front-250` with a 307 to an Internet
Archive storage node, and those nodes are slow. Measured against the
twelve albums on Explore's own shelves, a successful fetch took 14-16 s
and a failing one 13-17 s, against a client timeout of 10. So every
live fetch died, and a timeout writes nothing and says nothing -- which
is why this reads as "Explore has no album art" rather than as a slow
upstream. The timeout is 30 s, chosen to clear the measured range: the
fetch is off the critical path, so waiting costs nothing and giving up
early costs the whole page.

Two things beside it, both found on the way.

`writeCache(mbid, nil)` has recorded "the archive has no art for this"
as an empty file since it was written, and nothing has ever read it
back: `readCache` returns "" for an empty file, which is
indistinguishable from a miss. So every art-less release group was
re-fetched from CAA on every render that asked about it. A third of the
shelves are art-less, so that was a third of the page spending a live
request to be told again what the last one said. `knownMissing` reads
it, on both the release-group and the release path.

And the frontend marked a failed fetch as permanently answered for the
session, so a timed-out cover never retried within it. It drops the
marker instead; a genuine 404 is now answered from disk, so re-asking
one costs nothing.

Measured after: 23 of 24.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:09:14 -04:00
logan 786d9c6110 fix(release): seed the version floor on the parent, not on HEAD
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m38s
CI / e2e (pull_request) Successful in 6m14s
The floor tag marks what has already been released, so tagging the
commit being pushed leaves nothing between the floor and HEAD --
semantic-release then correctly reports there is nothing to release.
That is what the first run did: it seeded v0.0.0 on the merge commit
itself and cut no release.

HEAD^ is the first parent, so on a merge commit it is main as it was
before the merge and everything the merge brought in is releasable.

The tag has been moved to 6fb7b5e by hand; this is so the next repo
never needs that.
2026-08-17 20:37:43 -04:00
logan 0019310ca4 ci(release): cut releases from main automatically (#2)
Release / release (push) Failing after 40s
CI / check (push) Successful in 3m6s
CI / e2e (push) Successful in 7m16s
Implements .planning/plans/active/017-release-automation.md.

Merges to main now compute the version from Conventional Commits, cut
the tag and the Gitea release, and the four v* workflows publish and
attach their artifacts. First release is v0.0.1.
2026-08-18 00:22:13 +00:00
logan 1940cb548f fix(test): stop asserting a cache hit against a one-second deadline
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 3m51s
CI / e2e (pull_request) Successful in 6m21s
TestCacheTTLExpiry set a 1s TTL and immediately asserted a hit, so it
depended on an upper bound of elapsed wall-clock time between Set and
Get. Nothing can promise that: on the capacity-1 runner, with the rest
of the suite running in parallel, the goroutine can be descheduled for
longer than the TTL and the entry is then correctly gone.

It failed that way on this PR while passing five times out of five
locally, and it touches no code this branch changed.

Two entries now: one with an hour to live carries the presence
assertions, one with a second carries the expiry. Sleeping past a TTL
is always safe, so only the direction that cannot flake is timed.
2026-08-17 19:55:22 -04:00
logan 37e3373db9 docs: correct the workflow counts these comments name
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Failing after 1m35s
CI / e2e (pull_request) Skipped
Adding release.yml and desktop-assets.yml made 'the three workflows a
tag fires' wrong in three files that each said it slightly differently.
2026-08-17 19:47:55 -04:00
t 8d5d8af297 ci(release): keep the changelog out of a protected branch
CI / check (push) Skipped
CI / e2e (push) Skipped
main is protected (enable_push: false, empty whitelist), so
@semantic-release/git's commit-back is rejected by the pre-receive
hook -- and it would be rejected *after* the tag was pushed, leaving a
tagged release the run then reports as failed. Found by trying to push
this branch to main.

Whitelisting the CI user was the alternative and is declined: it
weakens a protection someone set deliberately and lets a bot push to
main without the checks every human PR has to pass.

So the release page is the changelog. The changelog plugin now writes a
gitignored .release-notes.md, which exists only to carry the notes into
gitea-release.sh without interpolating them into a shell command, and
CHANGELOG.md is a signpost -- a file claiming to be a changelog while
silently never updating is worse than no file.

Tags are not protected, so the tag push is unaffected.
2026-08-17 19:46:48 -04:00
logan 9ce79ee416 ci(release): release from a branch, not a detached HEAD
semantic-release resolves the release branch and then pushes a commit
and a tag to it, so a local branch named main is a better starting
point than the --detach the other five workflows use. Still pinned to
the pushed commit rather than to whatever main points at by the time
the container starts.

The floor tag falls back to the PAT when GITEA_TOKEN is unset, which is
safe rather than merely convenient: all four publishers skip v0.0.0
explicitly, so the worst case is four jobs that start and immediately
say there is nothing to build.
2026-08-17 19:21:19 -04:00
logan b3a0814f24 docs: describe the release pipeline where the claims used to be wrong
CLAUDE.md said .releaserc.yml was a config nothing ran and that there
were five workflows; both stop being true with this branch. The CI
section now names release.yml as the entry point and records the four
things in it that are load-bearing, including the two silent failure
modes worth pinning against.

packaging/homebrew/README.md and docs/android-release.md say where a
user would actually look that upgrading from 1.x needs a reinstall --
Homebrew offers nothing silently, and Android refuses outright.
2026-08-17 18:43:17 -04:00
logan 2c576fa1e8 ci(release): attach the Linux, Arch and Android builds to the release
A release page with nothing to download is one nobody can use. The
Arch package and the APK are already built and merely go unattached;
the plain Linux binary is new, and is what answers 'get the latest
version' without a package manager.

scripts/release-asset.sh waits for the release to exist first.
semantic-release pushes the tag in prepare and creates the release in
publish, so the tag push that starts these workflows happens before
there is an id to upload to -- and a capacity-1 runner serialises that
into working by accident, which is the worst kind of bug.

macOS is absent because it cannot be built here: GOOS=darwin
CGO_ENABLED=0 fails at wails/v3/pkg/mac, the darwin backend being
Objective-C behind cgo. Homebrew builds from source on the user's Mac
and stays the macOS channel. Windows cross-compiles cleanly and is
still withheld: no build of it has ever been run.

All three skip v0.0.0, which is semantic-release's version floor rather
than a shipment.
2026-08-17 18:39:06 -04:00
logan 544dbdb4db fix(packaging): stop publishing an Arch package on every merge to main
arch-package.yml ran on push to main and took its version from
`git describe`, so the pacman registry accumulated one package per
merge and not one of them corresponded to a version a user could be
told to install. It builds the tag release.yml cuts instead.

pkgver's literal drops to 0.0.1 with it. That is a downgrade from the
1.x already in the registry, so pacman offers no upgrade and an
existing install has to be removed once; epoch=1 would have avoided
that and is declined in a comment, because an epoch can never be
removed again.
2026-08-17 18:39:06 -04:00
logan 087eb77875 ci(release): cut a release from main with semantic-release
The config has been sitting in .releaserc.yml complete and uninvoked;
this is the workflow that runs it, and the one Gitea-shaped adaptation
it needs.

@semantic-release/github speaks GitHub's API, not Gitea's /api/v1, so
@semantic-release/exec calls scripts/gitea-release.sh instead. That
script reads the notes out of CHANGELOG.md rather than taking them as an
argument: release notes are rendered commit messages, so interpolating
the notes into a shell command would be an injection whose input is the
commit log.

The tag is pushed with a user PAT because Gitea does not start a
workflow from a ref pushed by a workflow's own token, and the three
publishing workflows are keyed on it.
2026-08-17 18:38:55 -04:00
yonlu 6fb7b5ea11 Merge pull request 'ci: trigger the catalog job deliberately, pin agent docs to one file' (#1) from chore/workflow-guardrails into main
Build & publish Arch package / arch-package (push) Successful in 2m40s
CI / check (push) Skipped
CI / e2e (push) Skipped
Build & publish the Android APK / apk (push) Failing after 56s
Sync Homebrew formula / sync-formula (push) Successful in 5s
Reviewed-on: #1
2026-08-17 20:19:38 +00:00
yonluandClaude Opus 5 369810e06b ci: stop testing every commit twice on a runner there is one of
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m53s
CI / e2e (pull_request) Successful in 6m26s
A branch push and its pull request are the same commit. With
`branches: ['**']` alongside `pull_request:`, opening a PR booked four
runs -- check and e2e for the branch, then both again for
refs/pull/N/head -- and this host has capacity 1, shared with an index
build that can hold it for three hours. PR #1's own checks queued two
runs deep behind exactly that.

`pull_request` covers feature branches. `main` stays because a
post-merge run is the record of the trunk's health, and now that main
refuses direct pushes it happens exactly once per merge.

The trade is that a branch pushed with no PR open gets no CI. That
matches the workflow this repo just committed to, and the signal returns
on the same commit the moment a PR exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 15:44:00 -04:00
yonluandClaude Opus 5 e51cb13662 ci: trigger the catalog job deliberately, pin agent docs to one file
CI / check (pull_request) Canceled after 0s
CI / e2e (pull_request) Canceled after 0s
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Two guardrails for the 2026-08-17 incident, and one is not about CI.

index-artifact.yml's `push` trigger was commented out that day with a
note to restore it once the rebuild completed. Restoring it is the bug.
A refresh is individually cheap, which is what made the trigger look
free; what it actually did was put an unattended job that mutates the
only copy of a ~205 GB catalog on the same trigger as an ordinary code
change, on a runner with capacity 1. The rule the file now states is the
general one -- a job that mutates state which cannot be rebuilt in ten
minutes is triggered deliberately -- so the next such job has somewhere
to look. The cron and workflow_dispatch lose nothing: indexbuild resumes
from its checkpoint either way.

Note what no branching or PR gate would have caught here. That change
was green on its branch, green on the merge and green on main; the fault
existed only against the persistent /cache database, which no fixture
reproduces. Code is gated by CI, irreplaceable state by refusing to
touch it and by docs/index-cache.md's restore.

The other half is the mismatch that started this: two harnesses reading
two files. AGENTS.md is a symlink to CLAUDE.md and skill-check asserts
the symlink rather than comparing contents, because a copy would satisfy
every other check in this repo while silently drifting -- which is the
failure being prevented. The same check now scans CLAUDE.md for make
targets, which it never did: 27 targets named in the file agents trust
most, none of them verified. Coverage goes 19 -> 46.

Scanning prose meant the line-start rule needed a fence. "Two green
branches do not / make a green merge" wrapped onto a line beginning
`make a` and duly failed on a target called `a`. Inside a fence it is
code; outside one it is a sentence that broke there, and a check that
fails on reflow gets disabled rather than fixed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 15:30:38 -04:00
logan 3d65da0529 test(download): stop racing a download these tests never wanted
Build & publish Arch package / arch-package (push) Successful in 2m36s
CI / e2e (push) Successful in 6m25s
CI / check (push) Successful in 2m43s
`check` failed on main with two failures in one package, and they are one
cause wearing two shapes:

    service_test.go:66: state = "satisfied", want wanted
    testing.go:1369: TempDir RemoveAll cleanup: ... directory not empty

Every test in service_test.go is about the durable Request that
StartDownload leaves behind, and none is about the download. But the
fixture is an anchored four-track request with a healthy provider, which
is precisely what AutoPickable says yes to -- so Manager.Start fired
`go m.grab(...)`, detached and with context.WithoutCancel, and the tests
raced it. Measured: the request reaches "satisfied" about 100ms after
StartDownload returns, so the first failure is the assertion reading the
next state, and the second is that same goroutine still writing into
t.TempDir() after the test returned.

The fixture now puts the candidate outside the auto-pick size window, so
the grab never starts. That is better than waiting for it: with no
goroutine there is nothing to be slow, and the tests state what they mean
without a timing assumption underneath. A test that does want the
download uses managerFixture and sets its own preferences.

It passed 20 runs under CPU load, but so did the broken version -- this
is a CI-only failure locally, so the cause was proved directly instead:
with the fixture's old preferences the request is observably "satisfied"
within 100ms of StartDownload, which is what CI read.
2026-08-17 14:16:45 -04:00
logan 52cbef27c4 docs: name the guard that covers every cache table
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m30s
The bullet added with the credit work names
`TestTheCatalogSurvivesAStaleShape`, which pins the table and shape that
failed. The general guard landed the same day and is the one that covers
a table nobody remembered -- flipping the policy back fails it on five,
including both artist-credit tables.
2026-08-17 14:01:28 -04:00
logan c03c0b8ec4 test(database): the next destructive repair fails a test, not a volume
Build & publish Arch package / arch-package (push) Successful in 2m33s
CI / check (push) Successful in 3m14s
CI / e2e (push) Canceled after 3m3s
The fix for the dropped catalog pins one table in one wrong shape, which
is the failure that happened. What cost the rebuild was more general: a
destructive repair added at `database.NewDB` -- the chokepoint every
binary in this project shares -- without asking which binary it runs in.
The next one will have a different name and a different reason.

So `TestNoCacheTableIsRetiredHere` asserts the outcome instead: put every
`datamap` Cache table into a shape the schema has moved past, open the
database the way cmd/indexbuild does, and require all of them to still be
there. Driving it from `datamap.ByKind` is what makes it cover tables
nobody remembered -- flipping the policy back fails on five, including
the two artist-credit tables added the same day, where the existing test
fails on one. It asserts the rows survive too, because SQLite does an
implicit DELETE before a DROP and a repair that recreated the table would
look identical. And it accepts an error from `NewDB`, because that is the
documented trade: loud is recoverable, gone is not.

`scripts/index-cache-snapshot.sh` covers the half no test can reach. The
volume holds the only copy of a catalog that costs hours of someone
else's bandwidth to re-derive. `VACUUM INTO` rather than `cp`, since a
byte copy of a live SQLite file is a corrupt file of plausible size; the
resumable staging directory is skipped; and each snapshot is reopened and
asked for its catalog row count before anything is rotated out. A corrupt
source and an empty catalog were both exercised: each exits non-zero,
removes its own output, and leaves the previous snapshots alone.

docs/index-cache.md is the restore, and the reason to bother: a restored
snapshot resolves to `refresh` and folds in the listens since, which is
minutes against the 3-23h this rebuild has been estimating.
2026-08-17 13:56:08 -04:00
yonlu 8c48105ca3 Merge remote-tracking branch 'origin/main' into wails-v3
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m31s
2026-08-17 13:52:34 -04:00
logan 1c4d6ca9a1 ci: stop booking three hours of runner on every push
Build & publish Arch package / arch-package (push) Successful in 2m34s
CI / check (push) Canceled after 53s
CI / e2e (push) Canceled after 0s
The catalog this job derives was dropped by the stale-shape repair (see
`fix(database): never retire the catalog the index build derives`, which
prevents a recurrence but cannot undo it), so `mode=auto` now resolves to
a full ~205 GB import from the dumps.

That import runs on every push to main with a 3h budget, on a runner of
capacity 1 -- so ordinary CI has been queuing behind it since the merge,
and each further push books another three hours. The damage is the
repetition, not the single job.

The `push` trigger is commented out until a run reports `complete=true`.
The weekly cron and workflow_dispatch still resume the build, which is
all it needs: indexbuild picks up from its checkpoint, so nothing already
imported is re-fetched.

Restoring the two commented lines is the entire revert, and the comment
beside them says so. NOTES.md carries the incident, including the two
things worth changing regardless: a destructive repair running inside
`database.NewDB` has to ask which binary it is in, and the only copy of a
205 GB derived asset is a single Docker volume with no snapshot.
2026-08-17 13:29:58 -04:00
yonluandClaude Opus 5 6bf832a4ba docs: record what credits are, and what the repair must never touch
Two mechanisms shipped today whose invariants are not visible from the
code, and one of them has already cost a rebuild.

Credits: why join phrases are assembly instructions rather than
disassembly ones, why credited_name is stored per row instead of joined
from artists, why the lookup is keyed on the recording MBID (and so
needed no local table), why an absent credit is cached as an answer,
and why the decomposition comes from a third dump at all — the
canonical dump has no join phrases and the JSON dumps overlap a real
library by zero rows. The measurements that justify the feature are
here too, including the correction that the "3 of 2,823" figure behind
plan 013 measured our own writer rather than any library.

The stale-shape repair gains the paragraph it should have shipped with:
retiring a Cache table is a build-tag decision, because the app
downloads its catalog and cmd/indexbuild derives it. Written as what
happened rather than as advice, since it dropped the real CI catalog on
its first run and the shape mismatch it found was there by design.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 13:13:05 -04:00
yonluandClaude Opus 5 4f8257ef72 fix(database): never retire the catalog the index build derives
Search index maintenance / maintain-index (push) Canceled after 0s
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m30s
The stale-shape repair dropped the CI catalog on its first run:

    retiring a table ... table=explore_index
      reason="column entity_type is TEXT, schema declares INTEGER"
    index maintenance mode=build reason="no completed import yet"

The mismatch was real and the drop was correct by the app's rule: a
client's catalog is *downloaded*, so a wrong shape costs a minute of
re-fetching the artifact, while keeping it costs every Explore read.

It is the wrong rule for one database. cmd/indexbuild's catalog is not
downloaded, it is what the artifact is cut from — the only way back is
the ~205 GB dump stream the /cache volume exists to avoid. And that
database is deliberately kept in the older encoding, which
`fix(indexexport): read an index older than the binary` exists to
tolerate, so the shape does not match by design and would have been
dropped on every run.

retireLibraryTables, right beside it, never touches the catalog for
exactly this reason. The repair reached past that protection because it
runs inside database.NewDB, which cmd/indexbuild also calls.

So the policy is a build tag, which is how this project already tells
the index tools apart (runtime_indexbuild.go, servicestartup.go,
dumpbuild_stub.go): Cache tables are rebuilt in the app and never in
cmd/indexbuild. Owned and Derived are still repaired in both — that is
the half this database can safely discard, and retireLibraryTables
already discards it.

The residual trade is deliberate: a future explore_index column will
now fail the index job loudly on applySchema rather than silently
costing it a 205 GB rebuild. A human should decide that one.

TestTheCatalogSurvivesAStaleShape is the accident, symptom first, with
the shape the real database is in — every current column, ids and
entity type still text. It fails with "the catalog was retired" when
the policy is flipped back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 12:24:49 -04:00
yonlu b505959934 Merge remote-tracking branch 'origin/main' into wails-v3
Build & publish Arch package / arch-package (push) Successful in 2m32s
CI / check (push) Successful in 3m10s
CI / e2e (push) Canceled after 0s
Search index maintenance / maintain-index (push) Canceled after 1h31m25s
2026-08-17 11:38:49 -04:00
yonluandClaude Opus 5 409bfd5e89 test(download): wait for the work, not for the state that precedes it
TestManagerEndToEndAutoPick waits for StateComplete and then asserts
that staging was released and the library was rescanned. Those happen
*after* the state is recorded: manager.go sets StateComplete, then
satisfies the request, then releases staging, then scans. So waiting on
the state is not waiting on either assertion, and on a loaded machine
the worker is descheduled in between and the test reads the world one
step too early:

    manager_test.go:209: staging not released: 1 dirs remain
    manager_test.go:218: library scans = 0, want 1

It passed alone every time and failed three times under a full-suite
run, which is the signature of a test race rather than a broken
manager — nothing here is wrong except what the test chose to wait on.
It blocks pushes, since the pre-push hook is exactly the loaded run.

It polls for the side effects now, through the waitFor this package
already has and already uses for the same reason one file over
(service_test.go waits for a request to become satisfied after the same
StateComplete).

Not reproduced on demand: eight spinners and -count=5 did not provoke
it with or without the fix, so this rests on the ordering being plain
in the code rather than on a red-to-green demonstration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 09:22:51 -04:00
yonluandClaude Opus 5 0eeef6048e feat(frontend): credit the artists on the full-screen now playing too
The phone shell's now-playing view landed on main while the credit
rendering was being written, so it arrived with the one call site that
still showed a multi-artist credit as a single link with the other
artists as punctuation inside it.

It is the same fix as the other ten: render from the parts, fall back
to the single link when there are fewer than two. The subscription is
what makes it show up at all — credits arrive after the track does, so
the name already on screen has to be re-rendered when they land.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:34:27 -04:00
yonlu 4fc0cdeab7 Merge remote-tracking branch 'origin/main' into wails-v3 2026-08-17 08:29:21 -04:00
yonluandClaude Opus 5 eb059a3d71 fix(database): retire a table whose shape the schema moved past
`applySchema` is CREATE ... IF NOT EXISTS and there is no migration
chain, so a *changed* table never migrates: the statement silently
no-ops against the old shape. Two plans had already landed on that, and
neither showed up in a test because a fresh install is perfectly
healthy.

- 014 added `total_tracks` to explore_index and to `indexRowFields`,
  the projection every explore read uses, so every search, browse,
  artist page and album page failed with "no such column: total_tracks"
  on any database that already had a catalog.
- 013 reshaped audio_files, so applySchema could not run at all and the
  app did not open.

staleshape.go runs before applySchema and drops what disagrees, so the
create is a create. It parses sql/schemas/ for the expectation rather
than writing the column list down a second time, and it notices a
changed *type* as well as a missing column — 013 moved mbid TEXT to
BLOB, which no ALTER could express and which SQLite will not coerce, so
a query against 16 raw bytes returns no rows rather than an error.

Only Authored tables are exempt. Cache is rebuildable by definition,
Owned is what a rescan rebuilds (plan 013's stated "delete and
rescan"), and a table the schema no longer describes at all goes too --
013 left seven behind plus schema_migrations.

Three things in it are load-bearing, and each was a bug first:

- The parser read `UNIQUE(mbid)` as a column, which made a healthy
  catalog look stale. That would have retired it on every launch and
  cost every user an artifact download per start.
- The drops are one transaction with defer_foreign_keys. Those legacy
  tables reference each other, so any order fails on whichever goes
  first; turning foreign keys off instead would suppress
  playlist_tracks.audio_file_id's ON DELETE SET NULL and leave entries
  pointing at ids a rescan reissues to *different songs*. Nulled
  entries are empty; stale ones are wrong, and wrong quietly.
- The order is sorted, so a failure reproduces. Map order is random,
  and the foreign-key bug passed its own regression test on two runs in
  three until the order was fixed.

Verified against a real pre-013 install: it opens, its 22 playlists
survive, 1,887 linked playlist entries become 0 rather than dangling,
and the legacy tables are swept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:27:05 -04:00
yonluandClaude Opus 5 dcabec8b1d feat(frontend): render a multi-artist credit as one link per artist
Every artist name in the app went through `artistLink(name, mbid)`, so
a track credited to several artists rendered one link and the rest as
punctuation — "2Pac feat. Snoop Dogg" linked 2Pac and left Snoop Dogg
as text inside it.

`creditLink(parts, fallbackName, fallbackMbid)` renders the credit from
its parts: one link per credited artist, join phrases as plain text
between them. The link boundaries are known by construction, which is
the point — locating a name inside the stored credit string would
reintroduce the mismatch the catalog exists to avoid, since that string
may come from the file's tags while the parts come from MusicBrainz and
the two disagree for ~1 in 3 multi-artist credits.

Fewer than two parts falls through to the previous behaviour exactly,
so a single-artist credit, a file with no recording MBID and a catalog
that has not answered yet all render as they did before. Nothing tries
to split the fallback string: "Simon & Garfunkel" is one artist, which
is why primaryArtist() does not split on "&" either.

The lookup is keyed on the recording MBID, which both sides already
carry — a catalog row has one and so does a local file — so one binding
serves Explore and the library's own lists, and no local table is
needed for this.

credit-store.ts, and three things in it are load-bearing:

- A miss is cached as an empty array. The backend returns nothing for a
  single-artist credit, which is ~87% of tracks, and caching only the
  hits would re-request the rest on every render forever.
- request() is per-row and coalesces into one call per frame. A
  virtualized list cannot hand over "the whole list": 50,000 rows would
  be 100 queries for the ~30 on screen.
- It is an LRU with a counted retainedChars probe, because a cache that
  grows with use is a leak with a schedule.

The virtualized lists push requestUpdate() into the virtualizer rather
than only the host, since its rows come from its own properties — a
host update alone would leave them exactly as they were. now-playing
marks its geometry dirty instead, because the marquee measures the text
it is about to scroll.

track-list keeps the single link while a search term is active: the
highlight spans are computed against the flat credit string, and
mapping them onto decomposed parts is a different problem.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:26:34 -04:00
yonluandClaude Opus 5 b3737d30af feat(explore): carry multi-artist credits in the catalog
A track credited to more than one artist has exactly one navigable
artist in this app and the rest are punctuation. `primaryArtist()`
string-parses the credit, strips a " feat. " clause and discards the
guest; it deliberately does not split on "&", "with" or "," because
those live inside real artist names.

Measured on a real 26,069-file library plus an 80+80 MusicBrainz
sample: 13% of recordings are multi-artist upstream, while only 0.86%
of files carry any structured multi-artist tag — mp3 carries zero
files with multiple MUSICBRAINZ_ARTISTID across 19,840. Of 1,286 files
saying "feat.", 90% have nothing structured behind it, and a sample of
80 such files was multi-artist in MB 80 times out of 80.

CLAUDE.md justified plan 013's removal of the credit tables with "3
credits of 2,823 listed more than one artist". That measured our own
*writer* — cachedLinkArtist was called once per credit, so a
collaboration could never have been recorded. Dropping the join table
was still right on cost; the evidence for "multi-artist is rare" was
not.

A credit is ordered parts and the credit string is derived from them,
so join phrases are assembly instructions, not disassembly ones.
Nothing here reconstructs a credit by searching a name inside a credit
string: the stored text may come from tags while the parts come from
the catalog, and those disagree for ~1 in 3 multi-artist credits.

Where it comes from, after two dead ends: the canonical dump CI
already streams has no join phrases and no as-credited names, and the
JSON dumps cover 153,691 recordings of ~35M with *zero* overlap
against a real library. So mbdump.tar.bz2 — 7.1 GB, ~13.7 min in
pure-Go bzip2, whose members are alphabetical, which is what lets one
pass resolve an entity's credit without buffering 35M recordings.

- artist_credit_part / artist_credit_ref, multi-artist credits only:
  a single-artist credit is already explore_index's own artist_name.
- Column layouts verified against the real 20260815 export;
  ErrDumpShape makes a wrong guess a failed build, not a wrong catalog.
- The pass runs on every mode, not just a build. The job picks its mode
  from the index's own state, and a complete import means "refresh",
  which never enters the importer — so credits could otherwise only
  arrive via a rebuild that re-downloads ~205 GB. It reports whether it
  populated anything, which is what flips `changed` and republishes.
- The importer asks whether an artifact carries the tables, on the
  writer where `core` is attached, so the artifact already published
  still imports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:25:36 -04:00
136 changed files with 10053 additions and 3167 deletions
+47
View File
@@ -129,6 +129,18 @@ jobs:
fi
v="${v#v}"
# v0.0.0 is semantic-release's version floor, not a shipment —
# see the bootstrap step in release.yml. It is skipped cleanly
# rather than failing the guard below, because a 45-minute red
# run against a tag that was never meant to ship is noise, and
# this is the most expensive of the four workflows a tag fires.
if [ "$v" = "0.0.0" ]; then
echo "v0.0.0 is the version floor, not a release; nothing to build"
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "skip=false" >> "$GITHUB_OUTPUT"
# Android orders releases by an integer and refuses anything
# not greater than what is installed. 1.3.1 -> 10301, which
# increases as long as minor and patch stay below 100.
@@ -143,9 +155,24 @@ jobs:
echo "version=$v" >> "$GITHUB_OUTPUT"
echo "code=$code" >> "$GITHUB_OUTPUT"
echo "tag=v$v" >> "$GITHUB_OUTPUT"
echo "building $v (versionCode $code)"
# Releases restarted at 0.0.1 when they became automatic (plan
# 017), so versionCode restarted at 1 — *below* the 10300 an
# installed 1.3.0 build carries. Android refuses a downgrade
# outright, and the only remedy is an uninstall, which takes the
# user's library with it. Said here because this is the file
# that computes the number.
if [ "$code" -lt 10600 ]; then
echo
echo "note: versionCode $code is below the 10600 that v1.6.0 shipped."
echo " An existing install must be removed before this one will"
echo " install, and that removal takes its library with it."
fi
- name: Go toolchain
if: steps.version.outputs.skip == 'false'
run: |
set -eu
if [ ! -x /cache/tool/go/bin/go ] || ! /cache/tool/go/bin/go version | grep -q "$GO_VERSION"; then
@@ -156,6 +183,7 @@ jobs:
/cache/tool/go/bin/go version
- name: Node toolchain
if: steps.version.outputs.skip == 'false'
run: |
set -eu
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
@@ -167,6 +195,7 @@ jobs:
# but still spends minutes verifying, so the guards are what make
# this cheap on every run after the first.
- name: Android SDK and NDK (cached)
if: steps.version.outputs.skip == 'false'
run: |
set -eu
mkdir -p "$ANDROID_HOME/cmdline-tools"
@@ -226,6 +255,7 @@ jobs:
# that are *verbatim* a secret, so a trimmed one could print in
# clear — or repeating the trimming logic in both.
- name: Build the signed APK
if: steps.version.outputs.skip == 'false'
working-directory: /src
env:
KEYSTORE_B64: ${{ secrets.ANDROID_KEYSTORE_B64 }}
@@ -349,6 +379,7 @@ jobs:
- name: Verify the APK
id: apk
if: steps.version.outputs.skip == 'false'
working-directory: /src
run: |
set -eu
@@ -390,6 +421,7 @@ jobs:
# file, so `latest` is deleted first. Credentials are the same
# OWNER/PACKAGE_TOKEN pair arch-package.yml publishes with.
- name: Publish to the Gitea package registry
if: steps.version.outputs.skip == 'false'
working-directory: /src
env:
VERSION: ${{ steps.version.outputs.version }}
@@ -425,3 +457,18 @@ jobs:
echo
echo "Obtainium URL:"
echo " $base/latest/yellowjacket.apk"
# The generic registry is what Obtainium polls; the release page is
# what a person looks at. Same file, already built and already
# verified by the step above — so this cannot publish something the
# signature check would have refused.
- name: Attach the APK to the release
if: steps.version.outputs.skip == 'false'
working-directory: /src
env:
TAG: ${{ steps.version.outputs.tag }}
VERSION: ${{ steps.version.outputs.version }}
run: |
set -eu
./scripts/release-asset.sh "$TAG" bin/yellowjacket.apk \
"yellowjacket-${VERSION}-android-arm64.apk"
+65 -2
View File
@@ -1,8 +1,23 @@
name: Build & publish Arch package
# Keyed on the tag, not on main. It used to publish on every push,
# deriving a version from `git describe` — so the registry accumulated a
# package per merge and none of them corresponded to anything a user
# could be told to install. release.yml decides what a release is now,
# and this builds the tag it cuts.
on:
push:
branches: [main]
tags: ["v*"]
workflow_dispatch:
inputs:
version:
description: "Version to build (default: the latest v* tag)"
required: false
concurrency:
group: arch-${{ github.ref }}
cancel-in-progress: true
jobs:
arch-package:
@@ -17,6 +32,7 @@ jobs:
REPO: ${{ github.repository }}
OWNER: ${{ github.repository_owner }}
SHA: ${{ github.sha }}
REF_NAME: ${{ github.ref_name }}
# Arch registry name (the "$repo" in clients' pacman.conf). Arbitrary label.
ARCH_REPO: stable
steps:
@@ -26,8 +42,9 @@ jobs:
# gtk3 was v2's stack and is now only the `-tags gtk3` escape hatch.
# These must match the PKGBUILD's depends=() — makepkg installs
# nothing itself, so a mismatch fails at link time, not at check time.
# jq is scripts/release-asset.sh's, not the build's.
pacman -Syu --noconfirm --needed \
base-devel git go nodejs pnpm curl sudo \
base-devel git go nodejs pnpm curl sudo jq \
webkitgtk-6.0 gtk4 alsa-lib
- name: Create unprivileged build user
@@ -36,15 +53,43 @@ jobs:
install -d -o builder -g builder /build
echo 'builder ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/builder
# v0.0.0 is semantic-release's version floor, not a shipment — see
# the bootstrap step in release.yml. A clean skip rather than a
# failure: a red run against a tag that was never meant to ship is
# noise, and this is one of the four workflows that would otherwise
# fire on it.
- name: Resolve the version
id: version
run: |
set -eu
v="${{ inputs.version }}"
[ -n "$v" ] || v="$REF_NAME"
case "$v" in v*) ;; *) v="v$v" ;; esac
if [ "$v" = "v0.0.0" ]; then
echo "v0.0.0 is the version floor, not a release; nothing to build"
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "tag=$v" >> "$GITHUB_OUTPUT"
echo "building $v"
- name: Clone repo at the pushed commit
if: steps.version.outputs.skip == 'false'
run: |
# Token auth works for private repos and needs no SSH key in CI.
sudo -u builder git clone \
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" \
/build/yellowjacket
# A tag push carries the tag's own commit in $SHA, so this checks
# out exactly what was tagged. pkgver() then reads the tag from
# the clone's own git history.
sudo -u builder git -C /build/yellowjacket checkout --detach "$SHA"
- name: Build package with makepkg
if: steps.version.outputs.skip == 'false'
run: |
cd /build/yellowjacket/packaging/arch
# Point the PKGBUILD at this local clone / exact commit; pkgver() then
@@ -54,6 +99,7 @@ jobs:
makepkg -f --noconfirm --cleanbuild
- name: Publish to the Gitea Arch registry
if: steps.version.outputs.skip == 'false'
run: |
cd /build/yellowjacket/packaging/arch
# makepkg also produces a -debug package (detached symbols); end users
@@ -67,3 +113,20 @@ jobs:
--upload-file "$pkg" \
"${SERVER_URL}/api/packages/${OWNER}/arch/${ARCH_REPO}"
done
# The pacman registry is for people who have added it to pacman.conf;
# the release page is for everyone else. Same file, and it is
# already built.
- name: Attach the package to the release
if: steps.version.outputs.skip == 'false'
env:
TAG: ${{ steps.version.outputs.tag }}
run: |
set -eu
cd /build/yellowjacket/packaging/arch
for pkg in yellowjacket-*.pkg.tar.zst; do
case "$pkg" in
yellowjacket-debug-*) continue ;;
esac
/build/yellowjacket/scripts/release-asset.sh "$TAG" "$(pwd)/$pkg"
done
+17 -3
View File
@@ -1,6 +1,6 @@
name: CI
# The other three workflows package and publish; none of them test
# The other five workflows package, publish or release; none of them test
# anything, so a green tick on this repo used to mean "the Arch package
# built", which is not the question anyone was asking. This is the
# workflow that gates.
@@ -9,9 +9,23 @@ name: CI
# before being written here, so every step below is a transcription of
# something observed working rather than something expected to.
# **A branch push and its PR are the same commit, and testing it twice
# costs the only runner there is.** `branches: ['**']` here meant every
# PR booked four runs — `check` and `e2e` for the branch push, then both
# again for `refs/pull/N/head` — on a host with capacity 1, where the
# queue is shared with an index build that can hold it for three hours.
#
# `pull_request` covers feature branches, and `main` is kept because a
# post-merge run is the record of the trunk's health. Since main now
# refuses direct pushes, that run happens exactly once per merge.
#
# The trade is explicit: a branch pushed with **no** PR open gets no CI.
# That is consistent with the workflow this repo committed to — every
# change goes through a PR — and the signal returns the moment one is
# opened, on the same commit.
on:
push:
branches: ['**']
branches: [main]
pull_request:
workflow_dispatch:
@@ -78,7 +92,7 @@ jobs:
# Cloned by hand rather than with actions/checkout: that is a JS
# action and needs node inside the job container before any step
# has had a chance to install it. Same approach as the other
# three workflows in this directory.
# other workflows in this directory.
- name: Clone repo at this commit
run: |
set -eu
+176
View File
@@ -0,0 +1,176 @@
name: Attach the desktop build to the release
# The Arch package goes to the pacman registry and the APK to the generic
# one, but a release page with nothing on it to download is a release page
# nobody can use. This builds the plain Linux x86_64 binary and attaches
# it, so "get the latest version" has an answer that needs no package
# manager at all.
#
# **Linux only, and macOS is not an oversight.** `GOOS=darwin
# CGO_ENABLED=0` fails at `wails/v3/pkg/mac: build constraints exclude all
# Go files` — the darwin backend is Objective-C behind cgo, so a .app
# needs a macOS host, and the runner is a Linux container. That is
# exactly why the Homebrew formula builds from source on the user's own
# Mac, and it stays the macOS channel.
#
# Windows *does* cross-compile (GOOS=windows CGO_ENABLED=0 succeeds in a
# couple of seconds — nothing in the audio, database or webview path needs
# cgo there), and is deliberately not published: no Windows build of this
# app has ever been run, and no tier here can exercise one. Shipping it
# would be a promise nothing in this repo can keep. Revisit when someone
# has actually booted it.
on:
push:
tags: ["v*"]
workflow_dispatch:
inputs:
version:
description: "Version to build and attach (default: the latest v* tag)"
required: false
concurrency:
group: desktop-assets-${{ github.ref }}
cancel-in-progress: true
jobs:
linux:
runs-on: ubuntu-latest
container:
image: ubuntu:24.04
volumes:
- /home/logan/docker/gitea/data/runner/cache/tool:/cache/tool
- /home/logan/docker/gitea/data/runner/cache/pnpm-store:/cache/pnpm-store
env:
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
SERVER_URL: ${{ github.server_url }}
REPO: ${{ github.repository }}
SHA: ${{ github.sha }}
REF_NAME: ${{ github.ref_name }}
DEBIAN_FRONTEND: noninteractive
GO_VERSION: '1.25.0'
npm_config_store_dir: /cache/pnpm-store
steps:
# The same set ci.yml's check job installs: the app is cgo, and
# without alsa.pc oto/v3 fails at `pkg-config --cflags -- alsa`
# before anything is compiled.
- name: System packages
run: |
set -eu
apt-get update -qq
apt-get install -y -qq --no-install-recommends \
ca-certificates curl git jq build-essential pkg-config \
libwebkitgtk-6.0-dev libgtk-4-dev libasound2-dev
- name: Clone repo at this commit
run: |
set -eu
git clone --quiet \
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" /src
git -C /src checkout --quiet --detach "$SHA"
git config --global --add safe.directory /src
git -C /src log --oneline -1
- name: Resolve the version
id: version
working-directory: /src
run: |
set -eu
v="${{ inputs.version }}"
if [ -z "$v" ]; then
case "$REF_NAME" in
v*) v="$REF_NAME" ;;
*) v=$(git describe --tags --abbrev=0 --match 'v[0-9]*') ;;
esac
fi
case "$v" in v*) ;; *) v="v$v" ;; esac
# v0.0.0 is semantic-release's version floor, not a shipment —
# see the bootstrap step in release.yml. Nothing is built for
# it, and this is a clean skip rather than a failure because a
# red run against a tag that was never meant to ship is noise.
if [ "$v" = "v0.0.0" ]; then
echo "v0.0.0 is the version floor, not a release; nothing to build"
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "tag=$v" >> "$GITHUB_OUTPUT"
echo "version=${v#v}" >> "$GITHUB_OUTPUT"
echo "building $v"
- name: Go toolchain
if: steps.version.outputs.skip == 'false'
run: |
set -eu
if [ ! -x /cache/tool/go/bin/go ] || ! /cache/tool/go/bin/go version | grep -q "$GO_VERSION"; then
mkdir -p /cache/tool && rm -rf /cache/tool/go
curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" | tar -C /cache/tool -xz
fi
echo "/cache/tool/go/bin" >> "$GITHUB_PATH"
/cache/tool/go/bin/go version
- name: Node toolchain
if: steps.version.outputs.skip == 'false'
run: |
set -eu
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
apt-get install -y -qq --no-install-recommends nodejs
corepack enable
node --version
# `make build-prod` is the production task: -trimpath and -w -s are
# already in it, so only the version stamp is passed, through the
# LDFLAGS_EXTRA variable this repo added to build/linux/Taskfile.yml.
# (`wails3 build` has no -ldflags of its own; that was v2.)
- name: Build
if: steps.version.outputs.skip == 'false'
working-directory: /src
env:
TAG: ${{ steps.version.outputs.tag }}
run: |
set -eu
export PATH="/src/scripts/toolbin:$PATH"
commit=$(git rev-parse --short HEAD)
go generate ./...
go tool wails3 task build \
LDFLAGS_EXTRA="-X 'main.version=${TAG}' -X 'main.commit=${commit}'"
# Described, never run: main.go has no flag parsing, so any
# invocation here would try to open a window in a container with
# no display and hang the job rather than printing a version.
test -x bin/yellowjacket
ls -la bin/yellowjacket
file bin/yellowjacket || true
# The .desktop file and the icon go in the tarball because without
# them the binary is a window with no menu entry — the Arch package
# installs both, and this is the same app for people not using it.
- name: Package the tarball
if: steps.version.outputs.skip == 'false'
working-directory: /src
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
set -eu
dir="yellowjacket-${VERSION}-linux-amd64"
mkdir -p "/tmp/$dir"
cp bin/yellowjacket "/tmp/$dir/"
cp packaging/arch/yellowjacket.desktop "/tmp/$dir/"
cp frontend/src/assets/images/icons/music/compact-disc.svg \
"/tmp/$dir/yellowjacket.svg"
tar -C /tmp -czf "/tmp/${dir}.tar.gz" "$dir"
ls -la "/tmp/${dir}.tar.gz"
- name: Attach it to the release
if: steps.version.outputs.skip == 'false'
working-directory: /src
env:
TAG: ${{ steps.version.outputs.tag }}
VERSION: ${{ steps.version.outputs.version }}
run: |
set -eu
./scripts/release-asset.sh "$TAG" \
"/tmp/yellowjacket-${VERSION}-linux-amd64.tar.gz"
+28 -2
View File
@@ -14,6 +14,15 @@ on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
version:
description: "Version to sync (default: the pushed tag)"
required: false
concurrency:
group: homebrew-${{ github.ref }}
cancel-in-progress: true
jobs:
sync-formula:
@@ -30,10 +39,25 @@ jobs:
uses: actions/checkout@v4
- name: Compute version and tarball checksum
id: version
run: |
set -euo pipefail
TAG="${GITHUB_REF_NAME}" # e.g. v1.3.0
VERSION="${TAG#v}" # e.g. 1.3.0
TAG="${{ inputs.version }}"
[ -n "$TAG" ] || TAG="${GITHUB_REF_NAME}" # e.g. v0.0.1
case "$TAG" in v*) ;; *) TAG="v$TAG" ;; esac
VERSION="${TAG#v}" # e.g. 0.0.1
# v0.0.0 is semantic-release's version floor, not a shipment —
# see the bootstrap step in release.yml. Skipped cleanly rather
# than failing: this one would otherwise push a formula for a
# version that does not exist into a *public* tap.
if [ "$VERSION" = "0.0.0" ]; then
echo "v0.0.0 is the version floor, not a release; nothing to sync"
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "skip=false" >> "$GITHUB_OUTPUT"
TARBALL="${SOURCE_TARBALL_BASE}/${TAG}.tar.gz"
echo "Fetching ${TARBALL}"
@@ -53,6 +77,7 @@ jobs:
echo "SHA256=${SHA256}" >> "$GITHUB_ENV"
- name: Render the formula with the new version and checksum
if: steps.version.outputs.skip == 'false'
run: |
set -euo pipefail
src="packaging/homebrew/Formula/yellowjacket.rb"
@@ -66,6 +91,7 @@ jobs:
cat yellowjacket.rb
- name: Push to the Homebrew tap repo
if: steps.version.outputs.skip == 'false'
run: |
set -euo pipefail
git clone "https://x-access-token:${TAP_TOKEN}@github.com/${TAP_REPO}.git" tap
+26 -4
View File
@@ -7,11 +7,29 @@ name: Search index maintenance
# import older than 6mo -> rebuild (re-import from the newest dump)
# otherwise -> refresh (fold in new incremental listens)
#
# A refresh is cheap and no-ops when nothing new has been published, so
# running it on every push to main is safe.
# **There is deliberately no `push` trigger, and restoring one is a
# decision rather than a cleanup.** A refresh is individually cheap, so
# running it on every push to main looked free; what it actually does is
# put an unattended job that mutates the only copy of a ~205 GB catalog
# on the same trigger as an ordinary code change, on a runner with
# capacity 1.
#
# That is not hypothetical. On 2026-08-17 `fix(database): retire a table
# whose shape the schema moved past` landed on main, green — the CI
# database is deliberately in the older encoding, so the stale-shape
# repair judged its `explore_index` stale and dropped it, and this job
# fell back to a full import from the dumps. `fix(database): never
# retire the catalog the index build derives` stops that specific repair
# and cannot undo it. Every push to main then booked another `budget`
# (3h) of the one runner while ordinary CI queued behind it.
#
# So the rule this file is an instance of: **a job that mutates state
# which cannot be rebuilt in ten minutes is triggered deliberately, not
# by a push.** The weekly cron keeps the catalog current, and
# workflow_dispatch resumes or forces a build — indexbuild picks up from
# its checkpoint either way, so nothing is lost by not running on every
# merge. See docs/index-cache.md for the snapshot and the restore.
on:
push:
branches: [main]
schedule:
# Weekly update pass. The 6-month rebuild is triggered by the same
# command when it notices the import has aged out.
@@ -33,6 +51,10 @@ on:
# Runs share one persistent working directory, so they must not overlap.
# A push landing mid-build waits rather than corrupting the checkpoint.
#
# That directory holds the only copy of a catalog nothing can cheaply
# re-derive: see docs/index-cache.md for the snapshot it takes and the
# restore, which is minutes against the hours a rebuild costs.
concurrency:
group: search-index
cancel-in-progress: false
+182
View File
@@ -0,0 +1,182 @@
name: Release
# The sixth workflow, and the one that decides whether the other three
# run at all. On every push to main it reads the Conventional Commits
# since the last tag, and if any of them is releasable it writes the
# changelog, pushes the tag, and creates the Gitea release whose body is
# that changelog section. The publishing workflows are keyed on `v*`, so
# the tag push is what starts them.
#
# **Why the tag is pushed with PACKAGE_TOKEN and not the Actions token.**
# Gitea, like GitHub, does not start a workflow from a ref pushed by a
# workflow's own token (go-gitea#33123). The token is what decides this,
# not the workflow — so semantic-release is handed a repositoryUrl
# carrying a *user* PAT, and the resulting push is attributed to a person
# and triggers the `v*` workflows normally.
#
# That limitation is used deliberately in the bootstrap step below, where
# a tag that must *not* trigger anything is pushed with the Actions token
# instead.
on:
push:
branches: [main]
workflow_dispatch:
# Cutting a tag is not a thing to cancel halfway: a superseded run must
# finish, not be killed between `git push --tags` and the release POST.
concurrency:
group: release-main
cancel-in-progress: false
jobs:
release:
runs-on: ubuntu-latest
container:
image: ubuntu:24.04
env:
SERVER_URL: ${{ github.server_url }}
OWNER: ${{ github.repository_owner }}
REPO: ${{ github.repository }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
DEBIAN_FRONTEND: noninteractive
steps:
- name: System packages
run: |
set -eu
apt-get update -qq
apt-get install -y -qq --no-install-recommends ca-certificates curl git jq
- name: Node toolchain
run: |
set -eu
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
apt-get install -y -qq --no-install-recommends nodejs
node --version
# By hand rather than actions/checkout, like the other five: that is
# a JS action and needs node inside the container before any step has
# installed it. The full history is required — semantic-release
# reads tags and walks commits, and a shallow clone silently makes
# every release look like the first one.
- name: Clone repo at this commit
run: |
set -eu
git clone --quiet \
"https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git" /src
# -B main rather than --detach, which the other five workflows
# use: semantic-release resolves the release branch and then
# pushes a commit and a tag to it, and a detached HEAD is a
# worse starting point for both than a local branch named after
# the one being released. Pinned to this commit, not to
# whatever main points at by the time the container started.
git -C /src checkout --quiet -B main "${{ github.sha }}"
git config --global --add safe.directory /src
git -C /src log --oneline -1
# Nothing currently pushes a `chore(release):` commit — main is a
# protected branch, so .releaserc.yml carries no @semantic-release/git
# and the release page is the changelog. This guard is kept for the
# day someone adds that plugin back: without it the commit-back is a
# push to the branch this workflow runs on, and the loop is a release
# per release. Six lines against that is cheap.
- name: Skip a changelog commit, if one ever exists
id: guard
working-directory: /src
run: |
set -eu
subject=$(git log -1 --format='%s')
case "$subject" in
"chore(release):"*)
echo "this is the release commit itself; nothing to do"
echo "skip=true" >> "$GITHUB_OUTPUT"
;;
*)
echo "skip=false" >> "$GITHUB_OUTPUT"
;;
esac
# semantic-release calls the first release of a repo with no tags
# 1.0.0, and offers no option to say otherwise. A floor tag is the
# only way to start at 0.0.1, so this creates one — once, ever.
#
# **It is pushed with the Actions token on purpose.** v0.0.0 is a
# floor, not a shipment: pushing it with a user PAT would start the
# Arch, Homebrew and Android workflows for a version that does not
# exist. The very limitation the header describes is what makes
# this inert.
- name: Seed the version floor
if: steps.guard.outputs.skip == 'false'
working-directory: /src
env:
ACTIONS_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
set -eu
git fetch --quiet --tags origin
if [ -n "$(git tag --list 'v[0-9]*')" ]; then
echo "floor already set; newest tag is $(git describe --tags --abbrev=0 --match 'v[0-9]*')"
exit 0
fi
# Prefer the Actions token because a ref it pushes starts no
# workflow, which is the whole point for a tag that is a floor
# rather than a shipment. Falling back to the PAT is safe
# rather than merely convenient: all four publishing workflows
# skip v0.0.0 explicitly, so the worst case is four jobs that
# start and immediately say there is nothing to build.
token="${ACTIONS_TOKEN:-$PACKAGE_TOKEN}"
[ -n "$ACTIONS_TOKEN" ] || echo "note: GITEA_TOKEN is unset; using the PAT"
# **On the parent, not on HEAD.** The floor marks what has
# already been released, so tagging the commit being pushed
# leaves nothing between the floor and HEAD — semantic-release
# then correctly reports there is nothing to release, which is
# exactly what the first run of this workflow did. HEAD^ is the
# first parent, so on the merge commit this fires for it is main
# as it was before the merge, and everything the merge brought
# in is releasable.
floor=$(git rev-parse "${{ github.sha }}^" 2>/dev/null || true)
if [ -z "$floor" ]; then
echo "HEAD has no parent, so no commit can precede the floor" >&2
exit 1
fi
echo "no v* tag exists — seeding v0.0.0 so the first release is 0.0.1"
git tag v0.0.0 "$floor"
git push --quiet \
"https://x-access-token:${token}@${SERVER_URL#https://}/${REPO}.git" \
refs/tags/v0.0.0
echo "seeded v0.0.0 at $floor (parent of ${{ github.sha }})"
# Pinned rather than installed into the repo: this is a Go project
# and a package.json at its root invites the npm plugin and every
# tool that looks for one. conventional-changelog-conventionalcommits
# is in the list because both the analyzer and the notes generator
# name that preset and neither depends on it.
#
# **That preset is held at 9 and the reason is worth keeping.** At
# 10 it is silently incompatible with the writer that
# release-notes-generator@14 pulls in (^8): every release note comes
# out as a bare `## 0.0.1 (date)` heading with **no sections and no
# commits under it**, and nothing errors. The version would have
# been right, the tag would have been right, every job would have
# been green, and the release body would have been empty. Check the
# notes, not the exit code, before moving any of these.
- name: Run semantic-release
if: steps.guard.outputs.skip == 'false'
working-directory: /src
run: |
set -eu
git config user.name "yellowjacket-ci"
git config user.email "yj@yellowjacket.app"
npx --yes \
-p semantic-release@25 \
-p @semantic-release/commit-analyzer@13 \
-p @semantic-release/release-notes-generator@14 \
-p @semantic-release/changelog@7 \
-p @semantic-release/exec@7 \
-p conventional-changelog-conventionalcommits@9 \
semantic-release \
--repository-url "https://x-access-token:${PACKAGE_TOKEN}@${SERVER_URL#https://}/${REPO}.git"
+107
View File
@@ -0,0 +1,107 @@
name: Unclaim
# A `Closes #N` footer in a commit body closes the issue on merge — and
# leaves `Status/In Progress` on it, because Gitea's auto-close touches
# state and nothing else. So #100 was closed and simultaneously marked
# as being actively worked on, and `scripts/issue.sh close` (which does
# drop the label) is exactly the thing the footer exists to avoid
# calling.
#
# **This hooks the close, not the merge.** Stripping the label in the
# PR would work and would be a per-PR habit; habits are what the footer
# removed. `issues: [closed]` covers every path an issue can close by —
# the footer on merge, `issue.sh close`, someone clicking Close in the
# web UI — and asks nothing of anyone at any of them.
#
# **Reopening deliberately does not restore it.** Reopening says the
# work was not finished, not that somebody is at a keyboard doing it
# now; the claim gets re-made by whoever picks it up.
#
# **This is not instant, and should not be described as it.** The
# runner has capacity 1 and is shared with an index build that can hold
# it for three hours, so a label tweak can queue behind one. Stale for
# an afternoon beats stale forever, which is what it was.
#
# The audit that answers "is this still firing" stays in CLAUDE.md and
# is one command:
#
# ./scripts/issue.sh list --state closed --label "Status/In Progress"
#
# A workflow that silently stops working is the failure mode this whole
# area has already produced once.
on:
issues:
types: [closed]
jobs:
unclaim:
runs-on: ubuntu-latest
container:
image: ubuntu:24.04
steps:
- name: Drop the claim label
# **Inside a container the act runner selects `sh`, not bash**, so
# `set -o pipefail` fails the job on its second line with "Illegal
# option" and the step never reaches the API. `homebrew-formula.yml`
# carries the same `set -euo pipefail` without trouble because it
# runs with **no container**, on the host image where bash is the
# default — so "another workflow does it" is not evidence here.
shell: bash
env:
# The automatic Actions token, as release.yml uses for the
# floor tag. It needs no more than write access to this repo.
TOKEN: ${{ secrets.GITEA_TOKEN }}
API: ${{ github.server_url }}/api/v1/repos/${{ github.repository }}
ISSUE: ${{ github.event.issue.number }}
run: |
set -euo pipefail
# `ca-certificates` is named because `--no-install-recommends`
# skips it, and `ubuntu:24.04` ships no CA bundle of its own —
# so curl comes up unable to verify TLS against our own Gitea
# and fails with "error setting certificate file" (exit 77).
# Every other containerised workflow here spells it out for the
# same reason; this one did not, and cost a release cycle.
apt-get update -qq
apt-get install -y -qq --no-install-recommends \
ca-certificates curl jq >/dev/null
label_id=$(
curl -sSf -H "Authorization: token $TOKEN" "$API/labels?limit=100" |
jq -r '.[] | select(.name == "Status/In Progress") | .id'
)
# The label not existing is a repo somebody reorganised, not a
# failure of this run — say so and stop, rather than failing a
# job on every close from then on.
if [ -z "$label_id" ]; then
echo "unclaim: no 'Status/In Progress' label in this repo; nothing to do"
exit 0
fi
# DELETE is idempotent here: an issue that never carried the
# label answers the same as one that did, which is what makes
# this safe to run on *every* close rather than only the ones
# that were claimed.
# The body is captured, not discarded, so a refusal is
# diagnosable from this log alone. Whether the automatic
# token carries issue-write scope is still unproven, and
# "DELETE returned 403" without Gitea's own sentence costs
# another merge to find out which of the two it is.
body=$(mktemp)
code=$(
curl -sS -o "$body" -w '%{http_code}' -X DELETE \
-H "Authorization: token $TOKEN" \
"$API/issues/$ISSUE/labels/$label_id"
)
case "$code" in
204) echo "unclaim: #$ISSUE is closed and unclaimed" ;;
*)
echo "unclaim: DELETE returned $code for #$ISSUE" >&2
cat "$body" >&2
exit 1
;;
esac
+4
View File
@@ -84,3 +84,7 @@ build/android/build/
build/android/.gradle/
build/android/gen/
build/android/overlay.json
# Written by @semantic-release/changelog purely to carry the release notes
# into scripts/gitea-release.sh; the release page is the changelog.
.release-notes.md
-118
View File
@@ -1,118 +0,0 @@
# Work log
Temporal memory: what happened and what's next. Structure lives in
`CLAUDE.md`, operational instructions in `.pi/skills/yellowjacket-dev/`,
measured discoveries in `.planning/NOTES.md`. Don't duplicate those here.
## Current state
Plan 005 (agent development harness) is **complete — all seven
phases**. Everything from phase 1 onward is still **uncommitted**: one
large but coherent working-tree diff, nothing pushed.
All four tiers verified green from a cold, cleaned state:
`make ui-test` 313 passed, `make lint` 0 issues × 3 configurations,
`make test` green × 3 passes, `make e2e` 19 passed. Both CI jobs
verified green in a bare `ubuntu:24.04` container, including 19/19 on
WebKit.
**Committed and pushed** as `5ca6cad` (the harness) + `ccacd67` (a CI
fix), and **green on the real runner**: job `check` ~4 min, job `e2e`
~3 min with 19/19 chromium *and* 19/19 webkit. One commit rather than
seven because the working tree was the end state, not per-phase
snapshots — `Makefile`, `CLAUDE.md` and `lefthook.yml` are touched by
nearly every phase, so a split would have been fabricated history.
Still unverified, because no run has failed yet: the
`actions/upload-artifact` step (`continue-on-error`, so it cannot mask
a real failure) and whether pnpm honours `npm_config_store_dir` for
store caching. Worth checking the next time a spec legitimately fails.
- [ ] `gitea_ci`'s `job_logs` returns 404 on Gitea 1.27.1 — the endpoint
is not exposed. Logs come from the VPS instead: `zstdcat` the file
under `gitea/actions_log/<owner>/<repo>/<xx>/<task_id>.log.zst`,
and note `zstdcat` is not in the gitea container, so
`docker cp` it out first. Job status is `action_run_job.status`
(1 success, 2 failure, 4 skipped, 5 waiting, 6 running).
Probably belongs in the `gitea` skill, not here.
Open items deliberately not fixed: WAV tags are write-only
(`TestWAVTagsAreNotReadableYet`), `themeStore.loadFromBackend`'s failure
handler cannot recover, `backend/playlist` has no CRUD suite.
## Log
### 2026-08-11 — cold skill run, then phase 7 (CI)
- **Followed the skill cold first**, as the last session asked. It
works: app up from a wiped `.dev/`, an undocumented flow driven
(queue panel + shuffle, asserted on `QueueModeChanged`), stopped —
~1 minute, no dead ends. One real config bug: `outputDir` in
`.playwright/cli.config.json` resolves against **cwd**, not the
config file's directory (only `initScript` does that), so snapshots
were landing above the repo and a *stale* one from the previous
session answered `ls -t` instead. That cost a DOM walk to disprove a
regression that did not exist. Four smaller doc gaps fixed
(`sandbox-seed` already runs `testdata`; `ui-setup`/`e2e-setup` were
undocumented prerequisites; `snapshot` prints a path; `dev-stop`
leaves the browser open), plus `dev-headless.sh`'s own banner, which
was suggesting the bare `window.go` call its next paragraph warns
against.
- **Built both CI jobs as container scripts before writing any YAML**,
then transcribed the YAML back out and re-ran it to prove the
transcription. Push-and-see is a bad loop on a self-hosted runner.
- **It found a real bug immediately**: `make lint` omitted
`webkit2_41` on all three passes, so it was linting configurations
nothing builds. Invisible on Arch (which still ships
`webkit2gtk-4.0.pc`), fatal on Ubuntu 24.04. Tag sets now match
`make test`.
- **Both open decisions settled by measurement**: ALSA `null` PCM for
audio (no daemon; the elapsed clock really advances), dead-address
stub for the explore artifact (and setting it for the *app* run, not
just seeding, is worth 8x on suite wall clock). **WebKit is a
required step** — it had never been run anywhere, so one throwaway
container run replaced a coin flip with 19/19 at +11 s.
### 2026-08-10 — phase 6, pi affordances
- Added `.pi/skills/yellowjacket-dev/` as a directory rather than a flat
file: only the description is always in context, so `SKILL.md` stays
short enough that reading it whole is never a decision, and the deeper
material sits in `references/{harness,fixtures,ui-tier,schema-change}.md`.
- Settled the CLAUDE.md-vs-skill split **grammatically, not topically**,
because a topical split is what rots — every new fact gets two
plausible homes. Three docs, three tenses: NOTES.md is past
(measured, dated, append-only), CLAUDE.md is present (what the system
is), the skill is imperative (what to run). A new paragraph's tense
decides where it goes.
- The five gotchas (binding timeouts, first-run wizard, `pkill -f`,
seeds-by-running, WebKit-is-CI-only) went **inline in SKILL.md**, not
into a reference: you need them before the failure, not after.
- Trimmed CLAUDE.md's "Fixtures and the headless harness" section by
about half — the command sequences and gotchas it was carrying are now
the skill's, and leaving both would have created exactly the duplicate
description this repo has a standing rule against.
- Added `make skill-check` / `scripts/skill-check.sh` + a pre-commit
hook: every command in `.pi/**/*.md` must be a real `make` target, so
the Makefile stays the source of truth for invocation and a renamed
target fails a commit instead of misleading an agent later. Verified
it fails (it caught its own not-yet-created target) and passes.
- Added the `/e2e` prompt template: promoting a hand-driven
`playwright-cli` session into a spec is a transcription with four
fixed substitutions (refs → testids, sleeps → `waitForEvent`, raw
`window.go``callBinding`, short fixture → `LONG_TRACK`), plus three
runs — pass, pass again, pass after a DB restore — because the usual
failure is a spec depending on state the hand-driving left behind.
- One shell trap: under `set -euo pipefail`, `x="$(make -pqRr | …)"`
fails the whole assignment, because `make -q` exits non-zero when a
target is out of date and `pipefail` propagates it.
### Earlier
Phases 15 of plan 005: fixture generator and manifest, headless launch
and seeds, the event bridge + `data-testid` pass + `backend/testctl` +
`e2e/`, the Vitest component tier + `make bindings-check`, and the
`events.Emit` wrapper with its in-process service-event tests. Recaps
and the five "verified end to end" blocks are in
`.planning/plans/active/005-agent-development-harness.md`; the lessons
are in `.planning/NOTES.md`.
+145
View File
@@ -3338,3 +3338,148 @@ Four things measured or corrected in the doing:
- **The gate belongs before the first status write.** Declining is a
no-op — no job in the indicator, no error tier to dismiss — which is
what makes the refusal safe to have on by default.
## The stale-shape repair dropped the CI catalog (2026-08-17)
Not our change, but it is the operational state everything else now runs
in, and the restore condition needs to be written down somewhere that is
not a commit message.
`fix(database): retire a table whose shape the schema moved past` added
`staleshape.go`: before `applySchema`, drop any non-Authored table whose
live shape disagrees with the schema. That is the right rule for an
install — a client's catalog is *downloaded*, so a stale one costs a
minute of re-fetching the artifact, and keeping it costs every Explore
read.
It runs inside `database.NewDB`, which `cmd/indexbuild` also calls. On
the first run after it landed, 19 seconds in:
```
16:15:51 retiring a table ... table=explore_index
reason="column entity_type is TEXT, schema declares INTEGER"
16:16:05 index maintenance mode=build reason="no completed import yet"
lastImported=never baselineSeries=0
```
**The premise was false for the one database where it was expensive.**
That catalog is not stale; it is deliberately kept in the older text
encoding, which `artifactStoresText` and `sourceColumns` exist to
tolerate — so it would have been judged stale and dropped on *every*
run. And `retireLibraryTables`, in the same package, already documents
the opposite rule for this database: drop everything the datamap does
**not** call Cache.
`fix(database): never retire the catalog the index build derives` makes
the policy a build tag (`retireStaleCache`, false under `indexbuild`),
which is how this project already separates the index tools. It prevents
recurrence and cannot undo the drop: that volume was the only copy.
**What it cost, and the shape of the cost.** A full re-import from the
MetaBrainz dumps, resumed across runs from a checkpoint, at a rate that
swung between 2 and 15 MB/s. The job runs on **every push to main** with
a 3 h budget on a runner of capacity 1 — so until the import completes,
every push books three hours and ordinary CI queues behind it. That is
the real damage: not one lost job, but a repeating one.
So the `push:` trigger in `index-artifact.yml` is **commented out**
until a run reports `complete=true`; the weekly cron and
`workflow_dispatch` still resume the build, which is all it needs.
Restoring those two lines is the whole revert.
Three things worth keeping from it:
- **A repair belongs where its assumptions hold.** `NewDB` is the one
chokepoint every binary in this project shares, including the one
whose database cannot be re-derived cheaply. Anything destructive
there needs to ask which binary it is in — the build tag was available
and is what the fix used.
- **The only copy of a 205 GB derived asset is one Docker volume.**
There is no snapshot, so the restore time is "however long
MetaBrainz takes today". A periodic copy would turn this class of
incident into twenty minutes.
- **The fix's residual trade is now the thing to watch**: with Cache
tables never retired under `indexbuild`, a future `explore_index`
column fails that job loudly at build time instead of silently
rebuilding. That is the right default, and it means the next schema
change touching `explore_index` needs a deliberate plan for this one
database rather than none.
## Two guards for the index cache, and what each one is worth (2026-08-17)
Both come out of the incident above, and they protect different halves
of it.
**`TestNoCacheTableIsRetiredHere` asserts the outcome, not the
mechanism.** The test that shipped with the fix pins one table in one
wrong shape, which is the failure that happened; what actually cost the
rebuild was a destructive repair added at `database.NewDB` — the
chokepoint every binary here shares — without asking which binary it was
in. The next one will have a different name and a different reason. So
this puts *every* `datamap` Cache table into a shape the schema has
moved past, opens the database the way `cmd/indexbuild` does, and
requires all of them to still be there.
Three things it got right by being written this way. The table list is
`datamap.ByKind(Cache)`, so the two credit tables added the same day
were covered without anyone adding them — flipping the policy back fails
on **five** tables including `artist_credit_part` and
`artist_credit_ref`, where the single-table test fails on one. It
asserts rows survive as well as the table, because SQLite does an
implicit DELETE before a DROP and a repair that recreated the table
would otherwise look identical. And it *accepts* an error from `NewDB`,
because that is the trade the fix documents: loud failure instead of a
silent day of downloading.
**`scripts/index-cache-snapshot.sh` covers the half no test can.** The
volume held the only copy of a catalog whose rebuild is hours of someone
else's bandwidth. `VACUUM INTO` rather than `cp`, because a byte copy of
a live SQLite file is a corrupt file of plausible size; the staging
directory is deliberately not copied, since a build resumes without it;
and the snapshot is reopened and asked for its catalog row count before
any rotation happens. Both failure paths were exercised rather than
argued: a corrupt source and an empty catalog each exit non-zero, delete
their own output, and leave the previous snapshots in place.
`docs/index-cache.md` is the restore procedure, and the number that
makes it worth having: a restored snapshot resolves to `refresh` and
folds in the incremental listens since — minutes, against the 323 h a
rebuild was estimating.
## A green release pipeline can ship an empty changelog (2026-08-18)
`conventional-changelog-conventionalcommits@10` is silently incompatible
with the writer `@semantic-release/release-notes-generator@14` depends on
(`conventional-changelog-writer@^8`). Every release note renders as a bare
`## 0.0.1 (date)` heading with **no sections and no commits under it**, no
step fails, and the release ships with an empty body.
It is pinned to `9` in `.gitea/workflows/release.yml` and in
`make release-dry`, which must stay identical. **Check the rendered notes,
never the exit code** — this is invisible to every tick in the pipeline.
## semantic-release needs push rights to the branch even when it never pushes to it (2026-08-18)
Core runs `git push --dry-run HEAD:<branch>` as a permission check, before
and independently of any plugin. With `@semantic-release/git` removed
nothing ever pushes to `main`, and the check still runs.
Two things this looked like and was not:
- **Not branch protection.** A `--dry-run` push does not reach the
pre-receive hook: pushing one to protected `main` with a write-scoped
token succeeds. So `main`'s `enable_push: false` is not what fails here.
- **A flat `403 Forbidden`, not Gitea's protection message.** That is the
tell. `PACKAGE_TOKEN` had package-write and repo-*read* — enough to
clone a private repo, so every other workflow was fine — and needed
`write:repository`.
## A tag-triggered workflow runs the workflow file at the *tagged* commit (2026-08-18)
Not the one on `main`. Moving `v0.0.0` onto a pre-merge commit ran that
commit's version of `homebrew-formula.yml`, which predated the `v0.0.0`
skip guard added in the same plan, and it pushed a `0.0.0` formula to the
public tap.
A guard added today does not protect a tag that points at yesterday. When
re-pointing a tag, check what the workflows looked like *there*.
+1 -1
View File
@@ -13,7 +13,7 @@ reviews. Nothing was changed.
Findings below are numbered `H-n` (hands-on) and cross-reference the
static reports where they overlap. The reconciliation plan built from
all four files is `.planning/plans/pending/007-ui-reconciliation.md`.
all four files is `.planning/plans/completed/007-ui-reconciliation.md`.
---
@@ -1,5 +1,7 @@
# 012 — What we ask the network for, and what we already had
> **Completed.** Findings 1, 2 and 4 shipped. Finding 3 — the bound-but-uncalled methods — is now **#86**.
**Status:** all four findings fixed. Lint (3 configs), Go tests (3
configs), `tsc` and 752 Vitest tests pass; **not driven against the
real app**, so the numbers below are read off the code, not measured.
@@ -1,5 +1,7 @@
# 015 — Android release pipeline
> **Completed.** The pipeline ships a signed APK from CI on every `v*` tag; `docs/android-release.md` is its operating document.
Ship an Android APK from CI on every version tag, published to the Gitea
generic package registry so Obtainium can poll a plain URL.
@@ -0,0 +1,339 @@
# 015 — Multi-artist credits, navigable
> **Completed.** Phases 1, 2 and 4 shipped. Running the ingest against the real dump and publishing an artifact that carries credits is **#88**; Phase 3 (`file_artists`) is **#89**, blocked on it.
## The problem
A track credited to more than one artist has exactly one navigable
artist in this app, and the others are punctuation.
`audio_files` carries `artist_credit` (the credit as tagged, for
display) and `artist_id` (one artist, for grouping and browsing).
`primaryArtist()` (`backend/library/artistcredit.go:53`) resolves that
one artist by *string-parsing* the credit: it strips a " feat. "
clause, and deliberately does not split on `&`, `x`, `with` or `,`
because those appear inside real artist names. So "Lana Del Rey ft.
Sean Lennon" stores Lana Del Rey and discards Sean Lennon entirely,
and "Alina Baraz & Galimatias" stores one artist whose name is the
whole credit.
### What the measurement says
Measured 2026-08-16 against a real 26,069-file library (19,840 mp3,
6,229 flac; 57 unreadable, m4a/ogg not examined), plus an 80+80
MusicBrainz `inc=artist-credits` sample.
- **13%** of a random sample of the library's recordings have more
than one credited artist in MusicBrainz (10 of 79 resolved).
Extrapolates to ~3,250 of the 24,989 files carrying a recording
MBID.
- **0.86%** of files (224) carry any structured multi-artist signal in
their own tags. mp3 carries **zero** files with multiple
`MUSICBRAINZ_ARTISTID` values across 19,840 files; flac has 87.
- **1,286** files say "feat." in `ARTIST`; **1,159 of them (90%)**
have nothing structured behind it. A sample of 80 such files was
multi-artist in MB **80 of 80 times**.
CLAUDE.md currently justifies plan 013's removal of `artist_credit` /
`artist_credit_artist` with "3 credits of 2,823 listed more than one
artist". That figure measured **our own writer**, not the library:
`cachedLinkArtist` was called exactly once per credit
(`e7748f1^:backend/library/library.go:1842`), so a collaboration could
never have been recorded, and the three were resolution collisions on
shared credit text. Dropping the join table was still correct — it only
ever held one row, so it was pure join cost — but the stated evidence
does not support "multi-artist is rare". Correcting that claim is part
of this plan.
### Why the tags cannot answer it
Deriving the decomposition locally, with no network, works **79% of the
time** (169 of 215 files with a multi-value `ARTISTS` tag: mp3 69/105,
flac 100/110), and the failures are systematic rather than random:
```
ARTIST = '2Pac feat. Snoop Dogg, Nate Dogg, Hussein Fatal & Yaki Kadafi'
ARTISTS = ['2Pac', 'Snoop Doggy Dogg', 'Nate Dogg', 'Fatal', 'Yaki Kadafi']
```
`ARTISTS` holds **canonical** artist names; `ARTIST` holds
**as-credited** names. Locating one inside the other fails on
"Snoop Doggy Dogg" vs "Snoop Dogg", on "Fatal" vs "Hussein Fatal", and
on Unicode (`Michel'le` vs `Michelle`, `K-Ci` vs `KCi` — U+2010, not
a hyphen). That distinction is precisely what a join phrase encodes,
and it is why this cannot be a tag-parsing feature.
Two format details that will mislead anyone re-running the probe:
Picard writes `ARTISTS` **slash-joined into one TXXX frame** on mp3 and
as **true repeated Vorbis keys** on flac, so a probe splitting only on
NUL undercounts mp3 to zero.
## The shape
MusicBrainz models a credit as ordered parts, and the credit *string*
is derived from them — `artist_credit.name` is a cached render, nothing
more. Each participant is `(position, artist, name, join_phrase)`,
where `artist` is the MBID (canonical, what you navigate to) and `name`
is the credited spelling (what you display).
**Join phrases are assembly instructions, not disassembly
instructions.** Rendering is a concatenation, never a search:
```
for each (position, artist_mbid, credited_name, join_phrase):
emit link(credited_name -> artist_mbid)
emit text(join_phrase)
```
The link positions are known **by construction**. This is load-bearing:
if we instead located each `credited_name` inside the stored
`artist_credit` text, we would reintroduce the mismatch above — the
stored string may have come from the tags while the parts come from the
catalog, and those **disagree for ~1 in 3 multi-artist files** (61 of
90 sampled credits rendered exactly equal to the tag string).
Divergences seen: `'Skrillex feat. Swae Lee'` tagged vs
`'Skrillex & Swae Lee'` in MB; `'STRFKR'` vs `'Starfucker'`;
`'Zedd feat. Hayley Williams'` vs `'... of Paramore'`. Either MB was
edited after tagging or Picard versions differ; either way the search
would miss or match the wrong span.
So `audio_files.artist_credit` stops being the source of truth and
becomes the **fallback**, used only where there are no parts.
## Where the data comes from
The catalog carries the decomposition; no user ever makes a
per-recording call. Two sources were ruled out first, both cheaply:
- **The canonical dump — which is what CI already pulls
(`dumpimport.go:84-85`) — does not have it.**
`canonical_musicbrainz_data.csv` gives `artist_mbids` (ordered list)
and `artist_credit_name`, but that last column is the *rendered*
string. Splitting it on CI needs the as-credited names, so CI would
fail exactly the way a local parse does.
- **The JSON dumps do not cover the catalog.**
`json-dumps/recording.tar.xz` is 31 MB / 368 MB uncompressed and
holds **153,691 recordings**, not ~35M. Measured against the test
library's 24,885 recording MBIDs: **0.00% overlap, zero rows**. It is
some other subset and is not usable.
That leaves the core dump, **`mbdump.tar.bz2`** (7.1 GB compressed at
the 20260815 export), from
`https://data.metabrainz.org/pub/musicbrainz/data/fullexport/`. Four
members are needed:
| member | why | approx rows |
| --- | --- | --- |
| `mbdump/artist_credit_name` | `(artist_credit, position, artist, name, join_phrase)` — the payload | ~4M |
| `mbdump/artist` | `id -> gid`, since the above references artist *row ids* | ~2.6M |
| `mbdump/recording` | `gid -> artist_credit`, to key credits by recording MBID | ~35M |
| `mbdump/release_group` | same, for album credits | ~2M |
### Coverage is not a concern
Of 24,885 distinct recording MBIDs in the test library, **24,808
(99.7%)** already have an `explore_index` recording row, measured
against a database at 2,052,200 rows — i.e. shipped-artifact coverage,
not a local build's. The popularity filter does not strand the long
tail here.
## Status
- **Phase 1 — done.** `backend/explore/dumpcredits.go` +
`dumpcreditswrite.go`, wired into `dumpimport.go`'s `run` behind its
own `credits_import_done` marker.
- **Phase 2 — done.** `cmd/indexexport` writes the two tables;
`artifactimport.go` reads them behind `artifactHasCredits()`.
- **Phase 4 — done, and it does not need Phase 3.** `explore.GetCredits`
reads the catalog tables keyed on the *recording* MBID, which both
sides of the app already carry — a catalog row has one and so does a
local file (`library.Track.RecordingMBID`). So one binding serves the
Explore pages and the library's own lists, and all ten artist-link
call sites render credits today without a local table.
- **Phase 3 (`file_artists`) — not started, and now an
offline-resilience task rather than a prerequisite.** The table is
deliberately *not* declared yet: nothing writes or reads it, and a
schema file plus a datamap note describing behaviour that does not
exist is a claim the code cannot back. Its remaining
value is that credits currently vanish when the catalog is absent or
still downloading, which is precisely the `no-index` state
`ShelfPage.State` exists to describe. Materialising into
`file_artists` is what makes a library stand on its own.
**Nothing renders yet in practice**, because no published artifact
carries credit tables — every credit falls back to its single link
until an index build with Phase 1 runs and is exported.
**Column layouts are verified against the real 20260815 export**, not
taken from the schema docs — `artist(id, gid, …)`,
`artist_credit(id, name, artist_count, …)`,
`artist_credit_name(credit, position, artist, name, join_phrase)` and
`recording(id, gid, name, artist_credit, …)` were each read out of the
dump. `release_group` shares `recording`'s first four columns and is
the one layout still taken on trust; `ErrDumpShape` turns a wrong guess
into a loud failure rather than a quietly wrong catalog.
**Still unrun: the ingest against the real 7.1 GB dump.** Everything is
covered by tests over a synthetic tar, which cannot catch a surprise in
the other ~35M rows.
### Phase 1 — Ingest credits on CI
New dump stage in `cmd/indexbuild`, behind the `indexbuild` tag with
the rest of `dumpimport.go`'s stages.
**Constraint from `b98840e`:** `cmd/indexbuild` is built
`CGO_ENABLED=0` in a plain `golang` container and must not reach the
Wails `application` package — `TestIndexToolsDoNotImportWails` walks
`go list -deps -tags indexbuild`. Nothing here should need it, but a
new `ServiceStartup` hook on a package this imports is how it comes
back. Go's `compress/bzip2` is pure Go and decompress-only, which is
all this needs.
**Measured, 20260815 export.** Tar members are **alphabetical**, and
that is favourable: `artist` (435 MB), `artist_credit` (414 MB) and
`artist_credit_name` (237 MB) all fall inside the first ~900 MB
compressed, while `recording` and `release_group` come later. So the
maps are complete before the rows that consume them arrive, and no
recording data is ever buffered.
Pure-Go `compress/bzip2` decompresses at **26 MB/s uncompressed /
8.7 MB/s compressed** (measured on a 250 MB prefix, 3.01x ratio) —
**~13.7 min** for the whole file single-threaded, and less because the
stream can stop after `release_group` rather than reading the
`series`/`tag`/`track`/`url`/`work` tail. The 2 MB/s origin throttle
dominates, as it already does for every other dump here.
Do not, however, *depend* on the ordering: assert it and fall back to
buffering if a future export reorders, rather than silently emitting
nothing.
- `artist` -> `map[int32]uuid16` (~2.6M x ~20 B = ~60 MB)
- `artist_credit_name` -> `map[int32][]creditPart` (~4M x ~40 B =
~200 MB)
- `recording` / `release_group` -> emit `gid -> credit_id` **only for
MBIDs already in `explore_index`** (the kept set is ~1.4M x 16 B =
~22 MB), which is what keeps 35M rows from being held
Peak ~300 MB, one sequential pass.
**Only multi-artist credits are stored.** A single-artist credit is
`(name, "")` and is already fully described by `explore_index`'s
`artist_name` / `artist_mbid`; storing it would triple the table for
nothing. Post-filter after loading, once the row count per credit is
known.
New tables (and `datamap` entries, or `TestCatalogCoversSchema` fails
the build — both are `Cache`, matching `explore_index`):
```
artist_credit_part(credit_id, position, artist_mbid, credited_name, join_phrase)
```
with `explore_index.artist_credit_id` as the link. Credits are
**shared** — an album's twelve tracks by one artist share one credit
row — which is the opposite of 013's local verdict, and correctly so:
1:1 in a local library, genuinely many-to-one at 2M-row catalog scale.
### Phase 2 — Ship them in the artifact
`cmd/indexexport` currently creates exactly two tables in the artifact
(`explore_index`, `artifact_meta`, at `cmd/indexexport/*.go:147,170`),
so this is a structural addition, not a column.
Estimated size: ~13% of 1.4M recordings, deduplicated by shared credit,
at ~2.3 parts each — order 400k rows, ~18 MB uncompressed. Against a
~0.6 GB install that is acceptable; it must be measured rather than
assumed before merge.
`artifactimport.go` must read it **only if present**, on the writer
handle where `core` is attached — the `artifactHasTotals()` /
`artifactStoresText()` pattern (`artifactimport.go:145-175`), one step
up from a column to a table. An artifact published before this exists
is still a perfectly good catalog and must import as one that declines
to answer. Adding this to the importer's SELECT list without the probe
is how every already-published artifact starts failing.
`artifactCatalogColumns` gains `artist_credit_id`; it is kept in sync
with the exporter by `TestArtifactColumnsMatchExporter`.
### Phase 3 — Materialize locally
```
file_artists(audio_file_id, position, artist_id, credited_name, join_phrase)
```
`credited_name` is stored **per row**, not looked up from
`artists.name` — that is the Snoop-Doggy-Dogg distinction, and it is
the whole point.
Filled at scan/import time by joining `audio_files.recording_mbid`
against the catalog. **Materialized rather than resolved live**,
because the catalog is a downloaded artifact that can be absent or
still arriving — that is why `ShelfPage.State` has a `no-index` value —
and a library whose track rows lose their artists when the catalog is
missing is worse than today.
That implies a backfill for the case where the catalog arrives *after*
the library was scanned. It registers with `jobs` (progress, cancel)
like every other long pass, and takes a **distinct kind** from
`index-build`, since `job-controls.ts` keys its "you will discard hours
of downloading" confirmation on that kind.
`artists` gains rows for guests who own no files. **This changes what
the artists grid shows** and is an open question below.
### Phase 4 — Render
`utils/explore-link.ts` gains a credit-rendering entry point taking
ordered parts and returning a `TemplateResult`. Every row and detail
view already renders artist names through it, so they inherit
multi-artist links without individually knowing credits exist — the
property that made centralising it worthwhile.
Its existing fallback philosophy already covers the no-parts case: "a
list where some rows are clickable and others silently are not reads as
a bug, not as a statement about metadata." Where there are no parts
(no recording MBID, or no catalog row — ~4% of the test library) render
today's behaviour: the flat `artist_credit` string with one link to the
primary artist. **Do not split the string there.** There is genuinely
no information to split on, and that is the one place the temptation
returns.
`primaryArtist()` stays exactly as it is. It remains the fallback and
is still what `artist_id` means.
## Open questions
1. **Catalog credit vs tagged credit, when they disagree** (~1 in 3
multi-artist files). Rendering the catalog's decomposition is what
makes names navigable; preserving the file's is what makes the app
reflect the user's files. Leaning toward: render the catalog
decomposition, keep `artist_credit` as the fallback string. Wants a
deliberate decision, not an accident.
2. **Do guest artists appear in the artists grid?** Phase 3 creates
`artists` rows for people who own no files. The grid currently means
"artists in your library" and joins `audio_files`. A guest on one
track is arguably in the library and arguably not. Whichever way,
the ownership question stays "is there a file" — that rule does not
bend.
3. **`release_group` credits** are ingested in the same pass for
nearly nothing, but album-artist rendering is a separate surface.
Ship the data in phase 1, render in a follow-up rather than widening
phase 4.
4. **Our own `tagwriter`** does not write `ARTISTS` or multiple
`MUSICBRAINZ_ARTISTID` frames, so autotagging a folder degrades the
very field this rests on — the same shape as the existing
track-totals note. Out of scope here; worth recording.
## Verification
- Coverage: re-run the library probe and assert `file_artists` is
populated for ~13% of files, not ~0.9%.
- `TestCatalogCoversSchema` / `TestLifetimesMatchSchema` for the new
tables.
- `TestIndexToolsDoNotImportWails` still passes with the new stage.
- An artifact **without** the credits table imports cleanly (the
`artifactHasTotals` regression shape).
- Round-trip: a known multi-artist recording renders each name as a
separate link with the correct join phrases between them.
@@ -1,5 +1,7 @@
# 016 — What Android parity would actually take
> **Completed.** Sections A, B1, B2 and B4 shipped. B3, writing tags on the device, is now **#87**; the device-found UI faults are #51#72, sequenced by #73.
> **Status: all of section A is done.** A1A3 landed with "let the app
> reach the user's music"; A4 (MediaSession, transport notification,
> audio focus) landed with "survive the screen locking". The direction
@@ -0,0 +1,87 @@
# 017 — Releases that happen by themselves
**Shipped as `v0.0.1`.** A merge to `main` now reads the Conventional
Commits since the last tag, cuts the tag and the Gitea release whose body
is the generated changelog, and the four publishing workflows build that
tag and attach their artifacts. Nothing is released by hand.
## What it looks like now
`release.yml` on push to `main` → semantic-release → tag → four `v*`
workflows in parallel (serialised in practice by the capacity-1 runner):
| workflow | publishes | attaches |
| --- | --- | --- |
| `arch-package` | pacman registry | `…-x86_64.pkg.tar.zst` |
| `android-apk` | generic registry (Obtainium) | `…-android-arm64.apk` |
| `desktop-assets` | — | `…-linux-amd64.tar.gz` |
| `homebrew-formula` | the public tap | — (builds from source) |
Verified on the real thing: all five green, three assets on the release,
the tap at `0.0.1`, and the Obtainium `latest` URL serving 200.
## The five decisions, and what they cost
1. **semantic-release, not a shell script.** The first draft of this plan
proposed hand-rolling it and the argument did not survive checking:
`@semantic-release/exec` is first-party and current, and the
Gitea-shaped part is one `curl`. What I would have hand-rolled —
commit parsing, semver ordering, note rendering — is the part with the
edge cases and none of it is Gitea-shaped.
2. **`@saithodev/semantic-release-gitea` is a dead end** and was offered
before it was checked: last published 2022, `got@10`, and no peer
dependency on semantic-release at all.
3. **No `@semantic-release/git`.** `main` is protected, so a changelog
commit-back is rejected by the pre-receive hook — and would be
rejected *after* the tag was pushed, leaving a tagged release the run
reports as failed. The release page is the changelog;
`.release-notes.md` is a gitignored carrier and `CHANGELOG.md` is a
signpost.
4. **Versions restart at `0.0.1`**, a downgrade on every channel. No
`epoch`, no `versionCode` offset: both are permanent, a reinstall is
once. Documented in `packaging/homebrew/README.md` and
`docs/android-release.md`.
5. **No macOS and no Windows.** `GOOS=darwin CGO_ENABLED=0` fails at
`wails/v3/pkg/mac` and there is no macOS runner, so Homebrew-from-source
stays that channel. Windows cross-compiles in ~2.5 s and is withheld
because no build of it has ever been *run*.
## Four things that only showed up by running it
- **`conventional-changelog-conventionalcommits@10` renders empty
notes.** Silently: right version, right tag, every step green, and a
release body that is a bare `## 0.0.1 (date)` heading with nothing
beneath it. Held at `9`, in `release.yml` and `make release-dry`, with
the reason beside both. **Check the rendered notes, never the exit
code.**
- **semantic-release core dry-run-pushes to the release branch** as a
permission check, independently of any plugin. `PACKAGE_TOKEN` had
package-write and repo-*read* — enough to clone, not enough for this —
and it failed with a flat `403 Forbidden` that reads exactly like
branch protection. It is not: a `--dry-run` push never reaches the
pre-receive hook, which a one-line experiment settled. The token needed
`write:repository`.
- **The floor tag must go on `HEAD^`, not `HEAD`.** Seeded on the merge
commit itself it leaves nothing between the floor and HEAD, and
semantic-release correctly reports there is nothing to release. The
first run did exactly that and cut nothing.
- **A tag-triggered workflow runs from the tagged commit's tree.**
Moving `v0.0.0` back to `6fb7b5e` ran the *pre-merge* homebrew
workflow, which predates the `v0.0.0` skip guard, and pushed a `0.0.0`
formula to the public tap. Self-corrected at `0.0.1`. The corollary is
general: a guard added today does not protect a tag pointing at
yesterday.
## Two mechanisms confirmed, having been assumptions
- **A tag pushed with a user PAT does start the `v*` workflows**; one
pushed with the Actions token does not (go-gitea#33123). Both halves
are load-bearing and both were observed: the floor seed triggered
nothing, and the release tag triggered all four.
- **Tags are not protected** on this repo, only `main` — which is what
lets semantic-release tag at all.
## Left behind deliberately
`v0.0.0` stays on `origin` as the floor. It carries no release, and all
four publishers skip it by name.
@@ -1,5 +1,7 @@
# Autotag (v1.3) — MusicBrainz Autotagger
> **Historical record.** Phases 008010 shipped, and the scoring engine was subsequently overhauled (`recommend.go`, `rank.go`, `mixedbag.go`), which makes the 011/012 sections below stale in their details. What is actually left is **#90** (auto-accept and entry points) and **#91** (settings, and a way back from the dismissed file-write warning).
The MusicBrainz autotagger, collectively **v1.3**. Builds on the explore-browser API client + cache foundation. Five sequential phases (008012), each depending on the prior one.
| Phase | Title | Status |
@@ -1,195 +0,0 @@
# 010 — Owned albums, offline
**Status:** not started — and **much smaller than when it was written**
**Branch:** none yet
**Created:** 2026-08-13
**Depends on:** nothing
**Related:** the `AlbumReleasesFailed` fix that prompted it, and the
tag-derived completeness that landed after it (same session)
---
## What already shipped, and what it leaves
The common case is solved without this plan. `GetAlbumCompleteness`
reads the "5/12" denominator off the files' own tags — persisted to
`release_group_recordings.total_tracks`, having been extracted at every
scan since forever and discarded — and an album that is **MBID-matched
and complete** now opens with **no catalog call at all**. Identity from
the MBID, tracklist from the tags; those were the two things the browse
was being spent on.
So the set this plan still has to serve is not "albums you own a track
of". It is:
- albums that are genuinely **incomplete** (the catalog is the only way
to say *which* tracks are missing — tags give the count, not the
names), and
- albums whose tags **never declared a total**, where completeness is
unknowable locally and the catalog is the only source.
On a well-tagged library that is a small minority, which changes the
economics below considerably: the run is shorter, and the rate limiter
contention that dominates this design is proportionally less severe.
Re-measure before building — the answer may now be "the prefetch is
enough".
---
## The problem
Opening an album detail page for an album **you already own** hits
MusicBrainz. Every time it is not in the response cache, which for most
of a library is every time, because nothing warms that cache except a
capped prefetch on the artist page.
The user's framing: *this is a classic example of an album we should
have had locally.*
## Why we do not have it, despite the discography backfill
`BackfillLibraryDiscographies` / `EnsureArtistDiscography`
(`backend/explore/searchindex.go:301`, `:397`) do less than the name
suggests. Per artist, `indexOneArtist` fetches:
- `fetchTopReleaseGroups` — capped at `indexMaxRGs` (50)
- `fetchTopRecordings` — capped at `indexMaxRecs` (200)
and writes them as **flat `explore_index` rows**. There is no release
group → tracklist relation anywhere in the index, and no release-level
rows at all. `explore_index` recordings carry `caa_release_mbid` and
`release_name`, which name the release used for cover art — not a
tracklist.
So "we have full discographies for library artists" means *we know
which albums the artist made, offline*. It has never meant we know
what is on any of them.
The only store of release-level catalog data in the app is `http_cache`
under `mb:browse:releases:<rg>` (90-day TTL, `musicbrainz.go:27`),
populated **only** by a live `BrowseReleases` with
`Includes: ["recordings", "media"]` at `MaxLimit` — the most expensive
call the app makes to MusicBrainz. It is warmed by exactly one thing:
`PrefetchReleases` (`explore.go:746`), capped at 8, called only when an
artist page renders.
An album opened from the library grid therefore always browses live.
## What to build
**A post-scan backfill that warms the release cache for release groups
that are owned but not known-complete** — bounded, resumable, and
shaped exactly like `BackfillLibraryDiscographies`, which is the proven
pattern for this in the codebase.
The scoping rule is the user's and it is the right one: not "every
album by every artist in the library" (50 release groups per artist,
mostly never opened) but albums with owned tracks — narrowed further,
now, to the ones a local answer cannot already cover. The query gains
one clause: skip release groups whose `GetAlbumCompleteness` reports
`complete`.
Sketch:
1. A query for release groups with ≥1 owned track and no warm release
cache entry. `release_groups.mbid` is the key; the owned-track join
is `audio_files → recordings → release_group_recordings`, the same
shape `unenrichedLibraryArtistMBIDs` already uses one table over.
2. Order by owned-track count descending, so the albums the user has
most of are warmed first — same reasoning as the discography
backfill's ordering, same benefit if a run is cut short.
3. Run through `releasesSF`, so it never double-fetches a release group
an interactive open is already handling.
4. Bound a run (`discogBackfillMaxPerRun` has a value to copy) and make
it resumable: the resume marker is the response cache itself —
`BrowseReleasesCached` already answers "is this one done", so unlike
the discography path this needs **no new flag column**.
5. Trigger it where `BackfillLibraryDiscographies` is triggered, and
register it with `jobs` so it has progress, pause and cancel like
every other long-running operation.
### The rate limiter is the whole design constraint
> **Update (2026-08-13): the priority half is built, and the sentence
> below is wrong on a detail.** `e.mb` runs on `mbSearchLimiter`
> (`NewRateLimiterBurst(3, 1)`); the 1 req/s `NewRateLimiter()` cited
> here is the *artist image* limiter. Both are shared and both were
> FIFO. `RateLimiter.WithBackgroundLane` + `WithBackgroundPriority(ctx)`
> now make a marked caller yield to interactive work and pace at 1/s,
> and `jobs.KindCatalogEnrich` + `startBackfillJob` give the existing
> backfills progress and cancel. **"Do not start until the priority
> question has an answer" is satisfied** — mark this backfill's context
> and register it the way `BackfillLibraryDiscographies` now is.
> `PrefetchReleases`' cap of 8 is still unrevisited.
One shared `NewRateLimiter()` at 1 req/s (`explore.go:84`) serves this,
`PrefetchReleases`, and every interactive browse. A backfill over a
few thousand owned albums is *hours* of wall clock at that rate — which
is fine for a background job, and not fine if it starves the album page
the user is looking at right now.
That is the real work in this plan, and it is not the query:
- Interactive browses need to **jump the queue**. Today they cannot;
there is one limiter and it is FIFO.
- `PrefetchReleases`' cap of 8 was sized when nothing else competed for
the limiter. Revisit it in the same change.
- The 60 s fallback the `AlbumReleasesFailed` fix installed is sized
for today's contention. If a backfill can queue behind it, that
number is wrong again — which is an argument for priority, not for a
bigger number.
Do not start the query until the priority question has an answer.
## The alternative that was considered and rejected
**Project release-group tracklists in the dump build and ship them in
the artifact.** The data is there: `canonical_musicbrainz_data.csv`
carries `release_mbid` *and* `recording_mbid`
(`dumpcatalog.go:520`), and `release_to_rg` already maps release →
release group. It is derivable from bytes the index build already
streams, with no new API surface at all, and it would work offline on
first launch with no per-user backfill.
It is rejected **for this plan** because the artifact is built
centrally and is byte-identical for every user, so "albums the user
owns a track of" cannot be a filter on it. Shipping tracklists for the
whole catalog means per-recording rows against a ~900 MB artifact
budget (~426 B/row measured), and gating on a popularity floor means it
is absent for exactly the obscure albums a local backfill would have
covered.
Worse than absent, in fact — and this is the argument that actually
kills it. The floor is not one number over artists; it is a **per
artist track budget** (`dumpcatalog.go:58-89`): 50 tracks for a tier-A
artist, 25 for tier B, 12 for tier C. A projected tracklist would
therefore be *whichever* of an album's tracks survived that budget,
with nothing marking the rest as absent — so the album page would count
owned against a truncated denominator and render "Play 7 of 9" for a
twelve-track album. That is a confident lie, where the honest states
this plan's alternative produces (complete / incomplete / unknown) are
at worst silent.
Note that `markLibraryArtists` (`dumpcatalog.go:246`) already grants
every library artist full coverage — 500 tracks, 100 release groups —
by reading the local library, so the per-user tailoring this option
supposedly cannot have does exist in code. It is a no-op in the CI
build (empty library), and reaching it means a **local** dump build:
the ~205 GB, half-a-day download the entire artifact design exists to
avoid. Whoever finds that function next should read this paragraph
before getting excited about it.
Worth revisiting if the artifact ever gains per-user tailoring, or if a
measurement shows the row count is smaller than feared. Note it also
yields the *canonical* tracklist rather than MusicBrainz's full version
list, so the versions dropdown would still browse live when opened.
## Done when
- Opening an owned album that has never been opened before renders its
catalog tracklist with no network call, after one backfill run.
- An interactive browse issued while the backfill is running is not
delayed by it.
- The backfill appears in the jobs indicator, and can be paused and
cancelled there.
- A second run after a completed one does approximately nothing.
+52 -16
View File
@@ -1,6 +1,20 @@
# semantic-release configuration
# Runs on main branch pushes to auto-determine version from conventional commits.
# Creates a git tag + GitHub Release draft; a separate workflow builds binaries.
# semantic-release configuration.
#
# Runs on pushes to main from .gitea/workflows/release.yml: determine the
# version from the Conventional Commits since the last tag, write the
# changelog, commit it, push the tag, and create the Gitea release.
#
# **There is no `@semantic-release/github` plugin here and there must not
# be.** Gitea's API is `/api/v1` and is not GitHub's surface. The Gitea
# community plugin (@saithodev/semantic-release-gitea) was considered and
# rejected: last published 2022, depends on got@10, and declares no peer
# dependency on semantic-release at all — i.e. untested against anything
# since v19, against a core now at v25. `exec` is first-party, current,
# and the Gitea-shaped part is one curl.
#
# The type list below is the one scripts/commit-check.sh enforces the
# grammar for — keep the two in step, or semantic-release will silently
# decline to release something the commit hook accepted.
branches:
- main
@@ -63,19 +77,41 @@ plugins:
section: Build
hidden: true
# Write CHANGELOG.md.
# Render the notes to a file.
#
# **This plugin is here to carry the notes, not to maintain a document.**
# It is how they reach the Gitea API *without being interpolated into a
# shell command*: release notes are rendered commit messages — arbitrary
# text carrying backticks, quotes and `$` — so templating
# ${nextRelease.notes} into `publishCmd` would be a shell injection with
# the commit log as its input. scripts/gitea-release.sh reads the top
# section of this file instead, and the only thing interpolated below is
# a semver string.
#
# The target is a gitignored build artifact rather than CHANGELOG.md,
# because nothing commits it back — see below.
- - "@semantic-release/changelog"
- changelogFile: CHANGELOG.md
- changelogFile: .release-notes.md
changelogTitle: "# Release notes"
# Commit the changelog back to the repo.
- - "@semantic-release/git"
- assets:
- CHANGELOG.md
message: "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
# Create the Gitea release, whose body is that section.
# `publish` runs after `prepare`, so the tag already exists by here.
- - "@semantic-release/exec"
- publishCmd: "./scripts/gitea-release.sh ${nextRelease.version}"
# **There is deliberately no @semantic-release/git here.**
#
# `main` is a protected branch with `enable_push: false` and an empty
# push whitelist, so a changelog commit-back would be rejected by the
# pre-receive hook — *after* the tag had already been pushed, leaving a
# tagged release the run then reported as failed. The alternative was to
# whitelist the CI user, which weakens a protection someone set on
# purpose and lets a bot push to main without passing the checks every
# human PR has to.
#
# So the release page is the changelog. Tags are not protected, so the
# tag push semantic-release does itself is unaffected. CHANGELOG.md in
# the repo is a signpost to the releases page and is not written by any
# of this; a file that claimed to be a changelog and silently stopped
# updating would be worse than no file at all.
# Create the GitHub Release (draft, so the build workflow can attach binaries).
- - "@semantic-release/github"
- draft: true
successComment: false
failComment: false
releasedLabels: false
Symlink
+1
View File
@@ -0,0 +1 @@
CLAUDE.md
+16 -372
View File
@@ -1,377 +1,21 @@
## [1.3.0](https://github.com/onion-4-dinner/yellowjacket/compare/v1.2.3...v1.3.0) (2026-03-20)
# Changelog
### Features
The changelog is the releases page:
* **09-01:** add scan control events and cancelled metrics field ([c695024](https://github.com/onion-4-dinner/yellowjacket/commit/c695024241a7513b8fedb3fbf7ff364d0515b392))
* **09-01:** add scan control fields and per-scan cancellable context ([cf22e52](https://github.com/onion-4-dinner/yellowjacket/commit/cf22e52a64850a80b9fcc63c21d81313e6bd56ab))
* **09-02:** add frontend keyboard shortcut service, store, and controller ([40d4815](https://github.com/onion-4-dinner/yellowjacket/commit/40d48151dd798b57eed9f54a572ae4735356d09e))
* **09-02:** add shortcuts config package with default bindings and Wails persistence ([6285ca9](https://github.com/onion-4-dinner/yellowjacket/commit/6285ca9dc4e6f211197e377d01c485b1ef65c300))
* **09-03:** add scan control UI with pause/resume/cancel and confirmation dialog ([3914369](https://github.com/onion-4-dinner/yellowjacket/commit/391436927c826f2f17a4523be7829aefc04a6b12))
* **09-04:** add keyboard shortcuts section to config page with conflict detection ([0451fb3](https://github.com/onion-4-dinner/yellowjacket/commit/0451fb38805ff2c27e43deb152daa892e733d2db))
* **10-01:** implement migration 6 and pre-migration backup ([1179f56](https://github.com/onion-4-dinner/yellowjacket/commit/1179f56c3680112692e71e8dc7ce946446fa8a8a))
* **10-01:** update SQL schema files for multi-library fresh installs ([535855b](https://github.com/onion-4-dinner/yellowjacket/commit/535855b383a457dd2be3298b4361313bef22b39d))
* **10-02:** add migration 6 integration tests and NewTestDBWithLibrary helper ([bc15189](https://github.com/onion-4-dinner/yellowjacket/commit/bc151891b50e59e41da2e00dbfafbecaad11b4ac))
* **10-02:** add sqlc queries for libraries and update playlist queries for phantom support ([02548dd](https://github.com/onion-4-dinner/yellowjacket/commit/02548dd55e59b28f3d6c8d9614f209140c979250))
* **11-01:** per-library scan pipeline with queue coordinator ([943db1c](https://github.com/onion-4-dinner/yellowjacket/commit/943db1cf274bdf59daf28ab6c20f78ef5ef53105))
* **11-02:** update config-page with per-library progress display and queue-aware cancel dialog ([d01591d](https://github.com/onion-4-dinner/yellowjacket/commit/d01591d6cc054a63b832c05a3164a72fdcaba342))
* **11-02:** update library-manager with per-library progress and Scan All button ([d61f122](https://github.com/onion-4-dinner/yellowjacket/commit/d61f122b567e8ac2b30fa96c637cbebc14493c89))
* **12-01:** add queue compaction method and wire removal hooks ([5995dfd](https://github.com/onion-4-dinner/yellowjacket/commit/5995dfd01d61cd4d2c0749eeeee2a1f93b739d68))
* **12-01:** implement library CRUD methods and orphan cleanup pipeline ([bd44f83](https://github.com/onion-4-dinner/yellowjacket/commit/bd44f8306c9129b9420ad81938bcf8105a1cb55a))
* **12-02:** make config sections collapsible with chevron dropdown ([12c6782](https://github.com/onion-4-dinner/yellowjacket/commit/12c678284c7582bd85cd52722f4d405b0bd0e20f))
* **12-02:** remove Libraries sidebar nav item and view routing ([e199712](https://github.com/onion-4-dinner/yellowjacket/commit/e199712a56e1cb3c0fc43d3340abb892a6f5fa7b))
* **12-02:** replace config-page library section with full library management UI ([ffc5d96](https://github.com/onion-4-dinner/yellowjacket/commit/ffc5d9639cf7c916a4f846590ae0d67cf13afe27))
* **12-02:** selectable library list with checkbox scan targeting ([13a42ae](https://github.com/onion-4-dinner/yellowjacket/commit/13a42aea2287d7ed0ec9ff9856f52c1fa7767338))
* **12-02:** show scan progress bar inline in library list entry ([df824c6](https://github.com/onion-4-dinner/yellowjacket/commit/df824c6989e92b2aefaa1ddf05b131ee319612d8))
* **13-01:** add library-filtered Go query methods and FTS search ([5f7de50](https://github.com/onion-4-dinner/yellowjacket/commit/5f7de5060a5bc557b96203267de694ef366ed507))
* **13-01:** add library-filtered sqlc queries for all browse views ([5cc58ce](https://github.com/onion-4-dinner/yellowjacket/commit/5cc58ce66ab70d8d5a570df5067f79ae2201037e))
* **13-02:** add library filter dropdown and wire all views to respect active filter ([42b8cf9](https://github.com/onion-4-dinner/yellowjacket/commit/42b8cf9f52133499ffcd7363bd39dd0c1069e091))
* **15-01:** migrate FTS5 search_index to contentless_delete=1 ([cb5155b](https://github.com/onion-4-dinner/yellowjacket/commit/cb5155b8906357ff77c5c579d57d02cf2eec6abe))
* **15-02:** create backend/fileutil package with AtomicWrite ([4d64b5d](https://github.com/onion-4-dinner/yellowjacket/commit/4d64b5dcfe43951e8ec63383bbf72c99107c63c4))
* **16-01:** add selectAll() to SelectionController and dispatch shortcut:select-all event ([f567762](https://github.com/onion-4-dinner/yellowjacket/commit/f5677628ef283b67370630b564f23178e43da3d2))
* **16-01:** wire shortcut:select-all listener in track-list, queue-panel, and playlist-view ([906ea28](https://github.com/onion-4-dinner/yellowjacket/commit/906ea28751ce9f96fdeeb9410ab5f6518f09fcb9))
* **16-02:** add go-flac dependencies and implement FLAC tag writer ([3642cbe](https://github.com/onion-4-dinner/yellowjacket/commit/3642cbe0d58f8912a786a4fc5380c40403add94a))
* **16-03:** implement DB sync module for tag write pipeline ([2966079](https://github.com/onion-4-dinner/yellowjacket/commit/2966079625cd42412411429af02184d015526e9b))
* **16-03:** WriteTrackTags pipeline with player safety, scan mutex, events, and app wiring ([64322f9](https://github.com/onion-4-dinner/yellowjacket/commit/64322f93538515d5a3e486dc14691b9c9dcf6f66))
* **17-01:** add TrackMetadataChanged handler and remove selection gate on Track Details ([fc5cf70](https://github.com/onion-4-dinner/yellowjacket/commit/fc5cf70e4c1be3d3f1545c140db5202601a08109))
* **17-01:** add WriteTrackTagsByPath and ImageFilePicker backend methods ([4235b4a](https://github.com/onion-4-dinner/yellowjacket/commit/4235b4a4d555882ce86628a88dd4e4eeee2c9097))
* **17-02:** implement save flow, cover art editing, and error handling ([265a9ea](https://github.com/onion-4-dinner/yellowjacket/commit/265a9ea8ceba893f956a03546e9ac4189adc7716))
* **18-01:** add BatchWriteProgress event constant ([3dba0e1](https://github.com/onion-4-dinner/yellowjacket/commit/3dba0e143c091327d305d39d2fa7a687ec47e172))
* **18-01:** add BatchWriteTrackTags with progress, cancellation, and partial failure ([f557ffd](https://github.com/onion-4-dinner/yellowjacket/commit/f557ffd652179b7cf8f8ff4a06824f30edf08007))
* **18-02:** add batch edit mode to track-details component ([6dab32b](https://github.com/onion-4-dinner/yellowjacket/commit/6dab32b36b497d54e8645e969aa79737ad3523ab))
* **18-02:** wire batch track-details to all 4 view context menus ([656985a](https://github.com/onion-4-dinner/yellowjacket/commit/656985add92663440baebb871f8cd6d5723117fd))
* **19-01:** implement WAV RIFF parser/writer and writeWavTags ([e6610ff](https://github.com/onion-4-dinner/yellowjacket/commit/e6610ff15e041213b6898ad48ff63b7060b312e7))
* **20-01:** implement OGG Vorbis tag writer with custom page parser and CRC32 ([5e98c03](https://github.com/onion-4-dinner/yellowjacket/commit/5e98c036342b9e174abdc6d00db21c2e2901f18b))
* **quick-17:** create playlist-details subpage component ([dc5c7d6](https://github.com/onion-4-dinner/yellowjacket/commit/dc5c7d6ca6cfbfac15546c048f1b33aaf47209c6))
* **quick-18:** replace track-info with multi-column grid layout in playlist-details ([ce23177](https://github.com/onion-4-dinner/yellowjacket/commit/ce2317722870f932792dc6456a63235ff4611466))
<https://git.ljones.me/yonlu/yellowjacket/releases>
### Bug Fixes
Every release there is generated from the Conventional Commits it
contains, by `.gitea/workflows/release.yml` on merge to `main`. Each one
carries its notes as its body, grouped by change type, with a link to the
commit behind every line.
* **09-05:** emit VolumeChanged event and persist state in ChangeVolume and MuteToggle ([bb3fd20](https://github.com/onion-4-dinner/yellowjacket/commit/bb3fd204f0895f357a14479b40754f397aae74c4))
* **10-01:** move library_id index to migration 6 to fix existing DB startup ([75b2a34](https://github.com/onion-4-dinner/yellowjacket/commit/75b2a349ebd6fada5cbc92bfae9854cc2cd53c63))
* **12-02:** claim orphaned tracks when adding library with matching path ([f60b6b5](https://github.com/onion-4-dinner/yellowjacket/commit/f60b6b525546ef77a3329fe92f03f336b7435a0e))
* **12-02:** count failed saves as skipped so scan progress bar advances ([b36e472](https://github.com/onion-4-dinner/yellowjacket/commit/b36e472212957ff089f4f5d35f3978a754e23502))
* **12-02:** delete artist_credit_artist before artist_credit in removal pipeline ([890284d](https://github.com/onion-4-dinner/yellowjacket/commit/890284ddb1d0fb95e423bddf27b40fb0db2d11e5))
* **12-02:** dismiss inline rename on click outside ([9272b06](https://github.com/onion-4-dinner/yellowjacket/commit/9272b060bf98118e37f19a8c0834034691bfe6a2))
* **12-02:** downgrade per-file save error to Debug, add warning count to scan summary ([cf18c39](https://github.com/onion-4-dinner/yellowjacket/commit/cf18c39dbd849d60218228cf1d2285ab2071e788))
* **12-02:** invalidate library store cache on LibraryRemoved event ([b093fbb](https://github.com/onion-4-dinner/yellowjacket/commit/b093fbb10a24054c4ef62b0bd13f28d9bfe6f121))
* **12-02:** keep Add Library button visible during scan ([649e516](https://github.com/onion-4-dinner/yellowjacket/commit/649e516aa30090665e9f10e89c1ccce378e36b96))
* **12-02:** move Add Library button inline with scan buttons ([771345d](https://github.com/onion-4-dinner/yellowjacket/commit/771345dd9d3870b3a907e1cce09c7456ab7ccd85))
* **12-02:** move scan buttons above library list, default to none selected ([ba3f840](https://github.com/onion-4-dinner/yellowjacket/commit/ba3f840a28fe2c6ca40c558305814d29c233d6e0))
* **12-02:** refresh library track counts after scan completes ([1f872aa](https://github.com/onion-4-dinner/yellowjacket/commit/1f872aa005a9405d9bc1f64a4b1dd2f1f1d4a16c))
* **12-02:** reorder orphan cleanup to delete FK children before recordings ([1d735c3](https://github.com/onion-4-dinner/yellowjacket/commit/1d735c3a5f5a78996d6ddbe5c787adf040fe2f21))
* **12-02:** replace removed Scan() import with ScanAllLibraries() ([0559822](https://github.com/onion-4-dinner/yellowjacket/commit/05598224e4d5532d2e2a3a7e5d3b5411240b1024))
* **12-02:** resolve phantom tracks caused by empty library root after TOML cleanup ([717e249](https://github.com/onion-4-dinner/yellowjacket/commit/717e249c368fd1cc8d5c8f945c352175708691cf))
* **12-02:** serialize ScanWarning.Err as string instead of error interface ([ac8cbb3](https://github.com/onion-4-dinner/yellowjacket/commit/ac8cbb3296bd561a305627668c211dce7209df25))
* **12-02:** soft scan claims orphaned library_id=0 tracks on startup ([1ad099a](https://github.com/onion-4-dinner/yellowjacket/commit/1ad099a9d35fc722475e238d3443fd5473566acd))
* **12-02:** soft scan on launch — only scan libraries with changed file counts ([92c4d23](https://github.com/onion-4-dinner/yellowjacket/commit/92c4d23a9a1e545fab497816ee3dce43a181cded))
* **12-02:** wait for scan to stop before library removal, surface errors in UI ([cf00498](https://github.com/onion-4-dinner/yellowjacket/commit/cf004986c95732d00208e83467267904ea3f2ef6))
* **13-02:** auto-resolve phantom playlist tracks after library scan ([93262b9](https://github.com/onion-4-dinner/yellowjacket/commit/93262b9ae0f737d2893839ac585776207b3b44b6))
* **13-02:** defer virtualizer event delegation until element exists ([f05d2bb](https://github.com/onion-4-dinner/yellowjacket/commit/f05d2bb603f5ea827164466fd0795a6c6e662529))
* **13-02:** resolve phantom playlist tracks using M3U8 paths after scan ([9f595b7](https://github.com/onion-4-dinner/yellowjacket/commit/9f595b7ac10c2191b5469004901cbbc1331c1abb))
* **14-01:** downgrade main-panel from contain:strict to layout+style+paint ([4b7d35d](https://github.com/onion-4-dinner/yellowjacket/commit/4b7d35d7ec4c8b14453a8f8250cd154b8c4c2537))
* **14-perf:** fix scroll jumping and input latency ([3b2e189](https://github.com/onion-4-dinner/yellowjacket/commit/3b2e189e7d0e6d00393d087565190fd307774257))
* **17-02:** fix cover art replace and remove ([d7c2965](https://github.com/onion-4-dinner/yellowjacket/commit/d7c2965752ae0ac9009d00f2431d5919a24558b7))
* **17-02:** handle float64 numeric values from Wails JSON deserialization ([900db2e](https://github.com/onion-4-dinner/yellowjacket/commit/900db2e56cca254873a3a5a7a384008feac4211b))
* **17-02:** refresh cover art URLs after save ([8cd4914](https://github.com/onion-4-dinner/yellowjacket/commit/8cd4914842f61c0c6b49e0216c7816e201a3c94a))
* **17-02:** refresh track-details dialog data after successful save ([ffcdc41](https://github.com/onion-4-dinner/yellowjacket/commit/ffcdc41b0d4fad8ed428dbaa55f6cdd38c096822))
* **18-02:** add field labels above title/artist/album inputs in batch edit mode ([9df2d67](https://github.com/onion-4-dinner/yellowjacket/commit/9df2d6764a0b0566dda33cff675debea4a61dea8))
* **18-02:** add field labels to all track-details states (single/batch, read/edit) ([d430ad8](https://github.com/onion-4-dinner/yellowjacket/commit/d430ad884bfd38bea93389d8be730ff00388a7be))
* **19-01:** add album_artist TPE2 mapping to applyTextChanges ([8f4c4a0](https://github.com/onion-4-dinner/yellowjacket/commit/8f4c4a0c2b14eeeaeccb972a40addb11f3d65437))
* preserve scroll position in cached grid views ([54df917](https://github.com/onion-4-dinner/yellowjacket/commit/54df917ffdd69c4f7ffaeccf2d161261ca80d84e))
* **queue-panel:** set flow layout _itemSize to match actual track item height ([288d9de](https://github.com/onion-4-dinner/yellowjacket/commit/288d9deae22d437fcd7857b368827db7b62c24f6))
* **queue-panel:** suppress virtualizer scroll corrections during scrollbar drag ([0bd8cef](https://github.com/onion-4-dinner/yellowjacket/commit/0bd8cefa00dcae2f8bd9579de2aefd58e0a9e6c9))
* **quick-19:** multi-root path resolution for playlist M3U8 tracks ([9144ded](https://github.com/onion-4-dinner/yellowjacket/commit/9144dedc2742925dc252d491763b4f2929238d0e))
* **S21/T01:** fix all lint warnings and upgrade wsl to wsl_v5 ([f16157a](https://github.com/onion-4-dinner/yellowjacket/commit/f16157a2134cbeb1787ff851d4875d77f2f3f86b))
**This file is not generated and is not a copy of that.** `main` is a
protected branch, so nothing pushes a changelog commit back to it — and a
file that claimed to be a changelog while silently never updating would
be worse than no file at all. `make release-dry` prints what the next
merge would release.
### Performance
* **12-02:** increase scan batch size from 50 to 300 ([21ea71e](https://github.com/onion-4-dinner/yellowjacket/commit/21ea71e2575d76258bd81d89ab8ac883aa3bed36))
* **12-02:** skip FTS5 rebuild during library removal ([30f4461](https://github.com/onion-4-dinner/yellowjacket/commit/30f4461e6957e20d3dc607fa0886a75b5c21b3cf))
* **14-01:** add CSS containment to app shell layout boundaries ([efa06f7](https://github.com/onion-4-dinner/yellowjacket/commit/efa06f7edf1e4acdc3d8865cad264403257ae40d))
* **14-01:** add GPU promotion and containment to all scroll containers ([ac8a52e](https://github.com/onion-4-dinner/yellowjacket/commit/ac8a52e110f9f8ebdc3433b60594370352126a18))
* **14-02:** replace innerHTML navigation with view caching system ([ad91043](https://github.com/onion-4-dinner/yellowjacket/commit/ad9104374a628342e0ea30cf409ff43de2c2f86e))
* **14-03:** add notification batching to queue store and granular change tracking to library store ([d0c05dc](https://github.com/onion-4-dinner/yellowjacket/commit/d0c05dc1d43a4fe12cc07f3cff25375b08a74ba0))
* **14-03:** eliminate per-item closure allocation in scroll render paths ([2f7ed70](https://github.com/onion-4-dinner/yellowjacket/commit/2f7ed7030425ed0ebb7a1a186917a79a7b26b850))
* **14-04:** RAF-throttle scroll position saves and add overflow-anchor to queue panel ([6ca0b3c](https://github.com/onion-4-dinner/yellowjacket/commit/6ca0b3c5a84769af064ebe45a6eaac014d1a270a))
* auto-detect NVIDIA+Wayland for DMABuf workaround ([915591a](https://github.com/onion-4-dinner/yellowjacket/commit/915591aea962beb60da2e96ac0f57307f646f675))
* inline SVGs, memoize grid slices, batch store notifications ([a4eac39](https://github.com/onion-4-dinner/yellowjacket/commit/a4eac394cebefd29d0ebcb4b1e331444dcb8fbaf))
* reduce software rendering overhead for NVIDIA+Wayland ([199c910](https://github.com/onion-4-dinner/yellowjacket/commit/199c91013fd806f6aefce49357df8a32b46faaa0))
### Refactoring
* **quick-17:** simplify playlist-view to navigate instead of expand ([955cd68](https://github.com/onion-4-dinner/yellowjacket/commit/955cd68be2dbf7a9071ef1c93084d687b59b6bd7))
## [1.2.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.2.1...v1.2.2) (2026-03-06)
### Bug Fixes
* recover from go-mp3 seek panic on startup ([#86](https://github.com/onion-4-dinner/yellowjacket/issues/86)) ([2f9d9f8](https://github.com/onion-4-dinner/yellowjacket/commit/2f9d9f8508b90b6188fe894c282c5b8e330e8046))
## [1.2.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.2.0...v1.2.1) (2026-03-06)
### Bug Fixes
* **deps:** pin go-webview2 to v1.0.21 for Wails v2 compat ([25f0fe8](https://github.com/onion-4-dinner/yellowjacket/commit/25f0fe81560eeff36a0b2beb52ce1bdf13d5e122))
## [1.2.0](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.3...v1.2.0) (2026-03-06)
### Features
* **02-02:** add ScanWarning type and reclassify scan errors as warnings ([e6866de](https://github.com/onion-4-dinner/yellowjacket/commit/e6866ded9dc0ea30ff942cd31b6c5ea3269e9584))
* **03-01:** create NewTestDB helper for in-memory SQLite test databases ([bae9d70](https://github.com/onion-4-dinner/yellowjacket/commit/bae9d70d23157ef4e79e60dd713d9a02ab63790b))
* **03-01:** extract shared applyPRAGMAs and add production PRAGMAs to NewDB ([d348815](https://github.com/onion-4-dinner/yellowjacket/commit/d34881530adda7fb75be84737798da46d17bfa8c))
* **06-01:** create track_metadata VIEW schema and migration 4 ([9c7e5a9](https://github.com/onion-4-dinner/yellowjacket/commit/9c7e5a96344a81bf132de487b4763f1dc3ff6df9))
* **06-02:** create Go→TypeScript event constant codegen tool ([3e9edd0](https://github.com/onion-4-dinner/yellowjacket/commit/3e9edd05e87395499ac24e456640d1f6d9b97f04))
* **06-03:** migrate lookupChunk to sqlc-generated LookupTrackMetaByPaths query ([2221a68](https://github.com/onion-4-dinner/yellowjacket/commit/2221a68459850a837c996c6e6d2bc95d41b20fb3))
* **08-01:** define design token CSS custom properties for icon sizes and type scale ([1444a66](https://github.com/onion-4-dinner/yellowjacket/commit/1444a66bb201ce5fdf16552a32bcd281089c64ed))
* **08-04:** apply design tokens to cover-grid, track-list, queue-panel, and detail components ([1303422](https://github.com/onion-4-dinner/yellowjacket/commit/1303422e69c27d528363900b3ca5287a48cc9f8e))
* **08-04:** convert sidebar em-based spacing to px and apply icon/type tokens ([aed90d7](https://github.com/onion-4-dinner/yellowjacket/commit/aed90d7b1710d0c5cece2e4956c0a6ce77b9a999))
* add scan progress bar with phase indicator ([a28b4d1](https://github.com/onion-4-dinner/yellowjacket/commit/a28b4d1e0673658824750d4c702359321dc9a78e))
* **quick-001:** add multi-file picker and batch import support ([c34e4ad](https://github.com/onion-4-dinner/yellowjacket/commit/c34e4ad029c119bff8f70a07ccc6bca58b11ea3c))
* **quick-001:** regenerate bindings and update frontend for multi-import ([2a542bf](https://github.com/onion-4-dinner/yellowjacket/commit/2a542bf3bcdc7772edb1aceb41f488774494f656))
* **quick-002:** add CountPlaylistsByName SQL query and regenerate sqlc ([04b2088](https://github.com/onion-4-dinner/yellowjacket/commit/04b2088b28b84a4d4df25b23d97112c5a955dff1))
* **quick-002:** add uniquePlaylistName helper and wire into ImportPlaylist ([8ba8bbe](https://github.com/onion-4-dinner/yellowjacket/commit/8ba8bbe7bed2ecff97613ebaa42a49a662050353))
* **quick-006:** remove list icon from playlists, add favorites icon to default ([3c19766](https://github.com/onion-4-dinner/yellowjacket/commit/3c19766fd0885d4171cf9929db6d69a3d5c1a3ff))
* **quick-11:** add configurable log level via YJ_LOG_LEVEL env var ([55b4902](https://github.com/onion-4-dinner/yellowjacket/commit/55b4902fac7b7f2c04ad5efac398ecedc5fedc2f))
* **quick-11:** add make dev-debug target for verbose logging ([c45bca4](https://github.com/onion-4-dinner/yellowjacket/commit/c45bca411ba1d4f32deea6027acf91237173dd15))
* **quick-12:** add favorite icon to album dropdown track rows ([12a0bbc](https://github.com/onion-4-dinner/yellowjacket/commit/12a0bbc89c19128485d597a61bd16bd0786450ad))
* **quick-15:** add BufferedStreamer with goroutine read-ahead ([85b23ac](https://github.com/onion-4-dinner/yellowjacket/commit/85b23acb24a048d2f7b85808e477bb991ae124e6))
* **quick-15:** insert BufferedStreamer into player pipeline and increase speaker buffer ([8a0b16a](https://github.com/onion-4-dinner/yellowjacket/commit/8a0b16a4ec08a95bfd3834c8216e21dce854432d))
* **quick-3:** add playlist-level multi-select state and selection handling ([e13151f](https://github.com/onion-4-dinner/yellowjacket/commit/e13151ffa5dc86e41ce242421679d65a740c3af0))
* **quick-3:** wire playlist context menu for batch delete of selected playlists ([c92ced2](https://github.com/onion-4-dinner/yellowjacket/commit/c92ced2c74e72bfc123c880c047462dc969cde34))
* **quick-4:** add 'Set as Default Playlist' context menu option ([9971b63](https://github.com/onion-4-dinner/yellowjacket/commit/9971b635b81fe3f8621c80a6664eccb3e1fc4bb8))
* **quick-5:** add CreatedAt/UpdatedAt to playlist Summary struct ([bdaff47](https://github.com/onion-4-dinner/yellowjacket/commit/bdaff478e802ee5c0745327c52dd9b190fcfef7d))
* **quick-5:** add sort dropdown UI and client-side sorting to playlist view ([5c07485](https://github.com/onion-4-dinner/yellowjacket/commit/5c074855351f1363cc7918837a78bbd3c0b7ebf5))
* **quick-7:** add PinDefault config field with backend getter/setter ([6e123bd](https://github.com/onion-4-dinner/yellowjacket/commit/6e123bd47f55e6d565f20bf7f19950e65f80787f))
* **quick-7:** wire frontend pin-default-playlist feature end-to-end ([e6378e1](https://github.com/onion-4-dinner/yellowjacket/commit/e6378e1f0d3b0f2a7604b8ef6097dba9050cdd16))
* **quick-8:** add FindDuplicateTracksInPlaylist backend method ([83de934](https://github.com/onion-4-dinner/yellowjacket/commit/83de934c39ca7d850a8b5925c90e6d0b3fe0a487))
* **quick-8:** create duplicate-tracks-dialog component ([9f3ba2b](https://github.com/onion-4-dinner/yellowjacket/commit/9f3ba2b9d474fa30dcb4934b01d4650e0d0d3cba))
* **quick-8:** wire duplicate detection into playlist-picker and playlist-view ([917a79a](https://github.com/onion-4-dinner/yellowjacket/commit/917a79a8d6e30dddd2170323bb26692386794872))
### Bug Fixes
* **01-01:** add mutex protection to Queue, Library, and Playlist SetContext methods ([daaa6b7](https://github.com/onion-4-dinner/yellowjacket/commit/daaa6b7f9779385979fe9dddae4e7bb388b3e5fb))
* **01-01:** collapse Player.SetContext double-lock into single acquisition ([3abaeba](https://github.com/onion-4-dinner/yellowjacket/commit/3abaeba3afb0f4d0edb81e26ca55b31bf59990ac))
* **02-01:** eliminate package-level startupErr and fix config file permissions ([2a86408](https://github.com/onion-4-dinner/yellowjacket/commit/2a864082017e489ffa086c136f1002277a77a7c4))
* **02-01:** log MPRIS callback errors instead of discarding them ([0860b2f](https://github.com/onion-4-dinner/yellowjacket/commit/0860b2fd4b2250da1eeb80c21f14fdf341697501))
* **08-02:** revert repeat() inside lit-virtualizer, restore .renderItem + .keyFunction ([72ef719](https://github.com/onion-4-dinner/yellowjacket/commit/72ef719ba70eeca0fa4bae47df092706f6fbaeed))
* drop+recreate contentless FTS5 index instead of DELETE ([8e9a616](https://github.com/onion-4-dinner/yellowjacket/commit/8e9a61603779eacbee7013b9bc760b315baf782a))
* **frontend:** reposition search indicator into toolbar and fix album cover art lookup ([a29137b](https://github.com/onion-4-dinner/yellowjacket/commit/a29137b2ba4c6b33ce9a5f868cbd6013e0e3b116))
* include full track metadata in GetAudioFilesByReleaseGroup query ([97f256d](https://github.com/onion-4-dinner/yellowjacket/commit/97f256d67f463d752f7adc5b400c4bf34eae1df1))
* **quick-10:** add migration 5 and fix entity cache for composite album key ([d43ba7b](https://github.com/onion-4-dinner/yellowjacket/commit/d43ba7bd0c7ace2a9ed71990a19498f8e9f90751))
* **quick-10:** update release_groups schema and queries for composite uniqueness ([999ab96](https://github.com/onion-4-dinner/yellowjacket/commit/999ab967beb9107a3f30ba287acbffad22f0b0de))
* **quick-13:** resolve lint issues in main source files ([e1a95e6](https://github.com/onion-4-dinner/yellowjacket/commit/e1a95e65a9f0f436b2e2d92befa9c881b6e8e430))
* **quick-14:** add roll-back-on-failure to queue index advancement ([2820de2](https://github.com/onion-4-dinner/yellowjacket/commit/2820de2510560fcd6d1015c18542d5ac30468247))
* **quick-9:** set fixed height on queue track items for stable virtualizer scroll ([ebde5e5](https://github.com/onion-4-dinner/yellowjacket/commit/ebde5e5a8bc4da8f40bef8f171c7ed86c213a336))
### Performance
* **07-01:** add incremental persistence helpers for queue mutations ([cdd17db](https://github.com/onion-4-dinner/yellowjacket/commit/cdd17db27509908514c21517631306655a2b3bd7))
* **07-01:** eliminate redundant lookups in SetQueue Phase 2 ([ced58fe](https://github.com/onion-4-dinner/yellowjacket/commit/ced58fe6a93d6f220137562b8ff09ffc33c69266))
* **07-02:** defer eagerFetch to after DOM ready for instant app shell ([cd98ad6](https://github.com/onion-4-dinner/yellowjacket/commit/cd98ad6dc8c2e4e6e0f01a48099b0c0511bf5a98))
* **08-01:** add queueMicrotask coalescing to library store and debounce search input ([3bf66ed](https://github.com/onion-4-dinner/yellowjacket/commit/3bf66ed125ed55bfbde95b0bc973710c2f2243b8))
* **08-02:** migrate cover-grid, artists-view, and genres-view virtualizers to repeat() directive ([1c3514d](https://github.com/onion-4-dinner/yellowjacket/commit/1c3514da1d0491b9758d7a6f9f72d59ef78fc8ed))
* **08-02:** migrate track-list and queue-panel virtualizers to repeat() directive ([d2d7d8c](https://github.com/onion-4-dinner/yellowjacket/commit/d2d7d8c6ce22923772cae4858b02804d15f74bb7))
* **08-03:** optimize column rendering and apply classMap to queue-panel renderTrackItem ([62f41c2](https://github.com/onion-4-dinner/yellowjacket/commit/62f41c24910632b270f9f5765e20e48db4b95ec9))
* **08-03:** replace class string construction with classMap directive in renderTrackRow ([ad21027](https://github.com/onion-4-dinner/yellowjacket/commit/ad210278fc20729dc76390e6bba9bff050549046))
### Refactoring
* **06-01:** consolidate search queries to use track_metadata VIEW ([9159b40](https://github.com/onion-4-dinner/yellowjacket/commit/9159b409dcd2afaa7dcc97bf5b0694edf85f06a4))
* **quick-14:** make playOrLoadCurrentTrack and playCurrentTrack return bool ([6eeddda](https://github.com/onion-4-dinner/yellowjacket/commit/6eeddda97669258cc5b7ba175a3c98d598a2871f))
## [1.1.3](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.2...v1.1.3) (2026-02-21)
### Bug Fixes
* add typescript as explicit devDependency and auto-install frontend deps in setup ([#70](https://github.com/onion-4-dinner/yellowjacket/issues/70)) ([7316587](https://github.com/onion-4-dinner/yellowjacket/commit/73165877fa79656ab9bc6f60bd8e9e52d6be206c))
* use local tsc binary in pre-commit hook to avoid PATH issues ([#71](https://github.com/onion-4-dinner/yellowjacket/issues/71)) ([6079e55](https://github.com/onion-4-dinner/yellowjacket/commit/6079e558ff913d38c7f1c4aeb52cc09474c4ed20))
## [1.1.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.1...v1.1.2) (2026-02-15)
### Bug Fixes
* r2 upload ([#69](https://github.com/onion-4-dinner/yellowjacket/issues/69)) ([0252466](https://github.com/onion-4-dinner/yellowjacket/commit/0252466f615b4e2fd9694790c6d311a9eac1ccf2))
## [1.1.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.1.0...v1.1.1) (2026-02-15)
### Bug Fixes
* **ci:** remove build-check job from CI workflow ([#66](https://github.com/onion-4-dinner/yellowjacket/issues/66)) ([42d3f45](https://github.com/onion-4-dinner/yellowjacket/commit/42d3f45d85afa694e9545997af3ff4ac814ad021))
## [1.1.0](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.3...v1.1.0) (2026-02-15)
### Features
* **ci:** upload release artifacts to Cloudflare R2 ([#65](https://github.com/onion-4-dinner/yellowjacket/issues/65)) ([8985084](https://github.com/onion-4-dinner/yellowjacket/commit/89850848cbf7783e5c85348ff18f7cd11d60231a))
## [1.0.3](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.2...v1.0.3) (2026-02-15)
### ⚠ BREAKING CHANGES
* **deps:** update module github.com/evilmartians/lefthook to v2 (#61)
* **deps:** update actions/checkout action to v6 (#45)
* **deps:** update dependency vite to v7 (#53)
### Bug Fixes
* resolve all lint errors and make linting a required CI check ([#62](https://github.com/onion-4-dinner/yellowjacket/issues/62)) ([30b2480](https://github.com/onion-4-dinner/yellowjacket/commit/30b2480df49f57878b0e8c923da6ad8d6fe99416))
* virtual list and cover grid ([#63](https://github.com/onion-4-dinner/yellowjacket/issues/63)) ([7579a76](https://github.com/onion-4-dinner/yellowjacket/commit/7579a768be84225ed46db4e7a90781f3e30e2953))
### Miscellaneous
* **deps:** update actions/checkout action to v6 ([#45](https://github.com/onion-4-dinner/yellowjacket/issues/45)) ([2d6e221](https://github.com/onion-4-dinner/yellowjacket/commit/2d6e22105d2daed1dc5b586c0442e2941949a165))
* **deps:** update dependency vite to v7 ([#53](https://github.com/onion-4-dinner/yellowjacket/issues/53)) ([f0006c4](https://github.com/onion-4-dinner/yellowjacket/commit/f0006c4c4335b60b58cccdd29de4792965e39694))
* **deps:** update module github.com/evilmartians/lefthook to v2 ([#61](https://github.com/onion-4-dinner/yellowjacket/issues/61)) ([e32b217](https://github.com/onion-4-dinner/yellowjacket/commit/e32b2179129ae7f26037697a125710ff7587566d))
## [1.0.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.1...v1.0.2) (2026-02-14)
### ⚠ BREAKING CHANGES
* **deps:** update actions/setup-node action to v6 (#48)
* **deps:** update dependency stylelint-config-standard to v40 (#52)
* **deps:** update dependency node to v24 (#51)
* **deps:** update dependency vite-plugin-static-copy to v3 (#54)
* **deps:** update golangci/golangci-lint-action action to v9 (#55)
* **deps:** update amannn/action-semantic-pull-request action to v6 (#50)
* **deps:** update actions/upload-artifact action to v6 (#49)
* **deps:** update actions/setup-go action to v6 (#47)
* **deps:** update actions/download-artifact action to v7 (#46)
### Bug Fixes
* **ci:** use allowedPostUpgradeCommands for Renovate post-upgrade tasks ([#60](https://github.com/onion-4-dinner/yellowjacket/issues/60)) ([0aef483](https://github.com/onion-4-dinner/yellowjacket/commit/0aef483b3cccd0616fd5be2d06d0856b46851d09))
### Miscellaneous
* **deps:** update actions/download-artifact action to v7 ([#46](https://github.com/onion-4-dinner/yellowjacket/issues/46)) ([1910f99](https://github.com/onion-4-dinner/yellowjacket/commit/1910f99cf64e9bdc5ce91e89cab254ecca15d030))
* **deps:** update actions/setup-go action to v6 ([#47](https://github.com/onion-4-dinner/yellowjacket/issues/47)) ([8911fb2](https://github.com/onion-4-dinner/yellowjacket/commit/8911fb2400047cf2f3dfa719edc1d1bf474cdaa5))
* **deps:** update actions/setup-node action to v6 ([#48](https://github.com/onion-4-dinner/yellowjacket/issues/48)) ([d7382fd](https://github.com/onion-4-dinner/yellowjacket/commit/d7382fd8444b6618dbfe991f5f97231528a07f13))
* **deps:** update actions/upload-artifact action to v6 ([#49](https://github.com/onion-4-dinner/yellowjacket/issues/49)) ([a2c644b](https://github.com/onion-4-dinner/yellowjacket/commit/a2c644b00eed83acc0ed38a2eb8c73868b7b79af))
* **deps:** update amannn/action-semantic-pull-request action to v6 ([#50](https://github.com/onion-4-dinner/yellowjacket/issues/50)) ([643ba27](https://github.com/onion-4-dinner/yellowjacket/commit/643ba27f066164aeb47e8d9aaf20fe98b9b69d30))
* **deps:** update dependency node to v24 ([#51](https://github.com/onion-4-dinner/yellowjacket/issues/51)) ([e7d3971](https://github.com/onion-4-dinner/yellowjacket/commit/e7d39711078ce86b0c029f0d03ff81162c5dc28a))
* **deps:** update dependency stylelint-config-standard to v40 ([#52](https://github.com/onion-4-dinner/yellowjacket/issues/52)) ([422aabc](https://github.com/onion-4-dinner/yellowjacket/commit/422aabcc07e9700ff189302b363e13d87c69163a))
* **deps:** update dependency vite-plugin-static-copy to v3 ([#54](https://github.com/onion-4-dinner/yellowjacket/issues/54)) ([77fa643](https://github.com/onion-4-dinner/yellowjacket/commit/77fa6435a5298f58ef83607d99c59b876132c66c))
* **deps:** update golangci/golangci-lint-action action to v9 ([#55](https://github.com/onion-4-dinner/yellowjacket/issues/55)) ([aedb7d1](https://github.com/onion-4-dinner/yellowjacket/commit/aedb7d1e6d204c56c468dd26b340752fd6bfeaeb))
## [1.0.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.0...v1.0.1) (2026-02-14)
### Bug Fixes
* resolve Renovate repo detection and pre-push hook hang ([#36](https://github.com/onion-4-dinner/yellowjacket/issues/36)) ([b205889](https://github.com/onion-4-dinner/yellowjacket/commit/b205889128f01e9eb75b607cf7c4034887cda3f4))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
### Bug Fixes
* allow library to initialize without config and fix lefthook lint flag ([5a958db](https://github.com/onion-4-dinner/yellowjacket/commit/5a958db16284a74e19c43259757b163b347cda7d))
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
* rename downloaded artifacts to platform-specific names for release ([e3bda0e](https://github.com/onion-4-dinner/yellowjacket/commit/e3bda0e2fc7700fad382cabe00aeb46f91fbb0a0))
* resolve frontend build failures in CI ([330a53c](https://github.com/onion-4-dinner/yellowjacket/commit/330a53c9f4b1292840ad0f75479b76b3d429c954))
* trigger build workflow from release event instead of tag push ([47772f7](https://github.com/onion-4-dinner/yellowjacket/commit/47772f73cc04093c55414bf20ebe2ef442418d19))
* use path.Join for embed.FS paths to fix Windows build ([672fe24](https://github.com/onion-4-dinner/yellowjacket/commit/672fe24ee99debf4a394fff7eec55f17b0e44476))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
### Bug Fixes
* allow library to initialize without config and fix lefthook lint flag ([5a958db](https://github.com/onion-4-dinner/yellowjacket/commit/5a958db16284a74e19c43259757b163b347cda7d))
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
* resolve frontend build failures in CI ([330a53c](https://github.com/onion-4-dinner/yellowjacket/commit/330a53c9f4b1292840ad0f75479b76b3d429c954))
* trigger build workflow from release event instead of tag push ([47772f7](https://github.com/onion-4-dinner/yellowjacket/commit/47772f73cc04093c55414bf20ebe2ef442418d19))
* use path.Join for embed.FS paths to fix Windows build ([672fe24](https://github.com/onion-4-dinner/yellowjacket/commit/672fe24ee99debf4a394fff7eec55f17b0e44476))
## [1.0.3](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.2...v1.0.3) (2026-02-14)
### Bug Fixes
* use path.Join for embed.FS paths to fix Windows build ([672fe24](https://github.com/onion-4-dinner/yellowjacket/commit/672fe24ee99debf4a394fff7eec55f17b0e44476))
## [1.0.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.1...v1.0.2) (2026-02-14)
### Bug Fixes
* resolve frontend build failures in CI ([330a53c](https://github.com/onion-4-dinner/yellowjacket/commit/330a53c9f4b1292840ad0f75479b76b3d429c954))
## [1.0.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.0...v1.0.1) (2026-02-14)
### Bug Fixes
* allow library to initialize without config and fix lefthook lint flag ([5a958db](https://github.com/onion-4-dinner/yellowjacket/commit/5a958db16284a74e19c43259757b163b347cda7d))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
### Bug Fixes
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
## [1.0.2](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.1...v1.0.2) (2026-02-14)
### Bug Fixes
* **ci:** fix golangci-lint version, skip player test in CI, remove standalone frontend build ([7317e09](https://github.com/onion-4-dinner/yellowjacket/commit/7317e093a7f92651ab65b2f83381d02105bdc0df))
## [1.0.1](https://github.com/onion-4-dinner/yellowjacket/compare/v1.0.0...v1.0.1) (2026-02-14)
### Bug Fixes
* **ci:** resolve CI failures for Go checks, codegen, and frontend type-checking ([d4f9361](https://github.com/onion-4-dinner/yellowjacket/commit/d4f936143ac75fbf3247cdbe2113bd89b0795d83))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
### Bug Fixes
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
### Bug Fixes
* **ci:** configure git credentials explicitly for semantic-release PAT ([24f21af](https://github.com/onion-4-dinner/yellowjacket/commit/24f21af8350227e77fc1fef9243c238e6417aca0))
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
### Bug Fixes
* **ci:** use PAT for semantic-release to trigger build workflow ([68d41c0](https://github.com/onion-4-dinner/yellowjacket/commit/68d41c0ff22fede57acab7a2bfed42df7814bb90))
## 1.0.0 (2026-02-14)
### Features
* **ci:** add semantic-release pipeline, cross-platform builds, and lefthook git hooks ([caf3e84](https://github.com/onion-4-dinner/yellowjacket/commit/caf3e843af7da37e05da36da5c41b6dc3c53ded1))
History before `v0.0.1` is in `git log`. The versions before it were cut
by hand and are not on the releases page; the entries this file used to
hold were generated against a GitHub remote this project no longer has,
and every link in them was dead.
+339 -18
View File
@@ -6,16 +6,121 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
YellowJacket is a cross-platform desktop music player built with Go (backend) and TypeScript/Lit (frontend), using the Wails framework to bridge them. It supports MP3, FLAC, OGG Vorbis, and WAV playback.
## Issues
**The tracker is the source of truth for what is wanted and what is
already being worked on**, and it is shared with a collaborator who
cannot see this session. `scripts/issue.sh` is the whole interface to
it (`list`, `mine`, `search`, `show`, `new`, `claim`, `unclaim`,
`comment`, `close`, `label`, `depends`, `labels`); it needs a
`GITEA_TOKEN` with `write:issue`.
**Search the tracker before starting any work, and claim what you
find.** Fifty-odd issues make that a real lookup rather than a
formality. `./scripts/issue.sh search <terms>` covers open and closed —
closed matters, because "that was fixed three weeks ago" is the
cheapest possible answer.
**Claiming happens before the first edit, not before the commit.** The
whole point is that the collaborator can see the work is taken *while
it is being done*, so `claim` sets the assignee, applies
`Status/In Progress` and posts a comment naming the branch and the
approach — all three, or none. It refuses outright if somebody else
already holds it, and that refusal is the feature: talk to them rather
than working around it.
**If no issue covers the work, open one first.** The issue exists
before the branch does. That is what makes the tracker a description
of the project rather than a description of the past.
**Findings get filed.** A bug tripped over while doing something else
is an issue with a reproduction, not a sentence in a chat message
nobody can search. So is a piece of work deliberately not done — the
issue is where "we decided not to, and here is why" survives.
Four conventions are already established and are not up for
reinvention:
- **The labels are a taxonomy**, not tags: `Kind/*`, `Area/*`,
`Priority/*`, `Platform/*`, plus `Reviewed/Confirmed` (the code was
read and the defect confirmed) and the `Status/*` family. `Status/*`
and `Reviewed/*` are **exclusive scopes** — one of each at most, so
applying a second replaces the first.
- **#73 is the roadmap.** It states the order the backlog should be
worked in and the soft relations that are not expressible as
blockers. Picking work off the open list by eye when a meta issue
states the sequence is how the sequence stops meaning anything.
- **Hard blockers are real Gitea dependencies**, which render on the
issue itself, and the blocked issue carries `Status/Blocked`.
- **A PR body carries a commit-to-issue table, the verification
actually run, and a `Closes` list** — PR #83 is the shape. That list
is for whoever reads the PR; what actually closes an issue is the
footer below.
**The closing keyword goes in the commit body, one issue per line.**
```
docs: delete four documents that contradict the code
<body>
Closes #98
```
**Gitea parses commit messages that reach `main`; it does not parse the
PR body**, which only closes anything if the merge happens to copy it
into the merge commit. Both halves of that were measured. #83's merge
commit carried `Closes #9, #13, #14, …` and closed **five of ten** — a
comma list is partially matched. #93's merge commit body was one
`Reviewed-on:` trailer, so #92 stayed open behind a perfectly correct
`Closes` line in the PR description.
A footer costs nothing elsewhere: Conventional Commits allows one,
`scripts/commit-check.sh` only regexes the subject, and
semantic-release reads the type from the subject — so this changes no
release decision. The rule that the issue number stays out of the
**subject** is unaffected, and was never about the body.
**Check it anyway.** A squash, or a merge message edited by hand,
still drops the footer. `./scripts/issue.sh list --state open` after a
merge, looking for what you just shipped; `./scripts/issue.sh close
<n>` for whatever did not take, with a comment naming the commit.
**Unclaiming is automatic, and it is hooked to the close rather than
to the merge.** Gitea's auto-close changes state and nothing else, so a
footer left `Status/In Progress` on a closed issue — #100 was closed
and marked as being actively worked on at the same time.
`.gitea/workflows/unclaim.yml` runs on `issues: [closed]`, which covers
the footer, `issue.sh close` and a click in the web UI alike; stripping
the label in the PR instead would have been a per-PR habit, and habits
are what the footer removed. It is not instant — the runner has
capacity 1 — and reopening deliberately does not restore the label.
`./scripts/issue.sh list --state closed --label "Status/In Progress"`
is how you find out it has stopped firing.
## Planning
Active and historical plans live in `.planning/`:
`.planning/` is **design documents and measured history**, not a queue
— the queue is the tracker, and a plan file that describes work nobody
has started is a second, staler answer to "what are we doing next".
- `.planning/NOTES.md` — gotchas, deferred items, open architecture questions, the "we already considered and rejected" list.
- `.planning/plans/active/` — work currently in progress (read first).
- `.planning/plans/pending/` — sequenced future work.
- `.planning/plans/completed/` — one concise recap per shipped milestone.
- `.planning/NOTES.md` — gotchas, measured facts, open architecture
questions, and the "we already considered and rejected" list. Dated,
because several are properties of someone else's server. **This is
where a decision reached on an issue gets written down** when it
outlives the issue.
- `.planning/plans/completed/` — one recap per shipped milestone, kept
for the arguments in it. Where a plan shipped incompletely, its
header says which issue carries the remainder.
- `.planning/audits/` — the read-only audits that produced the
reconciliation plans. Historical evidence; not a backlog.
- `.planning/plans/active/` — a multi-phase design document for work
**in flight**, linked from the issue that tracks it. Empty is the
normal state. There is no `pending/`: a plan nobody is executing is
an issue.
Numbering is sequential and stable across status moves (a plan keeps its `NNN-` prefix as it migrates between `pending → active → completed`). Abandoned plans are deleted; paused work stays in `pending/`.
Numbering is sequential and stable across status moves (a plan keeps
its `NNN-` prefix). Abandoned plans are deleted.
## Commands
@@ -233,6 +338,74 @@ rather than renaming them.
the drift it caused before — `sql/schemas/` and the migrations
disagreed, and sqlc generated against the stale one.
**What that costs an existing database is repaired once, at open.**
`CREATE ... IF NOT EXISTS` reaches an existing table only if its shape
already matches and otherwise silently no-ops, so a *changed* table
never migrates. Plan 014 added `total_tracks` to `explore_index` and
to `indexRowFields` — the projection every explore read uses — and no
database that already existed grew the column: **every** Explore
search, browse, artist and album page on such an install failed with
`no such column: total_tracks`, while a fresh install was perfectly
healthy, which is exactly why no test saw it. Plan 013 was worse on
the same install: `applySchema` could not be applied at all over a
pre-013 `audio_files`, so the app did not open.
`backend/database/staleshape.go` runs before `applySchema` and
retires what is stale, so the create is a create. Five things about
it are load-bearing:
- **It parses `sql/schemas/` for the expectation** rather than
writing the column list down a second time, because a second list
is a second thing to forget — the fault it exists to repair.
- **It notices a changed *type*, not just a missing column.** 013
moved `mbid` from TEXT to BLOB, and SQLite does not coerce between
them: a comparison against 16 raw bytes returns no rows rather than
an error. `ALTER TABLE ADD COLUMN` would have handled
`total_tracks` alone and cannot express this at all, which is why
the repair drops rather than migrates.
- **`Authored` is never retired**, and that boundary is a test
(`TestAuthoredTablesAreNeverRetired`), not a comment. Everything
else is rebuildable: `Cache` by definition, `Owned` by a rescan —
plan 013's stated "delete and rescan" — and `Derived` from Owned.
A table the schema no longer describes at all goes too; 013 left
seven behind plus `schema_migrations`.
- **Whether a stale `Cache` table may be rebuilt is a build tag**, and
it is the most expensive thing in this file to get wrong. In the app
the catalog is *downloaded*, so a wrong shape costs a minute of
re-fetching the artifact and keeping it costs every Explore read. In
`cmd/indexbuild` the catalog is *derived*, and the only way back is
the ~205 GB dump stream the `/cache` volume exists to avoid — so
`retireStaleCache` is false there (`staleshape_policy_indexbuild.go`)
and `TestTheCatalogSurvivesAStaleShape` fails the moment it is not.
`TestNoCacheTableIsRetiredHere` is the same assertion made of *every*
`datamap` Cache table rather than one, because the risk is not that
shape recurring — it is the next destructive repair added to
`database.NewDB`, the chokepoint every binary here shares, without
asking which binary it is in.
This is written down because it already happened: the repair shipped
without the distinction and dropped the real CI catalog on its first
run, with `reason="column entity_type is TEXT, schema declares
INTEGER"`. The mismatch was genuine — that database is deliberately
kept in the older encoding, which `fix(indexexport): read an index
older than the binary` exists to tolerate — so it would have been
dropped on *every* run. The consequence is that a future
`explore_index` column fails the index job loudly on `applySchema`
rather than silently costing it a rebuild, which is the trade a
human should get to make.
- **The drops are one transaction with `defer_foreign_keys`.** Those
legacy tables reference each other, so dropping them in any order
fails on whichever goes first, and turning foreign keys *off*
instead would silently take `playlist_tracks.audio_file_id`'s
ON DELETE SET NULL with it — leaving playlist entries pointing at
ids a rescan reissues to *different songs*. Nulled entries are
empty; stale ones are wrong, and wrong quietly.
- **The order is sorted, so a failure reproduces.** Map order is
random, and the foreign-key bug above passed its own regression
test on two runs in three until the order was fixed.
Retiring `explore_index` takes its FTS and its meta with it, because
the `dump_import_done` marker is what would otherwise stop the
artifact ever being fetched again.
**What that costs an existing database is that it does not open**, and
"delete and rescan" is the answer (plan 013, open question 1) — free
for everyone except one machine. The index job's `/cache` volume is a
@@ -508,6 +681,56 @@ selected as a literal `0`. Adding the column to the importer's SELECT
list without that is how a published artifact — which nobody can re-cut
retroactively — starts failing with `no such column`.
**A credit is ordered parts, and the string is derived from them.** A
track credited to several artists had exactly one navigable artist and
the rest were punctuation: `primaryArtist()` string-parses the credit,
strips a " feat. " clause and discards the guest, and deliberately does
not split on `&`, `with` or `,` because those live inside real artist
names ("Simon & Garfunkel"). Measured on a real 26,069-file library,
**13%** of recordings are multi-artist upstream while only **0.86%** of
files carry a structured multi-artist tag — mp3 carries *zero* files
with multiple `MUSICBRAINZ_ARTISTID` across 19,840 — so this cannot be
a tag-parsing feature. (The "3 credits of 2,823" figure that justified
plan 013's removal of the credit tables measured our own *writer*:
`cachedLinkArtist` ran once per credit, so a collaboration could never
have been recorded. Dropping the join table was still right on cost.)
`artist_credit_part` / `artist_credit_ref` carry the decomposition for
multi-artist credits only — a single-artist credit is already
`explore_index`'s own `artist_name`, and storing those would triple the
table to say nothing. Five things about it are load-bearing:
- **Join phrases are assembly instructions, not disassembly ones.**
`creditLink` concatenates parts, so link boundaries are known by
construction. Locating a `credited_name` *inside* the stored credit
string would reintroduce the fault this exists to fix: that string may
come from the file's tags while the parts come from the catalog, and
the two disagree for ~1 in 3 multi-artist credits (`'Skrillex feat.
Swae Lee'` tagged against `'Skrillex & Swae Lee'` upstream).
- **`credited_name` is stored per row**, never joined from `artists`:
MusicBrainz credits "Snoop Dogg" on a track by the artist called
"Snoop Doggy Dogg". Display follows the credit, navigation the MBID.
- **The lookup is keyed on the recording MBID**, which the catalog and a
local file both carry (`library.Track.RecordingMBID`), so one binding
serves Explore and the library's own lists — which is why this needed
no local table. `file_artists` remains the offline-resilience step and
is deliberately *not* declared until something writes it.
- **Absence is cached as an answer.** `credit-store.ts` stores `[]` for
a single-artist credit — *asked*, not *answered* — or the ~87% that
have nothing to decompose are re-requested on every render forever.
`request()` is per-row and coalesces into one call per frame, because
a virtualized list cannot hand over "the whole list": 50,000 rows is
100 queries for the ~30 on screen.
- **The dump is a third source, and it had to be.** The canonical dump
CI already streams has no join phrases and no as-credited names, and
the JSON dumps cover 153,691 recordings of ~35M with *zero* overlap
against a real library. So `mbdump.tar.bz2` — 7.1 GB, ~13.7 min in
pure-Go bzip2, whose members are alphabetical, which is what lets one
pass resolve an entity's credit without buffering 35M recordings. The
pass runs on **every** mode, because a complete import means
`refresh`, which never enters the importer at all, and it reports
whether it populated anything so `changed` republishes the artifact.
**A 0.6 GB download asks about the connection first.** `explore`'s
catalog artifact had no network awareness at all, which on a phone is a
month's data allowance spent without being asked (plan 016 B4).
@@ -1909,13 +2132,31 @@ Pre-commit hooks verify generated code is fresh — always run `make generate` a
two in step or semantic-release will decline to release something the
check accepted.
`.releaserc.yml` is a complete semantic-release config that **nothing
currently runs** — no workflow invokes it, and `CHANGELOG.md` is not
being written by it. That is deliberate for now (wiring it means pushing
tags, committing a changelog back, and interacting with the three
publish workflows); it is recorded here rather than implied, because
this file claimed for five phases that commitlint gated CI and that
semantic release ran, and neither was true.
`.releaserc.yml` **is** what runs now, from `release.yml`, and it is why
the commit grammar is load-bearing rather than decorative: a merge to
`main` whose commits are all `chore`/`ci`/`docs` releases nothing, and a
mistyped `feat` ships a minor version. `make release-dry` answers "what
would this merge release" without pushing.
**`@semantic-release/github` is not in that config and must not be.**
Gitea's API is `/api/v1` and is not GitHub's surface, so
`@semantic-release/exec` calls `scripts/gitea-release.sh` instead — one
`POST`, which is the whole of the Gitea-shaped work. The community
plugin (`@saithodev/semantic-release-gitea`) was considered and
rejected: last published 2022, on `got@10`, declaring no peer
dependency on semantic-release at all.
Two things in it fail *silently* and are therefore pinned with their
reasons. **The notes come from `CHANGELOG.md`, not from an argument**:
release notes are rendered commit messages — arbitrary text carrying
backticks, quotes and `$` — so templating `${nextRelease.notes}` into
`publishCmd` would be a shell injection whose input is the commit log.
And **`conventional-changelog-conventionalcommits` is held at 9**,
because at 10 it is quietly incompatible with the writer
`release-notes-generator@14` pulls in: every release note renders as a
bare `## 0.0.1 (date)` heading with no sections and no commits beneath
it, no step fails, and the release ships with an empty body. Check the
rendered notes, never the exit code.
## Testing
@@ -1924,17 +2165,97 @@ Tests use `database.NewTestDB(t)` for in-memory SQLite, built by the same
## Git Workflow
Feature branches and PRs are the norm, but direct pushes to `main` are allowed. Pre-commit runs vet, lint, codegen check, and frontend typecheck in parallel. Pre-push runs the full test suite.
Feature branches and PRs are the only way in: **`main` is a protected
branch** (`enable_push: false`, an empty push whitelist, and `CI / check*`
+ `CI / e2e*` as required status checks), so a direct push is rejected by
the pre-receive hook. This file said otherwise for a long time. Tags are
*not* protected, which is what lets `release.yml` push one.
**A branch answers a claimed issue** — see "Issues" above. The commit
grammar is unchanged and is load-bearing for a different reason
(semantic-release reads it), so the issue number lives in the branch
name and the PR body rather than in the commit subject.
**A batch of small fixes can be one PR**, which is what #83 did: eight
branches preserved as merges under one integration branch, so
authorship survives and the batch lands as one release rather than
eight. The cost is that its `Closes` list has to be checked afterwards
— it half-worked.
Pre-commit runs vet, lint, codegen check, and frontend typecheck in parallel. Pre-push runs the full test suite.
## CI
Five workflows in `.gitea/workflows/`. Four of them package and
Eight workflows in `.gitea/workflows/`. Five of them package and
publish (`arch-package`, `homebrew-formula`, `index-artifact`,
`android-apk`); only `ci.yml` gates, and it is the one to look at when
`android-apk`, `desktop-assets`); `release.yml` decides *whether* four of
those run at all; `unclaim.yml` is housekeeping on the tracker and
touches no code; only `ci.yml` gates, and it is the one to look at when
deciding whether a push was healthy.
**`android-apk.yml` is the only one keyed on a tag and the only one
that can lose something irrecoverable.** It builds the signed
**`release.yml` is the entry point for all of it.** On every push to
`main` it reads the Conventional Commits since the last tag and, if any
is releasable, writes the changelog, pushes the tag and creates the Gitea
release whose body is that changelog section. `arch-package`,
`homebrew-formula`, `android-apk` and `desktop-assets` are all keyed on
`v*`, so **the tag push is what starts them** — nothing is released by
hand any more.
Four things about it are load-bearing:
- **The tag is pushed with a user PAT, not the Actions token.** Gitea,
like GitHub, does not start a workflow from a ref pushed by a
workflow's own token (go-gitea#33123). The token is what decides this,
so `PACKAGE_TOKEN` is handed to semantic-release as the
`repositoryUrl` credential and the push is attributed to a person.
- **That same limitation is used deliberately, once.** semantic-release
calls the first release of a tagless repo `1.0.0` and offers no way to
say otherwise, so a `v0.0.0` floor tag is what makes the first release
`0.0.1` — and it is pushed with the *Actions* token precisely so it
triggers nothing. All four publishers additionally skip `v0.0.0`
explicitly, cleanly rather than by failing, because a floor is not a
shipment.
- **The release page is the changelog, and that follows from the branch
protection.** `@semantic-release/git` would push a `chore(release):`
commit back to `main`, which the pre-receive hook rejects — *after* the
tag had been pushed, leaving a tagged release the run then reports as
failed. Whitelisting the CI user was the alternative and was declined:
it weakens a protection someone set on purpose and lets a bot push to
`main` without the checks every human PR passes. So the plugin is
absent, `@semantic-release/changelog` writes to a gitignored
`.release-notes.md` purely to carry the notes into
`scripts/gitea-release.sh`, and `CHANGELOG.md` is a signpost to the
releases page rather than a file that would silently stop updating.
The workflow keeps its `chore(release):` guard anyway, for the day
someone adds the plugin back.
- **An asset upload waits for the release to exist.** semantic-release
pushes the tag in `prepare` and creates the release in `publish`, so
the tag push that starts these workflows happens *before* there is a
release id to attach to. `scripts/release-asset.sh` polls for it. The
capacity-1 runner serialises things enough that this would usually work
by accident, which is the worst kind of bug.
**Releases restarted at `0.0.1`, which is a downgrade on every channel.**
pacman and Homebrew both silently offer no upgrade from the old `1.x`,
and Android refuses the install outright — its remedy is an uninstall
that takes the user's library. This was chosen over pacman's `epoch` and
over offsetting `versionCode`, on the grounds that both are permanent and
a reinstall is once. `packaging/homebrew/README.md` and
`docs/android-release.md` say so where a user would look.
**`desktop-assets.yml` publishes Linux and nothing else, and macOS is not
an oversight.** `GOOS=darwin CGO_ENABLED=0` fails at
`wails/v3/pkg/mac: build constraints exclude all Go files` — the darwin
backend is Objective-C behind cgo, so a `.app` needs a macOS host and the
runner is a Linux container. That is exactly why the Homebrew formula
builds from source on the user's own Mac. Windows *does* cross-compile
cleanly (`GOOS=windows CGO_ENABLED=0`, a couple of seconds — oto uses
WinMM through `x/sys`, sqlite is modernc's pure-Go driver, WebView2 is
COM syscalls, MPRIS is `linux && !android`-tagged) and is deliberately
not published: no Windows build of this app has ever been *run*, and no
tier here can exercise one.
**`android-apk.yml` is the one that can lose something irrecoverable.** It builds the signed
`arm64-v8a` APK (the only ABI Android can run this app on — see
`app/build.gradle`) on every `v*` tag and publishes it to the *generic* registry, which is
readable without credentials — the reason Obtainium can poll a plain
+25 -5
View File
@@ -179,10 +179,12 @@ bindings-check: ## Fail if the generated bindings are stale
css-check: ## Fail if a css`` literal was ended early by a backtick in a comment
@cd frontend && node scripts/check-css-literals.mjs
# .pi/ documents commands, and a skill that documents a command wrongly
# is worse than no skill: an agent runs it confidently. Every command
# in there is a make target on purpose, so this is checkable.
skill-check: ## Fail if .pi/ documents a make target that does not exist
# .pi/ and CLAUDE.md document commands, and a doc that documents a
# command wrongly is worse than no doc: an agent runs it confidently.
# Every command in them is a make target on purpose, so this is
# checkable. It also asserts AGENTS.md is a symlink to CLAUDE.md, so the
# two harnesses cannot drift onto two descriptions of one project.
skill-check: ## Fail if the agent docs name a missing make target, or AGENTS.md is not a symlink
@./scripts/skill-check.sh
# Conventional Commits, which CLAUDE.md claimed CI enforced for a long
@@ -190,6 +192,24 @@ skill-check: ## Fail if .pi/ documents a make target that does not exist
commit-check: ## Fail if a commit subject is not a Conventional Commit
@./scripts/commit-check.sh $(if $(RANGE),--range $(RANGE))
# What a merge to main would release, without releasing it. Reads the
# same .releaserc.yml CI does, so "why did that not cut a version" is
# answerable locally instead of by pushing and watching. Needs no
# credentials: --dry-run neither tags nor publishes.
#
# The pins must stay identical to release.yml's, which is where the note
# on holding the conventionalcommits preset at 9 lives -- at 10 the
# release notes come out empty with everything green.
release-dry: ## Print the version a merge to main would release
@npx --yes \
-p semantic-release@25 \
-p @semantic-release/commit-analyzer@13 \
-p @semantic-release/release-notes-generator@14 \
-p @semantic-release/changelog@7 \
-p @semantic-release/exec@7 \
-p conventional-changelog-conventionalcommits@9 \
semantic-release --dry-run --no-ci
# v3 generates TypeScript into frontend/bindings/, nested by Go import
# path, rather than v2's frontend/wailsjs/. The `@go` alias absorbs the
# constant prefix, so a call site imports '@go/library/library.js'.
@@ -205,7 +225,7 @@ bindings: ## Regenerate frontend/bindings from the bound Go services
sandbox-seed sandbox-seed-bulk sandbox-seeds e2e e2e-setup e2e-report \
perf perf-compare \
ui-test ui-watch ui-visual ui-visual-update ui-setup \
bindings bindings-check skill-check commit-check
bindings bindings-check skill-check commit-check release-dry
# Base directory for fresh-install sandboxes. Deliberately NOT $TMPDIR:
# on most Linux distros /tmp is tmpfs (RAM-backed) and only a few GB, so
+5 -2
View File
@@ -106,5 +106,8 @@ make dev # run with hot-reload
make build-prod # produce a release binary
```
More detail for contributors lives in
[`docs/dev/overview.md`](./docs/dev/overview.md) and [`CLAUDE.md`](./CLAUDE.md).
More detail for contributors lives in [`CLAUDE.md`](./CLAUDE.md) — the
architecture, the conventions and the reasons behind them. What is
being worked on is [the issue
tracker](https://git.ljones.me/yonlu/yellowjacket/issues); #73 is the
roadmap.
+3 -2
View File
@@ -411,9 +411,10 @@ func (c *Config) SetDownloadPreferences(prefs download.AutoDownloadPrefs) error
formats = append(formats, string(f))
}
c.Downloads.MinFileSizeMB = prefs.MinSizeMB
c.Downloads.MinKbps = prefs.MinKbps
c.Downloads.MaxKbps = prefs.MaxKbps
c.Downloads.PreferredKbps = prefs.PreferredKbps
c.Downloads.MaxFileSizeMB = prefs.MaxSizeMB
c.Downloads.PreferredFileSizeMB = prefs.PreferredSizeMB
c.Downloads.AllowedFormats = formats
if err := c.Save(); err != nil {
+8
View File
@@ -89,6 +89,14 @@ func NewDB(logger *slog.Logger) (*DB, error) {
return nil, fmt.Errorf("could not apply PRAGMAs: %w", err)
}
// Before the schema is applied, not after: applySchema is
// CREATE ... IF NOT EXISTS, which no-ops against a table that
// already exists in an older shape. Retiring the stale one first is
// what turns that no-op into a create.
if err := retireStaleTables(dbCtx, db, logger); err != nil {
return nil, err
}
if err := applySchema(dbCtx, db); err != nil {
return nil, err
}
@@ -0,0 +1,56 @@
-- The decomposition of a multi-artist credit, from the MusicBrainz
-- dump. One row per credited artist, in credit order.
--
-- A credit is ordered parts, and the credit *string* is derived from
-- them -- MusicBrainz's own `artist_credit.name` is a cached render and
-- nothing more. Rendering is a concatenation:
--
-- for each part in position order:
-- emit link(credited_name -> artist_mbid)
-- emit text(join_phrase)
--
-- so the link boundaries are known by construction. That is the whole
-- reason this table exists, and it is why nothing may reconstruct a
-- credit by *searching* for a name inside a credit string: the stored
-- string may have come from a file's tags while the parts come from the
-- catalog, and measured on a real library those disagree for about one
-- in three multi-artist credits ("Skrillex feat. Swae Lee" tagged
-- against "Skrillex & Swae Lee" upstream). A search would miss, or
-- match the wrong span.
--
-- `credited_name` is the name *as credited*, which is not the artist's
-- canonical name: MusicBrainz credits "Snoop Dogg" on a track by the
-- artist whose name is "Snoop Doggy Dogg". It is stored per row rather
-- than joined from an artist table for exactly that reason.
--
-- Only *multi-artist* credits are stored. A single-artist credit is
-- (name, "") and is already fully described by explore_index's
-- artist_name and artist_mbid; storing those would roughly triple the
-- table to say nothing new.
--
-- Credits are shared: an album's twelve tracks by one artist reference
-- one credit_id. That is the opposite of the local library's verdict
-- in plan 013, and correctly so -- credit sharing is 1:1 in one
-- person's files and genuinely many-to-one across a 2M-row catalog.
--
-- MBIDs are the same 16 raw bytes explore_index stores, for the same
-- size reason and with the same CHECK, so a stringly write fails at the
-- insert that made it rather than reading back as no rows at all. See
-- backend/explore/mbid.go.
CREATE TABLE IF NOT EXISTS artist_credit_part (
credit_id INTEGER NOT NULL,
position INTEGER NOT NULL,
artist_mbid BLOB NOT NULL CHECK(length(artist_mbid) = 16),
-- The name as credited on this release, which may differ from the
-- artist's canonical name. Display uses this; navigation uses the
-- MBID above.
credited_name TEXT NOT NULL,
-- The literal connector that follows this part -- " feat. ", " & ",
-- ", ", or "" on the last part. Rendered as plain text between two
-- links.
join_phrase TEXT NOT NULL DEFAULT '',
PRIMARY KEY (credit_id, position)
) WITHOUT ROWID;
@@ -0,0 +1,30 @@
-- Which credit a catalog entity is credited to. One row per recording
-- or release group whose credit names more than one artist.
--
-- This is a table rather than an `explore_index.artist_credit_id`
-- column, and that is a deliberate consequence of how this app applies
-- its schema. `applySchema` is CREATE ... IF NOT EXISTS and there is
-- no migration chain (plan 013), so a *column* added to an existing
-- table never reaches a database that already has it -- while a new
-- *table* is created on every install, old or new, for free.
-- explore_index is the one table nobody can afford to drop and rebuild
-- on a schema change: it is the artifact users download rather than
-- derive.
--
-- Only multi-artist credits are referenced here, matching
-- artist_credit_part. An entity with no row is credited to exactly one
-- artist, which explore_index's own artist_name and artist_mbid already
-- describe -- so absence is the common case and means "nothing to
-- decompose", not "unknown".
--
-- `credit_id` is opaque and is only meaningful against the
-- artist_credit_part rows built or imported alongside it. The two are
-- always written together; nothing persists a credit_id anywhere else.
-- The local library stores resolved parts, never this id.
CREATE TABLE IF NOT EXISTS artist_credit_ref (
mbid BLOB NOT NULL PRIMARY KEY CHECK(length(mbid) = 16),
credit_id INTEGER NOT NULL
) WITHOUT ROWID;
CREATE INDEX IF NOT EXISTS idx_artist_credit_ref_credit
ON artist_credit_ref(credit_id);
+13
View File
@@ -27,6 +27,19 @@ type Artist struct {
Mbid sql.NullString
}
type ArtistCreditPart struct {
CreditID int64
Position int64
ArtistMbid []byte
CreditedName string
JoinPhrase string
}
type ArtistCreditRef struct {
Mbid []byte
CreditID int64
}
type ArtistEnrichment struct {
ArtistMbid string
BrowsedAt sql.NullTime
+560
View File
@@ -0,0 +1,560 @@
package database
import (
"context"
"database/sql"
"fmt"
"io/fs"
"log/slog"
"maps"
"path"
"slices"
"strings"
"yellowjacket/backend/datamap"
)
// This file repairs the one thing `CREATE TABLE IF NOT EXISTS` cannot.
//
// `sql/schemas/` is the single description of the schema and there is no
// migration chain (plan 013): a schema change is one edit to one file.
// That works perfectly for a *new* table, which every install then
// creates, and not at all for a changed one -- `IF NOT EXISTS` reaches
// an existing table only if its shape already matches, and otherwise
// silently no-ops. The user's answer to that is "delete and rescan"
// (plan 013, open question 1), which is free for everything a rescan
// rebuilds.
//
// It is not free for the catalog. explore_index is a *downloaded
// artifact*, not something derived from the user's files, and it is the
// largest thing this app stores. So it went stale instead: plan 014
// added `total_tracks` to the schema and to `indexRowFields` -- the one
// projection every explore read uses -- and no database that already
// existed ever grew the column. Every Explore search, browse, artist
// page and album page on such an install fails with
// "no such column: total_tracks", while a fresh install is perfectly
// healthy, which is why the tests did not see it. The same databases
// are stale a second way, from the same plan: their `mbid` columns are
// still TEXT where the schema now declares BLOB, and SQLite does not
// coerce between the two -- a comparison against 16 raw bytes simply
// returns no rows.
//
// The repair is to notice and drop, not to migrate. A dropped catalog
// costs one artifact download (about a minute); the alternative --
// ALTER TABLE ADD COLUMN, which would handle `total_tracks` alone
// cheaply -- cannot express the TEXT-to-BLOB half at all, and would
// leave those installs quietly broken while reporting success.
//
// Everything except `Authored` is eligible. `Cache` is rebuildable by
// definition; `Owned` is a projection of the user's files and a rescan
// rebuilds it, which is plan 013's stated answer to exactly this
// situation ("delete and rescan", open question 1); `Derived` is
// computed from Owned. No `Authored` table is ever dropped here --
// that is the whole point of the datamap, and it is asserted by
// TestAuthoredTablesAreNeverRetired rather than only stated.
//
// What that does *not* buy is immunity for authored rows that reference
// a retired table. `audio_files` is MIXED KIND: `play_count`,
// `last_played` and `tag_status` are authored columns on an Owned
// table, and they go with it. Playlists survive as playlists, and
// their entries survive pointing at nothing. That cost was weighed and
// accepted rather than overlooked -- the alternative is to carry the
// authored columns across the rebuild keyed on file_path, which stays a
// real option if this ever bites harder than it is worth.
//
// **This relies on foreign_keys being ON**, which applyPRAGMAs has
// already done by the time NewDB calls it, and the dependency is not
// cosmetic. SQLite performs an implicit DELETE before dropping a table
// when foreign keys are enabled, so `playlist_tracks.audio_file_id` --
// declared ON DELETE SET NULL -- is nulled. With foreign keys off, no
// action fires and those rows keep the ids they had, which a rescan
// then reissues starting from 1: every playlist would silently fill
// with *different songs*. Nulled entries are merely empty; stale ones
// are wrong, and wrong quietly. TestRetiringOwnedTablesDoesNotDangle
// is what stops a future reordering turning one into the other.
// retireGroups are tables that must be retired together. A catalog
// whose rows are gone must not keep the full-text index built over
// them, nor the metadata claiming the import that produced them
// finished -- that marker is exactly what stops the artifact being
// fetched again. applySchema recreates all three empty immediately
// afterwards, and the ordinary "no index yet" path takes over.
var retireGroups = [][]string{
{
"explore_index",
"explore_index_fts",
"explore_index_meta",
"explore_champion_fts",
},
}
// schemaColumn is one column as the schema file declares it.
type schemaColumn struct {
name string
typ string
}
// retireStaleTables drops every non-authored table whose live shape no
// longer matches what sql/schemas/ declares, plus any table the schema
// no longer describes at all, so applySchema can create the current
// shape afresh. It runs before applySchema and is a no-op on a new
// database, where the tables do not exist yet.
func retireStaleTables(
ctx context.Context, db *sql.DB, logger *slog.Logger,
) error {
declared, err := declaredTables()
if err != nil {
return err
}
stale := make(map[string]string)
for table, columns := range declared {
entry, ok := datamap.Lookup(table)
if !ok || entry.Kind == datamap.Authored || entry.FTS {
continue
}
// Whether a stale Cache table may be rebuilt is decided per
// binary, at compile time: the app re-downloads its catalog in
// about a minute, cmd/indexbuild would re-derive it from ~205 GB
// of dumps. See staleshape_policy.go.
if entry.Kind == datamap.Cache && !retireStaleCache {
continue
}
reason, err := staleReason(ctx, db, table, columns)
if err != nil {
return err
}
if reason != "" {
stale[table] = reason
}
}
obsolete, err := obsoleteTables(ctx, db)
if err != nil {
return err
}
maps.Copy(stale, obsolete)
if len(stale) == 0 {
return nil
}
return retireGroupsFor(ctx, db, logger, stale)
}
// obsoleteTables are live tables the schema no longer describes at all.
// TestCatalogCoversSchema makes the datamap a complete description of
// the current schema, so a table it does not know is one a past version
// created and this one does not -- plan 013 alone left seven behind
// (recordings, release_groups, artist_credit, artist_credit_artist,
// release_group_recordings, recording_genres) plus the
// schema_migrations table that squashing the chain retired. They are
// dead weight, and one of them holding a foreign key into a table being
// rebuilt is worse than dead weight.
//
// SQLite's own bookkeeping and FTS shadow tables are not obsolete:
// datamap.Lookup resolves a shadow table to its parent, and IsInternal
// covers the rest.
func obsoleteTables(ctx context.Context, db *sql.DB) (map[string]string, error) {
rows, err := db.QueryContext(
ctx, "SELECT name FROM sqlite_master WHERE type = 'table'",
)
if err != nil {
return nil, fmt.Errorf("could not list tables: %w", err)
}
defer func() { _ = rows.Close() }()
out := make(map[string]string)
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, fmt.Errorf("could not scan table name: %w", err)
}
if datamap.IsInternal(name) {
continue
}
if _, known := datamap.Lookup(name); !known {
out[name] = "the schema no longer describes this table"
}
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("could not read table list: %w", err)
}
return out, nil
}
// retireGroupsFor drops each stale table along with everything its
// retire group says must go with it.
func retireGroupsFor(
ctx context.Context, db *sql.DB, logger *slog.Logger,
stale map[string]string,
) error {
drop := make(map[string]string)
for table, reason := range stale {
drop[table] = reason
for _, group := range retireGroups {
if !slices.Contains(group, table) {
continue
}
for _, member := range group {
if _, already := drop[member]; !already {
drop[member] = "retired with " + table
}
}
}
}
return dropDeferred(ctx, db, logger, drop)
}
// dropDeferred drops every named table in one transaction with foreign
// key enforcement deferred to the commit.
//
// The deferral is required and the two obvious alternatives are both
// wrong. These tables reference each other -- pre-013 `audio_files`
// has a foreign key into `recordings`, which is itself being retired --
// so dropping them one at a time in an arbitrary order fails with
// "FOREIGN KEY constraint failed" on whichever is unlucky enough to go
// first, and there is no order that is safe in general. Turning
// foreign keys *off* for the duration would fix that and silently take
// the ON DELETE SET NULL on `playlist_tracks.audio_file_id` with it,
// leaving playlist entries pointing at ids a rescan reissues to
// different songs -- the exact failure
// TestRetiringOwnedTablesDoesNotDangle exists to prevent.
//
// Deferring keeps the actions firing while tolerating the inconsistency
// in the middle, and the commit then checks that the end state is
// sound. It is set inside the transaction because SQLite resets it at
// every commit.
func dropDeferred(
ctx context.Context, db *sql.DB, logger *slog.Logger,
drop map[string]string,
) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("could not begin the retire transaction: %w", err)
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx, "PRAGMA defer_foreign_keys = ON"); err != nil {
return fmt.Errorf("could not defer foreign keys: %w", err)
}
// Sorted, so a failure is reproducible. Map order is random, and a
// bug that depends on which table happens to go first reproduces on
// one run in three and passes review on the other two -- which is
// exactly how the foreign-key ordering above reached a real
// database. Sorting does not make any order *safe*; the deferral
// does that.
for _, table := range slices.Sorted(maps.Keys(drop)) {
logger.Warn(
"retiring a table the schema no longer describes",
"table", table,
"reason", drop[table],
)
if _, err := tx.ExecContext(
ctx, "DROP TABLE IF EXISTS "+quoteIdent(table),
); err != nil {
return fmt.Errorf("could not retire stale table %s: %w", table, err)
}
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("could not commit the retire: %w", err)
}
return nil
}
// staleReason reports why a live table disagrees with its declaration,
// or "" when it agrees. A column the live table does not have is the
// additive case; a column whose declared type changed is the one an
// ALTER could not fix anyway. Columns the live table has and the
// schema no longer declares are ignored: they cost nothing and dropping
// the table over one would retire a healthy catalog.
func staleReason(
ctx context.Context, db *sql.DB, table string, columns []schemaColumn,
) (string, error) {
live, err := liveColumns(ctx, db, table)
if err != nil {
return "", err
}
if len(live) == 0 {
// Not present at all: applySchema is about to create it.
return "", nil
}
for _, col := range columns {
liveType, present := live[col.name]
if !present {
return "missing column " + col.name, nil
}
if !sameDeclaredType(col.typ, liveType) {
return fmt.Sprintf(
"column %s is %s, schema declares %s",
col.name, liveType, col.typ,
), nil
}
}
return "", nil
}
// liveColumns returns the live table's columns and their declared types,
// empty when the table does not exist.
func liveColumns(
ctx context.Context, db *sql.DB, table string,
) (map[string]string, error) {
rows, err := db.QueryContext(
ctx, "SELECT name, type FROM pragma_table_info(?)", table,
)
if err != nil {
return nil, fmt.Errorf("could not inspect table %s: %w", table, err)
}
defer func() { _ = rows.Close() }()
out := make(map[string]string)
for rows.Next() {
var name, typ string
if err := rows.Scan(&name, &typ); err != nil {
return nil, fmt.Errorf("could not scan column of %s: %w", table, err)
}
out[name] = typ
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("could not read columns of %s: %w", table, err)
}
return out, nil
}
// sameDeclaredType compares two SQLite type names. They are compared
// case-insensitively and only on the leading word, so INTEGER matches
// INTEGER and VARCHAR(20) matches VARCHAR -- SQLite's affinity rules
// make finer distinctions meaningless, and a difference that fine is
// not worth retiring a catalog over. An empty declared type matches
// anything, which is what a column declared with only constraints has.
func sameDeclaredType(declared, live string) bool {
d := strings.ToUpper(strings.Fields(declared + " ")[0])
l := strings.ToUpper(strings.Fields(live + " ")[0])
if d == "" || l == "" {
return true
}
if i := strings.IndexByte(d, '('); i >= 0 {
d = d[:i]
}
if i := strings.IndexByte(l, '('); i >= 0 {
l = l[:i]
}
return d == l
}
// declaredTables parses every CREATE TABLE in sql/schemas/ into its
// column list. Parsing the schema rather than writing the expectation
// down a second time is the point: a second list is a second thing to
// forget, which is the fault this whole file exists to repair.
func declaredTables() (map[string][]schemaColumn, error) {
dirEntries, err := schemas.ReadDir("sql/schemas")
if err != nil {
return nil, fmt.Errorf("could not read schemas directory: %w", err)
}
out := make(map[string][]schemaColumn)
for _, dirEntry := range dirEntries {
if dirEntry.IsDir() {
continue
}
content, err := fs.ReadFile(schemas, path.Join("sql/schemas", dirEntry.Name()))
if err != nil {
return nil, fmt.Errorf("could not read %s: %w", dirEntry.Name(), err)
}
maps.Copy(out, parseCreateTables(string(content)))
}
return out, nil
}
// constraintKeywords begin a table constraint rather than a column.
var constraintKeywords = map[string]bool{
"PRIMARY": true, "FOREIGN": true, "UNIQUE": true,
"CHECK": true, "CONSTRAINT": true,
}
// parseCreateTables extracts the column names and declared types of
// every non-virtual CREATE TABLE in one schema file.
func parseCreateTables(content string) map[string][]schemaColumn {
out := make(map[string][]schemaColumn)
rest := stripLineComments(content)
for {
idx := indexFold(rest, "CREATE TABLE ")
if idx < 0 {
return out
}
rest = rest[idx+len("CREATE TABLE "):]
head, body, ok := splitTableBody(rest)
if !ok {
return out
}
if name := tableName(head); name != "" {
out[name] = parseColumns(body)
}
}
}
// tableName pulls the table name out of the text between "CREATE TABLE"
// and its opening parenthesis, dropping an IF NOT EXISTS and any
// quoting.
func tableName(head string) string {
head = strings.TrimSpace(head)
head = strings.TrimPrefix(head, "IF NOT EXISTS ")
head = strings.TrimPrefix(head, "if not exists ")
fields := strings.Fields(head)
if len(fields) == 0 {
return ""
}
return strings.Trim(fields[len(fields)-1], `"'`+"`")
}
// splitTableBody returns the text before the table's opening paren and
// the balanced text inside it.
func splitTableBody(s string) (head, body string, ok bool) {
open := strings.IndexByte(s, '(')
if open < 0 {
return "", "", false
}
depth := 0
for i := open; i < len(s); i++ {
switch s[i] {
case '(':
depth++
case ')':
depth--
if depth == 0 {
return s[:open], s[open+1 : i], true
}
}
}
return "", "", false
}
// parseColumns splits a table body on its top-level commas and keeps
// the parts that are columns rather than table constraints.
func parseColumns(body string) []schemaColumn {
var (
out []schemaColumn
depth int
start int
)
parts := make([]string, 0, 8)
for i := range len(body) {
switch body[i] {
case '(':
depth++
case ')':
depth--
case ',':
if depth == 0 {
parts = append(parts, body[start:i])
start = i + 1
}
}
}
parts = append(parts, body[start:])
for _, part := range parts {
fields := strings.Fields(part)
if len(fields) == 0 {
continue
}
// A table constraint need not be followed by a space --
// "UNIQUE(mbid)" is one field, and reading it as a column name
// makes an entirely healthy table look stale, which retires a
// catalog nobody asked to lose.
head := fields[0]
if i := strings.IndexByte(head, '('); i >= 0 {
head = head[:i]
}
if constraintKeywords[strings.ToUpper(head)] {
continue
}
col := schemaColumn{name: strings.Trim(head, `"'`+"`")}
if len(fields) > 1 {
col.typ = fields[1]
}
out = append(out, col)
}
return out
}
// stripLineComments removes -- comments, which otherwise contribute
// stray parentheses and commas to the parse.
func stripLineComments(s string) string {
lines := strings.Split(s, "\n")
for i, line := range lines {
if idx := strings.Index(line, "--"); idx >= 0 {
lines[i] = line[:idx]
}
}
return strings.Join(lines, "\n")
}
// indexFold is a case-insensitive strings.Index.
func indexFold(s, substr string) int {
return strings.Index(strings.ToUpper(s), strings.ToUpper(substr))
}
// quoteIdent quotes a table name for interpolation into DDL, which
// cannot take a bound parameter.
func quoteIdent(name string) string {
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
}
+15
View File
@@ -0,0 +1,15 @@
//go:build !indexbuild
package database
// retireStaleCache reports whether a Cache table whose shape no longer
// matches the schema may be dropped and rebuilt.
//
// In the app: yes. The only Cache table large enough to care about is
// the catalog, and the app does not derive it — it downloads it. A
// stale one costs about a minute of re-fetching the artifact, and
// keeping it costs every Explore read on the install, because a
// projection naming a column the table does not have fails outright.
//
// In cmd/indexbuild: no, and the file next to this one says why.
const retireStaleCache = true
@@ -0,0 +1,37 @@
//go:build indexbuild
package database
// retireStaleCache is false here, and this is the whole reason the
// policy is a build tag rather than a rule inside retireStaleTables.
//
// The index database is the one place in this project where the catalog
// is *derived* rather than downloaded. Rebuilding it is a ~205 GB dump
// stream over hours, resumed across runs from a checkpoint on a
// persistent volume; that volume exists for no other purpose. The app's
// answer to a stale catalog — drop it, fetch the artifact again — is
// not available here, because this database *is* what the artifact is
// cut from.
//
// This was not hypothetical. The repair shipped without it and dropped
// the CI catalog on its first run:
//
// retiring a table ... table=explore_index
// reason="column entity_type is TEXT, schema declares INTEGER"
// index maintenance mode=build reason="no completed import yet"
//
// The shape mismatch was real and the drop was correct by the app's
// rule. It was still wrong here: that database is deliberately kept in
// the older encoding, which is what `fix(indexexport): read an index
// older than the binary` exists to tolerate. A rule that is right for
// every install and catastrophic for one database has to be told which
// one it is in, and a build tag is how this project already tells the
// index tools apart (backend/events/runtime_indexbuild.go,
// backend/explore/servicestartup.go, dumpbuild_stub.go).
//
// cmd/indexbuild has its own repair for the half it *can* safely
// discard: retireLibraryTables drops every table the datamap does not
// classify as Cache, which is empty by construction in that database.
// Between the two, the library half is repaired and the catalog is
// never touched.
const retireStaleCache = false
+499
View File
@@ -0,0 +1,499 @@
package database
import (
"context"
"database/sql"
"log/slog"
"path"
"testing"
_ "modernc.org/sqlite"
)
// testLogger discards the repair's warnings; the tests assert on the
// database, not on the log.
func testLogger() *slog.Logger {
return slog.New(slog.DiscardHandler)
}
// openRaw opens a scratch database file with no schema applied, so a
// test can build an *old* shape and then let NewDB's repair meet it.
func openRaw(t *testing.T, dir string) *sql.DB {
t.Helper()
db, err := sql.Open("sqlite", path.Join(dir, "yj.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
return db
}
// TestRetiresIndexMissingAColumn is plan 014's bug, symptom first: an
// explore_index created before `total_tracks` existed, met by the
// projection every explore read uses. Before the repair this failed
// with "no such column: total_tracks" on every install that already had
// a catalog, while a fresh one was perfectly healthy.
func TestRetiresIndexMissingAColumn(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
// The pre-014 shape: the columns the projection needs, minus the
// one the plan added.
if _, err := db.ExecContext(ctx, `
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid BLOB NOT NULL
);
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid)
VALUES (1, x'00112233445566778899aabbccddeeff', 'x', 'y', x'');
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
// The column the projection needs is there now.
var n int
if err := db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM pragma_table_info('explore_index')
WHERE name = 'total_tracks'`,
).Scan(&n); err != nil {
t.Fatalf("inspect: %v", err)
}
if n != 1 {
t.Fatalf("explore_index still has no total_tracks column")
}
// And the catalog really was retired rather than patched, so the
// artifact is fetched again instead of half a catalog being served.
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM explore_index",
).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 0 {
t.Fatalf("stale rows survived the retire: %d", n)
}
}
// TestRetiresIndexWithTextMBIDs is the half an ALTER could not have
// repaired: plan 013 changed mbid from TEXT to BLOB, and SQLite does not
// coerce between them, so a query against 16 raw bytes returns no rows
// rather than an error.
func TestRetiresIndexWithTextMBIDs(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
if _, err := db.ExecContext(ctx, `
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type TEXT NOT NULL,
mbid TEXT NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid TEXT NOT NULL,
total_tracks INTEGER NOT NULL DEFAULT 0
);
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
var typ string
if err := db.QueryRowContext(ctx,
`SELECT type FROM pragma_table_info('explore_index') WHERE name = 'mbid'`,
).Scan(&typ); err != nil {
t.Fatalf("inspect: %v", err)
}
if typ != "BLOB" {
t.Fatalf("mbid is still %s, want BLOB", typ)
}
}
// TestRetiringTheIndexTakesItsMetaWithIt guards the thing that makes the
// repair actually repair: the marker saying the import finished is what
// stops the artifact being fetched again, so a catalog dropped without
// it would stay empty forever.
func TestRetiringTheIndexTakesItsMetaWithIt(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
if _, err := db.ExecContext(ctx, `
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL
);
CREATE TABLE explore_index_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
INSERT INTO explore_index_meta VALUES ('dump_import_done', '1');
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM explore_index_meta WHERE key = 'dump_import_done'",
).Scan(&n); err != nil {
t.Fatalf("meta: %v", err)
}
if n != 0 {
t.Fatalf("the import-done marker survived a retired catalog")
}
}
// TestHealthyDatabaseIsUntouched is the other half, and the one that
// would make this dangerous if it failed: a current schema must survive
// a launch with its catalog intact. A repair that retires a healthy
// catalog costs every user an artifact download on every start.
func TestHealthyDatabaseIsUntouched(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
if _, err := db.ExecContext(ctx, `
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid)
VALUES (1, x'00112233445566778899aabbccddeeff', 'x', 'y', x'')
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM explore_index",
).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 1 {
t.Fatalf("a healthy catalog was retired: %d rows left", n)
}
}
// TestAuthoredTablesAreNeverRetired states the boundary in a test rather
// than only in a comment: this mechanism deletes data, and the only
// thing standing between it and a user's playlists is the Kind filter.
func TestAuthoredTablesAreNeverRetired(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
// A playlists table missing most of its current columns.
if _, err := db.ExecContext(ctx, `
CREATE TABLE playlists (id INTEGER PRIMARY KEY, name TEXT NOT NULL);
INSERT INTO playlists (name) VALUES ('irreplaceable');
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM playlists",
).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 1 {
t.Fatalf("an authored table was retired; rows left: %d", n)
}
}
// TestRetiresTablesTheSchemaNoLongerDescribes covers what plan 013 left
// behind on every database that predates it: seven tables the schema
// stopped describing, plus the schema_migrations table that squashing
// the chain retired. They are not stale in shape — they are simply not
// ours any more.
func TestRetiresTablesTheSchemaNoLongerDescribes(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if _, err := db.ExecContext(ctx, `
CREATE TABLE recordings (id INTEGER PRIMARY KEY, name TEXT);
CREATE TABLE artist_credit (id INTEGER PRIMARY KEY, text TEXT);
CREATE TABLE schema_migrations (version INTEGER PRIMARY KEY);
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
for _, table := range []string{"recordings", "artist_credit", "schema_migrations"} {
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name = ?",
table,
).Scan(&n); err != nil {
t.Fatalf("inspect %s: %v", table, err)
}
if n != 0 {
t.Errorf("%s survived; the schema no longer describes it", table)
}
}
}
// TestFTSShadowTablesAreNotObsolete is the sweep's sharp edge: an FTS5
// virtual table is backed by four shadow tables that appear in
// sqlite_master under their own names and are in no schema file.
// Dropping one destroys the index it belongs to.
func TestFTSShadowTablesAreNotObsolete(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
obsolete, err := obsoleteTables(ctx, db)
if err != nil {
t.Fatalf("obsoleteTables: %v", err)
}
if len(obsolete) != 0 {
t.Fatalf("a freshly created schema reported obsolete tables: %v", obsolete)
}
}
// TestRetiringOwnedTablesDoesNotDangle pins the one behaviour that is
// silently wrong rather than loudly broken.
//
// Retiring audio_files leaves playlist entries behind. With
// foreign_keys ON — which applyPRAGMAs has done before NewDB gets here —
// SET NULL fires and they point at nothing. With it OFF they keep ids
// that the rescan reissues from 1, so every playlist quietly fills with
// different songs. Nothing about the schema makes that ordering
// obvious, so it is asserted rather than assumed.
func TestRetiringOwnedTablesDoesNotDangle(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if _, err := db.ExecContext(ctx, "PRAGMA foreign_keys = ON"); err != nil {
t.Fatalf("pragma: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
// Break audio_files' shape so it is retired, keeping a playlist
// entry that references it.
if _, err := db.ExecContext(ctx, `
INSERT INTO playlists (id, name) VALUES (1, 'keepme');
INSERT INTO libraries (id, name, path) VALUES (0, 'test', '/music');
INSERT INTO audio_files (id, file_path, file_type_id, length_milliseconds)
VALUES (7, '/music/a.flac', 1, 1000);
INSERT INTO playlist_tracks (playlist_id, audio_file_id, position)
VALUES (1, 7, 0);
DROP VIEW IF EXISTS track_metadata;
ALTER TABLE audio_files DROP COLUMN artist_credit;
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
var dangling int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM playlist_tracks WHERE audio_file_id IS NOT NULL",
).Scan(&dangling); err != nil {
t.Fatalf("count: %v", err)
}
if dangling != 0 {
t.Fatalf(
"%d playlist entries still point at retired audio_files ids; "+
"a rescan will reissue those ids to different tracks",
dangling,
)
}
// The playlist itself is authored and must be untouched.
var playlists int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM playlists",
).Scan(&playlists); err != nil {
t.Fatalf("playlists: %v", err)
}
if playlists != 1 {
t.Fatalf("authored playlist lost: %d", playlists)
}
}
// TestRetiringInterlinkedLegacyTables is the bug the unit tests missed
// and a real database found.
//
// The tables plan 013 retired reference each other -- pre-013
// audio_files has a foreign key into recordings -- so with foreign keys
// ON, dropping them one at a time fails with "FOREIGN KEY constraint
// failed" on whichever goes first, and map iteration order decides
// which that is. Every other test in this file ran with foreign keys
// off and passed happily; the app enables them in applyPRAGMAs before
// the repair runs, so only the real launch path showed it.
func TestRetiringInterlinkedLegacyTables(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if _, err := db.ExecContext(ctx, "PRAGMA foreign_keys = ON"); err != nil {
t.Fatalf("pragma: %v", err)
}
// The pre-013 shape, with the reference that makes ordering matter.
// release_group_recordings sorts *after* recordings and references
// it, so the deterministic order retires the parent while the child
// still holds rows pointing at it -- which is the case that fails
// without the deferral, rather than one that fails on some runs.
if _, err := db.ExecContext(ctx, `
CREATE TABLE recordings (id INTEGER PRIMARY KEY, name TEXT);
CREATE TABLE artist_credit (id INTEGER PRIMARY KEY, text TEXT);
CREATE TABLE release_group_recordings (
id INTEGER PRIMARY KEY,
recording_id INTEGER NOT NULL,
FOREIGN KEY(recording_id) REFERENCES recordings(id)
);
CREATE TABLE audio_files (
id INTEGER PRIMARY KEY,
file_path TEXT NOT NULL UNIQUE,
recording_id INTEGER,
FOREIGN KEY(recording_id) REFERENCES recordings(id)
);
INSERT INTO recordings (id, name) VALUES (1, 'x');
INSERT INTO release_group_recordings (id, recording_id) VALUES (1, 1);
INSERT INTO audio_files (id, file_path, recording_id)
VALUES (1, '/music/a.flac', 1);
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
for _, table := range []string{"recordings", "artist_credit"} {
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name = ?",
table,
).Scan(&n); err != nil {
t.Fatalf("inspect %s: %v", table, err)
}
if n != 0 {
t.Errorf("%s survived the retire", table)
}
}
// And the rebuilt audio_files is the current shape, which is the
// whole reason the old one had to go.
var n int
if err := db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM pragma_table_info('audio_files')
WHERE name = 'artist_credit'`,
).Scan(&n); err != nil {
t.Fatalf("inspect audio_files: %v", err)
}
if n != 1 {
t.Fatal("audio_files was not rebuilt in the current shape")
}
}
// TestParseCreateTablesReadsTheRealSchema keeps the parser honest
// against the files it actually runs on: a parser that silently found
// no columns would report every table healthy and repair nothing.
func TestParseCreateTablesReadsTheRealSchema(t *testing.T) {
declared, err := declaredTables()
if err != nil {
t.Fatalf("declaredTables: %v", err)
}
cols, ok := declared["explore_index"]
if !ok {
t.Fatal("explore_index was not parsed out of the schema files")
}
want := map[string]string{
"mbid": "BLOB",
"total_tracks": "INTEGER",
"artist_name": "TEXT",
}
got := make(map[string]string, len(cols))
for _, c := range cols {
got[c.name] = c.typ
}
for name, typ := range want {
if got[name] != typ {
t.Errorf("explore_index.%s parsed as %q, want %q", name, got[name], typ)
}
}
// A table constraint must not be mistaken for a column.
for _, c := range cols {
switch c.name {
case "PRIMARY", "FOREIGN", "UNIQUE", "CHECK", "CONSTRAINT":
t.Errorf("parsed table constraint %q as a column", c.name)
}
}
}
+15
View File
@@ -185,6 +185,21 @@ var tables = []Table{
Note: "Full-text index over the champion entities of the " +
"MusicBrainz dump. Rebuilt only by a full index build.",
},
{
Name: "artist_credit_part", Kind: Cache, Lifetime: Retained,
Note: "The decomposition of a multi-artist credit, from the " +
"MusicBrainz dump: one row per credited artist, with the " +
"name as credited and the join phrase that follows it. " +
"Arrives with the downloaded artifact, so rebuilding it " +
"costs a dump stream and it is never swept.",
},
{
Name: "artist_credit_ref", Kind: Cache, Lifetime: Retained,
Note: "Which credit a catalog recording or release group is " +
"credited to. Present only for multi-artist credits; " +
"absence means one artist, which explore_index already " +
"describes. Ships and dies with artist_credit_part.",
},
{
Name: "explore_index", Kind: Cache, Lifetime: Retained,
Note: "The offline MusicBrainz search index. Rebuilding costs a " +
+28 -11
View File
@@ -34,13 +34,29 @@ type UserConfig struct {
// in one burst that every provider sees as a flood.
WantedBatch int `toml:"WantedBatch"`
// MinFileSizeMB, MaxFileSizeMB and PreferredFileSizeMB bound and
// nudge what auto-pick (interactive or via the request list) may
// grab without asking. Zero on any of them is permissive: see
// AutoDownloadPrefs.
MinFileSizeMB int `toml:"MinFileSizeMB"`
MaxFileSizeMB int `toml:"MaxFileSizeMB"`
PreferredFileSizeMB int `toml:"PreferredFileSizeMB"`
// MinKbps, MaxKbps and PreferredKbps bound and nudge what auto-pick
// (interactive or via the request list) may grab without asking.
// Zero on any of them is permissive: see AutoDownloadPrefs.
//
// They replaced MinFileSizeMB / MaxFileSizeMB /
// PreferredFileSizeMB, which were megabytes and so said nothing
// without knowing how long the release was. The old keys are
// deliberately *not* read back: a number that meant "300 MB" cannot
// be reinterpreted as a bitrate without knowing the album it was
// aimed at, so migrating it would be inventing an intent the user
// never expressed. An existing config falls back to no window,
// which is the permissive default and matches a fresh install —
// and MaxFileSizeMB is the one that does carry over, because a
// ceiling on total bytes still means exactly what it did.
MinKbps int `toml:"MinKbps"`
MaxKbps int `toml:"MaxKbps"`
PreferredKbps int `toml:"PreferredKbps"`
// MaxFileSizeMB is a hard ceiling on a candidate's total size, kept
// in megabytes on purpose — it is a question about disk space, not
// about quality, and it has to apply to a candidate whose bitrate
// cannot be worked out at all.
MaxFileSizeMB int `toml:"MaxFileSizeMB"`
// AllowedFormats restricts auto-pick to these formats. Empty means
// no restriction. Values are Format strings ("flac", "mp3", ...).
@@ -56,10 +72,11 @@ func (c *UserConfig) AutoDownloadPrefs() AutoDownloadPrefs {
}
return AutoDownloadPrefs{
MinSizeMB: c.MinFileSizeMB,
MaxSizeMB: c.MaxFileSizeMB,
PreferredSizeMB: c.PreferredFileSizeMB,
AllowedFormats: formats,
MinKbps: c.MinKbps,
MaxKbps: c.MaxKbps,
PreferredKbps: c.PreferredKbps,
MaxSizeMB: c.MaxFileSizeMB,
AllowedFormats: formats,
}
}
+8 -10
View File
@@ -236,6 +236,12 @@ func (m *Manager) AutoPickable(dl Download, ranked []Candidate) bool {
return AutoPickable(dl, ranked, m.preferences())
}
// AutoPickVeto wraps the package function the same way, and is what the
// request list quotes back to the user.
func (m *Manager) AutoPickVeto(dl Download, ranked []Candidate) string {
return AutoPickVeto(dl, ranked, m.preferences())
}
// Reload rebuilds every provider from stored config. Called at startup
// and after any provider settings change.
//
@@ -612,16 +618,8 @@ func (m *Manager) Attempt(
return false, "", err
}
if !m.AutoPickable(dl, ranked) {
best := ranked[0]
return false, fmt.Sprintf(
"best of %d found is not a confident enough match "+
"(match %.0f%%, quality %.0f%%)",
len(ranked),
best.Match.Overall*100, //nolint:mnd // percent
best.Quality.Overall*100,
), nil
if veto := m.AutoPickVeto(dl, ranked); veto != "" {
return false, veto, nil
}
if err := m.store.CreateDownload(ctx, dl); err != nil {
+59 -22
View File
@@ -199,28 +199,36 @@ func TestManagerEndToEndAutoPick(t *testing.T) {
t.Errorf("expected imported file at %s: %v", want, err)
}
// Staging was released only after a successful import.
entries, err := os.ReadDir(f.staging.Root())
if err != nil {
t.Fatalf("read staging root: %v", err)
}
// Staging release and the rescan happen *after* the state is
// recorded (manager.go sets StateComplete, then releases, then
// scans), so waiting on the state is not waiting on these. Under
// load the worker is descheduled in between and asserting straight
// away reads the world one step too early -- which is exactly how
// this test failed on a busy machine while passing alone.
waitFor(t, func() bool {
entries, err := os.ReadDir(f.staging.Root())
if err != nil || len(entries) != 0 {
return false
}
if len(entries) != 0 {
t.Errorf("staging not released: %d dirs remain", len(entries))
}
f.lib.mu.Lock()
defer f.lib.mu.Unlock()
// The library was told to rescan.
f.lib.mu.Lock()
scanned := len(f.lib.scanned)
f.lib.mu.Unlock()
if scanned != 1 {
t.Errorf("library scans = %d, want 1", scanned)
}
return len(f.lib.scanned) == 1
}, "staging was never released, or the library was never rescanned")
}
// An ambiguous result set must park for the user rather than guess.
func TestManagerWaitsWhenAmbiguous(t *testing.T) {
// Two equally good copies are not an ambiguity — they are a spare.
//
// This asserted the opposite for as long as auto-pick required 0.08 of
// daylight over the runner-up, and that rule was wrong in exactly the
// case it fired hardest: a popular album turns up several *correct*
// copies, all matching the tracklist, differing only in format and
// seeders. There is no question there about what to fetch, only about
// which copy, and the ranking already answers that — closest to the
// preferred bitrate first. A candidate does not have to be better than
// the field, only good enough on its own terms.
func TestManagerAutoPicksAmongEquallyGoodCopies(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
@@ -238,11 +246,41 @@ func TestManagerWaitsWhenAmbiguous(t *testing.T) {
t.Fatalf("Start: %v", err)
}
if f.manager.AutoPickable(dl, ranked) {
t.Fatal("two equivalent candidates must not auto-pick")
if veto := f.manager.AutoPickVeto(dl, ranked); veto != "" {
t.Fatalf("two equally good copies must auto-pick, got veto: %s", veto)
}
waitForDownloadState(t, f.store, dl.ID, StateComplete)
// Exactly one of them was fetched, not both.
if grabs := a.GrabCalls + b.GrabCalls; grabs != 1 {
t.Errorf("grabs = %d, want exactly 1", grabs)
}
}
// The user can still pick explicitly when auto-pick is not what
// happened — a candidate the ranking did not choose is still grabbable.
func TestManagerPickIsExplicit(t *testing.T) {
t.Parallel()
f := newManagerFixture(t)
a := fakeWithAlbum(1, "source-a", ".flac")
b := fakeWithAlbum(2, "source-b", ".flac")
f.manager.installProvider(Config{ID: 1, Priority: 50}, a)
f.manager.installProvider(Config{ID: 2, Priority: 50}, b)
// No tracklist: never auto-picks, so the result set parks for the
// user and Pick is the only way anything is fetched.
dl := fourTrackDownload()
dl.Expected = nil
ranked, err := f.manager.Start(context.Background(), dl)
if err != nil {
t.Fatalf("Start: %v", err)
}
// Nothing was grabbed while waiting for the user.
if a.GrabCalls != 0 || b.GrabCalls != 0 {
t.Errorf(
"grabs happened without a pick: a=%d b=%d",
@@ -259,7 +297,6 @@ func TestManagerWaitsWhenAmbiguous(t *testing.T) {
t.Errorf("stored request id = %s, want %s", stored.ID, dl.ID)
}
// The user picks the second one explicitly.
if err := f.manager.Pick(
context.Background(), dl.ID, ranked[1].ID,
); err != nil {
+313 -77
View File
@@ -1,6 +1,7 @@
package download
import (
"fmt"
"math"
"sort"
"strings"
@@ -34,38 +35,102 @@ const (
weightArtistFit = 0.12
)
// Quality sub-weights. They sum to 1.0 along with weightSizeFit below.
// Quality sub-weights. Each set sums to 1.0.
//
// There are two of them because a stated preference changes what the
// other numbers are *for*. `formatRank` and `bitrateScore` are the
// app guessing at how good a copy is — FLAC over MP3, 320 over 128 —
// and that guess exists precisely because the user has not said. Once
// they have, the guess should not outvote them: with the old single set
// a preference of 320 kbps moved a candidate's score by at most 0.05
// against the 0.42 riding on format, so asking for 320 and being handed
// a FLAC every time was the *designed* behaviour. That is the same
// fault the megabyte window had — a preference the user can express and
// the ranking can ignore.
const (
weightFormat = 0.42
weightBitrate = 0.23
weightHealth = 0.20
weightPriority = 0.10
weightSizeFit = 0.05
weightFormat = 0.42
weightBitrate = 0.23
weightHealth = 0.20
weightPriority = 0.10
weightBitrateFit = 0.05
)
// Quality sub-weights when the user has named a preferred bitrate.
// The weight comes off format and bitrate — the two proxies the
// preference replaces — and health and priority are untouched, since
// neither is a stand-in for anything the user just said.
const (
statedWeightFormat = 0.20
statedWeightBitrate = 0.10
statedWeightHealth = 0.20
statedWeightPriority = 0.10
statedWeightBitrateFit = 0.40
)
// qualityWeights picks the set, in the order scoreQuality applies them.
func qualityWeights(p AutoDownloadPrefs) (
format, bitrate, health, priority, fit float64,
) {
if p.PreferredKbps > 0 {
return statedWeightFormat,
statedWeightBitrate,
statedWeightHealth,
statedWeightPriority,
statedWeightBitrateFit
}
return weightFormat,
weightBitrate,
weightHealth,
weightPriority,
weightBitrateFit
}
// unanchoredCap bounds the match score of a free-text request. Without
// an MBID there is no tracklist to be right about, so a confident-
// looking score would be a lie — and auto-pick keys off this.
const unanchoredCap = 0.65
// AutoDownloadPrefs gates and scores what AutoPickable may choose
// without asking. Zero values are permissive: no size window and no
// format restriction.
// without asking. Zero values are permissive: no bitrate window, no
// size ceiling and no format restriction.
//
// **The window is a rate, not a size.** It used to be three numbers in
// megabytes, which cannot mean anything on their own: 300 MB is a
// generous FLAC single and a suspiciously small boxset, and the user
// setting the number has no idea which release the pipeline will
// eventually apply it to. A bitrate is the same statement normalised
// by how long the music is, so one number holds across a 9-minute EP
// and a 3-hour opera — and it is the unit the thing being described is
// actually measured in. The runtime is known for every request
// auto-pick can act on (`Download.Expected` carries per-track lengths,
// and an anchored request is the only kind that reaches here), so this
// costs no extra lookup.
type AutoDownloadPrefs struct {
// MinSizeMB and MaxSizeMB bound what auto-pick will grab. Zero
// means no bound on that side. A candidate outside the window is
// filtered out of auto-pick entirely, not merely scored down — a
// tiny "sampler" torrent or a boxset ten times the expected size is
// usually the wrong thing entirely, not a worse copy of the right
// thing.
MinSizeMB int `json:"minSizeMb"`
MaxSizeMB int `json:"maxSizeMb"`
// MinKbps and MaxKbps bound the average bitrate auto-pick will
// grab. Zero means no bound on that side. A candidate outside the
// window is filtered out of auto-pick entirely, not merely scored
// down — a 96 kbps rip of the right album is not a worse copy the
// user might accept, it is one they said not to take unattended.
//
// For reference: 320 is the top of MP3, ~5001000 is FLAC depending
// on the material, and anything under ~128 is a transcode.
MinKbps int `json:"minKbps"`
MaxKbps int `json:"maxKbps"`
// PreferredSizeMB nudges the score toward a target size within the
// min/max window (a lossless rip and a heavily-padded lossless rip
// can both pass the window). Zero disables the nudge; sizeFit then
// returns a neutral value that does not affect ranking.
PreferredSizeMB int `json:"preferredSizeMb"`
// PreferredKbps nudges the score toward a target rate within the
// window, and breaks the tie when several candidates are equally
// good matches. Zero disables the nudge; bitrateFit then returns a
// neutral value that does not affect ranking.
PreferredKbps int `json:"preferredKbps"`
// MaxSizeMB is a hard ceiling on the whole candidate, and it is
// deliberately still a size. It answers a different question from
// the window above — not "is this the quality I want" but "is this
// going to fill the disk" — and it has to hold even for a candidate
// whose bitrate cannot be worked out, which is exactly the shape a
// mislabelled boxset arrives in. Zero means no ceiling.
MaxSizeMB int `json:"maxSizeMb"`
// AllowedFormats restricts auto-pick to candidates whose audio
// files are all in one of these formats. Empty means no
@@ -74,19 +139,33 @@ type AutoDownloadPrefs struct {
}
// eligible reports whether a candidate may be auto-picked under these
// preferences: within the size window (when set) and, when a format
// list is given, every audio file in an allowed format.
func (p AutoDownloadPrefs) eligible(c Candidate) bool {
// preferences: inside the bitrate window and the size ceiling (when
// set) and, when a format list is given, every audio file in an
// allowed format.
//
// `runtimeMillis` is how long the requested release is, and 0 means
// nobody knows. An unknown runtime **passes** the bitrate window
// rather than failing it: the window is a statement about quality, and
// refusing everything the moment a tracklist is missing a length would
// turn a gap in MusicBrainz into a silent embargo. The size ceiling
// still applies, which is why it exists separately.
func (p AutoDownloadPrefs) eligible(c Candidate, runtimeMillis int64) bool {
const bytesPerMB = 1 << 20
if p.MinSizeMB > 0 && c.TotalSize < int64(p.MinSizeMB)*bytesPerMB {
return false
}
if p.MaxSizeMB > 0 && c.TotalSize > int64(p.MaxSizeMB)*bytesPerMB {
return false
}
if kbps := candidateKbps(c, runtimeMillis); kbps > 0 {
if p.MinKbps > 0 && kbps < float64(p.MinKbps) {
return false
}
if p.MaxKbps > 0 && kbps > float64(p.MaxKbps) {
return false
}
}
if len(p.AllowedFormats) == 0 {
return true
}
@@ -107,11 +186,14 @@ func (p AutoDownloadPrefs) eligible(c Candidate) bool {
// filter returns only the candidates these preferences allow to be
// auto-picked, in the same (already ranked) order.
func (p AutoDownloadPrefs) filter(ranked []Candidate) []Candidate {
func (p AutoDownloadPrefs) filter(
ranked []Candidate,
runtimeMillis int64,
) []Candidate {
out := make([]Candidate, 0, len(ranked))
for _, c := range ranked {
if p.eligible(c) {
if p.eligible(c, runtimeMillis) {
out = append(out, c)
}
}
@@ -119,32 +201,116 @@ func (p AutoDownloadPrefs) filter(ranked []Candidate) []Candidate {
return out
}
// sizeFit scores how close totalSize is to PreferredSizeMB, 0..1,
// falling off linearly as the size doubles or halves away from it.
// Returns a neutral 0.5 when no preference is set, so the absence of a
// preference does not bias ranking.
func (p AutoDownloadPrefs) sizeFit(totalSize int64) float64 {
// bitrateFit scores how close a candidate's average bitrate is to
// PreferredKbps, falling off linearly as it doubles or halves away
// from it.
//
// The range is **0.5 to 1.0, not 0 to 1**, and the floor is the point.
// This carries 0.40 of the quality score once a preference is set, so a
// span down to zero would let a preference of 320 kbps push a perfectly
// good FLAC under `minQuality` and out of auto-pick altogether —
// turning "I like 320" into "never take anything else", silently. A
// preference may promote the copy that matches it; it may not
// disqualify the others. That is what `MinKbps`/`MaxKbps` are for, and
// they say so out loud.
//
// Returns the neutral floor when no preference is set or the rate
// cannot be worked out, so neither an absent preference nor an absent
// runtime biases ranking.
func (p AutoDownloadPrefs) bitrateFit(
c Candidate,
runtimeMillis int64,
) float64 {
const (
bytesPerMB = 1 << 20
neutral = 0.5
neutral = 0.5
span = 0.5
)
if p.PreferredSizeMB <= 0 || totalSize <= 0 {
if p.PreferredKbps <= 0 {
return neutral
}
preferred := float64(p.PreferredSizeMB) * bytesPerMB
ratio := float64(totalSize) / preferred
kbps := candidateKbps(c, runtimeMillis)
if kbps <= 0 {
return neutral
}
ratio := kbps / float64(p.PreferredKbps)
if ratio < 1 {
ratio = 1 / ratio
}
// ratio is now >= 1: 1.0 is an exact match, 2.0 is double or half
// the preferred size. Falls to 0 at 2x away and beyond.
fit := 1 - (ratio - 1)
// the preferred rate, where the closeness term reaches 0.
return neutral + span*clamp01(1-(ratio-1))
}
return clamp01(fit)
// candidateKbps is a candidate's average audio bitrate, or 0 when it
// cannot be worked out.
//
// Two sources, in this order, and the order matters:
//
// - **Derived from bytes over runtime**, which is the honest one. It
// covers lossless (where a stated bitrate rarely exists), it cannot
// be lied to by a filename, and it is what the user's window means.
// Only the *audio* files count: cover scans and a log file are not
// part of the bitrate, and a folder with 30 MB of artwork would
// otherwise read as a better rip than the same music without it.
// - **The mean stated bitrate**, when the runtime is unknown. Weaker
// — a provider that parses it from an MP3 header states it and one
// that guesses from the filename also "states" it — but a number
// from the file itself beats no number at all.
func candidateKbps(c Candidate, runtimeMillis int64) float64 {
const bitsPerByte = 8
audio := c.AudioFiles()
if len(audio) == 0 {
return 0
}
if runtimeMillis > 0 {
var bytes int64
for _, f := range audio {
bytes += f.Size
}
if bytes > 0 {
// bytes×8 bits over seconds, expressed in kbps: the two
// factors of 1000 (millis→seconds, bits→kilobits) cancel.
return float64(bytes) * bitsPerByte /
float64(runtimeMillis)
}
}
var (
sum int
count int
)
for _, f := range audio {
if f.Bitrate > 0 {
sum += f.Bitrate
count++
}
}
if count == 0 {
return 0
}
return float64(sum) / float64(count)
}
// runtimeMillis is how long the requested release is, summed over its
// expected tracklist. Zero when the tracklist is absent or carries no
// lengths, which is what every caller here treats as "unknown".
func (d Download) runtimeMillis() int64 {
var total int64
for _, t := range d.Expected {
total += t.LengthMillis
}
return total
}
// Score fills a candidate's Match, Quality and Score fields.
@@ -160,7 +326,9 @@ func Score(dl Download, c Candidate, priority int, prefs AutoDownloadPrefs) Cand
c.Files = mergeMatched(c.Files, matched)
c.Match = scoreMatch(dl, c, audio, titleFit)
c.Quality = scoreQuality(c, audio, priority, prefs)
c.Quality = scoreQuality(
c, audio, priority, prefs, dl.runtimeMillis(),
)
c.Score = weightMatch*c.Match.Overall + weightQuality*c.Quality.Overall
@@ -279,11 +447,12 @@ func scoreQuality(
audio []CandidateFile,
priority int,
prefs AutoDownloadPrefs,
runtimeMillis int64,
) QualityScore {
q := QualityScore{
Health: clamp01(c.Health),
Priority: clamp01(float64(priority) / 100.0),
SizeFit: prefs.sizeFit(c.TotalSize),
Health: clamp01(c.Health),
Priority: clamp01(float64(priority) / 100.0),
BitrateFit: prefs.bitrateFit(c, runtimeMillis),
}
if len(audio) == 0 {
@@ -310,11 +479,13 @@ func scoreQuality(
q.FormatRank = worst
q.Bitrate = bitrateScore(audio)
q.Overall = weightFormat*q.FormatRank +
weightBitrate*q.Bitrate +
weightHealth*q.Health +
weightPriority*q.Priority +
weightSizeFit*q.SizeFit
wFormat, wBitrate, wHealth, wPriority, wFit := qualityWeights(prefs)
q.Overall = wFormat*q.FormatRank +
wBitrate*q.Bitrate +
wHealth*q.Health +
wPriority*q.Priority +
wFit*q.BitrateFit
if q.Mixed {
q.Overall *= 0.9
@@ -444,6 +615,19 @@ func Rank(
return out[i].Match.Overall > out[j].Match.Overall
}
// Closest to the preferred bitrate wins the tie.
//
// This is what decides which copy is taken now that auto-pick
// no longer requires the winner to be clear of the field: when
// several candidates are equally good matches of equal overall
// quality, the one the user said they wanted the shape of is
// the answer, ahead of provider priority. With no preference
// set every BitrateFit is the same neutral value and this
// falls through, exactly as before.
if out[i].Quality.BitrateFit != out[j].Quality.BitrateFit {
return out[i].Quality.BitrateFit > out[j].Quality.BitrateFit
}
if out[i].Quality.Priority != out[j].Quality.Priority {
return out[i].Quality.Priority > out[j].Quality.Priority
}
@@ -454,19 +638,58 @@ func Rank(
return out
}
// AutoPickable reports whether a ranked list has a clear enough winner
// to grab without asking. It demands an anchored request, a high match,
// decent quality, and daylight between first and second place — if two
// candidates are close, the choice is the user's.
func AutoPickable(dl Download, ranked []Candidate, prefs AutoDownloadPrefs) bool {
const (
minMatch = 0.85
minQuality = 0.5
minLead = 0.08
)
// Auto-pick gates. Named rather than inlined because AutoPickVeto
// reports which of them refused, and a number in a sentence the user
// reads should be the same number the decision used.
const (
minMatch = 0.85
minQuality = 0.5
)
if !dl.Anchored() || len(ranked) == 0 {
return false
// AutoPickable reports whether a ranked list has a candidate worth
// grabbing without asking: an anchored request with a tracklist behind
// it, and a candidate that clears the match and quality bars inside the
// user's guardrails.
//
// **It does not require the winner to be better than the runner-up.**
// It used to demand 0.08 of daylight on the combined score, which meant
// the check fired hardest in the case it was never written for: a
// popular album turns up five *correct* copies, all matching the
// tracklist at 95%+ and differing only in format and seeders, their
// scores land within a point of each other, and auto-pick refused
// forever on the grounds that the choice was the user's. It was not.
// There was no question about *what* to fetch, only about which copy —
// and abundance is the one condition under which that question matters
// least. A candidate does not need to be the best one, only one that
// meets the criteria; where several do, `Rank` puts the one closest to
// the preferred bitrate first.
func AutoPickable(dl Download, ranked []Candidate, prefs AutoDownloadPrefs) bool {
return AutoPickVeto(dl, ranked, prefs) == ""
}
// AutoPickVeto returns the reason auto-pick declined, or "" when it
// would go ahead.
//
// It exists because "it rejected all of them" was indistinguishable
// from "it found nothing good". The request list's message was built
// from `ranked[0]` — the best candidate *before* the size and format
// guardrails, and before the lead check — so a request refused because
// the user's maximum size excluded every copy, or because three equally
// good copies were found, reported "best of 12 found is not a confident
// enough match (match 96%, quality 88%)". Numbers that clear both
// thresholds, beside a refusal, is a message that teaches the user the
// matcher is broken. Each gate names itself now.
func AutoPickVeto(
dl Download,
ranked []Candidate,
prefs AutoDownloadPrefs,
) string {
if len(ranked) == 0 {
return "nothing found"
}
if !dl.Anchored() {
return "the request is free text, so there is no release to be right about"
}
// An anchor with no tracklist behind it is an anchor in name only:
@@ -474,29 +697,42 @@ func AutoPickable(dl Download, ranked []Candidate, prefs AutoDownloadPrefs) bool
// is exactly the evidence a wrong-album candidate also has. This
// matters most for the request list, where nobody is watching.
if len(dl.Expected) == 0 {
return false
return "no tracklist for this release is known yet, so a candidate cannot be checked against it"
}
// The guardrails apply before the match/quality/lead checks: a
// candidate outside the allowed size or format is not a worse
// choice, it is not a choice auto-pick may make at all, so it must
// not count as "the winner" nor as "second place" for the lead
// check below.
eligible := prefs.filter(ranked)
// The guardrails apply before the match and quality checks: a
// candidate outside the allowed bitrate, size or format is not a
// worse choice, it is not a choice auto-pick may make at all, so it
// must not count as "the winner" either.
eligible := prefs.filter(ranked, dl.runtimeMillis())
if len(eligible) == 0 {
return false
return fmt.Sprintf(
"all %d found are outside the auto-download bitrate, size or format limits",
len(ranked),
)
}
best := eligible[0]
if best.Match.Overall < minMatch || best.Quality.Overall < minQuality {
return false
if best.Match.Overall < minMatch {
return fmt.Sprintf(
"best of %d found matches this release only %.0f%% (needs %.0f%%)",
len(ranked),
best.Match.Overall*100, //nolint:mnd // percent
minMatch*100, //nolint:mnd // percent
)
}
if len(eligible) > 1 && best.Score-eligible[1].Score < minLead {
return false
if best.Quality.Overall < minQuality {
return fmt.Sprintf(
"best of %d found is the right release but scores %.0f%% on quality (needs %.0f%%)",
len(ranked),
best.Quality.Overall*100, //nolint:mnd // percent
minQuality*100, //nolint:mnd // percent
)
}
return true
return ""
}
// mergeMatched copies MatchedTo assignments from the audio-only slice
+360 -57
View File
@@ -1,6 +1,34 @@
package download
import "testing"
import (
"strings"
"testing"
)
// trackMillis is five minutes; okComputer's four of them make a
// twenty-minute release, which is what turns a candidate's byte count
// into a bitrate the assertions below can name.
const trackMillis = 5 * 60 * 1000
// okComputerRuntime is that release's runtime, for the helpers that
// need it directly.
const okComputerRuntime = 4 * trackMillis
// kbpsCandidate builds an annotated candidate whose audio adds up to
// the given average bitrate over okComputer's runtime.
func kbpsCandidate(id, ext string, kbps int) Candidate {
// bits = kbps × 1000 × (runtimeMillis / 1000), so the thousands
// cancel and the byte count is kbps × runtimeMillis / 8.
const bitsPerByte = 8
total := int64(kbps) * okComputerRuntime / bitsPerByte
c := candidateFor(id, allTitles(), ext, total/int64(len(allTitles())))
c.Files = AnnotateFiles(c.Files)
c.TotalSize = total
return c
}
// okComputer is the reference request used across ranking tests.
func okComputer() Download {
@@ -8,11 +36,15 @@ func okComputer() Download {
ReleaseMBID: "mbid-ok-computer",
Artist: "Radiohead",
Album: "OK Computer",
// Four five-minute tracks: twenty minutes, so a candidate's
// bitrate is a number these tests can state exactly. Without
// lengths there is no runtime and the bitrate window has
// nothing to divide by.
Expected: []ExpectedTrack{
{Position: 1, Title: "Airbag"},
{Position: 2, Title: "Paranoid Android"},
{Position: 3, Title: "Subterranean Homesick Alien"},
{Position: 4, Title: "Exit Music (For a Film)"},
{Position: 1, Title: "Airbag", LengthMillis: trackMillis},
{Position: 2, Title: "Paranoid Android", LengthMillis: trackMillis},
{Position: 3, Title: "Subterranean Homesick Alien", LengthMillis: trackMillis},
{Position: 4, Title: "Exit Music (For a Film)", LengthMillis: trackMillis},
},
}
}
@@ -187,7 +219,7 @@ func TestUnanchoredMatchIsCapped(t *testing.T) {
}
}
func TestAutoPickableRequiresAnchorAndLead(t *testing.T) {
func TestAutoPickableRequiresAnchorAndTracklist(t *testing.T) {
t.Parallel()
dl := okComputer()
@@ -211,14 +243,18 @@ func TestAutoPickableRequiresAnchorAndLead(t *testing.T) {
}
})
t.Run("two close candidates are not", func(t *testing.T) {
// Two identical copies are a spare, not an ambiguity. This
// asserted the opposite while auto-pick required daylight over the
// runner-up — a rule that made abundance the thing that stopped a
// request being satisfied, which is backwards.
t.Run("two equally good candidates still are", func(t *testing.T) {
t.Parallel()
twin := best
twin.ID = "twin"
if AutoPickable(dl, []Candidate{best, twin}, AutoDownloadPrefs{}) {
t.Error("identical candidates must not auto-pick")
if !AutoPickable(dl, []Candidate{best, twin}, AutoDownloadPrefs{}) {
t.Error("identical good candidates must auto-pick")
}
})
@@ -300,18 +336,11 @@ func TestProviderPriorityBreaksTies(t *testing.T) {
}
}
const mb = 1 << 20
func TestAutoDownloadPrefsEligible(t *testing.T) {
t.Parallel()
flacCandidate := candidateFor("c", allTitles(), ".flac", 30_000_000)
flacCandidate.Files = AnnotateFiles(flacCandidate.Files)
flacCandidate.TotalSize = 300 * mb
mp3Candidate := candidateFor("c", allTitles(), ".mp3", 3_000_000)
mp3Candidate.Files = AnnotateFiles(mp3Candidate.Files)
mp3Candidate.TotalSize = 30 * mb
flacCandidate := kbpsCandidate("c", ".flac", 900)
mp3Candidate := kbpsCandidate("c", ".mp3", 128)
tests := []struct {
name string
@@ -321,18 +350,25 @@ func TestAutoDownloadPrefsEligible(t *testing.T) {
}{
{"zero value is permissive", AutoDownloadPrefs{}, flacCandidate, true},
{
"within min/max window",
AutoDownloadPrefs{MinSizeMB: 100, MaxSizeMB: 500},
"within the bitrate window",
AutoDownloadPrefs{MinKbps: 320, MaxKbps: 1200},
flacCandidate, true,
},
{
"below minimum",
AutoDownloadPrefs{MinSizeMB: 400},
"below the minimum bitrate",
AutoDownloadPrefs{MinKbps: 500},
mp3Candidate, false,
},
{
"above the maximum bitrate",
AutoDownloadPrefs{MaxKbps: 500},
flacCandidate, false,
},
{
"above maximum",
AutoDownloadPrefs{MaxSizeMB: 200},
// The ceiling is bytes, not a rate, and it is the guard
// that still works when the bitrate cannot be worked out.
"above the hard size ceiling",
AutoDownloadPrefs{MaxSizeMB: 50},
flacCandidate, false,
},
{
@@ -351,57 +387,131 @@ func TestAutoDownloadPrefsEligible(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := tt.prefs.eligible(tt.c); got != tt.want {
got := tt.prefs.eligible(tt.c, okComputerRuntime)
if got != tt.want {
t.Errorf("eligible() = %v, want %v", got, tt.want)
}
})
}
}
// A release nobody knows the length of cannot be judged on bitrate, and
// the window must not become a silent embargo because MusicBrainz is
// missing a track length. The size ceiling still applies — that is why
// it is a separate field.
func TestBitrateWindowPassesAnUnknownRuntime(t *testing.T) {
t.Parallel()
c := kbpsCandidate("c", ".mp3", 128)
prefs := AutoDownloadPrefs{MinKbps: 900}
if !prefs.eligible(c, 0) {
t.Error("an unknown runtime must pass the bitrate window")
}
if prefs.eligible(c, okComputerRuntime) {
t.Error("a known runtime must still be judged")
}
ceiling := AutoDownloadPrefs{MaxSizeMB: 1}
if ceiling.eligible(c, 0) {
t.Error("the size ceiling must apply even with no runtime")
}
}
// Artwork is not part of the bitrate. A folder carrying 30 MB of
// scans would otherwise read as a better rip than the same music
// without them, which is backwards.
func TestBitrateIgnoresNonAudioFiles(t *testing.T) {
t.Parallel()
c := kbpsCandidate("c", ".mp3", 320)
bare := candidateKbps(c, okComputerRuntime)
c.Files = append(c.Files, CandidateFile{
Path: "Radiohead - OK Computer/cover.jpg",
Size: 30 << 20,
})
c.Files = AnnotateFiles(c.Files)
if got := candidateKbps(c, okComputerRuntime); got != bare {
t.Errorf("bitrate with artwork = %f, want %f", got, bare)
}
}
// Where no runtime is known, a stated per-file bitrate is better than
// no answer at all.
func TestBitrateFallsBackToTheStatedRate(t *testing.T) {
t.Parallel()
c := candidateFor("c", allTitles(), ".mp3", 3_000_000)
for i := range c.Files {
c.Files[i].Bitrate = 192
}
c.Files = AnnotateFiles(c.Files)
if got := candidateKbps(c, 0); got != 192 {
t.Errorf("stated bitrate = %f, want 192", got)
}
}
func TestAutoDownloadPrefsFilter(t *testing.T) {
t.Parallel()
small := candidateFor("small", allTitles(), ".flac", 10_000_000)
small.TotalSize = 50 * mb
lossy := kbpsCandidate("lossy", ".mp3", 128)
lossless := kbpsCandidate("lossless", ".flac", 900)
big := candidateFor("big", allTitles(), ".flac", 30_000_000)
big.TotalSize = 500 * mb
prefs := AutoDownloadPrefs{MinKbps: 500}
prefs := AutoDownloadPrefs{MinSizeMB: 100, MaxSizeMB: 600}
filtered := prefs.filter(
[]Candidate{lossy, lossless}, okComputerRuntime,
)
filtered := prefs.filter([]Candidate{small, big})
if len(filtered) != 1 || filtered[0].ID != "big" {
if len(filtered) != 1 || filtered[0].ID != "lossless" {
t.Errorf("filter() = %v, want only the in-window candidate", filtered)
}
}
func TestAutoDownloadPrefsSizeFit(t *testing.T) {
func TestAutoDownloadPrefsBitrateFit(t *testing.T) {
t.Parallel()
const neutral = 0.5
tests := []struct {
name string
prefs AutoDownloadPrefs
totalSize int64
want float64
name string
prefs AutoDownloadPrefs
c Candidate
want float64
}{
{"no preference is neutral", AutoDownloadPrefs{}, 300 * mb, neutral},
{
"no preference is neutral",
AutoDownloadPrefs{},
kbpsCandidate("c", ".flac", 900), neutral,
},
{
"exact match scores 1",
AutoDownloadPrefs{PreferredSizeMB: 300},
300 * mb, 1.0,
AutoDownloadPrefs{PreferredKbps: 320},
kbpsCandidate("c", ".mp3", 320), 1.0,
},
{
"double the preferred size scores 0",
AutoDownloadPrefs{PreferredSizeMB: 300},
600 * mb, 0.0,
// The floor is neutral, not zero: this term carries 0.40
// of the quality score once a preference is set, and a
// span to zero would let "I like 320" quietly disqualify
// every FLAC from auto-pick.
"double the preferred rate falls to the neutral floor",
AutoDownloadPrefs{PreferredKbps: 320},
kbpsCandidate("c", ".flac", 640), neutral,
},
{
"half the preferred size scores 0",
AutoDownloadPrefs{PreferredSizeMB: 300},
150 * mb, 0.0,
"half the preferred rate falls to the neutral floor",
AutoDownloadPrefs{PreferredKbps: 320},
kbpsCandidate("c", ".mp3", 160), neutral,
},
{
"an unknowable rate is neutral",
AutoDownloadPrefs{PreferredKbps: 320},
kbpsCandidate("c", ".mp3", 320), neutral,
},
}
@@ -409,30 +519,223 @@ func TestAutoDownloadPrefsSizeFit(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := tt.prefs.sizeFit(tt.totalSize); got != tt.want {
t.Errorf("sizeFit(%d) = %f, want %f", tt.totalSize, got, tt.want)
// The last case deliberately withholds the runtime.
runtime := int64(okComputerRuntime)
if tt.name == "an unknowable rate is neutral" {
runtime = 0
}
if got := tt.prefs.bitrateFit(tt.c, runtime); got != tt.want {
t.Errorf("bitrateFit() = %f, want %f", got, tt.want)
}
})
}
}
// An otherwise-perfect candidate must not auto-pick when it falls
// outside the configured size guard: the guardrail applies before the
// match/quality/lead checks, not as one more input averaged into them.
func TestAutoPickableRejectsCandidateOutsideSizeGuard(t *testing.T) {
// outside the configured guardrails: they apply before the match and
// quality checks, not as one more input averaged into them.
func TestAutoPickableRejectsCandidateOutsideTheGuardrails(t *testing.T) {
t.Parallel()
dl := okComputer()
best := Score(dl, candidateFor("a", allTitles(), ".flac", 30_000_000), 50, AutoDownloadPrefs{})
best.TotalSize = 500 * mb
best := Score(dl, kbpsCandidate("a", ".flac", 900), 50, AutoDownloadPrefs{})
if !AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{}) {
t.Fatal("expected this candidate to be auto-pickable with no guardrails")
}
tight := AutoDownloadPrefs{MinSizeMB: 10, MaxSizeMB: 100}
if AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{MaxKbps: 320}) {
t.Error("candidate above the bitrate window must not auto-pick")
}
if AutoPickable(dl, []Candidate{best}, tight) {
t.Error("candidate outside the size guard must not auto-pick")
if AutoPickable(dl, []Candidate{best}, AutoDownloadPrefs{MaxSizeMB: 1}) {
t.Error("candidate above the size ceiling must not auto-pick")
}
}
// The refusal has to name the gate that refused.
//
// Before AutoPickVeto, every one of these came back as the same
// sentence built from `ranked[0]` — the best candidate before the size
// and format guardrails — so a request refused because the user's size
// window excluded every copy reported a match and a quality that both
// cleared their thresholds. A refusal quoting numbers that pass is
// what made the matcher look broken from outside.
func TestAutoPickVetoNamesTheGate(t *testing.T) {
t.Parallel()
dl := okComputer()
best := Score(
dl,
candidateFor("a", allTitles(), ".flac", 30_000_000),
50,
AutoDownloadPrefs{},
)
// candidateFor sizes the files and leaves TotalSize at 0, which is
// what the guardrails read.
sized := func(c Candidate, total int64) Candidate {
c.TotalSize = total
return c
}
tests := []struct {
name string
dl Download
ranked []Candidate
prefs AutoDownloadPrefs
wantSub string
}{
{
name: "nothing found",
dl: dl,
ranked: nil,
wantSub: "nothing found",
},
{
name: "free text",
dl: Download{Artist: "Radiohead", Album: "OK Computer"},
ranked: []Candidate{best},
wantSub: "free text",
},
{
name: "no tracklist behind the anchor",
dl: Download{
ReleaseMBID: "mbid-ok-computer",
Artist: "Radiohead",
Album: "OK Computer",
},
ranked: []Candidate{best},
wantSub: "no tracklist",
},
{
// The candidate is 120 MB and the window tops out at 1 MB:
// the old message reported its match and quality instead.
name: "outside the size window",
dl: dl,
ranked: []Candidate{sized(best, 120<<20)},
prefs: AutoDownloadPrefs{MaxSizeMB: 1},
wantSub: "bitrate, size or format limits",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := AutoPickVeto(tt.dl, tt.ranked, tt.prefs)
if !strings.Contains(got, tt.wantSub) {
t.Errorf("veto = %q, want it to mention %q", got, tt.wantSub)
}
})
}
}
// A clear winner has no veto at all — the sentence is empty, which is
// what AutoPickable reads.
func TestAutoPickVetoIsEmptyForAClearWinner(t *testing.T) {
t.Parallel()
dl := okComputer()
best := Score(
dl,
candidateFor("a", allTitles(), ".flac", 30_000_000),
50,
AutoDownloadPrefs{},
)
weak := Score(
dl,
candidateFor("b", allTitles()[:2], ".mp3", 1_000_000),
50,
AutoDownloadPrefs{},
)
if got := AutoPickVeto(dl, []Candidate{best, weak}, AutoDownloadPrefs{}); got != "" {
t.Errorf("veto = %q, want none", got)
}
}
// With several candidates that all clear the bar, the preferred
// bitrate decides which one is taken.
//
// This is what replaced the daylight requirement. Auto-pick no longer
// refuses when the field is close; it takes the copy nearest the shape
// the user asked for, which is the question they actually answered in
// Settings.
func TestPreferredBitrateBreaksTheTie(t *testing.T) {
t.Parallel()
dl := okComputer()
prefs := AutoDownloadPrefs{PreferredKbps: 320}
// Same album, same completeness, same health, same provider — the
// only difference between them is the rate.
lossless := kbpsCandidate("lossless", ".flac", 900)
perfect := kbpsCandidate("perfect", ".mp3", 320)
ranked := Rank(
dl, []Candidate{lossless, perfect}, nil, prefs,
)
if ranked[0].ID != "perfect" {
t.Errorf(
"winner = %q (fit %f) over %q (fit %f), want the 320 kbps copy",
ranked[0].ID, ranked[0].Quality.BitrateFit,
ranked[1].ID, ranked[1].Quality.BitrateFit,
)
}
if AutoPickVeto(dl, ranked, prefs) != "" {
t.Error("a close field must still auto-pick")
}
}
// With no preference set, nothing changes: BitrateFit is the same
// neutral value for every candidate and the older tie-breaks decide.
func TestNoPreferredBitrateLeavesRankingAlone(t *testing.T) {
t.Parallel()
dl := okComputer()
lossless := kbpsCandidate("lossless", ".flac", 900)
lossy := kbpsCandidate("lossy", ".mp3", 320)
ranked := Rank(
dl, []Candidate{lossy, lossless}, nil, AutoDownloadPrefs{},
)
if ranked[0].ID != "lossless" {
t.Errorf(
"winner = %q, want the lossless copy on format alone",
ranked[0].ID,
)
}
}
// A preferred bitrate promotes the copy that matches it and must never
// disqualify the ones that do not. It carries 0.40 of the quality
// score, so a fit spanning down to zero would put a perfectly good FLAC
// under minQuality and out of auto-pick — turning a preference into a
// prohibition without saying so. MinKbps and MaxKbps are how a user
// says that on purpose.
func TestAPreferredBitrateNeverDisqualifies(t *testing.T) {
t.Parallel()
dl := okComputer()
far := AutoDownloadPrefs{PreferredKbps: 128}
lossless := Score(dl, kbpsCandidate("flac", ".flac", 900), 50, far)
if lossless.Quality.Overall < minQuality {
t.Errorf(
"quality = %f under a far-off preference, want >= %f",
lossless.Quality.Overall, minQuality,
)
}
if veto := AutoPickVeto(dl, []Candidate{lossless}, far); veto != "" {
t.Errorf("a far-off preference vetoed the candidate: %s", veto)
}
}
+26 -6
View File
@@ -310,15 +310,35 @@ func (r *Reconciler) run(ctx context.Context, force bool) (Summary, error) {
summary.Synced = r.syncExternalLists(ctx)
attempted, started, err := r.attemptDue(ctx, force)
if err != nil {
return summary, err
// Nothing is searched for when there is nothing to search with, and
// the point is what that *does not* do to the list.
//
// Attempting anyway is not merely wasted work: every request comes
// back "no download clients are enabled", which RecordAttempt writes
// down as an attempt and schedules a retry for -- so a user who has
// deliberately built a wanted list with no client watched their
// requests accrue failures and announce "next check in 6 hours"
// about a check that cannot happen. Wanting something without a way
// to fetch it is a supported thing to do; being told it is being
// looked for is a lie.
//
// Everything above this line still runs: an artist subscription
// still expands, and a request the user satisfied by some other
// route -- ripped, bought, copied in -- is still retired, because
// neither needs a provider.
summary.NoProviders = len(r.manager.enabledProviders()) == 0
if !summary.NoProviders {
attempted, started, err := r.attemptDue(ctx, force)
if err != nil {
return summary, err
}
summary.Attempted = attempted
summary.Started = started
}
summary.Attempted = attempted
summary.Started = started
summary.Waiting = r.countWaiting(ctx)
summary.NoProviders = len(r.manager.enabledProviders()) == 0
r.logger.Info(
"reconciled request list",
+78
View File
@@ -453,6 +453,15 @@ func TestReconcileRespectsBatchSize(t *testing.T) {
f := newReconcileFixture(t)
ctx := context.Background()
// A client that searches and finds nothing. The batch size is about
// how many requests one pass *searches for*, which only means
// anything when there is something to search with -- a pass with no
// provider now attempts nothing at all, deliberately.
f.manager.installProvider(
Config{ID: 1, Priority: 50},
NewFakeProvider(1, "finds-nothing", Caps{CanSearch: true}),
)
f.reconciler.SetBatch(2)
for _, mbid := range []string{"rg-1", "rg-2", "rg-3", "rg-4"} {
@@ -593,3 +602,72 @@ func TestSummaryReportsNoProviders(t *testing.T) {
t.Error("summary did not report that no download client is enabled")
}
}
// ...and it does not search, which is the part the user sees.
//
// Attempting with no provider fails every request with "no download
// clients are enabled", and RecordAttempt writes that down as an
// attempt and schedules a retry -- so a wanted list built deliberately
// without a client accrued failures and announced "next check in 6
// hours" about a check that cannot happen. Wanting something with no
// way to fetch it is supported; being told it is being looked for is
// a lie.
func TestNoProvidersMeansNoAttempt(t *testing.T) {
t.Parallel()
f := newReconcileFixture(t)
ctx := context.Background()
id, err := f.store.AddRequest(ctx, Request{
MBID: "rg-1",
Entity: EntityReleaseGroup,
LibraryID: 1,
Title: "OK Computer",
})
if err != nil {
t.Fatalf("AddRequest: %v", err)
}
f.catalog.tracklists["rg-1"] = fourTrackDownload().Expected
summary, err := f.reconciler.RunNow(ctx)
if err != nil {
t.Fatalf("RunNow: %v", err)
}
if summary.Attempted != 0 {
t.Errorf("attempted %d requests with no client to search with, want 0",
summary.Attempted)
}
// The list still knows what is on it: "nothing happened" has to be
// reportable as "nothing was searched for, of the one thing you
// want" rather than as silence.
if summary.Waiting != 1 {
t.Errorf("summary reported %d waiting, want 1", summary.Waiting)
}
req, err := f.store.GetRequest(ctx, id)
if err != nil {
t.Fatalf("GetRequest: %v", err)
}
if req.Attempts != 0 {
t.Errorf("attempts = %d, want 0: a pass that could not search did not",
req.Attempts)
}
if req.LastError != "" {
t.Errorf("lastError = %q, want empty: the request did not fail, it "+
"was never tried", req.LastError)
}
// A new request is due immediately (next_try_at is set to now on
// insert), so the fault is not the presence of a time -- it is a
// time pushed into the future by a failed attempt, which is what the
// UI renders as "next check in 6 hours".
if req.NextTryAt.After(time.Now().Add(time.Minute)) {
t.Errorf("next try scheduled for %v: a check that cannot happen was "+
"put on the clock", req.NextTryAt)
}
}
+40
View File
@@ -20,6 +20,41 @@ func newServiceFixture(t *testing.T) serviceFixture {
mf := newManagerFixture(t)
svc := NewService(slogDiscard(), mf.manager, mf.store, NewMemSecretStore())
// Every test here is about the durable Request that `StartDownload`
// leaves behind, and none of them is about the download itself -- but
// their fixture is an anchored four-track request with a healthy
// provider, which is exactly what `AutoPickable` says yes to. So
// `Manager.Start` was firing `go m.grab(...)`, detached and with
// `context.WithoutCancel`, and the test then raced it.
//
// It lost, twice, in CI (`check` on c03c0b8, and nowhere locally):
//
// service_test.go:66: state = "satisfied", want wanted
// testing.go:1369: TempDir RemoveAll cleanup: ... directory not empty
//
// The first is the request reaching its *next* state before the
// assertion read it; the second is that same goroutine still writing
// into `t.TempDir()` after the test returned. One cause, two shapes.
//
// Putting the candidate outside the auto-pick guardrails stops the
// grab from ever starting, which is better than waiting for it: there
// is no goroutine to be slow, so the tests state what they mean
// ("the request exists, in this state") without a timing assumption
// underneath. A test that does want the download has `managerFixture`
// and sets its own preferences.
//
// The guard is a *format* the fake never produces, and it used to be
// `MaxSizeMB: 1`, which never fired: the size gates read
// `Candidate.TotalSize`, which real providers fill and the fake
// leaves at zero, and zero is under every ceiling. So the grab went
// ahead anyway and the second failure shape above — the TempDir
// cleanup race — kept happening, reproducibly, roughly one run in
// fifteen. A guard has to be keyed on something the fixture
// actually sets.
mf.manager.SetPreferences(AutoDownloadPrefs{
AllowedFormats: []Format{FormatWMA},
})
return serviceFixture{managerFixture: mf, svc: svc}
}
@@ -158,6 +193,11 @@ func TestManualDownloadSatisfiesRequestOnSuccess(t *testing.T) {
f := newServiceFixture(t)
ctx := context.Background()
// This is the one test here that is *about* the download, so it
// undoes the fixture's guard rather than relying on it — which is
// what it was doing implicitly while the guard did not work.
f.manager.SetPreferences(AutoDownloadPrefs{})
provider := fakeWithAlbum(1, "source", ".flac")
f.manager.installProvider(Config{ID: 1, Priority: 50}, provider)
+6 -1
View File
@@ -302,7 +302,12 @@ type QualityScore struct {
Bitrate float64 `json:"bitrate"`
Health float64 `json:"health"` // seeders, free slots
Priority float64 `json:"priority"` // user's per-provider preference
SizeFit float64 `json:"sizeFit"` // closeness to the preferred download size
// BitrateFit is closeness to the preferred *rate*, which is what
// the auto-download window is expressed in. It replaced a
// `SizeFit` measured in megabytes: a size means nothing without
// knowing how long the music is, so the same number described a
// generous single and a suspiciously small boxset.
BitrateFit float64 `json:"bitrateFit"`
// Mixed marks a candidate whose files are not all the same format,
// which usually means a hand-assembled folder rather than a rip.
+75
View File
@@ -283,6 +283,9 @@ func (si *SearchIndex) importCoreArtifact(ctx context.Context, path string) erro
}
merged, mergeErr := si.mergeArtifactRows(ctx, info.rows)
if mergeErr == nil {
si.mergeArtifactCredits(ctx)
}
if ftsSuspended {
start := time.Now()
@@ -473,3 +476,75 @@ func (si *SearchIndex) removeArtifactFile(path string) {
si.logger.Warn("core artifact: cleanup failed", "path", path, "error", err)
}
}
// artifactHasCredits reports whether the attached artifact carries the
// multi-artist credit tables.
//
// The same shape, and the same handle, as artifactHasTotals above: an
// artifact published before credits existed is still a perfectly good
// catalog, and there is one already out there. Selecting from a table
// that is not in it would fail an import that should have succeeded, so
// it is asked rather than assumed -- on the *writer*, because `core` is
// attached to that one connection and the read pool cannot see it.
func (si *SearchIndex) artifactHasCredits() bool {
var n int
err := si.db.QueryRowWriter(
`SELECT COUNT(*) FROM core.sqlite_master
WHERE type = 'table' AND name IN ('artist_credit_part', 'artist_credit_ref')`,
).Scan(&n)
return err == nil && n == 2
}
// mergeArtifactCredits copies the credit decomposition out of the
// attached artifact.
//
// Credits are replaced wholesale rather than merged: they are derived
// entirely from one dump build, they are keyed by ids that are only
// meaningful within the artifact that carried them, and a half-updated
// credit renders as the wrong artists rather than as missing ones.
//
// A failure here is logged and not returned. The catalog has already
// merged at this point, and a catalog without credits is the catalog
// this app had before them -- every credit falls back to its single
// artist, which is the same fallback an untagged file already gets.
func (si *SearchIndex) mergeArtifactCredits(ctx context.Context) {
if !si.artifactHasCredits() {
si.logger.Info("core artifact: no credit tables, keeping single-artist credits")
return
}
start := time.Now()
for _, stmt := range []string{
"DELETE FROM artist_credit_part",
"DELETE FROM artist_credit_ref",
`INSERT OR REPLACE INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
SELECT credit_id, position, artist_mbid, credited_name, join_phrase
FROM core.artist_credit_part`,
`INSERT OR REPLACE INTO artist_credit_ref (mbid, credit_id)
SELECT mbid, credit_id FROM core.artist_credit_ref`,
} {
if err := ctx.Err(); err != nil {
return
}
if _, err := si.db.ExecContext(stmt); err != nil {
si.logger.Warn("core artifact: credit merge failed", "error", err)
return
}
}
var refs int
_ = si.db.QueryRowWriter("SELECT COUNT(*) FROM artist_credit_ref").Scan(&refs)
si.logger.Info("core artifact: credits merged",
"entities", refs,
"elapsed", time.Since(start).Round(time.Millisecond),
)
}
+151
View File
@@ -3,6 +3,7 @@ package explore
import (
"context"
"database/sql"
"encoding/hex"
"os"
"path/filepath"
"strings"
@@ -597,3 +598,153 @@ func TestImportCoreArtifactReadsTotalsWhenPresent(t *testing.T) {
t.Errorf("TotalTracks = %d, want 0 (the catalog does not say)", old.TotalTracks)
}
}
// addArtifactCredits gives an artifact file the credit tables the
// exporter now writes, so the import path can be exercised against one
// that has them.
func addArtifactCredits(t *testing.T, path string) {
t.Helper()
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatalf("open artifact: %v", err)
}
defer func() { _ = db.Close() }()
for _, stmt := range []string{
`CREATE TABLE artist_credit_part (
credit_id INTEGER NOT NULL,
position INTEGER NOT NULL,
artist_mbid BLOB NOT NULL,
credited_name TEXT NOT NULL,
join_phrase TEXT NOT NULL DEFAULT '',
PRIMARY KEY (credit_id, position)
) WITHOUT ROWID`,
`CREATE TABLE artist_credit_ref (
mbid BLOB NOT NULL PRIMARY KEY,
credit_id INTEGER NOT NULL
) WITHOUT ROWID`,
} {
if _, err := db.Exec(stmt); err != nil {
t.Fatalf("create credit tables: %v", err)
}
}
// The packed form the catalog stores. uuid16/parseUUID live behind
// the indexbuild tag, so this file decodes for itself.
pack := func(mbid string) []byte {
raw, err := hex.DecodeString(strings.ReplaceAll(mbid, "-", ""))
if err != nil || len(raw) != 16 {
t.Fatalf("fixture MBID %q is not a UUID: %v", mbid, err)
}
return raw
}
a, b, rec := pack(artA), pack(artB), pack(recA)
for _, part := range [][]any{
{7, 0, a, "Artist A", " feat. "},
{7, 1, b, "Artist B", ""},
} {
if _, err := db.Exec(`INSERT INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
VALUES (?, ?, ?, ?, ?)`, part...); err != nil {
t.Fatalf("insert part: %v", err)
}
}
if _, err := db.Exec(
"INSERT INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)", rec, 7,
); err != nil {
t.Fatalf("insert ref: %v", err)
}
}
// TestImportCoreArtifactMergesCredits is the positive half of the
// compatibility pair: an artifact that carries credits delivers them,
// rendering back to the credit string they decompose.
func TestImportCoreArtifactMergesCredits(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
path := writeTestArtifact(t, validMeta(), []artifactRow{
{"recording", recA, "Song A", "Artist A feat. Artist B", artA, 2000},
})
addArtifactCredits(t, path)
if err := si.importCoreArtifact(context.Background(), path); err != nil {
t.Fatalf("importCoreArtifact: %v", err)
}
rows, err := db.QueryContext(
`SELECT p.credited_name, p.join_phrase
FROM artist_credit_ref r
JOIN artist_credit_part p ON p.credit_id = r.credit_id
ORDER BY p.position`,
)
if err != nil {
t.Fatalf("query credits: %v", err)
}
defer func() { _ = rows.Close() }()
var rendered strings.Builder
for rows.Next() {
var name, join string
if err := rows.Scan(&name, &join); err != nil {
t.Fatalf("scan: %v", err)
}
rendered.WriteString(name)
rendered.WriteString(join)
}
if got := rendered.String(); got != "Artist A feat. Artist B" {
t.Errorf("rendered credit = %q, want %q", got, "Artist A feat. Artist B")
}
}
// TestImportCoreArtifactWithoutCredits is the regression that matters
// most here: an artifact published before credits existed cannot be
// re-cut retroactively, so it must import as a catalog that declines to
// answer rather than failing outright. writeTestArtifact deliberately
// builds one without the tables.
func TestImportCoreArtifactWithoutCredits(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
path := writeTestArtifact(t, validMeta(), []artifactRow{
{"recording", recA, "Song A", "Artist A", artA, 2000},
})
if err := si.importCoreArtifact(context.Background(), path); err != nil {
t.Fatalf("an artifact without credit tables must still import: %v", err)
}
var rows int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM explore_index",
).Scan(&rows); err != nil {
t.Fatalf("count: %v", err)
}
if rows != 1 {
t.Errorf("catalog rows = %d, want 1", rows)
}
var refs int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_credit_ref",
).Scan(&refs); err != nil {
t.Fatalf("count refs: %v", err)
}
if refs != 0 {
t.Errorf("credit refs = %d, want 0", refs)
}
}
+25 -6
View File
@@ -46,23 +46,42 @@ func TestCacheMiss(t *testing.T) {
}
}
// TestCacheTTLExpiry checks both halves of the TTL contract, and uses two
// entries to do it.
//
// **No assertion here may depend on an upper bound of elapsed wall-clock
// time**, which is what the single-entry version of this test did: it set
// a 1s TTL and immediately asserted a *hit*, so on a loaded runner — one
// goroutine descheduled for over a second while the rest of the suite
// runs — the entry was correctly gone and the test failed with "expected
// cache hit immediately after set". It did exactly that in CI while
// passing five times out of five locally.
//
// Sleeping *past* a TTL is always safe, so the expiry half keeps a short
// one; the presence half gets a TTL nothing can outrun.
func TestCacheTTLExpiry(t *testing.T) {
c := newTestCache(t)
data := []byte(`{"ephemeral":true}`)
c.Set("ttl-test-key", data, 1*time.Second, "", "")
c.Set("ttl-live-key", data, time.Hour, "", "")
c.Set("ttl-expiring-key", data, 1*time.Second, "", "")
// Verify it's there immediately.
if _, ok := c.Get("ttl-test-key"); !ok {
t.Fatal("expected cache hit immediately after set")
if _, ok := c.Get("ttl-live-key"); !ok {
t.Fatal("expected a cache hit on an entry with an hour to live")
}
// Wait for expiry.
// Wait for the short one to expire.
time.Sleep(2 * time.Second)
if _, ok := c.Get("ttl-test-key"); ok {
if _, ok := c.Get("ttl-expiring-key"); ok {
t.Error("expected cache miss after TTL expiry, got hit")
}
// And the long-lived entry is still there, which is what says the
// sweep above expired an entry rather than the cache.
if _, ok := c.Get("ttl-live-key"); !ok {
t.Error("the hour-long entry expired too")
}
}
func TestCacheMBID(t *testing.T) {
+49 -5
View File
@@ -25,8 +25,26 @@ const (
// where cached cover art thumbnails are stored.
thumbnailDir = CoverArtCacheDirName
// thumbnailTimeout is the HTTP timeout for fetching a thumbnail.
thumbnailTimeout = 10 * time.Second
// thumbnailTimeout is the HTTP timeout for fetching a thumbnail,
// and it has to cover a redirect the Cover Art Archive does not
// serve itself.
//
// `coverartarchive.org` answers `front-250` with a 307 to an
// Internet Archive storage node (`dn######.us.archive.org`), and
// those nodes are routinely slow: measured against the twelve
// albums on Explore's own shelves, a successful fetch took 1416 s
// and a failing one 1317 s. At 10 s *every* cover on the page
// timed out — 24 cards, 5 of which had art, all of those from the
// disk cache — which reads as "Explore has no album art" rather
// than as a slow upstream, because a timeout writes nothing and
// says nothing.
//
// 30 s is chosen to clear that measured range with room, not to be
// generous: the fetch is off the critical path (each one is its own
// goroutine behind an 8/s limiter, and the frontend renders a
// placeholder until it lands), so the cost of waiting is nothing
// and the cost of giving up early is a blank page.
thumbnailTimeout = 30 * time.Second
// thumbnailMaxSize is the maximum image size to cache (2 MB).
thumbnailMaxSize = 2 * 1024 * 1024
@@ -97,6 +115,20 @@ func (p *CoverArtProxy) GetThumbnail(
return ""
}
// A 404 is an answer, and it is already on disk.
//
// `writeCache(mbid, nil)` has recorded "the archive has no art for
// this" as an empty file since this was written, and nothing has
// ever read it back: `readCache` returns "" for an empty file,
// which is indistinguishable from a miss, so every art-less release
// group was re-fetched from the network on every render that asked
// about it. On Explore's shelves a third of the cards are art-less,
// so that was a third of the page spending a live CAA request to be
// told again what the last one said.
if p.knownMissing(releaseGroupMBID) {
return ""
}
// Source 3: fetch from Cover Art Archive (slow, cached to disk).
url := CoverArtGroupURL(releaseGroupMBID)
data, cacheable, err := p.fetch(url)
@@ -177,8 +209,9 @@ func (p *CoverArtProxy) GetCandidateThumbnail(
}
}
// Network fetch on release group.
if releaseGroupMBID != "" {
// Network fetch on release group — unless a previous one was told
// there is none. See `knownMissing`.
if releaseGroupMBID != "" && !p.knownMissing(releaseGroupMBID) {
url := CoverArtGroupURL(releaseGroupMBID)
data, cacheable, err := p.fetch(url)
@@ -194,7 +227,7 @@ func (p *CoverArtProxy) GetCandidateThumbnail(
}
// Network fetch on release (fallback).
if releaseMBID != "" {
if releaseMBID != "" && !p.knownMissing(releaseMBID) {
url := CoverArtURL(releaseMBID)
data, cacheable, err := p.fetch(url)
@@ -285,6 +318,17 @@ func (p *CoverArtProxy) cachePath(mbid string) string {
return filepath.Join(p.cacheDir, mbid+".jpg")
}
// knownMissing reports whether a previous fetch was told the archive
// has no art for this MBID — the empty file `writeCache(mbid, nil)`
// leaves behind. It is deliberately separate from `readCache`, which
// answers "what are the bytes" and cannot express the difference
// between no answer and an answer of none.
func (p *CoverArtProxy) knownMissing(mbid string) bool {
info, err := os.Stat(p.cachePath(mbid))
return err == nil && info.Size() == 0
}
func (p *CoverArtProxy) readCache(mbid string) string {
path := p.cachePath(mbid)
+132
View File
@@ -0,0 +1,132 @@
package explore
import (
"fmt"
"strings"
)
// Reading multi-artist credits back out of the catalog.
//
// The tables are filled centrally (backend/explore/dumpcredits.go, and
// the artifact import) and hold only credits naming more than one
// artist: an entity with no rows here is credited to one artist, which
// explore_index's own artist_name and artist_mbid already describe.
// Absence is the common case and means "nothing to decompose", never
// "unknown".
//
// The lookup is keyed on the *recording* MBID, which both sides of the
// app already have -- a catalog row carries it and so does a local
// file (library.Track.RecordingMBID) -- so one query serves the Explore
// pages and the library's own lists without either needing to know
// where the other gets its rows.
// CreditPart is one credited artist within a credit, in credit order.
//
// CreditedName is the name *as credited*, which is not the artist's own
// name: MusicBrainz credits "Snoop Dogg" on a track by the artist
// called "Snoop Doggy Dogg". Display uses it; navigation uses
// ArtistMBID. JoinPhrase is the literal connector that follows this
// part, so a credit renders by concatenation and never by searching a
// name inside a credit string.
type CreditPart struct {
Position int `json:"position"`
ArtistMBID string `json:"artistMbid"`
CreditedName string `json:"creditedName"`
JoinPhrase string `json:"joinPhrase"`
}
// creditLookupBatch bounds how many MBIDs go into one IN clause. A
// tracklist is the caller here, so the realistic ceiling is a few
// hundred; the bound exists so a 50,000-row selection cannot build a
// statement SQLite refuses to parse.
const creditLookupBatch = 500
// GetCredits returns the decomposition of every multi-artist credit
// among the given entity MBIDs, keyed by MBID.
//
// MBIDs with a single-artist credit are simply absent from the result,
// which is what the caller wants: it renders its existing single link
// for those, and that is the same answer it would have rendered anyway.
func (si *SearchIndex) GetCredits(mbids []string) (map[string][]CreditPart, error) {
out := make(map[string][]CreditPart)
for start := 0; start < len(mbids); start += creditLookupBatch {
end := min(start+creditLookupBatch, len(mbids))
if err := si.appendCredits(mbids[start:end], out); err != nil {
return nil, err
}
}
return out, nil
}
// appendCredits runs one batch into the accumulating result.
func (si *SearchIndex) appendCredits(
mbids []string, out map[string][]CreditPart,
) error {
args := make([]any, 0, len(mbids))
holders := make([]string, 0, len(mbids))
for _, mbid := range mbids {
if mbid == "" {
continue
}
args = append(args, dbMBID(mbid))
holders = append(holders, "?")
}
if len(args) == 0 {
return nil
}
// Ordered by position because that ordering *is* the credit's
// meaning; the caller concatenates in the order it receives.
rows, err := si.db.QueryContext(
`SELECT r.mbid, p.position, p.artist_mbid, p.credited_name, p.join_phrase
FROM artist_credit_ref r
JOIN artist_credit_part p ON p.credit_id = r.credit_id
WHERE r.mbid IN (`+strings.Join(holders, ",")+`)
ORDER BY r.mbid, p.position`,
args...,
)
if err != nil {
return fmt.Errorf("read artist credits: %w", err)
}
defer func() { _ = rows.Close() }()
for rows.Next() {
var (
entity dbMBID
artist dbMBID
part CreditPart
)
if err := rows.Scan(
&entity, &part.Position, &artist, &part.CreditedName, &part.JoinPhrase,
); err != nil {
return fmt.Errorf("scan artist credit: %w", err)
}
part.ArtistMBID = string(artist)
out[string(entity)] = append(out[string(entity)], part)
}
if err := rows.Err(); err != nil {
return fmt.Errorf("read artist credits: %w", err)
}
return nil
}
// GetCredits is the bound form: the frontend asks for a tracklist's
// worth of MBIDs at once rather than one per row.
//
// Batched for the reason every other per-row backend question here is:
// asking on hover or on render turns a list into N IPC round trips, and
// this one is asked about every row of every list in the app.
func (e *Service) GetCredits(mbids []string) (map[string][]CreditPart, error) {
return e.index.GetCredits(mbids)
}
+128
View File
@@ -0,0 +1,128 @@
package explore
import (
"encoding/hex"
"fmt"
"strings"
"testing"
"yellowjacket/backend/database"
)
// seedCredit writes one multi-artist credit and points an entity at it,
// the way the dump import and the artifact import both do.
func seedCredit(t *testing.T, db *database.DB, entity string, id int, parts []CreditPart) {
t.Helper()
pack := func(mbid string) []byte {
raw, err := hex.DecodeString(strings.ReplaceAll(mbid, "-", ""))
if err != nil || len(raw) != 16 {
t.Fatalf("bad fixture mbid %q: %v", mbid, err)
}
return raw
}
if _, err := db.ExecContext(
"INSERT INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)",
pack(entity), id,
); err != nil {
t.Fatalf("seed ref: %v", err)
}
for _, p := range parts {
if _, err := db.ExecContext(
`INSERT INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
VALUES (?, ?, ?, ?, ?)`,
id, p.Position, pack(p.ArtistMBID), p.CreditedName, p.JoinPhrase,
); err != nil {
t.Fatalf("seed part: %v", err)
}
}
}
// TestGetCreditsDecomposes: the parts come back in position order and
// concatenate to the credit they describe.
func TestGetCreditsDecomposes(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
rec := testMBID("rec-1")
a, b := testMBID("artist-a"), testMBID("artist-b")
seedCredit(t, db, rec, 7, []CreditPart{
{Position: 0, ArtistMBID: a, CreditedName: "2Pac", JoinPhrase: " feat. "},
{Position: 1, ArtistMBID: b, CreditedName: "Snoop Dogg"},
})
got, err := si.GetCredits([]string{rec})
if err != nil {
t.Fatalf("GetCredits: %v", err)
}
parts := got[rec]
if len(parts) != 2 {
t.Fatalf("parts = %d, want 2", len(parts))
}
var rendered strings.Builder
for _, p := range parts {
rendered.WriteString(p.CreditedName)
rendered.WriteString(p.JoinPhrase)
}
if rendered.String() != "2Pac feat. Snoop Dogg" {
t.Errorf("rendered = %q, want %q", rendered.String(), "2Pac feat. Snoop Dogg")
}
// Dashed on the way out: a blob reaching the frontend is sixteen
// bytes of mojibake, and nothing above mbid.go speaks that.
if parts[0].ArtistMBID != a {
t.Errorf("artist mbid = %q, want %q", parts[0].ArtistMBID, a)
}
}
// TestGetCreditsOmitsSingleArtist: absence is the common case and means
// "nothing to decompose", so the caller renders its existing one link.
func TestGetCreditsOmitsSingleArtist(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
got, err := si.GetCredits([]string{testMBID("untagged"), ""})
if err != nil {
t.Fatalf("GetCredits: %v", err)
}
if len(got) != 0 {
t.Errorf("got %d credits, want none", len(got))
}
}
// TestGetCreditsBatches: the lookup is asked about whole tracklists, so
// it must not build one statement per row or one SQLite refuses to
// parse.
func TestGetCreditsBatches(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
mbids := make([]string, 0, creditLookupBatch*2+7)
for i := range creditLookupBatch*2 + 7 {
mbids = append(mbids, testMBID(fmt.Sprintf("batch-%d", i)))
}
// One real credit somewhere past the first batch boundary.
seedCredit(t, db, mbids[creditLookupBatch+3], 9, []CreditPart{
{Position: 0, ArtistMBID: testMBID("a"), CreditedName: "A", JoinPhrase: " & "},
{Position: 1, ArtistMBID: testMBID("b"), CreditedName: "B"},
})
got, err := si.GetCredits(mbids)
if err != nil {
t.Fatalf("GetCredits: %v", err)
}
if len(got[mbids[creditLookupBatch+3]]) != 2 {
t.Errorf("a credit past the first batch boundary was not returned")
}
}
+637
View File
@@ -0,0 +1,637 @@
//go:build indexbuild
package explore
import (
"archive/tar"
"bufio"
"compress/bzip2"
"context"
"errors"
"fmt"
"io"
"path"
"regexp"
"strconv"
"strings"
)
// Multi-artist credits, from the core MusicBrainz dump.
//
// A credit is ordered parts and the credit *string* is derived from
// them; MusicBrainz's own artist_credit.name is a cached render. What
// this pass extracts is the decomposition: for each catalog recording
// and release group whose credit names more than one artist, the
// credited artists in order, each with the name *as credited* and the
// join phrase that follows it. See artist_credit_part.sql for why that
// is stored rather than derived, and why nothing may reconstruct a
// credit by searching a name inside a credit string.
//
// It is a separate dump from everything else here, and it has to be.
// The canonical dump this importer already streams gives artist_mbids
// (an ordered list) and artist_credit_name (the *rendered* string) --
// no join phrases, and no per-artist as-credited names. Splitting the
// rendered string using canonical artist names fails on exactly the
// credits that matter: measured on a real library, 21% of multi-artist
// credits name an artist differently from the artist's own name
// ("Snoop Dogg" credited on a track by "Snoop Doggy Dogg"), so the
// substring is simply not there. The JSON dumps were checked too and
// cover 153,691 recordings of ~35M, with zero overlap against a real
// library. This dump is the only source.
//
// Cost, measured on the 20260815 export: 7.1 GB compressed, decompressed
// by pure-Go compress/bzip2 at ~26 MB/s uncompressed (~13.7 min for the
// whole file, single-threaded). cmd/indexbuild is built CGO_ENABLED=0,
// so the stdlib decompressor is what there is -- and it is fine, because
// the 2 MB/s origin throttle dominates, as it does for every other dump
// here.
const (
// defaultMBDumpBaseURL is the core MusicBrainz export. Only
// mbdump.tar.bz2 is fetched; the other tarballs there hold data this
// app has no use for.
defaultMBDumpBaseURL = "https://data.metabrainz.org/pub/musicbrainz/data/fullexport/"
)
var (
mbdumpDirRe = regexp.MustCompile(`^\d{8}-\d+$`)
mbdumpFileRe = regexp.MustCompile(`^mbdump\.tar\.bz2$`)
// ErrDumpShape is returned when a dump member does not have the
// columns this code was written against. It is deliberately fatal:
// reading the wrong column silently produces a catalog whose credits
// are subtly wrong, which is far worse than a failed build.
ErrDumpShape = errors.New("musicbrainz dump member has an unexpected shape")
)
// Column positions in the Postgres COPY output, verified against the
// 20260815 export. There is no header row to read them from, so they
// are asserted instead -- see checkShape.
const (
artistColID = 0
artistColGID = 1
artistColMin = 2
creditColID = 0
creditColArtistCount = 2
creditColMin = 3
partColCredit = 0
partColPosition = 1
partColArtist = 2
partColName = 3
partColJoin = 4
partColMin = 5
// recording and release_group share a layout in the columns this
// pass reads: id, gid, name, artist_credit, ...
entityColGID = 1
entityColCredit = 3
entityColMin = 4
)
// creditPart is one credited artist within a credit.
type creditPart struct {
position int
artistID int32
name string
join string
}
// creditScan is what one pass over the dump collects.
type creditScan struct {
// artistGIDs maps an artist row id to its MBID. artist_credit_name
// references artists by row id, and the tar orders `artist` before
// it, so this is complete by the time it is read.
artistGIDs map[int32]uuid16
// multiCredits are the credit ids naming more than one artist, from
// artist_credit.artist_count. Taking the count from the dump rather
// than counting parts means a credit can be rejected before its
// parts are stored.
multiCredits map[int32]struct{}
// parts are the decompositions of multiCredits, keyed by credit id.
parts map[int32][]creditPart
// refs maps a kept catalog entity to its credit. Only entities in
// explore_index and only multi-artist credits: everything else is
// already described by explore_index's own artist_name/artist_mbid.
refs map[uuid16]int32
// used are the credits some ref actually points at, which is a small
// fraction of multiCredits -- the catalog keeps ~1.8M entities of
// MusicBrainz's tens of millions.
used map[int32]struct{}
skippedUnknownArtist int
}
// creditsImportDoneKey marks in explore_index_meta that the credit pass
// has run against the current catalog.
//
// It is its own marker rather than part of the import's stage state for
// a resume reason: the credit pass runs *after* the catalog is
// assembled, and a failure in it must not send the next run back
// through the ~205 GB it just finished. Marking separately means a
// retry retries only this.
const creditsImportDoneKey = "credits_import_done"
// ensureArtistCredits runs the credit pass unless it has already run
// against this catalog, reporting whether it newly populated them.
//
// Called from both of run's paths -- the full import and the resume
// that finds the rows already assembled -- and from the maintenance
// entry point below, since a catalog built before credits existed is
// otherwise never offered a chance to gain them: the index job picks
// its mode from the index's own state, and a complete import means
// "refresh", which never enters run() at all.
//
// The return value is what tells the job there is something new worth
// publishing. A refresh otherwise reports "changed" only when the
// listens series advanced, so credits would sit in the CI database and
// never reach an artifact.
func (imp *dumpImporter) ensureArtistCredits(ctx context.Context) bool {
if imp.si.hasMeta(creditsImportDoneKey) {
return false
}
url, err := discoverDumpFile(
ctx, imp.httpClient, imp.mbdumpBaseURL, mbdumpDirRe, mbdumpFileRe,
)
if err != nil {
imp.logger.Warn("credit import: could not find the dump", "error", err)
return false
}
if err := imp.importArtistCredits(ctx, url); err != nil {
// A catalog without credits is the catalog this app shipped
// before them: every credit falls back to its single artist.
// That is worth far less than failing an import that otherwise
// succeeded.
imp.logger.Warn("credit import: failed", "error", err)
return false
}
imp.si.setMeta(creditsImportDoneKey, "1")
return true
}
// EnsureArtistCredits tops up the credit tables outside a full import.
//
// It exists because the index job's modes are decided from the index's
// own state: a cache holding a completed import chooses `refresh`,
// which folds in incremental listens and never enters the dump
// importer. Without this, a catalog built before the credit pass
// existed could only gain credits from a `rebuild` -- and a rebuild
// re-downloads ~205 GB to reproduce rows it already has, to add
// something that costs 7 GB on its own.
//
// Reports whether credits were newly populated, so the caller knows
// there is a new artifact worth publishing.
func (e *Service) EnsureArtistCredits(ctx context.Context) bool {
imp, err := newDumpImporter(e.index, e.lb)
if err != nil {
e.index.logger.Warn("credit import: could not start", "error", err)
return false
}
return imp.ensureArtistCredits(ctx)
}
// importArtistCredits streams the core MusicBrainz dump and fills
// artist_credit_part and artist_credit_ref for the entities the catalog
// kept.
//
// It runs after assembleIndex because it asks explore_index which
// entities those are: the popularity filter decides what is worth
// carrying credits for, and asking the table rather than the kept sets
// means this stays correct if that filter changes.
func (imp *dumpImporter) importArtistCredits(ctx context.Context, url string) error {
kept, err := imp.keptEntityMBIDs(ctx)
if err != nil {
return err
}
if len(kept) == 0 {
imp.logger.Warn("credit import: no catalog entities, skipping")
return nil
}
imp.logger.Info("credit import: starting", "url", url, "entities", len(kept))
imp.logJob("Streaming MusicBrainz dump for artist credits")
scan, err := imp.scanCreditDump(ctx, url, kept)
if err != nil {
return err
}
imp.logger.Info("credit import: scanned",
"multiArtistCredits", len(scan.multiCredits),
"entitiesWithMultiArtistCredit", len(scan.refs),
"creditsUsed", len(scan.used),
)
return imp.writeCredits(ctx, scan)
}
// keptEntityMBIDs is every recording and release group in the catalog.
// Artists are excluded: an artist is not credited to a credit.
func (imp *dumpImporter) keptEntityMBIDs(ctx context.Context) (map[uuid16]struct{}, error) {
rows, err := imp.si.db.QueryContextWith(ctx,
`SELECT mbid FROM explore_index
WHERE entity_type IN (2 /* release_group */, 3 /* recording */)`,
)
if err != nil {
return nil, fmt.Errorf("credit import: read catalog entities: %w", err)
}
defer func() { _ = rows.Close() }()
out := make(map[uuid16]struct{})
for rows.Next() {
var raw []byte
if err := rows.Scan(&raw); err != nil {
return nil, fmt.Errorf("credit import: scan mbid: %w", err)
}
if len(raw) != len(uuid16{}) {
continue
}
var id uuid16
copy(id[:], raw)
out[id] = struct{}{}
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("credit import: read catalog entities: %w", err)
}
return out, nil
}
// scanCreditDump makes one sequential pass over mbdump.tar.bz2.
//
// The tar's members are alphabetical, which is what makes a single pass
// possible without buffering the big ones: `artist` and
// `artist_credit_name` both arrive before `recording` and
// `release_group`, so by the time an entity names a credit, that
// credit's parts and their artists' MBIDs are already known and the
// entity can be resolved and dropped. 35M recording rows are never
// held.
//
// The order is not depended on blindly: an entity naming a credit that
// has not been seen is counted and reported rather than silently
// producing an empty catalog, which is what a reordered export would
// otherwise look like.
func (imp *dumpImporter) scanCreditDump(
ctx context.Context, url string, kept map[uuid16]struct{},
) (*creditScan, error) {
stream := imp.openDumpStream(ctx, url, 0)
defer func() { _ = stream.Close() }()
return imp.scanCreditTar(
ctx,
tar.NewReader(bzip2.NewReader(bufio.NewReaderSize(stream, 1<<20))),
kept,
)
}
// scanCreditTar is the parse, separated from the fetch so it can be
// driven by a tar built in a test. compress/bzip2 is decompress-only,
// so a test cannot produce the real container.
func (imp *dumpImporter) scanCreditTar(
ctx context.Context, tr *tar.Reader, kept map[uuid16]struct{},
) (*creditScan, error) {
scan := &creditScan{
artistGIDs: make(map[int32]uuid16),
multiCredits: make(map[int32]struct{}),
parts: make(map[int32][]creditPart),
refs: make(map[uuid16]int32),
used: make(map[int32]struct{}),
}
for {
if err := ctx.Err(); err != nil {
return nil, err
}
hdr, err := tr.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("credit import: tar: %w", err)
}
if hdr.Typeflag != tar.TypeReg {
continue
}
done, err := imp.scanCreditMember(ctx, hdr.Name, tr, kept, scan)
if err != nil {
return nil, err
}
if done {
// Everything this pass needs has been read; the rest of the
// tarball is other entities' data and decompressing it would
// cost minutes for nothing.
break
}
}
if scan.skippedUnknownArtist > 0 {
imp.logger.Warn("credit import: credits dropped for unknown artists",
"count", scan.skippedUnknownArtist,
)
}
return scan, nil
}
// scanCreditMember dispatches one tar member, reporting whether the
// pass has everything it needs.
func (imp *dumpImporter) scanCreditMember(
ctx context.Context, name string, r io.Reader,
kept map[uuid16]struct{}, scan *creditScan,
) (bool, error) {
switch path.Base(name) {
case "artist":
return false, imp.scanArtists(ctx, r, scan)
case "artist_credit":
return false, imp.scanCredits(ctx, r, scan)
case "artist_credit_name":
return false, imp.scanCreditParts(ctx, r, scan)
case "recording", "release_group":
if err := imp.scanCreditedEntities(ctx, r, kept, scan); err != nil {
return false, err
}
// release_group sorts after recording, so the pass is complete
// once it has been read.
return path.Base(name) == "release_group", nil
default:
return false, nil
}
}
// scanArtists records every artist's MBID by row id.
func (imp *dumpImporter) scanArtists(
ctx context.Context, r io.Reader, scan *creditScan,
) error {
return scanTSV(ctx, r, artistColMin, "artist", func(fields []string) error {
id, ok := parseInt32(fields[artistColID])
if !ok {
return nil
}
var gid uuid16
if !parseUUID(fields[artistColGID], gid[:]) {
return fmt.Errorf("%w: artist.gid is not a UUID: %q",
ErrDumpShape, truncate(fields[artistColGID]))
}
scan.artistGIDs[id] = gid
return nil
})
}
// scanCredits records which credits name more than one artist.
func (imp *dumpImporter) scanCredits(
ctx context.Context, r io.Reader, scan *creditScan,
) error {
return scanTSV(ctx, r, creditColMin, "artist_credit", func(fields []string) error {
id, ok := parseInt32(fields[creditColID])
if !ok {
return nil
}
count, ok := parseInt32(fields[creditColArtistCount])
if !ok {
return fmt.Errorf("%w: artist_credit.artist_count is not a number: %q",
ErrDumpShape, truncate(fields[creditColArtistCount]))
}
if count > 1 {
scan.multiCredits[id] = struct{}{}
}
return nil
})
}
// scanCreditParts records the decomposition of every multi-artist
// credit.
func (imp *dumpImporter) scanCreditParts(
ctx context.Context, r io.Reader, scan *creditScan,
) error {
return scanTSV(ctx, r, partColMin, "artist_credit_name", func(fields []string) error {
credit, ok := parseInt32(fields[partColCredit])
if !ok {
return nil
}
if _, multi := scan.multiCredits[credit]; !multi {
return nil
}
position, ok := parseInt32(fields[partColPosition])
if !ok {
return nil
}
artist, ok := parseInt32(fields[partColArtist])
if !ok {
return nil
}
scan.parts[credit] = append(scan.parts[credit], creditPart{
position: int(position),
artistID: artist,
name: fields[partColName],
join: fields[partColJoin],
})
return nil
})
}
// scanCreditedEntities resolves recordings and release groups against
// the catalog, keeping only those the catalog holds and whose credit
// names more than one artist.
func (imp *dumpImporter) scanCreditedEntities(
ctx context.Context, r io.Reader, kept map[uuid16]struct{}, scan *creditScan,
) error {
return scanTSV(ctx, r, entityColMin, "recording/release_group",
func(fields []string) error {
var gid uuid16
if !parseUUID(fields[entityColGID], gid[:]) {
return fmt.Errorf("%w: entity gid is not a UUID: %q",
ErrDumpShape, truncate(fields[entityColGID]))
}
if _, want := kept[gid]; !want {
return nil
}
credit, ok := parseInt32(fields[entityColCredit])
if !ok {
return fmt.Errorf("%w: entity artist_credit is not a number: %q",
ErrDumpShape, truncate(fields[entityColCredit]))
}
if _, multi := scan.multiCredits[credit]; !multi {
return nil
}
scan.refs[gid] = credit
scan.used[credit] = struct{}{}
return nil
})
}
// scanTSV reads Postgres COPY output a line at a time, unescaping each
// field and handing the row to fn.
//
// The shape is asserted on the first row rather than trusted: this dump
// has no header, so a column that moved would otherwise be read as a
// neighbouring one and produce a catalog that is quietly wrong.
func scanTSV(
ctx context.Context, r io.Reader, minCols int, member string,
fn func(fields []string) error,
) error {
sc := bufio.NewScanner(r)
sc.Buffer(make([]byte, 0, 1<<20), 1<<24)
checked := false
rows := 0
for sc.Scan() {
rows++
if rows%(1<<20) == 0 {
if err := ctx.Err(); err != nil {
return err
}
}
line := sc.Text()
if line == "" {
continue
}
fields := strings.Split(line, "\t")
if len(fields) < minCols {
if !checked {
return fmt.Errorf("%w: %s has %d columns, need at least %d",
ErrDumpShape, member, len(fields), minCols)
}
continue
}
checked = true
for i := range fields {
fields[i] = unescapeCopy(fields[i])
}
if err := fn(fields); err != nil {
return err
}
}
if err := sc.Err(); err != nil {
return fmt.Errorf("credit import: read %s: %w", member, err)
}
return nil
}
// unescapeCopy undoes Postgres COPY's text escaping. A NULL (\N) is
// returned as an empty string: every field this pass reads is either a
// number it will reject or a name whose absence means the same as
// empty.
func unescapeCopy(s string) string {
if s == `\N` {
return ""
}
if !strings.ContainsRune(s, '\\') {
return s
}
var b strings.Builder
b.Grow(len(s))
for i := 0; i < len(s); i++ {
if s[i] != '\\' || i+1 >= len(s) {
b.WriteByte(s[i])
continue
}
i++
switch s[i] {
case 'n':
b.WriteByte('\n')
case 't':
b.WriteByte('\t')
case 'r':
b.WriteByte('\r')
case 'b':
b.WriteByte('\b')
case 'f':
b.WriteByte('\f')
case 'v':
b.WriteByte('\v')
case '\\':
b.WriteByte('\\')
default:
b.WriteByte('\\')
b.WriteByte(s[i])
}
}
return b.String()
}
func parseInt32(s string) (int32, bool) {
n, err := strconv.ParseInt(s, 10, 32)
if err != nil {
return 0, false
}
return int32(n), true
}
// truncate bounds an error message built from dump data, which is
// attacker-free but can be long.
func truncate(s string) string {
const limit = 64
if len(s) <= limit {
return s
}
return s[:limit] + "..."
}
+406
View File
@@ -0,0 +1,406 @@
//go:build indexbuild
package explore
import (
"archive/tar"
"bytes"
"context"
"errors"
"strings"
"testing"
"yellowjacket/backend/database"
)
// tarOf builds an uncompressed tar of the named members, in the order
// given. Order is the point of several of these tests: the real dump's
// members are alphabetical, which is what lets one pass resolve an
// entity's credit without buffering 35M recordings.
func tarOf(t *testing.T, members ...[2]string) *tar.Reader {
t.Helper()
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
for _, m := range members {
body := []byte(m[1])
if err := tw.WriteHeader(&tar.Header{
Name: "mbdump/" + m[0],
Mode: 0o644,
Size: int64(len(body)),
Typeflag: tar.TypeReg,
}); err != nil {
t.Fatalf("tar header: %v", err)
}
if _, err := tw.Write(body); err != nil {
t.Fatalf("tar write: %v", err)
}
}
if err := tw.Close(); err != nil {
t.Fatalf("tar close: %v", err)
}
return tar.NewReader(&buf)
}
func tsv(rows ...[]string) string {
var b strings.Builder
for _, r := range rows {
b.WriteString(strings.Join(r, "\t"))
b.WriteByte('\n')
}
return b.String()
}
// mustMBID is testMBID in the packed form the catalog stores.
func mustMBID(label string) uuid16 {
var u uuid16
if !parseUUID(testMBID(label), u[:]) {
panic("testMBID did not produce a UUID for " + label)
}
return u
}
// The two artists of the worked example, and the entities they credit.
var (
creditRecMBID = mustMBID("recording-1")
creditRGMBID = mustMBID("release-group-1")
)
// sampleDump is the shape verified against the 20260815 export:
// artist(id, gid, ...), artist_credit(id, name, artist_count, ...),
// artist_credit_name(credit, position, artist, name, join_phrase),
// recording/release_group(id, gid, name, artist_credit, ...).
func sampleDump(t *testing.T) *tar.Reader {
t.Helper()
return tarOf(t,
[2]string{"artist", tsv(
[]string{"11", testMBID("artist-a"), "Snoop Doggy Dogg", "Snoop Doggy Dogg"},
[]string{"22", testMBID("artist-b"), "2Pac", "2Pac"},
)},
[2]string{"artist_credit", tsv(
[]string{"900", "2Pac feat. Snoop Dogg", "2", "1", "", "0", ""},
[]string{"901", "Solo Artist", "1", "1", "", "0", ""},
)},
[2]string{"artist_credit_name", tsv(
// Deliberately out of position order: the dump is not
// obliged to emit them sorted and the credit's meaning is
// the order, not the file's.
[]string{"900", "1", "11", "Snoop Dogg", ""},
[]string{"900", "0", "22", "2Pac", " feat. "},
[]string{"901", "0", "11", "Solo Artist", ""},
)},
[2]string{"recording", tsv(
[]string{"1", testMBID("recording-1"), "Some Song", "900", "180000"},
[]string{"2", testMBID("not-kept"), "Other", "900", "1"},
[]string{"3", testMBID("solo"), "Solo", "901", "1"},
)},
[2]string{"release_group", tsv(
[]string{"5", testMBID("release-group-1"), "Some Album", "900", "1"},
)},
)
}
func creditTestImporter(t *testing.T) *dumpImporter {
t.Helper()
db := database.NewTestDB(t)
return &dumpImporter{
si: NewSearchIndex(db, nil, nil, testLogger()),
logger: testLogger(),
}
}
// TestScanCreditDumpDecomposes is the worked example end to end: the
// credit's parts come back in position order, with the *credited*
// names and the join phrase between them.
func TestScanCreditDumpDecomposes(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{
creditRecMBID: {},
creditRGMBID: {},
}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if got := len(scan.refs); got != 2 {
t.Fatalf("refs = %d, want 2 (the recording and the release group)", got)
}
if scan.refs[creditRecMBID] != 900 {
t.Errorf("recording credit = %d, want 900", scan.refs[creditRecMBID])
}
parts := scan.parts[900]
if len(parts) != 2 {
t.Fatalf("parts = %d, want 2", len(parts))
}
// Sorting happens on write, so assert the pieces are all present
// and let the render test below check the order.
byPos := map[int]creditPart{}
for _, p := range parts {
byPos[p.position] = p
}
if byPos[0].name != "2Pac" || byPos[0].join != " feat. " {
t.Errorf("position 0 = %q/%q, want \"2Pac\"/\" feat. \"",
byPos[0].name, byPos[0].join)
}
// The credited name, not the artist's own name: this is the whole
// reason credited_name is stored per row.
if byPos[1].name != "Snoop Dogg" {
t.Errorf("position 1 credited name = %q, want \"Snoop Dogg\"", byPos[1].name)
}
}
// TestSingleArtistCreditsAreNotStored: a one-artist credit is already
// described by explore_index's artist_name/artist_mbid, and storing it
// would roughly triple the table to say nothing new.
func TestSingleArtistCreditsAreNotStored(t *testing.T) {
imp := creditTestImporter(t)
solo := mustMBID("solo")
kept := map[uuid16]struct{}{solo: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if len(scan.refs) != 0 {
t.Fatalf("a single-artist credit was referenced: %v", scan.refs)
}
if _, ok := scan.multiCredits[901]; ok {
t.Error("credit 901 has artist_count 1 and should not be multi")
}
}
// TestOnlyKeptEntitiesAreReferenced: the catalog's popularity filter
// decides what is worth carrying credits for, and an entity outside it
// must not produce a row pointing at nothing.
func TestOnlyKeptEntitiesAreReferenced(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{creditRecMBID: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if _, ok := scan.refs[mustMBID("not-kept")]; ok {
t.Error("an entity outside the catalog was referenced")
}
if len(scan.used) != 1 {
t.Errorf("used credits = %d, want 1", len(scan.used))
}
}
// TestWriteCreditsRoundTrips checks what the frontend will actually
// read: parts in position order, dashed MBIDs out of the 16 raw bytes,
// and a rendered credit that reassembles to the tagged string.
func TestWriteCreditsRoundTrips(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{creditRecMBID: {}, creditRGMBID: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if err := imp.writeCredits(context.Background(), scan); err != nil {
t.Fatalf("writeCredits: %v", err)
}
rows, err := imp.si.db.QueryContext(
`SELECT p.position, p.artist_mbid, p.credited_name, p.join_phrase
FROM artist_credit_ref r
JOIN artist_credit_part p ON p.credit_id = r.credit_id
WHERE r.mbid = ?
ORDER BY p.position`,
creditRecMBID[:],
)
if err != nil {
t.Fatalf("query: %v", err)
}
defer func() { _ = rows.Close() }()
var rendered strings.Builder
names := []string{}
for rows.Next() {
var (
pos int
mbid []byte
name string
join string
)
if err := rows.Scan(&pos, &mbid, &name, &join); err != nil {
t.Fatalf("scan row: %v", err)
}
if len(mbid) != 16 {
t.Fatalf("artist_mbid is %d bytes, want 16", len(mbid))
}
names = append(names, name)
rendered.WriteString(name)
rendered.WriteString(join)
}
if err := rows.Err(); err != nil {
t.Fatalf("rows: %v", err)
}
// Concatenation is the contract: names in order, join phrases
// between them, and no searching a name inside a credit string.
if got := rendered.String(); got != "2Pac feat. Snoop Dogg" {
t.Errorf("rendered credit = %q, want %q", got, "2Pac feat. Snoop Dogg")
}
if len(names) != 2 || names[0] != "2Pac" {
t.Errorf("parts came back out of position order: %v", names)
}
}
// TestCreditRefsNeverDangle: a ref whose parts were not stored renders
// as a credit with no artists at all, which is worse than the
// single-artist fallback it replaced.
func TestCreditRefsNeverDangle(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{creditRecMBID: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
// An artist the dump never named: the credit cannot be navigated to
// and must be dropped whole, taking its ref with it.
scan.artistGIDs = map[int32]uuid16{}
if err := imp.writeCredits(context.Background(), scan); err != nil {
t.Fatalf("writeCredits: %v", err)
}
var refs, parts int
if err := imp.si.db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_credit_ref",
).Scan(&refs); err != nil {
t.Fatalf("count refs: %v", err)
}
if err := imp.si.db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_credit_part",
).Scan(&parts); err != nil {
t.Fatalf("count parts: %v", err)
}
if refs != 0 || parts != 0 {
t.Fatalf("refs=%d parts=%d, want 0/0 when the artists are unknown", refs, parts)
}
}
// TestCreditDumpShapeIsAsserted: the dump has no header row, so a
// column that moved would be read as its neighbour and produce a
// catalog that is quietly wrong. Loud is the requirement.
func TestCreditDumpShapeIsAsserted(t *testing.T) {
imp := creditTestImporter(t)
short := tarOf(t, [2]string{"artist", tsv([]string{"11", "only-two-columns"})})
_, err := imp.scanCreditTar(context.Background(), short, map[uuid16]struct{}{})
if err == nil {
t.Fatal("a member with a non-UUID gid was accepted")
}
if !errors.Is(err, ErrDumpShape) {
t.Errorf("error = %v, want ErrDumpShape", err)
}
}
// TestUnescapeCopy covers Postgres COPY's text escaping, which reaches
// artist names routinely -- a tab or backslash in a name would
// otherwise shift every field after it.
func TestUnescapeCopy(t *testing.T) {
tests := []struct{ in, want string }{
{`plain`, `plain`},
{`\N`, ``},
{`a\tb`, "a\tb"},
{`a\nb`, "a\nb"},
{`back\\slash`, `back\slash`},
{`AC\/DC`, `AC\/DC`},
{`trailing\`, `trailing\`},
}
for _, tt := range tests {
if got := unescapeCopy(tt.in); got != tt.want {
t.Errorf("unescapeCopy(%q) = %q, want %q", tt.in, got, tt.want)
}
}
}
// TestEnsureArtistCreditsIsIdempotent pins what the index job depends
// on to decide whether to publish.
//
// The pass runs on every mode, including the `refresh` that a complete
// catalog always chooses — so it must be free when there is nothing to
// do, and it must say so. A `true` here republishes the artifact; a
// `true` on every run would republish an identical one weekly, and a
// permanent `false` would mean a catalog that never gains credits at
// all.
func TestEnsureArtistCreditsIsIdempotent(t *testing.T) {
imp := creditTestImporter(t)
// The marker is what "already done" means; with it set, the pass
// must not reach the network or report a change.
imp.si.setMeta(creditsImportDoneKey, "1")
if imp.ensureArtistCredits(context.Background()) {
t.Fatal("a second run reported new credits; the artifact would republish forever")
}
}
// TestEnsureArtistCreditsReportsFailureAsNoChange: a dump that cannot be
// reached leaves the catalog exactly as it was, and must not claim
// otherwise — publishing on it would ship an artifact with no credits
// and mark the work done.
func TestEnsureArtistCreditsReportsFailureAsNoChange(t *testing.T) {
imp := creditTestImporter(t)
imp.httpClient = newDumpHTTPClient()
imp.mbdumpBaseURL = "http://127.0.0.1:1/nonexistent/"
if imp.ensureArtistCredits(context.Background()) {
t.Fatal("an unreachable dump reported new credits")
}
if imp.si.hasMeta(creditsImportDoneKey) {
t.Error("a failed pass marked itself done; it would never retry")
}
}
+164
View File
@@ -0,0 +1,164 @@
//go:build indexbuild
package explore
import (
"context"
"database/sql"
"fmt"
"sort"
)
// writeCredits persists the scanned decompositions.
//
// Only credits some catalog entity actually points at are written: the
// dump has millions of multi-artist credits and the catalog keeps ~1.8M
// entities, so storing every credit would be most of a table nothing
// can reach.
//
// The two tables are written in one transaction, because a ref pointing
// at parts that are not there renders as a credit with no artists --
// worse than the single-artist fallback it replaced.
func (imp *dumpImporter) writeCredits(ctx context.Context, scan *creditScan) error {
tx, err := imp.si.db.BeginTx()
if err != nil {
return fmt.Errorf("credit import: begin: %w", err)
}
defer func() { _ = tx.Rollback() }()
// A rebuild replaces the previous pass wholesale. These are Cache
// tables derived entirely from the dump, so there is nothing to
// merge and a stale row is a wrong credit.
for _, table := range []string{"artist_credit_part", "artist_credit_ref"} {
if _, err := tx.ExecContext(ctx, "DELETE FROM "+table); err != nil {
return fmt.Errorf("credit import: clear %s: %w", table, err)
}
}
written, err := imp.writeCreditParts(ctx, tx, scan)
if err != nil {
return err
}
refs, err := imp.writeCreditRefs(ctx, tx, scan, written)
if err != nil {
return err
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("credit import: commit: %w", err)
}
imp.logger.Info("credit import: complete",
"credits", len(written),
"refs", refs,
)
return nil
}
// writeCreditParts inserts the parts of every used credit and returns
// the set of credits that were actually stored.
//
// A credit is stored whole or not at all. If any of its artists has no
// MBID -- which should not happen, the dump being self-consistent, but
// would leave a part that cannot be navigated to -- the credit is
// dropped and the entity falls back to explore_index's single artist,
// which is a worse answer rather than a broken one.
func (imp *dumpImporter) writeCreditParts(
ctx context.Context, tx *sql.Tx, scan *creditScan,
) (map[int32]struct{}, error) {
stmt, err := tx.PrepareContext(ctx,
`INSERT INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
VALUES (?, ?, ?, ?, ?)`,
)
if err != nil {
return nil, fmt.Errorf("credit import: prepare part insert: %w", err)
}
defer func() { _ = stmt.Close() }()
written := make(map[int32]struct{}, len(scan.used))
for credit := range scan.used {
parts := scan.parts[credit]
if len(parts) < 2 {
// artist_credit said more than one artist and
// artist_credit_name did not deliver them. Nothing to
// decompose, so leave the entity to its single artist.
continue
}
// Position order is the credit's meaning, and the dump is not
// obliged to emit it sorted.
sort.Slice(parts, func(i, j int) bool {
return parts[i].position < parts[j].position
})
resolved := make([][]any, 0, len(parts))
ok := true
for _, part := range parts {
gid, found := scan.artistGIDs[part.artistID]
if !found {
scan.skippedUnknownArtist++
ok = false
break
}
resolved = append(resolved, []any{
credit, part.position, gid[:], part.name, part.join,
})
}
if !ok {
continue
}
for _, args := range resolved {
if _, err := stmt.ExecContext(ctx, args...); err != nil {
return nil, fmt.Errorf("credit import: insert part: %w", err)
}
}
written[credit] = struct{}{}
}
return written, nil
}
// writeCreditRefs points each kept entity at its credit, skipping any
// whose credit was not stored so a ref never dangles.
func (imp *dumpImporter) writeCreditRefs(
ctx context.Context, tx *sql.Tx, scan *creditScan, written map[int32]struct{},
) (int, error) {
stmt, err := tx.PrepareContext(ctx,
"INSERT OR REPLACE INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)",
)
if err != nil {
return 0, fmt.Errorf("credit import: prepare ref insert: %w", err)
}
defer func() { _ = stmt.Close() }()
count := 0
for mbid, credit := range scan.refs {
if _, stored := written[credit]; !stored {
continue
}
id := mbid
if _, err := stmt.ExecContext(ctx, id[:], credit); err != nil {
return 0, fmt.Errorf("credit import: insert ref: %w", err)
}
count++
}
return count, nil
}
+8
View File
@@ -104,6 +104,7 @@ type dumpImporter struct {
canonicalBaseURL string
listensBaseURL string
mbdumpBaseURL string
// Disk safety floors (fields so tests can relax them).
minStartFreeBytes uint64
@@ -144,6 +145,7 @@ func newDumpImporter(si *SearchIndex, lb *ListenBrainzClient) (*dumpImporter, er
stagingDir: stagingDir,
canonicalBaseURL: defaultCanonicalBaseURL,
listensBaseURL: defaultListensBaseURL,
mbdumpBaseURL: defaultMBDumpBaseURL,
minStartFreeBytes: dumpMinStartFreeBytes,
abortFreeBytes: dumpAbortFreeBytes,
}, nil
@@ -171,6 +173,7 @@ func (imp *dumpImporter) run(ctx context.Context) error {
// Fast path: rows already assembled, only patch passes remain.
if state.Stage == dumpStageAssembled {
imp.si.MarkReadyIfPopulated()
imp.ensureArtistCredits(ctx)
imp.runPatchPasses(ctx)
if err := ctx.Err(); err != nil {
@@ -305,6 +308,11 @@ func (imp *dumpImporter) run(ctx context.Context) error {
imp.si.MarkReadyIfPopulated()
imp.si.refreshStatusCounts()
// Multi-artist credits, from a different dump. After the catalog,
// because it asks explore_index which entities are worth carrying
// credits for.
imp.ensureArtistCredits(ctx)
// Stage 4: API patch passes (idempotent).
imp.runPatchPasses(ctx)
+141 -15
View File
@@ -289,8 +289,13 @@ func (l *Library) scanInternal(
l.mu.Unlock()
}()
// The configured mode, not a hardcoded "auto". `ScanConcurrency`
// has been a validated config field with three values and one
// caller passing a constant, so choosing `ssd` or `hdd` by hand
// did nothing at all.
diskProfile := system.ProfileForPath(libraryPath)
workerCount := resolveScanWorkerCount(
ScanConcurrencyAuto,
l.conf.ScanConcurrency,
libraryPath,
)
@@ -300,6 +305,10 @@ func (l *Library) scanInternal(
"libraryName", libraryName,
"libraryPath", libraryPath,
"workers", workerCount,
"mode", l.conf.ScanConcurrency,
"device", diskProfile.Device,
"rotational", diskProfile.Rotational,
"queueDepth", diskProfile.QueueDepth,
)
// Helper to build a ScanProgress with library identification.
@@ -818,7 +827,7 @@ func (l *Library) scanInternal(
g := new(errgroup.Group)
g.SetLimit(workerCount)
for work := range workChan {
for work := range readaheadWork(scanCtx, workChan, diskProfile) {
g.Go(func() error {
if err := l.waitIfPaused(scanCtx); err != nil {
return err
@@ -1285,9 +1294,101 @@ func surveyAudioFiles(
return count, maxModTime
}
// hddWorkerCount is the maximum number of concurrent extraction
// workers when the library resides on a spinning disk.
const hddWorkerCount = 2
// How many extraction workers a spinning disk gets, and why it is two
// numbers rather than one.
//
// Extraction is not CPU work — every parser here reads headers and
// returns — so on a spinning disk the whole cost is seek latency, and
// the only question worth asking is how many reads should be in flight
// at once. That has two different right answers and the drive says
// which:
//
// - A drive with command queueing (NCQ: /sys/block/<dev>/device/
// queue_depth reports 31 or 32 on any SATA disk with it enabled)
// reorders outstanding reads into the order its head passes over
// them. Handing it several at once is most of why a parallel scan
// beats a serial one at all, and four is where the returns flatten:
// the drive needs a few requests to have anything to reorder, and
// past that it is queueing requests it was already going to
// service in that order.
// - A drive without it — queue_depth 1, which is what a USB bridge
// or a pre-2004 disk reports — services one command at a time in
// the order given. Every extra worker there is one more seek
// competing for one head, and the scan gets *slower* the harder it
// is pushed. Two is kept rather than one because the readahead
// hints (see readaheadWork) do the overlapping that concurrency
// was standing in for, and one worker cannot hide a stall.
//
// This used to be a flat 2 for anything rotational, which is a
// pre-NCQ assumption: it left a modern spinning disk with a quarter of
// the queue depth it can use.
const (
hddWorkerCountQueued = 4
hddWorkerCountSerial = 2
)
// Readahead tuning.
const (
// readaheadDepth is how many files ahead of the workers the
// prefetcher runs. It is the channel's buffer, so it is also the
// number of `WILLNEED` hints outstanding at once — comfortably more
// than a queueing drive's 32-command window is worth filling with
// one library, and small enough that a cancelled scan is not
// holding a long tail of queued reads.
readaheadDepth = 16
// readaheadBytes is how much of each file to pull in. Everything
// the scanner reads lives at the head: ID3v2 and FLAC's
// STREAMINFO/VORBIS_COMMENT/PICTURE blocks, and the first MPEG
// frame with its Xing header. 512 KB covers a tag carrying
// embedded cover art, which is the large case — and reading a
// little too much sequentially costs a spinning disk almost
// nothing next to the seek that got there.
readaheadBytes = 512 << 10
)
// readaheadWork forwards scan work while asking the kernel to fetch
// each file's header before a worker reaches it.
//
// The buffered channel *is* the lookahead: this goroutine runs ahead
// of the workers until the buffer fills, hinting every file as it goes,
// so by the time a worker takes an item the read it needs has been in
// flight for `readaheadDepth` files' worth of parsing. That is the
// only thing that helps a spinning disk here, because the per-file work
// is already header-only — every parser in `backend/metadata` reads a
// few hundred bytes and returns, so the scan is not waiting on CPU or
// on bytes, it is waiting on the head to arrive.
//
// It runs on rotational disks only. An SSD has no seek to hide and
// already has one worker per core; issuing hints there is pure syscall
// overhead against an OS readahead that is already ahead of us.
func readaheadWork(
ctx context.Context,
in <-chan scanWork,
profile system.DiskProfile,
) <-chan scanWork {
if !profile.Rotational {
return in
}
out := make(chan scanWork, readaheadDepth)
go func() {
defer close(out)
for work := range in {
hintReadahead(work.absolutePath, readaheadBytes)
select {
case out <- work:
case <-ctx.Done():
return
}
}
}()
return out
}
// resolveScanWorkerCount returns the number of concurrent
// extraction workers based on the configured concurrency mode
@@ -1296,20 +1397,45 @@ func resolveScanWorkerCount(
mode ScanConcurrency,
libraryPath string,
) int {
return workersForProfile(
mode,
system.ProfileForPath(libraryPath),
goruntime.NumCPU(),
)
}
// workersForProfile is the policy on its own, so it can be tested
// against drives this machine does not have.
//
// `hdd` and `ssd` override what the device says rather than being a
// separate branch: the mode is the user overruling detection, and
// detection is right about the queue depth either way — a user who
// picks `hdd` on a queueing drive still wants that drive's queue used.
func workersForProfile(
mode ScanConcurrency,
profile system.DiskProfile,
cpus int,
) int {
spinning := profile.Rotational
switch mode {
case ScanConcurrencySSD:
return goruntime.NumCPU()
spinning = false
case ScanConcurrencyHDD:
return min(hddWorkerCount, goruntime.NumCPU())
default: // auto
if system.IsRotationalDisk(libraryPath) {
return min(
hddWorkerCount, goruntime.NumCPU(),
)
}
return goruntime.NumCPU()
spinning = true
case ScanConcurrencyAuto:
}
if !spinning {
return cpus
}
workers := hddWorkerCountSerial
if profile.Queues() {
workers = hddWorkerCountQueued
}
return min(workers, cpus)
}
// scanWork represents a file to be processed by a worker.
+38
View File
@@ -0,0 +1,38 @@
//go:build linux
package library
import (
"os"
"golang.org/x/sys/unix"
)
// hintReadahead asks the kernel to start fetching the head of a file
// that is about to be read.
//
// `POSIX_FADV_WILLNEED` returns immediately and queues the read, which
// is the whole point: on a spinning disk the first access to a file
// costs a seek of several milliseconds, and that latency can only be
// hidden by having the next seek already in flight while the current
// file is being parsed. A drive with command queueing can then service
// the queued reads in head order rather than in the order they were
// asked for.
//
// Errors are dropped on purpose. This is a hint: a file that has since
// been deleted, a filesystem that does not implement fadvise, or a
// permission the walk saw and this open does not, all mean "no
// prefetch", never "fail the scan". The read that follows is what
// reports a genuine problem.
func hintReadahead(path string, bytes int64) {
f, err := os.Open(path)
if err != nil {
return
}
defer func() { _ = f.Close() }()
_ = unix.Fadvise(
int(f.Fd()), 0, bytes, unix.FADV_WILLNEED,
)
}
+13
View File
@@ -0,0 +1,13 @@
//go:build !linux
package library
// hintReadahead is a no-op off Linux.
//
// macOS has `F_RDADVISE` and Windows has `FILE_FLAG_SEQUENTIAL_SCAN`,
// and neither is wired up here for the reason the scan concurrency
// heuristic is not either: this package cannot tell a spinning disk
// from an SSD on those platforms (see system.ProfileForPath), so it
// would be prefetching without knowing whether prefetching is what the
// device wants.
func hintReadahead(_ string, _ int64) {}
+122
View File
@@ -0,0 +1,122 @@
package library
import (
"context"
"testing"
"yellowjacket/backend/system"
)
// How many workers a scan gets is decided by two facts about the
// device, and the second one is new: a spinning disk that can queue
// commands wants several reads in flight, and one that cannot wants
// almost none. Before this it was a flat 2 for anything rotational,
// which is a pre-NCQ assumption — a modern SATA disk reports a queue
// depth of 32 and was being given a quarter of what it can use.
func TestWorkersForProfile(t *testing.T) {
t.Parallel()
const cpus = 16
ssd := system.DiskProfile{Device: "sda", QueueDepth: 32}
hddQueued := system.DiskProfile{
Device: "sdb", Rotational: true, QueueDepth: 32,
}
hddSerial := system.DiskProfile{
Device: "sdc", Rotational: true, QueueDepth: 1,
}
// Neither NVMe nor a device-mapper volume publishes queue_depth.
// An unknown depth must not be read as "cannot queue", or every
// such device would be scanned as if it were a 2003 drive.
unknown := system.DiskProfile{Device: "dm-0", Rotational: true}
tests := []struct {
name string
mode ScanConcurrency
profile system.DiskProfile
want int
}{
{"ssd auto", ScanConcurrencyAuto, ssd, cpus},
{"queueing hdd auto", ScanConcurrencyAuto, hddQueued, hddWorkerCountQueued},
{"serial hdd auto", ScanConcurrencyAuto, hddSerial, hddWorkerCountSerial},
{"unknown depth queues", ScanConcurrencyAuto, unknown, hddWorkerCountQueued},
// The mode overrules detection about the *disk*, never about
// its queue: forcing hdd on a queueing drive still uses it.
{"forced hdd on an ssd", ScanConcurrencyHDD, ssd, hddWorkerCountQueued},
{"forced ssd on an hdd", ScanConcurrencySSD, hddQueued, cpus},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := workersForProfile(tt.mode, tt.profile, cpus); got != tt.want {
t.Errorf(
"workersForProfile(%q, %+v) = %d, want %d",
tt.mode, tt.profile, got, tt.want,
)
}
})
}
}
// A machine with fewer cores than the policy asks for gets its cores.
func TestWorkersNeverExceedTheCPUCount(t *testing.T) {
t.Parallel()
hdd := system.DiskProfile{Rotational: true, QueueDepth: 32}
if got := workersForProfile(ScanConcurrencyAuto, hdd, 1); got != 1 {
t.Errorf("single-core hdd = %d workers, want 1", got)
}
}
// The prefetch stage must forward every item and nothing else: it is a
// pass-through with a side effect, and a scan that drops a file because
// of a *hint* would be a spectacular way to lose part of a library.
func TestReadaheadForwardsEveryFile(t *testing.T) {
t.Parallel()
in := make(chan scanWork, 4)
for _, p := range []string{"/a", "/b", "/c", "/d"} {
in <- scanWork{absolutePath: p}
}
close(in)
var got []string
for w := range readaheadWork(
context.Background(),
in,
system.DiskProfile{Rotational: true, QueueDepth: 32},
) {
got = append(got, w.absolutePath)
}
want := []string{"/a", "/b", "/c", "/d"}
if len(got) != len(want) {
t.Fatalf("forwarded %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("item %d = %q, want %q", i, got[i], want[i])
}
}
}
// On an SSD the stage is not inserted at all — the channel comes back
// unchanged, so a scan there pays nothing for a feature it cannot use.
func TestReadaheadIsSkippedOnSolidState(t *testing.T) {
t.Parallel()
in := make(chan scanWork)
out := readaheadWork(
context.Background(), in, system.DiskProfile{QueueDepth: 32},
)
if out != (<-chan scanWork)(in) {
t.Error("an ssd must get the original channel, unwrapped")
}
}
+41 -10
View File
@@ -279,17 +279,19 @@ func (h *MPRISHandler) enqueue(fn func()) {
}
}
// UpdateMetadata pushes track metadata to D-Bus.
func (h *MPRISHandler) UpdateMetadata(meta Metadata) {
h.mu.Lock()
h.trackID++
tid := h.trackID
h.mu.Unlock()
m := map[string]interface{}{
// metadataMap builds the org.mpris.MediaPlayer2.Player Metadata value
// for one track.
//
// It is separated from UpdateMetadata, which needs a live D-Bus
// connection, so the map's contents can be asserted on: this file is
// behind a build tag and everything in it that touches h is reachable
// only from a session bus, which is the same reason the Android
// contract lives in an untagged androidpayload.go.
func metadataMap(meta Metadata, trackID uint64) map[string]any {
m := map[string]any{
"mpris:trackid": dbus.ObjectPath(
fmt.Sprintf(
"/org/yellowjacket/Track/%d", tid,
"/org/yellowjacket/Track/%d", trackID,
),
),
}
@@ -306,16 +308,45 @@ func (h *MPRISHandler) UpdateMetadata(meta Metadata) {
m["xesam:album"] = meta.Album
}
// Always present, even with nothing to point at.
//
// Every other key here can be omitted safely because a client
// reading the map sees a track with no title or no album and
// renders it that way. Art is different: KDE's applet (and
// others) treat an *absent* mpris:artUrl as "no news about the
// art" and keep drawing whatever the last track had, so playing
// something with no cover left the previous album's sleeve on
// screen — which reads as the wrong track playing rather than as
// missing artwork.
//
// An empty string is the honest answer and is what the spec's
// "URI" type degrades to; a client that cannot load it falls back
// to its own placeholder, which is the behaviour wanted.
artURL := ""
if meta.ArtFilePath != "" {
m["mpris:artUrl"] = "file://" + meta.ArtFilePath
artURL = "file://" + meta.ArtFilePath
}
m["mpris:artUrl"] = artURL
if meta.DurationSec > 0 {
m["mpris:length"] = int64(
meta.DurationSec,
) * usPerSec
}
return m
}
// UpdateMetadata pushes track metadata to D-Bus.
func (h *MPRISHandler) UpdateMetadata(meta Metadata) {
h.mu.Lock()
h.trackID++
tid := h.trackID
h.mu.Unlock()
m := metadataMap(meta, tid)
h.enqueue(func() {
h.props.SetMust(playerIf, "Metadata", m)
})
+86
View File
@@ -0,0 +1,86 @@
//go:build linux && !android
package mediacontrols
import "testing"
// The one key that must be present even when it is empty.
//
// Everything else in the map may be omitted, because a client reading
// it renders a track with no title as a track with no title. Art is
// different: KDE's applet treats an *absent* mpris:artUrl as no news
// about the art and keeps drawing the last one it saw, so a track with
// no cover wore the previous album's sleeve — which reads as the wrong
// track playing rather than as missing artwork.
func TestMetadataMapAlwaysCarriesArtURL(t *testing.T) {
t.Parallel()
tests := []struct {
name string
meta Metadata
want string
}{
{
name: "no art at all",
meta: Metadata{Title: "Blue in Green"},
want: "",
},
{
name: "art on disk",
meta: Metadata{
Title: "Blue in Green",
ArtFilePath: "/covers/kind-of-blue_lg.jpg",
},
want: "file:///covers/kind-of-blue_lg.jpg",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
m := metadataMap(tt.meta, 1)
got, ok := m["mpris:artUrl"]
if !ok {
t.Fatal("mpris:artUrl is absent; it must always be sent")
}
if got != tt.want {
t.Errorf("mpris:artUrl = %v, want %q", got, tt.want)
}
})
}
}
// The trackid has to change between tracks or a client is entitled to
// treat the metadata as describing the same track it already has.
func TestMetadataMapTrackIDVaries(t *testing.T) {
t.Parallel()
first := metadataMap(Metadata{Title: "A"}, 1)["mpris:trackid"]
second := metadataMap(Metadata{Title: "B"}, 2)["mpris:trackid"]
if first == second {
t.Errorf("trackid did not change: %v", first)
}
}
// The optional keys stay optional — this is what makes artUrl's
// always-present treatment a deliberate exception rather than drift.
func TestMetadataMapOmitsEmptyOptionalFields(t *testing.T) {
t.Parallel()
m := metadataMap(Metadata{}, 1)
for _, key := range []string{
"xesam:title",
"xesam:artist",
"xesam:album",
"mpris:length",
} {
if _, ok := m[key]; ok {
t.Errorf("%s is present for an empty Metadata", key)
}
}
}
+1
View File
@@ -81,6 +81,7 @@ func (q *Queue) emitTracksModified(
Index: index,
Positions: positions,
CurrentIndex: q.currentIndex,
Source: q.source,
},
)
}
+50
View File
@@ -219,6 +219,56 @@ func TestEmit_AddTrackSendsDeltaNotSnapshot(t *testing.T) {
}
}
// The append clears the source, and the delta is the only event those
// paths emit — so if it does not carry the source, the frontend keeps
// the label it was last given and goes on offering a link back to an
// album the queue no longer holds until something forces a full state.
func TestEmit_AppendDeltaCarriesClearedSource(t *testing.T) {
t.Parallel()
q, db, rec := setupRecordedQueue(t)
paths := seedAudioFiles(t, db, 4)
q.SetQueue(
paths[:3], 0, false,
Source{Type: "album", ID: 1, Label: "Abbey Road"},
)
if _, ok := rec.Wait(events.QueueChanged, waitFor); !ok {
t.Fatalf("no QueueChanged after SetQueue; got %v", rec.Names())
}
rec.Reset()
q.AddTrack(paths[3])
if got := modifiedOf(t, rec).Source; got != (Source{}) {
t.Errorf("delta source = %+v, want zero value", got)
}
}
// And a delta that did not clear it still reports the source it has,
// or the frontend would drop a perfectly good label on every removal.
func TestEmit_NonAppendDeltaCarriesSource(t *testing.T) {
t.Parallel()
q, db, rec := setupRecordedQueue(t)
paths := seedAudioFiles(t, db, 4)
album := Source{Type: "album", ID: 1, Label: "Abbey Road"}
q.SetQueue(paths, 0, false, album)
if _, ok := rec.Wait(events.QueueChanged, waitFor); !ok {
t.Fatalf("no QueueChanged after SetQueue; got %v", rec.Names())
}
rec.Reset()
q.RemoveTrack(3)
if got := modifiedOf(t, rec).Source; got != album {
t.Errorf("delta source = %+v, want %+v", got, album)
}
}
func TestEmit_RemoveTracksReportsPositions(t *testing.T) {
t.Parallel()
+44
View File
@@ -156,12 +156,21 @@ type PlaybackFailure struct {
}
// TracksModified is the payload for the QueueTracksModified event.
//
// Source is carried because an append is exactly what can *invalidate*
// it: a queue built from one album stops being that album the moment a
// track from somewhere else is added to it. The delta is the only event
// those paths emit, so without this the frontend would keep the label
// it was last given and go on saying "Playing from" an album that is no
// longer what is queued — an event carrying what its consumer needs, so
// nothing has to invalidate anything.
type TracksModified struct {
Action string `json:"action"`
Tracks []Track `json:"tracks,omitempty"`
Index int `json:"index"`
Positions []int `json:"positions,omitempty"`
CurrentIndex int `json:"currentIndex"`
Source Source `json:"source"`
}
// Queue manages an ordered list of tracks for playback.
@@ -455,6 +464,8 @@ func (q *Queue) AddTrack(filePath string) {
q.generateShuffleOrder()
}
q.dropSource()
q.persistAddTrack(track)
q.persistState()
q.emitTracksModified(
@@ -505,6 +516,8 @@ func (q *Queue) AddTracks(filePaths []string) {
q.generateShuffleOrder()
}
q.dropSource()
q.persistAddTracks(newTracks)
q.persistState()
q.emitTracksModified(
@@ -563,6 +576,8 @@ func (q *Queue) InsertNextTracks(filePaths []string) {
q.generateShuffleOrder()
}
q.dropSource()
q.persistInsertTracks(newTracks, insertPos)
q.persistState()
q.emitTracksModified(
@@ -613,6 +628,8 @@ func (q *Queue) InsertNext(filePath string) {
q.generateShuffleOrder()
}
q.dropSource()
q.persistInsertTracks([]Track{track}, insertPos)
q.persistState()
q.emitTracksModified(
@@ -680,6 +697,8 @@ func (q *Queue) InsertTracksAt(filePaths []string, index int) {
q.generateShuffleOrder()
}
q.dropSource()
q.persistInsertTracks(newTracks, index)
q.persistState()
q.emitTracksModified(
@@ -1537,6 +1556,31 @@ func (q *Queue) reindexPositions() {
}
}
// dropSource forgets which collection the queue was built from.
//
// A Source is a claim that everything queued came from one album,
// playlist, genre or artist, and the frontend renders it as a
// "Playing from X" link back to that page. Adding or inserting a track
// makes the claim false — the queue is now that album *plus* something
// else — so every path that does so calls this.
//
// It was set by SetQueue and cleared in exactly one place, Clear, so a
// label survived every append. It is persisted too (source_type /
// source_id / source_label on the queue state row), which is what made
// a wrong label outlive the session that earned it: an album queued on
// Monday, added to on Tuesday, still offered a link back to that album
// on Friday.
//
// Removing, reordering and shuffling deliberately do not call this. A
// queue with a track taken out of it, or played in another order, is
// still that album — the link still goes somewhere true. Only the
// arrival of a track from elsewhere makes it a lie.
//
// The caller must hold q.mu.
func (q *Queue) dropSource() {
q.source = Source{}
}
// commitMutation persists the current queue state after a mutation.
// When reindex is true, track positions are renumbered first.
// The caller must hold q.mu.
+111
View File
@@ -126,6 +126,117 @@ func TestClear_ResetsSource(t *testing.T) {
}
}
// A queue built from one album stops being that album the moment a
// track from somewhere else joins it, so every path that adds one
// drops the source. Before this, SetQueue was the only writer and
// Clear the only clearer, so "Playing from Abbey Road" outlived every
// append — and, being persisted, every restart too.
func TestAppendPathsDropSource(t *testing.T) {
t.Parallel()
album := Source{Type: "album", ID: 1, Label: "Abbey Road"}
tests := []struct {
name string
append func(q *Queue, paths []string)
}{
{
name: "AddTrack",
append: func(q *Queue, paths []string) {
q.AddTrack(paths[5])
},
},
{
name: "AddTracks",
append: func(q *Queue, paths []string) {
q.AddTracks(paths[5:7])
},
},
{
name: "InsertNext",
append: func(q *Queue, paths []string) {
q.InsertNext(paths[5])
},
},
{
name: "InsertNextTracks",
append: func(q *Queue, paths []string) {
q.InsertNextTracks(paths[5:7])
},
},
{
name: "InsertTracksAt",
append: func(q *Queue, paths []string) {
q.InsertTracksAt(paths[5:7], 1)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
q, db := setupTestQueue(t)
paths := seedAudioFiles(t, db, 8)
q.SetQueue(paths[:5], 0, false, album)
if got := q.GetState().Source; got != album {
t.Fatalf("source before append: got %+v, want %+v", got, album)
}
tt.append(q, paths)
if got := q.GetState().Source; got != (Source{}) {
t.Errorf(
"source after %s: got %+v, want zero value",
tt.name, got,
)
}
})
}
}
// Removing and reordering deliberately do not drop it: a queue with a
// track taken out of it is still that album, and the link still goes
// somewhere true.
func TestRemoveAndMoveKeepSource(t *testing.T) {
t.Parallel()
album := Source{Type: "album", ID: 1, Label: "Abbey Road"}
t.Run("RemoveTrack", func(t *testing.T) {
t.Parallel()
q, db := setupTestQueue(t)
paths := seedAudioFiles(t, db, 5)
q.SetQueue(paths, 0, false, album)
q.RemoveTrack(3)
if got := q.GetState().Source; got != album {
t.Errorf("source after RemoveTrack: got %+v, want %+v", got, album)
}
})
t.Run("MoveQueueTracks", func(t *testing.T) {
t.Parallel()
q, db := setupTestQueue(t)
paths := seedAudioFiles(t, db, 5)
q.SetQueue(paths, 0, false, album)
q.MoveQueueTracks([]int{0}, 3)
if got := q.GetState().Source; got != album {
t.Errorf(
"source after MoveQueueTracks: got %+v, want %+v",
got, album,
)
}
})
}
func TestSetQueue_WithStartIndex(t *testing.T) {
t.Parallel()
+76 -7
View File
@@ -28,6 +28,7 @@ var (
errUnsupportedOp = errors.New("unsupported operator")
errInvalidSortField = errors.New("invalid sort field: not in allowed field list")
errNotNumeric = errors.New("value must be numeric")
errInvalidMatch = errors.New("match must be \"all\" or \"any\"")
)
// Rule represents a single filter condition for a smart playlist.
@@ -37,13 +38,45 @@ type Rule struct {
Value string `json:"value"`
}
// MatchType decides how a rule set's conditions combine.
//
// The rules used to be joined with " AND " and nothing else, so a
// playlist could only ever narrow: "jazz released after 1960" was
// expressible and "jazz or blues" was not, which is most of what
// anyone reaches for a second rule to say.
type MatchType string
const (
// MatchAll requires every rule to hold — the historical behaviour,
// and what an empty match means so that every rule set written
// before this existed keeps the meaning it was saved with.
MatchAll MatchType = "all"
// MatchAny requires at least one rule to hold.
MatchAny MatchType = "any"
)
// joiner returns the SQL keyword that combines two conditions.
// An unrecognised value cannot reach here — ParseRuleSet rejects one
// — so the default is about the empty string, which is every rule set
// saved before this field existed.
func (m MatchType) joiner() string {
if m == MatchAny {
return " OR "
}
return " AND "
}
// RuleSet holds the complete filter configuration for a smart
// playlist, including optional sort and limit.
type RuleSet struct {
Rules []Rule `json:"rules"`
Limit int `json:"limit,omitempty"`
SortField string `json:"sort_field,omitempty"`
SortDir string `json:"sort_dir,omitempty"`
Rules []Rule `json:"rules"`
// Match is "all" or "any"; empty means "all". It is omitempty so
// an untouched playlist's stored JSON does not change shape.
Match MatchType `json:"match,omitempty"`
Limit int `json:"limit,omitempty"`
SortField string `json:"sort_field,omitempty"`
SortDir string `json:"sort_dir,omitempty"`
}
// fieldMap maps user-facing rule field names to track_metadata column
@@ -116,7 +149,12 @@ const genreDelimiter = "||"
// slice of rules. It is a pure function — no database access needed.
// Returns the clause (without the leading "WHERE"), the parameter
// args, and any validation error.
func BuildWhereClause(rules []Rule) (string, []any, error) {
//
// match decides how the conditions combine; an empty match is MatchAll,
// which is what every rule set saved before the field existed means.
func BuildWhereClause(
rules []Rule, match MatchType,
) (string, []any, error) {
if len(rules) == 0 {
return "", nil, nil
}
@@ -179,7 +217,28 @@ func BuildWhereClause(rules []Rule) (string, []any, error) {
args = append(args, condArgs...)
}
return strings.Join(conditions, " AND "), args, nil
// Under OR, each condition is parenthesised; under AND it is not.
//
// The asymmetry is deliberate rather than an omission. AND is the
// tighter operator in SQL, so an OR-join has to protect any
// condition that contains a top-level AND of its own or the halves
// come apart: `days_since_played less_than` is
// `last_played IS NOT NULL AND <expr> < ?`, which read without
// brackets under an OR-join happens to still parse correctly and
// would stop doing so the moment a condition grows a top-level OR.
// Bracketing under AND would be a no-op semantically and would
// rewrite the clause every existing test pins, so the brackets go
// exactly where they change something.
if match == MatchAny {
bracketed := make([]string, len(conditions))
for i, cond := range conditions {
bracketed[i] = "(" + cond + ")"
}
conditions = bracketed
}
return strings.Join(conditions, match.joiner()), args, nil
}
// validateOperator checks that the operator is valid for the field
@@ -599,7 +658,7 @@ func Evaluate(
start := time.Now()
logger := db.Logger()
where, args, err := BuildWhereClause(ruleSet.Rules)
where, args, err := BuildWhereClause(ruleSet.Rules, ruleSet.Match)
if err != nil {
return nil, fmt.Errorf(
"smart playlist rule error: %w", err,
@@ -1036,6 +1095,16 @@ func ParseRuleSet(jsonStr string) (RuleSet, error) {
)
}
// A match nobody recognises would otherwise fall through to AND,
// which is a playlist quietly returning the wrong tracks rather
// than refusing to be saved. This is the only place a rule set
// enters the backend, so it is the only place that has to ask.
if rs.Match != "" && rs.Match != MatchAll && rs.Match != MatchAny {
return RuleSet{}, fmt.Errorf(
"%w: %q", errInvalidMatch, rs.Match,
)
}
return rs, nil
}
+203 -24
View File
@@ -1,6 +1,7 @@
package smartplaylist
import (
"errors"
"strings"
"testing"
@@ -170,7 +171,7 @@ func TestBuildWhereClause_TextIs(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -189,7 +190,7 @@ func TestBuildWhereClause_TextIsNot(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "artist", Operator: "is_not", Value: "Queen"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -209,7 +210,7 @@ func TestBuildWhereClause_TextContains(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "title", Operator: "contains", Value: "Black"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -231,7 +232,7 @@ func TestBuildWhereClause_TextDoesNotContain(t *testing.T) {
Field: "title", Operator: "does_not_contain",
Value: "Black",
},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -251,7 +252,7 @@ func TestBuildWhereClause_TextStartsWith(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "title", Operator: "starts_with", Value: "Back"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -270,7 +271,7 @@ func TestBuildWhereClause_TextEndsWith(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "title", Operator: "ends_with", Value: "Black"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -292,7 +293,7 @@ func TestBuildWhereClause_TextIsAnyOf(t *testing.T) {
Field: "artist", Operator: "is_any_of",
Value: `["Queen","AC/DC"]`,
},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -312,7 +313,7 @@ func TestBuildWhereClause_NumericIs(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "year", Operator: "is", Value: "1980"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -331,7 +332,7 @@ func TestBuildWhereClause_NumericIsNot(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "year", Operator: "is_not", Value: "1980"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -350,7 +351,7 @@ func TestBuildWhereClause_NumericGreaterThan(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "year", Operator: "greater_than", Value: "2000"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -369,7 +370,7 @@ func TestBuildWhereClause_NumericLessThan(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "year", Operator: "less_than", Value: "1980"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -391,7 +392,7 @@ func TestBuildWhereClause_NumericBetween(t *testing.T) {
Field: "year", Operator: "between",
Value: "1975,1985",
},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -414,7 +415,7 @@ func TestBuildWhereClause_NumericBetweenJSON(t *testing.T) {
Field: "year", Operator: "between",
Value: `["1975","1985"]`,
},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -434,7 +435,7 @@ func TestBuildWhereClause_GenreIsProducesSubquery(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "genre", Operator: "is", Value: "Rock"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -466,7 +467,7 @@ func TestBuildWhereClause_GenreIsNotProducesSubquery(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "genre", Operator: "is_not", Value: "Rock"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -495,7 +496,7 @@ func TestBuildWhereClause_GenreIsAnyOfProducesSubquery(t *testing.T) {
Field: "genre", Operator: "is_any_of",
Value: `["Rock","Pop"]`,
},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -524,7 +525,7 @@ func TestBuildWhereClause_GenreContainsUsesSubquery(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "genre", Operator: "contains", Value: "Rock"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -557,7 +558,7 @@ func TestBuildWhereClause_MultipleRulesAND(t *testing.T) {
clause, args, err := BuildWhereClause([]Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
{Field: "year", Operator: "greater_than", Value: "1975"},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -572,6 +573,110 @@ func TestBuildWhereClause_MultipleRulesAND(t *testing.T) {
}
}
func TestBuildWhereClause_MultipleRulesOR(t *testing.T) {
t.Parallel()
clause, args, err := BuildWhereClause([]Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
{Field: "year", Operator: "greater_than", Value: "1975"},
}, MatchAny)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want := "(artist_name = ? COLLATE NOCASE) OR (year > ?)"
if clause != want {
t.Errorf("clause = %q, want %q", clause, want)
}
if len(args) != 2 || args[0] != "Queen" || args[1] != int64(1975) {
t.Errorf("args = %v, want [Queen 1975]", args)
}
}
// An empty match is what every rule set saved before the field existed
// carries, and it has to keep meaning AND — a playlist silently
// widening to OR on upgrade is the whole risk of adding this field.
func TestBuildWhereClause_EmptyMatchIsAll(t *testing.T) {
t.Parallel()
rules := []Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
{Field: "year", Operator: "greater_than", Value: "1975"},
}
empty, _, err := BuildWhereClause(rules, "")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
all, _, err := BuildWhereClause(rules, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if empty != all {
t.Errorf("empty match = %q, want the same as MatchAll %q",
empty, all)
}
}
// A condition carrying its own top-level AND is what makes the
// bracketing under OR load-bearing: `days_since_played less_than`
// is two predicates, and both belong to the same rule.
func TestBuildWhereClause_ORBracketsCompoundCondition(t *testing.T) {
t.Parallel()
clause, _, err := BuildWhereClause([]Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
{
Field: "days_since_played",
Operator: "less_than",
Value: "30",
},
}, MatchAny)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !strings.Contains(clause, "(last_played IS NOT NULL AND") {
t.Errorf(
"compound condition is not bracketed under OR: %q",
clause,
)
}
}
func TestParseRuleSet_RejectsUnknownMatch(t *testing.T) {
t.Parallel()
_, err := ParseRuleSet(`{"rules":[],"match":"either"}`)
if err == nil {
t.Fatal("expected an error for an unknown match type")
}
if !errors.Is(err, errInvalidMatch) {
t.Errorf("err = %v, want errInvalidMatch", err)
}
}
func TestParseRuleSet_AcceptsAnyAndAll(t *testing.T) {
t.Parallel()
for _, want := range []MatchType{MatchAll, MatchAny} {
rs, err := ParseRuleSet(
`{"rules":[],"match":"` + string(want) + `"}`,
)
if err != nil {
t.Fatalf("match %q: unexpected error: %v", want, err)
}
if rs.Match != want {
t.Errorf("match = %q, want %q", rs.Match, want)
}
}
}
func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
t.Parallel()
@@ -581,7 +686,7 @@ func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
Field: "genre", Operator: "does_not_contain",
Value: "Punk",
},
})
}, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -609,7 +714,7 @@ func TestBuildWhereClause_SameFieldMultipleTimes(t *testing.T) {
func TestBuildWhereClause_EmptyRules(t *testing.T) {
t.Parallel()
clause, args, err := BuildWhereClause(nil)
clause, args, err := BuildWhereClause(nil, MatchAll)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -631,7 +736,7 @@ func TestBuildWhereClause_InvalidField(t *testing.T) {
Field: "nonexistent", Operator: "is",
Value: "anything",
},
})
}, MatchAll)
if err == nil {
t.Fatal("expected error for invalid field, got nil")
}
@@ -654,7 +759,7 @@ func TestBuildWhereClause_InvalidOperatorForNumeric(t *testing.T) {
_, _, err := BuildWhereClause([]Rule{
{Field: "year", Operator: "contains", Value: "1980"},
})
}, MatchAll)
if err == nil {
t.Fatal(
"expected error for text operator on numeric field",
@@ -676,7 +781,7 @@ func TestBuildWhereClause_InvalidOperatorForText(t *testing.T) {
Field: "artist", Operator: "greater_than",
Value: "Queen",
},
})
}, MatchAll)
if err == nil {
t.Fatal(
"expected error for numeric operator on text field",
@@ -723,6 +828,80 @@ func TestEvaluate_TextIs(t *testing.T) {
}
}
// Two rules that share no track at all: under AND this is empty, and
// under OR it is the union. Before Match existed only the first was
// expressible, so a playlist could only ever narrow — "jazz or blues"
// had no way to be said.
func TestEvaluate_MatchAnyUnionsWhereMatchAllIntersects(t *testing.T) {
t.Parallel()
db := database.NewTestDB(t)
seedSmartPlaylistData(t, db)
// Queen has two tracks; Beyoncé has one; no track is by both.
rules := []Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
{Field: "artist", Operator: "is", Value: "Beyoncé"},
}
all, err := Evaluate(db, RuleSet{Rules: rules, Match: MatchAll})
if err != nil {
t.Fatalf("Evaluate(all): %v", err)
}
if len(all) != 0 {
t.Errorf("match=all returned %d tracks, want 0", len(all))
}
either, err := Evaluate(db, RuleSet{Rules: rules, Match: MatchAny})
if err != nil {
t.Fatalf("Evaluate(any): %v", err)
}
if len(either) != 3 {
t.Fatalf("match=any returned %d tracks, want 3", len(either))
}
for _, tr := range either {
if tr.ArtistName != "Queen" && tr.ArtistName != "Beyoncé" {
t.Errorf(
"track %q has artist %q, want Queen or Beyoncé",
tr.TrackName, tr.ArtistName,
)
}
}
}
// An empty match is what every playlist saved before the field existed
// carries, and it has to keep meaning AND all the way through Evaluate
// — a stored playlist silently widening on upgrade is the only real
// risk in adding this.
func TestEvaluate_EmptyMatchStillIntersects(t *testing.T) {
t.Parallel()
db := database.NewTestDB(t)
seedSmartPlaylistData(t, db)
tracks, err := Evaluate(db, RuleSet{
Rules: []Rule{
{Field: "artist", Operator: "is", Value: "Queen"},
{Field: "year", Operator: "greater_than", Value: "1979"},
},
})
if err != nil {
t.Fatalf("Evaluate: %v", err)
}
// Only "Another One Bites the Dust" (Queen, 1980) satisfies both.
if len(tracks) != 1 {
t.Fatalf("got %d tracks, want 1", len(tracks))
}
if want := "Another One Bites the Dust"; tracks[0].TrackName != want {
t.Errorf("got %q, want %q", tracks[0].TrackName, want)
}
}
// TestEvaluate_ArtworkEnrichment verifies the presentation-only
// cover-art and MusicBrainz-ID fields are attached to matched tracks
// by the batched fetchArtwork pass (they are no longer part of the
@@ -1340,7 +1519,7 @@ func TestSQLInjection_FieldName(t *testing.T) {
Field: "title; DROP TABLE playlists",
Operator: "is", Value: "x",
},
})
}, MatchAll)
if err == nil {
t.Fatal(
"expected error for injected field name, got nil",
+134 -57
View File
@@ -16,32 +16,107 @@ var errNoBlockDevice = errors.New(
"no matching block device found",
)
// IsRotationalDisk reports whether the block device backing the
// given path is a rotational (spinning) disk. Detection uses the
// Linux sysfs interface at /sys/block/<dev>/queue/rotational.
// Returns false on any error (assumes SSD).
func IsRotationalDisk(path string) bool {
dev, err := deviceForPath(path)
if err != nil {
return false
}
// DiskProfile is what the scanner needs to know about the device a
// library sits on. Both fields are about the same question — how many
// reads should be in flight at once — and they answer different halves
// of it, so they travel together rather than as two probes.
type DiskProfile struct {
// Device is the whole-disk kernel name ("sdb"), or "" when the
// path could not be resolved to one.
Device string
rotational, err := os.ReadFile(
filepath.Join(
"/sys/block", dev, "queue", "rotational",
),
)
if err != nil {
return false
}
// Rotational is /sys/block/<dev>/queue/rotational: true for a
// spinning disk, where a seek costs milliseconds.
Rotational bool
return strings.TrimSpace(string(rotational)) == "1"
// QueueDepth is /sys/block/<dev>/device/queue_depth — how many
// commands the drive will accept and reorder at once. This is
// NCQ: a SATA disk with it enabled reports 31 or 32, and one
// without reports 1. Zero means the file was not there to read,
// which is the case for anything that is not a SCSI/SATA device
// (NVMe, MMC, device-mapper, loop, a VM's virtio disk).
//
// It is the difference between concurrency helping and hurting.
// With queueing, several outstanding reads let the drive service
// them in the order its head passes over them, which is most of
// why a parallel scan is faster at all. Without it, every extra
// worker is one more seek competing for one head, and the scan
// gets slower the harder it is pushed.
QueueDepth int
}
// deviceForPath resolves a filesystem path to its underlying block
// device name (e.g. "sda") by matching the device major:minor
// from stat(2) against /sys/block/ entries.
func deviceForPath(path string) (string, error) {
// Queues reports whether the drive can reorder outstanding commands.
//
// An unknown depth (0) counts as queueing: everything that does not
// publish this file is a device where concurrency is fine — NVMe has
// its own queues, virtio and device-mapper are not the physical layer
// at all. The only case worth being careful about is the one that
// says so explicitly.
func (p DiskProfile) Queues() bool {
return p.QueueDepth != 1
}
// IsRotationalDisk reports whether the block device backing the
// given path is a rotational (spinning) disk. Returns false on any
// error (assumes SSD).
func IsRotationalDisk(path string) bool {
return ProfileForPath(path).Rotational
}
// ProfileForPath describes the device backing a filesystem path. A
// path that cannot be resolved yields the zero profile, which reads as
// "not rotational, queueing" — the permissive answer, since assuming a
// spinning disk on an SSD would halve a scan for nothing.
func ProfileForPath(path string) DiskProfile {
dev, err := diskForPath(path)
if err != nil {
return DiskProfile{}
}
return DiskProfile{
Device: dev,
Rotational: sysfsInt(dev, "queue", "rotational") == 1,
QueueDepth: sysfsInt(dev, "device", "queue_depth"),
}
}
// sysfsInt reads one small integer out of /sys/block/<dev>/<parts...>,
// returning 0 when it is absent or unparseable. Every attribute here
// is optional: sysfs layout varies by driver, and a missing file is
// "this device does not say", never an error worth propagating.
func sysfsInt(dev string, parts ...string) int {
p := filepath.Join(
append([]string{"/sys/block", dev}, parts...)...,
)
data, err := os.ReadFile(p) //nolint:gosec // sysfs, name from the kernel
if err != nil {
return 0
}
n, err := strconv.Atoi(strings.TrimSpace(string(data)))
if err != nil {
return 0
}
return n
}
// diskForPath resolves a filesystem path to the *whole disk* backing
// it — "sdb" for a file on "sdb3".
//
// It goes through /sys/dev/block/<major>:<minor>, which the kernel
// maintains as a symlink to the device's own sysfs directory, and then
// walks up to the parent when that directory turns out to be a
// partition. The previous implementation scanned /sys/block comparing
// dev numbers and, failing an exact match, took the first entry whose
// *major* agreed — and every SATA disk shares major 8. So a library on
// /dev/sdb3 resolved to whatever /sys/block listed first, which is
// alphabetical, which is sda. On the machine this was found on that
// meant a 6 TB spinning disk was read as the SSD next to it and scanned
// with one worker per core. Matching on major alone cannot be right
// whenever a machine has two disks, which is the case this exists for.
func diskForPath(path string) (string, error) {
var st syscall.Stat_t
if err := syscall.Stat(path, &st); err != nil {
return "", fmt.Errorf(
@@ -49,48 +124,50 @@ func deviceForPath(path string) (string, error) {
)
}
// Extract major and minor device numbers.
major := (st.Dev >> 8) & 0xff
minor := st.Dev & 0xff
// Linux packs dev_t as 12 bits of major and 20 of minor, split
// across the word. Masking the low byte of each — which is what
// this used to do — is right only for the first 256 of either.
major := unixMajor(uint64(st.Dev))
minor := unixMinor(uint64(st.Dev))
// Scan /sys/block/ for a matching device.
entries, err := os.ReadDir("/sys/block")
link := filepath.Join(
"/sys/dev/block",
strconv.FormatUint(major, 10)+":"+
strconv.FormatUint(minor, 10),
)
target, err := filepath.EvalSymlinks(link)
if err != nil {
return "", fmt.Errorf(
"could not read /sys/block: %w", err,
"%w: %s (%w)", errNoBlockDevice, link, err,
)
}
majorStr := strconv.FormatUint(major, 10)
devStr := majorStr + ":" +
strconv.FormatUint(minor, 10)
// A partition's directory sits inside its disk's, and only the
// disk carries `queue`. Climb at most one level: sysfs nests a
// partition exactly one deep under its disk.
name := filepath.Base(target)
for _, entry := range entries {
devFile := filepath.Join(
"/sys/block", entry.Name(), "dev",
)
data, err := os.ReadFile(devFile)
if err != nil {
continue
}
content := strings.TrimSpace(string(data))
if content == devStr {
return entry.Name(), nil
}
// The filesystem might be on a partition (e.g. sda1)
// whose parent block device is sda. Check if the
// major number matches.
parts := strings.SplitN(content, ":", 2)
if len(parts) == 2 && parts[0] == majorStr {
return entry.Name(), nil
}
if _, err := os.Stat(filepath.Join(target, "queue")); err != nil {
name = filepath.Base(filepath.Dir(target))
}
return "", fmt.Errorf(
"%w for %s", errNoBlockDevice, devStr,
)
if name == "" || name == "." || name == string(filepath.Separator) {
return "", fmt.Errorf(
"%w for %d:%d", errNoBlockDevice, major, minor,
)
}
return name, nil
}
// unixMajor and unixMinor decode a Linux dev_t. Spelled out rather
// than taken from golang.org/x/sys/unix so this file stays readable
// beside the encoding it is undoing.
func unixMajor(dev uint64) uint64 {
return (dev>>8)&0xfff | (dev >> 32 & ^uint64(0xfff))
}
func unixMinor(dev uint64) uint64 {
return dev&0xff | (dev >> 12 & ^uint64(0xff))
}
+25
View File
@@ -2,9 +2,34 @@
package system
// DiskProfile is what the scanner needs to know about the device a
// library sits on. See the Linux implementation for what each field
// means; off Linux nothing fills them, because neither macOS nor
// Windows publishes an equivalent of sysfs's `rotational` and
// `queue_depth` without going through platform APIs this package
// deliberately does not link.
type DiskProfile struct {
Device string
Rotational bool
QueueDepth int
}
// Queues reports whether the drive can reorder outstanding commands.
// Always true here: an unknown depth is the permissive answer, and
// assuming otherwise would halve every scan on every Mac.
func (p DiskProfile) Queues() bool {
return p.QueueDepth != 1
}
// IsRotationalDisk reports whether the block device backing the
// given path is a rotational (spinning) disk. On non-Linux
// platforms this always returns false (assumes SSD).
func IsRotationalDisk(_ string) bool {
return false
}
// ProfileForPath describes the device backing a filesystem path. Off
// Linux that is the zero profile, which reads as "an SSD that queues".
func ProfileForPath(_ string) DiskProfile {
return DiskProfile{}
}
+14 -1
View File
@@ -177,12 +177,25 @@ func run(o opts) error {
complete := svc.IndexImportComplete() && !errors.Is(err, errIncomplete)
// Credits are maintenance, not part of any one mode. They come from
// a different dump, they are keyed on entities the catalog already
// holds, and a catalog built before the pass existed would otherwise
// only gain them from a rebuild — which re-downloads ~205 GB to
// re-derive rows it already has. Skipped when the import is not
// complete, because there is nothing to key them against yet.
creditsAdded := false
if complete {
creditsAdded = svc.EnsureArtistCredits(context.Background())
}
// "Changed" means there is something new worth publishing, so it is
// only ever true for a finished import: a build stamps the listens
// series early, long before its rows are assembled, and reporting a
// change off that would be a lie about a half-built index.
changed := complete &&
(svc.IndexBaselineSeries() != seriesBefore || chosen != modeRefresh)
(svc.IndexBaselineSeries() != seriesBefore ||
chosen != modeRefresh ||
creditsAdded)
report(logger, svc, chosen, complete, changed)
+197 -4
View File
@@ -12,6 +12,7 @@ import (
_ "modernc.org/sqlite"
"yellowjacket/backend/database"
"yellowjacket/backend/datamap"
"yellowjacket/backend/system"
)
@@ -53,10 +54,19 @@ func TestRetireLibraryTables(t *testing.T) {
CREATE TABLE recordings (id INTEGER PRIMARY KEY, title TEXT);
`)
// The symptom, before the repair: the schema cannot be applied over
// a table whose shape has moved on.
if _, err := database.NewDB(logger); err == nil {
t.Fatal("expected the stale shape to fail to open; it did not")
// This used to assert the symptom -- that the schema cannot be
// applied over a table whose shape has moved on -- because at the
// time nothing repaired it and only this job did. The app-side
// repair (backend/database/staleshape.go) now retires a stale
// non-authored table before applySchema meets it, so opening
// succeeds and the symptom no longer reproduces from here.
//
// That does not make retireLibraryTables redundant, and the rest of
// this test is why: the app-side repair only removes what is *stale*,
// while this database wants its library half gone entirely, healthy
// or not, because nothing here scans, plays or authors.
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("the app-side repair should have opened this: %v", err)
}
if err := retireLibraryTables(context.Background(), logger); err != nil {
@@ -116,3 +126,186 @@ func count(t *testing.T, dbPath, query string) int {
return n
}
// TestTheCatalogSurvivesAStaleShape is the accident written down.
//
// The app repairs a stale Cache table by dropping it: its catalog is
// downloaded, so a wrong shape costs about a minute of re-fetching and
// keeping it costs every Explore read. Applied here that rule is
// catastrophic — this database is what the artifact is *cut from*, so
// there is nothing to re-fetch and the only way back is the ~205 GB
// dump stream the /cache volume exists to avoid.
//
// It shipped without that distinction and dropped the real CI catalog
// on the first run:
//
// retiring a table ... table=explore_index
// reason="column entity_type is TEXT, schema declares INTEGER"
// index maintenance mode=build reason="no completed import yet"
//
// The mismatch was real: that database is deliberately kept in the
// older encoding, which `fix(indexexport): read an index older than the
// binary` exists to tolerate. So the shape will not match, every run,
// by design — and the catalog must survive it anyway.
func TestTheCatalogSurvivesAStaleShape(t *testing.T) {
logger := slog.New(slog.DiscardHandler)
t.Setenv("YJ_HOME", t.TempDir())
dataDir, err := system.GetUserDataDirPath()
if err != nil {
t.Fatalf("resolve data dir: %v", err)
}
dbPath := filepath.Join(dataDir, "yj.db")
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("first open: %v", err)
}
// The shape the real index database is in: every current column,
// but the ids and the entity type still text. That is what the
// exporter's backward-compatibility fix tolerates, and it is what
// the repair saw and called stale.
exec(t, dbPath, `
DROP TABLE explore_index;
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type TEXT NOT NULL,
mbid TEXT NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid TEXT NOT NULL,
aliases TEXT NOT NULL DEFAULT '',
popularity INTEGER NOT NULL DEFAULT 0,
listener_count INTEGER NOT NULL DEFAULT 0,
duration INTEGER NOT NULL DEFAULT 0,
caa_release_mbid TEXT NOT NULL DEFAULT '',
release_name TEXT NOT NULL DEFAULT '',
primary_type TEXT NOT NULL DEFAULT '',
secondary_types TEXT NOT NULL DEFAULT '',
release_date TEXT NOT NULL DEFAULT '',
total_tracks INTEGER NOT NULL DEFAULT 0,
artist_type TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
disambiguation TEXT NOT NULL DEFAULT '',
sort_name TEXT NOT NULL DEFAULT '',
in_library INTEGER NOT NULL DEFAULT 0,
is_similar INTEGER NOT NULL DEFAULT 0,
local_artist_id INTEGER,
local_release_group_id INTEGER,
local_recording_id INTEGER,
discog_fetched INTEGER NOT NULL DEFAULT 0,
UNIQUE(mbid)
);
INSERT INTO explore_index
(entity_type, mbid, title, artist_name, artist_mbid)
VALUES ('artist', 'a-b-c', 'A Catalog Row', 'An Artist', 'd-e-f');
`)
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("open with a stale catalog shape: %v", err)
}
if got := count(t, dbPath, "SELECT COUNT(*) FROM explore_index"); got != 1 {
t.Fatalf(
"explore_index rows = %d, want 1 — the catalog was retired, "+
"which costs this database a ~205GB rebuild",
got,
)
}
}
// TestNoCacheTableIsRetiredHere is the general form of the accident
// above, and it exists because the specific one is not the risk.
//
// `TestTheCatalogSurvivesAStaleShape` pins one table in one wrong shape,
// which is the failure that happened. What cost the ~205 GB was not that
// shape: it was a destructive repair added to `database.NewDB` -- the
// one chokepoint every binary in this project shares -- without asking
// which binary it was running in. The next such repair will have a
// different name and a different reason, and this database still cannot
// afford it.
//
// So the assertion is about the *outcome* rather than the mechanism: put
// every Cache table in a shape the schema has certainly moved past, open
// the database the way cmd/indexbuild does, and require that all of them
// are still there afterwards. Any future repair that drops one fails
// here regardless of how it decides to.
//
// Two things about it are deliberate.
//
// The table list comes from `datamap.ByKind(Cache)` rather than being
// written out, so a Cache table added next year is covered by this test
// on the day it is added -- the same reason `TestCatalogCoversSchema`
// reads the schema instead of a list.
//
// And `NewDB` returning an error is *accepted*, because that is the
// trade the fix documents: with Cache tables no longer rebuilt here, a
// shape the schema moved past now fails this job loudly instead of
// silently costing it a day of downloading. Loud is fine. Gone is not.
func TestNoCacheTableIsRetiredHere(t *testing.T) {
logger := slog.New(slog.DiscardHandler)
t.Setenv("YJ_HOME", t.TempDir())
dataDir, err := system.GetUserDataDirPath()
if err != nil {
t.Fatalf("resolve data dir: %v", err)
}
dbPath := filepath.Join(dataDir, "yj.db")
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("first open: %v", err)
}
// An FTS table is four shadow tables and cannot be given a "wrong
// shape" meaningfully; the repair skips them for the same reason and
// retires them with their parent, which the parents below cover.
var cache []string
for _, table := range datamap.ByKind(datamap.Cache) {
if table.FTS {
continue
}
cache = append(cache, table.Name)
}
if len(cache) == 0 {
t.Fatal("no Cache tables to check: the datamap or this test is wrong")
}
for _, name := range cache {
// A shape nothing in the current schema describes. What matters
// is only that it disagrees; the real mismatch was one column's
// type.
exec(t, dbPath, `
DROP TABLE IF EXISTS `+name+`;
CREATE TABLE `+name+` (id INTEGER PRIMARY KEY, moved_past TEXT);
INSERT INTO `+name+` (moved_past) VALUES ('irreplaceable');
`)
}
// The error is not the assertion: see the note above.
_, _ = database.NewDB(logger)
for _, name := range cache {
rows := count(t, dbPath,
`SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = '`+name+`'`)
if rows == 0 {
t.Errorf("%s was retired: in this database a Cache table is derived, "+
"not downloaded, and dropping one costs the ~205 GB dump stream", name)
continue
}
// Present but emptied is the same loss wearing a different
// shape: SQLite does an implicit DELETE before a DROP, and a
// repair that recreated the table would look identical here.
if n := count(t, dbPath, `SELECT count(*) FROM `+name); n == 0 {
t.Errorf("%s survived but was emptied", name)
}
}
}
+77
View File
@@ -207,6 +207,26 @@ func createSchema(db *sql.DB) error {
key TEXT PRIMARY KEY,
value TEXT NOT NULL
)`,
// Multi-artist credits. Shipped as their own tables rather than
// as an explore_index column because a credit is a variable
// number of ordered parts, and because credits are *shared* --
// an album's tracks by one artist reference one credit, which is
// what keeps this to a few hundred thousand rows.
//
// An importer that predates these reads an artifact without
// them; artifactHasCredits is what asks.
`CREATE TABLE core.artist_credit_part (
credit_id INTEGER NOT NULL,
position INTEGER NOT NULL,
artist_mbid BLOB NOT NULL,
credited_name TEXT NOT NULL,
join_phrase TEXT NOT NULL DEFAULT '',
PRIMARY KEY (credit_id, position)
) WITHOUT ROWID`,
`CREATE TABLE core.artist_credit_ref (
mbid BLOB NOT NULL PRIMARY KEY,
credit_id INTEGER NOT NULL
) WITHOUT ROWID`,
}
for _, stmt := range stmts {
@@ -283,6 +303,63 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
fmt.Printf(" %-15s %d\n", sel.label+":", n)
}
return copyCredits(db)
}
// copyCredits ships the credit decomposition for the entities that made
// it into the artifact, and only those.
//
// The refs go first and the parts follow *from* the refs, so a credit is
// carried only if something in the artifact points at it. The source
// index holds credits for every catalog entity, while the artifact is a
// windowed subset -- copying all of them would carry a large table most
// of which nothing in the artifact can reach.
//
// A source index built before the credit pass simply has no rows here,
// which is not an error: the artifact then carries the tables empty, and
// every credit falls back to its single artist exactly as before.
func copyCredits(db *sql.DB) error {
// Asked, not assumed. A source index built before the credit pass
// has no such table, and "no such table" would fail an export whose
// catalog is otherwise complete.
for _, table := range []string{"artist_credit_ref", "artist_credit_part"} {
var n int
if err := db.QueryRow(
`SELECT COUNT(*) FROM main.sqlite_master
WHERE type = 'table' AND name = ?`, table,
).Scan(&n); err != nil {
return fmt.Errorf("probe %s: %w", table, err)
}
if n == 0 {
fmt.Printf(" %-15s none in source\n", "credits:")
return nil
}
}
refs, err := insertSelect(db, `
INSERT INTO core.artist_credit_ref (mbid, credit_id)
SELECT r.mbid, r.credit_id
FROM main.artist_credit_ref r
WHERE r.mbid IN (SELECT mbid FROM core.explore_index)`)
if err != nil {
return err
}
parts, err := insertSelect(db, `
INSERT INTO core.artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
SELECT p.credit_id, p.position, p.artist_mbid, p.credited_name, p.join_phrase
FROM main.artist_credit_part p
WHERE p.credit_id IN (SELECT credit_id FROM core.artist_credit_ref)`)
if err != nil {
return err
}
fmt.Printf(" %-15s %d refs, %d parts\n", "credits:", refs, parts)
return nil
}
+27 -1
View File
@@ -3,7 +3,33 @@
`.gitea/workflows/android-apk.yml` builds a signed `arm64-v8a` APK on
every `v*` tag and publishes it to Gitea's
**generic** package registry, which is readable without credentials —
which is what lets Obtainium poll a plain URL with no token.
which is what lets Obtainium poll a plain URL with no token. It also
attaches the same file to the Gitea release, which is what a person
looking at the release page downloads.
**Tags are not pushed by hand any more.** `.gitea/workflows/release.yml`
reads the Conventional Commits on every merge to `main`, decides the
version, and pushes the tag this workflow is keyed on — so releasing the
APK means merging a `fix:` or `feat:` commit, not running `git tag`. The
`workflow_dispatch` path below remains, for rebuilding a tag that already
exists.
## The 1.x installs cannot be upgraded to 0.0.x
Releases restarted at **0.0.1** when they became automatic (plan 017).
`versionCode` is computed as `maj*10000 + min*100 + pat`, so 0.0.1 is
**1** against the **10300** an installed 1.3.0 build carries — and
**Android refuses a downgrade outright**, with
`INSTALL_FAILED_VERSION_DOWNGRADE`.
The only way through is `adb uninstall app.yellowjacket` (or the
launcher's own uninstall) before installing 0.0.1, **and that takes the
device's library, playlists and play counts with it** — the same loss the
signing key section below exists to prevent, arrived at from the other
direction. This was chosen deliberately over offsetting `versionCode` by
a constant, on the grounds that the honest number is worth one reinstall
while an offset is permanent. The workflow prints a warning whenever the
code it computes is below 10600.
```
https://git.ljones.me/api/packages/yonlu/generic/yellowjacket-android/latest/yellowjacket.apk
-194
View File
@@ -1,194 +0,0 @@
# Config Improvement Suggestions
Remaining suggestions for improving the configuration system in YellowJacket.
## 2. Thread Safety Concerns
The current `Config` struct lacks synchronization:
- `Load()` and `Save()` can race with concurrent reads
- `handleConfigUpdate()` in library mutates `l.conf.DirectoryPath` without locks
**Suggestion:** Add a `sync.RWMutex` to protect config access, especially if config is read during scans.
```go
type Config struct {
mu sync.RWMutex
ctx context.Context
logger *slog.Logger
// ...
}
func (c *Config) Load() error {
c.mu.Lock()
defer c.mu.Unlock()
// ...
}
```
## 3. Nil Safety in Validation
In `config.go`, validation only runs if `c.Library != nil`, but `handleConfigPost` dereferences `postedConfig.Library` without checking for nil:
```go
if postedConfig.Library != nil {
c.Library = postedConfig.Library
// ...
}
```
**Status:** Partially addressed in the event refactor, but consider adding explicit nil checks in `Validate()` as well.
## 4. Inconsistent Error Handling on HTTP Responses
In `httphandler.go:28-31`, `WriteHeader` is called *after* rendering the error template, which won't work as expected (headers must be set before writing body):
```go
c.formSubmitError(err.Error()).Render(r.Context(), w)
w.WriteHeader(http.StatusInternalServerError) // Too late!
```
**Fix:** Set the status code before rendering:
```go
w.WriteHeader(http.StatusInternalServerError)
c.formSubmitError(err.Error()).Render(r.Context(), w)
```
## 5. Make `scanWorkerCount` Configurable
There's a TODO at `library.go:289`:
```go
// TODO: make configurable via Config.
var scanWorkerCount = goruntime.NumCPU()
```
**Suggestion:** Add this to `library.Config`:
```go
type Config struct {
DirectoryPath Directory `form:"Directory" schema:"directory,required"`
ScanWorkers int `form:"ScanWorkers" schema:"scan_workers"`
}
```
Then in `NewLibrary()` or `Scan()`:
```go
workers := l.conf.ScanWorkers
if workers <= 0 {
workers = goruntime.NumCPU()
}
```
## 6. Consider Config Defaults
Currently if no config exists, an empty one is saved. Consider providing sensible defaults (e.g., common music directories like `~/Music`).
```go
func (c *Config) setDefaults() {
if c.Library == nil {
c.Library = &library.Config{}
}
if c.Library.DirectoryPath == "" {
// Try common music directories
home, _ := os.UserHomeDir()
musicDir := filepath.Join(home, "Music")
if info, err := os.Stat(musicDir); err == nil && info.IsDir() {
c.Library.DirectoryPath = library.Directory(musicDir)
}
}
}
```
## 7. Config Reload/Watch Capability
The config is only loaded at startup. Consider adding:
- File watcher for external config changes (using `fsnotify`)
- Explicit reload method callable from UI
```go
func (c *Config) Watch() error {
watcher, err := fsnotify.NewWatcher()
if err != nil {
return err
}
go func() {
for event := range watcher.Events {
if event.Op&fsnotify.Write == fsnotify.Write {
c.Load()
// Emit event for listeners
}
}
}()
return watcher.Add(c.filePath)
}
```
## 8. Validation Should Return Structured Errors
Currently validation returns combined errors. Consider returning a structured validation result that the UI can map to specific fields for better user feedback.
```go
type ValidationError struct {
Field string
Message string
}
type ValidationResult struct {
Valid bool
Errors []ValidationError
}
func (c *Config) ValidateStructured() ValidationResult {
var result ValidationResult
result.Valid = true
if c.Library != nil {
if err := c.Library.Validate(); err != nil {
result.Valid = false
result.Errors = append(result.Errors, ValidationError{
Field: "Library.DirectoryPath",
Message: err.Error(),
})
}
}
return result
}
```
## 9. Use Standard Library for Config Paths
The path construction in `system/userdata.go` doesn't respect `$XDG_CONFIG_HOME` on Linux or use the standard Go `os.UserConfigDir()`.
**Current implementation:**
```go
case "linux":
return fmt.Sprintf("/home/%s/%s/yellowjacket", username, unixSubdirs[dt]), nil
```
**Suggested improvement:**
```go
func GetUserConfigDirPath() (string, error) {
baseDir, err := os.UserConfigDir() // Respects XDG_CONFIG_HOME
if err != nil {
return "", fmt.Errorf("could not get user config directory: %w", err)
}
path := filepath.Join(baseDir, "yellowjacket")
if err := os.MkdirAll(path, 0o755); err != nil {
return "", fmt.Errorf("could not create config directory: %w", err)
}
return path, nil
}
```
This approach:
- Respects `$XDG_CONFIG_HOME` on Linux
- Uses proper macOS paths (`~/Library/Application Support`)
- Uses `%AppData%` on Windows
- Is more portable and follows platform conventions
-53
View File
@@ -1,53 +0,0 @@
# Development Overview
YellowJacket is a moderately complex application. This document gives an overview of how development of it works.
## Logical Breakdown
YellowJacket can be thought about in a heirarchy of logical modules and components. The borders of these logical sections are mostly represented in the code and directory structure as well.
- Frontend
- UI Components (see [Lit](###lit-web-components))
- Backend
- App
- Asset Handler
- Logging
- System
- Player
- Library
- Config
- Database
- Queries (see [sqlc](###sqlc))
## Dependencies
YellowJacket uses many tools and libraries to provide its functionality.
This section lists each of these dependencies and explains how they are used.
### [Wails](https://wails.io)
Used to create desktop apps with Go and web technologies.
### [SQLite](https://github.com/mattn/go-sqlite3?tab=readme-ov-file#go-sqlite3)
Used for local database.
### [sqlc](https://sqlc.dev/)
Used to generate Go code from SQL.
### [Templ](https://templ.guide/)
Used to generate HTML templates with Go code.
### [Beep](https://github.com/gopxl/beep?tab=readme-ov-file#beep)
Used for audio playback.
### [Lit Web Components](https://lit.dev/)
Used for dynamic/reactive frontend components.
### [HTMX](https://htmx.org/)
Used for requesting HTML fragments from the backend and rendering them on the frontend.
-1648
View File
File diff suppressed because it is too large Load Diff
+99
View File
@@ -0,0 +1,99 @@
# The index cache, and why it has a snapshot
`/srv/yellowjacket/index-cache` on the Gitea host is the `YJ_HOME` the
search-index job keeps between runs — `.gitea/workflows/index-artifact.yml`
mounts it at `/cache`. It holds the catalog every user eventually
downloads, and it is the one database in this project that is
**derived rather than downloaded**.
That is the whole reason this document exists. An install with a broken
catalog re-fetches the ~0.6 GB artifact and is fine in a minute. This
database *is* what that artifact is cut from, so its only route back is
re-streaming the MetaBrainz dumps: hours, at a rate that belongs to
someone else's server, holding a runner of capacity 1 the entire time.
## What happened on 2026-08-17
A schema repair (`fix(database): retire a table whose shape the schema
moved past`) dropped every table whose live shape disagreed with the
schema, before `applySchema`. Correct for the app. Applied here it
deleted the catalog 19 seconds into the first run:
```
retiring a table ... table=explore_index
reason="column entity_type is TEXT, schema declares INTEGER"
index maintenance mode=build reason="no completed import yet"
```
The mismatch was real and deliberate: this database is kept in the older
text encoding, which `artifactStoresText` and `sourceColumns` exist to
tolerate. So it would have been judged stale on *every* run.
Two things came out of it. `retireStaleCache` is now a build tag —
false under `indexbuild`, true in the app — and
`TestNoCacheTableIsRetiredHere` asserts the outcome rather than the
mechanism, so the next destructive repair fails a test instead of a
production volume. And the volume got the snapshot it should always have
had, below.
## Taking snapshots
```sh
scripts/index-cache-snapshot.sh [SOURCE_HOME] [DEST_DIR] [KEEP]
```
Defaults: `/srv/yellowjacket/index-cache`, `/srv/yellowjacket/index-snapshots`,
keep 2. On the Gitea host, daily and away from the Monday 04:00 build:
```
30 5 * * * /path/to/index-cache-snapshot.sh >> /var/log/yj-index-snapshot.log 2>&1
```
Three properties worth knowing before trusting it:
- **It uses `VACUUM INTO`, not `cp`.** The database may be open, and a
byte copy of a live SQLite file is a corrupt file of plausible size.
`VACUUM INTO` takes a read lock and writes a consistent, compacted
copy; it is safe to run while a build is in progress.
- **It does not copy `data/explore-staging`.** That is a resumable
checkpoint of work in flight — large, constantly changing, and a build
resumes without it. What cannot be cheaply re-derived is the finished
catalog, which is in the database.
- **It verifies before it rotates.** Each snapshot is reopened and asked
for its catalog row count; a run that produces an unreadable or empty
file fails loudly, deletes its own output, and leaves the previous
snapshots alone. Both paths are exercised, not assumed.
## Restoring
Stop anything that might be using the volume first — the job holds it
for the length of a build, and the concurrency group (`search-index`)
means a queued run will start the moment one ends.
```sh
cd /srv/yellowjacket
mv index-cache/data/yj.db index-cache/data/yj.db.broken # keep it until you are sure
cp index-snapshots/yj-index-<stamp>.db index-cache/data/yj.db
chown --reference=index-cache/data/yj.db.broken index-cache/data/yj.db
```
Then dispatch the workflow with `mode=auto`. A restored snapshot is
older than the dumps, so `indexbuild` resolves to `refresh` and folds in
the incremental listens since — which is minutes, not hours.
Two notes on what a restore does *not* need. The staging directory can
be deleted; it will be rebuilt if a build is needed. And the published
artifact is untouched by any of this: users keep downloading the last
good one until a run reports `complete=true` and `changed=true`
republishes.
## The trade this leaves open
With Cache tables no longer retired under `indexbuild`, a future
`explore_index` column change will fail this job **loudly** — at
`applySchema`, or at the first query naming the column — rather than
silently rebuilding. That is the right default: loud is recoverable and
a silent day of downloading is not. It does mean the next schema change
touching `explore_index` needs a deliberate plan for this one database:
take a snapshot, apply the change to a copy, or accept a rebuild
knowingly.
+66
View File
@@ -0,0 +1,66 @@
import { test, expect } from '../support/fixtures.js';
/**
* The queue button says whether the queue is open.
*
* It used to look identical in both states, so the only way to tell
* what pressing it would do was to look at the other side of the window
* and infer it and for anyone not looking at all there was nothing to
* infer from: no `aria-expanded`, no `aria-controls`, no pressed state.
*
* The state is reflected *from the panel*, not kept beside the click,
* because the button is not the only thing that opens the queue
* `now-playing-view` sets the same attribute, since it hides the bar
* this button lives in. A flag maintained by the click handler would be
* right until something else opened the panel and then quietly wrong,
* which is the second test here.
*/
test.describe('the queue toggle', () => {
test('reports open and closed, and names what it controls', async ({
app,
}) => {
const toggle = app.locator('#queue-button');
await expect(toggle).toHaveAttribute('aria-controls', 'queue-panel');
await expect(toggle).toHaveAttribute('aria-expanded', 'false');
await toggle.click();
await expect(toggle).toHaveAttribute('aria-expanded', 'true');
// The state is not only in the accessibility tree: a control that
// announces a state it does not draw is half a fix.
//
// Background rather than colour, because the pointer is still on
// the button after the click and `:hover` paints it the same accent
// the open state does -- so a colour comparison here passes on the
// broken build and proves nothing.
const [open, closed] = await toggle.evaluate((el) => {
const now = getComputedStyle(el).backgroundColor;
el.setAttribute('aria-expanded', 'false');
const shut = getComputedStyle(el).backgroundColor;
el.setAttribute('aria-expanded', 'true');
return [now, shut];
});
expect(open).not.toBe(closed);
await toggle.click();
await expect(toggle).toHaveAttribute('aria-expanded', 'false');
});
test('follows the panel when something else opens it', async ({ app }) => {
const toggle = app.locator('#queue-button');
await expect(toggle).toHaveAttribute('aria-expanded', 'false');
// Exactly what `now-playing-view`'s queue button does.
await app.evaluate(() =>
document.getElementById('queue-panel')?.setAttribute('open', ''),
);
await expect(toggle).toHaveAttribute('aria-expanded', 'true');
});
});
+2 -2
View File
@@ -7,7 +7,7 @@ import { test, expect, callBinding } from '../support/fixtures.js';
* and produced two: every one of the eight call sites was a two-way
* ternary, so an album already on the request list showed a plus and
* said "is not in your library" on the same page, forty pixels from a
* filled button reading "Wanted".
* filled button reading "Requested".
*
* This spec exists at this tier rather than only in the component one
* because of what it drags in with it: reaching the requested state is
@@ -181,7 +181,7 @@ test.describe('the requested badge', () => {
const ds = document.querySelector('explore-album-details')
?.shadowRoot;
const btn = [...(ds?.querySelectorAll('wa-button') ?? [])].find(
(b) => /Wanted/.test(b.textContent ?? ''),
(b) => /Requested/.test(b.textContent ?? ''),
);
return btn?.querySelector('wa-icon')?.getAttribute('name') ?? '';
@@ -3,28 +3,52 @@
/**
* AutoDownloadPrefs gates and scores what AutoPickable may choose
* without asking. Zero values are permissive: no size window and no
* format restriction.
* without asking. Zero values are permissive: no bitrate window, no
* size ceiling and no format restriction.
*
* **The window is a rate, not a size.** It used to be three numbers in
* megabytes, which cannot mean anything on their own: 300 MB is a
* generous FLAC single and a suspiciously small boxset, and the user
* setting the number has no idea which release the pipeline will
* eventually apply it to. A bitrate is the same statement normalised
* by how long the music is, so one number holds across a 9-minute EP
* and a 3-hour opera and it is the unit the thing being described is
* actually measured in. The runtime is known for every request
* auto-pick can act on (`Download.Expected` carries per-track lengths,
* and an anchored request is the only kind that reaches here), so this
* costs no extra lookup.
*/
export interface AutoDownloadPrefs {
/**
* MinSizeMB and MaxSizeMB bound what auto-pick will grab. Zero
* means no bound on that side. A candidate outside the window is
* filtered out of auto-pick entirely, not merely scored down a
* tiny "sampler" torrent or a boxset ten times the expected size is
* usually the wrong thing entirely, not a worse copy of the right
* thing.
* MinKbps and MaxKbps bound the average bitrate auto-pick will
* grab. Zero means no bound on that side. A candidate outside the
* window is filtered out of auto-pick entirely, not merely scored
* down a 96 kbps rip of the right album is not a worse copy the
* user might accept, it is one they said not to take unattended.
*
* For reference: 320 is the top of MP3, ~5001000 is FLAC depending
* on the material, and anything under ~128 is a transcode.
*/
"minSizeMb": number;
"maxSizeMb": number;
"minKbps": number;
"maxKbps": number;
/**
* PreferredSizeMB nudges the score toward a target size within the
* min/max window (a lossless rip and a heavily-padded lossless rip
* can both pass the window). Zero disables the nudge; sizeFit then
* returns a neutral value that does not affect ranking.
* PreferredKbps nudges the score toward a target rate within the
* window, and breaks the tie when several candidates are equally
* good matches. Zero disables the nudge; bitrateFit then returns a
* neutral value that does not affect ranking.
*/
"preferredSizeMb": number;
"preferredKbps": number;
/**
* MaxSizeMB is a hard ceiling on the whole candidate, and it is
* deliberately still a size. It answers a different question from
* the window above not "is this the quality I want" but "is this
* going to fill the disk" and it has to hold even for a candidate
* whose bitrate cannot be worked out, which is exactly the shape a
* mislabelled boxset arrives in. Zero means no ceiling.
*/
"maxSizeMb": number;
/**
* AllowedFormats restricts auto-pick to candidates whose audio
@@ -487,9 +511,13 @@ export interface QualityScore {
"priority": number;
/**
* closeness to the preferred download size
* BitrateFit is closeness to the preferred *rate*, which is what
* the auto-download window is expressed in. It replaced a
* `SizeFit` measured in megabytes: a size means nothing without
* knowing how long the music is, so the same number described a
* generous single and a suspiciously small boxset.
*/
"sizeFit": number;
"bitrateFit": number;
/**
* Mixed marks a candidate whose files are not all the same format,
@@ -12,6 +12,7 @@ export {
export type {
AlbumCompleteFunc,
CreditPart,
IndexStatus,
LBSimilarArtist,
LBTopRecording,
@@ -7,6 +7,23 @@
*/
export type AlbumCompleteFunc = any;
/**
* CreditPart is one credited artist within a credit, in credit order.
*
* CreditedName is the name *as credited*, which is not the artist's own
* name: MusicBrainz credits "Snoop Dogg" on a track by the artist
* called "Snoop Doggy Dogg". Display uses it; navigation uses
* ArtistMBID. JoinPhrase is the literal connector that follows this
* part, so a credit renders by concatenation and never by searching a
* name inside a credit string.
*/
export interface CreditPart {
"position": number;
"artistMbid": string;
"creditedName": string;
"joinPhrase": string;
}
/**
* IndexStatus is the full index build status, exposed to the frontend.
*/
@@ -225,6 +225,18 @@ export function GetCandidateThumbnail(releaseMBID: string, releaseGroupMBID: str
return $Call.ByID(1946932424, releaseMBID, releaseGroupMBID);
}
/**
* GetCredits is the bound form: the frontend asks for a tracklist's
* worth of MBIDs at once rather than one per row.
*
* Batched for the reason every other per-row backend question here is:
* asking on hover or on render turns a list into N IPC round trips, and
* this one is asked about every row of every list in the app.
*/
export function GetCredits(mbids: string[] | null): $CancellablePromise<{ [_ in string]?: $models.CreditPart[] | null } | null> {
return $Call.ByID(225964099, mbids);
}
/**
* GetExploreShelves builds the page Explore shows before a query.
*
+11
View File
@@ -207,6 +207,17 @@ body div.sidebar {
color: var(--yj-accent, #ffd43b);
}
/* An open queue is a state this button can be in, and it used to
look exactly like the closed one -- so the only way to tell what
pressing it would do was to look at the other side of the window
and infer it. `aria-expanded` is the same fact for anyone not
looking at all, and it points at the panel it controls. */
#queue-button[aria-expanded='true'] {
color: var(--yj-accent, #ffd43b);
background: var(--yj-bg-overlay, #404040);
border-radius: 4px;
}
#queue-button.drag-over {
color: var(--yj-accent, #ffd43b);
outline: 2px dashed var(--yj-accent, #ffd43b);
+2 -1
View File
@@ -37,7 +37,8 @@
<footer class="bottom-bar">
<now-playing></now-playing>
<audio-player></audio-player>
<button aria-label="Toggle queue" id="queue-button">
<button aria-label="Toggle queue" aria-controls="queue-panel" aria-expanded="false"
id="queue-button">
<wa-icon name="list"></wa-icon>
</button>
</footer>
+22
View File
@@ -521,6 +521,28 @@ if (queueButton && queuePanel) {
}
});
// The button says whether the panel is open, and it learns that
// from the panel rather than from its own click handler.
//
// It is not the only thing that opens the queue -- `now-playing-view`
// sets the same attribute, because it hides the bar this button
// lives in -- so a state kept beside the click would be right until
// something else opened the panel and then quietly wrong. The panel's
// `open` attribute is the one fact; this reflects it.
const reflectQueueState = () => {
queueButton.setAttribute(
'aria-expanded',
String(queuePanel.hasAttribute('open')),
);
};
new MutationObserver(reflectQueueState).observe(queuePanel, {
attributes: true,
attributeFilter: ['open'],
});
reflectQueueState();
// ---------------------------------------------------------------
// Queue button as drop target (when queue panel is closed)
// ---------------------------------------------------------------
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M288.1-32c9 0 17.3 5.1 21.4 13.1L383 125.3 542.9 150.7c8.9 1.4 16.3 7.7 19.1 16.3s.5 18-5.8 24.4L441.7 305.9 467 465.8c1.4 8.9-2.3 17.9-9.6 23.2s-17 6.1-25 2L288.1 417.6 143.8 491c-8 4.1-17.7 3.3-25-2s-11-14.2-9.6-23.2L134.4 305.9 20 191.4c-6.4-6.4-8.6-15.8-5.8-24.4s10.1-14.9 19.1-16.3l159.9-25.4 73.6-144.2c4.1-8 12.4-13.1 21.4-13.1zm0 76.8L230.3 158c-3.5 6.8-10 11.6-17.6 12.8l-125.5 20 89.8 89.9c5.4 5.4 7.9 13.1 6.7 20.7l-19.8 125.5 113.3-57.6c6.8-3.5 14.9-3.5 21.8 0l113.3 57.6-19.8-125.5c-1.2-7.6 1.3-15.3 6.7-20.7l89.8-89.9-125.5-20c-7.6-1.2-14.1-6-17.6-12.8L288.1 44.8z"/></svg>

After

Width:  |  Height:  |  Size: 889 B

@@ -10,6 +10,7 @@ import type {
VisibilityChangedEvent,
} from '@lit-labs/virtualizer';
import { grid } from '@lit-labs/virtualizer/layouts/grid.js';
import { gridSpacingFor } from '@utils/grid-spacing';
import {
GetAlbumsByArtist,
GetFilePathsByAlbums,
@@ -147,8 +148,6 @@ export class ArtistsView
// ----- Grid spacing constants -----
private static readonly GRID_GAP = 8;
private static readonly GRID_PADDING = 8;
private static readonly CARD_PADDING = 5;
private get imageSize(): number {
@@ -177,20 +176,41 @@ export class ArtistsView
private createGridLayout() {
const w = this.cardSize ?? CARD_SIZE_DEFAULT;
const h = w + this.cardTextHeight;
const gap = ArtistsView.GRID_GAP;
const pad = ArtistsView.GRID_PADDING;
// One number for the gap, the row gap and the padding: whatever
// a row could not spend on another card, shared out equally, so
// the outside is never wider than the inside. See
// `utils/grid-spacing.ts`.
const spacing = this.spacingFor(this.containerWidth);
this.lastLayoutSpacing = spacing;
return grid({
itemSize: {
width: `${w}px`,
height: `${h}px`,
},
gap: `${gap}px`,
padding: `${pad}px`,
justify: 'center',
gap: `${spacing}px`,
padding: `${spacing}px`,
justify: 'start',
});
}
/** The width the grid lays itself out in. */
private get containerWidth(): number {
return (
this.renderRoot?.querySelector<HTMLElement>(
'.grid-scroll-container',
)?.clientWidth ||
this.clientWidth ||
0
);
}
private spacingFor(width: number): number {
return gridSpacingFor(width, this.cardSize);
}
/** Sort direction for the artist grid.
*
* There is only one key to sort by: `library.Artist` carries a
@@ -478,6 +498,8 @@ export class ArtistsView
override disconnectedCallback() {
super.disconnectedCallback();
this.detachWheelListener();
this.gridResizeObserver?.disconnect();
this.gridResizeObserver = null;
}
/** The wheel listener and the scroll debounce belong to the grid
@@ -730,10 +752,34 @@ export class ArtistsView
* ================================================================ */
private lastLayoutWidth = 0;
private lastLayoutSpacing = 0;
/** Watches the scroller so a window resize rebuilds the layout:
* the spacing is derived from its width, and nothing else asks
* this view to update when only that changes. */
private gridResizeObserver: ResizeObserver | null = null;
private observeGridWidth() {
const container =
this.renderRoot?.querySelector<HTMLElement>(
'.grid-scroll-container',
);
if (!container || this.gridResizeObserver) return;
this.gridResizeObserver = new ResizeObserver(() =>
this.requestUpdate(),
);
this.gridResizeObserver.observe(container);
}
private updateGridLayout() {
this.observeGridWidth();
if (
this.cardSize === this.lastLayoutWidth
this.cardSize === this.lastLayoutWidth &&
this.lastLayoutSpacing ===
this.spacingFor(this.containerWidth)
) {
return;
}
@@ -68,6 +68,29 @@ export class SeekBar extends LitElement {
align-items: center;
}
/* The clocks must not resize as they count.
Two things move them, and they need different answers. Digits in
a proportional font are different widths, so 1:11 is narrower
than 4:08 and the bar breathed once a second -- that is what
tabular figures fix. The character *count* changes too, at the
hundredth minute and whenever the right-hand clock is toggled to
remaining and grows a minus sign, and a figure width cannot fix
that -- so each clock also reserves the widest string this track
can put in it. The budget is per track rather than a constant
because reserving six characters on every track would push the
slider in by a character at each end for nothing. */
#seek-bar-container small,
.time-toggle {
font-variant-numeric: tabular-nums;
flex: 0 0 auto;
min-width: calc(var(--yj-clock-chars, 5) * 1ch);
}
#seek-bar-container small {
text-align: left;
}
.time-toggle {
background: none;
border: none;
@@ -76,6 +99,9 @@ export class SeekBar extends LitElement {
font: inherit;
font-size: var(--wa-font-size-s, 0.875rem);
cursor: pointer;
/* One more for the minus sign the remaining form carries. */
min-width: calc((var(--yj-clock-chars, 5) + 1) * 1ch);
text-align: right;
}
.time-toggle:hover,
@@ -219,8 +245,19 @@ export class SeekBar extends LitElement {
: formatSeconds(this.trackLength);
const rightTime = this.hasTrack ? rightLabel : '--:--';
// The widest string either clock can hold for *this* track. The
// duration is the longest elapsed value there can be, so its length
// is the budget; `--:--` is five, which is also the floor.
const clockChars = Math.max(
5,
this.hasTrack ? formatSeconds(this.trackLength).length : 0,
);
return html`
<div id="seek-bar-container">
<div
id="seek-bar-container"
style="--yj-clock-chars: ${clockChars}"
>
<small data-testid="elapsed-time">${elapsedTime}</small>
<wa-slider
label="Seek"
@@ -86,9 +86,10 @@ export class DownloadClients extends LitElement {
/** Working copy of the auto-download guardrails. */
@state()
private prefs: download.AutoDownloadPrefs = {
minSizeMb: 0,
minKbps: 0,
maxKbps: 0,
preferredKbps: 0,
maxSizeMb: 0,
preferredSizeMb: 0,
allowedFormats: [],
} as download.AutoDownloadPrefs;
@@ -284,25 +285,72 @@ export class DownloadClients extends LitElement {
: nothing}
<div class="form">
<!-- Bitrate, not megabytes. A size means nothing
on its own: 300 MB is a generous single and a
suspiciously small boxset, and whoever fills
this in has no idea which release it will be
applied to. A rate is the same statement
divided by how long the music is, so one number
holds across an EP and an opera. -->
<div class="field-row">
<wa-input
label="Minimum size (MB)"
label="Minimum bitrate (kbps)"
type="number"
min="0"
placeholder="No minimum"
.value=${this.prefs.minSizeMb ? String(this.prefs.minSizeMb) : ''}
.value=${this.prefs.minKbps ? String(this.prefs.minKbps) : ''}
@input=${(e: Event) => {
this.prefs = {
...this.prefs,
minSizeMb: Number((e.target as HTMLInputElement).value) || 0,
minKbps: Number((e.target as HTMLInputElement).value) || 0,
};
}}
></wa-input>
<wa-input
label="Maximum size (MB)"
label="Maximum bitrate (kbps)"
type="number"
min="0"
placeholder="No maximum"
.value=${this.prefs.maxKbps ? String(this.prefs.maxKbps) : ''}
@input=${(e: Event) => {
this.prefs = {
...this.prefs,
maxKbps: Number((e.target as HTMLInputElement).value) || 0,
};
}}
></wa-input>
<wa-input
label="Preferred bitrate (kbps)"
type="number"
min="0"
placeholder="No preference"
.value=${this.prefs.preferredKbps
? String(this.prefs.preferredKbps)
: ''}
@input=${(e: Event) => {
this.prefs = {
...this.prefs,
preferredKbps:
Number((e.target as HTMLInputElement).value) || 0,
};
}}
></wa-input>
</div>
<div class="requires">
320 is the top of MP3; a FLAC rip is usually
5001000 depending on the music. Preferred
decides between copies that are otherwise equally
good it never rules one out, which is what the
minimum and maximum are for.
</div>
<div class="field-row">
<wa-input
label="Never grab more than (MB)"
type="number"
min="0"
placeholder="No limit"
.value=${this.prefs.maxSizeMb ? String(this.prefs.maxSizeMb) : ''}
@input=${(e: Event) => {
this.prefs = {
@@ -311,22 +359,14 @@ export class DownloadClients extends LitElement {
};
}}
></wa-input>
<wa-input
label="Preferred size (MB)"
type="number"
min="0"
placeholder="No preference"
.value=${this.prefs.preferredSizeMb
? String(this.prefs.preferredSizeMb)
: ''}
@input=${(e: Event) => {
this.prefs = {
...this.prefs,
preferredSizeMb:
Number((e.target as HTMLInputElement).value) || 0,
};
}}
></wa-input>
</div>
<div class="requires">
A ceiling on the download itself, in case a
mislabelled boxset gets through. Still a size
because it is a question about disk space, and
because it has to apply to a candidate whose
bitrate cannot be worked out at all.
</div>
<div>
@@ -111,13 +111,32 @@ const gridStyles = css`
scale: 0.95;
}
/* Title and year on one line, and only the title truncates.
The year used to be part of the same run of text, so it was the
first thing an ellipsis ate: a card wide enough for a long album
name never showed its year, and browsing by year showed years
only for the albums with short names -- the sort said one thing
and the cards showed another.
A flex row rather than a second line, because the card's height
is what the virtualizer measures rows by. */
.album-name {
font-size: var(--album-name-font, 14px);
font-weight: 400;
color: var(--yj-text-primary, #fff);
display: flex;
justify-content: center;
align-items: baseline;
gap: 0.35em;
min-width: 0;
}
.album-title {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
min-width: 0;
}
.artist-name {
@@ -131,6 +150,8 @@ const gridStyles = css`
.album-year {
color: var(--yj-text-tertiary, #888);
flex: 0 0 auto;
white-space: nowrap;
}
/* ========================================
+116 -23
View File
@@ -19,6 +19,7 @@ import { LibraryController } from '@store/controllers/library-controller';
import { SearchController } from '@store/controllers/search-controller';
import { ViewLifecycleMixin } from '@utils/view-lifecycle';
import { RovingGridController } from '@utils/roving-grid';
import { gridColumnsFor, gridSpacingFor } from '@utils/grid-spacing';
import { queueStore } from '@store/queue-store';
import type { QueueSource } from '@store/queue-store';
import '@awesome.me/webawesome/dist/components/popup/popup.js';
@@ -52,7 +53,8 @@ import {
} from '@utils/context-menu-controller.js';
import type { ContextMenuHost } from '@utils/context-menu-controller.js';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { artistLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import {
createAlbumArtDragImage,
createDragImage,
@@ -96,19 +98,36 @@ export class CoverGrid
private lastAlbumsRef: library.Album[] | null =
null;
// Fixed grid spacing constants.
private static readonly GRID_GAP = 8;
private static readonly GRID_PADDING = 8;
private static readonly CARD_PADDING = 5;
private ctxMenu = new ContextMenuController(this);
private favCtrl = new FavoritesController(this);
private selMgr = new AlbumSelectionManager();
private scrollMgr = new ScrollManager(this, {
GRID_GAP: CoverGrid.GRID_GAP,
GRID_PADDING: CoverGrid.GRID_PADDING,
columnsFor: (width: number) => this.columnsFor(width),
spacingFor: (width: number) => this.spacingFor(width),
});
/**
* How many cards fit across `width`, by the same arithmetic the
* virtualizer's `space-evenly` grid uses no gap and no padding
* are reserved, because both come out of what is left over.
*
* The scroll manager restores a position by rebuilding the grid's
* geometry, so this and `spacingFor` must agree with the layout
* rather than approximate it; they were two constants that no
* longer describe anything once the spacing became elastic.
*/
columnsFor(width: number): number {
return gridColumnsFor(width, this.cardWidth);
}
/** The spacing that width produces: between columns, between rows,
* and around the outside, all the same number. */
spacingFor(width: number): number {
return gridSpacingFor(width, this.cardWidth);
}
private lastSelectedAlbumIndex: number | null = null;
private lastSelectedTrackIndex: number | null = null;
@@ -147,10 +166,30 @@ export class CoverGrid
}
// Virtualizer grid layout instance — recreated when
// the card size changes.
// the card size or the container width changes.
private gridLayout = this.createGridLayout();
private gridLayoutWidth = 0;
/** The spacing the current layouts were built with. */
private gridLayoutSpacing = 0;
/** Watches the scroll container so a window resize rebuilds the
* layout: the spacing is derived from its width, and nothing else
* asks this component to update when only that changes. */
private gridResizeObserver: ResizeObserver | null =
null;
private observeGridWidth(): void {
const container = this.scrollContainer;
if (!container || this.gridResizeObserver) return;
this.gridResizeObserver = new ResizeObserver(
() => this.requestUpdate(),
);
this.gridResizeObserver.observe(container);
}
/**
* Secondary layout for the "after" virtualizer in
* split mode. Uses zero top padding so there is no
@@ -168,22 +207,49 @@ export class CoverGrid
}
const h = w + this.cardTextHeight;
const gap = CoverGrid.GRID_GAP;
const pad = CoverGrid.GRID_PADDING;
// The spacing is whatever the row could not spend on another
// card, shared out equally — so it is the same number between
// two cards, between two rows, and down each outside edge.
// See `utils/grid-spacing.ts` for why it is computed rather
// than handed to the virtualizer as `space-evenly`.
const spacing = this.spacingFor(
this.containerWidth,
);
if (!noTopPad) {
this.gridLayoutSpacing = spacing;
}
return grid({
itemSize: {
width: `${w}px`,
height: `${h}px`,
},
gap: `${gap}px`,
gap: `${spacing}px`,
padding: noTopPad
? `0 ${pad}px ${pad}px`
: `${pad}px`,
justify: 'center',
? `0 ${spacing}px ${spacing}px`
: `${spacing}px`,
justify: 'start',
});
}
/**
* The width the grid lays itself out in.
*
* Read from the scroll container when there is one; before the
* first render there is not, and the fallback only has to be
* plausible the layout is rebuilt from the real width as soon as
* one exists.
*/
private get containerWidth(): number {
return (
this.scrollContainer?.clientWidth ||
this.clientWidth ||
0
);
}
private dragImageEl: HTMLElement | null = null;
// -- Memoisation caches for filtered albums --
@@ -425,8 +491,18 @@ export class CoverGrid
* Lifecycle
* ==================================================================== */
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback() {
super.connectedCallback();
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
// Two virtualizers when the grid is split; both draw rows.
this.renderRoot?.querySelectorAll('lit-virtualizer')
.forEach((v) => (v as unknown as { requestUpdate(): void }).requestUpdate());
});
this.restoreSortPreferences();
this.loadAlbums();
@@ -441,6 +517,8 @@ export class CoverGrid
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
this.removeEventListener(
'error',
@@ -453,6 +531,9 @@ export class CoverGrid
);
this.wheelListenerAttached = false;
this.gridResizeObserver?.disconnect();
this.gridResizeObserver = null;
this.scrollMgr.teardown();
this.scrollMgr.revealContainer(
this.scrollContainer,
@@ -590,10 +671,18 @@ export class CoverGrid
this.wheelListenerAttached = true;
}
// Recreate the virtualizer grid layout when
// the card size changes.
this.observeGridWidth();
// Recreate the virtualizer grid layout when the card size
// changes — or when the spacing the container width produces
// does, since that is now a derived number rather than a
// constant. Keyed on the spacing rather than on the width, or
// every pixel of a drag rebuilds a layout that would come out
// the same.
const cardSizeChanged =
this.gridLayoutWidth !== this.cardWidth;
this.gridLayoutWidth !== this.cardWidth ||
this.gridLayoutSpacing !==
this.spacingFor(this.containerWidth);
if (cardSizeChanged) {
this.gridLayout = this.createGridLayout();
@@ -1391,12 +1480,16 @@ export class CoverGrid
source: 'cover-grid',
});
// Single album: show cover art thumbnail.
// Multiple albums: show track-count badge.
// Single album: show its cover, badged with how many tracks are
// on the way -- an album is 1 track or 30 and the thumbnail is
// the same picture either way, so the number the drop is about
// was the one thing this drag did not say.
// Multiple albums: show the track-count badge alone.
if (isSingleAlbum && hit.album.CoverArtPath) {
this.dragImageEl =
createAlbumArtDragImage(
this.getCoverUrl(hit.album),
filePaths.length,
);
} else {
this.dragImageEl = createDragImage(
@@ -1809,10 +1902,10 @@ export class CoverGrid
class="album-name"
title="${album.Name}"
>
${album.Name}${album.Year
? html`
<span class="album-year">
(${album.Year})</span
<span class="album-title">${album.Name}</span
>${album.Year
? html`<span class="album-year"
>(${album.Year})</span
>`
: nothing}
</div>
@@ -1820,7 +1913,7 @@ export class CoverGrid
class="artist-name"
title="${album.ArtistName}"
>
${artistLink(album.ArtistName, album.ArtistMBID ?? '')}
${creditLink(creditStore.credits(album.MBID), album.ArtistName, album.ArtistMBID ?? '')}
</div>
</div>
</div>
@@ -6,12 +6,21 @@ import type { LibraryController } from '@store/controllers/library-controller';
import type { GridEntry } from './cover-grid-types.js';
/**
* Grid spacing constants shared between the scroll
* manager and the host component.
* Grid geometry, asked of the host rather than written down.
*
* These were two constants, `GRID_GAP` and `GRID_PADDING`, which stopped
* describing anything the moment the grid's spacing became elastic: the
* gap, the padding and the column count are all derived from the
* container width now, and a scroll position rebuilt from a stale 8px
* lands in the wrong row.
*/
export interface GridConstants {
readonly GRID_GAP: number;
readonly GRID_PADDING: number;
/** Columns that fit across `width`. */
columnsFor(width: number): number;
/** The spacing `width` produces between columns, between rows,
* and around the outside, all the same number. */
spacingFor(width: number): number;
}
/**
@@ -275,8 +284,8 @@ export class ScrollManager {
return;
}
const gap = this.gc.GRID_GAP;
const pad = this.gc.GRID_PADDING;
const gap = this.spacing(container);
const pad = gap;
const rowStep =
this.host.cardHeight + gap;
@@ -293,7 +302,7 @@ export class ScrollManager {
() => {
const rowStep =
this.host.cardHeight +
this.gc.GRID_GAP;
this.spacing(container);
if (this.pendingFocus === null) {
this.isResizing = true;
@@ -351,7 +360,7 @@ export class ScrollManager {
container: HTMLElement,
rowStep: number,
): void {
const pad = this.gc.GRID_PADDING;
const pad = this.spacing(container);
const cols = this.currentColumnCount;
const filtered =
this.host.cachedFilteredAlbums;
@@ -410,17 +419,15 @@ export class ScrollManager {
): number {
if (!container) return 1;
const gap = this.gc.GRID_GAP;
const pad = this.gc.GRID_PADDING;
const availableWidth =
container.clientWidth - pad * 2;
return this.gc.columnsFor(
container.clientWidth,
);
}
return Math.max(
1,
Math.floor(
(availableWidth + gap) /
(this.host.cardWidth + gap),
),
/** The grid's current spacing, which is also its padding. */
private spacing(container?: HTMLElement): number {
return this.gc.spacingFor(
container?.clientWidth ?? 800,
);
}
@@ -439,7 +446,7 @@ export class ScrollManager {
container?: HTMLElement,
): number {
const cols = this.getColumnCount(container);
const gap = this.gc.GRID_GAP;
const gap = this.spacing(container);
return (
cols * this.host.cardWidth +
@@ -460,7 +467,7 @@ export class ScrollManager {
const cols = this.getColumnCount(container);
const colIndex = idx % cols;
const gap = this.gc.GRID_GAP;
const gap = this.spacing(container);
return (
colIndex *
@@ -597,8 +604,8 @@ export class ScrollManager {
if (!this.host.splitMode) return raw;
const gap = this.gc.GRID_GAP;
const pad = this.gc.GRID_PADDING;
const gap = this.spacing(container);
const pad = gap;
const columns =
this.getColumnCount(container);
const rowStep = this.host.cardHeight + gap;
@@ -678,8 +685,8 @@ export class ScrollManager {
if (expandedIndex < 0) return;
const gap = this.gc.GRID_GAP;
const pad = this.gc.GRID_PADDING;
const gap = this.spacing(container);
const pad = gap;
const columns =
this.getColumnCount(container);
const rowStep = this.host.cardHeight + gap;
@@ -772,8 +779,8 @@ export class ScrollManager {
if (idx < 0) return;
const gap = this.gc.GRID_GAP;
const pad = this.gc.GRID_PADDING;
const gap = this.spacing(container);
const pad = gap;
const cols =
this.getColumnCount(container);
const rowStep = this.host.cardHeight + gap;
@@ -854,9 +861,8 @@ export class ScrollManager {
this.getExpandedAlbumIndex();
if (idx >= 0) {
const gap = this.gc.GRID_GAP;
const pad =
this.gc.GRID_PADDING;
const gap = this.spacing(container);
const pad = gap;
const cols =
this.getColumnCount(
container,
@@ -400,7 +400,7 @@ export class DownloadsView extends ViewLifecycleMixin(LitElement) {
private renderEmptyRequests() {
return html`
<div class="empty">
Nothing requested yet. Use Want this on an album or artist
Nothing requested yet. Use Request this on an album or artist
to add it here.
</div>
`;
@@ -512,7 +512,9 @@ export class DownloadsView extends ViewLifecycleMixin(LitElement) {
${request.artist ? `${request.artist}` : ''}${request.title ||
request.mbid}
</div>
<div class="detail">${requestDetail(request, this.nowMs)}</div>
<div class="detail">
${requestDetail(request, this.nowMs, this.canDownload)}
</div>
</div>
<div class="actions">
${request.state === 'satisfied'
@@ -706,10 +708,20 @@ export class DownloadsView extends ViewLifecycleMixin(LitElement) {
* looked for rather than as an error, because that is what it is the
* retry is already scheduled and there is nothing for the user to do.
*/
function requestDetail(request: Request, nowMs: number): string {
function requestDetail(
request: Request,
nowMs: number,
canDownload: boolean,
): string {
if (request.state === 'satisfied') return 'In your library';
if (request.state === 'paused') return 'Paused — not being looked for';
// With no client there is no search and no retry clock — the
// backend stopped scheduling one — so a row must not imply either.
// "Queued" and "next check in 6 hours" are both promises nothing is
// in a position to keep.
if (!canDownload) return 'On your list — no download client to search with';
if (request.attempts === 0) return 'Queued — not searched for yet';
const tries = `Searched ${request.attempts} time${request.attempts === 1 ? '' : 's'}`;
@@ -2,6 +2,7 @@ import { LitElement, html, css, nothing } from 'lit';
import { customElement, property, state, query } from 'lit/decorators.js';
import { classMap } from 'lit/directives/class-map.js';
import { designTokens } from '../../styles/tokens.css';
import { srOnly } from '../../styles/sr-only.css';
import {
LookupReleaseGroup,
BrowseReleases,
@@ -20,7 +21,8 @@ type MBRelease = explore.MBRelease;
type MBTrack = explore.MBTrack;
import { exploreCache } from '../../store/explore-cache';
import { libraryStore } from '../../store/library-store';
import { artistLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import { describeError } from '../../utils/describe-error';
import { EventsOn } from '@runtime/runtime';
import { Events } from '../../events';
@@ -288,6 +290,7 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
designTokens,
exploreLinkStyles,
contextMenuStyles,
srOnly,
css`
:host {
display: flex;
@@ -661,34 +664,21 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
font-weight: 400;
}
/* The request control is only offered where there is
* something to request, and only when the row is being
* attended to a column of plus signs down a mostly-owned
* album is the clutter the green ticks were.
/* The request control is offered on every row that has
* something to request, and is not revealed on hover.
*
* Hidden with opacity, never display:none or visibility,
* so it keeps its place in the layout (rows do not reflow
* as the pointer moves) and stays in the tab order and the
* accessibility tree. focus-within is what makes it
* reachable without a mouse: tabbing to the button reveals
* it, and the row's own focus reveals it before you get
* there. */
* It used to be transparent until the row was hovered or
* focused, on the reasoning that a column of plus signs
* down a mostly-owned album is clutter. That reasoning was
* inherited from the green ticks it replaced and does not
* survive the rule those were removed for: a tick marked
* the *common* case, while this marks the rows that are
* **not** here. A mark on the exception is the information
* on this page and one that appears only under the
* pointer cannot be seen, counted, or reached by anyone
* driving this with a finger. */
.track-row .track-request {
flex-shrink: 0;
opacity: 0;
transition: opacity 0.12s ease;
}
.track-row:hover .track-request,
.track-row:focus-within .track-request,
.track-row .track-request:focus-visible {
opacity: 1;
}
@media (prefers-reduced-motion: reduce) {
.track-row .track-request {
transition: none;
}
}
`,
];
@@ -704,22 +694,43 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
* BrowseReleases fetch never signals readiness. */
private releasesFallbackTimer?: number;
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback() {
super.connectedCallback();
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
});
if (this.releaseGroupMBID || this.localAlbumId) {
void this.loadAllData();
}
// The download button only appears once a client is connected,
// so this tracks the provider list rather than assuming.
//
// The `requestUpdate` is what makes the *tracklist's* badges
// move. Both assignments below are reactive fields, so Lit
// repaints when either changes — but a track request changes
// neither: `canDownload` is about providers and `isRequested`
// is about this album's own release group. Each row's badge
// reads `libraryStatusFor(false, track.mbid)` at render time,
// which is a dependency on the store that Lit cannot see, so
// clicking one filed the request and left the plus exactly
// where it was. The other three hosts rendering these badges
// (`explore-artist-details`, `explore-view`, `top-results-row`)
// have always asked for the repaint here; this one did not.
this.downloadUnsub = downloadStore.subscribe(() => {
this.canDownload = downloadStore.available;
this.syncRequested();
this.requestUpdate();
});
void downloadStore.init().then(() => {
this.canDownload = downloadStore.available;
this.syncRequested();
this.requestUpdate();
});
void this.resolveTargetLibraryId();
@@ -760,6 +771,8 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
this.downloadUnsub?.();
this.downloadUnsub = null;
this.unsubReleasesReady?.();
@@ -2670,7 +2683,7 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
slot="start"
name=${this.isRequested ? 'solid/bookmark' : 'regular/bookmark'}
></wa-icon>
${this.isRequested ? 'Wanted' : 'Want this'}
${this.isRequested ? 'Requested' : 'Request this'}
</wa-button>
`;
}
@@ -2850,7 +2863,11 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
return html`
${artist
? html`<div class="album-artist">
${artistLink(artist, artistMbid)}
${creditLink(
creditStore.credits(this.releaseGroupMBID),
artist,
artistMbid,
)}
</div>`
: nothing}
${metaParts.length > 0
@@ -3019,11 +3036,21 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
/* ── Tracklist ── */
/**
* The heading is there and is not drawn.
*
* A list of numbered titles with durations under an album's cover
* does not need a word above it saying what it is it was the
* only thing on this page labelling something already obvious. But
* the section is a landmark and the page's heading structure runs
* through it, so what goes is the *ink*, not the element: a reader
* jumping by heading still finds the tracklist.
*/
private renderTracklist() {
if (this.loadingReleases) {
return html`
<section>
<h3 class="section-header">Tracklist</h3>
<h3 class="sr-only">Tracklist</h3>
<div class="section-loading">Loading tracks\u2026</div>
</section>
`;
@@ -3036,7 +3063,7 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
if (!current) {
return html`
<section>
<h3 class="section-header">Tracklist</h3>
<h3 class="sr-only">Tracklist</h3>
<div class="section-error">
<wa-icon name="triangle-exclamation"></wa-icon>
No release data available.
@@ -3049,7 +3076,7 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
if (tracks.length === 0) {
return html`
<section>
<h3 class="section-header">Tracklist</h3>
<h3 class="sr-only">Tracklist</h3>
<div
style="color: var(--yj-text-tertiary, #888); font-size: var(--yj-text-md)"
>
@@ -3065,7 +3092,7 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
return html`
<section>
<h3 class="section-header">Tracklist</h3>
<h3 class="sr-only">Tracklist</h3>
<div class="tracklist">
${discNumbers.map((discNum) => {
const discTracks = discMap.get(discNum) ?? [];
@@ -2753,7 +2753,7 @@ export class ExploreArtistDetails extends LitElement implements ContextMenuHost
slot="icon"
name=${requested ? 'xmark' : 'bookmark'}
></wa-icon>
${requested ? 'Cancel Request' : 'Want This'}
${requested ? 'Cancel Request' : 'Request This'}
</wa-dropdown-item>
`
: nothing}
@@ -16,7 +16,8 @@ import { exploreCache, ARTIST_IMAGE_CACHE_LIMIT } from '../../store/explore-cach
import { queueStore } from '../../store/queue-store';
import { notificationStore } from '../../store/notification-store';
import '../notifications/inline-notice';
import { artistLink, trackLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, trackLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import { describeError } from '../../utils/describe-error';
import '@awesome.me/webawesome/dist/components/icon/icon.js';
import '../library-status-indicator/library-status-indicator.js';
@@ -774,6 +775,14 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
}
protected override onViewActivate(): void {
// A cached primary view, so this is torn down on the way out
// rather than on disconnect — which never fires here.
this.whileActive(
creditStore.subscribe(() => {
this.requestUpdate();
}),
);
// Fetched on arrival rather than on connect: this is a cached
// primary view, created and warmed at startup, so a fetch there
// is three catalog queries every user pays for whether or not
@@ -1500,9 +1509,24 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
if (url) {
this.thumbnailCache.set(req.mbid, url);
this.requestUpdate();
return;
}
// An empty answer is not necessarily "there
// is no art" — a slow Internet Archive node
// is answered by a timeout, which looks
// exactly the same from here. Drop the
// in-flight marker so the next time this
// release group is on screen it is asked
// again; the backend records a genuine 404
// on disk and answers that one instantly,
// so a real miss costs nothing to re-ask.
this.thumbnailCache.delete(req.mbid);
})
.catch(() => {});
.catch(() => {
this.thumbnailCache.delete(req.mbid);
});
}
})
.catch(() => {
@@ -2193,7 +2217,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
<div class="album-title" title="${rg.title}">
${rg.title}
</div>
<div class="album-artist">${artistLink(rg.artistCredit, rg.artistMbid ?? '')}</div>
<div class="album-artist">${creditLink(creditStore.credits(rg.mbid), rg.artistCredit, rg.artistMbid ?? '')}</div>
<div class="album-meta">
<div class="album-meta-text">
${rg.primaryType
@@ -2255,7 +2279,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
${trackLink(r.title, r.releaseName ?? '', r.releaseGroupMbid ?? '', r.mbid)}
</div>
<div class="track-artist">
${artistLink(r.artistCredit, r.artistMbid ?? '')}
${creditLink(creditStore.credits(r.mbid), r.artistCredit, r.artistMbid ?? '')}
</div>
</div>
<div class="track-meta">
@@ -10,6 +10,7 @@ import type {
VisibilityChangedEvent,
} from '@lit-labs/virtualizer';
import { grid } from '@lit-labs/virtualizer/layouts/grid.js';
import { gridSpacingFor } from '@utils/grid-spacing';
import {
GetFilePathsByGenres,
} from '@go/library/library.js';
@@ -155,8 +156,6 @@ export class GenresView
// ----- Grid spacing constants -----
private static readonly GRID_GAP = 8;
private static readonly GRID_PADDING = 8;
private static readonly CARD_PADDING = 5;
private get imageSize(): number {
@@ -185,20 +184,41 @@ export class GenresView
private createGridLayout() {
const w = this.cardSize ?? CARD_SIZE_DEFAULT;
const h = w + this.cardTextHeight;
const gap = GenresView.GRID_GAP;
const pad = GenresView.GRID_PADDING;
// One number for the gap, the row gap and the padding: whatever
// a row could not spend on another card, shared out equally, so
// the outside is never wider than the inside. See
// `utils/grid-spacing.ts`.
const spacing = this.spacingFor(this.containerWidth);
this.lastLayoutSpacing = spacing;
return grid({
itemSize: {
width: `${w}px`,
height: `${h}px`,
},
gap: `${gap}px`,
padding: `${pad}px`,
justify: 'center',
gap: `${spacing}px`,
padding: `${spacing}px`,
justify: 'start',
});
}
/** The width the grid lays itself out in. */
private get containerWidth(): number {
return (
this.renderRoot?.querySelector<HTMLElement>(
'.grid-scroll-container',
)?.clientWidth ||
this.clientWidth ||
0
);
}
private spacingFor(width: number): number {
return gridSpacingFor(width, this.cardSize);
}
/** Sort key and direction for the genre grid (H-19: it had none). */
@state()
private sortField: 'name' | 'tracks' = 'name';
@@ -483,6 +503,8 @@ export class GenresView
override disconnectedCallback() {
super.disconnectedCallback();
this.detachWheelListener();
this.gridResizeObserver?.disconnect();
this.gridResizeObserver = null;
}
/** See artists-view: off-screen the grid cannot be scrolled, and
@@ -737,10 +759,34 @@ export class GenresView
* ================================================================ */
private lastLayoutWidth = 0;
private lastLayoutSpacing = 0;
/** Watches the scroller so a window resize rebuilds the layout:
* the spacing is derived from its width, and nothing else asks
* this view to update when only that changes. */
private gridResizeObserver: ResizeObserver | null = null;
private observeGridWidth() {
const container =
this.renderRoot?.querySelector<HTMLElement>(
'.grid-scroll-container',
);
if (!container || this.gridResizeObserver) return;
this.gridResizeObserver = new ResizeObserver(() =>
this.requestUpdate(),
);
this.gridResizeObserver.observe(container);
}
private updateGridLayout() {
this.observeGridWidth();
if (
this.cardSize === this.lastLayoutWidth
this.cardSize === this.lastLayoutWidth &&
this.lastLayoutSpacing ===
this.spacingFor(this.containerWidth)
) {
return;
}
@@ -48,7 +48,7 @@ export type LibraryStatus =
*
* Colours and glyphs:
* - in-library green circle, check mark
* - queued amber circle, hourglass
* - queued amber circle, bookmark ("on your list")
* - not-in-library grey circle, plus sign
*
* Usage:
@@ -241,12 +241,23 @@ export class LibraryStatusIndicator extends LitElement {
}
`;
/**
* The glyph for each state.
*
* `queued` is a **bookmark**, not the hourglass it used to be. An
* hourglass says "wait, this is under way", which overstates what a
* request is: nothing may be downloading, nothing may ever be found,
* and the user can leave one sitting on the list indefinitely. A
* bookmark says the honest thing it is on your list and reads as
* the opposite of the plus that put it there, which is what a
* toggle's two states have to do.
*/
private iconName(): string {
switch (this.status) {
case 'in-library':
return 'check';
case 'queued':
return 'hourglass-half';
return 'bookmark';
default:
return 'plus';
}
@@ -276,7 +287,7 @@ export class LibraryStatusIndicator extends LitElement {
if (this.actionable) {
return this.status === 'queued'
? `Cancel the request for ${kind}${name}`
: `Want ${kind}${name}`;
: `Request ${kind}${name}`;
}
switch (this.status) {
@@ -354,25 +365,6 @@ export class LibraryStatusIndicator extends LitElement {
}
const title = this.tooltip();
const icon = this.iconName()
? html`<wa-icon name=${this.iconName()} aria-hidden="true"></wa-icon>`
: nothing;
if (this.actionable) {
return html`
<button
class="badge"
type="button"
title=${title}
aria-label=${title}
?disabled=${this.busy}
@click=${this.onActivate}
@keydown=${this.onKeydown}
>
${icon}
</button>
`;
}
// The ring stands in for the icon wherever the icon would go —
// including inside the button, because a partly-held album is
@@ -5,11 +5,12 @@ import '../audio-player/controls/player-controls';
import '../audio-player/seekbar/seek-bar';
import '../audio-player/volume-control/volume-control';
import {
artistLink,
creditLink,
albumLink,
exploreLinkStyles,
} from '@utils/explore-link';
import { PlayerController } from '@store/controllers/player-controller';
import { creditStore } from '@store/credit-store';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { designTokens } from '../../styles/tokens.css';
import { srOnly } from '../../styles/sr-only.css';
@@ -36,6 +37,22 @@ import { srOnly } from '../../styles/sr-only.css';
@customElement('now-playing-view')
export class NowPlayingView extends LitElement {
private player = new PlayerController(this);
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback(): void {
super.connectedCallback();
// Credits arrive after the track does, so the name this view is
// already showing has to be re-rendered when they land.
this.creditsUnsub = creditStore.subscribe(() => this.requestUpdate());
}
override disconnectedCallback(): void {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
}
private favCtrl = new FavoritesController(this);
static override styles = [designTokens, srOnly, exploreLinkStyles, css`
@@ -262,7 +279,11 @@ export class NowPlayingView extends LitElement {
${track.title || track.fileName}
</h2>
<p class="artist">
${artistLink(track.artist, track.artistMbid)}
${creditLink(
creditStore.credits(track.recordingMbid),
track.artist,
track.artistMbid,
)}
</p>
${track.album
? html`<p class="album">
@@ -286,7 +307,7 @@ export class NowPlayingView extends LitElement {
: `Add ${track.title} to ${this.favCtrl.playlistName}`}
@click=${this.toggleFavorite}
>
<wa-icon name=${this.favCtrl.iconName}></wa-icon>
<wa-icon name=${this.favCtrl.iconFor(favorited)}></wa-icon>
</button>
</div>
@@ -4,7 +4,7 @@ import '@awesome.me/webawesome/dist/components/icon/icon.js';
import '@awesome.me/webawesome/dist/components/popup/popup.js';
import type WaPopup from '@awesome.me/webawesome/dist/components/popup/popup.js';
import {
artistLink,
creditLink,
trackLink,
exploreLinkStyles,
} from '@utils/explore-link';
@@ -14,6 +14,7 @@ import {
navigateToQueueSource,
} from '@utils/queue-source-link';
import { PlayerController } from '@store/controllers/player-controller';
import { creditStore } from '@store/credit-store';
import { QueueController } from '@store/controllers/queue-controller';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { designTokens } from '../../styles/tokens.css';
@@ -325,6 +326,9 @@ export class NowPlaying extends LitElement {
}
`];
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback() {
super.connectedCallback();
this.loadScrollMode();
@@ -341,10 +345,21 @@ export class NowPlaying extends LitElement {
this.geometryDirty = true;
this.requestUpdate();
});
// A credit arriving changes the rendered text, and the marquee
// measures that text — so this is a geometry change, not just a
// repaint. Saying so is what stops the bar scrolling to the
// old width.
this.creditsUnsub = creditStore.subscribe(() => {
this.geometryDirty = true;
this.requestUpdate();
});
}
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
// A drag interrupted by the bar going away still has to clean up.
this.attachDragListeners(false);
window.removeEventListener(SCROLL_CHANGE_EVENT, this.handleScrollModeEvent);
@@ -483,7 +498,7 @@ export class NowPlaying extends LitElement {
@mouseleave=${this.handleArtistMouseLeave}
@transitionend=${() => this.onScrollCycleEnd('artist')}
>
<span class="scroll-content">${artistLink(track.artist, track.artistMbid) || 'Unknown Artist'}</span>
<span class="scroll-content">${creditLink(creditStore.credits(track.recordingMbid), track.artist, track.artistMbid) || 'Unknown Artist'}</span>
</span>
${describeQueueSource(this.queue.source)
? html`
@@ -512,7 +527,7 @@ export class NowPlaying extends LitElement {
)}
>
<wa-icon
name=${this.favCtrl.iconName}
name=${this.favCtrl.iconFor(isFav)}
variant=${favVariant}
></wa-icon>
</button>

Some files were not shown because too many files have changed in this diff Show More