Compare commits

...
30 Commits
Author SHA1 Message Date
yonlu 6fb7b5ea11 Merge pull request 'ci: trigger the catalog job deliberately, pin agent docs to one file' (#1) from chore/workflow-guardrails into main
Build & publish Arch package / arch-package (push) Successful in 2m40s
CI / check (push) Skipped
CI / e2e (push) Skipped
Build & publish the Android APK / apk (push) Failing after 56s
Sync Homebrew formula / sync-formula (push) Successful in 5s
Reviewed-on: #1
2026-08-17 20:19:38 +00:00
yonluandClaude Opus 5 369810e06b ci: stop testing every commit twice on a runner there is one of
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m53s
CI / e2e (pull_request) Successful in 6m26s
A branch push and its pull request are the same commit. With
`branches: ['**']` alongside `pull_request:`, opening a PR booked four
runs -- check and e2e for the branch, then both again for
refs/pull/N/head -- and this host has capacity 1, shared with an index
build that can hold it for three hours. PR #1's own checks queued two
runs deep behind exactly that.

`pull_request` covers feature branches. `main` stays because a
post-merge run is the record of the trunk's health, and now that main
refuses direct pushes it happens exactly once per merge.

The trade is that a branch pushed with no PR open gets no CI. That
matches the workflow this repo just committed to, and the signal returns
on the same commit the moment a PR exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 15:44:00 -04:00
yonluandClaude Opus 5 e51cb13662 ci: trigger the catalog job deliberately, pin agent docs to one file
CI / check (pull_request) Canceled after 0s
CI / e2e (pull_request) Canceled after 0s
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Two guardrails for the 2026-08-17 incident, and one is not about CI.

index-artifact.yml's `push` trigger was commented out that day with a
note to restore it once the rebuild completed. Restoring it is the bug.
A refresh is individually cheap, which is what made the trigger look
free; what it actually did was put an unattended job that mutates the
only copy of a ~205 GB catalog on the same trigger as an ordinary code
change, on a runner with capacity 1. The rule the file now states is the
general one -- a job that mutates state which cannot be rebuilt in ten
minutes is triggered deliberately -- so the next such job has somewhere
to look. The cron and workflow_dispatch lose nothing: indexbuild resumes
from its checkpoint either way.

Note what no branching or PR gate would have caught here. That change
was green on its branch, green on the merge and green on main; the fault
existed only against the persistent /cache database, which no fixture
reproduces. Code is gated by CI, irreplaceable state by refusing to
touch it and by docs/index-cache.md's restore.

The other half is the mismatch that started this: two harnesses reading
two files. AGENTS.md is a symlink to CLAUDE.md and skill-check asserts
the symlink rather than comparing contents, because a copy would satisfy
every other check in this repo while silently drifting -- which is the
failure being prevented. The same check now scans CLAUDE.md for make
targets, which it never did: 27 targets named in the file agents trust
most, none of them verified. Coverage goes 19 -> 46.

Scanning prose meant the line-start rule needed a fence. "Two green
branches do not / make a green merge" wrapped onto a line beginning
`make a` and duly failed on a target called `a`. Inside a fence it is
code; outside one it is a sentence that broke there, and a check that
fails on reflow gets disabled rather than fixed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 15:30:38 -04:00
logan 3d65da0529 test(download): stop racing a download these tests never wanted
Build & publish Arch package / arch-package (push) Successful in 2m36s
CI / e2e (push) Successful in 6m25s
CI / check (push) Successful in 2m43s
`check` failed on main with two failures in one package, and they are one
cause wearing two shapes:

    service_test.go:66: state = "satisfied", want wanted
    testing.go:1369: TempDir RemoveAll cleanup: ... directory not empty

Every test in service_test.go is about the durable Request that
StartDownload leaves behind, and none is about the download. But the
fixture is an anchored four-track request with a healthy provider, which
is precisely what AutoPickable says yes to -- so Manager.Start fired
`go m.grab(...)`, detached and with context.WithoutCancel, and the tests
raced it. Measured: the request reaches "satisfied" about 100ms after
StartDownload returns, so the first failure is the assertion reading the
next state, and the second is that same goroutine still writing into
t.TempDir() after the test returned.

The fixture now puts the candidate outside the auto-pick size window, so
the grab never starts. That is better than waiting for it: with no
goroutine there is nothing to be slow, and the tests state what they mean
without a timing assumption underneath. A test that does want the
download uses managerFixture and sets its own preferences.

It passed 20 runs under CPU load, but so did the broken version -- this
is a CI-only failure locally, so the cause was proved directly instead:
with the fixture's old preferences the request is observably "satisfied"
within 100ms of StartDownload, which is what CI read.
2026-08-17 14:16:45 -04:00
logan 52cbef27c4 docs: name the guard that covers every cache table
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m30s
The bullet added with the credit work names
`TestTheCatalogSurvivesAStaleShape`, which pins the table and shape that
failed. The general guard landed the same day and is the one that covers
a table nobody remembered -- flipping the policy back fails it on five,
including both artist-credit tables.
2026-08-17 14:01:28 -04:00
logan c03c0b8ec4 test(database): the next destructive repair fails a test, not a volume
Build & publish Arch package / arch-package (push) Successful in 2m33s
CI / check (push) Successful in 3m14s
CI / e2e (push) Canceled after 3m3s
The fix for the dropped catalog pins one table in one wrong shape, which
is the failure that happened. What cost the rebuild was more general: a
destructive repair added at `database.NewDB` -- the chokepoint every
binary in this project shares -- without asking which binary it runs in.
The next one will have a different name and a different reason.

So `TestNoCacheTableIsRetiredHere` asserts the outcome instead: put every
`datamap` Cache table into a shape the schema has moved past, open the
database the way cmd/indexbuild does, and require all of them to still be
there. Driving it from `datamap.ByKind` is what makes it cover tables
nobody remembered -- flipping the policy back fails on five, including
the two artist-credit tables added the same day, where the existing test
fails on one. It asserts the rows survive too, because SQLite does an
implicit DELETE before a DROP and a repair that recreated the table would
look identical. And it accepts an error from `NewDB`, because that is the
documented trade: loud is recoverable, gone is not.

`scripts/index-cache-snapshot.sh` covers the half no test can reach. The
volume holds the only copy of a catalog that costs hours of someone
else's bandwidth to re-derive. `VACUUM INTO` rather than `cp`, since a
byte copy of a live SQLite file is a corrupt file of plausible size; the
resumable staging directory is skipped; and each snapshot is reopened and
asked for its catalog row count before anything is rotated out. A corrupt
source and an empty catalog were both exercised: each exits non-zero,
removes its own output, and leaves the previous snapshots alone.

docs/index-cache.md is the restore, and the reason to bother: a restored
snapshot resolves to `refresh` and folds in the listens since, which is
minutes against the 3-23h this rebuild has been estimating.
2026-08-17 13:56:08 -04:00
yonlu 8c48105ca3 Merge remote-tracking branch 'origin/main' into wails-v3
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m31s
2026-08-17 13:52:34 -04:00
logan 1c4d6ca9a1 ci: stop booking three hours of runner on every push
Build & publish Arch package / arch-package (push) Successful in 2m34s
CI / check (push) Canceled after 53s
CI / e2e (push) Canceled after 0s
The catalog this job derives was dropped by the stale-shape repair (see
`fix(database): never retire the catalog the index build derives`, which
prevents a recurrence but cannot undo it), so `mode=auto` now resolves to
a full ~205 GB import from the dumps.

That import runs on every push to main with a 3h budget, on a runner of
capacity 1 -- so ordinary CI has been queuing behind it since the merge,
and each further push books another three hours. The damage is the
repetition, not the single job.

The `push` trigger is commented out until a run reports `complete=true`.
The weekly cron and workflow_dispatch still resume the build, which is
all it needs: indexbuild picks up from its checkpoint, so nothing already
imported is re-fetched.

Restoring the two commented lines is the entire revert, and the comment
beside them says so. NOTES.md carries the incident, including the two
things worth changing regardless: a destructive repair running inside
`database.NewDB` has to ask which binary it is in, and the only copy of a
205 GB derived asset is a single Docker volume with no snapshot.
2026-08-17 13:29:58 -04:00
yonluandClaude Opus 5 6bf832a4ba docs: record what credits are, and what the repair must never touch
Two mechanisms shipped today whose invariants are not visible from the
code, and one of them has already cost a rebuild.

Credits: why join phrases are assembly instructions rather than
disassembly ones, why credited_name is stored per row instead of joined
from artists, why the lookup is keyed on the recording MBID (and so
needed no local table), why an absent credit is cached as an answer,
and why the decomposition comes from a third dump at all — the
canonical dump has no join phrases and the JSON dumps overlap a real
library by zero rows. The measurements that justify the feature are
here too, including the correction that the "3 of 2,823" figure behind
plan 013 measured our own writer rather than any library.

The stale-shape repair gains the paragraph it should have shipped with:
retiring a Cache table is a build-tag decision, because the app
downloads its catalog and cmd/indexbuild derives it. Written as what
happened rather than as advice, since it dropped the real CI catalog on
its first run and the shape mismatch it found was there by design.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 13:13:05 -04:00
yonluandClaude Opus 5 4f8257ef72 fix(database): never retire the catalog the index build derives
Search index maintenance / maintain-index (push) Canceled after 0s
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m30s
The stale-shape repair dropped the CI catalog on its first run:

    retiring a table ... table=explore_index
      reason="column entity_type is TEXT, schema declares INTEGER"
    index maintenance mode=build reason="no completed import yet"

The mismatch was real and the drop was correct by the app's rule: a
client's catalog is *downloaded*, so a wrong shape costs a minute of
re-fetching the artifact, while keeping it costs every Explore read.

It is the wrong rule for one database. cmd/indexbuild's catalog is not
downloaded, it is what the artifact is cut from — the only way back is
the ~205 GB dump stream the /cache volume exists to avoid. And that
database is deliberately kept in the older encoding, which
`fix(indexexport): read an index older than the binary` exists to
tolerate, so the shape does not match by design and would have been
dropped on every run.

retireLibraryTables, right beside it, never touches the catalog for
exactly this reason. The repair reached past that protection because it
runs inside database.NewDB, which cmd/indexbuild also calls.

So the policy is a build tag, which is how this project already tells
the index tools apart (runtime_indexbuild.go, servicestartup.go,
dumpbuild_stub.go): Cache tables are rebuilt in the app and never in
cmd/indexbuild. Owned and Derived are still repaired in both — that is
the half this database can safely discard, and retireLibraryTables
already discards it.

The residual trade is deliberate: a future explore_index column will
now fail the index job loudly on applySchema rather than silently
costing it a 205 GB rebuild. A human should decide that one.

TestTheCatalogSurvivesAStaleShape is the accident, symptom first, with
the shape the real database is in — every current column, ids and
entity type still text. It fails with "the catalog was retired" when
the policy is flipped back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 12:24:49 -04:00
yonlu b505959934 Merge remote-tracking branch 'origin/main' into wails-v3
Build & publish Arch package / arch-package (push) Successful in 2m32s
CI / check (push) Successful in 3m10s
CI / e2e (push) Canceled after 0s
Search index maintenance / maintain-index (push) Canceled after 1h31m25s
2026-08-17 11:38:49 -04:00
logan d0250a2133 docs: confirm the phone track list on the phone
Build & publish Arch package / arch-package (push) Successful in 2m32s
Search index maintenance / maintain-index (push) Successful in 7s
CI / check (push) Successful in 2m26s
CI / e2e (push) Successful in 6m12s
Build & publish the Android APK / apk (push) Successful in 1m46s
Sync Homebrew formula / sync-formula (push) Successful in 7s
The arrangement and the width fix, measured on the device with the build
installed rather than at the same viewport in a browser: `24px 304px
80px`, 52px rows, no header, the title untruncated, no overflow. Same
numbers both places, which is why both were measured.
2026-08-17 10:51:23 -04:00
logan de2b324e20 feat(explore): refuse 0.6 GB on someone's mobile data
CI / check (push) Canceled after 0s
CI / e2e (push) Canceled after 0s
Search index maintenance / maintain-index (push) Canceled after 0s
Build & publish Arch package / arch-package (push) Successful in 2m30s
Plan 016 B4. The catalog artifact is about 0.6 GB and the app fetched it
with no awareness of the connection: on a desktop that is a minute of
bandwidth, on a phone it can be a month's allowance. It is now skipped on
a cellular connection unless `AllowMeteredCatalogDownload` is on, with
the toggle in Settings' Search Index section, where the text explaining
what the catalog is already lives.

The file layout is dictated by the cgo rule rather than by taste.
`explore` is imported by `cmd/indexbuild`, which builds with
CGO_ENABLED=0 and must not link Wails, so `netpolicy.go` holds the policy
and the JSON parsing -- tested on every platform -- and the single
platform call is a closure injected from `app.go`, which already names
`application` legitimately.

Three rules in it are load-bearing. An unknown answer is not a metered
one: only mobile answers at all, and treating silence as metered would
have disabled the download for every desktop user in the world. Cellular
is the only signal available, because the runtime reports
`wifi|cellular|ethernet|none` and no metered flag -- so a metered Wi-Fi
cannot be detected and is not refused, which is documented rather than
implied. And the gate runs before the first status write, so declining is
a no-op instead of a job in the indicator and an error tier to dismiss.

Two corrections to the plan while implementing it: the portable API is
`application.Mobile.NetworkJSON()`, not `application.Android`'s, which
exists only under the `android` build tag; and the permission is read at
the moment a download would start, so enabling it takes effect on the
next attempt rather than the next launch.
2026-08-17 10:48:00 -04:00
logan 2c78b58207 feat(ui): the track list a phone can read
Build & publish Arch package / arch-package (push) Successful in 2m33s
CI / check (push) Successful in 2m26s
Search index maintenance / maintain-index (push) Successful in 7s
CI / e2e (push) Successful in 6m15s
B2 phase 4, and the last of it. Measured on the device: at 424 CSS px
the four configured columns fit the row *exactly* -- `--grid-cols` came
out `24px 102px 101px 101px 80px` -- and not one of them fit its
content, with "Duration" too narrow for its own header. The columns were
never too wide; there were too many of them.

So a phone draws `titleArtist` (the title with the artist under it,
across the row's whole width) plus the duration, and drops the column
headers and the resize handles, which are a click-to-sort and a drag
with no touch equivalent. It is a **column set, not a second row
template**: the row, its delegated events, the selection semantics, the
playing marker and the virtualizer never learn anything changed, because
from their side only the number of columns did.

Three rules come with it. The row height is in two places
(`PHONE_ROW_HEIGHT` and the CSS rule) and must agree, since the
virtualizer positions rows from that number and a taller row overlaps
its neighbour. What is drawn and what can be sorted are different
questions, so the sort list is built from `configuredColumns` -- a phone
has no headers either, and building it from the drawn columns would
leave it able to sort by title and duration alone. And a phone's column
widths are neither loaded nor saved.

That third rule is the bug the device found with the arrangement already
passing five component tests and five e2e specs at the phone's own
viewport. `loadColumnWidths` is keyed by column *id* and fills a gap
with `MIN_COLUMN_WIDTH`, so the stacked column -- which nothing can ever
have saved a width for -- came out at 148px beside a duration column of
236. The mirror image was worse and unreachable from a phone at all:
saving would have written those widths back under the same ids,
replacing the width the user dragged on a desktop. The specs asserted
shape, and the fault depended on what `localStorage` held for a
different column set; the unit test now carries that map as a fixture.

Verified: 809 component tests, 112 e2e specs, and on the phone at
424x439 -- `24px 304px 80px`, 52px rows, no truncation, no overflow.
One full e2e run of three saw an unrelated autotag keypress spec flake
and pass on retry.
2026-08-17 10:36:29 -04:00
logan a9852c18a0 docs: the device answered both open questions, and neither as expected
Build & publish Arch package / arch-package (push) Successful in 2m39s
CI / check (push) Successful in 2m39s
Search index maintenance / maintain-index (push) Successful in 6s
CI / e2e (push) Successful in 5m57s
Both faults reported from the phone are now measured rather than
inferred, with the installed build and current main compared on the same
device.

"The controls are off screen" was literal and already fixed: the
installed build predates B2 phase 2, so its player bar still carried the
seek bar and volume at 424px and the transport ran past the right edge.
Current main measures no horizontal overflow and the controls at 200..380
inside 424, on the phone's own engine.

"No icons" was my own screenshot: taken six seconds after a cold start,
before the icon fetches landed. On the settled app every icon paints, and
the earlier black `fill` was the svg root rather than the path that
carries `fill="currentColor"`. Two conclusions from one misread node,
both corrected.

Chrome 113's missing Popover API does not break the menus, which was the
standing worry: a long-press opens the real panel with seven items,
positioned and painted -- so long-press is now verified on hardware over
a 1,744-track library, not just in a browser at a phone-shaped viewport.

What the device does add is a measurement for phase 4: the track list's
columns fit the host exactly and are simply too many for 424px.
2026-08-17 10:15:01 -04:00
yonluandClaude Opus 5 409bfd5e89 test(download): wait for the work, not for the state that precedes it
TestManagerEndToEndAutoPick waits for StateComplete and then asserts
that staging was released and the library was rescanned. Those happen
*after* the state is recorded: manager.go sets StateComplete, then
satisfies the request, then releases staging, then scans. So waiting on
the state is not waiting on either assertion, and on a loaded machine
the worker is descheduled in between and the test reads the world one
step too early:

    manager_test.go:209: staging not released: 1 dirs remain
    manager_test.go:218: library scans = 0, want 1

It passed alone every time and failed three times under a full-suite
run, which is the signature of a test race rather than a broken
manager — nothing here is wrong except what the test chose to wait on.
It blocks pushes, since the pre-push hook is exactly the loaded run.

It polls for the side effects now, through the waitFor this package
already has and already uses for the same reason one file over
(service_test.go waits for a request to become satisfied after the same
StateComplete).

Not reproduced on demand: eight spinners and -count=5 did not provoke
it with or without the fix, so this rests on the ordering being plain
in the code rather than on a red-to-green demonstration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 09:22:51 -04:00
yonluandClaude Opus 5 0eeef6048e feat(frontend): credit the artists on the full-screen now playing too
The phone shell's now-playing view landed on main while the credit
rendering was being written, so it arrived with the one call site that
still showed a multi-artist credit as a single link with the other
artists as punctuation inside it.

It is the same fix as the other ten: render from the parts, fall back
to the single link when there are fewer than two. The subscription is
what makes it show up at all — credits arrive after the track does, so
the name already on screen has to be re-rendered when they land.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:34:27 -04:00
yonlu 4fc0cdeab7 Merge remote-tracking branch 'origin/main' into wails-v3 2026-08-17 08:29:21 -04:00
yonluandClaude Opus 5 eb059a3d71 fix(database): retire a table whose shape the schema moved past
`applySchema` is CREATE ... IF NOT EXISTS and there is no migration
chain, so a *changed* table never migrates: the statement silently
no-ops against the old shape. Two plans had already landed on that, and
neither showed up in a test because a fresh install is perfectly
healthy.

- 014 added `total_tracks` to explore_index and to `indexRowFields`,
  the projection every explore read uses, so every search, browse,
  artist page and album page failed with "no such column: total_tracks"
  on any database that already had a catalog.
- 013 reshaped audio_files, so applySchema could not run at all and the
  app did not open.

staleshape.go runs before applySchema and drops what disagrees, so the
create is a create. It parses sql/schemas/ for the expectation rather
than writing the column list down a second time, and it notices a
changed *type* as well as a missing column — 013 moved mbid TEXT to
BLOB, which no ALTER could express and which SQLite will not coerce, so
a query against 16 raw bytes returns no rows rather than an error.

Only Authored tables are exempt. Cache is rebuildable by definition,
Owned is what a rescan rebuilds (plan 013's stated "delete and
rescan"), and a table the schema no longer describes at all goes too --
013 left seven behind plus schema_migrations.

Three things in it are load-bearing, and each was a bug first:

- The parser read `UNIQUE(mbid)` as a column, which made a healthy
  catalog look stale. That would have retired it on every launch and
  cost every user an artifact download per start.
- The drops are one transaction with defer_foreign_keys. Those legacy
  tables reference each other, so any order fails on whichever goes
  first; turning foreign keys off instead would suppress
  playlist_tracks.audio_file_id's ON DELETE SET NULL and leave entries
  pointing at ids a rescan reissues to *different songs*. Nulled
  entries are empty; stale ones are wrong, and wrong quietly.
- The order is sorted, so a failure reproduces. Map order is random,
  and the foreign-key bug passed its own regression test on two runs in
  three until the order was fixed.

Verified against a real pre-013 install: it opens, its 22 playlists
survive, 1,887 linked playlist entries become 0 rather than dangling,
and the legacy tables are swept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:27:05 -04:00
yonluandClaude Opus 5 dcabec8b1d feat(frontend): render a multi-artist credit as one link per artist
Every artist name in the app went through `artistLink(name, mbid)`, so
a track credited to several artists rendered one link and the rest as
punctuation — "2Pac feat. Snoop Dogg" linked 2Pac and left Snoop Dogg
as text inside it.

`creditLink(parts, fallbackName, fallbackMbid)` renders the credit from
its parts: one link per credited artist, join phrases as plain text
between them. The link boundaries are known by construction, which is
the point — locating a name inside the stored credit string would
reintroduce the mismatch the catalog exists to avoid, since that string
may come from the file's tags while the parts come from MusicBrainz and
the two disagree for ~1 in 3 multi-artist credits.

Fewer than two parts falls through to the previous behaviour exactly,
so a single-artist credit, a file with no recording MBID and a catalog
that has not answered yet all render as they did before. Nothing tries
to split the fallback string: "Simon & Garfunkel" is one artist, which
is why primaryArtist() does not split on "&" either.

The lookup is keyed on the recording MBID, which both sides already
carry — a catalog row has one and so does a local file — so one binding
serves Explore and the library's own lists, and no local table is
needed for this.

credit-store.ts, and three things in it are load-bearing:

- A miss is cached as an empty array. The backend returns nothing for a
  single-artist credit, which is ~87% of tracks, and caching only the
  hits would re-request the rest on every render forever.
- request() is per-row and coalesces into one call per frame. A
  virtualized list cannot hand over "the whole list": 50,000 rows would
  be 100 queries for the ~30 on screen.
- It is an LRU with a counted retainedChars probe, because a cache that
  grows with use is a leak with a schedule.

The virtualized lists push requestUpdate() into the virtualizer rather
than only the host, since its rows come from its own properties — a
host update alone would leave them exactly as they were. now-playing
marks its geometry dirty instead, because the marquee measures the text
it is about to scroll.

track-list keeps the single link while a search term is active: the
highlight spans are computed against the flat credit string, and
mapping them onto decomposed parts is a different problem.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:26:34 -04:00
yonluandClaude Opus 5 b3737d30af feat(explore): carry multi-artist credits in the catalog
A track credited to more than one artist has exactly one navigable
artist in this app and the rest are punctuation. `primaryArtist()`
string-parses the credit, strips a " feat. " clause and discards the
guest; it deliberately does not split on "&", "with" or "," because
those live inside real artist names.

Measured on a real 26,069-file library plus an 80+80 MusicBrainz
sample: 13% of recordings are multi-artist upstream, while only 0.86%
of files carry any structured multi-artist tag — mp3 carries zero
files with multiple MUSICBRAINZ_ARTISTID across 19,840. Of 1,286 files
saying "feat.", 90% have nothing structured behind it, and a sample of
80 such files was multi-artist in MB 80 times out of 80.

CLAUDE.md justified plan 013's removal of the credit tables with "3
credits of 2,823 listed more than one artist". That measured our own
*writer* — cachedLinkArtist was called once per credit, so a
collaboration could never have been recorded. Dropping the join table
was still right on cost; the evidence for "multi-artist is rare" was
not.

A credit is ordered parts and the credit string is derived from them,
so join phrases are assembly instructions, not disassembly ones.
Nothing here reconstructs a credit by searching a name inside a credit
string: the stored text may come from tags while the parts come from
the catalog, and those disagree for ~1 in 3 multi-artist credits.

Where it comes from, after two dead ends: the canonical dump CI
already streams has no join phrases and no as-credited names, and the
JSON dumps cover 153,691 recordings of ~35M with *zero* overlap
against a real library. So mbdump.tar.bz2 — 7.1 GB, ~13.7 min in
pure-Go bzip2, whose members are alphabetical, which is what lets one
pass resolve an entity's credit without buffering 35M recordings.

- artist_credit_part / artist_credit_ref, multi-artist credits only:
  a single-artist credit is already explore_index's own artist_name.
- Column layouts verified against the real 20260815 export;
  ErrDumpShape makes a wrong guess a failed build, not a wrong catalog.
- The pass runs on every mode, not just a build. The job picks its mode
  from the index's own state, and a complete import means "refresh",
  which never enters the importer — so credits could otherwise only
  arrive via a rebuild that re-downloads ~205 GB. It reports whether it
  populated anything, which is what flips `changed` and republishes.
- The importer asks whether an artifact carries the tables, on the
  writer where `core` is attached, so the artifact already published
  still imports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-17 08:25:36 -04:00
logan 0bfa2136be feat(dev): ask the phone instead of looking at it
Build & publish Arch package / arch-package (push) Successful in 2m29s
CI / check (push) Successful in 2m26s
Search index maintenance / maintain-index (push) Successful in 7s
CI / e2e (push) Successful in 5m53s
The device tier could only take a screenshot and read what Go chose to
log, and a screenshot cannot tell a dropped CSS declaration from a
missing asset. This adds the third thing: the page's own answer, from
the engine that is really rendering it.

`make android-screenshot` grabs the screen, `make android-inspect`
forwards the WebView's devtools socket, and `make android-eval EXPR=...`
evaluates in the real page.

Four details are load-bearing. Only a `debuggable` build opens that
socket, so the debug build type takes `applicationIdSuffix ".dev"` and
installs *beside* the release app -- the two carry different signing
certificates, and Android's only remedy for a changed certificate is an
uninstall, which takes the user's library with it. Playwright cannot
drive a WebView (`connectOverCDP` calls `Browser.setDownloadBehavior`,
which it answers "Browser context management is not supported"), so the
eval is raw CDP over Node's built-in WebSocket. The socket name carries
the pid, so it is resolved per launch rather than written down. And
`exec-out`, not `shell`, for the screenshot: a pty translates LF and
corrupts the PNG.

What it immediately established is why it was worth having. The phone
renders in Chrome 113 at 424x439 CSS px -- two years behind every
browser the other tiers use, with no Popover API and no relaxed CSS
nesting -- so a spec passing at that viewport says nothing about the
device, and two conclusions drawn from version numbers alone were wrong.
Both are corrected in NOTES.md and the plan.
2026-08-17 05:13:21 -04:00
logan b1cdef8769 docs: record what a phone said that no tier could
Build & publish Arch package / arch-package (push) Successful in 2m32s
Search index maintenance / maintain-index (push) Successful in 5s
CI / e2e (push) Successful in 6m10s
CI / check (push) Successful in 3m28s
The first device run of the published APK, and the first runtime
evidence any of the Android work has ever had -- A4 shipped entirely
reasoned from source.

It confirms A4 whole: playback survives the screen locking, and the
transport notification appears with cover art, which settles four
open questions at once (the service starts, the permission was granted
and the notification is visible, the lock screen picks up the session,
and art decoded from a MANAGE_EXTERNAL_STORAGE path by a service is
readable -- the one nobody could argue from documentation).

It also found the two faults fixed in the preceding commits, and the
lesson worth keeping is why *those two*: both are things the platform
adds rather than things the app draws. So the skill's Android tier now
says to ask a device about system bars, the back gesture, focus and
audio interruptions, permissions and the keyboard -- and not about
layout, which the other five tiers already cover.
2026-08-17 02:02:10 -04:00
logan d661836347 fix(android): keep the app out from under the system bars
Reported from the first device run: the playback controls are off
screen. `targetSdk 35` is Android 15, which lays every app out
edge-to-edge and ignores the deprecated `statusBarColor` and
`navigationBarColor` the scaffold's theme still sets -- so a
`match_parent` WebView draws the page's bottom band, which on a phone
is the transport *and* the tab bar, underneath the gesture bar.

`applyWindowInsets()` pads the container by
`systemBars | displayCutout | ime` and returns the insets rather than
consuming them, so the WebView is laid out inside them. The keyboard is
in the mask because a search box the keyboard covers is the same bug
one surface over.

The window background goes black to match the app's own default ramp:
that padding is what shows through, and a band of the scaffold's
blue-grey above and below reads as the app failing to fill the screen.

No tier we have can see this class of fault -- a browser viewport has
no system bars, so `phone-shell.spec.ts` at 390x844 renders a shell
that fits at the moment the device is clipping it. Verified only as far
as the APK building; the insets need the next build on a phone.
2026-08-17 02:02:10 -04:00
logan 28eecf0a97 fix(ui): the Android back button had nowhere to go
Reported from the first device run: back does not navigate back in the
app. The scaffold's `MainActivity.onBackPressed` asks
`webView.canGoBack()` and finishes the activity otherwise -- and this
app had never touched `history`, so that was false at every depth and
back quit from anywhere.

The fix is here rather than in Java, because the mechanism the scaffold
already uses is the one we were failing to feed: a navigation is a
history entry now, and `popstate` replays it. Nothing on the Android
side changes, and the behaviour becomes assertable in a browser with
`page.goBack()` instead of only on a phone.

The entry keeps the same URL -- the app has no routes, and a path a
reload cannot resolve is worse than none -- and carries the destination
in its state.

Two rules keep the stacks from disagreeing. The first navigation
*replaces* the launch entry rather than pushing one, or every launch
costs a back press before the app will close. And the in-app back
buttons go through `history.back()` rather than popping a stack of
their own: `navStack` is deleted, not kept alongside, because two
stacks is precisely how a detail view's own button and the phone's
gesture come to disagree about how far one press goes. The third spec
pins that invariant.
2026-08-17 02:01:57 -04:00
logan e8690476bd feat(ui): long-press opens the menus a right-click opens
Every context menu in the app opens from a `contextmenu` event, bound
three different ways across six components -- delegated on a
virtualizer, per row, per card. A phone has no right-click, so a phone
reached none of them (plan 016 B2 phase 3).

This is one document-capture listener installed once from `index.ts`,
not six components' worth of touch handling: a touch that holds still
for 500ms dispatches a synthetic `contextmenu` at the touch point, and
every existing handler runs unchanged. A seam no component has to opt
into is one no future component can forget.

Four details are load-bearing, each a way the obvious version fails.
The target is `composedPath()[0]`, not `elementFromPoint`, which stops
at the outermost shadow host -- every menu here is bound inside one, so
a host-targeted event reaches a delegated listener and no per-row one.
A browser that fires its own long-press `contextmenu` (Chromium does;
WebKit and the Android WebView vary) wins, and ours is told from theirs
by identity rather than `isTrusted`: `isTrusted` works in the app and
is untestable, which would leave the suppression path as the one thing
with no coverage. And the click ending the gesture is swallowed, keyed
on the gesture rather than a time window, or the first tap on the menu
it just opened is eaten too.

The e2e spec presses `.track-row`, not `[role="row"]`: the column
header is a row too, and it is the first one -- a press on it is
correctly ignored, which reads exactly like the gesture not working.
2026-08-17 02:01:46 -04:00
logan 7e0be8fa30 fix(indexexport): read an index older than the binary
Build & publish Arch package / arch-package (push) Successful in 2m24s
CI / check (push) Successful in 2m29s
Search index maintenance / maintain-index (push) Successful in 6s
CI / e2e (push) Successful in 6m38s
`maintain-index` failed with

    indexexport: copy rows: SQL logic error: no such column: total_tracks

three minutes into the one job that owns the ~205 GB checkpoint and
publishes the catalog every user downloads.

The cause is the exception that keeps that checkpoint alive. The job's
/cache is a real YJ_HOME that survives between runs, so its
explore_index is classified Cache and is deliberately *not* dropped and
recreated by cmd/indexbuild's schema repair -- which means a column
added to the schema afterwards is absent from it. total_tracks arrived
with the album-completeness work; the exporter selected it regardless.

The fix is the rule the importing side already follows.
artifactHasTotals exists because "adding a column to the importer's
SELECT is how you break every artifact already published"; the mirror
image, reading an index older than the binary, had no such guard.
sourceColumns asks pragma_table_info and selects a literal 0 when the
column is absent -- which is what that column already means by "the
catalog does not say", and what the app renders as unknown rather than
as incomplete. The artifact keeps every column, so an importer needs no
second shape.

The test reproduces the failure symptom first: with the fix removed it
fails with the CI message verbatim. Its own first version proved
nothing, though, and that is worth the comment it now carries --
`strings.Replace(catalogColumns, "total_tracks, ", …)` matches nothing,
because the list is formatted across lines and the name is followed by
a newline, so the "old" index was built with every current column.
2026-08-17 00:39:19 -04:00
logan 1b05dde382 feat(ui): the full-screen now playing a phone needs
CI / check (push) Successful in 2m25s
Search index maintenance / maintain-index (push) Failing after 2m53s
Build & publish Arch package / arch-package (push) Successful in 2m27s
CI / e2e (push) Successful in 5m55s
Plan 016 B2, phase 2. Phase 1 took the seek bar and the volume out of
the phone's bottom bar -- 4px of height is not a thumb target, and a
phone's volume belongs to its hardware keys -- and promised them a
full-screen view. This is it, reached from a button over the mini
player's cover art.

**It composes the transport rather than reimplementing it.** The same
`seek-bar`, `player-controls` and `volume-control` the desktop bar
uses; a phone layout that copies them is a second transport to fix
every bug in, and the seek bar in particular carries interpolation
rules that took a plan of their own to get right. The seek bar
thickens its own track below the breakpoint, in its own stylesheet,
because the track size lives on a wa-slider inside its shadow root
where a custom property from the host cannot reach.

**It is a detail view, not a primary one.** It is somewhere you go and
come back from, so index.ts pushes the current view and Back pops it --
which is also why it is not a fifth tab: a tab you cannot leave by
pressing it again is not a tab.

Two things came from reading a screenshot rather than from a failing
test, and both were invisible to assertions that were individually
correct.

**The mini player was still under the full-screen view**, repeating it
in 4em of an 844px phone. index.css hides the bottom bar while
`#main-content[data-active-view="now-playing"]`, through `:has()`
rather than a class toggled from index.ts, because the active view is
already published as an attribute. That takes the queue button with it,
so the view carries its own.

**And phase 1's shell rules had never applied.** A media query adds no
specificity, and the phone block sat above the plain rules it meant to
override, so at 390px the header kept its 2em gutters (32px), its 16px
gap and its 24px title, and the bottom bar kept a fixed 320px first
column. Nothing failed: the shell fits because of `min-width: 0` and
each component's own media query, which live in their own stylesheets
and have no later rule to lose to -- so what was dead was exactly the
cosmetic half no assertion looks at. The phone rules are one section at
the end of the file now, and it says why it is last. Measured after:
12px, 8px, 17.6px, `154px 187px 33px`.
2026-08-17 00:22:58 -04:00
logan 29299d17da fix(dev): run the local e2e tier against the app CI runs
Build & publish Arch package / arch-package (push) Successful in 2m26s
CI / check (push) Successful in 2m30s
Search index maintenance / maintain-index (push) Successful in 7s
CI / e2e (push) Successful in 6m5s
Two specs failed locally and passed in CI, which is the least useful
direction for a disagreement to point.

**`dev-headless.sh` was the only launcher not stubbing out the
catalog.** `seed-sandbox.sh` and `ci.yml` both send
`YJ_CORE_INDEX_URL` to a dead address; the dev launcher did not, so the
app downloaded and built the real ~1M-row Explore catalog into the
run's YJ_HOME and every local `make e2e` after that ran against a world
CI never sees. Found by reading the failure screenshot: the spec had
searched Explore for its fixture album and the page was full of real
ones. It defaults to the dead address now and takes an explicit one for
exploring by hand.

**And the shared backend carries spec state between runs.**
`explore-shelves` staged its catalog only `IfEmpty`, so one album row
left behind by `requested-badge` satisfied that gate: the shelves were
drawn from a single foreign row and the artist card the spec clicks did
not exist. It failed on the *second* local run and passed on the first,
and never in CI, where every run gets a fresh home.

"Is the catalog empty" was the wrong question and "are my rows there"
is the right one, so staging is unconditional (INSERT OR IGNORE keyed
on the MBID) and the assertion moved from *this insert wrote a row* to
*every fixture row is present*. That is both idempotent and stronger:
an MBID that fails CHECK(length(mbid) = 16) is silently dropped by OR
IGNORE, which the old per-insert count caught only on a cold catalog
and the new one catches always.

Verified by running the whole suite twice against one app: 97/3 before,
100 passed both times after.
2026-08-16 23:47:24 -04:00
logan 57fbbdf0d2 feat(ui): a shell a phone can be held in
Build & publish Arch package / arch-package (push) Successful in 2m33s
CI / check (push) Successful in 2m33s
Search index maintenance / maintain-index (push) Successful in 7s
CI / e2e (push) Successful in 5m40s
Plan 016 B2, phase 1. Below 600px the grid drops its sidebar column,
`bottom-nav` becomes the primary navigation, and the shell fits the
viewport instead of scrolling sideways out of it.

600 rather than the sidebar's own 900, because 900 is a laptop and the
answer there is a narrower sidebar, which is still a sidebar. Under 600
there is no room for one at all: 360px of viewport over a 200px nav is
not a layout.

**The tab bar is four destinations and a way to everything else.**
Three to five is where touch targets stop being thumb-sized -- eleven
over 360px is 32px each -- so the four are the ones plan 016's subset
says a phone is for, and "More" opens the *existing* `app-sidebar` in a
drawer rather than listing the destinations a second time. Two lists is
two places to add the next view to.

That reuse has a cost this found the hard way: a shared component
brings its `data-testid`s with it, so rendering the drawer's sidebar
unconditionally put a second `nav-home` (and ten siblings) in the DOM
and **failed 30 existing specs** with "resolved to 2 elements" -- on a
desktop viewport, where this element is `display: none` and the drawer
can never open. It renders only while the drawer is open, and the
component test asserts the absence, because the failure is invisible
from inside the component and lands in files nobody touched.

**What made the shell overflow was minimums, not padding.** Measured at
360px: the body was 652px wide, because a `min-width` in a flex row is
a hard floor and a grid item's implicit minimum is its content. So
`min-width: 0` on the boxes between the viewport and the content, and
each component stands its own non-essential parts down in its *own*
stylesheet -- search-bar's 200px floor, job-indicator's label (the
visible one; the live region that announces it is untouched),
audio-player's seek bar and volume. A media query inside a shadow root
is answered by the viewport, so this is the component saying what it
drops rather than the shell reaching in.

Volume goes because the hardware keys own it on a phone, which is the
same reason mediacontrols' Android handler implements no volume
callback. Seeking goes because 4px is not a thumb target; it belongs to
the full-screen now-playing view, which is the next phase.

An existing spec therefore asserts the opposite of what it did:
layout-overflow's 320px case used to require that the 464px behind
`overflow: hidden` could be *scrolled to*, which was the remedy
available while the shell had one layout. It reflows now -- 320px in a
320px viewport, exactly -- and reflow is what WCAG 1.4.10 asked for.
2026-08-16 23:19:26 -04:00
91 changed files with 9142 additions and 151 deletions
+15 -1
View File
@@ -9,9 +9,23 @@ name: CI
# before being written here, so every step below is a transcription of
# something observed working rather than something expected to.
# **A branch push and its PR are the same commit, and testing it twice
# costs the only runner there is.** `branches: ['**']` here meant every
# PR booked four runs — `check` and `e2e` for the branch push, then both
# again for `refs/pull/N/head` — on a host with capacity 1, where the
# queue is shared with an index build that can hold it for three hours.
#
# `pull_request` covers feature branches, and `main` is kept because a
# post-merge run is the record of the trunk's health. Since main now
# refuses direct pushes, that run happens exactly once per merge.
#
# The trade is explicit: a branch pushed with **no** PR open gets no CI.
# That is consistent with the workflow this repo committed to — every
# change goes through a PR — and the signal returns the moment one is
# opened, on the same commit.
on:
push:
branches: ['**']
branches: [main]
pull_request:
workflow_dispatch:
+26 -4
View File
@@ -7,11 +7,29 @@ name: Search index maintenance
# import older than 6mo -> rebuild (re-import from the newest dump)
# otherwise -> refresh (fold in new incremental listens)
#
# A refresh is cheap and no-ops when nothing new has been published, so
# running it on every push to main is safe.
# **There is deliberately no `push` trigger, and restoring one is a
# decision rather than a cleanup.** A refresh is individually cheap, so
# running it on every push to main looked free; what it actually does is
# put an unattended job that mutates the only copy of a ~205 GB catalog
# on the same trigger as an ordinary code change, on a runner with
# capacity 1.
#
# That is not hypothetical. On 2026-08-17 `fix(database): retire a table
# whose shape the schema moved past` landed on main, green — the CI
# database is deliberately in the older encoding, so the stale-shape
# repair judged its `explore_index` stale and dropped it, and this job
# fell back to a full import from the dumps. `fix(database): never
# retire the catalog the index build derives` stops that specific repair
# and cannot undo it. Every push to main then booked another `budget`
# (3h) of the one runner while ordinary CI queued behind it.
#
# So the rule this file is an instance of: **a job that mutates state
# which cannot be rebuilt in ten minutes is triggered deliberately, not
# by a push.** The weekly cron keeps the catalog current, and
# workflow_dispatch resumes or forces a build — indexbuild picks up from
# its checkpoint either way, so nothing is lost by not running on every
# merge. See docs/index-cache.md for the snapshot and the restore.
on:
push:
branches: [main]
schedule:
# Weekly update pass. The 6-month rebuild is triggered by the same
# command when it notices the import has aged out.
@@ -33,6 +51,10 @@ on:
# Runs share one persistent working directory, so they must not overlap.
# A push landing mid-build waits rather than corrupting the checkpoint.
#
# That directory holds the only copy of a catalog nothing can cheaply
# re-derive: see docs/index-cache.md for the snapshot it takes and the
# restore, which is minutes against the hours a rebuild costs.
concurrency:
group: search-index
cancel-in-progress: false
+9
View File
@@ -96,6 +96,15 @@ reference, because you need them *before* the failure, not after.
Run against the `bulk` seed a measurement session left behind and a
third of them fail (13 of 36, when it was measured), in a list that
reads exactly like a regression in whatever you are holding. `make dev-headless SEED=default` first.
- **The catalog is stubbed out locally now, like CI.**
`dev-headless.sh` defaults `YJ_CORE_INDEX_URL` to a dead address
because it was the only launcher that did not — `seed-sandbox.sh` and
`ci.yml` always have. Without it the app downloads the real ~1M-row
Explore catalog into the run's `YJ_HOME`, and specs that stage their
own catalog rows then search a million real ones and fail *locally
only*, which reads as a regression and is an environment. Pass
`YJ_CORE_INDEX_URL=<real url>` when you want the real catalog to
explore by hand.
- **…and the suite spends state it cannot always give back.**
`view-lifecycle.spec.ts` **skips an autotag album** on every run, out
of the eleven the seed has, and does not put it back — so around the
@@ -286,3 +286,65 @@ Related, and it will bite once: the launcher activity is
resolves the leading dot against the *applicationId* and fails with a
class-not-found that reads like a broken build. Always the
fully-qualified form.
## What only a device can answer
The emulator cannot run this app (three separate reasons, none of them
ours — see plan 016), so the phone in someone's pocket is a tier, and
asking for it is cheap. The first run of it, on 2026-08-17, confirmed
the whole of A4 and found two faults **no other tier can see**:
- **The back gesture.** `MainActivity.onBackPressed` asks
`webView.canGoBack()`. Nothing in a desktop shell has a back gesture,
so no spec had ever called `page.goBack()` and the app had never
pushed a history entry — back quit from any depth. It is a history
entry per navigation now, which is also what made it assertable in the
browser tier (`e2e/specs/back-navigation.spec.ts`).
- **The safe area.** `targetSdk 35` forces edge-to-edge, so the
transport and the tab bar sat under the gesture bar. **A browser
viewport has no system bars**: `phone-shell.spec.ts` at 390x844 will
keep passing on a build the device is clipping 48dp off. Insets are
handled in `applyWindowInsets()`.
So when asking for a device run, ask about what the platform *adds* —
system bars, the back gesture, focus and audio interruptions,
permission dialogs, the keyboard — not about what the app draws. The
drawing is what the other five tiers already cover.
## Asking the device, not just looking at it
A real phone can be inspected, and that turns this tier from "reported
symptoms" into evidence. Three commands:
```bash
make android-screenshot # what the screen shows (.dev/ by default)
make android-inspect # forward the WebView's devtools socket
make android-eval EXPR='JSON.stringify({vp:[innerWidth,innerHeight]})'
```
Four things about it, each of which costs an hour if met cold:
- **Only a `debuggable` build has a devtools socket**, and a debug build
carries `applicationIdSuffix ".dev"` so it installs **beside** the
release app. That matters more than convenience: the two are signed by
different certificates, and Android's only remedy for a changed
certificate is an uninstall, which takes the user's library with it.
Never uninstall to make room for a build.
- **Playwright cannot drive it.** `connectOverCDP` calls
`Browser.setDownloadBehavior`, a WebView answers "Browser context
management is not supported", and the connection dies before the first
evaluate. `scripts/android-eval.mjs` is raw CDP over Node's built-in
WebSocket for that reason.
- **Wireless adb drops when the screen sleeps.** The symptoms are
`device offline` mid-session and a `fetch failed` from the eval
script. Plug in over USB for anything longer than a couple of probes.
- **The socket name carries the pid**, which changes on every launch, so
it is resolved rather than remembered.
**And the reason to bother: the phone is an engine, not a screen.** The
first device here renders in **Chrome 113** at 424x439 CSS px. Every
other tier runs a current Chromium or WebKit, so a spec that passes at
that viewport says nothing about the phone — 113 has no Popover API and
no relaxed CSS nesting, and a dropped CSS declaration renders as
"present but wrong", which is the hardest failure to read from a
picture. Get the version first; it reframes every other symptom.
+617
View File
@@ -2828,3 +2828,620 @@ for boot ok". `pick_device` now resolves `ANDROID_SERIAL` from
`ro.boot.qemu.avd_name`, since serials are assigned in boot order and
the AVD name is the stable identity. Verified with both emulators
running: it selects `yj-test` and installs.
## The phone shell fits, and what it cost to make it fit (2026-08-16)
Plan 016 B2, phase 1: the shell below 600px. Measured at 360×780 and
390×844 against the real app (`make dev-headless` + Playwright, which
is the tier that can answer this — server mode serves the same document
an Android WebView renders).
**What overflowed, and by how much.** The body was 652px wide in a
360px viewport before any of this. Walking every element and its shadow
roots for a `right` past the viewport named the causes in order:
| element | width | why |
|---|---|---|
| `header.top-bar` | 580 | its children's minimums, summed |
| `search-bar` | 320 | `.search-container { min-width: 200px }` |
| `job-indicator` | 157 | the label, "3 background jobs" |
A `min-width` in a flex row is a *hard* floor — it does not shrink — and
a grid item's implicit minimum is `auto`, i.e. its content. So the
header could not get smaller than the sum of what it held, the body grew
to the header, and `overflow-x: hidden` would then have hidden a third
of the app rather than fitting it. `min-width: 0` on the boxes between
the viewport and the content, plus each component standing its own
non-essential parts down in its own stylesheet, takes 360 → 360 exactly.
At 320px (400% zoom, the width WCAG 1.4.10 names) it is also exact.
**So an existing spec now asserts the opposite of what it did**, and
that is the fix landing rather than the test being weakened.
`layout-overflow.spec.ts` used to assert that the 464px of app behind
`overflow: hidden` *could be scrolled to* with a wheel gesture, which
was the remedy available when the shell had one layout. It reflows now,
which is what 1.4.10 asks for; scrolling to the overflow was the
concession.
**And a shared component brings its test handles with it.**
`bottom-nav`'s "More" opens the *existing* `<app-sidebar>` in a drawer —
the whole point being not to write a second list of destinations — but
rendering it unconditionally put a second `data-testid="nav-home"` (and
ten siblings) in the DOM. **30 existing specs failed** with "strict mode
violation: resolved to 2 elements", on a *desktop* viewport where
`bottom-nav` is `display: none` and the drawer can never open. Lazy
rendering fixes it; the component test asserts the absence, because the
failure is invisible from inside the component and appears in files
nobody touched.
Three smaller things worth keeping:
- **A new icon name is a runtime failure, not a build one.** `bars` was
not in `src/icons/names.txt`, so `offline-icons.spec.ts` caught it —
the sweep asserts `window.__yjIconMisses` is empty. `node
frontend/scripts/fetch-icons.mjs` re-vendors after adding a line.
- **A `wa-drawer` animates, so a test asserts its events**, not its
`open` property: setting `open = false` starts a hide that has not
finished on the next microtask, and a test reading the property in
between sees the state it is leaving.
- **`update(el)` in the component tier takes two arguments**
(`update(el, {})`), which is only visible from `tsc`, not from a
failing test.
### The local e2e tier was not running the same app CI runs
`requested-badge.spec.ts` failed two of three tests locally while CI was
green, and the reason is worth more than the fix: **`dev-headless.sh`
was the only place that did not neutralise `YJ_CORE_INDEX_URL`.**
`seed-sandbox.sh` and `ci.yml` both point it at `127.0.0.1:1`; the dev
launcher did not, so the app downloaded and built the real ~1M-row
Explore catalog into the run's `YJ_HOME`, and a local `make e2e` then
ran against a world CI never sees.
Found by reading the failure screenshot: the spec had searched Explore
for its fixture album and the page was full of *real* ones — Real
Estate, Arrested Youth, The Yes Album. The staged row was there and
invisible among a million others.
`dev-headless.sh` now defaults the variable to the dead address and
takes an explicit one if you want the real catalog for exploring by
hand. `make e2e` locally: 97 passed / 3 failed before, 100 passed
after.
The second half of the same problem is that **the backend is one shared
process with one database, and specs leave rows in it.**
`explore-shelves` staged its catalog only `IfEmpty`, so a single album
row left behind by `requested-badge` satisfied that gate, the shelves
were drawn from one foreign row, and the artist card the spec clicks did
not exist. It fails on the *second* local run and passes on the first,
which is the least useful order, and never in CI, where every run gets a
fresh `YJ_HOME`.
"Is the catalog empty" was the wrong question; "are my rows there" is
the right one. The staging is unconditional now (`INSERT OR IGNORE`
keyed on the MBID) and the assertion moved from *this insert wrote a
row* to *every fixture row is present* — which is both idempotent and a
stronger check, since an MBID failing `CHECK(length(mbid) = 16)` is
silently dropped by OR IGNORE and would otherwise show up as an empty
page rather than a failed setup.
**Verified: the full suite runs twice against the same app, 100 passed
both times.** That is the property to keep — a spec tier whose second
run differs from its first is a tier that will one day blame the wrong
commit.
## A media query adds no specificity, and dead CSS looks like working CSS (2026-08-16)
Plan 016 B2 phase 2 shipped the full-screen now-playing view, and
checking it with a screenshot found that **phase 1's shell rules had
never applied**.
`index.css` is base rules then component rules, and the phone block had
been inserted in the middle — above the plain `.top-bar` and `.title`
rules it meant to override. A media query is not a specificity boost,
so with equal specificity the *later* declaration wins. Measured at
390px before the fix:
| declared for the phone | actually computed |
|---|---|
| `padding-left: 0.75em` | 32px (the 2em base) |
| `gap: 0.5em` | 16px (base) |
| `font-size: 1.1em` | 24px (the 1.5em base) |
| `grid-template-columns: minmax(0,1fr) auto auto` | `320px 1fr auto` (base) |
After moving the block to the end of the file: 12px, 8px, 17.6px, and
`154px 187px 33px`.
**Nothing failed while they were dead**, which is the part worth
keeping. The phone spec asserts that the shell does not scroll
sideways, and it did not — because the fitting was being done by
`min-width: 0` and by each component's *own* media query, which live in
their own stylesheets and so had no later rule to lose to. The
declarations that did nothing were the cosmetic ones, and no assertion
was ever going to see them. A screenshot did, in about ten seconds.
The file now ends with one phone section, and says why it is last.
### What the same screenshot found about the view itself
The bottom bar was still rendering the mini player *underneath* the
full-screen view — 4em of a 844px phone spent saying exactly what the
view above it says, and invisible to every assertion about either one
(both were correct on their own). `index.css` hides `.bottom-bar` while
`#main-content[data-active-view="now-playing"]`, through `:has()`
rather than a class toggled from `index.ts`: which view is showing is
already published as an attribute, and a second expression of the same
fact is a second thing to keep in step.
That took the queue button away with it, since that button lives in the
bar — so the view carries its own, toggling the same `open` attribute
on the same panel element.
**And a css`` literal cannot contain a backtick.** A comment reading
"the track size is set on the `wa-slider` inside its shadow root"
terminates the tagged template, and the failure arrives as
`Expected "]" but found "wa"` from the CSS parser, at a line number in
the *comment*. `make css-check` exists for this and named it
immediately.
## The index artifact could not be exported, and the reason is a rule this repo already had (2026-08-16)
`maintain-index` failed on an unrelated push:
```
indexexport: copy rows: SQL logic error: no such column: total_tracks (1)
```
Three minutes in, on the one job that owns the ~205 GB checkpoint and
publishes the catalog every user downloads.
**The cause is the exception that keeps that checkpoint alive.** The
index job's `/cache` is a real `YJ_HOME` that survives between runs, so
`explore_index` there is classified `Cache` and is deliberately *not*
dropped and recreated by `cmd/indexbuild`'s schema repair
(`staleschema.go`). A column added to the schema afterwards is
therefore simply absent from that database — and `total_tracks` was
added by the album-completeness work. The exporter selected it anyway.
**The fix is the rule the importer already follows.**
`artifactHasTotals()` exists precisely because "adding a column to the
importer's SELECT is how you break every artifact already published";
the mirror image — *reading* an index older than the binary — had no
such guard. `sourceColumns()` asks
`pragma_table_info('explore_index', 'main')` and selects a literal `0`
when the column is not there, which is what the column already means by
"the catalog does not say" and what the app already renders as unknown
rather than as incomplete. The destination keeps every column, so an
importer needs no second shape.
So the pattern generalises, and is worth stating once: **any query that
crosses a version boundary in either direction asks the schema rather
than trusting it.** There are now three of these — `artifactStoresText`
(encoding), `artifactHasTotals` (import), `sourceColumns` (export).
Two things about the test are worth keeping.
It reproduces the failure **symptom first**: with the fix removed it
fails with the CI message verbatim, `copy rows: SQL logic error: no
such column: total_tracks (1)`. That was checked, not assumed.
And its first version silently proved nothing. `oldColumns` was
`strings.Replace(catalogColumns, "total_tracks, ", "", 1)` — which
matches *nothing*, because the list is formatted across lines and the
name is followed by a newline rather than a space. So the "old" index
had every current column, the probe correctly said so, and the only
reason this was caught is that the assertion about the probe ran before
the assertion about the export. A fixture built by string surgery on a
formatted constant needs to be whitespace-independent; it filters the
list now.
## Long-press is one document listener, and the header row is a row (2026-08-17)
Plan 016 B2 phase 3. A phone has no right-click, and every context menu
in this app opens from a `contextmenu` event — six components' worth,
bound three different ways (delegated on a virtualizer, per row, per
card). `frontend/src/utils/long-press.ts` is one document-capture
listener installed once from `index.ts`: a touch that holds still for
500 ms dispatches a synthetic `contextmenu` at the touch point, and
**every existing handler runs unchanged**. No component opted in, and
none can forget to.
Four things it has to get right, and each is a way the obvious version
fails:
- **The target is `composedPath()[0]`, not `elementFromPoint`**, which
stops at the outermost shadow host. Every menu here is bound inside
one, so a host-targeted event reaches a delegated listener and no
per-row one.
- **A browser that fires its own must win.** Chromium already dispatches
`contextmenu` on long-press; WebKit and the WebView vary. One arriving
during the press cancels ours; one arriving after ours is swallowed at
document capture.
- **Ours is told from theirs by identity** (a `WeakSet`), not by
`isTrusted`. `isTrusted` would work in the app and is untestable — no
test can dispatch a trusted event — so the suppression path would have
been the one thing with no coverage.
- **The click ending the gesture is swallowed**, keyed on the gesture
(cleared by the next `pointerdown`) rather than a time window, or a
quick tap on the menu that just opened is eaten too.
**What cost the time was the assertion, not the code.** The e2e spec
pressed `[role="row"]` — which is the *column header*, and it is the
first one. The gesture fired correctly, the header correctly ignored it,
and the failure looked exactly like a menu that would not open. Found by
probing the running app (`playwright-cli eval`, dispatching the same
pointer events and logging what saw the `contextmenu`), which showed the
event reaching the row's own listener with no menu behind it — i.e. the
handler was refusing it, not missing it. `.track-row` is the selector.
Verified by execution: 8 component tests (real browser, real shadow
boundary, real timings) and 2 e2e specs against the running app, twice
in a row. Not verified: any of it under a real finger on a real
WebView — the pointer events are dispatched, because neither Desktop
Chrome nor Desktop Safari has touch and there is no device tier.
## The first device run: A4 works, and two things only a phone could say (2026-08-17)
The published v1.5.0 APK, on a real phone, owner-reported. **This is the
first runtime evidence any of the Android work has ever had** — A4
shipped entirely reasoned from source.
**What holds.** Playback survives the screen locking. The MediaSession
notification appears in the status pane *with album art* — which
answers, in one observation, four of the open questions from plan 016:
the foreground service starts, POST_NOTIFICATIONS was granted and the
notification is visible, the session is picked up, and **cover art
decoded from a `MANAGE_EXTERNAL_STORAGE` path by a service is
readable**. The last was the one nobody could argue from documentation.
**Two bugs, and neither is visible from any tier we have.**
*Back did not navigate back.* The scaffold's
`MainActivity.onBackPressed` asks `webView.canGoBack()` and finishes the
activity otherwise — and this app had never touched `history`, so that
was false at every depth and back quit from anywhere. The fix is in the
frontend, not in Java: a navigation is a `history` entry now
(`recordNavigation` in `index.ts`, same URL, the destination in the
entry's state) and `popstate` replays it with `_isBack`. The Java half
needs no change, because the mechanism it already uses is the one we
were failing to feed.
Two rules keep it honest. The **first** navigation replaces the launch
entry rather than pushing one, or every launch costs a back press before
the app will close. And the in-app back buttons go through
`history.back()` rather than popping a stack of their own — `navStack`
is **deleted**, not kept alongside, because two stacks is exactly how
the detail view's own button and the phone's gesture come to disagree
about how far back one press goes. `back-navigation.spec.ts` pins that
invariant.
*The transport was off screen.* **`targetSdk 35` is Android 15, which
lays every app out edge-to-edge**, ignores the deprecated
`statusBarColor`/`navigationBarColor` the theme still sets, and hands
the app a window the size of the screen. The WebView is `match_parent`,
so the page's bottom band — the transport, and on a phone the tab bar —
was drawn underneath the gesture bar. `applyWindowInsets()` pads the
container by `systemBars | displayCutout | ime` and returns the insets
rather than consuming them. The window background goes black to match
the app's own ramp, or the padding shows as a blue-grey band.
**Neither is findable in the browser tier, and that is the lesson worth
keeping**: a viewport has no system bars, so `phone-shell.spec.ts` at
390x844 renders a shell that fits perfectly while the device cuts 48dp
off the bottom — and `page.goBack()` was never called because nothing in
a desktop shell has a back gesture. The Android tier's own note says
failure there is invisible; this is the milder version, where the app
works and is simply wrong in ways only the platform can show you.
Verified by execution: the APK builds with the Java change; 3 e2e specs
cover the history behaviour, on Chromium locally and WebKit in CI.
Not verified: the insets themselves, which need the next APK on the
owner's phone. What to look for is one thing — the transport and the tab
bar clear of the gesture bar, and the header clear of the status bar.
## The phone is a Chrome 113 WebView, and that reframes everything (2026-08-17)
The device is reachable over adb now, so the tier can be *asked* rather
than reported on. `make android-inspect` + `make android-eval` are that:
a debug build (`applicationIdSuffix ".dev"`, so it installs **beside**
the release app rather than needing the uninstall that would take the
library with it) opens `webview_devtools_remote_<pid>`, and raw CDP over
Node's built-in WebSocket evaluates in the real page. **Playwright
cannot do this** — `connectOverCDP` calls `Browser.setDownloadBehavior`
and a WebView answers "Browser context management is not supported",
killing the connection before the first evaluate.
Measured on the device (Light Phone III, TLP301):
| fact | value |
| --- | --- |
| Android | 14, SDK 34 |
| screen | 1080x1240, density 408 |
| WebView viewport | **424 x 439 CSS px**, DPR 2.55 |
| WebView engine | **Chrome 113.0.5672.136** (mid-2023) |
**The first correction: the insets commit does not explain the report.**
Edge-to-edge is forced for apps *running on* Android 15, and this phone
is Android 14 — the screenshot shows the app correctly inset, with the
status bar and the gesture bar outside it. `applyWindowInsets()` is
right and stays (the next phone, or one OS update, is Android 15), but
it is **pre-emptive, not the fix for "the controls are off screen"**.
That was an inference from a version number, and the device disagreed.
**The second correction: the black `fill` proves nothing.** A wa-icon on
the device has the right `color` (#ffd43b) and an `<svg>` in its shadow
root, and `getComputedStyle(svg).fill` is black — but that is the *svg
root*, and every vendored Font Awesome path carries
`fill="currentColor"` itself, so the root's fill is irrelevant. Measuring
the wrong node produced a diagnosis-shaped result. `__yjIconMisses` is
empty, so no name is unbundled either. Why the icons do not appear in the
screenshot is **still open**.
**What the engine version does explain, and what to check next.**
Chrome 113 has `:has()`, `color-mix()` and `dialog.showModal()`, and
lacks three things this app's dependencies use:
- **Relaxed CSS nesting** (Chrome 120): a nested rule starting with a
bare element selector is dropped. `.x { svg { ... } }` parses to
nothing; `.x { & svg { ... } }` parses. Any Web Awesome or app
stylesheet written the modern way silently loses declarations here,
and dropped declarations are exactly the failure that looks like
"rendered but wrong".
- **The Popover API** (Chrome 114). Web Awesome's popup calls
`showPopover?.()` — optional, so nothing throws — but also sets
`popover="manual"`, which on 113 is an unknown attribute doing
nothing. Every context menu, dropdown and the whole menu keyboard
model rides on that, so it is the first thing to test with a library
present.
- `light-dark()` and relative colour syntax (`rgb(from ...)`).
**The lesson for the tier: a device is an engine, not just a screen.**
Every browser tier here runs a current Chromium or WebKit, and the phone
that will actually run this app is two years behind — so "it renders at
424x439 in Chromium" (checked, the transport is on screen) says nothing
about whether it renders on the phone. The e2e tier cannot be fixed by
resizing; the missing signal is version, and CDP against the device is
the only place to get it.
Verified by execution: every number in the table, the four feature
probes, and that the hardware back button no longer kills the app (the
`.dev` build carries the history fix; pid survived a BACK press).
Unverified: what happened to the icons and the transport controls, which
is where this resumes.
## What the device actually said, with both builds side by side (2026-08-17)
The phone inspectable and awake, the same Light Phone III running two
builds of this app in turn. This closes both questions the previous entry
left open, and **neither answer was the one the symptom suggested**.
**"The playback controls are off screen" was true, literal, and already
fixed.** The installed build is from B2 **phase 1** — it carries
`bottom-nav` and no `now-playing-view`, which dates it between 57bfbdf
and 1b05dde. Settled (30 s after launch, not 6), its player bar shows
art, title, favourite, shuffle, prev — and stops. Play/pause, next,
repeat and queue are past the right edge, because at 424 px the bar was
still carrying the seek bar and volume that **phase 2 moved into
`now-playing-view`**. On the current build, on the same phone and the
same engine, `document.body.scrollWidth` equals `clientWidth` (424) and
`player-controls` measures 200..380 inside 424. So the fix was already
on main, unreleased, and the device is what proved it rather than
argued it.
**"No icons" was an artefact of my own screenshot.** A `wa-icon` on the
device has `path` computed fill `rgb(255,212,59)` and paints; the first
capture was six seconds after a cold start, before the icon fetches had
landed. Two corrections in two entries from the same misreading: measure
the node that paints, and let the app settle before believing a picture.
**Chrome 113's missing Popover API does not break the menus.** This was
the leading worry and it is unfounded: a long-press on a row opens the
real panel at (212,145), 162x193, `visibility: visible`, seven
`role=menuitem`s, all seven inside the panel and clear of the player bar
— confirmed by screenshot as well as by measurement. Web Awesome's
`showPopover?.()` is an optional call and `wa-popup` positions itself,
so the attribute being inert costs nothing. **Long-press itself works on
real hardware**, over a real 1,744-track library, which is the phase 3
verification the browser tier could only approximate.
**The one genuine fault the device adds is phase 4's.** `track-list` at
424 px computes `--grid-cols: 24px 102px 101px 101px 80px` — which fits
the host exactly, so nothing overflows — but "Duration" does not fit in
80 px and neither does most content. The columns are not too wide; there
are simply too many of them for a phone, which is what phase 4 already
says. It is now a measurement rather than a prediction.
Two operational notes. The debug sibling scanned the phone's real music
and its data directory is **414 MB**, so it is worth uninstalling when
done (`adb uninstall app.yellowjacket.dev` — the sibling id is exactly
what makes that safe). And `am start` does not reliably take focus while
another app is foreground: check `topResumedActivity` before trusting a
screenshot, or you will read someone else's app.
## The phone track list, and the bug a viewport could not have found (2026-08-17)
B2 phase 4. A phone draws `titleArtist` — the title with the artist
under it — plus the duration, and drops the column headers and the
resize handles. It is a **column set, not a second row template**: the
row, its delegated events, the selection semantics, the playing marker
and the virtualizer never learn that anything changed, because from
their side only the number of columns did.
Three rules, each one a way it breaks otherwise. The row height is in
two places (`PHONE_ROW_HEIGHT` and the CSS) and they must agree, since
the virtualizer positions rows from that number. What is *drawn* and
what can be *sorted* are separate questions — the sort list is built
from `configuredColumns`, or a phone with no headers could sort by
nothing but title and duration. And a phone's widths are neither loaded
nor saved.
**That last one is the finding, and it came from the device.** With the
arrangement passing five component tests and five e2e specs at
424x439, the phone showed `24px 148px 236px`: the duration column with
55% of the row. `loadColumnWidths` is keyed by column *id* and fills a
gap with `MIN_COLUMN_WIDTH`, so the stacked column — which nothing can
ever have saved a width for, there being no handles to drag — came out
at the minimum while `trackLength` inherited a width saved for a
four-column desktop row. The mirror image is worse and was never
reachable from a phone at all: `saveColumnWidths` would have written the
computed phone widths back under the same ids, replacing the width the
user dragged on a desktop.
**Why every browser test missed it.** The specs assert the *shape* — how
many grid tracks, no header, no overflow, the title's share of the row —
and the width bug depends on what is in `localStorage` for a *different*
column set. dev-headless's seed happened to hold widths that split the
other way, so the same assertion passed in the browser and failed on the
phone. The unit test now carries the desktop map as a fixture, which is
the reproduction the browser needed to have.
**Confirmed on the phone afterwards**, with the fix installed:
`24px 304px 80px`, 52 px rows, no header row, the title 298 px and not
truncated, `body.scrollWidth == clientWidth`. The same numbers the
browser gives at that viewport, which is the point of having measured
both.
Two tooling notes worth keeping. `playwright-cli` holds its page across
a `make dev-headless` restart, so a probe after a rebuild can be
answering for the *old* bundle — it reported the desktop layout at 424 px
until the page was reopened. And wireless adb dropped twice more mid-
session when the screen slept; USB for anything longer than a few
probes.
## The catalog download now asks about the connection (2026-08-17)
Plan 016 B4. ~0.6 GB had no network awareness at all; it is skipped on a
cellular connection unless the user says otherwise
(`AllowMeteredCatalogDownload`, default false, toggle in Settings' Search
Index section).
**The shape is dictated by the cgo rule, not by taste.** `explore` is
imported by `cmd/indexbuild`, which builds with `CGO_ENABLED=0` and must
not link Wails, so `netpolicy.go` holds the policy and the JSON parsing —
tested on every platform — while the one platform call is a closure
injected from `app.go`, where naming `application` is already legitimate.
Four things measured or corrected in the doing:
- **The portable name is `application.Mobile`, not `application.Android`**
(which the plan and `CLAUDE.md` both named). `Android` exists only
under the `android` build tag; `Mobile`'s desktop implementation is a
stub whose `NetworkJSON()` returns `""`.
- **The runtime reports no metered flag.** `{"connected":bool,
"type":"wifi|cellular|ethernet|none"}` is all there is, so cellular is
the signal and a metered *Wi-Fi* — a phone hotspot, a hotel — cannot be
detected. Android itself knows (`NET_CAPABILITY_NOT_METERED`) and the
runtime does not pass it on. Documented gap, not an oversight.
- **An unknown answer must not read as metered.** Every desktop answers
`""`, so the obvious defensive default would have disabled the catalog
download for every desktop user in the world.
- **The gate belongs before the first status write.** Declining is a
no-op — no job in the indicator, no error tier to dismiss — which is
what makes the refusal safe to have on by default.
## The stale-shape repair dropped the CI catalog (2026-08-17)
Not our change, but it is the operational state everything else now runs
in, and the restore condition needs to be written down somewhere that is
not a commit message.
`fix(database): retire a table whose shape the schema moved past` added
`staleshape.go`: before `applySchema`, drop any non-Authored table whose
live shape disagrees with the schema. That is the right rule for an
install — a client's catalog is *downloaded*, so a stale one costs a
minute of re-fetching the artifact, and keeping it costs every Explore
read.
It runs inside `database.NewDB`, which `cmd/indexbuild` also calls. On
the first run after it landed, 19 seconds in:
```
16:15:51 retiring a table ... table=explore_index
reason="column entity_type is TEXT, schema declares INTEGER"
16:16:05 index maintenance mode=build reason="no completed import yet"
lastImported=never baselineSeries=0
```
**The premise was false for the one database where it was expensive.**
That catalog is not stale; it is deliberately kept in the older text
encoding, which `artifactStoresText` and `sourceColumns` exist to
tolerate — so it would have been judged stale and dropped on *every*
run. And `retireLibraryTables`, in the same package, already documents
the opposite rule for this database: drop everything the datamap does
**not** call Cache.
`fix(database): never retire the catalog the index build derives` makes
the policy a build tag (`retireStaleCache`, false under `indexbuild`),
which is how this project already separates the index tools. It prevents
recurrence and cannot undo the drop: that volume was the only copy.
**What it cost, and the shape of the cost.** A full re-import from the
MetaBrainz dumps, resumed across runs from a checkpoint, at a rate that
swung between 2 and 15 MB/s. The job runs on **every push to main** with
a 3 h budget on a runner of capacity 1 — so until the import completes,
every push books three hours and ordinary CI queues behind it. That is
the real damage: not one lost job, but a repeating one.
So the `push:` trigger in `index-artifact.yml` is **commented out**
until a run reports `complete=true`; the weekly cron and
`workflow_dispatch` still resume the build, which is all it needs.
Restoring those two lines is the whole revert.
Three things worth keeping from it:
- **A repair belongs where its assumptions hold.** `NewDB` is the one
chokepoint every binary in this project shares, including the one
whose database cannot be re-derived cheaply. Anything destructive
there needs to ask which binary it is in — the build tag was available
and is what the fix used.
- **The only copy of a 205 GB derived asset is one Docker volume.**
There is no snapshot, so the restore time is "however long
MetaBrainz takes today". A periodic copy would turn this class of
incident into twenty minutes.
- **The fix's residual trade is now the thing to watch**: with Cache
tables never retired under `indexbuild`, a future `explore_index`
column fails that job loudly at build time instead of silently
rebuilding. That is the right default, and it means the next schema
change touching `explore_index` needs a deliberate plan for this one
database rather than none.
## Two guards for the index cache, and what each one is worth (2026-08-17)
Both come out of the incident above, and they protect different halves
of it.
**`TestNoCacheTableIsRetiredHere` asserts the outcome, not the
mechanism.** The test that shipped with the fix pins one table in one
wrong shape, which is the failure that happened; what actually cost the
rebuild was a destructive repair added at `database.NewDB` — the
chokepoint every binary here shares — without asking which binary it was
in. The next one will have a different name and a different reason. So
this puts *every* `datamap` Cache table into a shape the schema has
moved past, opens the database the way `cmd/indexbuild` does, and
requires all of them to still be there.
Three things it got right by being written this way. The table list is
`datamap.ByKind(Cache)`, so the two credit tables added the same day
were covered without anyone adding them — flipping the policy back fails
on **five** tables including `artist_credit_part` and
`artist_credit_ref`, where the single-table test fails on one. It
asserts rows survive as well as the table, because SQLite does an
implicit DELETE before a DROP and a repair that recreated the table
would otherwise look identical. And it *accepts* an error from `NewDB`,
because that is the trade the fix documents: loud failure instead of a
silent day of downloading.
**`scripts/index-cache-snapshot.sh` covers the half no test can.** The
volume held the only copy of a catalog whose rebuild is hours of someone
else's bandwidth. `VACUUM INTO` rather than `cp`, because a byte copy of
a live SQLite file is a corrupt file of plausible size; the staging
directory is deliberately not copied, since a build resumes without it;
and the snapshot is reopened and asked for its catalog row count before
any rotation happens. Both failure paths were exercised rather than
argued: a corrupt source and an empty catalog each exit non-zero, delete
their own output, and leave the previous snapshots in place.
`docs/index-cache.md` is the restore procedure, and the number that
makes it worth having: a restored snapshot resolves to `refresh` and
folds in the incremental listens since — minutes, against the 323 h a
rebuild was estimating.
@@ -0,0 +1,337 @@
# 015 — Multi-artist credits, navigable
## The problem
A track credited to more than one artist has exactly one navigable
artist in this app, and the others are punctuation.
`audio_files` carries `artist_credit` (the credit as tagged, for
display) and `artist_id` (one artist, for grouping and browsing).
`primaryArtist()` (`backend/library/artistcredit.go:53`) resolves that
one artist by *string-parsing* the credit: it strips a " feat. "
clause, and deliberately does not split on `&`, `x`, `with` or `,`
because those appear inside real artist names. So "Lana Del Rey ft.
Sean Lennon" stores Lana Del Rey and discards Sean Lennon entirely,
and "Alina Baraz & Galimatias" stores one artist whose name is the
whole credit.
### What the measurement says
Measured 2026-08-16 against a real 26,069-file library (19,840 mp3,
6,229 flac; 57 unreadable, m4a/ogg not examined), plus an 80+80
MusicBrainz `inc=artist-credits` sample.
- **13%** of a random sample of the library's recordings have more
than one credited artist in MusicBrainz (10 of 79 resolved).
Extrapolates to ~3,250 of the 24,989 files carrying a recording
MBID.
- **0.86%** of files (224) carry any structured multi-artist signal in
their own tags. mp3 carries **zero** files with multiple
`MUSICBRAINZ_ARTISTID` values across 19,840 files; flac has 87.
- **1,286** files say "feat." in `ARTIST`; **1,159 of them (90%)**
have nothing structured behind it. A sample of 80 such files was
multi-artist in MB **80 of 80 times**.
CLAUDE.md currently justifies plan 013's removal of `artist_credit` /
`artist_credit_artist` with "3 credits of 2,823 listed more than one
artist". That figure measured **our own writer**, not the library:
`cachedLinkArtist` was called exactly once per credit
(`e7748f1^:backend/library/library.go:1842`), so a collaboration could
never have been recorded, and the three were resolution collisions on
shared credit text. Dropping the join table was still correct — it only
ever held one row, so it was pure join cost — but the stated evidence
does not support "multi-artist is rare". Correcting that claim is part
of this plan.
### Why the tags cannot answer it
Deriving the decomposition locally, with no network, works **79% of the
time** (169 of 215 files with a multi-value `ARTISTS` tag: mp3 69/105,
flac 100/110), and the failures are systematic rather than random:
```
ARTIST = '2Pac feat. Snoop Dogg, Nate Dogg, Hussein Fatal & Yaki Kadafi'
ARTISTS = ['2Pac', 'Snoop Doggy Dogg', 'Nate Dogg', 'Fatal', 'Yaki Kadafi']
```
`ARTISTS` holds **canonical** artist names; `ARTIST` holds
**as-credited** names. Locating one inside the other fails on
"Snoop Doggy Dogg" vs "Snoop Dogg", on "Fatal" vs "Hussein Fatal", and
on Unicode (`Michel'le` vs `Michelle`, `K-Ci` vs `KCi` — U+2010, not
a hyphen). That distinction is precisely what a join phrase encodes,
and it is why this cannot be a tag-parsing feature.
Two format details that will mislead anyone re-running the probe:
Picard writes `ARTISTS` **slash-joined into one TXXX frame** on mp3 and
as **true repeated Vorbis keys** on flac, so a probe splitting only on
NUL undercounts mp3 to zero.
## The shape
MusicBrainz models a credit as ordered parts, and the credit *string*
is derived from them — `artist_credit.name` is a cached render, nothing
more. Each participant is `(position, artist, name, join_phrase)`,
where `artist` is the MBID (canonical, what you navigate to) and `name`
is the credited spelling (what you display).
**Join phrases are assembly instructions, not disassembly
instructions.** Rendering is a concatenation, never a search:
```
for each (position, artist_mbid, credited_name, join_phrase):
emit link(credited_name -> artist_mbid)
emit text(join_phrase)
```
The link positions are known **by construction**. This is load-bearing:
if we instead located each `credited_name` inside the stored
`artist_credit` text, we would reintroduce the mismatch above — the
stored string may have come from the tags while the parts come from the
catalog, and those **disagree for ~1 in 3 multi-artist files** (61 of
90 sampled credits rendered exactly equal to the tag string).
Divergences seen: `'Skrillex feat. Swae Lee'` tagged vs
`'Skrillex & Swae Lee'` in MB; `'STRFKR'` vs `'Starfucker'`;
`'Zedd feat. Hayley Williams'` vs `'... of Paramore'`. Either MB was
edited after tagging or Picard versions differ; either way the search
would miss or match the wrong span.
So `audio_files.artist_credit` stops being the source of truth and
becomes the **fallback**, used only where there are no parts.
## Where the data comes from
The catalog carries the decomposition; no user ever makes a
per-recording call. Two sources were ruled out first, both cheaply:
- **The canonical dump — which is what CI already pulls
(`dumpimport.go:84-85`) — does not have it.**
`canonical_musicbrainz_data.csv` gives `artist_mbids` (ordered list)
and `artist_credit_name`, but that last column is the *rendered*
string. Splitting it on CI needs the as-credited names, so CI would
fail exactly the way a local parse does.
- **The JSON dumps do not cover the catalog.**
`json-dumps/recording.tar.xz` is 31 MB / 368 MB uncompressed and
holds **153,691 recordings**, not ~35M. Measured against the test
library's 24,885 recording MBIDs: **0.00% overlap, zero rows**. It is
some other subset and is not usable.
That leaves the core dump, **`mbdump.tar.bz2`** (7.1 GB compressed at
the 20260815 export), from
`https://data.metabrainz.org/pub/musicbrainz/data/fullexport/`. Four
members are needed:
| member | why | approx rows |
| --- | --- | --- |
| `mbdump/artist_credit_name` | `(artist_credit, position, artist, name, join_phrase)` — the payload | ~4M |
| `mbdump/artist` | `id -> gid`, since the above references artist *row ids* | ~2.6M |
| `mbdump/recording` | `gid -> artist_credit`, to key credits by recording MBID | ~35M |
| `mbdump/release_group` | same, for album credits | ~2M |
### Coverage is not a concern
Of 24,885 distinct recording MBIDs in the test library, **24,808
(99.7%)** already have an `explore_index` recording row, measured
against a database at 2,052,200 rows — i.e. shipped-artifact coverage,
not a local build's. The popularity filter does not strand the long
tail here.
## Status
- **Phase 1 — done.** `backend/explore/dumpcredits.go` +
`dumpcreditswrite.go`, wired into `dumpimport.go`'s `run` behind its
own `credits_import_done` marker.
- **Phase 2 — done.** `cmd/indexexport` writes the two tables;
`artifactimport.go` reads them behind `artifactHasCredits()`.
- **Phase 4 — done, and it does not need Phase 3.** `explore.GetCredits`
reads the catalog tables keyed on the *recording* MBID, which both
sides of the app already carry — a catalog row has one and so does a
local file (`library.Track.RecordingMBID`). So one binding serves the
Explore pages and the library's own lists, and all ten artist-link
call sites render credits today without a local table.
- **Phase 3 (`file_artists`) — not started, and now an
offline-resilience task rather than a prerequisite.** The table is
deliberately *not* declared yet: nothing writes or reads it, and a
schema file plus a datamap note describing behaviour that does not
exist is a claim the code cannot back. Its remaining
value is that credits currently vanish when the catalog is absent or
still downloading, which is precisely the `no-index` state
`ShelfPage.State` exists to describe. Materialising into
`file_artists` is what makes a library stand on its own.
**Nothing renders yet in practice**, because no published artifact
carries credit tables — every credit falls back to its single link
until an index build with Phase 1 runs and is exported.
**Column layouts are verified against the real 20260815 export**, not
taken from the schema docs — `artist(id, gid, …)`,
`artist_credit(id, name, artist_count, …)`,
`artist_credit_name(credit, position, artist, name, join_phrase)` and
`recording(id, gid, name, artist_credit, …)` were each read out of the
dump. `release_group` shares `recording`'s first four columns and is
the one layout still taken on trust; `ErrDumpShape` turns a wrong guess
into a loud failure rather than a quietly wrong catalog.
**Still unrun: the ingest against the real 7.1 GB dump.** Everything is
covered by tests over a synthetic tar, which cannot catch a surprise in
the other ~35M rows.
### Phase 1 — Ingest credits on CI
New dump stage in `cmd/indexbuild`, behind the `indexbuild` tag with
the rest of `dumpimport.go`'s stages.
**Constraint from `b98840e`:** `cmd/indexbuild` is built
`CGO_ENABLED=0` in a plain `golang` container and must not reach the
Wails `application` package — `TestIndexToolsDoNotImportWails` walks
`go list -deps -tags indexbuild`. Nothing here should need it, but a
new `ServiceStartup` hook on a package this imports is how it comes
back. Go's `compress/bzip2` is pure Go and decompress-only, which is
all this needs.
**Measured, 20260815 export.** Tar members are **alphabetical**, and
that is favourable: `artist` (435 MB), `artist_credit` (414 MB) and
`artist_credit_name` (237 MB) all fall inside the first ~900 MB
compressed, while `recording` and `release_group` come later. So the
maps are complete before the rows that consume them arrive, and no
recording data is ever buffered.
Pure-Go `compress/bzip2` decompresses at **26 MB/s uncompressed /
8.7 MB/s compressed** (measured on a 250 MB prefix, 3.01x ratio) —
**~13.7 min** for the whole file single-threaded, and less because the
stream can stop after `release_group` rather than reading the
`series`/`tag`/`track`/`url`/`work` tail. The 2 MB/s origin throttle
dominates, as it already does for every other dump here.
Do not, however, *depend* on the ordering: assert it and fall back to
buffering if a future export reorders, rather than silently emitting
nothing.
- `artist` -> `map[int32]uuid16` (~2.6M x ~20 B = ~60 MB)
- `artist_credit_name` -> `map[int32][]creditPart` (~4M x ~40 B =
~200 MB)
- `recording` / `release_group` -> emit `gid -> credit_id` **only for
MBIDs already in `explore_index`** (the kept set is ~1.4M x 16 B =
~22 MB), which is what keeps 35M rows from being held
Peak ~300 MB, one sequential pass.
**Only multi-artist credits are stored.** A single-artist credit is
`(name, "")` and is already fully described by `explore_index`'s
`artist_name` / `artist_mbid`; storing it would triple the table for
nothing. Post-filter after loading, once the row count per credit is
known.
New tables (and `datamap` entries, or `TestCatalogCoversSchema` fails
the build — both are `Cache`, matching `explore_index`):
```
artist_credit_part(credit_id, position, artist_mbid, credited_name, join_phrase)
```
with `explore_index.artist_credit_id` as the link. Credits are
**shared** — an album's twelve tracks by one artist share one credit
row — which is the opposite of 013's local verdict, and correctly so:
1:1 in a local library, genuinely many-to-one at 2M-row catalog scale.
### Phase 2 — Ship them in the artifact
`cmd/indexexport` currently creates exactly two tables in the artifact
(`explore_index`, `artifact_meta`, at `cmd/indexexport/*.go:147,170`),
so this is a structural addition, not a column.
Estimated size: ~13% of 1.4M recordings, deduplicated by shared credit,
at ~2.3 parts each — order 400k rows, ~18 MB uncompressed. Against a
~0.6 GB install that is acceptable; it must be measured rather than
assumed before merge.
`artifactimport.go` must read it **only if present**, on the writer
handle where `core` is attached — the `artifactHasTotals()` /
`artifactStoresText()` pattern (`artifactimport.go:145-175`), one step
up from a column to a table. An artifact published before this exists
is still a perfectly good catalog and must import as one that declines
to answer. Adding this to the importer's SELECT list without the probe
is how every already-published artifact starts failing.
`artifactCatalogColumns` gains `artist_credit_id`; it is kept in sync
with the exporter by `TestArtifactColumnsMatchExporter`.
### Phase 3 — Materialize locally
```
file_artists(audio_file_id, position, artist_id, credited_name, join_phrase)
```
`credited_name` is stored **per row**, not looked up from
`artists.name` — that is the Snoop-Doggy-Dogg distinction, and it is
the whole point.
Filled at scan/import time by joining `audio_files.recording_mbid`
against the catalog. **Materialized rather than resolved live**,
because the catalog is a downloaded artifact that can be absent or
still arriving — that is why `ShelfPage.State` has a `no-index` value —
and a library whose track rows lose their artists when the catalog is
missing is worse than today.
That implies a backfill for the case where the catalog arrives *after*
the library was scanned. It registers with `jobs` (progress, cancel)
like every other long pass, and takes a **distinct kind** from
`index-build`, since `job-controls.ts` keys its "you will discard hours
of downloading" confirmation on that kind.
`artists` gains rows for guests who own no files. **This changes what
the artists grid shows** and is an open question below.
### Phase 4 — Render
`utils/explore-link.ts` gains a credit-rendering entry point taking
ordered parts and returning a `TemplateResult`. Every row and detail
view already renders artist names through it, so they inherit
multi-artist links without individually knowing credits exist — the
property that made centralising it worthwhile.
Its existing fallback philosophy already covers the no-parts case: "a
list where some rows are clickable and others silently are not reads as
a bug, not as a statement about metadata." Where there are no parts
(no recording MBID, or no catalog row — ~4% of the test library) render
today's behaviour: the flat `artist_credit` string with one link to the
primary artist. **Do not split the string there.** There is genuinely
no information to split on, and that is the one place the temptation
returns.
`primaryArtist()` stays exactly as it is. It remains the fallback and
is still what `artist_id` means.
## Open questions
1. **Catalog credit vs tagged credit, when they disagree** (~1 in 3
multi-artist files). Rendering the catalog's decomposition is what
makes names navigable; preserving the file's is what makes the app
reflect the user's files. Leaning toward: render the catalog
decomposition, keep `artist_credit` as the fallback string. Wants a
deliberate decision, not an accident.
2. **Do guest artists appear in the artists grid?** Phase 3 creates
`artists` rows for people who own no files. The grid currently means
"artists in your library" and joins `audio_files`. A guest on one
track is arguably in the library and arguably not. Whichever way,
the ownership question stays "is there a file" — that rule does not
bend.
3. **`release_group` credits** are ingested in the same pass for
nearly nothing, but album-artist rendering is a separate surface.
Ship the data in phase 1, render in a follow-up rather than widening
phase 4.
4. **Our own `tagwriter`** does not write `ARTISTS` or multiple
`MUSICBRAINZ_ARTISTID` frames, so autotagging a folder degrades the
very field this rests on — the same shape as the existing
track-totals note. Out of scope here; worth recording.
## Verification
- Coverage: re-run the library probe and assert `file_artists` is
populated for ~13% of files, not ~0.9%.
- `TestCatalogCoversSchema` / `TestLifetimesMatchSchema` for the new
tables.
- `TestIndexToolsDoNotImportWails` still passes with the new stage.
- An artifact **without** the credits table imports cleanly (the
`artifactHasTotals` regression shape).
- Round-trip: a known multi-artist recording renders each name as a
separate link with the correct join phrases between them.
@@ -246,6 +246,13 @@ view's template is two templates to fix every bug in. Where a view
cannot serve both, the split belongs at the chunk boundary that already
exists.
Phase 1 followed that rule and found its cost: reusing `<app-sidebar>`
inside the drawer means reusing its `data-testid`s too, and a second
copy standing by in the DOM broke 30 specs that had nothing to do with
the phone. The rule holds — a second list of destinations would be
worse — but a shared component must be rendered only when it is wanted,
and the guard belongs in a test that names the reason.
## What is worth doing regardless of that decision
Cheap, independently useful, and each unblocks measurement:
@@ -308,12 +315,84 @@ places had to agree — `abiFilters`, the Makefile's `android:package`
anchor is what stops it also matching the fat APK's line. Adding the
ABI back, if modernc ever fixes `Xlstat64`, is those same three edits.
**B2, the desktop shell.** The largest remaining piece, and the scope
is now decided — see "The phone gets a subset" below.
**B2, the desktop shell.** Scope decided (below); **all four phases are
done.**
- *Phase 1, the shell.* Below 600px the sidebar column is gone,
`<bottom-nav>` is the primary navigation, and the shell fits 320px
exactly — measured, from 652px in a 360px viewport before.
- *Phase 2, the full-screen now-playing view.* Where phase 1's seek bar
and volume went. A detail view, so Back pops the nav stack; it
composes the real transport components rather than copying them; and
it hides the bottom bar while it is up, so it carries its own queue
button.
- *Phase 3, long-press.* `utils/long-press.ts`: one document-capture
listener, installed once from `index.ts`, which turns a 500 ms
stationary touch into a synthetic `contextmenu` at the touch point.
Every menu in the app opens from that event, so all six components
gained the gesture without one of them changing — which is the same
argument `ContextMenuController` rests on, one layer lower. The
details that are not obvious are in `NOTES.md` (2026-08-17); the one
worth repeating is that ours is told from the browser's own
long-press event by **identity**, not `isTrusted`, because a test
cannot dispatch a trusted event and that path would otherwise be the
only uncovered one.
- *Phase 4, the track list.* A phone draws `titleArtist` (title over
artist) plus the duration, and drops the column headers and the resize
handles — a column set rather than a second row template, so the row
and everything delegated on it is unchanged. Verified at the device's
own 424x439: `24px 304px 80px`, 52 px rows, no truncation, no
overflow. The device also found the bug in it, which no browser
viewport would have: saved *desktop* column widths reached the phone
through an id-keyed store and gave the duration column 55% of the row.
**B2 and B4 are complete.** B4 is `backend/explore/netpolicy.go`: the
catalog download is skipped on a cellular connection unless
`AllowMeteredCatalogDownload` is on, with the toggle in Settings' Search
Index section. The policy and the JSON parsing are in `explore` (tested
on every platform) and only the platform call is injected from `app.go`,
because `cmd/indexbuild` imports `explore` and must not link Wails. Two
things the plan got slightly wrong: the portable API is
`application.Mobile.NetworkJSON()` rather than `Android`'s, and it
reports no metered flag — so cellular is the signal and a metered Wi-Fi
cannot be seen.
What is left in this plan is B3 (tag writing, which needs a device) and
the standing question of the Light Phone's Chrome 113 — which so far has
cost nothing: menus, dialogs and long-press all work on it.
**B3/B4** are unchanged, and B3 is now *possible* where it was not:
with all-files access, `tagwriter` can write in place.
### What the first device run answered (2026-08-17)
A4 **works**: playback survives the screen locking, and the transport
notification appears with cover art — which also settles the service's
access to a `MANAGE_EXTERNAL_STORAGE` path, the permission grant and
the lock-screen session in one observation. Everything below in "what
none of section A answered" was written before this and is now answered
except the OEM permission-flow variance.
It also found two faults no browser tier can see, both fixed and both
awaiting the next APK for confirmation (`NOTES.md`, same date):
- **Back quit the app from any depth.** The scaffold asks
`webView.canGoBack()`; the frontend had never used `history`. A
navigation is a history entry now, and `navStack` is gone rather than
kept beside it.
- **The transport was under the gesture bar** — or so the version
number said. `applyWindowInsets()` in `MainActivity` is right and
stays, but the phone is **Android 14**, where the system still insets
the window: the fix is pre-emptive and the symptom has another cause.
Still open, along with icons that do not appear at all. The phone's
WebView is **Chrome 113**, which is the lead (no Popover API, no
relaxed CSS nesting), and `make android-inspect` / `android-eval` are
how it gets asked.
The standing item is unchanged in kind: **B3 (tag writing) and the
permission flow still need a device**, and so does confirming these two.
### What none of section A answered
Nothing here has been observed on a device. The permission flow in
Symlink
+1
View File
@@ -0,0 +1 @@
CLAUDE.md
+286
View File
@@ -233,6 +233,74 @@ rather than renaming them.
the drift it caused before — `sql/schemas/` and the migrations
disagreed, and sqlc generated against the stale one.
**What that costs an existing database is repaired once, at open.**
`CREATE ... IF NOT EXISTS` reaches an existing table only if its shape
already matches and otherwise silently no-ops, so a *changed* table
never migrates. Plan 014 added `total_tracks` to `explore_index` and
to `indexRowFields` — the projection every explore read uses — and no
database that already existed grew the column: **every** Explore
search, browse, artist and album page on such an install failed with
`no such column: total_tracks`, while a fresh install was perfectly
healthy, which is exactly why no test saw it. Plan 013 was worse on
the same install: `applySchema` could not be applied at all over a
pre-013 `audio_files`, so the app did not open.
`backend/database/staleshape.go` runs before `applySchema` and
retires what is stale, so the create is a create. Five things about
it are load-bearing:
- **It parses `sql/schemas/` for the expectation** rather than
writing the column list down a second time, because a second list
is a second thing to forget — the fault it exists to repair.
- **It notices a changed *type*, not just a missing column.** 013
moved `mbid` from TEXT to BLOB, and SQLite does not coerce between
them: a comparison against 16 raw bytes returns no rows rather than
an error. `ALTER TABLE ADD COLUMN` would have handled
`total_tracks` alone and cannot express this at all, which is why
the repair drops rather than migrates.
- **`Authored` is never retired**, and that boundary is a test
(`TestAuthoredTablesAreNeverRetired`), not a comment. Everything
else is rebuildable: `Cache` by definition, `Owned` by a rescan —
plan 013's stated "delete and rescan" — and `Derived` from Owned.
A table the schema no longer describes at all goes too; 013 left
seven behind plus `schema_migrations`.
- **Whether a stale `Cache` table may be rebuilt is a build tag**, and
it is the most expensive thing in this file to get wrong. In the app
the catalog is *downloaded*, so a wrong shape costs a minute of
re-fetching the artifact and keeping it costs every Explore read. In
`cmd/indexbuild` the catalog is *derived*, and the only way back is
the ~205 GB dump stream the `/cache` volume exists to avoid — so
`retireStaleCache` is false there (`staleshape_policy_indexbuild.go`)
and `TestTheCatalogSurvivesAStaleShape` fails the moment it is not.
`TestNoCacheTableIsRetiredHere` is the same assertion made of *every*
`datamap` Cache table rather than one, because the risk is not that
shape recurring — it is the next destructive repair added to
`database.NewDB`, the chokepoint every binary here shares, without
asking which binary it is in.
This is written down because it already happened: the repair shipped
without the distinction and dropped the real CI catalog on its first
run, with `reason="column entity_type is TEXT, schema declares
INTEGER"`. The mismatch was genuine — that database is deliberately
kept in the older encoding, which `fix(indexexport): read an index
older than the binary` exists to tolerate — so it would have been
dropped on *every* run. The consequence is that a future
`explore_index` column fails the index job loudly on `applySchema`
rather than silently costing it a rebuild, which is the trade a
human should get to make.
- **The drops are one transaction with `defer_foreign_keys`.** Those
legacy tables reference each other, so dropping them in any order
fails on whichever goes first, and turning foreign keys *off*
instead would silently take `playlist_tracks.audio_file_id`'s
ON DELETE SET NULL with it — leaving playlist entries pointing at
ids a rescan reissues to *different songs*. Nulled entries are
empty; stale ones are wrong, and wrong quietly.
- **The order is sorted, so a failure reproduces.** Map order is
random, and the foreign-key bug above passed its own regression
test on two runs in three until the order was fixed.
Retiring `explore_index` takes its FTS and its meta with it, because
the `dump_import_done` marker is what would otherwise stop the
artifact ever being fetched again.
**What that costs an existing database is that it does not open**, and
"delete and rescan" is the answer (plan 013, open question 1) — free
for everyone except one machine. The index job's `/cache` volume is a
@@ -508,6 +576,81 @@ selected as a literal `0`. Adding the column to the importer's SELECT
list without that is how a published artifact — which nobody can re-cut
retroactively — starts failing with `no such column`.
**A credit is ordered parts, and the string is derived from them.** A
track credited to several artists had exactly one navigable artist and
the rest were punctuation: `primaryArtist()` string-parses the credit,
strips a " feat. " clause and discards the guest, and deliberately does
not split on `&`, `with` or `,` because those live inside real artist
names ("Simon & Garfunkel"). Measured on a real 26,069-file library,
**13%** of recordings are multi-artist upstream while only **0.86%** of
files carry a structured multi-artist tag — mp3 carries *zero* files
with multiple `MUSICBRAINZ_ARTISTID` across 19,840 — so this cannot be
a tag-parsing feature. (The "3 credits of 2,823" figure that justified
plan 013's removal of the credit tables measured our own *writer*:
`cachedLinkArtist` ran once per credit, so a collaboration could never
have been recorded. Dropping the join table was still right on cost.)
`artist_credit_part` / `artist_credit_ref` carry the decomposition for
multi-artist credits only — a single-artist credit is already
`explore_index`'s own `artist_name`, and storing those would triple the
table to say nothing. Five things about it are load-bearing:
- **Join phrases are assembly instructions, not disassembly ones.**
`creditLink` concatenates parts, so link boundaries are known by
construction. Locating a `credited_name` *inside* the stored credit
string would reintroduce the fault this exists to fix: that string may
come from the file's tags while the parts come from the catalog, and
the two disagree for ~1 in 3 multi-artist credits (`'Skrillex feat.
Swae Lee'` tagged against `'Skrillex & Swae Lee'` upstream).
- **`credited_name` is stored per row**, never joined from `artists`:
MusicBrainz credits "Snoop Dogg" on a track by the artist called
"Snoop Doggy Dogg". Display follows the credit, navigation the MBID.
- **The lookup is keyed on the recording MBID**, which the catalog and a
local file both carry (`library.Track.RecordingMBID`), so one binding
serves Explore and the library's own lists — which is why this needed
no local table. `file_artists` remains the offline-resilience step and
is deliberately *not* declared until something writes it.
- **Absence is cached as an answer.** `credit-store.ts` stores `[]` for
a single-artist credit — *asked*, not *answered* — or the ~87% that
have nothing to decompose are re-requested on every render forever.
`request()` is per-row and coalesces into one call per frame, because
a virtualized list cannot hand over "the whole list": 50,000 rows is
100 queries for the ~30 on screen.
- **The dump is a third source, and it had to be.** The canonical dump
CI already streams has no join phrases and no as-credited names, and
the JSON dumps cover 153,691 recordings of ~35M with *zero* overlap
against a real library. So `mbdump.tar.bz2` — 7.1 GB, ~13.7 min in
pure-Go bzip2, whose members are alphabetical, which is what lets one
pass resolve an entity's credit without buffering 35M recordings. The
pass runs on **every** mode, because a complete import means
`refresh`, which never enters the importer at all, and it reports
whether it populated anything so `changed` republishes the artifact.
**A 0.6 GB download asks about the connection first.** `explore`'s
catalog artifact had no network awareness at all, which on a phone is a
month's data allowance spent without being asked (plan 016 B4).
`netpolicy.go` is the gate, and its shape is dictated by one constraint:
`explore` is imported by `cmd/indexbuild`, which is built with
`CGO_ENABLED=0` and must not link Wails — so the *policy* and the
*parsing* live here and are tested on every platform, while the platform
call is a closure injected from `app.go`. It is
`application.Mobile.NetworkJSON()`, not `application.Android`'s: the
latter exists only under the `android` build tag, and `Mobile`'s desktop
implementation is a stub returning `""`.
Three rules in it are load-bearing. **An unknown answer is not a metered
one** — only mobile answers at all, so treating silence as metered would
refuse the download on every desktop. **Cellular is the only signal
available**: the runtime reports `wifi|cellular|ethernet|none` and no
metered flag, so a metered *Wi-Fi* (a hotspot, a hotel) cannot be
detected and is not refused, which is a documented gap rather than an
oversight. And **the gate runs before anything is staged**, so declining
is a no-op rather than a job in the indicator and a status the user has
to dismiss. The permission (`AllowMeteredCatalogDownload`, default
false, so an existing config is careful without a migration) is read at
the moment a download would start, so turning it on takes effect on the
next attempt rather than the next launch.
**Background work yields, and says so in the context.** The post-scan
backfills share MusicBrainz's rate limiters with every page the user
can open, and both were FIFO — so a thousand-artist enrichment put an
@@ -683,6 +826,27 @@ moment it is most needed is the likeliest moment loading one fails.
`first-run-wizard` and the startup chrome are eager for the ordinary
reason — they are the first paint.
**A navigation is a history entry, and that is the whole back stack.**
`index.ts` records each navigation with `pushState` (same URL — the app
has no routes, and a path a reload cannot resolve is worse than none)
and replays `popstate` with `_isBack`. It exists for Android, whose back
button is not a key the page can bind: the scaffold's
`MainActivity.onBackPressed` asks `webView.canGoBack()` and finishes the
activity otherwise, so an app that never touched `history` quit from any
depth — which is what a device reported. Hooking the platform's own
mechanism rather than adding a JNI callback is also what makes it
testable in a browser (`page.goBack()`), and the Java half needed no
change at all.
Two rules hold it up. The **first** navigation *replaces* the launch
entry rather than pushing one, or every launch costs a back press before
the app will close. And the in-app back buttons (`navigate-back`, fired
by the detail views and `now-playing-view`) go through `history.back()`
rather than a stack of their own: the old `navStack` is **deleted**, not
kept beside it, because two stacks is precisely how a view's own back
button and the phone's gesture come to disagree about what one press
means.
**A primary view is cached, not unmounted.** `index.ts` keeps every
primary view in the DOM and toggles a `.view-hidden` class, because that
is what preserves `scrollTop` across navigation — so
@@ -838,6 +1002,20 @@ against the real components:
moving focus without setting it leaves the highlight on whichever
item the mouse last touched.
**And a menu opens from a finger, through the event it already has.**
`utils/long-press.ts` is one document-capture listener installed once
from `index.ts`: a touch that holds still for 500 ms dispatches a
synthetic `contextmenu` at the touch point, so all six components that
bind one — delegated on a virtualizer, per row, per card — gained the
gesture without changing. The target is `composedPath()[0]` rather than
`elementFromPoint`, which stops at the outermost shadow host and so
reaches a delegated listener and no per-row one; a browser that fires
its own long-press `contextmenu` (Chromium does, WebKit and the WebView
vary) wins, ours being told from theirs by **identity** rather than
`isTrusted`, since no test can dispatch a trusted event; and the click
that ends the gesture is swallowed, keyed on the gesture rather than on
a time window so the first tap on the menu it opened is not eaten too.
Three lists had no focused row to open a menu *from* — the queue panel
and both playlist detail views — and gained a roving tab stop through
`utils/roving-rows.ts`. **`track-list` deliberately does not use it**:
@@ -1008,6 +1186,65 @@ this app promises, no scrollbar appears. Note that `overflow: hidden`
still permits *programmatic* scrolling, so a probe that sets
`scrollLeft` passes on the broken build; the spec uses a wheel gesture.
**Below 600px it reflows instead, and that is the phone.** The sideways
scroll above was the concession available while the shell had one
layout; plan 016 B2 gives it a second. Under 600px the grid drops its
sidebar column, `<bottom-nav>` takes over as the primary navigation,
the header's controls shrink or stand down, and the shell measures
exactly 320px in a 320px viewport — so `layout-overflow.spec.ts` now
asserts *nothing needs scrolling to*, which is what WCAG 1.4.10 wanted
all along. 600 rather than the sidebar's 900 because 900 is a laptop:
the answer there is a narrower sidebar, which is still a sidebar.
Three rules in it are load-bearing, and the second cost 30 specs.
**A grid item's implicit minimum is its content**, so one child that
insists on 580px makes the *body* 580px wide inside a 360px viewport
and `overflow-x: hidden` then hides a third of the app rather than
fitting it. Every box between the viewport and the content that must
shrink carries `min-width: 0`, and the things that cannot shrink say so
in their own stylesheet — `search-bar`'s 200px floor, `job-indicator`'s
label, `audio-player`'s seek bar and volume. A media query inside a
shadow root is answered by the viewport, so a component states what it
drops at phone width itself rather than the shell reaching in.
**A duplicated component duplicates its handles.** `bottom-nav`'s
"More" opens the *same* `<app-sidebar>` in a `wa-drawer` rather than
listing the destinations again — but rendering it unconditionally put a
second copy of every `data-testid="nav-*"` in the DOM, and 30 existing
specs failed with "strict mode violation: resolved to 2 elements" on a
desktop viewport where the element is not even visible. It renders only
while the drawer is open, and `bottom-nav.test.ts` asserts its absence
before that.
**The tab bar is four destinations and a way to the rest.** Three to
five is where touch targets stop being thumb-sized; eleven over 360px
is 32px each. Which four is plan 016's committed subset, and everything
else — Settings included, because a phone still needs it — is behind
"More".
**The phone section of `index.css` is last on purpose.** A media query
adds no specificity, so a `@media (max-width: 599px)` block placed
above the plain rules it overrides loses to them — which is how phase 1
shipped a header that kept its 2em gutters and 24px title on a 390px
phone with every declaration dead and nothing failing. The shell fitted
anyway, because the fitting is done by `min-width: 0` and by each
component's own media query, which live in their own stylesheets and
have no later rule to lose to. Cosmetic declarations are exactly what
no assertion sees; a screenshot found it.
**`<now-playing-view>` is where the seek bar and volume went.** It is a
*detail* view (`DETAIL_LOADERS`, so the nav stack carries the way out —
a tab you cannot leave by pressing again is not a tab), reached from a
phone-only button over the mini player's art, and it **composes the
real `<seek-bar>`, `<player-controls>` and `<volume-control>`** rather
than reimplementing them. While it is up, `index.css` hides the bottom
bar through `body:has(#main-content[data-active-view="now-playing"])`
the active view is already published as an attribute, and a class
toggled from `index.ts` would be a second expression of the same fact.
The view therefore carries its own queue button, because that button
lives in the bar it hides.
**The playing row is a shape, not a hue.** `track-list` and
`queue-panel` draw a `::before` triangle in each row's own left
padding, plus `aria-current` — before, both rows were a background tint
@@ -1442,6 +1679,28 @@ by the three places that need them (the default widths, the
normaliser, and the resize handles' positions), because they were
written out separately and that is how they came to disagree.
**A phone draws one column of two lines, and that is a column set
rather than a second row template.** Measured on the device: at 424 px
the four configured columns fit the row *exactly* (`--grid-cols` came
out `24px 102px 101px 101px 80px`) and not one of them fit its content
— "Duration" did not fit its own header. The columns were never too
wide; there were too many of them. `PHONE_COLUMN_IDS` is `titleArtist`
(title over artist, sharing the row's whole width) plus the duration, so
the row, the delegated events, the selection semantics, the playing
marker and the virtualizer are all untouched: from their side only the
number of columns changed. Three rules come with it. **The row height
lives in two places and they must agree** — `PHONE_ROW_HEIGHT` and the
CSS rule — because the virtualizer positions rows from that number, so a
taller row overlaps its neighbour. **What is drawn and what can be
sorted are different questions**: the page header's sort list is built
from `configuredColumns`, or a phone (which has no column headers
either) could sort by nothing but title and duration. And **a phone's
widths are neither loaded nor saved**: `loadColumnWidths` is keyed by
column *id* and fills a gap with the minimum, so the stacked column —
which nothing can ever have saved a width for — came out at 148 px
beside a duration column of 236, and saving would have replaced the
width the user dragged on a desktop for the same id.
**The default columns are declared twice and must agree.**
`tracklist.DefaultColumns` is what a fresh install persists;
`DEFAULT_COLUMN_IDS` in `track-list/columns.ts` is what the list draws
@@ -1901,6 +2160,33 @@ like source** — it was generated once into a scratch directory and
copied across (plan 015), it carries one deliberate edit to its
`Taskfile.yml`, and only its output is gitignored. `build/ios/` is
still not carried and its `includes:` entry is still dropped.
**Its `MainActivity` owns the safe area, because `targetSdk 35` does
not leave that to the theme.** Android 15 lays every app out
edge-to-edge and ignores the `statusBarColor`/`navigationBarColor` the
scaffold's theme sets, and the WebView is `match_parent`, so the page's
bottom band — the transport and, on a phone, the tab bar — would be
drawn under the gesture bar. `applyWindowInsets()` pads the container by
`systemBars | displayCutout | ime` and returns the insets rather than
consuming them; the window background is black to match the app's own
ramp, since that padding is what shows through. It is **pre-emptive**:
the phone this was checked against is Android 14, where the system still
insets the window, and the enforcement applies to an app *running on*
15. No browser tier can see this class of fault either way — a viewport
has no system bars.
**And a device is an engine, not just a screen.** The phone this app was
first run on renders in **Chrome 113** — two years behind every browser
any other tier uses — at a 424x439 CSS px viewport. It has `:has()`,
`color-mix()` and `dialog.showModal()`; it does **not** have relaxed CSS
nesting (Chrome 120, so a nested rule beginning with a bare element
selector is silently dropped), the Popover API (114, which Web Awesome's
popups set `popover="manual"` for), `light-dark()` or relative colour
syntax. So "it renders at that size in Chromium" is not evidence about
the phone, and resizing a spec cannot recover the missing signal. `make
android-inspect` forwards the WebView's devtools socket and `make
android-eval` asks the real page — raw CDP, because `connectOverCDP`
calls `Browser.setDownloadBehavior` and a WebView refuses it.
`build/config.yml`'s `version` is the
*metadata* version and is not what the app reports — `main.version` is
stamped at link time from the packaging recipe's git-derived version.
+19 -4
View File
@@ -78,6 +78,19 @@ android-launch: ## Force-stop, clear logcat, and start the app
android-logs: ## Tail logcat, filtered to the app's own tags
@$(ANDROID_ENV) ./scripts/android-emulator.sh logs
# The only tier that can see the platform is the one you can look at.
android-screenshot: ## Grab the device screen (OUT=<path>)
@$(ANDROID_ENV) ./scripts/android-emulator.sh screenshot $(OUT)
# The page's own answer, from the engine that is really rendering it.
# Needs the debug build installed (it is a sibling id, so it does not
# disturb the release app): see scripts/android-eval.mjs.
android-inspect: ## Forward the device WebView's devtools socket
@$(ANDROID_ENV) ./scripts/android-emulator.sh inspect
android-eval: ## Evaluate JS in the device WebView (EXPR='...')
@node ./scripts/android-eval.mjs $(if $(EXPR),'$(EXPR)',)
# "Did it start" is the wrong question — a crash-looping app starts
# several times a second. This asserts the *same pid* is still there.
android-smoke: ## Launch and assert the app is still alive (SECONDS=<n>)
@@ -166,10 +179,12 @@ bindings-check: ## Fail if the generated bindings are stale
css-check: ## Fail if a css`` literal was ended early by a backtick in a comment
@cd frontend && node scripts/check-css-literals.mjs
# .pi/ documents commands, and a skill that documents a command wrongly
# is worse than no skill: an agent runs it confidently. Every command
# in there is a make target on purpose, so this is checkable.
skill-check: ## Fail if .pi/ documents a make target that does not exist
# .pi/ and CLAUDE.md document commands, and a doc that documents a
# command wrongly is worse than no doc: an agent runs it confidently.
# Every command in them is a make target on purpose, so this is
# checkable. It also asserts AGENTS.md is a symlink to CLAUDE.md, so the
# two harnesses cannot drift onto two descriptions of one project.
skill-check: ## Fail if the agent docs name a missing make target, or AGENTS.md is not a symlink
@./scripts/skill-check.sh
# Conventional Commits, which CLAUDE.md claimed CI enforced for a long
+18
View File
@@ -191,6 +191,24 @@ func NewYellowJacketApp(
yjApp.library.SetJobRegistry(yjApp.jobs)
yjApp.explore.SetJobRegistry(yjApp.jobs)
// Whether this connection is one to spend ~0.6 GB of catalog on
// (plan 016 B4). The probe is injected from here because `explore` is
// imported by `cmd/indexbuild`, which must not link Wails: naming
// `application` there is what `TestIndexToolsDoNotImportWails`
// forbids.
//
// `application.Mobile`, not `application.Android`: the latter exists
// only under the `android` build tag, while `Mobile` is the portable
// name whose desktop implementation is a stub returning "" — which
// parses to "unknown" and refuses nothing. Plan 016 named the tagged
// one; this is the same call by the name every build has.
yjApp.explore.SetNetworkPolicy(
func() explore.Network {
return explore.ParseNetworkJSON(application.Mobile.NetworkJSON())
},
yjApp.appConfig.GetAllowMeteredCatalogDownload,
)
// Let the release prefetch skip albums the user already owns in
// full — those open with no catalog call at all, so warming their
// tracklists spends the most expensive request in the app on
+45
View File
@@ -620,6 +620,51 @@ func (c *Config) SetQueueFallback(mode string) error {
return nil
}
// GetAllowMeteredCatalogDownload reports whether the ~0.6 GB Explore
// catalog may be fetched on a metered connection.
func (c *Config) GetAllowMeteredCatalogDownload() bool {
if c.General == nil {
return false
}
return c.General.AllowMeteredCatalogDownload
}
// SetAllowMeteredCatalogDownload saves the metered-download permission.
//
// There is nothing to validate and nothing to restart: the policy is
// read at the moment a download would start, so turning it on takes
// effect on the next attempt rather than needing this launch to be over.
func (c *Config) SetAllowMeteredCatalogDownload(allow bool) error {
if c.General == nil {
c.General = &GeneralConfig{}
c.General.ApplyDefaults()
}
c.General.AllowMeteredCatalogDownload = allow
if err := c.Save(); err != nil {
return fmt.Errorf(
"could not save config: %w", err,
)
}
events.Emit(
c.ctx,
events.GeneralConfigChanged,
map[string]any{
"AllowMeteredCatalogDownload": allow,
},
)
c.logger.Info(
"metered catalog download permission updated",
"allow", allow,
)
return nil
}
// GetTrackListColumns returns the configured track-list columns.
func (c *Config) GetTrackListColumns() []tracklist.Column {
if c.TrackList == nil {
+6
View File
@@ -48,6 +48,12 @@ var errUnknownQueueFallback = errors.New("unknown queue fallback")
type GeneralConfig struct {
DefaultPage DefaultPage `toml:"DefaultPage"`
QueueFallback QueueFallback `toml:"QueueFallback"`
// AllowMeteredCatalogDownload permits the ~0.6 GB Explore catalog to
// be fetched on a connection the platform calls cellular. It defaults
// to false, which is the whole point: the zero value is the safe one,
// so an existing config with no such key refuses by default rather
// than needing a migration to become careful.
AllowMeteredCatalogDownload bool `toml:"AllowMeteredCatalogDownload"`
}
// ApplyDefaults fills zero-value fields with sensible defaults.
+8
View File
@@ -89,6 +89,14 @@ func NewDB(logger *slog.Logger) (*DB, error) {
return nil, fmt.Errorf("could not apply PRAGMAs: %w", err)
}
// Before the schema is applied, not after: applySchema is
// CREATE ... IF NOT EXISTS, which no-ops against a table that
// already exists in an older shape. Retiring the stale one first is
// what turns that no-op into a create.
if err := retireStaleTables(dbCtx, db, logger); err != nil {
return nil, err
}
if err := applySchema(dbCtx, db); err != nil {
return nil, err
}
@@ -0,0 +1,56 @@
-- The decomposition of a multi-artist credit, from the MusicBrainz
-- dump. One row per credited artist, in credit order.
--
-- A credit is ordered parts, and the credit *string* is derived from
-- them -- MusicBrainz's own `artist_credit.name` is a cached render and
-- nothing more. Rendering is a concatenation:
--
-- for each part in position order:
-- emit link(credited_name -> artist_mbid)
-- emit text(join_phrase)
--
-- so the link boundaries are known by construction. That is the whole
-- reason this table exists, and it is why nothing may reconstruct a
-- credit by *searching* for a name inside a credit string: the stored
-- string may have come from a file's tags while the parts come from the
-- catalog, and measured on a real library those disagree for about one
-- in three multi-artist credits ("Skrillex feat. Swae Lee" tagged
-- against "Skrillex & Swae Lee" upstream). A search would miss, or
-- match the wrong span.
--
-- `credited_name` is the name *as credited*, which is not the artist's
-- canonical name: MusicBrainz credits "Snoop Dogg" on a track by the
-- artist whose name is "Snoop Doggy Dogg". It is stored per row rather
-- than joined from an artist table for exactly that reason.
--
-- Only *multi-artist* credits are stored. A single-artist credit is
-- (name, "") and is already fully described by explore_index's
-- artist_name and artist_mbid; storing those would roughly triple the
-- table to say nothing new.
--
-- Credits are shared: an album's twelve tracks by one artist reference
-- one credit_id. That is the opposite of the local library's verdict
-- in plan 013, and correctly so -- credit sharing is 1:1 in one
-- person's files and genuinely many-to-one across a 2M-row catalog.
--
-- MBIDs are the same 16 raw bytes explore_index stores, for the same
-- size reason and with the same CHECK, so a stringly write fails at the
-- insert that made it rather than reading back as no rows at all. See
-- backend/explore/mbid.go.
CREATE TABLE IF NOT EXISTS artist_credit_part (
credit_id INTEGER NOT NULL,
position INTEGER NOT NULL,
artist_mbid BLOB NOT NULL CHECK(length(artist_mbid) = 16),
-- The name as credited on this release, which may differ from the
-- artist's canonical name. Display uses this; navigation uses the
-- MBID above.
credited_name TEXT NOT NULL,
-- The literal connector that follows this part -- " feat. ", " & ",
-- ", ", or "" on the last part. Rendered as plain text between two
-- links.
join_phrase TEXT NOT NULL DEFAULT '',
PRIMARY KEY (credit_id, position)
) WITHOUT ROWID;
@@ -0,0 +1,30 @@
-- Which credit a catalog entity is credited to. One row per recording
-- or release group whose credit names more than one artist.
--
-- This is a table rather than an `explore_index.artist_credit_id`
-- column, and that is a deliberate consequence of how this app applies
-- its schema. `applySchema` is CREATE ... IF NOT EXISTS and there is
-- no migration chain (plan 013), so a *column* added to an existing
-- table never reaches a database that already has it -- while a new
-- *table* is created on every install, old or new, for free.
-- explore_index is the one table nobody can afford to drop and rebuild
-- on a schema change: it is the artifact users download rather than
-- derive.
--
-- Only multi-artist credits are referenced here, matching
-- artist_credit_part. An entity with no row is credited to exactly one
-- artist, which explore_index's own artist_name and artist_mbid already
-- describe -- so absence is the common case and means "nothing to
-- decompose", not "unknown".
--
-- `credit_id` is opaque and is only meaningful against the
-- artist_credit_part rows built or imported alongside it. The two are
-- always written together; nothing persists a credit_id anywhere else.
-- The local library stores resolved parts, never this id.
CREATE TABLE IF NOT EXISTS artist_credit_ref (
mbid BLOB NOT NULL PRIMARY KEY CHECK(length(mbid) = 16),
credit_id INTEGER NOT NULL
) WITHOUT ROWID;
CREATE INDEX IF NOT EXISTS idx_artist_credit_ref_credit
ON artist_credit_ref(credit_id);
+13
View File
@@ -27,6 +27,19 @@ type Artist struct {
Mbid sql.NullString
}
type ArtistCreditPart struct {
CreditID int64
Position int64
ArtistMbid []byte
CreditedName string
JoinPhrase string
}
type ArtistCreditRef struct {
Mbid []byte
CreditID int64
}
type ArtistEnrichment struct {
ArtistMbid string
BrowsedAt sql.NullTime
+560
View File
@@ -0,0 +1,560 @@
package database
import (
"context"
"database/sql"
"fmt"
"io/fs"
"log/slog"
"maps"
"path"
"slices"
"strings"
"yellowjacket/backend/datamap"
)
// This file repairs the one thing `CREATE TABLE IF NOT EXISTS` cannot.
//
// `sql/schemas/` is the single description of the schema and there is no
// migration chain (plan 013): a schema change is one edit to one file.
// That works perfectly for a *new* table, which every install then
// creates, and not at all for a changed one -- `IF NOT EXISTS` reaches
// an existing table only if its shape already matches, and otherwise
// silently no-ops. The user's answer to that is "delete and rescan"
// (plan 013, open question 1), which is free for everything a rescan
// rebuilds.
//
// It is not free for the catalog. explore_index is a *downloaded
// artifact*, not something derived from the user's files, and it is the
// largest thing this app stores. So it went stale instead: plan 014
// added `total_tracks` to the schema and to `indexRowFields` -- the one
// projection every explore read uses -- and no database that already
// existed ever grew the column. Every Explore search, browse, artist
// page and album page on such an install fails with
// "no such column: total_tracks", while a fresh install is perfectly
// healthy, which is why the tests did not see it. The same databases
// are stale a second way, from the same plan: their `mbid` columns are
// still TEXT where the schema now declares BLOB, and SQLite does not
// coerce between the two -- a comparison against 16 raw bytes simply
// returns no rows.
//
// The repair is to notice and drop, not to migrate. A dropped catalog
// costs one artifact download (about a minute); the alternative --
// ALTER TABLE ADD COLUMN, which would handle `total_tracks` alone
// cheaply -- cannot express the TEXT-to-BLOB half at all, and would
// leave those installs quietly broken while reporting success.
//
// Everything except `Authored` is eligible. `Cache` is rebuildable by
// definition; `Owned` is a projection of the user's files and a rescan
// rebuilds it, which is plan 013's stated answer to exactly this
// situation ("delete and rescan", open question 1); `Derived` is
// computed from Owned. No `Authored` table is ever dropped here --
// that is the whole point of the datamap, and it is asserted by
// TestAuthoredTablesAreNeverRetired rather than only stated.
//
// What that does *not* buy is immunity for authored rows that reference
// a retired table. `audio_files` is MIXED KIND: `play_count`,
// `last_played` and `tag_status` are authored columns on an Owned
// table, and they go with it. Playlists survive as playlists, and
// their entries survive pointing at nothing. That cost was weighed and
// accepted rather than overlooked -- the alternative is to carry the
// authored columns across the rebuild keyed on file_path, which stays a
// real option if this ever bites harder than it is worth.
//
// **This relies on foreign_keys being ON**, which applyPRAGMAs has
// already done by the time NewDB calls it, and the dependency is not
// cosmetic. SQLite performs an implicit DELETE before dropping a table
// when foreign keys are enabled, so `playlist_tracks.audio_file_id` --
// declared ON DELETE SET NULL -- is nulled. With foreign keys off, no
// action fires and those rows keep the ids they had, which a rescan
// then reissues starting from 1: every playlist would silently fill
// with *different songs*. Nulled entries are merely empty; stale ones
// are wrong, and wrong quietly. TestRetiringOwnedTablesDoesNotDangle
// is what stops a future reordering turning one into the other.
// retireGroups are tables that must be retired together. A catalog
// whose rows are gone must not keep the full-text index built over
// them, nor the metadata claiming the import that produced them
// finished -- that marker is exactly what stops the artifact being
// fetched again. applySchema recreates all three empty immediately
// afterwards, and the ordinary "no index yet" path takes over.
var retireGroups = [][]string{
{
"explore_index",
"explore_index_fts",
"explore_index_meta",
"explore_champion_fts",
},
}
// schemaColumn is one column as the schema file declares it.
type schemaColumn struct {
name string
typ string
}
// retireStaleTables drops every non-authored table whose live shape no
// longer matches what sql/schemas/ declares, plus any table the schema
// no longer describes at all, so applySchema can create the current
// shape afresh. It runs before applySchema and is a no-op on a new
// database, where the tables do not exist yet.
func retireStaleTables(
ctx context.Context, db *sql.DB, logger *slog.Logger,
) error {
declared, err := declaredTables()
if err != nil {
return err
}
stale := make(map[string]string)
for table, columns := range declared {
entry, ok := datamap.Lookup(table)
if !ok || entry.Kind == datamap.Authored || entry.FTS {
continue
}
// Whether a stale Cache table may be rebuilt is decided per
// binary, at compile time: the app re-downloads its catalog in
// about a minute, cmd/indexbuild would re-derive it from ~205 GB
// of dumps. See staleshape_policy.go.
if entry.Kind == datamap.Cache && !retireStaleCache {
continue
}
reason, err := staleReason(ctx, db, table, columns)
if err != nil {
return err
}
if reason != "" {
stale[table] = reason
}
}
obsolete, err := obsoleteTables(ctx, db)
if err != nil {
return err
}
maps.Copy(stale, obsolete)
if len(stale) == 0 {
return nil
}
return retireGroupsFor(ctx, db, logger, stale)
}
// obsoleteTables are live tables the schema no longer describes at all.
// TestCatalogCoversSchema makes the datamap a complete description of
// the current schema, so a table it does not know is one a past version
// created and this one does not -- plan 013 alone left seven behind
// (recordings, release_groups, artist_credit, artist_credit_artist,
// release_group_recordings, recording_genres) plus the
// schema_migrations table that squashing the chain retired. They are
// dead weight, and one of them holding a foreign key into a table being
// rebuilt is worse than dead weight.
//
// SQLite's own bookkeeping and FTS shadow tables are not obsolete:
// datamap.Lookup resolves a shadow table to its parent, and IsInternal
// covers the rest.
func obsoleteTables(ctx context.Context, db *sql.DB) (map[string]string, error) {
rows, err := db.QueryContext(
ctx, "SELECT name FROM sqlite_master WHERE type = 'table'",
)
if err != nil {
return nil, fmt.Errorf("could not list tables: %w", err)
}
defer func() { _ = rows.Close() }()
out := make(map[string]string)
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, fmt.Errorf("could not scan table name: %w", err)
}
if datamap.IsInternal(name) {
continue
}
if _, known := datamap.Lookup(name); !known {
out[name] = "the schema no longer describes this table"
}
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("could not read table list: %w", err)
}
return out, nil
}
// retireGroupsFor drops each stale table along with everything its
// retire group says must go with it.
func retireGroupsFor(
ctx context.Context, db *sql.DB, logger *slog.Logger,
stale map[string]string,
) error {
drop := make(map[string]string)
for table, reason := range stale {
drop[table] = reason
for _, group := range retireGroups {
if !slices.Contains(group, table) {
continue
}
for _, member := range group {
if _, already := drop[member]; !already {
drop[member] = "retired with " + table
}
}
}
}
return dropDeferred(ctx, db, logger, drop)
}
// dropDeferred drops every named table in one transaction with foreign
// key enforcement deferred to the commit.
//
// The deferral is required and the two obvious alternatives are both
// wrong. These tables reference each other -- pre-013 `audio_files`
// has a foreign key into `recordings`, which is itself being retired --
// so dropping them one at a time in an arbitrary order fails with
// "FOREIGN KEY constraint failed" on whichever is unlucky enough to go
// first, and there is no order that is safe in general. Turning
// foreign keys *off* for the duration would fix that and silently take
// the ON DELETE SET NULL on `playlist_tracks.audio_file_id` with it,
// leaving playlist entries pointing at ids a rescan reissues to
// different songs -- the exact failure
// TestRetiringOwnedTablesDoesNotDangle exists to prevent.
//
// Deferring keeps the actions firing while tolerating the inconsistency
// in the middle, and the commit then checks that the end state is
// sound. It is set inside the transaction because SQLite resets it at
// every commit.
func dropDeferred(
ctx context.Context, db *sql.DB, logger *slog.Logger,
drop map[string]string,
) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("could not begin the retire transaction: %w", err)
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx, "PRAGMA defer_foreign_keys = ON"); err != nil {
return fmt.Errorf("could not defer foreign keys: %w", err)
}
// Sorted, so a failure is reproducible. Map order is random, and a
// bug that depends on which table happens to go first reproduces on
// one run in three and passes review on the other two -- which is
// exactly how the foreign-key ordering above reached a real
// database. Sorting does not make any order *safe*; the deferral
// does that.
for _, table := range slices.Sorted(maps.Keys(drop)) {
logger.Warn(
"retiring a table the schema no longer describes",
"table", table,
"reason", drop[table],
)
if _, err := tx.ExecContext(
ctx, "DROP TABLE IF EXISTS "+quoteIdent(table),
); err != nil {
return fmt.Errorf("could not retire stale table %s: %w", table, err)
}
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("could not commit the retire: %w", err)
}
return nil
}
// staleReason reports why a live table disagrees with its declaration,
// or "" when it agrees. A column the live table does not have is the
// additive case; a column whose declared type changed is the one an
// ALTER could not fix anyway. Columns the live table has and the
// schema no longer declares are ignored: they cost nothing and dropping
// the table over one would retire a healthy catalog.
func staleReason(
ctx context.Context, db *sql.DB, table string, columns []schemaColumn,
) (string, error) {
live, err := liveColumns(ctx, db, table)
if err != nil {
return "", err
}
if len(live) == 0 {
// Not present at all: applySchema is about to create it.
return "", nil
}
for _, col := range columns {
liveType, present := live[col.name]
if !present {
return "missing column " + col.name, nil
}
if !sameDeclaredType(col.typ, liveType) {
return fmt.Sprintf(
"column %s is %s, schema declares %s",
col.name, liveType, col.typ,
), nil
}
}
return "", nil
}
// liveColumns returns the live table's columns and their declared types,
// empty when the table does not exist.
func liveColumns(
ctx context.Context, db *sql.DB, table string,
) (map[string]string, error) {
rows, err := db.QueryContext(
ctx, "SELECT name, type FROM pragma_table_info(?)", table,
)
if err != nil {
return nil, fmt.Errorf("could not inspect table %s: %w", table, err)
}
defer func() { _ = rows.Close() }()
out := make(map[string]string)
for rows.Next() {
var name, typ string
if err := rows.Scan(&name, &typ); err != nil {
return nil, fmt.Errorf("could not scan column of %s: %w", table, err)
}
out[name] = typ
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("could not read columns of %s: %w", table, err)
}
return out, nil
}
// sameDeclaredType compares two SQLite type names. They are compared
// case-insensitively and only on the leading word, so INTEGER matches
// INTEGER and VARCHAR(20) matches VARCHAR -- SQLite's affinity rules
// make finer distinctions meaningless, and a difference that fine is
// not worth retiring a catalog over. An empty declared type matches
// anything, which is what a column declared with only constraints has.
func sameDeclaredType(declared, live string) bool {
d := strings.ToUpper(strings.Fields(declared + " ")[0])
l := strings.ToUpper(strings.Fields(live + " ")[0])
if d == "" || l == "" {
return true
}
if i := strings.IndexByte(d, '('); i >= 0 {
d = d[:i]
}
if i := strings.IndexByte(l, '('); i >= 0 {
l = l[:i]
}
return d == l
}
// declaredTables parses every CREATE TABLE in sql/schemas/ into its
// column list. Parsing the schema rather than writing the expectation
// down a second time is the point: a second list is a second thing to
// forget, which is the fault this whole file exists to repair.
func declaredTables() (map[string][]schemaColumn, error) {
dirEntries, err := schemas.ReadDir("sql/schemas")
if err != nil {
return nil, fmt.Errorf("could not read schemas directory: %w", err)
}
out := make(map[string][]schemaColumn)
for _, dirEntry := range dirEntries {
if dirEntry.IsDir() {
continue
}
content, err := fs.ReadFile(schemas, path.Join("sql/schemas", dirEntry.Name()))
if err != nil {
return nil, fmt.Errorf("could not read %s: %w", dirEntry.Name(), err)
}
maps.Copy(out, parseCreateTables(string(content)))
}
return out, nil
}
// constraintKeywords begin a table constraint rather than a column.
var constraintKeywords = map[string]bool{
"PRIMARY": true, "FOREIGN": true, "UNIQUE": true,
"CHECK": true, "CONSTRAINT": true,
}
// parseCreateTables extracts the column names and declared types of
// every non-virtual CREATE TABLE in one schema file.
func parseCreateTables(content string) map[string][]schemaColumn {
out := make(map[string][]schemaColumn)
rest := stripLineComments(content)
for {
idx := indexFold(rest, "CREATE TABLE ")
if idx < 0 {
return out
}
rest = rest[idx+len("CREATE TABLE "):]
head, body, ok := splitTableBody(rest)
if !ok {
return out
}
if name := tableName(head); name != "" {
out[name] = parseColumns(body)
}
}
}
// tableName pulls the table name out of the text between "CREATE TABLE"
// and its opening parenthesis, dropping an IF NOT EXISTS and any
// quoting.
func tableName(head string) string {
head = strings.TrimSpace(head)
head = strings.TrimPrefix(head, "IF NOT EXISTS ")
head = strings.TrimPrefix(head, "if not exists ")
fields := strings.Fields(head)
if len(fields) == 0 {
return ""
}
return strings.Trim(fields[len(fields)-1], `"'`+"`")
}
// splitTableBody returns the text before the table's opening paren and
// the balanced text inside it.
func splitTableBody(s string) (head, body string, ok bool) {
open := strings.IndexByte(s, '(')
if open < 0 {
return "", "", false
}
depth := 0
for i := open; i < len(s); i++ {
switch s[i] {
case '(':
depth++
case ')':
depth--
if depth == 0 {
return s[:open], s[open+1 : i], true
}
}
}
return "", "", false
}
// parseColumns splits a table body on its top-level commas and keeps
// the parts that are columns rather than table constraints.
func parseColumns(body string) []schemaColumn {
var (
out []schemaColumn
depth int
start int
)
parts := make([]string, 0, 8)
for i := range len(body) {
switch body[i] {
case '(':
depth++
case ')':
depth--
case ',':
if depth == 0 {
parts = append(parts, body[start:i])
start = i + 1
}
}
}
parts = append(parts, body[start:])
for _, part := range parts {
fields := strings.Fields(part)
if len(fields) == 0 {
continue
}
// A table constraint need not be followed by a space --
// "UNIQUE(mbid)" is one field, and reading it as a column name
// makes an entirely healthy table look stale, which retires a
// catalog nobody asked to lose.
head := fields[0]
if i := strings.IndexByte(head, '('); i >= 0 {
head = head[:i]
}
if constraintKeywords[strings.ToUpper(head)] {
continue
}
col := schemaColumn{name: strings.Trim(head, `"'`+"`")}
if len(fields) > 1 {
col.typ = fields[1]
}
out = append(out, col)
}
return out
}
// stripLineComments removes -- comments, which otherwise contribute
// stray parentheses and commas to the parse.
func stripLineComments(s string) string {
lines := strings.Split(s, "\n")
for i, line := range lines {
if idx := strings.Index(line, "--"); idx >= 0 {
lines[i] = line[:idx]
}
}
return strings.Join(lines, "\n")
}
// indexFold is a case-insensitive strings.Index.
func indexFold(s, substr string) int {
return strings.Index(strings.ToUpper(s), strings.ToUpper(substr))
}
// quoteIdent quotes a table name for interpolation into DDL, which
// cannot take a bound parameter.
func quoteIdent(name string) string {
return `"` + strings.ReplaceAll(name, `"`, `""`) + `"`
}
+15
View File
@@ -0,0 +1,15 @@
//go:build !indexbuild
package database
// retireStaleCache reports whether a Cache table whose shape no longer
// matches the schema may be dropped and rebuilt.
//
// In the app: yes. The only Cache table large enough to care about is
// the catalog, and the app does not derive it — it downloads it. A
// stale one costs about a minute of re-fetching the artifact, and
// keeping it costs every Explore read on the install, because a
// projection naming a column the table does not have fails outright.
//
// In cmd/indexbuild: no, and the file next to this one says why.
const retireStaleCache = true
@@ -0,0 +1,37 @@
//go:build indexbuild
package database
// retireStaleCache is false here, and this is the whole reason the
// policy is a build tag rather than a rule inside retireStaleTables.
//
// The index database is the one place in this project where the catalog
// is *derived* rather than downloaded. Rebuilding it is a ~205 GB dump
// stream over hours, resumed across runs from a checkpoint on a
// persistent volume; that volume exists for no other purpose. The app's
// answer to a stale catalog — drop it, fetch the artifact again — is
// not available here, because this database *is* what the artifact is
// cut from.
//
// This was not hypothetical. The repair shipped without it and dropped
// the CI catalog on its first run:
//
// retiring a table ... table=explore_index
// reason="column entity_type is TEXT, schema declares INTEGER"
// index maintenance mode=build reason="no completed import yet"
//
// The shape mismatch was real and the drop was correct by the app's
// rule. It was still wrong here: that database is deliberately kept in
// the older encoding, which is what `fix(indexexport): read an index
// older than the binary` exists to tolerate. A rule that is right for
// every install and catastrophic for one database has to be told which
// one it is in, and a build tag is how this project already tells the
// index tools apart (backend/events/runtime_indexbuild.go,
// backend/explore/servicestartup.go, dumpbuild_stub.go).
//
// cmd/indexbuild has its own repair for the half it *can* safely
// discard: retireLibraryTables drops every table the datamap does not
// classify as Cache, which is empty by construction in that database.
// Between the two, the library half is repaired and the catalog is
// never touched.
const retireStaleCache = false
+499
View File
@@ -0,0 +1,499 @@
package database
import (
"context"
"database/sql"
"log/slog"
"path"
"testing"
_ "modernc.org/sqlite"
)
// testLogger discards the repair's warnings; the tests assert on the
// database, not on the log.
func testLogger() *slog.Logger {
return slog.New(slog.DiscardHandler)
}
// openRaw opens a scratch database file with no schema applied, so a
// test can build an *old* shape and then let NewDB's repair meet it.
func openRaw(t *testing.T, dir string) *sql.DB {
t.Helper()
db, err := sql.Open("sqlite", path.Join(dir, "yj.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
return db
}
// TestRetiresIndexMissingAColumn is plan 014's bug, symptom first: an
// explore_index created before `total_tracks` existed, met by the
// projection every explore read uses. Before the repair this failed
// with "no such column: total_tracks" on every install that already had
// a catalog, while a fresh one was perfectly healthy.
func TestRetiresIndexMissingAColumn(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
// The pre-014 shape: the columns the projection needs, minus the
// one the plan added.
if _, err := db.ExecContext(ctx, `
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid BLOB NOT NULL
);
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid)
VALUES (1, x'00112233445566778899aabbccddeeff', 'x', 'y', x'');
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
// The column the projection needs is there now.
var n int
if err := db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM pragma_table_info('explore_index')
WHERE name = 'total_tracks'`,
).Scan(&n); err != nil {
t.Fatalf("inspect: %v", err)
}
if n != 1 {
t.Fatalf("explore_index still has no total_tracks column")
}
// And the catalog really was retired rather than patched, so the
// artifact is fetched again instead of half a catalog being served.
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM explore_index",
).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 0 {
t.Fatalf("stale rows survived the retire: %d", n)
}
}
// TestRetiresIndexWithTextMBIDs is the half an ALTER could not have
// repaired: plan 013 changed mbid from TEXT to BLOB, and SQLite does not
// coerce between them, so a query against 16 raw bytes returns no rows
// rather than an error.
func TestRetiresIndexWithTextMBIDs(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
if _, err := db.ExecContext(ctx, `
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type TEXT NOT NULL,
mbid TEXT NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid TEXT NOT NULL,
total_tracks INTEGER NOT NULL DEFAULT 0
);
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
var typ string
if err := db.QueryRowContext(ctx,
`SELECT type FROM pragma_table_info('explore_index') WHERE name = 'mbid'`,
).Scan(&typ); err != nil {
t.Fatalf("inspect: %v", err)
}
if typ != "BLOB" {
t.Fatalf("mbid is still %s, want BLOB", typ)
}
}
// TestRetiringTheIndexTakesItsMetaWithIt guards the thing that makes the
// repair actually repair: the marker saying the import finished is what
// stops the artifact being fetched again, so a catalog dropped without
// it would stay empty forever.
func TestRetiringTheIndexTakesItsMetaWithIt(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
if _, err := db.ExecContext(ctx, `
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL
);
CREATE TABLE explore_index_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
INSERT INTO explore_index_meta VALUES ('dump_import_done', '1');
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM explore_index_meta WHERE key = 'dump_import_done'",
).Scan(&n); err != nil {
t.Fatalf("meta: %v", err)
}
if n != 0 {
t.Fatalf("the import-done marker survived a retired catalog")
}
}
// TestHealthyDatabaseIsUntouched is the other half, and the one that
// would make this dangerous if it failed: a current schema must survive
// a launch with its catalog intact. A repair that retires a healthy
// catalog costs every user an artifact download on every start.
func TestHealthyDatabaseIsUntouched(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
if _, err := db.ExecContext(ctx, `
INSERT INTO explore_index (entity_type, mbid, title, artist_name, artist_mbid)
VALUES (1, x'00112233445566778899aabbccddeeff', 'x', 'y', x'')
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM explore_index",
).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 1 {
t.Fatalf("a healthy catalog was retired: %d rows left", n)
}
}
// TestAuthoredTablesAreNeverRetired states the boundary in a test rather
// than only in a comment: this mechanism deletes data, and the only
// thing standing between it and a user's playlists is the Kind filter.
func TestAuthoredTablesAreNeverRetired(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
db := openRaw(t, dir)
// A playlists table missing most of its current columns.
if _, err := db.ExecContext(ctx, `
CREATE TABLE playlists (id INTEGER PRIMARY KEY, name TEXT NOT NULL);
INSERT INTO playlists (name) VALUES ('irreplaceable');
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM playlists",
).Scan(&n); err != nil {
t.Fatalf("count: %v", err)
}
if n != 1 {
t.Fatalf("an authored table was retired; rows left: %d", n)
}
}
// TestRetiresTablesTheSchemaNoLongerDescribes covers what plan 013 left
// behind on every database that predates it: seven tables the schema
// stopped describing, plus the schema_migrations table that squashing
// the chain retired. They are not stale in shape — they are simply not
// ours any more.
func TestRetiresTablesTheSchemaNoLongerDescribes(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if _, err := db.ExecContext(ctx, `
CREATE TABLE recordings (id INTEGER PRIMARY KEY, name TEXT);
CREATE TABLE artist_credit (id INTEGER PRIMARY KEY, text TEXT);
CREATE TABLE schema_migrations (version INTEGER PRIMARY KEY);
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
for _, table := range []string{"recordings", "artist_credit", "schema_migrations"} {
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name = ?",
table,
).Scan(&n); err != nil {
t.Fatalf("inspect %s: %v", table, err)
}
if n != 0 {
t.Errorf("%s survived; the schema no longer describes it", table)
}
}
}
// TestFTSShadowTablesAreNotObsolete is the sweep's sharp edge: an FTS5
// virtual table is backed by four shadow tables that appear in
// sqlite_master under their own names and are in no schema file.
// Dropping one destroys the index it belongs to.
func TestFTSShadowTablesAreNotObsolete(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
obsolete, err := obsoleteTables(ctx, db)
if err != nil {
t.Fatalf("obsoleteTables: %v", err)
}
if len(obsolete) != 0 {
t.Fatalf("a freshly created schema reported obsolete tables: %v", obsolete)
}
}
// TestRetiringOwnedTablesDoesNotDangle pins the one behaviour that is
// silently wrong rather than loudly broken.
//
// Retiring audio_files leaves playlist entries behind. With
// foreign_keys ON — which applyPRAGMAs has done before NewDB gets here —
// SET NULL fires and they point at nothing. With it OFF they keep ids
// that the rescan reissues from 1, so every playlist quietly fills with
// different songs. Nothing about the schema makes that ordering
// obvious, so it is asserted rather than assumed.
func TestRetiringOwnedTablesDoesNotDangle(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if _, err := db.ExecContext(ctx, "PRAGMA foreign_keys = ON"); err != nil {
t.Fatalf("pragma: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
// Break audio_files' shape so it is retired, keeping a playlist
// entry that references it.
if _, err := db.ExecContext(ctx, `
INSERT INTO playlists (id, name) VALUES (1, 'keepme');
INSERT INTO libraries (id, name, path) VALUES (0, 'test', '/music');
INSERT INTO audio_files (id, file_path, file_type_id, length_milliseconds)
VALUES (7, '/music/a.flac', 1, 1000);
INSERT INTO playlist_tracks (playlist_id, audio_file_id, position)
VALUES (1, 7, 0);
DROP VIEW IF EXISTS track_metadata;
ALTER TABLE audio_files DROP COLUMN artist_credit;
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
var dangling int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM playlist_tracks WHERE audio_file_id IS NOT NULL",
).Scan(&dangling); err != nil {
t.Fatalf("count: %v", err)
}
if dangling != 0 {
t.Fatalf(
"%d playlist entries still point at retired audio_files ids; "+
"a rescan will reissue those ids to different tracks",
dangling,
)
}
// The playlist itself is authored and must be untouched.
var playlists int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM playlists",
).Scan(&playlists); err != nil {
t.Fatalf("playlists: %v", err)
}
if playlists != 1 {
t.Fatalf("authored playlist lost: %d", playlists)
}
}
// TestRetiringInterlinkedLegacyTables is the bug the unit tests missed
// and a real database found.
//
// The tables plan 013 retired reference each other -- pre-013
// audio_files has a foreign key into recordings -- so with foreign keys
// ON, dropping them one at a time fails with "FOREIGN KEY constraint
// failed" on whichever goes first, and map iteration order decides
// which that is. Every other test in this file ran with foreign keys
// off and passed happily; the app enables them in applyPRAGMAs before
// the repair runs, so only the real launch path showed it.
func TestRetiringInterlinkedLegacyTables(t *testing.T) {
ctx := context.Background()
db := openRaw(t, t.TempDir())
if _, err := db.ExecContext(ctx, "PRAGMA foreign_keys = ON"); err != nil {
t.Fatalf("pragma: %v", err)
}
// The pre-013 shape, with the reference that makes ordering matter.
// release_group_recordings sorts *after* recordings and references
// it, so the deterministic order retires the parent while the child
// still holds rows pointing at it -- which is the case that fails
// without the deferral, rather than one that fails on some runs.
if _, err := db.ExecContext(ctx, `
CREATE TABLE recordings (id INTEGER PRIMARY KEY, name TEXT);
CREATE TABLE artist_credit (id INTEGER PRIMARY KEY, text TEXT);
CREATE TABLE release_group_recordings (
id INTEGER PRIMARY KEY,
recording_id INTEGER NOT NULL,
FOREIGN KEY(recording_id) REFERENCES recordings(id)
);
CREATE TABLE audio_files (
id INTEGER PRIMARY KEY,
file_path TEXT NOT NULL UNIQUE,
recording_id INTEGER,
FOREIGN KEY(recording_id) REFERENCES recordings(id)
);
INSERT INTO recordings (id, name) VALUES (1, 'x');
INSERT INTO release_group_recordings (id, recording_id) VALUES (1, 1);
INSERT INTO audio_files (id, file_path, recording_id)
VALUES (1, '/music/a.flac', 1);
`); err != nil {
t.Fatalf("seed: %v", err)
}
if err := retireStaleTables(ctx, db, testLogger()); err != nil {
t.Fatalf("retire: %v", err)
}
if err := applySchema(ctx, db); err != nil {
t.Fatalf("applySchema: %v", err)
}
for _, table := range []string{"recordings", "artist_credit"} {
var n int
if err := db.QueryRowContext(ctx,
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name = ?",
table,
).Scan(&n); err != nil {
t.Fatalf("inspect %s: %v", table, err)
}
if n != 0 {
t.Errorf("%s survived the retire", table)
}
}
// And the rebuilt audio_files is the current shape, which is the
// whole reason the old one had to go.
var n int
if err := db.QueryRowContext(ctx,
`SELECT COUNT(*) FROM pragma_table_info('audio_files')
WHERE name = 'artist_credit'`,
).Scan(&n); err != nil {
t.Fatalf("inspect audio_files: %v", err)
}
if n != 1 {
t.Fatal("audio_files was not rebuilt in the current shape")
}
}
// TestParseCreateTablesReadsTheRealSchema keeps the parser honest
// against the files it actually runs on: a parser that silently found
// no columns would report every table healthy and repair nothing.
func TestParseCreateTablesReadsTheRealSchema(t *testing.T) {
declared, err := declaredTables()
if err != nil {
t.Fatalf("declaredTables: %v", err)
}
cols, ok := declared["explore_index"]
if !ok {
t.Fatal("explore_index was not parsed out of the schema files")
}
want := map[string]string{
"mbid": "BLOB",
"total_tracks": "INTEGER",
"artist_name": "TEXT",
}
got := make(map[string]string, len(cols))
for _, c := range cols {
got[c.name] = c.typ
}
for name, typ := range want {
if got[name] != typ {
t.Errorf("explore_index.%s parsed as %q, want %q", name, got[name], typ)
}
}
// A table constraint must not be mistaken for a column.
for _, c := range cols {
switch c.name {
case "PRIMARY", "FOREIGN", "UNIQUE", "CHECK", "CONSTRAINT":
t.Errorf("parsed table constraint %q as a column", c.name)
}
}
}
+15
View File
@@ -185,6 +185,21 @@ var tables = []Table{
Note: "Full-text index over the champion entities of the " +
"MusicBrainz dump. Rebuilt only by a full index build.",
},
{
Name: "artist_credit_part", Kind: Cache, Lifetime: Retained,
Note: "The decomposition of a multi-artist credit, from the " +
"MusicBrainz dump: one row per credited artist, with the " +
"name as credited and the join phrase that follows it. " +
"Arrives with the downloaded artifact, so rebuilding it " +
"costs a dump stream and it is never swept.",
},
{
Name: "artist_credit_ref", Kind: Cache, Lifetime: Retained,
Note: "Which credit a catalog recording or release group is " +
"credited to. Present only for multi-artist credits; " +
"absence means one artist, which explore_index already " +
"describes. Ships and dies with artist_credit_part.",
},
{
Name: "explore_index", Kind: Cache, Lifetime: Retained,
Note: "The offline MusicBrainz search index. Rebuilding costs a " +
+15 -16
View File
@@ -199,24 +199,23 @@ func TestManagerEndToEndAutoPick(t *testing.T) {
t.Errorf("expected imported file at %s: %v", want, err)
}
// Staging was released only after a successful import.
entries, err := os.ReadDir(f.staging.Root())
if err != nil {
t.Fatalf("read staging root: %v", err)
}
// Staging release and the rescan happen *after* the state is
// recorded (manager.go sets StateComplete, then releases, then
// scans), so waiting on the state is not waiting on these. Under
// load the worker is descheduled in between and asserting straight
// away reads the world one step too early -- which is exactly how
// this test failed on a busy machine while passing alone.
waitFor(t, func() bool {
entries, err := os.ReadDir(f.staging.Root())
if err != nil || len(entries) != 0 {
return false
}
if len(entries) != 0 {
t.Errorf("staging not released: %d dirs remain", len(entries))
}
f.lib.mu.Lock()
defer f.lib.mu.Unlock()
// The library was told to rescan.
f.lib.mu.Lock()
scanned := len(f.lib.scanned)
f.lib.mu.Unlock()
if scanned != 1 {
t.Errorf("library scans = %d, want 1", scanned)
}
return len(f.lib.scanned) == 1
}, "staging was never released, or the library was never rescanned")
}
// An ambiguous result set must park for the user rather than guess.
+24
View File
@@ -20,6 +20,30 @@ func newServiceFixture(t *testing.T) serviceFixture {
mf := newManagerFixture(t)
svc := NewService(slogDiscard(), mf.manager, mf.store, NewMemSecretStore())
// Every test here is about the durable Request that `StartDownload`
// leaves behind, and none of them is about the download itself -- but
// their fixture is an anchored four-track request with a healthy
// provider, which is exactly what `AutoPickable` says yes to. So
// `Manager.Start` was firing `go m.grab(...)`, detached and with
// `context.WithoutCancel`, and the test then raced it.
//
// It lost, twice, in CI (`check` on c03c0b8, and nowhere locally):
//
// service_test.go:66: state = "satisfied", want wanted
// testing.go:1369: TempDir RemoveAll cleanup: ... directory not empty
//
// The first is the request reaching its *next* state before the
// assertion read it; the second is that same goroutine still writing
// into `t.TempDir()` after the test returned. One cause, two shapes.
//
// Putting the candidate outside the auto-pick size window stops the
// grab from ever starting, which is better than waiting for it: there
// is no goroutine to be slow, so the tests state what they mean
// ("the request exists, in this state") without a timing assumption
// underneath. A test that does want the download has `managerFixture`
// and sets its own preferences.
mf.manager.SetPreferences(AutoDownloadPrefs{MaxSizeMB: 1})
return serviceFixture{managerFixture: mf, svc: svc}
}
+14
View File
@@ -27,6 +27,20 @@ var artifactStageNames = [...]string{
// failure path is non-fatal by design: the caller falls back, and a
// fresh install with no network still gets its own library in Explore.
func (si *SearchIndex) tryCoreArtifact(ctx context.Context) error {
// Before anything is staged: ~0.6 GB is not a download to start on
// someone's cellular allowance without being asked (plan 016 B4).
// This is checked first so no job appears and no status changes --
// declining is a no-op, not a failure the user has to dismiss.
if si.netPolicy.refuses() {
si.logIndexJob(
jobs.LevelInfo,
"Skipping the catalog download on a metered connection. "+
"Enable it in Settings to download anyway.",
)
return ErrMeteredNetwork
}
si.mu.Lock()
si.buildStatus = IndexStatus{
Building: true,
+75
View File
@@ -283,6 +283,9 @@ func (si *SearchIndex) importCoreArtifact(ctx context.Context, path string) erro
}
merged, mergeErr := si.mergeArtifactRows(ctx, info.rows)
if mergeErr == nil {
si.mergeArtifactCredits(ctx)
}
if ftsSuspended {
start := time.Now()
@@ -473,3 +476,75 @@ func (si *SearchIndex) removeArtifactFile(path string) {
si.logger.Warn("core artifact: cleanup failed", "path", path, "error", err)
}
}
// artifactHasCredits reports whether the attached artifact carries the
// multi-artist credit tables.
//
// The same shape, and the same handle, as artifactHasTotals above: an
// artifact published before credits existed is still a perfectly good
// catalog, and there is one already out there. Selecting from a table
// that is not in it would fail an import that should have succeeded, so
// it is asked rather than assumed -- on the *writer*, because `core` is
// attached to that one connection and the read pool cannot see it.
func (si *SearchIndex) artifactHasCredits() bool {
var n int
err := si.db.QueryRowWriter(
`SELECT COUNT(*) FROM core.sqlite_master
WHERE type = 'table' AND name IN ('artist_credit_part', 'artist_credit_ref')`,
).Scan(&n)
return err == nil && n == 2
}
// mergeArtifactCredits copies the credit decomposition out of the
// attached artifact.
//
// Credits are replaced wholesale rather than merged: they are derived
// entirely from one dump build, they are keyed by ids that are only
// meaningful within the artifact that carried them, and a half-updated
// credit renders as the wrong artists rather than as missing ones.
//
// A failure here is logged and not returned. The catalog has already
// merged at this point, and a catalog without credits is the catalog
// this app had before them -- every credit falls back to its single
// artist, which is the same fallback an untagged file already gets.
func (si *SearchIndex) mergeArtifactCredits(ctx context.Context) {
if !si.artifactHasCredits() {
si.logger.Info("core artifact: no credit tables, keeping single-artist credits")
return
}
start := time.Now()
for _, stmt := range []string{
"DELETE FROM artist_credit_part",
"DELETE FROM artist_credit_ref",
`INSERT OR REPLACE INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
SELECT credit_id, position, artist_mbid, credited_name, join_phrase
FROM core.artist_credit_part`,
`INSERT OR REPLACE INTO artist_credit_ref (mbid, credit_id)
SELECT mbid, credit_id FROM core.artist_credit_ref`,
} {
if err := ctx.Err(); err != nil {
return
}
if _, err := si.db.ExecContext(stmt); err != nil {
si.logger.Warn("core artifact: credit merge failed", "error", err)
return
}
}
var refs int
_ = si.db.QueryRowWriter("SELECT COUNT(*) FROM artist_credit_ref").Scan(&refs)
si.logger.Info("core artifact: credits merged",
"entities", refs,
"elapsed", time.Since(start).Round(time.Millisecond),
)
}
+151
View File
@@ -3,6 +3,7 @@ package explore
import (
"context"
"database/sql"
"encoding/hex"
"os"
"path/filepath"
"strings"
@@ -597,3 +598,153 @@ func TestImportCoreArtifactReadsTotalsWhenPresent(t *testing.T) {
t.Errorf("TotalTracks = %d, want 0 (the catalog does not say)", old.TotalTracks)
}
}
// addArtifactCredits gives an artifact file the credit tables the
// exporter now writes, so the import path can be exercised against one
// that has them.
func addArtifactCredits(t *testing.T, path string) {
t.Helper()
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatalf("open artifact: %v", err)
}
defer func() { _ = db.Close() }()
for _, stmt := range []string{
`CREATE TABLE artist_credit_part (
credit_id INTEGER NOT NULL,
position INTEGER NOT NULL,
artist_mbid BLOB NOT NULL,
credited_name TEXT NOT NULL,
join_phrase TEXT NOT NULL DEFAULT '',
PRIMARY KEY (credit_id, position)
) WITHOUT ROWID`,
`CREATE TABLE artist_credit_ref (
mbid BLOB NOT NULL PRIMARY KEY,
credit_id INTEGER NOT NULL
) WITHOUT ROWID`,
} {
if _, err := db.Exec(stmt); err != nil {
t.Fatalf("create credit tables: %v", err)
}
}
// The packed form the catalog stores. uuid16/parseUUID live behind
// the indexbuild tag, so this file decodes for itself.
pack := func(mbid string) []byte {
raw, err := hex.DecodeString(strings.ReplaceAll(mbid, "-", ""))
if err != nil || len(raw) != 16 {
t.Fatalf("fixture MBID %q is not a UUID: %v", mbid, err)
}
return raw
}
a, b, rec := pack(artA), pack(artB), pack(recA)
for _, part := range [][]any{
{7, 0, a, "Artist A", " feat. "},
{7, 1, b, "Artist B", ""},
} {
if _, err := db.Exec(`INSERT INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
VALUES (?, ?, ?, ?, ?)`, part...); err != nil {
t.Fatalf("insert part: %v", err)
}
}
if _, err := db.Exec(
"INSERT INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)", rec, 7,
); err != nil {
t.Fatalf("insert ref: %v", err)
}
}
// TestImportCoreArtifactMergesCredits is the positive half of the
// compatibility pair: an artifact that carries credits delivers them,
// rendering back to the credit string they decompose.
func TestImportCoreArtifactMergesCredits(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
path := writeTestArtifact(t, validMeta(), []artifactRow{
{"recording", recA, "Song A", "Artist A feat. Artist B", artA, 2000},
})
addArtifactCredits(t, path)
if err := si.importCoreArtifact(context.Background(), path); err != nil {
t.Fatalf("importCoreArtifact: %v", err)
}
rows, err := db.QueryContext(
`SELECT p.credited_name, p.join_phrase
FROM artist_credit_ref r
JOIN artist_credit_part p ON p.credit_id = r.credit_id
ORDER BY p.position`,
)
if err != nil {
t.Fatalf("query credits: %v", err)
}
defer func() { _ = rows.Close() }()
var rendered strings.Builder
for rows.Next() {
var name, join string
if err := rows.Scan(&name, &join); err != nil {
t.Fatalf("scan: %v", err)
}
rendered.WriteString(name)
rendered.WriteString(join)
}
if got := rendered.String(); got != "Artist A feat. Artist B" {
t.Errorf("rendered credit = %q, want %q", got, "Artist A feat. Artist B")
}
}
// TestImportCoreArtifactWithoutCredits is the regression that matters
// most here: an artifact published before credits existed cannot be
// re-cut retroactively, so it must import as a catalog that declines to
// answer rather than failing outright. writeTestArtifact deliberately
// builds one without the tables.
func TestImportCoreArtifactWithoutCredits(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
path := writeTestArtifact(t, validMeta(), []artifactRow{
{"recording", recA, "Song A", "Artist A", artA, 2000},
})
if err := si.importCoreArtifact(context.Background(), path); err != nil {
t.Fatalf("an artifact without credit tables must still import: %v", err)
}
var rows int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM explore_index",
).Scan(&rows); err != nil {
t.Fatalf("count: %v", err)
}
if rows != 1 {
t.Errorf("catalog rows = %d, want 1", rows)
}
var refs int
if err := db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_credit_ref",
).Scan(&refs); err != nil {
t.Fatalf("count refs: %v", err)
}
if refs != 0 {
t.Errorf("credit refs = %d, want 0", refs)
}
}
+132
View File
@@ -0,0 +1,132 @@
package explore
import (
"fmt"
"strings"
)
// Reading multi-artist credits back out of the catalog.
//
// The tables are filled centrally (backend/explore/dumpcredits.go, and
// the artifact import) and hold only credits naming more than one
// artist: an entity with no rows here is credited to one artist, which
// explore_index's own artist_name and artist_mbid already describe.
// Absence is the common case and means "nothing to decompose", never
// "unknown".
//
// The lookup is keyed on the *recording* MBID, which both sides of the
// app already have -- a catalog row carries it and so does a local
// file (library.Track.RecordingMBID) -- so one query serves the Explore
// pages and the library's own lists without either needing to know
// where the other gets its rows.
// CreditPart is one credited artist within a credit, in credit order.
//
// CreditedName is the name *as credited*, which is not the artist's own
// name: MusicBrainz credits "Snoop Dogg" on a track by the artist
// called "Snoop Doggy Dogg". Display uses it; navigation uses
// ArtistMBID. JoinPhrase is the literal connector that follows this
// part, so a credit renders by concatenation and never by searching a
// name inside a credit string.
type CreditPart struct {
Position int `json:"position"`
ArtistMBID string `json:"artistMbid"`
CreditedName string `json:"creditedName"`
JoinPhrase string `json:"joinPhrase"`
}
// creditLookupBatch bounds how many MBIDs go into one IN clause. A
// tracklist is the caller here, so the realistic ceiling is a few
// hundred; the bound exists so a 50,000-row selection cannot build a
// statement SQLite refuses to parse.
const creditLookupBatch = 500
// GetCredits returns the decomposition of every multi-artist credit
// among the given entity MBIDs, keyed by MBID.
//
// MBIDs with a single-artist credit are simply absent from the result,
// which is what the caller wants: it renders its existing single link
// for those, and that is the same answer it would have rendered anyway.
func (si *SearchIndex) GetCredits(mbids []string) (map[string][]CreditPart, error) {
out := make(map[string][]CreditPart)
for start := 0; start < len(mbids); start += creditLookupBatch {
end := min(start+creditLookupBatch, len(mbids))
if err := si.appendCredits(mbids[start:end], out); err != nil {
return nil, err
}
}
return out, nil
}
// appendCredits runs one batch into the accumulating result.
func (si *SearchIndex) appendCredits(
mbids []string, out map[string][]CreditPart,
) error {
args := make([]any, 0, len(mbids))
holders := make([]string, 0, len(mbids))
for _, mbid := range mbids {
if mbid == "" {
continue
}
args = append(args, dbMBID(mbid))
holders = append(holders, "?")
}
if len(args) == 0 {
return nil
}
// Ordered by position because that ordering *is* the credit's
// meaning; the caller concatenates in the order it receives.
rows, err := si.db.QueryContext(
`SELECT r.mbid, p.position, p.artist_mbid, p.credited_name, p.join_phrase
FROM artist_credit_ref r
JOIN artist_credit_part p ON p.credit_id = r.credit_id
WHERE r.mbid IN (`+strings.Join(holders, ",")+`)
ORDER BY r.mbid, p.position`,
args...,
)
if err != nil {
return fmt.Errorf("read artist credits: %w", err)
}
defer func() { _ = rows.Close() }()
for rows.Next() {
var (
entity dbMBID
artist dbMBID
part CreditPart
)
if err := rows.Scan(
&entity, &part.Position, &artist, &part.CreditedName, &part.JoinPhrase,
); err != nil {
return fmt.Errorf("scan artist credit: %w", err)
}
part.ArtistMBID = string(artist)
out[string(entity)] = append(out[string(entity)], part)
}
if err := rows.Err(); err != nil {
return fmt.Errorf("read artist credits: %w", err)
}
return nil
}
// GetCredits is the bound form: the frontend asks for a tracklist's
// worth of MBIDs at once rather than one per row.
//
// Batched for the reason every other per-row backend question here is:
// asking on hover or on render turns a list into N IPC round trips, and
// this one is asked about every row of every list in the app.
func (e *Service) GetCredits(mbids []string) (map[string][]CreditPart, error) {
return e.index.GetCredits(mbids)
}
+128
View File
@@ -0,0 +1,128 @@
package explore
import (
"encoding/hex"
"fmt"
"strings"
"testing"
"yellowjacket/backend/database"
)
// seedCredit writes one multi-artist credit and points an entity at it,
// the way the dump import and the artifact import both do.
func seedCredit(t *testing.T, db *database.DB, entity string, id int, parts []CreditPart) {
t.Helper()
pack := func(mbid string) []byte {
raw, err := hex.DecodeString(strings.ReplaceAll(mbid, "-", ""))
if err != nil || len(raw) != 16 {
t.Fatalf("bad fixture mbid %q: %v", mbid, err)
}
return raw
}
if _, err := db.ExecContext(
"INSERT INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)",
pack(entity), id,
); err != nil {
t.Fatalf("seed ref: %v", err)
}
for _, p := range parts {
if _, err := db.ExecContext(
`INSERT INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
VALUES (?, ?, ?, ?, ?)`,
id, p.Position, pack(p.ArtistMBID), p.CreditedName, p.JoinPhrase,
); err != nil {
t.Fatalf("seed part: %v", err)
}
}
}
// TestGetCreditsDecomposes: the parts come back in position order and
// concatenate to the credit they describe.
func TestGetCreditsDecomposes(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
rec := testMBID("rec-1")
a, b := testMBID("artist-a"), testMBID("artist-b")
seedCredit(t, db, rec, 7, []CreditPart{
{Position: 0, ArtistMBID: a, CreditedName: "2Pac", JoinPhrase: " feat. "},
{Position: 1, ArtistMBID: b, CreditedName: "Snoop Dogg"},
})
got, err := si.GetCredits([]string{rec})
if err != nil {
t.Fatalf("GetCredits: %v", err)
}
parts := got[rec]
if len(parts) != 2 {
t.Fatalf("parts = %d, want 2", len(parts))
}
var rendered strings.Builder
for _, p := range parts {
rendered.WriteString(p.CreditedName)
rendered.WriteString(p.JoinPhrase)
}
if rendered.String() != "2Pac feat. Snoop Dogg" {
t.Errorf("rendered = %q, want %q", rendered.String(), "2Pac feat. Snoop Dogg")
}
// Dashed on the way out: a blob reaching the frontend is sixteen
// bytes of mojibake, and nothing above mbid.go speaks that.
if parts[0].ArtistMBID != a {
t.Errorf("artist mbid = %q, want %q", parts[0].ArtistMBID, a)
}
}
// TestGetCreditsOmitsSingleArtist: absence is the common case and means
// "nothing to decompose", so the caller renders its existing one link.
func TestGetCreditsOmitsSingleArtist(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
got, err := si.GetCredits([]string{testMBID("untagged"), ""})
if err != nil {
t.Fatalf("GetCredits: %v", err)
}
if len(got) != 0 {
t.Errorf("got %d credits, want none", len(got))
}
}
// TestGetCreditsBatches: the lookup is asked about whole tracklists, so
// it must not build one statement per row or one SQLite refuses to
// parse.
func TestGetCreditsBatches(t *testing.T) {
db := database.NewTestDB(t)
si := NewSearchIndex(db, nil, nil, testLogger())
mbids := make([]string, 0, creditLookupBatch*2+7)
for i := range creditLookupBatch*2 + 7 {
mbids = append(mbids, testMBID(fmt.Sprintf("batch-%d", i)))
}
// One real credit somewhere past the first batch boundary.
seedCredit(t, db, mbids[creditLookupBatch+3], 9, []CreditPart{
{Position: 0, ArtistMBID: testMBID("a"), CreditedName: "A", JoinPhrase: " & "},
{Position: 1, ArtistMBID: testMBID("b"), CreditedName: "B"},
})
got, err := si.GetCredits(mbids)
if err != nil {
t.Fatalf("GetCredits: %v", err)
}
if len(got[mbids[creditLookupBatch+3]]) != 2 {
t.Errorf("a credit past the first batch boundary was not returned")
}
}
+637
View File
@@ -0,0 +1,637 @@
//go:build indexbuild
package explore
import (
"archive/tar"
"bufio"
"compress/bzip2"
"context"
"errors"
"fmt"
"io"
"path"
"regexp"
"strconv"
"strings"
)
// Multi-artist credits, from the core MusicBrainz dump.
//
// A credit is ordered parts and the credit *string* is derived from
// them; MusicBrainz's own artist_credit.name is a cached render. What
// this pass extracts is the decomposition: for each catalog recording
// and release group whose credit names more than one artist, the
// credited artists in order, each with the name *as credited* and the
// join phrase that follows it. See artist_credit_part.sql for why that
// is stored rather than derived, and why nothing may reconstruct a
// credit by searching a name inside a credit string.
//
// It is a separate dump from everything else here, and it has to be.
// The canonical dump this importer already streams gives artist_mbids
// (an ordered list) and artist_credit_name (the *rendered* string) --
// no join phrases, and no per-artist as-credited names. Splitting the
// rendered string using canonical artist names fails on exactly the
// credits that matter: measured on a real library, 21% of multi-artist
// credits name an artist differently from the artist's own name
// ("Snoop Dogg" credited on a track by "Snoop Doggy Dogg"), so the
// substring is simply not there. The JSON dumps were checked too and
// cover 153,691 recordings of ~35M, with zero overlap against a real
// library. This dump is the only source.
//
// Cost, measured on the 20260815 export: 7.1 GB compressed, decompressed
// by pure-Go compress/bzip2 at ~26 MB/s uncompressed (~13.7 min for the
// whole file, single-threaded). cmd/indexbuild is built CGO_ENABLED=0,
// so the stdlib decompressor is what there is -- and it is fine, because
// the 2 MB/s origin throttle dominates, as it does for every other dump
// here.
const (
// defaultMBDumpBaseURL is the core MusicBrainz export. Only
// mbdump.tar.bz2 is fetched; the other tarballs there hold data this
// app has no use for.
defaultMBDumpBaseURL = "https://data.metabrainz.org/pub/musicbrainz/data/fullexport/"
)
var (
mbdumpDirRe = regexp.MustCompile(`^\d{8}-\d+$`)
mbdumpFileRe = regexp.MustCompile(`^mbdump\.tar\.bz2$`)
// ErrDumpShape is returned when a dump member does not have the
// columns this code was written against. It is deliberately fatal:
// reading the wrong column silently produces a catalog whose credits
// are subtly wrong, which is far worse than a failed build.
ErrDumpShape = errors.New("musicbrainz dump member has an unexpected shape")
)
// Column positions in the Postgres COPY output, verified against the
// 20260815 export. There is no header row to read them from, so they
// are asserted instead -- see checkShape.
const (
artistColID = 0
artistColGID = 1
artistColMin = 2
creditColID = 0
creditColArtistCount = 2
creditColMin = 3
partColCredit = 0
partColPosition = 1
partColArtist = 2
partColName = 3
partColJoin = 4
partColMin = 5
// recording and release_group share a layout in the columns this
// pass reads: id, gid, name, artist_credit, ...
entityColGID = 1
entityColCredit = 3
entityColMin = 4
)
// creditPart is one credited artist within a credit.
type creditPart struct {
position int
artistID int32
name string
join string
}
// creditScan is what one pass over the dump collects.
type creditScan struct {
// artistGIDs maps an artist row id to its MBID. artist_credit_name
// references artists by row id, and the tar orders `artist` before
// it, so this is complete by the time it is read.
artistGIDs map[int32]uuid16
// multiCredits are the credit ids naming more than one artist, from
// artist_credit.artist_count. Taking the count from the dump rather
// than counting parts means a credit can be rejected before its
// parts are stored.
multiCredits map[int32]struct{}
// parts are the decompositions of multiCredits, keyed by credit id.
parts map[int32][]creditPart
// refs maps a kept catalog entity to its credit. Only entities in
// explore_index and only multi-artist credits: everything else is
// already described by explore_index's own artist_name/artist_mbid.
refs map[uuid16]int32
// used are the credits some ref actually points at, which is a small
// fraction of multiCredits -- the catalog keeps ~1.8M entities of
// MusicBrainz's tens of millions.
used map[int32]struct{}
skippedUnknownArtist int
}
// creditsImportDoneKey marks in explore_index_meta that the credit pass
// has run against the current catalog.
//
// It is its own marker rather than part of the import's stage state for
// a resume reason: the credit pass runs *after* the catalog is
// assembled, and a failure in it must not send the next run back
// through the ~205 GB it just finished. Marking separately means a
// retry retries only this.
const creditsImportDoneKey = "credits_import_done"
// ensureArtistCredits runs the credit pass unless it has already run
// against this catalog, reporting whether it newly populated them.
//
// Called from both of run's paths -- the full import and the resume
// that finds the rows already assembled -- and from the maintenance
// entry point below, since a catalog built before credits existed is
// otherwise never offered a chance to gain them: the index job picks
// its mode from the index's own state, and a complete import means
// "refresh", which never enters run() at all.
//
// The return value is what tells the job there is something new worth
// publishing. A refresh otherwise reports "changed" only when the
// listens series advanced, so credits would sit in the CI database and
// never reach an artifact.
func (imp *dumpImporter) ensureArtistCredits(ctx context.Context) bool {
if imp.si.hasMeta(creditsImportDoneKey) {
return false
}
url, err := discoverDumpFile(
ctx, imp.httpClient, imp.mbdumpBaseURL, mbdumpDirRe, mbdumpFileRe,
)
if err != nil {
imp.logger.Warn("credit import: could not find the dump", "error", err)
return false
}
if err := imp.importArtistCredits(ctx, url); err != nil {
// A catalog without credits is the catalog this app shipped
// before them: every credit falls back to its single artist.
// That is worth far less than failing an import that otherwise
// succeeded.
imp.logger.Warn("credit import: failed", "error", err)
return false
}
imp.si.setMeta(creditsImportDoneKey, "1")
return true
}
// EnsureArtistCredits tops up the credit tables outside a full import.
//
// It exists because the index job's modes are decided from the index's
// own state: a cache holding a completed import chooses `refresh`,
// which folds in incremental listens and never enters the dump
// importer. Without this, a catalog built before the credit pass
// existed could only gain credits from a `rebuild` -- and a rebuild
// re-downloads ~205 GB to reproduce rows it already has, to add
// something that costs 7 GB on its own.
//
// Reports whether credits were newly populated, so the caller knows
// there is a new artifact worth publishing.
func (e *Service) EnsureArtistCredits(ctx context.Context) bool {
imp, err := newDumpImporter(e.index, e.lb)
if err != nil {
e.index.logger.Warn("credit import: could not start", "error", err)
return false
}
return imp.ensureArtistCredits(ctx)
}
// importArtistCredits streams the core MusicBrainz dump and fills
// artist_credit_part and artist_credit_ref for the entities the catalog
// kept.
//
// It runs after assembleIndex because it asks explore_index which
// entities those are: the popularity filter decides what is worth
// carrying credits for, and asking the table rather than the kept sets
// means this stays correct if that filter changes.
func (imp *dumpImporter) importArtistCredits(ctx context.Context, url string) error {
kept, err := imp.keptEntityMBIDs(ctx)
if err != nil {
return err
}
if len(kept) == 0 {
imp.logger.Warn("credit import: no catalog entities, skipping")
return nil
}
imp.logger.Info("credit import: starting", "url", url, "entities", len(kept))
imp.logJob("Streaming MusicBrainz dump for artist credits")
scan, err := imp.scanCreditDump(ctx, url, kept)
if err != nil {
return err
}
imp.logger.Info("credit import: scanned",
"multiArtistCredits", len(scan.multiCredits),
"entitiesWithMultiArtistCredit", len(scan.refs),
"creditsUsed", len(scan.used),
)
return imp.writeCredits(ctx, scan)
}
// keptEntityMBIDs is every recording and release group in the catalog.
// Artists are excluded: an artist is not credited to a credit.
func (imp *dumpImporter) keptEntityMBIDs(ctx context.Context) (map[uuid16]struct{}, error) {
rows, err := imp.si.db.QueryContextWith(ctx,
`SELECT mbid FROM explore_index
WHERE entity_type IN (2 /* release_group */, 3 /* recording */)`,
)
if err != nil {
return nil, fmt.Errorf("credit import: read catalog entities: %w", err)
}
defer func() { _ = rows.Close() }()
out := make(map[uuid16]struct{})
for rows.Next() {
var raw []byte
if err := rows.Scan(&raw); err != nil {
return nil, fmt.Errorf("credit import: scan mbid: %w", err)
}
if len(raw) != len(uuid16{}) {
continue
}
var id uuid16
copy(id[:], raw)
out[id] = struct{}{}
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("credit import: read catalog entities: %w", err)
}
return out, nil
}
// scanCreditDump makes one sequential pass over mbdump.tar.bz2.
//
// The tar's members are alphabetical, which is what makes a single pass
// possible without buffering the big ones: `artist` and
// `artist_credit_name` both arrive before `recording` and
// `release_group`, so by the time an entity names a credit, that
// credit's parts and their artists' MBIDs are already known and the
// entity can be resolved and dropped. 35M recording rows are never
// held.
//
// The order is not depended on blindly: an entity naming a credit that
// has not been seen is counted and reported rather than silently
// producing an empty catalog, which is what a reordered export would
// otherwise look like.
func (imp *dumpImporter) scanCreditDump(
ctx context.Context, url string, kept map[uuid16]struct{},
) (*creditScan, error) {
stream := imp.openDumpStream(ctx, url, 0)
defer func() { _ = stream.Close() }()
return imp.scanCreditTar(
ctx,
tar.NewReader(bzip2.NewReader(bufio.NewReaderSize(stream, 1<<20))),
kept,
)
}
// scanCreditTar is the parse, separated from the fetch so it can be
// driven by a tar built in a test. compress/bzip2 is decompress-only,
// so a test cannot produce the real container.
func (imp *dumpImporter) scanCreditTar(
ctx context.Context, tr *tar.Reader, kept map[uuid16]struct{},
) (*creditScan, error) {
scan := &creditScan{
artistGIDs: make(map[int32]uuid16),
multiCredits: make(map[int32]struct{}),
parts: make(map[int32][]creditPart),
refs: make(map[uuid16]int32),
used: make(map[int32]struct{}),
}
for {
if err := ctx.Err(); err != nil {
return nil, err
}
hdr, err := tr.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("credit import: tar: %w", err)
}
if hdr.Typeflag != tar.TypeReg {
continue
}
done, err := imp.scanCreditMember(ctx, hdr.Name, tr, kept, scan)
if err != nil {
return nil, err
}
if done {
// Everything this pass needs has been read; the rest of the
// tarball is other entities' data and decompressing it would
// cost minutes for nothing.
break
}
}
if scan.skippedUnknownArtist > 0 {
imp.logger.Warn("credit import: credits dropped for unknown artists",
"count", scan.skippedUnknownArtist,
)
}
return scan, nil
}
// scanCreditMember dispatches one tar member, reporting whether the
// pass has everything it needs.
func (imp *dumpImporter) scanCreditMember(
ctx context.Context, name string, r io.Reader,
kept map[uuid16]struct{}, scan *creditScan,
) (bool, error) {
switch path.Base(name) {
case "artist":
return false, imp.scanArtists(ctx, r, scan)
case "artist_credit":
return false, imp.scanCredits(ctx, r, scan)
case "artist_credit_name":
return false, imp.scanCreditParts(ctx, r, scan)
case "recording", "release_group":
if err := imp.scanCreditedEntities(ctx, r, kept, scan); err != nil {
return false, err
}
// release_group sorts after recording, so the pass is complete
// once it has been read.
return path.Base(name) == "release_group", nil
default:
return false, nil
}
}
// scanArtists records every artist's MBID by row id.
func (imp *dumpImporter) scanArtists(
ctx context.Context, r io.Reader, scan *creditScan,
) error {
return scanTSV(ctx, r, artistColMin, "artist", func(fields []string) error {
id, ok := parseInt32(fields[artistColID])
if !ok {
return nil
}
var gid uuid16
if !parseUUID(fields[artistColGID], gid[:]) {
return fmt.Errorf("%w: artist.gid is not a UUID: %q",
ErrDumpShape, truncate(fields[artistColGID]))
}
scan.artistGIDs[id] = gid
return nil
})
}
// scanCredits records which credits name more than one artist.
func (imp *dumpImporter) scanCredits(
ctx context.Context, r io.Reader, scan *creditScan,
) error {
return scanTSV(ctx, r, creditColMin, "artist_credit", func(fields []string) error {
id, ok := parseInt32(fields[creditColID])
if !ok {
return nil
}
count, ok := parseInt32(fields[creditColArtistCount])
if !ok {
return fmt.Errorf("%w: artist_credit.artist_count is not a number: %q",
ErrDumpShape, truncate(fields[creditColArtistCount]))
}
if count > 1 {
scan.multiCredits[id] = struct{}{}
}
return nil
})
}
// scanCreditParts records the decomposition of every multi-artist
// credit.
func (imp *dumpImporter) scanCreditParts(
ctx context.Context, r io.Reader, scan *creditScan,
) error {
return scanTSV(ctx, r, partColMin, "artist_credit_name", func(fields []string) error {
credit, ok := parseInt32(fields[partColCredit])
if !ok {
return nil
}
if _, multi := scan.multiCredits[credit]; !multi {
return nil
}
position, ok := parseInt32(fields[partColPosition])
if !ok {
return nil
}
artist, ok := parseInt32(fields[partColArtist])
if !ok {
return nil
}
scan.parts[credit] = append(scan.parts[credit], creditPart{
position: int(position),
artistID: artist,
name: fields[partColName],
join: fields[partColJoin],
})
return nil
})
}
// scanCreditedEntities resolves recordings and release groups against
// the catalog, keeping only those the catalog holds and whose credit
// names more than one artist.
func (imp *dumpImporter) scanCreditedEntities(
ctx context.Context, r io.Reader, kept map[uuid16]struct{}, scan *creditScan,
) error {
return scanTSV(ctx, r, entityColMin, "recording/release_group",
func(fields []string) error {
var gid uuid16
if !parseUUID(fields[entityColGID], gid[:]) {
return fmt.Errorf("%w: entity gid is not a UUID: %q",
ErrDumpShape, truncate(fields[entityColGID]))
}
if _, want := kept[gid]; !want {
return nil
}
credit, ok := parseInt32(fields[entityColCredit])
if !ok {
return fmt.Errorf("%w: entity artist_credit is not a number: %q",
ErrDumpShape, truncate(fields[entityColCredit]))
}
if _, multi := scan.multiCredits[credit]; !multi {
return nil
}
scan.refs[gid] = credit
scan.used[credit] = struct{}{}
return nil
})
}
// scanTSV reads Postgres COPY output a line at a time, unescaping each
// field and handing the row to fn.
//
// The shape is asserted on the first row rather than trusted: this dump
// has no header, so a column that moved would otherwise be read as a
// neighbouring one and produce a catalog that is quietly wrong.
func scanTSV(
ctx context.Context, r io.Reader, minCols int, member string,
fn func(fields []string) error,
) error {
sc := bufio.NewScanner(r)
sc.Buffer(make([]byte, 0, 1<<20), 1<<24)
checked := false
rows := 0
for sc.Scan() {
rows++
if rows%(1<<20) == 0 {
if err := ctx.Err(); err != nil {
return err
}
}
line := sc.Text()
if line == "" {
continue
}
fields := strings.Split(line, "\t")
if len(fields) < minCols {
if !checked {
return fmt.Errorf("%w: %s has %d columns, need at least %d",
ErrDumpShape, member, len(fields), minCols)
}
continue
}
checked = true
for i := range fields {
fields[i] = unescapeCopy(fields[i])
}
if err := fn(fields); err != nil {
return err
}
}
if err := sc.Err(); err != nil {
return fmt.Errorf("credit import: read %s: %w", member, err)
}
return nil
}
// unescapeCopy undoes Postgres COPY's text escaping. A NULL (\N) is
// returned as an empty string: every field this pass reads is either a
// number it will reject or a name whose absence means the same as
// empty.
func unescapeCopy(s string) string {
if s == `\N` {
return ""
}
if !strings.ContainsRune(s, '\\') {
return s
}
var b strings.Builder
b.Grow(len(s))
for i := 0; i < len(s); i++ {
if s[i] != '\\' || i+1 >= len(s) {
b.WriteByte(s[i])
continue
}
i++
switch s[i] {
case 'n':
b.WriteByte('\n')
case 't':
b.WriteByte('\t')
case 'r':
b.WriteByte('\r')
case 'b':
b.WriteByte('\b')
case 'f':
b.WriteByte('\f')
case 'v':
b.WriteByte('\v')
case '\\':
b.WriteByte('\\')
default:
b.WriteByte('\\')
b.WriteByte(s[i])
}
}
return b.String()
}
func parseInt32(s string) (int32, bool) {
n, err := strconv.ParseInt(s, 10, 32)
if err != nil {
return 0, false
}
return int32(n), true
}
// truncate bounds an error message built from dump data, which is
// attacker-free but can be long.
func truncate(s string) string {
const limit = 64
if len(s) <= limit {
return s
}
return s[:limit] + "..."
}
+406
View File
@@ -0,0 +1,406 @@
//go:build indexbuild
package explore
import (
"archive/tar"
"bytes"
"context"
"errors"
"strings"
"testing"
"yellowjacket/backend/database"
)
// tarOf builds an uncompressed tar of the named members, in the order
// given. Order is the point of several of these tests: the real dump's
// members are alphabetical, which is what lets one pass resolve an
// entity's credit without buffering 35M recordings.
func tarOf(t *testing.T, members ...[2]string) *tar.Reader {
t.Helper()
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
for _, m := range members {
body := []byte(m[1])
if err := tw.WriteHeader(&tar.Header{
Name: "mbdump/" + m[0],
Mode: 0o644,
Size: int64(len(body)),
Typeflag: tar.TypeReg,
}); err != nil {
t.Fatalf("tar header: %v", err)
}
if _, err := tw.Write(body); err != nil {
t.Fatalf("tar write: %v", err)
}
}
if err := tw.Close(); err != nil {
t.Fatalf("tar close: %v", err)
}
return tar.NewReader(&buf)
}
func tsv(rows ...[]string) string {
var b strings.Builder
for _, r := range rows {
b.WriteString(strings.Join(r, "\t"))
b.WriteByte('\n')
}
return b.String()
}
// mustMBID is testMBID in the packed form the catalog stores.
func mustMBID(label string) uuid16 {
var u uuid16
if !parseUUID(testMBID(label), u[:]) {
panic("testMBID did not produce a UUID for " + label)
}
return u
}
// The two artists of the worked example, and the entities they credit.
var (
creditRecMBID = mustMBID("recording-1")
creditRGMBID = mustMBID("release-group-1")
)
// sampleDump is the shape verified against the 20260815 export:
// artist(id, gid, ...), artist_credit(id, name, artist_count, ...),
// artist_credit_name(credit, position, artist, name, join_phrase),
// recording/release_group(id, gid, name, artist_credit, ...).
func sampleDump(t *testing.T) *tar.Reader {
t.Helper()
return tarOf(t,
[2]string{"artist", tsv(
[]string{"11", testMBID("artist-a"), "Snoop Doggy Dogg", "Snoop Doggy Dogg"},
[]string{"22", testMBID("artist-b"), "2Pac", "2Pac"},
)},
[2]string{"artist_credit", tsv(
[]string{"900", "2Pac feat. Snoop Dogg", "2", "1", "", "0", ""},
[]string{"901", "Solo Artist", "1", "1", "", "0", ""},
)},
[2]string{"artist_credit_name", tsv(
// Deliberately out of position order: the dump is not
// obliged to emit them sorted and the credit's meaning is
// the order, not the file's.
[]string{"900", "1", "11", "Snoop Dogg", ""},
[]string{"900", "0", "22", "2Pac", " feat. "},
[]string{"901", "0", "11", "Solo Artist", ""},
)},
[2]string{"recording", tsv(
[]string{"1", testMBID("recording-1"), "Some Song", "900", "180000"},
[]string{"2", testMBID("not-kept"), "Other", "900", "1"},
[]string{"3", testMBID("solo"), "Solo", "901", "1"},
)},
[2]string{"release_group", tsv(
[]string{"5", testMBID("release-group-1"), "Some Album", "900", "1"},
)},
)
}
func creditTestImporter(t *testing.T) *dumpImporter {
t.Helper()
db := database.NewTestDB(t)
return &dumpImporter{
si: NewSearchIndex(db, nil, nil, testLogger()),
logger: testLogger(),
}
}
// TestScanCreditDumpDecomposes is the worked example end to end: the
// credit's parts come back in position order, with the *credited*
// names and the join phrase between them.
func TestScanCreditDumpDecomposes(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{
creditRecMBID: {},
creditRGMBID: {},
}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if got := len(scan.refs); got != 2 {
t.Fatalf("refs = %d, want 2 (the recording and the release group)", got)
}
if scan.refs[creditRecMBID] != 900 {
t.Errorf("recording credit = %d, want 900", scan.refs[creditRecMBID])
}
parts := scan.parts[900]
if len(parts) != 2 {
t.Fatalf("parts = %d, want 2", len(parts))
}
// Sorting happens on write, so assert the pieces are all present
// and let the render test below check the order.
byPos := map[int]creditPart{}
for _, p := range parts {
byPos[p.position] = p
}
if byPos[0].name != "2Pac" || byPos[0].join != " feat. " {
t.Errorf("position 0 = %q/%q, want \"2Pac\"/\" feat. \"",
byPos[0].name, byPos[0].join)
}
// The credited name, not the artist's own name: this is the whole
// reason credited_name is stored per row.
if byPos[1].name != "Snoop Dogg" {
t.Errorf("position 1 credited name = %q, want \"Snoop Dogg\"", byPos[1].name)
}
}
// TestSingleArtistCreditsAreNotStored: a one-artist credit is already
// described by explore_index's artist_name/artist_mbid, and storing it
// would roughly triple the table to say nothing new.
func TestSingleArtistCreditsAreNotStored(t *testing.T) {
imp := creditTestImporter(t)
solo := mustMBID("solo")
kept := map[uuid16]struct{}{solo: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if len(scan.refs) != 0 {
t.Fatalf("a single-artist credit was referenced: %v", scan.refs)
}
if _, ok := scan.multiCredits[901]; ok {
t.Error("credit 901 has artist_count 1 and should not be multi")
}
}
// TestOnlyKeptEntitiesAreReferenced: the catalog's popularity filter
// decides what is worth carrying credits for, and an entity outside it
// must not produce a row pointing at nothing.
func TestOnlyKeptEntitiesAreReferenced(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{creditRecMBID: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if _, ok := scan.refs[mustMBID("not-kept")]; ok {
t.Error("an entity outside the catalog was referenced")
}
if len(scan.used) != 1 {
t.Errorf("used credits = %d, want 1", len(scan.used))
}
}
// TestWriteCreditsRoundTrips checks what the frontend will actually
// read: parts in position order, dashed MBIDs out of the 16 raw bytes,
// and a rendered credit that reassembles to the tagged string.
func TestWriteCreditsRoundTrips(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{creditRecMBID: {}, creditRGMBID: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
if err := imp.writeCredits(context.Background(), scan); err != nil {
t.Fatalf("writeCredits: %v", err)
}
rows, err := imp.si.db.QueryContext(
`SELECT p.position, p.artist_mbid, p.credited_name, p.join_phrase
FROM artist_credit_ref r
JOIN artist_credit_part p ON p.credit_id = r.credit_id
WHERE r.mbid = ?
ORDER BY p.position`,
creditRecMBID[:],
)
if err != nil {
t.Fatalf("query: %v", err)
}
defer func() { _ = rows.Close() }()
var rendered strings.Builder
names := []string{}
for rows.Next() {
var (
pos int
mbid []byte
name string
join string
)
if err := rows.Scan(&pos, &mbid, &name, &join); err != nil {
t.Fatalf("scan row: %v", err)
}
if len(mbid) != 16 {
t.Fatalf("artist_mbid is %d bytes, want 16", len(mbid))
}
names = append(names, name)
rendered.WriteString(name)
rendered.WriteString(join)
}
if err := rows.Err(); err != nil {
t.Fatalf("rows: %v", err)
}
// Concatenation is the contract: names in order, join phrases
// between them, and no searching a name inside a credit string.
if got := rendered.String(); got != "2Pac feat. Snoop Dogg" {
t.Errorf("rendered credit = %q, want %q", got, "2Pac feat. Snoop Dogg")
}
if len(names) != 2 || names[0] != "2Pac" {
t.Errorf("parts came back out of position order: %v", names)
}
}
// TestCreditRefsNeverDangle: a ref whose parts were not stored renders
// as a credit with no artists at all, which is worse than the
// single-artist fallback it replaced.
func TestCreditRefsNeverDangle(t *testing.T) {
imp := creditTestImporter(t)
kept := map[uuid16]struct{}{creditRecMBID: {}}
scan, err := imp.scanCreditTar(context.Background(), sampleDump(t), kept)
if err != nil {
t.Fatalf("scan: %v", err)
}
// An artist the dump never named: the credit cannot be navigated to
// and must be dropped whole, taking its ref with it.
scan.artistGIDs = map[int32]uuid16{}
if err := imp.writeCredits(context.Background(), scan); err != nil {
t.Fatalf("writeCredits: %v", err)
}
var refs, parts int
if err := imp.si.db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_credit_ref",
).Scan(&refs); err != nil {
t.Fatalf("count refs: %v", err)
}
if err := imp.si.db.QueryRowWriter(
"SELECT COUNT(*) FROM artist_credit_part",
).Scan(&parts); err != nil {
t.Fatalf("count parts: %v", err)
}
if refs != 0 || parts != 0 {
t.Fatalf("refs=%d parts=%d, want 0/0 when the artists are unknown", refs, parts)
}
}
// TestCreditDumpShapeIsAsserted: the dump has no header row, so a
// column that moved would be read as its neighbour and produce a
// catalog that is quietly wrong. Loud is the requirement.
func TestCreditDumpShapeIsAsserted(t *testing.T) {
imp := creditTestImporter(t)
short := tarOf(t, [2]string{"artist", tsv([]string{"11", "only-two-columns"})})
_, err := imp.scanCreditTar(context.Background(), short, map[uuid16]struct{}{})
if err == nil {
t.Fatal("a member with a non-UUID gid was accepted")
}
if !errors.Is(err, ErrDumpShape) {
t.Errorf("error = %v, want ErrDumpShape", err)
}
}
// TestUnescapeCopy covers Postgres COPY's text escaping, which reaches
// artist names routinely -- a tab or backslash in a name would
// otherwise shift every field after it.
func TestUnescapeCopy(t *testing.T) {
tests := []struct{ in, want string }{
{`plain`, `plain`},
{`\N`, ``},
{`a\tb`, "a\tb"},
{`a\nb`, "a\nb"},
{`back\\slash`, `back\slash`},
{`AC\/DC`, `AC\/DC`},
{`trailing\`, `trailing\`},
}
for _, tt := range tests {
if got := unescapeCopy(tt.in); got != tt.want {
t.Errorf("unescapeCopy(%q) = %q, want %q", tt.in, got, tt.want)
}
}
}
// TestEnsureArtistCreditsIsIdempotent pins what the index job depends
// on to decide whether to publish.
//
// The pass runs on every mode, including the `refresh` that a complete
// catalog always chooses — so it must be free when there is nothing to
// do, and it must say so. A `true` here republishes the artifact; a
// `true` on every run would republish an identical one weekly, and a
// permanent `false` would mean a catalog that never gains credits at
// all.
func TestEnsureArtistCreditsIsIdempotent(t *testing.T) {
imp := creditTestImporter(t)
// The marker is what "already done" means; with it set, the pass
// must not reach the network or report a change.
imp.si.setMeta(creditsImportDoneKey, "1")
if imp.ensureArtistCredits(context.Background()) {
t.Fatal("a second run reported new credits; the artifact would republish forever")
}
}
// TestEnsureArtistCreditsReportsFailureAsNoChange: a dump that cannot be
// reached leaves the catalog exactly as it was, and must not claim
// otherwise — publishing on it would ship an artifact with no credits
// and mark the work done.
func TestEnsureArtistCreditsReportsFailureAsNoChange(t *testing.T) {
imp := creditTestImporter(t)
imp.httpClient = newDumpHTTPClient()
imp.mbdumpBaseURL = "http://127.0.0.1:1/nonexistent/"
if imp.ensureArtistCredits(context.Background()) {
t.Fatal("an unreachable dump reported new credits")
}
if imp.si.hasMeta(creditsImportDoneKey) {
t.Error("a failed pass marked itself done; it would never retry")
}
}
+164
View File
@@ -0,0 +1,164 @@
//go:build indexbuild
package explore
import (
"context"
"database/sql"
"fmt"
"sort"
)
// writeCredits persists the scanned decompositions.
//
// Only credits some catalog entity actually points at are written: the
// dump has millions of multi-artist credits and the catalog keeps ~1.8M
// entities, so storing every credit would be most of a table nothing
// can reach.
//
// The two tables are written in one transaction, because a ref pointing
// at parts that are not there renders as a credit with no artists --
// worse than the single-artist fallback it replaced.
func (imp *dumpImporter) writeCredits(ctx context.Context, scan *creditScan) error {
tx, err := imp.si.db.BeginTx()
if err != nil {
return fmt.Errorf("credit import: begin: %w", err)
}
defer func() { _ = tx.Rollback() }()
// A rebuild replaces the previous pass wholesale. These are Cache
// tables derived entirely from the dump, so there is nothing to
// merge and a stale row is a wrong credit.
for _, table := range []string{"artist_credit_part", "artist_credit_ref"} {
if _, err := tx.ExecContext(ctx, "DELETE FROM "+table); err != nil {
return fmt.Errorf("credit import: clear %s: %w", table, err)
}
}
written, err := imp.writeCreditParts(ctx, tx, scan)
if err != nil {
return err
}
refs, err := imp.writeCreditRefs(ctx, tx, scan, written)
if err != nil {
return err
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("credit import: commit: %w", err)
}
imp.logger.Info("credit import: complete",
"credits", len(written),
"refs", refs,
)
return nil
}
// writeCreditParts inserts the parts of every used credit and returns
// the set of credits that were actually stored.
//
// A credit is stored whole or not at all. If any of its artists has no
// MBID -- which should not happen, the dump being self-consistent, but
// would leave a part that cannot be navigated to -- the credit is
// dropped and the entity falls back to explore_index's single artist,
// which is a worse answer rather than a broken one.
func (imp *dumpImporter) writeCreditParts(
ctx context.Context, tx *sql.Tx, scan *creditScan,
) (map[int32]struct{}, error) {
stmt, err := tx.PrepareContext(ctx,
`INSERT INTO artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
VALUES (?, ?, ?, ?, ?)`,
)
if err != nil {
return nil, fmt.Errorf("credit import: prepare part insert: %w", err)
}
defer func() { _ = stmt.Close() }()
written := make(map[int32]struct{}, len(scan.used))
for credit := range scan.used {
parts := scan.parts[credit]
if len(parts) < 2 {
// artist_credit said more than one artist and
// artist_credit_name did not deliver them. Nothing to
// decompose, so leave the entity to its single artist.
continue
}
// Position order is the credit's meaning, and the dump is not
// obliged to emit it sorted.
sort.Slice(parts, func(i, j int) bool {
return parts[i].position < parts[j].position
})
resolved := make([][]any, 0, len(parts))
ok := true
for _, part := range parts {
gid, found := scan.artistGIDs[part.artistID]
if !found {
scan.skippedUnknownArtist++
ok = false
break
}
resolved = append(resolved, []any{
credit, part.position, gid[:], part.name, part.join,
})
}
if !ok {
continue
}
for _, args := range resolved {
if _, err := stmt.ExecContext(ctx, args...); err != nil {
return nil, fmt.Errorf("credit import: insert part: %w", err)
}
}
written[credit] = struct{}{}
}
return written, nil
}
// writeCreditRefs points each kept entity at its credit, skipping any
// whose credit was not stored so a ref never dangles.
func (imp *dumpImporter) writeCreditRefs(
ctx context.Context, tx *sql.Tx, scan *creditScan, written map[int32]struct{},
) (int, error) {
stmt, err := tx.PrepareContext(ctx,
"INSERT OR REPLACE INTO artist_credit_ref (mbid, credit_id) VALUES (?, ?)",
)
if err != nil {
return 0, fmt.Errorf("credit import: prepare ref insert: %w", err)
}
defer func() { _ = stmt.Close() }()
count := 0
for mbid, credit := range scan.refs {
if _, stored := written[credit]; !stored {
continue
}
id := mbid
if _, err := stmt.ExecContext(ctx, id[:], credit); err != nil {
return 0, fmt.Errorf("credit import: insert ref: %w", err)
}
count++
}
return count, nil
}
+8
View File
@@ -104,6 +104,7 @@ type dumpImporter struct {
canonicalBaseURL string
listensBaseURL string
mbdumpBaseURL string
// Disk safety floors (fields so tests can relax them).
minStartFreeBytes uint64
@@ -144,6 +145,7 @@ func newDumpImporter(si *SearchIndex, lb *ListenBrainzClient) (*dumpImporter, er
stagingDir: stagingDir,
canonicalBaseURL: defaultCanonicalBaseURL,
listensBaseURL: defaultListensBaseURL,
mbdumpBaseURL: defaultMBDumpBaseURL,
minStartFreeBytes: dumpMinStartFreeBytes,
abortFreeBytes: dumpAbortFreeBytes,
}, nil
@@ -171,6 +173,7 @@ func (imp *dumpImporter) run(ctx context.Context) error {
// Fast path: rows already assembled, only patch passes remain.
if state.Stage == dumpStageAssembled {
imp.si.MarkReadyIfPopulated()
imp.ensureArtistCredits(ctx)
imp.runPatchPasses(ctx)
if err := ctx.Err(); err != nil {
@@ -305,6 +308,11 @@ func (imp *dumpImporter) run(ctx context.Context) error {
imp.si.MarkReadyIfPopulated()
imp.si.refreshStatusCounts()
// Multi-artist credits, from a different dump. After the catalog,
// because it asks explore_index which entities are worth carrying
// credits for.
imp.ensureArtistCredits(ctx)
// Stage 4: API patch passes (idempotent).
imp.runPatchPasses(ctx)
+137
View File
@@ -0,0 +1,137 @@
package explore
import (
"encoding/json"
"errors"
"strings"
"sync"
)
// Whether the catalog artifact may be downloaded on this connection
// (plan 016 B4).
//
// The artifact is ~0.6 GB. On a desktop that is a minute of someone
// else's bandwidth; on a phone it can be a month's allowance, and the
// app had no awareness of the difference at all.
//
// Three decisions shape this file.
//
// **The policy lives here and the platform call does not.** `explore` is
// imported by `cmd/indexbuild`, which is built with `CGO_ENABLED=0` in a
// plain Go container, so naming `application` here would break the one
// job that must not fail (see `TestIndexToolsDoNotImportWails`). What is
// injected is a closure; what is *tested* is the parsing and the
// decision, on every platform.
//
// **An unknown answer is not a metered one.** Only mobile answers this
// question — the desktop stub returns an empty string — so a policy that
// treated silence as "metered" would refuse the download on every
// desktop in the world. Silence means "no reason to refuse".
//
// **Cellular is the signal, and it is the only one available.** Wails
// reports `{"connected":bool,"type":"wifi|cellular|ethernet|none"}` and
// no metered flag, so a metered *wifi* — a phone hotspot, a hotel — is
// invisible to us and will not be refused. That is a known gap rather
// than an oversight: Android knows (`NET_CAPABILITY_NOT_METERED`) and
// the runtime does not pass it on.
// ErrMeteredNetwork is returned instead of downloading the catalog when
// the connection looks metered and the user has not opted in. Every
// failure path in `tryCoreArtifact` is already non-fatal, so this
// behaves like any other reason the artifact is not available yet.
var ErrMeteredNetwork = errors.New(
"explore: catalog download declined on a metered connection",
)
// Network is what the platform can say about the connection.
type Network struct {
// Known is false when nothing answered — every desktop, and any
// mobile build whose bridge is not up yet.
Known bool
// Connected reports a usable connection of any kind.
Connected bool
// Metered reports a connection the user is plausibly paying for by
// the byte. See the note above on what this cannot see.
Metered bool
}
// NetworkProbe answers "what kind of connection is this", or an unknown
// Network when the platform does not say.
type NetworkProbe func() Network
// ParseNetworkJSON reads the runtime's network payload.
//
// Anything unparseable is `Known: false` rather than an error: this
// decides whether to *skip* an optional download, and a malformed
// payload is not a reason to refuse one.
func ParseNetworkJSON(payload string) Network {
var raw struct {
Connected bool `json:"connected"`
Type string `json:"type"`
}
if strings.TrimSpace(payload) == "" {
return Network{}
}
if err := json.Unmarshal([]byte(payload), &raw); err != nil {
return Network{}
}
return Network{
Known: true,
Connected: raw.Connected,
Metered: strings.EqualFold(raw.Type, "cellular"),
}
}
// networkPolicy is the injected half: how to ask, and whether the user
// has said yes anyway.
type networkPolicy struct {
mu sync.RWMutex
probe NetworkProbe
allowMetered func() bool
}
func (p *networkPolicy) set(probe NetworkProbe, allowMetered func() bool) {
p.mu.Lock()
defer p.mu.Unlock()
p.probe = probe
p.allowMetered = allowMetered
}
// refuses reports whether a large optional download should be skipped.
func (p *networkPolicy) refuses() bool {
p.mu.RLock()
probe, allow := p.probe, p.allowMetered
p.mu.RUnlock()
if probe == nil {
return false
}
if allow != nil && allow() {
return false
}
state := probe()
return state.Known && state.Metered
}
// SetNetworkPolicy wires how the catalog download decides whether this
// connection is one to spend 0.6 GB on. Both arguments may be nil, which
// is the desktop's answer: never refuse.
//
//wails:ignore // internal wiring, not part of the app's IPC surface.
func (si *SearchIndex) SetNetworkPolicy(probe NetworkProbe, allowMetered func() bool) {
si.netPolicy.set(probe, allowMetered)
}
// SetNetworkPolicy wires the metered-connection policy into the index.
//
//wails:ignore // internal wiring, not part of the app's IPC surface.
func (e *Service) SetNetworkPolicy(probe NetworkProbe, allowMetered func() bool) {
e.index.SetNetworkPolicy(probe, allowMetered)
}
+158
View File
@@ -0,0 +1,158 @@
package explore
import (
"errors"
"testing"
)
// The catalog is ~0.6 GB and the decision not to fetch it is the only
// part of plan 016 B4 that can be tested anywhere but on a phone: the
// platform call is a one-line closure injected from app.go, and
// everything that decides anything is here.
func TestParseNetworkJSON(t *testing.T) {
t.Parallel()
tests := []struct {
name string
payload string
want Network
}{{
name: "cellular is metered",
payload: `{"connected":true,"type":"cellular"}`,
want: Network{Known: true, Connected: true, Metered: true},
}, {
name: "wifi is not",
payload: `{"connected":true,"type":"wifi"}`,
want: Network{Known: true, Connected: true},
}, {
name: "ethernet is not",
payload: `{"connected":true,"type":"ethernet"}`,
want: Network{Known: true, Connected: true},
}, {
name: "the case is the platform's business, not ours",
payload: `{"connected":true,"type":"Cellular"}`,
want: Network{Known: true, Connected: true, Metered: true},
}, {
name: "offline is known and unmetered",
payload: `{"connected":false,"type":"none"}`,
want: Network{Known: true},
}, {
// The desktop stub. This is the case that must not read as
// "metered": every desktop in the world answers this way.
name: "an empty payload is unknown",
payload: "",
want: Network{},
}, {
name: "so is a malformed one",
payload: `{"connected":`,
want: Network{},
}}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if got := ParseNetworkJSON(tt.payload); got != tt.want {
t.Errorf("ParseNetworkJSON(%q) = %+v, want %+v", tt.payload, got, tt.want)
}
})
}
}
func TestNetworkPolicyRefuses(t *testing.T) {
t.Parallel()
cellular := func() Network {
return Network{Known: true, Connected: true, Metered: true}
}
wifi := func() Network { return Network{Known: true, Connected: true} }
unknown := func() Network { return Network{} }
yes := func() bool { return true }
no := func() bool { return false }
tests := []struct {
name string
probe NetworkProbe
allowMetered func() bool
want bool
}{{
name: "no probe wired refuses nothing",
probe: nil,
want: false,
}, {
name: "an unknown connection refuses nothing",
probe: unknown,
want: false,
}, {
name: "wifi refuses nothing",
probe: wifi,
want: false,
}, {
name: "cellular refuses by default",
probe: cellular,
want: true,
}, {
name: "cellular with no permission refuses",
probe: cellular,
allowMetered: no,
want: true,
}, {
name: "cellular the user opted into does not",
probe: cellular,
allowMetered: yes,
want: false,
}, {
// The permission is read at decision time rather than captured,
// so turning it on takes effect on the next attempt instead of
// the next launch.
name: "permission is asked, not remembered",
probe: cellular,
allowMetered: yes,
want: false,
}}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
var p networkPolicy
p.set(tt.probe, tt.allowMetered)
if got := p.refuses(); got != tt.want {
t.Errorf("refuses() = %v, want %v", got, tt.want)
}
})
}
}
// The gate has to come before anything is staged: a declined download is
// a no-op, not a job in the indicator or a status the user must dismiss.
func TestTryCoreArtifactDeclinesMeteredWithoutStaging(t *testing.T) {
t.Parallel()
si := &SearchIndex{}
si.SetNetworkPolicy(
func() Network { return Network{Known: true, Connected: true, Metered: true} },
nil,
)
err := si.tryCoreArtifact(t.Context())
if !errors.Is(err, ErrMeteredNetwork) {
t.Fatalf("tryCoreArtifact() error = %v, want ErrMeteredNetwork", err)
}
// Nothing announced itself: no build status, no tiers, no job. A
// SearchIndex with no database would panic on any of the work below
// the gate, which is itself part of the assertion.
if si.buildStatus.Building {
t.Error("declining a metered download still reported a build in progress")
}
if len(si.buildStatus.Tiers) != 0 {
t.Errorf("declining staged %d tiers, want none", len(si.buildStatus.Tiers))
}
}
+5
View File
@@ -212,6 +212,11 @@ type SearchIndex struct {
cancel context.CancelFunc
done chan struct{}
// netPolicy decides whether this connection is one to spend ~0.6 GB
// of catalog on. Its own lock: it is written once at startup and read
// from the build goroutine (netpolicy.go).
netPolicy networkPolicy
mu sync.RWMutex
ready bool
+10
View File
@@ -83,6 +83,16 @@ android {
}
debug {
debuggable true
// Its own application id, so it installs *beside* the release
// app rather than needing an uninstall to replace it. The two
// are signed by different certificates (the release one comes
// from a keystore CI holds), and Android's remedy for a
// certificate change is an uninstall -- which takes the
// user's library with it. This is also what makes the WebView
// inspectable on a real phone: `debuggable` is what turns on
// `setWebContentsDebuggingEnabled`, and `make android-inspect`
// drives it.
applicationIdSuffix ".dev"
}
}
@@ -31,9 +31,16 @@ import android.webkit.WebSettings;
import android.webkit.WebView;
import android.webkit.WebViewClient;
import android.view.View;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.content.FileProvider;
import androidx.core.graphics.Insets;
import androidx.core.view.ViewCompat;
import androidx.core.view.WindowInsetsCompat;
import androidx.core.view.WindowCompat;
import androidx.core.view.WindowInsetsControllerCompat;
import androidx.webkit.WebViewAssetLoader;
import org.json.JSONObject;
@@ -88,6 +95,10 @@ public class MainActivity extends AppCompatActivity {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// Before anything renders: the page is laid out inside the
// window, and on Android 15 the window is the whole screen.
applyWindowInsets();
// Initialize the native Go library
bridge = new WailsBridge(this);
bridge.initialize();
@@ -892,8 +903,61 @@ public class MainActivity extends AppCompatActivity {
}
}
/**
* Keep the web content inside the safe area.
*
* <p>targetSdk 35 is Android 15, which lays every app out
* edge-to-edge and ignores the {@code statusBarColor} and
* {@code navigationBarColor} this app's theme still sets. The
* WebView is {@code match_parent}, so the page's bottom band -- the
* transport and, on a phone, the tab bar -- was drawn underneath the
* gesture bar and reported from a device as "I can't see the
* playback controls, they seem to be off screen".
*
* <p>No web-tier test can see this: a browser viewport has no system
* bars, so the phone specs at 390x844 render a shell that fits
* while the device does not.
*
* <p>The insets are applied as padding and the window insets are
* returned rather than consumed, so the WebView is laid out inside
* them. {@code ime()} is in the mask because the same reasoning
* covers the keyboard: a focused search box that the keyboard
* covers is the same bug one surface over.
*/
private void applyWindowInsets() {
final View container = findViewById(R.id.main_container);
if (container == null) {
return;
}
ViewCompat.setOnApplyWindowInsetsListener(container, (view, windowInsets) -> {
Insets insets = windowInsets.getInsets(
WindowInsetsCompat.Type.systemBars()
| WindowInsetsCompat.Type.displayCutout()
| WindowInsetsCompat.Type.ime());
view.setPadding(insets.left, insets.top, insets.right, insets.bottom);
return windowInsets;
});
// The padded band shows the window background, which is dark
// (this app's own default ramp is black), so the system's icons
// have to be the light set or they vanish into it. The theme is
// DayNight and would otherwise ask for dark icons in light mode.
WindowInsetsControllerCompat controller =
WindowCompat.getInsetsController(getWindow(), getWindow().getDecorView());
controller.setAppearanceLightStatusBars(false);
controller.setAppearanceLightNavigationBars(false);
}
@Override
public void onBackPressed() {
// The frontend records every navigation as a history entry, so
// this is the app's own back stack: `canGoBack()` is false only
// at the launch entry, which is where back should leave.
if (webView != null && webView.canGoBack()) {
webView.goBack();
} else {
@@ -2,7 +2,12 @@
<resources>
<color name="wails_blue">#3574D4</color>
<color name="wails_blue_dark">#2C5FB8</color>
<color name="wails_background">#1B2636</color>
<!-- The window background, which is what the launch screen shows and
what the system-bar padding leaves visible. Black rather than the
scaffold's blue-grey because this app's own default ramp is
black: a band of #1B2636 above and below it reads as the app
failing to fill the screen. -->
<color name="wails_background">#000000</color>
<color name="white">#FFFFFFFF</color>
<color name="black">#FF000000</color>
</resources>
+14 -1
View File
@@ -177,12 +177,25 @@ func run(o opts) error {
complete := svc.IndexImportComplete() && !errors.Is(err, errIncomplete)
// Credits are maintenance, not part of any one mode. They come from
// a different dump, they are keyed on entities the catalog already
// holds, and a catalog built before the pass existed would otherwise
// only gain them from a rebuild — which re-downloads ~205 GB to
// re-derive rows it already has. Skipped when the import is not
// complete, because there is nothing to key them against yet.
creditsAdded := false
if complete {
creditsAdded = svc.EnsureArtistCredits(context.Background())
}
// "Changed" means there is something new worth publishing, so it is
// only ever true for a finished import: a build stamps the listens
// series early, long before its rows are assembled, and reporting a
// change off that would be a lie about a half-built index.
changed := complete &&
(svc.IndexBaselineSeries() != seriesBefore || chosen != modeRefresh)
(svc.IndexBaselineSeries() != seriesBefore ||
chosen != modeRefresh ||
creditsAdded)
report(logger, svc, chosen, complete, changed)
+197 -4
View File
@@ -12,6 +12,7 @@ import (
_ "modernc.org/sqlite"
"yellowjacket/backend/database"
"yellowjacket/backend/datamap"
"yellowjacket/backend/system"
)
@@ -53,10 +54,19 @@ func TestRetireLibraryTables(t *testing.T) {
CREATE TABLE recordings (id INTEGER PRIMARY KEY, title TEXT);
`)
// The symptom, before the repair: the schema cannot be applied over
// a table whose shape has moved on.
if _, err := database.NewDB(logger); err == nil {
t.Fatal("expected the stale shape to fail to open; it did not")
// This used to assert the symptom -- that the schema cannot be
// applied over a table whose shape has moved on -- because at the
// time nothing repaired it and only this job did. The app-side
// repair (backend/database/staleshape.go) now retires a stale
// non-authored table before applySchema meets it, so opening
// succeeds and the symptom no longer reproduces from here.
//
// That does not make retireLibraryTables redundant, and the rest of
// this test is why: the app-side repair only removes what is *stale*,
// while this database wants its library half gone entirely, healthy
// or not, because nothing here scans, plays or authors.
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("the app-side repair should have opened this: %v", err)
}
if err := retireLibraryTables(context.Background(), logger); err != nil {
@@ -116,3 +126,186 @@ func count(t *testing.T, dbPath, query string) int {
return n
}
// TestTheCatalogSurvivesAStaleShape is the accident written down.
//
// The app repairs a stale Cache table by dropping it: its catalog is
// downloaded, so a wrong shape costs about a minute of re-fetching and
// keeping it costs every Explore read. Applied here that rule is
// catastrophic — this database is what the artifact is *cut from*, so
// there is nothing to re-fetch and the only way back is the ~205 GB
// dump stream the /cache volume exists to avoid.
//
// It shipped without that distinction and dropped the real CI catalog
// on the first run:
//
// retiring a table ... table=explore_index
// reason="column entity_type is TEXT, schema declares INTEGER"
// index maintenance mode=build reason="no completed import yet"
//
// The mismatch was real: that database is deliberately kept in the
// older encoding, which `fix(indexexport): read an index older than the
// binary` exists to tolerate. So the shape will not match, every run,
// by design — and the catalog must survive it anyway.
func TestTheCatalogSurvivesAStaleShape(t *testing.T) {
logger := slog.New(slog.DiscardHandler)
t.Setenv("YJ_HOME", t.TempDir())
dataDir, err := system.GetUserDataDirPath()
if err != nil {
t.Fatalf("resolve data dir: %v", err)
}
dbPath := filepath.Join(dataDir, "yj.db")
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("first open: %v", err)
}
// The shape the real index database is in: every current column,
// but the ids and the entity type still text. That is what the
// exporter's backward-compatibility fix tolerates, and it is what
// the repair saw and called stale.
exec(t, dbPath, `
DROP TABLE explore_index;
CREATE TABLE explore_index (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type TEXT NOT NULL,
mbid TEXT NOT NULL,
title TEXT NOT NULL,
artist_name TEXT NOT NULL,
artist_mbid TEXT NOT NULL,
aliases TEXT NOT NULL DEFAULT '',
popularity INTEGER NOT NULL DEFAULT 0,
listener_count INTEGER NOT NULL DEFAULT 0,
duration INTEGER NOT NULL DEFAULT 0,
caa_release_mbid TEXT NOT NULL DEFAULT '',
release_name TEXT NOT NULL DEFAULT '',
primary_type TEXT NOT NULL DEFAULT '',
secondary_types TEXT NOT NULL DEFAULT '',
release_date TEXT NOT NULL DEFAULT '',
total_tracks INTEGER NOT NULL DEFAULT 0,
artist_type TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
disambiguation TEXT NOT NULL DEFAULT '',
sort_name TEXT NOT NULL DEFAULT '',
in_library INTEGER NOT NULL DEFAULT 0,
is_similar INTEGER NOT NULL DEFAULT 0,
local_artist_id INTEGER,
local_release_group_id INTEGER,
local_recording_id INTEGER,
discog_fetched INTEGER NOT NULL DEFAULT 0,
UNIQUE(mbid)
);
INSERT INTO explore_index
(entity_type, mbid, title, artist_name, artist_mbid)
VALUES ('artist', 'a-b-c', 'A Catalog Row', 'An Artist', 'd-e-f');
`)
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("open with a stale catalog shape: %v", err)
}
if got := count(t, dbPath, "SELECT COUNT(*) FROM explore_index"); got != 1 {
t.Fatalf(
"explore_index rows = %d, want 1 — the catalog was retired, "+
"which costs this database a ~205GB rebuild",
got,
)
}
}
// TestNoCacheTableIsRetiredHere is the general form of the accident
// above, and it exists because the specific one is not the risk.
//
// `TestTheCatalogSurvivesAStaleShape` pins one table in one wrong shape,
// which is the failure that happened. What cost the ~205 GB was not that
// shape: it was a destructive repair added to `database.NewDB` -- the
// one chokepoint every binary in this project shares -- without asking
// which binary it was running in. The next such repair will have a
// different name and a different reason, and this database still cannot
// afford it.
//
// So the assertion is about the *outcome* rather than the mechanism: put
// every Cache table in a shape the schema has certainly moved past, open
// the database the way cmd/indexbuild does, and require that all of them
// are still there afterwards. Any future repair that drops one fails
// here regardless of how it decides to.
//
// Two things about it are deliberate.
//
// The table list comes from `datamap.ByKind(Cache)` rather than being
// written out, so a Cache table added next year is covered by this test
// on the day it is added -- the same reason `TestCatalogCoversSchema`
// reads the schema instead of a list.
//
// And `NewDB` returning an error is *accepted*, because that is the
// trade the fix documents: with Cache tables no longer rebuilt here, a
// shape the schema moved past now fails this job loudly instead of
// silently costing it a day of downloading. Loud is fine. Gone is not.
func TestNoCacheTableIsRetiredHere(t *testing.T) {
logger := slog.New(slog.DiscardHandler)
t.Setenv("YJ_HOME", t.TempDir())
dataDir, err := system.GetUserDataDirPath()
if err != nil {
t.Fatalf("resolve data dir: %v", err)
}
dbPath := filepath.Join(dataDir, "yj.db")
if _, err := database.NewDB(logger); err != nil {
t.Fatalf("first open: %v", err)
}
// An FTS table is four shadow tables and cannot be given a "wrong
// shape" meaningfully; the repair skips them for the same reason and
// retires them with their parent, which the parents below cover.
var cache []string
for _, table := range datamap.ByKind(datamap.Cache) {
if table.FTS {
continue
}
cache = append(cache, table.Name)
}
if len(cache) == 0 {
t.Fatal("no Cache tables to check: the datamap or this test is wrong")
}
for _, name := range cache {
// A shape nothing in the current schema describes. What matters
// is only that it disagrees; the real mismatch was one column's
// type.
exec(t, dbPath, `
DROP TABLE IF EXISTS `+name+`;
CREATE TABLE `+name+` (id INTEGER PRIMARY KEY, moved_past TEXT);
INSERT INTO `+name+` (moved_past) VALUES ('irreplaceable');
`)
}
// The error is not the assertion: see the note above.
_, _ = database.NewDB(logger)
for _, name := range cache {
rows := count(t, dbPath,
`SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = '`+name+`'`)
if rows == 0 {
t.Errorf("%s was retired: in this database a Cache table is derived, "+
"not downloaded, and dropping one costs the ~205 GB dump stream", name)
continue
}
// Present but emptied is the same loss wearing a different
// shape: SQLite does an implicit DELETE before a DROP, and a
// repair that recreated the table would look identical here.
if n := count(t, dbPath, `SELECT count(*) FROM `+name); n == 0 {
t.Errorf("%s survived but was emptied", name)
}
}
}
+191
View File
@@ -0,0 +1,191 @@
//go:build indexbuild
package main
import (
"database/sql"
"path/filepath"
"strings"
"testing"
_ "modernc.org/sqlite"
)
// The columns an index built before the completeness work has: every
// current one except total_tracks.
//
// Filtered rather than string-replaced, because the list is formatted
// across lines: `strings.Replace(catalogColumns, "total_tracks, ", …)`
// matches nothing (the name is followed by a newline, not a space) and
// silently yields the *current* list -- so the test built a modern
// source index and proved nothing while passing its own premise.
var oldColumns = withoutTotals(catalogColumns)
func withoutTotals(cols string) string {
kept := make([]string, 0, 20)
for _, part := range strings.Split(cols, ",") {
if strings.TrimSpace(part) == "total_tracks" {
continue
}
kept = append(kept, strings.TrimSpace(part))
}
return strings.Join(kept, ", ")
}
// TestExportFromAnIndexWithoutTotals reproduces the failure that broke
// the index-artifact job, symptom first.
//
// The job's /cache volume is a real YJ_HOME that survives between runs
// and holds ~205 GB, so its explore_index is Cache and is deliberately
// not dropped by cmd/indexbuild's schema repair -- which means a column
// added to the schema afterwards is simply absent from it. The exporter
// selected it anyway and the whole run died with
//
// indexexport: copy rows: SQL logic error: no such column: total_tracks
//
// after three minutes of work, on a job that publishes the catalog
// every user downloads.
func TestExportFromAnIndexWithoutTotals(t *testing.T) {
t.Parallel()
db := openWithSource(t, oldColumns)
if got := sourceColumns(db); strings.Contains(got, "total_tracks") {
t.Fatalf("source list still names total_tracks: %s", got)
}
if err := copyRows(db, 10, 5, 5); err != nil {
t.Fatalf("export from an index without total_tracks: %v", err)
}
// Zero, not absent: the artifact keeps every column so an importer
// needs no second shape, and 0 is what the column already means by
// "the catalog does not say".
var total int
if err := db.QueryRow(
`SELECT total_tracks FROM core.explore_index WHERE entity_type = 2`,
).Scan(&total); err != nil {
t.Fatalf("read exported total_tracks: %v", err)
}
if total != 0 {
t.Errorf("total_tracks = %d, want 0", total)
}
}
// TestExportCarriesTotalsWhenTheIndexHasThem is the other half: the
// probe must not cost the totals of an index that does have them.
func TestExportCarriesTotalsWhenTheIndexHasThem(t *testing.T) {
t.Parallel()
db := openWithSource(t, catalogColumns)
if got := sourceColumns(db); !strings.Contains(got, "total_tracks") {
t.Fatalf("source list dropped total_tracks: %s", got)
}
if err := copyRows(db, 10, 5, 5); err != nil {
t.Fatalf("export: %v", err)
}
var total int
if err := db.QueryRow(
`SELECT total_tracks FROM core.explore_index WHERE entity_type = 2`,
).Scan(&total); err != nil {
t.Fatalf("read exported total_tracks: %v", err)
}
if total != 12 {
t.Errorf("total_tracks = %d, want 12", total)
}
}
// openWithSource builds a source index carrying exactly `columns`, with
// one artist and one of its release groups, and attaches a fresh
// artifact database as `core`.
func openWithSource(t *testing.T, columns string) *sql.DB {
t.Helper()
dir := t.TempDir()
db, err := sql.Open("sqlite", filepath.Join(dir, "src.db"))
if err != nil {
t.Fatalf("open source: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
// The source's shape is the point of the test, so it is spelled
// out here rather than taken from the app's schema, which is
// always current by definition.
create := `CREATE TABLE explore_index (
id INTEGER PRIMARY KEY,
entity_type INTEGER NOT NULL,
mbid BLOB NOT NULL,
title TEXT NOT NULL DEFAULT '',
artist_name TEXT NOT NULL DEFAULT '',
artist_mbid BLOB NOT NULL DEFAULT x'',
aliases TEXT NOT NULL DEFAULT '',
popularity INTEGER NOT NULL DEFAULT 0,
listener_count INTEGER NOT NULL DEFAULT 0,
duration INTEGER NOT NULL DEFAULT 0,
caa_release_mbid BLOB NOT NULL DEFAULT x'',
release_name TEXT NOT NULL DEFAULT '',
primary_type TEXT NOT NULL DEFAULT '',
secondary_types TEXT NOT NULL DEFAULT '',
release_date TEXT NOT NULL DEFAULT '',
total_tracks INTEGER NOT NULL DEFAULT 0,
artist_type TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
disambiguation TEXT NOT NULL DEFAULT '',
sort_name TEXT NOT NULL DEFAULT '',
discog_fetched INTEGER NOT NULL DEFAULT 0
)`
if !strings.Contains(columns, "total_tracks") {
create = strings.Replace(
create, "total_tracks INTEGER NOT NULL DEFAULT 0,\n", "", 1,
)
}
if _, err := db.Exec(create); err != nil {
t.Fatalf("create source: %v", err)
}
seed := `INSERT INTO explore_index (` + columns + `) VALUES `
if strings.Contains(columns, "total_tracks") {
seed += `(1, x'00000000000000000000000000000001', 'A', 'A',
x'00000000000000000000000000000001', '', 100, 100, 0, x'',
'', '', '', '', 12, '', '', '', '', 0),
(2, x'00000000000000000000000000000002', 'RG', 'A',
x'00000000000000000000000000000001', '', 90, 90, 0, x'',
'', 'Album', '', '', 12, '', '', '', '', 0)`
} else {
seed += `(1, x'00000000000000000000000000000001', 'A', 'A',
x'00000000000000000000000000000001', '', 100, 100, 0, x'',
'', '', '', '', '', '', '', '', 0),
(2, x'00000000000000000000000000000002', 'RG', 'A',
x'00000000000000000000000000000001', '', 90, 90, 0, x'',
'', 'Album', '', '', '', '', '', '', 0)`
}
if _, err := db.Exec(seed); err != nil {
t.Fatalf("seed source: %v", err)
}
if _, err := db.Exec(
`ATTACH DATABASE ? AS core`, filepath.Join(dir, "core.db"),
); err != nil {
t.Fatalf("attach core: %v", err)
}
if err := createSchema(db); err != nil {
t.Fatalf("create artifact schema: %v", err)
}
return db
}
+119 -2
View File
@@ -25,6 +25,7 @@ import (
"os"
"path/filepath"
"strconv"
"strings"
"time"
_ "modernc.org/sqlite"
@@ -43,6 +44,41 @@ const catalogColumns = `entity_type, mbid, title, artist_name, artist_mbid,
release_name, primary_type, secondary_types, release_date, total_tracks,
artist_type, country, disambiguation, sort_name, discog_fetched`
// sourceColumns is catalogColumns as read *from* the built index,
// which is not always shaped like the one this binary was compiled
// against.
//
// The index job's /cache volume is a real YJ_HOME that survives
// between runs and holds ~205 GB nobody can re-download casually, so
// its explore_index is classified Cache and is deliberately **not**
// dropped and recreated by cmd/indexbuild's schema repair. A column
// added to the schema after that database was built is therefore
// absent from it, and selecting it fails the whole export with
// "no such column: total_tracks" -- which is what happened the first
// time the job ran after the completeness work.
//
// So the source list is asked for rather than assumed, exactly as
// artifactHasTotals does on the importing side. Zero is what the
// column means by "the catalog does not say", and the app already
// renders that as unknown rather than as incomplete.
func sourceColumns(db *sql.DB) string {
var n int
err := db.QueryRow(
`SELECT COUNT(*) FROM pragma_table_info('explore_index', 'main')
WHERE name = 'total_tracks'`,
).Scan(&n)
if err == nil && n > 0 {
return catalogColumns
}
fmt.Println(
" note: this index predates total_tracks; exporting 0 for it",
)
return strings.Replace(catalogColumns, "total_tracks", "0", 1)
}
var errNoHome = errors.New(
"YJ_HOME must be set to the directory holding the built index",
)
@@ -171,6 +207,26 @@ func createSchema(db *sql.DB) error {
key TEXT PRIMARY KEY,
value TEXT NOT NULL
)`,
// Multi-artist credits. Shipped as their own tables rather than
// as an explore_index column because a credit is a variable
// number of ordered parts, and because credits are *shared* --
// an album's tracks by one artist reference one credit, which is
// what keeps this to a few hundred thousand rows.
//
// An importer that predates these reads an artifact without
// them; artifactHasCredits is what asks.
`CREATE TABLE core.artist_credit_part (
credit_id INTEGER NOT NULL,
position INTEGER NOT NULL,
artist_mbid BLOB NOT NULL,
credited_name TEXT NOT NULL,
join_phrase TEXT NOT NULL DEFAULT '',
PRIMARY KEY (credit_id, position)
) WITHOUT ROWID`,
`CREATE TABLE core.artist_credit_ref (
mbid BLOB NOT NULL PRIMARY KEY,
credit_id INTEGER NOT NULL
) WITHOUT ROWID`,
}
for _, stmt := range stmts {
@@ -189,6 +245,10 @@ func createSchema(db *sql.DB) error {
// dumpcatalog.go — a flat global top-N would give a handful of
// superstars everything and everyone else nothing.
func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
// The destination is created by this binary and always has every
// column; only the source may be older.
srcColumns := sourceColumns(db)
if _, err := db.Exec(`
CREATE TEMP TABLE core_artists AS
SELECT mbid FROM main.explore_index
@@ -201,7 +261,7 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
copied, err := insertSelect(db, `
INSERT INTO core.explore_index (`+catalogColumns+`)
SELECT `+catalogColumns+`
SELECT `+srcColumns+`
FROM main.explore_index
WHERE entity_type = 1 /* artist */
AND mbid IN (SELECT mbid FROM core_artists)`)
@@ -228,7 +288,7 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
// most `limit` rows, ranked by their own listen counts.
n, err := insertSelect(db, `
INSERT INTO core.explore_index (`+catalogColumns+`)
SELECT `+catalogColumns+` FROM (
SELECT `+srcColumns+` FROM (
SELECT *, ROW_NUMBER() OVER (
PARTITION BY artist_mbid ORDER BY popularity DESC
) AS rn
@@ -243,6 +303,63 @@ func copyRows(db *sql.DB, artists, perArtistRGs, perArtistRecs int) error {
fmt.Printf(" %-15s %d\n", sel.label+":", n)
}
return copyCredits(db)
}
// copyCredits ships the credit decomposition for the entities that made
// it into the artifact, and only those.
//
// The refs go first and the parts follow *from* the refs, so a credit is
// carried only if something in the artifact points at it. The source
// index holds credits for every catalog entity, while the artifact is a
// windowed subset -- copying all of them would carry a large table most
// of which nothing in the artifact can reach.
//
// A source index built before the credit pass simply has no rows here,
// which is not an error: the artifact then carries the tables empty, and
// every credit falls back to its single artist exactly as before.
func copyCredits(db *sql.DB) error {
// Asked, not assumed. A source index built before the credit pass
// has no such table, and "no such table" would fail an export whose
// catalog is otherwise complete.
for _, table := range []string{"artist_credit_ref", "artist_credit_part"} {
var n int
if err := db.QueryRow(
`SELECT COUNT(*) FROM main.sqlite_master
WHERE type = 'table' AND name = ?`, table,
).Scan(&n); err != nil {
return fmt.Errorf("probe %s: %w", table, err)
}
if n == 0 {
fmt.Printf(" %-15s none in source\n", "credits:")
return nil
}
}
refs, err := insertSelect(db, `
INSERT INTO core.artist_credit_ref (mbid, credit_id)
SELECT r.mbid, r.credit_id
FROM main.artist_credit_ref r
WHERE r.mbid IN (SELECT mbid FROM core.explore_index)`)
if err != nil {
return err
}
parts, err := insertSelect(db, `
INSERT INTO core.artist_credit_part
(credit_id, position, artist_mbid, credited_name, join_phrase)
SELECT p.credit_id, p.position, p.artist_mbid, p.credited_name, p.join_phrase
FROM main.artist_credit_part p
WHERE p.credit_id IN (SELECT credit_id FROM core.artist_credit_ref)`)
if err != nil {
return err
}
fmt.Printf(" %-15s %d refs, %d parts\n", "credits:", refs, parts)
return nil
}
+99
View File
@@ -0,0 +1,99 @@
# The index cache, and why it has a snapshot
`/srv/yellowjacket/index-cache` on the Gitea host is the `YJ_HOME` the
search-index job keeps between runs — `.gitea/workflows/index-artifact.yml`
mounts it at `/cache`. It holds the catalog every user eventually
downloads, and it is the one database in this project that is
**derived rather than downloaded**.
That is the whole reason this document exists. An install with a broken
catalog re-fetches the ~0.6 GB artifact and is fine in a minute. This
database *is* what that artifact is cut from, so its only route back is
re-streaming the MetaBrainz dumps: hours, at a rate that belongs to
someone else's server, holding a runner of capacity 1 the entire time.
## What happened on 2026-08-17
A schema repair (`fix(database): retire a table whose shape the schema
moved past`) dropped every table whose live shape disagreed with the
schema, before `applySchema`. Correct for the app. Applied here it
deleted the catalog 19 seconds into the first run:
```
retiring a table ... table=explore_index
reason="column entity_type is TEXT, schema declares INTEGER"
index maintenance mode=build reason="no completed import yet"
```
The mismatch was real and deliberate: this database is kept in the older
text encoding, which `artifactStoresText` and `sourceColumns` exist to
tolerate. So it would have been judged stale on *every* run.
Two things came out of it. `retireStaleCache` is now a build tag —
false under `indexbuild`, true in the app — and
`TestNoCacheTableIsRetiredHere` asserts the outcome rather than the
mechanism, so the next destructive repair fails a test instead of a
production volume. And the volume got the snapshot it should always have
had, below.
## Taking snapshots
```sh
scripts/index-cache-snapshot.sh [SOURCE_HOME] [DEST_DIR] [KEEP]
```
Defaults: `/srv/yellowjacket/index-cache`, `/srv/yellowjacket/index-snapshots`,
keep 2. On the Gitea host, daily and away from the Monday 04:00 build:
```
30 5 * * * /path/to/index-cache-snapshot.sh >> /var/log/yj-index-snapshot.log 2>&1
```
Three properties worth knowing before trusting it:
- **It uses `VACUUM INTO`, not `cp`.** The database may be open, and a
byte copy of a live SQLite file is a corrupt file of plausible size.
`VACUUM INTO` takes a read lock and writes a consistent, compacted
copy; it is safe to run while a build is in progress.
- **It does not copy `data/explore-staging`.** That is a resumable
checkpoint of work in flight — large, constantly changing, and a build
resumes without it. What cannot be cheaply re-derived is the finished
catalog, which is in the database.
- **It verifies before it rotates.** Each snapshot is reopened and asked
for its catalog row count; a run that produces an unreadable or empty
file fails loudly, deletes its own output, and leaves the previous
snapshots alone. Both paths are exercised, not assumed.
## Restoring
Stop anything that might be using the volume first — the job holds it
for the length of a build, and the concurrency group (`search-index`)
means a queued run will start the moment one ends.
```sh
cd /srv/yellowjacket
mv index-cache/data/yj.db index-cache/data/yj.db.broken # keep it until you are sure
cp index-snapshots/yj-index-<stamp>.db index-cache/data/yj.db
chown --reference=index-cache/data/yj.db.broken index-cache/data/yj.db
```
Then dispatch the workflow with `mode=auto`. A restored snapshot is
older than the dumps, so `indexbuild` resolves to `refresh` and folds in
the incremental listens since — which is minutes, not hours.
Two notes on what a restore does *not* need. The staging directory can
be deleted; it will be rebuilt if a build is needed. And the published
artifact is untouched by any of this: users keep downloading the last
good one until a run reports `complete=true` and `changed=true`
republishes.
## The trade this leaves open
With Cache tables no longer retired under `indexbuild`, a future
`explore_index` column change will fail this job **loudly** — at
`applySchema`, or at the first query naming the column — rather than
silently rebuilding. That is the right default: loud is recoverable and
a silent day of downloading is not. It does mean the next schema change
touching `explore_index` needs a deliberate plan for this one database:
take a snapshot, apply the change to a copy, or accept a rebuild
knowingly.
+92
View File
@@ -0,0 +1,92 @@
import { test, expect } from '../support/fixtures.js';
/**
* Back is the platform's, and the app has to have somewhere for it to
* go (reported from a device: "the Android back button does not
* navigate back in the app").
*
* The scaffold's `MainActivity.onBackPressed` asks `webView.canGoBack()`
* and finishes the activity otherwise. This app never touched
* `history`, so that was always false and back quit from any depth. A
* navigation is a history entry now, which is why this is assertable
* here at all: `page.goBack()` is the same `popstate` the phone's
* gesture produces, so the browser tier can answer a question that
* otherwise needs a device.
*
* What it cannot answer is whether Android's *gesture* reaches the
* WebView, which is between the OS and the scaffold.
*/
type Page = import('@playwright/test').Page;
const activeView = (page: Page) =>
page.getByTestId('main-content');
/**
* Open an artist's detail view, which is the deepest ordinary route.
*
* A library artist opens `explore-artist-details` -- the catalog panel
* standing in for a library one, as `explore-link.ts` describes -- and
* the view name follows the component, not the source of the click.
*/
async function openAnArtist(app: Page): Promise<void> {
await app.getByTestId('nav-artists').click();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'artists');
// A card, by the name on it: the grid is virtualized and positioned
// by transform, so a click at coordinates is a click at whatever
// happens to be there.
await app.locator('artists-view').getByText('Aurora Fields').first().click();
await expect(activeView(app)).toHaveAttribute(
'data-active-view',
'explore-artist-details',
);
}
test.describe('the back gesture', () => {
test('leaves a detail view for the view it was opened from', async ({
app,
}) => {
await openAnArtist(app);
await app.goBack();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'artists');
});
test('walks back through primary views, one press per navigation', async ({
app,
}) => {
await app.getByTestId('nav-tracks').click();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'tracks');
await app.getByTestId('nav-albums').click();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'albums');
await app.goBack();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'tracks');
// Forward is free once back works, and it is what proves the entry
// was restored rather than the view merely re-rendered.
await app.goForward();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'albums');
});
test('an in-app back button consumes exactly one entry', async ({ app }) => {
await app.getByTestId('nav-tracks').click();
await openAnArtist(app);
// The detail view's own back button and the phone's gesture are the
// same press: if each popped its own stack, this would land two
// navigations back instead of one.
await app
.locator('explore-artist-details')
.getByRole('button', { name: 'Back to explore' })
.click();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'artists');
await app.goBack();
await expect(activeView(app)).toHaveAttribute('data-active-view', 'tracks');
});
});
+58 -11
View File
@@ -26,9 +26,7 @@ import type { Page } from '@playwright/test';
*/
test.describe('Explore before anyone has typed', () => {
test.beforeEach(async ({ app }) => {
// Idempotent, so running it per test costs one count query when a
// catalog is already there — which is every developer machine.
await stageCatalogIfEmpty(app);
await stageCatalog(app);
await app.getByTestId('nav-explore').click();
await expect(app.getByTestId('main-content')).toHaveAttribute(
@@ -125,8 +123,7 @@ test.describe('Explore before anyone has typed', () => {
});
/**
* Give the app a catalog if it has none, so the empty-index environment
* still exercises the shelves rather than skipping them.
* Give the app the catalog these shelves are written against.
*
* Deliberately shaped: two artists with albums (one of them with
* three), and a third with none. The three albums are what "one album
@@ -135,9 +132,23 @@ test.describe('Explore before anyone has typed', () => {
* above it and correctly skipped the first version of this fixture
* had only two, and the artists shelf was rightly omitted, which read
* as a broken page.
*
* **Unconditional, and it used to ask whether the catalog was empty.**
* "Any rows at all" is the wrong question: the backend is one shared
* process with one database, so a *single* row left by another spec
* file `requested-badge` stages one album satisfies that gate and
* this suite then draws a shelf page with no artist card on it and
* times out looking for one. It survives a suite run, so it is the
* second local `make e2e` that fails and the first that passes, which
* is the least useful order. CI never sees it: every run there is a
* fresh YJ_HOME.
*
* The inserts are `INSERT OR IGNORE` keyed on the MBID, so running
* this per test is idempotent, and adding seven low-popularity rows to
* a developer machine's real million-row catalog changes nothing the
* shelves show.
*/
async function stageCatalogIfEmpty(app: Page): Promise<void> {
if ((await catalogRows(app)) > 0) return;
async function stageCatalog(app: Page): Promise<void> {
// The catalog stores an MBID as its 16 raw bytes and an entity type
// as a small integer, so a staged row has to be spelled the way the
@@ -185,15 +196,51 @@ async function stageCatalogIfEmpty(app: Page): Promise<void> {
expect(result.status, `staging failed: ${result.body}`).toBe(200);
// …and `OR IGNORE` means a 200 is not a write. A CHECK the row
// violates is *ignored*, not reported, so the count below is the
// violates is *ignored*, not reported, so the check below is the
// only thing that can tell staging from silence.
//
// It is `0 or 1`, not `1`, because this helper is now
// unconditional: the second call of a run legitimately writes
// nothing. What must hold either way is that the rows are *there*,
// which is what the assertion after the loop says — a stronger
// statement than "this insert wrote something", and the one that
// actually protects the fixture.
expect(
(JSON.parse(result.body) as { rowsAffected?: number }).rowsAffected,
`staged nothing: ${result.body}`,
).toBe(1);
`staging error: ${result.body}`,
).toBeLessThanOrEqual(1);
}
expect(await catalogRows(app)).toBeGreaterThan(0);
// Every staged row is present, whoever put it there. An MBID that
// fails `CHECK(length(mbid) = 16)` is silently dropped by OR IGNORE,
// and this is where that shows up.
expect(await stagedRowCount(app), 'the staged catalog is incomplete')
.toBe(rows.length);
}
/** How many of the staged fixture rows are in the catalog. */
async function stagedRowCount(app: Page): Promise<number> {
const result = await app.evaluate(async () => {
const res = await fetch('/__test/sql', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
sql: `SELECT COUNT(*) AS n FROM explore_index
WHERE artist_name IN ('Staged Alpha', 'Staged Beta',
'Staged Gamma')`,
}),
});
return { status: res.status, body: await res.text() };
});
expect(result.status, `count failed: ${result.body}`).toBe(200);
const parsed = JSON.parse(result.body) as {
rows?: { n?: number }[];
};
return parsed.rows?.[0]?.n ?? 0;
}
/**
+23 -19
View File
@@ -199,35 +199,39 @@ test.describe('the shell reflows rather than hiding what does not fit', () => {
});
});
test('what does not fit sideways can be scrolled to', async ({ app }) => {
test('nothing needs scrolling to at 320px, because it all fits', async ({ app }) => {
// 320 CSS px is 400% page zoom of a 1280px viewport, which is the
// size 1.4.10 names. The shell is 784px wide there, so 464px of the
// app — the job indicator and the queue button among it — used to
// be behind `overflow: hidden` with no way to reach it.
// size 1.4.10 names.
//
// **This assertion is the inverse of the one it replaces, and that
// is the fix landing rather than the test being weakened.** The
// shell used to be 784px wide here, so 464px of the app — the job
// indicator and the queue button among it — sat behind
// `overflow: hidden` with no way to reach it; making the axis
// scrollable was the remedy available at the time. 016 B2's phone
// layout reflows instead: below 600px the sidebar becomes a bottom
// tab bar, the header's controls shrink, and the shell measures
// exactly 320px in a 320px viewport. Reflow is what 1.4.10 asks
// for; being able to scroll to the overflow was the concession.
await app.setViewportSize({ width: 320, height: 256 });
// A *gesture*, not `scrollLeft = 9999`: `overflow: hidden` still
// permits programmatic scrolling, so the obvious probe passes on
// the build that has the bug. It did, first time.
await app.mouse.move(160, 20);
await app.mouse.wheel(400, 400);
await app.waitForTimeout(200);
const reach = await app.evaluate(() => {
const fit = await app.evaluate(() => {
const se = document.scrollingElement!;
return { left: se.scrollLeft, top: se.scrollTop };
return {
scrollWidth: se.scrollWidth,
clientWidth: document.documentElement.clientWidth,
scrollHeight: se.scrollHeight,
clientHeight: document.documentElement.clientHeight,
};
});
expect(reach.left).toBeGreaterThan(0);
expect(fit.scrollWidth).toBeLessThanOrEqual(fit.clientWidth);
// And the vertical axis stays fixed, which is what keeps the
// transport where a desktop player's transport belongs.
expect(reach.top).toBe(0);
// transport where a player's transport belongs.
expect(fit.scrollHeight).toBeLessThanOrEqual(fit.clientHeight);
await app.evaluate(() => {
document.scrollingElement!.scrollLeft = 0;
});
await app.setViewportSize({ width: 1440, height: 900 });
});
+127
View File
@@ -0,0 +1,127 @@
import { test, expect } from '../support/fixtures.js';
/**
* Long-press is the touch route to a context menu (plan 016 B2 phase 3).
*
* The component tier proves the gesture in isolation, against markup it
* built itself. What it cannot prove is the half that made this one
* listener instead of six: that the synthetic event reaches the handler
* a *real* component bound `track-list` delegates its `contextmenu`
* on the `lit-virtualizer` rather than binding one per row and that
* the real `wa-popup` menu opens from it, which is a path with its own
* history of opening and then refusing to work (see
* `menu-keyboard.spec.ts`).
*
* The pointer events are dispatched rather than performed: this project
* runs Desktop Chrome and Desktop Safari, neither of which has touch,
* and a device tier does not exist. So this is honest about what it
* checks the app's own listeners, on the app's own DOM, from the
* events a touch would produce and not about a real finger.
*/
/** A common small phone, as in `phone-shell.spec.ts`. */
const PHONE = { width: 390, height: 844 };
/** Comfortably past the module's 500ms hold. */
const HELD = 900;
type Page = import('@playwright/test').Page;
/** The track list's menu panel, or null while it is not rendered. */
const panel = (page: Page) =>
page.evaluate(() => {
const el = document
.querySelector('track-list')
?.shadowRoot?.querySelector('.context-menu-panel');
if (!el) return null;
return {
role: el.getAttribute('role'),
label: el.getAttribute('aria-label'),
items: el.querySelectorAll('[role="menuitem"]').length,
};
});
/**
* Press the first track row, optionally dragging partway through the
* shape of a scroll that begins on a row, which must not open a menu.
*/
async function pressFirstRow(
page: Page,
opts: { driftY?: number } = {},
): Promise<void> {
await page.evaluate((drift) => {
// `.track-row`, not `[role="row"]`: the column header is a row too,
// and it is the *first* one -- a press on it is correctly ignored,
// which reads exactly like the gesture not working.
const row = document
.querySelector('track-list')
?.shadowRoot?.querySelector('.track-row');
if (!row) throw new Error('no track row to press');
const box = row.getBoundingClientRect();
const x = Math.round(box.left + box.width / 2);
const y = Math.round(box.top + box.height / 2);
const send = (type: string, dy = 0) =>
row.dispatchEvent(
new PointerEvent(type, {
bubbles: true,
composed: true,
cancelable: true,
pointerType: 'touch',
isPrimary: true,
clientX: x,
clientY: y + dy,
}),
);
send('pointerdown');
if (drift) send('pointermove', drift);
}, opts.driftY ?? 0);
}
test.describe('long-press opens the track menu', () => {
test.beforeEach(async ({ app }) => {
await app.setViewportSize(PHONE);
await app.getByTestId('tab-tracks').click();
await expect(app.getByTestId('main-content')).toHaveAttribute(
'data-active-view',
'tracks',
);
});
test.afterEach(async ({ app }) => {
// Every other spec file runs against a desktop, and the viewport
// belongs to the shared context rather than to this file.
await app.setViewportSize({ width: 1440, height: 900 });
});
test('reaches the delegated handler and opens the real menu', async ({
app,
}) => {
await expect.poll(() => panel(app)).toBeNull();
await pressFirstRow(app);
await expect
.poll(() => panel(app), { timeout: HELD + 2000 })
.toMatchObject({ role: 'menu', label: 'Track actions' });
// The same panel Shift+F10 opens, items and all -- not an empty
// popup that happened to become visible.
expect((await panel(app))?.items).toBeGreaterThan(0);
});
test('does not open one for a press that turns into a scroll', async ({
app,
}) => {
await pressFirstRow(app, { driftY: 40 });
await app.waitForTimeout(HELD);
expect(await panel(app)).toBeNull();
});
});
+170
View File
@@ -0,0 +1,170 @@
import { test, expect } from '../support/fixtures.js';
/**
* The phone shell (plan 016 B2, phase 1).
*
* This is the tier that can actually answer the question. Wails v3's
* server mode serves the real frontend, so a Chromium at 390×844 is the
* same document an Android WebView renders the only thing a device
* adds here is the WebView's own quirks, and CI runs the WebKit half
* for exactly that reason.
*
* The assertions are the three things B2 is *for*: the eleven-item
* sidebar is gone, the four destinations plan 016 committed to are
* reachable with a thumb, and nothing scrolls sideways. The last one is
* the one that hides: `overflow-x: auto` on `body` means a shell that
* does not fit produces a scrollbar rather than a broken layout, which
* looks survivable in a screenshot and is not.
*/
/** A common small phone. Narrower than any device this is likely to meet. */
const PHONE = { width: 390, height: 844 };
/** The narrowest thing still sold, near enough. */
const SMALL_PHONE = { width: 360, height: 780 };
const horizontalOverflow = (page: import('@playwright/test').Page) =>
page.evaluate(() => ({
scrollWidth: document.body.scrollWidth,
clientWidth: document.body.clientWidth,
}));
test.describe('the shell on a phone', () => {
test.beforeEach(async ({ app }) => {
await app.setViewportSize(PHONE);
});
test('replaces the sidebar with a bottom tab bar', async ({ app }) => {
await expect(app.locator('div.sidebar')).toBeHidden();
const nav = app.locator('bottom-nav');
await expect(nav).toBeVisible();
// Four tabs and a way to everything else, which is the shape the
// plan argues for: a tab bar is 3-5 items before the targets stop
// being thumb-sized.
for (const id of ['home', 'albums', 'tracks', 'playlists', 'more']) {
await expect(app.getByTestId(`tab-${id}`)).toBeVisible();
}
});
test('navigates from a tab', async ({ app }) => {
await app.getByTestId('tab-albums').click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'albums');
await app.getByTestId('tab-home').click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'home');
});
test('reaches the views with no tab through the drawer', async ({ app }) => {
await app.getByTestId('tab-more').click();
// Scoped to the drawer: the desktop sidebar is still in the DOM
// (hidden by the media query, not removed), so an unscoped testid
// matches two elements and Playwright's strict mode refuses --
// which is the right complaint, since the two really are different
// buttons.
//
// The drawer holds the *same* sidebar the desktop uses, so Settings
// -- which a phone still needs occasionally -- is reachable without
// a second list of destinations to keep in step.
const settings = app
.getByTestId('nav-drawer')
.getByTestId('nav-settings');
await expect(settings).toBeVisible();
await settings.click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'settings');
// And the drawer gets out of the way once it has done its job.
await expect(app.getByTestId('nav-drawer')).toBeHidden();
});
test('has a named drawer', async ({ app }) => {
await app.getByTestId('tab-more').click();
// The a11y snapshot never prints a dialog's name, so this asks for
// the role and the name together -- which is the check that caught
// eleven unnamed dialogs.
await expect(
app.getByRole('dialog', { name: 'All views' }),
).toBeVisible();
});
for (const vp of [PHONE, SMALL_PHONE]) {
test(`does not scroll sideways at ${vp.width}×${vp.height}`, async ({ app }) => {
await app.setViewportSize(vp);
await app.getByTestId('tab-tracks').click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'tracks');
const { scrollWidth, clientWidth } = await horizontalOverflow(app);
expect(scrollWidth, `body overflows by ${scrollWidth - clientWidth}px`)
.toBeLessThanOrEqual(clientWidth);
});
}
test('opens the full-screen now playing, and comes back', async ({ app }) => {
// Something has to be playing for the mini player to be a way in.
await app.getByTestId('tab-tracks').click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'tracks');
await app.locator('track-list .track-row').first().dblclick();
await expect(app.getByTestId('now-playing-title')).not.toBeEmpty();
await app.getByTestId('open-now-playing').click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'now-playing');
// The seek bar and volume that phase 1 took out of the bottom bar
// are here, and they are the *same* components -- this view
// composes the transport rather than reimplementing it.
await expect(app.locator('now-playing-view seek-bar')).toBeVisible();
await expect(app.locator('now-playing-view volume-control')).toBeVisible();
// Back goes where the user came from, through the nav stack.
await app.getByTestId('npv-back').click();
await expect(app.getByTestId('main-content'))
.toHaveAttribute('data-active-view', 'tracks');
});
test('offers no way in on a desktop, where the bar is whole', async ({ app }) => {
await app.setViewportSize({ width: 1440, height: 900 });
// The button exists in the markup at every size; CSS decides. If
// this becomes visible on a desktop it is a 48px hit target over
// the cover art, swallowing the clicks that open the preview.
await expect(app.getByTestId('open-now-playing')).toBeHidden();
});
test('keeps the transport, minus what a thumb cannot use', async ({ app }) => {
// The player bar stays: this is a music player, and what is playing
// has to be visible and pausable from every view.
await expect(app.locator('audio-player')).toBeVisible();
await expect(app.locator('now-playing')).toBeVisible();
// Volume is the hardware keys' job on a phone, and a 4px seek bar
// is not a thumb target -- both belong to a later phase's
// full-screen now-playing view.
await expect(app.locator('audio-player volume-control')).toBeHidden();
});
});
test.describe('the desktop shell is unchanged', () => {
test('keeps the sidebar and hides the tab bar', async ({ app }) => {
await app.setViewportSize({ width: 1440, height: 900 });
await expect(app.locator('div.sidebar')).toBeVisible();
await expect(app.locator('bottom-nav')).toBeHidden();
});
});
+137
View File
@@ -0,0 +1,137 @@
import { test, expect } from '../support/fixtures.js';
/**
* The track list on a phone (plan 016 B2 phase 4).
*
* The component tier pins the arrangement; this pins it in the real
* shell, at the viewport of the device the work was measured on 424 x
* 439, a Light Phone III because the fault it fixes was invisible to
* every assertion the app had. The columns *fit*: `--grid-cols` summed
* to exactly the host width, nothing overflowed, and every column was
* still unreadable. Only a measurement of what a cell can hold, or a
* screenshot, shows that.
*/
type Page = import('@playwright/test').Page;
/** The phone this was built against, in CSS pixels. */
const DEVICE = { width: 424, height: 439 };
/** A common small phone, as the shell specs use. */
const PHONE = { width: 390, height: 844 };
const list = (page: Page) => page.locator('track-list');
/** The row's grid tracks and the widest text a cell can show. */
const rowGeometry = (page: Page) =>
page.evaluate(() => {
const sr = document.querySelector('track-list')?.shadowRoot;
const row = sr?.querySelector('.track-row');
if (!row) return null;
const title = row.querySelector('.stacked-title');
const sub = row.querySelector('.stacked-sub');
return {
tracks: getComputedStyle(row)
.gridTemplateColumns.split(/\s+/)
.filter(Boolean).length,
rowHeight: Math.round(row.getBoundingClientRect().height),
headerRow: !!sr?.querySelector('.header-row'),
handles: sr?.querySelectorAll('.col-resize-handle').length ?? 0,
titleWidth: title ? Math.round(title.getBoundingClientRect().width) : 0,
// A truncated cell is the fault; a cell wider than its text is fine.
titleTruncated: title ? title.scrollWidth > title.clientWidth + 1 : null,
subText: sub?.textContent?.trim() ?? null,
};
});
test.describe('the track list on a phone', () => {
test.beforeEach(async ({ app }) => {
await app.setViewportSize(DEVICE);
await app.getByTestId('tab-tracks').click();
await expect(app.getByTestId('main-content')).toHaveAttribute(
'data-active-view',
'tracks',
);
await expect(list(app).first()).toBeVisible();
});
test.afterEach(async ({ app }) => {
await app.setViewportSize({ width: 1440, height: 900 });
});
test('stacks the title over the artist and drops the pointer affordances', async ({
app,
}) => {
const geo = await rowGeometry(app);
expect(geo).not.toBeNull();
// Favourite + one stacked column + duration.
expect(geo?.tracks).toBe(3);
expect(geo?.headerRow).toBe(false);
expect(geo?.handles).toBe(0);
expect(geo?.subText).toBeTruthy();
// The row height has to match the virtualizer's item size, or rows
// overlap; 52 is that number.
expect(geo?.rowHeight).toBe(52);
});
test('gives the title most of the row instead of a quarter of it', async ({
app,
}) => {
const geo = await rowGeometry(app);
// Four columns at this width gave a title ~102px. The measurement
// that matters is the share of the row, not the pixel count.
expect(geo?.titleWidth ?? 0).toBeGreaterThan(DEVICE.width * 0.55);
});
test('needs no sideways scrolling, and neither does the shell', async ({
app,
}) => {
const overflow = await app.evaluate(() => ({
body: [document.body.scrollWidth, document.body.clientWidth],
list: (() => {
const sr = document.querySelector('track-list')?.shadowRoot;
const row = sr?.querySelector('.track-row');
return row ? [row.scrollWidth, row.clientWidth] : null;
})(),
}));
expect(overflow.body[0]).toBe(overflow.body[1]);
expect(overflow.list?.[0]).toBe(overflow.list?.[1]);
});
test('keeps the sorts a phone has no headers to reach', async ({ app }) => {
// With no column headers, the page header's sort control is the only
// route to sort-by-artist — so it must still offer the columns the
// phone does not draw.
const ids = await app.evaluate(() => {
const header = document
.querySelector('track-list')
?.shadowRoot?.querySelector('page-header') as
| (Element & { sortOptions?: { id: string }[] })
| null;
return (header?.sortOptions ?? []).map((o) => o.id);
});
expect(ids).toContain('artistName');
expect(ids).toContain('album');
});
test('is the desktop list again above the breakpoint', async ({ app }) => {
await app.setViewportSize(PHONE);
await expect.poll(async () => (await rowGeometry(app))?.tracks).toBe(3);
await app.setViewportSize({ width: 1024, height: 800 });
// The same element, re-laid-out: this is one component with two
// column sets, not two components.
await expect.poll(async () => (await rowGeometry(app))?.headerRow).toBe(true);
await expect.poll(async () => (await rowGeometry(app))?.tracks).toBe(5);
});
});
@@ -17,6 +17,14 @@ import * as download$0 from "../download/models.js";
// @ts-ignore: Unused imports
import * as tracklist$0 from "../tracklist/models.js";
/**
* GetAllowMeteredCatalogDownload reports whether the ~0.6 GB Explore
* catalog may be fetched on a metered connection.
*/
export function GetAllowMeteredCatalogDownload(): $CancellablePromise<boolean> {
return $Call.ByID(2258585139);
}
/**
* GetDefaultPage returns the view the app opens to on launch.
*/
@@ -127,6 +135,17 @@ export function Save(): $CancellablePromise<void> {
return $Call.ByID(1988945736);
}
/**
* SetAllowMeteredCatalogDownload saves the metered-download permission.
*
* There is nothing to validate and nothing to restart: the policy is
* read at the moment a download would start, so turning it on takes
* effect on the next attempt rather than needing this launch to be over.
*/
export function SetAllowMeteredCatalogDownload(allow: boolean): $CancellablePromise<void> {
return $Call.ByID(192700351, allow);
}
/**
* SetDefaultPage validates and saves a new launch page.
*/
@@ -12,6 +12,7 @@ export {
export type {
AlbumCompleteFunc,
CreditPart,
IndexStatus,
LBSimilarArtist,
LBTopRecording,
@@ -7,6 +7,23 @@
*/
export type AlbumCompleteFunc = any;
/**
* CreditPart is one credited artist within a credit, in credit order.
*
* CreditedName is the name *as credited*, which is not the artist's own
* name: MusicBrainz credits "Snoop Dogg" on a track by the artist
* called "Snoop Doggy Dogg". Display uses it; navigation uses
* ArtistMBID. JoinPhrase is the literal connector that follows this
* part, so a credit renders by concatenation and never by searching a
* name inside a credit string.
*/
export interface CreditPart {
"position": number;
"artistMbid": string;
"creditedName": string;
"joinPhrase": string;
}
/**
* IndexStatus is the full index build status, exposed to the frontend.
*/
@@ -225,6 +225,18 @@ export function GetCandidateThumbnail(releaseMBID: string, releaseGroupMBID: str
return $Call.ByID(1946932424, releaseMBID, releaseGroupMBID);
}
/**
* GetCredits is the bound form: the frontend asks for a tracklist's
* worth of MBIDs at once rather than one per row.
*
* Batched for the reason every other per-row backend question here is:
* asking on hover or on render turns a list into N IPC round trips, and
* this one is asked about every row of every list in the app.
*/
export function GetCredits(mbids: string[] | null): $CancellablePromise<{ [_ in string]?: $models.CreditPart[] | null } | null> {
return $Call.ByID(225964099, mbids);
}
/**
* GetExploreShelves builds the page Explore shows before a query.
*
+121
View File
@@ -41,6 +41,18 @@ body {
overflow-y: hidden;
}
/* Above the phone breakpoint the tab bar does not exist. It is in the
markup unconditionally and eagerly, for the reason notification-host
is: navigation that has to fetch a chunk before it can navigate is
not navigation. */
@media (min-width: 600px) {
bottom-nav {
display: none;
}
}
p {
margin: 0;
/* I want to set paragraph margins myself */
@@ -130,6 +142,8 @@ body div.sidebar {
contain: layout style paint;
}
.bottom-bar {
grid-area: bottom-bar;
padding: 0.25em;
@@ -241,3 +255,110 @@ body div.sidebar {
pointer-events: none !important;
contain: strict !important;
}
/* ===================================================================
The phone shell (plan 016 B2).
**This section is last on purpose.** A media query adds no
specificity, so `@media (max-width: 599px) { .title { } }` placed
above the plain `.title` rule loses to it -- which is exactly what
happened when this landed in the middle of the file: the header kept
its 2em gutters, its 16px gap and its 24px title on a 390px phone,
and every one of these declarations was dead. Nothing failed,
because the shell fits for a different reason (the `min-width: 0`
below and each component's own media query), so a screenshot was
what caught it.
600px, not the sidebar's 900: 900 is a *laptop* and the response to
it is a narrower sidebar, which is still a sidebar. Below 600 there
is no room for one at all -- 360px of viewport over a 200px nav is
not a layout -- so the navigation moves to the bottom, where a thumb
is, and the eleven-item list moves into `bottom-nav`'s drawer.
=================================================================== */
@media (max-width: 599px) {
body {
grid-template:
"top-bar" 3.25em
"main-panel" 1fr
"bottom-bar" auto
"bottom-nav" auto
/ 1fr;
/* Nothing may scroll sideways here. On a desktop the shell is
allowed to overflow a zoomed-in window (a11y.21 above); a
phone *is* the small viewport, so the shell has to fit it. */
overflow-x: hidden;
}
body div.sidebar {
display: none;
}
bottom-nav {
grid-area: bottom-nav;
}
/* The 2em gutters are half a thumb each at this width, and the
subtitle is already gone from 900 down.
`min-width: 0` is the load-bearing half. A grid item's implicit
minimum is `auto` -- its content -- so a header whose children
ask for 580px makes the *body* 580px wide inside a 360px
viewport, and `overflow-x: hidden` then hides the right-hand
third of the app rather than fitting it. Every box between the
viewport and the content that must shrink needs this. */
.top-bar {
padding-left: 0.75em;
padding-right: 0.75em;
gap: 0.5em;
min-width: 0;
overflow: hidden;
}
.content-area,
.main-panel,
.bottom-bar {
min-width: 0;
}
.title {
font-size: 1.1em;
}
/* The search box is the one header control worth its width; the
library filter is a rarely-changed setting and reachable from
the drawer's Settings. */
.top-bar library-filter {
display: none;
}
/* The full-screen now-playing view *is* the transport, so the bar
repeating it underneath is 4em of a small screen spent saying
the same thing twice -- visible in a screenshot, invisible to
every assertion about either one.
`:has()` rather than a class toggled from index.ts: which view
is showing is already published as an attribute, and a second
expression of the same fact is a second thing to keep in step.
The view carries its own queue button, because this is where
that one lived. */
body:has(#main-content[data-active-view="now-playing"]) .bottom-bar {
display: none;
}
.top-bar search-bar {
flex: 1 1 auto;
min-width: 0;
}
}
@media (max-width: 599px) {
.bottom-bar {
grid-template-columns: minmax(0, 1fr) auto auto;
gap: 0.25em;
}
.bottom-bar audio-player {
margin: 0.25em;
}
}
+7
View File
@@ -41,6 +41,13 @@
<wa-icon name="list"></wa-icon>
</button>
</footer>
<!-- The phone's primary navigation, hidden above 600px by
index.css. Eager rather than a chunk, for the reason
notification-host is: it is the only way to move around the
app on a phone. After the footer, because that is where it
renders -- the tab bar sits below the transport, and DOM order
is what a screen reader and the tab sequence follow. -->
<bottom-nav></bottom-nav>
<first-run-wizard></first-run-wizard>
<notification-host></notification-host>
<shortcuts-overlay></shortcuts-overlay>
+95 -25
View File
@@ -21,6 +21,7 @@ import '@components/audio-player/audio-player.ts';
import '@components/track-list/track-list.ts';
import '@components/now-playing/now-playing.ts';
import '@components/sidebar/app-sidebar.ts';
import '@components/bottom-nav/bottom-nav.ts';
import '@components/queue-panel/queue-panel.ts';
import '@components/search-bar/search-bar.ts';
import '@components/library-filter/library-filter.ts';
@@ -49,6 +50,7 @@ import '@store/theme-store';
// registers the document keydown listener for global shortcuts.
import './src/services/keyboard-shortcut-service';
import { activateView, deactivateView } from '@utils/view-lifecycle';
import { installLongPressContextMenu } from '@utils/long-press';
import {
hasTrackPayload,
getDragPayload,
@@ -63,6 +65,11 @@ setBasePath('/dist/webawesome');
// the session.
registerBundledIcons();
// The touch equivalent of a right-click, installed once for every menu
// in the app rather than per component. Harmless on a desktop: it acts
// on `pointerType === 'touch'` only.
installLongPressContextMenu();
// ---------------------------------------------------------------------------
// View caching navigation system
// ---------------------------------------------------------------------------
@@ -126,6 +133,11 @@ const DETAIL_LOADERS: Record<string, () => Promise<unknown>> = {
import('@components/explore-artist-details/explore-artist-details.js'),
'explore-album-details': () =>
import('@components/explore-album-details/explore-album-details.js'),
// A detail view rather than a primary one on purpose: it is
// somewhere you go and come back from, so the nav stack carries
// the way out (016 B2 phase 2).
'now-playing': () =>
import('@components/now-playing-view/now-playing-view.ts'),
};
// Opened from a menu rather than by navigating, so they have no entry
@@ -148,10 +160,6 @@ const viewCache = new Map<string, HTMLElement>();
let currentViewEl: HTMLElement | null = null;
let currentDetailEl: HTMLElement | null = null;
/** Navigation history stack for back-button support in detail views. */
const navStack: Array<{ view: string; [key: string]: any }> = [];
/** The current navigation detail (so we can push it onto the stack). */
let currentNavDetail: { view: string; [key: string]: any } = { view: 'home' };
const mainContent = document.getElementById('main-content');
@@ -184,6 +192,71 @@ document.addEventListener('navigate', (e: Event) => {
void handleNavigate((e as CustomEvent).detail);
});
// ---------------------------------------------------------------------------
// The platform's back gesture
// ---------------------------------------------------------------------------
// Android's back button is not a keystroke the page can bind: the
// scaffold's `MainActivity.onBackPressed` asks `webView.canGoBack()` and
// otherwise finishes the activity. This app never touched `history`, so
// that was always false and back quit the app from any depth -- reported
// from a device as "back does not navigate back".
//
// So a navigation is a history entry, and back is `popstate`. It hooks
// the platform's own mechanism rather than a JNI callback of our own,
// which is the same reason `events.ts` hooks the runtime's transport:
// the Java half needs no change, and the behaviour is testable in a
// browser (`page.goBack()`) instead of only on a phone.
//
// Two rules keep the two stacks from disagreeing. A navigation that
// *came from* history pushes nothing (`_isBack`), or going back would
// deepen the stack it is unwinding. And the in-app back buttons --
// `navigate-back`, which the detail views and `now-playing-view` fire --
// go through `history.back()` rather than popping `navStack`
// themselves, so one press cannot consume two entries.
/** The navigation an entry stands for. `undefined` on the entry that
* predates the app's own routing, which is the one back exits from. */
type NavState = { yjNav?: { view: string; [key: string]: any } };
/** Whether the app's first navigation has been recorded. It *replaces*
* the launch entry rather than pushing, or every launch would cost one
* back press before the app would exit. */
let historyStarted = false;
/** How many entries this session has pushed beyond that first one --
* i.e. how deep back can go while staying inside the app. */
let pushedEntries = 0;
function recordNavigation(detail: { view: string; [key: string]: any }): void {
// `_isBack` is bookkeeping, not destination: keeping it in the entry
// would make a replayed navigation claim to be a back-navigation.
const { _isBack: _ignored, ...nav } = detail;
const state: NavState = { yjNav: nav };
// Same URL, deliberately: the app has no routes, and a path a
// reload cannot resolve is worse than no path at all.
if (historyStarted) {
history.pushState(state, '');
pushedEntries += 1;
} else {
history.replaceState(state, '');
historyStarted = true;
}
}
window.addEventListener('popstate', (e: PopStateEvent) => {
const nav = (e.state as NavState | null)?.yjNav;
// Before the app's first navigation, or an entry somebody else
// pushed: nothing to restore, and the activity should be free to
// finish.
if (!nav) return;
pushedEntries = Math.max(0, pushedEntries - 1);
void handleNavigate({ ...nav, _isBack: true });
});
async function handleNavigate(
detail: { view: string; [key: string]: any },
): Promise<void> {
@@ -193,6 +266,8 @@ async function handleNavigate(
const seq = ++navSeq;
if (!detail._isBack) recordNavigation(detail);
// Bookkeeping stays synchronous with the click: the search box's
// scope and the active-view attribute describe the navigation that
// was *asked for*, and are what the rest of the app and the e2e
@@ -206,9 +281,6 @@ async function handleNavigate(
// --- Primary (cacheable) views ----------------------------------------
if (view in VIEW_TAGS) {
// Navigating to a primary view clears the history stack.
navStack.length = 0;
// Remove any active detail view first
if (currentDetailEl) {
deactivateView(currentDetailEl);
@@ -241,7 +313,6 @@ async function handleNavigate(
// the way out. Either way this is the call that starts it.
activateView(target);
currentViewEl = target;
currentNavDetail = { view };
return;
}
@@ -250,12 +321,6 @@ async function handleNavigate(
if (seq !== navSeq) return;
// --- Detail (ephemeral) views -----------------------------------------
// Push the current view onto the nav stack before switching
// (unless this is a back-navigation, which already popped).
if (!detail._isBack) {
navStack.push({ ...currentNavDetail });
}
// Hide the current primary view
if (currentViewEl) {
currentViewEl.classList.add('view-hidden');
@@ -268,8 +333,6 @@ async function handleNavigate(
currentDetailEl = null;
}
currentNavDetail = { ...detail };
switch (view) {
case 'artist-details': {
const { artistId, artistName } = detail;
@@ -304,6 +367,13 @@ async function handleNavigate(
currentDetailEl = spEl;
break;
}
case 'now-playing': {
const npEl = document.createElement('now-playing-view');
mainContent.appendChild(npEl);
currentDetailEl = npEl;
break;
}
case 'genre-details': {
const { genreName } = detail;
const genreEl = document.createElement('genre-details');
@@ -406,16 +476,16 @@ function schedule(fn: () => void): void {
setTimeout(fn, 200);
}
// Navigate-back: pop the nav stack and re-dispatch as a regular navigate.
// Navigate-back: the in-app back buttons, which are the same press as
// the phone's. It goes through the history rather than a stack of its
// own, so one press is one entry however it arrived -- two stacks is
// how a detail view's own button and the back gesture come to disagree.
//
// At the root there is nothing of ours to go back to, and going back
// anyway would leave the app: the depth check is what stops a stray
// `navigate-back` closing it.
document.addEventListener('navigate-back', () => {
const prev = navStack.pop();
if (prev) {
document.dispatchEvent(new CustomEvent('navigate', {
bubbles: true,
composed: true,
detail: { ...prev, _isBack: true },
}));
}
if (pushedEntries > 0) history.back();
});
// Navigate to the user's configured launch page. Falls back to 'home'
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M0 96C0 78.3 14.3 64 32 64l384 0c17.7 0 32 14.3 32 32s-14.3 32-32 32L32 128C14.3 128 0 113.7 0 96zM0 256c0-17.7 14.3-32 32-32l384 0c17.7 0 32 14.3 32 32s-14.3 32-32 32L32 288c-17.7 0-32-14.3-32-32zM448 416c0 17.7-14.3 32-32 32L32 448c-17.7 0-32-14.3-32-32s14.3-32 32-32l384 0c17.7 0 32 14.3 32 32z"/></svg>

After

Width:  |  Height:  |  Size: 608 B

@@ -32,6 +32,23 @@ export class AudioPlayer extends LitElement {
flex: 1;
}
/* The phone transport (plan 016 B2): the buttons, and nothing
else. A media query inside a shadow root is answered by the
viewport, not by the host, so this is the component saying what
it drops at phone width rather than the shell reaching in.
Volume goes because the hardware keys own it on a phone --
Android routes them to the media stream, which is also why
mediacontrols' Android handler implements no volume callback.
The seek bar goes because a 4px-tall target dragged with a thumb
is not a seek control; seeking belongs to the full-screen
now-playing view, which is the next phase. */
@media (max-width: 599px) {
volume-control,
seek-bar {
display: none;
}
}
`];
override render() {
@@ -27,6 +27,18 @@ export class SeekBar extends LitElement {
private showRemaining: boolean = true;
static override styles = [designTokens, waSliderLabel, css`
/* 12px below the phone breakpoint. The bottom bar's seek bar is
display:none there (016 B2 phase 1), so the only instance a
viewport media query can reach at that width is the full-screen
now-playing view's -- which is exactly the one a thumb uses.
The track size lives on wa-slider inside this shadow root, so a
custom property set by the host would not reach it. */
@media (max-width: 599px) {
wa-slider {
--track-size: 12px;
}
}
wa-slider {
--track-size: 6px;
flex: 1;
@@ -0,0 +1,253 @@
import { LitElement, html, css, nothing } from 'lit';
import { customElement, state, query } from 'lit/decorators.js';
import '@awesome.me/webawesome/dist/components/icon/icon.js';
import '@awesome.me/webawesome/dist/components/drawer/drawer.js';
import type WaDrawer from '@awesome.me/webawesome/dist/components/drawer/drawer.js';
import { designTokens } from '../../styles/tokens.css';
import '../sidebar/app-sidebar.js';
import { nameDialog } from '@utils/name-dialog';
type View = 'home' | 'albums' | 'tracks' | 'playlists';
interface Tab {
id: View;
label: string;
icon: string;
}
/**
* The phone's primary navigation: a bottom tab bar, shown only below
* the phone breakpoint (index.css owns that; this element is
* `display: none` above it).
*
* **Four destinations and a way to everything else.** A tab bar is
* three to five items before the targets stop being thumb-sized
* 360 px over eleven sidebar entries is 32 px each so the four here
* are the ones plan 016's subset says a phone is *for*, and "More"
* opens the existing `<app-sidebar>` in a drawer. That is deliberately
* a reuse rather than a second nav: two lists of destinations is two
* places to add the next view to, and the sidebar already carries the
* drag-to-navigate behaviour, the active state and the labels.
*
* It emits the same bubbling, composed `navigate` event the sidebar
* does, so `index.ts` needs no knowledge of it, and it listens for that
* event globally for the same reason the sidebar does: a navigation it
* did not send (a card click, a detail view, the drawer) still has to
* move the highlight.
*/
@customElement('bottom-nav')
export class BottomNav extends LitElement {
static override styles = [designTokens, css`
:host {
display: block;
background-color: var(--yj-bg-elevated, #343a40);
border-top: 1px solid var(--yj-border, #495057);
/* The home indicator on a gesture-navigation phone sits
under the last few pixels of the viewport, so the bar
pads itself out of the way where the browser reports
one and by nothing where it does not. */
padding-bottom: env(safe-area-inset-bottom, 0);
}
nav ul {
display: grid;
grid-auto-flow: column;
grid-auto-columns: 1fr;
margin: 0;
padding: 0;
list-style: none;
}
button {
width: 100%;
/* 48px is the smallest target this should ever be; the
label sits under the icon rather than beside it, which
is what keeps five of them legible at 360px. */
min-height: 48px;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 2px;
padding: 4px 0;
background: none;
border: none;
color: var(--yj-text-secondary, #adb5bd);
cursor: pointer;
font-family: inherit;
font-size: var(--yj-font-size-xs, 0.7rem);
}
button wa-icon {
font-size: 1.15rem;
}
button.active {
color: var(--yj-accent, #ffd43b);
}
button:focus-visible {
outline: 2px solid var(--yj-accent, #ffd43b);
outline-offset: -2px;
}
.label {
/* A tab label is an aid, not the name: the button's own
accessible name comes from its text, and truncating it
visually does not change that. */
max-width: 100%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
wa-drawer::part(body) {
padding: 0;
}
app-sidebar {
/* The sidebar sizes itself inline and collapses to icons
below 900px, which is every phone. In the drawer there
is room for the labels, so it is told not to. */
height: 100%;
}
`];
@state()
private activeView = 'home';
/**
* Whether the drawer has been asked for.
*
* The sidebar inside it is rendered only while this is true, and
* that is not an optimisation. `app-sidebar` carries a
* `data-testid` per destination, so a second copy standing by in
* the DOM makes every `nav-*` testid ambiguous **for the whole
* app** -- 30 existing specs failed with "strict mode violation:
* resolved to 2 elements" on a desktop viewport where this element
* is not even visible. A duplicate of a shared component is a
* duplicate of its handles.
*/
@state()
private drawerOpen = false;
@query('wa-drawer')
private drawer?: WaDrawer;
private static readonly TABS: Tab[] = [
{ id: 'home', label: 'Home', icon: 'house' },
{ id: 'albums', label: 'Albums', icon: 'compact-disc' },
{ id: 'tracks', label: 'Tracks', icon: 'music' },
{ id: 'playlists', label: 'Playlists', icon: 'list' },
];
override connectedCallback() {
super.connectedCallback();
document.addEventListener(
'navigate',
this.onGlobalNavigate as EventListener,
);
}
override disconnectedCallback() {
super.disconnectedCallback();
document.removeEventListener(
'navigate',
this.onGlobalNavigate as EventListener,
);
}
override updated() {
// Web Awesome renders its heading into its own shadow root and
// never points aria-labelledby at it, so the drawer would
// otherwise be announced unnamed -- the same fix, and the same
// reason, as every wa-dialog in the app. A drawer's shadow root
// has the same shape, so the helper needs no change.
nameDialog(this.drawer);
}
private onGlobalNavigate = (e: Event) => {
const detail = (e as CustomEvent<{ view?: string }>).detail;
if (detail?.view) this.activeView = detail.view;
// A navigation from inside the drawer is the drawer's job done.
this.drawerOpen = false;
};
private openDrawer = () => {
this.drawerOpen = true;
};
/**
* Web Awesome closes itself on Escape and on a click outside, and
* tells us afterwards rather than asking -- so the flag follows the
* element, or the next `open` would be a no-op against a drawer
* that thinks it is already open.
*/
private onDrawerHide = () => {
this.drawerOpen = false;
};
private navigate(view: View) {
this.dispatchEvent(new CustomEvent('navigate', {
detail: { view },
bubbles: true,
composed: true,
}));
}
override render() {
return html`
<nav aria-label="Primary">
<ul>
${BottomNav.TABS.map((tab) => html`
<li>
<button
type="button"
class=${this.activeView === tab.id ? 'active' : ''}
data-testid="tab-${tab.id}"
aria-current=${this.activeView === tab.id
? 'page'
: 'false'}
@click=${() => this.navigate(tab.id)}
>
<wa-icon name=${tab.icon}></wa-icon>
<span class="label">${tab.label}</span>
</button>
</li>
`)}
<li>
<button
type="button"
data-testid="tab-more"
aria-haspopup="dialog"
@click=${this.openDrawer}
>
<wa-icon name="bars"></wa-icon>
<span class="label">More</span>
</button>
</li>
</ul>
</nav>
<wa-drawer
placement="start"
label="All views"
data-testid="nav-drawer"
?open=${this.drawerOpen}
@wa-after-hide=${this.onDrawerHide}
>
${this.drawerOpen
? html`<app-sidebar expanded></app-sidebar>`
: nothing}
</wa-drawer>
`;
}
}
declare global {
interface HTMLElementTagNameMap {
'bottom-nav': BottomNav;
}
}
@@ -17,6 +17,8 @@ import {
SetDefaultPage,
GetQueueFallback,
SetQueueFallback,
GetAllowMeteredCatalogDownload,
SetAllowMeteredCatalogDownload,
} from '@go/config/config.js';
import { GetIndexStatus } from '@go/explore/service.js';
import { notificationStore } from '@store/notification-store';
@@ -75,6 +77,9 @@ export class ConfigPage extends ViewLifecycleMixin(LitElement) {
// --- Now Playing state ---
@state() private scrollMode = 'hover';
/** Whether the ~0.6 GB catalog may be fetched on mobile data. */
@state() private allowMeteredCatalogDownload = false;
// --- Favorites state ---
@state() private playlists: playlist.Summary[] = [];
@@ -888,17 +893,20 @@ export class ConfigPage extends ViewLifecycleMixin(LitElement) {
private async loadLibraries(): Promise<void> {
try {
const [libs, mode, defaultPage, queueFallback] = await Promise.all([
GetAllLibrariesWithTrackCounts(),
GetScanConcurrency(),
GetDefaultPage(),
GetQueueFallback(),
]);
const [libs, mode, defaultPage, queueFallback, allowMetered] =
await Promise.all([
GetAllLibrariesWithTrackCounts(),
GetScanConcurrency(),
GetDefaultPage(),
GetQueueFallback(),
GetAllowMeteredCatalogDownload(),
]);
this.libraries = libs ?? [];
this.concurrencyMode = mode;
this.defaultPage = defaultPage;
this.queueFallback = queueFallback;
this.allowMeteredCatalogDownload = allowMetered;
} catch (err) {
console.error(
@@ -1500,10 +1508,53 @@ export class ConfigPage extends ViewLifecycleMixin(LitElement) {
</div>`
: html`<div class="index-loading">Loading status…</div>`}
</div>
<config-field
.schema=${{
key: 'allowMeteredCatalogDownload',
label: 'Download the catalog on mobile data',
description:
'The catalog is about 0.6 GB. It is skipped on a '
+ 'cellular connection unless this is on; a '
+ 'metered Wi-Fi network cannot be detected.',
type: 'toggle' as const,
}}
.value=${this.allowMeteredCatalogDownload}
@config-change=${this.handleAllowMeteredChange}
></config-field>
</config-section>
`;
}
/**
* The catalog download's one permission (plan 016 B4).
*
* It is in this section rather than General because it is about
* *this* download and nothing else, and because the section already
* explains what the catalog is the toggle would be unreadable
* beside "Default page".
*/
private handleAllowMeteredChange = (
e: CustomEvent<ConfigFieldChangeEvent>,
): void => {
const allow = Boolean(e.detail.value);
const previous = this.allowMeteredCatalogDownload;
this.allowMeteredCatalogDownload = allow;
void SetAllowMeteredCatalogDownload(allow).catch((err: unknown) => {
console.error('failed to save metered download permission', err);
// The visible state reverted, so this is the Transient case:
// a small action the user can simply repeat.
this.allowMeteredCatalogDownload = previous;
notificationStore.transient({
key: 'metered-catalog-setting',
title: 'Setting not saved',
text: describeError(err, 'That setting could not be saved.'),
});
});
};
private tierIcon(state: string): string {
switch (state) {
case 'complete':
@@ -52,7 +52,8 @@ import {
} from '@utils/context-menu-controller.js';
import type { ContextMenuHost } from '@utils/context-menu-controller.js';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { artistLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import {
createAlbumArtDragImage,
createDragImage,
@@ -425,8 +426,18 @@ export class CoverGrid
* Lifecycle
* ==================================================================== */
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback() {
super.connectedCallback();
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
// Two virtualizers when the grid is split; both draw rows.
this.renderRoot?.querySelectorAll('lit-virtualizer')
.forEach((v) => (v as unknown as { requestUpdate(): void }).requestUpdate());
});
this.restoreSortPreferences();
this.loadAlbums();
@@ -441,6 +452,8 @@ export class CoverGrid
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
this.removeEventListener(
'error',
@@ -1820,7 +1833,7 @@ export class CoverGrid
class="artist-name"
title="${album.ArtistName}"
>
${artistLink(album.ArtistName, album.ArtistMBID ?? '')}
${creditLink(creditStore.credits(album.MBID), album.ArtistName, album.ArtistMBID ?? '')}
</div>
</div>
</div>
@@ -20,7 +20,8 @@ type MBRelease = explore.MBRelease;
type MBTrack = explore.MBTrack;
import { exploreCache } from '../../store/explore-cache';
import { libraryStore } from '../../store/library-store';
import { artistLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import { describeError } from '../../utils/describe-error';
import { EventsOn } from '@runtime/runtime';
import { Events } from '../../events';
@@ -704,8 +705,15 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
* BrowseReleases fetch never signals readiness. */
private releasesFallbackTimer?: number;
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback() {
super.connectedCallback();
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
});
if (this.releaseGroupMBID || this.localAlbumId) {
void this.loadAllData();
}
@@ -760,6 +768,8 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
this.downloadUnsub?.();
this.downloadUnsub = null;
this.unsubReleasesReady?.();
@@ -2850,7 +2860,11 @@ export class ExploreAlbumDetails extends LitElement implements ContextMenuHost {
return html`
${artist
? html`<div class="album-artist">
${artistLink(artist, artistMbid)}
${creditLink(
creditStore.credits(this.releaseGroupMBID),
artist,
artistMbid,
)}
</div>`
: nothing}
${metaParts.length > 0
@@ -16,7 +16,8 @@ import { exploreCache, ARTIST_IMAGE_CACHE_LIMIT } from '../../store/explore-cach
import { queueStore } from '../../store/queue-store';
import { notificationStore } from '../../store/notification-store';
import '../notifications/inline-notice';
import { artistLink, trackLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, trackLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import { describeError } from '../../utils/describe-error';
import '@awesome.me/webawesome/dist/components/icon/icon.js';
import '../library-status-indicator/library-status-indicator.js';
@@ -774,6 +775,14 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
}
protected override onViewActivate(): void {
// A cached primary view, so this is torn down on the way out
// rather than on disconnect — which never fires here.
this.whileActive(
creditStore.subscribe(() => {
this.requestUpdate();
}),
);
// Fetched on arrival rather than on connect: this is a cached
// primary view, created and warmed at startup, so a fetch there
// is three catalog queries every user pays for whether or not
@@ -2193,7 +2202,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
<div class="album-title" title="${rg.title}">
${rg.title}
</div>
<div class="album-artist">${artistLink(rg.artistCredit, rg.artistMbid ?? '')}</div>
<div class="album-artist">${creditLink(creditStore.credits(rg.mbid), rg.artistCredit, rg.artistMbid ?? '')}</div>
<div class="album-meta">
<div class="album-meta-text">
${rg.primaryType
@@ -2255,7 +2264,7 @@ export class ExploreView extends ViewLifecycleMixin(LitElement) implements Conte
${trackLink(r.title, r.releaseName ?? '', r.releaseGroupMbid ?? '', r.mbid)}
</div>
<div class="track-artist">
${artistLink(r.artistCredit, r.artistMbid ?? '')}
${creditLink(creditStore.credits(r.mbid), r.artistCredit, r.artistMbid ?? '')}
</div>
</div>
<div class="track-meta">
@@ -149,6 +149,19 @@ export class JobIndicator extends LitElement {
text-overflow: ellipsis;
}
/* On a phone the ring is the whole indicator: "3 background
jobs" is 114px of a 360px header, and it pushed the
header past the viewport. Only the *visible* label
goes -- the live region in render() is what announces
this, and it is unaffected, so the ring keeps its
accessible name and screen readers keep hearing the
state change. */
@media (max-width: 599px) {
.label {
display: none;
}
}
.alert-dot {
width: 6px;
height: 6px;
@@ -0,0 +1,353 @@
import { LitElement, html, css, nothing } from 'lit';
import { customElement } from 'lit/decorators.js';
import '@awesome.me/webawesome/dist/components/icon/icon.js';
import '../audio-player/controls/player-controls';
import '../audio-player/seekbar/seek-bar';
import '../audio-player/volume-control/volume-control';
import {
creditLink,
albumLink,
exploreLinkStyles,
} from '@utils/explore-link';
import { PlayerController } from '@store/controllers/player-controller';
import { creditStore } from '@store/credit-store';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { designTokens } from '../../styles/tokens.css';
import { srOnly } from '../../styles/sr-only.css';
/**
* What is playing, at the size a phone has room for (plan 016 B2,
* phase 2).
*
* Phase 1 took the seek bar and the volume out of the bottom bar,
* because 4px of height is not a thumb target and a phone's volume
* belongs to its hardware keys. This is where they went: the same
* `<seek-bar>`, `<player-controls>` and `<volume-control>` elements the
* desktop transport uses, given room. **Not copies of them** a phone
* layout that reimplements the transport is a second transport to fix
* every bug in, and the seek bar in particular carries the
* interpolation rules that took a plan of their own to get right.
*
* It is a *detail* view rather than a primary one: it is somewhere you
* go and come back from, so `index.ts` pushes the current view onto the
* nav stack and Back pops it. That is also why it is not in the tab
* bar a tab you cannot leave by pressing the same tab again is not a
* tab.
*/
@customElement('now-playing-view')
export class NowPlayingView extends LitElement {
private player = new PlayerController(this);
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback(): void {
super.connectedCallback();
// Credits arrive after the track does, so the name this view is
// already showing has to be re-rendered when they land.
this.creditsUnsub = creditStore.subscribe(() => this.requestUpdate());
}
override disconnectedCallback(): void {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
}
private favCtrl = new FavoritesController(this);
static override styles = [designTokens, srOnly, exploreLinkStyles, css`
:host {
display: flex;
flex-direction: column;
height: 100%;
box-sizing: border-box;
padding: 0.75em 1em 1.25em;
gap: 0.75em;
background-color: var(--yj-bg-surface, #212529);
overflow-y: auto;
}
header {
display: flex;
align-items: center;
gap: 0.5em;
flex: 0 0 auto;
}
.context {
flex: 1 1 auto;
}
.back {
background: none;
border: none;
color: var(--yj-text-primary, #f8f9fa);
/* 48px is the touch-target floor, and this is the control
that gets a user out of a full-screen view. */
min-width: 48px;
min-height: 48px;
font-size: 1.1rem;
cursor: pointer;
border-radius: 6px;
}
.back:focus-visible {
outline: 2px solid var(--yj-accent, #ffd43b);
outline-offset: -2px;
}
.context {
font-size: var(--yj-font-size-xs, 0.75rem);
letter-spacing: 0.08em;
text-transform: uppercase;
color: var(--yj-text-secondary, #adb5bd);
}
.art {
flex: 1 1 auto;
display: flex;
align-items: center;
justify-content: center;
min-height: 0;
}
.art img,
.art .placeholder {
/* Square, and never taller than the room left over: the
art is the one thing here that would happily push the
transport off the bottom of a short phone. */
width: min(100%, 60vh);
aspect-ratio: 1;
object-fit: cover;
border-radius: 12px;
background-color: var(--yj-bg-elevated, #343a40);
}
.art .placeholder {
display: flex;
align-items: center;
justify-content: center;
font-size: 3rem;
color: var(--yj-text-tertiary, #868e96);
}
.meta {
flex: 0 0 auto;
display: flex;
align-items: center;
gap: 0.75em;
min-width: 0;
}
.names {
flex: 1 1 auto;
min-width: 0;
}
.title {
font-size: 1.15rem;
font-weight: 600;
margin: 0;
/* Two lines, then an ellipsis. A marquee is the bottom
bar's answer to a 320px box; here there is room to wrap,
and wrapping does not move. */
display: -webkit-box;
-webkit-line-clamp: 2;
-webkit-box-orient: vertical;
overflow: hidden;
}
.artist,
.album {
margin: 0;
font-size: 0.9rem;
color: var(--yj-text-secondary, #adb5bd);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.favorite {
background: none;
border: none;
color: var(--yj-text-secondary, #adb5bd);
min-width: 48px;
min-height: 48px;
font-size: 1.25rem;
cursor: pointer;
border-radius: 6px;
}
.favorite.on {
color: var(--yj-accent, #ffd43b);
}
.favorite:focus-visible {
outline: 2px solid var(--yj-accent, #ffd43b);
outline-offset: -2px;
}
.transport {
flex: 0 0 auto;
display: flex;
flex-direction: column;
gap: 0.5em;
}
/* The seek bar is the reason this view exists. Its own
stylesheet thickens the track below the phone breakpoint --
the track size is set on the wa-slider inside its shadow
root, so a custom property set from here would not reach
it. */
seek-bar {
display: block;
}
.empty {
flex: 1 1 auto;
display: flex;
align-items: center;
justify-content: center;
color: var(--yj-text-secondary, #adb5bd);
text-align: center;
}
`];
private back() {
this.dispatchEvent(new CustomEvent('navigate-back', {
bubbles: true,
composed: true,
}));
}
/**
* Open the queue.
*
* This view hides the bottom bar (index.css), and the bar is where
* the queue button lives -- so without this, going full-screen
* would take the queue away. It toggles the same `open` attribute
* `index.ts` does, because the panel's state is an attribute on one
* element and a second mechanism for it is a second thing to keep
* in step.
*/
private openQueue() {
document.getElementById('queue-panel')?.setAttribute('open', '');
}
private toggleFavorite() {
const path = this.player.currentTrack?.filePath;
if (path) void this.favCtrl.toggleFavorite(path);
}
override render() {
const track = this.player.currentTrack;
if (!track) {
return html`
${this.renderHeader()}
<p class="empty" data-testid="npv-empty">
Nothing is playing.
</p>
`;
}
const favorited = this.favCtrl.isFavorited(track.filePath);
// The largest kept tier, which is what `saveCoverArt` records as
// the path -- there is no full-resolution original to reach for.
const art = track.coverArtLarge || track.coverArt;
return html`
${this.renderHeader()}
<div class="art">
${art
? html`<img
src=${art}
alt=""
decoding="async"
data-testid="npv-art"
/>`
: html`<div class="placeholder" aria-hidden="true">
<wa-icon name="compact-disc"></wa-icon>
</div>`}
</div>
<div class="meta">
<div class="names">
<h2 class="title" data-testid="npv-title">
${track.title || track.fileName}
</h2>
<p class="artist">
${creditLink(
creditStore.credits(track.recordingMbid),
track.artist,
track.artistMbid,
)}
</p>
${track.album
? html`<p class="album">
${albumLink(
track.album,
track.releaseGroupMbid,
undefined,
track.artist,
)}
</p>`
: nothing}
</div>
<button
type="button"
class="favorite ${favorited ? 'on' : ''}"
data-testid="npv-favorite"
aria-pressed=${favorited ? 'true' : 'false'}
aria-label=${favorited
? `Remove ${track.title} from ${this.favCtrl.playlistName}`
: `Add ${track.title} to ${this.favCtrl.playlistName}`}
@click=${this.toggleFavorite}
>
<wa-icon name=${this.favCtrl.iconName}></wa-icon>
</button>
</div>
<div class="transport">
<seek-bar></seek-bar>
<player-controls></player-controls>
<volume-control></volume-control>
</div>
`;
}
private renderHeader() {
return html`
<header>
<button
type="button"
class="back"
data-testid="npv-back"
aria-label="Back"
@click=${this.back}
>
<wa-icon name="chevron-down"></wa-icon>
</button>
<span class="context">Now playing</span>
<button
type="button"
class="back"
data-testid="npv-queue"
aria-label="Show the queue"
@click=${this.openQueue}
>
<wa-icon name="list"></wa-icon>
</button>
</header>
`;
}
}
declare global {
interface HTMLElementTagNameMap {
'now-playing-view': NowPlayingView;
}
}
@@ -4,7 +4,7 @@ import '@awesome.me/webawesome/dist/components/icon/icon.js';
import '@awesome.me/webawesome/dist/components/popup/popup.js';
import type WaPopup from '@awesome.me/webawesome/dist/components/popup/popup.js';
import {
artistLink,
creditLink,
trackLink,
exploreLinkStyles,
} from '@utils/explore-link';
@@ -14,6 +14,7 @@ import {
navigateToQueueSource,
} from '@utils/queue-source-link';
import { PlayerController } from '@store/controllers/player-controller';
import { creditStore } from '@store/credit-store';
import { QueueController } from '@store/controllers/queue-controller';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { designTokens } from '../../styles/tokens.css';
@@ -146,6 +147,41 @@ export class NowPlaying extends LitElement {
position: relative;
}
/* The phone's way into the full-screen now-playing view (016 B2
phase 2). It sits over the cover art rather than being a
thirteenth control in a 360px bar, and it is a *button* rather
than a click handler on the art because it is an action with a
name -- the art itself is decorative and the title beside it
already navigates somewhere else (the catalog page).
CSS owns whether it exists, the same way it does for bottom-nav:
there is no viewport check in the component. */
.expand {
display: none;
}
@media (max-width: 599px) {
.expand {
position: absolute;
inset: 0;
display: block;
width: 100%;
height: 100%;
padding: 0;
background: none;
border: none;
border-radius: 4px;
cursor: pointer;
/* The art shows through; this is a target, not a picture. */
color: transparent;
}
.expand:focus-visible {
outline: 2px solid var(--yj-accent, #ffd43b);
outline-offset: 2px;
}
}
.cover-preview-panel {
width: 500px;
height: 500px;
@@ -290,6 +326,9 @@ export class NowPlaying extends LitElement {
}
`];
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback() {
super.connectedCallback();
this.loadScrollMode();
@@ -306,10 +345,21 @@ export class NowPlaying extends LitElement {
this.geometryDirty = true;
this.requestUpdate();
});
// A credit arriving changes the rendered text, and the marquee
// measures that text — so this is a geometry change, not just a
// repaint. Saying so is what stops the bar scrolling to the
// old width.
this.creditsUnsub = creditStore.subscribe(() => {
this.geometryDirty = true;
this.requestUpdate();
});
}
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
// A drag interrupted by the bar going away still has to clean up.
this.attachDragListeners(false);
window.removeEventListener(SCROLL_CHANGE_EVENT, this.handleScrollModeEvent);
@@ -377,6 +427,13 @@ export class NowPlaying extends LitElement {
<div class="sr-only" role="status" aria-live="polite">${announcement}</div>
<div class="now-playing">
<div class="cover-art-wrapper">
<button
type="button"
class="expand"
data-testid="open-now-playing"
aria-label="Open now playing"
@click=${this.openNowPlaying}
></button>
<div
class="cover-art"
@mouseenter=${this.handleCoverMouseEnter}
@@ -441,7 +498,7 @@ export class NowPlaying extends LitElement {
@mouseleave=${this.handleArtistMouseLeave}
@transitionend=${() => this.onScrollCycleEnd('artist')}
>
<span class="scroll-content">${artistLink(track.artist, track.artistMbid) || 'Unknown Artist'}</span>
<span class="scroll-content">${creditLink(creditStore.credits(track.recordingMbid), track.artist, track.artistMbid) || 'Unknown Artist'}</span>
</span>
${describeQueueSource(this.queue.source)
? html`
@@ -485,6 +542,15 @@ export class NowPlaying extends LitElement {
`;
}
/** Open the full-screen view. Phone only; see `.expand`. */
private openNowPlaying = () => {
this.dispatchEvent(new CustomEvent('navigate', {
detail: { view: 'now-playing' },
bubbles: true,
composed: true,
}));
};
// ===================================================================
// SCROLL LOGIC
// ===================================================================
@@ -24,6 +24,7 @@ import type * as playlist from '@go/playlist/models.js';
import { EventsOn } from '@runtime/runtime';
import { Events } from '../../events';
import { queueStore } from '@store/queue-store';
import { creditStore } from '@store/credit-store';
import { PlayerController } from '@store/controllers/player-controller';
import { SearchController } from '@store/controllers/search-controller';
import { SelectionController } from '@utils/selection-controller';
@@ -61,7 +62,8 @@ import '@components/duplicate-tracks-dialog/duplicate-tracks-dialog.js';
import type { DuplicateTracksDialog } from '@components/duplicate-tracks-dialog/duplicate-tracks-dialog.js';
import { formatMilliseconds } from '@utils/time';
import {
artistLink,
creditLink,
creditText,
albumLink,
trackLink,
exploreLinkStyles,
@@ -106,6 +108,9 @@ export class PlaylistDetails
* rather than guessed. Without the hint the flow layout's 100 px
* default drives constant scroll-error correction, which reads as
* the list jumping under the pointer. */
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
@query('lit-virtualizer')
private virtualizer?: LitVirtualizer;
@@ -224,6 +229,14 @@ export class PlaylistDetails
override connectedCallback() {
super.connectedCallback();
// Credits arrive after the rows that asked for them, and a
// virtualizer repaints from its *own* properties — a host
// update alone leaves the rows exactly as they were.
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
this.virtualizer?.requestUpdate();
});
this.loadTracks();
this.tracksChangedCleanup = EventsOn(
@@ -248,6 +261,8 @@ export class PlaylistDetails
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
if (this.tracksChangedCleanup) {
this.tracksChangedCleanup();
@@ -1575,7 +1590,7 @@ export class PlaylistDetails
: nothing}
</div>
<span class="cell col-title" title="${track.Title || track.FilePath}">${trackLink(track.Title, track.Album, track.ReleaseGroupMBID, track.RecordingMBID, undefined, track.Artist) || track.FilePath}</span>
<span class="cell col-artist" title="${track.Artist}">${artistLink(track.Artist, track.ArtistMBID)}</span>
<span class="cell col-artist" title="${creditText(creditStore.credits(track.RecordingMBID), track.Artist)}">${creditLink(creditStore.credits(track.RecordingMBID), track.Artist, track.ArtistMBID)}</span>
<span class="cell col-album" title="${track.Album}">${albumLink(track.Album, track.ReleaseGroupMBID, undefined, track.Artist)}</span>
<span class="cell col-duration">${formatMilliseconds(track.Duration)}</span>`}
</div>
@@ -12,6 +12,7 @@ import '@awesome.me/webawesome/dist/components/popup/popup.js';
import type WaPopup from '@awesome.me/webawesome/dist/components/popup/popup.js';
import '@awesome.me/webawesome/dist/components/dropdown-item/dropdown-item.js';
import { QueueController } from '@store/controllers/queue-controller';
import { creditStore } from '@store/credit-store';
import {
describeQueueSource,
isQueueSourceNavigable,
@@ -55,7 +56,7 @@ import { tracksByFilePath } from '@utils/track-index.js';
import type { TrackDetails } from '@components/track-details/track-details.js';
import type { CoverArtUrls } from '@components/track-details/track-details.js';
import {
artistLink,
creditLink,
trackLink,
exploreLinkStyles,
} from '@utils/explore-link';
@@ -108,6 +109,9 @@ export class QueuePanel
@query('#playlist-submenu')
private playlistSubmenuPopup!: WaPopup;
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
@query('lit-virtualizer')
private virtualizer!: LitVirtualizer;
@@ -635,6 +639,14 @@ export class QueuePanel
override connectedCallback() {
super.connectedCallback();
// Credits arrive after the rows that asked for them, and a
// virtualizer repaints from its *own* properties — a host
// update alone leaves the rows exactly as they were.
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
this.virtualizer?.requestUpdate();
});
this.style.setProperty(
'--queue-width',
`${this.panelWidth}px`,
@@ -671,6 +683,8 @@ export class QueuePanel
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
document.removeEventListener(
'mousemove',
this.handleMouseMove,
@@ -1675,7 +1689,7 @@ export class QueuePanel
${trackLink(title, track.album, track.releaseGroupMbid, track.recordingMbid, undefined, track.artist)}
</span>
<span class="track-artist" title=${artist}>
${artistLink(track.artist, track.artistMbid) || 'Unknown Artist'}
${creditLink(creditStore.credits(track.recordingMbid), track.artist, track.artistMbid) || 'Unknown Artist'}
</span>
</div>
<button
@@ -67,6 +67,21 @@ export class SearchBar extends LitElement {
transition: border-color 0.15s ease;
}
/* The 200px floor is a desktop floor. On a phone the header is
the whole width there is, and a min-width in a flex row is a
*hard* one -- it does not shrink, so the header stayed 580px
wide inside a 360px viewport and the shell scrolled
sideways. Measured at 360px: 580 -> 360. */
@media (max-width: 599px) {
:host {
min-width: 0;
}
.search-container {
min-width: 0;
}
}
.search-container:focus-within {
border-color: var(--yj-accent, #ffd43b);
}
+14 -2
View File
@@ -1,5 +1,5 @@
import { LitElement, html, css } from 'lit';
import { customElement, state } from 'lit/decorators.js';
import { customElement, state, property } from 'lit/decorators.js';
import '@awesome.me/webawesome/dist/components/icon/icon.js';
import { designTokens } from '../../styles/tokens.css';
@@ -166,6 +166,17 @@ export class AppSidebar extends LitElement {
@state()
private collapsed = false;
/**
* Keep the labels regardless of the viewport, for a host that has
* made room for them -- `bottom-nav`'s drawer, which is the whole
* screen wide on the phone where this would otherwise auto-collapse
* to icons. The auto-collapse is a *width* response to a narrow
* shell, and inside a drawer the shell is not what the sidebar is
* sharing space with.
*/
@property({ type: Boolean, reflect: true })
expanded = false;
/** The width the user chose, restored when the window grows back. */
private userWidth = DEFAULT_WIDTH;
@@ -344,7 +355,8 @@ export class AppSidebar extends LitElement {
*/
private applyViewportWidth() {
const narrow =
this.narrowViewport?.matches ?? false;
!this.expanded &&
(this.narrowViewport?.matches ?? false);
const width = narrow
? MIN_WIDTH
: this.userWidth;
@@ -15,6 +15,7 @@ import {
import { EventsOn } from '@runtime/runtime';
import { Events } from '../../events';
import { queueStore } from '@store/queue-store';
import { creditStore } from '@store/credit-store';
import { PlayerController } from '@store/controllers/player-controller';
import { SearchController } from '@store/controllers/search-controller';
import { SelectionController } from '@utils/selection-controller';
@@ -51,7 +52,8 @@ import type { CoverArtUrls } from '@components/track-details/track-details.js';
import { libraryStore } from '@store/library-store';
import { formatMilliseconds } from '@utils/time';
import {
artistLink,
creditLink,
creditText,
albumLink,
trackLink,
exploreLinkStyles,
@@ -136,6 +138,9 @@ export class SmartPlaylistDetails
* rather than guessed: without the hint the flow layout's 100 px
* default drives constant scroll-error correction, which reads as
* the list jumping under the pointer. */
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
@query('lit-virtualizer')
private virtualizer?: LitVirtualizer;
@@ -609,6 +614,14 @@ export class SmartPlaylistDetails
override connectedCallback() {
super.connectedCallback();
// Credits arrive after the rows that asked for them, and a
// virtualizer repaints from its *own* properties — a host
// update alone leaves the rows exactly as they were.
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
this.virtualizer?.requestUpdate();
});
if (this.autoEdit) {
// Skip evaluation for new playlists — go straight to editor.
this.autoEdit = false;
@@ -649,6 +662,8 @@ export class SmartPlaylistDetails
override disconnectedCallback() {
super.disconnectedCallback();
this.creditsUnsub?.();
this.creditsUnsub = undefined;
if (this.playlistDeletedCleanup) {
this.playlistDeletedCleanup();
@@ -1423,7 +1438,7 @@ export class SmartPlaylistDetails
: nothing}
</div>
<span class="cell col-title" title="${track.Title || track.FilePath}">${trackLink(track.Title, track.Album, track.ReleaseGroupMBID, track.RecordingMBID, undefined, track.Artist) || track.FilePath}</span>
<span class="cell col-artist" title="${track.Artist}">${artistLink(track.Artist, track.ArtistMBID)}</span>
<span class="cell col-artist" title="${creditText(creditStore.credits(track.RecordingMBID), track.Artist)}">${creditLink(creditStore.credits(track.RecordingMBID), track.Artist, track.ArtistMBID)}</span>
<span class="cell col-album" title="${track.Album}">${albumLink(track.Album, track.ReleaseGroupMBID, undefined, track.Artist)}</span>
<span class="cell col-duration">${formatMilliseconds(track.Duration)}</span>`}
</div>
@@ -9,7 +9,8 @@ import {
} from '@go/explore/service.js';
import '../library-status-indicator/library-status-indicator.js';
import type { LibraryStatus } from '../library-status-indicator/library-status-indicator.js';
import { artistLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditLink, exploreLinkStyles } from '../../utils/explore-link';
import { creditStore } from '@store/credit-store';
import { libraryStatusFor } from '../../utils/library-status';
import { downloadStore } from '../../store/download-store';
@@ -61,14 +62,23 @@ export class TopResultsRow extends LitElement {
* the property and never updates this element. One subscription for
* the row, not one per card.
*/
/** Unsubscribes the credit-arrival repaint. */
private creditsUnsub?: () => void;
override connectedCallback(): void {
super.connectedCallback();
this.creditsUnsub = creditStore.subscribe(() => {
this.requestUpdate();
});
this.unsubRequests = downloadStore.subscribe(() =>
this.requestUpdate(),
);
}
override disconnectedCallback(): void {
this.creditsUnsub?.();
this.creditsUnsub = undefined;
this.unsubRequests?.();
this.unsubRequests = undefined;
super.disconnectedCallback();
@@ -327,7 +337,7 @@ export class TopResultsRow extends LitElement {
${artistPart || metaPart
? html`<span class="card-subtitle"
>${artistPart
? artistLink(artistPart, r.artistMbid ?? '')
? creditLink(creditStore.credits(r.mbid), artistPart, r.artistMbid ?? '')
: nothing}${artistPart && metaPart
? ' · '
: ''}${metaPart}</span
+52 -2
View File
@@ -9,6 +9,8 @@ import {
import { formatMilliseconds } from '@utils/time';
import { html, nothing } from 'lit';
import { highlightText } from './search-ranking';
/** Compares two strings using locale-aware ordering. */
const compareStr = (
a: string,
@@ -36,8 +38,17 @@ export interface ColumnDef {
defaultWidth: string;
/** Text alignment. Defaults to left. */
align?: 'left' | 'right';
/** Optional custom render function returning an HTML template. */
renderCell?: (track: library.Track) => unknown;
/**
* Optional custom render function returning an HTML template.
*
* `term` is the active search term, for a cell that wants to
* highlight its own text: the default path applies `highlightText`
* to `accessor`'s value, and a cell that renders itself has to do
* that itself or silently lose the highlight. Only `titleArtist`
* needs it, which is why it is optional rather than a second
* required parameter on all of them.
*/
renderCell?: (track: library.Track, term?: string) => unknown;
/**
* Comparison function for sorting two tracks by this column.
* Returns negative if a < b, positive if a > b, zero if equal.
@@ -85,6 +96,27 @@ export const COLUMN_DEFS: Record<string, ColumnDef> = {
/>`;
},
},
titleArtist: {
id: 'titleArtist',
// Named for what it sorts by, since that is the only place the
// label is user-visible: the phone has no column headers, and
// the page header's sort list is built from the *configured*
// columns rather than the drawn ones.
label: 'Track Name',
accessor: (t) => t.TrackName,
defaultWidth: '1fr',
comparator: (a, b) => compareStr(a.TrackName, b.TrackName),
renderCell: (t, term) => html`
<div class="stacked">
<span class="stacked-title"
>${term ? highlightText(t.TrackName, term) : t.TrackName}</span
>
<span class="stacked-sub"
>${term ? highlightText(t.ArtistName, term) : t.ArtistName}</span
>
</div>
`,
},
trackName: {
id: 'trackName',
label: 'Track Name',
@@ -249,6 +281,24 @@ export const CORE_SEARCH_COLUMN_IDS: string[] = [
'album',
];
/**
* The one column a phone shows, and it is two lines.
*
* At 424 CSS px -- the width of the phone this was measured on -- four
* columns fit the row exactly and none of them fits its *content*:
* `--grid-cols` came out `24px 102px 101px 101px 80px`, so "Duration"
* did not fit its own header and a title had ~20 characters. The
* columns were never too wide; there were too many of them.
*
* So the phone gets the title with the artist under it, which is the
* shape every phone music list has, and the full row width to put them
* in. It is a *column definition* rather than a second row template on
* purpose: the row, the delegated events, the selection semantics, the
* playing marker and the virtualizer all keep working, because from
* their side nothing has changed except how many columns there are.
*/
export const PHONE_COLUMN_IDS: string[] = ['titleArtist', 'trackLength'];
/**
* Default column IDs. Album is in them (H-15): without it, the three
* `Tideline / Aurora Fields / 00:06` rows in this app's own fixture
+160 -18
View File
@@ -25,11 +25,13 @@ import type { SortOption } from '@components/page-header/page-header';
import { TrackListController } from '@store/controllers/tracklist-controller';
import { FavoritesController } from '@store/controllers/favorites-controller';
import { queueStore } from '@store/queue-store';
import { creditStore } from '@store/credit-store';
import type { QueueSource } from '@store/queue-store';
import { LibraryController } from '@store/controllers/library-controller';
import {
COLUMN_DEFS,
DEFAULT_COLUMN_IDS,
PHONE_COLUMN_IDS,
} from './columns';
import type { ColumnDef } from './columns';
import { classMap } from 'lit/directives/class-map.js';
@@ -39,6 +41,7 @@ import {
} from './search-ranking';
import {
artistLink,
creditLink,
albumLink,
trackLink,
exploreLinkStyles,
@@ -90,6 +93,18 @@ const ROW_PADDING_X = 8;
const ROW_CHROME_WIDTH =
FAV_COL_WIDTH + ROW_PADDING_X * 2;
/**
* Row heights, in the same relationship as the widths above: the number
* is read by the CSS *and* by the virtualizer's layout, so they cannot
* disagree. A phone row is two lines (title over artist).
*/
const ROW_HEIGHT = 33;
const PHONE_ROW_HEIGHT = 52;
/** The shell's phone breakpoint, as `index.css` and every component
* stylesheet spells it. */
const PHONE_QUERY = '(max-width: 599px)';
// Inline SVG paths for favorite icons — eliminates wa-icon shadow DOM
// overhead (30-50 shadow roots during scroll). Font Awesome 6 paths.
const FAV_ICONS = {
@@ -175,19 +190,34 @@ export class TrackList
* Resolved column definitions for the currently configured
* column IDs. Falls back to defaults for any unknown ID.
*/
private get activeColumns(): ColumnDef[] {
/**
* The columns the user has chosen what a desktop draws, and what
* *anything* may be sorted by.
*
* This is deliberately separate from `activeColumns`: "which columns
* are drawn" and "what can I sort by" are different questions, and
* the phone is exactly where they diverge. Building the sort list
* from the drawn columns would silently take sort-by-artist and
* sort-by-album away from the phone, which has no other route to
* them since it has no column headers either.
*/
private get configuredColumns(): ColumnDef[] {
const ids = this.trackListCtrl.columnIds;
const chosen = !ids || ids.length === 0 ? DEFAULT_COLUMN_IDS : ids;
if (!ids || ids.length === 0) {
return DEFAULT_COLUMN_IDS
.map((id) => COLUMN_DEFS[id])
.filter(
(d): d is ColumnDef =>
d !== undefined,
);
}
return chosen
.map((id) => COLUMN_DEFS[id])
.filter(
(d): d is ColumnDef =>
d !== undefined,
);
}
return ids
/** The columns actually drawn: two stacked lines on a phone. */
private get activeColumns(): ColumnDef[] {
if (!this.phone) return this.configuredColumns;
return PHONE_COLUMN_IDS
.map((id) => COLUMN_DEFS[id])
.filter(
(d): d is ColumnDef =>
@@ -374,9 +404,42 @@ export class TrackList
// doesn't need to measure items. Without this hint, the default 100px
// estimate causes constant scroll error correction (scrollTo() calls)
// that produce visible jumping/skipping during scroll.
/**
* The virtualizer's item size and the CSS row height are the same
* number in two places, and they must agree: the layout positions
* rows from this figure, so a row that is really taller overlaps its
* neighbour and a shorter one leaves a gap. Both come from here.
*/
private flowLayout = flow({
_itemSize: { width: 100, height: 33 },
_itemSize: { width: 100, height: ROW_HEIGHT },
} as Parameters<typeof flow>[0]);
private phoneFlowLayout = flow({
_itemSize: { width: 100, height: PHONE_ROW_HEIGHT },
} as Parameters<typeof flow>[0]);
private get rowLayout(): Parameters<typeof flow>[0] {
return this.phone ? this.phoneFlowLayout : this.flowLayout;
}
/**
* Phone width, from the shell's own breakpoint.
*
* A media query *inside* a shadow root is answered by the viewport,
* which is what lets every other component state what it drops at
* phone width in its own stylesheet. This list cannot: its grid is
* computed in JS from the host width, so the same threshold has to
* be readable from JS as well. One breakpoint, two expressions of
* it, and the reason is written here rather than inferred.
*/
@state()
private phone = matchMedia(PHONE_QUERY).matches;
private phoneQuery = matchMedia(PHONE_QUERY);
private onPhoneChange = (e: MediaQueryListEvent): void => {
this.phone = e.matches;
};
private hasRestoredScroll = false;
private scrollSaveRAFId: number | null = null;
@@ -543,6 +606,20 @@ export class TrackList
}
private initColumnWidths() {
// A phone's widths are never the saved ones. `loadColumnWidths`
// is keyed by column *id* and fills a gap with
// `MIN_COLUMN_WIDTH`, so the phone's stacked column -- which
// nothing has ever saved a width for, there being no handles to
// drag -- came out at the minimum while the duration column
// inherited a width saved for a four-column desktop row. Found
// on the device: `24px 148px 236px`, the duration column with
// 55% of a phone's row.
if (this.phone) {
this.computeDefaultWidths();
return;
}
const saved = this.loadColumnWidths();
const cols = this.activeColumns;
@@ -673,6 +750,13 @@ export class TrackList
}
private saveColumnWidths() {
// And a phone's widths are never *saved*: they are computed from
// a column set the user did not choose, and writing them would
// overwrite the width they dragged for the same column on a
// desktop. Nothing on a phone can resize a column anyway, so
// this is only reachable by a window crossing the breakpoint.
if (this.phone) return;
try {
const cols = this.activeColumns;
@@ -1045,6 +1129,35 @@ export class TrackList
contain: strict;
}
/* A phone row is two lines, and this height must equal
PHONE_ROW_HEIGHT: the virtualizer positions rows from that number,
so a taller row overlaps its neighbour and a shorter one gaps. */
@media (max-width: 599px) {
.track-row {
height: 52px;
}
}
.stacked {
display: flex;
flex-direction: column;
justify-content: center;
gap: 2px;
min-width: 0;
}
.stacked-title,
.stacked-sub {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.stacked-sub {
font-size: var(--yj-text-xs);
color: var(--yj-text-secondary, #b3b3b3);
}
.track-row > * {
min-width: 0;
}
@@ -1171,9 +1284,17 @@ export class TrackList
);
this.resizeObserver.observe(this);
// Connection, not the view lifecycle: this only sets state, so
// it is harmless (and wanted) while the list is off screen -- a
// rotation on another view must not leave this one laid out for
// the wrong width when the user comes back to it.
this.phoneQuery.addEventListener('change', this.onPhoneChange);
}
override disconnectedCallback() {
this.phoneQuery.removeEventListener('change', this.onPhoneChange);
// Remove delegated event handlers from virtualizer.
const virt = this.virtualizer;
if (virt) {
@@ -1220,6 +1341,17 @@ export class TrackList
'shortcut:tracklist-delete',
this.handleShortcutDelete,
);
// Credits arrive after the rows that asked for them. The
// virtualizer produces its rows from its *own* properties, so a
// host re-render alone repaints nothing — the same reason a
// selection change pushes requestUpdate() into it.
this.whileActive(
creditStore.subscribe(() => {
this.requestUpdate();
this.virtualizer?.requestUpdate();
}),
);
}
/**
@@ -2017,7 +2149,7 @@ export class TrackList
</svg>
</div>
${cols.map((col) => {
const customCell = col.renderCell?.(track);
const customCell = col.renderCell?.(track, term);
if (customCell !== undefined && customCell !== nothing) {
return html`<div role="gridcell" class="cell">${customCell}</div>`;
}
@@ -2031,7 +2163,15 @@ export class TrackList
if (col.id === 'trackName') {
display = trackLink(track.TrackName, track.Album, track.ReleaseGroupMBID, track.RecordingMBID, display as any, track.ArtistName);
} else if (col.id === 'artistName') {
display = artistLink(track.ArtistName, track.ArtistMBID, display as any);
// A search term highlights the *flat* credit string,
// and mapping those spans onto decomposed parts is a
// different problem from rendering the credit. While
// filtering, the single link is the honest answer.
creditStore.request(track.RecordingMBID);
const parts = term ? undefined : creditStore.get(track.RecordingMBID);
display = parts && parts.length > 1
? creditLink(parts, track.ArtistName, track.ArtistMBID)
: artistLink(track.ArtistName, track.ArtistMBID, display as any);
} else if (col.id === 'album') {
display = albumLink(track.Album, track.ReleaseGroupMBID, display as any, track.ArtistName);
}
@@ -2065,7 +2205,7 @@ export class TrackList
private renderPageHeader() {
const options: SortOption[] = [
{ id: '', label: 'Default' },
...this.activeColumns
...this.configuredColumns
.filter((c) => c.comparator)
.map((c) => ({ id: c.id, label: c.label })),
];
@@ -2115,7 +2255,7 @@ export class TrackList
aria-busy=${this.loadingTracks}
@keydown=${this.onListKeydown}
>
<div class="header-row" role="row">
${this.phone ? nothing : html`<div class="header-row" role="row">
<div role="columnheader" aria-label="Favourite"></div>
${cols.map(
(col) => html`
@@ -2149,7 +2289,7 @@ export class TrackList
</div>
`,
)}
</div>
</div>`}
${visibleTracks.length === 0
? html`<p class="no-results">
No tracks match your search.
@@ -2160,12 +2300,14 @@ export class TrackList
.items=${visibleTracks}
.renderItem=${this.renderTrackRow}
.keyFunction=${(track: library.Track) => track.FilePath}
.layout=${this.flowLayout}
.layout=${this.rowLayout}
></lit-virtualizer>
`}
<!-- Resizing is a pointer gesture with no touch equivalent, and
the phone's two columns are not the user's to arrange. -->
<div class="resize-overlay">
${this.colBoundaryPositions.map(
${(this.phone ? [] : this.colBoundaryPositions).map(
(pos, i) => html`
<div
class="col-resize-handle ${this.resizingColumn === i ? 'active' : ''}"
+1
View File
@@ -22,6 +22,7 @@ solid/arrow-rotate-right
solid/arrows-rotate
solid/arrow-up-short-wide
solid/backward-step
solid/bars
regular/bookmark
solid/bookmark
solid/box-open
+226
View File
@@ -0,0 +1,226 @@
/**
* Multi-artist credits, keyed by recording MBID.
*
* A credit is ordered parts and the credit *string* is derived from
* them. This store holds the parts for entities that have more than
* one credited artist; everything else renders the single link it
* always did.
*
* Three things about it are load-bearing.
*
* **Absence is an answer, and it is cached as one.** The backend
* returns nothing for a single-artist credit, which is the common case
* by a wide margin measured on a real library, 13% of tracks are
* multi-artist. Caching only the hits would re-request the other 87%
* on every render, forever, which is the same shape as the bug that
* made `explore-album-details` ask the backend on hover. A miss is
* stored as an empty array: *asked*, not *answered*.
*
* **The lookup is batched, and coalesced across callers.** Every row
* of every tracklist asks this question, and one IPC round trip per row
* is how a 5,000-row list becomes unusable. A virtualized list cannot
* hand over "the whole list" either 50,000 rows is 100 queries for
* the ~30 on screen. So `request()` is per-row and cheap: it collects
* into a pending set and flushes once on the next frame, which turns a
* screenful of rows into exactly one call. `ensure()` remains for a
* caller that genuinely has a bounded list in hand.
*
* **It is bounded.** A cache that grows with use is a leak with a
* schedule; a browsing afternoon touches far more credits than a
* screenful. The cap is entries rather than bytes because a credit is
* a handful of short strings, unlike the art caches next door.
*/
import { GetCredits } from '@go/explore/service.js';
import type { CreditPart } from '../utils/explore-link';
import { LRUMap } from '../utils/lru-map';
import { compact } from '../utils/binding';
import { registerCacheProbe } from '../utils/cache-stats';
/**
* Entries retained. A credit is ~4 short strings, so this is well
* under a megabyte sized to comfortably exceed any single list the
* app renders, because a cap below the visible count evicts rows that
* are still on screen and the re-render fetches them straight back.
*/
export const CREDIT_CACHE_LIMIT = 20_000;
/** An empty parts array is the negative marker: asked, no decomposition. */
type CachedParts = readonly CreditPart[];
class CreditStore {
private cache = new LRUMap<string, CachedParts>(CREDIT_CACHE_LIMIT);
/** MBIDs with a request in flight, so a re-render does not refetch. */
private inFlight = new Set<string>();
private listeners = new Set<() => void>();
/** Collected by request(), flushed as one batch on the next frame. */
private pending = new Set<string>();
private flushHandle: number | null = null;
constructor() {
registerCacheProbe('credits', () => ({
entries: this.cache.size,
chars: this.retainedChars(),
limit: CREDIT_CACHE_LIMIT,
}));
}
/**
* The strings actually retained, counted rather than estimated
* a bound that is only checkable against a guess is not checkable.
*/
private retainedChars(): number {
let total = 0;
for (const parts of this.cache.values()) {
for (const part of parts) {
total +=
part.creditedName.length +
part.joinPhrase.length +
part.artistMbid.length;
}
}
return total;
}
/**
* Subscribe to "some credits arrived".
*
* Deliberately not per-MBID: a list fetches its rows in one call and
* re-renders once, so a fine-grained signal would buy nothing and
* cost a listener per row.
*/
subscribe(fn: () => void): () => void {
this.listeners.add(fn);
return () => this.listeners.delete(fn);
}
/**
* The parts for one entity, or undefined when it has not been asked
* about yet.
*
* An entity with a single-artist credit returns an empty array, and
* `creditLink` treats fewer than two parts as the fallback so a
* caller does not have to distinguish "not asked" from "one artist"
* to render correctly, only to decide whether to ask.
*/
get(mbid: string | undefined): readonly CreditPart[] | undefined {
if (!mbid) return undefined;
return this.cache.get(mbid);
}
/**
* Ask about one entity, joining whatever batch is forming.
*
* Safe to call from a render: it is a set insert and a scheduled
* flush, and an entity already cached or in flight is dropped. The
* loop it looks like it might cause does not happen after a flush
* every requested MBID is cached, so the re-render's requests are
* all dropped and nothing notifies again.
*/
request(mbid: string | undefined): void {
if (!mbid) return;
if (this.cache.has(mbid)) return;
if (this.inFlight.has(mbid)) return;
if (this.pending.has(mbid)) return;
this.pending.add(mbid);
if (this.flushHandle !== null) return;
// A frame, not a microtask: the point is to collect every row a
// virtualizer renders in this pass, and those happen across the
// whole update, not within one microtask checkpoint.
this.flushHandle = requestAnimationFrame(() => {
this.flushHandle = null;
const batch = [...this.pending];
this.pending.clear();
void this.ensure(batch);
});
}
/**
* Ask and read in one call, for use inside a template.
*
* A getter with a side effect, deliberately: the alternative is
* every call site writing `request(x)` beside `get(x)` and one of
* them eventually forgetting, which renders a permanently
* single-artist credit that looks exactly like an entity with one
* artist. Making the request the same act as the read is what
* stops the two drifting apart.
*/
credits(mbid: string | undefined): readonly CreditPart[] | undefined {
this.request(mbid);
return this.get(mbid);
}
/**
* Fetch the credits for a list, skipping anything already known or
* already being fetched.
*
* `has` rather than `get` for the membership test: probing must not
* mark an entry recently-used, or scrolling past a row would keep
* it alive ahead of one actually being rendered.
*/
async ensure(mbids: readonly (string | undefined)[]): Promise<void> {
const wanted = new Set<string>();
for (const mbid of mbids) {
if (!mbid) continue;
if (this.cache.has(mbid)) continue;
if (this.inFlight.has(mbid)) continue;
wanted.add(mbid);
}
if (wanted.size === 0) return;
const batch = [...wanted];
for (const mbid of batch) this.inFlight.add(mbid);
try {
const found = compact(await GetCredits(batch));
for (const mbid of batch) {
// Every MBID asked for gets an entry, present or not:
// the absent ones are the answer "one artist", and not
// recording that is what would re-ask forever.
this.cache.set(mbid, found[mbid] ?? []);
}
this.notify();
} catch (err) {
// A credit is an enrichment: without it every name renders
// as the single link it did before, which is a worse answer
// rather than a broken one. Nothing user-facing is worth
// interrupting for, so this stays in the console.
console.error('Failed to load artist credits', err);
} finally {
for (const mbid of batch) this.inFlight.delete(mbid);
}
}
/** Drop everything. The tags on disk changed, so credits may have. */
invalidate(): void {
this.cache = new LRUMap<string, CachedParts>(CREDIT_CACHE_LIMIT);
this.notify();
}
private notify(): void {
for (const fn of this.listeners) fn();
}
}
export const creditStore = new CreditStore();
+78
View File
@@ -294,3 +294,81 @@ async function openAlbum(
navigate(target, detail);
}
/**
* One credited artist within a multi-artist credit.
*
* Mirrors `artist_credit_part` / `file_artists`: the name **as
* credited** (which is not the artist's own name MusicBrainz credits
* "Snoop Dogg" on a track by the artist called "Snoop Doggy Dogg"), the
* MBID to navigate to, and the literal connector that follows this
* part.
*/
export interface CreditPart {
/** The name as credited. Display uses this. */
creditedName: string;
/** The artist's MusicBrainz ID. Navigation uses this. */
artistMbid: string;
/** The connector following this part: " feat. ", " & ", ", ", "". */
joinPhrase: string;
}
/**
* Render a credit as links, one per credited artist, with the join
* phrases as plain text between them.
*
* Join phrases are **assembly instructions, not disassembly
* instructions**. This concatenates parts; it never searches for a
* name inside a credit string. That distinction is the whole point:
* the stored credit text may have come from a file's tags while the
* parts come from the catalog, and measured on a real library those
* disagree for about one in three multi-artist credits ("Skrillex
* feat. Swae Lee" tagged against "Skrillex & Swae Lee" upstream). A
* search would miss, or match the wrong span. Building from parts,
* the link boundaries are known by construction.
*
* Falls back to `artistLink(fallbackName, fallbackMbid)` today's
* behaviour exactly when there are no parts. That is the common
* case and not a degraded one: a single-artist credit *is* one link,
* and a file with no recording MBID or no catalog row has nothing to
* decompose. Do not try to split the fallback string; there is
* genuinely no information in it to split on.
*
* @param parts - The credit's parts in position order, if known.
* @param fallbackName - The credit as a single string.
* @param fallbackMbid - The primary artist's MBID.
*/
export function creditLink(
parts: readonly CreditPart[] | undefined,
fallbackName: string,
fallbackMbid: string,
): TemplateResult | string {
// One part is one link, so it is the fallback rather than a special
// case — and a zero-part credit reaching here would otherwise
// render as nothing at all, which is worse than the single-artist
// answer it replaced.
if (!parts || parts.length < 2) {
return artistLink(fallbackName, fallbackMbid);
}
return html`${parts.map(
(part) =>
html`${artistLink(part.creditedName, part.artistMbid)}${part.joinPhrase}`,
)}`;
}
/**
* The plain-text form of a credit, for `title=` attributes and any
* other place that needs a string rather than a template.
*
* Rendered from the same parts by the same concatenation, so the
* tooltip cannot disagree with the links beneath it.
*/
export function creditText(
parts: readonly CreditPart[] | undefined,
fallbackName: string,
): string {
if (!parts || parts.length < 2) return fallbackName;
return parts.map((p) => p.creditedName + p.joinPhrase).join('');
}
+201
View File
@@ -0,0 +1,201 @@
/**
* Long-press as the touch equivalent of a right-click (plan 016 B2,
* phase 3).
*
* Every context menu in the app opens from a `contextmenu` event
* `track-list` and `queue-panel` delegate one on their virtualizer,
* the card grids and both playlist detail views bind one per row, and
* `explore-artist-details` binds three. A phone has no right-click, so
* a phone reached none of them.
*
* **This is one document listener, not six components' worth of touch
* handling.** A press that stays still for `LONG_PRESS_MS` dispatches a
* synthetic `contextmenu` at the touch point on the element the touch
* actually landed on, and every existing handler delegated or
* per-row, in any shadow root runs unchanged. Six implementations of
* a gesture is exactly the fault `ContextMenuController` exists to
* prevent, and a seam that needs no component to opt in cannot be
* forgotten by the next component.
*
* Three things about it are load-bearing.
*
* **The target comes from `composedPath()[0]`, not from
* `elementFromPoint`**, which stops at the outermost shadow host: every
* menu in this app is bound inside one, so a synthetic event dispatched
* on the host reaches a delegated listener and no per-row one.
*
* **A browser that already does this must win.** Chromium fires a
* `contextmenu` on long-press itself; WebKitGTK and the Android WebView
* vary. So one arriving during the press cancels ours, and one arriving
* just after ours is swallowed at document capture where nothing else
* has seen it yet. The two are told apart by **identity** (a `WeakSet`
* of the events this module made) rather than by `isTrusted`, so the
* suppressor cannot eat the event it exists to deliver, the rule holds
* for anything else in the app that synthesises one, and a test can
* stand in for a browser that fires its own.
*
* **The click that ends the gesture is swallowed.** A row's click
* selects, and a card's plays; without this, opening a menu also
* activates the thing under it. It is keyed on the gesture (cleared by
* the next `pointerdown`) rather than on a time window, so a quick tap
* on the menu that just opened is not eaten too.
*/
/** How long a press must hold still to mean "menu". */
export const LONG_PRESS_MS = 500;
/**
* How far a press may drift and still count. Below a finger's own
* jitter is a gesture nobody can perform; above ~12px it starts
* stealing the first frames of a scroll.
*/
export const MOVE_TOLERANCE_PX = 10;
/** The active installation, so a second call is a no-op rather than a
* second listener set. */
let uninstall: (() => void) | null = null;
/** The events this module dispatched. Identity, not `isTrusted`: see
* the note above. */
const ours = new WeakSet<Event>();
/**
* Install the gesture. Idempotent; returns the uninstaller (which the
* tests use the app installs once and never removes it).
*/
export function installLongPressContextMenu(): () => void {
if (uninstall) return uninstall;
let timer: ReturnType<typeof setTimeout> | null = null;
let originX = 0;
let originY = 0;
let target: EventTarget | null = null;
/** A trusted `contextmenu` arrived for this press: the browser has
* it covered. */
let nativeSeen = false;
/** We opened a menu, and the click ending that gesture is not a
* click on anything. */
let swallowClick = false;
/** We dispatched one, so a trusted one arriving now is a duplicate. */
let justFired = false;
const cancel = (): void => {
if (timer !== null) clearTimeout(timer);
timer = null;
target = null;
};
const fire = (): void => {
timer = null;
const el = target;
target = null;
if (nativeSeen || !el) return;
justFired = true;
swallowClick = true;
const menu = new MouseEvent('contextmenu', {
bubbles: true,
cancelable: true,
// Or it stops at the shadow root the row lives in, and the
// delegated listeners never see it.
composed: true,
clientX: originX,
clientY: originY,
button: 2,
});
ours.add(menu);
el.dispatchEvent(menu);
};
const onPointerDown = (e: PointerEvent): void => {
// A new gesture: whatever the last one left behind is stale.
swallowClick = false;
justFired = false;
nativeSeen = false;
cancel();
if (e.pointerType !== 'touch' || !e.isPrimary) return;
originX = e.clientX;
originY = e.clientY;
target = e.composedPath()[0] ?? e.target;
timer = setTimeout(fire, LONG_PRESS_MS);
};
const onPointerMove = (e: PointerEvent): void => {
if (timer === null) return;
const drifted =
Math.abs(e.clientX - originX) > MOVE_TOLERANCE_PX ||
Math.abs(e.clientY - originY) > MOVE_TOLERANCE_PX;
if (drifted) cancel();
};
const onContextMenu = (e: Event): void => {
// Ours. Everything below is about somebody else's.
if (ours.has(e)) return;
if (timer !== null) {
// The browser got there first, so stand down rather than
// opening the same menu twice.
nativeSeen = true;
cancel();
return;
}
if (justFired) {
justFired = false;
e.preventDefault();
e.stopImmediatePropagation();
}
};
const onClick = (e: Event): void => {
if (!swallowClick) return;
swallowClick = false;
e.preventDefault();
e.stopImmediatePropagation();
};
// Capture throughout: a component handler that stops propagation
// (every context-menu handler in the app does) must not be able to
// hide the gesture from this, and the suppressors have to run
// before anything that would act on the event.
const opts = { capture: true } as const;
document.addEventListener('pointerdown', onPointerDown, opts);
document.addEventListener('pointermove', onPointerMove, opts);
document.addEventListener('pointerup', cancel, opts);
document.addEventListener('pointercancel', cancel, opts);
document.addEventListener('contextmenu', onContextMenu, opts);
document.addEventListener('click', onClick, opts);
// A scroll started by something other than the finger (momentum, a
// programmatic reveal) still means the press was not a press.
document.addEventListener('scroll', cancel, { capture: true, passive: true });
uninstall = () => {
cancel();
document.removeEventListener('pointerdown', onPointerDown, opts);
document.removeEventListener('pointermove', onPointerMove, opts);
document.removeEventListener('pointerup', cancel, opts);
document.removeEventListener('pointercancel', cancel, opts);
document.removeEventListener('contextmenu', onContextMenu, opts);
document.removeEventListener('click', onClick, opts);
document.removeEventListener('scroll', cancel, opts);
uninstall = null;
};
return uninstall;
}
+165
View File
@@ -0,0 +1,165 @@
/**
* The phone's primary navigation (plan 016 B2).
*
* Three of these are about the thing that makes a second nav dangerous:
* it has to agree with the first one. `bottom-nav` emits the same
* bubbling, composed `navigate` event `app-sidebar` does and listens
* for that event globally, so a navigation from anywhere a card, a
* detail view, the drawer's own sidebar moves its highlight too. A
* tab bar that only tracks its own clicks looks right until the moment
* the user arrives somewhere by another route.
*/
import { describe, expect, it, beforeEach } from 'vitest';
import '@components/bottom-nav/bottom-nav';
import type { BottomNav } from '@components/bottom-nav/bottom-nav';
import { fixture, shadow, shadowAll, update } from '@test/support/render';
import { resetHarness } from '@test/support/harness';
type Nav = BottomNav;
const tabs = (el: HTMLElement) =>
shadowAll<HTMLButtonElement>(el, 'nav button');
/** Resolve on one occurrence of an event, or reject loudly on time. */
const once = (el: Element, name: string, timeoutMs = 2000) =>
new Promise<void>((resolve, reject) => {
const timer = setTimeout(
() => reject(new Error(`${name} never fired`)),
timeoutMs,
);
el.addEventListener(name, () => {
clearTimeout(timer);
resolve();
}, { once: true });
});
describe('bottom-nav', () => {
beforeEach(() => {
resetHarness();
});
it('offers the four phone destinations and a way to the rest', async () => {
const el = await fixture<Nav>('bottom-nav');
expect(tabs(el).map((b) => b.dataset.testid)).toEqual([
'tab-home',
'tab-albums',
'tab-tracks',
'tab-playlists',
'tab-more',
]);
});
it('emits a navigate event that escapes its shadow root', async () => {
const el = await fixture<Nav>('bottom-nav');
const seen: string[] = [];
document.addEventListener('navigate', (e) => {
seen.push((e as CustomEvent<{ view: string }>).detail.view);
});
shadow<HTMLButtonElement>(el, '[data-testid="tab-albums"]')?.click();
// Composed and bubbling, or index.ts's document-level listener --
// the only thing that actually changes the view -- never hears it.
expect(seen).toEqual(['albums']);
});
it('follows a navigation it did not send', async () => {
const el = await fixture<Nav>('bottom-nav');
document.dispatchEvent(new CustomEvent('navigate', {
detail: { view: 'tracks' },
bubbles: true,
composed: true,
}));
await update(el, {});
const current = tabs(el)
.filter((b) => b.getAttribute('aria-current') === 'page')
.map((b) => b.dataset.testid);
expect(current).toEqual(['tab-tracks']);
});
it('marks exactly one tab current, and none for a view it has no tab for', async () => {
const el = await fixture<Nav>('bottom-nav');
document.dispatchEvent(new CustomEvent('navigate', {
detail: { view: 'settings' },
bubbles: true,
composed: true,
}));
await update(el, {});
// Settings lives in the drawer, so nothing in the bar is current.
// Leaving Home highlighted would be a tab bar lying about where
// the user is.
expect(
tabs(el).filter((b) => b.getAttribute('aria-current') === 'page'),
).toHaveLength(0);
});
it('closes the drawer when a navigation happens', async () => {
const el = await fixture<Nav>('bottom-nav');
const drawer = shadow<HTMLElement & { open: boolean }>(el, 'wa-drawer');
if (!drawer) throw new Error('no drawer');
// The drawer animates, so the assertion is its own event rather
// than the `open` property: setting `open = false` starts a hide
// that has not finished on the next microtask, and a test that
// reads the property in between sees the state it is leaving.
const shown = once(drawer, 'wa-after-show');
shadow<HTMLButtonElement>(el, '[data-testid="tab-more"]')?.click();
await shown;
const hidden = once(drawer, 'wa-after-hide');
document.dispatchEvent(new CustomEvent('navigate', {
detail: { view: 'settings' },
bubbles: true,
composed: true,
}));
await hidden;
expect(drawer.open).toBe(false);
});
it('gives every tab a name and a target big enough to hit', async () => {
const el = await fixture<Nav>('bottom-nav');
for (const button of tabs(el)) {
expect(button.textContent?.trim()).not.toBe('');
// 48px is the floor for a touch target; the bar is the one
// surface in this app that has no pointer to fall back on.
expect(button.getBoundingClientRect().height).toBeGreaterThanOrEqual(48);
}
});
it('holds no second sidebar until the drawer is asked for', async () => {
const el = await fixture<Nav>('bottom-nav');
// `app-sidebar` carries a data-testid per destination, so a spare
// copy standing by makes every `nav-*` testid ambiguous for the
// *whole app*: rendering it unconditionally failed 30 existing
// specs with "strict mode violation: resolved to 2 elements", on a
// desktop viewport where this element is not even visible.
expect(shadow(el, 'app-sidebar')).toBeNull();
});
it('keeps the drawer sidebar expanded, where there is room for labels', async () => {
const el = await fixture<Nav>('bottom-nav');
shadow<HTMLButtonElement>(el, '[data-testid="tab-more"]')?.click();
await update(el, {});
// Without this the sidebar's own auto-collapse (a response to a
// narrow *shell*) would render icons in a full-width drawer.
expect(shadow<HTMLElement>(el, 'app-sidebar')?.hasAttribute('expanded'))
.toBe(true);
});
});
+212
View File
@@ -0,0 +1,212 @@
/**
* Long-press as the touch route to a context menu (plan 016 B2 phase 3).
*
* These run in a real browser with real event dispatch, which is the
* only place the two things that make this hard are true: the synthetic
* event has to cross a shadow boundary to reach the listener a
* component actually bound, and the suppressors have to tell a trusted
* event from ours at document capture without eating the one they exist
* to deliver.
*
* The timings are real rather than faked, because the thing under test
* *is* a timing, and 600 ms twice is cheaper than a fake-timer harness
* that would also have to fake the pointer events.
*/
import { describe, expect, it, afterEach, beforeEach } from 'vitest';
import {
installLongPressContextMenu,
LONG_PRESS_MS,
MOVE_TOLERANCE_PX,
} from '@utils/long-press';
/** A press that has certainly resolved, either way. */
const HELD = LONG_PRESS_MS + 120;
/** A press that has certainly not. */
const BRIEF = Math.round(LONG_PRESS_MS / 4);
const wait = (ms: number) => new Promise((r) => setTimeout(r, ms));
let uninstall: (() => void) | null = null;
let host: HTMLElement;
let inner: HTMLElement;
/** A row inside a shadow root, which is where every menu in this app
* is bound an element in the light DOM would pass a weaker test. */
function mountRow(): { host: HTMLElement; inner: HTMLElement } {
const el = document.createElement('div');
const root = el.attachShadow({ mode: 'open' });
const row = document.createElement('div');
row.textContent = 'a track';
root.append(row);
document.body.append(el);
return { host: el, inner: row };
}
function press(
el: EventTarget,
type: string,
init: PointerEventInit = {},
): void {
el.dispatchEvent(
new PointerEvent(type, {
bubbles: true,
composed: true,
cancelable: true,
pointerType: 'touch',
isPrimary: true,
clientX: 40,
clientY: 60,
...init,
}),
);
}
/**
* Record every `contextmenu` that reaches the listener, *as the
* listener sees it*.
*
* `target` is retargeted for the scope reading it, so an assertion made
* after dispatch has finished reports the shadow host however the event
* was dispatched - which is the same answer a broken implementation
* gives. It has to be read from inside the handler, where the component
* reads it.
*/
function recordMenus(el: EventTarget): { event: MouseEvent; target: EventTarget | null }[] {
const seen: { event: MouseEvent; target: EventTarget | null }[] = [];
el.addEventListener('contextmenu', (e) => {
e.preventDefault();
// Every real handler does this; the gesture must work anyway.
e.stopPropagation();
seen.push({ event: e as MouseEvent, target: e.target });
});
return seen;
}
describe('long-press opens a context menu', () => {
beforeEach(() => {
uninstall = installLongPressContextMenu();
({ host, inner } = mountRow());
});
afterEach(() => {
uninstall?.();
uninstall = null;
host.remove();
});
it('dispatches one at the touch point, on the element touched', async () => {
const seen = recordMenus(inner);
press(inner, 'pointerdown');
await wait(HELD);
expect(seen).toHaveLength(1);
expect(seen[0]?.event.clientX).toBe(40);
expect(seen[0]?.event.clientY).toBe(60);
// Dispatched on the row itself, not on its shadow host - which is
// the difference between a per-row handler firing and only a
// delegated one firing.
expect(seen[0]?.target).toBe(inner);
});
it('is cancelled by a press that moves', async () => {
const seen = recordMenus(inner);
press(inner, 'pointerdown');
press(inner, 'pointermove', {
clientX: 40 + MOVE_TOLERANCE_PX + 5,
clientY: 60,
});
await wait(HELD);
expect(seen).toHaveLength(0);
});
it('tolerates the jitter a finger cannot help', async () => {
const seen = recordMenus(inner);
press(inner, 'pointerdown');
press(inner, 'pointermove', { clientX: 43, clientY: 62 });
await wait(HELD);
expect(seen).toHaveLength(1);
});
it('is cancelled by lifting early, and by a scroll', async () => {
const seen = recordMenus(inner);
press(inner, 'pointerdown');
await wait(BRIEF);
press(inner, 'pointerup');
await wait(HELD);
expect(seen).toHaveLength(0);
press(inner, 'pointerdown');
press(inner, 'pointercancel');
await wait(HELD);
expect(seen).toHaveLength(0);
});
it('ignores a mouse, which has a right button of its own', async () => {
const seen = recordMenus(inner);
press(inner, 'pointerdown', { pointerType: 'mouse' });
await wait(HELD);
expect(seen).toHaveLength(0);
});
it('swallows the click that ends the gesture, and only that one', async () => {
let clicks = 0;
inner.addEventListener('click', () => {
clicks += 1;
});
press(inner, 'pointerdown');
await wait(HELD);
press(inner, 'pointerup');
inner.click();
expect(clicks).toBe(0);
// The next tap is a tap: on a phone that is the user choosing an
// item in the menu that just opened, so eating it would make the
// gesture useless.
press(inner, 'pointerdown');
press(inner, 'pointerup');
inner.click();
expect(clicks).toBe(1);
});
it('stands down where the browser fires its own', async () => {
const seen = recordMenus(inner);
press(inner, 'pointerdown');
await wait(BRIEF);
// Chromium does this itself on touch; WebKit and the Android
// WebView vary, which is the whole reason both halves exist. A
// test cannot dispatch a *trusted* event, which is why the module
// tells its own apart by identity rather than by `isTrusted`.
inner.dispatchEvent(
new MouseEvent('contextmenu', {
bubbles: true,
composed: true,
cancelable: true,
}),
);
await wait(HELD);
// One menu: the browser's. Not two.
expect(seen).toHaveLength(1);
});
});
@@ -0,0 +1,126 @@
/**
* The full-screen now-playing view (plan 016 B2, phase 2).
*
* What is worth pinning here is not the layout but the *composition*:
* it renders the same `<seek-bar>`, `<player-controls>` and
* `<volume-control>` the desktop transport does, rather than its own.
* A phone layout that reimplements the transport is a second transport
* to fix every bug in and the seek bar in particular carries
* interpolation rules that took a plan of their own to get right.
*/
import { describe, expect, it, beforeEach } from 'vitest';
import '@components/now-playing-view/now-playing-view';
import { Events } from '../../src/events';
import { emit, resetHarness, stub } from '@test/support/harness';
import { fixture, shadow, text } from '@test/support/render';
import type { TrackInfo } from '@store/player-store';
const TRACK: TrackInfo = {
fileName: 'tideline.mp3',
filePath: '/music/tideline.mp3',
trackLength: 245,
seekPosition: 0,
state: 'playing',
title: 'Tideline',
artist: 'Sea Change',
album: 'Ebb',
coverArt: '/covers/ebb.jpg',
coverArtSmall: '/covers/ebb_sm.jpg',
coverArtMedium: '/covers/ebb_md.jpg',
coverArtLarge: '/covers/ebb_lg.jpg',
trackChangeId: 1,
artistMbid: '',
releaseGroupMbid: '',
recordingMbid: '',
};
describe('now-playing-view', () => {
beforeEach(() => {
resetHarness();
});
it('reuses the real transport components', async () => {
emit(Events.TrackChanged, TRACK);
const el = await fixture('now-playing-view');
for (const tag of ['seek-bar', 'player-controls', 'volume-control']) {
expect(shadow(el, tag), `${tag} is not rendered`).not.toBeNull();
}
});
it('shows the track, and the largest cover tier that is kept', async () => {
emit(Events.TrackChanged, TRACK);
const el = await fixture('now-playing-view');
expect(text(el, '[data-testid="npv-title"]')).toBe('Tideline');
// `saveCoverArt` records the largest *tier* as the path; there is
// no full-resolution original on disk to reach for.
expect(
shadow<HTMLImageElement>(el, '[data-testid="npv-art"]')?.getAttribute('src'),
).toBe('/covers/ebb_lg.jpg');
});
it('says so when nothing is playing, rather than rendering an empty frame', async () => {
// The player store is a singleton and outlives a test, so "no
// track" has to be stated rather than assumed from a fresh mount.
emit(Events.TrackChanged, null);
const el = await fixture('now-playing-view');
expect(shadow(el, '[data-testid="npv-empty"]')).not.toBeNull();
expect(shadow(el, '[data-testid="npv-art"]')).toBeNull();
// …and the way out is still there, which is the whole point of
// rendering the header in both branches.
expect(shadow(el, '[data-testid="npv-back"]')).not.toBeNull();
});
it('leaves by the nav stack, not by guessing where it came from', async () => {
emit(Events.TrackChanged, TRACK);
const el = await fixture('now-playing-view');
let backs = 0;
document.addEventListener('navigate-back', () => {
backs += 1;
});
shadow<HTMLButtonElement>(el, '[data-testid="npv-back"]')?.click();
// `navigate-back` pops what index.ts pushed. Dispatching a
// `navigate` to a hardcoded view would strand anyone who arrived
// here from a detail page.
expect(backs).toBe(1);
});
it('gives the favourite button a target and a state', async () => {
stub('playlist.Service.ToggleFavorite', undefined);
emit(Events.TrackChanged, TRACK);
const el = await fixture('now-playing-view');
const fav = shadow<HTMLButtonElement>(el, '[data-testid="npv-favorite"]');
expect(fav).not.toBeNull();
expect(fav?.getAttribute('aria-pressed')).toBe('false');
// A button that says only "heart" says nothing; the name carries
// the track and the playlist it goes to.
expect(fav?.getAttribute('aria-label')).toContain('Tideline');
expect(fav!.getBoundingClientRect().height).toBeGreaterThanOrEqual(48);
});
it('gives the way out a thumb-sized target', async () => {
emit(Events.TrackChanged, TRACK);
const el = await fixture('now-playing-view');
const back = shadow<HTMLButtonElement>(el, '[data-testid="npv-back"]');
expect(back!.getBoundingClientRect().height).toBeGreaterThanOrEqual(48);
expect(back?.getAttribute('aria-label')).toBe('Back');
});
});
@@ -0,0 +1,161 @@
/**
* The track list on a phone (plan 016 B2 phase 4).
*
* Measured on the device this was built for: at 424 CSS px the four
* configured columns fit the row *exactly* `--grid-cols` came out
* `24px 102px 101px 101px 80px` and not one of them fit its content.
* "Duration" did not fit its own header. The columns were never too
* wide; there were too many of them.
*
* So a phone draws one column of two lines plus the duration, and the
* split is a *column set* rather than a second row template: everything
* about a row that is not "how many columns" keeps working, which is
* what these tests pin. The `matchMedia` stub is the same one
* `now-playing.test.ts` uses the component reads the breakpoint from
* JS because its grid is computed in JS, so this is the seam.
*/
import { describe, expect, it, afterEach } from 'vitest';
import type { LitElement } from 'lit';
import '@components/track-list/track-list';
import { fixture, shadow, shadowAll } from '@test/support/render';
const TRACKS = Array.from({ length: 12 }, (_, i) => ({
FilePath: `/music/track-${i}.mp3`,
TrackName: `Track ${i}`,
ArtistName: `Artist ${i}`,
Album: 'An Album',
Duration: 180 + i,
})) as never[];
const real = window.matchMedia.bind(window);
/** Force the shell's phone breakpoint on or off. */
function stubPhone(phone: boolean): void {
window.matchMedia = ((q: string) =>
q.includes('max-width: 599px')
? {
matches: phone,
media: q,
addEventListener() {},
removeEventListener() {},
}
: real(q)) as typeof window.matchMedia;
}
async function mount(phone: boolean): Promise<LitElement> {
stubPhone(phone);
// Narrow, so a desktop layout at this width would be the cramped one
// the plan describes rather than a comfortable one.
const el = await fixture<LitElement>('track-list', {
externalTracks: TRACKS,
});
el.style.width = '424px';
el.style.height = '400px';
await el.updateComplete;
return el;
}
afterEach(() => {
window.matchMedia = real;
localStorage.removeItem('track-list-column-widths');
});
describe('the track list at phone width', () => {
it('draws the title with the artist under it, and the duration', async () => {
const el = await mount(true);
const row = shadow(el, '.track-row');
expect(row).not.toBeNull();
expect(shadow(el, '.stacked-title')?.textContent?.trim()).toBe('Track 0');
expect(shadow(el, '.stacked-sub')?.textContent?.trim()).toBe('Artist 0');
// Two drawn columns plus the favourite: three grid tracks, not five.
const tracks = getComputedStyle(row as Element)
.gridTemplateColumns.split(/\s+/)
.filter(Boolean);
expect(tracks).toHaveLength(3);
});
it('drops the column headers and the resize handles', async () => {
const el = await mount(true);
// Both are pointer affordances: a header is where a click sorts and
// a handle is where a drag resizes, and a phone can do neither.
expect(shadow(el, '.header-row')).toBeNull();
expect(shadowAll(el, '.col-resize-handle')).toHaveLength(0);
});
it('keeps every sort the desktop offers', async () => {
const el = await mount(true);
const header = shadow(el, 'page-header') as
| (Element & { sortOptions?: { id: string }[] })
| null;
// The regression this guards: building the sort list from the
// *drawn* columns would leave a phone able to sort by title and
// duration only, with no column headers to reach the rest by.
const ids = (header?.sortOptions ?? []).map((o) => o.id);
expect(ids).toContain('artistName');
expect(ids).toContain('album');
});
it('still marks the row a screen reader has to understand', async () => {
const el = await mount(true);
const row = shadow(el, '.track-row');
// The row is the same row: only the cells inside it changed, which
// is the entire argument for doing this as a column set.
expect(row?.getAttribute('role')).toBe('row');
expect(row?.getAttribute('aria-selected')).toBe('false');
expect(row?.getAttribute('data-testid')).toBe('track-row');
expect(shadowAll(el, '[role="gridcell"]').length).toBeGreaterThan(0);
});
it('ignores widths saved for the desktop, and does not overwrite them', async () => {
// The bug the device found, in the fixture that reproduces it.
// `loadColumnWidths` is keyed by column *id* and fills a gap with
// MIN_COLUMN_WIDTH, so the phone's stacked column -- which nothing
// can ever have saved a width for -- came out at the minimum while
// the duration column inherited a width dragged on a wide window:
// measured `24px 148px 236px` on a 424px phone.
const desktop = { trackName: 300, artistName: 200, album: 200, trackLength: 236 };
localStorage.setItem('track-list-column-widths', JSON.stringify(desktop));
const el = await mount(true);
const tracks = getComputedStyle(shadow(el, '.track-row') as Element)
.gridTemplateColumns.split(/\s+/)
.filter(Boolean)
.map((t) => Math.round(parseFloat(t)));
// Favourite, then the stacked column, then the duration -- and the
// stacked one is the widest thing in the row.
expect(tracks[1]).toBeGreaterThan(tracks[2] ?? 0);
// And the desktop's own widths survive being on a phone: writing the
// computed phone widths back would silently replace the width the
// user dragged for the same column id.
expect(JSON.parse(localStorage.getItem('track-list-column-widths') ?? '{}'))
.toMatchObject(desktop);
});
it('leaves the desktop alone', async () => {
const el = await mount(false);
const row = shadow(el, '.track-row');
expect(shadow(el, '.header-row')).not.toBeNull();
expect(shadow(el, '.stacked-title')).toBeNull();
const tracks = getComputedStyle(row as Element)
.gridTemplateColumns.split(/\s+/)
.filter(Boolean);
expect(tracks).toHaveLength(5);
});
});
+121
View File
@@ -0,0 +1,121 @@
/**
* A track credited to more than one artist has one navigable artist in
* this app and the rest are punctuation. `creditLink` is the fix: it
* renders a credit as one link per credited artist with the join
* phrases as plain text between them.
*
* The rule these tests exist to pin is that join phrases are
* **assembly** instructions, not disassembly instructions the credit
* is built from its parts, never found by searching a name inside a
* credit string. Measured on a real library, the stored credit text and
* the catalog's parts disagree for about one in three multi-artist
* credits, so a search would miss or match the wrong span.
*/
import { describe, expect, it } from 'vitest';
import { html, render } from 'lit';
import { creditLink, creditText, type CreditPart } from '@utils/explore-link';
const TUPAC = '11111111-1111-4111-8111-111111111111';
const SNOOP = '22222222-2222-4222-8222-222222222222';
const parts: CreditPart[] = [
{ creditedName: '2Pac', artistMbid: TUPAC, joinPhrase: ' feat. ' },
{ creditedName: 'Snoop Dogg', artistMbid: SNOOP, joinPhrase: '' },
];
function renderToEl(value: unknown): HTMLElement {
const host = document.createElement('div');
render(html`${value}`, host);
return host;
}
describe('creditLink', () => {
it('renders one link per credited artist', () => {
const el = renderToEl(creditLink(parts, '2Pac feat. Snoop Dogg', TUPAC));
const links = el.querySelectorAll('a.explore-link');
expect(links).toHaveLength(2);
expect(links[0]?.textContent).toBe('2Pac');
expect(links[1]?.textContent).toBe('Snoop Dogg');
});
it('puts the join phrase between the links as plain text', () => {
const el = renderToEl(creditLink(parts, '2Pac feat. Snoop Dogg', TUPAC));
// The whole credit reads correctly...
expect(el.textContent?.replace(/\s+/g, ' ').trim()).toBe(
'2Pac feat. Snoop Dogg',
);
// ...and " feat. " is not inside either link, which is the
// difference between a credit and a link with punctuation in it.
for (const link of el.querySelectorAll('a.explore-link')) {
expect(link.textContent).not.toMatch(/feat/);
}
});
it('falls back to a single link when there are no parts', () => {
const el = renderToEl(creditLink(undefined, 'Alina Baraz & Galimatias', TUPAC));
const links = el.querySelectorAll('a.explore-link');
expect(links).toHaveLength(1);
expect(links[0]?.textContent).toBe('Alina Baraz & Galimatias');
});
it('does not split the fallback string on its separators', () => {
// "&" and "with" appear inside real artist names — "Simon &
// Garfunkel" is one artist — so a credit with no parts is one
// link, always. This is the whole reason primaryArtist() does
// not split on them either.
const el = renderToEl(creditLink(undefined, 'Simon & Garfunkel', TUPAC));
expect(el.querySelectorAll('a.explore-link')).toHaveLength(1);
});
it('treats a one-part credit as the single-link case', () => {
// A zero- or one-part credit reaching the multi-artist branch
// would render as nothing, or as a link with a dangling join
// phrase after it.
const one: CreditPart[] = [
{ creditedName: 'Solo', artistMbid: TUPAC, joinPhrase: '' },
];
const el = renderToEl(creditLink(one, 'Solo', TUPAC));
expect(el.querySelectorAll('a.explore-link')).toHaveLength(1);
expect(el.textContent?.trim()).toBe('Solo');
});
it('renders the credited name, not the artist name', () => {
// MusicBrainz credits "Snoop Dogg" on a track by the artist
// called "Snoop Doggy Dogg". Display follows the credit;
// navigation follows the MBID.
const el = renderToEl(creditLink(parts, 'anything', TUPAC));
expect(el.textContent).toContain('Snoop Dogg');
expect(el.textContent).not.toContain('Snoop Doggy Dogg');
});
});
describe('creditText', () => {
it('reassembles the credit as a string', () => {
expect(creditText(parts, 'ignored')).toBe('2Pac feat. Snoop Dogg');
});
it('is the fallback string when there are no parts', () => {
expect(creditText(undefined, 'Alina Baraz & Galimatias')).toBe(
'Alina Baraz & Galimatias',
);
});
it('agrees with what creditLink renders', () => {
// The tooltip and the links come from the same parts by the same
// concatenation, so they cannot disagree.
const el = renderToEl(creditLink(parts, 'ignored', TUPAC));
expect(el.textContent?.replace(/\s+/g, ' ').trim()).toBe(
creditText(parts, 'ignored'),
);
});
});
+5 -1
View File
@@ -26,7 +26,11 @@ pre-commit:
go generate ./...
if [ -n "$(git diff --name-only)" ]; then
echo "Generated code is out of date. Run 'make generate' and stage the changes."
git diff --stat
# --no-pager, or this blocks forever on `less` waiting for a
# keypress that a hook run without a tty will never get: the
# commit hangs at exactly the moment it is trying to tell you
# why it failed.
git --no-pager diff --stat
exit 1
fi
+66 -1
View File
@@ -35,6 +35,8 @@ cd "$(dirname "$0")/.."
AVD="${YJ_AVD:-yj-test}"
SDK="${ANDROID_SDK_ROOT:-${ANDROID_HOME:-$HOME/Android/Sdk}}"
PKG="${YJ_ANDROID_PKG:-app.yellowjacket}"
# Where `make android-inspect` forwards the WebView's devtools socket.
CDP_PORT="${YJ_ANDROID_CDP_PORT:-9222}"
# **Not "$PKG/.MainActivity".** A leading-dot activity is resolved
# relative to the *applicationId*, and the scaffold's activity lives in
# the Java package `com.wails.app`, which is deliberately not the
@@ -264,6 +266,67 @@ cmd_logs() {
WailsBridge:V "$PKG":V GoLog:V AndroidRuntime:E DEBUG:V libc:F ActivityManager:I '*:S'
}
# Forward the WebView's devtools socket, so the page can be asked things.
#
# Only a `debuggable` build opens that socket, and a debug build carries
# `applicationIdSuffix ".dev"` precisely so it can be installed *beside*
# the release app: the two are signed by different certificates, and
# Android's remedy for a certificate change is an uninstall, which takes
# the user's library with it. So this looks for the sibling first and the
# release id second.
#
# The socket name carries the pid, which changes on every launch -- which
# is why this resolves it rather than documenting a number.
cmd_inspect() {
need_sdk
pick_device || die "no device -- plug a phone in (USB debugging on) or run 'make android-emulator'"
local pkg pid
pid=""
for pkg in "$PKG.dev" "$PKG"; do
pid=$("$ADB" shell pidof "$pkg" 2>/dev/null | tr -d '\r' | awk '{print $1}')
[ -n "$pid" ] && break
done
[ -n "$pid" ] || die "neither $PKG.dev nor $PKG is running; launch it first"
"$ADB" forward --remove-all >/dev/null 2>&1 || true
"$ADB" forward "tcp:$CDP_PORT" "localabstract:webview_devtools_remote_$pid" >/dev/null \
|| die "adb forward failed"
echo "android: $pkg (pid $pid) devtools on http://localhost:$CDP_PORT"
echo " make android-eval EXPR='JSON.stringify({vp:[innerWidth,innerHeight]})'"
echo " (a release build has no devtools socket: build and install the debug one)"
}
# What the phone is actually showing.
#
# This tier exists because no other one can see the platform: system
# bars, the safe area, the keyboard, an OEM's permission dialog. All of
# those are things you have to *look* at, and two of the three faults
# found so far were found by reading a picture rather than an assertion
# (`android-tier.md`).
#
# `exec-out` and not `shell`: `adb shell` runs the output through a pty
# on some platforms, which translates LF and corrupts the PNG -- for
# which the symptom is an image viewer refusing a file that downloaded
# perfectly.
cmd_screenshot() {
need_sdk
pick_device || die "no device \u2014 plug a phone in (USB debugging on) or run 'make android-emulator'"
local out="${1:-}"
[ -n "$out" ] || out=".dev/android-$(date +%Y%m%d-%H%M%S).png"
mkdir -p "$(dirname "$out")"
"$ADB" exec-out screencap -p > "$out" || die "screencap failed"
[ -s "$out" ] || die "screencap produced nothing (is the screen locked?)"
echo "android: screenshot -> $out"
}
# Start the app and assert it is *still the same process* a few seconds
# later. "It started" is not the question — a crash-looping app starts
# continuously.
@@ -316,9 +379,11 @@ stop) cmd_stop ;;
install) cmd_install ;;
launch) cmd_launch ;;
logs) cmd_logs ;;
screenshot) cmd_screenshot "${2:-}" ;;
inspect) cmd_inspect ;;
smoke) cmd_smoke "${2:-10}" ;;
*)
echo "usage: $0 {setup|start|stop|install|launch|logs|smoke [seconds]}" >&2
echo "usage: $0 {setup|start|stop|install|launch|logs|screenshot [path]|inspect|smoke [seconds]}" >&2
exit 2
;;
esac
+97
View File
@@ -0,0 +1,97 @@
/**
* Evaluate an expression inside the app's WebView on a real device.
*
* The device tier could only ever *look* at the app (a screenshot) or
* read what Go chose to log. This is the third thing: the page's own
* answer, from the engine that is actually rendering it which is how
* "the icons are missing" stops being a guess about assets and becomes a
* computed style.
*
* Two facts make it work at all. A `debuggable` build calls
* `WebView.setWebContentsDebuggingEnabled(true)`, which opens an abstract
* unix socket per process (`webview_devtools_remote_<pid>`); `make
* android-inspect` forwards it to localhost. And **Playwright cannot use
* it** `connectOverCDP` immediately calls `Browser.setDownloadBehavior`,
* which a WebView answers with "Browser context management is not
* supported", so the connection dies before the first evaluate. Raw CDP
* over Node's built-in WebSocket is a dozen lines and has no such
* opinion.
*
* Usage: node scripts/android-eval.mjs '<js expression>'
* make android-eval EXPR='...'
*
* The expression is evaluated with `awaitPromise`, so an async probe is
* fine. Return a string (`JSON.stringify(...)`) for anything structured:
* `returnByValue` will not serialise a DOM node.
*/
const PORT = process.env.YJ_ANDROID_CDP_PORT ?? '9222';
const expression = process.argv[2];
if (!expression) {
console.error("usage: node scripts/android-eval.mjs '<js expression>'");
process.exit(2);
}
const endpoint = `http://localhost:${PORT}/json`;
let targets;
try {
targets = await (await fetch(endpoint)).json();
} catch (err) {
console.error(
`android-eval: nothing on :${PORT} (${err.message})\n` +
" run 'make android-inspect' first, and check the phone is " +
'awake -- wireless adb drops when the screen sleeps',
);
process.exit(1);
}
const page = targets.find((t) => t.type === 'page');
if (!page) {
console.error('android-eval: no page target; is the app in the foreground?');
process.exit(1);
}
const ws = new WebSocket(page.webSocketDebuggerUrl);
await new Promise((resolve, reject) => {
ws.onopen = resolve;
ws.onerror = () => reject(new Error('websocket refused'));
});
const answer = await new Promise((resolve, reject) => {
const timer = setTimeout(() => reject(new Error('evaluate timed out')), 20_000);
ws.onmessage = (m) => {
const msg = JSON.parse(m.data);
if (msg.id !== 1) return;
clearTimeout(timer);
resolve(msg.result);
};
ws.send(
JSON.stringify({
id: 1,
method: 'Runtime.evaluate',
params: { expression, awaitPromise: true, returnByValue: true },
}),
);
});
ws.close();
if (answer.exceptionDetails) {
console.error(
'android-eval: threw:',
answer.exceptionDetails.exception?.description ??
answer.exceptionDetails.text,
);
process.exit(1);
}
const value = answer.result?.value;
console.log(typeof value === 'string' ? value : JSON.stringify(value, null, 2));
+15
View File
@@ -157,7 +157,22 @@ fi
# YJ_TESTCTL mounts backend/testctl's /__test/ endpoints. It is opt-in
# rather than implied by the dev build so that a human's `make dev` does
# not carry an arbitrary-SQL endpoint on a listening port.
#
# YJ_CORE_INDEX_URL points at a dead address, which is what
# seed-sandbox.sh and ci.yml already do and what this script was the
# only one *not* doing. Without it the app downloads and builds the
# real ~1M-row Explore catalog into the run's YJ_HOME, so a local `make
# e2e` runs against a different world than CI: the specs that stage
# their own catalog rows (requested-badge) then search a catalog full
# of real albums, fail to find their fixture, and report it as a
# regression in whatever was last changed. A spec tier whose result
# depends on what a previous run downloaded is not a result -- the same
# rule as the emulator's `-no-snapshot`.
#
# Set YJ_CORE_INDEX_URL yourself to opt back in, for exploring Explore
# by hand.
YJ_TESTCTL=1 \
YJ_CORE_INDEX_URL="${YJ_CORE_INDEX_URL:-http://127.0.0.1:1/none.tar.zst}" \
WAILS_SERVER_PORT="$PORT" \
YJ_LOG_LEVEL="$LOG_LEVEL" setsid dbus-run-session -- \
"$BIN" \
+89
View File
@@ -0,0 +1,89 @@
#!/bin/sh
# Snapshot the index build's database, which is the only copy of it.
#
# `/srv/yellowjacket/index-cache` is the `YJ_HOME` the index job keeps
# between runs (`.gitea/workflows/index-artifact.yml` mounts it at
# `/cache`). Its catalog is *derived*, not downloaded: the only way to
# rebuild it is to re-stream the MetaBrainz dumps, which is hours at a
# rate that is someone else's to decide. On 2026-08-17 a schema repair
# dropped it and cost exactly that.
#
# So it gets a snapshot, and this is the script a cron on that host runs.
# It is deliberately not part of the workflow: a backup that only exists
# while the thing it protects is being modified is not a backup.
#
# Usage (on the Gitea host):
#
# scripts/index-cache-snapshot.sh [SOURCE_HOME] [DEST_DIR] [KEEP]
#
# SOURCE_HOME default /srv/yellowjacket/index-cache
# DEST_DIR default /srv/yellowjacket/index-snapshots
# KEEP how many to retain, default 2
#
# Suggested cron — daily, and nowhere near the Monday 04:00 build:
#
# 30 5 * * * /path/to/index-cache-snapshot.sh >> /var/log/yj-index-snapshot.log 2>&1
#
# Three things about it are load-bearing.
#
# **`VACUUM INTO`, not `cp`.** The database may be open, and a byte copy
# of a live SQLite file is a corrupt file with a plausible size.
# `VACUUM INTO` takes a read lock, writes a consistent compacted copy,
# and is safe while the index job is running — it costs the snapshot's
# own write, not the source's availability.
#
# **The staging directory is not copied.** `/cache/data/explore-staging`
# is a resumable checkpoint of work in flight; it is large, it changes
# constantly, and a build resumes without it. What cannot be re-derived
# cheaply is the finished catalog, which is in the database.
#
# **A snapshot that is not verified is a belief.** Each one is opened
# and asked for its catalog row count before the old ones are rotated
# out, so a run that produced an unreadable file leaves the previous
# good snapshot in place and fails loudly.
set -eu
SOURCE_HOME="${1:-/srv/yellowjacket/index-cache}"
DEST_DIR="${2:-/srv/yellowjacket/index-snapshots}"
KEEP="${3:-2}"
DB="$SOURCE_HOME/data/yj.db"
STAMP=$(date +%Y%m%d-%H%M%S)
OUT="$DEST_DIR/yj-index-$STAMP.db"
die() { echo "index-snapshot: $*" >&2; exit 1; }
command -v sqlite3 >/dev/null 2>&1 || die "sqlite3 is not installed"
[ -f "$DB" ] || die "no database at $DB (is SOURCE_HOME right?)"
mkdir -p "$DEST_DIR"
# Headroom: the copy is at most the size of the source, usually less
# (VACUUM compacts). Refusing here beats a half-written snapshot.
need_kb=$(du -k "$DB" | cut -f1)
free_kb=$(df -Pk "$DEST_DIR" | awk 'NR == 2 { print $4 }')
[ "$free_kb" -gt "$need_kb" ] || die "not enough space in $DEST_DIR (need ~${need_kb}K, have ${free_kb}K)"
# A failed snapshot must leave nothing behind. `VACUUM INTO` refuses an
# existing file, so a partial one from a disk-full write would block
# every later run -- and worse, rotation counts files by name, so it
# would eventually be kept *instead of* a good one.
cleanup() { [ -n "${KEPT:-}" ] || rm -f "$OUT"; }
trap cleanup EXIT
echo "index-snapshot: $DB -> $OUT"
sqlite3 "$DB" "VACUUM INTO '$OUT'" || die "VACUUM INTO failed"
rows=$(sqlite3 "$OUT" "SELECT count(*) FROM explore_index" 2>/dev/null) \
|| die "snapshot is unreadable: keeping the previous ones"
[ "${rows:-0}" -gt 0 ] || die "snapshot has an empty catalog: keeping the previous ones"
KEPT=1
echo "index-snapshot: ok, $rows catalog rows, $(du -h "$OUT" | cut -f1)"
# Rotate only after the new one has been verified.
ls -1t "$DEST_DIR"/yj-index-*.db 2>/dev/null | tail -n +"$((KEEP + 1))" | while read -r old; do
echo "index-snapshot: removing $old"
rm -f "$old"
done
+71 -14
View File
@@ -1,17 +1,48 @@
#!/usr/bin/env bash
#
# Every command in .pi/ is a `make` target on purpose: the Makefile is
# the source of truth for *how* to invoke something, and the skill only
# decides *which* and *in what order*. This check keeps that honest —
# a renamed or deleted target turns into a failing commit rather than
# into an agent confidently running a command that no longer exists.
# Every command in the agent-facing docs is a `make` target on purpose:
# the Makefile is the source of truth for *how* to invoke something, and
# the docs only decide *which* and *in what order*. This check keeps
# that honest — a renamed or deleted target turns into a failing commit
# rather than into an agent confidently running a command that no longer
# exists.
#
# It extracts every `make <target>` mentioned under .pi/ and asserts the
# target exists. Usage: scripts/skill-check.sh
# It checks two things. Usage: scripts/skill-check.sh
#
# **Every `make <target>` named in an agent-facing doc exists.** The
# scanned set is `.pi/` *and* CLAUDE.md, which is the half that was
# missing: CLAUDE.md names 27 targets and nothing verified one of them,
# so the file the agents trust most was the file least checked.
#
# **AGENTS.md is a symlink to CLAUDE.md.** This repo is worked on by
# two agent harnesses that read different files by convention — Claude
# Code reads CLAUDE.md, others read AGENTS.md — and two harnesses
# reading two descriptions of one project is how they come to hold
# different beliefs about it. A symlink makes that impossible by
# construction; a *copy* would pass every other check in this repo while
# silently drifting, which is exactly the failure being prevented, so
# the symlink itself is asserted rather than its contents compared.
set -euo pipefail
cd "$(dirname "$0")/.."
# The symlink half runs even without .pi/, since it is not about .pi/.
if [ -e AGENTS.md ] || [ -L AGENTS.md ]; then
if [ ! -L AGENTS.md ]; then
echo "skill-check: AGENTS.md is a regular file, not a symlink to CLAUDE.md." >&2
echo " Two harnesses would read two descriptions of one project." >&2
echo " Fix: rm AGENTS.md && ln -s CLAUDE.md AGENTS.md" >&2
exit 1
fi
target="$(readlink AGENTS.md)"
if [ "$target" != "CLAUDE.md" ]; then
echo "skill-check: AGENTS.md points at '$target', expected CLAUDE.md." >&2
exit 1
fi
fi
[ -d .pi ] || exit 0
# `make -pq` prints the database including every rule, without running
@@ -21,11 +52,37 @@ targets="$({ make -pqRr 2>/dev/null || true; } |
awk '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ {sub(/:.*/, "", $0); print}' |
sort -u)"
# A mention counts only when it is code: backticked (`make ui-test`) or
# the first thing on a line, as in a fenced block. Bare prose is not
# scanned, because English says things like "a renamed make target".
mentioned="$(grep -rhoE '(`|^)make [a-z][a-z0-9-]*' .pi --include='*.md' |
sed 's/^`//' | awk '{print $2}' | sort -u)"
# A mention counts only when it is code: backticked (`make ui-test`)
# anywhere, or at the start of a line **inside a fenced block**. Bare
# prose is not scanned, because English says things like "a renamed make
# target".
#
# The fence is why this is awk rather than one grep. Line-start alone is
# not evidence of code in a file that is mostly hard-wrapped prose: the
# sentence "Two green branches do not / make a green merge" wrapped onto
# a line beginning `make a`, and the check duly failed on a target called
# `a`. Inside a fence it is code; outside one it is a sentence that
# happened to break there, and a check that fails on reflow gets
# disabled rather than fixed.
#
# AGENTS.md is deliberately not in this list: it is a symlink to
# CLAUDE.md, asserted above, so scanning it would report every failure
# twice under two names.
mentioned="$({ find .pi -name '*.md' 2>/dev/null; echo CLAUDE.md; } |
xargs awk '
FNR == 1 { fence = 0 }
/^```/ { fence = !fence; next }
{
rest = $0
while (match(rest, /`make [a-z][a-z0-9-]*/)) {
print substr(rest, RSTART + 6, RLENGTH - 6)
rest = substr(rest, RSTART + RLENGTH)
}
if (fence && match($0, /^make [a-z][a-z0-9-]*/)) {
print substr($0, 6, RLENGTH - 5)
}
}
' | sort -u)"
missing=""
@@ -36,10 +93,10 @@ for t in $mentioned; do
done
if [ -n "$missing" ]; then
echo "skill-check: .pi/ documents make targets that do not exist:" >&2
echo "skill-check: the agent docs name make targets that do not exist:" >&2
for t in $missing; do
echo " make $t" >&2
grep -rln "make $t" .pi --include='*.md' | sed 's/^/ /' >&2
grep -rln "make $t" .pi CLAUDE.md --include='*.md' | sed 's/^/ /' >&2
done
echo "Fix the docs, or restore the target." >&2
exit 1