Commit Graph
22 Commits
Author SHA1 Message Date
yonluandClaude Opus 5 e7748f1fd5 feat(database): shape the library like files, and shrink the catalog
CI / check (push) Successful in 3m7s
CI / e2e (push) Canceled after 1m45s
Plans 013 and 014, the album page that prompted them, and the smaller
fixes they turned up. Changelog, largest first.

## The local library is shaped like files, not like MusicBrainz

`audio_files` carries its own tags and points at `albums` and
`artists`; `file_genres` is the one real many-to-many. `recordings`,
`release_group_recordings`, `artist_credit`, `artist_credit_artist`,
`recording_genres`, `release_groups` and `release_to_rg` are gone from
the local side, and with them a six-way join in every read, a
`MIN(release_group_id)` subquery in eleven queries and a
first-credited-artist subquery in nine. Measured on a real 25,966-file
library, every many-to-many that model expressed was 1:1 in the data.

- Ownership is a file. `GetFilePathsByRecordingMBIDs`,
  `LibraryMBIDIndex.CheckMBIDs`, `collectLibraryEntities` and
  `pruneStaleLocalCrossReferences` all join `audio_files`, so the 812
  orphaned recordings, 216 release groups and 260 artists that library
  carried are now structurally impossible.
- One projection: every track query selects from the `track_metadata`
  view, one row type, one mapper. Nine hand-rolled copies had drifted
  far enough to report different years on different screens.
- `library_id = 0` means every library, so each list query exists once
  instead of scoped and unscoped with a branch at every call site.
- No migration chain. `sql/schemas/` is the one description of the
  shape; `sql/migrations/`, `applyMigrations` and `schema_migrations`
  are squashed away, along with the drift between them that had sqlc
  generating against a stale schema.
- `database.InsertTestTrack` is the one test seeder; twenty test files
  had been assembling the old FK chain each in its own order.

## The catalog stores its ids as bytes

`explore_index`'s three 36-char MBID columns and its entity-type text
are 16 raw bytes and a small integer. The table and its six indexes go
780 MB to 405 MB on a real 2,052,200-row catalog, which is why a fresh
install is ~0.6 GB rather than ~1.0 GB.

- `backend/explore/mbid.go` is the only place the encoding is known;
  everything above it speaks dashed strings.
- `CHECK(length(mbid) = 16)` makes a stringly write fail at the insert
  rather than silently returning no rows, since SQLite does not coerce
  between TEXT and BLOB.
- The importer asks the artifact what encoding it carries and converts
  on the way in, so the artifact already published keeps working and no
  format bump is needed.
- `indexRowColumns`/`scanIndexRow` replace four copies of a 22-column
  list, and `TestStoredEncodingRoundTrips` sweeps every read path.

## An album page that says how much of the album is yours

- One question, asked once: is there a file. `filePaths` is filled by a
  single batched lookup when the tracklist settles, and the badge, the
  Play count, the dimmed rows and every menu item read it — replacing
  four claims of decreasing confidence that could show a green tick on
  an album whose every action did nothing.
- Play, Play 7 of 12, or no play button at all.
- `total_tracks` on `explore_index` (~2 bytes over 400,677 release
  groups) and on `audio_files` from tags that have always carried it:
  a complete MBID-matched album now makes no catalog call at all, where
  it used to spend the most expensive request the app makes.
- A merged cluster shows the running order the most releases agree on,
  and the version list marks the release you own rather than standing a
  synthetic entry in for it.
- `AlbumReleasesFailed`: a slow fetch is no longer reported as a failed
  one by a 12-second timer.
- Rows not in the library are dimmed in place (with `aria-disabled`)
  instead of the owned ones wearing a green tick and a legend.

## Caches and cover art get ceilings

- Only the three tiers of a cover are stored; the full-resolution copy
  nothing rendered was 1,134 MB of a 1.4 GB covers directory.
- One artist portrait is downloaded and the rest are remembered as
  URLs — 4.1 GB of a 5.3 GB cache was candidates no code path reads.
- `browsedArtBudget` and `httpCacheBudget` bound what an age cannot:
  the same install held art for 5,770 artists in a 1,301-artist
  library.
- `OrphanedArtistImagesJob` joined a bare MBID onto a sharded
  directory, so it deleted the rows that were the only record of the
  files it left behind. `explore.ArtistImageDir` is that layout's one
  definition now.

## The autotag queue asks whether there is work

`tagging_items` was a row per album folder, not a queue, and no query
read the `tag_status` column that held the answer. The four queue
queries ask the files, which matters most where it is least visible:
`startPrefetch` was scoring every album in a tagged library against
MusicBrainz.

## Phantom playlist tracks resolve in place

An M3U8 imported before its files leaves phantom rows; they now match
by path and fall back to position, keep their place in the playlist
when resolved, and pair best-first so two phantoms cannot claim the
same file.

## Playing a track plays the list it is in

Double-click, and Play on a single row's menu, queue the list as
displayed with `startIndex` on that row — the album page and the track
list used to queue one track and discard the album around it. A
multi-row selection still plays exactly itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AfVYUVExXsx1nSWrXN8mAh
2026-08-16 13:58:15 -04:00
yonluandClaude Opus 5 4471db3aef feat(wails): move the Go side to v3
Phases 2 and 3 of plan 009, plus the parts of phase 1 that could not
land before them. Nothing in the tree imports wails/v2 any more; all
three lint and test configurations are green and `go build .` produces
a running binary.

The point of the migration is one file. backend/events/emit.go probed
ctx.Value("events") — a v2-*private* context key — to decide whether
emitting was safe, because runtime.EventsEmit called log.Fatalf on a
context without the runtime and took the process down with it. v3's
emit takes no context, so that is now application.Get() == nil. D1
held: events.Emit keeps its ctx as the WithSink test seam, and all 45
call sites and 7 test files are untouched.

The bootstrap splits into application.New + Window.NewWithOptions +
Run. Ten bound services implement ServiceStartup instead of being
handed a context by hand from OnStartup, which also stops ten
SetContext methods being exported as bindings. jobs.Registry and
explore.SearchIndex keep theirs — neither is bound, so converting them
would be churn for no binding removed.

Four things differed from the plan and are written up in it: GPU policy
moved to the per-window LinuxWindow options rather than surviving on
LinuxOptions; there is no OnStartup/OnDomReady option, so app-level
wiring hangs off ApplicationStarted; application.NewService is generic,
so FEBindings []any could not survive (the binding generator is a
static analyser and would have seen nothing); and the quit veto had to
be restructured, because v3's dialog answers on a callback rather than
returning the button, so ShouldQuit vetoes, asks, and quits again from
the callback.

Window state saving moves to a WindowClosing hook — the size has to be
read while the window still exists, and v3's OnShutdown has neither
context nor window. backend/logging is deleted rather than ported:
v3 takes a *slog.Logger directly, so the v2 logger.Logger adapter had
no caller left.

Phase 1's tail rides along, now that it can: the Makefile's wails
invocations, all 50 webkit2_41 sites, lefthook, both packaging recipes
and ci.yml's apt lists. v3 builds against GTK4 + WebKitGTK 6.0, which
Arch and ubuntu:24.04 both ship, so the tag is a deletion rather than
a translation.

Phase 4 is next and the branch is not usable until it lands: the app
builds, but frontend/wailsjs/ is v2's tree and nothing regenerates it,
so the frontend cannot reach the backend yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UDCbcCZQepnpSQYJ6SxxZm
2026-08-14 14:01:02 -04:00
logan 5ca6cad45a feat(harness): agent-drivable dev harness and CI that gates
Build & publish Arch package / arch-package (push) Successful in 2m8s
CI / check (push) Failing after 1m56s
CI / e2e (push) Skipped
Search index maintenance / maintain-index (push) Successful in 13s
A coding agent could develop this repo's Go packages and could not
develop the application: every path to running YellowJacket ended in a
blocking GTK window, so 265 bound methods, 46 events, 33 component
directories and 13 stores had exactly one form of verification
available — `tsc --noEmit`.

The unlock is that `wails dev`'s dev server on :34115 serves the real
frontend with the real generated bindings against the same Go backend a
desktop window attaches to, so a plain Chromium under Xvfb gets a fully
functional app. Four test tiers now exist, cheapest first:

- `make ui-test` — 313 Vitest tests in a real browser in ~2 s, no app,
  no backend, no display. Works because `frontend/wailsjs/` is a pure
  passthrough to `window.go`/`window.runtime`, so faking just those two
  globals runs the real bindings and the real store code.
- `make test` — services in-process, asserting on the payload the
  frontend would receive, via a new `events.Emit` wrapper.
- `make dev-headless` + `playwright-cli` — the real app, driven
  interactively, with an event bridge on `window.__yjEvents` and a
  dev-only control surface at `/__test/`.
- `make e2e` — 19 of those flows frozen as Playwright specs.

`events.Emit(ctx, …)` replaces all 35 direct `runtime.EventsEmit` call
sites: wails' `getEvents` `log.Fatalf`s on any context without its
runtime, so those paths could not run under test and a background
worker could take the app down. Four packages had each hand-rolled the
same guard; nine more guarded on `ctx != nil`, which does not help.
`TestNoDirectRuntimeEmits` fails the build on a new one.

Fixtures are generated, not committed (`make testdata`), and seeds are
built by *running the app* — never by hand-writing config and DB rows,
which would be a second description of a valid YJ_HOME.

`.gitea/workflows/ci.yml` is the first workflow here that tests
anything; the other three only package, so `gitea_ci` reported only
packaging jobs and misled anyone asking whether a push was healthy.
Both jobs were prototyped to green in a bare ubuntu:24.04 container
before the YAML was written, which immediately caught `make lint`
linting three configurations that nothing builds: all three passes
omitted `webkit2_41`, so wails resolved webkit2gtk-4.0 — which Arch
still ships and Ubuntu 24.04 dropped.

Operational instructions live in `.pi/skills/yellowjacket-dev/`,
measured discoveries in `.planning/NOTES.md`, and architecture in
`CLAUDE.md` — split by tense, not by topic, because a topical split
gives every new fact two plausible homes. `make skill-check` fails a
commit if the skill cites a make target that does not exist.
2026-08-10 23:20:42 -04:00
yonluandClaude Sonnet 5 e190fd75b9 feat: data lifecycle rewrite, download clients, wanted list, and central catalog index
Build & publish Arch package / arch-package (push) Successful in 2m12s
Search index maintenance / maintain-index (push) Successful in 2h22m28s
Ships the fresh-start schema cleanup: rebuilt explore catalog index
pipeline (dump import, artifact fetch/build, incremental listen-count
refresh), a new download subsystem (Lidarr/Prowlarr/qBittorrent/SABnzbd/
slskd/yt-dlp providers, staging, reconciliation, wanted list), and the
supporting schema/query/store changes across backend and frontend.

Also includes two smaller follow-ups: bump the central index's
rebuild-after cadence from 90 to 180 days, and remove the Explore
"library only" online/offline toggle entirely (frontend-only, no
backend counterpart) rather than carry unused UI/state.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2Agd9af5hE7qzti2ackiS
2026-08-06 17:12:01 -04:00
yonlu f16157a213 fix(S21/T01): fix all lint warnings and upgrade wsl to wsl_v5
Files:
- .golangci.yml
- backend/events/cmd/genevents/main.go
- backend/fileutil/atomicwrite_test.go
- backend/library/library.go
- backend/player/buffered_streamer_test.go
- backend/player/player.go
- backend/tagwriter/dbsync.go
- backend/tagwriter/mp3_test.go
- backend/tagwriter/ogg.go
- backend/tagwriter/ogg_test.go
- backend/tagwriter/ogg_vorbis.go
- backend/tagwriter/pipeline.go
- backend/tagwriter/tagwriter.go
- backend/tagwriter/wav_test.go
2026-03-20 14:36:18 -04:00
yonlu f13aa7086b test(20-02): add round-trip tests for all OGG requirements
- TestWriteOggTags_TextFields: all 9 text fields round-trip via dhowden/tag (OGG-01)
- TestWriteOggTags_CoverArt: METADATA_BLOCK_PICTURE embed verified (OGG-04)
- TestWriteOggTags_ClearCoverArt: cover art clear via nil (OGG-04)
- TestWriteOggTags_PartialUpdate: non-edited fields preserved (OGG-02)
- TestWriteOggTags_AudioPreservation: audio page data byte-identical (OGG-03)
- TestWriteOggTags_AtomicSafety: corrupt file untouched on failure (OGG-05)
- TestWriteOggTags_RejectNonVorbis: Theora OGG rejected (OGG-03 error path)
- TestWriteOggTags_RejectMultiStream: multi-serial rejected
2026-03-19 14:02:02 -04:00
yonlu 246a991c75 test(20-02): add OGG test fixture builder and CRC32 validation
- createTestOGG builds minimal valid OGG Vorbis file programmatically
- TestOggCRC_KnownVectors validates CRC32 against known vectors and fixture pages
- TestCreateTestOGG_Valid verifies fixture structure and metadata.ExtractTags compatibility
2026-03-19 14:00:47 -04:00
yonlu 5e98c03634 feat(20-01): implement OGG Vorbis tag writer with custom page parser and CRC32
- Custom OGG page parser/writer with MSB-first CRC32 lookup table (ogg.go)
- Vorbis Comment packet parse/serialize with raw byte preservation (ogg_vorbis.go)
- METADATA_BLOCK_PICTURE base64 encoding for cover art with legacy field stripping
- Multi-stream and non-Vorbis OGG rejection with clear error messages
- Pipeline integration: FormatOGG constant, .ogg in DetectFormat, writeOggTags dispatch
- Lenient-read/strict-write: warn on CRC mismatch, always write correct CRCs
- Page sequence renumbering and crash-safe writes via AtomicWrite
2026-03-19 13:53:36 -04:00
yonlu f11b523afb style(19-02): fix wsl lint warnings in WAV test and writer
- Add blank lines before cuddled expressions in wav_test.go
- Fix cuddled copy expression in wav.go writeRIFF
- All WAV lint issues resolved (remaining are pre-existing in other files)
2026-03-19 08:57:57 -04:00
yonlu 1b28882af4 test(19-02): add round-trip tests for all WAV tag writer requirements
- TestWriteWavTags_TextFields: 9 field round-trip (WAV-01)
- TestWriteWavTags_CoverArt: JPEG embed/read-back (WAV-04)
- TestWriteWavTags_ClearCoverArt: embed then clear (WAV-04)
- TestWriteWavTags_PartialUpdate: 2-of-9 change, 7 preserved (WAV-01)
- TestWriteWavTags_ChunkPreservation: fmt/data/LIST/bext preserved (WAV-02, WAV-03)
- TestWriteWavTags_AtomicSafety: failed write leaves file untouched (WAV-05)
- TestWriteWavTags_RejectsRF64: RF64 rejected with clear error (WAV-02)
- readWavID3Tags uses bogem/id3v2 ParseReader for reliable read-back
2026-03-19 08:53:36 -04:00
yonlu 21fe17212d test(19-02): add WAV test fixture builder and read-back helper
- createTestWAV: builds minimal valid WAV with optional ID3v2 tag
- readWavID3Tags: extracts ID3v2 from RIFF via parseRIFF + dhowden/tag
- createTestWAVWithExtraChunks: adds LIST INFO and bext chunks for preservation tests
- makePCMFmtData: generates 16-byte PCM format data
2026-03-19 08:50:20 -04:00
yonlu e6610ff15e feat(19-01): implement WAV RIFF parser/writer and writeWavTags
- Add FormatWAV constant and .wav DetectFormat case
- Add FormatWAV dispatch in pipeline WriteTrackTags switch
- Create wav.go with custom RIFF chunk parser/writer
- parseRIFF: lenient read, RF64 rejection, case-insensitive ID3 chunk detection
- writeRIFF: strict write with correct padding and 4GB size check
- writeWavTags: merge existing ID3v2, reuse applyTextChanges/applyCoverArtChanges
- Atomic write via fileutil.AtomicWrite for crash safety
2026-03-19 08:44:38 -04:00
yonlu 8f4c4a0c2b fix(19-01): add album_artist TPE2 mapping to applyTextChanges
- Map FieldAlbumArtist to TPE2 frame via CommonID
- Follows same pattern as FieldComposer → TCOM
- Fixes latent gap in MP3 writing, enables WAV reuse
2026-03-19 08:42:53 -04:00
yonlu f557ffd652 feat(18-01): add BatchWriteTrackTags with progress, cancellation, and partial failure
- Add BatchFailure and BatchResult types for structured batch outcomes
- Add cancelBatch channel and suppressEvents flag to TagWriter struct
- Add CancelBatchWrite method for mid-batch cancellation from frontend
- Add BatchWriteTrackTags method processing tracks sequentially
- Emit BatchWriteProgress event per-track with current/total/succeeded/failed
- Suppress per-track TrackMetadataChanged; emit single event after batch
- Wails bindings auto-generated for BatchWriteTrackTags and CancelBatchWrite
- tagwriter namespace with BatchResult/BatchFailure in models.ts
2026-03-18 13:02:30 -04:00
yonlu d7c2965752 fix(17-02): fix cover art replace and remove
Three issues fixed:

1. asBytes() helper for []interface{} → []byte conversion — same
   float64 deserialization issue as numeric fields. Cover art data
   from the frontend arrives as []interface{} of float64, not []byte.

2. DB sync for cover art — was a placeholder no-op. Now saves image
   to covers cache dir (content-hash dedup + thumbnail generation),
   upserts cover_art row, and updates release_groups.cover_art_id.
   Clear sets cover_art_id to NULL on linked release groups.

3. Frontend ReadFile returns base64 string (Go []byte JSON encoding),
   not number[]. Decode with atob() before creating Uint8Array for
   preview blob URL.
2026-03-18 10:36:17 -04:00
yonlu 900db2e56c fix(17-02): handle float64 numeric values from Wails JSON deserialization
Wails deserializes JSON numbers from JavaScript as float64, not int.
All .(int) type assertions on year, track_number, and disc_number
silently failed. Add asInt() helper, replace all assertions.
2026-03-18 07:46:29 -04:00
yonlu 4235b4a4d5 feat(17-01): add WriteTrackTagsByPath and ImageFilePicker backend methods
- WriteTrackTagsByPath resolves filePath to trackID via GetAudioFileByPath
- ImageFilePicker opens native file dialog filtered to JPEG/PNG
- Added Wails TypeScript bindings for both new methods
2026-03-17 21:03:17 -04:00
yonlu 64322f9353 feat(16-03): WriteTrackTags pipeline with player safety, scan mutex, events, and app wiring
- TagWriter struct with WriteTrackTags entry point orchestrating full pipeline
- PlayerStopper interface + playerAdapter for import-cycle-free player safety
- PipelineLocker interface + pipelineMu on Library for scan/write mutual exclusion
- TrackMetadataChanged event constant + auto-generated TypeScript events
- TagWriter wired into app.go as Wails binding with SetContext in OnStartup
- 5 pipeline integration tests: player safety, scan mutex, orphan cleanup, genre relink, full DB sync
- Lint fixes: static error, else-if, wsl cuddle, golines
2026-03-17 10:55:10 -04:00
yonlu 2966079625 feat(16-03): implement DB sync module for tag write pipeline
- syncDatabase runs entity relink + FTS5 + orphan cleanup in single tx
- Handles artist, album, genre changes with upsert-and-relink pattern
- FTS5 delete + insert within transaction for search index consistency
- Orphan cleanup for artist_credits, release_groups, and genres
- All hand-crafted SQL has SAFETY comments per codebase convention
2026-03-17 10:48:03 -04:00
yonlu 6bd65a6f43 docs(16-02): complete FLAC tag writer plan
- Create 16-02-SUMMARY.md with execution results
- Update STATE.md with Phase 16 progress and decisions
- Update ROADMAP.md plan progress
- Mark WRITE-02 and WRITE-04 requirements complete
2026-03-17 10:35:38 -04:00
yonlu a677a44d49 test(16-02): add FLAC writer round-trip tests with 7 test functions
- TestWriteFlacTags_TextFields: round-trip 9 text fields via metadata.ExtractTags
- TestWriteFlacTags_CoverArt: JPEG embed and read-back verification
- TestWriteFlacTags_ClearCoverArt: embed then clear art, verify removal
- TestWriteFlacTags_PartialUpdate: update 2 of 8 fields, verify others preserved
- TestWriteFlacTags_PreservesStreamInfo: verify StreamInfo block survives tag writes
- TestWriteFlacTags_ReplaceComment: write twice, verify no duplicate Vorbis Comments
- TestWriteFlacTags_AtomicSafety: verify corrupt/invalid writes leave original intact
- Create shared helpers_test.go with testLogger, tinyJPEG, assertEqual, etc.
2026-03-17 10:32:46 -04:00
yonlu 3642cbe0d5 feat(16-02): add go-flac dependencies and implement FLAC tag writer
- Add go-flac/go-flac/v2, flacvorbis/v2, flacpicture/v2 dependencies
- Create tagwriter.go with TagChanges type, field constants, format detection, MIME detection
- Create flac.go with writeFlacTags using Vorbis Comments + PICTURE blocks + AtomicWrite
- Implement replaceVorbisComment helper for case-insensitive field replacement
- Handle cover art add/replace/clear via PICTURE metadata blocks
2026-03-17 10:32:28 -04:00