From 765204e8ebe9960f575251332a9e4ed2b134c4b8 Mon Sep 17 00:00:00 2001 From: Caleb Allen Date: Mon, 9 Mar 2026 09:54:12 -0400 Subject: [PATCH] docs(10-02): complete sqlc queries & migration tests plan - SUMMARY: Library CRUD queries, phantom-aware playlist queries, 5 migration tests - STATE: Phase 10 complete (2/2 plans), ready for Phase 11 - ROADMAP: Phase 10 marked complete - REQUIREMENTS: LIB-04, LIB-05 marked complete --- .planning/REQUIREMENTS.md | 8 +- .planning/ROADMAP.md | 10 +- .planning/STATE.md | 21 +-- .../10-schema-migration/10-02-SUMMARY.md | 132 ++++++++++++++++++ 4 files changed, 153 insertions(+), 18 deletions(-) create mode 100644 .planning/phases/10-schema-migration/10-02-SUMMARY.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 9759df1..ade3a9b 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -26,8 +26,8 @@ Requirements for v1.1 Multi-Library Support milestone. Each maps to roadmap phas - [ ] **LIB-01**: User can add a new library directory via a folder picker dialog - [ ] **LIB-02**: User can rename a library (display name) - [ ] **LIB-03**: User can remove a library — tracks are deleted from DB, shared entities (artists, albums, genres) are cleaned up only if no other library references them -- [ ] **LIB-04**: Libraries are stored in SQLite (not TOML config) with CRUD through the UI -- [ ] **LIB-05**: Existing single-directory config is migrated seamlessly to the libraries table on first run after upgrade +- [x] **LIB-04**: Libraries are stored in SQLite (not TOML config) with CRUD through the UI +- [x] **LIB-05**: Existing single-directory config is migrated seamlessly to the libraries table on first run after upgrade - [ ] **LIB-06**: Library list is displayed in a management UI (settings or sidebar section) ### Library Scanning @@ -183,8 +183,8 @@ Which phases cover which requirements. Updated during roadmap creation. | LIB-01 | Phase 12 | Pending | | LIB-02 | Phase 12 | Pending | | LIB-03 | Phase 12 | Pending | -| LIB-04 | Phase 10 | Pending | -| LIB-05 | Phase 10 | Pending | +| LIB-04 | Phase 10 | Complete | +| LIB-05 | Phase 10 | Complete | | LIB-06 | Phase 12 | Pending | | LSCAN-01 | Phase 11 | Pending | | LSCAN-02 | Phase 11 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 8fd516c..bcec0c4 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -28,7 +28,7 @@ ### v1.1 Multi-Library Support (Phases 9-13) - [x] **Phase 9: Scan Cancellation & Keyboard Shortcuts** — Cancellable library scans and configurable keyboard shortcuts -- [ ] **Phase 10: Schema & Migration** — Libraries table, library_id FK, playlist_tracks phantom rebuild, config migration +- [x] **Phase 10: Schema & Migration** — Libraries table, library_id FK, playlist_tracks phantom rebuild, config migration (completed 2026-03-09) - [ ] **Phase 11: Per-Library Scan Pipeline** — Scan pipeline refactored for per-library scanning with sequential coordination - [ ] **Phase 12: Library CRUD & Data Integrity** — Library management API, orphan cleanup, queue/playlist lifecycle, library manager UI - [ ] **Phase 13: Library Views & Phantom Tracks** — Filtered presentation across all views, search, browse, and phantom track display @@ -62,10 +62,10 @@ Plans: 2. An existing user's database is migrated on first launch: their single directory becomes a named library, all existing audio_files get that library_id, and everything works without any user action 3. The `playlist_tracks` table supports nullable `audio_file_id` with phantom metadata columns — the schema is ready for phantom track preservation 4. All migration operations complete atomically — a crash mid-migration leaves the database unchanged (not half-migrated) -**Plans:** 1/2 plans executed +**Plans:** 2/2 plans complete Plans: -- [ ] 10-01-PLAN.md — Schema definitions + Migration 6 (libraries table, library_id FK, phantom columns, track_metadata VIEW, backup, TOML migration) -- [ ] 10-02-PLAN.md — sqlc queries for libraries + updated playlist phantom queries + migration integration tests +- [x] 10-01-PLAN.md — Schema definitions + Migration 6 (libraries table, library_id FK, phantom columns, track_metadata VIEW, backup, TOML migration) +- [x] 10-02-PLAN.md — sqlc queries for libraries + updated playlist phantom queries + migration integration tests ### Phase 11: Per-Library Scan Pipeline **Goal:** Users can scan individual libraries independently with proper sequential coordination @@ -115,7 +115,7 @@ Plans: | 7. Backend Performance | v1.0 | 2/2 | Complete | 2026-03-05 | | 8. Frontend Performance & UX | v1.0 | 4/4 | Complete | 2026-03-05 | | 9. Scan Cancellation & Keyboard Shortcuts | v1.1 | 5/5 | Complete | 2026-03-07 | -| 10. Schema & Migration | 1/2 | In Progress| | - | +| 10. Schema & Migration | 2/2 | Complete | 2026-03-09 | - | | 11. Per-Library Scan Pipeline | v1.1 | 0/? | Not started | - | | 12. Library CRUD & Data Integrity | v1.1 | 0/? | Not started | - | | 13. Library Views & Phantom Tracks | v1.1 | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index a3fbfd0..204dc67 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -23,17 +23,17 @@ See: .planning/PROJECT.md (updated 2026-03-08) ## Current Position Phase: 10 — Schema & Migration -Plan: 1 of 2 in Phase (Plan 01 complete) -Status: In progress — executing Phase 10 -Progress: ████░░░░░░░░░░░░░░░░ 1/5 phases complete (Phase 9) -Last activity: 2026-03-09 — Completed 10-01 schema files + migration 6 +Plan: 2 of 2 in Phase (Plan 02 complete) +Status: Phase 10 complete — ready for Phase 11 +Progress: ████████░░░░░░░░░░░░ 2/5 phases complete (Phase 10) +Last activity: 2026-03-09 — Completed 10-02 sqlc queries + migration tests ### Phase Overview | Phase | Status | |-------|--------| | 9. Scan Cancellation & Keyboard Shortcuts | Complete (5/5 plans) ✅ | -| 10. Schema & Migration | In progress (1/2 plans) | +| 10. Schema & Migration | Complete (2/2 plans) ✅ | | 11. Per-Library Scan Pipeline | Not started | | 12. Library CRUD & Data Integrity | Not started | | 13. Library Views & Phantom Tracks | Not started | @@ -51,6 +51,7 @@ Last activity: 2026-03-09 — Completed 10-01 schema files + migration 6 | 09-03 | scan control UI | 2 min | 1 | 3 | | 09-05 | integration testing & verification | 3 min | 2 | 1 | | Phase 10-01 P01 | 11 min | 2 tasks | 10 files | +| Phase 10-02 P02 | 5 min | 2 tasks | 9 files | ## Accumulated Context @@ -87,6 +88,8 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns | Sentinel library id=0 in test DB | Existing tests use DEFAULT library_id=0; sentinel row satisfies FK without modifying every test | | TOML cleanup via generic map[string]any | Preserves all config sections when removing only DirectoryPath; no dependency on full Config struct | | sql.NullInt64 for nullable playlist_tracks.audio_file_id | Phantom tracks have NULL audio_file_id; generated sqlc code requires sql.Null types | +| COALESCE fallback chain: live → phantom → empty | Playlist queries use 3-level COALESCE so callers always get usable string values | +| is_phantom computed column via CASE WHEN | Eliminates null-checking logic in callers; simple int64 boolean (0/1) | ### Warnings (carry forward) @@ -110,9 +113,9 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns ### Last Session **Date:** 2026-03-09 -**What happened:** Executed Phase 10, Plan 01 — created multi-library schema files (_libraries.sql, updated audio_files/playlist_tracks/track_metadata_view) and implemented migration 6 with backup, TOML config migration, phantom metadata backfill. 3 auto-fixed deviations (sqlc regen, test FK fix, linter fix). -**Where we stopped:** Completed 10-01-PLAN.md (schema files + migration 6) -**Next action:** Execute Phase 10, Plan 02 or continue to Phase 11 +**What happened:** Executed Phase 10, Plan 02 — added 7 library CRUD queries, updated all playlist queries for phantom support (LEFT JOINs, COALESCE fallback, is_phantom column), added library-filtered audio file queries, wrote 5 migration 6 integration tests, added NewTestDBWithLibrary helper. +**Where we stopped:** Completed 10-02-PLAN.md (Phase 10 complete) +**Next action:** Plan or execute Phase 11 (Per-Library Scan Pipeline) --- *State initialized: 2026-02-27* @@ -125,4 +128,4 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns | 18 | add multi-column metadata display to playlist-details | 2026-03-08 | ce23177 | [18-add-multi-column-metadata-display-to-pla](./quick/18-add-multi-column-metadata-display-to-pla/) | Last activity: 2026-03-08 - Completed quick task 18: add multi-column metadata display to playlist-details -*Last updated: 2026-03-09 — Completed 10-01-PLAN.md (schema + migration 6)* +*Last updated: 2026-03-09 — Completed 10-02-PLAN.md (Phase 10 complete)* diff --git a/.planning/phases/10-schema-migration/10-02-SUMMARY.md b/.planning/phases/10-schema-migration/10-02-SUMMARY.md new file mode 100644 index 0000000..35e06dd --- /dev/null +++ b/.planning/phases/10-schema-migration/10-02-SUMMARY.md @@ -0,0 +1,132 @@ +--- +phase: 10-schema-migration +plan: 02 +subsystem: database +tags: [sqlite, sqlc, queries, phantom-tracks, migration-tests, multi-library] + +# Dependency graph +requires: + - phase: 10-schema-migration plan 01 + provides: libraries table, audio_files.library_id, playlist_tracks phantom columns, migration 6 +provides: + - sqlc CRUD queries for libraries table (7 queries) + - Updated playlist queries with phantom metadata support and LEFT JOINs + - GetTrackPhantomMetadata helper query for eager phantom population + - Audio file queries filtered by library_id + - Migration 6 integration tests (5 test functions) + - NewTestDBWithLibrary helper for downstream test usage +affects: [11-per-library-scan, 12-library-crud, 13-library-views] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "LEFT JOIN for nullable FK columns in sqlc queries" + - "COALESCE fallback chain: live metadata → phantom metadata → empty string" + - "is_phantom computed column via CASE WHEN for phantom track detection" + - "NewTestDBWithLibrary helper for tests needing pre-populated library" + +key-files: + created: + - backend/database/sql/queries/libraries.sql + - backend/database/sql/sqlcgen/libraries.sql.go + - backend/database/database_test.go + modified: + - backend/database/sql/queries/audio_files.sql + - backend/database/sql/queries/playlists.sql + - backend/database/sql/sqlcgen/audio_files.sql.go + - backend/database/sql/sqlcgen/playlists.sql.go + - backend/database/testhelper.go + +key-decisions: + - "COALESCE fallback chain for phantom metadata: prefer live data over phantom data over empty string" + - "Computed is_phantom column via CASE WHEN rather than requiring callers to check audio_file_id" + - "GetPlaylistTrackFilePaths filters out NULLs with audio_file_id IS NOT NULL" + +patterns-established: + - "LEFT JOIN + COALESCE pattern for nullable FK queries" + - "is_phantom computed column pattern for phantom track detection" + - "NewTestDBWithLibrary(t, name, path) for integration tests needing libraries" + +requirements-completed: [LIB-04, LIB-05] + +# Metrics +duration: 5min +completed: 2026-03-09 +--- + +# Phase 10 Plan 2: sqlc Queries & Migration Tests Summary + +**Library CRUD queries, phantom-aware playlist queries with LEFT JOIN + COALESCE fallback, and 5 migration 6 integration tests** + +## Performance + +- **Duration:** 5 min +- **Started:** 2026-03-09T13:45:05Z +- **Completed:** 2026-03-09T13:50:34Z +- **Tasks:** 2 +- **Files modified:** 9 + +## Accomplishments +- Created 7 library CRUD queries (create, get, get-by-path, list, update, delete, count) with sqlc-generated Go code +- Updated all playlist track queries to use LEFT JOIN for nullable audio_file_id, with COALESCE fallback chain from live metadata to phantom metadata +- Added GetTrackPhantomMetadata helper query for eager phantom population at insert time +- Added is_phantom computed column to GetPlaylistTracksWithMetadata and GetAllPlaylistTracksWithMetadata +- Added GetAudioFilesByLibrary and CountAudioFilesByLibrary queries +- Created 5 comprehensive migration 6 integration tests covering fresh DB, CRUD, phantom tracks, FK enforcement, and VIEW validation +- Added NewTestDBWithLibrary helper for downstream test usage + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Add sqlc queries for libraries and update playlist queries** - `02548dd` (feat) +2. **Task 2: Migration integration tests and NewTestDB update** - `bc15189` (feat) + +## Files Created/Modified +- `backend/database/sql/queries/libraries.sql` - 7 CRUD queries for libraries table +- `backend/database/sql/queries/playlists.sql` - Updated with phantom support, LEFT JOINs, GetTrackPhantomMetadata +- `backend/database/sql/queries/audio_files.sql` - Added GetAudioFilesByLibrary, CountAudioFilesByLibrary +- `backend/database/sql/sqlcgen/libraries.sql.go` - Generated Go code for library queries +- `backend/database/sql/sqlcgen/playlists.sql.go` - Regenerated with phantom columns, is_phantom, LEFT JOINs +- `backend/database/sql/sqlcgen/audio_files.sql.go` - Regenerated with library filter queries +- `backend/database/database_test.go` - 5 migration 6 integration tests +- `backend/database/testhelper.go` - Added NewTestDBWithLibrary helper + +## Decisions Made +- COALESCE fallback chain: live data → phantom data → empty string ensures callers always get usable values regardless of whether a track is phantom or not +- Added `is_phantom` as a computed column (`CASE WHEN pt.audio_file_id IS NULL THEN 1 ELSE 0 END`) to eliminate null-checking logic in callers +- GetPlaylistTrackFilePaths now filters `WHERE audio_file_id IS NOT NULL` to exclude phantom tracks from file path lists + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Fixed NewTestDBWithLibrary path collision with sentinel library** +- **Found during:** Task 2 (migration tests) +- **Issue:** Tests using `NewTestDBWithLibrary(t, "Test", "/test")` collided with the sentinel library at `(0, 'Test', '/test')` from NewTestDB, causing UNIQUE constraint violation +- **Fix:** Changed test paths to unique values (`/test/music`, `/test/fk-lib`, `/test/view-lib`) to avoid collision with sentinel +- **Files modified:** backend/database/database_test.go +- **Verification:** All 5 TestMigration6 tests pass +- **Committed in:** bc15189 (Task 2 commit) + +--- + +**Total deviations:** 1 auto-fixed (1 bug) +**Impact on plan:** Minor path collision fix in tests. No scope creep. + +## Issues Encountered +None + +## User Setup Required +None - no external service configuration required. + +## Next Phase Readiness +- Phase 10 complete: schema files, migration 6, sqlc queries, and migration tests all in place +- Ready for Phase 11 (per-library scan pipeline) — libraries table and library_id queries available +- Ready for Phase 12 (library CRUD API) — all 7 library queries generated and tested +- Ready for Phase 13 (library views & phantom tracks) — phantom metadata queries with is_phantom column available + +--- +*Phase: 10-schema-migration* +*Completed: 2026-03-09*