diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 76bc07d..9759df1 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -36,7 +36,7 @@ Requirements for v1.1 Multi-Library Support milestone. Each maps to roadmap phas - [ ] **LSCAN-02**: Scanning is sequential — only one library scans at a time (SQLite single-writer) - [ ] **LSCAN-03**: Scan progress UI shows which library is being scanned - [ ] **LSCAN-04**: Existing scan cancellation and pause/resume work per-library -- [ ] **LSCAN-05**: Audio files are associated with their library via `library_id` foreign key +- [x] **LSCAN-05**: Audio files are associated with their library via `library_id` foreign key ### Unified Presentation @@ -54,10 +54,10 @@ Requirements for v1.1 Multi-Library Support milestone. Each maps to roadmap phas ### Data Integrity -- [ ] **DATA-01**: Schema migration adds `libraries` table and `library_id` FK on `audio_files` +- [x] **DATA-01**: Schema migration adds `libraries` table and `library_id` FK on `audio_files` - [ ] **DATA-02**: Orphan cleanup after library removal: reference-counting bottom-up deletes for artists, albums, genres only referenced by removed library's tracks - [ ] **DATA-03**: FTS5 index entries for removed tracks are cleaned up (handling contentless table limitations) -- [ ] **DATA-04**: All library operations are transactional — no partial state on failure +- [x] **DATA-04**: All library operations are transactional — no partial state on failure ## Future Requirements @@ -190,7 +190,7 @@ Which phases cover which requirements. Updated during roadmap creation. | LSCAN-02 | Phase 11 | Pending | | LSCAN-03 | Phase 11 | Pending | | LSCAN-04 | Phase 11 | Pending | -| LSCAN-05 | Phase 10 | Pending | +| LSCAN-05 | Phase 10 | Complete | | VIEW-01 | Phase 13 | Pending | | VIEW-02 | Phase 13 | Pending | | VIEW-03 | Phase 13 | Pending | @@ -199,10 +199,10 @@ Which phases cover which requirements. Updated during roadmap creation. | PLAY-02 | Phase 13 | Pending | | PLAY-03 | Phase 13 | Pending | | PLAY-04 | Phase 12 | Pending | -| DATA-01 | Phase 10 | Pending | +| DATA-01 | Phase 10 | Complete | | DATA-02 | Phase 12 | Pending | | DATA-03 | Phase 12 | Pending | -| DATA-04 | Phase 10 | Pending | +| DATA-04 | Phase 10 | Complete | **Coverage:** - v1.1 requirements: 31 total (8 complete + 23 pending) diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 77a4383..8fd516c 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -62,7 +62,7 @@ Plans: 2. An existing user's database is migrated on first launch: their single directory becomes a named library, all existing audio_files get that library_id, and everything works without any user action 3. The `playlist_tracks` table supports nullable `audio_file_id` with phantom metadata columns — the schema is ready for phantom track preservation 4. All migration operations complete atomically — a crash mid-migration leaves the database unchanged (not half-migrated) -**Plans:** 2 plans +**Plans:** 1/2 plans executed Plans: - [ ] 10-01-PLAN.md — Schema definitions + Migration 6 (libraries table, library_id FK, phantom columns, track_metadata VIEW, backup, TOML migration) - [ ] 10-02-PLAN.md — sqlc queries for libraries + updated playlist phantom queries + migration integration tests @@ -115,7 +115,7 @@ Plans: | 7. Backend Performance | v1.0 | 2/2 | Complete | 2026-03-05 | | 8. Frontend Performance & UX | v1.0 | 4/4 | Complete | 2026-03-05 | | 9. Scan Cancellation & Keyboard Shortcuts | v1.1 | 5/5 | Complete | 2026-03-07 | -| 10. Schema & Migration | v1.1 | 0/2 | Planned | - | +| 10. Schema & Migration | 1/2 | In Progress| | - | | 11. Per-Library Scan Pipeline | v1.1 | 0/? | Not started | - | | 12. Library CRUD & Data Integrity | v1.1 | 0/? | Not started | - | | 13. Library Views & Phantom Tracks | v1.1 | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index e2d47a5..a3fbfd0 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,13 +2,13 @@ gsd_state_version: 1.0 milestone: v1.1 milestone_name: Multi-Library Support -status: planning -last_updated: "2026-03-08" +status: unknown +last_updated: "2026-03-09T13:43:03.790Z" progress: - total_phases: 5 + total_phases: 2 completed_phases: 1 - total_plans: 5 - completed_plans: 5 + total_plans: 7 + completed_plans: 6 --- # YellowJacket — Project State @@ -22,18 +22,18 @@ See: .planning/PROJECT.md (updated 2026-03-08) ## Current Position -Phase: 10 — Schema & Migration (next up) -Plan: TBD (awaiting plan creation) -Status: Roadmap created — ready for phase planning +Phase: 10 — Schema & Migration +Plan: 1 of 2 in Phase (Plan 01 complete) +Status: In progress — executing Phase 10 Progress: ████░░░░░░░░░░░░░░░░ 1/5 phases complete (Phase 9) -Last activity: 2026-03-08 — Multi-library roadmap created (Phases 10-13) +Last activity: 2026-03-09 — Completed 10-01 schema files + migration 6 ### Phase Overview | Phase | Status | |-------|--------| | 9. Scan Cancellation & Keyboard Shortcuts | Complete (5/5 plans) ✅ | -| 10. Schema & Migration | Not started | +| 10. Schema & Migration | In progress (1/2 plans) | | 11. Per-Library Scan Pipeline | Not started | | 12. Library CRUD & Data Integrity | Not started | | 13. Library Views & Phantom Tracks | Not started | @@ -50,6 +50,7 @@ Last activity: 2026-03-08 — Multi-library roadmap created (Phases 10-13) | 09-04 | keyboard shortcuts settings UI | 5 min | 2 | 2 | | 09-03 | scan control UI | 2 min | 1 | 3 | | 09-05 | integration testing & verification | 3 min | 2 | 1 | +| Phase 10-01 P01 | 11 min | 2 tasks | 10 files | ## Accumulated Context @@ -78,6 +79,15 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns | Backend filtering, not frontend | Don't load 150K tracks when viewing one library | | 4 multi-library phases (10-13) | Natural delivery boundaries: schema → scan → CRUD → views, each phase delivers verifiable capability | +### Phase 10 Decisions + +| Decision | Rationale | +|----------|-----------| +| Underscore prefix `_libraries.sql` for schema ordering | Go embed.FS ReadDir sorts alphabetically; `_` < `a` ensures libraries table created before audio_files FK | +| Sentinel library id=0 in test DB | Existing tests use DEFAULT library_id=0; sentinel row satisfies FK without modifying every test | +| TOML cleanup via generic map[string]any | Preserves all config sections when removing only DirectoryPath; no dependency on full Config struct | +| sql.NullInt64 for nullable playlist_tracks.audio_file_id | Phantom tracks have NULL audio_file_id; generated sqlc code requires sql.Null types | + ### Warnings (carry forward) - Player lock ordering (`p.mu` before `speaker.Lock()`, goroutine dispatch in beep callback) — carry forward @@ -99,10 +109,10 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns ### Last Session -**Date:** 2026-03-08 -**What happened:** Created multi-library roadmap (Phases 10-13) from 20 requirements. Phase 10 = Schema & Migration (DATA-01, DATA-04, LIB-04, LIB-05, LSCAN-05). Phase 11 = Per-Library Scan Pipeline (LSCAN-01..04). Phase 12 = Library CRUD & Data Integrity (LIB-01..03, LIB-06, DATA-02, DATA-03, PLAY-04). Phase 13 = Library Views & Phantom Tracks (VIEW-01..04, PLAY-01..03). -**Where we stopped:** Roadmap written, ready for phase planning -**Next action:** Run `gsd-plan-phase 10` to create plans for Schema & Migration +**Date:** 2026-03-09 +**What happened:** Executed Phase 10, Plan 01 — created multi-library schema files (_libraries.sql, updated audio_files/playlist_tracks/track_metadata_view) and implemented migration 6 with backup, TOML config migration, phantom metadata backfill. 3 auto-fixed deviations (sqlc regen, test FK fix, linter fix). +**Where we stopped:** Completed 10-01-PLAN.md (schema files + migration 6) +**Next action:** Execute Phase 10, Plan 02 or continue to Phase 11 --- *State initialized: 2026-02-27* @@ -115,4 +125,4 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns | 18 | add multi-column metadata display to playlist-details | 2026-03-08 | ce23177 | [18-add-multi-column-metadata-display-to-pla](./quick/18-add-multi-column-metadata-display-to-pla/) | Last activity: 2026-03-08 - Completed quick task 18: add multi-column metadata display to playlist-details -*Last updated: 2026-03-08 — Multi-library roadmap created (Phases 10-13)* +*Last updated: 2026-03-09 — Completed 10-01-PLAN.md (schema + migration 6)* diff --git a/.planning/phases/10-schema-migration/10-01-SUMMARY.md b/.planning/phases/10-schema-migration/10-01-SUMMARY.md new file mode 100644 index 0000000..c9343a6 --- /dev/null +++ b/.planning/phases/10-schema-migration/10-01-SUMMARY.md @@ -0,0 +1,151 @@ +--- +phase: 10-schema-migration +plan: 01 +subsystem: database +tags: [sqlite, migration, multi-library, phantom-tracks, schema] + +# Dependency graph +requires: [] +provides: + - libraries table (name, path, created_at) + - audio_files.library_id FK column with index + - playlist_tracks phantom metadata columns (6 fields) + - playlist_tracks SET NULL FK (was CASCADE) + - track_metadata VIEW with library_id + - migration 6 function (multi-library upgrade) + - pre-migration backup function + - TOML config cleanup (DirectoryPath removal) +affects: [11-per-library-scan, 12-library-crud, 13-library-views] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Underscore prefix for schema file ordering (_libraries.sql sorts before audio_files.sql)" + - "Sentinel library row (id=0) in test DB for FK satisfaction" + - "Dynamic DEFAULT in ALTER TABLE ADD COLUMN for backfill" + - "TOML read/write with generic map[string]any to preserve unknown sections" + +key-files: + created: + - backend/database/sql/schemas/_libraries.sql + modified: + - backend/database/database.go + - backend/database/sql/schemas/audio_files.sql + - backend/database/sql/schemas/playlist_tracks.sql + - backend/database/sql/schemas/track_metadata_view.sql + - backend/database/sql/sqlcgen/audio_files.sql.go + - backend/database/sql/sqlcgen/models.go + - backend/database/sql/sqlcgen/playlists.sql.go + - backend/database/testhelper.go + - backend/playlist/playlist.go + +key-decisions: + - "Underscore prefix _libraries.sql for embedded FS sort order (libraries table must exist before audio_files FK)" + - "Sentinel library id=0 in NewTestDB so existing tests using DEFAULT library_id=0 continue working" + - "TOML cleanup uses generic map[string]any to preserve all config sections, only deletes DirectoryPath" + - "Backup skipped for in-memory databases (test environments)" + +patterns-established: + - "_libraries.sql naming convention for schema ordering" + - "sql.NullInt64 for nullable FK columns in playlist_tracks" + +requirements-completed: [DATA-01, DATA-04, LSCAN-05] + +# Metrics +duration: 11min +completed: 2026-03-09 +--- + +# Phase 10 Plan 1: Schema & Migration Summary + +**Libraries table, audio_files.library_id FK, playlist_tracks phantom columns with SET NULL FK, migration 6 with pre-backup and TOML config cleanup** + +## Performance + +- **Duration:** 11 min +- **Started:** 2026-03-09T13:29:50Z +- **Completed:** 2026-03-09T13:41:26Z +- **Tasks:** 2 +- **Files modified:** 10 + +## Accomplishments +- Created libraries table schema with name, path, created_at columns +- Added library_id FK to audio_files with index for filter performance +- Rebuilt playlist_tracks with nullable audio_file_id (SET NULL FK) and 6 phantom metadata columns +- Implemented migration 6 with 14-step process: backup, TOML read, FK OFF, create table, insert default library, add column, rebuild playlist_tracks, backfill phantom metadata, recreate VIEW, FK ON, TOML cleanup, version bump +- Updated track_metadata VIEW to include library_id +- Regenerated sqlc code and fixed all callers for nullable AudioFileID + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Update SQL schema files for fresh installs** - `535855b` (feat) +2. **Task 2: Implement migration 6 and pre-migration backup** - `1179f56` (feat) + +## Files Created/Modified +- `backend/database/sql/schemas/_libraries.sql` - New libraries table DDL +- `backend/database/sql/schemas/audio_files.sql` - Added library_id column and FK +- `backend/database/sql/schemas/playlist_tracks.sql` - Nullable audio_file_id, SET NULL FK, 6 phantom columns +- `backend/database/sql/schemas/track_metadata_view.sql` - Added af.library_id to SELECT +- `backend/database/database.go` - migration6MultiLibrary(), backupDatabase(), TOML helpers +- `backend/database/sql/sqlcgen/models.go` - Library struct, updated AudioFile and PlaylistTrack +- `backend/database/sql/sqlcgen/audio_files.sql.go` - Updated queries for library_id column +- `backend/database/sql/sqlcgen/playlists.sql.go` - sql.NullInt64 for AudioFileID, phantom fields +- `backend/database/testhelper.go` - Sentinel library row, updated runMigrations call +- `backend/playlist/playlist.go` - sql.NullInt64 wrapping for AddPlaylistTrack calls + +## Decisions Made +- Used underscore prefix `_libraries.sql` to ensure correct embedded FS sort order (libraries must exist before audio_files FK reference) +- Sentinel library row at id=0 in NewTestDB for backward compatibility with existing test data using DEFAULT library_id=0 +- TOML config cleanup uses generic `map[string]any` decode to preserve all config sections when removing only DirectoryPath +- Backup function skips for in-memory databases (`:memory:` path check) to support test environments + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] Regenerated sqlc code and fixed compilation errors** +- **Found during:** Task 1 (SQL schema updates) +- **Issue:** Pre-commit hook auto-ran `sqlc generate` which updated generated code — AudioFileID changed from `int64` to `sql.NullInt64`, breaking 4 call sites in playlist.go +- **Fix:** Added `database/sql` import to playlist.go and wrapped all AudioFileID assignments with `sql.NullInt64{Int64: id, Valid: true}` +- **Files modified:** backend/database/sql/sqlcgen/{models,audio_files.sql,playlists.sql}.go, backend/playlist/playlist.go +- **Verification:** `go build ./...` passes +- **Committed in:** 535855b (Task 1 commit) + +**2. [Rule 3 - Blocking] Fixed test FK constraint failures** +- **Found during:** Task 2 (migration implementation) +- **Issue:** Existing tests insert audio_files with DEFAULT library_id=0 but no library with id=0 exists after schema changes — FK constraint violated +- **Fix:** Added sentinel library row (id=0, name='Test', path='/test') in NewTestDB() so all tests have a valid FK target +- **Files modified:** backend/database/testhelper.go +- **Verification:** `go test ./backend/database/... -count=1` passes (all 10+ test functions) +- **Committed in:** 1179f56 (Task 2 commit) + +**3. [Rule 1 - Bug] Fixed unchecked error returns on file Close()** +- **Found during:** Task 2 (linter pre-commit check) +- **Issue:** `src.Close()` and `dst.Close()` in backupDatabase() had unchecked error returns, caught by errcheck linter +- **Fix:** Changed to `defer func() { _ = src.Close() }()` pattern (explicit discard) +- **Files modified:** backend/database/database.go +- **Verification:** `golangci-lint` passes with 0 issues +- **Committed in:** 1179f56 (Task 2 commit) + +--- + +**Total deviations:** 3 auto-fixed (2 blocking, 1 bug) +**Impact on plan:** All fixes necessary for correctness and build health. No scope creep — sqlc regeneration and test fixes are direct consequences of the schema changes. + +## Issues Encountered +None — migration 6 follows established patterns from migration 5. + +## User Setup Required +None - no external service configuration required. + +## Next Phase Readiness +- Schema foundation complete for multi-library support +- Ready for Plan 02 (sqlc query updates, if applicable) or Phase 11 (per-library scan pipeline) +- All existing tests pass with new schema + +--- +*Phase: 10-schema-migration* +*Completed: 2026-03-09*