yonlu/yellowjacket
1
1
Fork 0
Code Issues 55 Pull Requests 6 Actions Packages Projects Releases 12 Wiki Activity

docs(10-01): complete schema & migration plan

Browse Source
This commit is contained in:
yonlu
2026-03-09 09:44:05 -04:00
parent 1179f56c36
commit 2c77368482
4 changed files with 184 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/REQUIREMENTS.md
+6 -6
View File
@@ -36,7 +36,7 @@ Requirements for v1.1 Multi-Library Support milestone. Each maps to roadmap phas
- [ ] **LSCAN-02**: Scanning is sequential — only one library scans at a time (SQLite single-writer)
- [ ] **LSCAN-03**: Scan progress UI shows which library is being scanned
- [ ] **LSCAN-04**: Existing scan cancellation and pause/resume work per-library
- [ ] **LSCAN-05**: Audio files are associated with their library via `library_id` foreign key
- [x] **LSCAN-05**: Audio files are associated with their library via `library_id` foreign key
### Unified Presentation
@@ -54,10 +54,10 @@ Requirements for v1.1 Multi-Library Support milestone. Each maps to roadmap phas
### Data Integrity
- [ ] **DATA-01**: Schema migration adds `libraries` table and `library_id` FK on `audio_files`
- [x] **DATA-01**: Schema migration adds `libraries` table and `library_id` FK on `audio_files`
- [ ] **DATA-02**: Orphan cleanup after library removal: reference-counting bottom-up deletes for artists, albums, genres only referenced by removed library's tracks
- [ ] **DATA-03**: FTS5 index entries for removed tracks are cleaned up (handling contentless table limitations)
- [ ] **DATA-04**: All library operations are transactional — no partial state on failure
- [x] **DATA-04**: All library operations are transactional — no partial state on failure
## Future Requirements
@@ -190,7 +190,7 @@ Which phases cover which requirements. Updated during roadmap creation.
| LSCAN-02 | Phase 11 | Pending |
| LSCAN-03 | Phase 11 | Pending |
| LSCAN-04 | Phase 11 | Pending |
| LSCAN-05 | Phase 10 | Pending |
| LSCAN-05 | Phase 10 | Complete |
| VIEW-01 | Phase 13 | Pending |
| VIEW-02 | Phase 13 | Pending |
| VIEW-03 | Phase 13 | Pending |
@@ -199,10 +199,10 @@ Which phases cover which requirements. Updated during roadmap creation.
| PLAY-02 | Phase 13 | Pending |
| PLAY-03 | Phase 13 | Pending |
| PLAY-04 | Phase 12 | Pending |
| DATA-01 | Phase 10 | Pending |
| DATA-01 | Phase 10 | Complete |
| DATA-02 | Phase 12 | Pending |
| DATA-03 | Phase 12 | Pending |
| DATA-04 | Phase 10 | Pending |
| DATA-04 | Phase 10 | Complete |
**Coverage:**
- v1.1 requirements: 31 total (8 complete + 23 pending)
.planning/ROADMAP.md
+2 -2
View File
@@ -62,7 +62,7 @@ Plans:
2. An existing user's database is migrated on first launch: their single directory becomes a named library, all existing audio_files get that library_id, and everything works without any user action
3. The `playlist_tracks` table supports nullable `audio_file_id` with phantom metadata columns — the schema is ready for phantom track preservation
4. All migration operations complete atomically — a crash mid-migration leaves the database unchanged (not half-migrated)
**Plans:** 2 plans
**Plans:** 1/2 plans executed
Plans:
- [ ] 10-01-PLAN.md — Schema definitions + Migration 6 (libraries table, library_id FK, phantom columns, track_metadata VIEW, backup, TOML migration)
- [ ] 10-02-PLAN.md — sqlc queries for libraries + updated playlist phantom queries + migration integration tests
@@ -115,7 +115,7 @@ Plans:
| 7. Backend Performance | v1.0 | 2/2 | Complete | 2026-03-05 |
| 8. Frontend Performance & UX | v1.0 | 4/4 | Complete | 2026-03-05 |
| 9. Scan Cancellation & Keyboard Shortcuts | v1.1 | 5/5 | Complete | 2026-03-07 |
| 10. Schema & Migration | v1.1 | 0/2 | Planned | - |
| 10. Schema & Migration | 1/2 | In Progress| | - |
| 11. Per-Library Scan Pipeline | v1.1 | 0/? | Not started | - |
| 12. Library CRUD & Data Integrity | v1.1 | 0/? | Not started | - |
| 13. Library Views & Phantom Tracks | v1.1 | 0/? | Not started | - |
.planning/STATE.md
+25 -15
View File
@@ -2,13 +2,13 @@
gsd_state_version: 1.0
milestone: v1.1
milestone_name: Multi-Library Support
status: planning
last_updated: "2026-03-08"
status: unknown
last_updated: "2026-03-09T13:43:03.790Z"
progress:
total_phases: 5
total_phases: 2
completed_phases: 1
total_plans: 5
completed_plans: 5
total_plans: 7
completed_plans: 6
---
# YellowJacket — Project State
@@ -22,18 +22,18 @@ See: .planning/PROJECT.md (updated 2026-03-08)
## Current Position
Phase: 10 — Schema & Migration (next up)
Plan: TBD (awaiting plan creation)
Status: Roadmap created — ready for phase planning
Phase: 10 — Schema & Migration
Plan: 1 of 2 in Phase (Plan 01 complete)
Status: In progress — executing Phase 10
Progress: ████░░░░░░░░░░░░░░░░ 1/5 phases complete (Phase 9)
Last activity: 2026-03-08 — Multi-library roadmap created (Phases 10-13)
Last activity: 2026-03-09 — Completed 10-01 schema files + migration 6
### Phase Overview
| Phase | Status |
|-------|--------|
| 9. Scan Cancellation & Keyboard Shortcuts | Complete (5/5 plans) ✅ |
| 10. Schema & Migration | Not started |
| 10. Schema & Migration | In progress (1/2 plans) |
| 11. Per-Library Scan Pipeline | Not started |
| 12. Library CRUD & Data Integrity | Not started |
| 13. Library Views & Phantom Tracks | Not started |
@@ -50,6 +50,7 @@ Last activity: 2026-03-08 — Multi-library roadmap created (Phases 10-13)
| 09-04 | keyboard shortcuts settings UI | 5 min | 2 | 2 |
| 09-03 | scan control UI | 2 min | 1 | 3 |
| 09-05 | integration testing & verification | 3 min | 2 | 1 |
| Phase 10-01 P01 | 11 min | 2 tasks | 10 files |
## Accumulated Context
@@ -78,6 +79,15 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns
| Backend filtering, not frontend | Don't load 150K tracks when viewing one library |
| 4 multi-library phases (10-13) | Natural delivery boundaries: schema → scan → CRUD → views, each phase delivers verifiable capability |
### Phase 10 Decisions
| Decision | Rationale |
|----------|-----------|
| Underscore prefix `_libraries.sql` for schema ordering | Go embed.FS ReadDir sorts alphabetically; `_` < `a` ensures libraries table created before audio_files FK |
| Sentinel library id=0 in test DB | Existing tests use DEFAULT library_id=0; sentinel row satisfies FK without modifying every test |
| TOML cleanup via generic map[string]any | Preserves all config sections when removing only DirectoryPath; no dependency on full Config struct |
| sql.NullInt64 for nullable playlist_tracks.audio_file_id | Phantom tracks have NULL audio_file_id; generated sqlc code requires sql.Null types |
### Warnings (carry forward)
- Player lock ordering (`p.mu` before `speaker.Lock()`, goroutine dispatch in beep callback) — carry forward
@@ -99,10 +109,10 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns
### Last Session
**Date:** 2026-03-08
**What happened:** Created multi-library roadmap (Phases 10-13) from 20 requirements. Phase 10 = Schema & Migration (DATA-01, DATA-04, LIB-04, LIB-05, LSCAN-05). Phase 11 = Per-Library Scan Pipeline (LSCAN-01..04). Phase 12 = Library CRUD & Data Integrity (LIB-01..03, LIB-06, DATA-02, DATA-03, PLAY-04). Phase 13 = Library Views & Phantom Tracks (VIEW-01..04, PLAY-01..03).
**Where we stopped:** Roadmap written, ready for phase planning
**Next action:** Run `gsd-plan-phase 10` to create plans for Schema & Migration
**Date:** 2026-03-09
**What happened:** Executed Phase 10, Plan 01 — created multi-library schema files (_libraries.sql, updated audio_files/playlist_tracks/track_metadata_view) and implemented migration 6 with backup, TOML config migration, phantom metadata backfill. 3 auto-fixed deviations (sqlc regen, test FK fix, linter fix).
**Where we stopped:** Completed 10-01-PLAN.md (schema files + migration 6)
**Next action:** Execute Phase 10, Plan 02 or continue to Phase 11
---
*State initialized: 2026-02-27*
@@ -115,4 +125,4 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns
| 18 | add multi-column metadata display to playlist-details | 2026-03-08 | ce23177 | [18-add-multi-column-metadata-display-to-pla](./quick/18-add-multi-column-metadata-display-to-pla/) |
Last activity: 2026-03-08 - Completed quick task 18: add multi-column metadata display to playlist-details
*Last updated: 2026-03-08 — Multi-library roadmap created (Phases 10-13)*
*Last updated: 2026-03-09 — Completed 10-01-PLAN.md (schema + migration 6)*
.planning/phases/10-schema-migration/10-01-SUMMARY.md
+151
View File
@@ -0,0 +1,151 @@
---
phase: 10-schema-migration
plan: 01
subsystem: database
tags: [sqlite, migration, multi-library, phantom-tracks, schema]
# Dependency graph
requires: []
provides:
- libraries table (name, path, created_at)
- audio_files.library_id FK column with index
- playlist_tracks phantom metadata columns (6 fields)
- playlist_tracks SET NULL FK (was CASCADE)
- track_metadata VIEW with library_id
- migration 6 function (multi-library upgrade)
- pre-migration backup function
- TOML config cleanup (DirectoryPath removal)
affects: [11-per-library-scan, 12-library-crud, 13-library-views]
# Tech tracking
tech-stack:
added: []
patterns:
- "Underscore prefix for schema file ordering (_libraries.sql sorts before audio_files.sql)"
- "Sentinel library row (id=0) in test DB for FK satisfaction"
- "Dynamic DEFAULT in ALTER TABLE ADD COLUMN for backfill"
- "TOML read/write with generic map[string]any to preserve unknown sections"
key-files:
created:
- backend/database/sql/schemas/_libraries.sql
modified:
- backend/database/database.go
- backend/database/sql/schemas/audio_files.sql
- backend/database/sql/schemas/playlist_tracks.sql
- backend/database/sql/schemas/track_metadata_view.sql
- backend/database/sql/sqlcgen/audio_files.sql.go
- backend/database/sql/sqlcgen/models.go
- backend/database/sql/sqlcgen/playlists.sql.go
- backend/database/testhelper.go
- backend/playlist/playlist.go
key-decisions:
- "Underscore prefix _libraries.sql for embedded FS sort order (libraries table must exist before audio_files FK)"
- "Sentinel library id=0 in NewTestDB so existing tests using DEFAULT library_id=0 continue working"
- "TOML cleanup uses generic map[string]any to preserve all config sections, only deletes DirectoryPath"
- "Backup skipped for in-memory databases (test environments)"
patterns-established:
- "_libraries.sql naming convention for schema ordering"
- "sql.NullInt64 for nullable FK columns in playlist_tracks"
requirements-completed: [DATA-01, DATA-04, LSCAN-05]
# Metrics
duration: 11min
completed: 2026-03-09
---
# Phase 10 Plan 1: Schema & Migration Summary
**Libraries table, audio_files.library_id FK, playlist_tracks phantom columns with SET NULL FK, migration 6 with pre-backup and TOML config cleanup**
## Performance
- **Duration:** 11 min
- **Started:** 2026-03-09T13:29:50Z
- **Completed:** 2026-03-09T13:41:26Z
- **Tasks:** 2
- **Files modified:** 10
## Accomplishments
- Created libraries table schema with name, path, created_at columns
- Added library_id FK to audio_files with index for filter performance
- Rebuilt playlist_tracks with nullable audio_file_id (SET NULL FK) and 6 phantom metadata columns
- Implemented migration 6 with 14-step process: backup, TOML read, FK OFF, create table, insert default library, add column, rebuild playlist_tracks, backfill phantom metadata, recreate VIEW, FK ON, TOML cleanup, version bump
- Updated track_metadata VIEW to include library_id
- Regenerated sqlc code and fixed all callers for nullable AudioFileID
## Task Commits
Each task was committed atomically:
1. **Task 1: Update SQL schema files for fresh installs** - `535855b` (feat)
2. **Task 2: Implement migration 6 and pre-migration backup** - `1179f56` (feat)
## Files Created/Modified
- `backend/database/sql/schemas/_libraries.sql` - New libraries table DDL
- `backend/database/sql/schemas/audio_files.sql` - Added library_id column and FK
- `backend/database/sql/schemas/playlist_tracks.sql` - Nullable audio_file_id, SET NULL FK, 6 phantom columns
- `backend/database/sql/schemas/track_metadata_view.sql` - Added af.library_id to SELECT
- `backend/database/database.go` - migration6MultiLibrary(), backupDatabase(), TOML helpers
- `backend/database/sql/sqlcgen/models.go` - Library struct, updated AudioFile and PlaylistTrack
- `backend/database/sql/sqlcgen/audio_files.sql.go` - Updated queries for library_id column
- `backend/database/sql/sqlcgen/playlists.sql.go` - sql.NullInt64 for AudioFileID, phantom fields
- `backend/database/testhelper.go` - Sentinel library row, updated runMigrations call
- `backend/playlist/playlist.go` - sql.NullInt64 wrapping for AddPlaylistTrack calls
## Decisions Made
- Used underscore prefix `_libraries.sql` to ensure correct embedded FS sort order (libraries must exist before audio_files FK reference)
- Sentinel library row at id=0 in NewTestDB for backward compatibility with existing test data using DEFAULT library_id=0
- TOML config cleanup uses generic `map[string]any` decode to preserve all config sections when removing only DirectoryPath
- Backup function skips for in-memory databases (`:memory:` path check) to support test environments
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] Regenerated sqlc code and fixed compilation errors**
- **Found during:** Task 1 (SQL schema updates)
- **Issue:** Pre-commit hook auto-ran `sqlc generate` which updated generated code — AudioFileID changed from `int64` to `sql.NullInt64`, breaking 4 call sites in playlist.go
- **Fix:** Added `database/sql` import to playlist.go and wrapped all AudioFileID assignments with `sql.NullInt64{Int64: id, Valid: true}`
- **Files modified:** backend/database/sql/sqlcgen/{models,audio_files.sql,playlists.sql}.go, backend/playlist/playlist.go
- **Verification:** `go build ./...` passes
- **Committed in:** 535855b (Task 1 commit)
**2. [Rule 3 - Blocking] Fixed test FK constraint failures**
- **Found during:** Task 2 (migration implementation)
- **Issue:** Existing tests insert audio_files with DEFAULT library_id=0 but no library with id=0 exists after schema changes — FK constraint violated
- **Fix:** Added sentinel library row (id=0, name='Test', path='/test') in NewTestDB() so all tests have a valid FK target
- **Files modified:** backend/database/testhelper.go
- **Verification:** `go test ./backend/database/... -count=1` passes (all 10+ test functions)
- **Committed in:** 1179f56 (Task 2 commit)
**3. [Rule 1 - Bug] Fixed unchecked error returns on file Close()**
- **Found during:** Task 2 (linter pre-commit check)
- **Issue:** `src.Close()` and `dst.Close()` in backupDatabase() had unchecked error returns, caught by errcheck linter
- **Fix:** Changed to `defer func() { _ = src.Close() }()` pattern (explicit discard)
- **Files modified:** backend/database/database.go
- **Verification:** `golangci-lint` passes with 0 issues
- **Committed in:** 1179f56 (Task 2 commit)
---
**Total deviations:** 3 auto-fixed (2 blocking, 1 bug)
**Impact on plan:** All fixes necessary for correctness and build health. No scope creep — sqlc regeneration and test fixes are direct consequences of the schema changes.
## Issues Encountered
None — migration 6 follows established patterns from migration 5.
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Schema foundation complete for multi-library support
- Ready for Plan 02 (sqlc query updates, if applicable) or Phase 11 (per-library scan pipeline)
- All existing tests pass with new schema
---
*Phase: 10-schema-migration*
*Completed: 2026-03-09*