yonlu/yellowjacket
1
1
Fork 0
Code Issues 55 Pull Requests 6 Actions Packages Projects Releases 12 Wiki Activity

docs(11-01): complete per-library scan pipeline plan

Browse Source 
- Create 11-01-SUMMARY.md with execution results
- Update STATE.md: advance to phase 11 plan 1/3
- Update ROADMAP.md: phase 11 at 1/3 plans complete
- Mark LSCAN-01, LSCAN-02, LSCAN-04 complete in REQUIREMENTS.md
This commit is contained in:
yonlu
2026-03-09 16:05:55 -04:00
parent 943db1cf27
commit 9308848936
4 changed files with 159 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/REQUIREMENTS.md
+6 -6
View File
@@ -32,10 +32,10 @@ Requirements for v1.1 Multi-Library Support milestone. Each maps to roadmap phas
### Library Scanning
- [ ] **LSCAN-01**: User can trigger a scan for a specific library (not all-or-nothing)
- [ ] **LSCAN-02**: Scanning is sequential — only one library scans at a time (SQLite single-writer)
- [x] **LSCAN-01**: User can trigger a scan for a specific library (not all-or-nothing)
- [x] **LSCAN-02**: Scanning is sequential — only one library scans at a time (SQLite single-writer)
- [ ] **LSCAN-03**: Scan progress UI shows which library is being scanned
- [ ] **LSCAN-04**: Existing scan cancellation and pause/resume work per-library
- [x] **LSCAN-04**: Existing scan cancellation and pause/resume work per-library
- [x] **LSCAN-05**: Audio files are associated with their library via `library_id` foreign key
### Unified Presentation
@@ -186,10 +186,10 @@ Which phases cover which requirements. Updated during roadmap creation.
| LIB-04 | Phase 10 | Complete |
| LIB-05 | Phase 10 | Complete |
| LIB-06 | Phase 12 | Pending |
| LSCAN-01 | Phase 11 | Pending |
| LSCAN-02 | Phase 11 | Pending |
| LSCAN-01 | Phase 11 | Complete |
| LSCAN-02 | Phase 11 | Complete |
| LSCAN-03 | Phase 11 | Pending |
| LSCAN-04 | Phase 11 | Pending |
| LSCAN-04 | Phase 11 | Complete |
| LSCAN-05 | Phase 10 | Complete |
| VIEW-01 | Phase 13 | Pending |
| VIEW-02 | Phase 13 | Pending |
.planning/ROADMAP.md
+2 -2
View File
@@ -76,7 +76,7 @@ Plans:
2. Only one library scans at a time — requesting a second scan while one is running either queues it or is rejected with clear feedback
3. Scan progress UI identifies which library is currently being scanned (library name visible in progress indicator)
4. Existing cancel and pause/resume controls work correctly for per-library scans — cancelling one library's scan doesn't affect others
**Plans:** 3 plans
**Plans:** 1/3 plans executed
Plans:
- [ ] 11-01-PLAN.md — Backend scan queue coordinator, per-library scan methods, CreateAudioFile with library_id
- [ ] 11-02-PLAN.md — Frontend progress UI with library name, cancel scope modal, Scan All button
@@ -120,7 +120,7 @@ Plans:
| 8. Frontend Performance & UX | v1.0 | 4/4 | Complete | 2026-03-05 |
| 9. Scan Cancellation & Keyboard Shortcuts | v1.1 | 5/5 | Complete | 2026-03-07 |
| 10. Schema & Migration | 2/2 | Complete | 2026-03-09 | - |
| 11. Per-Library Scan Pipeline | v1.1 | 0/? | Not started | - |
| 11. Per-Library Scan Pipeline | 1/3 | In Progress| | - |
| 12. Library CRUD & Data Integrity | v1.1 | 0/? | Not started | - |
| 13. Library Views & Phantom Tracks | v1.1 | 0/? | Not started | - |
.planning/STATE.md
+24 -14
View File
@@ -3,12 +3,12 @@ gsd_state_version: 1.0
milestone: v1.1
milestone_name: Multi-Library Support
status: unknown
last_updated: "2026-03-09T13:57:33.492Z"
last_updated: "2026-03-09T20:03:14Z"
progress:
total_phases: 2
completed_phases: 2
total_plans: 7
completed_plans: 7
total_plans: 8
completed_plans: 8
---
# YellowJacket — Project State
@@ -18,15 +18,15 @@ progress:
See: .planning/PROJECT.md (updated 2026-03-08)
**Core value:** The music player works reliably and feels solid — every interaction is correct, responsive, and trustworthy.
**Current focus:** v1.1 Multi-Library Support — Phase 10 (Schema & Migration)
**Current focus:** v1.1 Multi-Library Support — Phase 11 (Per-Library Scan Pipeline)
## Current Position
Phase: 10 — Schema & Migration
Plan: 2 of 2 in Phase (Plan 02 complete)
Status: Phase 10 complete — ready for Phase 11
Progress: ████████░░░░░░░░░░░░ 2/5 phases complete (Phase 10)
Last activity: 2026-03-09 — Completed 10-02 sqlc queries + migration tests
Phase: 11 — Per-Library Scan Pipeline
Plan: 1 of 3 in Phase (Plan 01 complete)
Status: Phase 11 in progress — 2 plans remaining
Progress: ████████████░░░░░░░░ 2/5 phases complete (Phase 11 in progress)
Last activity: 2026-03-09 — Completed 11-01 per-library scan pipeline
### Phase Overview
@@ -34,7 +34,7 @@ Last activity: 2026-03-09 — Completed 10-02 sqlc queries + migration tests
|-------|--------|
| 9. Scan Cancellation & Keyboard Shortcuts | Complete (5/5 plans) ✅ |
| 10. Schema & Migration | Complete (2/2 plans) ✅ |
| 11. Per-Library Scan Pipeline | Not started |
| 11. Per-Library Scan Pipeline | In Progress (1/3 plans) |
| 12. Library CRUD & Data Integrity | Not started |
| 13. Library Views & Phantom Tracks | Not started |
@@ -52,6 +52,7 @@ Last activity: 2026-03-09 — Completed 10-02 sqlc queries + migration tests
| 09-05 | integration testing & verification | 3 min | 2 | 1 |
| Phase 10-01 P01 | 11 min | 2 tasks | 10 files |
| Phase 10-02 P02 | 5 min | 2 tasks | 9 files |
| Phase 11-01 P01 | 7 min | 2 tasks | 11 files |
## Accumulated Context
@@ -91,6 +92,15 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns
| COALESCE fallback chain: live → phantom → empty | Playlist queries use 3-level COALESCE so callers always get usable string values |
| is_phantom computed column via CASE WHEN | Eliminates null-checking logic in callers; simple int64 boolean (0/1) |
### Phase 11 Decisions
| Decision | Rationale |
|----------|-----------|
| Library ID threaded via importResult | Explicit data flow through scan pipeline, avoids mutating shared Library struct state |
| scanInternal returns *ScanMetrics only | Called from goroutine in scan queue, error return impractical; errors logged + accumulated in Warnings |
| Auto worker count per library path | Each library may be on different storage (SSD/HDD), auto-detect per scan |
| Backward-compatible Scan() wrapper | Keeps handleConfigUpdate and FullRescan working without changes |
### Warnings (carry forward)
- Player lock ordering (`p.mu` before `speaker.Lock()`, goroutine dispatch in beep callback) — carry forward
@@ -113,9 +123,9 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns
### Last Session
**Date:** 2026-03-09
**What happened:** Executed Phase 10, Plan 02 — added 7 library CRUD queries, updated all playlist queries for phantom support (LEFT JOINs, COALESCE fallback, is_phantom column), added library-filtered audio file queries, wrote 5 migration 6 integration tests, added NewTestDBWithLibrary helper.
**Where we stopped:** Completed 10-02-PLAN.md (Phase 10 complete)
**Next action:** Plan or execute Phase 11 (Per-Library Scan Pipeline)
**What happened:** Executed Phase 11, Plan 01 — created scan queue coordinator (scan_queue.go), refactored Scan() to scanInternal() with per-library parameters, added ScanLibrary(id)/ScanAllLibraries() Wails bindings, FIFO queue with silent dedup, queue-aware cancel/pause, library_id in CreateAudioFile, library identification in all events.
**Where we stopped:** Completed 11-01-PLAN.md
**Next action:** Execute Phase 11, Plan 02
---
*State initialized: 2026-02-27*
@@ -128,4 +138,4 @@ Decisions from v1.0 are archived in PROJECT.md Key Decisions table. Key patterns
| 18 | add multi-column metadata display to playlist-details | 2026-03-08 | ce23177 | [18-add-multi-column-metadata-display-to-pla](./quick/18-add-multi-column-metadata-display-to-pla/) |
Last activity: 2026-03-08 - Completed quick task 18: add multi-column metadata display to playlist-details
*Last updated: 2026-03-09 — Completed 10-02-PLAN.md (Phase 10 complete)*
*Last updated: 2026-03-09 — Completed 11-01-PLAN.md (Phase 11 complete)*
.planning/phases/11-per-library-scan-pipeline/11-01-SUMMARY.md
+127
View File
@@ -0,0 +1,127 @@
---
phase: 11-per-library-scan-pipeline
plan: 01
subsystem: library
tags: [scan-queue, per-library, wails-bindings, sqlc, events]
# Dependency graph
requires:
- phase: 10-schema-migration
provides: libraries table, library_id column on audio_files, GetLibrary/GetAllLibraries/GetLibraryByPath queries
provides:
- ScanLibrary(id) Wails-bound method for per-library scanning
- ScanAllLibraries() Wails-bound method for bulk sequential scanning
- Scan queue coordinator with FIFO sequential execution and silent dedup
- CancelCurrentScan() and CancelAllScans() for queue-aware cancellation
- GetScanQueueLength() and QueuedLibraryNames() for UI display
- Library-aware ScanProgress and ScanMetrics with libraryId, libraryName, queuedCount
- LibraryScanQueued and LibraryScanQueueDrained events
- CreateAudioFile with library_id parameter
affects: [12-library-crud-data-integrity, 13-library-views-phantom-tracks]
# Tech tracking
tech-stack:
added: []
patterns:
- "Scan queue coordinator pattern: FIFO queue with single-active-scan mutex"
- "scanInternal() as reusable per-library scan engine"
- "Silent dedup for scan requests (no-op if already scanning or queued)"
key-files:
created:
- backend/library/scan_queue.go
modified:
- backend/library/library.go
- backend/library/scan_control.go
- backend/library/metrics.go
- backend/events/events.go
- backend/database/sql/queries/audio_files.sql
- backend/database/sql/sqlcgen/audio_files.sql.go
- frontend/src/events.ts
- frontend/wailsjs/go/library/Library.d.ts
- frontend/wailsjs/go/library/Library.js
key-decisions:
- "Library identification threaded through importResult.libraryID rather than adding field to Library struct"
- "scanInternal returns *ScanMetrics instead of (*ScanMetrics, error) — errors are logged and warnings accumulated"
- "Worker count auto-detected per library path (ScanConcurrencyAuto) rather than using global config value"
- "Backward-compatible Scan() retained as deprecated wrapper for handleConfigUpdate"
patterns-established:
- "Scan queue coordinator: scanQueue []scanQueueEntry + drainQueue() pattern for sequential execution"
- "mkProgress closure for DRY ScanProgress event construction with library identification"
requirements-completed: [LSCAN-01, LSCAN-02, LSCAN-04]
# Metrics
duration: 7min
completed: 2026-03-09
---
# Phase 11 Plan 01: Per-Library Scan Pipeline Summary
**ScanLibrary(id) with FIFO queue coordinator, per-library file association via library_id, and queue-aware cancel/pause controls**
## Performance
- **Duration:** 7 min
- **Started:** 2026-03-09T19:56:10Z
- **Completed:** 2026-03-09T20:03:14Z
- **Tasks:** 2
- **Files modified:** 11
## Accomplishments
- `ScanLibrary(id)` resolves library path from DB and scans only that directory, associating files with library_id
- FIFO scan queue ensures only one scan runs at a time, with silent dedup for duplicate requests
- `ScanAllLibraries()` queries all libraries and queues them sequentially
- `CancelCurrentScan()` stops current library and auto-starts next queued; `CancelAllScans()` clears queue too
- Pause freezes current scan AND queue (drainQueue only runs on scan completion)
- All scan events (progress, started, complete, cancelled) include library name and queue count
## Task Commits
Each task was committed atomically (note: lint fix amend merged both into single commit):
1. **Task 1: Add library_id to CreateAudioFile + update events and progress types** - `943db1c` (feat)
2. **Task 2: Create scan queue coordinator and refactor Library for per-library scanning** - `943db1c` (feat)
_Note: Tasks were merged into a single commit due to lint fix amend during pre-commit hook._
## Files Created/Modified
- `backend/library/scan_queue.go` - Scan queue coordinator: ScanLibrary, ScanAllLibraries, CancelCurrentScan, CancelAllScans, drainQueue
- `backend/library/library.go` - Refactored Scan() → scanInternal() with library ID/name/path parameters, per-library DB queries
- `backend/library/scan_control.go` - Deprecated CancelScan() in favor of queue-aware methods
- `backend/library/metrics.go` - Added LibraryID, LibraryName to ScanMetrics; LibraryID, LibraryName, QueuedCount to ScanProgress
- `backend/events/events.go` - Added LibraryScanQueued and LibraryScanQueueDrained constants
- `backend/database/sql/queries/audio_files.sql` - Added library_id to CreateAudioFile INSERT
- `backend/database/sql/sqlcgen/audio_files.sql.go` - Regenerated with LibraryID in CreateAudioFileParams
- `frontend/src/events.ts` - Regenerated with scan queue events
- `frontend/wailsjs/go/library/Library.d.ts` - Auto-generated Wails bindings for new methods
- `frontend/wailsjs/go/library/Library.js` - Auto-generated Wails bindings for new methods
- `frontend/wailsjs/go/models.ts` - Auto-generated model updates
## Decisions Made
- **Library ID threading via importResult:** Rather than adding a libraryID field to the Library struct, the ID is threaded through the scan pipeline via the importResult struct and set in the DB writer goroutine. This keeps the data flow explicit and avoids mutation of shared state.
- **scanInternal returns only metrics:** Changed signature from `(*ScanMetrics, error)` to `*ScanMetrics` since the scan queue coordinator calls it in a goroutine where error return is impractical. Errors are logged and accumulated in ScanMetrics.Warnings.
- **Auto worker count per library:** Each library path may reside on different storage (SSD vs HDD), so worker count uses `ScanConcurrencyAuto` with per-path detection rather than the global config value.
- **Backward-compatible Scan():** Retained as deprecated wrapper that resolves the library from `l.conf.DirectoryPath` via `GetLibraryByPath`. This keeps `handleConfigUpdate` and `FullRescan` working without changes.
## Deviations from Plan
None - plan executed exactly as written.
## Issues Encountered
None
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Per-library scan pipeline complete, ready for Phase 11 Plan 02 (if exists) or Phase 12 (Library CRUD & Data Integrity)
- Frontend can now call `ScanLibrary(id)`, `ScanAllLibraries()`, `CancelCurrentScan()`, `CancelAllScans()`
- Progress events include library identification for UI display
- Phase 12 can build library management UI on top of these Wails bindings
---
*Phase: 11-per-library-scan-pipeline*
*Completed: 2026-03-09*