docs(06-03): complete lookupChunk migration & SAFETY comments plan
- SUMMARY.md with execution results and self-check - STATE.md updated: Phase 6 complete (3/3 plans), decisions recorded - ROADMAP.md updated: Phase 6 marked Complete - REQUIREMENTS.md: QUAL-03 and QUAL-04 marked complete
This commit is contained in:
@@ -23,8 +23,8 @@ Requirements for the consolidation milestone. Each maps to roadmap phases.
|
||||
|
||||
- [x] **QUAL-01**: Duplicated FTS5 JOIN pattern (5+ copies) is consolidated into a single SQLite VIEW (track_metadata or similar)
|
||||
- [x] **QUAL-02**: Event name constants are generated from Go source (backend/events/events.go) to TypeScript (frontend/src/events.ts) via codegen, wired into go generate and pre-commit hook
|
||||
- [ ] **QUAL-03**: Queue batch lookups in persistence.go use sqlc.slice() instead of fmt.Sprintf placeholder construction where feasible
|
||||
- [ ] **QUAL-04**: Intentional hand-crafted SQL exceptions (batch INSERT, dynamic IN clauses) are documented with // SAFETY: comments explaining why they bypass sqlc
|
||||
- [x] **QUAL-03**: Queue batch lookups in persistence.go use sqlc.slice() instead of fmt.Sprintf placeholder construction where feasible
|
||||
- [x] **QUAL-04**: Intentional hand-crafted SQL exceptions (batch INSERT, dynamic IN clauses) are documented with // SAFETY: comments explaining why they bypass sqlc
|
||||
|
||||
### Performance
|
||||
|
||||
@@ -100,8 +100,8 @@ Which phases cover which requirements. Updated during roadmap creation.
|
||||
| CORR-09 | Phase 2: Backend Correctness | Complete |
|
||||
| QUAL-01 | Phase 6: SQL Consolidation & Code Quality | Complete |
|
||||
| QUAL-02 | Phase 6: SQL Consolidation & Code Quality | Complete |
|
||||
| QUAL-03 | Phase 6: SQL Consolidation & Code Quality | Pending |
|
||||
| QUAL-04 | Phase 6: SQL Consolidation & Code Quality | Pending |
|
||||
| QUAL-03 | Phase 6: SQL Consolidation & Code Quality | Complete |
|
||||
| QUAL-04 | Phase 6: SQL Consolidation & Code Quality | Complete |
|
||||
| PERF-01 | Phase 7: Backend Performance | Pending |
|
||||
| PERF-02 | Phase 7: Backend Performance | Pending |
|
||||
| PERF-03 | Phase 7: Backend Performance | Pending |
|
||||
|
||||
+24
-22
@@ -2,13 +2,13 @@
|
||||
gsd_state_version: 1.0
|
||||
milestone: v1.0
|
||||
milestone_name: milestone
|
||||
status: in-progress
|
||||
last_updated: "2026-03-05T00:23:19Z"
|
||||
status: executing
|
||||
last_updated: "2026-03-05T00:35:08.150Z"
|
||||
progress:
|
||||
total_phases: 8
|
||||
completed_phases: 5
|
||||
total_phases: 6
|
||||
completed_phases: 6
|
||||
total_plans: 11
|
||||
completed_plans: 10
|
||||
completed_plans: 11
|
||||
---
|
||||
|
||||
# YellowJacket — Consolidation Milestone State
|
||||
@@ -16,26 +16,26 @@ progress:
|
||||
## Project Reference
|
||||
|
||||
**Core value:** The music player works reliably and feels solid — every interaction is correct, responsive, and trustworthy.
|
||||
**Current focus:** Phase 6 in progress — track_metadata VIEW + event codegen complete, 1 plan remaining (SAFETY comments).
|
||||
**Current focus:** Phase 6 complete — VIEW consolidation, event codegen, SAFETY comments all done. Ready for Phase 7.
|
||||
**Milestone:** Consolidation (correctness, performance, code quality, UX polish, test coverage)
|
||||
|
||||
## Current Position
|
||||
|
||||
**Phase:** 06-sql-consolidation-code-quality (in progress)
|
||||
**Plan:** 2/3 (06-01, 06-02 complete)
|
||||
**Status:** In progress
|
||||
**Phase:** 06-sql-consolidation-code-quality (complete)
|
||||
**Plan:** 3/3 (all complete)
|
||||
**Status:** Phase complete
|
||||
|
||||
```
|
||||
Phase Progress: [######..] 6/8 phases — Phase 6: 2/3 plans complete
|
||||
Phase Progress: [######..] 6/8 phases — Phase 6: 3/3 plans complete ✓
|
||||
```
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Phases complete | 5/8 |
|
||||
| Plans complete | 1/2 (Phase 5) |
|
||||
| Requirements delivered | 16/26 |
|
||||
| Phases complete | 6/8 |
|
||||
| Plans complete | 3/3 (Phase 6) |
|
||||
| Requirements delivered | 18/26 |
|
||||
| Tests added | 84 |
|
||||
| Bugs fixed | 9 |
|
||||
| 01-01 duration | 11 min |
|
||||
@@ -50,6 +50,7 @@ Phase Progress: [######..] 6/8 phases — Phase 6: 2/3 plans complete
|
||||
| Phase 05 P02 | 4 min | 2 tasks | 1 files |
|
||||
| Phase 06 P01 | 2 min | 2 tasks | 4 files |
|
||||
| Phase 06 P02 | 2 min | 2 tasks | 3 files |
|
||||
| Phase 06 P03 | 6 min | 2 tasks | 7 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -76,6 +77,8 @@ Phase Progress: [######..] 6/8 phases — Phase 6: 2/3 plans complete
|
||||
| Contentless FTS5 DELETE limitation | DeleteSearchIndex errors on content='' tables; production logs warning, stale entries are harmless | Phase 5 |
|
||||
| SQLite VIEW for JOIN dedup | track_metadata VIEW consolidates 5-table JOIN; migration2 keeps inline JOIN for upgrade path | Phase 6 |
|
||||
| AST-based event codegen | Iterate f.Decls directly for deterministic declaration-order output; atomic writes via temp+rename | Phase 6 |
|
||||
| sqlc.slice() for batch lookups | LookupTrackMetaByPaths uses track_metadata VIEW; chunking preserved at 900 since sqlc.slice() doesn't auto-chunk | Phase 6 |
|
||||
| SAFETY comment convention | Two-part format (why + safety assurance); cross-references from library.go/rescan.go to search.go | Phase 6 |
|
||||
|
||||
### TODOs
|
||||
|
||||
@@ -119,19 +122,18 @@ None currently.
|
||||
### Last Session
|
||||
|
||||
**Date:** 2026-03-05
|
||||
**What happened:** Executed Phase 6 Plan 02 — Go→TypeScript event codegen tool with go/ast, fixing LibraryConfigChanged gap
|
||||
**Where we stopped:** Completed 06-02-PLAN.md (2 tasks, all verification passed). Phase 6 plan 2/3 done.
|
||||
**Next action:** `/gsd-execute-phase 06` to continue with 06-03-PLAN.md
|
||||
**What happened:** Executed Phase 6 Plan 03 — migrated lookupChunk to sqlc-generated query, added SAFETY comments to all 12 hand-crafted SQL statements
|
||||
**Where we stopped:** Completed 06-03-PLAN.md (2 tasks, all verification passed). Phase 6 complete (3/3 plans).
|
||||
**Next action:** `/gsd-plan-phase 07` to plan Phase 7 (performance/startup optimization)
|
||||
|
||||
### Context for Next Session
|
||||
|
||||
- Event codegen tool at backend/events/cmd/genevents/main.go
|
||||
- LibraryConfigChanged gap automatically fixed by codegen
|
||||
- `go generate ./...` completes in ~1.8s, codegen-check hook works end-to-end
|
||||
- `codegen-check` pre-commit hook no longer hangs — can use LEFTHOOK=1 for commits
|
||||
- Phase 6: 2/3 plans complete, SAFETY comments plan remaining
|
||||
- Phase 6 fully complete: VIEW consolidation, event codegen, SAFETY comments
|
||||
- All hand-crafted SQL documented with // SAFETY: comments
|
||||
- lookupChunk now uses sqlc-generated LookupTrackMetaByPaths
|
||||
- Ready for Phase 7 performance/startup optimization
|
||||
|
||||
---
|
||||
*State initialized: 2026-02-27*
|
||||
Last activity: 2026-03-05 - Completed 06-02: Go→TypeScript event codegen with go/ast and pre-commit hook
|
||||
Last activity: 2026-03-05 - Completed 06-03: lookupChunk sqlc migration + SAFETY comments on all hand-crafted SQL
|
||||
*Last updated: 2026-03-05*
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
phase: 06-sql-consolidation-code-quality
|
||||
plan: 03
|
||||
subsystem: database
|
||||
tags: [sqlite, sqlc, fts5, sql-safety, code-quality]
|
||||
|
||||
# Dependency graph
|
||||
requires:
|
||||
- phase: 06-sql-consolidation-code-quality
|
||||
provides: "track_metadata VIEW for sqlc query migration"
|
||||
provides:
|
||||
- "sqlc-generated LookupTrackMetaByPaths query with sqlc.slice()"
|
||||
- "SAFETY comments on all 12 hand-crafted SQL statements"
|
||||
affects: [07-performance-startup-optimization]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: ["sqlc.slice() for variable-length IN clauses", "SAFETY comment convention for hand-crafted SQL"]
|
||||
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- "backend/database/sql/queries/audio_files.sql"
|
||||
- "backend/database/sql/sqlcgen/audio_files.sql.go"
|
||||
- "backend/queue/persistence.go"
|
||||
- "backend/database/search.go"
|
||||
- "backend/library/library.go"
|
||||
- "backend/library/rescan.go"
|
||||
|
||||
key-decisions:
|
||||
- "Used sqlc.slice() with track_metadata VIEW for type-safe batch lookups"
|
||||
- "Preserved chunking at maxSQLiteVars=900 since sqlc.slice() does not auto-chunk"
|
||||
- "Two-part SAFETY comment format: why sqlc can't handle it + what makes it safe"
|
||||
|
||||
patterns-established:
|
||||
- "SAFETY comment convention: // SAFETY: [reason sqlc can't handle] + [safety assurance]"
|
||||
- "Cross-reference pattern: library.go/rescan.go SAFETY comments reference search.go canonical implementations"
|
||||
|
||||
requirements-completed: [QUAL-03, QUAL-04]
|
||||
|
||||
# Metrics
|
||||
duration: 6min
|
||||
completed: 2026-03-05
|
||||
---
|
||||
|
||||
# Phase 6 Plan 3: SQL Consolidation — lookupChunk Migration & SAFETY Comments Summary
|
||||
|
||||
**Migrated queue lookupChunk from fmt.Sprintf IN clause to sqlc-generated LookupTrackMetaByPaths query via track_metadata VIEW, and documented all 12 hand-crafted SQL statements with // SAFETY: comments**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 6 min
|
||||
- **Started:** 2026-03-05T00:27:52Z
|
||||
- **Completed:** 2026-03-05T00:34:10Z
|
||||
- **Tasks:** 2
|
||||
- **Files modified:** 7
|
||||
|
||||
## Accomplishments
|
||||
- Replaced hand-crafted `fmt.Sprintf` IN clause in `lookupChunk` with sqlc-generated `LookupTrackMetaByPaths` query using `sqlc.slice()` and `track_metadata` VIEW
|
||||
- Added `// SAFETY:` comments to all 12 hand-crafted SQL statements across 4 files (7 in search.go, 3 in library.go, 1 in rescan.go, 1 in persistence.go)
|
||||
- All existing tests pass unchanged: database (15), library (13), queue (29) — all with `-race`
|
||||
- Zero hand-crafted SQL in lookupChunk; the only remaining hand-crafted SQL in queue is `insertTrackBatch` (documented with SAFETY comment)
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Migrate lookupChunk to sqlc with sqlc.slice()** - `2221a68` (feat)
|
||||
2. **Task 2: Add SAFETY comments to all hand-crafted SQL** - `7dfe003` (docs)
|
||||
|
||||
## Files Created/Modified
|
||||
- `backend/database/sql/queries/audio_files.sql` - Added LookupTrackMetaByPaths query using track_metadata VIEW
|
||||
- `backend/database/sql/sqlcgen/audio_files.sql.go` - sqlc-generated Go code for LookupTrackMetaByPaths
|
||||
- `backend/queue/persistence.go` - lookupChunk now uses sqlc query; insertTrackBatch has SAFETY comment
|
||||
- `backend/database/search.go` - 7 SAFETY comments on all FTS5 operations
|
||||
- `backend/library/library.go` - 3 SAFETY comments on FTS5 INSERT/DELETE in commitNewAudioFile and updateAudioFileMetadata
|
||||
- `backend/library/rescan.go` - 1 SAFETY comment on FTS5 DELETE in clearAllLibraryData
|
||||
|
||||
## Decisions Made
|
||||
- Used `sqlc.slice()` with `track_metadata` VIEW — the VIEW already provides the exact columns needed (id, file_path, title, artist_name), eliminating the need for an inline JOIN
|
||||
- Preserved `lookupTrackMetaBatch` chunking at `maxSQLiteVars` (900) because `sqlc.slice()` does NOT auto-chunk large parameter lists
|
||||
- Two-part SAFETY comment format: (1) why sqlc can't handle it, (2) what makes the query safe — makes it clear these are intentional exceptions, not oversights
|
||||
- Cross-references in library.go/rescan.go point back to canonical search.go implementations to avoid divergent documentation
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written.
|
||||
|
||||
## Issues Encountered
|
||||
None
|
||||
|
||||
## User Setup Required
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
- Phase 6 complete: all 3 plans executed (VIEW consolidation, event codegen, SAFETY comments)
|
||||
- All hand-crafted SQL documented; future maintainers can see why each exception exists
|
||||
- Ready for Phase 7 (performance/startup optimization)
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
All created/modified files exist on disk. All commit hashes verified in git log.
|
||||
|
||||
---
|
||||
*Phase: 06-sql-consolidation-code-quality*
|
||||
*Completed: 2026-03-05*
|
||||
Reference in New Issue
Block a user