yonlu/yellowjacket
1
1
Fork 0
Code Issues 48 Pull Requests Actions Packages Projects Releases 12 Wiki Activity

chore: complete v1.0 Consolidation milestone

Browse Source 
Archive milestone artifacts:
- milestones/v1.0-ROADMAP.md (full roadmap archive)
- milestones/v1.0-REQUIREMENTS.md (26/26 requirements complete)
- milestones/v1.0-phases/ (8 phase directories with plans, summaries, verifications)

Updated:
- PROJECT.md: full evolution review, all consolidation requirements validated
- ROADMAP.md: collapsed to milestone summary with archive link
- STATE.md: reset for next milestone
- MILESTONES.md: created with stats and accomplishments
- RETROSPECTIVE.md: created with lessons learned

Deleted:
- REQUIREMENTS.md (archived, fresh for next milestone)

8 phases, 17 plans, 34 tasks, 84 tests added, 6 days
This commit is contained in:
yonlu
2026-03-05 09:34:43 -05:00
parent 5ef45f91ed
commit 6ce0661fca
58 changed files with 348 additions and 294 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-01-PLAN.md
+241
View File
@@ -0,0 +1,241 @@
---
phase: 06-sql-consolidation-code-quality
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
- backend/database/database.go
- backend/database/search.go
- backend/database/sql/schemas/track_metadata_view.sql
- backend/database/sql/sqlcgen/models.go
autonomous: true
requirements: [QUAL-01]
must_haves:
truths:
- "All FTS5 search queries (SearchFTS, SearchFTSByFilename, SearchFTSTracks) use the track_metadata VIEW instead of inline 5-table JOINs"
- "RebuildSearchIndex SELECTs from track_metadata VIEW instead of duplicating the JOIN"
- "Migration 4 creates the track_metadata VIEW for existing databases"
- "sqlc generate succeeds with the VIEW schema file and produces updated models"
- "Existing FTS5 search tests (15 tests) pass unchanged after VIEW consolidation"
artifacts:
- path: "backend/database/sql/schemas/track_metadata_view.sql"
provides: "VIEW definition for sqlc schema awareness"
contains: "CREATE VIEW IF NOT EXISTS track_metadata"
- path: "backend/database/database.go"
provides: "Migration 4 creating VIEW for existing databases"
contains: "migration4TrackMetadataView"
- path: "backend/database/search.go"
provides: "Consolidated search queries using VIEW"
contains: "track_metadata"
key_links:
- from: "backend/database/search.go"
to: "track_metadata VIEW"
via: "JOIN track_metadata tm ON tm.id = si.rowid"
pattern: "JOIN track_metadata"
- from: "backend/database/database.go"
to: "track_metadata VIEW"
via: "migration 4 CREATE VIEW"
pattern: "CREATE VIEW IF NOT EXISTS track_metadata"
---
<objective>
Consolidate the duplicated 5-table FTS5 JOIN pattern into a single SQLite VIEW named `track_metadata`, and update all search queries to use it.
Purpose: Eliminate 4+ copies of the same complex JOIN across search.go and database.go. A single VIEW is the source of truth for audio file metadata JOINs — changes to the schema only need updating in one place.
Output: Migration 4 (VIEW creation), sqlc schema file, consolidated search.go queries, updated sqlc-generated code.
</objective>
<execution_context>
@/home/caleb/.config/opencode/get-shit-done/workflows/execute-plan.md
@/home/caleb/.config/opencode/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/06-sql-consolidation-code-quality/06-RESEARCH.md
@backend/database/database.go
@backend/database/search.go
@backend/database/sql/schemas/
@backend/database/sqlc.yaml
<interfaces>
<!-- Key types and contracts the executor needs. -->
From backend/database/database.go:
- Migrations are Go functions registered in a slice, applied sequentially by PRAGMA user_version
- Pattern: `migration2BasenameAndFTS`, `migration3UniqueArtistCreditArtist` — each bumps user_version
- Current highest migration: 3 (user_version=3)
- `//go:generate go tool sqlc generate` directive at line 21
From backend/database/search.go:
- `func (d *DB) SearchFTS(query string, limit int) ([]SearchResult, error)` — line 22
- `func (d *DB) SearchFTSByFilename(query string, limit int) ([]SearchResult, error)` — line 72
- `func (d *DB) RebuildSearchIndex() error` — line 161
- `func (d *DB) SearchFTSTracks(query string, limit int) ([]SearchTrackResult, error)` — line 222
- All 4 functions contain inline 5-table JOINs (audio_files → recordings → artist_credit → release_group_recordings subquery → release_groups)
From backend/database/sql/schemas/ directory:
- Schema files sorted alphabetically; sqlc processes them in filesystem order
- Tables: artist_credit.sql, artists.sql, audio_files.sql, cover_art.sql, file_types.sql, genres.sql, recordings.sql, release_group_recordings.sql, release_groups.sql, etc.
- `track_metadata_view.sql` will sort after all table schemas (t > all existing prefixes)
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: Create track_metadata VIEW schema and migration</name>
<files>
backend/database/sql/schemas/track_metadata_view.sql
backend/database/database.go
</files>
<action>
1. Create `backend/database/sql/schemas/track_metadata_view.sql` with the VIEW definition:
```sql
CREATE VIEW IF NOT EXISTS track_metadata AS
SELECT
af.id,
af.file_path,
af.length_milliseconds,
COALESCE(r.name, '') AS title,
COALESCE(ac.text, '') AS artist_name,
r.track_number,
r.disc_number,
COALESCE(rg.name, '') AS album,
CAST(COALESCE(
(SELECT GROUP_CONCAT(g.name, '||')
FROM recording_genres rg_sub
JOIN genres g ON rg_sub.genre_id = g.id
WHERE rg_sub.recording_id = r.id),
''
) AS TEXT) AS genre,
COALESCE(r.year, 0) AS year,
COALESCE(r.composer, '') AS composer,
COALESCE(ft.extension, '') AS file_type,
af.sample_rate,
af.bit_depth,
af.channels,
af.bitrate,
af.file_size
FROM audio_files af
LEFT JOIN recordings r ON af.recording_id = r.id
LEFT JOIN artist_credit ac ON r.artist_credit_id = ac.id
LEFT JOIN (
SELECT recording_id,
MIN(release_group_id) AS release_group_id
FROM release_group_recordings
GROUP BY recording_id
) rgr ON r.id = rgr.recording_id
LEFT JOIN release_groups rg ON rgr.release_group_id = rg.id
LEFT JOIN file_types ft ON af.file_type_id = ft.id;
```
2. In `backend/database/database.go`, add migration 4 (`migration4TrackMetadataView`):
- The migration function should execute `CREATE VIEW IF NOT EXISTS track_metadata AS ...` (same SQL as the schema file)
- Register it in the migrations slice after migration 3
- Follow the existing migration function pattern (takes `*sql.DB` and `context.Context`, returns `error`)
3. Run `go tool sqlc generate` from `backend/database/` to regenerate code with VIEW awareness.
4. **CRITICAL:** Do NOT change `migration2BasenameAndFTS` to use the VIEW — migration 2 runs before migration 4 for databases upgrading from version 1. The inline JOIN in migration 2 must stay as-is.
5. Verify sqlc generate succeeds without errors.
</action>
<verify>
<automated>cd backend/database && go tool sqlc generate && echo "sqlc OK"</automated>
</verify>
<done>
- `track_metadata_view.sql` exists in schemas directory with the VIEW definition
- Migration 4 registered in database.go, creates the VIEW for existing databases
- `sqlc generate` succeeds and recognizes the VIEW
- migration2 code is unchanged (still uses inline JOIN)
</done>
</task>
<task type="auto">
<name>Task 2: Consolidate search queries to use track_metadata VIEW</name>
<files>
backend/database/search.go
</files>
<action>
Update all 4 search functions in `search.go` to use the `track_metadata` VIEW instead of inline JOINs:
1. **SearchFTS** (line ~22): Replace the inline 5-table JOIN with:
```sql
SELECT tm.file_path, tm.length_milliseconds, tm.title, tm.artist_name, tm.album
FROM search_index si
JOIN track_metadata tm ON tm.id = si.rowid
WHERE search_index MATCH ?
ORDER BY rank
LIMIT ?
```
Only select the 5 columns the function actually uses — SQLite optimizes away unused VIEW columns.
2. **SearchFTSByFilename** (line ~72): Same pattern as SearchFTS but with the filename-specific FTS query logic. Replace the inline JOIN with `JOIN track_metadata tm ON tm.id = si.rowid`. Keep the same column selection.
3. **SearchFTSTracks** (line ~222): Replace the inline 6-table JOIN (includes file_types) with the VIEW. The VIEW already includes `file_type` (from the file_types JOIN), so this becomes simpler. Select the columns needed by `SearchTrackResult`: file_path, length_milliseconds, title, artist_name, album, track_number, disc_number, genre, year, composer, file_type, sample_rate, bit_depth, channels, bitrate, file_size.
4. **RebuildSearchIndex** (line ~161): Replace the inline JOIN with:
```sql
INSERT INTO search_index(rowid, file_path, title, artist, album)
SELECT id, file_path, title, artist_name, album
FROM track_metadata
```
**Preserve:** All FTS5 MATCH syntax, ORDER BY rank, LIMIT clauses, error handling, row scanning, and function signatures remain identical. Only the FROM/JOIN clauses change.
**Do NOT touch:** `InsertSearchIndex`, `DeleteSearchIndex`, `ClearSearchIndex` — these are single-row FTS5 operations that don't use JOINs.
</action>
<verify>
<automated>cd backend && go test -tags webkit2_41 -race -count=1 -timeout 60s ./database/...</automated>
</verify>
<done>
- SearchFTS, SearchFTSByFilename, SearchFTSTracks, and RebuildSearchIndex all use `track_metadata` VIEW
- No inline 5-table JOIN patterns remain in search.go (except in comments)
- All 15 existing FTS5 search tests pass with -race
- Function signatures unchanged — callers are unaffected
</done>
</task>
</tasks>
<verification>
```bash
# 1. Verify sqlc generates cleanly
cd backend/database && go tool sqlc generate
# 2. Verify all database tests pass (15 search tests + migrations)
cd backend && go test -tags webkit2_41 -race -count=1 -timeout 60s ./database/...
# 3. Verify no inline JOIN duplication remains in search.go
grep -c "LEFT JOIN recordings" backend/database/search.go # Should be 0
# 4. Verify VIEW is referenced
grep -c "track_metadata" backend/database/search.go # Should be 4+
# 5. Verify migration2 is unchanged
grep "LEFT JOIN recordings" backend/database/database.go # Should still exist (migration2 only)
# 6. Full build check
go build -tags webkit2_41 ./...
```
</verification>
<success_criteria>
- The duplicated 5-table JOIN pattern is eliminated from search.go (0 copies remain)
- All search queries use the `track_metadata` VIEW
- Migration 4 creates the VIEW for existing databases
- sqlc schema file enables future sqlc queries against the VIEW
- All 15 existing database tests pass with -race
- Full project builds without errors
</success_criteria>
<output>
After completion, create `.planning/phases/06-sql-consolidation-code-quality/06-01-SUMMARY.md`
</output>
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-01-SUMMARY.md
+102
View File
@@ -0,0 +1,102 @@
---
phase: 06-sql-consolidation-code-quality
plan: 01
subsystem: database
tags: [sqlite, view, fts5, sql-consolidation, sqlc]
# Dependency graph
requires:
- phase: 05-database-library-tests
provides: "15 FTS5 search tests as safety net for VIEW consolidation"
provides:
- "track_metadata VIEW consolidating 5-table metadata JOIN"
- "Migration 4 for existing databases"
- "sqlc schema awareness of track_metadata VIEW"
affects: [07-performance-startup-optimization, 08-frontend-polish-accessibility]
# Tech tracking
tech-stack:
added: []
patterns: ["SQLite VIEW for JOIN deduplication", "migration-backed VIEW creation"]
key-files:
created:
- "backend/database/sql/schemas/track_metadata_view.sql"
modified:
- "backend/database/database.go"
- "backend/database/search.go"
- "backend/database/sql/sqlcgen/models.go"
key-decisions:
- "VIEW uses CREATE VIEW IF NOT EXISTS for idempotent schema application"
- "migration2 inline JOIN preserved — runs before migration 4 for upgrade path"
patterns-established:
- "SQLite VIEW as single source of truth for complex multi-table JOINs"
requirements-completed: [QUAL-01]
# Metrics
duration: 2min
completed: 2026-03-05
---
# Phase 6 Plan 1: SQL Consolidation — track_metadata VIEW Summary
**Consolidated 4 duplicated 5-table FTS5 JOINs into a single `track_metadata` SQLite VIEW with migration 4 and sqlc schema awareness**
## Performance
- **Duration:** 2 min
- **Started:** 2026-03-05T00:20:53Z
- **Completed:** 2026-03-05T00:23:19Z
- **Tasks:** 2
- **Files modified:** 4
## Accomplishments
- Created `track_metadata` VIEW consolidating the 5-table audio metadata JOIN pattern
- Added migration 4 to create the VIEW for existing databases (user_version 3→4)
- Replaced all 4 inline JOINs in search.go (SearchFTS, SearchFTSByFilename, SearchFTSTracks, RebuildSearchIndex) with VIEW references
- All 15 existing FTS5 search tests pass unchanged with `-race`
- Net reduction: 60 lines of duplicated SQL eliminated
## Task Commits
Each task was committed atomically:
1. **Task 1: Create track_metadata VIEW schema and migration** - `9c7e5a9` (feat)
2. **Task 2: Consolidate search queries to use track_metadata VIEW** - `9159b40` (refactor)
## Files Created/Modified
- `backend/database/sql/schemas/track_metadata_view.sql` - VIEW definition for sqlc schema awareness
- `backend/database/database.go` - Migration 4 (track_metadata VIEW creation for existing databases)
- `backend/database/search.go` - All 4 search functions now use `JOIN track_metadata` instead of inline JOINs
- `backend/database/sql/sqlcgen/models.go` - sqlc-generated TrackMetadatum model from VIEW
## Decisions Made
- VIEW uses `CREATE VIEW IF NOT EXISTS` for idempotent schema application (safe for both fresh and migrated databases)
- migration2 inline JOIN intentionally preserved — it runs at user_version=1→2 before the VIEW exists at version=3→4
- TrackMetadatum sqlc model generated automatically but not used in Go code yet (available for future sqlc queries against the VIEW)
## Deviations from Plan
None - plan executed exactly as written.
## Issues Encountered
None
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- VIEW consolidation complete, search.go has zero duplicated JOINs
- Ready for remaining Phase 6 plans (code quality improvements)
- Track metadata VIEW available for future sqlc queries
## Self-Check: PASSED
All created files exist on disk. All commit hashes verified in git log.
---
*Phase: 06-sql-consolidation-code-quality*
*Completed: 2026-03-05*
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-02-PLAN.md
+253
View File
@@ -0,0 +1,253 @@
---
phase: 06-sql-consolidation-code-quality
plan: 02
type: execute
wave: 1
depends_on: []
files_modified:
- backend/events/events.go
- backend/events/cmd/genevents/main.go
- frontend/src/events.ts
- lefthook.yml
autonomous: true
requirements: [QUAL-02]
must_haves:
truths:
- "Running `go generate ./backend/events/...` produces frontend/src/events.ts that exactly matches the Go constants"
- "The generated events.ts includes LibraryConfigChanged (currently missing from hand-maintained TS file)"
- "The codegen-check pre-commit hook detects stale events.ts and fails"
- "Output is deterministic — running the generator twice produces identical output"
artifacts:
- path: "backend/events/cmd/genevents/main.go"
provides: "Go→TypeScript event constant generator"
contains: "go/ast"
- path: "backend/events/events.go"
provides: "go:generate directive for event codegen"
contains: "go:generate"
- path: "frontend/src/events.ts"
provides: "Generated TypeScript event constants"
contains: "LibraryConfigChanged"
key_links:
- from: "backend/events/events.go"
to: "frontend/src/events.ts"
via: "go:generate directive running genevents"
pattern: "go:generate go run"
- from: "lefthook.yml"
to: "go generate"
via: "codegen-check pre-commit hook"
pattern: "go generate"
---
<objective>
Build a Go code generator that reads event constants from `backend/events/events.go` using `go/ast` and produces `frontend/src/events.ts`, then wire it into `go generate` and the pre-commit hook.
Purpose: Eliminate manual synchronization of event names between Go and TypeScript. The generator automatically catches drift (like the missing `LibraryConfigChanged`) and the pre-commit hook prevents stale files from being committed.
Output: Generator tool, `//go:generate` directive, updated events.ts with missing constant, working codegen-check hook.
</objective>
<execution_context>
@/home/caleb/.config/opencode/get-shit-done/workflows/execute-plan.md
@/home/caleb/.config/opencode/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/06-sql-consolidation-code-quality/06-RESEARCH.md
@backend/events/events.go
@frontend/src/events.ts
@lefthook.yml
<interfaces>
<!-- Current event constants for reference -->
From backend/events/events.go (21 constants in 5 groups):
```go
// Playback events (backend → frontend push).
const (
PlaybackStateChanged = "PlaybackStateChanged"
PlaybackFinished = "PlaybackFinished"
TrackChanged = "TrackChanged"
SeekFailed = "SeekFailed"
VolumeChanged = "VolumeChanged"
)
// Queue events (backend → frontend push).
const (
QueueChanged = "QueueChanged"
QueueIndexChanged = "QueueIndexChanged"
QueueModeChanged = "QueueModeChanged"
QueueTracksModified = "QueueTracksModified"
)
// Config events.
const (
LibraryConfigChanged = "LibraryConfigChanged" // <-- MISSING from TS
ThemeConfigChanged = "ThemeConfigChanged"
TrackListConfigChanged = "TrackListConfigChanged"
FavoritesConfigChanged = "FavoritesConfigChanged"
)
// Playlist events.
const (
PlaylistCreated = "PlaylistCreated"
PlaylistDeleted = "PlaylistDeleted"
PlaylistRenamed = "PlaylistRenamed"
PlaylistTracksChanged = "PlaylistTracksChanged"
PlaylistsRestored = "PlaylistsRestored"
DefaultPlaylistChanged = "DefaultPlaylistChanged"
)
// Library events.
const (
LibraryScanStarted = "LibraryScanStarted"
LibraryScanComplete = "LibraryScanComplete"
)
```
From frontend/src/events.ts (20 constants — missing LibraryConfigChanged):
- Format: `export const Events = { ... } as const;`
- Followed by: `export type EventName = (typeof Events)[keyof typeof Events];`
- Comment groups match Go groups (Playback, Queue, Playlist, Config, Library)
From lefthook.yml:
- codegen-check hook runs `go generate ./...` then checks `git diff --name-only`
- Hook currently hangs per STATE.md but research shows `go generate ./...` now completes in <1s
Existing go:generate directives:
- `backend/app.go:4` — `//go:generate go tool templ generate`
- `backend/database/database.go:21` — `//go:generate go tool sqlc generate`
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: Create event codegen tool</name>
<files>
backend/events/cmd/genevents/main.go
backend/events/events.go
</files>
<action>
1. Create `backend/events/cmd/genevents/main.go` — a standalone Go program (package main) that:
- Uses `go/ast`, `go/parser`, `go/token` to parse `events.go` in the same directory as the source
- Accepts a `-source` flag (path to events.go, default: the events.go file relative to the generator location) and an `-output` flag (path to output .ts file)
- Walks the AST in declaration order (NOT map iteration — deterministic output is critical)
- For each `const` block: extracts the doc comment above the block (e.g., "// Playback events (backend → frontend push).") and each constant name + string value
- Generates TypeScript output matching the current `events.ts` format exactly:
```typescript
// Code generated by genevents from backend/events/events.go. DO NOT EDIT.
export const Events = {
// Playback events (backend → frontend push)
PlaybackStateChanged: "PlaybackStateChanged",
...
} as const;
export type EventName = (typeof Events)[keyof typeof Events];
```
- Preserves comment group separation with blank lines between groups
- Strips the trailing period from Go doc comments (Go convention) for TypeScript comments
- Writes output atomically (write to temp file, then rename)
2. Add `//go:generate` directive to `backend/events/events.go`:
```go
//go:generate go run ./cmd/genevents -source events.go -output ../../frontend/src/events.ts
```
Place it after the package doc comment and before the first const block. Use a relative path from the events package directory to the frontend output.
3. Run `go generate ./backend/events/...` and verify the output matches the expected format.
4. Verify the generated events.ts now includes `LibraryConfigChanged` (the constant missing from the hand-maintained file).
**Key constraint:** AST iteration must be in source declaration order (iterate `f.Decls` directly, NOT collect into a map). This ensures deterministic output so the codegen-check hook doesn't produce false diffs.
</action>
<verify>
<automated>go generate ./backend/events/... && diff <(cat frontend/src/events.ts) <(go run ./backend/events/cmd/genevents -source backend/events/events.go -output /dev/stdout) && echo "Deterministic OK" && grep -q "LibraryConfigChanged" frontend/src/events.ts && echo "Missing constant fixed"</automated>
</verify>
<done>
- Generator exists at backend/events/cmd/genevents/main.go
- `//go:generate` directive added to events.go
- Running `go generate ./backend/events/...` produces valid events.ts
- Output includes all 21 constants (including LibraryConfigChanged)
- Output is deterministic (running twice produces identical files)
- Comment groups match Go source ordering
</done>
</task>
<task type="auto">
<name>Task 2: Wire codegen-check pre-commit hook</name>
<files>
lefthook.yml
</files>
<action>
1. The existing `codegen-check` hook in `lefthook.yml` already runs `go generate ./...` and diffs. Per research, `go generate ./...` now completes in <1 second (previous hanging appears resolved). The hook structure should work as-is with the new event generator wired in.
2. Test the hook end-to-end:
- Run `go generate ./...` and verify it completes quickly (<5 seconds)
- Verify no unstaged changes exist after generation (all generated code is up-to-date)
- Manually introduce a drift: add a test constant to events.go, verify `go generate` updates events.ts, then verify the hook would detect the diff
3. If the hook still hangs (unlikely per research): narrow the `codegen-check` glob to only trigger on event-related files, or split into a separate event-specific check. Update lefthook.yml accordingly.
4. Run the full pre-commit hook to verify all hooks pass:
```bash
LEFTHOOK=1 lefthook run pre-commit
```
Note: If the hook takes >10 seconds, investigate and optimize. Expected: <5s total.
5. Clean up any test changes (remove test constant if added).
**Important:** The hook runs `go generate ./...` which triggers ALL generators (templ, sqlc, events). This is the correct behavior — it ensures all generated code is fresh. The <1s completion time makes this acceptable.
</action>
<verify>
<automated>go generate ./... && test -z "$(git diff --name-only)" && echo "codegen-check would pass"</automated>
</verify>
<done>
- `go generate ./...` completes in <5 seconds
- codegen-check hook detects stale events.ts (adding Go constant without regenerating TS fails the hook)
- All existing pre-commit hooks still pass
- No leftover test changes in the working tree
</done>
</task>
</tasks>
<verification>
```bash
# 1. Generator produces valid output
go generate ./backend/events/...
# 2. Output includes all 21 constants
grep -c ":" frontend/src/events.ts # Should be 21+ (constants + type line)
# 3. LibraryConfigChanged is present
grep "LibraryConfigChanged" frontend/src/events.ts
# 4. Deterministic output
go generate ./backend/events/...
git diff --name-only # Should be empty (no changes on second run)
# 5. Full generate works
go generate ./...
# 6. Frontend typecheck passes with new events.ts
cd frontend && ./node_modules/.bin/tsc --noEmit
# 7. Full build
go build -tags webkit2_41 ./...
```
</verification>
<success_criteria>
- Event codegen tool parses Go constants and generates matching TypeScript
- LibraryConfigChanged gap is automatically fixed
- `go generate` directive wired into events.go
- codegen-check hook works end-to-end (detects drift, passes when clean)
- Frontend TypeScript compiles with generated events.ts
- Output is deterministic across runs
</success_criteria>
<output>
After completion, create `.planning/phases/06-sql-consolidation-code-quality/06-02-SUMMARY.md`
</output>
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-02-SUMMARY.md
+98
View File
@@ -0,0 +1,98 @@
---
phase: 06-sql-consolidation-code-quality
plan: 02
subsystem: codegen
tags: [go-ast, codegen, typescript, go-generate, lefthook]
# Dependency graph
requires: []
provides:
- Go→TypeScript event constant generator (genevents)
- go:generate directive for automatic event sync
- LibraryConfigChanged gap automatically fixed
- Pre-commit codegen-check hook covers event constants
affects: [frontend, backend-events]
# Tech tracking
tech-stack:
added: [go/ast, go/parser, go/token]
patterns: [AST-based codegen for cross-language constant sync, atomic file writes via temp+rename]
key-files:
created:
- backend/events/cmd/genevents/main.go
modified:
- backend/events/events.go
- frontend/src/events.ts
key-decisions:
- "Iterate f.Decls directly (not map) for deterministic declaration-order output"
- "Strip trailing period from Go doc comments for cleaner TypeScript comments"
- "Atomic writes via temp file + os.Rename to prevent partial output"
patterns-established:
- "Cross-language constant sync: Go source of truth → go/ast parser → TypeScript codegen"
- "go:generate directive per package with relative paths to output"
requirements-completed: [QUAL-02]
# Metrics
duration: 2min
completed: 2026-03-05
---
# Phase 06 Plan 02: Event Codegen Summary
**Go→TypeScript event constant generator using go/ast, fixing LibraryConfigChanged gap and wiring pre-commit drift detection**
## Performance
- **Duration:** 2 min
- **Started:** 2026-03-05T00:20:57Z
- **Completed:** 2026-03-05T00:23:43Z
- **Tasks:** 2
- **Files modified:** 3
## Accomplishments
- Built `genevents` codegen tool parsing Go AST for deterministic TypeScript output
- Fixed missing `LibraryConfigChanged` constant — now automatically generated from Go source
- Verified codegen-check pre-commit hook detects drift when Go constants change without regenerating TS
- All 21 event constants synced between Go and TypeScript, frontend typecheck passes
## Task Commits
Each task was committed atomically:
1. **Task 1: Create event codegen tool** - `3e9edd0` (feat)
2. **Task 2: Wire codegen-check pre-commit hook** - No changes needed (lefthook.yml already correctly configured; task was verification-only)
## Files Created/Modified
- `backend/events/cmd/genevents/main.go` - Go→TypeScript event constant generator using go/ast
- `backend/events/events.go` - Added `//go:generate` directive for automatic codegen
- `frontend/src/events.ts` - Regenerated with all 21 constants including LibraryConfigChanged
## Decisions Made
- Iterated `f.Decls` directly (not collected into map) for deterministic declaration-order output
- Stripped trailing periods from Go doc comments for cleaner TypeScript comments
- Used atomic writes (temp file + `os.Rename`) to prevent partial output on failure
- No lefthook.yml changes needed — existing `codegen-check` hook already runs `go generate ./...` which now includes the event generator
## Deviations from Plan
None - plan executed exactly as written.
## Issues Encountered
None
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Event codegen complete, ready for remaining Phase 6 plans
- Pre-commit hook validates all generated code (templ, sqlc, events) in <2 seconds
## Self-Check: PASSED
---
*Phase: 06-sql-consolidation-code-quality*
*Completed: 2026-03-05*
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-03-PLAN.md
+317
View File
@@ -0,0 +1,317 @@
---
phase: 06-sql-consolidation-code-quality
plan: 03
type: execute
wave: 2
depends_on: [06-01]
files_modified:
- backend/database/sql/queries/audio_files.sql
- backend/database/sql/sqlcgen/audio_files.sql.go
- backend/database/sql/sqlcgen/models.go
- backend/queue/persistence.go
- backend/database/search.go
- backend/library/library.go
- backend/library/rescan.go
autonomous: true
requirements: [QUAL-03, QUAL-04]
must_haves:
truths:
- "lookupChunk no longer uses fmt.Sprintf for IN clause construction — it calls a sqlc-generated query via the track_metadata VIEW"
- "Every hand-crafted SQL statement that bypasses sqlc has a // SAFETY: comment with two parts: why sqlc can't handle it AND what makes it safe"
- "All 12 identified hand-crafted SQL statements have SAFETY comments"
- "Queue tests and database tests pass unchanged after the migration"
artifacts:
- path: "backend/database/sql/queries/audio_files.sql"
provides: "sqlc query for batch track metadata lookup"
contains: "LookupTrackMetaByPaths"
- path: "backend/queue/persistence.go"
provides: "Updated lookupChunk using sqlc-generated query"
contains: "SAFETY"
- path: "backend/database/search.go"
provides: "SAFETY comments on all FTS5 queries"
contains: "SAFETY"
- path: "backend/library/library.go"
provides: "SAFETY comments on FTS5 insert/delete operations"
contains: "SAFETY"
- path: "backend/library/rescan.go"
provides: "SAFETY comments on FTS5 delete operation"
contains: "SAFETY"
key_links:
- from: "backend/queue/persistence.go"
to: "backend/database/sql/sqlcgen/"
via: "sqlc-generated LookupTrackMetaByPaths query"
pattern: "LookupTrackMetaByPaths"
- from: "backend/database/sql/queries/audio_files.sql"
to: "track_metadata VIEW"
via: "SELECT FROM track_metadata WHERE file_path IN (sqlc.slice)"
pattern: "sqlc.slice"
---
<objective>
Migrate the queue's `lookupChunk` from hand-crafted SQL with `fmt.Sprintf` to a sqlc-generated query using the `track_metadata` VIEW and `sqlc.slice()`, then add `// SAFETY:` comments to all remaining hand-crafted SQL statements.
Purpose: Replace the only hand-crafted SQL that CAN be migrated to sqlc (lookupChunk), and document all intentional exceptions so future maintainers understand why each hand-crafted statement exists.
Output: sqlc query file, regenerated code, updated persistence.go, SAFETY comments on all 12 hand-crafted SQL statements.
</objective>
<execution_context>
@/home/caleb/.config/opencode/get-shit-done/workflows/execute-plan.md
@/home/caleb/.config/opencode/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/06-sql-consolidation-code-quality/06-RESEARCH.md
@.planning/phases/06-sql-consolidation-code-quality/06-01-SUMMARY.md
@backend/queue/persistence.go
@backend/database/search.go
@backend/library/library.go
@backend/library/rescan.go
@backend/database/sql/queries/audio_files.sql
@backend/database/sqlc.yaml
<interfaces>
<!-- Key types the executor needs -->
From backend/queue/persistence.go:
```go
type trackMeta struct {
AudioFileID int64
FilePath string
Title string
Artist string
}
// lookupTrackMetaBatch — chunks at maxSQLiteVars (900) and calls lookupChunk per chunk
// lookupChunk — hand-crafted SELECT with fmt.Sprintf IN clause (TARGET for sqlc migration)
// insertTrackBatch — multi-row INSERT with variable VALUES count (STAYS hand-crafted)
const maxSQLiteVars = 900
```
From backend/database/search.go (after Plan 01 consolidation):
- SearchFTS — FTS5 MATCH query using track_metadata VIEW
- SearchFTSByFilename — FTS5 MATCH query using track_metadata VIEW
- InsertSearchIndex — single-row INSERT INTO search_index
- DeleteSearchIndex — DELETE FROM search_index WHERE rowid = ?
- ClearSearchIndex — DELETE FROM search_index
- RebuildSearchIndex — INSERT INTO search_index SELECT FROM track_metadata
- SearchFTSTracks — FTS5 MATCH query using track_metadata VIEW
From backend/library/library.go:
- commitNewAudioFile (~line 798) — INSERT INTO search_index VALUES (single row)
- updateAudioFileMetadata (~line 879) — DELETE FROM search_index WHERE rowid = ?
- updateAudioFileMetadata (~line 893) — INSERT INTO search_index VALUES (single row)
From backend/library/rescan.go:
- clearAllLibraryData (~line 165) — DELETE FROM search_index
Complete SAFETY comment inventory (12 statements):
| # | File | Function | Operation | Why hand-crafted |
|---|------|----------|-----------|-----------------|
| 1 | search.go | SearchFTS | FTS5 MATCH | FTS5 unsupported by sqlc |
| 2 | search.go | SearchFTSByFilename | FTS5 MATCH | FTS5 unsupported by sqlc |
| 3 | search.go | InsertSearchIndex | FTS5 INSERT | FTS5 virtual table |
| 4 | search.go | DeleteSearchIndex | FTS5 DELETE | FTS5 virtual table |
| 5 | search.go | ClearSearchIndex | FTS5 DELETE | FTS5 virtual table |
| 6 | search.go | RebuildSearchIndex | FTS5 INSERT SELECT | FTS5 virtual table |
| 7 | search.go | SearchFTSTracks | FTS5 MATCH | FTS5 unsupported by sqlc |
| 8 | library.go | commitNewAudioFile | FTS5 INSERT | FTS5 virtual table |
| 9 | library.go | updateAudioFileMetadata | FTS5 DELETE | FTS5 virtual table |
| 10 | library.go | updateAudioFileMetadata | FTS5 INSERT | FTS5 virtual table |
| 11 | rescan.go | clearAllLibraryData | FTS5 DELETE | FTS5 virtual table |
| 12 | persistence.go | insertTrackBatch | Variable-count multi-row INSERT | sqlc can't generate variable-length batch INSERTs |
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: Migrate lookupChunk to sqlc with sqlc.slice()</name>
<files>
backend/database/sql/queries/audio_files.sql
backend/database/sql/sqlcgen/audio_files.sql.go
backend/database/sql/sqlcgen/models.go
backend/queue/persistence.go
</files>
<action>
1. Add the sqlc query to `backend/database/sql/queries/audio_files.sql`:
```sql
-- name: LookupTrackMetaByPaths :many
SELECT id, file_path, title, artist_name
FROM track_metadata
WHERE file_path IN (sqlc.slice('paths'));
```
This uses the `track_metadata` VIEW created by Plan 01. The VIEW's columns `title` and `artist_name` match the data lookupChunk currently fetches via its inline JOIN.
2. Run `go tool sqlc generate` from `backend/database/` to generate the Go code.
3. Update `backend/queue/persistence.go`:
a. Replace the `lookupChunk` method body. Instead of building `fmt.Sprintf` placeholders, call the sqlc-generated `LookupTrackMetaByPaths` method:
```go
func (q *Queue) lookupChunk(
paths []string,
result map[string]trackMeta,
) {
if len(paths) == 0 {
return
}
rows, err := q.db.Queries.LookupTrackMetaByPaths(q.db.Ctx, paths)
if err != nil {
q.logger.Error("Batch metadata lookup failed", "err", err)
return
}
for _, row := range rows {
result[row.FilePath] = trackMeta{
AudioFileID: row.ID,
FilePath: row.FilePath,
Title: row.Title,
Artist: row.ArtistName,
}
}
}
```
b. The `lookupTrackMetaBatch` function stays unchanged — it still chunks at `maxSQLiteVars` and calls `lookupChunk` per chunk. The chunking is still necessary because `sqlc.slice()` does NOT auto-chunk.
c. Remove the now-unused imports: `"fmt"` and `"strings"` may become unused if `insertTrackBatch` is the only remaining user. Check import usage — `fmt` is still needed for `insertTrackBatch` (line ~200 `fmt.Errorf`), and `strings` is still needed for `insertTrackBatch` (line ~196 `strings.Join`). Keep both if still referenced.
4. Verify the field name mapping is correct:
- VIEW column `id` → sqlc field `ID` → `trackMeta.AudioFileID`
- VIEW column `file_path` → sqlc field `FilePath` → `trackMeta.FilePath`
- VIEW column `title` → sqlc field `Title` → `trackMeta.Title`
- VIEW column `artist_name` → sqlc field `ArtistName` → `trackMeta.Artist`
5. Run queue tests to verify the migration doesn't break metadata resolution.
</action>
<verify>
<automated>cd backend/database && go tool sqlc generate && cd ../.. && go test -tags webkit2_41 -race -count=1 -timeout 60s ./backend/queue/...</automated>
</verify>
<done>
- sqlc query `LookupTrackMetaByPaths` exists in audio_files.sql
- lookupChunk uses the sqlc-generated query instead of fmt.Sprintf
- lookupTrackMetaBatch still chunks at maxSQLiteVars (900)
- All queue tests pass with -race (29 tests)
- No hand-crafted SQL remains in lookupChunk
</done>
</task>
<task type="auto">
<name>Task 2: Add SAFETY comments to all hand-crafted SQL</name>
<files>
backend/database/search.go
backend/library/library.go
backend/library/rescan.go
backend/queue/persistence.go
</files>
<action>
Add `// SAFETY:` comments to all 12 hand-crafted SQL statements. Each comment has two parts: (1) WHY sqlc can't handle it, and (2) what makes the query safe. Cross-reference related operations where applicable.
**backend/database/search.go** (7 statements):
1. Before SearchFTS query (~line 34):
`// SAFETY: FTS5 MATCH syntax unsupported by sqlc. Query is parameterized; no string interpolation.`
2. Before SearchFTSByFilename query (~line 92):
`// SAFETY: FTS5 MATCH syntax unsupported by sqlc. Query is parameterized; no string interpolation.`
3. Before InsertSearchIndex query (~line 133):
`// SAFETY: FTS5 virtual table INSERT unsupported by sqlc. All values are parameterized.`
4. Before DeleteSearchIndex query (~line 143):
`// SAFETY: FTS5 virtual table DELETE unsupported by sqlc. Rowid is parameterized.`
5. Before ClearSearchIndex query (~line 152):
`// SAFETY: FTS5 virtual table DELETE unsupported by sqlc. No parameters; unconditional delete.`
6. Before RebuildSearchIndex query (~line 168):
`// SAFETY: FTS5 virtual table INSERT unsupported by sqlc. All values sourced from track_metadata VIEW; no user input.`
7. Before SearchFTSTracks query (~line 232):
`// SAFETY: FTS5 MATCH syntax unsupported by sqlc. Query is parameterized; no string interpolation.`
**backend/library/library.go** (3 statements):
8. Before commitNewAudioFile FTS INSERT (~line 798):
`// SAFETY: FTS5 virtual table, see search.go:InsertSearchIndex. All values parameterized.`
9. Before updateAudioFileMetadata FTS DELETE (~line 879):
`// SAFETY: FTS5 virtual table, see search.go:DeleteSearchIndex. Rowid parameterized.`
10. Before updateAudioFileMetadata FTS INSERT (~line 893):
`// SAFETY: FTS5 virtual table, see search.go:InsertSearchIndex. All values parameterized.`
**backend/library/rescan.go** (1 statement):
11. Before clearAllLibraryData FTS DELETE (~line 165):
`// SAFETY: FTS5 virtual table, see search.go:ClearSearchIndex. No parameters; unconditional delete.`
**backend/queue/persistence.go** (1 statement):
12. Before insertTrackBatch query (~line 195):
`// SAFETY: Multi-row INSERT with variable row count unsupported by sqlc. Placeholder count matches args length; no string interpolation.`
**Rules:**
- Place each SAFETY comment on the line immediately before the SQL string literal (the query variable or inline string)
- Use the exact `// SAFETY:` prefix (capital, colon, space)
- Two-part format: reason + safety assurance
- Cross-reference related operations in library.go/rescan.go back to search.go
</action>
<verify>
<automated>test $(grep -r "// SAFETY:" backend/database/search.go backend/library/library.go backend/library/rescan.go backend/queue/persistence.go | wc -l) -eq 12 && echo "All 12 SAFETY comments present" && go build -tags webkit2_41 ./...</automated>
</verify>
<done>
- All 12 hand-crafted SQL statements have SAFETY comments
- Comments follow two-part format (why + safety assurance)
- Cross-references link library.go/rescan.go back to search.go
- Code compiles without errors
- No SAFETY comments on migration DDL (migration2, migration3, migration4)
</done>
</task>
</tasks>
<verification>
```bash
# 1. sqlc generates cleanly
cd backend/database && go tool sqlc generate
# 2. All queue tests pass
go test -tags webkit2_41 -race -count=1 -timeout 60s ./backend/queue/...
# 3. All database tests pass
go test -tags webkit2_41 -race -count=1 -timeout 60s ./backend/database/...
# 4. All library tests pass
go test -tags webkit2_41 -race -count=1 -timeout 60s ./backend/library/...
# 5. Verify all 12 SAFETY comments exist
grep -r "// SAFETY:" backend/database/search.go backend/library/library.go backend/library/rescan.go backend/queue/persistence.go | wc -l # Should be 12
# 6. Verify no fmt.Sprintf remains in lookupChunk
grep -A5 "func.*lookupChunk" backend/queue/persistence.go | grep -c "fmt.Sprintf" # Should be 0
# 7. Full build
go build -tags webkit2_41 ./...
```
</verification>
<success_criteria>
- lookupChunk uses sqlc-generated `LookupTrackMetaByPaths` query against track_metadata VIEW
- fmt.Sprintf placeholder construction eliminated from lookupChunk
- Chunking logic preserved (maxSQLiteVars = 900)
- All 12 hand-crafted SQL statements documented with // SAFETY: comments
- All existing tests pass (queue: 29, database: 15, library: 13)
- Full project builds without errors
</success_criteria>
<output>
After completion, create `.planning/phases/06-sql-consolidation-code-quality/06-03-SUMMARY.md`
</output>
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-03-SUMMARY.md
+107
View File
@@ -0,0 +1,107 @@
---
phase: 06-sql-consolidation-code-quality
plan: 03
subsystem: database
tags: [sqlite, sqlc, fts5, sql-safety, code-quality]
# Dependency graph
requires:
- phase: 06-sql-consolidation-code-quality
provides: "track_metadata VIEW for sqlc query migration"
provides:
- "sqlc-generated LookupTrackMetaByPaths query with sqlc.slice()"
- "SAFETY comments on all 12 hand-crafted SQL statements"
affects: [07-performance-startup-optimization]
# Tech tracking
tech-stack:
added: []
patterns: ["sqlc.slice() for variable-length IN clauses", "SAFETY comment convention for hand-crafted SQL"]
key-files:
created: []
modified:
- "backend/database/sql/queries/audio_files.sql"
- "backend/database/sql/sqlcgen/audio_files.sql.go"
- "backend/queue/persistence.go"
- "backend/database/search.go"
- "backend/library/library.go"
- "backend/library/rescan.go"
key-decisions:
- "Used sqlc.slice() with track_metadata VIEW for type-safe batch lookups"
- "Preserved chunking at maxSQLiteVars=900 since sqlc.slice() does not auto-chunk"
- "Two-part SAFETY comment format: why sqlc can't handle it + what makes it safe"
patterns-established:
- "SAFETY comment convention: // SAFETY: [reason sqlc can't handle] + [safety assurance]"
- "Cross-reference pattern: library.go/rescan.go SAFETY comments reference search.go canonical implementations"
requirements-completed: [QUAL-03, QUAL-04]
# Metrics
duration: 6min
completed: 2026-03-05
---
# Phase 6 Plan 3: SQL Consolidation — lookupChunk Migration & SAFETY Comments Summary
**Migrated queue lookupChunk from fmt.Sprintf IN clause to sqlc-generated LookupTrackMetaByPaths query via track_metadata VIEW, and documented all 12 hand-crafted SQL statements with // SAFETY: comments**
## Performance
- **Duration:** 6 min
- **Started:** 2026-03-05T00:27:52Z
- **Completed:** 2026-03-05T00:34:10Z
- **Tasks:** 2
- **Files modified:** 7
## Accomplishments
- Replaced hand-crafted `fmt.Sprintf` IN clause in `lookupChunk` with sqlc-generated `LookupTrackMetaByPaths` query using `sqlc.slice()` and `track_metadata` VIEW
- Added `// SAFETY:` comments to all 12 hand-crafted SQL statements across 4 files (7 in search.go, 3 in library.go, 1 in rescan.go, 1 in persistence.go)
- All existing tests pass unchanged: database (15), library (13), queue (29) — all with `-race`
- Zero hand-crafted SQL in lookupChunk; the only remaining hand-crafted SQL in queue is `insertTrackBatch` (documented with SAFETY comment)
## Task Commits
Each task was committed atomically:
1. **Task 1: Migrate lookupChunk to sqlc with sqlc.slice()** - `2221a68` (feat)
2. **Task 2: Add SAFETY comments to all hand-crafted SQL** - `7dfe003` (docs)
## Files Created/Modified
- `backend/database/sql/queries/audio_files.sql` - Added LookupTrackMetaByPaths query using track_metadata VIEW
- `backend/database/sql/sqlcgen/audio_files.sql.go` - sqlc-generated Go code for LookupTrackMetaByPaths
- `backend/queue/persistence.go` - lookupChunk now uses sqlc query; insertTrackBatch has SAFETY comment
- `backend/database/search.go` - 7 SAFETY comments on all FTS5 operations
- `backend/library/library.go` - 3 SAFETY comments on FTS5 INSERT/DELETE in commitNewAudioFile and updateAudioFileMetadata
- `backend/library/rescan.go` - 1 SAFETY comment on FTS5 DELETE in clearAllLibraryData
## Decisions Made
- Used `sqlc.slice()` with `track_metadata` VIEW — the VIEW already provides the exact columns needed (id, file_path, title, artist_name), eliminating the need for an inline JOIN
- Preserved `lookupTrackMetaBatch` chunking at `maxSQLiteVars` (900) because `sqlc.slice()` does NOT auto-chunk large parameter lists
- Two-part SAFETY comment format: (1) why sqlc can't handle it, (2) what makes the query safe — makes it clear these are intentional exceptions, not oversights
- Cross-references in library.go/rescan.go point back to canonical search.go implementations to avoid divergent documentation
## Deviations from Plan
None - plan executed exactly as written.
## Issues Encountered
None
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Phase 6 complete: all 3 plans executed (VIEW consolidation, event codegen, SAFETY comments)
- All hand-crafted SQL documented; future maintainers can see why each exception exists
- Ready for Phase 7 (performance/startup optimization)
## Self-Check: PASSED
All created/modified files exist on disk. All commit hashes verified in git log.
---
*Phase: 06-sql-consolidation-code-quality*
*Completed: 2026-03-05*
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-CONTEXT.md
+73
View File
@@ -0,0 +1,73 @@
# Phase 6: SQL Consolidation & Code Quality - Context
**Gathered:** 2026-03-04
**Status:** Ready for planning
<domain>
## Phase Boundary
Eliminate duplicated SQL patterns (FTS5 5-table JOIN), automate Go-to-TypeScript event constant synchronization, migrate eligible hand-crafted SQL to sqlc, and document all intentional sqlc exceptions with SAFETY comments. No new features, no schema changes beyond the VIEW migration.
</domain>
<decisions>
## Implementation Decisions
### FTS5 VIEW Design
- Single VIEW named `track_metadata` with all 16 columns (file_path, length, title, artist, album, track_number, disc_number, genre, year, composer, file_type, sample_rate, bit_depth, channels, bitrate, file_size)
- VIEW rooted on `audio_files` (not `search_index`) so it's usable for both search queries (JOIN search_index to VIEW) and index rebuilds (SELECT directly from VIEW)
- Created as migration 4 (next sequential PRAGMA user_version bump)
- Lightweight search queries (SearchFTS, SearchFTSByFilename) SELECT only the 5 columns they need from the VIEW — SQLite optimizes unused columns away
- RebuildSearchIndex and migration2 INSERT INTO search_index use the VIEW instead of duplicating the JOIN
### Event Codegen Approach
- Go constants in `backend/events/events.go` are the source of truth
- Generator written in Go, using `go/ast` to parse the const block from events.go
- Wired into `go generate` via `//go:generate` directive on events.go
- Output format matches current `frontend/src/events.ts` structure exactly: `export const Events = { ... } as const;` — zero changes needed in frontend import sites
- Fix the existing `codegen-check` lefthook pre-commit hook (currently hangs) to run the generator and diff the output — fail if TypeScript file is stale
- Note: `LibraryConfigChanged` exists in Go but is missing from TypeScript — the generator will fix this automatically
### sqlc Migration Scope
- Migrate `lookupChunk()` query fully to sqlc: the SELECT + JOINs + `sqlc.slice()` for the IN clause — use the new `track_metadata` VIEW instead of hand-crafted JOINs
- `insertTrackBatch()` (multi-row VALUES with variable row count) stays hand-crafted — sqlc cannot generate variable-length batch INSERTs. Document as exception.
- All FTS5 operations (~11 statements across search.go, library.go, rescan.go) stay hand-crafted — sqlc does not support FTS5 virtual tables (MATCH, rank, content='' tables). Document all as exceptions.
### SAFETY Comment Convention
- Format: two parts — WHY sqlc can't handle it AND what makes it safe
- Example: `// SAFETY: FTS5 MATCH syntax unsupported by sqlc. Query is parameterized; no string interpolation.`
- Scope: runtime query SQL only — migration DDL (ALTER TABLE, CREATE INDEX, PRAGMA) does NOT need SAFETY comments
- Per-statement annotation only — no central registry file. The comments ARE the documentation.
- Cross-reference related operations: FTS5 INSERT/DELETE in library.go and rescan.go should reference search.go functions they relate to (e.g., `// SAFETY: FTS5 virtual table, see search.go:RebuildSearchIndex. Parameterized.`)
### Claude's Discretion
- Exact VIEW column ordering and COALESCE/NULL handling
- Generator CLI interface (flags, output path defaults)
- How to structure the sqlc query file for lookupChunk (naming, placement)
- Exact wording of SAFETY comments (as long as they follow the two-part format)
- How to handle the lefthook codegen-check fix (may need to investigate why it hangs)
</decisions>
<specifics>
## Specific Ideas
- The `track_metadata` VIEW name matches the roadmap suggestion — keep it familiar
- Generator should use `go/ast` for reliable parsing, not regex/string matching on the Go source
- The existing `codegen-check` hook hangs per STATE.md — fixing it is part of this phase, not a separate effort
- `lookupChunk` uses chunking at `maxSQLiteVars = 900` — the sqlc migration must preserve this chunking logic even if the SQL itself moves to sqlc
- The migration code in database.go (migration2) that duplicates the rebuild JOIN should also switch to the VIEW once migration 4 creates it — but since migration 2 runs before migration 4 in sequence, the migration2 code may need to stay as-is for existing databases (Claude should handle this ordering carefully)
</specifics>
<deferred>
## Deferred Ideas
None — discussion stayed within phase scope
</deferred>
---
*Phase: 06-sql-consolidation-code-quality*
*Context gathered: 2026-03-04*
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-RESEARCH.md
+528
View File
@@ -0,0 +1,528 @@
# Phase 6: SQL Consolidation & Code Quality - Research
**Researched:** 2026-03-04
**Domain:** SQLite VIEW consolidation, Go codegen, sqlc advanced features
**Confidence:** HIGH
## Summary
Phase 6 eliminates duplicated SQL JOIN patterns, automates Go→TypeScript event synchronization, migrates eligible hand-crafted SQL to sqlc, and documents all intentional sqlc exceptions. The codebase has a well-defined 5-table JOIN pattern (audio_files → recordings → artist_credit → release_group_recordings subquery → release_groups) duplicated across **10+ locations** in both hand-crafted Go SQL and sqlc query files. This pattern can be consolidated into a single SQLite VIEW named `track_metadata`.
Verification confirms that sqlc v1.30.0 (the project's current version) fully supports querying from VIEWs and using `sqlc.slice()` for IN clauses with the SQLite engine — both features were tested directly against the project's toolchain. The event codegen task is straightforward: `go/ast` can parse the 4 const blocks in `events.go` (21 constants) and produce the matching TypeScript `events.ts` output. The existing `codegen-check` lefthook hook currently runs `go generate ./...` which was observed to hang in earlier phases (templ generation timeout), but testing now shows it completes in under 1 second — the fix may simply be wiring the new generator into the existing hook and verifying it works end-to-end.
**Primary recommendation:** Create the `track_metadata` VIEW as migration 4, update all search/rebuild queries to use it, write the event codegen tool using `go/ast`, migrate `lookupChunk` to sqlc with `sqlc.slice()`, and annotate all remaining hand-crafted SQL with `// SAFETY:` comments.
<user_constraints>
## User Constraints (from CONTEXT.md)
### Locked Decisions
- Single VIEW named `track_metadata` with all 16 columns (file_path, length, title, artist, album, track_number, disc_number, genre, year, composer, file_type, sample_rate, bit_depth, channels, bitrate, file_size)
- VIEW rooted on `audio_files` (not `search_index`) so it's usable for both search queries (JOIN search_index to VIEW) and index rebuilds (SELECT directly from VIEW)
- Created as migration 4 (next sequential PRAGMA user_version bump)
- Lightweight search queries (SearchFTS, SearchFTSByFilename) SELECT only the 5 columns they need from the VIEW — SQLite optimizes unused columns away
- RebuildSearchIndex and migration2 INSERT INTO search_index use the VIEW instead of duplicating the JOIN
- Go constants in `backend/events/events.go` are the source of truth
- Generator written in Go, using `go/ast` to parse the const block from events.go
- Wired into `go generate` via `//go:generate` directive on events.go
- Output format matches current `frontend/src/events.ts` structure exactly: `export const Events = { ... } as const;` — zero changes needed in frontend import sites
- Fix the existing `codegen-check` lefthook pre-commit hook (currently hangs) to run the generator and diff the output — fail if TypeScript file is stale
- Note: `LibraryConfigChanged` exists in Go but is missing from TypeScript — the generator will fix this automatically
- Migrate `lookupChunk()` query fully to sqlc: the SELECT + JOINs + `sqlc.slice()` for the IN clause — use the new `track_metadata` VIEW instead of hand-crafted JOINs
- `insertTrackBatch()` (multi-row VALUES with variable row count) stays hand-crafted — sqlc cannot generate variable-length batch INSERTs. Document as exception.
- All FTS5 operations (~11 statements across search.go, library.go, rescan.go) stay hand-crafted — sqlc does not support FTS5 virtual tables (MATCH, rank, content='' tables). Document all as exceptions.
- Format: two parts — WHY sqlc can't handle it AND what makes it safe
- Example: `// SAFETY: FTS5 MATCH syntax unsupported by sqlc. Query is parameterized; no string interpolation.`
- Scope: runtime query SQL only — migration DDL (ALTER TABLE, CREATE INDEX, PRAGMA) does NOT need SAFETY comments
- Per-statement annotation only — no central registry file. The comments ARE the documentation.
- Cross-reference related operations: FTS5 INSERT/DELETE in library.go and rescan.go should reference search.go functions they relate to
### Claude's Discretion
- Exact VIEW column ordering and COALESCE/NULL handling
- Generator CLI interface (flags, output path defaults)
- How to structure the sqlc query file for lookupChunk (naming, placement)
- Exact wording of SAFETY comments (as long as they follow the two-part format)
- How to handle the lefthook codegen-check fix (may need to investigate why it hangs)
### Deferred Ideas (OUT OF SCOPE)
None — discussion stayed within phase scope
</user_constraints>
<phase_requirements>
## Phase Requirements
| ID | Description | Research Support |
|----|-------------|-----------------|
| QUAL-01 | Duplicated FTS5 JOIN pattern (5+ copies) consolidated into single SQLite VIEW | VIEW `track_metadata` verified working with sqlc v1.30.0; 10+ duplicate JOIN sites identified across search.go, database.go, audio_files.sql, playlists.sql, genres.sql, persistence.go |
| QUAL-02 | Event constants generated from Go to TypeScript via codegen, wired into go generate and pre-commit hook | 21 Go constants in 4 const blocks parseable by `go/ast`; TypeScript has 20 (missing `LibraryConfigChanged`); `go generate ./...` completes in <1s; lefthook codegen-check hook exists but needs generator wiring |
| QUAL-03 | Queue batch lookups use sqlc.slice() instead of fmt.Sprintf placeholder construction | `sqlc.slice()` confirmed working with SQLite engine in sqlc v1.30.0 (tested directly); `lookupChunk` in persistence.go is the target; chunking logic must be preserved at caller level |
| QUAL-04 | Hand-crafted SQL exceptions documented with // SAFETY: comments | ~11 FTS5 statements + 1 insertTrackBatch identified; two-part comment format decided |
</phase_requirements>
## Standard Stack
### Core
| Tool | Version | Purpose | Why Standard |
|------|---------|---------|--------------|
| sqlc | v1.30.0 | SQL-to-Go codegen | Already in use (`go tool sqlc`); supports VIEWs and `sqlc.slice()` for SQLite |
| go/ast | stdlib (Go 1.25) | Parse Go const blocks for event codegen | Standard library, no dependencies; reliable AST parsing |
| go/parser | stdlib (Go 1.25) | Parse Go source files | Used with go/ast for the event generator |
| go/token | stdlib (Go 1.25) | Token positions for AST parsing | Required by go/parser |
### Supporting
| Tool | Version | Purpose | When to Use |
|------|---------|---------|-------------|
| lefthook | v1.13.6+ | Pre-commit hook runner | Wire event codegen check into existing `codegen-check` hook |
| modernc.org/sqlite | v1.45.0 | SQLite driver (pure Go) | Already in use; VIEW support is standard SQLite |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| go/ast | Regex parsing of events.go | Fragile, breaks on comments/formatting changes; go/ast is robust |
| SQLite VIEW | Rewrite all queries in sqlc | FTS5 queries can't use sqlc; VIEW gives partial consolidation |
| sqlc.slice() | Keep hand-crafted lookupChunk | sqlc.slice() is cleaner and eliminates manual placeholder construction |
## Architecture Patterns
### VIEW Schema Location
```
backend/database/sql/schemas/
├── ...existing schema files...
└── track_metadata_view.sql # CREATE VIEW IF NOT EXISTS track_metadata
```
The VIEW SQL file goes in the schemas directory so sqlc can see it during code generation. File naming should sort after the tables it depends on (alphabetical ordering puts `track_metadata_view.sql` after all table schemas).
**Important:** `CREATE VIEW IF NOT EXISTS` is the correct DDL for the schema file. The VIEW will also be created by migration 4 for existing databases, but the schema file ensures sqlc knows about it and new databases get it automatically.
### Pattern 1: VIEW Definition
**What:** The `track_metadata` VIEW consolidates the 5-table JOIN into a reusable SQL object
**When to use:** Any query needing audio file metadata with title/artist/album
**Example:**
```sql
-- In backend/database/sql/schemas/track_metadata_view.sql
CREATE VIEW IF NOT EXISTS track_metadata AS
SELECT
af.id,
af.file_path,
af.length_milliseconds,
COALESCE(r.name, '') AS title,
COALESCE(ac.text, '') AS artist_name,
r.track_number,
r.disc_number,
COALESCE(rg.name, '') AS album,
CAST(COALESCE(
(SELECT GROUP_CONCAT(g.name, '||')
FROM recording_genres rg_sub
JOIN genres g ON rg_sub.genre_id = g.id
WHERE rg_sub.recording_id = r.id),
''
) AS TEXT) AS genre,
COALESCE(r.year, 0) AS year,
COALESCE(r.composer, '') AS composer,
COALESCE(ft.extension, '') AS file_type,
af.sample_rate,
af.bit_depth,
af.channels,
af.bitrate,
af.file_size
FROM audio_files af
LEFT JOIN recordings r ON af.recording_id = r.id
LEFT JOIN artist_credit ac ON r.artist_credit_id = ac.id
LEFT JOIN (
SELECT recording_id,
MIN(release_group_id) AS release_group_id
FROM release_group_recordings
GROUP BY recording_id
) rgr ON r.id = rgr.recording_id
LEFT JOIN release_groups rg ON rgr.release_group_id = rg.id
LEFT JOIN file_types ft ON af.file_type_id = ft.id;
```
**Note:** The VIEW includes `af.id` (needed for FTS5 rowid matching and queue lookups). The `id` column is included in the VIEW but queries don't have to select it. It also uses LEFT JOIN throughout (not INNER JOIN) to match the existing pattern — some audio files may have recording_id=0 (no metadata yet).
### Pattern 2: Search Queries Using VIEW
**What:** FTS5 search queries JOIN search_index to the VIEW
**When to use:** SearchFTS, SearchFTSByFilename, SearchFTSTracks
**Example:**
```sql
-- Hand-crafted (stays in search.go — FTS5 MATCH unsupported by sqlc)
SELECT
tm.file_path,
tm.length_milliseconds,
tm.title,
tm.artist_name,
tm.album
FROM search_index si
JOIN track_metadata tm ON tm.id = si.rowid
WHERE search_index MATCH ?
ORDER BY rank
LIMIT ?
```
### Pattern 3: Rebuild Using VIEW
**What:** RebuildSearchIndex selects directly from VIEW
**When to use:** Full FTS5 index rebuild, migration 2 FTS population
**Example:**
```sql
-- Hand-crafted (stays in search.go — FTS5 INSERT unsupported by sqlc)
INSERT INTO search_index(rowid, file_path, title, artist, album)
SELECT id, file_path, title, artist_name, album
FROM track_metadata
```
### Pattern 4: sqlc.slice() for Batch Lookups
**What:** Queue lookupChunk migrated to sqlc query using VIEW + sqlc.slice()
**When to use:** Batch file path lookups in queue persistence
**Example:**
```sql
-- In backend/database/sql/queries/queue.sql (or audio_files.sql)
-- name: LookupTrackMetaBatch :many
SELECT id, file_path, title, artist_name
FROM track_metadata
WHERE file_path IN (sqlc.slice('paths'));
```
**Critical note:** The generated sqlc code does NOT handle chunking — it generates a single query with all placeholders. The caller (`lookupTrackMetaBatch`) must still chunk the paths array at `maxSQLiteVars = 900` before calling the generated method. The chunking loop stays; only the inner SQL construction moves to sqlc.
### Pattern 5: Event Codegen with go/ast
**What:** Go program reads events.go const blocks, generates events.ts
**When to use:** Automated via `//go:generate` directive
**Example structure:**
```go
// backend/events/gen_events_ts.go (or cmd/gen-events/main.go)
package main
import (
"go/ast"
"go/parser"
"go/token"
// ...
)
func main() {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "events.go", nil, parser.ParseComments)
// Walk AST, extract const declarations
// Group by comment blocks (Playback, Queue, Config, Playlist, Library)
// Generate TypeScript output matching current format
}
```
### Anti-Patterns to Avoid
- **Don't put the VIEW in a migration-only file without the schema file:** sqlc needs the VIEW definition in the schema directory to generate code against it. The migration creates it for existing DBs; the schema file teaches sqlc about it.
- **Don't remove chunking from lookupTrackMetaBatch:** `sqlc.slice()` doesn't auto-chunk. SQLite has a bind variable limit (~32766 in newer versions, but the project uses a conservative 900). The chunking loop must remain.
- **Don't try to make FTS5 queries use sqlc:** FTS5 MATCH syntax, `content=''` virtual tables, and rank ordering are unsupported by sqlc's parser. These must stay hand-crafted.
- **Don't change the migration2 code to use the VIEW for DB version < 4:** Migration 2 runs before migration 4 in sequence. For databases upgrading from version 1→4, migration 2 must still work without the VIEW. Only databases already at version ≥ 4 (including fresh DBs) should use the VIEW in the rebuild path.
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| Go AST parsing | Regex/string matching on events.go | `go/ast` + `go/parser` + `go/token` | Handles comments, multiline, formatting robustly |
| SQL IN clause placeholder construction | `fmt.Sprintf` with manual `?` joining | `sqlc.slice()` | Generates correct placeholder expansion; type-safe |
| Duplicate JOIN patterns | Copy-paste SQL across files | SQLite VIEW | Single source of truth; SQLite optimizes unused columns |
**Key insight:** The manual placeholder construction in `lookupChunk` is exactly the pattern `sqlc.slice()` was designed to replace — sqlc generates the same `strings.Replace` / `strings.Repeat` code but with type safety and no manual `args` slice building.
## Common Pitfalls
### Pitfall 1: Migration Ordering with VIEW
**What goes wrong:** migration2 tries to SELECT from `track_metadata` VIEW before migration 4 creates it
**Why it happens:** Migrations run sequentially by version number. A database at version 1 runs migration 2 (which populates FTS) before migration 4 (which creates the VIEW).
**How to avoid:** Keep the existing inline JOIN in `migration2BasenameAndFTS`. Only the `RebuildSearchIndex` function (called at runtime, not during migration) should use the VIEW. The VIEW schema file handles fresh databases; migration 4 handles existing databases.
**Warning signs:** `no such table: track_metadata` error during migration
### Pitfall 2: sqlc Schema File Ordering
**What goes wrong:** sqlc fails to parse the VIEW definition because it references tables not yet defined
**Why it happens:** sqlc processes schema files in filesystem order. If `track_metadata_view.sql` sorts before the tables it references, sqlc can't resolve them.
**How to avoid:** Name the file so it sorts after all dependencies. `track_metadata_view.sql` sorts after `recordings.sql`, `release_groups.sql`, etc. (all start with lowercase letters before 't'). Alternatively, prefix with `zz_` if needed, but alphabetical ordering of `track_metadata_view.sql` already works.
**Warning signs:** sqlc generate errors about unknown tables/columns
### Pitfall 3: VIEW Column Mismatch with Existing Queries
**What goes wrong:** Queries that used INNER JOINs (e.g., `GetAllTracksWithFullMetadata` uses `JOIN recordings r` not `LEFT JOIN`) return different results when switched to the VIEW (which uses LEFT JOINs)
**Why it happens:** The VIEW uses LEFT JOINs to handle audio files without metadata. Existing sqlc queries that use INNER JOINs implicitly filter out unmatched rows.
**How to avoid:** Only replace queries that already use LEFT JOINs (search queries, playlist metadata queries, SearchAudioFilesByBasename). Leave queries with intentional INNER JOINs (like `GetAllTracksWithFullMetadata`) as-is, or add `WHERE r.id IS NOT NULL` to preserve INNER JOIN semantics. Carefully review each query's JOIN type before converting.
**Warning signs:** Extra rows with empty metadata appearing in results
### Pitfall 4: codegen-check Hook Scope
**What goes wrong:** The event generator is added to `go generate` but the codegen-check hook still runs the full `go generate ./...` which includes templ and sqlc, making it slow
**Why it happens:** The hook runs all generators, not just the event one
**How to avoid:** The hook currently runs `go generate ./...` and then diffs. This approach is actually fine — testing shows `go generate ./...` completes in <1 second when nothing has changed. The hanging issue from earlier phases appears to be resolved. Verify the hook works end-to-end after wiring in the new generator.
**Warning signs:** Hook taking >5 seconds (should be <2s)
### Pitfall 5: sqlc.slice() Empty Slice Behavior
**What goes wrong:** Passing an empty slice to a `sqlc.slice()` query
**Why it happens:** The generated code replaces the placeholder with `NULL` for empty slices, which means `WHERE file_path IN (NULL)` — this matches nothing (correct behavior), but the caller should still handle it
**How to avoid:** The chunking logic in `lookupTrackMetaBatch` already handles empty input (returns empty map). The sqlc-generated code also handles empty slices gracefully (returns empty results). No action needed, but be aware of the behavior.
**Warning signs:** N/A — behavior is correct
### Pitfall 6: Generated TypeScript File Must Be Deterministic
**What goes wrong:** The event generator produces different output on different runs (e.g., map iteration order), causing the codegen-check hook to always fail
**Why it happens:** Go maps don't have deterministic iteration order
**How to avoid:** Use `ast.Inspect` or iterate `f.Decls` in source order (AST preserves declaration order). Don't collect into a map and iterate — iterate the AST directly and emit in declaration order.
**Warning signs:** `codegen-check` hook always shows diff even when events.go hasn't changed
## Code Examples
### Example 1: Migration 4 — Create track_metadata VIEW
```sql
-- In migration 4 (backend/database/database.go)
CREATE VIEW IF NOT EXISTS track_metadata AS
SELECT
af.id,
af.file_path,
af.length_milliseconds,
COALESCE(r.name, '') AS title,
COALESCE(ac.text, '') AS artist_name,
r.track_number,
r.disc_number,
COALESCE(rg.name, '') AS album,
CAST(COALESCE(
(SELECT GROUP_CONCAT(g.name, '||')
FROM recording_genres rg_sub
JOIN genres g ON rg_sub.genre_id = g.id
WHERE rg_sub.recording_id = r.id),
''
) AS TEXT) AS genre,
COALESCE(r.year, 0) AS year,
COALESCE(r.composer, '') AS composer,
COALESCE(ft.extension, '') AS file_type,
af.sample_rate,
af.bit_depth,
af.channels,
af.bitrate,
af.file_size
FROM audio_files af
LEFT JOIN recordings r ON af.recording_id = r.id
LEFT JOIN artist_credit ac ON r.artist_credit_id = ac.id
LEFT JOIN (
SELECT recording_id,
MIN(release_group_id) AS release_group_id
FROM release_group_recordings
GROUP BY recording_id
) rgr ON r.id = rgr.recording_id
LEFT JOIN release_groups rg ON rgr.release_group_id = rg.id
LEFT JOIN file_types ft ON af.file_type_id = ft.id;
```
### Example 2: Consolidated SearchFTS Using VIEW
```go
// In search.go — replaces the inline 5-table JOIN
rows, err := d.db.QueryContext(d.Ctx, `
SELECT
tm.file_path,
tm.length_milliseconds,
tm.title,
tm.artist_name,
tm.album
FROM search_index si
JOIN track_metadata tm ON tm.id = si.rowid
WHERE search_index MATCH ?
ORDER BY rank
LIMIT ?
`, ftsQuery, limit)
```
### Example 3: Consolidated RebuildSearchIndex Using VIEW
```go
// In search.go — replaces inline JOIN for rebuild
_, err := d.db.ExecContext(d.Ctx, `
INSERT INTO search_index(rowid, file_path, title, artist, album)
SELECT id, file_path, title, artist_name, album
FROM track_metadata
`)
```
### Example 4: sqlc Query for lookupChunk Replacement
```sql
-- In backend/database/sql/queries/queue.sql (or a new track_metadata.sql)
-- name: LookupTrackMetaByPaths :many
SELECT id, file_path, title, artist_name
FROM track_metadata
WHERE file_path IN (sqlc.slice('paths'));
```
### Example 5: Event Generator Core Logic
```go
// Using go/ast to extract constants from events.go
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, eventsGoPath, nil, parser.ParseComments)
if err != nil {
log.Fatal(err)
}
type eventConst struct {
Name string
Value string
}
var events []eventConst
ast.Inspect(f, func(n ast.Node) bool {
genDecl, ok := n.(*ast.GenDecl)
if !ok || genDecl.Tok != token.CONST {
return true
}
for _, spec := range genDecl.Specs {
vs, ok := spec.(*ast.ValueSpec)
if !ok || len(vs.Names) == 0 || len(vs.Values) == 0 {
continue
}
lit, ok := vs.Values[0].(*ast.BasicLit)
if !ok || lit.Kind != token.STRING {
continue
}
name := vs.Names[0].Name
value := strings.Trim(lit.Value, `"`)
events = append(events, eventConst{Name: name, Value: value})
}
return true
})
```
### Example 6: SAFETY Comment Examples
```go
// SAFETY: FTS5 MATCH syntax unsupported by sqlc. Query is parameterized; no string interpolation.
rows, err := d.db.QueryContext(d.Ctx, `SELECT ... FROM search_index si ... WHERE search_index MATCH ?`, ...)
// SAFETY: FTS5 virtual table INSERT unsupported by sqlc. All values come from track_metadata VIEW; no user input.
_, err := d.db.ExecContext(d.Ctx, `INSERT INTO search_index(rowid, ...) SELECT ... FROM track_metadata`)
// SAFETY: FTS5 virtual table, see search.go:InsertSearchIndex. Parameterized.
_, err := tx.ExecContext(l.ctx, `INSERT INTO search_index(rowid, ...) VALUES (?, ?, ?, ?, ?)`, ...)
// SAFETY: Multi-row INSERT with variable row count unsupported by sqlc. Placeholder count matches args length; no string interpolation.
_, err := tx.ExecContext(q.db.Ctx, query, args...)
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Manual IN clause placeholder | `sqlc.slice()` | sqlc v1.18+ | Type-safe slice parameters for MySQL/SQLite |
| Duplicate JOINs everywhere | SQLite VIEWs | Always available | Single source of truth, optimizer handles unused columns |
| Manual event sync | Codegen from Go→TS | This phase | Eliminates drift (LibraryConfigChanged already missing) |
**Deprecated/outdated:**
- None relevant — all tools are current versions
## Existing Duplicate JOIN Inventory
All locations with the 5-table audio metadata JOIN pattern:
### Hand-Crafted SQL in Go (stay hand-crafted, get SAFETY comments)
| File | Function/Line | Pattern | VIEW Applicable? |
|------|--------------|---------|-----------------|
| `backend/database/search.go:34` | SearchFTS | FTS5 MATCH + 5-table JOIN | Yes — replace JOIN with `JOIN track_metadata` |
| `backend/database/search.go:92` | SearchFTSByFilename | FTS5 MATCH + 5-table JOIN | Yes — replace JOIN with `JOIN track_metadata` |
| `backend/database/search.go:232` | SearchFTSTracks | FTS5 MATCH + 6-table JOIN (+ file_types) | Yes — replace JOIN with `JOIN track_metadata` |
| `backend/database/search.go:168` | RebuildSearchIndex | INSERT INTO FTS from 5-table JOIN | Yes — `SELECT FROM track_metadata` |
| `backend/database/database.go:344` | migration2BasenameAndFTS | INSERT INTO FTS from 5-table JOIN | **No** — must keep inline (runs before migration 4) |
| `backend/library/library.go:798` | commitNewAudioFile | FTS5 INSERT VALUES | No — single-row parameterized insert, no JOIN |
| `backend/library/library.go:879` | updateAudioFileMetadata | FTS5 DELETE + INSERT | No — single-row operations, no JOIN |
| `backend/library/rescan.go:165` | clearAllLibraryData | FTS5 DELETE all | No — simple DELETE, no JOIN |
| `backend/queue/persistence.go:64` | lookupChunk | 3-table JOIN + fmt.Sprintf IN | Yes — migrate to sqlc with VIEW |
| `backend/queue/persistence.go:195` | insertTrackBatch | Multi-row INSERT with variable VALUES | No — stays hand-crafted (no JOINs) |
### sqlc Query Files (already managed by sqlc, may benefit from VIEW)
| File | Query Name | Pattern | VIEW Applicable? |
|------|-----------|---------|-----------------|
| `audio_files.sql:106` | SearchAudioFilesByBasename | 5-table JOIN (same subquery pattern) | Yes — could use VIEW |
| `audio_files.sql:75` | GetAllTracksWithFullMetadata | 6-table JOIN (INNER JOINs) | Partial — uses INNER JOINs (different semantics) |
| `playlists.sql:37` | GetPlaylistTracksWithMetadata | 6-table JOIN + cover_art | Partial — includes cover_art JOIN not in VIEW |
| `playlists.sql:63` | GetAllPlaylistTracksWithMetadata | 6-table JOIN + cover_art | Partial — includes cover_art JOIN not in VIEW |
| `genres.sql:26` | GetTracksByGenre | 7-table JOIN (genre-rooted) | Partial — rooted on genres, not audio_files |
| `queue.sql:15` | GetQueueTracks | 3-table JOIN | Partial — simpler pattern (no rgr subquery) |
### Scope Decision for sqlc Queries
The VIEW consolidation primarily targets the **hand-crafted Go SQL** where the duplication is most problematic (search.go has 3 copies of the identical pattern). For sqlc queries, converting to use the VIEW is optional and should be done case-by-case:
- `SearchAudioFilesByBasename` — good candidate (exact same pattern)
- Playlist/genre queries — involve additional JOINs (cover_art, genre tables) beyond what the VIEW provides, so the benefit is lower
- `GetAllTracksWithFullMetadata` — uses INNER JOINs intentionally, semantics differ from VIEW's LEFT JOINs
## FTS5 Statements Requiring SAFETY Comments
Complete inventory of hand-crafted FTS5 SQL statements:
| # | File | Line | Operation | Comment Needed |
|---|------|------|-----------|---------------|
| 1 | `search.go` | 34 | SearchFTS — `WHERE search_index MATCH ?` | Yes |
| 2 | `search.go` | 92 | SearchFTSByFilename — `WHERE search_index MATCH ?` | Yes |
| 3 | `search.go` | 133 | InsertSearchIndex — `INSERT INTO search_index` | Yes |
| 4 | `search.go` | 143 | DeleteSearchIndex — `DELETE FROM search_index WHERE rowid = ?` | Yes |
| 5 | `search.go` | 152 | ClearSearchIndex — `DELETE FROM search_index` | Yes |
| 6 | `search.go` | 168 | RebuildSearchIndex — `INSERT INTO search_index ... SELECT FROM` | Yes |
| 7 | `search.go` | 232 | SearchFTSTracks — `WHERE search_index MATCH ?` | Yes |
| 8 | `library.go` | 798 | commitNewAudioFile — `INSERT INTO search_index ... VALUES` | Yes (cross-ref search.go) |
| 9 | `library.go` | 879 | updateAudioFileMetadata — `DELETE FROM search_index` | Yes (cross-ref search.go) |
| 10 | `library.go` | 891 | updateAudioFileMetadata — `INSERT INTO search_index ... VALUES` | Yes (cross-ref search.go) |
| 11 | `rescan.go` | 165 | clearAllLibraryData — `DELETE FROM search_index` | Yes (cross-ref search.go) |
| 12 | `persistence.go` | 195 | insertTrackBatch — multi-row `INSERT INTO queue_tracks` | Yes (variable VALUES count) |
## Event Constant Inventory
### Go (backend/events/events.go) — 21 constants in 4 blocks
```
Playback: PlaybackStateChanged, PlaybackFinished, TrackChanged, SeekFailed, VolumeChanged
Queue: QueueChanged, QueueIndexChanged, QueueModeChanged, QueueTracksModified
Config: LibraryConfigChanged, ThemeConfigChanged, TrackListConfigChanged, FavoritesConfigChanged
Playlist: PlaylistCreated, PlaylistDeleted, PlaylistRenamed, PlaylistTracksChanged, PlaylistsRestored, DefaultPlaylistChanged
Library: LibraryScanStarted, LibraryScanComplete
```
### TypeScript (frontend/src/events.ts) — 20 constants
Missing: `LibraryConfigChanged` (exists in Go, absent from TypeScript)
### Generator Output Format Target
```typescript
export const Events = {
// Playback events (backend → frontend push)
PlaybackStateChanged: "PlaybackStateChanged",
// ... preserving comment groups and ordering
} as const;
export type EventName = (typeof Events)[keyof typeof Events];
```
## Open Questions
1. **Should sqlc queries (SearchAudioFilesByBasename, etc.) also be updated to use the VIEW?**
- What we know: The VIEW consolidation is primarily targeting hand-crafted Go SQL in search.go. Sqlc queries are already managed and less prone to drift.
- What's unclear: Whether updating sqlc queries provides enough benefit to justify the churn and testing.
- Recommendation: Update `SearchAudioFilesByBasename` (exact same pattern). Leave playlist/genre queries as-is (they have additional JOINs the VIEW doesn't cover). This is Claude's discretion per CONTEXT.md.
2. **Where should the event generator Go file live?**
- What we know: It needs to be a `main` package (standalone executable for `go:generate`). Options: `backend/events/cmd/gen-events-ts/main.go` or `cmd/gen-events-ts/main.go` or inline in `backend/events/`.
- What's unclear: Project convention for codegen tools (none exist yet).
- Recommendation: `backend/events/cmd/genevents/main.go` — keeps it close to the source of truth. The `//go:generate` directive on events.go runs it.
3. **codegen-check hook — is it actually fixed?**
- What we know: `go generate ./...` now completes in <1 second in testing. Previous hanging was during Phase 2 (Feb 2026).
- What's unclear: Whether the fix was a templ version update, environment change, or something else.
- Recommendation: After wiring the event generator, test the full hook manually (`lefthook run pre-commit`) before declaring it fixed. If it still hangs, narrow the hook scope to only run event codegen check (not full `go generate ./...`).
## Sources
### Primary (HIGH confidence)
- sqlc v1.30.0 official docs — [select.html#mysql-and-sqlite](https://docs.sqlc.dev/en/stable/howto/select.html#mysql-and-sqlite) — `sqlc.slice()` syntax and generated code
- sqlc v1.30.0 official docs — [ddl.html](https://docs.sqlc.dev/en/stable/howto/ddl.html) — Schema handling including VIEWs
- Direct verification: `go tool sqlc generate` tested with VIEW + `sqlc.slice()` against project's sqlc v1.30.0 — both work correctly
- Go stdlib `go/ast`, `go/parser`, `go/token` documentation — standard library, stable API
### Secondary (MEDIUM confidence)
- Codebase analysis: 10+ duplicate JOIN instances identified by grep across .go and .sql files
- lefthook.yml examination: `codegen-check` hook structure and `go generate ./...` command
- `go generate ./...` timing test: completes in <1s (2 templ + 1 sqlc generators, all no-op)
### Tertiary (LOW confidence)
- None — all findings verified against primary sources or direct testing
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH — sqlc v1.30.0 verified directly; go/ast is stable stdlib
- Architecture: HIGH — VIEW + sqlc.slice() both tested against project toolchain
- Pitfalls: HIGH — migration ordering verified by reading database.go; JOIN semantics verified by reading query files
**Research date:** 2026-03-04
**Valid until:** 2026-04-04 (stable tools, no fast-moving dependencies)
.planning/milestones/v1.0-phases/06-sql-consolidation-code-quality/06-VERIFICATION.md
+95
View File
@@ -0,0 +1,95 @@
---
phase: 06-sql-consolidation-code-quality
verified: 2026-03-04T23:45:00Z
status: passed
score: 4/4 must-haves verified
re_verification: false
---
# Phase 6: SQL Consolidation & Code Quality Verification Report
**Phase Goal:** Duplicated SQL patterns are eliminated, event names are provably synchronized between Go and TypeScript, and intentional SQL exceptions are documented
**Verified:** 2026-03-04T23:45:00Z
**Status:** passed
**Re-verification:** No — initial verification
## Goal Achievement
### Observable Truths
| # | Truth | Status | Evidence |
|---|-------|--------|----------|
| 1 | The duplicated 5-table FTS5 JOIN pattern is consolidated into a single SQLite VIEW (`track_metadata`), and all search queries use the VIEW instead of inline JOINs | ✓ VERIFIED | `track_metadata_view.sql` has full VIEW definition (37 lines). `search.go` has 5 `track_metadata` references and 0 `LEFT JOIN recordings`. Migration 4 registered in `database.go` with `CREATE VIEW IF NOT EXISTS track_metadata`. Migration 2 inline JOIN intentionally preserved (2 `LEFT JOIN recordings` in database.go). |
| 2 | A code generator reads Go event constants from `backend/events/events.go` and produces `frontend/src/events.ts`, wired into `go generate` and the pre-commit hook — adding an event in Go without regenerating TypeScript fails the hook | ✓ VERIFIED | `genevents/main.go` exists (166 lines), uses `go/ast`, `go/parser`, `go/token`. `events.go` has `//go:generate go run ./cmd/genevents -source events.go -output ../../frontend/src/events.ts`. `events.ts` has "Code generated by genevents" header, 21 constants (matches Go's 21), includes `LibraryConfigChanged`. `lefthook.yml` codegen-check runs `go generate ./...` and fails on diff. |
| 3 | Queue batch lookups in `persistence.go` use `sqlc.slice()` for IN clauses where sqlc supports it, replacing `fmt.Sprintf` placeholder construction | ✓ VERIFIED | `audio_files.sql` has `LookupTrackMetaByPaths` query with `sqlc.slice('paths')`. `persistence.go` `lookupChunk` calls `q.db.Queries.LookupTrackMetaByPaths`. `fmt.Sprintf` count in persistence.go is 0. sqlc-generated `audio_files.sql.go` has `LookupTrackMetaByPaths` function. Chunking preserved at `maxSQLiteVars`. |
| 4 | Every hand-crafted SQL statement that intentionally bypasses sqlc has a `// SAFETY:` comment explaining why (batch INSERT, dynamic IN clauses, etc.) | ✓ VERIFIED | Exactly 12 `// SAFETY:` comments found across 4 files: 7 in `search.go`, 3 in `library.go`, 1 in `rescan.go`, 1 in `persistence.go`. All follow two-part format (reason + safety assurance). Cross-references from library.go/rescan.go back to search.go. |
**Score:** 4/4 truths verified
### Required Artifacts
| Artifact | Expected | Status | Details |
|----------|----------|--------|---------|
| `backend/database/sql/schemas/track_metadata_view.sql` | VIEW definition for sqlc schema awareness | ✓ VERIFIED | 37-line file with `CREATE VIEW IF NOT EXISTS track_metadata` consolidating 5-table JOIN with all 16 columns |
| `backend/database/database.go` | Migration 4 creating VIEW for existing databases | ✓ VERIFIED | `migration4TrackMetadataView` function registered, sets `user_version = 4`, VIEW SQL matches schema file |
| `backend/database/search.go` | Consolidated search queries using VIEW | ✓ VERIFIED | All 4 search functions (SearchFTS, SearchFTSByFilename, SearchFTSTracks, RebuildSearchIndex) use `JOIN track_metadata tm`, 7 SAFETY comments |
| `backend/events/cmd/genevents/main.go` | Go→TypeScript event constant generator | ✓ VERIFIED | 166-line program using go/ast, parses declaration order, writes atomically, strips trailing periods |
| `backend/events/events.go` | go:generate directive for event codegen | ✓ VERIFIED | `//go:generate go run ./cmd/genevents -source events.go -output ../../frontend/src/events.ts` |
| `frontend/src/events.ts` | Generated TypeScript event constants | ✓ VERIFIED | Generated header present, 21 constants matching Go source, includes LibraryConfigChanged, `EventName` type exported |
| `backend/database/sql/queries/audio_files.sql` | sqlc query for batch track metadata lookup | ✓ VERIFIED | `LookupTrackMetaByPaths` query using `track_metadata` VIEW with `sqlc.slice('paths')` |
| `backend/queue/persistence.go` | Updated lookupChunk using sqlc-generated query | ✓ VERIFIED | `lookupChunk` calls `LookupTrackMetaByPaths`, no fmt.Sprintf, SAFETY comment on `insertTrackBatch` |
| `backend/library/library.go` | SAFETY comments on FTS5 operations | ✓ VERIFIED | 3 SAFETY comments (lines 796, 878, 893) cross-referencing search.go |
| `backend/library/rescan.go` | SAFETY comment on FTS5 delete operation | ✓ VERIFIED | 1 SAFETY comment (line 164) cross-referencing search.go:ClearSearchIndex |
| `backend/database/sql/sqlcgen/audio_files.sql.go` | sqlc-generated Go code | ✓ VERIFIED | `LookupTrackMetaByPaths` function, `LookupTrackMetaByPathsRow` struct generated |
### Key Link Verification
| From | To | Via | Status | Details |
|------|----|-----|--------|---------|
| `search.go` | `track_metadata` VIEW | `JOIN track_metadata tm ON tm.id = si.rowid` | ✓ WIRED | All 4 search functions use VIEW; RebuildSearchIndex also selects from VIEW directly |
| `database.go` | `track_metadata` VIEW | Migration 4 CREATE VIEW | ✓ WIRED | `migration4TrackMetadataView` creates VIEW, registered in migration sequence after migration 3 |
| `events.go` | `events.ts` | `//go:generate go run ./cmd/genevents` | ✓ WIRED | Directive present, output file has generated header and all 21 constants |
| `lefthook.yml` | `go generate` | codegen-check pre-commit hook | ✓ WIRED | Hook runs `go generate ./...`, checks `git diff --name-only`, fails on stale generated code |
| `persistence.go` | `sqlcgen/` | `LookupTrackMetaByPaths` query | ✓ WIRED | `lookupChunk` calls `q.db.Queries.LookupTrackMetaByPaths(q.db.Ctx, paths)` |
| `audio_files.sql` | `track_metadata` VIEW | `SELECT FROM track_metadata WHERE file_path IN (sqlc.slice)` | ✓ WIRED | Query references VIEW and uses `sqlc.slice('paths')` for variable-length IN clause |
### Requirements Coverage
| Requirement | Source Plan | Description | Status | Evidence |
|-------------|------------|-------------|--------|----------|
| QUAL-01 | 06-01 | Duplicated FTS5 JOIN consolidated into SQLite VIEW | ✓ SATISFIED | VIEW schema exists, migration 4 creates it, all search queries use it, 0 inline JOINs remain in search.go |
| QUAL-02 | 06-02 | Event names generated from Go to TypeScript via codegen | ✓ SATISFIED | genevents tool exists, go:generate directive wired, 21/21 constants synced, LibraryConfigChanged gap fixed, pre-commit hook detects drift |
| QUAL-03 | 06-03 | Queue batch lookups use sqlc.slice() for IN clauses | ✓ SATISFIED | LookupTrackMetaByPaths uses sqlc.slice, lookupChunk calls sqlc-generated query, fmt.Sprintf eliminated |
| QUAL-04 | 06-03 | Hand-crafted SQL exceptions documented with SAFETY comments | ✓ SATISFIED | 12/12 SAFETY comments across 4 files, two-part format, cross-references |
No orphaned requirements — all 4 QUAL requirements mapped to this phase are accounted for in plans and verified.
### Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
|------|------|---------|----------|--------|
| — | — | — | — | No anti-patterns found |
No TODO/FIXME/placeholder/empty-implementation patterns detected in any modified files.
### Human Verification Required
No items require human verification. All success criteria are programmatically verifiable:
- VIEW definition and migration are structural code
- Event constant count matching is numeric
- SAFETY comment presence is textual
- sqlc.slice usage is code-level
### Gaps Summary
No gaps found. All 4 success criteria are fully verified:
1. **VIEW consolidation** — track_metadata VIEW exists, migration 4 registered, all search queries use VIEW, 0 duplicated inline JOINs remain
2. **Event codegen** — genevents parses Go AST, generates matching TypeScript, go:generate wired, pre-commit hook runs `go generate ./...` and fails on drift, 21/21 constants including previously-missing LibraryConfigChanged
3. **sqlc.slice migration** — lookupChunk uses sqlc-generated LookupTrackMetaByPaths, fmt.Sprintf eliminated, chunking preserved
4. **SAFETY documentation** — 12/12 hand-crafted SQL statements documented with two-part SAFETY comments
---
_Verified: 2026-03-04T23:45:00Z_
_Verifier: Claude (gsd-verifier)_