From abaf46ef1f4694a837802d596645fad6cb93f1bb Mon Sep 17 00:00:00 2001 From: Caleb Allen Date: Mon, 2 Mar 2026 21:21:52 -0500 Subject: [PATCH] docs(03): create phase plan --- .planning/ROADMAP.md | 6 +- .../03-test-infrastructure/03-01-PLAN.md | 207 ++++++++++++++++++ 2 files changed, 211 insertions(+), 2 deletions(-) create mode 100644 .planning/phases/03-test-infrastructure/03-01-PLAN.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index a789c31..8b7688c 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -55,7 +55,9 @@ Plans: 2. Production SQLite connection applies `synchronous=NORMAL`, `cache_size=-8000`, and `mmap_size=67108864` PRAGMAs at database open 3. Each test gets an isolated database instance — no shared state between test functions 4. Tests using `NewTestDB` pass with `-race` flag enabled -**Plans:** TBD +**Plans:** 1 plan +Plans: +- [ ] 03-01-PLAN.md — Extract shared applyPRAGMAs, add production PRAGMAs, and create NewTestDB helper ### Phase 4: Queue, Config & Player Tests **Goal:** The queue, config, and player packages have comprehensive unit tests that characterize current behavior and serve as a safety net for later refactoring @@ -117,7 +119,7 @@ Plans: |-------|----------------|--------|-----------| | 1. Concurrency Race Fixes | 1/1 | Complete | 2026-02-28 | | 2. Backend Correctness | 2/2 | Complete | 2026-03-03 | -| 3. Test Infrastructure | 0/? | Not started | — | +| 3. Test Infrastructure | 0/1 | Planned | — | | 4. Queue, Config & Player Tests | 0/? | Not started | — | | 5. Database & Library Tests | 0/? | Not started | — | | 6. SQL Consolidation & Code Quality | 0/? | Not started | — | diff --git a/.planning/phases/03-test-infrastructure/03-01-PLAN.md b/.planning/phases/03-test-infrastructure/03-01-PLAN.md new file mode 100644 index 0000000..da70e57 --- /dev/null +++ b/.planning/phases/03-test-infrastructure/03-01-PLAN.md @@ -0,0 +1,207 @@ +--- +phase: 03-test-infrastructure +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - backend/database/database.go + - backend/database/testhelper.go +autonomous: true +requirements: + - TEST-01 + - PERF-04 + +must_haves: + truths: + - "Production SQLite connection applies synchronous=NORMAL, cache_size=-8000, and mmap_size=67108864 PRAGMAs at database open" + - "NewTestDB(t) returns a clean in-memory SQLite DB with the same migrations and PRAGMAs as production NewDB()" + - "Each test gets an isolated database instance — no shared state between test functions" + - "Tests using NewTestDB pass with -race flag enabled" + artifacts: + - path: "backend/database/database.go" + provides: "Shared applyPRAGMAs function + production PRAGMA application in NewDB" + contains: "applyPRAGMAs" + - path: "backend/database/testhelper.go" + provides: "NewTestDB test helper for in-memory SQLite with production-mirror setup" + exports: ["NewTestDB"] + key_links: + - from: "backend/database/testhelper.go" + to: "backend/database/database.go" + via: "shared applyPRAGMAs function" + pattern: "applyPRAGMAs\\(" + - from: "backend/database/testhelper.go" + to: "backend/database/database.go" + via: "shared schema application (schemas embed + runMigrations)" + pattern: "schemas\\.ReadDir|runMigrations" +--- + + +Create a production-mirroring SQLite test helper and apply performance PRAGMAs to the production database connection. + +Purpose: Establish the test foundation that all subsequent test phases (4-5) depend on. Tests need real database instances with identical configuration to production — same PRAGMAs, same migrations, same constraints — so test results are trustworthy. + +Output: Modified `database.go` with shared PRAGMA function + production PRAGMAs applied, and new `testhelper.go` with `NewTestDB(t)`. + + + +@/home/caleb/.config/opencode/get-shit-done/workflows/execute-plan.md +@/home/caleb/.config/opencode/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md + +@backend/database/database.go + + + + + +From backend/database/database.go: +```go +// DB wraps the SQLite database connection and queries. +type DB struct { + db *sql.DB + Ctx context.Context + Queries *sqlcgen.Queries + logger *slog.Logger +} + +// NewDB opens the database and applies schema migrations. +func NewDB(logger *slog.Logger) (*DB, error) + +// BeginTx starts a new database transaction. +func (d *DB) BeginTx() (*sql.Tx, error) + +// ExecContext executes a query without returning any rows. +func (d *DB) ExecContext(query string, args ...any) (sql.Result, error) + +// QueryContext executes a query that returns rows. +func (d *DB) QueryContext(query string, args ...any) (*sql.Rows, error) +``` + +From backend/database/database.go (internal): +```go +//go:embed sql/schemas/*.sql +var schemas embed.FS + +func runMigrations(ctx context.Context, db *sql.DB, logger *slog.Logger) error +func isDuplicateColumnErr(err error) bool +``` + + + + + + Task 1: Extract shared applyPRAGMAs and add production PRAGMAs to NewDB + backend/database/database.go + + In `backend/database/database.go`: + + 1. Create an unexported `applyPRAGMAs(ctx context.Context, db *sql.DB) error` function that executes these PRAGMAs in order: + - `PRAGMA foreign_keys = ON` (already exists in NewDB — extract it) + - `PRAGMA synchronous = NORMAL` + - `PRAGMA cache_size = -8000` + - `PRAGMA mmap_size = 67108864` + + Use a slice of PRAGMA strings and loop over them with `db.ExecContext`. Wrap errors with `fmt.Errorf("could not apply PRAGMA %q: %w", pragma, err)`. + + 2. Modify `NewDB()` to call `applyPRAGMAs(dbCtx, db)` instead of the inline `PRAGMA foreign_keys = ON` exec. Insert the call right after `db.SetMaxOpenConns(1)` — PRAGMAs before schema creation, per CONTEXT.md decision. + + 3. Remove the standalone `foreign_keys` PRAGMA block that currently exists in `NewDB()` (lines 58-65) since it's now handled by `applyPRAGMAs`. + + 4. Add a doc comment on `applyPRAGMAs`: `// applyPRAGMAs configures SQLite connection settings. Called by both NewDB and NewTestDB to ensure identical behavior.` + + Follow existing conventions: error wrapping with `fmt.Errorf`, blank line after early returns (`nlreturn`), keep lines under 100 chars (`golines`). + + + cd /mnt/vault/dev/golang/yellowjacket && go build -tags webkit2_41 ./backend/database/ && go vet -tags webkit2_41 ./backend/database/ + + + - `applyPRAGMAs` function exists in `database.go` with all 4 PRAGMAs (foreign_keys, synchronous, cache_size, mmap_size) + - `NewDB()` calls `applyPRAGMAs` instead of inline foreign_keys PRAGMA + - Package compiles and passes vet + + + + + Task 2: Create NewTestDB helper in testhelper.go + backend/database/testhelper.go + + Create `backend/database/testhelper.go` with: + + 1. Package declaration: `package database` + + 2. Imports: `context`, `database/sql`, `fmt`, `io/fs`, `log/slog`, `path`, `testing`, `modernc.org/sqlite` (blank import for driver), and `yellowjacket/backend/database/sql/sqlcgen`. + + 3. Exported function `NewTestDB(t *testing.T) *DB`: + - Call `t.Helper()` at the start + - Open in-memory SQLite: `sql.Open("sqlite", ":memory:?_busy_timeout=5000&_journal_mode=WAL")` + - If open fails, `t.Fatalf("could not open test database: %v", err)` + - `db.SetMaxOpenConns(1)` — same as production + - Create context: `ctx := t.Context()` (use `t.Context()` per usetesting linter) + - Call `applyPRAGMAs(ctx, db)` — if error, `t.Fatalf("could not apply PRAGMAs: %v", err)` + - Apply schemas: iterate `schemas.ReadDir("sql/schemas")`, read each .sql file, `db.ExecContext(ctx, string(sqlContent))` — mirror the exact loop from `NewDB()`. If error, `t.Fatalf`. + - Call `runMigrations(ctx, db, slog.Default())` — if error, `t.Fatalf("could not run migrations: %v", err)` + - Do NOT run orphan cleanup query (CONTEXT.md decision: "test DBs start empty, no orphans to clean") + - Create queries: `queries := sqlcgen.New(db)` + - Register cleanup: `t.Cleanup(func() { db.Close() })` + - Return `&DB{db: db, Ctx: ctx, Queries: queries, logger: slog.Default()}` + + 4. Add doc comment: `// NewTestDB returns an in-memory SQLite database that mirrors the production setup (PRAGMAs + all migrations). The database is automatically closed when the test completes via t.Cleanup.` + + Note: Do NOT expose raw `*sql.DB` — tests use `DB.ExecContext()` / `DB.Queries` like production code (per CONTEXT.md decision). No functional options. No error return — failures are fatal via `t.Fatalf`. + + Follow conventions: `t.Helper()`, `t.Context()`, blank import comment, doc comments ending with period, `nlreturn` spacing. + + + cd /mnt/vault/dev/golang/yellowjacket && go build -tags webkit2_41 ./backend/database/ && go vet -tags webkit2_41 ./backend/database/ && go test -tags webkit2_41 -race -count=1 -run TestNewTestDB ./backend/database/ 2>&1 || echo "No test yet — build+vet passed" + + + - `backend/database/testhelper.go` exists with exported `NewTestDB(t *testing.T) *DB` + - Function opens `:memory:` DB, applies PRAGMAs via shared `applyPRAGMAs`, applies schemas, runs migrations + - No orphan cleanup, no health check, no error return, no functional options + - Cleanup registered via `t.Cleanup()` + - Package compiles, passes vet, and passes `-race` flag + + + + + + +After both tasks complete, run the full verification: + +```bash +# 1. Build the database package +go build -tags webkit2_41 ./backend/database/ + +# 2. Vet the database package +go vet -tags webkit2_41 ./backend/database/ + +# 3. Run all existing tests with race detector to confirm no regressions +make test + +# 4. Verify applyPRAGMAs is called from both NewDB and NewTestDB +grep -n "applyPRAGMAs" backend/database/database.go backend/database/testhelper.go + +# 5. Verify production PRAGMAs are all present +grep -c "PRAGMA" backend/database/database.go +``` + + + +1. `backend/database/database.go` has a shared `applyPRAGMAs` function with all 4 PRAGMAs +2. `NewDB()` calls `applyPRAGMAs` (no more inline foreign_keys PRAGMA) +3. `backend/database/testhelper.go` exports `NewTestDB(t *testing.T) *DB` +4. `NewTestDB` uses `:memory:` with same connection params, calls `applyPRAGMAs` + schema loop + `runMigrations` +5. `NewTestDB` registers `t.Cleanup(func() { db.Close() })` +6. `make test` passes (all existing tests green, race detector clean) +7. No orphan cleanup in `NewTestDB`, no health check, no functional options + + + +After completion, create `.planning/phases/03-test-infrastructure/03-01-SUMMARY.md` +