yellowjacket/.planning/milestones/v1.2-phases/15-schema-migration-write-safety/15-01-PLAN.md

---
phase: 15-schema-migration-write-safety
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - backend/database/sql/schemas/search_index.sql
  - backend/database/database.go
  - backend/database/search.go
  - backend/database/search_test.go
  - backend/library/library.go
autonomous: true
requirements: [SCHEMA-01]

must_haves:
  truths:
    - "FTS5 search_index uses contentless_delete=1 after migration 8"
    - "DeleteSearchIndex performs a real DELETE for individual rows"
    - "Existing search queries return identical results after migration"
    - "Migration is idempotent — safe to re-run if interrupted"
    - "ClearSearchIndex still works for full rebuilds"
  artifacts:
    - path: "backend/database/sql/schemas/search_index.sql"
      provides: "Updated FTS5 schema with contentless_delete=1"
      contains: "contentless_delete=1"
    - path: "backend/database/database.go"
      provides: "Migration 8 function"
      contains: "migration8"
    - path: "backend/database/search.go"
      provides: "Real DeleteSearchIndex implementation"
      exports: ["DeleteSearchIndex"]
    - path: "backend/database/search_test.go"
      provides: "Tests for delete, insert-update cycle, and search correctness"
      min_lines: 50
  key_links:
    - from: "backend/database/database.go"
      to: "backend/database/search.go"
      via: "migration 8 calls RebuildSearchIndex"
      pattern: "RebuildSearchIndex"
    - from: "backend/database/search.go"
      to: "backend/database/sql/schemas/search_index.sql"
      via: "ClearSearchIndex CREATE statement matches schema file"
      pattern: "contentless_delete=1"
    - from: "backend/library/library.go"
      to: "backend/database/search.go"
      via: "library calls InsertSearchIndex and DeleteSearchIndex"
      pattern: "DeleteSearchIndex"
---

<objective>
Migrate FTS5 search_index to contentless_delete=1 and implement real row-level DELETE support.

Purpose: Currently, DeleteSearchIndex is a no-op because contentless FTS5 tables cannot delete rows. After adding `contentless_delete=1`, individual rows can be deleted/updated — a prerequisite for inline tag edit → DB sync in Phase 16.

Output: Migration 8 function, updated schema, real DeleteSearchIndex, passing tests.
</objective>

<execution_context>
@/home/caleb/.config/opencode/get-shit-done/workflows/execute-plan.md
@/home/caleb/.config/opencode/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/15-schema-migration-write-safety/15-CONTEXT.md

@backend/database/database.go
@backend/database/search.go
@backend/database/search_test.go
@backend/database/sql/schemas/search_index.sql
@backend/library/library.go

<interfaces>
<!-- Key types and contracts the executor needs. Extracted from codebase. -->

From backend/database/database.go:
```go
type DB struct {
    db      *sql.DB
    Ctx     context.Context
    Queries *sqlcgen.Queries
    logger  *slog.Logger
}

func (d *DB) runMigrations() error  // sequential if version < N blocks
// Current: PRAGMA user_version ends at 7
// Migration 2 (migration2BasenameAndFTS) rebuilds FTS5 on startup
```

From backend/database/search.go:
```go
func (d *DB) InsertSearchIndex(rowid int64, filePath, title, artist, album string) error
func (d *DB) DeleteSearchIndex(_ int64) error  // CURRENT: no-op, discards rowid
func (d *DB) ClearSearchIndex() error           // DROP + recreate FTS5 table
func (d *DB) RebuildSearchIndex() error          // ClearSearchIndex + bulk insert from track_metadata
func (d *DB) SearchFTS(query string) ([]SearchResult, error)
func (d *DB) SearchFTSByFilename(query string) ([]SearchResult, error)
func (d *DB) SearchFTSTracks(query string) ([]Track, error)
func (d *DB) SearchFTSTracksByLibrary(query string, libraryID int64) ([]Track, error)
```

From backend/database/search_test.go:
```go
func seedSearchData(t *testing.T, db *DB)  // Seeds 7 tracks with full FK chains
// Tests use NewTestDB(t), t.Parallel(), t.Errorf/t.Fatalf patterns
```

From backend/library/library.go (raw FTS5 SQL):
```go
// Line ~1013-1017: INSERT INTO search_index(rowid, file_path, title, artist, album) VALUES (?, ?, ?, ?, ?)
// Line ~1100-1103: Same INSERT pattern for metadata updates
// Line ~1080-1084: Comment explaining stale FTS entries are harmless
```
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Migrate FTS5 schema and add migration 8</name>
  <files>
    backend/database/sql/schemas/search_index.sql
    backend/database/database.go
    backend/database/search.go
    backend/library/library.go
  </files>
  <action>
**1. Update the FTS5 schema file** (`backend/database/sql/schemas/search_index.sql`):

Change `content=''` to `content='', contentless_delete=1`. The full CREATE statement becomes:

```sql
CREATE VIRTUAL TABLE IF NOT EXISTS search_index USING fts5(
    file_path,
    title,
    artist,
    album,
    content='',
    contentless_delete=1,
    tokenize='unicode61 remove_diacritics 2'
);
```

Note: `content=''` is still required — `contentless_delete=1` is an addition, not a replacement. Both options must be present together per SQLite docs.

**2. Update ClearSearchIndex** in `backend/database/search.go`:

Update the inline CREATE VIRTUAL TABLE statement in ClearSearchIndex to match the schema file exactly (add `contentless_delete=1`). This is the second place the FTS5 schema is defined.

**3. Implement real DeleteSearchIndex** in `backend/database/search.go`:

Replace the no-op with a real implementation. With `contentless_delete=1`, the correct DELETE syntax is:

```go
func (d *DB) DeleteSearchIndex(rowid int64) error {
    _, err := d.db.ExecContext(d.Ctx,
        `DELETE FROM search_index WHERE rowid = ?`, rowid,
    )
    if err != nil {
        return fmt.Errorf("could not delete search index entry: %w", err)
    }

    return nil
}
```

Update the doc comment to remove the "no-op" explanation and document the new behavior.

**4. Add migration 8** to `runMigrations()` in `backend/database/database.go`:

Add a new `if version < 8` block after the existing migration 7 block. The migration must:

- Call `d.ClearSearchIndex()` to DROP the old `content=''` table
- The schema file (already embedded and applied at startup before migrations) creates the new `content='', contentless_delete=1` table — BUT since schemas run first, the old table already exists and `IF NOT EXISTS` skips the creation. So the migration needs to explicitly DROP and recreate.
- After dropping, recreate using the new schema. Don't call ClearSearchIndex here (which has the updated schema) — instead, drop the table and let `RebuildSearchIndex()` handle both recreate + repopulate:

```go
if version < 8 {
    d.logger.Info("migration 8: rebuilding FTS5 search_index with contentless_delete=1")

    if err := d.RebuildSearchIndex(); err != nil {
        return fmt.Errorf("migration 8: could not rebuild search index: %w", err)
    }

    if _, err := d.db.ExecContext(d.Ctx,
        `PRAGMA user_version = 8`,
    ); err != nil {
        return fmt.Errorf("migration 8: could not set user_version: %w", err)
    }
}
```

This is naturally idempotent per the CONTEXT.md decision — if interrupted, re-running drops and rebuilds again.

**5. Update library.go raw SQL comments** in `backend/library/library.go`:

Around lines 1080-1084, update the comment that says "stale entries are harmless" to note that `DeleteSearchIndex` now works and Phase 16 will use it for inline updates. The INSERT statements themselves don't change — they already use the correct column names and rowid binding.

**What to avoid:** Do NOT change any column names in the FTS5 table (file_path, title, artist, album). Do NOT modify the tokenizer. Do NOT change InsertSearchIndex or any search query SQL — the only changes are to the table options and DeleteSearchIndex.
  </action>
  <verify>
    <automated>cd /mnt/vault/dev/golang/yellowjacket && go test -tags webkit2_41 -run TestSearch -count=1 -timeout 30s ./backend/database/ && go test -tags webkit2_41 -run TestMigration -count=1 -timeout 30s ./backend/database/</automated>
  </verify>
  <done>
    - search_index.sql contains `contentless_delete=1`
    - ClearSearchIndex CREATE statement matches schema file
    - DeleteSearchIndex performs a real DELETE (not a no-op)
    - Migration 8 exists and sets PRAGMA user_version = 8
    - All existing search tests pass unchanged (SearchFTS, SearchFTSByFilename, etc.)
    - `go vet -tags webkit2_41 ./backend/database/` and `go vet -tags webkit2_41 ./backend/library/` pass
  </done>
</task>

<task type="auto">
  <name>Task 2: Add tests for FTS5 row deletion and update cycle</name>
  <files>
    backend/database/search_test.go
  </files>
  <action>
Add new test functions to `backend/database/search_test.go` that verify the new DeleteSearchIndex behavior and the insert-delete-reinsert cycle needed for tag editing.

**Tests to add:**

1. **TestDeleteSearchIndex** — Table-driven test:
   - Seed data with `seedSearchData(t, db)` (7 tracks)
   - Delete one row by rowid
   - Verify searching for that track's title returns no results
   - Verify searching for other tracks still works
   - Cases: delete existing rowid (success), delete non-existent rowid (no error — DELETE WHERE with no match is fine in SQLite)

2. **TestSearchIndexUpdateCycle** — Simulates tag edit flow:
   - Insert a track into search_index with rowid=100, title="Old Title", artist="Old Artist"
   - Verify search for "Old Title" returns rowid 100
   - Delete rowid 100 from search_index
   - Verify search for "Old Title" returns no results
   - Re-insert rowid 100 with title="New Title", artist="New Artist"
   - Verify search for "New Title" returns rowid 100
   - Verify search for "Old Title" returns no results (no ghost/stale entries)

3. **TestClearSearchIndexPreservesSchema** — Verify ClearSearchIndex still works:
   - Seed data
   - Call ClearSearchIndex()
   - Verify search returns no results
   - Insert new data
   - Verify search works again (table was recreated with correct schema including contentless_delete=1)

All tests must follow existing patterns:
- Use `t.Parallel()` at top level
- Use `NewTestDB(t)` for DB setup
- Use `t.Errorf` / `t.Fatalf` (no assertion libraries)
- Use `seedSearchData(t, db)` where appropriate

Note: The seedSearchData helper creates full FK chains (audio_files → recordings → artists → etc.) that satisfy the track_metadata VIEW's JOINs. For TestSearchIndexUpdateCycle, you'll need to insert a minimal audio_file + recording chain to have valid data in track_metadata for the search JOIN. Look at seedSearchData for the exact pattern.
  </action>
  <verify>
    <automated>cd /mnt/vault/dev/golang/yellowjacket && go test -tags webkit2_41 -v -run "TestDeleteSearchIndex|TestSearchIndexUpdateCycle|TestClearSearchIndexPreservesSchema" -count=1 -timeout 30s ./backend/database/</automated>
  </verify>
  <done>
    - TestDeleteSearchIndex passes — deleting a row removes it from search results
    - TestSearchIndexUpdateCycle passes — delete + reinsert produces no ghost entries
    - TestClearSearchIndexPreservesSchema passes — drop/recreate preserves new schema
    - All existing search_test.go tests continue to pass
    - `make test` passes (full test suite)
  </done>
</task>

</tasks>

<verification>
1. `make test` — full test suite passes (includes race detector)
2. `make lint` — no new lint violations
3. `go vet -tags webkit2_41 ./backend/database/ ./backend/library/` — no issues
4. Grep verification: `grep -n 'contentless_delete=1' backend/database/sql/schemas/search_index.sql backend/database/search.go` shows both locations updated
5. Grep verification: `grep -n 'no-op\|no.op\|NOOP' backend/database/search.go` returns no matches (no-op comment removed)
</verification>

<success_criteria>
- FTS5 search_index table uses `content='', contentless_delete=1` in both schema file and ClearSearchIndex
- DeleteSearchIndex performs `DELETE FROM search_index WHERE rowid = ?` (no longer a no-op)
- Migration 8 drops and rebuilds the FTS5 table with the new schema
- All existing search tests pass unchanged
- New tests verify row deletion, update cycle (delete + reinsert), and ClearSearchIndex
- Full `make test` and `make lint` pass
</success_criteria>

<output>
After completion, create `.planning/phases/15-schema-migration-write-safety/15-01-SUMMARY.md`
</output>