yellowjacket/.planning/milestones/v1.2-phases/15-schema-migration-write-safety/15-01-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
15-schema-migration-write-safety	01	execute	1		backend/database/sql/schemas/search_index.sql backend/database/database.go backend/database/search.go backend/database/search_test.go backend/library/library.go	true	SCHEMA-01	truths artifacts key_links FTS5 search_index uses contentless_delete=1 after migration 8 DeleteSearchIndex performs a real DELETE for individual rows Existing search queries return identical results after migration Migration is idempotent — safe to re-run if interrupted ClearSearchIndex still works for full rebuilds path provides contains backend/database/sql/schemas/search_index.sql Updated FTS5 schema with contentless_delete=1 contentless_delete=1 path provides contains backend/database/database.go Migration 8 function migration8 path provides exports backend/database/search.go Real DeleteSearchIndex implementation DeleteSearchIndex path provides min_lines backend/database/search_test.go Tests for delete, insert-update cycle, and search correctness 50 from to via pattern backend/database/database.go backend/database/search.go migration 8 calls RebuildSearchIndex RebuildSearchIndex from to via pattern backend/database/search.go backend/database/sql/schemas/search_index.sql ClearSearchIndex CREATE statement matches schema file contentless_delete=1 from to via pattern backend/library/library.go backend/database/search.go library calls InsertSearchIndex and DeleteSearchIndex DeleteSearchIndex

Migrate FTS5 search_index to contentless_delete=1 and implement real row-level DELETE support.

Purpose: Currently, DeleteSearchIndex is a no-op because contentless FTS5 tables cannot delete rows. After adding contentless_delete=1, individual rows can be deleted/updated — a prerequisite for inline tag edit → DB sync in Phase 16.

Output: Migration 8 function, updated schema, real DeleteSearchIndex, passing tests.

<execution_context> @/home/caleb/.config/opencode/get-shit-done/workflows/execute-plan.md @/home/caleb/.config/opencode/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/15-schema-migration-write-safety/15-CONTEXT.md

@backend/database/database.go @backend/database/search.go @backend/database/search_test.go @backend/database/sql/schemas/search_index.sql @backend/library/library.go

From backend/database/database.go:

type DB struct {
    db      *sql.DB
    Ctx     context.Context
    Queries *sqlcgen.Queries
    logger  *slog.Logger
}

func (d *DB) runMigrations() error  // sequential if version < N blocks
// Current: PRAGMA user_version ends at 7
// Migration 2 (migration2BasenameAndFTS) rebuilds FTS5 on startup

From backend/database/search.go:

func (d *DB) InsertSearchIndex(rowid int64, filePath, title, artist, album string) error
func (d *DB) DeleteSearchIndex(_ int64) error  // CURRENT: no-op, discards rowid
func (d *DB) ClearSearchIndex() error           // DROP + recreate FTS5 table
func (d *DB) RebuildSearchIndex() error          // ClearSearchIndex + bulk insert from track_metadata
func (d *DB) SearchFTS(query string) ([]SearchResult, error)
func (d *DB) SearchFTSByFilename(query string) ([]SearchResult, error)
func (d *DB) SearchFTSTracks(query string) ([]Track, error)
func (d *DB) SearchFTSTracksByLibrary(query string, libraryID int64) ([]Track, error)

From backend/database/search_test.go:

func seedSearchData(t *testing.T, db *DB)  // Seeds 7 tracks with full FK chains
// Tests use NewTestDB(t), t.Parallel(), t.Errorf/t.Fatalf patterns

From backend/library/library.go (raw FTS5 SQL):

// Line ~1013-1017: INSERT INTO search_index(rowid, file_path, title, artist, album) VALUES (?, ?, ?, ?, ?)
// Line ~1100-1103: Same INSERT pattern for metadata updates
// Line ~1080-1084: Comment explaining stale FTS entries are harmless

Task 1: Migrate FTS5 schema and add migration 8 backend/database/sql/schemas/search_index.sql backend/database/database.go backend/database/search.go backend/library/library.go **1. Update the FTS5 schema file** (`backend/database/sql/schemas/search_index.sql`):

Change content='' to content='', contentless_delete=1. The full CREATE statement becomes:

CREATE VIRTUAL TABLE IF NOT EXISTS search_index USING fts5(
    file_path,
    title,
    artist,
    album,
    content='',
    contentless_delete=1,
    tokenize='unicode61 remove_diacritics 2'
);

Note: content='' is still required — contentless_delete=1 is an addition, not a replacement. Both options must be present together per SQLite docs.

2. Update ClearSearchIndex in backend/database/search.go:

Update the inline CREATE VIRTUAL TABLE statement in ClearSearchIndex to match the schema file exactly (add contentless_delete=1). This is the second place the FTS5 schema is defined.

3. Implement real DeleteSearchIndex in backend/database/search.go:

Replace the no-op with a real implementation. With contentless_delete=1, the correct DELETE syntax is:

func (d *DB) DeleteSearchIndex(rowid int64) error {
    _, err := d.db.ExecContext(d.Ctx,
        `DELETE FROM search_index WHERE rowid = ?`, rowid,
    )
    if err != nil {
        return fmt.Errorf("could not delete search index entry: %w", err)
    }

    return nil
}

Update the doc comment to remove the "no-op" explanation and document the new behavior.

4. Add migration 8 to runMigrations() in backend/database/database.go:

Add a new if version < 8 block after the existing migration 7 block. The migration must:

• Call d.ClearSearchIndex() to DROP the old content='' table
• The schema file (already embedded and applied at startup before migrations) creates the new content='', contentless_delete=1 table — BUT since schemas run first, the old table already exists and IF NOT EXISTS skips the creation. So the migration needs to explicitly DROP and recreate.
• After dropping, recreate using the new schema. Don't call ClearSearchIndex here (which has the updated schema) — instead, drop the table and let RebuildSearchIndex() handle both recreate + repopulate:

if version < 8 {
    d.logger.Info("migration 8: rebuilding FTS5 search_index with contentless_delete=1")

    if err := d.RebuildSearchIndex(); err != nil {
        return fmt.Errorf("migration 8: could not rebuild search index: %w", err)
    }

    if _, err := d.db.ExecContext(d.Ctx,
        `PRAGMA user_version = 8`,
    ); err != nil {
        return fmt.Errorf("migration 8: could not set user_version: %w", err)
    }
}

This is naturally idempotent per the CONTEXT.md decision — if interrupted, re-running drops and rebuilds again.

5. Update library.go raw SQL comments in backend/library/library.go:

Around lines 1080-1084, update the comment that says "stale entries are harmless" to note that DeleteSearchIndex now works and Phase 16 will use it for inline updates. The INSERT statements themselves don't change — they already use the correct column names and rowid binding.

What to avoid: Do NOT change any column names in the FTS5 table (file_path, title, artist, album). Do NOT modify the tokenizer. Do NOT change InsertSearchIndex or any search query SQL — the only changes are to the table options and DeleteSearchIndex. cd /mnt/vault/dev/golang/yellowjacket && go test -tags webkit2_41 -run TestSearch -count=1 -timeout 30s ./backend/database/ && go test -tags webkit2_41 -run TestMigration -count=1 -timeout 30s ./backend/database/ - search_index.sql contains contentless_delete=1 - ClearSearchIndex CREATE statement matches schema file - DeleteSearchIndex performs a real DELETE (not a no-op) - Migration 8 exists and sets PRAGMA user_version = 8 - All existing search tests pass unchanged (SearchFTS, SearchFTSByFilename, etc.) - go vet -tags webkit2_41 ./backend/database/ and go vet -tags webkit2_41 ./backend/library/ pass

Task 2: Add tests for FTS5 row deletion and update cycle backend/database/search_test.go Add new test functions to `backend/database/search_test.go` that verify the new DeleteSearchIndex behavior and the insert-delete-reinsert cycle needed for tag editing.

Tests to add:

1. TestDeleteSearchIndex — Table-driven test:

   • Seed data with seedSearchData(t, db) (7 tracks)
   • Delete one row by rowid
   • Verify searching for that track's title returns no results
   • Verify searching for other tracks still works
   • Cases: delete existing rowid (success), delete non-existent rowid (no error — DELETE WHERE with no match is fine in SQLite)

2. TestSearchIndexUpdateCycle — Simulates tag edit flow:

   • Insert a track into search_index with rowid=100, title="Old Title", artist="Old Artist"
   • Verify search for "Old Title" returns rowid 100
   • Delete rowid 100 from search_index
   • Verify search for "Old Title" returns no results
   • Re-insert rowid 100 with title="New Title", artist="New Artist"
   • Verify search for "New Title" returns rowid 100
   • Verify search for "Old Title" returns no results (no ghost/stale entries)

3. TestClearSearchIndexPreservesSchema — Verify ClearSearchIndex still works:

   • Seed data
   • Call ClearSearchIndex()
   • Verify search returns no results
   • Insert new data
   • Verify search works again (table was recreated with correct schema including contentless_delete=1)

All tests must follow existing patterns:

• Use t.Parallel() at top level
• Use NewTestDB(t) for DB setup
• Use t.Errorf / t.Fatalf (no assertion libraries)
• Use seedSearchData(t, db) where appropriate

Note: The seedSearchData helper creates full FK chains (audio_files → recordings → artists → etc.) that satisfy the track_metadata VIEW's JOINs. For TestSearchIndexUpdateCycle, you'll need to insert a minimal audio_file + recording chain to have valid data in track_metadata for the search JOIN. Look at seedSearchData for the exact pattern. cd /mnt/vault/dev/golang/yellowjacket && go test -tags webkit2_41 -v -run "TestDeleteSearchIndex|TestSearchIndexUpdateCycle|TestClearSearchIndexPreservesSchema" -count=1 -timeout 30s ./backend/database/ - TestDeleteSearchIndex passes — deleting a row removes it from search results - TestSearchIndexUpdateCycle passes — delete + reinsert produces no ghost entries - TestClearSearchIndexPreservesSchema passes — drop/recreate preserves new schema - All existing search_test.go tests continue to pass - make test passes (full test suite)

1. `make test` — full test suite passes (includes race detector) 2. `make lint` — no new lint violations 3. `go vet -tags webkit2_41 ./backend/database/ ./backend/library/` — no issues 4. Grep verification: `grep -n 'contentless_delete=1' backend/database/sql/schemas/search_index.sql backend/database/search.go` shows both locations updated 5. Grep verification: `grep -n 'no-op\|no.op\|NOOP' backend/database/search.go` returns no matches (no-op comment removed)

<success_criteria>

• FTS5 search_index table uses content='', contentless_delete=1 in both schema file and ClearSearchIndex
• DeleteSearchIndex performs DELETE FROM search_index WHERE rowid = ? (no longer a no-op)
• Migration 8 drops and rebuilds the FTS5 table with the new schema
• All existing search tests pass unchanged
• New tests verify row deletion, update cycle (delete + reinsert), and ClearSearchIndex
• Full make test and make lint pass </success_criteria>

After completion, create `.planning/phases/15-schema-migration-write-safety/15-01-SUMMARY.md`