yonlu/yellowjacket
1
1
Fork 0
Code Issues 55 Pull Requests 6 Actions Packages Projects Releases 12 Wiki Activity

Retiring a stale audio_files empties every playlist, where a rescan would preserve it #183

New Issue
Open
opened 2026-08-21 14:22:24 +00:00 by yonlu · 1 comment
yonlu commented 2026-08-21 14:22:24 +00:00
Owner
Copy Link
Copy Source

Report

retireStaleTables (backend/database/staleshape.go) drops a stale
audio_files and lets ON DELETE SET NULL null out
playlist_tracks.audio_file_id. It does not first populate the
phantom_* columns — so every playlist entry survives as an empty row
that nothing can ever re-link, and the playlist is silently emptied.

The full-rescan path, which is the documented answer to the same
situation ("delete and rescan", plan 013 open question 1), gets this
right. backend/library/rescan.go:136:

// Preserve playlist tracks across rescan: populate phantom
// metadata for all linked tracks before audio_files are deleted.
// ON DELETE SET NULL will null out audio_file_id, converting them
// to phantoms that ResolvePhantomTracksAfterScan can re-link.

So the manual repair preserves playlists and the automatic repair that
exists to spare the user that manual work does not.

Findings

  • staleshape.go contains zero references to phantom (grep).
  • datamap.go:282 states the contract this breaks: playlist_tracks is
    Authored, SetNull, "Deliberately survives its track: audio_file_id
    is nulled and phantom_* columns preserve the entry so a rescan can
    re-link it."
  • staleshape.go:56 acknowledges the entries "survive pointing at
    nothing" and weighs that as accepted cost — but it was weighed against
    losing them, not against the phantom mechanism that already exists
    twenty lines away in another file.
  • The rest of the sequence already works: libraries is
    Authored, Retained so the chosen folders survive, and
    SoftScanAllLibraries (scan_queue.go:109) sees dbCount 0 vs
    diskCount N and queues a full scan automatically. The user launches
    and watches a scan; they do not have to delete YJ_HOME.

Why now

Pre-1.0 the project's stated posture is that schema breakage is
acceptable between releases. That makes this the load-bearing safety
net rather than an edge case: the next audio_files column (added_at,
for #46) triggers exactly this path on every existing install.

Direction

Run the same UPDATE playlist_tracks SET phantom_* = ... that
rescan.go runs, inside the retire transaction, before the drop, and
only when audio_files is among the tables going. It must be inside
that transaction, which already sets defer_foreign_keys and is
already ordered deterministically.

Test: seed a playlist with a linked track, retire audio_files,
rescan, assert the entry re-linked — the symptom first, in the shape of
TestRetiringOwnedTablesDoesNotDangle.

Also worth deciding, separately

The auto-queued full scan calls ClearExcludedPaths (rescan.go:132),
so "removed from library" decisions are dropped on that first launch
too. excluded_paths is Authored and survives the retire itself; it
is the scan that clears it. Probably acceptable, but it is a second
authored thing going away quietly and should be a deliberate answer.

**Report** `retireStaleTables` (`backend/database/staleshape.go`) drops a stale `audio_files` and lets `ON DELETE SET NULL` null out `playlist_tracks.audio_file_id`. It does **not** first populate the `phantom_*` columns — so every playlist entry survives as an empty row that nothing can ever re-link, and the playlist is silently emptied. The full-rescan path, which is the *documented* answer to the same situation ("delete and rescan", plan 013 open question 1), gets this right. `backend/library/rescan.go:136`: ```go // Preserve playlist tracks across rescan: populate phantom // metadata for all linked tracks before audio_files are deleted. // ON DELETE SET NULL will null out audio_file_id, converting them // to phantoms that ResolvePhantomTracksAfterScan can re-link. ``` So the manual repair preserves playlists and the *automatic* repair that exists to spare the user that manual work does not. **Findings** - `staleshape.go` contains zero references to `phantom` (grep). - `datamap.go:282` states the contract this breaks: `playlist_tracks` is `Authored, SetNull`, "Deliberately survives its track: audio_file_id is nulled and phantom_* columns preserve the entry so a rescan can re-link it." - `staleshape.go:56` acknowledges the entries "survive pointing at nothing" and weighs that as accepted cost — but it was weighed against losing them, not against the phantom mechanism that already exists twenty lines away in another file. - The rest of the sequence already works: `libraries` is `Authored, Retained` so the chosen folders survive, and `SoftScanAllLibraries` (`scan_queue.go:109`) sees dbCount 0 vs diskCount N and queues a full scan automatically. The user launches and watches a scan; they do not have to delete `YJ_HOME`. **Why now** Pre-1.0 the project's stated posture is that schema breakage is acceptable between releases. That makes this the load-bearing safety net rather than an edge case: the next `audio_files` column (`added_at`, for #46) triggers exactly this path on every existing install. **Direction** Run the same `UPDATE playlist_tracks SET phantom_* = ...` that `rescan.go` runs, inside the retire transaction, before the drop, and only when `audio_files` is among the tables going. It must be inside that transaction, which already sets `defer_foreign_keys` and is already ordered deterministically. Test: seed a playlist with a linked track, retire `audio_files`, rescan, assert the entry re-linked — the symptom first, in the shape of `TestRetiringOwnedTablesDoesNotDangle`. **Also worth deciding, separately** The auto-queued full scan calls `ClearExcludedPaths` (`rescan.go:132`), so "removed from library" decisions are dropped on that first launch too. `excluded_paths` is `Authored` and survives the retire itself; it is the scan that clears it. Probably acceptable, but it is a second authored thing going away quietly and should be a deliberate answer.
yonlu self-assigned this 2026-08-21 14:23:24 +00:00
yonlu added the
Status
In Progress
 label 2026-08-21 14:23:24 +00:00
yonlu commented 2026-08-21 14:23:24 +00:00
Author
Owner
Copy Link
Copy Source

Starting on this on fix/183-phantom-across-retire.

Approach: run the same UPDATE playlist_tracks SET phantom_* = ...
that rescan.go:136 already runs, inside the retire transaction and
before the drops, gated on audio_files being among the tables being
retired. Inside that transaction because it already sets
defer_foreign_keys and is already deterministically ordered.

Doing it ahead of #46, which adds audio_files.added_at and is exactly
what triggers this path on every existing install.

Starting on this on `fix/183-phantom-across-retire`. Approach: run the same `UPDATE playlist_tracks SET phantom_* = ...` that `rescan.go:136` already runs, inside the retire transaction and before the drops, gated on `audio_files` being among the tables being retired. Inside that transaction because it already sets `defer_foreign_keys` and is already deterministically ordered. Doing it ahead of #46, which adds `audio_files.added_at` and is exactly what triggers this path on every existing install.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Area/Design
Area/Downloads
Area/Explore
Area/Library-UI
Area/Metadata
Area/Packaging
Area/Player
Area/Queue
Area/Settings
Area/Shell-Nav
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
 Platform/Android
Platform/Desktop
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Medium
3
The priority is medium
Priority
Low
4
The priority is low
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
In Progress
Somebody is actively working on this right now
No labels Kind/Bug Platform/Android Platform/Desktop
Priority
High
2
Reviewed
Confirmed
1
Status
In Progress
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
logan yonlu
No Assignees yonlu
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: yonlu/yellowjacket#183