Android has no /tmp and hands an app no TMPDIR. Go's os.TempDir() falls
back to "/tmp" when the variable is unset, so every library in this
process that wants scratch space was being handed a path that has never
existed.
SQLite is the one that noticed, and it said so precisely:
W/yellowjacket: msg="champion index rebuild failed"
explore.search-index.error="populate champion fts: disk I/O error (6410)"
6410 is not a generic I/O error. `6410 & 0xff` is 10, SQLITE_IOERR, and
`6410 >> 8` is 25 -- SQLITE_IOERR_GETTEMPPATH. SQLite could not work out
where to put a temporary file. Two measurements on the device say why:
`ls -d /tmp` does not exist, and the app process's environment carries
no TMPDIR. A shell's does (/data/local/tmp), which is why this is easy
to miss from `adb shell`.
The cost was a silent performance cliff on the slowest device this app
runs on: `championReady` stayed false, so every Explore search took the
generic path over the whole 1,079,667-row index instead of the champion
subset, and the rebuild was re-attempted on every launch.
**The class is fixed rather than the statement.** The trigger is the
*size* of the work, not that query -- anything that spills fails the
same way there, so large sorts, large joins and VACUUM were all waiting
their turn. The repair belongs at the process's one answer to "where do
temporary files go".
`PRAGMA temp_store = MEMORY` was the alternative: cheaper, more local,
and a promise that every future spill fits in RAM on a phone. The
catalog is the largest thing in this app and that is not a promise
worth making silently.
UseTempDir sits beside UseHomeOverride and carries its two rules for
the same reasons. **An empty base is a no-op**, because that is what
application.Mobile.StoragePath() returns on desktop -- so this needs no
build tag and changes nothing off mobile, where /tmp is real. And **an
explicit TMPDIR wins**, so anyone who set one deliberately gets it;
nothing sets it on the platform this exists for. It needs no new Wails
API and no Java change: StoragePath() is already what YJ_HOME is
pointed at, and the directory goes under it.
Two things beyond the rename of a variable.
**Writability is probed, not assumed.** MkdirAll on an existing
unwritable directory succeeds, so without the probe this could set
TMPDIR to a directory nothing can use -- which is the same bug one
directory over, and just as quiet.
**It returns its error, and main logs it.** A temp directory that could
not be created is the same silent failure one step earlier. A failure
is not fatal: it leaves the platform's answer in place, which is what
every release before this one ran with. That log line is readable on
the platform only because of #160.
Verified on the reference device, where the same launch that used to
print the failure now prints:
I/yellowjacket: msg="champion index rebuilt"
explore.search-index.elapsed=6.496s
Closes#190
Every slog line the app wrote on Android went to /dev/null, including
the one naming the error it was about to os.Exit on. #52 is what that
cost: a process that vanished with no tombstone, no AndroidRuntime
stack and nothing in `logcat -b crash`, at Priority/Critical for
months, whose entire diagnosis was one sLogger.Error main.go was
already writing.
backend/androidlog is a slog.Handler over __android_log_write, chosen
in main() by build tag rather than by a runtime check so that a desktop
binary links no cgo for a platform it cannot run on.
**Everything except the write itself is untagged.** That is
androidpayload.go's discipline pushed as far as it goes: the only
toolchain that compiles the android tag is a cross-compiler and the
only thing that runs it is a phone, so the priority mapping, the
formatting, the chunking and the handler's own attr and group
bookkeeping are ordinary Go that `go test` exercises everywhere, and
android.go is fifteen lines that hand a string to liblog.
Four things in it are load-bearing.
**The tag is a fixed string, not the application id.** The debug build
carries `applicationIdSuffix ".dev"` so it can be installed beside the
release app, and it is the only build whose WebView can be inspected --
so a tag derived from the id is a different tag on the one build
anybody debugging this app is running, and the filter meant to show
these lines would hide them exactly where they were being looked for.
**The priorities are android/log.h's own values, asserted twice.**
android.go carries constant expressions that do not compile as uint if
the header renumbers; the untagged test writes the six numbers out
longhand, because comparing a constant to itself passes on any
renumbering. A wrong priority is the failure that hides rather than
breaks -- logcat prints whatever number it is handed, so an Error filed
as Info is present, correct, and invisible to every filter.
**Formatting is delegated to slog's TextHandler.** WithAttrs and
WithGroup are the half of slog.Handler that is easy to get subtly
wrong, and a logger whose groups are wrong is a logger nobody reads.
The derived handlers share the parent's buffer *and its mutex*: a
second mutex would guard nothing, and two loggers derived from one
would splice their bytes into a single line under load.
**A line is chunked, because liblog drops what does not fit.** The
kernel logger's entry is 4068 bytes for tag and message together and
the remainder goes without comment, so a long record would be truncated
in the middle of the thing worth reading.
Time and level are dropped from the formatted line, since logcat stamps
every entry with both -- and dropping them by *key* also ate a caller's
own "level" attribute, which the on-device probe caught and
TestACallersOwnLevelAttrSurvives now holds. ReplaceAttr sees an empty
group path for the built-ins and for every top-level attribute alike,
so the kinds are what separate them.
Verified on the reference device (TLP301, Android 14): a debug build
logs I/W/E under the `yellowjacket` tag at the right priorities, and
the first thing it surfaced was a real warning nobody could previously
see -- `champion index rebuild failed ... disk I/O error (6410)`.
Closes#160
Wails' Android entry point is `nativeInit`, which `MainActivity.onCreate`
calls, and it runs `go mainFunc()` every time. Android destroys and
recreates an activity **without restarting the process** -- a
configuration change the manifest does not declare, memory pressure, or
every background under "Don't keep activities" -- so main() ran again on
a live app.
Every path out of that is fatal. `application.New` returns the existing
app rather than building a second one, `app.Run()` then refuses because
`a.starting` is still true behind Android's `select{}`, and the
`os.Exit(1)` under that error takes the **first**, healthy app down with
it: its database, its queue, and the audio a mediaPlayback foreground
service is holding the process alive to play. ActivityManager restarts
the app, which is the report.
Measured on a Light Phone III (Android 14, arm64): conditional on the
activity actually being recreated, the process died 8 times out of 8.
The runs that "passed" were runs where no recreation happened, which is
the whole of the report's "sometimes". After this, 5/5 recreations
survive on one pid, plus six background/foreground cycles.
It never left evidence because os.Exit is not a crash: no tombstone, no
AndroidRuntime stack, nothing in `logcat -b crash`, and the slog line
naming the error went to /dev/null with the rest of fd 1.
The latch is first in main() because everything below it -- above all
NewYellowJacketApp, which opens the SQLite database -- is work that must
not happen twice in one process. It is inert off Android.
Returning early is not a degraded mode: nativeInit has already
re-pointed the JNI reference at the new bridge, so the recreated
WebView talks to the app that is still running, with its queue and
playback position intact. Verified by hooking dispatchWailsEvent on the
recreated page: IndexStatusChanged, JobsChanged, android:storageAccess.
No tier here runs main() on Android, so the guard is a source sweep, in
the spirit of TestNoDirectRuntimeEmits. The failure it exists for is not
the latch being deleted -- that is loud -- but a line creeping in above
it.
Closes#52
backend/system resolves config and data from $HOME or the OS
equivalent, and Android has neither: buildUserDirPath switches on
runtime.GOOS with cases for darwin, linux and windows and a default
returning errUnsupportedOS. So NewYellowJacketApp failed and main()
called os.Exit(1) about six milliseconds after the JNI bridge came up.
That failure is invisible in all three places anyone would look. There
is no panic, no AndroidRuntime stack and no tombstone, because os.Exit
is not a crash; Go's stdout does not reach logcat, so the slog line
naming the error is discarded; and ActivityManager respawns the process
fast enough that pidof always answers, so a crash-looping app looks
alive.
main() now sets the override before anything asks for a path.
application.Mobile.StoragePath() is the platform's own answer --
getFilesDir() on Android, Application Support on iOS -- and returns ""
on desktop, where UseHomeOverride is a no-op, so this needs no build
tag and changes nothing off mobile. resolveUserDirPath already honours
YJ_HOME on every OS, so there was a seam for it.
The knowledge stays in main(): backend/system gains no import of the
Wails application package, for the same reason backend/events is split
by the indexbuild tag.
UseHomeOverride's two rules are tested because nothing else would
notice them breaking. An empty base does nothing, which is exactly the
desktop case. And an override already set wins, so YJ_HOME still
relocates a sandbox on the one platform that would otherwise decide for
itself.
This is not the end of the port. The app now reaches the database and
takes SIGSYS on the x86_64 emulator -- modernc.org/libc issues a raw
lstat syscall on linux/amd64 and Android's seccomp forbids it. arm64,
which is what ships to phones, has no lstat syscall at all and routes
through fstatat, so it is structurally unaffected. See NOTES.md.
make e2e is green on chromium: 92 passed. The harness is rebuilt on
what v3 actually offers, and three of the four things it replaced turn
out to be better than what they replaced.
The headless launch is v3's own server mode. scripts/dev-headless.sh
ran a `-tags dev` binary whose app_dev.go parsed -devserver/-assetdir
out of os.Args; that file went with v2, so the harness had no server at
all. `-tags dev,server` is a first-class mode and needs no display, so
Xvfb is gone from the script and from CI.
The bridge hooks two places, neither of them EventsOn. Inbound is
window._wails.dispatchWailsEvent, wrapped by pre-creating the object
the runtime keeps and putting an accessor on the one property.
Outbound is fetch: v3 routes every runtime call through one POST, so
the bridge sees binding calls and event emits from any module, needs no
walk of an object graph, and cannot miss a call made before it looked.
__yjEvents.call posts to that endpoint by method name, so it depends on
nothing in the app's bundle and works on a page with no init script.
That is what lets seed-sandbox.sh drop playwright-cli entirely — it
drove AddLibrary through a browser only because window.go was v2's one
way in — and with it a global npm install and a second Chromium in CI.
measure.mjs and one spec lose their window.go walks and read the
bridge's log instead; e2e/support/method-ids.mjs derives id -> name
from frontend/bindings/ (phase 6b option 1, so it cannot go stale
silently). Plain .mjs because measure.mjs runs under bare node and one
derivation beats two that can disagree.
Four bugs surfaced, and the migration is how.
The cross-service wiring never ran headless. It hung off
Common.ApplicationStarted, which server mode never emits —
setupCommonEvents is an explicit no-op there — so the queue had no
TrackLoader and playing a track changed the queue and then silently did
nothing. It is a service registered last now (backend/startup.go):
services start in registration order, which is the ordering the wiring
needs, in every mode.
Six specs called SetQueue with 3 of its 4 arguments. v2 accepted that
and filled the gap; v3 answers "expects 4 arguments, got 3".
requested-badge's cleanup read window.go and returned early on
`if (!svc)` — the silent cleanup its own comment was written to
prevent, one migration later. It posts to the runtime endpoint now,
which any page can do.
SearchIndex.Search trusted a startup latch, so rows a spec staged
afterwards were unsearchable and three specs passed only when an
earlier one happened to flip it. shelves.go fixed exactly this and left
hasCatalogRows behind; the search path now uses it as the fallback,
with the latch still the fast path.
Two spec edits are deletions of assertions about v2. harness.spec
checked Object.keys(window.go) and that a bad call *hung*; it now
checks the real runtime is loaded and that the backend rejects with a
TypeError naming the argument. album-actions asserted a tracklist
legend that dcc40b1 deleted on main — that spec has been failing since,
and what replaced it is covered in frontend/test/components.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UDCbcCZQepnpSQYJ6SxxZm
Phases 2 and 3 of plan 009, plus the parts of phase 1 that could not
land before them. Nothing in the tree imports wails/v2 any more; all
three lint and test configurations are green and `go build .` produces
a running binary.
The point of the migration is one file. backend/events/emit.go probed
ctx.Value("events") — a v2-*private* context key — to decide whether
emitting was safe, because runtime.EventsEmit called log.Fatalf on a
context without the runtime and took the process down with it. v3's
emit takes no context, so that is now application.Get() == nil. D1
held: events.Emit keeps its ctx as the WithSink test seam, and all 45
call sites and 7 test files are untouched.
The bootstrap splits into application.New + Window.NewWithOptions +
Run. Ten bound services implement ServiceStartup instead of being
handed a context by hand from OnStartup, which also stops ten
SetContext methods being exported as bindings. jobs.Registry and
explore.SearchIndex keep theirs — neither is bound, so converting them
would be churn for no binding removed.
Four things differed from the plan and are written up in it: GPU policy
moved to the per-window LinuxWindow options rather than surviving on
LinuxOptions; there is no OnStartup/OnDomReady option, so app-level
wiring hangs off ApplicationStarted; application.NewService is generic,
so FEBindings []any could not survive (the binding generator is a
static analyser and would have seen nothing); and the quit veto had to
be restructured, because v3's dialog answers on a callback rather than
returning the button, so ShouldQuit vetoes, asks, and quits again from
the callback.
Window state saving moves to a WindowClosing hook — the size has to be
read while the window still exists, and v3's OnShutdown has neither
context nor window. backend/logging is deleted rather than ported:
v3 takes a *slog.Logger directly, so the v2 logger.Logger adapter had
no caller left.
Phase 1's tail rides along, now that it can: the Makefile's wails
invocations, all 50 webkit2_41 sites, lefthook, both packaging recipes
and ci.yml's apt lists. v3 builds against GTK4 + WebKitGTK 6.0, which
Arch and ubuntu:24.04 both ship, so the tag is a deletion rather than
a translation.
Phase 4 is next and the branch is not usable until it lands: the app
builds, but frontend/wailsjs/ is v2's tree and nothing regenerates it,
so the frontend cannot reach the backend yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UDCbcCZQepnpSQYJ6SxxZm
Consolidates in-progress work across autotag, explore, and library:
- autotag: beets/Picard-informed scoring engine — ID-first matching, VA
handling, recommendation tiers, and a merged distance/rank cascade, with
an eval harness for regression tracking.
- explore: offline MusicBrainz dump import/incremental refresh replaces the
legacy tier crawl; index-first local search with fuzzy matching and a
dedicated ranker; disk-free guards for dump downloads.
- library: artist-credit extraction and matching.
- lyrics: owned-library lyric search (FTS) with LRCLIB backfill.
Also: rewrite README to be user-focused, and migrate upstream to
git.ljones.me/yonlu/yellowjacket.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Instead of disabling DMABuf for all users (hurting AMD/Intel/X11
performance), detect NVIDIA GPU + Wayland session at startup and
only apply WEBKIT_DISABLE_DMABUF_RENDERER=1 for that combo.
Detection checks /proc/driver/nvidia/version first (fast, no
subprocess), falls back to scanning /proc/modules for nvidia.
GPU policy remains Always for everyone — the DMABuf workaround
only affects buffer sharing between WebKitGTK and the display
server, not GPU-accelerated CSS/layout/paint.
- Change dev mode default from Debug to Info to prevent stdout flooding
- Add resolveLogLevel() function with YJ_LOG_LEVEL env var support
- Accept debug/info/warn/error values (case-insensitive)
- Prevents neovim display corruption during library scans
Wire up Go's standard profiling toolkit so it's automatically available
in dev builds and completely absent from production. The profiling
package uses build tags (dev/!dev) to eliminate all pprof, trace, and
timing code from release binaries with zero new dependencies.
- backend/profiling: pprof HTTP server on :6060, /debug/trace endpoint,
block/mutex profiling, and TimeOp helper for structured operation timing
- scripts/profile.sh: interactive menu-driven script that auto-selects
free ports (8080-8089) so multiple profiles can be open simultaneously
- Instrumented key operations: app init, database init, player load/restore,
queue set/restore
- Makefile targets: profile, profile-cpu, profile-heap, profile-trace
- .gitignore: exclude trace-*.out and *.pprof artifacts
logan
yonluandClaude Opus 5