diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index 7dfee4c..85fd4dd 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -171,7 +171,7 @@ jobs: working-directory: /src run: make ui-test - # frontend/wailsjs is generated by `wails`, not by `go generate`, + # frontend/bindings is generated by `wails3`, not by `go generate`, # so the codegen pre-commit hook does not cover it. - name: Bindings are current working-directory: /src @@ -184,7 +184,9 @@ jobs: run: make skill-check # ---------------------------------------------------------------- # - # Job 2: the real app, under a virtual display. # + # Job 2: the real app, headless. v3's `-tags server` needs no # + # display, so the Xvfb this job used to wrap everything in is gone. # + # `dbus-run-session` stays, for MPRIS. # # ---------------------------------------------------------------- # e2e: runs-on: ubuntu-latest diff --git a/.planning/plans/active/009-wails-v3-migration.md b/.planning/plans/active/009-wails-v3-migration.md index cc8e1fb..7d6da4d 100644 --- a/.planning/plans/active/009-wails-v3-migration.md +++ b/.planning/plans/active/009-wails-v3-migration.md @@ -1,12 +1,15 @@ # 009 — Wails v3 migration -**Status:** Phases 0–6 complete; **Phase 7 (CI and packaging) is all -that is left**, and most of it rode along. Locally green: `make lint` -and `make test` (all three build configurations), `tsc --noEmit`, 757 -Vitest tests across all 63 files, `make e2e` 92/92 on chromium, -`make bindings-check`, `make skill-check`, and `make perf` runs and -reports real binding counts again. **WebKit is unverified** — it is -CI-only by design — so the merge waits on a green CI run. +**Status:** Phases 0–7 complete. CI needed nothing (it had ridden along +with Phases 1–6); **packaging needed everything** — both recipes still +called v2's CLI, installed from v2's output path, and built from a +scaffold's `yjref` metadata. Locally green: `make lint` and `make test` +(all three build configurations), `tsc --noEmit`, `make build-prod`, +757 Vitest tests across all 58 files, `make e2e` 92/92 on chromium, +`make bindings-check`, `make skill-check`, `make commit-check`, +`make css-check`, and `make perf` runs and reports real binding counts +again. **WebKit is unverified** — it is CI-only by design — so the +merge waits on a green CI run. **Branch:** `wails-v3`, off `main` at `edb13a6`. **Created:** 2026-08-13 **Phase 0 run:** 2026-08-13 against **v3.0.0-beta.8** @@ -994,6 +997,83 @@ matters. **Est.** Half a session. +### Phase 7 — what actually landed + +Most of the bullets above had already ridden along with Phases 1–6: +`ci.yml`'s apt lists were `libwebkitgtk-6.0-dev libgtk-4-dev` on both +jobs, Xvfb and `@playwright/cli` were gone from it, the PKGBUILD's +`depends=()` was `webkitgtk-6.0 gtk4`, and `nfpm.yaml` shipped the GTK4 +dependency set. **What had not been checked is whether any of the +packaging recipes still work**, and neither did. + +**Both of them were still calling v2's CLI.** `go tool wails3 build +-clean -trimpath -ldflags …` fails outright — `flag provided but not +defined: -clean`. v3's `build` takes `-tags`, `-obfuscated` and +`-garbleargs` and nothing else, because the build is a Taskfile tree +now and `-trimpath`/`-w -s` live in the production task's own flags. +Both recipes also installed from `build/bin/`, which is v2's output +path; v3 writes to `bin/`, and `build/` is *tracked build assets*. +Neither had been run since Phase 1 — the same gap the corrected note +under Phase 1 records about `make build-dev`. + +**The version stamp needed a seam, and it is this repo's one edit to +the scaffold Taskfiles.** `wails3 build` has no `-ldflags`, and +`build:native` computes `BUILD_FLAGS` in its own `vars:`, so a CLI +variable cannot override it. `LDFLAGS_EXTRA` is appended *inside* the +production `-ldflags` string in `build/linux/Taskfile.yml` and +`build/darwin/Taskfile.yml` (identical on both, so the Homebrew formula +has one invocation rather than two), empty by default so `make +build-dev` and `make build-prod` are unchanged. Verified end to end, +not by reading the template: `wails3 task build LDFLAGS_EXTRA="-X +'main.version=v9.9.9' …"` produces a binary that logs `version: v9.9.9` +on startup. + +**The tasks invoke `wails3` by bare name**, which is why the Makefile +has `scripts/toolbin` — and neither packaging recipe had it, so both +would have died at the first sub-task even with correct flags. Both +prepend it now. + +**Bundling is a separate step from building in v3**, which the formula +did not know: `task build` produces a bare binary on *both* platforms, +and the macOS `.app` is `task package`. The formula's `Dir["build/bin/ +*.app"]` would have found nothing and `odie`'d. + +**The build assets were a scaffold's, not this app's** — and this is +the find that mattered most. `build/darwin/Info.plist` named +`CFBundleExecutable` **`yjref`** (the scratchpad app Phase 1 scaffolded +from) and `com.example.yjref`; `nfpm.yaml` packaged `./bin/yjref` to +`/usr/local/bin/yjref`; the `.desktop` template said "A yjref +application"; the Windows manifest said `com.example.yjref`. A macOS +`.app` built from that plist would not have launched. They are +generated from `build/config.yml`, so the fix is +`wails3 task common:update:build-assets` and filling that file's `info` +block — which had also never been filled from `wails.json`, contrary to +Phase 1 step 3. Two consequences recorded in place: nfpm's `homepage` +and `license` are **not** derived from `config.yml` and survive the +refresh (they were `wails.io` / `MIT`), and the refresh regenerates +`build/ios/` and `build/android/`, which are gitignored rather than +deleted-and-rediscovered on every run. + +**Documentation.** `make skill-check` passes; `.pi/` needed no changes +(Phase 6 had already retired `window.go` and Xvfb from it). `README.md` +still told a contributor to `go install wails/v2/cmd/wails` and +`apt-get install libgtk-3-dev libwebkit2gtk-4.1-dev`. CLAUDE.md gained +a **Packaging** section for the four Taskfile facts above, and its +lifecycle, bindings, harness, events and CI sections were rewritten +onto v3 — including the point of the whole migration, that +`TestNoDirectRuntimeEmits`'s justification is now the *weaker* one. + +**Locally green after the change:** `make lint` and `make test` (all +three build configurations), `tsc --noEmit`, `make css-check`, +`make bindings-check`, `make skill-check`, `make commit-check`, +`make build-prod`, 757 Vitest tests across all 58 files (batched — the +single-run failure still reproduces and is still this machine's +resource limit, unchanged from Phase 5), and `make e2e` **92/92 on +chromium**. + +**Still unverified: WebKit**, and it is CI-only by design. That is the +one thing standing between this branch and a merge. + --- ## Phase 8 — What v3 unlocks (explicitly out of scope) diff --git a/CLAUDE.md b/CLAUDE.md index 769e81c..86d98b6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,7 +39,7 @@ make e2e-setup # Install the e2e runner + its browser (once) make ui-test # Vitest component/store suite in a real browser (no app) make ui-visual # Same, including toMatchScreenshot comparisons make ui-setup # Install the Vitest provider's own Chromium (once) -make bindings-check # Fail if frontend/wailsjs is stale vs the Go bindings +make bindings-check # Fail if frontend/bindings is stale vs the Go bindings make skill-check # Fail if .pi/ documents a make target that doesn't exist make commit-check # Fail if a commit subject is not a Conventional Commit make lint # golangci-lint v2 (strict), all three build configurations @@ -87,9 +87,12 @@ through `internal/testfixtures`, selecting files by *case* path, and skip themselves when it has not been generated. The app itself can be run without a blocking window — `make -dev-headless` — and driven with `playwright-cli` against the dev server -on `:34115`, which is the real app with real bindings on `window.go`, -bridged to the same Go backend a desktop window would use. +dev-headless` — and driven with `playwright-cli` against it on +`:34115`. That is wails v3's first-class `-tags server` mode: the real +app, the real bindings, the same Go backend a desktop window would use, +served over HTTP with **no display at all**. The Xvfb this used to +require is gone, from the script and from CI; `dbus-run-session` stays, +for MPRIS. **The operational half of all this lives in the `yellowjacket-dev` skill** (`.pi/skills/yellowjacket-dev/`): which tier @@ -106,8 +109,16 @@ phase 3: backend event on `window.__yjEvents`. Half this app is push-driven, so assertions **await an event, not a timeout**: `await window.__yjEvents.wait('LibraryScanComplete', {timeoutMs: 60000})`. - It also provides `ready()` and `call('queue.Queue.GetState', [])`, - which times out instead of hanging. + It also provides `ready()` and `call('queue.Queue.GetState', [])`. + Both hook v3's own seams rather than an internal: inbound is + `window._wails.dispatchWailsEvent`, the entry point the backend's push + uses, and outbound is **`fetch`** — v3 routes every runtime call + through one POST to `/wails/runtime`, so one hook sees calls from any + module and cannot miss one made before the harness looked. `call()` + posts by method name, so it depends on nothing in the app's bundle and + works on a page with no init script — which is why `seed-sandbox.sh` + is `curl` now and needs no browser. It no longer races a timeout + either: v3 rejects bad arguments and unknown methods cleanly. - **The dev-only control surface**, `backend/testctl`, mounted at `/__test/` on the same port: `health`, `db/snapshot`, `db/restore`, `emit` (force any backend event, which renders push-driven views @@ -133,18 +144,29 @@ meaningless against the seed's one empty playlist, so it builds ten It wraps every bound Go method, so "did that refetch the library" is a fact rather than an inference. It is not a spec and does not run in CI. -**The cheapest tier needs none of that.** `make ui-test` runs 672 -Vitest tests in a real Chromium in ~2 s with no Wails, no backend, no -seeded library and no virtual display, because `frontend/wailsjs/` is a -pure passthrough to `window.go` / `window.runtime` and -`frontend/test/support/wails-fake.ts` replaces just those two globals — -so the tests exercise the real generated bindings and the real store -code. +**The cheapest tier needs none of that.** `make ui-test` runs 757 +Vitest tests in a real Chromium with no Wails, no backend, no seeded +library and no virtual display, because **v3 routes every runtime call +— bindings, event emits, window, dialogs, clipboard — through one IPC +transport**, and `frontend/test/support/wails-fake.ts` replaces it via +`setTransport()`, which is public documented API. So the fake covers +strictly more than v2's two globals did while being shorter, and the +tests exercise the real generated bindings, the real runtime and the +real store code. A binding carries an **ID**, not a name +(`$Call.ByID(2822423495)` is FNV-1a over +`yellowjacket/backend/home.Service.GetShelves`), so the fake derives +that map from the generated tree rather than writing it down. -**`frontend/wailsjs/` is generated by `wails`, not `go generate`**, so +**`frontend/bindings/` is generated by `wails3`, not `go generate`**, so the pre-commit codegen check does not cover it. `make bindings-check` -(~1.5 s, also a pre-commit hook) regenerates it and fails on a dirty -tree; `make bindings` regenerates it for real. +(~3.5 s warm, ~20 s on a cold build cache, also a pre-commit hook) +regenerates it and fails on a dirty tree; `make bindings` regenerates it +for real. It is slower than v2's because v3's generator is a **static +analyser** over the whole package graph rather than runtime reflection +— which is also why the tag set it runs under matters, and why it is +the *default* one: that is the configuration users run, and neither +`indexbuild` nor `dev` adds a bound service. See +`scripts/bindings-check.sh`. **Seeds are produced by running the app**, never by hand-writing a `config.toml` and DB rows — the same discipline `sql/schemas/` gets, @@ -154,7 +176,46 @@ See `.planning/plans/completed/005-agent-development-harness.md`. ## Architecture -**Wails app lifecycle** (`main.go` → `backend/app.go`): `YellowJacketApp` is the root struct bound to Wails. Its methods are callable from the frontend. Lifecycle hooks: `OnStartup` (init audio), `OnDomReady` (start library scan), `OnBeforeClose` (save window state), `OnShutdown` (persist player/queue state). +**Wails app lifecycle** (`main.go` → `backend/app.go`): `main.go` is +`application.New(opts)` → `app.Window.NewWithOptions(…)` → `app.Run()`. +Each bound service takes its context from `ServiceStartup(ctx, +application.ServiceOptions{})` — v3 calls it on every service, in +registration order, on the main goroutine — and gives it back in +`ServiceShutdown()`. That context is **cancelled on app shutdown**, +which `SetContext` never was. + +Three things about it are load-bearing. + +**`ServiceShutdown()` takes no context.** A method with a +`context.Context` parameter does not satisfy the interface and is +**silently never called** — no error, no warning. + +**The cross-service wiring is a service, not an event.** v3 has no +`OnStartup`/`OnDomReady` option, and the obvious replacement — +`app.Event.OnApplicationEvent(events.Common.ApplicationStarted, …)` — +is the right *moment* and the wrong *mechanism*: **server mode emits no +application events at all** (`setupCommonEvents` is an explicit no-op +under `-tags server`), so the desktop build wired itself and the +headless harness did not. `backend/startup.go` is registered last +instead, which takes the ordering from the mechanism rather than from +an event and therefore holds in every mode. Anything else keyed on +`Common.*` is suspect for the same reason. + +**The quit veto is asynchronous now.** v2's `MessageDialog` blocked and +returned the button; v3's `Show()` returns immediately and the answer +arrives on a `Button.OnClick` callback, so `ShouldQuit` cannot ask and +answer in one call — it vetoes, shows the dialog, and calls +`app.Quit()` from the callback. `quitConfirmed` is what stops that +second `Quit()` asking again; `quitAsking` stops a second close attempt +stacking dialogs. Window state moved off that path entirely, onto a +`Common.WindowClosing` hook, because the size has to be read while the +window still exists and `OnShutdown` has neither a context nor a +window. + +`internalServiceMethods` auto-excludes `ServiceStartup`, +`ServiceShutdown`, `ServiceName` and `ServeHTTP` from bindings, so this +shape **removed** 12 spurious bindings and the bogus `context` model +rather than renaming them. **Backend packages** (under `backend/`): - `player` — Audio playback via beep. `BufferedStreamer` provides a ring buffer for smooth seeking. @@ -444,7 +505,11 @@ that invents its own flat layout agrees with the bug. keeping only `explore.ArtistImageKeepNames()` and refusing an empty keep set for the reason the covers sweep refuses an empty live set. -**Frontend** (`frontend/`): Lit 3.2 web components + Web Awesome UI library + HTMX. State management via singleton reactive stores in `src/store/`. Wails bindings auto-generated in `frontend/wailsjs/` — don't edit by hand. +**Frontend** (`frontend/`): Lit 3.2 web components + Web Awesome UI library + HTMX. State management via singleton reactive stores in `src/store/`. Wails bindings auto-generated as TypeScript in `frontend/bindings/`, nested by Go import path — don't edit by hand. The `@go` alias absorbs the constant prefix, so a call site imports `@go/library/library.js`. + +**One seam states what the generated types get wrong, rather than 78 patches.** v3's generator is honest where v2's lied: a Go `nil` slice marshals to JSON `null` and always has (v2 typed it `T[]`), and a Go named string type is a closed set (v2 typed it `string`). There is no flag to turn either off, correctly. So `utils/binding.ts` states the app's actual contract at the only place it is true — `list` yields `[]` for a nil slice, `dict`/`dictByName` yield `{}` for a nil map and drop null-valued keys (which loses nothing: `noUncheckedIndexedAccess` already makes every read `V | undefined`), and `compact` is the same for a map arriving as a *field*. Where a nullable slice is a model field there is no boundary to put it at, and those are `?? []` at the point of use. + +All three also return a **plain `Promise`**: v3 bindings return a `CancellablePromise` and nothing in this app cancels one, so letting it inward would put a Wails type in every store signature for a capability none of them use. **A view is a chunk, and three components are not.** `index.ts` holds a loader table (`VIEW_LOADERS`, `DETAIL_LOADERS`) and `await`s a view's @@ -1427,11 +1492,26 @@ on every change, or `scrollToIndex` from calling `scrollIntoView()` on something invisible. Emit through **`events.Emit(ctx, name, data...)`**, never -`runtime.EventsEmit` — wails `log.Fatalf`s (unrecoverably) on any -context that does not carry its runtime, which includes every -`context.Background()`, so a direct call cannot run under test and can -kill the app from a background worker. `TestNoDirectRuntimeEmits` fails -the build on a direct call anywhere outside `backend/events`. +`app.Event.Emit`. `TestNoDirectRuntimeEmits` fails the build on a +direct call anywhere outside `backend/events`. + +**Its justification changed with v3 and is now the weaker one.** Under +v2 this was a safety rule: `runtime.EventsEmit` `log.Fatalf`'d — +unrecoverably, taking the process down — on any context not carrying +the runtime, which includes every `context.Background()`, so a +background worker could kill the app by emitting. That is gone. v3's +emit takes **no context at all**, and `application.Get()` with no app +running returns `nil` rather than dying, so `Deliver` is +`if app == nil { return ErrNoRuntime }` where it used to probe the +**v2-private context key** `ctx.Value("events")`. What remains worth +pinning is narrower and still real: one emit path is what lets +`emitStatus` drop an unchanged payload for every caller at once. + +**`events.Emit` keeps its `ctx` anyway, and it is now purely a test +seam.** Delivery does not go through it; `events.WithSink(ctx, rec)` +does, which is how a service is asserted on in-process +(`backend/queue/emit_test.go` is the model). Dropping the parameter +would have churned 45 call sites and every test for no gain. That wrapper is what makes services testable in-process: install a recorder with `events.WithSink(ctx, rec)` and assert on the payload the @@ -1489,8 +1569,10 @@ Two jobs, both in an `ubuntu:24.04` container: `make lint` and `make test` (three build configurations each), `tsc --noEmit`, `make ui-test`, `make bindings-check`, `make skill-check`. -- **`e2e`** — under Xvfb and a private D-Bus: fixtures, a seed built by - running the app, `make dev-headless`, then the Playwright suite +- **`e2e`** — under a private D-Bus and **no display at all**, since + v3's `-tags server` is a real headless mode: fixtures, a seed built by + running the app (`curl` against the runtime endpoint — no browser), + `make dev-headless`, then the Playwright suite against **both** Chromium and WebKit. Playwright's Linux WebKit links Ubuntu 24.04 libraries that Arch does not provide, so CI is the only place it can run, and it is the closest available approximation of @@ -1528,3 +1610,44 @@ WebKitGTK 6.0, which Arch and ubuntu:24.04 both ship, so the `webkit2_41` tag that used to be mandatory everywhere is gone. A machine without `webkitgtk-6.0` can still build with `-tags gtk3`, but that is an escape hatch, not what CI or a release builds. + +## Packaging + +**The Makefile is the front door and `Taskfile.yml` is an +implementation detail behind it.** `make dev`, `make build-dev`, +`make build-prod`, `make bindings` and `make e2e` all keep their names; +what changed underneath is that a build is now a Taskfile tree +(`Taskfile.yml` → `build//Taskfile.yml`) rather than one +`wails build` invocation, and `wails3` is still a **vendored Go tool** +(`go tool wails3`), never a global install. + +Four things about that tree bite anything outside the Makefile, and all +four bit the packaging recipes: + +- **The tasks invoke `wails3` by bare name**, in 54 places across the + scaffold files. `scripts/toolbin/wails3` puts that name on PATH + pointing back at the vendored tool; without it a build dies at its + first sub-task with `wails3: command not found`. The Makefile + prepends it, and so must `packaging/arch/PKGBUILD` and the Homebrew + formula. +- **`wails3 build` has no `-ldflags`, `-trimpath` or `-clean`** — those + were v2's. `-trimpath` and `-w -s` are already in the production + task's own flags; the version stamp goes through **`LDFLAGS_EXTRA`**, + this repo's one edit to the scaffold Taskfiles (linux and darwin + alike), passed as `wails3 task build LDFLAGS_EXTRA="-X 'main.version=…'"`. +- **The output is `bin/`, not v2's `build/bin/`.** `build/` is *tracked + build assets* now. +- **Bundling is a separate step from building.** `task build` produces + a bare binary on every platform; the macOS `.app` is `task package`. + +**`build/`'s platform metadata is generated from `build/config.yml`.** +`wails3 task common:update:build-assets` rewrites `Info.plist`, the +`.desktop` template, `nfpm.yaml` and the Windows manifest from that +one file — so a hand edit to any of them is lost on the next refresh, +and the two fields it does *not* own (nfpm's `homepage` and `license`) +say so in place. That refresh also regenerates `build/ios/` and +`build/android/`, which this repo does not carry: they are gitignored +rather than deleted-and-rediscovered, and their `includes:` entries +are dropped from `Taskfile.yml`. `build/config.yml`'s `version` is the +*metadata* version and is not what the app reports — `main.version` is +stamped at link time from the packaging recipe's git-derived version. diff --git a/Makefile b/Makefile index 4351b3c..c8409e5 100644 --- a/Makefile +++ b/Makefile @@ -21,10 +21,11 @@ dev-debug: setup generate clean if [ -f .env ]; then set -a; . ./.env; set +a; fi; : "$${YJ_HOME:=$(DEV_YJ_HOME)}"; export YJ_HOME; YJ_LOG_LEVEL=debug PATH="$(TOOLBIN):$$PATH" go tool wails3 dev -config ./build/config.yml # ── Headless harness (plan 005) ────────────────────────────────────── -# The same dev server `make dev` runs, minus the blocking GTK window: -# Xvfb gives it the display it insists on, and the script returns once -# :34115 answers. This is the only entry point an agent can use, since -# every other one blocks the terminal forever. +# The same app `make dev` runs, minus the window: v3's `-tags server` +# is a first-class headless mode that needs no display at all, so the +# Xvfb this used to require is gone. The script returns once :34115 +# answers. This is the only entry point an agent can use, since every +# other one blocks the terminal forever. dev-headless: ## Start the app headless in the background (SEED= to seed) @./scripts/dev-headless.sh $(if $(SEED),--seed $(SEED),) $(HEADLESS_ARGS) diff --git a/README.md b/README.md index 394987b..a540431 100644 --- a/README.md +++ b/README.md @@ -78,16 +78,25 @@ YellowJacket is built with [Go](https://go.dev/) and a | Go | 1.25+ | | Node.js | 22+ | | pnpm | 10+ | -| Wails CLI | v2 (`go install github.com/wailsapp/wails/v2/cmd/wails@latest`) | +| Wails CLI | v3 — vendored, no install needed (`go tool wails3`) | -On Linux, install the system libraries Wails needs: +The Wails v3 CLI resolves from the `tool` block in `go.mod`, so there is nothing +to install globally; `make setup` fetches it with the rest of the tooling. + +On Linux, install the system libraries Wails needs. v3 builds against GTK4 + +WebKitGTK 6.0 by default: ```bash -sudo apt-get install libasound2-dev libgtk-3-dev libwebkit2gtk-4.1-dev +sudo apt-get install libasound2-dev libgtk-4-dev libwebkitgtk-6.0-dev # Debian/Ubuntu +sudo pacman -S alsa-lib gtk4 webkitgtk-6.0 # Arch ``` -macOS and Windows need no extra system packages. Run `wails doctor` to check your -environment. +A machine without `webkitgtk-6.0` can still build with `-tags gtk3` against the +older WebKit2GTK 4.1 stack, but that is an escape hatch, not what CI or a +release builds. + +macOS and Windows need no extra system packages. Run `go tool wails3 doctor` to +check your environment. **Build**