From 2ed9e725e34ff6270538f36f15f42061c01fbd77 Mon Sep 17 00:00:00 2001 From: Caleb Allen Date: Tue, 6 Oct 2026 01:59:57 -0400 Subject: [PATCH] docs(planning): what the library payloads cost, measured MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan 023 (#279–#284): the before/after table on 50 000 tracks, and the six things worth keeping — including two measurements that pointed away from where the work had been (mmap is a bound, and the remaining second is Wails encoding every result twice). --- .planning/NOTES.md | 84 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/.planning/NOTES.md b/.planning/NOTES.md index b429d49..8643283 100644 --- a/.planning/NOTES.md +++ b/.planning/NOTES.md @@ -5097,3 +5097,87 @@ beside what they explain. These three did not: The drawer-style gutter would buy the affordance by taking width off a full-screen surface on a 424px viewport; back and a 44px close button answer it instead. + +## What the library payloads actually cost (measured 2026-10-05, desktop dev build) + +Plan 023 (#279–#284). Measured against `make sandbox-seed-bulk` +(50 000 tracks, 371 MB `yj.db` in the smaller copy) with the desktop +dev build and `e2e/perf/measure.mjs`, which grew a `memory` section for +it: backend RSS and peak from `/proc`, the Go heap from pprof, the +page's JS heap and DOM counters from CDP, and the bytes every binding +returned, before and after the first open of Tracks. + +| | before | after #281 | after #280 | +|---|---|---|---| +| Backend RSS at rest | 543 MB | 296 MB | 189 MB | +| Backend peak RSS | 571 MB | 337 MB | 281 MB | +| Go heap held | 361 MB | 125 MB | 7 MB | +| JS heap at rest | 31.8 MB | 18.0 MB | 5.2 MB | +| Binding bytes at rest | 35.9 MB | 12.1 MB | 1.7 MB | +| JS heap after a browse | 36.5 MB | 22.7 MB | 23.1 MB | +| Tracks first open → first row | 38 ms | 26 ms | 1 255 ms | +| Slowest first view open | 44 ms | 57 ms | 76 ms | + +Six things worth keeping: + +- **The number that fell was not the number being optimised.** The + binding payload fell 20.5 → 10.45 MB (dictionary-encoded columns) and + 35.9 → 12.1 MB, but what made the backend's RSS fall by 354 MB was + the *fetch not happening*: `GetTracks` was the only thing in the app + that allocated 170 MB transiently (`sqlcgen.GetTracks` 80 MB, `json/v2` + 128 MB, `slices.Grow` 77 MB, `bytes.Clone` 47 MB of 484 MB total + `alloc_space`). Encoding smaller would not have touched that. +- **A `dictionary` is what makes dropping fields the wrong trade.** The + first version of the column table left out `LastPlayed` and the three + larger cover tiers, and then a "select all → edit tags" over 50 000 + tracks had to fetch every track back to open the dialog: 3 071 ms + against 88 ms before. Interning means the repeated strings are nearly + free, so the fields belong in the table; what fixed it in the end was + neither — the Tracks view hands the dialog the rows it already has. + **A payload optimisation that removes a field is a new fetch waiting + to be written.** +- **`mmap_size` is a bound, not an allocation.** #283 read as "64 MB of + mapping per connection", and the RSS of the database mapping is 46 MB + whether the bound is 64 MB or 16 MB, because a mapping costs what the + working set touches. Quartering it moved no query either (1 172 → + 1 197 ms for the whole track list, 106 → 124 ms for the album list, 6 + → 8 ms for an FTS search — all inside the run-to-run spread). Kept for + the phone, where the bound is the address space the low-memory killer + reads. +- **A view that is first paint is a view that is active.** `index.html` + renders a ``, so the track list was connected at launch + and fetched 12 MB whoever was looking at — and the fix was not in the + component but in the shell: markup is not a decision about which view + the launch lands on, so the seed is `view-hidden` and the first + navigation is what activates it. Hover and keyboard focus on a nav + item prefetch, which is the ~100 ms before the click. +- **The remaining second is the transport, not the data.** A cold open + of Tracks on 50 000 tracks is 1 255 ms, and a *raw* binding call for + the same table is 1 118–1 382 ms: none of it is the TypeScript decode + or the render. It is the Go-side query, the column encode and Wails + encoding every result twice — once for a debug log that is off + (#286). Splitting that number is what says where to go next, and the + answer was not where the payload work had been. +- **A test named for the behaviour caught the thing the design missed.** + The source sweep that pins "the whole-library track array has one + reader" was written to catch the four call sites #279 had already + converted; it found `smart-playlist-editor`, which built its value + suggestions from the same array and would have gone silently empty + once nothing loaded it. The suggestion box is a backend query now + (`SuggestSmartPlaylistValues`). + +And two things about this machine, since they cost a cycle each: + +- **`make ui-test` is not reliable at load average 14.** Under the + workstation's own background services, the browser provider's module + fetches fail in a different handful of files each run ("Failed to + import test file", "Cannot connect to the iframe") while every test + that runs passes. `--maxWorkers=1 --retry=2` reduces it; individual + files always pass. A failure list that changes between runs is the + environment, not the branch. +- **A measurement run inherits the machine's mood.** The first + after-#281 numbers said view opens had doubled (albums 27 → 78 ms, + settings 44 → 179 ms, Tracks first row 38 → 105 ms). Re-running + unchanged gave 26 ms and 57 ms. The tell was the same one + `NOTES.md` already records twice: before and after suspiciously + equal, or suspiciously worse, across *unrelated* measurements.