yonlu/yellowjacket
1
1
Fork 0
Code Issues 55 Pull Requests 6 Actions Packages Projects Releases 12 Wiki Activity

make ui-test: folder-picker and explore-track-details time out in full-suite runs on main #140

New Issue
Open
opened 2026-08-19 20:35:43 +00:00 by logan · 1 comment
logan commented 2026-08-19 20:35:43 +00:00
Collaborator
Copy Link
Copy Source

Report

A full make ui-test run on pristine main fails four or five tests
that pass in isolation. Reproduced twice while verifying an unrelated
branch, by stashing the branch's changes so the tree was clean main
(f967916):

FAIL |chromium| test/components/explore-track-details.test.ts > opens the dialog on the library track behind the row
FAIL |chromium| test/components/folder-picker.test.ts > browses in-app on Android, and never opens the platform dialog
FAIL |chromium| test/components/folder-picker.test.ts > resolves null when the browser is cancelled
FAIL |chromium| test/components/folder-picker.test.ts > descends into a folder and returns the one it is showing
FAIL |chromium| test/components/folder-picker.test.ts > stays put and explains when a folder cannot be opened

The failures are Error: Test timed out in 15000ms., not assertion
failures — which is what distinguishes this from #138, where the
assertion is a real ordering disagreement.

Reproduction

git checkout main            # f967916, clean tree
make ui-test                 # 2 of 2 runs failed; the exact set varied
cd frontend && npx vitest run --browser.headless \
    test/components/folder-picker.test.ts \
    test/components/explore-track-details.test.ts
# 11 passed, 2 runs of 2

What is ruled out

  • Not introduced by any local change: reproduced with a stashed tree on
    main, twice, and the same two runs of the same files in isolation
    passed.
  • Not #138: that one fails an assertion about sort order and did not
    appear in either of these runs. The two may still share a cause —
    both need the full suite's timing to appear.

The lead, stated as a lead

The run's own numbers say the machine is saturated before these files
start: setup 160.94s and import 69.05s against tests 53.68s, for
a wall time of 38s across 82 parallel browser files. A 15s per-test
timeout is measured from when vitest starts the test, so a file that
waits on a browser context that is not scheduled fails without anything
in it being wrong. folder-picker awaits a dialog and
explore-track-details awaits a lazily-imported chunk — both are
"wait for something that has to be scheduled", which is the shape that
loses a race for the CPU rather than a race in the code.

If that is right, the fix is a per-test timeout or a concurrency cap
rather than anything in the two suites; if it is not, the flake is
theirs. Worth measuring --fileParallelism settings before assuming.

Why it matters

make ui-test runs in CI's check job, and CI is capacity 1 — a
suite that fails for reasons unrelated to the push costs a ~3 minute
re-run and, worse, trains everyone to re-run rather than read.

**Report** A full `make ui-test` run on pristine `main` fails four or five tests that pass in isolation. Reproduced twice while verifying an unrelated branch, by stashing the branch's changes so the tree was clean `main` (`f967916`): ``` FAIL |chromium| test/components/explore-track-details.test.ts > opens the dialog on the library track behind the row FAIL |chromium| test/components/folder-picker.test.ts > browses in-app on Android, and never opens the platform dialog FAIL |chromium| test/components/folder-picker.test.ts > resolves null when the browser is cancelled FAIL |chromium| test/components/folder-picker.test.ts > descends into a folder and returns the one it is showing FAIL |chromium| test/components/folder-picker.test.ts > stays put and explains when a folder cannot be opened ``` The failures are `Error: Test timed out in 15000ms.`, not assertion failures — which is what distinguishes this from #138, where the assertion is a real ordering disagreement. **Reproduction** ``` git checkout main # f967916, clean tree make ui-test # 2 of 2 runs failed; the exact set varied cd frontend && npx vitest run --browser.headless \ test/components/folder-picker.test.ts \ test/components/explore-track-details.test.ts # 11 passed, 2 runs of 2 ``` **What is ruled out** - Not introduced by any local change: reproduced with a stashed tree on `main`, twice, and the same two runs of the same files in isolation passed. - Not #138: that one fails an assertion about sort order and did not appear in either of these runs. The two may still share a cause — both need the *full* suite's timing to appear. **The lead, stated as a lead** The run's own numbers say the machine is saturated before these files start: `setup 160.94s` and `import 69.05s` against `tests 53.68s`, for a wall time of 38s across 82 parallel browser files. A 15s per-test timeout is measured from when vitest starts the test, so a file that waits on a browser context that is not scheduled fails without anything in it being wrong. `folder-picker` awaits a dialog and `explore-track-details` awaits a lazily-imported chunk — both are "wait for something that has to be scheduled", which is the shape that loses a race for the CPU rather than a race in the code. If that is right, the fix is a per-test timeout or a concurrency cap rather than anything in the two suites; if it is not, the flake is theirs. Worth measuring `--fileParallelism` settings before assuming. **Why it matters** `make ui-test` runs in CI's `check` job, and CI is capacity 1 — a suite that fails for reasons unrelated to the push costs a ~3 minute re-run and, worse, trains everyone to re-run rather than read.
logan added the
Priority
Medium
3
Reviewed
Confirmed
1
Area/Library-UIKind/Testing labels 2026-08-19 20:35:43 +00:00
logan commented 2026-08-24 11:40:43 +00:00
Author
Collaborator
Copy Link
Copy Source

Not picked up: it did not reproduce here, in seven full runs, including
four at this issue's own saturation numbers.

I went looking for this specifically, because #219 (open, fix/138-ui-test-storage-leak)
found that @vitest/browser-playwright reuses one BrowserContext across
test files, so localStorage leaks between them — 24 polluted test-starts in
one full run, cascading into 248 failures — and the question was whether #140
is the same root cause wearing a different symptom.

What I ran. npx vitest run (the whole of make ui-test), 24-core
machine, vitest 4.1.10:

# tree machine wall setup import tests result
1–3 #219's branch idle 13.1s ~43s ~18.7s ~32s 1092 passed
4 #219's branch loaded 30.4s 143.0s 69.8s 37.3s 1092 passed
5–7 test/setup.ts + play-in-context.test.ts reverted to origin/main loaded 28.9–30.8s 133.0–149.0s 65.2–70.0s ~37s 1092 passed

The load was 36 spinning processes on 24 cores, picked to land on the
profile quoted in the report (setup 160.94s, import 69.05s,
tests 53.68s, wall 38s). Setup and import matched within ~10%; tests
came in at 37s against the reported 53.7s, so the saturation is in the
right region but not identical.

Runs 5–7 are the counterfactual for the #219 question: those two files are
the entirety of #219's diff outside CLAUDE.md, so reverting them is
pristine main as far as this tier can tell. Neither #140's five timeouts
nor #138's assertion appeared, with or without the fix. So this measurement
does not show that #219 fixes #140 — it shows I could not reproduce #140 at
all, which is the reason I am leaving it rather than claiming it. Per the
scheduled-run rules, an intermittent failure that will not reproduce on
demand is an unbounded task.

What it is not. The localStorage leak's signature is a wrong value —
a sort order, a column width, a cover size — and #140's is
Test timed out in 15000ms. Neither named file mounts a component that
persists anything: folder-picker reads no storage at all, and
explore-track-details mounts explore-album-details, which renders its
own .track-row list rather than <track-list>, so none of the three
polluted key families (track-list-sort-*, track-list-column-widths,
cover-grid-size) can reach it. I would not treat #219 as closing this.

One lead, from reading the two files rather than from a measurement.
Every poll in them is finite and setTimeout-based — folder-picker's
host() is 50 × 10ms, explore-track-details's dialogTrack() and
pickerPaths() are 100 × setTimeout(0) — and exhausting one of those
budgets fails as an assertion (folder-picker did not mount itself,
expected undefined to be '/music/an-album/01.flac'), never as a 15s
timeout. So a timeout means some single await did not settle at all, and
it is not "the poll is too short": raising a poll budget or the per-test
timeout would be treating the symptom of something else.

The one unbounded wait in both files is the same one: a dynamic import()
of a lazily-loaded chunk — pickDirectory() → import('../components/folder-picker/folder-picker'),
and showTrackDetailsForPath() → the track-details chunk. Both are
source modules, so they are transformed on demand by the Vite dev server
(a single Node process shared with 82 parallel browser files) rather than
being covered by optimizeDeps.include, which only pre-bundles
node_modules deps. That is the shape the config's own optimizeDeps
comment already exists for one layer up. Worth checking a failing run's
output for new dependencies optimized / a page reload before assuming;
for what it is worth, none of my seven runs logged either.

Leaving it open, unlabelled and unclaimed.

**Not picked up: it did not reproduce here, in seven full runs, including four at this issue's own saturation numbers.** I went looking for this specifically, because #219 (open, `fix/138-ui-test-storage-leak`) found that `@vitest/browser-playwright` reuses one `BrowserContext` across test files, so `localStorage` leaks between them — 24 polluted test-starts in one full run, cascading into 248 failures — and the question was whether #140 is the same root cause wearing a different symptom. **What I ran.** `npx vitest run` (the whole of `make ui-test`), 24-core machine, vitest 4.1.10: | # | tree | machine | wall | setup | import | tests | result | |---|------|---------|------|-------|--------|-------|--------| | 1–3 | #219's branch | idle | 13.1s | ~43s | ~18.7s | ~32s | 1092 passed | | 4 | #219's branch | loaded | 30.4s | 143.0s | 69.8s | 37.3s | 1092 passed | | 5–7 | `test/setup.ts` + `play-in-context.test.ts` reverted to `origin/main` | loaded | 28.9–30.8s | 133.0–149.0s | 65.2–70.0s | ~37s | 1092 passed | The load was 36 spinning processes on 24 cores, picked to land on the profile quoted in the report (`setup 160.94s`, `import 69.05s`, `tests 53.68s`, wall 38s). Setup and import matched within ~10%; `tests` came in at 37s against the reported 53.7s, so the saturation is in the right region but not identical. Runs 5–7 are the counterfactual for the #219 question: those two files are the entirety of #219's diff outside `CLAUDE.md`, so reverting them is pristine `main` as far as this tier can tell. **Neither #140's five timeouts nor #138's assertion appeared, with or without the fix.** So this measurement does not show that #219 fixes #140 — it shows I could not reproduce #140 at all, which is the reason I am leaving it rather than claiming it. Per the scheduled-run rules, an intermittent failure that will not reproduce on demand is an unbounded task. **What it is not.** The localStorage leak's signature is a wrong *value* — a sort order, a column width, a cover size — and #140's is `Test timed out in 15000ms`. Neither named file mounts a component that persists anything: `folder-picker` reads no storage at all, and `explore-track-details` mounts `explore-album-details`, which renders its own `.track-row` list rather than `<track-list>`, so none of the three polluted key families (`track-list-sort-*`, `track-list-column-widths`, `cover-grid-size`) can reach it. I would not treat #219 as closing this. **One lead, from reading the two files rather than from a measurement.** Every poll in them is finite and `setTimeout`-based — `folder-picker`'s `host()` is 50 × 10ms, `explore-track-details`'s `dialogTrack()` and `pickerPaths()` are 100 × `setTimeout(0)` — and exhausting one of those budgets fails as an *assertion* (`folder-picker did not mount itself`, `expected undefined to be '/music/an-album/01.flac'`), never as a 15s timeout. So a timeout means some single `await` did not settle at all, and it is not "the poll is too short": raising a poll budget or the per-test timeout would be treating the symptom of something else. The one unbounded wait in both files is the same one: a dynamic `import()` of a lazily-loaded chunk — `pickDirectory()` → `import('../components/folder-picker/folder-picker')`, and `showTrackDetailsForPath()` → the `track-details` chunk. Both are *source* modules, so they are transformed on demand by the Vite dev server (a single Node process shared with 82 parallel browser files) rather than being covered by `optimizeDeps.include`, which only pre-bundles `node_modules` deps. That is the shape the config's own `optimizeDeps` comment already exists for one layer up. Worth checking a failing run's output for `new dependencies optimized` / a page reload before assuming; for what it is worth, none of my seven runs logged either. Leaving it open, unlabelled and unclaimed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Area/Design
Area/Downloads
Area/Explore
Area/Library-UI
Area/Metadata
Area/Packaging
Area/Player
Area/Queue
Area/Settings
Area/Shell-Nav
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
 Platform/Android
Platform/Desktop
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Medium
3
The priority is medium
Priority
Low
4
The priority is low
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
In Progress
Somebody is actively working on this right now
No labels Area/Library-UI Kind/Testing
Priority
Medium
3
Reviewed
Confirmed
1
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
logan yonlu
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: yonlu/yellowjacket#140