yonlu/yellowjacket
1
1
Fork 0
Code Issues 55 Pull Requests 6 Actions Packages Projects Releases 12 Wiki Activity

e2e state leaks between spec files: a staged job outlives the spec that staged it #168

New Issue
Open
opened 2026-08-21 00:19:34 +00:00 by logan · 1 comment
logan commented 2026-08-21 00:19:34 +00:00
Collaborator
Copy Link
Copy Source

Report

The e2e app is one long-lived process shared by every spec file and by
every make e2e invocation, and /__test/emit writes to stores that
nothing clears. So state staged by one spec file is still there for the
next, and for the next run.

Tripped over while doing #57. A draft assertion read "with the top bar
gone, the content starts at y=0". It passed run after run in isolation
and failed inside a suite run, because top-bar-fit.spec.ts had staged
a long-titled scan with testctl.emit('JobsChanged', …) and <job-band>
is a real grid row whenever work is in flight (#62) — so the content
started at 55.

Findings

  • dev-headless.sh daemonises and make e2e drives the app that is
    already running; this is deliberate and documented (starting one per
    run would rebuild the frontend every time).
  • /__test/emit is exactly the right tool and is not the problem —
    the point of it is to force a backend event without staging the work.
    What is missing is any statement of who owns the state afterwards.
  • jobs-on-a-phone.spec.ts and top-bar-fit.spec.ts both stage jobs
    and neither clears them; nothing has needed them to until a third
    spec measured a layout.
  • The failure mode is the worst kind: the spec that leaks passes,
    and an unrelated spec fails     — and it fails only in a suite run and
    only in file order, so it reads as a flake.

Direction

Two halves, and the second is the one that generalises.

Give testctl a way to say "no jobs" and have the specs that stage
them clean up — emit('JobsChanged', []) in an afterEach is enough
and needs no backend change.

And write down the rule that actually prevents this, which #57 arrived
at by accident: measure against the thing next to you, not against an
absolute coordinate. contentTop === 0 was quietly also asserting
"and no background job is running", which is not something that spec
was about or could arrange; contentTop === jobBandBottom is true
either way. An absolute coordinate in a shell measurement is an implicit
claim about everything above it.

Filed rather than fixed here because the cleanup belongs with whoever
next touches the job specs, and because the rule is worth stating
before it is enforced.

**Report** The e2e app is one long-lived process shared by every spec file and by every `make e2e` invocation, and `/__test/emit` writes to stores that nothing clears. So state staged by one spec file is still there for the next, and for the next *run*. Tripped over while doing #57. A draft assertion read "with the top bar gone, the content starts at y=0". It passed run after run in isolation and failed inside a suite run, because `top-bar-fit.spec.ts` had staged a long-titled scan with `testctl.emit('JobsChanged', …)` and `<job-band>` is a real grid row whenever work is in flight (#62) — so the content started at 55. **Findings** - `dev-headless.sh` daemonises and `make e2e` drives the app that is already running; this is deliberate and documented (starting one per run would rebuild the frontend every time). - `/__test/emit` is exactly the right tool and is not the problem — the point of it is to force a backend event without staging the work. What is missing is any statement of who owns the state afterwards. - `jobs-on-a-phone.spec.ts` and `top-bar-fit.spec.ts` both stage jobs and neither clears them; nothing has needed them to until a third spec measured a layout. - The failure mode is the worst kind: **the spec that leaks passes, and an unrelated spec fails** — and it fails only in a suite run and only in file order, so it reads as a flake. **Direction** Two halves, and the second is the one that generalises. Give `testctl` a way to say "no jobs" and have the specs that stage them clean up — `emit('JobsChanged', [])` in an `afterEach` is enough and needs no backend change. And write down the rule that actually prevents this, which #57 arrived at by accident: **measure against the thing next to you, not against an absolute coordinate.** `contentTop === 0` was quietly also asserting "and no background job is running", which is not something that spec was about or could arrange; `contentTop === jobBandBottom` is true either way. An absolute coordinate in a shell measurement is an implicit claim about everything above it. Filed rather than fixed here because the cleanup belongs with whoever next touches the job specs, and because the rule is worth stating before it is enforced.
logan added the Kind/Testing
Reviewed
Confirmed
1
Priority
Low
4
 labels 2026-08-21 00:19:34 +00:00
logan commented 2026-08-26 11:41:30 +00:00
Author
Collaborator
Copy Link
Copy Source

The leak half does not reproduce, measured under this issue's own
conditions. Posting the evidence rather than acting on it, because
the fix this asks for — an afterEach clearing jobs — would be a
change that fixes nothing, and the second half is worth keeping either
way.

What was measured. A throwaway probe spec was added that makes
exactly the draft assertion described here — at 390px, with the top bar
gone, where does the content start — and reports the numbers rather
than asserting them. zz- so it sorts last, i.e. after both
jobs-on-a-phone.spec.ts and top-bar-fit.spec.ts.

alone:                       PROBE {"contentTop":0,"bandBottom":0,"bandRows":0}
after the two job specs:     PROBE {"contentTop":0,"bandBottom":0,"bandRows":0}
in a full `make e2e` run:    PROBE {"contentTop":0,"bandBottom":0,"bandRows":0}   (245 passed)

contentTop is 0 in a suite run, which is the value this issue says
fails there. A second probe asked the backend directly, on a fresh page
straight after top-bar-fit.spec.ts had staged its long-titled scan:

{"stagedIds":[],"activeFromBackend":[],"bandRows":0,"bandVisible":true,"bandHTML":"<!---->"}

No top-bar-fit or phone: job in jobs.Service.GetJobs, and the band
renders an empty template.

Why, and it is structural rather than lucky. /__test/emit calls
events.Deliver and nothing else — it does not touch the job registry,
which is the point of it ("force a backend event without staging the
work"). Playwright gives each test a fresh context and page, and
job-store is a singleton in that page which rebuilds itself from
GetJobs() on load. So a staged JobsChanged reaches exactly the one
page that was open when it was emitted, and the next test — in the same
file or a different one — starts from the backend's own registry, which
never heard about it.

So there is no state for an afterEach to clear. Two smaller
consequences fall out of the same fact and are worth knowing before
someone writes another one: top-bar-fit.spec.ts's closing "Leave the
app as the next spec expects to find it" viewport reset and
queue-selection.spec.ts's comment that "an open panel outlives the
page" are both written on this belief. The queue one is genuinely
different — the queue is backend state and does outlive the page —
but the open panel is not.

What is still true is the rule, and it is the half that generalises:
contentTop === 0 was quietly also asserting "and no background job is
running". That is not a leak between files; it is an absolute
coordinate carrying an implicit claim about everything above it, and
the current spec is right to measure main.top - band.bottom instead.
It happens to be the case that the job it measures against is one its
own test stages.

Left for a decision rather than taken: whether this becomes a
docs-only issue for the rule (.pi/skills/yellowjacket-dev/references/
would be the place), or is closed as Reviewed/Invalid with the rule
folded into whichever spec next needs it. Either is defensible and
neither is mine to pick; nothing has been changed and no branch was
pushed.

Measured on main at 245647f, against make dev-headless SEED=default
with YJ_CORE_INDEX_URL pointed at a dead address.

**The leak half does not reproduce, measured under this issue's own conditions.** Posting the evidence rather than acting on it, because the fix this asks for — an `afterEach` clearing jobs — would be a change that fixes nothing, and the second half is worth keeping either way. **What was measured.** A throwaway probe spec was added that makes exactly the draft assertion described here — at 390px, with the top bar gone, where does the content start — and reports the numbers rather than asserting them. `zz-` so it sorts last, i.e. after both `jobs-on-a-phone.spec.ts` and `top-bar-fit.spec.ts`. ``` alone: PROBE {"contentTop":0,"bandBottom":0,"bandRows":0} after the two job specs: PROBE {"contentTop":0,"bandBottom":0,"bandRows":0} in a full `make e2e` run: PROBE {"contentTop":0,"bandBottom":0,"bandRows":0} (245 passed) ``` `contentTop` is 0 in a suite run, which is the value this issue says fails there. A second probe asked the backend directly, on a fresh page straight after `top-bar-fit.spec.ts` had staged its long-titled scan: ``` {"stagedIds":[],"activeFromBackend":[],"bandRows":0,"bandVisible":true,"bandHTML":"<!---->"} ``` No `top-bar-fit` or `phone:` job in `jobs.Service.GetJobs`, and the band renders an empty template. **Why, and it is structural rather than lucky.** `/__test/emit` calls `events.Deliver` and nothing else — it does not touch the job registry, which is the point of it ("force a backend event without staging the work"). Playwright gives each *test* a fresh context and page, and `job-store` is a singleton in that page which rebuilds itself from `GetJobs()` on load. So a staged `JobsChanged` reaches exactly the one page that was open when it was emitted, and the next test — in the same file or a different one — starts from the backend's own registry, which never heard about it. So there is no state for an `afterEach` to clear. Two smaller consequences fall out of the same fact and are worth knowing before someone writes another one: `top-bar-fit.spec.ts`'s closing "Leave the app as the next spec expects to find it" viewport reset and `queue-selection.spec.ts`'s comment that "an open panel outlives the page" are both written on this belief. The queue one is genuinely different — the *queue* is backend state and does outlive the page — but the open panel is not. **What is still true is the rule**, and it is the half that generalises: `contentTop === 0` was quietly also asserting "and no background job is running". That is not a leak between files; it is an absolute coordinate carrying an implicit claim about everything above it, and the current spec is right to measure `main.top - band.bottom` instead. It happens to be the case that the job it measures against is one its own test stages. **Left for a decision** rather than taken: whether this becomes a docs-only issue for the rule (`.pi/skills/yellowjacket-dev/references/` would be the place), or is closed as `Reviewed/Invalid` with the rule folded into whichever spec next needs it. Either is defensible and neither is mine to pick; nothing has been changed and no branch was pushed. Measured on `main` at 245647f, against `make dev-headless SEED=default` with `YJ_CORE_INDEX_URL` pointed at a dead address.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Area/Design
Area/Downloads
Area/Explore
Area/Library-UI
Area/Metadata
Area/Packaging
Area/Player
Area/Queue
Area/Settings
Area/Shell-Nav
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
 Platform/Android
Platform/Desktop
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Medium
3
The priority is medium
Priority
Low
4
The priority is low
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
In Progress
Somebody is actively working on this right now
No labels Kind/Testing
Priority
Low
4
Reviewed
Confirmed
1
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
logan yonlu
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: yonlu/yellowjacket#168