The issue.TestYtDlpSearchSkipsUnparseableLines failed once in a
full make test run and once in CI, both times on a tree whose diff
contained no Go at all, with fork/exec …/yt-dlp: text file busy. #146 diagnosed the mechanism:
the kernel refuses to exec a file that is open for writing anywhere in
the process, and these tests are t.Parallel(), so a sibling's fork duplicates stubYtDlp's write descriptor in the moment it is
open and carries it past our close.
What changed. The write in stubYtDlp is held under syscall.ForkLock — six lines and a comment, in one test file. No
production code, nothing user-visible.
Why that and not either fix the issue named. "Close the file before
any sibling can fork" is not available: os.WriteFile has already
closed it before anything execs, and the offending descriptor is a copy taken by a fork that was already in flight. "Write the stub once
per package into a shared dir" narrows the window without removing it,
since the first write of each script still races. ForkLock is the
lock syscall.forkExec takes across the fork itself, so holding it
over the write means no child can exist while the descriptor does, and
a child forked earlier cannot hold one for a file that did not yet
exist. It is declared on Windows and darwin as well as Unix, which
matters because this helper skips at runtime rather than behind a build
tag.
Verification.
tier
result
throwaway probe in backend/download (12 goroutines × 200 real stubYtDlp calls + exec)
189 / 176 / 178 ETXTBSY of 2400 before, 0 / 0 / 0 after
standalone harness outside the repo
271–302 of 4800 before, 0 of 4800 after
go test -race ./backend/download/, ×3
green
make lint (three build configurations)
0 issues
make test (all three passes), ×2
green
GOOS=windows go vet ./backend/download/
passes (GOOS=darwin fails for the documented wails/v3/pkg/mac cgo reason, unrelated)
The probe is not committed: it is a measurement, and a stress test
that only sometimes fails is the class of thing this issue is about.
The before/after numbers above are what it produced.
Deliberately not done.
make ui-test, make e2e, make generate, make bindings — no
frontend, no .sql, no .templ, no bound signature.
CLAUDE.md untouched: it documents no test helper at this level, and
four open PRs are each editing that file in a different place.
The original failure is still not reproducible as a failure; what
is now impossible is the mechanism. A recurrence would mean a
different cause and deserves its own issue.
No new issues filed — stubYtDlp is the only test helper in the repo
that writes an executable and then execs it (cmd/indexbuild/deps_test.go
execs go list), so there is no second site.
**The issue.** `TestYtDlpSearchSkipsUnparseableLines` failed once in a
full `make test` run and once in CI, both times on a tree whose diff
contained no Go at all, with
`fork/exec …/yt-dlp: text file busy`. #146 diagnosed the mechanism:
the kernel refuses to exec a file that is open for writing anywhere in
the process, and these tests are `t.Parallel()`, so a **sibling's**
`fork` duplicates `stubYtDlp`'s write descriptor in the moment it is
open and carries it past our close.
**What changed.** The write in `stubYtDlp` is held under
`syscall.ForkLock` — six lines and a comment, in one test file. No
production code, nothing user-visible.
**Why that and not either fix the issue named.** "Close the file before
any sibling can fork" is not available: `os.WriteFile` has already
closed it before anything execs, and the offending descriptor is a
*copy* taken by a fork that was already in flight. "Write the stub once
per package into a shared dir" narrows the window without removing it,
since the first write of each script still races. `ForkLock` is the
lock `syscall.forkExec` takes across the fork itself, so holding it
over the write means no child can exist while the descriptor does, and
a child forked earlier cannot hold one for a file that did not yet
exist. It is declared on Windows and darwin as well as Unix, which
matters because this helper skips at runtime rather than behind a build
tag.
**Verification.**
| tier | result |
| --- | --- |
| throwaway probe in `backend/download` (12 goroutines × 200 real `stubYtDlp` calls + exec) | **189 / 176 / 178** ETXTBSY of 2400 before, **0 / 0 / 0** after |
| standalone harness outside the repo | 271–302 of 4800 before, 0 of 4800 after |
| `go test -race ./backend/download/`, ×3 | green |
| `make lint` (three build configurations) | 0 issues |
| `make test` (all three passes), ×2 | green |
| `GOOS=windows go vet ./backend/download/` | passes (`GOOS=darwin` fails for the documented `wails/v3/pkg/mac` cgo reason, unrelated) |
The probe is **not** committed: it is a measurement, and a stress test
that only sometimes fails is the class of thing this issue is about.
The before/after numbers above are what it produced.
**Deliberately not done.**
- `make ui-test`, `make e2e`, `make generate`, `make bindings` — no
frontend, no `.sql`, no `.templ`, no bound signature.
- `CLAUDE.md` untouched: it documents no test helper at this level, and
four open PRs are each editing that file in a different place.
- The original failure is still not reproducible *as a failure*; what
is now impossible is the mechanism. A recurrence would mean a
different cause and deserves its own issue.
No new issues filed — `stubYtDlp` is the only test helper in the repo
that writes an executable and then execs it (`cmd/indexbuild/deps_test.go`
execs `go list`), so there is no second site.
Closes #146
The kernel refuses to exec a file that is open for writing anywhere in
the process, and these tests are parallel: a sibling's fork duplicates
stubYtDlp's write descriptor in the moment it is open and carries it
past our close, so the exec a moment later fails with ETXTBSY. That is
the flake seen once locally and once in CI, both times on a tree with
no Go in its diff.
Closing sooner is not available -- os.WriteFile has already closed the
file before anything execs it -- and O_CLOEXEC does not help, because
the window is between another goroutine's fork and its own exec.
syscall.ForkLock is the lock forkExec takes across that fork, so
holding it over the write means no child can exist while the
descriptor does.
Measured on the helper itself under 12 concurrent writers: 176-189 of
2400 execs refused before, 0 of 2400 after, three runs each.
Closes#146
CI is green. Run 18482: check success, e2e success (chromium
and WebKit both ran).
Not merging — leaving it for a human, with Status/In Progress still on #146.
**CI is green.** Run 18482: `check` success, `e2e` success (chromium
and WebKit both ran).
Not merging — leaving it for a human, with `Status/In Progress` still on
#146.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
The issue.
TestYtDlpSearchSkipsUnparseableLinesfailed once in afull
make testrun and once in CI, both times on a tree whose diffcontained no Go at all, with
fork/exec …/yt-dlp: text file busy. #146 diagnosed the mechanism:the kernel refuses to exec a file that is open for writing anywhere in
the process, and these tests are
t.Parallel(), so a sibling'sforkduplicatesstubYtDlp's write descriptor in the moment it isopen and carries it past our close.
What changed. The write in
stubYtDlpis held undersyscall.ForkLock— six lines and a comment, in one test file. Noproduction code, nothing user-visible.
Why that and not either fix the issue named. "Close the file before
any sibling can fork" is not available:
os.WriteFilehas alreadyclosed it before anything execs, and the offending descriptor is a
copy taken by a fork that was already in flight. "Write the stub once
per package into a shared dir" narrows the window without removing it,
since the first write of each script still races.
ForkLockis thelock
syscall.forkExectakes across the fork itself, so holding itover the write means no child can exist while the descriptor does, and
a child forked earlier cannot hold one for a file that did not yet
exist. It is declared on Windows and darwin as well as Unix, which
matters because this helper skips at runtime rather than behind a build
tag.
Verification.
backend/download(12 goroutines × 200 realstubYtDlpcalls + exec)go test -race ./backend/download/, ×3make lint(three build configurations)make test(all three passes), ×2GOOS=windows go vet ./backend/download/GOOS=darwinfails for the documentedwails/v3/pkg/maccgo reason, unrelated)The probe is not committed: it is a measurement, and a stress test
that only sometimes fails is the class of thing this issue is about.
The before/after numbers above are what it produced.
Deliberately not done.
make ui-test,make e2e,make generate,make bindings— nofrontend, no
.sql, no.templ, no bound signature.CLAUDE.mduntouched: it documents no test helper at this level, andfour open PRs are each editing that file in a different place.
is now impossible is the mechanism. A recurrence would mean a
different cause and deserves its own issue.
No new issues filed —
stubYtDlpis the only test helper in the repothat writes an executable and then execs it (
cmd/indexbuild/deps_test.goexecs
go list), so there is no second site.Closes #146
CI is green. Run 18482:
checksuccess,e2esuccess (chromiumand WebKit both ran).
Not merging — leaving it for a human, with
Status/In Progressstill on#146.