Commit Graph
1 Commits
Author SHA1 Message Date
yonluandClaude Opus 5 590a0d86dd perf(library): size the scan to the drive, and prefetch what it reads
CI / check (push) Skipped
CI / e2e (push) Skipped
CI / check (pull_request) Successful in 2m47s
CI / e2e (pull_request) Successful in 5m59s
Every parser in `backend/metadata` is header-only -- a few hundred
bytes and return -- so on a spinning disk a scan is not waiting on CPU
or on bytes, it is waiting on the head to arrive. Two things follow,
and the drive says which.

**How many reads should be in flight.** This was a flat 2 for anything
rotational, which is a pre-NCQ assumption: a modern SATA disk reports a
queue depth of 32 and reorders outstanding reads into the order its
head passes over them, and was being handed a quarter of what it can
use. It gets 4 now. A drive that reports 1 -- a USB bridge, a pre-2004
disk -- services one command at a time in the order given, where every
extra worker is one more seek competing for one head and the scan gets
*slower* the harder it is pushed; that keeps 2.

**And that the next seek should already be queued.** A prefetch stage
between the walk and the workers issues `POSIX_FADV_WILLNEED` over the
first 512 KB of each file -- enough for an ID3v2 tag carrying cover
art, or FLAC's STREAMINFO and PICTURE blocks. The buffered channel *is*
the lookahead: the goroutine runs 16 files ahead of the workers,
hinting as it goes, so the read a worker needs has been in flight for
sixteen files' worth of parsing by the time it asks. Rotational only;
an SSD gets the channel back unwrapped and pays nothing, since it has
no seek to hide and already has one worker per core.

`workersForProfile` is the policy on its own so it can be tested
against drives this machine does not have, and the scan logs the
device, its rotational flag and its queue depth, so the decision is
inspectable rather than inferred.

Also: `ScanConcurrency` has been a validated three-value config field
with exactly one caller, passing the constant `auto` -- so choosing
`ssd` or `hdd` by hand did nothing at all. It reads the config now.
The two modes overrule detection about the *disk* and not about its
queue, since a user who picks `hdd` on a queueing drive still wants
that drive's queue used.

What is not here is inode-ordered dispatch. It needs the streaming walk
restructured to buffer per directory, and with queueing the drive is
already reordering what the hints put in front of it; that wants a
measurement on real hardware before the complexity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeQt5hgXg5YGoNZQ9ozG7L
2026-08-17 22:12:15 -04:00