feat(bench): add matrix runner, gamescope and per-session controls
- bench.py: gamescope-headless display (default, 1920x1080; ~2% cost vs desktop for isolation from the shared Wayland desktop), --shader, --dh, --disable-mods/--add-mods, --world-zip, crash-safe restore journal, spectator mode in the plan (elevated cameras fell to the sea floor and S4 measured underwater darkness) - matrix.toml/matrix.py: declarative, resumable B0-B14 runner keyed on mods hash (warns when B0 must be re-run) - build.py: bench-only resource-pack variants (C1-fa, C1-fa-ext, emf-lod*-fa, C1-bl); every bench variant pins resourcePacks so packs don't leak between sessions; shipped tier output unchanged - analyze.py: tagged labels use their own block's baseline
This commit is contained in:
1 parent
8f56e2f780
commit
a15c912ad0
10 files changed
+2196
-58
No files matched your search
@@ -21,6 +21,41 @@ python3 -m unittest discover -s tests # offline tests (from this d
|
||||
`session --dry-run` prints the datapack it would write and changes nothing. `bench.py meta`
|
||||
prints the environment snapshot; `bench.py disarm` removes a leftover plan pack.
|
||||
|
||||
## Per-session instance changes (undone after every session)
|
||||
|
||||
```sh
|
||||
python3 bench.py session --variant C1 --plan s1,s2,s4 --shader ComplementaryReimagined_r5.9.3.zip \
|
||||
[--shader-options profile.txt] --label B7
|
||||
python3 bench.py session --variant C1 --plan s1 --dh off
|
||||
python3 bench.py session --variant C1 --plan s1 --disable-mods 'badoptimizations*,particle_core*'
|
||||
python3 bench.py session --variant C1 --plan s2 --add-mods ~/trials/gnetum.jar
|
||||
python3 bench.py session --variant C1 --plan s4 --world-zip ~/onion-bench/worlds/bench-world-v1-s4-1.zip
|
||||
python3 bench.py restore # after a hard kill: replay .harness-pending.json
|
||||
```
|
||||
|
||||
- `--shader NAME|off` (default `off`): `config/iris.properties` `enableShaders` /
|
||||
`shaderPack`; NAME must exist in `shaderpacks/`. `--shader-options FILE` is copied to
|
||||
`shaderpacks/<NAME>.txt` (Iris' per-pack options file; journaled whenever a shader is set).
|
||||
- `--dh on|off` (default `on`): `[client.advanced.debugging] rendererMode` =
|
||||
`DEFAULT` / `DISABLED` in `config/DistantHorizons.toml`, set after the variant (DH's
|
||||
`quickEnableRendering` is GUI-only and maps to this key).
|
||||
- `--disable-mods GLOB,..`: case-insensitive globs over `mods/*.jar`, renamed to
|
||||
`.jar.disabled`; a glob that matches nothing is an error. `--add-mods PATH,..` copies jars
|
||||
in (name collision = error). `mods_hash` is taken after these changes.
|
||||
- `--world-zip PATH`: `saves/onionbench` -> `saves/onionbench.harness-bak`, zip extracted
|
||||
(world at the zip root or under one top directory, needs `level.dat`); sha256 recorded.
|
||||
- Every session first resets `options.txt` `resourcePacks` / `incompatibleResourcePacks`
|
||||
to `["vanilla","mod_resources"]` / `[]` (variants may override); not undone.
|
||||
- Crash safety: before the first change `session` writes `.minecraft/.harness-pending.json`
|
||||
(write-ahead: each entry precedes its change; file backups in `.minecraft/.harness-backup/`).
|
||||
The normal `finally` and `bench.py restore` run the same restore, newest entry first;
|
||||
failed entries stay in the journal. `session` and `boot` refuse to start while it exists.
|
||||
SIGTERM unwinds through the restore; SIGKILL leaves the journal for `bench.py restore`.
|
||||
- Rows record `shader`, `shader_options(_sha256)`, `dh`, `disabled_mods`, `added_mods`,
|
||||
`world_zip(_sha256)`, `resource_packs`, `restore_errors`; warn flags `gpu_dpm_auto` (RX
|
||||
7800 XT `power_dpm_force_performance_level` != `high`), `server_running`,
|
||||
`restore_failed`.
|
||||
|
||||
- **boot**: N launches; waits for `Game took N seconds` (or `joined the game` with
|
||||
`--join`); outcome `ok` / `crash` / `timeout`. Crash = new file in `crash-reports/` or
|
||||
`hs_err_pid*.log`, client java exited, or a log line with `Failed to create mod
|
||||
@@ -84,8 +119,9 @@ join seconds, block-atlas size, `level.dat` hash.
|
||||
only reaches Xwayland windows, `windowactivate --sync` hangs under kwin, focus-steal
|
||||
prevention can drop keystrokes, and typed chat lands in whatever has focus. Hence no
|
||||
input injection at all in v2.
|
||||
- Still manual: **S6** (JEI/GUI overlay needs a key press), **B7** shader profiles (set in
|
||||
Iris' GUI), `dh-off`-style checks needing F3 reads, `/sparkc` heap/GC reads (B1/B4).
|
||||
- Still manual: **S6** (JEI/GUI overlay needs a key press), producing B7 shader option
|
||||
profiles (export once from Iris' GUI, then pass `--shader-options`), checks needing F3
|
||||
reads, `/sparkc` heap/GC reads (B1/B4).
|
||||
- MangoHud's `log_duration=1800` in `~/.local/bin/mangohud-local` caps one CSV at 30 min;
|
||||
`session` warns when a plan is longer (B5's 45-min decay run needs a split or a larger
|
||||
value).
|
||||
@@ -94,9 +130,10 @@ join seconds, block-atlas size, `level.dat` hash.
|
||||
the start of each capture (identical across variants).
|
||||
- **FTB Backups 3** backs up the world right after join (its index
|
||||
`ftbbackups3/backups.json` doesn't parse, so it treats the world as never backed up)
|
||||
and every `backup_timer` = 120 min. For the bench instance only, set `auto: false` in
|
||||
`saves/onionbench/serverconfig/ftbbackups3-server.snbt` (copy of
|
||||
`config/ftbbackups3-server.snbt`; the world override wins and is never touched by pack
|
||||
updates). Not done by the harness. Never edit the repo's `config/` for this.
|
||||
- The dedicated server in tmux `onion-mc` keeps running during benches and was observed
|
||||
using ~78 % of a core while idle; it is recorded per row as a noise covariate.
|
||||
and every `backup_timer` = 120 min. **Done (bench instance only):** `auto: false` in
|
||||
the instance's `config/ftbbackups3-server.snbt`. Never edit the repo's `config/` for this.
|
||||
- Stop the dedicated server (tmux `onion-mc`) while benching (owner-approved); it used up
|
||||
to ~78 % of a core idle. Rows are flagged `server_running` if it is up.
|
||||
- amdgpu DPM is `auto` (no root); rows are flagged `gpu_dpm_auto`. For tighter noise:
|
||||
`echo high | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level`.
|
||||
- Scenes run in **spectator** mode (set by the plan) so elevated cameras don't fall.
|
||||
Reference in new issue
Block a user