chore(bench): add client bench harness and Phase 2 handoff
The client now boots cleanly on the bench machine (the CAC spawn-key
crash was fixed in 8e8483d). Add the harness used to drive bench scenes
and capture frametimes, the bench-world generation workflow, and a
self-contained handoff brief. All files live under analysis/ (packwizignored),
so nothing player-facing changes.
- harness/rcon.py: minimal RCON client for the temp worldgen server
- harness/run_scene.py: apply variant -> launch Prism under MangoHud ->
xdotool scene commands -> p50/p95/p99/1%-low from the CSV
- HANDOFF.md: state, gotchas, next steps
- build.py: disable DH autoUpdater for bench variants (suppress modal)
This commit is contained in:
1 parent
8e8483d072
commit
e5e369a693
7 files changed
+611
-1
No files matched your search
@@ -411,3 +411,40 @@ machine has.
|
||||
2. Build the bench world (§2), then run B0 → B14. Write the winners into `build.py`, ship,
|
||||
refresh, commit.
|
||||
3. Report shader-policy data (B7) and any trial-mod verdicts (B12b) to the owner.
|
||||
|
||||
## Session 2026-10-15 — Phase 2 begins: client boots, bench harness built
|
||||
|
||||
Machine: **Manjaro**, Java 21 (OpenJDK), Prism 11.0.3, RX 7800 XT / Ryzen 9 9900X / 91 GB.
|
||||
Unlike the previous machine this one runs the client. Full brief: `analysis/client-perf/HANDOFF.md`.
|
||||
|
||||
### Boot check (client) — essentially PASS
|
||||
Pack reaches the main menu cleanly (ModernFix "Game took 29.3 s"; atlas 4096x4096x2;
|
||||
Default Options seeds `options.txt` correctly). No missing deps, no mixin apply failures,
|
||||
no `ftbteams`, no fishermans-haven errors, no YACL "Failed to deserialize". 22 benign ERRORs.
|
||||
The one real blocker — **Critters & Companions spawn keys camelCase vs YACL's snake_case
|
||||
policy**, NPE in `CACSpawnConfig$SpawnEntry.isTag`, broken mod state — was fixed and pushed
|
||||
by the owner as **`8e8483d`** while this session ran. Alex's Mobs / Fowl Play spawn tuning
|
||||
verified applied in-game.
|
||||
|
||||
**Open:** intermittent **world-load** crash in WATUT
|
||||
(`ShaderInstanceBlur` → `FileNotFoundException: minecraft:shaders/core/particle.json`).
|
||||
First load worked, next crashed. Needs repro; owner says remove no mods.
|
||||
|
||||
### Built this session (all under `analysis/client-perf/`, packwizignored)
|
||||
- `harness/rcon.py`, `harness/run_scene.py` — apply variant → launch Prism into the bench
|
||||
world under user-local MangoHud → drive scenes via xdotool → slice p99/1%-low from CSV.
|
||||
Verified end-to-end once (C1/S1, but vsync-locked; `vblank_mode=0` added after).
|
||||
- User-local MangoHud 0.8.4 (`~/.local/mangohud`, wrapper `~/.local/bin/mangohud-local`).
|
||||
- Bench world `saves/onionbench` generated by temp server `~/onion-bench/server-bench`
|
||||
(S2 Create floor, S3 mob pen placed+verified). S4 airships not built.
|
||||
- `build.py` BENCH layer: `enableAutoUpdater=false` (suppress DH update modal). Tiers
|
||||
unchanged; **nothing shipped**.
|
||||
|
||||
### Not done
|
||||
The matrix blocks B0–B14. Fleet P0/P1/P2 are unreachable → any data here is a high-tier
|
||||
proxy. Only one non-decision-grade run exists. Next agent: kill the WATUT crash, confirm
|
||||
uncapped frametimes, then run B0→B14 per `BENCH-MATRIX.md`.
|
||||
|
||||
### Pushed
|
||||
`8e8483d` (owner, CAC keys). This session's analysis/harness commit follows; it touches no
|
||||
shipped file.
|
||||
@@ -0,0 +1,109 @@
|
||||
# Onion MC client-perf — handoff (2026-10-15)
|
||||
|
||||
Self-contained brief for the next agent. Read `.pi/journal.md` (last sections) and
|
||||
`BENCH-MATRIX.md` too, but this file is the live state on this machine.
|
||||
|
||||
## Goal
|
||||
|
||||
Finish the **Phase 2 client-performance loop**: measure the staged configs, decide by
|
||||
`BENCH-MATRIX.md`'s rules, write winners into `proposed-configs/build.py`
|
||||
(`DEFAULT` / `DECK` / `HIGH`), `build.py --ship`, `packwiz refresh`, commit (Conventional
|
||||
Commits) and push. Pushes go live to players on their next Prism launch.
|
||||
|
||||
## Repo / pack state
|
||||
|
||||
- Pack: **217 mods**, MC 1.21.1, NeoForge 21.1.251. Git `HEAD == origin/main == 8e8483d`.
|
||||
- `8e8483d` fixed the one real boot blocker: the Critters & Companions partial spawn config
|
||||
used camelCase keys (`biomeSpec`) but YACL serialises with
|
||||
`FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES`; every entry loaded `biomeSpec = null` and
|
||||
`CACSpawnConfig$SpawnEntry.isTag()` NPE'd in `FMLCommonSetupEvent`, leaving the client in a
|
||||
"broken mod state" (119 refused events). Fixed to snake_case; **already pushed**.
|
||||
- The Alex's Mobs (TOML) and Fowl Play (YACL camelCase) spawn files are correct and were
|
||||
verified applied in-game. Do not commit the mod/server **full re-saves** of
|
||||
`config/*` (the running server rewrites them): always `git checkout -- config/` before
|
||||
staging.
|
||||
- Nothing was shipped this session. Shipped `config/defaultoptions/**` is unchanged from
|
||||
`a01bc3b`. `build.py` was only patched in the **BENCH** layer (below); the tiers are
|
||||
untouched.
|
||||
|
||||
## Boot check (client, this machine) — essentially PASS
|
||||
|
||||
Reached main menu cleanly. `Sound engine started` 01:26:53; ModernFix "Game took 29.3 s to
|
||||
start"; blocks atlas `4096x4096x2`; Default Options correctly seeded `options.txt`
|
||||
(renderDistance 8, maxFps 120, vsync true). No missing deps, no mixin apply failures, no
|
||||
`ftbteams` refs, no fishermans-haven biome errors, no YACL "Failed to deserialize". 22 ERROR
|
||||
lines, all benign (pride access-transformer, Controlify @Shadow, FTB Library legacy
|
||||
textures, 4 GeckoLib molang parses, icys-better-horses pack path, Xaero version expiry,
|
||||
Veil failing to recompile 2 lodestone `nineslice` shaders).
|
||||
|
||||
**Open boot-adjacent issue (not yet fixed):** an intermittent crash on **world load**:
|
||||
```
|
||||
java.io.FileNotFoundException: minecraft:shaders/core/particle.json
|
||||
at com.corosus.watut.ShaderInstanceBlur.<init>(ShaderInstanceBlur.java:27)
|
||||
at GameRenderer.handler$jhl000$watut$onLoadShaders(...)
|
||||
```
|
||||
First world load succeeded and measured; the next crashed during "Rendering overlay". WATUT
|
||||
is a LEVERS §5 removal candidate but the owner says **remove no mods** — look for a WATUT
|
||||
config/patch or a reliable retry. Confirm whether real players hit it before shipping.
|
||||
|
||||
## Tooling installed on this machine (all user-local, no root)
|
||||
|
||||
- **Prism Launcher 11.0.3** (`/usr/bin/prismlauncher`). Data root `~/onion-bench/prism`,
|
||||
instance id **`OnionBench`**, game dir
|
||||
`~/onion-bench/prism/instances/OnionBench/.minecraft` (214 jars). A valid Microsoft
|
||||
account is in `~/onion-bench/prism/accounts.json` (device-code login, owner-approved).
|
||||
- **MangoHud 0.8.4** extracted to `~/.local/mangohud`; wrapper **`~/.local/bin/mangohud-local`**
|
||||
(exports `MANGOHUD_CONFIG`, `LD_PRELOAD` shim, `vblank_mode=0`). CSVs land in
|
||||
`~/onion-bench/mangologs/`. The instance has `WrapperCommand=mangohud-local`.
|
||||
- **Bench world**: `~/onion-bench/prism/instances/OnionBench/.minecraft/saves/onionbench`
|
||||
(seed `onionbench`, creative, cheats). S2 Create floor (2304 kinetic BEs) and S3 mob pen
|
||||
(30 villagers/15 cows/15 sheep/10 allays) were placed via RCON and verified. S4 airships
|
||||
NOT built. Datapack lives in the world's `datapacks/onion-bench`.
|
||||
- **Temp server** for worldgen: `~/onion-bench/server-bench` (symlinks repo `mods`/
|
||||
`libraries`, copies `config`), port 25566 / RCON 25576, currently **stopped**.
|
||||
- **Harness** (this is new, untracked, packwizignored): `analysis/client-perf/harness/`
|
||||
- `rcon.py` — minimal RCON client.
|
||||
- `run_scene.py` — applies a variant, launches Prism into the bench world under MangoHud,
|
||||
drives scenes with xdotool chat, slices the CSV into p50/p95/p99/1%-low. Verified working
|
||||
end-to-end once (C1/S1).
|
||||
|
||||
## Measurements so far — NOT decision-grade
|
||||
|
||||
One run: **C1 / S1** p50 16.72 ms, p99 17.24 ms, peak 17.60 ms, avg 59.8 FPS, 1% low 57.5,
|
||||
n=1889. It was **vsync-locked at ~60 FPS** (tight 16.7 ms floor). `vblank_mode=0` was added
|
||||
afterwards but the next launch crashed on world load, so uncapped data is not collected yet.
|
||||
Treat all of this as pipeline validation only.
|
||||
|
||||
## Next steps (in order)
|
||||
|
||||
1. Resolve the WATUT world-load crash; re-run one C1/S1 and confirm FPS > 60 (vblank off).
|
||||
2. Run the matrix in order, per `BENCH-MATRIX.md`: **B0** noise floor (C0/S1 ×3) → **B1**
|
||||
baseline (C0, S1–S6) → **B2** composite (C1, and C1-deck/C1-high) → **B3** DH radius
|
||||
(`dh-r256`, `dh-r192`, `C1`, `dh-r64`, `dh-off` on S1/S5) → **B4** heap/GC (use `jcmd`/
|
||||
`jstat`, or spark, at 5/6 G and J0–J3) → **B6** ModernFix → **B7** shaders → **B8**
|
||||
entities → **B9** Flywheel → **B10** SPR → **B11** Sodium/blend → **B12a/b** ablation and
|
||||
trials → **B13** spark profile → **B14** spawn counts.
|
||||
3. Write winners into `build.py`, `python3 build.py --ship`, `~/go/bin/packwiz refresh`,
|
||||
commit. Ship `config/modernfix-mixins.properties` only if B6 passes (see SHIP comment).
|
||||
4. Update `PLAYER-TUNING.md` (tiers, heap/GC, shader profiles), `LEVERS.md`,
|
||||
`BENCH-MATRIX.md`, and append a journal section.
|
||||
|
||||
## Fleet note
|
||||
|
||||
P0 (Steam Deck), P1 (7700K/1060) and P2 (6700K/1080 Ti) are **not reachable** from this box
|
||||
(no routes / no SSH targets). Everything measured here is a **high-tier proxy** on a Ryzen 9
|
||||
9900X + RX 7800 XT, 91 GB RAM. Label results as proxies; absolute FPS floors do not map to
|
||||
tier decisions.
|
||||
|
||||
## Gotchas learned this session
|
||||
|
||||
- Prism 11 `--offline` fails a fresh instance with "Required libraries are missing"
|
||||
(`EnsureOfflineLibraries`). Launch **online** once to download libs/assets; after that
|
||||
offline works.
|
||||
- `xdotool windowactivate --sync` **hangs** under kwin; use `windowactivate` + `windowfocus`.
|
||||
- MangoHud needs `MANGOHUD_CONFIG` set explicitly and `LD_PRELOAD` **exported**.
|
||||
- The owner's dedicated server runs from the repo root (`java @user_jvm_args.txt
|
||||
@libraries/.../unix_args.txt`, port 25565). Never kill it; never touch `world/`. The temp
|
||||
bench server used separate ports/paths and is stopped.
|
||||
- `analysis/` is packwizignored (safe to commit); `config/` is server-rewritten (restore
|
||||
before staging).
|
||||
@@ -0,0 +1,76 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Minimal Minecraft RCON client (stdlib only).
|
||||
|
||||
Usage: python3 rcon.py <host> <port> <password> <command> [command ...]
|
||||
Exit non-zero if the server reports a failure.
|
||||
"""
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
|
||||
|
||||
class Rcon:
|
||||
def __init__(self, host: str, port: int, password: str, timeout: float = 10.0):
|
||||
self.host, self.port, self.password = host, port, password
|
||||
self.timeout = timeout
|
||||
self._id = 0
|
||||
|
||||
def _send(self, sock: socket.socket, ptype: int, body: str) -> int:
|
||||
self._id += 1
|
||||
rid = self._id
|
||||
payload = struct.pack("<ii", rid, ptype) + body.encode("utf-8") + b"\x00\x00"
|
||||
sock.sendall(struct.pack("<i", len(payload)) + payload)
|
||||
return rid
|
||||
|
||||
def _recv(self, sock: socket.socket):
|
||||
raw = b""
|
||||
while len(raw) < 4:
|
||||
chunk = sock.recv(4 - len(raw))
|
||||
if not chunk:
|
||||
raise ConnectionError("RCON connection closed")
|
||||
raw += chunk
|
||||
(length,) = struct.unpack("<i", raw)
|
||||
body = b""
|
||||
while len(body) < length:
|
||||
chunk = sock.recv(length - len(body))
|
||||
if not chunk:
|
||||
raise ConnectionError("RCON connection closed")
|
||||
body += chunk
|
||||
rid, ptype = struct.unpack("<ii", body[:8])
|
||||
return rid, ptype, body[8:-2].decode("utf-8", "replace")
|
||||
|
||||
def command(self, cmd: str) -> str:
|
||||
with socket.create_connection((self.host, self.port), timeout=self.timeout) as sock:
|
||||
sock.settimeout(self.timeout)
|
||||
rid = self._send(sock, 3, self.password)
|
||||
rid2, _, _ = self._recv(sock)
|
||||
if rid2 == -1:
|
||||
raise PermissionError("RCON auth failed")
|
||||
self._send(sock, 2, cmd)
|
||||
_, _, text = self._recv(sock)
|
||||
return text
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 5:
|
||||
print(__doc__)
|
||||
return 2
|
||||
host, port, password = argv[1], int(argv[2]), argv[3]
|
||||
rc = 0
|
||||
for cmd in argv[4:]:
|
||||
try:
|
||||
out = Rcon(host, port, password).command(cmd)
|
||||
except Exception as exc: # noqa: BLE001 - CLI surface
|
||||
print(f"! {cmd}: {exc}", file=sys.stderr)
|
||||
rc = 1
|
||||
continue
|
||||
if out.strip():
|
||||
print(out.strip())
|
||||
lowered = out.lower()
|
||||
if "unknown" in lowered or "failed" in lowered or "error" in lowered:
|
||||
rc = 1
|
||||
return rc
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
@@ -0,0 +1,267 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Client-perf bench harness: drive one variant/scene and extract frametime metrics.
|
||||
|
||||
Applies a generated variant, launches the Prism instance under the user-local
|
||||
MangoHud wrapper straight into the bench world, runs the scene via xdotool chat
|
||||
commands, records wall-clock markers, then slices the MangoHud CSV.
|
||||
|
||||
Usage:
|
||||
python3 run_scene.py --variant C1 --scene s1 [--warmup 15] [--capture 30]
|
||||
[--runs 1] [--record out.csv] [--pre storm_on]
|
||||
|
||||
Scenes: s1, s2, s3, s4, s5 (tour = 120s capture, no warmup), s2s (= s2 + storm).
|
||||
Only stdlib. Must run on the desktop session that owns DISPLAY :1.
|
||||
"""
|
||||
import argparse
|
||||
import bisect
|
||||
import csv as csvmod
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
PROPOSED = HERE.parent / "proposed-configs"
|
||||
ROOT = Path("/home/logan/onion-bench/prism")
|
||||
INSTANCE = "OnionBench"
|
||||
GAME = ROOT / "instances" / INSTANCE / ".minecraft"
|
||||
LOGS = GAME / "logs" / "latest.log"
|
||||
MANGOLOGS = Path("/home/logan/onion-bench/mangologs")
|
||||
WORLD = "onionbench"
|
||||
ENV = {
|
||||
**os.environ,
|
||||
"DISPLAY": ":1",
|
||||
"XAUTHORITY": "/run/user/1000/xauth_HvvPAq",
|
||||
"XDG_RUNTIME_DIR": "/run/user/1000",
|
||||
"PATH": "/home/logan/.local/bin:" + os.environ.get("PATH", ""),
|
||||
}
|
||||
|
||||
SCENE_FN = {
|
||||
"s1": "bench:s1",
|
||||
"s2": "bench:s2",
|
||||
"s3": "bench:s3",
|
||||
"s4": "bench:s4",
|
||||
"s5": "bench:tour_start",
|
||||
"s2s": "bench:s2",
|
||||
}
|
||||
|
||||
|
||||
def log(msg):
|
||||
print(f"[run_scene] {msg}", flush=True)
|
||||
|
||||
|
||||
def xdo(*args):
|
||||
subprocess.run(["xdotool", *args], env=ENV, check=False,
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
|
||||
|
||||
def mc_window():
|
||||
out = subprocess.run(["xdotool", "search", "--name", "Minecraft"],
|
||||
env=ENV, capture_output=True, text=True).stdout.split()
|
||||
return out[-1] if out else None
|
||||
|
||||
|
||||
def client_java_pids():
|
||||
out = subprocess.run(["ps", "-C", "java", "-o", "pid=,args="],
|
||||
capture_output=True, text=True).stdout
|
||||
pids = []
|
||||
for line in out.splitlines():
|
||||
if f"instances/{INSTANCE}/.minecraft" in line or f"instances/{INSTANCE}/natives" in line:
|
||||
pids.append(int(line.split()[0]))
|
||||
return pids
|
||||
|
||||
|
||||
def apply_variant(vid):
|
||||
subprocess.run([sys.executable, str(PROPOSED / "apply_variant.py"), vid, str(GAME)],
|
||||
check=True, stdout=subprocess.DEVNULL)
|
||||
|
||||
|
||||
def launch_client():
|
||||
if LOGS.exists():
|
||||
LOGS.unlink()
|
||||
p = subprocess.Popen(
|
||||
["setsid", "prismlauncher", "-d", str(ROOT), "--launch", INSTANCE, "--world", WORLD],
|
||||
env=ENV, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
)
|
||||
return p
|
||||
|
||||
|
||||
def wait_for_world(timeout=240):
|
||||
t0 = time.time()
|
||||
while time.time() - t0 < timeout:
|
||||
if LOGS.exists():
|
||||
txt = LOGS.read_text(errors="replace")
|
||||
if "Sound engine started" in txt:
|
||||
return True
|
||||
time.sleep(2)
|
||||
return False
|
||||
|
||||
|
||||
def chat(command, settle=0.6):
|
||||
wid = mc_window()
|
||||
if wid:
|
||||
xdo("windowactivate", wid)
|
||||
xdo("windowfocus", wid)
|
||||
xdo("key", "--clearmodifiers", "t")
|
||||
time.sleep(0.4)
|
||||
xdo("type", "--clearmodifiers", "--delay", "12", command)
|
||||
time.sleep(0.2)
|
||||
xdo("key", "--clearmodifiers", "Return")
|
||||
time.sleep(settle)
|
||||
|
||||
|
||||
def newest_csv():
|
||||
files = [p for p in MANGOLOGS.glob("*.csv") if not p.name.endswith("_summary.csv")]
|
||||
return max(files, key=lambda p: p.stat().st_mtime) if files else None
|
||||
|
||||
|
||||
def parse_csv(path):
|
||||
"""Return (columns, rows) where rows are lists of floats; line 3 is the header."""
|
||||
lines = path.read_text(errors="replace").splitlines()
|
||||
hdr_i = None
|
||||
for i, line in enumerate(lines[:10]):
|
||||
if "frametime" in line and "elapsed" in line:
|
||||
hdr_i = i
|
||||
break
|
||||
if hdr_i is None:
|
||||
raise RuntimeError(f"no frametime header in {path}")
|
||||
cols = lines[hdr_i].split(",")
|
||||
e_i = cols.index("elapsed")
|
||||
ft_i = cols.index("frametime")
|
||||
fps_i = cols.index("fps")
|
||||
rows = []
|
||||
for line in lines[hdr_i + 1:]:
|
||||
parts = line.split(",")
|
||||
if len(parts) != len(cols):
|
||||
continue
|
||||
try:
|
||||
rows.append((float(parts[e_i]), float(parts[ft_i]), float(parts[fps_i]), parts))
|
||||
except ValueError:
|
||||
continue
|
||||
return cols, rows
|
||||
|
||||
|
||||
def pct(values, p):
|
||||
if not values:
|
||||
return float("nan")
|
||||
s = sorted(values)
|
||||
k = (len(s) - 1) * p
|
||||
lo, hi = int(k), min(int(k) + 1, len(s) - 1)
|
||||
return s[lo] + (s[hi] - s[lo]) * (k - lo)
|
||||
|
||||
|
||||
def metrics_between(rows, t_start, t_end, run_start):
|
||||
"""rows: list of (elapsed_ns, frametime_ms, fps, raw). Wall = run_start + elapsed/1e9."""
|
||||
ft, fps = [], []
|
||||
for e, frametime, f, raw in rows:
|
||||
wall = run_start + e / 1e9
|
||||
if t_start <= wall <= t_end:
|
||||
ft.append(frametime)
|
||||
fps.append(f)
|
||||
if not ft:
|
||||
return None
|
||||
fps_sorted = sorted(fps)
|
||||
n1 = max(1, int(len(fps_sorted) * 0.01))
|
||||
one_low = sum(fps_sorted[:n1]) / n1
|
||||
return {
|
||||
"n": len(ft),
|
||||
"p50_ms": round(pct(ft, 0.50), 3),
|
||||
"p95_ms": round(pct(ft, 0.95), 3),
|
||||
"p99_ms": round(pct(ft, 0.99), 3),
|
||||
"peak_ms": round(max(ft), 3),
|
||||
"avg_fps": round(sum(fps) / len(fps), 2),
|
||||
"one_low_fps": round(one_low, 2),
|
||||
}
|
||||
|
||||
|
||||
def kill_client(prism):
|
||||
try:
|
||||
os.killpg(os.getpgid(prism.pid), signal.SIGTERM)
|
||||
except Exception:
|
||||
pass
|
||||
for pid in client_java_pids():
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
time.sleep(5)
|
||||
for pid in client_java_pids():
|
||||
try:
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
|
||||
|
||||
def run_once(variant, scene, warmup, capture, pre):
|
||||
apply_variant(variant)
|
||||
p = launch_client()
|
||||
run_start = time.time()
|
||||
try:
|
||||
if not wait_for_world():
|
||||
log("ERROR: world never loaded")
|
||||
return None
|
||||
time.sleep(8) # let the world settle / place player
|
||||
fn = SCENE_FN[scene]
|
||||
if scene != "s5":
|
||||
chat(f"/function {fn}", settle=1.0)
|
||||
if pre:
|
||||
chat(f"/function bench:{pre}", settle=1.0)
|
||||
if scene == "s5":
|
||||
capture = 120
|
||||
warmup = 0
|
||||
chat("/function bench:tour_start", settle=0.5)
|
||||
time.sleep(warmup)
|
||||
t_start = time.time()
|
||||
chat(f"/say BENCH_CAPTURE_START {variant} {scene}")
|
||||
time.sleep(capture)
|
||||
t_end = time.time()
|
||||
chat("/say BENCH_CAPTURE_END")
|
||||
chat("/save-all", settle=2.0)
|
||||
finally:
|
||||
csv_path = newest_csv()
|
||||
kill_client(p)
|
||||
cols, rows = parse_csv(csv_path)
|
||||
m = metrics_between(rows, t_start, t_end, run_start)
|
||||
if m:
|
||||
m.update({"variant": variant, "scene": scene, "csv": csv_path.name,
|
||||
"run_start": round(run_start, 3)})
|
||||
return m
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--variant", required=True)
|
||||
ap.add_argument("--scene", required=True, choices=sorted(SCENE_FN))
|
||||
ap.add_argument("--warmup", type=int, default=15)
|
||||
ap.add_argument("--capture", type=int, default=30)
|
||||
ap.add_argument("--runs", type=int, default=1)
|
||||
ap.add_argument("--pre", default=None)
|
||||
ap.add_argument("--record", default=None)
|
||||
a = ap.parse_args()
|
||||
results = []
|
||||
for i in range(a.runs):
|
||||
log(f"run {i + 1}/{a.runs}: {a.variant} {a.scene}")
|
||||
m = run_once(a.variant, a.scene, a.warmup, a.capture, a.pre)
|
||||
if m:
|
||||
results.append(m)
|
||||
print("RESULT " + str(m), flush=True)
|
||||
time.sleep(3)
|
||||
if a.record and results:
|
||||
fields = ["variant", "scene", "n", "p50_ms", "p95_ms", "p99_ms", "peak_ms",
|
||||
"avg_fps", "one_low_fps", "csv", "run_start"]
|
||||
exists = Path(a.record).exists()
|
||||
with open(a.record, "a", newline="") as fh:
|
||||
w = csvmod.DictWriter(fh, fieldnames=fields)
|
||||
if not exists:
|
||||
w.writeheader()
|
||||
for m in results:
|
||||
w.writerow({k: m.get(k, "") for k in fields})
|
||||
return 0 if results else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,57 @@
|
||||
# Phase 0 distill — capture protocol, sheet columns, per-machine prep, LEVERS §7 corrections
|
||||
Sources: `phase0/P0-steamdeck.md`, `phase0/P1-windows-7700k-1060.md`, `phase0/P2-linux-6700k-1080ti.md`, `LEVERS.md` §7.
|
||||
|
||||
## (a) Benchmark capture procedure (shared protocol, all machines)
|
||||
- [ ] **Uncapped, vsync OFF during measurement.** Apply the FPS cap only *after* tuning; with a cap/vsync every config "hits" the target.
|
||||
- [ ] Cap/vsync settings: in-game **Max framerate: Unlimited**, **VSync: Off**; MangoHud `fps_limit=0`; SteamOS **Framerate Limit → Off**; P1: NVIDIA panel Vertical sync → Off too (**two vsync switches**, check both each session); P2: compositor vsync disabled.
|
||||
- [ ] **Metric: p99 frametime + 1% low FPS, not average FPS** (sheet also logs p50/p95, avg FPS, peak frametime).
|
||||
- [ ] **Discard 10–15 s of warmup per scene** (JIT, chunk load, initial GC).
|
||||
- [ ] **Settle after any render-distance or DH change** — both trigger rebuilds; otherwise you measure the rebuild, not steady state.
|
||||
- [ ] **3 runs per configuration; report median + spread.** A single run is noise. **One variable per run.**
|
||||
- [ ] Capture duration: MangoHud `log_duration=30` (seconds; match scene length); PresentMon Option B `--timed 30`. P0 extra: ~5 min thermal soak before first measured run. P0 runtime log toggle: `toggle_logging` (default `Shift_L+F2`).
|
||||
- [ ] Invalid-run guardrails: FPS limiter active; governor moved (P0/P2); RAM headroom shifted materially (P0/P2); power plan changed or GPU throttling (P1) — `nvidia-smi -q -d PERFORMANCE` must show no `SW Power Cap` / `HW Slowdown`.
|
||||
- [ ] GPU background logging (P1+P2, identical command): `nvidia-smi --query-gpu=timestamp,memory.used,clocks.sm,clocks.mem,power.draw,temperature.gpu,utilization.gpu --format=csv -l 1` → `gpu-<timestamp>.csv` (`memory.used` = VRAM series).
|
||||
- [ ] Fleet-wide JVM: Java 21, **same JVM build on all three** (Temurin 21); identical G1 args except `-Xmx` (P0 `-Xmx5G` start/6 G max, P1 `-Xmx8G`, P2 `-Xmx6G`); **omit `-XX:+AlwaysPreTouch` on P0 and P2**.
|
||||
|
||||
## (b) Recording-sheet columns (one row per run)
|
||||
- **P0**: `Date / session`, `Machine`, `Config ID`, `Scene`, `Run #`, `CSV file`, `p50 / p95 / p99 frametime (ms)`, `1% low FPS`, `Avg FPS`, `Peak measured frametime (ms)`, `Notes`.
|
||||
- **P1**: same plus `CSV file (raw + summary)`, `VRAM peak (MB)` (headline metric), `Throttle flag` (none / SW Power Cap / HW Slowdown).
|
||||
- **P2**: same plus `VRAM peak (MB)`, `Throttle flag`, `MemAvailable before / after`, `Swap delta` (must be 0).
|
||||
- Example values: Machine `P0-SteamDeck-LCD` / `P1-7700K-1060` / `P2-6700K-1080Ti`; Config ID `baseline / dh-off / emf-off / …`; Scene `A-base / B-terrain / C-gui`; Run # `1 / 2 / 3`; CSV e.g. `Minecraft_2026-09-28_14-22-01.csv`.
|
||||
|
||||
## (c) Per-machine prep specifics
|
||||
### P0 — Steam Deck (GPU/shared-RAM bound; LCD 60 Hz can only validate the 30/60 ladder rows)
|
||||
- [ ] Charger plugged in; **Desktop Mode** (Gaming Mode routes through MangoApp, which cannot log); instance on **internal storage**; record panel refresh rate.
|
||||
- [ ] SteamOS overlay: Quick Access `...` → Performance → **Overlay Level 0**, **Framerate Limit Off**, **TDP Off or pinned (e.g. 15 W)** — recorded, constant; check no per-game profile re-enables them; disable Decky MangoApp-routed plugins.
|
||||
- [ ] Governor: leave `schedutil` as set but record it (`/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor`); pinning `performance` risks thermal throttling.
|
||||
- [ ] MangoHud install (re-check every session — SteamOS updates remove pacman packages): `sudo steamos-readonly disable && sudo pacman -S --needed mangohud && sudo steamos-readonly enable`; find real paths with `pacman -Ql mangohud | grep -E 'mangohud$|libMangoHud'`. Alternative: MangoHudPy (verify it can write CSV first).
|
||||
- [ ] Config from packaged example → `~/.config/MangoHud/MangoHud.conf`: `output_folder=/home/deck/mcbench/logs`, `autostart_log=1`, `log_duration=30`, `fps_limit=0`, `frametime=1`, `frame_timing=1`; percentile key/syntax: **not stated** — confirm in packaged example.
|
||||
- [ ] Prism wrapper command (Custom Commands): `mangohud`; fallback `mangohud --dlsym`.
|
||||
|
||||
### P1 — Windows 7700K / GTX 1060 6 GB (VRAM bound; only Windows box)
|
||||
- [ ] `powercfg /setactive SCHEME_MIN` (High performance); Game Bar off; Game DVR background recording off; Defender real-time exclusion for instance folder (record it); close browsers/Discord/OneDrive.
|
||||
- [ ] NVIDIA per-program settings for **`javaw.exe`** (verify process name in Task Manager): Power management → **Prefer maximum performance**; Vertical sync → **Off**; Low Latency Mode, Threaded Optimization, Texture filtering → fixed + recorded; HAGS recorded + constant; disable *Fullscreen optimizations* for `javaw.exe`; same fullscreen display mode every session.
|
||||
- [ ] `nvidia-smi` (PowerShell, same command as P2) + `-q -d PERFORMANCE` throttle check; VRAM logging = headline metric (peak VRAM on 6 GB).
|
||||
- [ ] Tooling: **PresentMon** (no MangoHud on Windows), run **as Administrator**. Option A: Frame Time Analysis `Run.bat` → `presentmon.exe --multi_csv --no_console_stats --hotkey "F10"` (F10 starts/stops; new CSV per capture). Option B: `presentmon.exe --process_name javaw.exe --output_file ~\mcbench\logs\pm.csv --timed 30`. Emits raw CSV + stats summary CSV (avg/min/max/percentile FPS). Analysis: Frame Time Analysis (browser, reads PresentMon/CapFrameX/OCAT/MangoHud/FrameView, merges CSVs).
|
||||
- [ ] Cross-OS caveat: ETW vs MangoHud don't measure identically — absolute numbers not cross-comparable; compare orderings per OS.
|
||||
|
||||
### P2 — Linux 6700K / GTX 1080 Ti (CPU main-thread + RAM/GC bound; ~8 GB taken by Docker)
|
||||
- [ ] **Do not alter Docker or any running process** — treat load as constant; verify instead.
|
||||
- [ ] Proprietary NVIDIA driver confirmed (`nvidia-smi`; `glxinfo -B | grep -E 'OpenGL renderer|OpenGL version'`); compositor vsync off; record resolution/refresh.
|
||||
- [ ] `nvidia-settings -a '[gpu:0]/GPUPowerMizerMode=1'`; throttle check via `nvidia-smi -q -d PERFORMANCE`; start the shared `nvidia-smi --query-gpu=…` logging.
|
||||
- [ ] **Noise floor FIRST**: same config, same scene, 3 runs, no changes — p50/p95/p99 spread = detection threshold; any delta below it is not evidence.
|
||||
- [ ] RAM baseline: record `MemAvailable` (`free -h`, `/proc/meminfo`) at start and before/after each block; drift = runs not comparable; `grep -E 'pswpin|pswpout' /proc/vmstat` must not move (swap during a run invalidates p99 outright).
|
||||
- [ ] Governor: `sudo cpupower frequency-set -g performance` (record; revert afterwards). Bench at stock clocks.
|
||||
- [ ] MangoHud: config keys same as P0 (`output_folder=/home/<user>/mcbench/logs`, `autostart_log=1`, `log_duration=30`, `fps_limit=0`, `frametime=1`, `frame_timing=1`); percentile key: **not stated**. Prism wrapper `mangohud`, fallback `mangohud --dlsym`.
|
||||
- [ ] Heap rule: `-Xmx` = min(6 GB, `MemAvailable` − 2 GB); rig **over-weights RAM/GC ladder steps** (Docker) — record as rig property, don't attribute to a mod.
|
||||
|
||||
## (d) LEVERS §7 corrections (every bullet, corrected fact)
|
||||
- **DH default threads is 4 on the fleet, not 2** — default is `ceil(nproc × 0.5)` (`ThreadPresetConfigEventHandler` @ 3.3.2); the observed `numberOfThreads = 2` came from the owner's 4-thread machine (nproc=4). All other observed DH values match source defaults. *(Phase 1b)*
|
||||
- **Puzzles Lib 21.1.64 has no perf change on NeoForge** (lever 14 is a bugfix only) — r9's lead and this file's earlier "the fix is in 21.1.64 for 1.21.1" were wrong for our loader. *(Phase 1b)*
|
||||
- **r6's Shenandoah flags crash JDK 21**: `-XX:ShenandoahGCMode=iu` needs `-XX:+UnlockExperimentalVMOptions` (tested OpenJDK 21.0.12); use plain `-XX:+UseShenandoahGC`. *(Phase 1b)*
|
||||
- **SSRD's distance is a COMMON config**, not a Sodium option: `config/ssrd-common.toml`, `physicsRenderDistance`, default 128, slider floor 16; the server reads it too. *(Phase 1b)*
|
||||
- **r4 "remove one CTM engine" is wrong** — Chipped requires Athena and Rechiseled requires Fusion; both are hard dependencies (§5).
|
||||
- **r9 "Puzzles Lib ~8 % render-thread overhead"** was measured on MC 26.2/Fabric; the fix is in 21.1.64 for 1.21.1; gain size unknown. *(superseded by the Phase 1b Puzzles bullet above — both appear in §7 as written)*
|
||||
- **r3/r8's EMF #443 magnitude** is from the closed Oculus/Forge era; treat the 4× figure as directional only.
|
||||
- **"Sable overrides vanilla core shaders"**: real but small — just `worldborder.*` from the nested Veil; the substantive part is Sable's runtime patching of Flywheel's internal shaders.
|
||||
- **r1's Flywheel-under-shaders claim**: confirmed in `Backends.java` on the 1.21.1 branch (with any shaderpack active, Flywheel marks both GPU backends unsupported → Create renders kinetic BEs on CPU + shadow pass).
|
||||
@@ -0,0 +1,60 @@
|
||||
# Triage: /home/logan/onion-mc/logs/latest.log (dedicated server)
|
||||
|
||||
7165 lines, 986,551 bytes. **Covers mid-session only**: starts `14Oct2026 00:00:29` with
|
||||
RCON keepalive lines (server was already running); no mod-loading/startup phase is in
|
||||
this file. Ends ~19:50. NOTE: log timestamps say 2026 — quoted as-is.
|
||||
|
||||
## 1) Missing dependency / missing mod / "Mod file ... requires" errors
|
||||
**no hits.** Searched (case-insensitive): `missing (mod|dependency)`, `Mod file .* requires`,
|
||||
`mandatory`, `dependency`, `requires` — 0 matches in the whole file.
|
||||
|
||||
## 2) Mixin apply failures / fatal crashes
|
||||
**no hits.** 0 lines containing `mixin` at all; 0 `/FATAL]` lines; no `Critical injection
|
||||
failure`, `crash report`, `Exception`, `Segfault`, or `Failed to apply`. Server ran
|
||||
continuously (periodic Sable sub-level saves every ~5 min) with no crash.
|
||||
|
||||
## 3) "fishermans" / biome-modifier parse errors or warnings
|
||||
**no hits.** 0 matches for `fisher` (any case) and 0 for `biome` — so no biome-modifier
|
||||
parse errors/warnings either.
|
||||
|
||||
## 4) ftbteams / dev/ftb/mods/ftbteams
|
||||
**no hits** for `ftbteams` (any case). `ftb` case-insensitively matches 15 lines, **all**
|
||||
from `dev.ftb.mods.ftbbackups.Backups` (routine FTB Backups runs at 12:28, 14:28, 16:28),
|
||||
e.g. line 4465:
|
||||
```
|
||||
4465:[14Oct2026 12:28:03.472] [Server thread/INFO] [dev.ftb.mods.ftbbackups.Backups/]: backup starting: Server
|
||||
```
|
||||
No errors or references to ftbteams jar paths.
|
||||
|
||||
## 5) YACL "Failed to deserialize" (fowlplay / crittersandcompanions)
|
||||
**no hits.** 0 lines contain `yacl`, 0 contain `Failed to deserialize`, 0 mention
|
||||
`fowlplay` or `crittersandcompanions`.
|
||||
|
||||
## 6) Startup-complete marker / "Done (N.NNNs)!"
|
||||
**Absent from this file.** No `Done (`, `Starting minecraft server`, `Preparing level`,
|
||||
or startup banner — the file begins mid-run (`[14Oct2026 00:00:29.211] ... Thread RCON
|
||||
Client /127.0.0.1 started`, line 1). Startup output is in an earlier rotated log
|
||||
(e.g. `logs/2026-10-1*.log.gz`). The server was up throughout; line 68 shows a LAN
|
||||
connection attempt refused by whitelist:
|
||||
```
|
||||
68:[14Oct2026 00:12:07.186] [Server thread/INFO] [net.minecraft.server.network.ServerLoginPacketListenerImpl/]: Disconnecting FishNet (/127.0.0.1:51840): You are not white-listed on this server!
|
||||
```
|
||||
|
||||
## 7) Error/fatal counts + sample
|
||||
- Literal `[ERROR]`: **0** — this log's format is `[Server thread/ERROR]`; counting `/ERROR]` gives **8 lines**.
|
||||
- `[FATAL]` / `/FATAL]`: **0**. `[WARN]`: **0**.
|
||||
- All 8 ERROR lines collapse to 4 distinct messages (each twice), all benign
|
||||
`ServerRecipeBook` noise at 12:28:02 (right before the 12:28:03 backup), lines 4450–4457:
|
||||
```
|
||||
4450:[14Oct2026 12:28:02.967] [Server thread/ERROR] [net.minecraft.stats.ServerRecipeBook/]: Tried to load unrecognized recipe: etched:radio removed now.
|
||||
4451:[14Oct2026 12:28:02.968] [Server thread/ERROR] [net.minecraft.stats.ServerRecipeBook/]: Tried to load unrecognized recipe: etched:music_label removed now.
|
||||
4452:[14Oct2026 12:28:02.968] [Server thread/ERROR] [net.minecraft.stats.ServerRecipeBook/]: Tried to load unrecognized recipe: etched:boombox removed now.
|
||||
4453:[14Oct2026 12:28:02.968] [Server thread/ERROR] [net.minecraft.stats.ServerRecipeBook/]: Tried to load unrecognized recipe: etched:blank_music_disc removed now.
|
||||
```
|
||||
|
||||
## Bottom line
|
||||
No evidence here of missing-mods, mixin failures, fishermans/biome-modifier parse issues,
|
||||
ftbteams references, or YACL deserialize problems — and no crash. If client-side symptom
|
||||
reports stem from startup-time mod loading, the signal will be in the **rotated startup
|
||||
log** or the **client** log, not this mid-session server `latest.log`. No files were
|
||||
modified; no processes touched.
|
||||
@@ -182,7 +182,11 @@ HIGH = { # strong GPU, but still a 4C/8T CPU (true of every fleet machine)
|
||||
BENCH = {
|
||||
"options.txt": {"maxFps": 260, "enableVsync": "false", "renderDistance": 8,
|
||||
"simulationDistance": 8},
|
||||
"config/DistantHorizons.toml": {(DH_W, "generatorPlan"): "DISABLED"},
|
||||
"config/DistantHorizons.toml": {
|
||||
(DH_W, "generatorPlan"): "DISABLED",
|
||||
# Harness: suppress DH's "new update available" modal on every launch.
|
||||
("client.advanced.autoUpdater", "enableAutoUpdater"): False,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user