12 KiB
Bench results — proxy machine, run 1 (B0–B12a)
Proxy data only. One machine (Ryzen 9 9900X / RX 7800 XT, Manjaro), not the fleet (P0 Deck,
P1 7700K+1060, P2 6700K+1080 Ti). Run 1: 2026-10-16 22:25 → ~2026-10-17 05:00 EDT, 56 sessions
(57 launches incl. one auto-retry), 306/309 capture windows valid. Source:
~/onion-bench/results/session.jsonl (ts >= 2026-10-16T22:24), verdicts from
harness/analyze.py --baseline C1 --noise B0 --since 2026-10-16T22:24.
Run 2 (B4h, B7 shader rows, B12b) is in progress; B13/B14 are manual and not done.
Method
- Harness:
harness/bench.py+matrix.py. Every session is one cold launch at 1920×1080 in headless gamescope, world restored frombench-world-v1*.zip(DH LODs pre-generated,generatorPlan=DISABLED), scenes driven by an in-world datapack, spectator camera, FTB Backups off, dedicated server stopped. - Captures: per scene 15 s warm-up + 30 s capture (S5 tour 120 s, no warm-up) × 3 reps, 45 s settle after join. Frametimes from MangoHud; metrics = p50/p99 frametime, 1 % low (frametime-based), avg fps.
- Win rule (BENCH-MATRIX §4): win = median p99 improves by more than the B0 p99 spread AND 1 % low doesn't drop by more than the B0 1 %-low spread; loss = worse beyond either; else tie (ties keep the current/upstream value).
- Noise floor (B0, C0 × 3 sessions on S1): p99 spread 0.40 ms, 1 %-low spread 6.6 fps (p99 9.64 ms, 1 % low 88.3 fps).
- Baselines: each variant is compared with the C1 session(s) of its own block
(
C1@B3,C1@B11.baseline, …), i.e. same JVM and same time window.
Highlights
-
The shipped tuning (C1) beats upstream defaults (C0) on every scene. C0 (B1) vs C1 (B2), same JVM (Prism default, 8 GB):
Scene p99 C0 → C1 (ms) 1 % low C0 → C1 (fps) avg fps C0 → C1 S1 vista 9.33 → 8.23 93.0 → 104.9 218 → 272 S2 Create floor 3.79 → 3.22 218.7 → 259.8 694 → 975 S3 mob pen 7.43 → 4.46 110.1 → 186.0 255 → 395 S4 airships 2.93 → 2.44 298.0 → 341.3 789 → 1073 S5 tour 3.47 → 2.95 194.2 → 223.8 788 → 1017 Startup ("Game took"): C0 31.38 s → C1 25.36 s. Atlas: C0 4096×4096×4 → C1 4096×4096×2.
-
Distant Horizons is the biggest single render cost. Turning DH off (B3.dh-off) gives S1 p99 7.99 → 5.38 ms, 1 % low 105.5 → 144.0; S5 3.09 → 2.34 ms, 211.2 → 282.4. DH's LOW quality preset at C1's radius 128 (
dh-qlow) is a win on S1 (−0.96 ms, +9.5 fps). SSAO costs 1.25 ms p99 on S1. -
Fresh Animations is expensive. S3 p99 4.32 → 8.65 ms (FA) / 10.20 ms (FA + Extensions); 1 % low 184.6 → 104.9 / 89.0. It is opt-in (not enabled by default) — keep it that way and document the cost.
-
The added performance mods earn their place. Removing BadOptimizations, Particle Core or Sodium Extra each loses (Particle Core / Sodium Extra: S3 1 % low −24 / −25 fps).
-
ModernFix
dynamic_resources(B6): startup 7.6 s faster (26.32 vs 33.92 s median) and better frametimes with it on. Ship is gated only on the manual missing-texture check (see B6 row).
Per-block verdicts
Numbers are p99 ms / 1 % low fps; Δ is vs the block's C1 baseline (+ = better p99).
| Block | Change tested | Result | Verdict | Ship action |
|---|---|---|---|---|
| B0 | noise floor, C0 × 3 on S1 | p99 9.64 (spread 0.40), 1 % low 88.3 (spread 6.6) | — | thresholds for all blocks |
| B1 | C0 baseline, S1–S5 | see highlights table; startup 31.38 s | reference | — |
| B2 | C1 vs C0 | C1 wins S1–S5 (table above) | win | C1 stays shipped; no bisect needed |
| B2 | C1-deck vs C1 | S1 7.08/115.0 vs 8.23/104.9 (win); S2/S4/S5 tie; S3 4.84/167.5 vs 4.46/186.0 (loss) | mixed | keep Deck tier as is (S3 loss: check entityShadows/particles on fleet P0) |
| B2 | C1-high vs C1 | S1 tie; S2–S5 losses (e.g. S4 2.66/311.6 vs 2.44/341.3) | expected cost | keep High as opt-in quality tier |
| B3 | DH radius 64 | S1 7.46/111.1 vs 7.99/105.5 (win); S5 tie | win S1 | already the Deck value; no change to C1 |
| B3 | DH radius 192 / 256 | S1 tie (7.69/105.5, 8.07/101.5); S5 tie | tie | decision: on this GPU 256 (upstream) ties 128; fleet floors decide — keep 128 until fleet data |
| B3 | DH LOW quality preset @ r128 (dh-qlow) |
S1 7.03/115.0 vs 7.99/105.5 (+0.96, +9.5) | win (S1 only) | candidate for DEFAULT after an S5 run + visual check |
| B3 | DH SSAO on | S1 9.24/95.3 (−1.25, −10.2) | loss | confirms C1 SSAO off |
| B3 | DH threads 4 | S1 tie; S5 3.32/197.4 vs 3.09/211.2 (loss) | loss | threads stay 2 |
| B3 | DH off (reference) | S1 5.38/144.0; S5 2.34/282.4 | — | cost reference only |
| B4 | GC J0/J1/J2/J3 @ 6 GB, S2+S5 | S5: J0 3.07/226.8, J1 3.11/228.1 (tie), J2 3.40/216.6 (loss), J3 3.75/173.4 (loss); S2 all tie | J0 | keep J0; heap ladder = B4h (run 2) |
| B5 | Sable ships 0/1/4 (proxy) | S4 3.00/280.6, 3.12/261.6, 2.97/273.2 | — | ships nearly free on this GPU |
| B5 | SSRD 32/64/128, sable-noshade (4 ships) | 3.02/265.0, 3.03/264.7, 3.11/261.9, 3.04/263.0 vs pooled C1 (−8 to −11 fps 1 % low) | loss/noise | keep C1 (SSRD 16, shading on); decay run deferred (fleet P0/P1) |
| B6 | ModernFix dynamic_resources off vs on (C1) | S1 9.11/83.8 vs 8.48/98.6; S2 3.29/236.1 vs 3.18/264.0; startup 33.92 vs 26.32 s | dynres on wins | ship modernfix-mixins.properties after the manual missing-texture check |
| B8 | Fresh Animations (+Ext) on S3 | 8.65/104.9 (FA), 10.20/89.0 (FA+Ext) vs 4.32/184.6 | large cost | keep opt-in; document in PLAYER-TUNING |
| B8 | EMF animation LOD 20 / 8 (with FA) | vs C1-fa (LOD 12) 8.65/104.9: LOD 20 8.91/101.8 (tie), LOD 8 8.16/108.0 (+0.49, +3.1 → win) | LOD 8 wins under FA | optional: LOD 8 for FA users (PLAYER-TUNING note) |
| B8 | LambDynamicLights fancy / fastest | 4.41/181.8, 4.35/187.7 vs 4.32/184.6 | tie | keep C1 fast |
| B9 | Flywheel light flat / smooth (S2) | 3.45/246.6 (tie), 3.46/238.9 (−8.6, loss) vs 3.44/247.5 | tie/loss | keep C1 TRI_LINEAR |
| B10 | Sound Physics rays up / strict / no caps (S2s) | 3.69/216.2 (loss), 3.58/233.7 (tie), 3.64/216.9 (loss) vs 3.52/237.6 | loss/tie | keep C1 |
| B11 | Sodium 3 builder threads, blend 0/2, mipmaps 4 | all losses except blend 2 (S5 tie); e.g. mip4 S1 8.82/97.6 vs 8.12/104.2 | loss/tie | keep C1 |
| B12a | remove BadOptimizations / Particle Core / Sodium Extra | each loses ≥1 scene (e.g. S3 3.93/205.3, 4.63/188.4, 4.61/187.5 vs 3.87/212.5) | keep all | no change |
| B4h, B7, B12b, B14 | see Run 2 below | |||
| B13 | spark profile | — | pending (manual) | — |
Run 2 (2026-10-17/18): B4h, B7, B12b, B14
Same machine and method; default JVM (run 1's B5–B12a ran on a leaked J3/6 GB JVM —
harness bug fixed in 60e10b3; within-block verdicts unaffected). Verdicts use the B0
thresholds (p99 0.40 ms, 1 % low 6.6 fps); one session per row, so treat single-scene
margins near the threshold as soft. Numbers: p99 ms / 1 % low fps.
B4h — heap ladder (J0, S2 + S5)
| heap | S2 | S5 |
|---|---|---|
| 4 GB | 3.58 / 229.1 | 3.32 / 188.7 |
| 5 GB | 3.58 / 238.6 | 3.13 / 209.1 |
| 6 GB | 3.63 / 225.7 | 3.14 / 211.8 |
| 8 GB | 3.58 / 234.5 | 3.18 / 209.2 |
Smallest heap within the winner's spread: 5 GB (S5 1 % low 209.1 vs best 211.8).
4 GB loses on S5 (−23 fps). Heap is a launcher setting, not a shipped config →
PLAYER-TUNING recommendation (≥ 5 GB), not a build.py change.
B7 — shaders (data for owner; no auto-decision)
| profile | S1 vegetation | S2 Create floor | S4 airships |
|---|---|---|---|
| none (run-2 re-run) | 8.13 / 103.6 (274 fps avg) | 3.28 / 260.4 (974) | 2.50 / 332.5 (1072) |
| MakeUp shadowless-medium | 8.69 / 97.3 (263) | 12.88 / 67.9 (120) | 2.58 / 314.0 (1050) |
| MakeUp medium | 10.04 / 88.4 (213) | 12.32 / 71.9 (121) | 2.94 / 280.4 (779) |
| Complementary LOW | 10.85 / 83.2 (182) | 13.23 / 64.0 (120) | 4.16 / 205.2 (444) |
| Complementary MEDIUM | 10.92 / 78.5 (175) | 13.27 / 67.4 (118) | 4.33 / 192.8 (401) |
All shader rows loaded the pack (no shader_not_loaded). The dominant cost is S2:
any shaderpack makes Flywheel drop its GPU backends, so the Create floor goes from
974 to ~120 fps average (p99 3.3 → ~13 ms) regardless of profile. Elsewhere MakeUp
shadowless is nearly free; Complementary costs ~2.7 ms p99 on S1 and ~2× frame time on S4.
→ Supports opt-in shaders, MakeUp first, and Iris Flywheel Compat (B12b).
B12b — trial mods (each vs its own baseline)
| trial | result | verdict |
|---|---|---|
| Iris Flywheel Compat 2.4.0 (MakeUp default, vs shader-baseline) | S2 3.98 / 218.3 vs 13.26 / 65.8 (avg 644 vs 117 fps); S1 9.92 / 84.8 vs 10.23 / 83.8 (tie); S4 3.10 / 266.1 vs 3.01 / 277.6 (1 % low −11.5, loss) | win under shaders (restores Flywheel instancing) — add after a visual check of Create contraptions/airships with a shaderpack (unofficial mod; breaks on Create/Iris updates) |
| Flerovium 1.1.2 (vs C1) | S3 3.48 / 229.9 vs 5.71 / 147.4; S1/S2/S5 tie | win on entities — but the B12b C1 S3 baseline (5.71) is worse than every run-1 C1 S3 (4.32–4.46), so the true gain is ~0.9 ms / +45 fps vs run 1. Add after one confirmation pair |
| Gnetum 3.3.6 | S3 4.41 / 189.3 (vs anomalous 5.71; ≈ run-1 C1) ; S1/S2/S5 tie | tie in practice → no add |
| Better Block Entities 1.3.4 | S2 3.51 / 245.3 (loss), S3 loss; with FA 9.86 / 89.8 vs 9.52 / 87.4 (tie) | reject |
| Super Resolution 0.9.2-alpha.2 | S1 8.74 / 99.7 (loss), S5 1 % low −16.6 (loss) | reject on this proxy (P0-only lever; its first launch rewrites fml.toml and demands a restart) |
| Ixeris 4.6.8 | S1 p99 −0.49 (loss), S5 1 % low −12.2 (loss); one crash (NPE in GameRules.getInt during server tick, retry ok) |
reject |
| Cull Leaves 4.1.1 (vs Better Leaves baseline) | S1 tie; S5 1 % low 208.8 vs 217.1 (loss) | reject |
New mod adds (Iris Flywheel Compat, Flerovium) ⇒ re-run B0 before any later matrix work.
B14 — spawn tuning (defaults vs shipped, 10 min per site, 1 run each)
| site | defaults total / tuned | shipped total / tuned |
|---|---|---|
| jungle (−8 107 −201) | 123 / 5 | 118 / 3 |
| river (904 63 88) | 208 / 29 | 208 / 25 |
| warm ocean (−200 63 56) | 155 / 22 | 130 / 0 |
Automated rule says back off, but it is driven by one Fowl Play herd: at the warm-ocean
site defaults spawned 14 geese + 8 ducks (on nearby land/water inside the 128-block
sphere), shipped none; the other "vanished" species (gazelle, dragonfly) had max count 1.
Sea bunny — the warm-ocean target — appeared under neither config, so that site doesn't
exercise it. Inconclusive at n = 1: needs ≥ 3 interleaved runs per config (≈ 3 h)
before any back-off; if it persists, the candidate is Fowl Play duck/goose weights.
All runs: 33 count rows, biome_ok=1, config check ok.
Caveats
-
Run 2: Super Resolution's first launch flipped
earlyWindowControlin the benchfml.toml(not journaled by the harness); later run-2 sessions started with it off. Loading-screen only — frametimes unaffected; startup times after 01:54 are not comparable. Reset 2026-10-18. -
Single proxy machine. Fleet machines (P0–P2) were unreachable; the fleet fps floors (P0 1 % low ≥ 30, P1/P2 ≥ 60) don't map to a 7800 XT — every verdict here is relative. B2's "C1 wins on ≥ 2 machines" gate is satisfied on one machine only.
-
This GPU hides render costs. Airships (B5), Flywheel lighting and DH radius are cheap here; the fleet's older GPUs may rank them differently. B5 decay is deferred to P0/P1.
-
GPU clocks unpinned (amdgpu DPM
auto, no root); every row carriesgpu_dpm_auto. -
JVM differed between block groups (harness bug). B0–B3 ran Prism's default args at 8 GB; B4 set each GC at 6 GB;
bench.pynever restoredinstance.cfg, so B5–B12a all ran on J3 Shenandoah, 6 GB. Every verdict compares sessions inside one block (same JVM), so verdicts stand, but absolute numbers aren't comparable across those groups and the B0 noise floor was measured under the other JVM. -
One crash: B10
spr-nocap, first attempt —OutOfMemoryError: Ran out of memory reading level dataafter an unusually slow (~1.5 min) resource load, on the leaked Shenandoah/6 GB JVM. The automatic retry was clean. -
Single sessions per variant (except B0/B6, 3 each); 3 reps per scene per session.