onion-mc/analysis/client-perf/BENCH-MATRIX.md

Phase 2 benchmark matrix

This replaces LEVERS.md §6. It is meant to run without judgement calls: every config is a generated directory, every scene is a datapack function, and every block ends in a decision rule. The measurement protocol and machine prep are in ../../phase0/P{0,1,2}-*.md (uncapped, vsync off, p99 frametime + 1 % low, 15 s warm-up discarded, 3 runs, one variable per comparison). This file adds what phase0 leaves open: the world, the scenes, the configs and the order.

Machines: P0 Steam Deck · P1 7700K + GTX 1060 6 GB (Windows) · P2 6700K + 1080 Ti, ~8 GB free RAM.

---

0. Prerequisite: boot check

The pack has never booted at its current size: 217 mods after a01bc3b added Default Options, Sodium Extra, Reese's Sodium Options, BadOptimizations and Particle Core. Before any timing, launch once and check latest.log for:

• missing deps and mixin apply failures;
• fishermans-haven biome-modifier errors;
• ftbteams references;
• Default Options' Populated … with default options lines (proves the seeding works);
• no YACL "Failed to deserialize" lines for fowlplay or crittersandcompanions (spawn tuning, SPAWN-TUNING.md);
• a working Sodium video-settings screen (Reese's Sodium Options replaces it; SSRD injects a slider into it).

r5 also flags a reported Iris 1.8.14-beta.1 ↔ Sodium 0.8.13 mixin incompatibility on NeoForge 1.21.1, with a third-party "Iris-Sodium Compat" patch on CurseForge. Check for it with a shaderpack enabled. If the pack doesn't boot cleanly, fix that first. Bisect the five new mods before anything older.

1. Fixed inputs

Input	Value
Pack	one commit, recorded in every row (git rev-parse --short HEAD). The Puzzles Lib bump goes in before B0 if approved, never mid-matrix.
Bench instance	a separate Prism instance per machine with the packwiz PreLaunchCommand removed, so it can't self-update mid-matrix. Mods are copied from a normal install of the recorded commit.
Java	Temurin 21 on all three (phase0). Check java -XX:+UseShenandoahGC -version works (needed for J3).
World	bench-world-v1.zip, built once on P2 (§2), sha256 recorded, unpacked identically on every machine
Configs	analysis/client-perf/proposed-configs/out/<ID>/, generated by build.py at the recorded commit
In-game pins	phase0 Step 7/8. The options.txt keys that matter are already in every bench variant (maxFps:260, enableVsync:false, renderDistance:8, simulationDistance:8). GUI scale, FOV, resolution and fullscreen are recorded per machine and never changed.
Resource packs / shaders	none, unless the block says otherwise

Applying a config

With the game closed, run python3 proposed-configs/apply_variant.py <ID> <instance>/.minecraft. It copies all 10 config files (every variant has every file, so nothing from a previous variant survives) and merges options.txt key by key, so GUI scale, resolution and keybinds are kept. It also writes the ID to .minecraft/BENCH_VARIANT.txt; check it before each run. On Windows without Python, copy the config/ tree by hand and edit the listed options.txt keys in game. Default Options seeds only missing files, so it never interferes. Relaunch for every config change, even ones a mod could hot-reload. Run all of that config's scenes in one session, in the order S1 → S2 → S3 → S4 → S5 → S6.

JVM sets (Prism → Java arguments; -Xmx per block)

All four were checked to parse on OpenJDK 21.0.12.

ID	Arguments (plus -Xmx)
J0 (phase0 default)	-XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+DisableExplicitGC -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=40 -XX:G1HeapRegionSize=8M -XX:G1ReservePercent=20 -XX:G1HeapWastePercent=5 -XX:G1MixedGCCountTarget=4 -XX:InitiatingHeapOccupancyPercent=15 -XX:G1MixedGCLiveThresholdPercent=90 -XX:G1RSetUpdatingPauseTimePercent=5 -XX:SurvivorRatio=32 -XX:+PerfDisableSharedMem -XX:MaxTenuringThreshold=1
J1 (client G1, brucethemoose)	-XX:+UseG1GC -XX:MaxGCPauseMillis=37 -XX:G1HeapRegionSize=16M -XX:+UnlockExperimentalVMOptions -XX:G1NewSizePercent=23 -XX:G1ReservePercent=20 -XX:G1MixedGCCountTarget=3 -XX:G1HeapWastePercent=20 -XX:InitiatingHeapOccupancyPercent=10 -XX:G1RSetUpdatingPauseTimePercent=0 -XX:MaxTenuringThreshold=1 -XX:GCTimeRatio=99 -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem
J2 (Generational ZGC)	-XX:+UseZGC -XX:+ZGenerational -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem. Plain -XX:+UseZGC on JDK 21 is the old non-generational ZGC.
J3 (Shenandoah)	-XX:+UseShenandoahGC -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem. Not r6's -XX:ShenandoahGCMode=iu: on JDK 21 that aborts VM startup unless -XX:+UnlockExperimentalVMOptions is added (tested), and the mode is gone in later JDKs.

Default heap outside B4 (phase0): P0 -Xmx5G, P1 -Xmx8G, P2 -Xmx6G, all with J0.

---

2. Bench world (built once on P2)

1. Copy bench/onion-bench/ into a new world's datapacks/ folder. Create the world: Creative, cheats on, seed onionbench, default world type. Run /function bench:setup.
2. S2: /function bench:s2, then /function bench:s2_setup. Check that the shafts and cogwheels rotate. If the /fill-placed network didn't initialise, break and re-place one motor per row by hand. Accept when F3 shows Flywheel instances and every row turns.
3. S3: /function bench:s3, then /function bench:s3_setup. Check the counts with /execute if entity @e[type=villager,x=1999,y=150,z=-1,dx=25,dy=8,dz=25] (expect 30).
4. S4: /function bench:s4 (camera at 3000 172 -10, facing south). Build one Aeronautics test ship (about 5×7×15 blocks, planks + glass + 1 propeller) and assemble it so it hovers at y≈170. Copy it with a Create schematic to get 4 ships, centred at z = 32, 96, 192 and 384 (2, 6, 12 and 24 chunks away). SSRD 16 then cuts only the 384 one. Record the sub-level count Sable shows. This is the only hand-built scene. It's baked into the zip, so runs never rebuild it. To get the 0- and 1-ship states, break ships into blocks in a copy of the world: bench-world-v1-s4-0.zip and -s4-1.zip.
5. DH pre-generation: use the upstream DH file (data/observed-default-configs/DistantHorizons.toml: SURFACE_THEN_CHUNKS, radius 256), then stand at S1 until DH's F3 generation queue reads 0. Run /function bench:tour_start with the same settings, waiting at each hop until the queue drains (re-issue bench:tour/stepN by hand). Do the same at S4. Every bench config then runs with generatorPlan = "DISABLED", so no world generation happens during measurement.
6. Leave the world at S1, save, quit and zip it.

Scenes

ID	Command	What it loads	Capture
S1 vista	/function bench:s1	0 200 0, facing south, 20° down. Pure terrain + DH LODs	15 s warm-up + 30 s, hands off
S2 Create floor	/function bench:s2	2,304 kinetic BEs (Flywheel instancing; vanilla BE path under shaders)	15 + 30 s
S2s sound storm	S2 + /function bench:storm_on (stop: storm_off)	24 sounds/tick from 3 capped Create IDs (8 per ID per tick vs cap 4)	15 + 30 s, sound on
S3 mob pen	/function bench:s3	30 villagers, 15 cows, 15 sheep, 10 allays carrying torches (moving LDL sources), 10 dropped torches	15 + 30 s
S4 airships	/function bench:s4	0 / 1 / 4 Sable sub-levels at 2–24 chunks	15 + 30 s
S5 tour	/function bench:s1, wait 10 s, then /function bench:tour_start	24 teleports of 160 blocks every 5 s along +x: chunk + LOD streaming, GC churn	all 120 s, no warm-up discard
S6 GUI	at S1, press E with the JEI overlay open; mouse parked over the JEI list	GUI / JEI / tooltip path	15 + 30 s

---

3. Config IDs

Generated by proposed-configs/build.py. out/MANIFEST.md lists every changed key. C0 = every file at upstream default; C1 = the proposed pack default (= mid tier). All other IDs are C1 with one change.

ID	Change vs C1
C0	everything back to upstream (DH 256 / SSAO on / 4 threads, Flywheel SMOOTH, LDL fancy, SPR 32/4, mip 4, blend 2, ModernFix dynres off, …)
C1-deck, C1-high	the tier deltas from proposed-configs/README.md §2
dh-r256 / dh-r192 / dh-r64	DH LOD radius (C1 = 128)
dh-off	C1, then DH options → Enable Rendering off after each launch. That toggle is GUI-only and never written to the file.
dh-ssao / dh-thr4 / dh-qlow	SSAO on / 4 DH threads / DH's LOW quality preset at radius 128
fw-flat / fw-smooth	Flywheel lightSmoothness (C1 = TRI_LINEAR)
mf-dynres-off	ModernFix dynamic_resources=false
ssrd-32 / -64 / -128	SSRD physicsRenderDistance (C1 = 16)
sable-noshade	Sable sub_level_dynamic_shading=false
emf-lod20 / emf-lod8	EMF LOD 20 + Average (= upstream) / LOD 8 + retainDetailOnLowFps=false
ldl-fancy / ldl-fastest	LambDynamicLights mode (C1 = fast); fastest also turns entity light sources off
spr-up / spr-strict / spr-nocap	SPR rays 32/4 (upstream) / strict_occlusion=true / Create sound caps removed
sod-thr3	Sodium chunk_builder_threads=3 (C1 = 0 = auto = 2)
opt-blend2 / opt-blend0 / opt-mip4	biome blend 5×5 / off; mipmap 4

---

4. Blocks, in run order

"Win" means the median p99 frametime improves by more than that machine's B0 spread, and the 1 % low doesn't get worse by more than the spread. Ties keep the upstream value (less change). FPS floors for tier picks: P0 1 % low ≥ 30, P1/P2 1 % low ≥ 60.

Block	Machines	Configs (× JVM)	Scenes	Records	Decision rule
B0 noise floor	all	C0 ×3 sessions	S1	p99 / 1 % spread	Defines each machine's threshold. P2 also checks MemAvailable drift (phase0 Step 4).
B1 baseline	all	C0	S1–S6	+ Game took N seconds to start (ModernFix log line), the Created: …blocks.png-atlas line, /sparkc heapsummary after S1 loads, VRAM peak (P1/P2)	Reference numbers for everything below
B2 composite	all; tiers on their machine	C1 (all), C1-deck (P0), C1-high (P2)	S1–S6	as B1	If C1 isn't a win on ≥ 2 machines, stop and bisect with B3–B11 before shipping anything
B3 Distant Horizons	all	dh-r256, dh-r192, C1, dh-r64, dh-off	S1, S5	+ DH memory in F3	Radius per tier = the largest radius meeting the FPS floor on S1 and S5: P0 → deck, P1 → default, P2 → high
	P0, P1	dh-ssao, dh-qlow	S1		Adopt SSAO-off / LOW-quality for that tier on a win
	all	dh-thr4	S1, S5		Keep 2 threads unless 4 is a win on S5 and not a loss on S1
B4 heap & GC	P2, then P0	C1 (C1-deck on P0) × J0, J1, J2, J3 at -Xmx6G (P2) / -Xmx5G (P0); then the winner at 4 / 5 / 6 G	S5 (120 s), S2	+ /sparkc gc after S5 (pause count / max / total), swap delta (P2)	GC = best S5 1 % low. Heap = the smallest size within the spread of the best. Feeds PLAYER-TUNING.md §2. P1 isn't run (32 GB, not a constraint).
B5 Sable / SSRD	P0, P1	world s4-0 / s4-1 / s4-4 × C1; s4-4 × ssrd-32, ssrd-64, ssrd-128, sable-noshade	S4	sub-level count	SSRD per tier = the largest value meeting the floor with 4 ships. sable-noshade ships only on a win and if it looks acceptable.
	P1	C1, s4-4, J0	S4, 45 min (2 runs)	FPS at 0 / 15 / 30 / 45 min, /sparkc heapsummary at each	Decay > spread is a Sable #1011-class finding: report it, no config fix
B6 ModernFix	P2, P0	C1 vs mf-dynres-off, 3 cold launches each	S1, S2, S6	startup, heap after load, missing-texture check (below)	Ship dynamic_resources=true only if: heap −300 MB or more or startup faster beyond spread, and zero missing textures, and no crash in 3 launches
B7 shaders	P1, P2: all rows; P0: MakeUp rows only	C1 with: none / MakeUp shadowless_medium / MakeUp medium / Complementary LOW / Complementary MEDIUM (profiles set in Iris' shader settings)	S2, S1, S4 (s4-4)	Flywheel backend line in F3 (expect off with shaders); visual check for Iris #3164 on ships	Data for the owner's shader policy, no auto-decision. Report the FPS cost per profile and whether ship corruption appears with shadow profiles
B8 entities	P2, P0 (P1: FA rows only)	C1 / C1 + Fresh Animations / C1 + FA + Extensions / emf-lod20 + FA / emf-lod8 + FA; ldl-fancy, ldl-fastest (no FA)	S3		EMF: adopt the smallest LOD that's a win with FA on. LDL: per tier, the cheapest mode that's a win. FA guidance → PLAYER-TUNING.md §4
B9 Flywheel	P0, P1	fw-flat, C1, fw-smooth (no shaders)	S2		Per tier: the smoothest value meeting the floor
B10 Sound Physics	P2, P0	spr-up, C1, spr-strict, spr-nocap	S2s		Adopt each change that's a win. Listen for obvious audio breakage.
B11 vanilla / Sodium	P1, P2	sod-thr3	S5, S1		3 threads only if S5 wins and S1 doesn't lose
	all	opt-blend2, opt-blend0	S5		Lowest blend radius that's a win, per tier
	P1	opt-mip4	S1, S2	VRAM peak	Mip 4 for mid/high only if VRAM peak ≤ 5.5 GB and no loss
B12a added-mod ablation	P2, P0 (P1: Sodium Extra only)	C1 with one of BadOptimizations / Particle Core / Sodium Extra removed from mods/	S1, S3, S2s		Keep the mod unless removing it is a win (it costs FPS) or it breaks something visible. Sodium Extra + RSO stay regardless for their per-machine options.
B12b trials	per mod: Gnetum (all), Better Block Entities (all, with FA on and off), Flerovium (all), Super Resolution (P0), Ixeris (P1), Iris Flywheel Compat (with shaders, B7 scenes), Cull Leaves + MidnightLib (with Better Leaves)	C1 + one mod	the scenes that mod targets		Owner pre-approved helpful adds. Add with packwiz modrinth add only on a win with no breakage. Not More Culling: its NeoForge build declares minecraft [1.21,1.21.1). A new mod set means re-running B0 first.
B13 profile	P2	the worst scene from B2 under C1	that scene	/sparkc profiler start --timeout 60, upload or save the report	Flag any mod over 5 % of render-thread time that isn't Sodium, Iris, DH or Flywheel
B14 spawn tuning	P2	singleplayer, defaults vs the shipped spawn files	river, jungle, warm ocean (10 min each)	per-species and total entity counts	Procedure and keep/back-off rule: SPAWN-TUNING.md §B14

Missing-texture check (B6): in creative, open JEI, search @chipped, @rechiseled and @handcrafted, scroll each list, and place a 4×4 wall of 3 connected-texture Rechiseled blocks. Any magenta/black texture or broken CTM seam fails the check.

Rough budget

Assumes ~4 min per launch (B1 measures the real figure) and ~3 min per scene for 3 × 45 s.

Block	Launches (all machines)	Hours
B0–B2	~12	~5
B3	~20	~4
B4	~14	~3
B5	~12 + 1.5 h decay	~4
B6	12 cold launches	~2
B7–B11	~45	~7
Total		~25 h machine time, ~10 h per machine

5. Output

One CSV row per run, using the phase0 recording sheet plus config_id, jvm_set, xmx, scene, world_sha, pack_commit. When the matrix is done, write the winners back into proposed-configs/build.py (DEFAULT/DECK/HIGH), regenerate, and only then present the files for shipping.