logan
611a1a5952
Everything the next agent needs to run the Phase 2 optimisation loop on a machine that can launch the client: - analysis/client-perf/: LEVERS (ranked levers + corrections), BENCH-MATRIX (world, scenes, config IDs, blocks B0-B14 with decision rules), PLAYER-TUNING, SPAWN-TUNING, research briefs, the jar scanner and its output, observed default configs, and proposed-configs/ (build.py generates tiers and bench variants and fails on unknown keys; apply_variant.py applies a variant and merges options.txt). - analysis/client-perf/bench/onion-bench/: bench-world datapack (scene teleports, Create grid, mob pen, sound storm, 120 s teleport tour). - analysis/audit/: the static recipe/dependency audit, including gen_fixes.py, which generates the shipped Paxi datapack and was never tracked. - phase0/: per-machine measurement harness (Steam Deck, 7700K/1060, 6700K/1080 Ti). - .pi/journal.md: running work log; read it first. All of it is packwizignored, so nothing here ships to players.
191 lines
15 KiB
Markdown
191 lines
15 KiB
Markdown
# Phase 2 benchmark matrix
|
||
|
||
This replaces `LEVERS.md` §6. It is meant to run without judgement calls: every config is a
|
||
generated directory, every scene is a datapack function, and every block ends in a decision
|
||
rule. The measurement protocol and machine prep are in `../../phase0/P{0,1,2}-*.md`
|
||
(uncapped, vsync off, p99 frametime + 1 % low, 15 s warm-up discarded, 3 runs, one variable
|
||
per comparison). This file adds what phase0 leaves open: the world, the scenes, the configs
|
||
and the order.
|
||
|
||
Machines: **P0** Steam Deck · **P1** 7700K + GTX 1060 6 GB (Windows) · **P2** 6700K + 1080 Ti,
|
||
~8 GB free RAM.
|
||
|
||
---
|
||
|
||
## 0. Prerequisite: boot check
|
||
|
||
The pack has never booted at its current size: 217 mods after `a01bc3b` added Default
|
||
Options, Sodium Extra, Reese's Sodium Options, BadOptimizations and Particle Core. Before
|
||
any timing, launch once and check `latest.log` for:
|
||
- missing deps and mixin apply failures;
|
||
- `fishermans-haven` biome-modifier errors;
|
||
- `ftbteams` references;
|
||
- Default Options' `Populated … with default options` lines (proves the seeding works);
|
||
- no YACL "Failed to deserialize" lines for `fowlplay` or `crittersandcompanions`
|
||
(spawn tuning, `SPAWN-TUNING.md`);
|
||
- a working Sodium video-settings screen (Reese's Sodium Options replaces it; SSRD injects a
|
||
slider into it).
|
||
|
||
r5 also flags a reported Iris 1.8.14-beta.1 ↔ Sodium 0.8.13 mixin incompatibility on
|
||
NeoForge 1.21.1, with a third-party "Iris-Sodium Compat" patch on CurseForge. Check for it
|
||
with a shaderpack enabled. **If the pack doesn't boot cleanly, fix that first. Bisect the
|
||
five new mods before anything older.**
|
||
|
||
## 1. Fixed inputs
|
||
|
||
| Input | Value |
|
||
| --- | --- |
|
||
| Pack | one commit, recorded in every row (`git rev-parse --short HEAD`). The Puzzles Lib bump goes in before B0 if approved, never mid-matrix. |
|
||
| Bench instance | a **separate** Prism instance per machine with the packwiz `PreLaunchCommand` **removed**, so it can't self-update mid-matrix. Mods are copied from a normal install of the recorded commit. |
|
||
| Java | Temurin 21 on all three (phase0). Check `java -XX:+UseShenandoahGC -version` works (needed for J3). |
|
||
| World | `bench-world-v1.zip`, built once on P2 (§2), sha256 recorded, unpacked identically on every machine |
|
||
| Configs | `analysis/client-perf/proposed-configs/out/<ID>/`, generated by `build.py` at the recorded commit |
|
||
| In-game pins | phase0 Step 7/8. The `options.txt` keys that matter are already in every bench variant (`maxFps:260`, `enableVsync:false`, `renderDistance:8`, `simulationDistance:8`). GUI scale, FOV, resolution and fullscreen are recorded per machine and never changed. |
|
||
| Resource packs / shaders | none, unless the block says otherwise |
|
||
|
||
### Applying a config
|
||
|
||
With the game closed, run
|
||
`python3 proposed-configs/apply_variant.py <ID> <instance>/.minecraft`. It copies all 10
|
||
config files (every variant has every file, so nothing from a previous variant survives)
|
||
and **merges** `options.txt` key by key, so GUI scale, resolution and keybinds are kept.
|
||
It also writes the ID to `.minecraft/BENCH_VARIANT.txt`; check it before each run. On
|
||
Windows without Python, copy the `config/` tree by hand and edit the listed `options.txt`
|
||
keys in game. Default Options seeds only missing files, so it never interferes.
|
||
**Relaunch for every config change**, even ones a mod could hot-reload. Run all of that
|
||
config's scenes in one session, in the order S1 → S2 → S3 → S4 → S5 → S6.
|
||
|
||
### JVM sets (Prism → Java arguments; `-Xmx` per block)
|
||
|
||
All four were checked to parse on OpenJDK 21.0.12.
|
||
|
||
| ID | Arguments (plus `-Xmx`) |
|
||
| --- | --- |
|
||
| **J0** (phase0 default) | `-XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+DisableExplicitGC -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=40 -XX:G1HeapRegionSize=8M -XX:G1ReservePercent=20 -XX:G1HeapWastePercent=5 -XX:G1MixedGCCountTarget=4 -XX:InitiatingHeapOccupancyPercent=15 -XX:G1MixedGCLiveThresholdPercent=90 -XX:G1RSetUpdatingPauseTimePercent=5 -XX:SurvivorRatio=32 -XX:+PerfDisableSharedMem -XX:MaxTenuringThreshold=1` |
|
||
| **J1** (client G1, brucethemoose) | `-XX:+UseG1GC -XX:MaxGCPauseMillis=37 -XX:G1HeapRegionSize=16M -XX:+UnlockExperimentalVMOptions -XX:G1NewSizePercent=23 -XX:G1ReservePercent=20 -XX:G1MixedGCCountTarget=3 -XX:G1HeapWastePercent=20 -XX:InitiatingHeapOccupancyPercent=10 -XX:G1RSetUpdatingPauseTimePercent=0 -XX:MaxTenuringThreshold=1 -XX:GCTimeRatio=99 -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem` |
|
||
| **J2** (Generational ZGC) | `-XX:+UseZGC -XX:+ZGenerational -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem`. Plain `-XX:+UseZGC` on JDK 21 is the old non-generational ZGC. |
|
||
| **J3** (Shenandoah) | `-XX:+UseShenandoahGC -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem`. **Not** r6's `-XX:ShenandoahGCMode=iu`: on JDK 21 that aborts VM startup unless `-XX:+UnlockExperimentalVMOptions` is added (tested), and the mode is gone in later JDKs. |
|
||
|
||
Default heap outside B4 (phase0): P0 `-Xmx5G`, P1 `-Xmx8G`, P2 `-Xmx6G`, all with J0.
|
||
|
||
---
|
||
|
||
## 2. Bench world (built once on P2)
|
||
|
||
1. Copy `bench/onion-bench/` into a new world's `datapacks/` folder. Create the world:
|
||
Creative, cheats on, seed `onionbench`, default world type. Run `/function bench:setup`.
|
||
2. **S2**: `/function bench:s2`, then `/function bench:s2_setup`. Check that the shafts and
|
||
cogwheels **rotate**. If the `/fill`-placed network didn't initialise, break and re-place
|
||
one motor per row by hand. Accept when F3 shows Flywheel instances and every row turns.
|
||
3. **S3**: `/function bench:s3`, then `/function bench:s3_setup`. Check the counts with
|
||
`/execute if entity @e[type=villager,x=1999,y=150,z=-1,dx=25,dy=8,dz=25]` (expect 30).
|
||
4. **S4**: `/function bench:s4` (camera at 3000 172 -10, facing south). Build one
|
||
Aeronautics test ship (about 5×7×15 blocks, planks + glass + 1 propeller) and assemble it
|
||
so it hovers at y≈170. Copy it with a Create schematic to get **4 ships, centred at z =
|
||
32, 96, 192 and 384** (2, 6, 12 and 24 chunks away). SSRD 16 then cuts only the 384 one.
|
||
Record the sub-level count Sable shows. This is the only hand-built scene. It's baked into
|
||
the zip, so runs never rebuild it. To get the 0- and 1-ship states, break ships into
|
||
blocks *in a copy of the world*: `bench-world-v1-s4-0.zip` and `-s4-1.zip`.
|
||
5. **DH pre-generation**: use the upstream DH file
|
||
(`data/observed-default-configs/DistantHorizons.toml`: `SURFACE_THEN_CHUNKS`, radius
|
||
256), then stand at S1 until DH's F3 generation queue reads 0.
|
||
Run `/function bench:tour_start` with the same settings, waiting at each hop until the
|
||
queue drains (re-issue `bench:tour/stepN` by hand). Do the same at S4. Every bench config
|
||
then runs with `generatorPlan = "DISABLED"`, so no world generation happens during
|
||
measurement.
|
||
6. Leave the world at S1, save, quit and zip it.
|
||
|
||
### Scenes
|
||
|
||
| ID | Command | What it loads | Capture |
|
||
| --- | --- | --- | --- |
|
||
| **S1** vista | `/function bench:s1` | 0 200 0, facing south, 20° down. Pure terrain + DH LODs | 15 s warm-up + 30 s, hands off |
|
||
| **S2** Create floor | `/function bench:s2` | 2,304 kinetic BEs (Flywheel instancing; vanilla BE path under shaders) | 15 + 30 s |
|
||
| **S2s** sound storm | S2 + `/function bench:storm_on` (stop: `storm_off`) | 24 sounds/tick from 3 capped Create IDs (8 per ID per tick vs cap 4) | 15 + 30 s, sound on |
|
||
| **S3** mob pen | `/function bench:s3` | 30 villagers, 15 cows, 15 sheep, 10 allays carrying torches (moving LDL sources), 10 dropped torches | 15 + 30 s |
|
||
| **S4** airships | `/function bench:s4` | 0 / 1 / 4 Sable sub-levels at 2–24 chunks | 15 + 30 s |
|
||
| **S5** tour | `/function bench:s1`, wait 10 s, then `/function bench:tour_start` | 24 teleports of 160 blocks every 5 s along +x: chunk + LOD streaming, GC churn | **all 120 s, no warm-up discard** |
|
||
| **S6** GUI | at S1, press E with the JEI overlay open; mouse parked over the JEI list | GUI / JEI / tooltip path | 15 + 30 s |
|
||
|
||
---
|
||
|
||
## 3. Config IDs
|
||
|
||
Generated by `proposed-configs/build.py`. `out/MANIFEST.md` lists every changed key.
|
||
**C0** = every file at upstream default; **C1** = the proposed pack default (= mid tier).
|
||
All other IDs are C1 with **one** change.
|
||
|
||
| ID | Change vs C1 |
|
||
| --- | --- |
|
||
| C0 | everything back to upstream (DH 256 / SSAO on / 4 threads, Flywheel SMOOTH, LDL fancy, SPR 32/4, mip 4, blend 2, ModernFix dynres off, …) |
|
||
| C1-deck, C1-high | the tier deltas from `proposed-configs/README.md` §2 |
|
||
| dh-r256 / dh-r192 / dh-r64 | DH LOD radius (C1 = 128) |
|
||
| dh-off | C1, then DH options → *Enable Rendering* off after each launch. That toggle is GUI-only and never written to the file. |
|
||
| dh-ssao / dh-thr4 / dh-qlow | SSAO on / 4 DH threads / DH's LOW quality preset at radius 128 |
|
||
| fw-flat / fw-smooth | Flywheel `lightSmoothness` (C1 = TRI_LINEAR) |
|
||
| mf-dynres-off | ModernFix `dynamic_resources=false` |
|
||
| ssrd-32 / -64 / -128 | SSRD `physicsRenderDistance` (C1 = 16) |
|
||
| sable-noshade | Sable `sub_level_dynamic_shading=false` |
|
||
| emf-lod20 / emf-lod8 | EMF LOD 20 + Average (= upstream) / LOD 8 + `retainDetailOnLowFps=false` |
|
||
| ldl-fancy / ldl-fastest | LambDynamicLights mode (C1 = fast); `fastest` also turns entity light sources off |
|
||
| spr-up / spr-strict / spr-nocap | SPR rays 32/4 (upstream) / `strict_occlusion=true` / Create sound caps removed |
|
||
| sod-thr3 | Sodium `chunk_builder_threads=3` (C1 = 0 = auto = 2) |
|
||
| opt-blend2 / opt-blend0 / opt-mip4 | biome blend 5×5 / off; mipmap 4 |
|
||
|
||
---
|
||
|
||
## 4. Blocks, in run order
|
||
|
||
"Win" means the median p99 frametime improves by more than that machine's **B0 spread**, and
|
||
the 1 % low doesn't get worse by more than the spread. Ties keep the upstream value (less
|
||
change). FPS floors for tier picks: **P0 1 % low ≥ 30**, **P1/P2 1 % low ≥ 60**.
|
||
|
||
| Block | Machines | Configs (× JVM) | Scenes | Records | Decision rule |
|
||
| --- | --- | --- | --- | --- | --- |
|
||
| **B0** noise floor | all | C0 ×3 sessions | S1 | p99 / 1 % spread | Defines each machine's threshold. P2 also checks `MemAvailable` drift (phase0 Step 4). |
|
||
| **B1** baseline | all | C0 | S1–S6 | + `Game took N seconds to start` (ModernFix log line), the `Created: …blocks.png-atlas` line, `/sparkc heapsummary` after S1 loads, VRAM peak (P1/P2) | Reference numbers for everything below |
|
||
| **B2** composite | all; tiers on their machine | C1 (all), C1-deck (P0), C1-high (P2) | S1–S6 | as B1 | If C1 isn't a win on ≥ 2 machines, stop and bisect with B3–B11 before shipping anything |
|
||
| **B3** Distant Horizons | all | dh-r256, dh-r192, C1, dh-r64, dh-off | S1, S5 | + DH memory in F3 | **Radius per tier** = the largest radius meeting the FPS floor on S1 *and* S5: P0 → deck, P1 → default, P2 → high |
|
||
| | P0, P1 | dh-ssao, dh-qlow | S1 | | Adopt SSAO-off / LOW-quality for that tier on a win |
|
||
| | all | dh-thr4 | S1, S5 | | Keep 2 threads unless 4 is a win on S5 *and* not a loss on S1 |
|
||
| **B4** heap & GC | P2, then P0 | C1 (C1-deck on P0) × J0, J1, J2, J3 at `-Xmx6G` (P2) / `-Xmx5G` (P0); then the winner at 4 / 5 / 6 G | S5 (120 s), S2 | + `/sparkc gc` after S5 (pause count / max / total), swap delta (P2) | GC = best S5 1 % low. Heap = the smallest size within the spread of the best. Feeds `PLAYER-TUNING.md` §2. P1 isn't run (32 GB, not a constraint). |
|
||
| **B5** Sable / SSRD | P0, P1 | world s4-0 / s4-1 / s4-4 × C1; s4-4 × ssrd-32, ssrd-64, ssrd-128, sable-noshade | S4 | sub-level count | SSRD per tier = the largest value meeting the floor with 4 ships. sable-noshade ships only on a win *and* if it looks acceptable. |
|
||
| | P1 | C1, s4-4, J0 | S4, **45 min** (2 runs) | FPS at 0 / 15 / 30 / 45 min, `/sparkc heapsummary` at each | Decay > spread is a Sable #1011-class finding: report it, no config fix |
|
||
| **B6** ModernFix | P2, P0 | C1 vs mf-dynres-off, 3 cold launches each | S1, S2, S6 | startup, heap after load, missing-texture check (below) | Ship `dynamic_resources=true` only if: heap −300 MB or more *or* startup faster beyond spread, **and** zero missing textures, **and** no crash in 3 launches |
|
||
| **B7** shaders | P1, P2: all rows; P0: MakeUp rows only | C1 with: none / MakeUp `shadowless_medium` / MakeUp `medium` / Complementary `LOW` / Complementary `MEDIUM` (profiles set in Iris' shader settings) | S2, S1, S4 (s4-4) | `Flywheel` backend line in F3 (expect off with shaders); visual check for Iris #3164 on ships | Data for the owner's shader policy, no auto-decision. Report the FPS cost per profile and whether ship corruption appears with shadow profiles |
|
||
| **B8** entities | P2, P0 (P1: FA rows only) | C1 / C1 + Fresh Animations / C1 + FA + Extensions / emf-lod20 + FA / emf-lod8 + FA; ldl-fancy, ldl-fastest (no FA) | S3 | | EMF: adopt the smallest LOD that's a win with FA on. LDL: per tier, the cheapest mode that's a win. FA guidance → `PLAYER-TUNING.md` §4 |
|
||
| **B9** Flywheel | P0, P1 | fw-flat, C1, fw-smooth (no shaders) | S2 | | Per tier: the smoothest value meeting the floor |
|
||
| **B10** Sound Physics | P2, P0 | spr-up, C1, spr-strict, spr-nocap | S2s | | Adopt each change that's a win. Listen for obvious audio breakage. |
|
||
| **B11** vanilla / Sodium | P1, P2 | sod-thr3 | S5, S1 | | 3 threads only if S5 wins and S1 doesn't lose |
|
||
| | all | opt-blend2, opt-blend0 | S5 | | Lowest blend radius that's a win, per tier |
|
||
| | P1 | opt-mip4 | S1, S2 | VRAM peak | Mip 4 for mid/high only if VRAM peak ≤ 5.5 GB and no loss |
|
||
| **B12a** added-mod ablation | P2, P0 (P1: Sodium Extra only) | C1 with **one** of BadOptimizations / Particle Core / Sodium Extra removed from `mods/` | S1, S3, S2s | | Keep the mod unless removing it is a win (it costs FPS) or it breaks something visible. Sodium Extra + RSO stay regardless for their per-machine options. |
|
||
| **B12b** trials | per mod: Gnetum (all), Better Block Entities (all, with FA on and off), Flerovium (all), Super Resolution (P0), Ixeris (P1), Iris Flywheel Compat (with shaders, B7 scenes), Cull Leaves + MidnightLib (with Better Leaves) | C1 + one mod | the scenes that mod targets | | Owner pre-approved helpful adds. Add with `packwiz modrinth add` only on a win with no breakage. **Not More Culling**: its NeoForge build declares `minecraft [1.21,1.21.1)`. A new mod set means re-running B0 first. |
|
||
| **B13** profile | P2 | the worst scene from B2 under C1 | that scene | `/sparkc profiler start --timeout 60`, upload or save the report | Flag any mod over 5 % of render-thread time that isn't Sodium, Iris, DH or Flywheel |
|
||
| **B14** spawn tuning | P2 | singleplayer, defaults vs the shipped spawn files | river, jungle, warm ocean (10 min each) | per-species and total entity counts | Procedure and keep/back-off rule: `SPAWN-TUNING.md` §B14 |
|
||
|
||
**Missing-texture check (B6)**: in creative, open JEI, search `@chipped`, `@rechiseled` and
|
||
`@handcrafted`, scroll each list, and place a 4×4 wall of 3 connected-texture Rechiseled
|
||
blocks. Any magenta/black texture or broken CTM seam fails the check.
|
||
|
||
### Rough budget
|
||
|
||
Assumes ~4 min per launch (B1 measures the real figure) and ~3 min per scene for 3 × 45 s.
|
||
|
||
| Block | Launches (all machines) | Hours |
|
||
| --- | --- | --- |
|
||
| B0–B2 | ~12 | ~5 |
|
||
| B3 | ~20 | ~4 |
|
||
| B4 | ~14 | ~3 |
|
||
| B5 | ~12 + 1.5 h decay | ~4 |
|
||
| B6 | 12 cold launches | ~2 |
|
||
| B7–B11 | ~45 | ~7 |
|
||
| **Total** | | **~25 h machine time**, ~10 h per machine |
|
||
|
||
## 5. Output
|
||
|
||
One CSV row per run, using the phase0 recording sheet plus `config_id`, `jvm_set`, `xmx`,
|
||
`scene`, `world_sha`, `pack_commit`. When the matrix is done, write the winners back into
|
||
`proposed-configs/build.py` (DEFAULT/DECK/HIGH), regenerate, and only then present the files
|
||
for shipping.
|