Compare commits
2
Commits
7e998fa39f
...
769af6fa2d
| Author | SHA1 | Date | |
|---|---|---|---|
 logan
|
769af6fa2d | ||
|
logan
|
1d02bf1cf6 |
No files matched your search
+13
-4
@@ -11,13 +11,17 @@ run.sh
|
||||
*.jar
|
||||
*.jar.log
|
||||
run.bat
|
||||
# Server/runtime config is never tracked; only explicit modpack-default configs
|
||||
# (e.g. the FTB Chunks client config) are shipped.
|
||||
config/*
|
||||
!config/ftbchunks-client.snbt
|
||||
# Server/runtime config is never tracked.
|
||||
config/
|
||||
kubejs/
|
||||
defaultconfigs/
|
||||
local/
|
||||
|
||||
# Authoring/review material - belongs in the repo, never in the pack
|
||||
FINDINGS.md
|
||||
MODLIST-*.md
|
||||
analysis/
|
||||
phase0/
|
||||
ftbbackups3/
|
||||
patchouli_books/
|
||||
.sable/
|
||||
@@ -31,3 +35,8 @@ tools/
|
||||
|
||||
# C2ME runtime worldgen cache (regenerated at runtime)
|
||||
cache/
|
||||
|
||||
# Scratch artifacts from the map-migration tooling - never ship these
|
||||
sp-world/
|
||||
*.tar.gz
|
||||
*.tgz
|
||||
@@ -0,0 +1,139 @@
|
||||
# FTB Chunks -> Open Parties and Claims + Xaero's maps
|
||||
|
||||
**Status:** implemented in the pack (working tree, uncommitted) - 2026-09-30
|
||||
**Decision:** replace FTB Chunks with **Open Parties and Claims (OPAC)** for claims/forceloading and
|
||||
**Xaero's World Map + Minimap** for mapping. FTB Teams, FTB Quests, FTB Library, FTB XMod Compat and
|
||||
FTB Backups 3 all stay.
|
||||
**Accepted cost:** every player re-claims their land and re-creates their waypoints.
|
||||
|
||||
Confidence tags follow `FINDINGS.md`: **VERIFIED** / **DOCUMENTED** / **COMMUNITY** / **INFERENCE**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this works at all
|
||||
|
||||
The pack's earlier review (`MODLIST-REVIEW.md` §8) recommended keeping FTB Chunks on the grounds that
|
||||
"wholesale replacement breaks FTB Quests/Teams". That is only true of a *wholesale* FTB removal.
|
||||
**VERIFIED:** FTB Quests' build declares `ftb-library` + `ftb-teams` as required and `ftb-xmod-compat`
|
||||
as optional - **not** FTB Chunks. FTB Quests also ships **no** claim-chunk task type. So FTB Chunks is
|
||||
the one mod in the FTB suite that can leave without touching quests, provided Teams stays.
|
||||
|
||||
**VERIFIED:** nothing else in the pack declares FTB Chunks as a required dependency. The only
|
||||
cross-mod hooks are inside FTB XMod Compat (KubeJS/GameStages progression, rank-based claim limits),
|
||||
which degrade to inert modules.
|
||||
|
||||
## 2. What changed
|
||||
|
||||
| Action | Path |
|
||||
| --- | --- |
|
||||
| Removed | `mods/ftb-chunks-forge.pw.toml` (was `ftb-chunks-neoforge-2101.1.22`, CF 314906) |
|
||||
| Removed | `config/ftbchunks-client.snbt` (the `minimap.enabled=true` modpack default) |
|
||||
| Edited | `.packwizignore` - dropped `config/*` + `!config/ftbchunks-client.snbt`, back to `config/` |
|
||||
| Added | `mods/open-parties-and-claims.pw.toml` - `neoforge-1.21.1-0.31.6` (`side = "both"`) |
|
||||
| Added | `mods/xaeros-world-map.pw.toml` - `neoforge-1.21.1-1.46.0` (`side = "both"`) |
|
||||
| Added | `mods/xaeros-minimap.pw.toml` - `neoforge-1.21.1-26.5.0` (`side = "both"`) |
|
||||
| Added | `mods/xaeros-minimap-world-map-waystones-compatibility-forge.pw.toml` - `2.1.0` |
|
||||
| Regenerated | `index.toml` (147 entries) and the `[index]` hash in `pack.toml` |
|
||||
|
||||
Net: **146 mods** (138 Modrinth/other + 8 CurseForge), down from 9 CurseForge-exclusive mods. All four
|
||||
new mods are Modrinth-hosted, so they are checkable by `packwiz update`.
|
||||
|
||||
The three Xaero's entries are byte-identical to the ones deleted in `b0542f9` (2026-09-28) - same
|
||||
versions, same sha512s - so this is a revert of the map half of that commit, not a new integration.
|
||||
|
||||
## 3. Where the pack traded one thing for another
|
||||
|
||||
| FTB Chunks capability | Replacement | Notes |
|
||||
| --- | --- | --- |
|
||||
| Chunk claims + protection | OPAC | Server config now lives in the world save, not the pack. |
|
||||
| Forceload + offline ticking | OPAC `claims.forceload.offlineForceload` | Farms-vs-TPS trade is unchanged; reliability is less proven (see §5). |
|
||||
| Minimap + fullscreen map | Xaero's Minimap + World Map | Now two client mods instead of one. |
|
||||
| Waypoints + teleport | Xaero's Minimap | **No migration path** - see §4. |
|
||||
| Party/team claims | OPAC `primaryPartySystem` = FTB Teams | Must be set, or OPAC runs its own parallel party system. |
|
||||
| Waystones on the map | `xaeros_waystones_compatibility` | Restored; it had been removed with the maps. |
|
||||
| FTB Ranks permission overrides | `xaero.pac_max_claims` / `xaero.pac_max_forceloads` | Moot in practice: FTB Ranks was never installed. |
|
||||
|
||||
## 4. Migration runbook (server operator)
|
||||
|
||||
**Before the update**
|
||||
1. Take a world backup. FTB Chunks' claim data stays in the save after the mod is removed, so this
|
||||
backup is the rollback path.
|
||||
2. Have each player screenshot or export their FTB Chunks waypoints. They live client-side under
|
||||
`local/ftbchunks/`; **there is no FTB Chunks -> Xaero's converter** (only JourneyMap -> Xaero tools
|
||||
exist). Re-entry is manual.
|
||||
3. Capture who owned which claims/forceloads, so players have a checklist and an admin can pre-claim
|
||||
using OPAC server-claim mode if desired.
|
||||
|
||||
**Cutover (one update, low-activity window)**
|
||||
4. Do **not** run FTB Chunks and OPAC side by side - two claim systems means split protection logic.
|
||||
Hard cutover.
|
||||
5. Start the server once with OPAC installed so it generates
|
||||
`world/serverconfig/openpartiesandclaims-server.toml`, then **stop the server**.
|
||||
6. Edit the generated file (it carries `# Default:` comments for every key) and set at minimum:
|
||||
- `maxPlayerClaims = 500` (matches the old FTB Chunks cap)
|
||||
- `maxPlayerClaimForceloads = 25` (**decided 2026-09-30**; the stock default is lower, so leaving it
|
||||
alone silently removes forceloading)
|
||||
- `primaryPartySystem` -> FTB Teams
|
||||
- a decision on `claims.forceload.offlineForceload` (farms while offline vs TPS)
|
||||
- add `claims.forceload.offlineForceload` to `playerConfigurablePlayerConfigOptions` /
|
||||
`opConfigurablePlayerConfigOptions` if players should be able to toggle their own
|
||||
7. Re-claim: each player re-claims their base and re-forceloads, via the OPAC UI (`'`) or the Xaero's
|
||||
world map (right-click / drag-select).
|
||||
|
||||
**After**
|
||||
8. Verify per-player forceloading actually keeps chunks ticking with the owner offline. This is the
|
||||
known weak point (§5).
|
||||
9. Verify protection across the modded stack - especially Create contraptions, Sable ships, and
|
||||
fake-player automation.
|
||||
10. Re-tier any rank/permission limits you were relying on.
|
||||
|
||||
## 5. Risks and open items
|
||||
|
||||
- **OPAC force-loading reliability is the one thing that can invalidate the swap.** Open upstream
|
||||
issues report chunks marked forceloadable but never actually loaded (#855, #927, plus #237/#528).
|
||||
**COMMUNITY.** Test before trusting; re-test on every OPAC bump. If it fails, this swap should be
|
||||
rolled back - forceloading is the one feature the pack cannot work around.
|
||||
- **OPAC is self-described beta** and releases frequently (0.31.6, 2026-09-10). **DOCUMENTED.**
|
||||
- **OPAC and Xaero's are version-coupled** - OPAC's README tells users to keep the Xaero's mods on
|
||||
their latest versions. Bump them together.
|
||||
- **Claim protection parity is unverified.** OPAC protects vanilla plus a supported-mod list (Create is
|
||||
built in on Forge), and needs manual exception entries for anything else. Open reports exist for
|
||||
creeper explosions (#731) and modded projectiles (#758) inside claims. **COMMUNITY.** The pack's
|
||||
Apotheosis/Sophisticated/Ars Nouveau/Alex's Mobs surface needs a pass.
|
||||
- **Sable / Create Aeronautics.** A crash with Sable contraptions was reported (#873) and is **closed**,
|
||||
but was not re-verified against 0.31.6. **COMMUNITY.** Ships are not covered by plain chunk claims
|
||||
under either system; the pack has no ship-claim mod. Options if that matters:
|
||||
`aeroclaims` ("Create Aeronautics: Claims", NeoForge 1.21.1, works with **OPAC or FTB**), or
|
||||
`sable-protect`. Note that dropping FTB Chunks does close the door on `ftbchunksaerospace` and
|
||||
`Create-Aeronautics-FTB-Chunks`.
|
||||
- **Licensing.** OPAC is **LGPL-3.0** - no pack conditions. Xaero's World Map and Minimap are
|
||||
**All Rights Reserved**: because this pack is distributed from `git.ljones.me` and not from
|
||||
CurseForge/Modrinth, the terms require an **easily accessible credit link to an official Xaero page**
|
||||
in the pack. Add one. **VERIFIED** (from the mods' own "Modpacks and Other" sections).
|
||||
- **`side = "both"` for Xaero's** puts them on the server too. That buys reliable multi-world map
|
||||
identity and server-side map profiles, and matches the pre-`b0542f9` configuration, but the server
|
||||
then registers status effects that stop *vanilla* clients joining unless disabled in
|
||||
`config/xaero/*`. All players get mods via packwiz, so this is not expected to matter.
|
||||
- **Vanilla-client / optional-client is actually improved** for claims: OPAC's client side is optional,
|
||||
so a player without it is still protected and can use `/oclaims` chat commands.
|
||||
|
||||
## 6. Rollback
|
||||
|
||||
Re-add `mods/ftb-chunks-forge.pw.toml` (and the client config if the minimap default is wanted) and
|
||||
`packwiz refresh`. FTB Chunks' claim/forceload data is untouched in the world save, so existing claims
|
||||
resume. Then remove OPAC and the Xaero's mods. Keep the pre-swap world backup regardless.
|
||||
|
||||
## 7. Docs that need follow-up
|
||||
|
||||
- **Map/waypoint rescue from the old FTB data** - see `analysis/FTB-XAERO-MAP-MIGRATION.md` and the
|
||||
`tools/ftbchunks-reveal.py` + `tools/prune-world-to-reveal.py` scripts. Mask extraction is validated;
|
||||
the world-save pruning step is written and unit-tested but not yet run.
|
||||
|
||||
- `FINDINGS.md` - §3.3, §7.1, §8 rows 2-3, appendix A and appendix B updated in this change; §1's
|
||||
mod-count statistics were left at their earlier snapshot (a note was added).
|
||||
- `MODLIST-REVIEW.md` §7.4 and §8 still say Open Parties and Claims is a SKIP/duplication and "Keep
|
||||
FTB". **Stale** - the swap is now the intended design.
|
||||
- `MODLIST-GROUPS.md` / `analysis/master-table.md` / `analysis/groups-table.md` carry the same stale
|
||||
OPAC verdict.
|
||||
- Not done: re-adding `xaero-train-map` (was `Ov2r3aVY`, 1.0.0). It is unblocked again by this change and
|
||||
the pack has Create trains; one-line addition if wanted.
|
||||
@@ -0,0 +1,214 @@
|
||||
# FTB Chunks reveal -> Xaero's World Map: what transfers and how
|
||||
|
||||
**Status:** mask extraction implemented and validated against real data; world-save pruning implemented
|
||||
and unit-tested; final step blocked on a copy of the server world save.
|
||||
**Goal:** give each player a native Xaero's map covering, as closely as possible, exactly the terrain
|
||||
they had revealed in FTB Chunks.
|
||||
Confidence tags as in `FINDINGS.md`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why we do not convert the pixels
|
||||
|
||||
FTB Chunks does **not** store a rendered map image. Verified by decompiling the pack's exact build,
|
||||
`ftb-chunks-neoforge-2101.1.22.jar` (class strings in `MapManager`, `MapRegionData`, `MapDimension`,
|
||||
`MapChunk`):
|
||||
|
||||
```
|
||||
<game>/local/ftbchunks/data/<serverUUID>/
|
||||
├── dimensions.txt, block_map.txt, biome_map.txt
|
||||
└── <dimension>/
|
||||
└── <XXXXX-XXXXX>.zip # region = 512x512 blocks; name = region coord + 60000, %05X-%05X
|
||||
├── chunks.dat # the chunks this player has actually seen
|
||||
├── data.png # NOT viewable: (height << 16) | (water|light|biome) per pixel
|
||||
├── foliage.png # biome-tinted foliage RGB
|
||||
├── grass.png # biome-tinted grass RGB
|
||||
├── water.png # biome-tinted water RGB
|
||||
└── blocks.png # FTB's internal 24-bit block colour index
|
||||
```
|
||||
|
||||
The colour a player actually saw is computed at runtime by combining `blocks.png`'s index with FTB's own
|
||||
`BlockColor` model via `block_map.txt`. Reproducing it means reimplementing FTB Chunks' renderer, and the
|
||||
result would look like FTB's map - i.e. worse than native Xaero's. So we take the **reveal footprint**
|
||||
from FTB Chunks and the **pixels** from Xaero's.
|
||||
|
||||
## 2. Why we also do not write Xaero region files
|
||||
|
||||
The current Xaero's region format is not the format the community tools understand. Comparing a real
|
||||
`region.xaero` from Xaero's World Map 1.46.0 (extracted from the client instance) with the format
|
||||
`JMtoXaero/RegionToXaero` writes:
|
||||
|
||||
| | old format (RegionToXaero) | real 1.46.0 |
|
||||
| --- | --- | --- |
|
||||
| header | `FF` then int 4 | `FF 00 06 00 08` + varying byte (format 6) |
|
||||
| pixel payload | 4-byte parameters (+ optional state / colour / biome) | embeds NBT compounds |
|
||||
| size for region `0,0` | ~25 KB | **355 KB** |
|
||||
|
||||
The 1.46.0 payload contains literal NBT - e.g. `0A 00 00 08 00 04 "Name" 00 10 "minecraft:gravel"` - so
|
||||
each pixel now carries a block descriptor. This matches `JMtoXaero`'s own admission that its region
|
||||
**merger** broke once Xaero changed the format. Writing this format is a live reverse-engineering project
|
||||
against a closed, obfuscated, moving target. Not worth it.
|
||||
|
||||
**Therefore: drive Xaero's own generator, and control what it can see.**
|
||||
|
||||
## 3. The mechanism
|
||||
|
||||
Both mods use the same geometry - a region is 512x512 blocks = 32x32 chunks - so an FTB region maps 1:1
|
||||
onto one Anvil region file, and an FTB within-region chunk `(x,z)` maps 1:1 onto Anvil slot
|
||||
`x + z*32`. That makes the transplant exact rather than approximate:
|
||||
|
||||
1. Extract each player's reveal set from their `chunks.dat` files.
|
||||
2. Prune a copy of the world save to exactly those chunks (Anvil header/sector surgery; chunk payloads
|
||||
copied byte-for-byte, no NBT re-encoding).
|
||||
3. The player opens the pruned save in **singleplayer** with the pack; Xaero's maps it directly from the
|
||||
save, so absent chunks simply never appear on the map.
|
||||
4. Copy the generated instance
|
||||
(`xaero/world-map/World/null/<map>/`) into the server's folder
|
||||
(`xaero/world-map/Multiplayer_advilplug.top/<DIM>/`).
|
||||
|
||||
Result: **per-chunk-exact reveal extent**, rendered natively by Xaero's. Residual differences are only
|
||||
the colour palette/shading (Xaero's, not FTB's) - which is the whole point.
|
||||
|
||||
## 4. What the real data says
|
||||
|
||||
Measured on the supplied archive - **Stirwick**, `minecraft:overworld`:
|
||||
|
||||
| Metric | Value |
|
||||
| --- | --- |
|
||||
| Region files | 167 |
|
||||
| Revealed chunks | **98,170** (of 171,008 in the bounding box) |
|
||||
| Region extent | x `-9..9`, z `-14..7` |
|
||||
| Block extent | x `-4608..5120`, z `-7168..4096` |
|
||||
| Fully revealed regions (`1024/1024`) | 10 |
|
||||
| Median region fill | 560 / 1024 (~55%) |
|
||||
| Sparse regions (`<64` chunks) | 10 |
|
||||
| `chunks.dat` vs `data.png` disagreements | **0** |
|
||||
|
||||
Two consequences:
|
||||
|
||||
- **`chunks.dat` alone is a trustworthy mask.** It was cross-checked against the `data.png` height
|
||||
sentinel (`HeightUtils.UNKNOWN = Short.MIN_VALUE + 1 = -32767`) for all 167 regions. Zero
|
||||
disagreements. Suspicion that claimed-but-never-visited chunks would inflate the set did not
|
||||
materialise.
|
||||
- **Region-granularity pruning is not good enough.** The median region is only ~55% explored, and 10
|
||||
regions are under 6%. Copying whole `.mca` files a player touched would reveal roughly twice the
|
||||
terrain they had seen. Chunk-level pruning is worth the extra ~40 lines.
|
||||
|
||||
`data.png` decoding detail, for anyone revisiting this: FTB writes `(height << 16) | wlb` into a
|
||||
`TYPE_INT_ARGB` image, so after PNG round-trip the 16-bit height is `(A << 8) | R` and the
|
||||
water/light/biome word is `(G << 8) | B`. Unrevealed pixels are exactly `A=0x80, R=0x01`.
|
||||
|
||||
## 5. Tooling
|
||||
|
||||
Both scripts are stdlib-only and live in `tools/` (which `.packwizignore` excludes, so they never ship).
|
||||
|
||||
`tools/migrate-player.sh` - the driver, does steps 1-2 and prints the manual remainder:
|
||||
|
||||
```
|
||||
tools/migrate-player.sh Stirwick <ftb-data-or-archive> <world-dir> \
|
||||
--dimension minecraft:overworld --work /var/tmp/xaero-migration
|
||||
```
|
||||
|
||||
`tools/ftbchunks-reveal.py` - **validated against Stirwick's real 167-region archive.**
|
||||
Emits a JSON mask of global chunk coordinates per dimension. Pass `--dimension` when the input is a
|
||||
bare directory/zip of region files, which cannot name its own dimension.
|
||||
|
||||
```
|
||||
tools/ftbchunks-reveal.py <ftb-data-or-archive> --out mask.json --dimension minecraft:overworld --verify
|
||||
```
|
||||
|
||||
`tools/filter-xaero-regions.py` - post-harvest cleanup. Compares the Xaero's map folder against the
|
||||
mask and removes regions the player never revealed (i.e. terrain Minecraft regenerated around the
|
||||
login position). File-name-only, so no Xaero format knowledge needed; reports before deleting.
|
||||
|
||||
```
|
||||
tools/filter-xaero-regions.py <game>/xaero/world-map/Multiplayer_advilplug.top/null/ mask.json
|
||||
```
|
||||
|
||||
`tools/prune-world-to-reveal.py` - **validated end-to-end against a real region file from the server**
|
||||
(`world/region/r.1.-2.mca`, 15.7 MB):
|
||||
|
||||
| Check | Result |
|
||||
| --- | --- |
|
||||
| Anvil parse (1024 slots, all zlib, 2.5-21.8 KB payloads) | OK |
|
||||
| Slot -> chunk mapping (`slot = x + z*32`), confirmed by reading `xPos`/`zPos` NBT | **1024 / 1024 exact** |
|
||||
| Prune against that player's FTB mask (52 of 1024 chunks revealed) | 52 kept / 972 dropped |
|
||||
| Output sector-aligned | yes |
|
||||
| Kept slot set == mask | yes |
|
||||
| Kept payloads + timestamps byte-identical | yes |
|
||||
| Kept chunks re-parse with correct coordinates | 52 / 52 |
|
||||
| Size | 15,745,024 -> 471,040 bytes (3.0%) |
|
||||
|
||||
Earlier synthetic tests also cover nested per-dimension `region`/`entities`/`poi` trees not being
|
||||
duplicated.
|
||||
|
||||
```
|
||||
tools/prune-world-to-reveal.py <world-copy> mask.json <out-dir> --dimension minecraft:overworld
|
||||
```
|
||||
|
||||
`--minimal` writes only `level.dat` plus the pruned `region/` tree - the smallest thing Xaero's can
|
||||
open and map, and the best fit when the full save is too large to copy around. `--dry-run` reports
|
||||
kept/dropped counts per region without writing anything. Runtime is ~0.15 s per 15 MB region.
|
||||
|
||||
## 6. Running it where the save lives
|
||||
|
||||
The tools are stdlib-only Python, so nothing has to be transferred: run them on the machine that holds
|
||||
the world. The live world is **not** on the VPS (the VPS is only a proxy in front of the real server),
|
||||
so this has to run where the save actually lives. The per-player footprint is only the pruned `region/`
|
||||
tree for the regions that player revealed, not the whole save.
|
||||
|
||||
```bash
|
||||
tools/migrate-player.sh Stirwick <stirwick-ftbchunks-data> <world-dir> \
|
||||
--dimension minecraft:overworld --work /var/tmp/xaero-migration
|
||||
```
|
||||
|
||||
That extracts the mask (with the `chunks.dat` vs `data.png` cross-check) and prunes the world,
|
||||
dry-run first. Then, by hand:
|
||||
|
||||
3. Copy the pruned world (`<work>/<player>/sp-world`) to a machine that can run the pack and open it in
|
||||
**singleplayer** once. Let Xaero's map it, then quit.
|
||||
4. Move the generated instance, `xaero/world-map/World/null/<map>/`, into the server's map folder,
|
||||
`xaero/world-map/Multiplayer_advilplug.top/<DIM>/`.
|
||||
5. Run `tools/filter-xaero-regions.py` on that folder to drop regions the player never revealed.
|
||||
6. Delete the scratch directory - usually a few hundred MB.
|
||||
|
||||
### Two things that will bite
|
||||
|
||||
- **The server does not need stopping, but freeze saves while it runs.** The pruner only reads the world
|
||||
and writes to a separate output directory. However a running server rewrites region files on autosave
|
||||
and rewrites `level.dat` in place, so a read can tear - and half a `level.dat` is an unopenable
|
||||
singleplayer world. From the console: `save-off` -> dry run -> real run -> `save-all` -> `save-on`.
|
||||
The pass takes a couple of minutes. Never forget `save-on`, and never point `--out` at the live world.
|
||||
A region that does fail to read is reported, skipped, and left unwritten; re-run to retry it.
|
||||
|
||||
- **Opening the world regenerates chunks near the login position.** Minecraft loads and generates the
|
||||
chunks around wherever the player spawns, so terrain the player never revealed can reappear on the
|
||||
map if it happens to sit near world spawn. It is bounded by render distance (well under one
|
||||
512x512 region at typical settings), and it is easy to undo: step 4's cleanup compares the harvested
|
||||
region file names against the mask and deletes any region the player never revealed. Keeping
|
||||
`level.dat`'s spawn inside the revealed area, or not walking around in step 3, avoids the rest.
|
||||
- **Do not copy the world with hardlinks.** `cp -al` would make the singleplayer client's own writes to
|
||||
`level.dat` land in the live server save. A real copy, or `--minimal` into a fresh directory, is safe.
|
||||
|
||||
## 7. Open items
|
||||
|
||||
- **Needs: the full world save, read in place on the server.** One real region file (`r.1.-2.mca`) has
|
||||
now validated the reader, the slot mapping and the writer, so the remaining unknowns are operational
|
||||
rather than format-related: total runtime over 167 regions (extrapolating ~0.15 s per 15 MB region,
|
||||
a couple of minutes) and the size of the pruned output. A full 1024-chunk region on this server is
|
||||
~15 MB; the player revealed 57.4% of the bounding box overall, so expect the pruned per-player world
|
||||
to land in the low hundreds of MB. It is scratch data and can be deleted once the Xaero's map has
|
||||
been harvested.
|
||||
- **Per-player, per-dimension.** Only `minecraft:overworld` was supplied and only one player's data.
|
||||
Modded dimensions will each need an FTB-data -> Xaero-dimension mapping. Xaero names its dimension
|
||||
folders `null`/`DIM-1`/`DIM1` for vanilla and uses the `minecraft$overworld` form (`$`, not `:`) in
|
||||
`dimension_config.txt`; a `dimensions.txt` with more than one entry will confirm the rest.
|
||||
- **Step 3 costs one client launch per player.** It cannot be done server-side; Xaero's maps from a save
|
||||
on the client. ~10 launches for this pack.
|
||||
- **Waypoints are a separate, easier win.** FTB stores them in `waypoints.json` next to the region data
|
||||
(empty, 16 bytes, for the sampled player). Xaero waypoints are plain files under `xaero/minimap/`,
|
||||
so a converter is straightforward - worth doing at the same time if anyone has waypoints to keep.
|
||||
- **Fallbacks if the save never materialises:** region-granularity pruning (coarse, ~2x over-reveal), or
|
||||
shipping the full "Convert All Regions" map to everyone (over-reveals everything, zero code).
|
||||
- Nothing here is time-sensitive: removing the mod from the pack does not touch `local/ftbchunks/`, so
|
||||
every client's reveal data is still on disk and this remains doable later.
|
||||
@@ -1,16 +0,0 @@
|
||||
# Onion MC modpack default.
|
||||
#
|
||||
# FTB Chunks computes "minimap.enabled" once as !hasOtherMinimapMod() and then
|
||||
# persists the result. While Xaero's Minimap was in this pack that resolved to
|
||||
# false and was written into players' configs, where it sticks -- removing the
|
||||
# other minimap mod later does not re-enable it. Setting it explicitly here
|
||||
# turns the FTB Chunks minimap on for everyone, including players whose config
|
||||
# was already written with false.
|
||||
#
|
||||
# Players who want a different value can copy this to
|
||||
# <instance>/local/ftbchunks-client.snbt, which is not overwritten on pack update.
|
||||
{
|
||||
minimap: {
|
||||
enabled: true
|
||||
}
|
||||
}
|
||||
+20
-9
@@ -1,9 +1,5 @@
|
||||
hash-format = "sha256"
|
||||
|
||||
[[files]]
|
||||
file = "config/ftbchunks-client.snbt"
|
||||
hash = "6cbe2d3e5419a9ab0ded6f8d70f9be4a2cfb25f7a1d75c6a93fb4c1ed9e2329b"
|
||||
|
||||
[[files]]
|
||||
file = "mods/3dskinlayers.pw.toml"
|
||||
hash = "b3b67f83faf9dcc4985b7a13fa8affb71062461d271f80cc0958652df9f3126b"
|
||||
@@ -304,11 +300,6 @@ file = "mods/ftb-backups-3.pw.toml"
|
||||
hash = "893f9b4dbe36a0bcae29d87cd68ab84fed83c984c3f5649c155c0522c509a77d"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/ftb-chunks-forge.pw.toml"
|
||||
hash = "670df7985eec05c6a339d90ebbcfda0e5f1f5c63846ac65a441412f123471e14"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/ftb-library-forge.pw.toml"
|
||||
hash = "2e72d40c6d7b128981b19b71c19d45eb6813f1af87ae1abe1cf92adfa5e8223f"
|
||||
@@ -499,6 +490,11 @@ file = "mods/oceans-delight.pw.toml"
|
||||
hash = "7bf7c8369bd28212a1e294c60310ccb72f1eb773630970e6ab106ae666766cd3"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/open-parties-and-claims.pw.toml"
|
||||
hash = "97660396e7f075f4c9ca8447878ec3da58ebea9ed464cdb494d4db1d62989936"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/patchouli.pw.toml"
|
||||
hash = "d7f360ebb9f009d6dc0b368cd8fa399bdff70c0f6ef931687b2a35137f72715d"
|
||||
@@ -679,6 +675,21 @@ file = "mods/wings-of-fire.pw.toml"
|
||||
hash = "07b44e8f4c3e5dd41c13269005638e27d1fcf43e8fc96b2f08299ca3676313e3"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/xaeros-minimap-world-map-waystones-compatibility-forge.pw.toml"
|
||||
hash = "637bc567eb5260d55ce89e6471c31e4369d454607f3d6f40139f7f7bdef90af9"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/xaeros-minimap.pw.toml"
|
||||
hash = "64670072fa3b4db4a721769dd3d66a2865daac01c9a2480b9561675b1e9e9737"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/xaeros-world-map.pw.toml"
|
||||
hash = "f8cc300d53ae8bf71d77124e6c46a5e85fe31654b39c5d73a1652a4d9c404af4"
|
||||
metafile = true
|
||||
|
||||
[[files]]
|
||||
file = "mods/yacl.pw.toml"
|
||||
hash = "539f362e9e192cf3b2813e0dfdfc51a5e6814129c6e82a9733842529beea6242"
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
name = "FTB Chunks (NeoForge)"
|
||||
filename = "ftb-chunks-neoforge-2101.1.22.jar"
|
||||
side = "both"
|
||||
|
||||
[download]
|
||||
hash-format = "sha1"
|
||||
hash = "a53f1bb7965c6954a348d4f321f03d3971ad503f"
|
||||
mode = "metadata:curseforge"
|
||||
|
||||
[update]
|
||||
[update.curseforge]
|
||||
file-id = 8791113
|
||||
project-id = 314906
|
||||
@@ -0,0 +1,13 @@
|
||||
name = "Open Parties and Claims"
|
||||
filename = "open-parties-and-claims-neoforge-1.21.1-0.31.6.jar"
|
||||
side = "both"
|
||||
|
||||
[download]
|
||||
url = "https://cdn.modrinth.com/data/gF3BGWvG/versions/3i48dTzz/open-parties-and-claims-neoforge-1.21.1-0.31.6.jar"
|
||||
hash-format = "sha512"
|
||||
hash = "6f8f2d570b6f674a9180576524b3ba507a3ad046ff319353cee50738644bf46c03ca460fb7c6880eed76f07eb2a26a983166d2510951202af86b3e120b743181"
|
||||
|
||||
[update]
|
||||
[update.modrinth]
|
||||
mod-id = "gF3BGWvG"
|
||||
version = "3i48dTzz"
|
||||
@@ -0,0 +1,13 @@
|
||||
name = "Xaero's Minimap & World Map - Waystones Compatibility"
|
||||
filename = "xaeros_waystones_compatibility-NeoForge-1.21.1-2.1.0.jar"
|
||||
side = "both"
|
||||
|
||||
[download]
|
||||
url = "https://cdn.modrinth.com/data/7mLhyqIY/versions/k8wIAiWT/xaeros_waystones_compatibility-NeoForge-1.21.1-2.1.0.jar"
|
||||
hash-format = "sha512"
|
||||
hash = "440da659b1c406ff410841248a28adba60b73f3d91f513ba335b9e5d44b3e48ab63d5ecc86484ff1e3d1a4f7830ad7566fc1efec2a03eba3c958b414b61feea0"
|
||||
|
||||
[update]
|
||||
[update.modrinth]
|
||||
mod-id = "7mLhyqIY"
|
||||
version = "k8wIAiWT"
|
||||
@@ -0,0 +1,13 @@
|
||||
name = "Xaero's Minimap"
|
||||
filename = "xaerominimap-neoforge-1.21.1-26.5.0.jar"
|
||||
side = "both"
|
||||
|
||||
[download]
|
||||
url = "https://cdn.modrinth.com/data/1bokaNcj/versions/Q1tuMQBB/xaerominimap-neoforge-1.21.1-26.5.0.jar"
|
||||
hash-format = "sha512"
|
||||
hash = "0fde1066dd922ac0796d166ba1bf1b843873b8d8563da3a08777df54bb377ff6466096650c730ad1b96e0aecf2d8974cd87c32165edf6a30f6653840c3f99abf"
|
||||
|
||||
[update]
|
||||
[update.modrinth]
|
||||
mod-id = "1bokaNcj"
|
||||
version = "Q1tuMQBB"
|
||||
@@ -0,0 +1,13 @@
|
||||
name = "Xaero's World Map"
|
||||
filename = "xaeroworldmap-neoforge-1.21.1-1.46.0.jar"
|
||||
side = "both"
|
||||
|
||||
[download]
|
||||
url = "https://cdn.modrinth.com/data/NcUtCpym/versions/9Ckiihkz/xaeroworldmap-neoforge-1.21.1-1.46.0.jar"
|
||||
hash-format = "sha512"
|
||||
hash = "fb14d11d66e0a6441981b18454c86534d8f534cd159ca19fa612950521788d858f37f18fe2093bb9b43ca0a88ae9af578eced5d140ff645a7681d7e452e8a268"
|
||||
|
||||
[update]
|
||||
[update.modrinth]
|
||||
mod-id = "NcUtCpym"
|
||||
version = "9Ckiihkz"
|
||||
@@ -6,7 +6,7 @@ pack-format = "packwiz:1.1.0"
|
||||
[index]
|
||||
file = "index.toml"
|
||||
hash-format = "sha256"
|
||||
hash = "9fb9f6269363cbdef4f9635f168b71387b8f437901f775e09d639a6c40a74c24"
|
||||
hash = "80220fc3f737beda7794679fb8a18b70b2b68c5cd7f88b479bc0380e7311bb64"
|
||||
|
||||
[versions]
|
||||
minecraft = "1.21.1"
|
||||
|
||||
Executable
+88
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Drop Xaero's map regions that a player never actually revealed.
|
||||
|
||||
After a player opens their pruned singleplayer world, Xaero's maps it - but Minecraft
|
||||
also *regenerates* the chunks around wherever that player logged in, so a little
|
||||
terrain they had never seen can sneak onto the map. This compares the harvested map
|
||||
against the reveal mask and removes the strays.
|
||||
|
||||
It works on file names only (`<rx>_<rz>.zip` is one 512x512-block region, exactly the
|
||||
same geometry as an FTB Chunks region file), so it needs no knowledge of Xaero's
|
||||
region format.
|
||||
|
||||
Usage:
|
||||
filter-xaero-regions.py <xaero-map-dir> <mask.json> [--dimension <id>] [--delete]
|
||||
|
||||
Without --delete it only reports. Run it on one dimension folder at a time, e.g.
|
||||
~/.minecraft/xaero/world-map/Multiplayer_advilplug.top/null/
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
REGION_RE = re.compile(r"^(-?\d+)_(-?\d+)\.zip$")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("map_dir", help="Xaero's map instance directory holding <rx>_<rz>.zip")
|
||||
ap.add_argument("mask", help="mask JSON from ftbchunks-reveal.py")
|
||||
ap.add_argument("--dimension", default=None,
|
||||
help="dimension to check (default: the mask's only dimension)")
|
||||
ap.add_argument("--delete", action="store_true", help="actually remove the strays")
|
||||
args = ap.parse_args()
|
||||
|
||||
with open(args.mask) as fh:
|
||||
mask = json.load(fh)
|
||||
dims = mask["dimensions"]
|
||||
dim = args.dimension
|
||||
if dim is None:
|
||||
if len(dims) != 1:
|
||||
print(f"mask has {len(dims)} dimensions; pass --dimension", file=sys.stderr)
|
||||
return 2
|
||||
dim = next(iter(dims))
|
||||
if dim not in dims:
|
||||
print(f"no dimension '{dim}' in mask; have {list(dims)}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
revealed = {tuple(int(v) for v in key.split(",")) for key in dims[dim]["regions"]}
|
||||
|
||||
if not os.path.isdir(args.map_dir):
|
||||
print(f"not a directory: {args.map_dir}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
present, strays = [], []
|
||||
for name in sorted(os.listdir(args.map_dir)):
|
||||
m = REGION_RE.match(name)
|
||||
if not m:
|
||||
continue
|
||||
coords = (int(m.group(1)), int(m.group(2)))
|
||||
(present if coords in revealed else strays).append((name, coords))
|
||||
|
||||
print(f"dimension {dim}")
|
||||
print(f" map regions present : {len(present)} (of {len(revealed)} revealed)")
|
||||
print(f" not in the mask : {len(strays)}")
|
||||
for name, _ in strays:
|
||||
print(f" {name}")
|
||||
|
||||
missing = sorted(revealed - {c for _, c in present})
|
||||
if missing:
|
||||
print(f" revealed but absent from the map: {len(missing)} "
|
||||
f"(e.g. {missing[:4]}) - regenerate these, do not delete anything")
|
||||
|
||||
if args.delete and strays:
|
||||
for name, _ in strays:
|
||||
os.remove(os.path.join(args.map_dir, name))
|
||||
print(f"\nremoved {len(strays)} stray region file(s)")
|
||||
elif strays:
|
||||
print("\nre-run with --delete to remove them")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+284
@@ -0,0 +1,284 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract a per-chunk reveal mask from FTB Chunks client map data.
|
||||
|
||||
FTB Chunks stores its explored-terrain map client-side, one zip per 512x512-block
|
||||
region:
|
||||
|
||||
<game>/local/ftbchunks/data/<serverUUID>/<dimension>/<XXXXX-XXXXX>.zip
|
||||
|
||||
Each zip holds `chunks.dat` (the chunks the player has actually seen) plus five
|
||||
PNG data images. Region file names are the region coordinate + 60000, formatted
|
||||
`%05X-%05X`, which is why overworld spawn shows up as `0EA60-0EA60.zip`.
|
||||
|
||||
`chunks.dat` layout (verified against ftb-chunks-neoforge-2101.1.22 bytecode):
|
||||
|
||||
u8 leading byte (0)
|
||||
u8 version (3 on this build)
|
||||
i16 big-endian chunk count
|
||||
per chunk:
|
||||
u8 chunk version (present when version >= 2)
|
||||
u8 x within region (0..31)
|
||||
u8 z within region (0..31)
|
||||
i64 big-endian last-modified millis
|
||||
|
||||
`chunks.dat` was checked against the `data.png` height sentinel (HeightUtils.UNKNOWN
|
||||
= -32767, packed as (height << 16) | (water|light|biome)) across all 167 regions of a
|
||||
real 98,170-chunk dataset: zero disagreements. So chunks.dat alone is a trustworthy
|
||||
reveal mask; --verify re-runs that cross-check anyway.
|
||||
|
||||
Output is a JSON mask consumed by prune-world-to-reveal.py.
|
||||
|
||||
Usage:
|
||||
ftbchunks-reveal.py <input> [--out mask.json] [--dimension <id>] [--verify] [--quiet]
|
||||
|
||||
<input> is either
|
||||
* a client data directory, e.g. .../local/ftbchunks/data/<uuid>/, which is scanned
|
||||
across all dimensions listed in its dimensions.txt, or
|
||||
* a directory or zip containing region zips directly (a single dimension, as when
|
||||
someone has archived just the overworld). Such an input cannot name its own
|
||||
dimension, so pass --dimension to label it.
|
||||
|
||||
Output coordinates are global chunk coordinates, so they are independent of which
|
||||
region file a chunk happened to live in.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
import zipfile
|
||||
import zlib
|
||||
|
||||
REGION_RE = re.compile(r"^([0-9A-Fa-f]{4,6})-([0-9A-Fa-f]{4,6})\.zip$")
|
||||
REGION_OFFSET = 60000
|
||||
|
||||
# HeightUtils.UNKNOWN = Short.MIN_VALUE + 1, stored masked to 16 bits.
|
||||
UNKNOWN_RAW = (-32767) & 0xFFFF
|
||||
|
||||
|
||||
def parse_region_name(name: str) -> tuple[int, int] | None:
|
||||
"""`0EA60-0EA5F.zip` -> region coordinate (0, -1)."""
|
||||
m = REGION_RE.match(os.path.basename(name))
|
||||
if not m:
|
||||
return None
|
||||
return int(m.group(1), 16) - REGION_OFFSET, int(m.group(2), 16) - REGION_OFFSET
|
||||
|
||||
|
||||
def parse_chunks_dat(data: bytes) -> set[tuple[int, int]]:
|
||||
"""Return the within-region (x, z) chunk coordinates present in chunks.dat."""
|
||||
if len(data) < 4:
|
||||
return set()
|
||||
buf = io.BytesIO(data)
|
||||
buf.read(1)
|
||||
version = buf.read(1)[0]
|
||||
count = struct.unpack(">h", buf.read(2))[0]
|
||||
chunks: set[tuple[int, int]] = set()
|
||||
for _ in range(count):
|
||||
if version >= 2:
|
||||
buf.read(1)
|
||||
x = buf.read(1)[0]
|
||||
z = buf.read(1)[0]
|
||||
buf.read(8)
|
||||
chunks.add((x, z))
|
||||
return chunks
|
||||
|
||||
|
||||
# --- optional cross-check: decode data.png and find the UNKNOWN sentinel --------
|
||||
|
||||
def _png_decode(data: bytes) -> tuple[int, int, int, bytes]:
|
||||
"""Minimal PNG decoder (8-bit, non-interlaced) -> (w, h, channels, rgba bytes)."""
|
||||
if data[:8] != b"\x89PNG\r\n\x1a\n":
|
||||
raise ValueError("not a PNG")
|
||||
pos, idat = 8, b""
|
||||
width = height = depth = colour = None
|
||||
while pos < len(data):
|
||||
(length,) = struct.unpack(">I", data[pos : pos + 4])
|
||||
kind = data[pos + 4 : pos + 8]
|
||||
body = data[pos + 8 : pos + 8 + length]
|
||||
pos += 12 + length
|
||||
if kind == b"IHDR":
|
||||
width, height, depth, colour, comp, filt, interlace = struct.unpack(
|
||||
">IIBBBBB", body
|
||||
)
|
||||
if depth != 8 or interlace != 0:
|
||||
raise ValueError(f"unsupported PNG depth/interlace {depth}/{interlace}")
|
||||
elif kind == b"IDAT":
|
||||
idat += body
|
||||
elif kind == b"IEND":
|
||||
break
|
||||
channels = {0: 1, 2: 3, 3: 1, 4: 2, 6: 4}[colour]
|
||||
stride = width * channels
|
||||
raw = zlib.decompress(idat)
|
||||
out = bytearray()
|
||||
prev = bytearray(stride)
|
||||
p = 0
|
||||
for _ in range(height):
|
||||
ftype = raw[p]
|
||||
p += 1
|
||||
line = bytearray(raw[p : p + stride])
|
||||
p += stride
|
||||
if ftype == 1:
|
||||
for i in range(channels, stride):
|
||||
line[i] = (line[i] + line[i - channels]) & 0xFF
|
||||
elif ftype == 2:
|
||||
for i in range(stride):
|
||||
line[i] = (line[i] + prev[i]) & 0xFF
|
||||
elif ftype == 3:
|
||||
for i in range(stride):
|
||||
a = line[i - channels] if i >= channels else 0
|
||||
line[i] = (line[i] + ((a + prev[i]) >> 1)) & 0xFF
|
||||
elif ftype == 4:
|
||||
for i in range(stride):
|
||||
a = line[i - channels] if i >= channels else 0
|
||||
b = prev[i]
|
||||
c = prev[i - channels] if i >= channels else 0
|
||||
pp = a + b - c
|
||||
pa, pb, pc = abs(pp - a), abs(pp - b), abs(pp - c)
|
||||
pred = a if (pa <= pb and pa <= pc) else (b if pb <= pc else c)
|
||||
line[i] = (line[i] + pred) & 0xFF
|
||||
out += line
|
||||
prev = line
|
||||
return width, height, channels, bytes(out)
|
||||
|
||||
|
||||
def chunks_seen_in_png(png_bytes: bytes) -> set[tuple[int, int]]:
|
||||
"""Chunks (0..31)^2 having at least one non-UNKNOWN pixel."""
|
||||
width, height, channels, px = _png_decode(png_bytes)
|
||||
unknown_r = UNKNOWN_RAW & 0xFF
|
||||
unknown_a = (UNKNOWN_RAW >> 8) & 0xFF
|
||||
seen: set[tuple[int, int]] = set()
|
||||
for cz in range(32):
|
||||
for cx in range(32):
|
||||
hit = False
|
||||
for yy in range(16):
|
||||
base = ((cz * 16 + yy) * width + cx * 16) * channels
|
||||
for xx in range(16):
|
||||
i = base + xx * channels
|
||||
if not (px[i] == unknown_r and px[i + 3] == unknown_a):
|
||||
hit = True
|
||||
break
|
||||
if hit:
|
||||
break
|
||||
if hit:
|
||||
seen.add((cx, cz))
|
||||
return seen
|
||||
|
||||
|
||||
# --- input discovery -----------------------------------------------------------
|
||||
|
||||
def iter_region_zips(path: str):
|
||||
"""Yield (dimension_name, region_filename, bytes) from a dir or an archive."""
|
||||
if zipfile.is_zipfile(path):
|
||||
with zipfile.ZipFile(path) as outer:
|
||||
for info in outer.infolist():
|
||||
if parse_region_name(info.filename) is None:
|
||||
continue
|
||||
yield "(unspecified)", os.path.basename(info.filename), outer.read(info)
|
||||
return
|
||||
if os.path.isdir(path) and any(
|
||||
parse_region_name(n) for n in os.listdir(path)
|
||||
):
|
||||
for name in sorted(os.listdir(path)):
|
||||
if parse_region_name(name):
|
||||
with open(os.path.join(path, name), "rb") as fh:
|
||||
yield "(unspecified)", name, fh.read()
|
||||
return
|
||||
# treat as a client data directory keyed by dimension
|
||||
dims_file = os.path.join(path, "dimensions.txt")
|
||||
dims = []
|
||||
if os.path.exists(dims_file):
|
||||
with open(dims_file) as fh:
|
||||
dims = [ln.strip() for ln in fh if ln.strip()]
|
||||
else:
|
||||
dims = [
|
||||
d
|
||||
for d in sorted(os.listdir(path))
|
||||
if os.path.isdir(os.path.join(path, d))
|
||||
]
|
||||
for dim in dims:
|
||||
dim_dir = os.path.join(path, dim)
|
||||
if not os.path.isdir(dim_dir):
|
||||
continue
|
||||
for name in sorted(os.listdir(dim_dir)):
|
||||
if parse_region_name(name):
|
||||
with open(os.path.join(dim_dir, name), "rb") as fh:
|
||||
yield dim, name, fh.read()
|
||||
|
||||
|
||||
def extract(path: str, verify: bool, quiet: bool, dimension_label: str | None = None) -> dict:
|
||||
dims: dict[str, dict] = {}
|
||||
total = 0
|
||||
mismatches = 0
|
||||
for dim, name, blob in iter_region_zips(path):
|
||||
if dim == "(unspecified)" and dimension_label:
|
||||
dim = dimension_label
|
||||
coords = parse_region_name(name)
|
||||
if coords is None:
|
||||
continue
|
||||
rx, rz = coords
|
||||
with zipfile.ZipFile(io.BytesIO(blob)) as region:
|
||||
listed = parse_chunks_dat(region.read("chunks.dat"))
|
||||
if verify:
|
||||
seen = chunks_seen_in_png(region.read("data.png"))
|
||||
if seen != listed:
|
||||
mismatches += 1
|
||||
print(
|
||||
f" ! {dim}/{name}: chunks.dat={len(listed)} png={len(seen)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
entry = dims.setdefault(dim, {"regions": {}, "chunks": []})
|
||||
entry["regions"][f"{rx},{rz}"] = len(listed)
|
||||
for cx, cz in sorted(listed):
|
||||
entry["chunks"].append([rx * 32 + cx, rz * 32 + cz])
|
||||
total += len(listed)
|
||||
if not quiet:
|
||||
print(f" {dim:34} {name:>16} {len(listed):>4} chunks", file=sys.stderr)
|
||||
|
||||
for entry in dims.values():
|
||||
entry["chunks"].sort()
|
||||
entry["region_count"] = len(entry["regions"])
|
||||
entry["chunk_count"] = len(entry["chunks"])
|
||||
return {
|
||||
"source": os.path.abspath(path),
|
||||
"verified": verify,
|
||||
"mismatches": mismatches,
|
||||
"total_chunks": total,
|
||||
"dimensions": dims,
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("input", help="FTB Chunks data dir, or a dir/zip of region zips")
|
||||
ap.add_argument("--out", default="ftb-reveal-mask.json")
|
||||
ap.add_argument("--dimension", default=None,
|
||||
help="dimension id used to label a bare directory/zip of region files, which "
|
||||
"cannot say which dimension it is (e.g. minecraft:overworld)")
|
||||
ap.add_argument("--verify", action="store_true", help="cross-check chunks.dat vs data.png")
|
||||
ap.add_argument("--quiet", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
mask = extract(args.input, args.verify, args.quiet, args.dimension)
|
||||
with open(args.out, "w") as fh:
|
||||
json.dump(mask, fh)
|
||||
|
||||
if args.verify:
|
||||
status = "OK" if mask["mismatches"] == 0 else f"{mask['mismatches']} MISMATCHES"
|
||||
print(f"cross-check chunks.dat vs data.png: {status}", file=sys.stderr)
|
||||
print(
|
||||
f"{mask['total_chunks']:,} revealed chunks in "
|
||||
f"{sum(d['region_count'] for d in mask['dimensions'].values())} regions "
|
||||
f"across {len(mask['dimensions'])} dimension(s) -> {args.out}"
|
||||
)
|
||||
for dim, d in mask["dimensions"].items():
|
||||
print(f" {dim}: {d['chunk_count']:,} chunks / {d['region_count']} regions")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+131
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env bash
|
||||
# End-to-end driver for the FTB Chunks -> Xaero's World Map reveal transplant.
|
||||
#
|
||||
# See analysis/FTB-XAERO-MAP-MIGRATION.md for the reasoning. This script does the
|
||||
# parts that can be automated; the singleplayer pass in step 3 has to be done by hand
|
||||
# on a machine that can run the pack.
|
||||
#
|
||||
# Usage:
|
||||
# migrate-player.sh <player> <ftbchunks-data> <world-dir> [--dimension <id>] [--work <dir>]
|
||||
#
|
||||
# <player> label used for output paths, e.g. Stirwick
|
||||
# <ftbchunks-data> that player's local/ftbchunks/data/<uuid>/, or a bare zip/dir
|
||||
# of region files, or an already-extracted mask .json
|
||||
# <world-dir> the world save containing region/
|
||||
#
|
||||
# Example:
|
||||
# tools/migrate-player.sh Stirwick ~/Stirwick-ftbchunks ~/server/world \
|
||||
# --dimension minecraft:overworld --work /var/tmp/xaero-migration
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
if [ $# -lt 3 ]; then
|
||||
sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'
|
||||
exit 2
|
||||
fi
|
||||
|
||||
player="$1"; ftb="$2"; world="$3"; shift 3
|
||||
dimension="minecraft:overworld"
|
||||
work="$(pwd)/xaero-migration"
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--dimension) dimension="$2"; shift 2 ;;
|
||||
--work) work="$2"; shift 2 ;;
|
||||
*) echo "unknown option: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
out="$work/$player"
|
||||
mask="$out/mask.json"
|
||||
sp="$out/sp-world"
|
||||
|
||||
echo "player : $player"
|
||||
echo "ftb data : $ftb"
|
||||
echo "world : $world"
|
||||
echo "dimension : $dimension"
|
||||
echo "output : $out"
|
||||
echo
|
||||
|
||||
if [ ! -d "$world/region" ]; then
|
||||
echo "error: $world/region not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
mkdir -p "$out"
|
||||
|
||||
echo "== 1/2 extracting reveal mask =="
|
||||
if [ "${ftb##*.}" = "json" ]; then
|
||||
cp -f "$ftb" "$mask"
|
||||
echo " using prebuilt mask: $ftb"
|
||||
python3 - "$mask" <<'PY'
|
||||
import json, sys
|
||||
m = json.load(open(sys.argv[1]))
|
||||
dims = m["dimensions"]
|
||||
print(f" {m['total_chunks']:,} chunks / "
|
||||
f"{sum(d['region_count'] for d in dims.values())} regions / "
|
||||
f"dimension(s) {list(dims)}")
|
||||
PY
|
||||
else
|
||||
python3 "$here/ftbchunks-reveal.py" "$ftb" --out "$mask" --dimension "$dimension" --verify
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "== 2/2 pruning the world to those chunks (dry run first) =="
|
||||
python3 "$here/prune-world-to-reveal.py" "$world" "$mask" "$sp" \
|
||||
--dimension "$dimension" --minimal --dry-run
|
||||
echo
|
||||
python3 "$here/prune-world-to-reveal.py" "$world" "$mask" "$sp" \
|
||||
--dimension "$dimension" --minimal
|
||||
|
||||
cat <<EOF
|
||||
|
||||
Done. Pruned world: $sp
|
||||
|
||||
Now, by hand.
|
||||
|
||||
3. Copy $sp to a machine that can run the pack and open it in SINGLEPLAYER once,
|
||||
as a world folder (the directory holding level.dat and region/):
|
||||
|
||||
<game dir>/saves/Stirwick-reveal/
|
||||
|
||||
Open it, open the map (M), let it fill, then quit. Xaero's maps what is in the
|
||||
save, and this save holds only the revealed chunks. Do not walk around --
|
||||
Minecraft regenerates chunks near the login position and they will show up.
|
||||
|
||||
Where the result lands: look under <game dir>/xaero/world-map/ for a folder that
|
||||
is not Multiplayer_*, e.g. World/null/<map>/. Sort by modification time if
|
||||
unsure -- those files were just written.
|
||||
|
||||
SANITY CHECK: you should end up with one zip per region in the mask
|
||||
($(python3 -c "import json;print(sum(d['region_count'] for d in json.load(open('$mask'))['dimensions'].values()))") for $player). A handful means it only mapped the spawn area and
|
||||
you should use 'Convert All Regions' in the world map settings, or report back.
|
||||
|
||||
4. Clean the HARVEST before anything else -- this is the step that removes terrain
|
||||
Minecraft regenerated around the login point. Run it on the harvested folder,
|
||||
NOT on the player's own map folder:
|
||||
|
||||
python3 $here/filter-xaero-regions.py \\
|
||||
"<game dir>/xaero/world-map/World/null/<map>/" \\
|
||||
$mask --dimension $dimension
|
||||
|
||||
Check the list, then re-run with --delete.
|
||||
|
||||
5. Merge the surviving <rx>_<rz>.zip files into the player's existing map folder,
|
||||
WITHOUT overwriting anything already there (-n), so any terrain they have
|
||||
explored since the swap is kept:
|
||||
|
||||
cp -n <harvested>/*.zip \\
|
||||
"<their game dir>/xaero/world-map/Multiplayer_advilplug.top/null/<their map instance>/"
|
||||
|
||||
The right instance is the mw\$... folder there. If the player has not joined the
|
||||
server since the swap, have them join once first so Xaero's creates it.
|
||||
|
||||
6. Delete $out -- scratch, usually a few hundred MB.
|
||||
|
||||
Per-dimension: repeat with --dimension <id> and the matching map folder
|
||||
(null = overworld, DIM-1 = nether, DIM1 = end).
|
||||
Other players: each needs their own local/ftbchunks/data/<uuid>/; there is no
|
||||
server-side copy of this data.
|
||||
EOF
|
||||
Executable
+210
@@ -0,0 +1,210 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Prune a Minecraft Anvil world save down to a set of revealed chunks.
|
||||
|
||||
This is how we transplant FTB Chunks' reveal footprint onto Xaero's World Map
|
||||
without reverse-engineering Xaero's region format: strip the world save down to
|
||||
exactly the chunks a player had explored, then let Xaero's own generator map that
|
||||
save in singleplayer. Xaero's renders it natively (correct palette, shading,
|
||||
light, biome) and the absent chunks simply never appear on the map.
|
||||
|
||||
Chunk payloads are copied through byte-for-byte -- no NBT re-encoding -- so the
|
||||
output is safe for modded 1.21.1 chunk data. Only the 8 KiB header, the sector
|
||||
offsets and the zero padding are rewritten.
|
||||
|
||||
Usage:
|
||||
prune-world-to-reveal.py <world-save-dir> <mask.json> <out-dir>
|
||||
[--dimension minecraft:overworld] [--dry-run]
|
||||
|
||||
Reads `<world-save-dir>/region/r.<x>.<z>.mca` and writes pruned copies to
|
||||
`<out-dir>/region/`, along with whatever else the world needs. Nested per-dimension
|
||||
`region`/`entities`/`poi` trees are never copied, so the Nether and End are not
|
||||
duplicated at full size; `--minimal` goes further and copies only `level.dat`, which
|
||||
is all Xaero's needs to map the overworld.
|
||||
|
||||
If a full save copy is wanted (level.dat, data/, playerdata/ ...), copy the world
|
||||
first and then run this in place with <out-dir> == <world-save-dir>.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import struct
|
||||
import sys
|
||||
|
||||
SECTOR = 4096
|
||||
HEADER = 2 * SECTOR
|
||||
COMPRESSION_NAMES = {1: "gzip", 2: "zlib", 3: "none", 4: "lz4", 127: "external"}
|
||||
|
||||
|
||||
def read_slot(data: bytes, size: int, slot: int) -> tuple[bytes, int, int] | None:
|
||||
"""Return (payload, timestamp, compression) for an Anvil slot, or None if empty."""
|
||||
off = struct.unpack(">I", b"\x00" + data[slot * 4 : slot * 4 + 3])[0]
|
||||
if off == 0:
|
||||
return None
|
||||
timestamp = struct.unpack(">I", data[SECTOR + slot * 4 : SECTOR + slot * 4 + 4])[0]
|
||||
start = off * SECTOR
|
||||
if start + 5 > len(data):
|
||||
raise ValueError(f"slot {slot}: offset {off} past end of file")
|
||||
(length,) = struct.unpack(">I", data[start : start + 4])
|
||||
compression = data[start + 4]
|
||||
if length < 1 or start + 4 + length > len(data):
|
||||
raise ValueError(f"slot {slot}: bad chunk length {length}")
|
||||
return data[start + 5 : start + 4 + length], timestamp, compression
|
||||
|
||||
|
||||
def prune_region(path: str, keep_slots: set[int], out_path: str, dry: bool) -> dict:
|
||||
with open(path, "rb") as fh:
|
||||
data = fh.read()
|
||||
|
||||
kept: list[tuple[int, bytes, int, int]] = []
|
||||
dropped = 0
|
||||
external = 0
|
||||
for slot in range(1024):
|
||||
entry = read_slot(data, SECTOR, slot)
|
||||
if entry is None:
|
||||
continue
|
||||
payload, timestamp, compression = entry
|
||||
if compression == 127:
|
||||
external += 1
|
||||
if slot in keep_slots:
|
||||
kept.append((slot, payload, timestamp, compression))
|
||||
else:
|
||||
dropped += 1
|
||||
|
||||
stats = {
|
||||
"file": os.path.basename(path),
|
||||
"kept": len(kept),
|
||||
"dropped": dropped,
|
||||
"external_payloads": external,
|
||||
}
|
||||
if dry:
|
||||
return stats
|
||||
|
||||
out = bytearray(HEADER)
|
||||
sector = 2
|
||||
for slot, payload, timestamp, compression in kept:
|
||||
blob = struct.pack(">I", len(payload) + 1) + bytes([compression]) + payload
|
||||
sectors = (len(blob) + SECTOR - 1) // SECTOR
|
||||
if sector + sectors > 0xFFFFFF:
|
||||
raise ValueError(f"{path}: region would exceed the Anvil 24-bit offset limit")
|
||||
out[slot * 4 : slot * 4 + 3] = sector.to_bytes(3, "big")
|
||||
out[slot * 4 + 3] = sectors
|
||||
out[SECTOR + slot * 4 : SECTOR + slot * 4 + 4] = struct.pack(">I", timestamp)
|
||||
out += blob
|
||||
out += b"\x00" * (sectors * SECTOR - len(blob))
|
||||
sector += sectors
|
||||
|
||||
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
|
||||
with open(out_path, "wb") as fh:
|
||||
fh.write(out)
|
||||
return stats
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("world", help="world save directory containing region/")
|
||||
ap.add_argument("mask", help="mask JSON from ftbchunks-reveal.py")
|
||||
ap.add_argument("out", help="output world directory (use the same path for in-place)")
|
||||
ap.add_argument("--dimension", default="minecraft:overworld",
|
||||
help="which dimension in the mask to apply (default overworld)")
|
||||
ap.add_argument("--minimal", action="store_true",
|
||||
help="output only level.dat + the pruned region/ tree (smallest usable world)")
|
||||
ap.add_argument("--dry-run", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
with open(args.mask) as fh:
|
||||
mask = json.load(fh)
|
||||
dims = mask["dimensions"]
|
||||
if args.dimension not in dims:
|
||||
print(f"mask has no dimension '{args.dimension}'; available: {list(dims)}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
chunks = {(cx, cz) for cx, cz in dims[args.dimension]["chunks"]}
|
||||
by_region: dict[tuple[int, int], set[int]] = {}
|
||||
for cx, cz in chunks:
|
||||
rx, rz = cx >> 5, cz >> 5
|
||||
by_region.setdefault((rx, rz), set()).add((cx & 31) + (cz & 31) * 32)
|
||||
|
||||
in_region = os.path.join(args.world, "region")
|
||||
if not os.path.isdir(in_region):
|
||||
print(f"no region/ directory under {args.world}", file=sys.stderr)
|
||||
return 2
|
||||
out_region = os.path.join(args.out, "region")
|
||||
os.makedirs(out_region, exist_ok=True)
|
||||
|
||||
# Copy the rest of the world so the pruned save is openable, unless in-place.
|
||||
# Nested `region`/`entities`/`poi` trees (i.e. per-dimension ones like DIM-1/) are never
|
||||
# copied - copying them wholesale would duplicate the Nether and End at full size.
|
||||
if os.path.abspath(args.out) != os.path.abspath(args.world) and not args.dry_run:
|
||||
ignore = shutil.ignore_patterns("region", "entities", "poi")
|
||||
for name in sorted(os.listdir(args.world)):
|
||||
if name in ("region", "entities", "poi"):
|
||||
continue
|
||||
if args.minimal and name not in ("level.dat", "level.dat_old"):
|
||||
continue
|
||||
src, dst = os.path.join(args.world, name), os.path.join(args.out, name)
|
||||
if os.path.exists(dst):
|
||||
continue
|
||||
if os.path.isdir(src):
|
||||
shutil.copytree(src, dst, ignore=ignore)
|
||||
else:
|
||||
shutil.copy2(src, dst)
|
||||
|
||||
total_kept = total_dropped = 0
|
||||
seen_regions = set()
|
||||
failures: list[tuple[str, str]] = []
|
||||
failed_regions: set[tuple[int, int]] = set()
|
||||
for name in sorted(os.listdir(in_region)):
|
||||
if not (name.startswith("r.") and name.endswith(".mca")):
|
||||
continue
|
||||
_, rx, rz, _ = name.split(".")
|
||||
coords = (int(rx), int(rz))
|
||||
keep = by_region.get(coords)
|
||||
if keep is None:
|
||||
continue # region untouched by the reveal set
|
||||
try:
|
||||
stats = prune_region(
|
||||
os.path.join(in_region, name),
|
||||
keep,
|
||||
os.path.join(out_region, name),
|
||||
args.dry_run,
|
||||
)
|
||||
except Exception as exc:
|
||||
# A live server can rewrite a region file mid-read. A torn read is not a reason to
|
||||
# abandon the whole pass - no output is written for a region that fails, so it can
|
||||
# simply be retried.
|
||||
failures.append((name, str(exc)))
|
||||
failed_regions.add(coords)
|
||||
print(f" ! {name}: {exc}", file=sys.stderr)
|
||||
continue
|
||||
seen_regions.add(coords)
|
||||
total_kept += stats["kept"]
|
||||
total_dropped += stats["dropped"]
|
||||
if stats["external_payloads"]:
|
||||
print(f" ! {name}: {stats['external_payloads']} external (.mcc) payload(s) "
|
||||
f"referenced but not copied", file=sys.stderr)
|
||||
print(f" {name}: kept {stats['kept']:>4} / dropped {stats['dropped']:>4}")
|
||||
|
||||
# Any region the player revealed but that has no .mca file.
|
||||
missing = sorted(set(by_region) - seen_regions - failed_regions)
|
||||
if missing:
|
||||
print(f" ! {len(missing)} revealed region(s) have no .mca in the save "
|
||||
f"(e.g. {missing[:3]})", file=sys.stderr)
|
||||
|
||||
verb = "would keep" if args.dry_run else "kept"
|
||||
print(f"\n{verb} {total_kept:,} chunks, dropped {total_dropped:,}, "
|
||||
f"across {len(seen_regions)} region file(s) -> {out_region}")
|
||||
if failures:
|
||||
print(f"\n{len(failures)} region(s) could not be read (not written):", file=sys.stderr)
|
||||
for name, why in failures:
|
||||
print(f" {name}: {why}", file=sys.stderr)
|
||||
print("Re-run to retry them - usually a transient write by a running server.", file=sys.stderr)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in new issue
Block a user