onion-mc/analysis/FTB-XAERO-MAP-MIGRATION.md

FTB Chunks reveal -> Xaero's World Map: what transfers and how

Status: mask extraction implemented and validated against real data; world-save pruning implemented and unit-tested; final step blocked on a copy of the server world save. Goal: give each player a native Xaero's map covering, as closely as possible, exactly the terrain they had revealed in FTB Chunks. Confidence tags as in analysis/archive/FINDINGS.md.

---

1. Why we do not convert the pixels

FTB Chunks does not store a rendered map image. Verified by decompiling the pack's exact build, ftb-chunks-neoforge-2101.1.22.jar (class strings in MapManager, MapRegionData, MapDimension, MapChunk):

<game>/local/ftbchunks/data/<serverUUID>/
├── dimensions.txt, block_map.txt, biome_map.txt
└── <dimension>/
    └── <XXXXX-XXXXX>.zip          # region = 512x512 blocks; name = region coord + 60000, %05X-%05X
        ├── chunks.dat             # the chunks this player has actually seen
        ├── data.png               # NOT viewable: (height << 16) | (water|light|biome) per pixel
        ├── foliage.png            # biome-tinted foliage RGB
        ├── grass.png              # biome-tinted grass RGB
        ├── water.png              # biome-tinted water RGB
        └── blocks.png             # FTB's internal 24-bit block colour index

The colour a player actually saw is computed at runtime by combining blocks.png's index with FTB's own BlockColor model via block_map.txt. Reproducing it means reimplementing FTB Chunks' renderer, and the result would look like FTB's map - i.e. worse than native Xaero's. So we take the reveal footprint from FTB Chunks and the pixels from Xaero's.

2. Why we also do not write Xaero region files

The current Xaero's region format is not the format the community tools understand. Comparing a real region.xaero from Xaero's World Map 1.46.0 (extracted from the client instance) with the format JMtoXaero/RegionToXaero writes:

	old format (RegionToXaero)	real 1.46.0
header	FF then int 4	FF 00 06 00 08 + varying byte (format 6)
pixel payload	4-byte parameters (+ optional state / colour / biome)	embeds NBT compounds
size for region 0,0	~25 KB	355 KB

The 1.46.0 payload contains literal NBT - e.g. 0A 00 00 08 00 04 "Name" 00 10 "minecraft:gravel" - so each pixel now carries a block descriptor. This matches JMtoXaero's own admission that its region merger broke once Xaero changed the format. Writing this format is a live reverse-engineering project against a closed, obfuscated, moving target. Not worth it.

Therefore: drive Xaero's own generator, and control what it can see.

3. The mechanism

Both mods use the same geometry - a region is 512x512 blocks = 32x32 chunks - so an FTB region maps 1:1 onto one Anvil region file, and an FTB within-region chunk (x,z) maps 1:1 onto Anvil slot x + z*32. That makes the transplant exact rather than approximate:

1. Extract each player's reveal set from their chunks.dat files.
2. Prune a copy of the world save to exactly those chunks (Anvil header/sector surgery; chunk payloads copied byte-for-byte, no NBT re-encoding).
3. The player opens the pruned save in singleplayer with the pack; Xaero's maps it directly from the save, so absent chunks simply never appear on the map.
4. Copy the generated instance (xaero/world-map/World/null/<map>/) into the server's folder (xaero/world-map/Multiplayer_advilplug.top/<DIM>/).

Result: per-chunk-exact reveal extent, rendered natively by Xaero's. Residual differences are only the colour palette/shading (Xaero's, not FTB's) - which is the whole point.

4. What the real data says

Measured on the supplied archive - Stirwick, minecraft:overworld:

Metric	Value
Region files	167
Revealed chunks	98,170 (of 171,008 in the bounding box)
Region extent	x -9..9, z -14..7
Block extent	x -4608..5120, z -7168..4096
Fully revealed regions (1024/1024)	10
Median region fill	560 / 1024 (~55%)
Sparse regions (<64 chunks)	10
chunks.dat vs data.png disagreements	0

Two consequences:

• chunks.dat alone is a trustworthy mask. It was cross-checked against the data.png height sentinel (HeightUtils.UNKNOWN = Short.MIN_VALUE + 1 = -32767) for all 167 regions. Zero disagreements. Suspicion that claimed-but-never-visited chunks would inflate the set did not materialise.
• Region-granularity pruning is not good enough. The median region is only ~55% explored, and 10 regions are under 6%. Copying whole .mca files a player touched would reveal roughly twice the terrain they had seen. Chunk-level pruning is worth the extra ~40 lines.

data.png decoding detail, for anyone revisiting this: FTB writes (height << 16) | wlb into a TYPE_INT_ARGB image, so after PNG round-trip the 16-bit height is (A << 8) | R and the water/light/biome word is (G << 8) | B. Unrevealed pixels are exactly A=0x80, R=0x01.

5. Tooling

Both scripts are stdlib-only and live in tools/ (which .packwizignore excludes, so they never ship).

tools/migrate-player.sh - the driver, does steps 1-2 and prints the manual remainder:

tools/migrate-player.sh Stirwick <ftb-data-or-archive> <world-dir> \
    --dimension minecraft:overworld --work /var/tmp/xaero-migration

tools/ftbchunks-reveal.py - validated against Stirwick's real 167-region archive. Emits a JSON mask of global chunk coordinates per dimension. Pass --dimension when the input is a bare directory/zip of region files, which cannot name its own dimension.

tools/ftbchunks-reveal.py <ftb-data-or-archive> --out mask.json --dimension minecraft:overworld --verify

tools/filter-xaero-regions.py - post-harvest cleanup. Compares the Xaero's map folder against the mask and removes regions the player never revealed (i.e. terrain Minecraft regenerated around the login position). File-name-only, so no Xaero format knowledge needed; reports before deleting.

tools/filter-xaero-regions.py <game>/xaero/world-map/Multiplayer_advilplug.top/null/ mask.json

tools/prune-world-to-reveal.py - validated end-to-end against a real region file from the server (world/region/r.1.-2.mca, 15.7 MB):

Check	Result
Anvil parse (1024 slots, all zlib, 2.5-21.8 KB payloads)	OK
Slot -> chunk mapping (slot = x + z*32), confirmed by reading xPos/zPos NBT	1024 / 1024 exact
Prune against that player's FTB mask (52 of 1024 chunks revealed)	52 kept / 972 dropped
Output sector-aligned	yes
Kept slot set == mask	yes
Kept payloads + timestamps byte-identical	yes
Kept chunks re-parse with correct coordinates	52 / 52
Size	15,745,024 -> 471,040 bytes (3.0%)

Earlier synthetic tests also cover nested per-dimension region/entities/poi trees not being duplicated.

tools/prune-world-to-reveal.py <world-copy> mask.json <out-dir> --dimension minecraft:overworld

--minimal writes only level.dat plus the pruned region/ tree - the smallest thing Xaero's can open and map, and the best fit when the full save is too large to copy around. --dry-run reports kept/dropped counts per region without writing anything. Runtime is ~0.15 s per 15 MB region.

6. Running it where the save lives

The tools are stdlib-only Python, so nothing has to be transferred: run them on the machine that holds the world. The live world is not on the VPS (the VPS is only a proxy in front of the real server), so this has to run where the save actually lives. The per-player footprint is only the pruned region/ tree for the regions that player revealed, not the whole save.

tools/migrate-player.sh Stirwick <stirwick-ftbchunks-data> <world-dir> \
    --dimension minecraft:overworld --work /var/tmp/xaero-migration

That extracts the mask (with the chunks.dat vs data.png cross-check) and prunes the world, dry-run first. Then, by hand:

3. Copy the pruned world (<work>/<player>/sp-world) to a machine that can run the pack and open it in singleplayer once. Let Xaero's map it, then quit.
4. Move the generated instance, xaero/world-map/World/null/<map>/, into the server's map folder, xaero/world-map/Multiplayer_advilplug.top/<DIM>/.
5. Run tools/filter-xaero-regions.py on that folder to drop regions the player never revealed.
6. Delete the scratch directory - usually a few hundred MB.

Two things that will bite

• The server does not need stopping, but freeze saves while it runs. The pruner only reads the world and writes to a separate output directory. However a running server rewrites region files on autosave and rewrites level.dat in place, so a read can tear - and half a level.dat is an unopenable singleplayer world. From the console: save-off -> dry run -> real run -> save-all -> save-on. The pass takes a couple of minutes. Never forget save-on, and never point --out at the live world. A region that does fail to read is reported, skipped, and left unwritten; re-run to retry it.

• Opening the world regenerates chunks near the login position. Minecraft loads and generates the chunks around wherever the player spawns, so terrain the player never revealed can reappear on the map if it happens to sit near world spawn. It is bounded by render distance (well under one 512x512 region at typical settings), and it is easy to undo: step 4's cleanup compares the harvested region file names against the mask and deletes any region the player never revealed. Keeping level.dat's spawn inside the revealed area, or not walking around in step 3, avoids the rest.

• Do not copy the world with hardlinks. cp -al would make the singleplayer client's own writes to level.dat land in the live server save. A real copy, or --minimal into a fresh directory, is safe.

7. Open items

• Needs: the full world save, read in place on the server. One real region file (r.1.-2.mca) has now validated the reader, the slot mapping and the writer, so the remaining unknowns are operational rather than format-related: total runtime over 167 regions (extrapolating ~0.15 s per 15 MB region, a couple of minutes) and the size of the pruned output. A full 1024-chunk region on this server is ~15 MB; the player revealed 57.4% of the bounding box overall, so expect the pruned per-player world to land in the low hundreds of MB. It is scratch data and can be deleted once the Xaero's map has been harvested.
• Per-player, per-dimension. Only minecraft:overworld was supplied and only one player's data. Modded dimensions will each need an FTB-data -> Xaero-dimension mapping. Xaero names its dimension folders null/DIM-1/DIM1 for vanilla and uses the minecraft$overworld form ($, not :) in dimension_config.txt; a dimensions.txt with more than one entry will confirm the rest.
• Step 3 costs one client launch per player. It cannot be done server-side; Xaero's maps from a save on the client. ~10 launches for this pack.
• Waypoints are a separate, easier win. FTB stores them in waypoints.json next to the region data (empty, 16 bytes, for the sampled player). Xaero waypoints are plain files under xaero/minimap/, so a converter is straightforward - worth doing at the same time if anyone has waypoints to keep.
• Fallbacks if the save never materialises: region-granularity pruning (coarse, ~2x over-reveal), or shipping the full "Convert All Regions" map to everyone (over-reveals everything, zero code).
• Nothing here is time-sensitive: removing the mod from the pack does not touch local/ftbchunks/, so every client's reveal data is still on disk and this remains doable later.