mirror/Shipwright
1
0
Fork 0
mirror of https://github.com/HarbourMasters/Shipwright synced 2026-08-12 12:18:32 -04:00
Code Issues Packages Projects Releases Wiki Activity

Phase 0: flesh out the verification gates

Browse Source 
Adds torch-migration/PHASE0.md with the full gate detail: what each gate's
variable is, the driver source, exact commands, cost, and exit criteria.

Two gates the plan was missing:
  - Gate C (Release build) - the harness 14/14 is a Debug measurement; CI
    and releases build -O3.
  - Gate A2 (two extractions in one process) - free once the driver exists,
    retires the re-entrancy risk offline instead of by hand in the game.

Also: gates run full-tree + check.sh rather than test_assets.py (which
hardcodes the torch path and copies a filtered yml subset), 19 ROM dumps
not 17, and libgfxd being USE_STANDALONE-only is called out as the real
hazard in Gate A.
This commit is contained in:
briaguya
2026-07-24 21:55:57 -04:00
parent 1c1a3b16a1
commit e4163e1ecd
2 changed files with 503 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
torch-migration/PHASE0.md
+460
View File
@@ -0,0 +1,460 @@
# Phase 0 — Verification gates
Detail for [`PLAN.md`](PLAN.md) Phase 0. **Nothing in Shipwright changes until every gate here is
green.** All work happens in
[`briaguya0/zapd-to-torch-test-harness`](https://github.com/briaguya0/zapd-to-torch-test-harness).
---
## Why there are gates at all
The harness proves 14/14 byte-for-byte parity — but for exactly **one** build of Torch, driven
exactly **one** way:
| | Harness measured | Shipwright will use |
|---|---|---|
| Build kind | executable (`USE_STANDALONE=ON`) | static lib (`OFF`) |
| Games compiled in | all 9 (defaults) | OoT only |
| Build type | `Debug` (`-g`) | `Release` (`-O3`) |
| Driver | `main.cpp` + CLI11 | our own `Companion` calls |
| Extractions per process | one | up to two (vanilla, then MQ) |
| `soh.o2r` | n/a (OTRExporter makes it) | a new in-tree packer |
Every row is an unproven variable. Each gate closes exactly one of them, **one at a time**, so a
failure names its own cause. Running only the final combination would tell us "something broke"
without saying what.
Reading the source says all six should be fine. Reading is how we got here; running is how we know.
---
## Preconditions
- Everything builds inside the **`soh` distrobox** (`distrobox enter soh`) — `cmake`, `ninja`, and
the compilers are not on the host `PATH`.
- Harness at `~/code/zapd-to-torch-test-harness`, submodules initialised.
- `roms/` populated: **19 ROM dumps → 14 version directories** (`ntsc_1-0`, `ntsc_1-1` and
`ntsc_1-2` have two dumps each; `pal_mq_dbg` has three).
*(`PLAN.md` says 17 hashes in a couple of places — it's 19. Corrected there.)*
- `assets/yml/` generated: 20,353 `.yml` / 119 MB across the 14 version dirs, plus the committed
`config.yml`. Gitignored — regenerate with `zapd_to_torch.py` if absent.
- `o2r/` holds all 19 OTRExporter reference archives (one per ROM dump, from Shipwright `95d8f7e`).
- Existing baseline build at `torch/build/` (`USE_STANDALONE=ON`, all games ON, `Debug`,
`PORT_VERSION_ENDIANNESS=ON`) — leave it alone; it *is* the control.
---
## How the gates are actually run
**Do not use `tools/test_assets.py` for these gates.** It hardcodes `TORCH` to
`torch/build/torch`, and it copies a *filtered subset* of the yml into a scratch dir (1,320 of
1,450 files for `pal_gc`) — which is not the invocation SoH will make. It's the right tool for
bisecting a single failing asset, not for proving a configuration.
Use the **full-tree + `check.sh`** path instead. It points Torch at the entire yml tree — the same
shape as Phase 1's `srcdir = assets/` — and diffs the two archives file-by-file:
```sh
OUT=$(mktemp -d) # fresh every run — see below
"$TORCH_BIN" o2r -s assets/yml -d "$OUT" -u 9.2.3 roms/<rom>.z64
cp "$OUT/oot.o2r" o2r/torch.o2r
cp "o2r/<rom>.o2r" o2r/reference.o2r
./check.sh # missing / extra / mismatched, all three
```
`check.sh` never invokes Torch, so it takes any binary — Gate A's driver drops straight in with no
harness patching.
> **The fresh destdir is mandatory, not hygiene.** `Process()` writes `destdir/torch.hash.yml` and
> reads it back on the next run to skip unchanged files. Reuse a destdir and the second run
> silently produces a partial archive. (This is the same constraint that forces Phase 3's
> `Mkdtemp()`.)
### New: `tools/matrix.sh`
One script, used by every gate:
```sh
tools/matrix.sh <torch-bin> <label>
```
- loops all 19 ROM dumps, fresh destdir each
- writes `logs/matrix-<label>/<rom>.log`
- prints a 19-row PASS/FAIL table and exits non-zero on any failure
Cost: ~20 s per ROM (19 s of that is Torch), so **~7 min per configuration**. Four configurations
plus preflight is well under an hour of wall-clock; the builds dominate.
---
## Preflight — reproduce the known-good result
Before changing a single variable, re-run the *current* configuration on this machine, today:
```sh
tools/matrix.sh torch/build/torch baseline
```
Expect 19/19 `PASS: All N files match!`. Without this, a later gate failure is ambiguous between
"the variable broke it" and "something drifted in the environment."
Also confirm the reference archives are what we think they are (the README claims this for
`pal_gc`; spot-check one more):
```sh
./manifest.sh o2r/ntsc_u_gc_b82710.o2r /tmp/regen.json
diff <(jq -S . manifests/ntsc_u_gc.json) <(jq -S . /tmp/regen.json) && echo MATCH
```
---
## Gate D — the upstream pin *(already green)*
`HarbourMasters/Torch` main and the harness's pinned fork commit `9422bf4` have identical trees
(`5d247f78…`; `git diff --stat` = 0 lines). Nothing needs upstreaming for parity.
**Action:** record the merge SHA `4cae44160693e1beb562e39dc301bd42278be1f9` and re-run
`git diff 4cae4416 9422bf4` immediately before opening the Shipwright PR, in case main has moved.
No build, no matrix run.
---
## Gate A — static lib, driven the way SoH will drive it
**Variable:** `USE_STANDALONE=OFF` *and* the hand-rolled driver, together.
They're one gate on purpose. Testing "does the static lib link" without also testing "does driving
`Companion` by hand produce the same bytes" would leave the more interesting half unproven — and
the driver has to exist either way, because `USE_STANDALONE=OFF` compiles out `int main`.
### The driver *is* a dry run of Phase 3
`torchlib-driver/main.cpp` is not throwaway scaffolding. It executes the exact sequence
`soh/soh/Extractor/TorchExtract.cpp` will use, so if any assumption in Phase 3 is wrong, it fails
here — offline, against a reference archive — instead of in the game with a progress bar spinning.
```
zapd-to-torch-test-harness/
torchlib-driver/
CMakeLists.txt
main.cpp
```
**`CMakeLists.txt`** — note it deliberately does *not* set the `BUILD_<game>` flags or the build
type; those are Gate B's and Gate C's variables and get passed on the command line:
```cmake
cmake_minimum_required(VERSION 3.26)
project(torchlib-driver CXX)
set(CMAKE_CXX_STANDARD 20)
set(USE_STANDALONE OFF CACHE BOOL "" FORCE)
set(PORT_VERSION_ENDIANNESS ON CACHE BOOL "" FORCE)
set(ROM_CRC_BSWAP OFF CACHE BOOL "" FORCE)
set(BUILD_UI OFF CACHE BOOL "" FORCE)
set(BUILD_STORMLIB OFF CACHE BOOL "" FORCE)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/../torch ${CMAKE_BINARY_DIR}/torch)
add_executable(torchlib-driver main.cpp)
target_link_libraries(torchlib-driver PRIVATE torch)
```
`torch` exports `src/`, `lib/`, and the yaml-cpp includes as `PUBLIC` under `USE_STANDALONE=OFF`
(`torch/CMakeLists.txt:437-442`), and spdlog becomes `PUBLIC` too — so nothing else is needed.
**`main.cpp`** — argv-compatible with `torch o2r -s S -d D -u V <rom>` so `matrix.sh` can't tell
the two binaries apart:
```cpp
#include <atomic>
#include <cstdio>
#include <filesystem>
#include <string>
#include "Companion.h"
static int RunOnce(const std::string& rom, const std::string& src,
const std::string& dest, const std::string& ver) {
std::atomic<size_t> phases{ 0 };
try {
auto* c = new Companion(std::filesystem::path(rom), ArchiveType::O2R, false, src, dest);
Companion::Instance = c; // bare global; factories deref it unconditionally
c->SetVersion(ver);
c->SetPhaseCallback([&phases](int) { ++phases; });
c->Init(ExportType::Binary); // Init is the whole run; it calls Process() itself
} catch (const std::exception& e) {
fprintf(stderr, "driver: exception: %s\n", e.what());
return 2;
}
if (!std::filesystem::exists(std::filesystem::path(dest) / "oot.o2r")) {
fprintf(stderr, "driver: no oot.o2r produced\n"); // Process() returns void and has
return 3; // several log-and-return failure paths
}
fprintf(stderr, "driver: phases=%zu\n", phases.load());
return 0;
}
```
…plus ~20 lines of argv parsing and, for Gate A2, a `--second <rom> <dest>` flag that calls
`RunOnce` a second time in the same process.
Four Phase 3 claims this proves or refutes:
1. **`Companion::Instance` can be assigned by hand.** It's a raw global defined at
`src/main.cpp:15`, *outside* the `#if defined(STANDALONE)` guard — so the static lib carries the
storage but no entry point. Deliberate, but untested from outside.
2. **`try/catch` is sufficient.** 127 `throw std::runtime_error` sites across `torch/src` and
nothing catches at the top level; an escape is `std::terminate`, not a popup.
3. **Statting the output is necessary.** `Process()` returns `void` and just logs-and-returns on
missing `config.yml` (`Companion.cpp:1304`), unknown ROM hash (`:1334`), no `config:` node
(`:1390`), bad GBI (`:1484`). A silent no-archive must be caught.
4. **`SetPhaseCallback` is a usable progress denominator.** One call site
(`gPhaseCallback(2)`, `Companion.cpp:1282-1283`), fired once per yml file at the parse→export
transition. Compare the printed `phases=` against
`find assets/yml/<version> -name '*.yml' | wc -l` (1,449–1,480 depending on version). If they
match, the Phase 3 progress bar is monotonic 0→100; if they don't, the denominator theory is
wrong and it's better to know now.
### What Gate A is hunting
**libgfxd.** This is the one that could actually fail. `torch/CMakeLists.txt:60-81` fetches and
compiles libgfxd **only `if(USE_STANDALONE)`** — under `OFF` it isn't in the build at all. Reading
the source says that's safe: every `gfxd_*` call sits inside `#ifdef STANDALONE`
(`DisplayListFactory.cpp:81-199` = `DListCodeExporter` + `DebugDisplayList`;
`DisplayListOverrides.cpp:18-224` = the `GFXDOverride` callbacks), while `DListBinaryExporter`
(`:239`) and `DListFactory::parse` (`:526`) are outside it. Display lists are the largest asset
class in the archive, so if that reading is wrong the gate fails immediately and unmistakably.
**Other `STANDALONE` deltas, all expected inert:**
- `Companion.cpp:1524` gates `enums:` parsing — `config.yml` has no `enums:` key.
- `DListCodeExporter` isn't registered — we export `Binary`.
- yaml-cpp gains `YAML_CPP_STATIC_DEFINE`; spdlog moves `PRIVATE`→`PUBLIC`; the standalone-only
`find_package(ZLIB REQUIRED)` disappears (`zlibstatic` from FetchContent remains).
**One landmine worth writing down:**
`if(NOT USE_STANDALONE AND EXISTS "/mnt/c/WINDOWS/system32/wsl.exe")` (`torch/CMakeLists.txt:295`)
FetchContents Microsoft GSL. It fires **only inside WSL**, and only in the static-lib
configuration — i.e. exactly what Shipwright will use. Irrelevant in the distrobox; it will bite
the first person who builds SoH under WSL after this lands.
### Running it
```sh
cmake -S torchlib-driver -B torchlib-driver/build -GNinja -DCMAKE_BUILD_TYPE=Debug
cmake --build torchlib-driver/build -j
tools/matrix.sh torchlib-driver/build/torchlib-driver static-lib
```
**Pass:** 19/19 identical, and `phases=` matches the yml count for each version.
Smoke-test `pal_gc` alone before committing to the full matrix.
---
## Gate A2 — two extractions in one process
**Variable:** re-entrancy. Free to add once the driver exists, and it retires
[`PLAN.md`](PLAN.md) risk #6 offline instead of by hand in the game.
`RunExtract` can extract more than once per process (`PS_FIRST` → `PS_SECOND`, and
`ES_EXTRACT_ARGS` loops over CLI args), so a real user extracting vanilla and then MQ in one
session hits state SoH has never exercised:
- `gProcessedFiles` (`Companion.h:300`) is **never cleared** — a second `Process()` on the *same*
instance skips every file. Phase 3's answer is a fresh `Companion` per call; this gate is what
proves that's enough.
- `gAddrMap` / `gParseResults` / `gWriteMap` also accumulate.
- `AliasManager::Instance` isn't cleared at the end of `Process` (believed inert for OoT — no OoT
factory calls `Register` — but unproven).
- `AudioManager::Instance` leaks per run (`Companion.cpp:1584`).
- File-scope statics in `TextureFactory.cpp:14-15`, `CompressedTextureFactory.cpp:17-18`,
`DisplayListFactory.cpp:82`.
```sh
tools/matrix.sh --pair pal_gc_0227d7 pal_mq_f46239 <driver> reentrancy
```
One process, two fresh `Companion`s, two destdirs; both archives compared against their own
references. **Pass:** both identical — specifically, the *second* one must be identical, which is
the whole point.
---
## Gate B — OoT-only build
**Variable:** the eight non-OoT `BUILD_<game>` flags. Still `USE_STANDALONE=ON`, still `Debug`, so
this is measured against the preflight baseline with exactly one thing changed.
```sh
cmake -S torch -B torch/build-oot -GNinja -DCMAKE_BUILD_TYPE=Debug \
-DPORT_VERSION_ENDIANNESS=ON \
-DBUILD_SM64=OFF -DBUILD_MK64=OFF -DBUILD_SF64=OFF -DBUILD_PM64=OFF \
-DBUILD_FZERO=OFF -DBUILD_BK64=OFF -DBUILD_MARIO_ARTIST=OFF -DBUILD_NAUDIO=OFF
cmake --build torch/build-oot -j
tools/matrix.sh torch/build-oot/torch oot-only
```
Each `BUILD_X=OFF` drops `-DX_SUPPORT` and filters `src/factories/x/*` out of the source glob
(`torch/CMakeLists.txt:158-207`). Two effects are real:
- **`BUILD_BK64=OFF`** removes `BK64::TrySynthesizeRomConfig`, Companion's fallback for a ROM hash
absent from `config.yml`. All 19 of ours are present, so it's unreachable here — but it means an
*unknown* dump behaves differently, which is precisely why Phase 3 must stat the output rather
than trust the run.
- **`BUILD_NAUDIO=OFF`** removes Torch's only tinyxml2 consumers. It does **not** remove the
tinyxml2 `FetchContent` — that's unconditional — so Phase 2's `OVERRIDE_FIND_PACKAGE` concern
stands regardless.
Nothing else should touch a shared path. A cross-game factory registration or a shared header
behind an `#ifdef` would show up as a diff, and that's what the gate is for.
---
## Gate C — Release build
**Variable:** `-DCMAKE_BUILD_TYPE=Release`. Otherwise the baseline configuration.
**This gate isn't in `PLAN.md` and should be.** The harness's 14/14 is a `Debug` measurement
(`torch/build/CMakeCache.txt`: `CMAKE_BUILD_TYPE=Debug`), and so is the local Shipwright build.
Shipwright's CI and every release build `Release` — Torch sets `CMAKE_CXX_FLAGS_RELEASE "-O3"` for
non-MSVC (`torch/CMakeLists.txt:242`). Any UB, uninitialised read, or unspecified evaluation order
in Torch can change bytes between `-g` and `-O3`.
```sh
cmake -S torch -B torch/build-release -GNinja -DCMAKE_BUILD_TYPE=Release -DPORT_VERSION_ENDIANNESS=ON
cmake --build torch/build-release -j
tools/matrix.sh torch/build-release/torch release
```
Seven minutes to find out, versus finding out from a user whose release-build archive doesn't match
their friend's. If it fails it's an upstream Torch bug and everything else stops until it's fixed.
---
## Gate A′ — the shipping configuration
**Variable:** all of them at once — static lib + driver + OoT-only + Release.
Not implied by A ∧ A2 ∧ B ∧ C. `#ifdef` interactions are combinatorial, and this is the only
configuration that will ever ship.
```sh
cmake -S torchlib-driver -B torchlib-driver/build-ship -GNinja -DCMAKE_BUILD_TYPE=Release \
-DBUILD_SM64=OFF -DBUILD_MK64=OFF -DBUILD_SF64=OFF -DBUILD_PM64=OFF \
-DBUILD_FZERO=OFF -DBUILD_BK64=OFF -DBUILD_MARIO_ARTIST=OFF -DBUILD_NAUDIO=OFF
cmake --build torchlib-driver/build-ship -j
tools/matrix.sh torchlib-driver/build-ship/torchlib-driver shipping
tools/matrix.sh --pair pal_gc_0227d7 pal_mq_f46239 torchlib-driver/build-ship/torchlib-driver shipping-pair
```
**When A′ is green, that flag set is the spec.** Phase 2's root-`CMakeLists.txt` block is a
transcription of it, not a re-derivation — and `main.cpp`'s `RunOnce` is a transcription into
`TorchExtract.cpp`. Copy, don't rewrite.
---
## Gate E — the `soh.o2r` baseline
Different in kind from the others: it doesn't test Torch at all. It captures the artifact the
Phase 4 packer has to reproduce, from a pipeline that is about to be deleted. **Do this while
ZAPD still builds.**
Current state: `soh.o2r` at the repo root, **1,042 entries** = 1,041 files under
`soh/assets/custom/` + `portVersion`. No `version` file — matching
`sohArchiveVersionMatch` (`OTRGlobals.cpp:282-283`), which only reads `portVersion`.
1. **Rebuild rather than trust the file on disk**, so provenance is unambiguous:
```sh
cmake --build build --target GenerateSohOtr # inside the soh distrobox
```
> `GenerateSohOtr` **writes into the source tree**: it `rm -rf`s and re-copies
> `soh/assets/custom/shaders/` from `libultraship/src/fast/shaders/` (3 files —
> `directx/default.shader.hlsl`, `metal/default.shader.metal`, `opengl/default.shader.glsl`).
> That directory is gitignored (`soh/assets/.gitignore:8`), which is why `git status` stays
> clean and why the on-disk count (1,041) exceeds `git ls-files` (1,038). Phase 4's packer must
> run after the same copy or the archives differ by exactly those 3 entries.
2. **Record the output** — `manifest.sh` already does exactly this job:
```sh
./manifest.sh /var/home/briaguya/code/Shipwright/soh.o2r manifests/soh_o2r.json
```
Lives in the harness alongside the other 14 manifests. (Not in the yml repo — that holds yml and
nothing else.)
3. **Record the inputs too**, or a future mismatch is ambiguous between "the packer is wrong" and
"the assets changed":
- sha256 of every file under `soh/assets/custom/` → `manifests/soh_custom_inputs.json`
- `CMAKE_PROJECT_VERSION` = **9.2.3** (`CMakeLists.txt:9`), the `portVersion` payload
- the `develop` SHA the archive was built from
4. **Call out the 31 files that matter.** Only these hit the conversion path — everything else is a
byte copy, and a byte copy that fails is a trivial bug:
| Format | Count |
|---|---|
| `.rgba32.png` | 15 |
| `.ia8.png` | 8 |
| `.ia16.png` | 5 |
| `.ia4.png` | 3 |
The other 1,010: 923 objects, 38 plain `.png`, 23 `.rgba16`, 22 json, 4 ttf, 3 shaders, 2 xml.
The trap is quantisation, not decoding — `n64graphics`' `SCALE_8_4(x)` (`x*15/255`) vs ZAPD's
`(x >> 4) & 0xF` disagree (`0xF0` → `0xE` vs `0xF`). Phase 4 hand-ports ZAPD's arithmetic; this
manifest is what catches it if the port is wrong.
---
## Exit criteria
| Gate | Pass condition |
|---|---|
| Preflight | 19/19 identical with the existing baseline binary; one manifest re-derivation matches |
| D | pin SHA recorded; `git diff` vs upstream main still empty |
| A | 19/19 identical; `phases=` matches the per-version yml count |
| A2 | both archives in a two-extraction process identical — especially the second |
| B | 19/19 identical |
| C | 19/19 identical |
| A′ | 19/19 identical, plus the pair run |
| E | `manifests/soh_o2r.json` (1,042 entries) + input hashes + version + SHA committed |
"Identical" means `check.sh` prints `PASS: All N files match!` — **0 missing, 0 extra,
0 mismatched**, not "close enough".
Write the results up as a second table in the harness README, next to the existing 14/14 one. That
table is the evidence the Shipwright PR points at.
---
## Cost
| | Build | Matrix |
|---|---|---|
| Preflight | — | ~7 min |
| A | full Torch rebuild | ~7 min |
| A2 | — (reuses A) | ~1 min |
| B | full Torch rebuild | ~7 min |
| C | full Torch rebuild | ~7 min |
| A′ | full Torch rebuild | ~8 min |
| E | Shipwright `GenerateSohOtr` | ~1 min |
An evening, dominated by four full Torch builds. Four extra build trees; the `Debug` Torch binary
alone is 125 MB, so either build them sequentially and delete as you go or budget a few GB.
---
## What Phase 0 deliberately does **not** cover
Being explicit so these don't get mistaken for retired risk:
- **Linking Torch into Shipwright.** The zlib / stb / `StringHelper` / `StrHash64` / tinyxml2 /
`storm` collisions in `PLAN.md` Phase 2 need SoH's own build to surface. A standalone driver
links Torch against nothing else, so it can't see any of them. That's Phase 2's first day.
- **spdlog global-state stomping.** Torch calls `spdlog::set_level`/`set_pattern` on the *default*
logger (`Companion.cpp:193-194, 1810-1814`), and our `config.yml` sets `logging: CRITICAL` — so
this fires on every run, it just doesn't matter until there's a SoH logger to clobber.
- **Windows and macOS.** Every gate runs Linux/gcc in the distrobox. Torch forces `/MT` under MSVC,
adds `-DSTORMLIB_NO_AUTO_LINK` and `/bigobj`, and carries a `cmake_minimum_required(3.12)` +
`CMAKE_POLICY_VERSION_MINIMUM` shim under a CMake 4 host. If a Windows machine is available,
running Gate A′ there is the cheapest possible insurance on the one platform that can't be
predicted from reading — but it isn't a blocker for starting Phase 2.
- **Anything in-game.** The driver proves the bytes; it says nothing about the ImGui progress bar,
the ROM picker, or `ValidHashes`. That's the Phase 3 manual matrix.
torch-migration/PLAN.md
+43 -26
View File
@@ -67,35 +67,52 @@ that header regeneration is temporarily unavailable; file the Torch PR in parall
## Phase 0 — Verification gates (no Shipwright changes)
All run in `zapd-to-torch-test-harness` against the committed `manifests/*.json`.
> **Detail: [`PHASE0.md`](PHASE0.md)** — driver source, exact commands, cost, exit criteria, and
> what Phase 0 deliberately doesn't cover.
**Gate A — `USE_STANDALONE=OFF` is byte-identical.**
The static-lib build has no CLI, so add a throwaway driver *in the harness*
(`torchlib-driver/`: `add_subdirectory(../torch)` with `USE_STANDALONE=OFF`, ~40-line `main.cpp`
constructing `Companion` and calling `Init(ExportType::Binary)`). Build both ways from the same
tree with `-DPORT_VERSION_ENDIANNESS=ON`, run all 14 versions, compare via `check.sh`.
Expected identical: the only reachable `#ifdef STANDALONE` difference is `enums:` parsing
(`Companion.cpp:1524`) and the Code exporter — and `assets/yml/config.yml` has no `enums:` key.
Smoke-test `pal_gc` first.
The harness's 14/14 was measured on **one** build of Torch (`USE_STANDALONE=ON`, all 9 games,
`Debug`, driven by the CLI) — Shipwright will use a different build in a different way. Each gate
below changes exactly **one** variable against that baseline, so a failure names its own cause.
All run in `zapd-to-torch-test-harness`; nothing touches Shipwright.
**Gate B — OoT-only build is byte-identical.**
`-DBUILD_{SM64,MK64,SF64,PM64,FZERO,BK64,MARIO_ARTIST,NAUDIO}=OFF`. Run the 14-target matrix
(`ROM_VERSION=<v> python3 tools/test_assets.py roms/<rom>.z64`). The 14/14 parity was measured
with defaults ON, so this is a genuine change. Two real effects: `BUILD_BK64=OFF` drops the
`BK64::TrySynthesizeRomConfig` fallback (irrelevant — all 17 hashes are in `config.yml`), and
`BUILD_NAUDIO=OFF` removes Torch's only tinyxml2 consumers.
**Preflight.** Re-run the existing baseline binary over all 19 ROM dumps first, so a later failure
can't be confused with environment drift.
**Gate A′ — the shipping configuration.** Re-run Gate A *with* the Gate B flags. This exact
combination is what ships.
**Gate A — static lib, driven the way SoH will drive it.** `USE_STANDALONE=OFF` plus a
`torchlib-driver/` in the harness that executes the exact `Companion` sequence Phase 3 will use
(assign `Companion::Instance`, `SetVersion`, `SetPhaseCallback`, `Init(Binary)`, try/catch, stat
the output) — so it's a dry run of `TorchExtract.cpp`, not just a link test. The real hazard:
**libgfxd is fetched and compiled only `if(USE_STANDALONE)`**. Reading says every `gfxd_*` call is
inside `#ifdef STANDALONE` (Code exporter + debug dump) and the Binary exporter is outside it — if
that reading is wrong, display lists break loudly. Also confirms the phase-callback count matches
the on-disk yml count, which is the Phase 3 progress denominator.
**Gate E — `soh.o2r` baseline.** From current `develop`, build `GenerateSohOtr` and record a
sha256 manifest of all 1,041 entries. This is the acceptance test for the new packer. All 31
convertible PNGs are 8-bit / colour-type 6, so decode risk is low; the risk is quantisation
(see Phase 4).
**Gate A2 — two extractions in one process.** Free once the driver exists; retires risk #6
(`gProcessedFiles` is never cleared, `AliasManager` isn't reset, factory-local statics) offline
instead of by hand in the game.
**Gate D — upstream pin.** Already satisfied (trees equal). Just record the SHA.
**Gate B — OoT-only build.** `-DBUILD_{SM64,MK64,SF64,PM64,FZERO,BK64,MARIO_ARTIST,NAUDIO}=OFF`.
Two real effects: `BUILD_BK64=OFF` drops the `BK64::TrySynthesizeRomConfig` fallback (unreachable —
all 19 hashes are in `config.yml`), and `BUILD_NAUDIO=OFF` removes Torch's only tinyxml2 consumers
(but not the unconditional tinyxml2 `FetchContent`).
**Nothing in Shipwright changes until A, B, A′, E are green.**
**Gate C — Release build.** The 14/14 is a `Debug` measurement; CI and releases build `-O3`. Any UB
or unspecified evaluation order in Torch can change bytes between the two. Seven minutes to find
out here instead of from a user.
**Gate A′ — the shipping configuration.** Static lib + driver + OoT-only + Release, all at once.
Not implied by the others — `#ifdef` interactions are combinatorial. When it's green, that flag set
*is* the spec: Phase 2's CMake block and Phase 3's `TorchExtract.cpp` are transcriptions of it.
**Gate D — upstream pin.** Already satisfied (trees equal). Record the SHA; re-diff before opening
the PR.
**Gate E — `soh.o2r` baseline.** Rebuild `GenerateSohOtr` on current `develop` **while ZAPD still
builds** and record a sha256 manifest of all 1,042 entries (1,041 files + `portVersion`), plus the
input hashes and version that produced it. Only 31 files hit the conversion path; the risk is
quantisation, not decoding (see Phase 4).
**Nothing in Shipwright changes until A, A2, B, C, A′, E are green.**
---
@@ -109,7 +126,7 @@ empty). Submoduled at **`assets/`** (top level, sibling of `soh/`) so the eventu
**It holds yml and nothing else** — no generators, no generator inputs, no manifests:
```
config.yml 17 ROM SHA1 → 14 version dirs (gbi: F3DEX2_OoT, sort: OFFSET,
config.yml 19 ROM SHA1 → 14 version dirs (gbi: F3DEX2_OoT, sort: OFFSET,
primary_virtual_segment: 0x80, strict_declarations: true)
<14 version dirs>/ 20,353 .yml, 111 MB
README.md what this is; which harness commit produced the initial import
@@ -285,7 +302,7 @@ because a fresh temp destdir has no `torch.hash.yml`. **Nothing in the ImGui pro
straight into the app dir would litter it *and* make a second in-session extraction skip files it
thinks are unchanged.
**MQ naming:** leave `config.yml` at `binary: oot.o2r` for all 17 hashes and rename on the way out
**MQ naming:** leave `config.yml` at `binary: oot.o2r` for all 19 hashes and rename on the way out
of the temp dir based on `IsMasterQuest()`. Torch offers no output-filename override
(`-d` sets only the directory; you'd get a *directory* named `oot-mq.o2r`), and this keeps SoH the
single source of truth for MQ-ness and preserves the harness's `<out>/oot.o2r` expectation.
@@ -443,7 +460,7 @@ validated automatically. Needs a ROM, so self-hosted or encrypted secret.
```
0. Torch PR: gate the zlib fetch, make stb/StringHelper guards unconditional [only if they fire]
1. Harness: Gates A, B, A′
1. Harness: preflight, then Gates A, A2, B, C, A′ (see PHASE0.md)
2. Seed briaguya0/soh-asset-yml: one-shot conversion output (config.yml + 14 version dirs)
3. Gate E: soh.o2r manifest from current develop
--- single Shipwright PR from here ---