Shipwright/torch-migration/PHASE0.md

# Phase 0 — Verification gates

Detail for [`PLAN.md`](PLAN.md) Phase 0. **Nothing in Shipwright changes until every gate here is
green.** All work happens in
[`briaguya0/zapd-to-torch-test-harness`](https://github.com/briaguya0/zapd-to-torch-test-harness).

---

## Why there are gates at all

The harness proves 14/14 byte-for-byte parity — but for exactly **one** build of Torch, driven
exactly **one** way:

| | Harness measured | Shipwright will use |
|---|---|---|
| Build kind | executable (`USE_STANDALONE=ON`) | static lib (`OFF`) |
| Games compiled in | all 9 (defaults) | OoT only |
| Build type | `Debug` (`-g`) | `Release` (`-O3`) |
| Driver | `main.cpp` + CLI11 | our own `Companion` calls |
| Extractions per process | one | up to two (vanilla, then MQ) |
| `soh.o2r` | n/a (OTRExporter makes it) | a new in-tree packer |

Every row is an unproven variable. Each gate closes exactly one of them, **one at a time**, so a
failure names its own cause. Running only the final combination would tell us "something broke"
without saying what.

Reading the source says all six should be fine. Reading is how we got here; running is how we know.

---

## Preconditions

- Everything builds inside the **`soh` distrobox** (`distrobox enter soh`) — `cmake`, `ninja`, and
  the compilers are not on the host `PATH`.
- Harness at `~/code/zapd-to-torch-test-harness`, submodules initialised.
- `roms/` populated: **19 ROM dumps → 14 version directories** (`ntsc_1-0`, `ntsc_1-1` and
  `ntsc_1-2` have two dumps each; `pal_mq_dbg` has three).
  *(`PLAN.md` says 17 hashes in a couple of places — it's 19. Corrected there.)*
- `assets/yml/` generated: 20,353 `.yml` / 119 MB across the 14 version dirs, plus the committed
  `config.yml`. Gitignored — regenerate with `zapd_to_torch.py` if absent.
- `o2r/` holds all 19 OTRExporter reference archives (one per ROM dump, from Shipwright `95d8f7e`).
- Existing baseline build at `torch/build/` (`USE_STANDALONE=ON`, all games ON, `Debug`,
  `PORT_VERSION_ENDIANNESS=ON`) — leave it alone; it *is* the control.

---

## How the gates are actually run

**`tools/test_assets.py`, unfiltered — the same tool that produced the existing 14/14.** Using
anything else would make the gate results not directly comparable to the baseline they're being
measured against.

An unfiltered run is a genuine full-tree, both-directions comparison:

- `list_assets` with no `--category/--file/--type` returns **every** key in
  `manifests/<version>.json` — all 35,386 for `pal_gc`.
- `collect_yaml_files` reverse-maps those asset paths to yml files (1,320), then `setup_scratch`'s
  `copy_yaml_with_externals` recursion follows `external_files:` and pulls in the remaining 130 —
  **all 1,450 land in the scratch dir.** Verified by replaying the logic: `copied == on-disk`, zero
  yml files omitted.
- It hashes what Torch produced and diffs it against the manifest in *both* directions —
  `not generated` (in the reference, missing from ours) and `not in reference` (extras, computed
  over the whole zip, not just the filtered set).
- The destdir is a fresh `mkdtemp` per run, which matters: `Process()` writes
  `destdir/torch.hash.yml` and reads it back next run to skip unchanged files. Reuse a destdir and
  the second run silently produces a partial archive. (Same constraint that forces Phase 3's
  `Mkdtemp()`.)

> The `Copying 1320 YAMLs to scratch dir` line in the logs is printed *before* the `external_files`
> recursion runs — it's the pre-recursion count, not what actually gets copied. Easy to misread as
> a filtered subset. It isn't one.

**One harness patch is needed:** the torch binary path is hardcoded in two places —
`tools/test_assets.py:34` and `lib.sh:12`. Make both honour a `TORCH_BIN` environment override so
Gate A's driver can be dropped in without editing files between runs:

```python
TORCH = os.environ.get("TORCH_BIN") or os.path.join(SOH_DIR, "torch", "build", "torch")
```

`./check.sh` stays as the secondary whole-archive check — it compares two extracted `.o2r`
directly, so it's the tool to reach for if a gate fails and you want the raw file-list diff rather
than a per-asset table.

### New: `tools/matrix.sh`

There's no 19-ROM loop script today; every gate needs one. It's a thin wrapper, not new comparison
logic:

```sh
TORCH_BIN=<binary> tools/matrix.sh <label>
```

- loops all 19 ROM dumps, mapping each to its `--rom-version`
- calls `test_assets.py <rom> --rom-version <ver> --failures-only`
- writes `logs/matrix-<label>/<rom>.log`, prints a 19-row PASS/FAIL table, exits non-zero on any
  failure

Cost: ~20 s per ROM (19 s of that is Torch), so **~7 min per configuration**. Four configurations
plus preflight is well under an hour of wall-clock; the builds dominate.

---

## Preflight — reproduce the known-good result

Before changing a single variable, re-run the *current* configuration on this machine, today:

```sh
TORCH_BIN=torch/build/torch tools/matrix.sh baseline
```

Expect 19/19 `0 failed, 0 not generated, 0 not in reference`. Without this, a later gate failure is ambiguous between
"the variable broke it" and "something drifted in the environment."

Also confirm the reference archives are what we think they are (the README claims this for
`pal_gc`; spot-check one more):

```sh
./manifest.sh o2r/ntsc_u_gc_b82710.o2r /tmp/regen.json
diff <(jq -S . manifests/ntsc_u_gc.json) <(jq -S . /tmp/regen.json) && echo MATCH
```

---

## Gate D — the upstream pin *(already green)*

`HarbourMasters/Torch` main and the harness's pinned fork commit `9422bf4` have identical trees
(`5d247f78…`; `git diff --stat` = 0 lines). Nothing needs upstreaming for parity.

**Action:** record the merge SHA `4cae44160693e1beb562e39dc301bd42278be1f9` and re-run
`git diff 4cae4416 9422bf4` immediately before opening the Shipwright PR, in case main has moved.
No build, no matrix run.

---

## Gate A — static lib, driven the way SoH will drive it

**Variable:** `USE_STANDALONE=OFF` *and* the hand-rolled driver, together.

They're one gate on purpose. Testing "does the static lib link" without also testing "does driving
`Companion` by hand produce the same bytes" would leave the more interesting half unproven — and
the driver has to exist either way, because `USE_STANDALONE=OFF` compiles out `int main`.

### The driver *is* a dry run of Phase 3

`torchlib-driver/main.cpp` is not throwaway scaffolding. It executes the exact sequence
`soh/soh/Extractor/TorchExtract.cpp` will use, so if any assumption in Phase 3 is wrong, it fails
here — offline, against a reference archive — instead of in the game with a progress bar spinning.

```
zapd-to-torch-test-harness/
  torchlib-driver/
    CMakeLists.txt
    main.cpp
```

**`CMakeLists.txt`** — note it deliberately does *not* set the `BUILD_<game>` flags or the build
type; those are Gate B's and Gate C's variables and get passed on the command line:

```cmake
cmake_minimum_required(VERSION 3.26)
project(torchlib-driver CXX)
set(CMAKE_CXX_STANDARD 20)

set(USE_STANDALONE          OFF CACHE BOOL "" FORCE)
set(PORT_VERSION_ENDIANNESS ON  CACHE BOOL "" FORCE)
set(ROM_CRC_BSWAP           OFF CACHE BOOL "" FORCE)
set(BUILD_UI                OFF CACHE BOOL "" FORCE)
set(BUILD_STORMLIB          OFF CACHE BOOL "" FORCE)

add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/../torch ${CMAKE_BINARY_DIR}/torch)
add_executable(torchlib-driver main.cpp)
target_link_libraries(torchlib-driver PRIVATE torch)
```

`torch` exports `src/`, `lib/`, and the yaml-cpp includes as `PUBLIC` under `USE_STANDALONE=OFF`
(`torch/CMakeLists.txt:437-442`), and spdlog becomes `PUBLIC` too — so nothing else is needed.

**`main.cpp`** — must be argv-compatible with `torch o2r -s S -d D -u V <rom>`, because that's
exactly what `test_assets.py:run_torch` execs (`[TORCH, "o2r", "-s", …, "-d", …, "-u", "9.2.3",
rom]`). Get that right and `TORCH_BIN` is the only thing that changes between the baseline and
every gate:

```cpp
#include <atomic>
#include <cstdio>
#include <filesystem>
#include <string>
#include "Companion.h"

static int RunOnce(const std::string& rom, const std::string& src,
                   const std::string& dest, const std::string& ver) {
    std::atomic<size_t> phases{ 0 };
    try {
        auto* c = new Companion(std::filesystem::path(rom), ArchiveType::O2R, false, src, dest);
        Companion::Instance = c;              // bare global; factories deref it unconditionally
        c->SetVersion(ver);
        c->SetPhaseCallback([&phases](int) { ++phases; });
        c->Init(ExportType::Binary);          // Init is the whole run; it calls Process() itself
    } catch (const std::exception& e) {
        fprintf(stderr, "driver: exception: %s\n", e.what());
        return 2;
    }
    if (!std::filesystem::exists(std::filesystem::path(dest) / "oot.o2r")) {
        fprintf(stderr, "driver: no oot.o2r produced\n");   // Process() returns void and has
        return 3;                                           // several log-and-return failure paths
    }
    fprintf(stderr, "driver: phases=%zu\n", phases.load());
    return 0;
}
```

…plus ~20 lines of argv parsing and, for Gate A2, a `--second <rom> <dest>` flag that calls
`RunOnce` a second time in the same process.

Four Phase 3 claims this proves or refutes:

1. **`Companion::Instance` can be assigned by hand.** It's a raw global defined at
   `src/main.cpp:15`, *outside* the `#if defined(STANDALONE)` guard — so the static lib carries the
   storage but no entry point. Deliberate, but untested from outside.
2. **`try/catch` is sufficient.** 127 `throw std::runtime_error` sites across `torch/src` and
   nothing catches at the top level; an escape is `std::terminate`, not a popup.
3. **Statting the output is necessary.** `Process()` returns `void` and just logs-and-returns on
   missing `config.yml` (`Companion.cpp:1304`), unknown ROM hash (`:1334`), no `config:` node
   (`:1390`), bad GBI (`:1484`). A silent no-archive must be caught.
4. **`SetPhaseCallback` is a usable progress denominator.** One call site
   (`gPhaseCallback(2)`, `Companion.cpp:1282-1283`), fired once per yml file at the parse→export
   transition. Compare the printed `phases=` against
   `find assets/yml/<version> -name '*.yml' | wc -l` (1,449–1,480 depending on version). If they
   match, the Phase 3 progress bar is monotonic 0→100; if they don't, the denominator theory is
   wrong and it's better to know now.

### What Gate A is hunting

**libgfxd.** This is the one that could actually fail. `torch/CMakeLists.txt:60-81` fetches and
compiles libgfxd **only `if(USE_STANDALONE)`** — under `OFF` it isn't in the build at all. Reading
the source says that's safe: every `gfxd_*` call sits inside `#ifdef STANDALONE`
(`DisplayListFactory.cpp:81-199` = `DListCodeExporter` + `DebugDisplayList`;
`DisplayListOverrides.cpp:18-224` = the `GFXDOverride` callbacks), while `DListBinaryExporter`
(`:239`) and `DListFactory::parse` (`:526`) are outside it. Display lists are the largest asset
class in the archive, so if that reading is wrong the gate fails immediately and unmistakably.

**Other `STANDALONE` deltas, all expected inert:**

- `Companion.cpp:1524` gates `enums:` parsing — `config.yml` has no `enums:` key.
- `DListCodeExporter` isn't registered — we export `Binary`.
- yaml-cpp gains `YAML_CPP_STATIC_DEFINE`; spdlog moves `PRIVATE`→`PUBLIC`; the standalone-only
  `find_package(ZLIB REQUIRED)` disappears (`zlibstatic` from FetchContent remains).

**One landmine worth writing down:**
`if(NOT USE_STANDALONE AND EXISTS "/mnt/c/WINDOWS/system32/wsl.exe")` (`torch/CMakeLists.txt:295`)
FetchContents Microsoft GSL. It fires **only inside WSL**, and only in the static-lib
configuration — i.e. exactly what Shipwright will use. Irrelevant in the distrobox; it will bite
the first person who builds SoH under WSL after this lands.

### Running it

```sh
cmake -S torchlib-driver -B torchlib-driver/build -GNinja -DCMAKE_BUILD_TYPE=Debug
cmake --build torchlib-driver/build -j
TORCH_BIN=torchlib-driver/build/torchlib-driver tools/matrix.sh static-lib
```

**Pass:** 19/19 identical, and `phases=` matches the yml count for each version.
Smoke-test `pal_gc` alone before committing to the full matrix.

---

## Gate A2 — two extractions in one process

**Variable:** re-entrancy. Free to add once the driver exists, and it retires
[`PLAN.md`](PLAN.md) risk #6 offline instead of by hand in the game.

`RunExtract` can extract more than once per process (`PS_FIRST` → `PS_SECOND`, and
`ES_EXTRACT_ARGS` loops over CLI args), so a real user extracting vanilla and then MQ in one
session hits state SoH has never exercised:

- `gProcessedFiles` (`Companion.h:300`) is **never cleared** — a second `Process()` on the *same*
  instance skips every file. Phase 3's answer is a fresh `Companion` per call; this gate is what
  proves that's enough.
- `gAddrMap` / `gParseResults` / `gWriteMap` also accumulate.
- `AliasManager::Instance` isn't cleared at the end of `Process` (believed inert for OoT — no OoT
  factory calls `Register` — but unproven).
- `AudioManager::Instance` leaks per run (`Companion.cpp:1584`).
- File-scope statics in `TextureFactory.cpp:14-15`, `CompressedTextureFactory.cpp:17-18`,
  `DisplayListFactory.cpp:82`.

This is the one gate `test_assets.py` **can't** drive — it execs the binary once per ROM, so two
extractions in one process has to come from the driver itself. Hence the `--second` flag, and a
`--pair` mode in `matrix.sh` that invokes the driver directly and then hands each of the two
outputs to `check.sh`:

```sh
DEST_A=$(mktemp -d); DEST_B=$(mktemp -d)
"$TORCH_BIN" o2r -s assets/yml -d "$DEST_A" -u 9.2.3 roms/pal_gc_0227d7.z64 \
             --second roms/pal_mq_f46239.z64 "$DEST_B"
# then, for each of (DEST_A, pal_gc_0227d7) and (DEST_B, pal_mq_f46239):
cp "$DEST/oot.o2r" o2r/torch.o2r; cp "o2r/<rom>.o2r" o2r/reference.o2r; ./check.sh
```

One process, two fresh `Companion`s, two destdirs. Deliberately a vanilla ROM then an MQ one — the
same sequence a user hits when they answer "Yes" to "extract another". **Pass:** `check.sh` prints
`PASS: All N files match!` for both — specifically for the *second*, which is the whole point.

---

## Gate B — OoT-only build

**Variable:** the eight non-OoT `BUILD_<game>` flags. Still `USE_STANDALONE=ON`, still `Debug`, so
this is measured against the preflight baseline with exactly one thing changed.

```sh
cmake -S torch -B torch/build-oot -GNinja -DCMAKE_BUILD_TYPE=Debug \
      -DPORT_VERSION_ENDIANNESS=ON \
      -DBUILD_SM64=OFF -DBUILD_MK64=OFF -DBUILD_SF64=OFF -DBUILD_PM64=OFF \
      -DBUILD_FZERO=OFF -DBUILD_BK64=OFF -DBUILD_MARIO_ARTIST=OFF -DBUILD_NAUDIO=OFF
cmake --build torch/build-oot -j
TORCH_BIN=torch/build-oot/torch tools/matrix.sh oot-only
```

Each `BUILD_X=OFF` drops `-DX_SUPPORT` and filters `src/factories/x/*` out of the source glob
(`torch/CMakeLists.txt:158-207`). Two effects are real:

- **`BUILD_BK64=OFF`** removes `BK64::TrySynthesizeRomConfig`, Companion's fallback for a ROM hash
  absent from `config.yml`. All 19 of ours are present, so it's unreachable here — but it means an
  *unknown* dump behaves differently, which is precisely why Phase 3 must stat the output rather
  than trust the run.
- **`BUILD_NAUDIO=OFF`** removes Torch's only tinyxml2 consumers. It does **not** remove the
  tinyxml2 `FetchContent` — that's unconditional — so Phase 2's `OVERRIDE_FIND_PACKAGE` concern
  stands regardless.

Nothing else should touch a shared path. A cross-game factory registration or a shared header
behind an `#ifdef` would show up as a diff, and that's what the gate is for.

---

## Gate C — Release build

**Variable:** `-DCMAKE_BUILD_TYPE=Release`. Otherwise the baseline configuration.

**This gate isn't in `PLAN.md` and should be.** The harness's 14/14 is a `Debug` measurement
(`torch/build/CMakeCache.txt`: `CMAKE_BUILD_TYPE=Debug`), and so is the local Shipwright build.
Shipwright's CI and every release build `Release` — Torch sets `CMAKE_CXX_FLAGS_RELEASE "-O3"` for
non-MSVC (`torch/CMakeLists.txt:242`). Any UB, uninitialised read, or unspecified evaluation order
in Torch can change bytes between `-g` and `-O3`.

```sh
cmake -S torch -B torch/build-release -GNinja -DCMAKE_BUILD_TYPE=Release -DPORT_VERSION_ENDIANNESS=ON
cmake --build torch/build-release -j
TORCH_BIN=torch/build-release/torch tools/matrix.sh release
```

Seven minutes to find out, versus finding out from a user whose release-build archive doesn't match
their friend's. If it fails it's an upstream Torch bug and everything else stops until it's fixed.

---

## Gate A′ — the shipping configuration

**Variable:** all of them at once — static lib + driver + OoT-only + Release.

Not implied by A ∧ A2 ∧ B ∧ C. `#ifdef` interactions are combinatorial, and this is the only
configuration that will ever ship.

```sh
cmake -S torchlib-driver -B torchlib-driver/build-ship -GNinja -DCMAKE_BUILD_TYPE=Release \
      -DBUILD_SM64=OFF -DBUILD_MK64=OFF -DBUILD_SF64=OFF -DBUILD_PM64=OFF \
      -DBUILD_FZERO=OFF -DBUILD_BK64=OFF -DBUILD_MARIO_ARTIST=OFF -DBUILD_NAUDIO=OFF
cmake --build torchlib-driver/build-ship -j
export TORCH_BIN=torchlib-driver/build-ship/torchlib-driver
tools/matrix.sh shipping
tools/matrix.sh --pair pal_gc_0227d7 pal_mq_f46239 shipping-pair   # Gate A2, shipping config
```

**When A′ is green, that flag set is the spec.** Phase 2's root-`CMakeLists.txt` block is a
transcription of it, not a re-derivation — and `main.cpp`'s `RunOnce` is a transcription into
`TorchExtract.cpp`. Copy, don't rewrite.

---

## Gate E — the `soh.o2r` baseline

Different in kind from the others: it doesn't test Torch at all. It captures the artifact the
Phase 4 packer has to reproduce, from a pipeline that is about to be deleted. **Do this while
ZAPD still builds.**

Current state: `soh.o2r` at the repo root, **1,042 entries** = 1,041 files under
`soh/assets/custom/` + `portVersion`. No `version` file — matching
`sohArchiveVersionMatch` (`OTRGlobals.cpp:282-283`), which only reads `portVersion`.

1. **Rebuild rather than trust the file on disk**, so provenance is unambiguous:
   ```sh
   cmake --build build --target GenerateSohOtr     # inside the soh distrobox
   ```
   > `GenerateSohOtr` **writes into the source tree**: it `rm -rf`s and re-copies
   > `soh/assets/custom/shaders/` from `libultraship/src/fast/shaders/` (3 files —
   > `directx/default.shader.hlsl`, `metal/default.shader.metal`, `opengl/default.shader.glsl`).
   > That directory is gitignored (`soh/assets/.gitignore:8`), which is why `git status` stays
   > clean and why the on-disk count (1,041) exceeds `git ls-files` (1,038). Phase 4's packer must
   > run after the same copy or the archives differ by exactly those 3 entries.

2. **Record the output** — `manifest.sh` already does exactly this job:
   ```sh
   ./manifest.sh /var/home/briaguya/code/Shipwright/soh.o2r manifests/soh_o2r.json
   ```
   Lives in the harness alongside the other 14 manifests. (Not in the yml repo — that holds yml and
   nothing else.)

3. **Record the inputs too**, or a future mismatch is ambiguous between "the packer is wrong" and
   "the assets changed":
   - sha256 of every file under `soh/assets/custom/` → `manifests/soh_custom_inputs.json`
   - `CMAKE_PROJECT_VERSION` = **9.2.3** (`CMakeLists.txt:9`), the `portVersion` payload
   - the `develop` SHA the archive was built from

4. **Call out the 31 files that matter.** Only these hit the conversion path — everything else is a
   byte copy, and a byte copy that fails is a trivial bug:

   | Format | Count |
   |---|---|
   | `.rgba32.png` | 15 |
   | `.ia8.png` | 8 |
   | `.ia16.png` | 5 |
   | `.ia4.png` | 3 |

   The other 1,010: 923 objects, 38 plain `.png`, 23 `.rgba16`, 22 json, 4 ttf, 3 shaders, 2 xml.
   The trap is quantisation, not decoding — `n64graphics`' `SCALE_8_4(x)` (`x*15/255`) vs ZAPD's
   `(x >> 4) & 0xF` disagree (`0xF0` → `0xE` vs `0xF`). Phase 4 hand-ports ZAPD's arithmetic; this
   manifest is what catches it if the port is wrong.

---

## Exit criteria

| Gate | Pass condition | Status |
|---|---|---|
| Preflight | 19/19 identical with the existing baseline binary; one manifest re-derivation matches | ✅ **green** — see below |
| D | pin SHA recorded; `git diff` vs upstream main still empty | ✅ green (trees equal) |
| A | 19/19 identical; `phases=` matches the per-version yml count | ✅ **green** — see below |
| A2 | both archives in a two-extraction process identical — especially the second | ✅ **green** — see below |
| B | 19/19 identical | ✅ **green** — see below |
| C | 19/19 identical | ✅ **green** — see below |
| A′ | 19/19 identical, plus the pair run | ✅ **green** — see below |
| E | `manifests/soh_o2r.json` (1,042 entries) + input hashes + version + SHA committed | ⬜ |

"Identical" means `test_assets.py` reports **`0 failed, 0 not generated, 0 not in reference`** —
not "close enough". `--failures-only` keeps the logs to the point; drop it when bisecting.

Write the results up as a second table in the harness README, next to the existing 14/14 one. That
table is the evidence the Shipwright PR points at.

### Preflight result — 2026-07-24

`TORCH_BIN=torch/build/torch tools/matrix.sh baseline`, harness `phase0-gates` off
`bump-torch-9422bf4`. **19 passed, 0 failed, of 19 ROM dumps**, every one
`0 failed, 0 not generated, 0 not in reference`.

| Version | Dumps | Assets |
|---|---|---:|
| `ntsc_1-0` | `ntsc_j_1-0_c892bb`, `ntsc_u_1-0_ad69c9` | 38,390 |
| `ntsc_1-1` | `ntsc_j_1-1_dbfc81`, `ntsc_u_1-1_d3ecb2` | 38,526 |
| `ntsc_1-2` | `ntsc_j_1-2_fa5f59`, `ntsc_u_1-2_41b3bd` | 39,032 |
| `ntsc_j_gc` | `ntsc_j_gc_0769c8` | 39,064 |
| `ntsc_j_gc_collection` | `ntsc_j_gc_collection_2ce2d1` | 39,066 |
| `ntsc_j_mq` | `ntsc_j_mq_dd14e1` | 39,026 |
| `ntsc_u_gc` | `ntsc_u_gc_b82710` | 39,066 |
| `ntsc_u_mq` | `ntsc_u_mq_8b5d13` | 39,029 |
| `pal_1-0` | `pal_1-0_328a1f` | 35,362 |
| `pal_1-1` | `pal_1-1_cfbb98` | 35,362 |
| `pal_gc` | `pal_gc_0227d7` | 35,386 |
| `pal_gc_dbg` | `pal_gc_dbg_cee6bc` | 35,646 |
| `pal_mq` | `pal_mq_f46239` | 35,352 |
| `pal_mq_dbg` | `pal_mq_dbg_079b85`, `_50bebe`, `_cfecfd` | 35,612 |

Counts match the harness README's 14-target table exactly. The **19**-dump matrix adds something
the 14-target one never showed: the duplicate dumps agree with each other — both 1.0 dumps produce
the same 38,390 assets against the same manifest, and all three `pal_mq_dbg` dumps produce the same
35,612. Since `PLAN.md`'s MQ handling keys off `IsMasterQuest()` rather than the dump, that's worth
having on record.

Reference archives spot-checked as well — `manifest.sh` re-derivation of
`o2r/ntsc_u_gc_b82710.o2r` (39,066 entries) and `o2r/pal_mq_dbg_079b85.o2r` (35,612) reproduces the
committed manifests byte-for-byte. The baseline the gates measure against hasn't drifted.

### Gate A result — 2026-07-24

`TORCH_BIN=torchlib-driver/build/torchlib-driver tools/matrix.sh static-lib`, harness
`phase0-gates` @ `504f086`. **19 passed, 0 failed, of 19 ROM dumps** — identical counts to the
preflight table above, every one `0 failed, 0 not generated, 0 not in reference`.

**`USE_STANDALONE=OFF` changes nothing about the bytes**, and driving `Companion` by hand produces
exactly what the CLI does. Confirmed rather than assumed:

- **libgfxd really is absent.** `torchlib-driver/build/_deps/` contains only `tinyxml2`, `yaml-cpp`
  and `zlib` — no `libgfxd-src`, because the `FetchContent` sits inside `if(USE_STANDALONE)`. The
  Binary export path is genuinely gfxd-free; all 19 archives extract identically without it.
- **The progress denominator is exact.** `phases=` matched `find assets/yml/<version> -name '*.yml'
  | wc -l` for all three distinct counts: `pal_gc` 1450/1450, `ntsc_u_gc` 1449/1449, `pal_gc_dbg`
  1480/1480. So Phase 3's bar is monotonic 0→100 with a denominator counted off disk, and
  `OTRGlobals.cpp:750-755` needs no change.
- **Archive size matches the CLI build's** (33,154,599 bytes for `pal_gc`). It's ~343 KB larger
  than the OTRExporter reference: 76 KB of that is Torch's duplicate archive entries (see Gate A2
  below) and the remaining ~267 KB is zip-compression difference, not content. Which is why
  comparison is always file-by-file inside the archive.
- The `Companion::Instance` hand-assignment, `SetVersion`, `SetPhaseCallback`, `Init(Binary)`
  sequence works from outside the library, so `TorchExtract.cpp` can be a transcription of
  `RunOnce()`.

**Also learned, for Phase 2:** zlib *is* fetched under `USE_STANDALONE=OFF` with
`BUILD_STORMLIB=OFF` — `_deps/zlib-src` exists despite zero `zlib.h` includes in `torch/src`,
because that `FetchContent_Declare(... OVERRIDE_FIND_PACKAGE)` is unconditional. Risk #1 is
observed, not hypothetical. Conversely spdlog was *not* fetched (`find_package(spdlog QUIET)` found
the distrobox's), so the second-copy ABI concern doesn't apply — only the global-logger stomping.

### Gate A2 result — 2026-07-24 — ✅ green

`TORCH_BIN=torchlib-driver/build/torchlib-driver tools/matrix.sh --pair pal_gc_0227d7
pal_mq_f46239 reentrancy`. One process, vanilla then MQ, a fresh `Companion` and destdir each:

| | assets | missing | extra | mismatched |
|---|---|---|---|---|
| `pal_gc` (first) | 35,386 / 35,386 | 0 | 0 | 0 |
| `pal_mq` (second) | 35,352 / 35,352 | 0 | 0 | 0 |

The load-bearing number is the **second** run's `phases=1450`, exactly `pal_mq`'s on-disk yml
count. Had `gProcessedFiles` leaked across runs the second pass would have skipped files it
believed already processed — a collapsed phase count and a truncated archive. It processed all
1,450 and reproduced the reference exactly.

**A fresh `Companion` per extraction is sufficient.** `AliasManager::Instance` not being cleared,
`AudioManager::Instance` leaking, and the file-scope statics in `TextureFactory.cpp:14-15`,
`CompressedTextureFactory.cpp:17-18`, `DisplayListFactory.cpp:82` are all confirmed inert for
OoT/Binary. [`PLAN.md`](PLAN.md) risk #6 is retired offline; the in-game two-ROM session (§9 of
the verification matrix) is now a confirmation, not a discovery.

#### Found along the way: Torch writes duplicate archive entries

The first attempt failed both comparisons — not on parity, but because `check.sh` never got to
compare. `unzip` had aborted mid-extract.

**Torch writes some assets more than once under the same archive path.** `pal_mq`: 35,377 physical
zip entries for 35,352 unique names; 24 names duplicated (23 cutscenes + `link_home_room_0Back
ground_002480`, which appears three times) = 25 redundant entries. `pal_gc`: also 24. Every copy is
byte-identical to its siblings. All 19 OTRExporter reference archives have **zero**. Identical
under `USE_STANDALONE` ON and OFF, so it is not a static-lib artifact — pre-existing Torch
behavior, and Gate A's conclusions are unaffected.

Zip permits this: the format is a flat sequence of member records plus a central directory, with
the path as an ordinary string field and no uniqueness constraint. Only a reader materializing to
a real filesystem has to reconcile them — hence `unzip` prompting `replace ...?`, reading EOF from
a non-interactive stdin, and exiting non-zero into `set -e`.

**Benign for SoH.** `O2rArchive::LoadFile` resolves via `zip_name_locate`, which returns one entry
index, and the copies are byte-identical. `O2rArchive::Open` is the only place LUS walks entries by
raw index, and it only feeds `Archive::IndexFile`, which does `(*mHashes)[CRC64(path)] = path` — a
repeat insert is an idempotent overwrite. `ListFiles`/`HasFile` derive from that same map. So the
duplicates collapse at the archive's front door: no double-loading, no double-counting, no
duplicate list entries. Cost is 76 KB compressed on a 33 MB archive.

Filed as item 5 of [HarbourMasters/Torch#233](https://github.com/HarbourMasters/Torch/issues/233#issuecomment-5076597596)
for post-migration cleanup. **Not a blocker and not something the migration needs to fix.**

The lasting lesson is about the *harness*, not Torch: a comparison keyed on archive path — which is
both `test_assets.py` and `check.sh` — structurally cannot see duplicate entries, because they
collapse into one key. Asset parity genuinely holds (identical unique-path sets, identical bytes at
every path), but "19/19 identical" was never a statement about the zip container. `check.sh` now
extracts with `unzip -o` and prints a duplicate-name count for both archives so this can't go quiet
again.

### Gates B and C result — 2026-07-24 — ✅ both green

Run back to back; builds in the `soh` distrobox, matrices on the host. **19/19 each**, every dump
`0 failed, 0 not generated, 0 not in reference`, asset counts identical to the preflight table.

The `option()` defaults matter for the claim that each gate moves exactly one variable:
`BUILD_UI`, `BUILD_STORMLIB` and `ROM_CRC_BSWAP` all default `OFF` and so already match the
baseline cache; only `PORT_VERSION_ENDIANNESS` (default `OFF`) has to be passed explicitly. So
Gate B differs from the preflight baseline in the eight `BUILD_<game>` flags and nothing else, and
Gate C in `CMAKE_BUILD_TYPE` and nothing else.

**Gate B — OoT-only.** No shared path is touched by dropping the other eight games. In particular
`BUILD_BK64=OFF` removing `BK64::TrySynthesizeRomConfig` changes nothing for our 19 dumps, all of
which are in `config.yml` — but it does mean an *unknown* dump behaves differently, which is why
Phase 3 must stat the output rather than trust the run.

**Gate C — Release.** No UB, uninitialised read, or evaluation-order dependence in Torch changes
bytes between `-g` and `-O3`. This is the gate that wasn't in `PLAN.md`, and it was worth adding:
every prior parity measurement, including the harness's original 14/14, was a `Debug` measurement,
while CI and every release build `-O3`.

Builds were verified real rather than assumed — the three binaries are distinct, freshly
timestamped, and sized as expected (baseline Debug all-games 125 MB, OoT-only Debug 49 MB,
Release 7.2 MB). They configure fast because Fedora puts `/usr/lib64/ccache` on `PATH`; ccache
emits identical objects, so this doesn't weaken the result.

#### Bonus: extraction wall-clock, from Torch's own `Took NNNNms`

| Build | Mean per ROM | vs baseline |
|---|---|---|
| baseline — Debug, all games | 26.7 s | — |
| Gate B — Debug, OoT-only | 26.2 s | unchanged |
| Gate C — **Release** | **11.4 s** | **2.3× faster** |

Directly relevant to [`PLAN.md`](PLAN.md) risk #5. SoH ships Release, so in-game extraction is a
~11 s operation on this machine, not the ~27 s every Debug measurement so far has implied — and
the OoT-only flags cost nothing. What's still unmeasured is **ZAPD's** time on the same machine,
so "does it regress?" remains formally open; but at ~11 s the answer is unlikely to need a
"this takes N minutes" note in the UI.

### Gate A′ result — 2026-07-24 — ✅ green

The shipping configuration, all variables at once. **19/19 identical**, plus the pair run
**PASS/PASS**. Verified as genuinely that combination, not assumed:

```
USE_STANDALONE=OFF   CMAKE_BUILD_TYPE=Release   BUILD_OOT=ON   PORT_VERSION_ENDIANNESS=ON
BUILD_{SM64,MK64,SF64,PM64,FZERO,BK64,MARIO_ARTIST,NAUDIO}=OFF   BUILD_UI=OFF
BUILD_STORMLIB=OFF   ROM_CRC_BSWAP=OFF
```

**That flag set is now the spec.** Phase 2's root-`CMakeLists.txt` block is a transcription of it
and `main.cpp`'s `RunOnce` a transcription into `TorchExtract.cpp` — copy, don't re-derive.

- **libgfxd is absent here too**, in the exact combination that ships: `_deps/` holds only
  tinyxml2, yaml-cpp and zlib. The Binary export path is gfxd-free under Release as well as Debug.
- **Archive bytes are configuration-independent.** The pair produced 33,154,599 (`pal_gc`) and
  33,120,244 (`pal_mq`) — identical to the Debug static-lib run in Gate A2, and both runs reported
  `phases=1450` including the second. Debug/Release and CLI/static-lib all agree.
- **Extraction: 11.8 s mean per ROM**, matching Gate C's Release number. The static-lib wrapper
  costs nothing.

**Both Phase 2 dependency risks are now confirmed in the shipping configuration, not merely
predicted:**

- **zlib is fetched** under `USE_STANDALONE=OFF` + `BUILD_STORMLIB=OFF` despite zero `zlib.h`
  includes in `torch/src` (risk #1).
- **tinyxml2 is fetched** even though `BUILD_NAUDIO=OFF` removes its only Torch consumers — the
  `FetchContent_Declare(... OVERRIDE_FIND_PACKAGE)` is unconditional, so it will hijack
  libultraship's `find_package(tinyxml2 REQUIRED)` regardless.

Neither affects archive bytes; both are link/configure concerns for Phase 2, and both are now
observed facts rather than readings of the CMake.

---

## Cost

| | Build | Matrix |
|---|---|---|
| Preflight | — | ~7 min |
| A | full Torch rebuild | ~7 min |
| A2 | — (reuses A) | ~1 min |
| B | full Torch rebuild | ~7 min |
| C | full Torch rebuild | ~7 min |
| A′ | full Torch rebuild | ~8 min |
| E | Shipwright `GenerateSohOtr` | ~1 min |

An evening, dominated by four full Torch builds. Four extra build trees; the `Debug` Torch binary
alone is 125 MB, so either build them sequentially and delete as you go or budget a few GB.

---

## What Phase 0 deliberately does **not** cover

Being explicit so these don't get mistaken for retired risk:

- **Linking Torch into Shipwright.** The zlib / stb / `StringHelper` / `StrHash64` / tinyxml2 /
  `storm` collisions in `PLAN.md` Phase 2 need SoH's own build to surface. A standalone driver
  links Torch against nothing else, so it can't see any of them. That's Phase 2's first day.
- **spdlog global-state stomping.** Torch calls `spdlog::set_level`/`set_pattern` on the *default*
  logger (`Companion.cpp:193-194, 1810-1814`), and our `config.yml` sets `logging: CRITICAL` — so
  this fires on every run, it just doesn't matter until there's a SoH logger to clobber.
- **Windows and macOS.** Every gate runs Linux/gcc in the distrobox. Torch forces `/MT` under MSVC,
  adds `-DSTORMLIB_NO_AUTO_LINK` and `/bigobj`, and carries a `cmake_minimum_required(3.12)` +
  `CMAKE_POLICY_VERSION_MINIMUM` shim under a CMake 4 host. If a Windows machine is available,
  running Gate A′ there is the cheapest possible insurance on the one platform that can't be
  predicted from reading — but it isn't a blocker for starting Phase 2.
- **Anything in-game.** The driver proves the bytes; it says nothing about the ImGui progress bar,
  the ROM picker, or `ValidHashes`. That's the Phase 3 manual matrix.