From e4163e1ecdae9f1e9e1d2235ae6d14d2cc9d222c Mon Sep 17 00:00:00 2001 From: briaguya <70942617+briaguya0@users.noreply.github.com> Date: Fri, 24 Jul 2026 21:55:57 -0400 Subject: [PATCH] Phase 0: flesh out the verification gates Adds torch-migration/PHASE0.md with the full gate detail: what each gate's variable is, the driver source, exact commands, cost, and exit criteria. Two gates the plan was missing: - Gate C (Release build) - the harness 14/14 is a Debug measurement; CI and releases build -O3. - Gate A2 (two extractions in one process) - free once the driver exists, retires the re-entrancy risk offline instead of by hand in the game. Also: gates run full-tree + check.sh rather than test_assets.py (which hardcodes the torch path and copies a filtered yml subset), 19 ROM dumps not 17, and libgfxd being USE_STANDALONE-only is called out as the real hazard in Gate A. --- torch-migration/PHASE0.md | 460 ++++++++++++++++++++++++++++++++++++++ torch-migration/PLAN.md | 69 +++--- 2 files changed, 503 insertions(+), 26 deletions(-) create mode 100644 torch-migration/PHASE0.md diff --git a/torch-migration/PHASE0.md b/torch-migration/PHASE0.md new file mode 100644 index 0000000000..b36c93192c --- /dev/null +++ b/torch-migration/PHASE0.md @@ -0,0 +1,460 @@ +# Phase 0 — Verification gates + +Detail for [`PLAN.md`](PLAN.md) Phase 0. **Nothing in Shipwright changes until every gate here is +green.** All work happens in +[`briaguya0/zapd-to-torch-test-harness`](https://github.com/briaguya0/zapd-to-torch-test-harness). + +--- + +## Why there are gates at all + +The harness proves 14/14 byte-for-byte parity — but for exactly **one** build of Torch, driven +exactly **one** way: + +| | Harness measured | Shipwright will use | +|---|---|---| +| Build kind | executable (`USE_STANDALONE=ON`) | static lib (`OFF`) | +| Games compiled in | all 9 (defaults) | OoT only | +| Build type | `Debug` (`-g`) | `Release` (`-O3`) | +| Driver | `main.cpp` + CLI11 | our own `Companion` calls | +| Extractions per process | one | up to two (vanilla, then MQ) | +| `soh.o2r` | n/a (OTRExporter makes it) | a new in-tree packer | + +Every row is an unproven variable. Each gate closes exactly one of them, **one at a time**, so a +failure names its own cause. Running only the final combination would tell us "something broke" +without saying what. + +Reading the source says all six should be fine. Reading is how we got here; running is how we know. + +--- + +## Preconditions + +- Everything builds inside the **`soh` distrobox** (`distrobox enter soh`) — `cmake`, `ninja`, and + the compilers are not on the host `PATH`. +- Harness at `~/code/zapd-to-torch-test-harness`, submodules initialised. +- `roms/` populated: **19 ROM dumps → 14 version directories** (`ntsc_1-0`, `ntsc_1-1` and + `ntsc_1-2` have two dumps each; `pal_mq_dbg` has three). + *(`PLAN.md` says 17 hashes in a couple of places — it's 19. Corrected there.)* +- `assets/yml/` generated: 20,353 `.yml` / 119 MB across the 14 version dirs, plus the committed + `config.yml`. Gitignored — regenerate with `zapd_to_torch.py` if absent. +- `o2r/` holds all 19 OTRExporter reference archives (one per ROM dump, from Shipwright `95d8f7e`). +- Existing baseline build at `torch/build/` (`USE_STANDALONE=ON`, all games ON, `Debug`, + `PORT_VERSION_ENDIANNESS=ON`) — leave it alone; it *is* the control. + +--- + +## How the gates are actually run + +**Do not use `tools/test_assets.py` for these gates.** It hardcodes `TORCH` to +`torch/build/torch`, and it copies a *filtered subset* of the yml into a scratch dir (1,320 of +1,450 files for `pal_gc`) — which is not the invocation SoH will make. It's the right tool for +bisecting a single failing asset, not for proving a configuration. + +Use the **full-tree + `check.sh`** path instead. It points Torch at the entire yml tree — the same +shape as Phase 1's `srcdir = assets/` — and diffs the two archives file-by-file: + +```sh +OUT=$(mktemp -d) # fresh every run — see below +"$TORCH_BIN" o2r -s assets/yml -d "$OUT" -u 9.2.3 roms/.z64 +cp "$OUT/oot.o2r" o2r/torch.o2r +cp "o2r/.o2r" o2r/reference.o2r +./check.sh # missing / extra / mismatched, all three +``` + +`check.sh` never invokes Torch, so it takes any binary — Gate A's driver drops straight in with no +harness patching. + +> **The fresh destdir is mandatory, not hygiene.** `Process()` writes `destdir/torch.hash.yml` and +> reads it back on the next run to skip unchanged files. Reuse a destdir and the second run +> silently produces a partial archive. (This is the same constraint that forces Phase 3's +> `Mkdtemp()`.) + +### New: `tools/matrix.sh` + +One script, used by every gate: + +```sh +tools/matrix.sh