diff --git a/phase-ends/DIGEST.md b/phase-ends/DIGEST.md index dc6d662..b6791f6 100644 --- a/phase-ends/DIGEST.md +++ b/phase-ends/DIGEST.md @@ -23,3 +23,7 @@ Phase 4 replaced the data-only representation with a documented, ROM-safe assemb ## Phase 5 — Original Toolchain Identification and First C Match (2026-09-23) Phase 5 identified the original build toolchain from byte evidence and produced the project's first C match. The compiler is **`egcs-2.91.66 19990314` (egcs-1.1.2), target `mips-sony-psx`** — the `CC1PSX.EXE` of PsyQ SDK 4.5, matching the executable's loader-detected SDK signature. It was confirmed both from the SDK's own README banner and by executing the real SDK binary (under the ignored `wibo` Win32 loader) and comparing output; the open `decompals/old-gcc` 0.17 `gcc-2.91.66-psx` build is instruction-identical to it across twelve probe files (~990 instructions), so no proprietary compiler is needed. Working flags are `-quiet -O2 -G0 -mno-split-addresses` with a mandatory `cpp` preprocessing stage (`cc1` rejects comments and directives); the SDK assembler is ASPSX 2.81, with `maspsx` the open emulator (not yet wired in). An earlier in-phase selection (`gcc-2.7.2-psx`) was wrong — an artifact of the invalid `-mcpu=3000` spelling, which perturbs only `2.91.66` — and was corrected with its cause recorded. The Phase 4 ordered workflow was promoted into tracked tooling as `tools/sf3_match` (`range`/`plan`/`build`/`gate`) driven by the tracked registry `config/regions.tsv`, with `make test` and `make check` added. The plan's intended first target, the entry `[0x800FB368,0x800FB410)`, was determined to be CRT startup rather than compiler output (`ra` round-tripped through an absolute global, `break` termination, `sp` built from linker globals, clear loop falling through with no `jr ra`); two bounded compile experiments failed and were stopped, and no C is claimed for it. The first match is `func_80017AD4` (`0x80017AD4..0x80017AE8`, 20 bytes, `src/func_80017AD4.c`): instruction-identical and `make gate` green at `c_regions=1`, 0 differing bytes, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`. A duplicate-sharing mechanism (N registry rows to one source, enabled by symbol-localizing region objects) is documented and tested. The clean Phase 3 data baseline is unaffected. Final gates: 53 synthetic tests pass; `make clean`, `make all`, `cmp` and SHA-1 green; 57 tracked files with none under a prohibited root. Knowledge is recorded in `docs/MATCHING_COOKBOOK.md` (nine byte-proven findings), `docs/MATCHING_CONVENTIONS.md`, `docs/PHASE5_*.md`, and a new `README.md`. Unresolved: registry symbol support, `maspsx`/ASPSX `la` behaviour (GNU `as` expands `la` with `ori` where the original uses `addiu`), per-region flag overrides, the `-G` small-data threshold, broader function segmentation, and the `0x8005DEF8` constant-multiply reconstruction. No rules were added. + +## Phase 6 — From One Match to a Matching Pipeline (2026-09-23) + +Phase 6 turned the single match into a pipeline and corrected the project's compiler identification. The harness gained a tracked symbol registry (`config/symbols.tsv`) resolved at **link** time, per-region compiler/assembler flag overrides in `config/regions.tsv`, a **maspsx** ASPSX-emulation stage between `cc1` and `as`, and a per-symbol `gp` marker that forces explicit `%gp_rel` for small-data accesses. A tracked, synthetic-tested `tools/sf3_boundaries` produces `config/function_inventory.tsv`: 2,875 candidate function starts graded `entry`/`jal`/`prologue`/`ghidra` (addresses only), of which 575 `jal` targets are absent from Ghidra's function set; library-versus-game-code is explicitly unresolved. Trying to match a framed function exposed that Phase 5's compiler (`egcs-2.91.66`, PsyQ 4.5) could not produce the game's epilogues (322 framed functions end `lw ra` / `addiu sp,sp,N` / `jr ra` / `nop`). The real PsyQ **4.0** and 4.1 SDKs were obtained (ignored `tools/psyq/`; checksums in `docs/SETUP.md`), and the original compiler is **PsyQ 4.0 `CC1PSX`** (`GNU C 2.7.2.SN32.3.7.0002`), for which the open `gcc-2.7.2-psx` is instruction-identical across **21/21** probe files (next best 6); the SDK 4.0 assembler is **ASPSX 2.56**, and Phase 5's `-mno-split-addresses` requirement was an artifact of the wrong compiler. The loader's `PsyQ Version = 4.5.0` is the runtime library version. The batch is **12 registered regions / 11 distinct functions** across three shapes — leaf getter/setter, `la`/`gp`-relative, and call with a frame — with duplicate sharing demonstrated on a real body that occurs exactly twice (`0x800262E0`/`0x800262EC`). The Ghidra-draft workflow (decompiler skeleton → first-draft C → byte gate) was adopted and documented; it reproduced the previously unexplained real `mult` by 68 in `0x8005DEF8` via a non-const local (`short k = 68;`), reaching a 5-byte near-match that is deliberately not registered. Final gates: 86 synthetic tests pass; `make clean`, `make all`, `cmp` and SHA-1 green; `make gate` `c_regions=12`, 0 differing bytes, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`; 0 tracked paths under any prohibited root. Unresolved and recorded: `0x8005DEF8` (5-byte register tie-break), `0x800F3160` (store-in-delay-slot scheduling), the numeric `-G`, the CRT entry, library-versus-game-code, and broader segmentation. No rules were added. diff --git a/phase-ends/PhaseEnd_Phase6.md b/phase-ends/PhaseEnd_Phase6.md new file mode 100644 index 0000000..64cf15d --- /dev/null +++ b/phase-ends/PhaseEnd_Phase6.md @@ -0,0 +1,116 @@ +# PhaseEnd — Phase 6: From One Match to a Matching Pipeline + +**Date:** 2026-09-23 +**Phase Status:** Complete +**Milestone confirmed by developer:** yes + +## Completed Checklist + +- P6-T1 — Created the Phase 6 control record, revalidated the clean baseline, and triaged the Phase 5 + open items into class-blocking versus single-function. +- P6-T2 — Added the tracked symbol registry and per-region flag overrides; registered `0x8002D2A0`. +- P6-T3 — Wired `maspsx` between `cc1` and `as` and moved symbol resolution to the linker; registered + `0x8002D2BC`. +- P6-T4 — Reproduced a `gp`-relative function and recorded the small-data evidence; registered + `0x80012780`. +- Rules check — Re-read `AGENTS.md` mandatory behavior after P6-T4 and stated the required notice. +- P6-T5 — Added the tracked evidence-graded function-boundary inventory (`tools/sf3_boundaries`, + `config/function_inventory.tsv`). +- P6-T6 — Matched the batch (12 regions / 11 distinct functions, three shapes), demonstrated + duplicate sharing on a real shared body, and — on developer direction — corrected the compiler + identification and adopted the Ghidra-draft workflow. +- P6-T7 — Wrote the verification record, ran every clean gate, and obtained developer confirmation. + +## Verified Results + +### Matching batch + +**12 registered regions / 11 distinct functions**, `make gate` → `c_regions=12`, 0 differing bytes, +SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +| Shape | Functions | +|---|---| +| leaf getter/setter | `func_80012780`, `func_80017AD4`, `func_80017AE8`, `func_80026264`, `func_800262E0` | +| `la` / `gp`-relative | `func_8002D2A0`, `func_8002D2BC`, `func_80012780` | +| call with a frame | `func_80017DD0`, `func_80024C14`, `func_80036308`, `func_800697A4` | +| duplicate body | `func_800262E0` / `func_800262EC` (one source; the body occurs exactly twice) | + +### Toolchain corrected + +The Phase 5 identification (`egcs-2.91.66`, PsyQ 4.5) did not explain the executable. The original +compiler is **PsyQ 4.0 `CC1PSX`** (`GNU C 2.7.2.SN32.3.7.0002`); the open `gcc-2.7.2-psx` is +instruction-identical to it across **all 21 probe files** (the next best candidate matched 6). The SDK +4.0 assembler is **ASPSX 2.56**. The loader's `PsyQ Version = 4.5.0` is the runtime library version. +Phase 5's `-mno-split-addresses` requirement was an artifact of the wrong compiler. + +### Toolchain gaps closed + +| Gap | Closure | +|---|---| +| registry symbols | `config/symbols.tsv`; link-time `--defsym` and HI16 carry adjustment | +| per-region flags | optional fourth field in `config/regions.tsv` | +| `maspsx` / ASPSX `la` | maspsx stage between `cc1` and `as` (`--aspsx-version=2.56`) | +| `-G` small data | per-symbol `gp` marker and explicit `%gp_rel` rewrite | + +### Inventory and workflow + +- `tools/sf3_boundaries` + `config/function_inventory.tsv`: 2,875 candidates graded + `entry`/`jal`/`prologue`/`ghidra` (addresses only). 575 `jal` targets are absent from Ghidra's + function set. +- Ghidra-draft workflow adopted: decompiler skeleton → first-draft C → byte gate. See + `docs/PHASE6_GHIDRA_WORKFLOW.md`. + +## Deviations and Bounded Results + +| Item | Plan | Actual | Reason / limit | +|---|---|---|---| +| Compiler identity | carried from Phase 5 | **corrected to PsyQ 4.0** | framed epilogues (322 functions) and the real-SDK comparison; Phase 5's oracles did not discriminate | +| Framed shape | required in the batch | delivered after the correction | was blocked until the compiler was fixed | +| `0x8005DEF8` | re-attempt once | **near-match, 5 bytes differ** | real `mult` by 68 reproduced via a non-const local (`short k = 68;`); residual is a v0/v1 register tie-break; not registered | +| `0x800F3160` | — | not reproduced | its store-in-delay-slot scheduling matches no tested compiler+assembler pair | +| `-G` threshold | determine numerically | **not recoverable** | the choice is per object size/section, unobservable; per-symbol `gp` form used instead | +| Library vs game code | mark unresolved | unresolved | no evidence class separates them | + +## Verification and Firewall + +- `make clean` exit 0; `make all` exit 0; `cmp` exit 0; both files SHA-1 + `e173426c157384ebf1b6caf8c6fea18a85a14af9`. +- `make gate` with 12 regions: `c_regions=12`, `differing_bytes=0`, `result=MATCH`. +- `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests` — exit 0; **86 tests pass** + (33 added this phase). +- 0 tracked paths under any prohibited root; `git diff --check` exit 0. +- No game bytes, disassembly, generated assembly, build outputs, expected binaries, Ghidra material, + dumps, or proprietary SDK material are tracked. The PsyQ 4.0/4.1/4.4/4.5/4.6 SDKs and `wibo` remain + ignored; only checksums and provenance are in `docs/SETUP.md`. + +## Rules Added This Phase + +None. + +## Next + +Phase 6 is closed. A future phase must begin in a fresh session with an approved task-by-task plan. It +should preserve the all-payload data baseline, the ordered-layout harness, the corrected toolchain, and +the byte gate, and must treat the following as unresolved: `0x8005DEF8` (5-byte register tie-break), +`0x800F3160` (scheduling), the numeric `-G`, the CRT entry, library-versus-game-code, and broader +function segmentation beyond the 2,875 graded candidates. + +## Plain-English Recap + +Phase 6 turned a single matched function into a working pipeline and then used that pipeline to find a +mistake in the project's earlier conclusion. It added a symbol registry, per-region compiler flags, an +ASPSX emulation stage, and a `gp`-relative mechanism, then matched eleven functions across three +shapes — leaf setters, address/small-data accesses, and calls with a stack frame — with one shared body +registered twice. While trying to match a framed function it became clear that the compiler identified +in Phase 5 could not produce the game's epilogues; downloading the real PsyQ 4.0 and 4.1 SDKs settled +it: the game was built with PsyQ 4.0's compiler, which an open build reproduces instruction-for- +instruction across every probe. The phase also replaced guesswork about function boundaries with a +graded inventory, and adopted a workflow where Ghidra drafts a function's shape and the byte gate +decides whether it is right. Two functions remain unmatched for precise, recorded reasons — one differs +by five bytes of register allocation, one by an assembler scheduling quirk — and the whole executable +still rebuilds byte-for-byte. + +## 🛑 Stop Here + +`CURRENT_PHASE.md` is archived as `phase-ends/logs/Phase6.md`; this PhaseEnd and the digest update +belong in the closure commit. Do not begin Phase 7 in this session. diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/logs/Phase6.md similarity index 100% rename from phase-ends/CURRENT_PHASE.md rename to phase-ends/logs/Phase6.md