diff --git a/README.md b/README.md index c4bf127..7c1d7ac 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,10 @@ byte**. instruction-identical to the original and the whole rebuilt executable must be SHA-1 identical. Nothing "functionally equivalent" counts, and unmatched C never enters the default build. -> **Status: early.** The project is in Phase 5. The executable already rebuilds byte-identically from -> an all-assembly baseline, and the original compiler has been identified from byte evidence. -> **No function has been matched to C yet**, and no C is in the default build. +> **Status: early.** The project is in Phase 5. The executable rebuilds byte-identically, the +> original compiler has been identified from byte evidence, and the first C function has been matched +> and gated. The default build is still almost entirely an assembly/data baseline: **one function out +> of roughly 1,700 is C.** ## Where it stands @@ -21,8 +22,9 @@ Nothing "functionally equivalent" counts, and unmatched C never enters the defau | All-assembly byte-identical rebuild of the executable | done (Phase 3–4) | | Address-ordered code-recovery path (entry as real MIPS) | done (Phase 4) | | Original compiler identified from byte evidence | done (Phase 5) | +| Matching harness and full-binary gate | done (Phase 5) | | Code/function segmentation | entry range only | -| Functions matched to C | **0** (Ghidra's analyzer reports ~1,721 function candidates) | +| Functions matched to C | **1** — `func_80017AD4`, byte-identical and gated (Ghidra's analyzer reports ~1,721 function candidates) | The validated target is the USA executable `SCUS_946.40;1`: diff --git a/config/regions.tsv b/config/regions.tsv index 5b51a93..4552f11 100644 --- a/config/regions.tsv +++ b/config/regions.tsv @@ -1,3 +1,9 @@ # Code-region registry: one C region per matched function. # Columns: startendsource (addresses are hex) -# Empty means the build is the all-payload data baseline. +# +# A region's compiled .text must be exactly the declared length, and the whole +# executable must stay byte-identical (see `make gate`). An empty registry means +# the build is the all-payload data baseline and contains no C. +# +# Matched so far: +0x80017AD4 0x80017AE8 src/func_80017AD4.c diff --git a/docs/PHASE5_FIRST_MATCH.md b/docs/PHASE5_FIRST_MATCH.md new file mode 100644 index 0000000..9fe0a86 --- /dev/null +++ b/docs/PHASE5_FIRST_MATCH.md @@ -0,0 +1,84 @@ +# Phase 5 — Entry Determination and First C Match + +**Scope:** the isolated entry experiment and the project's first byte-identical C function match. +**Task:** P5-T5 +**Status:** complete. One function is matched; the entry is determined **not** to be compiler output and stays fallback. + +## Part A — the entry is CRT startup, not C + +The Phase 5 plan named the entry `[0x800FB368, 0x800FB410)` as the first C target. The bounded +experiment says it is not compiler output, and no C is claimed for it. + +**Structural evidence** (all from the validated executable): + +| Observation | Why it is not compiler output | +|---|---| +| The return address is saved to and reloaded from an absolute global (`sw ra,…(at)` before the first call, `lw ra,…(ra)` after it) | A compiler spills `ra` into the function's own stack frame, never through a global round-trip. | +| `break 0x0,0x1` terminates the range | Program termination, not a C return. | +| The stack pointer is built from two linker-provided words with an `or` of `0x80000000` | Linker/BIOS memory layout, not a source-level expression. | +| `gp` is initialized from an absolute address and `s8` is set to `sp` | Startup conventions. | +| The clear loop falls through with no `jr ra` | It is inline code inside a larger routine, not a callable function. | + +**Bounded compile experiments** (two, then stopped): + +1. The clear-loop sub-range `[0x800FB368, 0x800FB38C)` from a constant-address `for` loop — + `LENGTH-MISMATCH` (the candidate is a function with a return; the original is not). +2. The same range from `extern` symbols — again a different shape: an up-front empty-range guard, + `addu` instead of the original's `addiu`, `bne` instead of `bnez`, and a trailing `jr ra`. + +**Conclusion:** the entry is PsyQ `crt0`-style startup. It remains in the data fallback and is not +registered as a C region. Its true origin (hand-written assembly assembled by ASPSX) is a hypothesis +consistent with the evidence above, not a proven fact. + +## Part B — the first C match + +| | | +|---|---| +| Function | `func_80017AD4` | +| Range | `0x80017AD4..0x80017AE8` (20 bytes) | +| Source | `src/func_80017AD4.c` | +| Registry | `config/regions.tsv` | +| Instruction comparison | byte-identical (`sf3_match range`, 0 differing bytes) | +| Full-binary gate | `make gate` → `c_regions=1`, 0 differing bytes, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9` | +| Clean rebuild | `make clean`, `make all`, `cmp` and SHA-1 all green (baseline unaffected) | + +The function stores a boolean derived from an **unsigned `char`** argument and then a second +argument, with the compiler scheduling the second store into the `jr ra` delay slot. It was chosen +because it exercises the original's default unsigned-`char` ABI and the delay-slot filler on a body +that is unique in the executable. + +**Duplicate and reference review (before matching):** + +- The 20-byte body occurs exactly once in the payload (`0x80017AD4`); no duplicate function body, so + no rule-8 sharing is required for this match. +- One direct caller: `jal 0x80017AD4` from `0x80014C14`. +- A near-miss was checked and rejected as a duplicate: `0x80012DAC` shares the same three-store tail + but is the tail of `FUN_80012d8c`, not a separate function. + +Two further reconstructions were also verified byte-identical by `sf3_match range` but are **not yet +registered**: + +- `0x8002D2A0` (28 bytes, `return (unsigned)(g - a) >= (unsigned)b;`) — needs a symbol definition, and + the registry has no symbol support yet. +- `0x80017AE8` (16 bytes, three stores) — verified, deliberately left for the next batch. + +## Codegen findings (cookbook seeds) + +Verified during this task; to be formalised in P5-T6. + +| # | Finding | Basis | +|---|---|---| +| 1 | `-mno-split-addresses` is required. It makes `cc1` emit assembler **macros** (`lw $2,sym`, `sw $4,sym`, `la $2,sym`); the default `-msplit-addresses` emits explicit `lui $3,%hi` / `lw $2,%lo($3)` sequences that do **not** match. | Real `CC1PSX` and `gcc-2.91.66-psx`; oracle ranges. | +| 2 | `lw rt,sym` expands using `rt` itself as the address scratch (`lui v0,…; lw v0,…`), which is what the original does. | `0x80085B80` byte-identical. | +| 3 | `sw rt,sym` cannot clobber its source, so the assembler expands it through `$at` (`lui at,…; sw a0,…`), which is what the original does. | `0x800F3160` byte-identical. | +| 4 | The `la` macro differs between assemblers: **GNU `as` expands it with `ori`**, while the original (ASPSX 2.81) uses **`addiu`** (`%lo`, sign-extended). Any function that materialises a symbol *address* will therefore mismatch until `maspsx` is wired in. | Direct comparison of `la $3,__bss_start` output against the original's clear loop. | +| 5 | `cc1` consumes **preprocessed** input only; comments and `#` directives must be removed by `cpp` first. | Observed parse failures. | +| 6 | A constant address is materialised with `lui`+`ori`, whereas a *symbol* address uses the macro/`%hi`/`%lo` split — so a probe written with a literal address will not reproduce the original's `addiu` form. | Clear-loop experiments 1 and 2. | + +## Limits + +- The function name, struct name and all field names are hypotheses reconstructed from disassembly; + only the compiled bytes are evidence. +- Only 20 bytes of the executable are C. Everything else remains the data fallback. +- The registry has no symbol support, so symbol-referencing functions cannot be registered yet. +- `maspsx` is not wired into the harness; `la`-using functions are expected to mismatch (finding 4). diff --git a/docs/PHASE5_MATCH_HARNESS.md b/docs/PHASE5_MATCH_HARNESS.md index 83db542..9c06c6d 100644 --- a/docs/PHASE5_MATCH_HARNESS.md +++ b/docs/PHASE5_MATCH_HARNESS.md @@ -89,8 +89,8 @@ The harness defaults to the identified toolchain and flags, all overridable: ## Verification -- `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v` — exit 0; 50 synthetic - tests pass (22 of them new: header/registry/layout/compare logic, plus end-to-end tests that build +- `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v` — exit 0; 52 synthetic + tests pass (24 of them new: header/registry/layout/compare/summary logic, plus end-to-end tests that build a synthetic executable from real compiler output and skip cleanly when the ignored toolchain is absent). - The harness **fails safely** on deliberately incorrect input: a corrupted candidate yields diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index 54665f7..b1f19b4 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -2,8 +2,8 @@ **Phase:** Phase 5 — Original Toolchain Identification and First C Match **Plan:** `phase-ends/Phase5_PLAN.md` -**Status:** active; P5-T4 complete — the matching harness and ordered-layout gate are tracked and green -**Active task:** P5-T4 complete; next is P5-T5 (first isolated C match) +**Status:** active; P5-T5 complete — the entry is CRT startup (not C) and the first C match is byte-identical +**Active task:** P5-T5 complete; next is P5-T6 (cookbook, registry conventions, phase gate) > Approval note: `Phase5_PLAN.md` was authored as a DRAFT requiring explicit developer > approval. The developer's instruction to begin Phase 5 is being treated as that approval @@ -52,11 +52,27 @@ address-ordered executable from the tracked region registry `config/regions.tsv` The Phase 4 ordered workflow is now tracked tooling rather than an ignored experiment. +## P5-T5 completed evidence + +`docs/PHASE5_FIRST_MATCH.md` records both halves. + +- **Entry `[0x800FB368,0x800FB410)` is not compiler output.** The return address is round-tripped + through an absolute global around the first call, the range ends in `break`, the stack pointer is + built from linker globals, and the clear loop falls through with no `jr ra`. Two bounded compile + experiments failed and were stopped. The entry stays fallback; no C is claimed for it. +- **First C match: `func_80017AD4`, `0x80017AD4..0x80017AE8` (20 bytes), `src/func_80017AD4.c`.** + Byte-identical instruction range and `make gate` → `c_regions=1`, 0 differing bytes, SHA-1 + `e173426c157384ebf1b6caf8c6fea18a85a14af9`. Duplicate review: the body is unique, one caller at + `0x80014C14`. The clean Phase 3 baseline rebuild is unaffected and still green. +- Two more reconstructions were verified byte-identical by `sf3_match range` but are not yet + registered: `0x8002D2A0` (needs symbol support) and `0x80017AE8`. +- Six codegen findings were recorded, including the key one: GNU `as` expands the `la` macro with + `ori` while the original assembler (ASPSX 2.81) uses `addiu`, so `la`-using functions need `maspsx`. + ## Next task -P5-T5 — one isolated C candidate for the entry range behind the default fallback, compared with -`tools/sf3_match range` and gated with `make gate`. Open item carried forward: `0x8005DEF8` -reconstruction (see `docs/PHASE5_TOOLCHAIN_FINGERPRINT.md`). +P5-T6 — matching cookbook, registry/verification conventions, and the phase gate. Known gaps to +close: symbol support in the region registry, `maspsx` integration, and a per-region flag override. ## P5-T2 completed evidence diff --git a/phase-ends/Phase5_PLAN.md b/phase-ends/Phase5_PLAN.md index e383671..c3667c1 100644 --- a/phase-ends/Phase5_PLAN.md +++ b/phase-ends/Phase5_PLAN.md @@ -58,7 +58,7 @@ Identify and verify the original Syphon Filter 3 (USA) build toolchain — compi - [ ] **Rules check** - Re-read `AGENTS.md` mandatory behavior after P5-T4 and state: `Rules check — re-read complete. Continuing with [next task].` -- [ ] **P5-T5 — Isolated first C-matching experiment for the entry** *(xHigh)* +- [x] **P5-T5 — Isolated first C-matching experiment for the entry** *(xHigh)* — complete; the entry is determined not to be compiler output, and the first C match is `func_80017AD4`. See `docs/PHASE5_FIRST_MATCH.md`. - Write at most one C candidate for the entry routine behind an explicit `NON_MATCHING`/default-fallback guard; compile with the selected toolchain and flags; compare its exact `[0x800FB368, 0x800FB410)` instruction stream against the validated USA range. - Inspect references and potential duplicates before matching; supply only the nine evidenced relocations; never define the rejected `0x80000004` label. - Iterate only on comparison-supported root causes; stop after two distinct unexplained failures. diff --git a/src/func_80017AD4.c b/src/func_80017AD4.c new file mode 100644 index 0000000..5129c41 --- /dev/null +++ b/src/func_80017AD4.c @@ -0,0 +1,30 @@ +/* + * func_80017AD4 — 20 bytes at 0x80017AD4..0x80017AE8 + * + * Byte-identical reconstruction of a leaf routine. The original stores a + * boolean derived from an unsigned byte argument and then a second argument; + * the compiler schedules the second store into the `jr ra` delay slot. + * + * The observed instructions are: + * andi a2,a2,0xff flag &= 0xff + * sltu a2,zero,a2 flag = (0 < flag) + * sw a2,4(a0) p->flag = flag + * jr ra + * sw a1,8(a0) p->value = value (delay slot) + * + * LIMITS: the function name, the struct name and every field name are + * hypotheses reconstructed from the disassembly. Only the compiled bytes are + * evidence. The `unsigned char` parameter is not cosmetic — the `andi` mask is + * how the original's default unsigned-char ABI shows through. + */ + +typedef struct { + int unknown_00; /* offset 0x00 — not touched by this routine */ + int flag; /* offset 0x04 */ + int value; /* offset 0x08 */ +} func_80017AD4_args; + +void func_80017AD4(func_80017AD4_args *p, int value, unsigned char flag) { + p->flag = flag != 0; + p->value = value; +}