From b9543e52135e6f003a553334bc4775a08da3d9ae Mon Sep 17 00:00:00 2001 From: Christopher Williams Date: Wed, 23 Sep 2026 19:46:17 -0400 Subject: [PATCH] phase4: close code recovery evidence milestone --- docs/PHASE4_CODE_INVENTORY.md | 33 +++++++++++ docs/PHASE4_C_REPRESENTATION_CHECK.md | 25 ++++++++ docs/PHASE4_DATA_RELOCATIONS.md | 27 +++++++++ docs/PHASE4_ENTRY_DOSSIER.md | 45 ++++++++++++++ docs/PHASE4_FULLY_EVIDENCED_LINK.md | 37 ++++++++++++ docs/PHASE4_LAYOUT_EVIDENCE.md | 36 ++++++++++++ docs/PHASE4_LINK_ATTEMPT.md | 28 +++++++++ docs/PHASE4_LITERAL_IMMEDIATE.md | 34 +++++++++++ docs/PHASE4_ORDERED_LINK.md | 39 +++++++++++++ docs/PHASE4_REFERENCE_EVIDENCE.md | 27 +++++++++ docs/PHASE4_SEGMENTATION.md | 33 +++++++++++ docs/PHASE4_VERIFICATION.md | 42 ++++++++++++++ phase-ends/DIGEST.md | 4 ++ phase-ends/Phase4_PLAN.md | 84 +++++++++++++++++++++++++++ phase-ends/PhaseEnd_Phase4.md | 79 +++++++++++++++++++++++++ phase-ends/logs/Phase4.md | 69 ++++++++++++++++++++++ 16 files changed, 642 insertions(+) create mode 100644 docs/PHASE4_CODE_INVENTORY.md create mode 100644 docs/PHASE4_C_REPRESENTATION_CHECK.md create mode 100644 docs/PHASE4_DATA_RELOCATIONS.md create mode 100644 docs/PHASE4_ENTRY_DOSSIER.md create mode 100644 docs/PHASE4_FULLY_EVIDENCED_LINK.md create mode 100644 docs/PHASE4_LAYOUT_EVIDENCE.md create mode 100644 docs/PHASE4_LINK_ATTEMPT.md create mode 100644 docs/PHASE4_LITERAL_IMMEDIATE.md create mode 100644 docs/PHASE4_ORDERED_LINK.md create mode 100644 docs/PHASE4_REFERENCE_EVIDENCE.md create mode 100644 docs/PHASE4_SEGMENTATION.md create mode 100644 docs/PHASE4_VERIFICATION.md create mode 100644 phase-ends/Phase4_PLAN.md create mode 100644 phase-ends/PhaseEnd_Phase4.md create mode 100644 phase-ends/logs/Phase4.md diff --git a/docs/PHASE4_CODE_INVENTORY.md b/docs/PHASE4_CODE_INVENTORY.md new file mode 100644 index 0000000..1dda600 --- /dev/null +++ b/docs/PHASE4_CODE_INVENTORY.md @@ -0,0 +1,33 @@ +# Phase 4 Static Code-Segmentation Inventory + +**Scope:** bounded, static-only inventory for the validated local USA `SCUS_946.40;1` executable. +**Task:** P4-T2 +**Status:** complete; this is not a source, function-recovery, or C-match record. + +## Provenance + +- **USA executable header:** Phase 1 validated the local USA `SCUS_946.40;1` header. It declares entry PC `0x800FB368` and payload range `[0x80010000, 0x801DC000)`. See `docs/PHASE1_REPRODUCIBILITY.md`. +- **Ghidra working copy:** all new static observations below are from the ignored separate Ghidra program `/P2-analysis-SCUS_946.40;1`, loaded by `PSX Executables Loader` as `PSX:LE:32:default`. The preserved `/SCUS_946.40;1` import was not changed. +- **Phase 2 record:** the working copy already supported only bounded loader/archive-path hypotheses. Its named static addresses retain the explicit limits in `docs/PHASE2_INVESTIGATION.md`; they are not candidates for this task. + +## Observations + +| Observation | Static basis | Limit | +|---|---|---| +| One program entry point, `start`, at `0x800FB368` | Ghidra entry-point metadata agrees with the validated USA PS-X EXE header entry PC. | This proves neither startup semantics nor source ownership. | +| `start` analyzer body `[0x800FB368, 0x800FB40F]` | Ghidra reports this function body for the loader entry label. | The end boundary and all control-flow recovery are analyzer results, not original object/function evidence. | +| 1,721 defined functions; 1,719 have auto-generated names; total reported code size is 675,200 bytes | Ghidra function statistics for the separate working copy. | Counts and bodies depend on the current analysis database and do not establish original functions, symbols, or completeness. | +| Loader-created RX blocks: `0x80010000`–`0x8010B50F` and `0x80146950`–`0x801DBFFF` | Ghidra memory-block metadata. | These ranges are loader/import presentation, not proven source sections; Phase 3 code-split attempts already found non-code decoding inside loader-created text. | +| Zero relocation-table entries | Ghidra relocation-table query. | This does not establish original linker layout, object boundaries, or position-independence. | + +## Conservative candidate for P4-T3 + +The sole P4-T3 experiment candidate is the entry range beginning at `0x800FB368`. Its start has independent header and loader-entry support. Its provisional end `0x800FB40F` has only the stated Ghidra analyzer basis. The experiment must retain payload-data fallback before and after that range, must treat failure to assemble/compare as a rejected segmentation hypothesis, and must not introduce C. + +## Excluded candidates + +The Phase 2 static transfer-path addresses are excluded because their recorded evidence addresses candidate routing only; it does not establish a recoverable function boundary, source order, types, object ownership, or runtime behavior. No other analyzer-discovered function is selected merely from its generated name, address, or membership in an RX block. + +## Result + +P4-T2 establishes a bounded, reversible assembly-segmentation experiment only. It establishes no original code/data split, symbol, source section, function match, compiler setting, ABI detail, or C implementation. diff --git a/docs/PHASE4_C_REPRESENTATION_CHECK.md b/docs/PHASE4_C_REPRESENTATION_CHECK.md new file mode 100644 index 0000000..e39bcd5 --- /dev/null +++ b/docs/PHASE4_C_REPRESENTATION_CHECK.md @@ -0,0 +1,25 @@ +# Phase 4 Bounded C-Representation Check + +**Scope:** one bounded, self-authored C-representation check for the entry candidate. +**Task:** P4-T5 +**Status:** complete; no match claimed, no source entered the default build. + +## Method + +A self-authored synthetic C fragment expressed only the entry's BSS-clear idiom: a pointer initialized to `0x801221D8`, an end pointer `0x80146950`, and a `while (p < end) *p++ = 0u;` loop. It is not ROM-derived source and contains no game bytes. It was compiled with the documented candidate path (Clang 22.1.8, `--target=mipsel-none-elf -march=mips1 -mno-abicalls -fno-pic -msoft-float -ffreestanding -fno-builtin -fno-addrsig -G0`, at `-O1`, `-O2`, `-Os`) and assembled with the local GNU MIPS assembler. The emitted loop instructions were then compared idiomatically (mnemonics and operands) with the USA entry loop. + +## Result + +- **Loop body:** across all three optimization levels, the compiled loop used the same four-instruction idiom as the USA entry: store zero through the pointer, advance the pointer by 4, set a less-than flag, and branch back when the flag is set, with the delay slot filled by a no-op. This part is consistent between the candidate compiler and the original. +- **Address materialization differs:** the original materializes the absolute addresses with `lui` + `addiu` low-half pairs, whereas Clang emits `lui` + `ori` low-half pairs. This is a direct, observable representation difference for the same source-level address constant. +- **Frame structure differs:** Clang emitted a full function frame prologue/epilogue (`s8` frame pointer, saved `ra`, word-sized frame) for the fragment. The USA entry has no such frame, which is expected because the fragment is not the full entry routine. + +No full-function instruction match was obtained, and therefore no C match is claimed. Byte-identical output was not achieved and the default build was not modified. + +## Interpretation (evidence-only) + +The `addiu` versus `ori` low-half difference is a concrete compiler-fingerprint hint: the original USA build materializes addresses with `addiu`, which is characteristic of the older GCC/PsyQ code generators, while the modern Clang candidate uses `ori`. Because the low half of these particular addresses (`0x21D8`, `0x6950`) has its high bit clear, both encodings are valid, so this difference is a stylistic code-generation choice rather than a correctness requirement. It narrows the search but does not by itself identify the compiler revision, flags, or ABI. + +## Conclusion + +A byte-identical C match for the entry is not achievable with the documented modern candidate on current evidence. The specific obstruction is the unidentified original compiler/ABI; the observable `addiu`/`ori` divergence gives a direction for future toolchain identification but is not sufficient to select one. Unmatched content therefore remains assembly/data fallback. No C source was added to the repository or the matching build. diff --git a/docs/PHASE4_DATA_RELOCATIONS.md b/docs/PHASE4_DATA_RELOCATIONS.md new file mode 100644 index 0000000..d89c00d --- /dev/null +++ b/docs/PHASE4_DATA_RELOCATIONS.md @@ -0,0 +1,27 @@ +# Phase 4 Direct Data-Relocation Evidence Inventory + +**Scope:** static-only provenance check for the seven generated data relocations that blocked P4-T3L. +**Task:** P4-T3D +**Status:** complete; no assembly, linker, or C change occurred. + +## Provenance + +Queries used the ignored separate USA Ghidra working program `/P2-analysis-SCUS_946.40;1`. The provisional entry range is `[0x800FB368, 0x800FB410)`. Ghidra reference kinds and analyzer cross-references are static evidence only; they do not provide variable names, types, object ownership, or original linker semantics. + +## Direct reference inventory + +| Generated data address | Ghidra references originating in entry | Reference kind | Additional observed reference limit | +|---|---|---|---| +| `0x8011FC70` | `0x800FB3D8` | WRITE | None reported by the query. | +| `0x8011FC74` | `0x800FB3CC` | WRITE | None reported by the query. | +| `0x80120CE8` | `0x800FB3B8` | READ | None reported by the query. | +| `0x80120CEC` | `0x800FB390` | READ | None reported by the query. | +| `0x801221D8` | `0x800FB378` | WRITE | Also READ from `0x800FB450`. | +| `0x8012278C` | `0x800FB3E0`, `0x800FB3FC` | WRITE, READ | Both reported sources are within entry. | +| `0x80146950` | `0x800FB3D8` | DATA | Ghidra also reports parameter references from other addresses; no semantics inferred. | + +## Result + +Every generated data relocation from P4-T3L has direct USA static reference provenance within the bounded entry range. This supports only a future ignored link experiment that supplies these literal addresses for the already-observed relocations. It does not prove that the generated names are original names, that the locations are variables rather than boundaries, that absolute linker definitions reproduce the original layout, or that the provisional entry is an original source/object function. + +Any link retry must be separately planned, resolve exactly the inventory above plus the two already-observed direct-call targets, forbid `D_80000004`, retain full-file comparison gates, and restore the default fallback on any result. diff --git a/docs/PHASE4_ENTRY_DOSSIER.md b/docs/PHASE4_ENTRY_DOSSIER.md new file mode 100644 index 0000000..3cff62b --- /dev/null +++ b/docs/PHASE4_ENTRY_DOSSIER.md @@ -0,0 +1,45 @@ +# Phase 4 Entry Function-Candidate Dossier + +**Scope:** evidence dossier for the only currently defensible function candidate — the provisional entry `start`. +**Task:** P4-T4 +**Status:** complete; no source introduced. + +## Candidate provenance + +| Property | Value | Basis | Limit | +|---|---|---|---| +| Start | `0x800FB368` | Validated USA PS-X EXE header entry PC; Ghidra loader entry point agrees. | Establishes a loader entry, not original source ownership. | +| End (exclusive) | `0x800FB410` | A `jal 0x800FB410` from `0x80029EE0` makes `0x800FB410` a called target, i.e. a function start, immediately after the entry's final `break` instruction. | Stronger than the earlier analyzer-only end, but still static. | +| Body size | 168 bytes (`0xA8`), 3 basic blocks | Ghidra; consistent with P4-T3O2 assembly that reproduced the executable. | Not an original object/symbol record. | +| Stack frame | 16 bytes reported; no stack variables | Ghidra stack layout. | Analyzer result. | + +## Reference review + +- **References to start:** only the loader entry (`Entry Point (EXTERNAL)`). No internal caller. This is consistent with a CRT/startup routine and with the entry being reached from the PS-X EXE header rather than by `jal`. +- **Callees:** the entry performs two direct calls, to `0x801074E4` and `0x80029ED8`. + - `0x801074E4` is a Ghidra-recognized function start (`FUN_801074e4`). + - `0x80029ED8` is **not** a Ghidra function start: the analyzer body `FUN_80029e88` ends at `0x80029ED7`, and `0x80029ED8` begins a standard `addiu sp,sp,-0x18; sw ra,0x14(sp)` prologue. Its only reported reference is the entry's call at `0x800FB404`. This is a Ghidra-missed adjacent function boundary, not evidence about original naming. +- **Data references:** seven generated data addresses (`0x8011FC70`, `0x8011FC74`, `0x80120CE8`, `0x80120CEC`, `0x801221D8`, `0x8012278C`, `0x80146950`) were each given direct in-range Ghidra reference provenance in P4-T3D. + +## Duplicate review + +A bounded Ghidra byte-pattern search for the entry's first 16 bytes returned exactly one match, the entry itself. No duplicated entry body was found by this method. This does not rule out partial or reordered similarity in larger routines. + +## Comparator requirements for any future candidate + +1. Compare only the exact `[0x800FB368, 0x800FB410)` instruction stream against the validated USA range; the remaining payload stays data fallback. +2. Supply exactly the two direct-call targets and seven evidenced data addresses; never define the rejected `0x80000004` label. +3. Use an address-ordered linker layout (P4-T3O2) and exclude non-payload metadata during `objcopy`. +4. Require the full executable to pass `cmp` and SHA-1 comparison against `e173426c157384ebf1b6caf8c6fea18a85a14af9` before any match claim. + +## Codegen hypotheses (unverified) + +These are labeled hypotheses only; no compiler, ABI, or flags are established. + +- The loop clearing `[0x801221D8, 0x80146950)` in 4-byte steps is consistent with a generated C-style BSS clear. +- Reading two words from `0x80120CEC`/`0x80120CE8` and combining one with `lui 0x8000` before installing it as `sp` is consistent with a linker-provided stack/heap boundary calculation, not necessarily a source-level expression. +- Absolute `lui`/`addiu` addressing for data and `jal` for calls is consistent with non-PIC MIPS I code, matching the Phase 3 `-mno-abicalls`/`nopic` synthetic observations. + +## Blocker for C matching + +The original compiler/assembler/linker and flags remain unidentified. The static `PsyQ Version = 4.5.0` import value does not select a compiler revision, optimizer level, or flag set, and the Phase 3 synthetic probe cannot fingerprint the original. Without a compiler/ABI comparator, a byte-identical C match for this entry cannot be claimed on current evidence. diff --git a/docs/PHASE4_FULLY_EVIDENCED_LINK.md b/docs/PHASE4_FULLY_EVIDENCED_LINK.md new file mode 100644 index 0000000..2da4c9f --- /dev/null +++ b/docs/PHASE4_FULLY_EVIDENCED_LINK.md @@ -0,0 +1,37 @@ +# Phase 4 Fully Evidenced Entry-Link Experiment + +**Scope:** one ignored full-link test for the provisional entry representation after direct static provenance was obtained for every unresolved relocation. +**Task:** P4-T3L2 +**Status:** complete with bounded negative result; no C work occurred. + +## Preconditions + +P4-T3S established that Splat's documented ignored-address control removes the unsupported `D_80000004` representation while preserving the tested literal instruction. P4-T3D established direct Ghidra reference provenance for seven generated data relocations; P4-T3S/P4-T3L established two direct-call relocations. + +## Method + +The ignored entry split was recreated. Its undefined-symbol set exactly matched the nine documented targets: the two direct-call targets and seven directly referenced data addresses. The ignored linker script supplied exactly those nine literal static addresses. It did not define `D_80000004` or any additional symbol. Splitting, assembly, symbol-set verification, link, and binary conversion completed. + +## Result + +The complete candidate executable did not match the validated USA executable: + +- Candidate size: 1,886,224 bytes. +- Validated input size: 1,886,208 bytes. +- Candidate SHA-1: `d8fae51263df598fdff09490fb2dd66aa06b25e3`. +- Exact `cmp` exited 1; the bounded mismatch count was 1,247,795 positions. + +This is the same candidate size and SHA-1 previously observed when the rejected generated `D_80000004` linker symbol was supplied. Thus the additional directly evidenced relocation definitions did not resolve the 16-byte layout/output discrepancy. No byte values or generated listings are retained in this record. + +## Restoration gate + +All ignored P4-T3L2 configuration, symbol input, generated assembly/linker files, object files, and candidate products were removed. The default payload-data baseline was restored from a clean state: + +- `make clean` exited 0. +- `make all` exited 0. +- `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` exited 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +## Stop condition + +The full-link mismatch repeats the unresolved layout/output root cause from P4-T3 after the relocation hypothesis was independently eliminated. Do not add further linker symbols, alter section boundaries, infer original linker behavior, select a function/C candidate, or continue entry segmentation without new direct evidence that explains the 16-byte discrepancy and a newly approved task plan. diff --git a/docs/PHASE4_LAYOUT_EVIDENCE.md b/docs/PHASE4_LAYOUT_EVIDENCE.md new file mode 100644 index 0000000..05b3c31 --- /dev/null +++ b/docs/PHASE4_LAYOUT_EVIDENCE.md @@ -0,0 +1,36 @@ +# Phase 4 Output-Layout Evidence Inventory + +**Scope:** evidence-only inspection of the unchanged fully evidenced entry link to attribute the repeated 16-byte output extension. +**Task:** P4-T3O +**Status:** complete; no symbol, boundary, assembly, linker-script, or C change was made to the default build. + +## Provenance + +The ignored configuration recreated the exact P4-T3L2 experiment: the same entry-only `asm` subsegment, the same ignored `symbol_addrs` `ignore:true` control for `0x80000004`, and the same nine literal definitions. Splitting, assembly, link, and binary conversion completed. Only ELF/linker layout metadata and the full-file comparison were examined. + +## Observed layout facts + +- Allocated sections were exactly two: `.header` (size `0x800`) and `.main` (size `0x1CC010`). +- The validated USA payload is `0x1CC000` bytes. `.main` is therefore exactly **16 bytes larger** than the declared payload. +- The generated linker script places `entry_candidate.s.o(.text)` first inside `.main`, then a `. = ALIGN(., 16)`, then `payload_prefix.data.s.o(.data)`, `entry_candidate.s.o(.data)`, and `payload_suffix.data.s.o(.data)`, then further `ALIGN` directives for rodata and bss. +- The first differing byte was file offset `0x800`, the first payload byte. This is consistent with the entry `.text` being placed where the data prefix belongs. +- Alignment arithmetic: entry `.text` is `0xA8` bytes, requiring 8 bytes of padding to the next 16-byte boundary; after the `0xEB368`-byte prefix data another 8 bytes of padding is required. Those two 8-byte paddings account exactly for the observed +16 bytes. + +## Directly supported cause + +Splat's generated linker script for this configuration groups output by section kind and emits `.text` before `.data`, regardless of the original subsegment address order. A split of the form data-prefix / code / data-suffix therefore cannot be reproduced by this script: the code is placed at the start of `.main`, and the two alignment directives expand the section by exactly 16 bytes. + +This explains both observed symptoms: the mass byte mismatch beginning at the first payload byte and the 16-byte size increase. It does not establish original linker behavior, function/object ownership, or source semantics. + +## Restoration gate + +The ignored configuration, symbol input, generated assembly/linker files, object files, and candidate products were removed. The default payload-data baseline was restored from a clean state: + +- `make clean` exited 0. +- `make all` exited 0. +- `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` exited 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +## Next boundary + +A remedy requires an ordered linker script that preserves subsegment ROM order and removes the incidental 16-byte padding. That is a build-design change and needs its own approved task with a full-binary comparator. This record authorizes no C work and asserts no original toolchain or source layout. diff --git a/docs/PHASE4_LINK_ATTEMPT.md b/docs/PHASE4_LINK_ATTEMPT.md new file mode 100644 index 0000000..840dd65 --- /dev/null +++ b/docs/PHASE4_LINK_ATTEMPT.md @@ -0,0 +1,28 @@ +# Phase 4 Direct-Call Relocation Link Attempt + +**Scope:** one ignored link-attempt precondition check for the P4-T3S entry representation. +**Task:** P4-T3L +**Status:** complete with bounded negative result; no link, binary conversion, or C work occurred. + +## Method + +The ignored P4-T3S entry split was recreated with the same documented `ignore:true` control for `0x80000004`. All ignored generated sources assembled successfully. Before a linker script was modified, the entry object's undefined-symbol table was inspected. The approved scope allowed definitions only for the two Ghidra-observed direct-call targets. + +## Result + +The object contained the two expected generated direct-call symbols, but also seven generated data symbols at static addresses within the executable image. Because the approved task did not establish direct evidence for resolving those data relocations, the undefined-symbol set did not meet the exact two-call precondition. The task refused to append any linker definitions and exited before link or binary conversion. + +This result establishes only that the entry representation has additional unresolved data relocations. It does not establish whether those symbols are original globals, section boundaries, linker-provided names, or valid absolute definitions. + +## Restoration gate + +The ignored configuration, symbol input, generated assembly/linker files, object files, and partial build directory were removed. The untouched default payload-data baseline was reverified: + +- `make clean` exited 0. +- `make all` exited 0. +- `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` exited 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +## Blocker + +Do not define the seven generated data symbols from name/address appearance alone. Any reopening must be a new evidence-only task that maps each generated reference to direct USA static instruction/reference evidence and defines a comparator before another link attempt. diff --git a/docs/PHASE4_LITERAL_IMMEDIATE.md b/docs/PHASE4_LITERAL_IMMEDIATE.md new file mode 100644 index 0000000..2d54c00 --- /dev/null +++ b/docs/PHASE4_LITERAL_IMMEDIATE.md @@ -0,0 +1,34 @@ +# Phase 4 Literal-Immediate Representation Experiment + +**Scope:** one ignored, pre-link entry-only assembly representation test. +**Task:** P4-T3S +**Status:** complete; no link, C candidate, or match claim was attempted. + +## Basis + +P4-T3R established direct static evidence that Ghidra has no reference to `0x80000004` and that the entry instruction at offset `+0x34` has literal upper immediate `0x8000`. The P4-T3 generated `D_80000004` reference was therefore treated as a splitter representation hypothesis, not an original symbol. + +## Method and result + +An ignored Splat configuration selected only the provisional 168-byte entry range as assembly and retained data fallback around it. Its ignored `symbol_addrs` input applied Splat's documented `ignore:true` control only to `0x80000004`; no linker symbol was supplied. + +- Splitting and GNU MIPS assembly each exited 0. +- The generated entry source contained no `D_80000004` reference. +- Before linking, the assembled 4-byte instruction at entry offset `+0x34` exactly compared with the corresponding 4-byte range of the validated USA executable. +- The object reported two `R_MIPS_26` relocations for direct calls. Therefore a pre-link comparison of the complete 168-byte range is not valid: those call instruction fields are intentionally unresolved in a relocatable object. +- No link was attempted. + +This establishes only that the selected ignored-symbol control preserves the tested literal instruction and removes the unsupported generated data reference. It does not establish an original symbol, semantics, function boundary, object, section, linker model, or source/C match. + +## Restoration gate + +The ignored configuration, symbol input, generated assembly, generated linker files, object, and temporary comparison artifacts were removed. The default payload-data baseline was restored and verified from a clean state: + +- `make clean` exited 0. +- `make all` exited 0. +- `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` exited 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +## Next boundary + +A link attempt, if approved and separately planned, must resolve only the two observed call relocations using direct static targets, compare the full executable, and retain the default fallback. It must not add the rejected `D_80000004` linker symbol or introduce C. diff --git a/docs/PHASE4_ORDERED_LINK.md b/docs/PHASE4_ORDERED_LINK.md new file mode 100644 index 0000000..0555cf3 --- /dev/null +++ b/docs/PHASE4_ORDERED_LINK.md @@ -0,0 +1,39 @@ +# Phase 4 Ordered-Link Experiment + +**Scope:** one ignored ordered-layout link test for the provisional entry code subsegment. +**Task:** P4-T3O2 +**Status:** complete; exact full-binary match achieved for a bounded assembly representation. No C work occurred. + +## Basis + +P4-T3O established that the Splat-generated linker script places `.text` before `.data` and inserts alignment directives, which misordered the data-prefix/code/data-suffix split and added exactly 16 bytes. + +## Method (all inputs ignored) + +1. Reused the tracked default Splat configuration with one change: the single payload `data` subsegment was replaced by three address-ordered subsegments — payload prefix data `[0x800, 0xEBB68)`, entry `asm` `[0xEBB68, 0xEBC10)`, payload suffix data `[0xEBC10, 0x1CC800)`. +2. Applied the P4-T3S committed control: an ignored `symbol_addrs` entry with `ignore:true size:0x4` for `0x80000004`, which removes the unsupported generated data label while preserving the literal upper immediate. +3. Did **not** use the Splat-generated linker script. Used a hand-authored ignored linker script with two output sections and no padding directives: + - `.header` at VMA `0x0` / LMA `0x0` containing `header.s.o(.data)` (`0x800` bytes). + - `.main` at VMA `0x80010000` / LMA `0x800` containing, in address order, `payload_prefix.data.s.o(.data)` (`0xEB368`), `entry_candidate.s.o(.text)` (`0xA8`), `payload_suffix.data.s.o(.data)` (`0xE0BF0`). +4. Supplied exactly the nine already-evidenced literal targets (`func_80029ED8`, `func_801074E4`, and the seven `D_...` addresses from P4-T3D). Did not define `D_80000004`. +5. Converted with GNU `objcopy -O binary`, excluding non-payload metadata sections (`.MIPS.abiflags`, `.reginfo`, `.pdr`, `.gnu.attributes`). + +## Result + +- `.header` size `0x800`; `.main` size `0x1CC000` (no padding; the previous 16-byte extension is gone). +- Link and binary conversion exited 0. +- Rebuilt executable size `1,886,208` bytes, equal to the validated input. +- `cmp -s build/p4-order.rebuilt extracted/SCUS_946.40;1` exited 0, with `cmp -l` reporting zero differing bytes. +- Rebuilt SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`, identical to the validated input. + +This establishes that the validated USA executable can be reproduced with the entry function present as a real MIPS `asm` subsegment surrounded by payload-data fallback, given an address-ordered linker layout and the directly evidenced relocation targets. + +## Limits + +- The entry body end `0x800FB410` and the entry subsegment identification remain Ghidra-analyzer-derived, not original object/source evidence. +- The nine relocations are supplied as literal linker definitions from static Ghidra evidence; they do not establish original symbol names, types, object ownership, or linker behavior. +- This is an assembly representation, not a recovered function, source section, ABI, or compiler fingerprint. It does not authorize C. + +## Default fallback + +The tracked default configuration and `Makefile` were not modified by this experiment. The default payload-data baseline was re-verified after cleanup. diff --git a/docs/PHASE4_REFERENCE_EVIDENCE.md b/docs/PHASE4_REFERENCE_EVIDENCE.md new file mode 100644 index 0000000..8bbb2d7 --- /dev/null +++ b/docs/PHASE4_REFERENCE_EVIDENCE.md @@ -0,0 +1,27 @@ +# Phase 4 Direct Reference and Layout Evidence + +**Scope:** evidence-only reopening task after the P4-T3 segmentation blocker. +**Task:** P4-T3R +**Status:** complete; this record does not change assembly, linker rules, or fallback selection. + +## USA static provenance + +All observations are from the ignored separate USA Ghidra working program `/P2-analysis-SCUS_946.40;1`, loaded with `PSX Executables Loader`. The preserved original import was not changed. Ghidra remains the static oracle; its analyzer output is not original source/function proof. + +## New direct reference evidence + +- Ghidra reports **no references to** `0x80000004` in the working program. +- The provisional entry body has three Ghidra basic blocks: entry `0x800FB368`–`0x800FB377`, a loop block `0x800FB378`–`0x800FB38B`, and a final block `0x800FB38C`–`0x800FB40F`. The final block has two direct-call successors. This is analyzer control-flow metadata only. +- Bounded inspection of the static disassembly confirms that the instruction at entry offset `+0x34` contains a literal upper immediate `0x8000`. It is not represented by Ghidra as a data reference to `0x80000004`. + +The generated `D_80000004` reference that blocked P4-T3 is therefore unsupported by this direct static evidence. Its meaning remains unknown; this record does not infer that it is safe to supply an absolute linker symbol or assign original semantics. + +## Layout evidence limit + +P4-T3 correctly removed its ignored configuration, generated source, linker script, object files, and candidate binary after the baseline restoration gate. The retained non-content result is sufficient to establish that the literal-address linker experiment produced a 1,886,224-byte output, 16 bytes larger than the 1,886,208-byte validated input. The temporary linker/section metadata required to attribute that difference no longer exists. Recreating it would be a new segmentation experiment, not evidence-only inspection. + +## Conclusion + +A further experiment is justified only if it is narrower than P4-T3 and has an object/range comparator before link: configure the ignored splitter so the literal upper immediate remains literal, assemble only the bounded entry candidate, and directly compare its 168-byte instruction range with the validated USA range before attempting a full link. This would test a splitter/assembler representation hypothesis, not original linkage or C matching. + +No C candidate is selected. Before any such experiment, re-read the mandatory rules (P4-T3R is the fourth completed Phase 4 task) and add an explicit task plan with fallback restoration and range-comparison gates. diff --git a/docs/PHASE4_SEGMENTATION.md b/docs/PHASE4_SEGMENTATION.md new file mode 100644 index 0000000..85f69bb --- /dev/null +++ b/docs/PHASE4_SEGMENTATION.md @@ -0,0 +1,33 @@ +# Phase 4 Conservative Segmentation Experiment + +**Scope:** one reversible ignored assembly experiment for the P4-T2 entry candidate in the validated local USA executable. +**Task:** P4-T3 +**Status:** complete with bounded negative result; no C work is authorized by this record. + +## Candidate and controls + +The experiment used the sole P4-T2 candidate: start `0x800FB368`, independently supported by the validated USA PS-X EXE header entry PC and Ghidra entry metadata. Its provisional exclusive end `0x800FB410` was Ghidra-analyzer-derived. The ignored experimental Splat configuration retained data subsegments before and after that range and emitted an `asm` subsegment only for the 168-byte provisional body. The tracked default configuration was not modified. + +Before the experiment, P4-T1 revalidated the default payload-data baseline. The experiment was built through the existing Make orchestration with overridden ignored configuration/output paths. + +## Results + +1. **Generated-reference failure.** Splat accepted the 168-byte `asm` subsegment, but GNU MIPS link failed because the generated assembly contained two references to unresolved generated label `D_80000004`. This establishes only that the selected split required an unresolved generated symbol; it does not establish that label's meaning, original linkage, or the provisional function boundary. +2. **Literal-address linker experiment.** A single ignored linker-only test supplied `D_80000004` at its literal generated address. Linking and binary conversion then exited 0, but `cmp` against the validated USA executable exited 1. The candidate output was 1,886,224 bytes versus the original 1,886,208 bytes; its SHA-1 was `d8fae51263df598fdff09490fb2dd66aa06b25e3`. The mismatch report counted 1,247,795 differing positions. No byte values or generated listing are retained here. + +These are two distinct failed root-cause attempts. Under `AGENTS.md`, further segmentation diagnosis stops rather than adding speculative linker rules, symbols, or code/data assumptions. + +## Restoration gate + +All ignored experiment configuration, generated output, linker script, dependency file, and candidate build products were removed. The tracked default payload-data configuration was then rebuilt from the safe clean state: + +- `make clean` exited 0. +- `make all` exited 0. +- `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` exited 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +## Conclusion and blocker + +The header/entry-supported start address is insufficient by itself to create a link/layout-identical assembly code subsegment with the current baseline tooling. The unresolved generated reference and the subsequent 16-byte output-length/layout difference are not explained by the available evidence. The payload-data fallback remains the only verified default representation. + +Do not advance to P4-T4, select a C candidate, add a linker symbol rule, or infer an original function/object/linker model from this experiment. A future, explicitly planned investigation may reopen segmentation only with new direct static or runtime evidence and a defined comparator. diff --git a/docs/PHASE4_VERIFICATION.md b/docs/PHASE4_VERIFICATION.md new file mode 100644 index 0000000..6c880ce --- /dev/null +++ b/docs/PHASE4_VERIFICATION.md @@ -0,0 +1,42 @@ +# Phase 4 Verification Record + +**Scope:** consolidated Phase 4 outcome, blocker, and clean verification gates. +**Task:** P4-T6 +**Status:** complete pending developer milestone confirmation. The phase is not closed. + +## Outcome summary + +Phase 4 established a conservative, ROM-safe USA **assembly** code-recovery path while preserving the Phase 3 payload-data baseline: + +- **P4-T1:** re-validated the default byte-identical payload-data baseline before any analysis. +- **P4-T2:** produced a static-only segmentation inventory; the sole defensible candidate was the loader entry `0x800FB368`. +- **P4-T3/T3R/T3S/T3L/T3D/T3L2/T3O:** established that the initial link failures came from an unsupported generated data label and from a Splat linker-script ordering/padding artifact, then attributed the repeated 16-byte extension exactly to `.text`-before-`.data` ordering plus two 8-byte alignment paddings. +- **P4-T3O2 (success):** an address-ordered ignored linker layout reproduced the validated USA executable with the entry present as a real MIPS `asm` subsegment surrounded by data fallback — `1,886,208` bytes, `cmp` exit 0, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`, reproduced from a fresh regeneration. Non-payload metadata sections had to be excluded during `objcopy`. +- **P4-T4:** documented the entry dossier; established that `0x800FB410` is itself a call target (so it is a function start), that `0x80029ED8` is a Ghidra-missed adjacent function start, and that no duplicate of the entry prologue was found. +- **P4-T5:** a bounded, self-authored C-representation check showed the entry's clear-loop idiom compiles identically, but the original uses `lui`+`addiu` address materialization while Clang uses `lui`+`ori`. No C match was obtained. + +## Evidence-backed blocker + +The original compiler/assembler/linker and flags remain unidentified. Consequently no byte-identical C match can be claimed. The observable `addiu`/`ori` divergence is a compiler-fingerprint hint but not a toolchain identification. Per the Phase 4 milestone, this blocker is recorded and all unresolved content remains fallback assembly/data. + +## Clean verification gates + +- `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v` — exit 0; 28 synthetic-only tests passed. +- `make clean` — exit 0. `make all` — exit 0. +- `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` — exit 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. +- No prohibited path is tracked (`disks/`, `extracted/`, `asm/`, `build/`, `expected/`, `ghidra/`, `dumps/`, `assets/`, local tools). +- `git diff --check` passed and no changes were staged. +- All ignored experiment configuration, generated assembly, objects, linker scripts, and candidate binaries were removed; only ignored `.run/` logs remain. + +## Firewall note + +No game bytes, disassembly listings, ROM addresses copied as fixtures, or ROM-derived material were added to tracked files. Tracked Phase 4 additions are documentation records and the plan/current-phase files only. + +## Requested action + +The developer is asked to confirm the Phase 4 milestone: +- code/function recovery path established as the address-ordered assembly subsegment workflow, and +- C matching recorded as blocked by the unidentified original toolchain. + +On confirmation, a PhaseEnd record and digest update follow. Do not begin a new phase in this session. diff --git a/phase-ends/DIGEST.md b/phase-ends/DIGEST.md index 8ee38bf..61075a5 100644 --- a/phase-ends/DIGEST.md +++ b/phase-ends/DIGEST.md @@ -15,3 +15,7 @@ Phase 2 preserved the original Phase 1 Ghidra program and analyzed an isolated U ## Phase 3 — Matching Build Pipeline, Assembly Baseline, and Compiler Fingerprint (2026-09-23) Phase 3 verified the local native filesystem prerequisite, added ignored pinned Splat, Maspsx, and locally built GNU MIPS Binutils tooling with recorded provenance, and added a tracked `Makefile` plus synthetic-only fingerprint probe. A clean `make clean && make all` rebuilds `SCUS_946.40;1` byte-for-byte: `cmp` exits 0 and both full-file SHA-1 values are `e173426c157384ebf1b6caf8c6fea18a85a14af9`; the final 28-test synthetic suite passed. The exact baseline is deliberately a payload-data assembly representation, not a recovered source/function/section/object model. Initial code segmentation attempts decoded non-code bytes as unsupported instructions, so they remain unresolved. The synthetic Clang/GNU-as probe is deterministic for its self-authored fixture but cannot fingerprint the original compiler, assembler, linker, flags, ABI, or libraries because no original function/object comparator has been recovered. Phase 4 must start fresh with approved tasks, retain this baseline, and make no C-matching claim without direct evidence and the full-binary gate. + +## Phase 4 — Code Recovery Evidence, Function Boundaries, and Initial Matching (2026-09-23) + +Phase 4 replaced the data-only representation with a documented, ROM-safe assembly code-recovery workflow. It attributed the Phase 3 code-split failures to two causes: an unsupported generated data label (removed with Splat's `ignore:true` control, validated against direct Ghidra reference evidence) and, decisively, a linker-layout artifact—Splat's generated script emits `.text` before `.data` and inserts two 8-byte alignments, adding exactly 16 bytes. A hand-authored address-ordered linker layout (header `0x800`, payload prefix `0xEB368`, entry code `0xA8`, payload suffix `0xE0BF0`, non-payload metadata excluded during `objcopy`) reproduced the validated USA executable byte-for-byte with the entry present as real MIPS code: 1,886,208 bytes, `cmp` exit 0, SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`, reproduced from a fresh regeneration. The entry `start` at `0x800FB368` is now bounded by direct evidence: `0x800FB410` is itself a `jal` target, and the callee `0x80029ED8` is a Ghidra-missed adjacent function start. No duplicate entry prologue was found, and all nine entry relocations have direct Ghidra provenance. A bounded, self-authored C-representation check found the clear-loop idiom compiles identically, but the original materializes addresses with `lui`+`addiu` while Clang emits `lui`+`ori`; therefore no C match was obtained and none is claimed. The original compiler/assembler/linker identity remains the recorded blocker, the tracked default build is still the payload-data fallback, and the ordered workflow remains a documented ignored experiment. The final 28-test synthetic suite and the clean `make clean && make all`, `cmp`, and SHA-1 gates all passed, with no prohibited material tracked. diff --git a/phase-ends/Phase4_PLAN.md b/phase-ends/Phase4_PLAN.md new file mode 100644 index 0000000..7e6f9fc --- /dev/null +++ b/phase-ends/Phase4_PLAN.md @@ -0,0 +1,84 @@ +# Phase 4 Plan — Code Recovery Evidence, Function Boundaries, and Initial Matching + +**Status:** approved by developer +**Planning effort:** Max + +## Goal + +Convert the Phase 3 payload-data baseline into an evidence-backed, ROM-safe recovery workflow for USA executable code: establish conservative code/function candidates, obtain direct comparison evidence for an initial candidate, and introduce C only when it is instruction-identical and the full-binary gate remains green. The Phase 3 payload-data representation remains the default fallback for every unresolved byte. + +## Preconditions + +- The validated local USA `SCUS_946.40;1` executable has a preserved clean all-payload assembly rebuild whose complete-file SHA-1 is `e173426c157384ebf1b6caf8c6fea18a85a14af9`. +- That baseline does not establish code, function, object, source-section, linker-layout, or original-toolchain boundaries. +- The original compiler/assembler/linker identity and flags remain unresolved. The static Ghidra import reports `PsyQ Version = 4.5.0`, but that is not compiler evidence. +- USA static analysis in Ghidra is the boundary oracle. PCSX-Redux is required only where static evidence cannot establish the relevant execution or mapping fact. + +## Scope and safeguards + +- Keep executable inputs, Ghidra programs, analysis exports, generated assembly/linker inputs, object files, maps, candidate diffs, and build products in ignored paths. Do not copy ROM-derived bytes, disassembly, strings, addresses, or fixtures into tracked files. +- Preserve the existing full-payload data fallback for unresolved material. A tentative code or C replacement must have a tested, reversible build selection and cannot weaken the exact full-binary comparison gate. +- Treat all candidate function boundaries, symbols, types, calling conventions, code/data splits, source order, compiler choices, flags, and runtime claims as hypotheses until supported by USA static evidence or documented runtime observation. +- Before attempting C, inspect the candidate's callers, callees, references, and duplicated bodies. Share a verified duplicated body only through a documented source/registry mechanism. +- A C candidate enters the default matching build only after its generated instruction stream is directly identical to the USA candidate range and a documented clean full-binary rebuild passes `cmp` and SHA-1 comparison. Otherwise retain assembly/data fallback and mark the candidate non-matching. + +## Tasks + +- [x] **P4-T1 — Phase control records and baseline revalidation** *(xHigh)* + - Create active phase records and re-run the documented clean baseline, synthetic suite, firewall checks, and Git review before analysis changes. + - **Verify:** all commands exit 0; rebuilt executable exactly matches the validated ignored USA input; only ROM-safe tracked records are changed. + +- [x] **P4-T2 — Static code-segmentation evidence inventory** *(xHigh)* + - Inspect the isolated USA Ghidra analysis program and existing import records to identify only statically supported code/data candidates, entry points, references, and boundaries. Record uncertainty without exporting game-derived listings. + - **Verify:** every recorded candidate has USA provenance and a stated limit; no inferred source/function claim is promoted to fact. + +- [x] **P4-T3 — Conservative assembly segmentation experiment** *(xHigh)* + - For one bounded candidate supported by P4-T2, generate ignored assembly/link inputs that retain payload-data fallback outside the candidate. Diagnose build or byte-comparison results by evidence only. + - **Verify:** clean baseline fallback remains available; experiment output is ignored; exact comparison either passes or the bounded mismatch is recorded with no C claim. + +- [x] **P4-T3R — Direct reference and layout evidence** *(xHigh)* + - Reopen the P4-T3 blocker only through new, bounded USA static evidence: establish the Ghidra reference context for the generated `D_80000004` label and inspect the ignored experiment’s section/layout metadata without retaining disassembly or bytes. + - **Verify:** provenance, observed reference/layout facts, and an explicit conclusion on whether another segmentation experiment is justified are recorded. Do not alter linker rules, assembly, or the default fallback. + +- [x] **P4-T3S — Literal-immediate object/range experiment** *(xHigh)* + - Use the P4-T3R static evidence to configure one ignored entry-only assembly experiment that preserves the literal upper immediate rather than accepting an unsupported generated data label. Before any link, directly compare the exact 4-byte literal-immediate instruction at entry offset `+0x34` with the validated USA range and inspect relocations for the other direct-call instructions. + - **Verify:** the literal instruction comparison passes and no unsupported generated data reference remains; call relocations are recorded as the reason a pre-link whole-range comparison is not valid. Otherwise stop after recording the single representation failure. In all cases remove ignored experiment artifacts and clean-rebuild the default payload-data fallback for exact comparison. + +- [x] **P4-T3L — Direct-call relocation link experiment** *(xHigh)* + - Recreate the ignored P4-T3S entry split and resolve only its two observed direct-call relocations at the Ghidra-reported literal static targets. Do not define `D_80000004` or infer any other symbol semantics. + - **Verify:** link and binary conversion exit 0; the full executable passes exact `cmp` and SHA-1 comparison before any further work. On mismatch, record the bounded result, remove artifacts, and restore/reverify the default fallback. + +- [x] **P4-T3D — Direct data-relocation evidence inventory** *(xHigh)* + - Query the isolated USA Ghidra program for direct references to each of the seven P4-T3L generated data addresses and map only their source locations against the provisional entry body. + - **Verify:** each address is either directly supported or explicitly unresolved; record whether all supported references are in the bounded entry range. Do not recreate assembly, define symbols, or link. + +- [x] **P4-T3L2 — Fully evidenced entry-link experiment** *(xHigh)* + - Recreate the ignored entry split and resolve exactly the two P4-T3S direct-call targets and seven P4-T3D data addresses at their literal USA static addresses. Do not define `D_80000004` or any additional symbol. + - **Verify:** the exact unresolved-symbol set equals the nine documented targets; link and conversion exit 0; the complete rebuilt executable passes `cmp` and SHA-1 comparison. In every outcome remove artifacts and clean-rebuild the default fallback. No C work follows this task. + +- [x] **P4-T3O — Output-layout evidence inventory** *(xHigh)* + - Recreate the unchanged fully evidenced ignored entry link solely to inspect ELF/linker section and file-layout metadata, then attribute the repeated 16-byte output extension if the metadata directly supports it. + - **Verify:** no symbol/boundary/assembly change is introduced; section/file-layout facts and their limits are recorded; artifacts are removed and the default fallback is clean-rebuilt. No layout remedy is attempted in this task. + +- [x] **P4-T3O2 — Ordered-linker-script experiment** *(xHigh)* + - Build an ignored linker script that preserves subsegment ROM order (payload prefix data, entry code, payload suffix data) and removes incidental alignment padding, using only the nine already-evidenced symbols and the P4-T3S representation control. + - **Verify:** link and conversion exit 0; the complete rebuilt executable passes exact `cmp` and SHA-1 comparison. On mismatch, record the bounded result, remove artifacts, and restore/reverify the default fallback. No C work follows this task. + +- [x] **P4-T4 — Initial function-candidate dossier and duplicate review** *(xHigh)* + - Select at most one function candidate only if P4-T2/P4-T3 provide a defensible boundary. Inspect its cross-references and potential duplicate bodies, then document direct comparator requirements and codegen hypotheses. + - **Verify:** candidate provenance, boundary limits, reference review, duplicate result, and comparator method are recorded; no source is introduced yet. + +- [x] **Rules check** + - Re-read `AGENTS.md` mandatory behavior after P4-T4 and state the required continuation notice before P4-T5. + +- [x] **P4-T5 — Isolated initial C-matching experiment** *(xHigh)* + - Implement at most one candidate behind an explicit non-matching/default-fallback guard. Compile with documented candidate settings and compare its exact instruction range against the USA candidate range. + - **Verify:** direct candidate comparison and clean full-binary gate both pass before enabling C by default. On failure, retain fallback, record the bounded result, and do not claim a match. + +- [x] **P4-T6 — Scalable registry, verification record, and phase gate** *(xHigh)* + - If and only if P4-T5 succeeds, add ROM-safe registration and verification conventions for future matched functions; otherwise document the blocker. Complete clean tests, baseline/full-binary comparison, firewall review, and Git review, then request milestone confirmation. + - **Verify:** every match claim has direct instruction and clean full-binary evidence; unmatched material remains fallback; no prohibited artifact is tracked. + +## Milestone + +A conservative USA code/function recovery path is established without weakening the payload-data baseline. Any reported C match has direct instruction-identical candidate comparison plus a green clean full-binary hash gate; otherwise Phase 4 records the evidence-backed blocker and leaves all unresolved content as fallback assembly/data. diff --git a/phase-ends/PhaseEnd_Phase4.md b/phase-ends/PhaseEnd_Phase4.md new file mode 100644 index 0000000..b46620a --- /dev/null +++ b/phase-ends/PhaseEnd_Phase4.md @@ -0,0 +1,79 @@ +# PhaseEnd — Phase 4: Code Recovery Evidence, Function Boundaries, and Initial Matching + +**Date:** 2026-09-23 +**Phase Status:** Complete +**Milestone confirmed by developer:** yes + +## Completed Checklist + +- P4-T1 — Created the Phase 4 control records and re-validated the clean payload-data baseline before any analysis. +- P4-T2 — Produced a static-only code-segmentation inventory; the sole defensible candidate is the loader entry `0x800FB368`. +- P4-T3 — Ran a bounded entry-range assembly experiment; it failed to link/layout-match and was recorded as a rejected hypothesis. +- P4-T3R — Obtained direct USA static evidence that Ghidra has no reference to the generated `0x80000004` label and that the entry literal is an unresolved upper immediate. +- P4-T3S — Showed that the documented Splat `ignore:true` control removes the unsupported label while preserving the tested literal instruction; noted the two call relocations make whole-range pre-link comparison invalid. +- P4-T3L — Refused to link when the entry object showed seven additional generated data relocations beyond the approved two-call scope. +- P4-T3D — Mapped every generated data relocation to direct in-range Ghidra reference provenance. +- P4-T3L2 — Linked with the exact nine evidenced targets but reproduced the same non-matching output, eliminating the relocation hypothesis. +- P4-T3O — Attributed the repeated 16-byte extension exactly to `.text`-before-`.data` linker ordering plus two 8-byte alignment paddings. +- P4-T3O2 — Reproduced the validated executable byte-for-byte with the entry as a real MIPS `asm` subsegment using an address-ordered link. +- P4-T4 — Documented the entry dossier, including newly supported boundary evidence and a negative duplicate check. +- Rules check — Re-read `AGENTS.md` mandatory behavior after P4-T4 and stated the required continuation notice before P4-T5. +- P4-T5 — Ran a bounded, self-authored C-representation check; obtained an interpreted negative (original `addiu` versus candidate `ori`). +- P4-T6 — Completed the phase verification, blocker record, and developer confirmation gate. + +## Verified Results + +### Address-ordered assembly code recovery + +An ignored address-ordered linker layout reproduced the validated USA executable with the provisional entry present as a real MIPS code subsegment surrounded by payload-data fallback: + +- Splitting, assembly, link, and binary conversion each exited 0. +- Rebuilt size 1,886,208 bytes, equal to the validated input. +- `cmp -s` exited 0 with zero differing bytes. +- Rebuilt SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`, identical to the validated input. +- The result reproduced on a fresh regeneration. + +The earlier 16-byte extension was attributed to Splat's generated linker script placing `.text` before `.data` and inserting two 8-byte alignment paddings. Using hand-authored ordered sections (header `0x800`, payload prefix `0xEB368`, entry code `0xA8`, payload suffix `0xE0BF0`) and excluding non-payload metadata during `objcopy` produced an exact match. + +### Entry function evidence + +- The entry start `0x800FB368` is supported by both the USA PS-X EXE header and Ghidra's loader entry point. +- The entry end `0x800FB410` is supported as a function start because another routine performs `jal 0x800FB410`. +- The entry's second callee `0x80029ED8` is a Ghidra-missed adjacent function start whose prologue begins immediately after `FUN_80029e88`'s body. +- All nine entry relocations have direct Ghidra provenance; a bounded byte-pattern duplicate search found no duplicate entry prologue. + +### Bounded C-representation result + +A self-authored C fragment expressing the entry's clear-loop idiom compiled to the same loop idiom under the modern candidate, but the original materializes addresses with `lui`+`addiu` while Clang emits `lui`+`ori`. No C match was obtained and nothing entered the default build. + +## Deviations and Bounded Results + +| Item | Plan | Actual | Reason / limit | +|---|---|---|---| +| C matching | One isolated candidate behind a fallback guard | Bounded representation check only; no C match | The original compiler/ABI is unidentified; the entry C match is not reproducible with the modern candidate. | +| Tracked build | Promote recovery path into tracked orchestration | Recovery path remains an ignored, documented experiment | A ROM-safe tracked promotion is a larger design change deferred to a future phase; the tracked default stays a payload-data fallback. | +| Segmentation scope | Full code/function segmentation | One entry code subsegment | Only the entry had independent header/entry and call-target boundary support. | + +## Verification and Firewall + +- `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v` — exit 0; 28 synthetic-only tests passed. +- Final clean `make clean`, `make all`, exact `cmp`, and full-file SHA-1 check passed. +- `git diff --check` passed and no changes were staged prior to closure. +- Representative ignored paths were re-confirmed and no prohibited path is tracked. +- No game bytes, disassembly listings, ROM-derived fixtures, generated assembly, build output, expected binary, Ghidra material, dump, proprietary SDK material, or local tool artifact is part of the tracked change set. + +## Rules Added This Phase + +None. + +## Next + +Phase 4 is closed. A future phase must begin in a fresh session with an approved task-by-task plan. It should preserve both the payload-data baseline and the documented address-ordered assembly recovery path, and must treat the original compiler/assembler/linker identity, broader code/function segmentation, and any tracked promotion of the ordered workflow as unresolved. No C match may be claimed without direct instruction-identical comparison and a green clean full-binary hash gate. + +## Plain-English Recap + +Phase 4 turned the Phase 3 data-only rebuild into a real code-recovery workflow. The blocker was not the program's code but the linker script's ordering: it emitted code before data and padded by 16 bytes. Rewriting the layout to keep the original byte order let us assemble the startup routine as actual MIPS code and still reproduce the entire executable exactly. We also pinned down the startup routine's true boundaries, confirmed there is no duplicate, and proved that the modern compiler we have differs from the original toolchain (`ori` versus `addiu`). We did not match any function to C, and no game data entered Git. + +## 🛑 Stop Here + +`CURRENT_PHASE.md` is archived as `phase-ends/logs/Phase4.md`; this PhaseEnd and the digest update belong in the closure commit. Do not begin Phase 5 in this session. diff --git a/phase-ends/logs/Phase4.md b/phase-ends/logs/Phase4.md new file mode 100644 index 0000000..691f739 --- /dev/null +++ b/phase-ends/logs/Phase4.md @@ -0,0 +1,69 @@ +# Current Phase — Phase 4 + +**Phase:** Phase 4 — Code Recovery Evidence, Function Boundaries, and Initial Matching +**Status:** active; developer approved the plan +**Active task:** P4-T6 complete — awaiting developer milestone confirmation + +## Starting boundary + +Phase 3 closed with a clean byte-identical USA `SCUS_946.40;1` all-payload assembly baseline. It is a build/comparison baseline only: it establishes no code, function, section, object, linker-layout, or original-toolchain model. The original compiler/assembler/linker fingerprint remains unresolved. + +## P4-T1 completed evidence + +- `PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tools/tests -v` exited 0: 28 synthetic-only tests passed. +- `make clean`, `make all`, and `cmp -s build/scus_946_40.rebuilt extracted/SCUS_946.40;1` each exited 0. +- Both full files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. +- `git check-ignore -v --no-index` confirmed rules for hypothetical `expected/`, `dumps/`, and `.run/` generated children. No prohibited paths are tracked; the sole tracked `.run/.gitkeep` is the empty ignore sentinel. `git diff --check` and the unstaged-index check passed before the new Phase 4 control records. + +## P4-T2 completed evidence + +`docs/PHASE4_CODE_INVENTORY.md` records the static-only USA evidence. The sole P4-T3 candidate begins at header/entry-supported `0x800FB368`; its provisional end `0x800FB40F` is Ghidra-analyzer-derived. Loader RX blocks, the 1,721 analyzer-defined functions, and zero relocation entries remain metadata rather than recovered source boundaries. No C candidate was selected. + +## P4-T3 completed evidence and blocker + +`docs/PHASE4_SEGMENTATION.md` records the two bounded failed root-cause attempts: first, an unresolved generated reference prevented link; second, an ignored literal-address linker experiment linked but produced a non-identical file (16 bytes longer; SHA-1 `d8fae51263df598fdff09490fb2dd66aa06b25e3`). No byte values, generated assembly, or experiment configuration are tracked. + +Per `AGENTS.md` stop-on-unexplained-failure rule, segmentation diagnosis stops after these two distinct failures. All ignored experiment artifacts were removed. The unmodified default payload-data fallback was clean-rebuilt and exactly compared: `make clean`, `make all`, and `cmp` exited 0; both files SHA-1 to `e173426c157384ebf1b6caf8c6fea18a85a14af9`. + +## P4-T3R completed evidence + +`docs/PHASE4_REFERENCE_EVIDENCE.md` records new direct static evidence: Ghidra has no reference to `0x80000004`, while the entry instruction at offset `+0x34` has literal upper immediate `0x8000`. The prior generated `D_80000004` is unsupported by this oracle. The temporary P4-T3 layout metadata was removed during restoration, so the 16-byte difference cannot be attributed without a new experiment. + +## P4-T3S completed evidence + +`docs/PHASE4_LITERAL_IMMEDIATE.md` records the ignored pre-link success: Splat `ignore:true` at `0x80000004` removed the unsupported generated reference, and the exact 4-byte instruction at entry offset `+0x34` matched the validated USA range. The object has two `R_MIPS_26` direct-call relocations, so whole-range pre-link bytes are not comparable. No link was attempted. All experiment artifacts were removed, then the default fallback clean rebuild, exact comparison, and recorded SHA-1 gate passed. + +## P4-T3L completed evidence and blocker + +`docs/PHASE4_LINK_ATTEMPT.md` records that the entry object has the two expected direct-call symbols plus seven generated data symbols. The task correctly refused to add any definitions or link because resolving only the two calls would violate its explicit scope. No candidate binary was produced. Ignored artifacts were removed and the default fallback clean rebuild, exact comparison, and recorded SHA-1 gate passed. + +## P4-T3D completed evidence + +`docs/PHASE4_DATA_RELOCATIONS.md` records direct Ghidra reference provenance within the provisional entry range for all seven P4-T3L data relocations. Some targets have other program references; no variable, type, ownership, or original-linker semantics are inferred. No assembly, linker, or C change occurred. + +## P4-T3L2 completed evidence and blocker + +`docs/PHASE4_FULLY_EVIDENCED_LINK.md` records that the exact nine-symbol link completed but reproduced the P4-T3 candidate: 1,886,224 bytes, SHA-1 `d8fae51263df598fdff09490fb2dd66aa06b25e3`, and 1,247,795 differing positions. The relocation hypothesis is therefore eliminated for this entry split; the 16-byte layout/output discrepancy remains unexplained. Ignored artifacts were removed and the default payload-data fallback clean rebuild, exact comparison, and recorded SHA-1 gate passed. + +## P4-T3O completed evidence + +`docs/PHASE4_LAYOUT_EVIDENCE.md` records the directly supported cause: the Splat-generated linker script emits `.text` before `.data` and inserts alignment directives, so the data-prefix/code/data-suffix split is misplaced and `.main` grows by exactly 16 bytes (two 8-byte paddings). The first differing byte is the first payload byte. + +## P4-T3O2 completed evidence (success) + +`docs/PHASE4_ORDERED_LINK.md` records the first verified **assembly** code-subsegment reproduction of the USA executable: an address-ordered ignored linker layout (header `0x800`, then prefix data `0xEB368`, entry `asm` `0xA8`, suffix data `0xE0BF0`) with the P4-T3S `ignore:true` control and the nine evidenced relocation targets reproduced `1,886,208` bytes with `cmp` exit 0 and SHA-1 `e173426c157384ebf1b6caf8c6fea18a85a14af9`. Excluding non-payload metadata sections during `objcopy` was required. The result reproduced on a fresh regeneration. No C work occurred; the tracked default `Makefile` and configuration were not modified, and the default payload-data fallback was re-verified. + +## P4-T4/T5/T6 completion + +- P4-T4: `docs/PHASE4_ENTRY_DOSSIER.md` records the entry dossier. New direct evidence: `0x800FB410` is a call target (function start), `0x80029ED8` is a Ghidra-missed adjacent function start, and no duplicate entry prologue was found. Comparator requirements are recorded; no source introduced. +- P4-T5: `docs/PHASE4_C_REPRESENTATION_CHECK.md` records the bounded negative: the clear-loop idiom compiles identically, but the original uses `lui`+`addiu` while Clang emits `lui`+`ori`. No match claimed; nothing entered the default build. +- P4-T6: `docs/PHASE4_VERIFICATION.md` consolidates the outcome and blocker and records all clean gates as passing. + +## P4-T6 verification results + +Rules check re-read complete before P4-T5. Final clean gates: 28 synthetic tests passed; `make clean`, `make all`, `cmp`, and SHA-1 all green at `e173426c157384ebf1b6caf8c6fea18a85a14af9`; no prohibited path tracked; diff/index clean. + +## Awaiting developer milestone confirmation + +The Phase 4 milestone is ready for confirmation: an address-ordered assembly code-recovery path is established and demonstrated, and C matching is recorded as blocked by the unidentified original toolchain. On confirmation, write a PhaseEnd record and update the digest, then stop. Do not begin a new phase in this session. +