BFM-decomp/phase-ends/PhaseEnd_Phase6.md

PhaseEnd — Phase 6: Compiler fingerprint + first matched functions

Date: 2026-06-14 · Project Version: 1.6.0 · Phase Status: Complete (substantively; the LZSS milestone gate is carried to Phase 7 per Drew — it uniquely needs the jump-table-in-rodata workflow)

Build Log

Files created/changed and complete — do not recreate:

• config/splat.us.exe.yaml — text subsegment flipped [0x800, asm, 800] → [0x800, c, 800]. splat now emits src/800.c (file-scope INCLUDE_ASM stubs) + per-function asm/nonmatchings/800/*.s. Consumed by make extract.
• include/common.h — new, committed prelude (#include "include_asm.h" + u8..f32 typedefs). Every src/*.c includes it; splat references but does NOT generate it.
• Makefile — added: MAPFILE + -Map on the ld line (asm-differ symbol lookup); C_SRCS; OBJS excludes asm/nonmatchings/* (those are textually .included by the .c); the build/src/%.o: src/%.c c-rule (cpp→cc1→maspsx→as); CPPFLAGS/CC1FLAGS (pinned §5.4 flags); MASPSX_FLAGS := --expand-div (pinned). check-env untouched.
• diff_settings.py — new, repo root. asm-differ config: arch mipsel (R3000), object mode vs expected/, gnu map, mipsel-linux-gnu-objdump.
• tools/decompile.py — new, m2c wrapper (fn → asm/nonmatchings/**/<fn>.s → m2c mipsel-gcc-c scaffold).
• tools/permuter/compile.sh + tools/permuter/bin/mips-linux-gnu-objdump — new, committed decomp-permuter harness (build-faithful compile command + the objdump shim the permuter's hardcoded mips- name needs). Per-function setup recipe in cookbook §3.
• docs/matching-cookbook.md — new, the evolvable idiom catalog (I1–I4 idioms, T1–T3 techniques, §3 permuter recipe + -j sweet-spot, §4 flag gotchas, §5 hard-residual classes). The compounding knowledge base.
• requirements-python.txt — new, committed venv freeze (the Phase-4-deferred freeze). Notably pycparser held <3.0 (3.0 removed plyparser, breaking the pinned permuter — hard-won). pip install -r reproduces the venv.
• docs/SETUP.md — §6.6 (the matching loop + NON_MATCHING guard convention) and §5.4 PINNED block (the compiler triple, G8).
• CLAUDE.md — step 5 in the Session Start Protocol: matching phases read the cookbook + §5.4, and evolve the cookbook + tooling (the flywheel).
• src/800.c — the matched-C record (committed): 14 real functions in C (func_80018F20 byte-clamp + 13 accessors), 2 #ifdef NON_MATCHING-guarded near-misses (func_80015A74, func_80016714), and 14 splat-auto empty no-ops (void f(void){}).
• phase-ends/PhaseEnd_Phase6.md — this file. phase-ends/CURRENT_PHASE.md — absorbed into this file and deleted (P8).

Local artifacts (gitignored / regenerable — do NOT recreate):

• asm/nonmatchings/800/*.s (2054 per-function disasm) + include/*.inc + undefined_*_auto.txt + build/ + expected/ — make extract/make build/make expected regenerate them.
• .run/permuter/func_80015A74/ — permuter scratch (recipe in cookbook §3; the C also lives guarded in src/800.c).
• .venv/ — now reproducible from requirements-python.txt.
• Memory (outside repo): matching-cookbook (the flywheel pointer) + MEMORY.md index line.

Tools/packages installed (this phase): venv pip — asm-differ runtime (watchdog 6.0.0, Levenshtein 0.27.3+RapidFuzz, cxxfilt 0.3.0; colorama already present) and decomp-permuter (pycparser 2.23 (<3.0 pin), toml 0.10.2, PyNaCl 1.6.2+cffi). All pinned in requirements-python.txt.

Verification results (literal):

• make clean && make extract && make build && make check → build/us/SLUS_007.26, SHA1 143dbb89f34491258bbc27810d0a12ec8b43a8dd == original (BYTE-IDENTICAL) — through the cpp→cc1→maspsx→as c-path, both at 100% INCLUDE_ASM (the flip) and with 14 real C functions linked.
• Compiler triple PINNED + confirmed (G8): func_80018F20 byte-exact (asm-differ score 0 — the sltiu range-check probe); func_80015A74 division idioms exact (the --expand-div find — divu+break + multu 0xCCCCCCCD); func_80016714 every instruction identical (phantom-frame residual). psx≈cdk and 2.56≈2.67 are byte-equivalent on non-discriminating functions.
• 14 real functions matched (asm-differ score 0): func_80018F20 + 13 accessors (func_80019198/378/388/398/3A8, func_8001AA78/AA88, func_800168B4, func_80019018, func_8001B22C/B374/B384/B85C). +14 splat-auto empty no-ops = 28 byte-matched total.
• decomp-permuter harness validated: base score = 60 on func_80015A74 (== asm-differ); -j 8 ≈ 6600 candidates/30 s (parallel works); -j 30 oversubscribes → exit 144 (sweet spot ~8–16).
• 2 instruction-identical near-misses guarded NON_MATCHING (func_80015A74 hoisted-invariant scheduling; func_80016714 phantom empty stack frame) — decomp-permuter candidates (cookbook §5).
• LZSS spike (read-only recon): LzssDecodeSector needs gcc's switch jump table placed byte-exact at 0x80072A38, inside a mixed blob of 102 jump tables + 161 other data labels → the jump-table-in-rodata workflow → Phase 7 opener (unlocks ~102 switch-functions).
• git status: only config/, Makefile, src/, include/common.h, diff_settings.py, tools/decompile.py, tools/permuter/, docs/, requirements-python.txt, CLAUDE.md tracked; zero ROM-derived/generated bulk staged; all tooling committable; deps frozen.

Milestone achieved: The matching loop is proven and industrialized — the asm→c flip rebuilds byte-identically, the compiler triple is pinned by evidence, and 14 real functions (+14 splat-auto empties) compile to byte-identical machine code (asm-differ score 0; make check BYTE-IDENTICAL). The roadmap's LZSS gate is deferred to Phase 7 (Drew's framing — LZSS uniquely needs the jump-table-in-rodata workflow).

Next: Phase 7 — open in plan mode with a fresh context window. Start with the jump-table-in-rodata workflow → LZSS (first table) → the other ~101 switch-functions; then matching-loop-at-scale (difficulty/duplicate/progress reports, the auto-match harvester) + the Gen1 close-out (≥25 matched, overlay map emulator-proven).

Deviations

Item	Plan	Actual	Reason
LZSS milestone gate	match in Phase 6	deferred to Phase 7	needs the jump-table-in-rodata workflow (roadmap-scoped Phase 7); Drew's milestone-framing decision
Task D "≥3 score-0 probes"	3 score-0 probes	1 score-0 probe + 2 instruction-identical near-misses + 13 tiny-leaf score-0 matches	the medium idiom-rich probes hit gcc byte-exactness quirks (scheduling, phantom frame); the pin is established by instruction-level identity across 3 idiom classes
Task F approach	match probes + LZSS	pivot to tiny trivial leaves (185 available)	medium probes quirk-prone; tiny leaves one-shot from a pattern (the harvester triage, live)
Empty functions	(n/a)	splat auto-emitted empty C for 14 no-ops → free byte-matches	splat c-mode behavior (discovered)
Execution order	A→B→C→D… (checklist)	harness (C) before the ladder (D)	the ladder is scored by asm-differ — the harness is a prerequisite (per the approved plan)
--expand-div	(not anticipated)	pinned globally	the division idiom requires it (bare divu never matches); key fingerprint find
Tooling (Task T / #9)	not in the original plan	added mid-phase (Drew-directed): permuter harness built+validated; m2c --context seed deferred to Phase 7	Drew directive; the flywheel's tier-2
venv requirements freeze	Phase-4-deferred	done at Phase-6 close (requirements-python.txt)	Drew flagged the dep-loss risk; the planned time

Commit Message

feat(phase-6): compiler pinned + matching loop proven (14 matches) — LZSS to Phase 7

- splat.us.exe.yaml: text subseg asm->c (INCLUDE_ASM scaffold); include/common.h
  (committed prelude); Makefile c-rule cpp->cc1->maspsx->as + -Map
- PIN (G8): gcc-2.7.2-psx -O2 -G0 -mips1 -mcpu=3000 -mgas -msoft-float -fgnu-linker
  + maspsx --aspsx-version=2.56 --expand-div; byte-exact on func_80018F20 +
  instruction-identical across 3 idiom classes; --expand-div REQUIRED for div/rem
  (SETUP.md §5.4 PINNED, Makefile MASPSX_FLAGS)
- harness: diff_settings.py (asm-differ mipsel object mode), tools/decompile.py
  (m2c wrapper), tools/permuter/ (decomp-permuter harness + objdump shim), expected/
- 14 real functions matched (asm-differ score 0): func_80018F20 + 13 accessors;
  +14 splat-auto empty no-ops; make check BYTE-IDENTICAL throughout
- 2 instruction-identical near-misses guarded NON_MATCHING (func_80015A74 scheduling,
  func_80016714 phantom frame) -> decomp-permuter candidates
- flywheel: docs/matching-cookbook.md, memory + CLAUDE.md step 5 (consult+evolve), R16
- requirements-python.txt: venv freeze (pycparser<3.0 pinned — permuter needs plyparser)
- LZSS spike: needs jump-table-in-rodata workflow (102 tables, mixed blob) -> Phase 7
- venv: asm-differ + permuter deps installed
- bumps project version 1.5.0 -> 1.6.0

Rules Added This Phase

Rule

Reason

R16 — The matching automation flywheel. Consult the knowledge base (docs/matching-cookbook.md + the pinned triple §5.4 + the decomp-permuter harness + the m2c --context) before each match, and after each match — especially a hard-won near-miss ground to byte-perfect — feed the GENERALIZABLE lesson back into both the cookbook (the idiom/technique) and the tooling (a permuter PERM_* recipe / weight, an m2c context entry). Don't over-encode one-offs; the target is the average function trending to one-shot + a shrinking hard tail, not literal zero-intervention. Wired into the matching-cookbook memory + CLAUDE.md step 5.

Drew-directed (2026-06-14). The ~99.9%-autonomous matching model only scales if each match makes the next one cheaper; this makes "consult + evolve" a standing obligation, not a one-off.

PhaseEnd Changelog

v1.5.0 → v1.6.0 — Phase 6 complete (substantively; the LZSS gate is carried to Phase 7, which it fits better — it uniquely needs the jump-table-in-rodata workflow). The matching loop is proven and industrialized: the asm→c flip rebuilds byte-identically; the compiler triple is PINNED by evidence (gcc-2.7.2-psx -O2 -G0 … + maspsx --aspsx-version=2.56 --expand-div), with --expand-div the key discovery; the full harness is wired (asm-differ object mode + tools/decompile.py + the decomp-permuter harness + the expected/ baseline); 14 real functions match at asm-differ score 0 (a byte-clamp + 13 accessors) plus 14 splat-auto empties, make check BYTE-IDENTICAL throughout. The tiny-leaf harvest validated the harvester triage (easy=auto, medium=permuter, hard=defer) — there are 185 tiny leaves and ~102 switch-functions waiting. A knowledge flywheel is now standing (cookbook + memory + CLAUDE.md step 5 + R16), and the venv is reproducible (requirements-python.txt, with the hard-won pycparser<3.0 pin). Two instruction-identical near-misses are guarded as permuter candidates. The LZSS recon scoped Phase 7's opener.

🛑 Stop Here

PhaseEnd written; CURRENT_PHASE.md absorbed and removed. Drew commits AND pushes this PhaseEnd plus the whole Phase-6 change set (R6/R8 — one commit lands: config/, Makefile, src/800.c, include/common.h, diff_settings.py, tools/decompile.py, tools/permuter/, docs/ (SETUP §5.4/§6.6 + matching-cookbook.md), requirements-python.txt, CLAUDE.md, PhaseEnd_Phase6.md). Start a fresh session, in plan mode, for Phase 7 — opening with the jump-table-in-rodata workflow → LZSS. Keep this file forever.