BFM-decomp/phase-ends/PhaseEnd_Phase26.md

# PhaseEnd — Phase 26: The Family-Endgame → the tooling-integrity audit + the §52 discovery-flywheel (mechanical harvest byte-proven exhausted)
**Date:** 2026-07-15 · **Project Version:** 1.25.0 · **Phase Status:** Complete (a PIVOT close — the mechanical/templating thesis reached its byte-gate ceiling; structural completion deferred to Phase 27; Phase-16 precedent) · **Generation:** Gen2 (18th phase of the arc; Phase 14 public-flip deferred to Gen3+)

> Gen2 phase 18 (8→…→25→**26**). The **longest phase**, run across **~13 sessions (2026-07-11 → 07-15)** and containing an **inserted half-phase (26-A, the tooling-integrity audit)** — the Phase-3.5 precedent. The granular per-task/per-session crash-recovery trail (Tasks 0–6, the A0–A11 audit ledger, the §52 flywheel waves) is preserved on-demand at **`phase-ends/logs/Phase26.md`** (R19 — NOT auto-loaded). This file is the synthesis. Owner decisions (Drew): open from the megaplan → insert the audit as a half-phase (gate-1 2026-07-14) → resume Task 7 at xHigh → single-Fable5 test → redirect to the §52 cheap-Opus flywheel → on the byte-proven exhaustion of the mechanical harvest, **close the phase and open Phase 27 with fresh byte-gate-honest scans**.

## Build Log

**Files created/changed and complete — do not recreate** (the durable Phase-26 output; the archived log has the exhaustive per-commit list):

*The mechanical family engine (Tasks 1–6, the phase's original substance):*
- `tools/family_remap.py` — extended `reloc_targets` for the add/addu indexed-global idiom (the "reach-1 tail" was largely this tracker blind spot); backward-compatible cross-address `to_addr` + `imm_map` hook; single-pass simultaneous substitution (fixed the latent chained-rename bug); the shared word-diff classifier (`stream_words`/`classify_member` — PURE/IMM/STRUCT). `norm_stream`/`h_norm` UNTOUCHED.
- `tools/family_hseq.py` — full-frontier h_seq survey (the reframe engine) → `.run/family_hseq.json` + `docs/family-hseq.md`.
- `tools/family_sweep.py` — the two-phase stage-then-gate driver: `hseq_sweep()` (crack one exemplar → positionally remap per-overlay reloc/imm symbols → whole-binary byte-gate), `--only`, `--band`, `--reconcile-raw` (§41c per-sibling reconcile), the §42e static pin guard. **This session: two fixes** — the pin-guard now strips comments before the `__asm__("$` check (a body that DOCUMENTS a removed pin is pin-free code, was false-skipping 133 members); and `--allow-pins` (bypass the guard, let the byte-gate arbitrate — added to TEST the func_8017A4AC pinned-×134 precedent).
- `tools/family_manifest.py`, `tools/canon_sig_reconcile.py` v3.2, `tools/rtu_match.py`, `tools/t7_bank.py`, `tools/bank_exemplar.py` — the curriculum + recovery + real-TU-faithful match tooling (Tasks 4–6).

*The tooling-integrity audit (26-A — the inserted half-phase, A0–A11; the phase's most consequential work):*
- **`tools/corpus.py`** — ONE derived corpus oracle (derives the binary's `.c` set + each function's `.s` home from the FILESYSTEM and `matched = sig − stubs` from the PROVEN INVARIANT, never re-parsed). Coverage-asserted (R32). Deleted ~4 decaying scanners.
- **`tools/cdecl.py`** — the ONE coverage-asserting C-declaration parser (recursive descent over C's total declarator grammar; ~250 lines replacing 15 divergent regexes). `make audit-cdecl` gate. gcc adjudicates its own coverage gap.
- **`make audit-corpus`** — the SECOND ORACLE (R34): cross-checks splat's boundaries against `sig_image`'s independent ones; reproduced the A2 audit's 193-slice number from an independently-written tool.
- `config/symbols.us.ram.txt` — the `listCdBuffer` fix (A4): a symbol whose address falls inside another binary's vram window must never enter that binary's symbol stack (193 unmatchable slices → 0).
- `tools/reconcile_tu.py` (wired into the bank paths, retiring the fleet-majority oracle), `tools/masked_diff.py` (PC16 fix: 150 closeness lies → 4), `build_engine_types` (unblocked from 81% hard-exit), `jr_isolate_all`/`overlay_src_split`/`jr_inventory` (silent byte-changer + coverage fixes), `lint_symbol_refs` (green + wired), `extract` invalidates stale objects (a stale `.o` can produce a FALSE PASS). **DELETED:** `census_conflict_callees`, `derive_canonical_sigs` (R33). Full ledger: `docs/tooling-audit.md` (AUDIT-CLOSE LEDGER).

*The §52 discovery-flywheel (Task 7, this session — the endgame lever):*
- `src/ov_SC01_077/**` + the 134 overlay `.c` files + `config/dedup.us.yaml` — the banked matches. Task-7 this session banked **5 pin-free regalloc-order cores ×134 = 670 function-instances** (`func_80171FFC`, `func_801775E0`, `func_801379FC`, `func_801497A8`, `func_801495C4`).
- `docs/matching-cookbook.md` **§40–§52 + §52a + §52b** — the mature family + gcc-2.7.2 regalloc toolkit. **§52** (the walker-family skeleton, distilled from the Fable5 `func_80178004` wall) + **§52a/§52b** (the sibling-wave de-pin levers + FOUR named intrinsic-wall classes: flagship `$s0` local-alloc, caller-saved priority-first-fit, non-coalescing delay-slot copy, symbol-address-base wins-low-needs-high).
- `docs/decision-log.md` (R31 — the audit thesis, the single-Fable5 wall, the §52 flywheel, the mechanical-harvest-exhausted pivot), `docs/tooling-audit.md`, `docs/family-endgame-megaplan.md`.
- `phase-ends/CURRENT_PHASE.md` → **`phase-ends/logs/Phase26.md`** (R19). `phase-ends/PhaseEnd_Phase26.md` — this file.

**Tools/packages installed:** None — the whole phase used the existing Phase-4/6/…/25 toolchain + venv. Fable5/Opus agents via the Agent tool.

**Verification results (literal) — the milestone proof:**
- **`make clean && make extract-all && make check-all` → 136 passed, 0 failed of 136** BYTE-IDENTICAL from a genuinely-clean tree (R22), reproduced after every banked batch this phase. The whole-binary byte-gate (`harvest_verify`, G3/P9) was the sole per-bank arbiter and **never accepted a wrong match**.
- **Fleet: instr-weighted 58.2% → 68.9%** (the phase arc: family engine + sessions 6–8 cracks 58.2→66.5; the 26-A audit fixes 66.5→68.6; the §52 flywheel this session 68.6→68.9) · **distinct-code 30.3% → 49.2%** (53,429 / 84,996 unique fns) · fn-count → 84.47%. **0 NON_MATCHING** in any default build (G4); **dedup 1840 validated / 0 failed**; 53,371 INCLUDE_ASM stubs.
- **`make report` / `make audit-corpus` (0 slices) / `make audit-cdecl` all green.**
- **The byte-proven pivot finding (R14/P9):** the matched-sib mechanical harvest is **exhausted** — 3 byte-gate probes returned 0% (tiny-IMM 0/241, PURE reach-134 0/134, pinned-PURE-with-pins 0/133). The manifest's ~13,075 "templatable" member-slots are an h_seq *prediction* the whole-binary gate refuses (collision / register-drift / pin-crash). Phase 26's "crack ~986 exemplars → template ×120" thesis is spent at this ceiling.
- `git status`: only `config/`/`tools/`/`src/`/`docs/`/`phase-ends/` tracked across the phase; zero ROM-derived/generated bulk staged; the `db.*.gbf` churn is R23 restart-noise (never staged); **no Ghidra DB change** (matching used cached asm).

**Milestone (honest — a PIVOT, not the stated milestone):** Phase 26's stated milestone was *structural completion*, and it is **NOT reached** (68.9% instr / 49.2% distinct). What the phase DID deliver: (1) the mechanical family engine + the h_seq reframe; (2) the **tooling-integrity audit** that found the tools *were* several of the walls, fixed ~15 of them, deleted decaying scanners, and built two derived oracles + a second disagreeing oracle (R32/R33/R34); (3) the **§52 discovery-flywheel** — a single Fable5 pass that *failed* to crack the flagship `func_80178004` nonetheless distilled the walker-family idiom that cheap-Opus waves then applied to bank **670 instances** and precisely characterize four intrinsic-wall classes; and (4) the **byte-proven finding** that the mechanical/templating harvest is exhausted at this ceiling. The structural remainder (hard-wall cracks + hand-decomp of the genuine unique residue) is a distinct phase, opened fresh in Phase 27 with byte-gate-honest re-scans. A Phase-16-style honest, finding-driven close.

**Next:** **Phase 27 — the hard-remainder endgame, re-scanned honest.** A fresh session, **plan mode, Max** (Tier-1). Open by **re-scanning the frontier with the now-bugfree tools AND validating templatability against the byte-gate** (not h_seq prediction), so the plan is built on reality, not a manifest that over-promises ~13k phantom members. Then scope the genuine remainder: the harder crack tail (Fable5 discovery → §52-flywheel application, diminishing), the documented giant walls (permanent INCLUDE_ASM per G4 unless a new idiom dissolves them), and hand-decomp of the true unique residue.

## Deviations
| Item | Plan | Actual | Reason |
|---|---|---|---|
| Phase shape | the megaplan: crack ~986 h_seq exemplars → template ×120 | family engine built + a **tooling audit inserted (26-A)** + a **§52 discovery-flywheel** + the templating thesis **byte-proven exhausted** | the megaplan's scans were run on BUGGY pre-audit tools; the audit found the tools were the walls; the byte-gate then refuted the templatability the manifest predicted |
| 26-A audit | (not in the original plan) | **inserted as a half-phase** (Drew, Phase-3.5 precedent) | seven silent-skip tool bugs in one session revealed a structural blind spot; a hole in a SELECTION tool makes work invisible to planning — fix before matching on top of it |
| Task 7 model tier | Fable5 per giant (window ~7/13) | window extended to 7/19; **single-Fable5 test (xHigh) → wall → redirect to cheap-Opus §52 flywheel** | the flagship is an intrinsic 3-integer regalloc wall (Fable5 at 477k tok couldn't crack it); but its idiom made cheap-Opus waves productive (Fable5 DISCOVERS, cheap-Opus APPLIES) |
| Task 8 (mechanical harvest) | the big deterministic lever (~2.6M templatable ins) | **byte-proven TAPPED** (3 probes, 0%) | A3h + the wave propagations banked everything cleanly templatable; the remaining manifest "templatable" is collision/drift the gate refuses (R14: verify predictions vs the bytes) |
| Tasks 9/11 | mid-band harvest + residue map | **folded into the Phase-27 fresh-scan re-plan** | the mechanical bands are the same tapped thesis; an honest residue map needs the byte-gate-validated re-scan, so it belongs at the Phase-27 open |
| Milestone | structural completion | **NOT reached (68.9%); honest PIVOT close** | the mechanical thesis reached its byte-gate ceiling; structural completion is the distinct hard phase (Phase-16 precedent — a finding is a valid phase deliverable, P9) |
| Effort | Max throughout | Max (planning/audit/synthesis) · xHigh (the §52 flywheel orchestration + waves) · Fable5 (the single wall-breaker test) · Ultracode (the A2 audit fan-out) | R26/R27 — breadth under Ultracode, discovery under Fable5, deep synthesis at Max, routine orchestration at xHigh |

## Commit Message
```
(Phase 26 landed across ~13 sessions of checkpoint commits; the 26-A audit close = commit:0622;
this session's §52 flywheel = commit:0623..commit:0626; this close = the archived log + this PhaseEnd.)

feat(phase-26): close — family engine + the tooling-integrity audit + the §52 discovery-flywheel;
mechanical harvest byte-proven exhausted; fleet 58.2->68.9% instr (v1.25.0)

- FAMILY ENGINE (Tasks 1-6): family_remap (extended reloc tracker + single-pass subst) +
  family_hseq (the h_seq reframe) + family_sweep (crack-one -> template-x134 -> byte-gate) +
  canon_sig_reconcile v3.2 + rtu_match. Sessions 6-8 cracked 13 cores (58.2->66.5% instr).
- 26-A TOOLING-INTEGRITY AUDIT (inserted half-phase, A0-A11): the tools WERE several of the
  walls. Fixed ~15; DELETED decaying scanners (R33); built corpus.py + cdecl.py (derived,
  coverage-asserted oracles) + make audit-corpus (a SECOND, disagreeing oracle, R34); the
  listCdBuffer 193-slice corpus defect -> 0; masked_diff 150 closeness-lies -> 4; the stale-
  object false-pass closed. Payoff 66.5->68.6% instr. docs/tooling-audit.md AUDIT-CLOSE LEDGER.
- 52 DISCOVERY-FLYWHEEL (Task 7): single Fable5 on func_80178004 = intrinsic 3-integer regalloc
  wall, BUT distilled the walker-family idiom (52); two cheap-Opus waves applied it -> 5 pin-free
  cores banked x134 = 670 instances (68.6->68.9%). 4 named wall classes; 52/52a/52b. pin-guard
  comment false-positive fixed; family_sweep --allow-pins.
- FINDING (R14/P9, 3 probes 0%): the matched-sib mechanical harvest is EXHAUSTED; the manifest's
  ~13k "templatable" members are an h_seq prediction the byte-gate refuses. Phase-26's templating
  thesis is spent -> close, open Phase 27 with byte-gate-honest re-scans.
- R22 clean-fleet 136/136 BYTE-IDENTICAL throughout; dedup 1840/0; 0 NON_MATCHING (G4).
- rules R32 (coverage assertion) / R33 (derive, don't re-derive) / R34 (a second, disagreeing
  oracle). worklog -> phase-ends/logs/Phase26.md (R19). bumps 1.24.0 -> 1.25.0.
```

## Rules Added This Phase
| Rule | Reason |
|---|---|
| **R32 — Assert your COVERAGE.** A tool that scans the corpus must compare what it found against an **over-approximating** candidate set and fail on the gap. A silent skip is a DEFECT, not a no-op — and note (the corrected form): **a loud failure that nobody counts is exactly as invisible as a silent one** (`build_engine_types` hard-exited on 81% of its corpus for four phases while "failing loud"). | The audit's root cause: a scanner extracts N items, the true count is M>N, and nobody ever compared N to M. Seven silent-skip bugs in one session were one structural blind spot. (Drew: "agreed", 2026-07-14.) |
| **R33 — Derive, don't re-derive.** Where a proven invariant answers the question, derive the answer from it rather than re-parse the source. **The best outcome is a DELETED SCANNER, not a fixed regex.** Before adding a coverage assertion to a scanner, first ask whether the scanner should exist at all. | 28 audit findings collapsed to one derived oracle (`corpus.py`) + ~10 deleted/retired scanners. The build invariant (`INCLUDE_ASM` pastes the ORIGINAL asm ⇒ a function not wrapped in it is byte-exact) makes most re-parsing scanners unnecessary. |
| **R34 — A second, DISAGREEING oracle — not a better assertion.** When an oracle is *structurally* blind to a class of error, no assertion inside it can help; add an INDEPENDENT oracle that can disagree with it, and make them argue. | The whole-binary byte-gate is a perfect CORRECTNESS oracle and a NULL COVERAGE oracle (green since Phase 5 at 0% decomp, because `INCLUDE_ASM` pastes the original). `sig_image` disagreeing with splat is the only reason the 193 `listCdBuffer` slices were ever visible — we had both oracles all along and never compared them. |

*(No OTHER new governance rules — per the Phase-8+ precedent, the phase's other lessons are techniques (→ cookbook §40–§52b) + findings (→ decision-log + the megaplan). The **§52 discovery-flywheel** ("a frontier model that FAILS to crack a wall still distills the idiom that lets a cheap tier crack its non-wall siblings; Fable5 DISCOVERS, cheap-Opus APPLIES") is captured as cookbook §52 + the `matching-is-solved-integration-is-the-bottleneck` / `structural-family-mechanical-remap` memories. The **"a manifest's h_seq prediction is not a bank — validate templatability against the byte-gate"** lesson is a direct application of R14/G3. G3/P9/R14/R16/R22/R26/R27/R30/R31 governed the phase.)*

## PhaseEnd Changelog
**v1.24.0 → v1.25.0 — Phase 26 complete (Gen2 phase 18; the longest phase; a PIVOT close).** Opened from the family-endgame megaplan ("crack ~986 h_seq exemplars → template ×120"), the phase built the **mechanical family engine** (`family_remap`/`family_hseq`/`family_sweep`, crack-one-then-template-×134-under-the-byte-gate) and, mid-arc, **inserted the tooling-integrity audit (26-A)** — the phase's most consequential work: the tools *were* several of the walls the project had "byte-proven" across 26 phases. The audit fixed ~15 tools, **deleted** decaying scanners (R33), built **derived, coverage-asserted oracles** (`corpus.py`, `cdecl.py`) and a **second, disagreeing oracle** (`make audit-corpus`, R34), and dissolved corpus defects (the 193 `listCdBuffer` slices; 150 `masked_diff` closeness-lies; the stale-object false-pass), banking 66.5→68.6% instr. Task 7 then delivered the **§52 discovery-flywheel**: a single Fable5 pass *failed* to crack the flagship `func_80178004` (a genuine 3-integer regalloc wall) but distilled the **walker-family idiom** that two cheap-Opus waves applied to bank **5 pin-free cores ×134 = 670 instances** (68.6→68.9%) and precisely characterize **four intrinsic-wall classes** — the flywheel principle *Fable5 DISCOVERS, cheap-Opus APPLIES*, proven end-to-end. The phase then **byte-proved (3 probes, 0%) that the matched-sib mechanical harvest is exhausted** — the manifest's ~13k "templatable" members are an h_seq prediction the whole-binary gate refuses. So Phase 26's templating thesis is spent at **68.9% instr / 49.2% distinct** (from 58.2%/30.3% at Phase-25 close — a large arc), the *structural completion* milestone is **honestly not reached**, and the phase closes on that finding, handing the hard remainder (wall-cracks + hand-decomp) to a **Phase 27 that re-scans the frontier honest** (byte-gate-validated, not h_seq-predicted). Three governance rules (**R32/R33/R34** — the audit's real deliverable, the derived-oracle discipline that makes the next 26 phases unable to manufacture a wall out of a lookup miss). 136/136 byte-identical, 0 NON_MATCHING, dedup 1840/0 throughout.

## Plain-English Recap
This was the longest stretch of the project, and it ended by *changing what we believe about the finish line*. Three things happened. **First**, we built a "stamp machine" — reconstruct one copy of an engine function, and the tool stamps the matching copy into all ~130 game locations, each checked bit-for-bit. **Second — the big one — we audited our own tools and discovered that many of the "unbeatable compiler walls" we'd been recording for months were actually *bugs in our own scanners*: tools that silently skipped work and made it look like there was nothing there.** We fixed them, deleted the worst ones, and built new tools that *derive* the truth from the build instead of guessing, plus a second independent tool whose whole job is to *disagree* and catch the first one's blind spots. That alone recovered a couple percent of the game. **Third**, we hit a real wall on the single hardest function even with the best AI in the world — but that failed attempt *taught us the compiler's exact behavior*, which cheaper AI then used to reconstruct **670 more pieces** and to explain precisely which remaining ones are genuinely impossible. Finally, we proved — by actually testing it — that the "stamp everything" approach has now given all it can (the leftover functions aren't clean copies of each other; they genuinely differ). So we're **closing this phase honestly**: we went from ~58% to ~69% of the game rebuilt bit-for-bit, we made our tools trustworthy, and we learned that the rest of the work is a *different kind* of work — hard, one-at-a-time reconstruction — which deserves a fresh start with clean tools and an honest re-scan of exactly what's left. The most important lasting win isn't the percentage; it's that our instruments no longer lie to us, and we wrote down three rules to keep it that way. Everything still rebuilds perfectly; nothing broke.

## 🛑 Stop Here
PhaseEnd written; `CURRENT_PHASE.md` archived → `phase-ends/logs/Phase26.md` (R19, via `git mv` — left uncommitted for Drew's milestone-close commit). The Phase-26 matching + audit work is committed across the phase (26-A close `commit:0622`; this session `commit:0623`..`commit:0626`); **Drew commits AND pushes** this PhaseEnd + the archived log (R6/R8 — the milestone-close is the human-controlled gate-2 sign-off). No Ghidra DB change this phase (R23 no-op — the `db.*.gbf` churn is restart-noise; do NOT stage). Gen2 continues — do **NOT** start Phase 27 here. Start a **fresh session** (effort **Max**, **plan mode**) for **Phase 27 — the hard-remainder endgame**, opening with a byte-gate-honest re-scan of the frontier (the now-bugfree tools + templatability validated against the gate, not h_seq prediction), then scope the wall-cracks + hand-decomp residue. Keep this file forever.