BFM-decomp/phase-ends/PhaseEnd_Phase25.md

# PhaseEnd — Phase 25: The structural-family endgame → the giant endgame + the h_seq family reframe
**Date:** 2026-07-11 · **Project Version:** 1.24.0 · **Phase Status:** Complete (milestone confirmed by Drew, gate 2) · **Generation:** Gen2 (17th phase of the arc; Phase 14 public-flip deferred to Gen3+)

> Gen2 phase 17 (8→…→24→**25**). Ran across **many sessions (2026-07-08 → 07-11)** — the longest matching phase.
> The granular per-task/per-session crash-recovery trail (T0–T7 mechanical tiers, the wave rounds, the giant
> campaign, the permuter batch, the reach/family surveys) is preserved on-demand at
> **`phase-ends/logs/Phase25.md`** (R19 — NOT auto-loaded; consult only when researching a mechanism). This file
> is the synthesis. Owner decisions (Drew): exemplar-per-family sweep, cheap-tier-first with Opus/Fable5/permuter
> escalation → giant campaign → on the **h_seq family reframe**, close the phase and hand a **family-endgame
> megaplan** to a fresh plan-mode + Fable5 **Phase 26**.

## Build Log

**Files created/changed and complete — do not recreate** (the durable Phase-25 output; the archived log has the exhaustive per-commit list):

*The mechanical family engine (the phase's economic core, T2–T7):*
- `tools/family_manifest.py` — regroup the unmatched frontier by `h_norm`; per family the ov_SC01_077 drafting exemplar + instances (×N leverage) + byte-weight → `docs/family-manifest.md`.
- `tools/family_remap.py` — crack ONE exemplar → positionally substitute each member's per-overlay reloc symbols → plain `harvest_verify` byte-gate. ~0 agent tokens/member. **The T3 proof that h_norm families are TEMPLATES (reloc-symbol-remap), not free dedup.**
- `tools/family_sweep.py` — two-phase stage-then-gate driver; `--only`, `--reconcile` (per-sibling canon_sig_reconcile, the Q5 law), `--edit-remap` (§42e out-of-body //@EDIT + once-global engine_core.h flips), `--no-preclassify`.
- `tools/canon_sig_reconcile.py` **v3.2** — the def-side loose-typing-wall cracker (strip ambient dups → rewrite def to canonical sig → cast changed params AT USE; uniquify draft type names). The T6 headline tool.
- `tools/t7_bank.py` (M1 driver, reconcile-at-bank-time), `tools/recover_giant.py` (canonical-extern block-scope recovery), `tools/rtu_match.py` (real-TU-faithful parallel-safe match check — closes match_one's isolation blind spot), `tools/progress.py --weighted` (the honest byte/instruction-weighted metrics).

*Config / source (the harvest — fleet 66.02% → ~74.6% fn-count):*
- The 134 overlay `src/ov_*/*.c` files — thousands of member-matches banked via the mechanical family sweeps (T7 M1 37 + M2 4,389 + M3 268; waves 2–4; the matched-free +16,512 / type-lift +1,729 / h_exact stragglers). `src/shared/engine_core.h` + `engine_types.h` (typedef/struct lifts). `config/dedup.us.yaml` (1813 groups, 0 failed).
- **The giant campaign** (Task A, tonight + prior): all matchable region-a giants ×134 — batch 1 (`func_80166994` 369 §43 K&R-s16, `func_80135480` 258, `func_80163EC8` 234) + **the flagship `func_80133CD4` (399 ins) ×134** (§45 Fable5 crack, `commit:0515`). `func_8014FBC0` (22×1996 inline-asm trampoline) + `func_80132144` (27×539) among the wave banks.

*Docs / knowledge base (the mature gcc-2.7.2 + family toolkit):*
- `docs/matching-cookbook.md` **§40–§45**: §40/§40a (structural-family mechanical-remap), §41/§41a–c (canon_sig_reconcile + the ×134 per-sibling re-reconcile law + the object-probe≠gate correction), §42/§42a–e (density-lever catalog, the stale-object gate trap, rtu_match, the wave-4 levers, the `--edit-remap` propagation), §43 (K&R s16-param dissolves the narrow-param wall), §44 (cheap-Opus giant batch, 5 levers + "each giant is its own class"), **§45 (the flagship crack — merged-accumulator-variables + the 1-death local-alloc in-out-asm + offset-0 /s + goto-shared-return; the template for the family-core cracks)**.
- `docs/phase25-t6-curriculum.md` (Fable5-authored crack curriculum), `docs/phase25-frontier-map.md` (rtu-measured endgame buckets), **`docs/family-endgame-megaplan.md`** (the Phase-26 brief — the h_seq family reframe + the finish plan), `docs/decision-log.md` (R31, strategic-pivot rationale), `docs/backlog.md` / `docs/worklist.md`.
- `phase-ends/CURRENT_PHASE.md` → **`phase-ends/logs/Phase25.md`** (R19). `phase-ends/PhaseEnd_Phase25.md` — this file.

**Tools/packages installed:** None — the whole phase used the existing Phase-4/6/…/24 toolchain + venv (decomp-permuter, m2c, rabbitizer, cc1 candidates). Fable5/Opus agents via the Agent tool.

**Verification results (literal):**
- **`make check-all` → 136 passed, 0 failed of 136** — reproduced from a genuinely-clean R22 tree (`make clean` + extract-all-136 + check-all) after every banked batch. The whole-binary byte-gate (`harvest_verify`, G3/P9) was the sole per-bank arbiter and **never accepted a wrong match** across the phase.
- **Fleet: 66.02% → ~74.6% fn-count** (the arc: 66.02 → 70.82 → 71.36 → 72.18 → 73.66 → 73.97 → 74.36 → 74.48 → 74.58 → tonight's flagship). **Weighted (the honest metrics, `progress.py --weighted` introduced this phase): instr-weighted 58.2% (decomp.dev-display), distinct-code 30.3%** (29,322 / 84,996 unique fns). **0 NON_MATCHING** in any default build (G4); `dedup-check` 1813 validated / 0 failed.
- **Flagship `func_80133CD4` (399 ins) byte-identical** (whole-binary gate `d19c9580`), `family_sweep` **133/133** siblings ×134, PIN-FREE — the ~22-phase "unsteerable whole-function register permutation" retired.
- **The reframe (byte-verified tonight, R14):** the "36k unique / 87.3% distinct-code tail" is a **strict-`h_norm` artifact** — re-clustered by `h_seq` (mnemonic skeleton), **90% collapses into ~754 families** (986 substantial nins≥80 / 1.88M ins, top-20 = 52%), all per-location-shaped, templatability confirmed (identical nbytes/ncalls/call-sequence across ~120 members).
- `git status`: only `config/`/`tools/`/`src/`/`docs/`/`phase-ends/` tracked across the phase; zero ROM-derived/generated bulk staged; the `db.*.gbf` churn is R23 restart-noise (never staged); **no Ghidra DB change** (matching used cached Ghidra-C + asm).

**Milestone achieved (confirmed by Drew, gate 2):** The structural-family endgame drove the fleet **66.02% → ~74.6% fn-count (58.2% instr-weighted / 30.3% distinct-code)**, banking the mechanical `h_norm`-family sweep engine (`family_remap`/`family_sweep`, ~20k+ member-matches for ~0 agent tokens), the T6/T7 curriculum + recovery tooling (`canon_sig_reconcile` v3.2, `rtu_match`), and the **complete giant endgame** (every matchable region-a giant ×134, capped by the 399-ins flagship `func_80133CD4` via the §45 Fable5 gdb-on-cc1 crack) — then **closed on the strategic reframe** (R14) that the remaining "unique tail" is really **~986 templatable `h_seq` families**, the discovery that defines the finish plan and is handed forward as `docs/family-endgame-megaplan.md`. 136/136 byte-identical, 0 NON_MATCHING throughout.

**Next:** **Phase 26 — the family-endgame (finish the decomp).** A fresh session, **plan-mode + Fable5** (Tier-1 Phase-Start). Start from **`docs/family-endgame-megaplan.md`**: Step A (cheap `h_seq` family map + templatability validation + exemplar triage — the measured foundation) → Step B (Fable5 hard family CORES + the 2 permanent walls + the 3 walled reach-134 giants) → Step C (`family_remap` immediate-substitution + the mechanical ×120 harvest + the 5.9 MB matched-free propagation) → Step D (the genuine unique residue + the 5 behemoths — last). **Fable5 is gone after ~2026-07-13** — spend it only on the hard family cores + the class-walls (permanent idioms), never the unique behemoths.

## Deviations
| Item | Plan | Actual | Reason |
|---|---|---|---|
| Milestone shape | "close when top-family ROI drops" (open-ended) | closed on the **h_seq reframe** + a giant-endgame checkpoint | the cheap tiers were byte-proven exhausted on the giant/near-miss tail, and the reframe redirects to a distinct approach (a new phase) |
| The "unique tail" | assumed ~36k unique hand-decomps (strict h_norm) | **90% is ~754 h_seq families** (R14 — the grouping was the artifact, not the code) | Drew's intuition + a byte-verified re-cluster; the single most consequential finding of the phase |
| Giant route | permuter/cheap-Opus first, Fable5 on walls | flagship cracked by **Fable5 gdb-on-cc1** (§45); 3 flywheel giants + F-band permuter batch all WALLED → Fable5 | RC-6/S3/allocno-tie intrinsic ties; the free tiers improve *toward* but can't close them (byte-proven) |
| Fable5 usage doctrine | (not pre-specified) | **parallel-isolated agents, distill-between, NOT batched** (context-accumulation is ~N× costlier; the §31 map already amortizes the source-read) | measured this session; the flagship = 125 tool-calls / 506k tokens solo |
| v4 local model | retrain + gate | **FAILED/negative — keep v3** (7B capacity-bound) | corpus-quality>size; the local tier is a $0 ≤15-ins mop-up, off the critical path |
| Phase close | write PhaseEnd + continue | close + hand a **megaplan** to a fresh plan-mode/Fable5 phase | the finish is a distinct plan-mode Phase Start (two-gate model); Drew's call |

## Commit Message
```
(Phase 25 landed across many sessions of checkpoint commits; key closes: commit:0514 giant batch 1,
commit:0515 flagship ×134 + §45, commit:0516 family-endgame megaplan + checkpoint, + this PhaseEnd.)

feat(phase-25): close — structural-family endgame + the giant endgame + the h_seq family reframe (v1.24.0)

- MECHANICAL FAMILY ENGINE: family_manifest/family_remap/family_sweep + canon_sig_reconcile v3.2 +
  rtu_match; ~20k+ member-matches banked for ~0 agent tokens (h_norm families are TEMPLATES, T3).
  Fleet 66.02% -> ~74.6% fn-count (58.2% instr-weighted / 30.3% distinct-code, progress.py --weighted).
- GIANT ENDGAME COMPLETE: every matchable region-a giant ×134 incl. the 399-ins flagship func_80133CD4
  (§45 Fable5 gdb-on-cc1 crack: merged-accumulator-variables + 1-death local-alloc in-out-asm + /s +
  goto-ret; the ~22-phase 'unsteerable' verdict retired). 3 flywheel giants + F-band permuter batch
  WALLED -> Fable5 (recon-done pin-free seeds + gdb specs).
- THE REFRAME (R14, byte-verified): the '36k unique / 87% distinct-code tail' is a strict-h_norm
  artifact; re-clustered by h_seq it collapses 90% into ~754 families (986 substantial / 1.88M ins,
  top-20=52%), templatability confirmed. Endgame = crack ~986 exemplars -> template ×120.
- cookbook §40-§45; docs/{phase25-t6-curriculum,phase25-frontier-map,family-endgame-megaplan}.md.
  136/136 byte-identical (R22), 0 NON_MATCHING (G4), dedup 1813/0. No new governance rules (techniques
  -> cookbook; the reframe -> the megaplan). worklog -> logs/Phase25.md (R19). bumps 1.23.0 -> 1.24.0.
```

## Rules Added This Phase
| Rule | Reason |
|---|---|
| **R31 — Capture the WHY behind strategic pivots in `docs/decision-log.md`, while fresh** (confirmed by Drew 2026-07-08, binding). At each major direction change / dead-end / reversal, log (during the producing session): context+belief → what failed → the pivot → the byte/measurement-grounded why → a hindsight "better path." Extends R30 (technical artifacts) to STRATEGIC/decision rationale — the perishable judgment the terse Deviations table can't hold. Forward-only. The substrate for the eventual retrospective + the public "how to AI-decomp" wiki. | The terse PhaseEnd loses the reasoning; a fresh session reconstructing from compressed summaries would lose it. Exercised all phase (the T3 pivot, the giant campaign, the h_seq reframe). |

*(No OTHER new governance rules — per the Phase-8+ precedent, the phase's lessons are techniques (→ cookbook §40–§45) + findings (→ the megaplan + the `structural-family-mechanical-remap` memory, updated with the h_seq refinement). The **h_seq re-cluster methodology** ("before declaring code unique, regroup by the looser fingerprint — strict h_norm over-fragments per-location families on immediates") is a direct application of **R14** (verify the grouping against the bytes) and is captured in the megaplan + memory. The **Fable5 parallel-isolated / distill-between doctrine** → effort-map + the `breadth-isolated-agents-not-serial` memory. G3/P9/R14/R16/R22/R26/R27/R30 governed the phase.)*

## PhaseEnd Changelog
**v1.23.0 → v1.24.0 — Phase 25 complete (Gen2 phase 17; the structural-family endgame + the giant endgame + the reframe).** The phase built and ran the **mechanical family engine** — `family_remap`/`family_sweep` (crack one exemplar → positionally remap per-overlay symbols → whole-binary byte-gate), banking **~20k+ member-matches for ~0 agent tokens** (proving h_norm families are TEMPLATES, not free dedup) — plus the T6/T7 curriculum and the def-side-wall recovery tooling (`canon_sig_reconcile` v3.2, `rtu_match`, the object-probe≠gate correction). It **completed the giant endgame** (every matchable region-a giant ×134, capped by the 399-ins flagship `func_80133CD4` cracked PIN-FREE via a Fable5 gdb-on-cc1 read of the allocator → cookbook §45), driving the fleet **66.02% → ~74.6% fn-count (58.2% instr-weighted / 30.3% distinct-code)** at 136/136 byte-identical, 0 NON_MATCHING throughout. The mature gcc-2.7.2 toolkit is now §17–§45. **The decisive close (R14):** regrouping the unmatched frontier by the looser **`h_seq`** fingerprint proved the "36k unique / 87% distinct-code tail" is a strict-`h_norm` artifact — **90% is ~754 templatable per-location families** (986 substantial / 1.88M ins; top-20 = 52%; same function at one address × ~120 overlays, differing only in per-location constants). That turns the endgame from "36k hand-decomps" into "crack ~986 exemplars → template ×~120," and it is handed forward as **`docs/family-endgame-megaplan.md`** for a fresh plan-mode + Fable5 **Phase 26**. One rule was added mid-phase (**R31**, capture-the-why); no other governance rules (techniques → cookbook, the reframe → the megaplan). The 3 walled reach-134 giants, the 2 permanent walls, and the F-band residuals are all carried forward in the megaplan §6 (recon-done seeds + specs), so nothing is lost on close.

## Plain-English Recap
This was a huge, multi-day push, and it ended on a discovery that changes the whole finish line. First, we built a "stamp" machine: once we reconstruct one copy of a piece of engine code, the tool automatically produces the matching copy for all ~120 game locations and checks each is bit-perfect — so **tens of thousands of functions got reconstructed for almost no extra effort**. That pushed the game from ~66% to ~75% (by function count). We also finished off the "giant" functions — the biggest, most-reused engine routines — including a 400-instruction monster we'd literally called "impossible" for most of the project; the top AI model, reading the actual 1990s compiler's source code, cracked it cleanly, and we wrote the trick down so cheaper AI can reuse it. But the **biggest win was a course-correction Drew pushed for**: the ~36,000 leftover functions looked like a hopeless one-by-one grind, but that was an illusion caused by how strictly we were grouping them. When we grouped them more loosely — by their *instruction skeleton* instead of exact details — **90% of them turned out to be the same handful of functions repeated once per game location**, just with different numbers filled in. So instead of 36,000 unique puzzles, it's really about **986 puzzles, each of which stamps into ~120 places once solved.** That makes "finish the decomp this weekend" genuinely plausible. We wrote all of this — the data, the plan, and exactly what to point the best AI model at before it goes away in two days — into a single handoff document for the next planning session. Everything still rebuilds perfectly; nothing broke.

## 🛑 Stop Here
PhaseEnd written; `CURRENT_PHASE.md` archived → `phase-ends/logs/Phase25.md` (R19, via `git mv` — left uncommitted). **Drew commits AND pushes** this PhaseEnd + the archived log (R6/R8 — the milestone-close is the human-controlled gate-2 sign-off; tonight's matching/docs commits `commit:0514`..`commit:0516` are already landed). No Ghidra DB change this phase (R23 no-op — the `db.*.gbf` churn is restart-noise; do NOT stage). Gen2 continues — do **NOT** start Phase 26 here. Start a **fresh session** (effort **Max**, **plan mode**, Fable5 for the plan-draft) for **Phase 26 — the family-endgame (finish the decomp)**, opening from `docs/family-endgame-megaplan.md` (Step A cheap map+triage first — the measured foundation). Keep this file forever.