BFM-decomp/phase-ends/CURRENT_PHASE.md

CURRENT_PHASE — Phase 24: Fix + evolve the permuter (§31-class-aware) · integration-recovery tool · scale the §31 wave

Started: 2026-07-02 · Effort: Max · Gen2 phase 16 of the arc (8→…→23→24) · Open-ended matching phase (checkpoint-close, like 15/20/21/22). Plan: approved (Drew, sequencing = permuter-first, staged). Full plan: /home/musashi/.claude/plans/plan-mode-enabled-max-tender-bengio.md. Baseline at phase start: fleet 64.86% byte-identical-from-source (~30% byte-weighted); 136/136 binaries byte-identical; 0 NON_MATCHING; dedup groups per Phase-23 close. main 143dbb89…, resident 8e17e02f…, ov_SC01_077 d19c9580….

Invariant (every task)

Whole-binary byte-gate (tools/harvest_verify.py) is the SOLE arbiter (G3/P9). Tools only reshape declarations/plumbing or PROPOSE candidates; nothing banks except a byte-identical SHA1 rebuild, verified CLEAN (R22): make clean && (extract all 136) && make check-all → 136/136; tools/dedup_integrate.py --check 0-failed; tools/progress.py --fleet monotonic up. No Ghidra DB change (R23 restart-noise — do NOT stage db.*.gbf).

Task checklist

• T1 — Re-log map-wave results + backlog hygiene (xHigh)
• T2 — Permuter Stage-0: floor-free masked scorer (-drz) (Max)
• T3 — Permuter setup fixes: pins · typedefs · -O0 (xHigh)
• T4 — Probe the flagship func_80132784 (4/400) + count-exact seeds (Max)
• T5 — Permuter evolve: §31-directed mutation + grinder fixes (Max) ← CURRENT
• T5b — Fable5 §31 map-extension spike: crack an intrinsic class (S11/RC-6) (Max; agent model:fable) — added 2026-07-02 (Drew)
• T6 — Build the integration-recovery tool (Max)
• T7 — Unblock the 8 compile-blocked region-a giants (xHigh)
• T8 — Scale the §31 wave (breadth → prompt for /effort toggle, R27)
• T9 — Distill + PhaseEnd (Max, Tier-1)

Current task: T5. Rules check ✓ after T4 (P6); next due after T8.

Fable5 policy (Drew, 2026-07-02): Fable5 (Agent(model:fable)) is the reserved discovery / wall-breaker tier (reads the gcc-2.7.2 source; ~375k tok/giant), NOT the workhorse. Cheap-tier-first: permuter + Opus applying §31 → Fable5 only when they stall. Two uses this phase: (1) on-demand escalation on any giant the fixed permuter can't close (T4/T8 — no task); (2) T5b proactive spike to test whether S11/RC-6 are truly intrinsic. A Fable5-cracked lever goes into §31 (R16) so Opus applies it thereafter.

R14 corrections carried in (do NOT re-adopt the stale Phase-23 backlog framings)

• "split-file-blind glob" is refuted — the real defect is compile.sh/match_one.py are -O2 only (can't match _o0). → -O0 compile.sh variant (T3).
• "churn-without-blacklist" is refuted — the real churn is floor-victim idle re-tries (grinder.py:120 tried.clear()). → scorer (T2) + input-changed gating (T5).
• func_80144090 ×1 is a type-lift case (OtBlk not in engine_types.h), NOT a straggler. → build_engine_types.py (T6).
• backlog.jsonl closeness is stale/regressed for the wave seeds (true values in .run/wave/*.c headers; func_8014E048 logged failed → wave_targets skips it). → re-log first (T1).
• The "~95/80/31" class counts are aspirational; real canon SCHEDULE=30/REGALLOC=84/REMAT=3, steerable near-misses scattered in OTHER/None. → wave text-scans where_stuck (T8).
• gcc-papermario is gcc 2.8.1, not 2.7.2; vanilla 2.7.2 at tools/reference/gcc-2.7.2/ (cite it). loop.md on vanilla; sched/regalloc/cse map files still cite 2.8.1 lines.
• No decomp-permuter submodule edit for the overhaul (scorer rebind + #pragma _permuter b64literal pin carrier + weight_overrides, all in tools/). The pin-scope randomizer pass (only submodule edit) is deferred.

Progress log

(append one line per task on completion — the crash-recovery trail)

• T4 ✓ (2026-07-02): 🎯 FLAGSHIP BANKED — func_80132784 (400 ins, "HARD-DEFER/irreducible" for 22 phases) matched whole-binary + committed (commit:0438); make check BINARY=ov_SC01_077 BYTE-IDENTICAL, dedup-check 1783/0. The permuter overhaul (T2+T3) is validated end-to-end by a real bank. Banked ×1 — the ×134 propagation is capped by per-overlay declaration plumbing (ov_SC02_005 cross-overlay straggler + an extern/type mismatch), NOT codegen → T6 target (the +1.6% byte-weight lands when T6's integration-recovery reconciles it). Hand-fix that unblocked the ×1: dropped the draft's redundant Blk16 typedef (already in engine_types.h:442) — T6 automates this. Count-exact seeds (func_8014E048 35 / func_80176D94 52 / func_80148094 72 / func_801412A8 110): permuter setup works (parse/compile/search clean, 0 internal failures), but random mutation didn't close them (func_80148094 only 77→74 in T2) — they're intrinsic residuals (RC-6/S11) → T5 directed mutation + T5b Fable5. Fleet 64.86% (×1 negligible). Commit: (checkpoint).
• T3 ✓ (2026-07-02): permuter setup fixes (all in tools/, no submodule edit): p16_permute.hide_asm — a b64-pragma carrier for register pins AND GTE __asm__ blocks (pycparser parses the pragma; decomp-permuter's own process_pragmas decodes it back so cc1 sees the real asm); custom-typedef/#define preservation (drop only #include); f32 typedef gap fixed; compile_o0.sh -O0 variant auto-selected for _o0 targets; run_masked.py pin-var fallback (expr_type→int on an unknown/hidden-pin id → 0 "internal permuter failures", was >0). ROOT CAUSE of the GTE fail: compile.sh now prepends .include "macro.inc" so mvmva assembles (the real build gets it via include_asm.h, which base.c omits + -DPERMUTER disables) — byte-neutral for non-GTE (count-exact scores unchanged). ALL 5 seeds parse+compile (base 4/110/36/52/77). 🎯 The flagship func_80132784 (4/400) CLOSED to a masked-0 that match_one independently confirms MATCH (400 ins) — winner preserved .run/wave/func_80132784.win.c for the T4 whole-binary gate → ×134 (≈ +1.6% byte-weight). No build-input changed. Commit: (checkpoint).
• T2 ✓ (2026-07-02): built the floor-free relocation-masked scorer, ENTIRELY in the tools/ layer (no decomp-permuter submodule edit): tools/masked_diff.py (shared objdump -drz oracle + masking), refactored tools/match_one.py to use it (-dr→-drz), tools/masked_scorer.py (MaskedScorer drop-in), tools/permuter/run_masked.py (rebinds src.main.Scorer), wired p16_permute.run_permuter. VALIDATED: masked_self=0; masked_cand≈match_one (7772/3635/52=52); STOCK floor 1750–2100 vs masked 36–77 (the wander cause, removed); live permuter base score = masked 77 (not stock 1930), descends to 74. -drz also fixed match_one's GTE under-count (regression-clean on count-exact seeds; func_80132784 now true 400 ins / 4-off). match_one output format unchanged → gate_stage/grinder compatible. No build-input changed. Commit: (checkpoint). T3 next handles the 2 compile FAILs surfaced (func_80132784 GTE mvmva/common.h; func_801412A8 custom-typedef drop).
• T1 ✓ (2026-07-02): re-logged 7 map-wave seeds with BYTE-VERIFIED closeness (dropped 19 stale/unreproducible records from .run/backlog.jsonl, .bak kept; best_draft → .run/backlog_drafts/). Ground truth (match_one -dr): func_8014E048=35 (was mis-logged failed), func_80176D94=52, func_80148094=72 (best draft = vG2.c, NOT the named file), func_801412A8=110; T6 leaf-MATCHes func_8014F4C0/func_80155800=0. func_80144090 excluded (banked ×1, not a stub → T6 OtBlk type-lift). Fixed gate_stage sig_unify/--src-file doc-drift. wave_targets --class REGALLOC + grinder now surface all 4 count-exact permuter seeds with true closeness; func_80132784 correctly kept out of blind selection. No build-input changed → 136/136 invariant untouched. Commit: (checkpoint).

T6 targets (banked leaf-MATCH, ×N propagation capped — declaration plumbing)

• func_80132784 — banked ×1 in ov_SC01_077_a.c; ×134 propagation blocked by ov_SC02_005 cross-overlay straggler + an extern/type mismatch. The auto-from batch is all-or-nothing (poisoned by pre-existing capped fns 0x8012A018/0x80165CA0); T6's tool must (a) drop-straggler per-function cleanly, (b) reconcile the extern/type conflict per overlay. The redundant-Blk16-typedef reconcile is the class-c type dedup T6 automates.
• Carried from Phase-23 backlog: func_8014F4C0 (fwd-decl reconcile + Vec4u lift), func_80155800 (TU-context), func_80144090 (OtBlk type-lift). Test cases in .run/wave/.

Blockers / open findings

• RESOLVED in T2 (R14 self-correction): the T1 finding that "func_80132784 is 204-off / not 4/400" was itself the -dr artifact. Once match_one uses -drz (T2), func_80132784 reads its true 400 ins and is 4/400 count-exact — the Phase-23 "4 instructions away" premise is CONFIRMED. Backlog re-logged closeness 4 (source T2-drz). The residual is a prologue save/init order swap (S7/S11) — a pure register/schedule swap, prime permuter/pin + T4 target. The realistic count-exact near-permuter seeds are: func_80132784 (4), func_8014E048 (35), func_80176D94 (52), func_80148094/vG2 (72), func_801412A8 (110). S11 blocks func_801412A8 + (partly) func_80132784 → T5b Fable5 spike still well-motivated. Lesson: the -drz fix (T2) is load-bearing for measurement honesty, not just the permuter's gradient.