BFM-decomp/phase-ends/logs/Phase20.md

CURRENT_PHASE — Phase 20: Close the propagation cap, crack the gcc residual classes, then scale the waves

Status: In progress (plan approved, gate 1 passed) · Generation: Gen2 (12th phase) · Started: 2026-06-20 Effort: Max (planning done); per-task below (R7). Owner decisions: class-crack first; skip the permuter background grind this phase. Project version at phase start: 1.18.0 (fleet 58.00% byte-identical, 136/136 binaries).

Open-ended phase (Phase 15/19 precedent): fleet % rises monotonically, 136/136 byte-identical throughout, 0 NON_MATCHING; closed at a clean checkpoint, not a fixed %. Plan file: ~/.claude/plans/max-effort-set-plan-giggly-pony.md.

Task checklist

• T1 — Close the propagation cap (typedef/local-type lift) · DONE (xHigh) Extended tools/build_engine_types.py with find_typedefs() + collision/overlap guards; lifted 14 typedefs to engine_types.h (byte-neutral, ov_SC01_077 stays d19c9580); dedup_propagate --auto-from propagated 9 reach-134 fns ×134. check-all 136/136, fleet 58.00%→58.35% (+0.35%), dedup 1485→1494, 0 NON_MATCHING. FINDING (R14): cap was 16; only 9 were type-blocked (all freed) — the other 7 are callee/data-extern plumbing (func_80142C84, D_800AE620…), the §19 declaration-plumbing class, NOT the type-lift → recovery-tooling lever (see Notes).
• T2 — Exemplar-miner / residual router · DONE (xHigh) tools/exemplar_miner.py → docs/exemplar_curriculum.md + .run/exemplar_routing.json (reach computed from sigs like dedup_propagate). 835 residual stubs routed: WAVE 472 (218 reach-134, T6 fuel) · STRUCT 159 · PINS 114 (45 reach-134) · STUB 90 (74 ARITY_WALL + 16 NONFAITHFUL). Key T3 input: 17 reach-134 fns at m2c-mismatch=1 = the cleanest regalloc/schedule isolates (likely §17-pins wins + the genuine class residuals). Named exemplars confirmed reach-134. Caveat: census is the Jun-19 snapshot (pre-Phase-19 waves, ~7% stale); T5/T6 regen gives the authoritative fresh WAVE list.
• T3 — gcc class-crack research block · Max each (R17) — the phase hinge
  • T3a — %lo-folding at -O0 CRACKED (not cited!): the array-of-STRUCT idiom folds %lo (extern Struct base[], sizeof==stride, base[i].field; NOT *(T*)(&sym+i*stride) which materializes). func_8013B7AC byte-gated + banked (×1, cluster is overlay-local). Cookbook §18/§20 closed. The other 9 -O0 stubs: idiom applies but need per-fn drafting (3 tried failed on -O0 codegen detail; 4 are big) → backlog/fuel. Reusable idiom fleet-wide for any indexed-global access.
  • [~] T3b — loop-guard (func_8012C2D0, worked in ov_SC03_014). R14: exemplar MISLABELED — it's gcc strength-reduction/addressing (end = base D_80120194 + 0x658C as an induction-var final-value), NOT operand-order. 2 drafts: base-relative-end + lhu/u16 now correct; residual = gcc materializes the base separately before +0x658C (resists the constant-fold my C produces). Likely crackable via indexed-struct-array, but a deep per-fn induction-var grind. Prior "irreducible" was an incomplete framing. PAUSED for the ROI checkpoint below.
  • T3c — store-vs-load scheduling tie-break (func_8014F2E0; sched.c) Each: read the gcc-papermario pass → try the C lever → match_one → crack (byte-gate + propagate + cookbook idiom) OR cite-irreducible (cookbook verdict).
• T4 — Distill the gcc-2.7.2 codegen map · DONE (Max, R16) Wrote cookbook §20 (the wave-at-scale gate cap = loose-typing call-graph wall; store-vs-load + §10 hoist-vs-remat + IV-combine + hoisted-invariant-order CONFIRMED-irreducible with byte-evidence; the -O1 class; the stale-.o + args-as-array + --chunk 1 gotchas; the T1/T2 wins) + an R14 correction to the §17a loop-guard entry (strength-reduction, not operand-order). %lo (T3a) left as a placeholder to append when cracked.
• T5 — Ghidra-C regen for fresh wave targets · xHigh + R23/R29 Stop MCP → DecompileFunctions.java over fresh tractable reach-134 stubs → restart → pause + prompt /mcp → get_binary_info (G2).
• [~] T6 — Enriched waves (batch 1 done) · Ultracode Wave on 48 cached reach-134 ≤90-ins targets → 41/48 match_one MATCH (85%). Whole-binary gate: 8/41 banked (7 ×134 + 1 local), fleet 58.35%→58.63% (+0.28%), dedup 1494→1501, check-all 136/136. KEY FINDING (byte-verified, R14/P9): the 33 lost are the LOOSE-TYPING WALL IN THE CALL GRAPH. A callee (e.g. func_80153C74) is declared with CONFLICTING types at different overlay sites, so the draft's extern hits conflicting types in-TU (a COMPILE error, not a byte miss). match_one over-predicts because it compiles STANDALONE (draft's own externs) and masks jal/%hi/%lo. ALL recovery levers fail: sig_unify/canon_draft_decls impose a single "canonical" that's wrong for some sites; fix_arity_callers (0, not arity); no-proto externs (0, incompatible with the overlay's narrow decls); strip-externs (0, implicit-int ≠ target). The ONLY fix is per-site function-pointer casts (((ret(*)(args))func_X)(…), §17a) = Drew's backlog item 2 (auto call-site-cast recovery), the precisely-diagnosed Phase-21 cap. Diagnostic lesson: a failed in-TU build leaves a STALE .o (objdump misleads — trust the whole-binary SHA, not the disasm). The 7 DIFFs gave gold gcc diagnoses (do T3c/T4 for free): T3c store-vs-load func_8014F2E0/func_80150528 CONFIRMED unsteerable (sched.c); §10 hoist-vs-remat func_80149374/func_801493D0; IV-combine func_80177AD4; hoisted-invariant order func_80177F84; func_80161A90 = -O1 (build-infra, needs an -O1 split like -O0).

Optional / conditional (decide live, else → Phase 21): -O0 ×134 rollout (only if T3a cracks %lo); the 28 giants (deferred to a focused Phase-21 deep session).

Current task pointer — MULTI-SESSION CONTINUE (Drew, 2026-06-21: not closing; resume in a fresh session)

Done + committed: T1 (cap, +0.35%), T2 (router), T3b (cited — R14 reframe), T3c (cited — written up in §20), T4 (codegen map §20 + §17a correction), T6 batch-1 (8 banked, +0.28%), THE CAP TOOL cast_call_sites.py (built+validated; 6 callee-cast recoveries, 5 ×134 + 1 local, +0.19%). Fleet 58.00%→58.82%, 136/136 byte-identical, 0 NON_MATCHING.

Remaining, prioritized for the fresh session (the plan is approved — resume autonomously, P3/§Session-Start step 4):

1. [THE CAP — DONE] Built tools/cast_call_sites.py (cookbook §20 RESOLUTION). The §17a-1 per-site fn-ptr cast: per draft, callee whose canonical TU sig ≠ draft's intended sig → decl→canonical + call→((ret(*)(args))func_X)(…) cast (codegen-neutral, gcc folds → direct jal). Pipeline canon_resident_calls → cast_call_sites → sig_unify → harvest_verify --chunk 1. Recovered 6/33 byte-identical (5 ×134 + 1 local), fleet 58.63%→58.82%, check-all 136/136. R14 CORRECTION of §20's "~33" projection (byte-proven): the 33 are NOT mostly callee-conflicts — ~6 callee (cast-recovered), ~18 DEF-side loose-typing wall (def's matchable sig conflicts with the unchangeable caller-canonical; caller-side fix BLOCKED — INCLUDE_ASM declares nothing, so a shared macro's internal extern is load-bearing in the 133 stub overlays), + a small DATA tail. The def-side residual needs RE-DRAFTING under the caller-canonical sig (future wave w/ canonical pinned), not a text transform; the arity-mismatch subset is a hard wall. Open extension: data-cast (*(T*)&D_x) — 1 case here, low yield. Drafts: .run/drafts-t6-cn/ → .run/drafts-t6-cast/ (cast) → .run/drafts-t6-cast-uni/ (cast+sig_unify); gate good-sha d19c9580….
2. T3a — %lo-folding -O0 (func_8013C360 cluster; cookbook §18 open residual; entangled with the -O0 split infra). Append the verdict/crack to cookbook §20's placeholder.
3. T5 — Ghidra-C regen (R23/R29: stop MCP → DecompileFunctions.java → /mcp) for fresh reach-134 targets → more T6 wave batches (now with the cast-recovery → full yield). Cache is 300; ov_SC01_077 has 668 stubs.
4. [build-infra] -O1/-O0 split files to bank the -O1 (func_80161A90) + the 6 matched -O0 fns ×134 (per-overlay -O0 splits, uniform offsets, scriptable).

Reusable wave recipe (proven this session): gen_harvest_targets --min-reach 134 --max-nins 90 → filter to cached Ghidra-C → Workflow (one agent/target, §17 toolkit, pass args as a JSON array NOT a string — the gotcha that bit us) → canon_resident_calls → the new cast-recovery → harvest_verify --chunk 1 (chunk>1 mis-attributes compile-erroring drafts) → dedup_propagate --auto-from.

Carry from T1 → recovery tooling (Phase-20 backlog item 2 / T2 router): the 7 callee/data-plumbing capped fns (reach-134, matched-but-local): 0x80142B2C 0x801535F4 0x80155E30 0x80157580 0x801576A8 0x80168F40 0x80170B48. They fail compiles_standalone on undeclared callees/data, not types. A dedup_propagate macro-extern-injection (or canonical-callee-sig embed, the gen_harvest_targets approach) would free them ×134 (~+0.3%). NOT type-lift; do not reopen T1 for them.

Candidate rules (propose at the Phase-20 PhaseEnd)

• R30 — Capture context-dependent knowledge artifacts before a fresh session. Cookbook entries, byte-verified findings, codegen-map distillations, and doc corrections whose quality depends on the current session's full live context MUST be written DURING that session, before any fresh-session handoff (the fresh session inherits only compressed summaries and loses the detail/nuance/byte-evidence). Defer only mechanical/continuable work (build the next tool, run the next wave). Extends R16 (flywheel — adds the timing constraint); memory capture-knowledge-before-fresh-session. (Drew, 2026-06-21 — prompted by nearly deferring T4's §20 distillation.)

Blockers

None.

Progress log

• 2026-06-21 (fresh session, Max): Built tools/cast_call_sites.py (the cap tool, §17a-1 callee-cast). Validated on T6 batch-1's 33 gate-fails → 6 byte-identical recoveries (5 propagated ×134 + func_8015F948 local), fleet 58.63%→58.82%, check-all 136/136, dedup 1501→1508. Diagnosed the residual exhaustively (R14): the §20 "~33" projection was wrong — the batch is dominated by the DEF-side loose-typing wall (caller-side blocked by INCLUDE_ASM-declares-nothing), not callee-conflicts. Captured in cookbook §20 RESOLUTION + SETUP inventory (R16/R21/R30). Local commit-checkpoint (Drew pushes, R6).
• 2026-06-21 (Drew order ① -O0 rollout): INVALID PREMISE (R14, byte-proven). The 6 matched -O0 cluster fns are NOT reach-134 byte-identical — only 1/134 overlays shares the bytes; they reference per-overlay data addresses → overlay-LOCAL, not ×134-shareable. The Phase-18/19 "reach-134" was a mislabel. Built+byte-validated the §18 split infra (rollout tool + Makefile %_o0.o pattern + PROVIDE .ld), gate exposed the wall, reverted. -O0 rollout = ×1 per-overlay-unique work (~0.2-0.3%), SKIP. Cookbook §18 corrected. Fleet unchanged 58.82%.
• 2026-06-21 (Drew order ② data-cast): MOOT (R14, byte-proven). 0 drafts have an extern struct/union data conflict; the lone "DATA-conflict" was a typedef-redef (Blk20) masking the def-sig wall. data-cast = code for 0 cases → NOT built. Cookbook §20 corrected. CONSOLIDATED STATE: the reach-134 ≤90-ins CACHED tail is byte-EXHAUSTED of cheap recoveries — cap tool banked the callee-conflict fraction (6, +0.19%); the rest are the def-side loose-typing wall + overlay-local + genuine codegen quirks (all byte-proven non-mechanical). Remaining levers: T5 fresh waves (uncached reach-134 via Ghidra-C regen — needs MCP cycle + Ultracode, highest potential) · T3a %lo (value dropped — targets are overlay-local; reusable idiom only) · the giants (byte-weighted lever, deep/hard). Fleet 58.82%, 136/136, 0 NON_MATCHING. Reported to Drew for the steer.
• 2026-06-20: Session start; read full load order (PROJECT_CONTEXT + 19 PhaseEnds). Phase 19 closed at 58.00%. Brainstormed strategy with Drew (ML/Wine/debug-symbols/gcc-papermario threads → honest verdicts; the wall is propagation-cap + ~3 uncracked gcc classes, not comprehension). Max set, plan mode. 3 Explore agents mapped the propagation/wave/quirk subsystems; verified the type-lift crux firsthand. Reframed (R14/P9): class-cracking = ~3 bounded spikes. Drew chose class-crack first + skip permuter grind. Plan approved (gate 1). CURRENT_PHASE + task list created (R28).

Notes / decisions

• R14/P9 reframe (recorded): of 916 residual stubs — STRUCTURAL_MISS 368 (→waves), PERMUTER_CLASS 146 (→permuter/pins), ARITY_WALL 74 (→stub, fundamental loose-typing), fn-ptr/jtbl 144 (→struct context), NONFAITHFUL 16. Novel uncracked gcc classes = 3 (loop-guard, store-vs-load, %lo-folding); 2 carry a single prior "irreducible" verdict (worth ONE Max re-examination each, since the sibling regalloc-order class cracked with pins).
• Honest % expectation: T1 ~+0.6%; T3a maybe ~+0.6% (if %lo cracks); T3b/c may be 0 (cited-irreducible). Durable value = the codegen map + a higher wave close-rate, then T6 banks the scale %.
• Commit cadence: per-task/per-batch local checkpoints (long-phase precedent), each clean-rebuild-verified (R22). Drew pushes (R6).