The S38 checkpoint gated the phase's best lever ("do NOT scale the alias lever") on
distinct-code falling 89.3 -> 89.2. It never fell.
PROOF (each commit's metric recomputed from its OWN committed tree, 0 unresolved):
commit:1426 TRUE : instr 12394533 distinct 5022306 (77895 uniq)
commit:1426 COMMITTED: instr 12402412 distinct 5029324 (78025 uniq) <- stale
HEAD TRUE == COMMITTED: instr 12405402 distinct 5025082 (77952 uniq)
=> true delta 843->HEAD: instr +10869, distinct +2776 ins / +57 uniq. ALL ROSE.
The 843 digest was generated from a working tree still holding work REVERTED before the
commit landed (+7,879 ins / +130 uniq overstated) and never regenerated, so the next
HONEST digest read as a fall. => THE ALIAS LEVER IS UNGATED (scale it, §61 small batches).
Both recorded leads were wrong (R14): progress.py:423's SIG regex feeds fn-count ONLY
(neither weighted metric sees a C identifier — both derive matched = sig - corpus.stubs),
and "the harvest reverted functions to INCLUDE_ASM" died on one grep (483 removed, 0 added).
The 3-grep proof: identical sigs + unchanged tools/ + zero +INCLUDE_ASM => HEAD's stub set
is a strict subset => both numerators are FORBIDDEN to fall.
THREE INSTRUMENT DEFECTS, all one class (a bare except around a fail-CLOSED oracle):
- progress.py stub_addrs wrapped corpus.stubs in `except Exception: return set()`. An empty
stub set means "could not answer", not "no stubs", so matched = sig - stubs credited EVERY
function. Byte-witnessed: instr 100.00% / distinct 100.00% in a tree with no asm/. Now
propagates.
- cast_call_sites.tu_for + reconcile_tu.tu_for had the identical swallow, falling back to the
default <ov>.c instead of the jr/-O0 split TU — silently reinstating the exact bug
cast_call_sites' own docstring says it exists to fix. A wrong-TU reconcile fails the gate,
and this phase's base rate is ~24k PLUMBING vs 4,917 DIFF, so it presents as a codegen wall.
Now propagate CorpusError; ValueError fallback for curated names preserved; derived-TU path
re-verified (a _jr_ split stub resolves correctly, both tools agree).
NEW GATE (R34 — the byte-gate is a null oracle for DOCUMENTS; check-all stays 140/140 over a
stale digest forever): tools/audit_digest.py + `make audit-digest`, wired into tools-health
after report. Recomputes the three headline metrics from the current tree and fails if the
committed digest disagrees. Compares INTEGERS, not percentages — the +7,879-instruction
staleness printed as "94.4%" on both sides. Negative-control-proven against the stale 843
digest (fails, exit 1) and green on HEAD.
Verified: make report exit 0 (dedup-check 1910 validated / 0 failed, C1 coverage
241216/241216); audit-digest OK; cookbook-index OK (398 sections); metrics unchanged by the
fix (94.40% / 89.18%). No src/ or config/ edits — no bytes touched, nothing banked.
cookbook §140 · decision-log 2026-08-04 · SETUP.md inventory (R21) · R14/R32/R34/R35.
812 KiB
Matching Cookbook — reusable compiler idioms & C-writing techniques
⚠️ SOURCE-CITATION PROVENANCE (swept 2026-07-28,
tools/sweep_citations.py). This file's ~57 gcc-source citations are MIXED provenance — some are exact for our gcc 2.7.2 (tools/reference/gcc-2.7.2/), others came fromgcc-papermario, which is gcc 2.8.1. Spot- verified exact:loop.c:5556(emit_iv_add_mult),local-alloc.c:1765/1795/1825(combine_regs),global.c:906/917/924/1000(find_reg),local-alloc.c:1021/1064(update_equiv_regs). Known miss — NOW FIXED (2026-07-28):expr.c:5535wasMIN_EXPR/MAX_EXPRoptab code. The two sites that idiom actually means are 4577 (INDIRECT_REF, guarded 4570-4576) and 4888 (COMPONENT_REF, unconditional). Both corrected in place below, re-derived twice independently. (4904is theOFFSET_REFgrant — a third site, not the one that idiom is about.) Why this is lower-stakes here than indocs/gcc-2.7.2-map/*: those files are source-derived reasoning, so a bad read means bad advice. This file's idioms are byte-proven, with citations attached as explanation — a drifted cite corrupts the why, not the lever. Re-derive a citation (grep -n '^sym (' tools/reference/gcc-2.7.2/*.c) before building new reasoning on it.
Evolvable reference (docs/ layer). Created Phase 6, 2026-06-14. Append an entry every time a reusable nuance is found — these recur across nearly every function, so capturing them once accelerates all future matching. Companion to
SETUP.md§5 (the pinned triple) and §6.6 (the matching loop). Consult this at session start before matching.
Pinned toolchain (the context all entries assume): tools/bin/gcc-2.7.2-psx/cc1
-O2 -G0 -mips1 -mcpu=3000 -mgas -msoft-float -fgnu-linker → maspsx --aspsx-version=2.56 --expand-div → mipsel-as -march=r3000 -mtune=r3000 -no-pad-sections -O1 -G0. (See SETUP.md
§5.4. psx≈cdk and 2.56≈2.67 produce identical bytes on functions without the discriminating
idioms — they only diverge on sltu+$at / div-expansion specifics.)
How to use this
- Scaffold (
tools/decompile.py <fn>or Ghidra MCP), then consult the entries below to shape the C toward the target idioms before iterating asm-differ — it saves rounds. - When you discover a new asm↔C correspondence or a "what makes gcc emit X" trick, add it here.
- When a residual diff is pure instruction scheduling (provably-independent ops reordered), that's a decomp-permuter job, not a hand-iteration job — note it and move on.
§1 Idiom catalog (asm pattern → C that produces it)
I1 — Unsigned range check: (x - lo) < (hi-lo) → addiu+sltiu
Target: addiu v0,v1,-0x51 ; sltiu v0,v0,0x5f (true iff x in [0x51, 0xaf]).
C: if ((u32)(x - 0x51) < 0x5f). The subtract-then-unsigned-compare is the canonical
single-branch range test. sltiu (immediate) is emitted for < constant.
Example: func_80018F20.
I2 — Byte mask forces andi even after lbu
Target: a redundant-looking andi v1,v1,0xff on a value already loaded by lbu.
C: write (x & 0xff) explicitly on the later uses of a byte value (gcc 2.7.2 -O2 does
NOT prove the upper bits zero across pseudo-registers, so the & 0xff survives as andi).
If you omit it, you lose the andi and the diff won't close.
Example: func_80018F20 (first compare uses raw x, the 2nd/3rd use x & 0xff).
I3 — Division by a constant → magic multiply
Target: lui a3,0xcccc ; ori a3,a3,0xcccd ; multu a1,a3 ; mfhi t0 ; srl a1,t0,3 = a1 / 10.
C: just write x / 10 (constant divisor). gcc emits the reciprocal-multiply (0xCCCCCCCD,
shift 3 for ÷10). The magic constant is loop-invariant and gets hoisted to the preheader.
Example: func_80015A74.
I4 — Runtime (variable) division → divu + zero-check break (NEEDS --expand-div)
Target: divu zero,a0,a1 ; bnez a1,.+12 ; nop ; break 0x7 ; mflo <q> ; mfhi <r>.
C: x / y and x % y on the same operands → one divu, mflo=quotient, mfhi=remainder.
Critical: maspsx must run with --expand-div or it emits a bare divu with no zero-check
and the function can never match. This is PINNED in the Makefile (MASPSX_FLAGS).
Example: func_80015A74.
§2 Writing matching C (what makes gcc emit X)
T1 — Loop pointer: top-of-body for addu induction, not constant-folded addiu
If you init a loop pointer before the loop from a constant index (p = base + 2), gcc
constant-folds it to addiu p,base,2. The original often recomputes it from the induction
variable, giving addu p,base,i at both entry and loop-back. Fix: compute the pointer at
the top of the loop body (p = base + i;) so gcc keeps the induction form.
Example: func_80018F20 (365→0 hinged on this).
T2 — Source statement order drives instruction scheduling
gcc 2.7.2's scheduler largely follows source order for independent setup/init statements.
When pre-loop inits are mis-ordered vs the target (e.g. a counter li landing before vs after
a hoisted loop-invariant), reordering the C statements moves them. When it doesn't (the op is
placed by the optimizer, e.g. a hoisted invariant), it becomes a permuter job — see §3.
Open example: func_80015A74 residual (counter init vs hoisted magic constant).
T3 — Types
u8/u16/u32/s8/s16/s32 from include/common.h. A u8* deref → lbu (zero-extend); s8* →
lb (sign-extend). Pick the load width/sign that matches the asm, then layer & 0xff (I2) /
casts as needed.
T4 — Branch polarity: invert the source condition to flip gcc's chosen branch
Two source forms can be logically identical but emit opposite branches:
if (x & m) return A; return B; vs if ((x & m) == 0) return B; return A;. gcc -O2 picks one
polarity (beqz vs bnez); it may be the opposite of the target. Symptom in asm-differ: the right
structure but a lone beqz↔bnez flip with the two return constants swapped between the branch
and its j/delay slot. Fix: rewrite the condition with the other polarity. Also: which arm of an
if/else becomes the fall-through follows source order — put the target's fall-through block in the
if, the branched-to block in the else (e.g. if (a != b){…} else {…} if the == block sits
last). Example: CdQueueBusy (1405 → 210 via if/else order, 210 → 0 via the 0x20 polarity flip).
Likewise multi-exit functions: write the success/main return LAST (it becomes the fall-through
into the shared epilogue) and error cases as early returns (they branch in). Reversing this —
if (ok) { … return good; } return 0; — makes return 0 the fall-through and duplicates the
j epilogue/move v0,zero tail. Example: CdReadRequest (305 → 0 by flipping to if (busy) return 0; … return cdReq_result;).
§3 When a diff is pure scheduling → decomp-permuter (harness built, Phase 6)
A residual diff of provably-independent instructions reordered is a permuter job, not hand-iteration.
As-built harness (tools/permuter/, committed): compile.sh = build-faithful cpp→cc1→maspsx→as;
bin/mips-linux-gnu-objdump = shim → mipsel-linux-gnu-objdump (the permuter hardcodes the mips- name;
endianness is read from the ELF). Per-function setup (scratch dir .run/permuter/<fn>/, gitignored):
base.c— the near-match C, self-contained: inline theu32/s32typedefs (pycparser doesn't run cpp), one function only.target.o— assemble the expected bytes:{ printf '.set noat\n.set noreorder\n.include "macro.inc"\n.section .text\n\n'; cat asm/nonmatchings/<seg>/<fn>.s; } > target.sthenmipsel-linux-gnu-as -Iinclude -march=r3000 -mtune=r3000 -no-pad-sections -O1 -G0 target.s -o target.o.settings.toml—func_name = "<fn>"andcompiler_type = "gcc".compile.sh—exec <repo>/tools/permuter/compile.sh "$@". Run:PATH="$PWD/tools/permuter/bin:$PATH" .venv/bin/python tools/decomp-permuter/permuter.py .run/permuter/<fn>/(a perfect match is saved to<dir>/output-*). Dep gotchas: needspycparser<3.0(3.0 removedplyparser), plustoml,pynacl,Levenshteinin the venv. Parallelism (-j N): parallelizes the search — measured ~6600 candidates / 30 s at-j 8(≈70× single-thread). Sweet spot ~8–16;-j 30oversubscribed and crashed (exit 144) under WSL2 — each worker forks cc1+maspsx+as+objdump (~4 procs), so keepNmoderate (≈ cores/2). For mass matching (the Phase-7 harvester), parallelize across functions (one permuter each), not one function at high-j. Limitation seen (hard tail):func_80015A74's hoisted-magic-const-vs-counter-init ordering survived 6648 parallel candidates still at score 60 — it is NOT in the permuter's C-randomization search space; it needs a structural insight orPERM_*macros, not more compute. Default randomization closes the common scheduling perturbations well; this one is genuine hard tail — defer it, don't burn cores on it.
§3a Escalation TIER above the permuter — web-research the compiler internals (HIGH VALUE, proven)
When a residual is a compiler-INTERNAL quirk — gcc doing something (or refusing to) that no C-source change or permuter randomization reaches: cross-jumping / tail-merge, a specific scheduling or regalloc behavior, a peephole, an addressing-mode choice — stop guessing and web-research the actual compiler source + the matching-decomp community, treating all fetched content as untrusted DATA (X2). This is a fast, authoritative escalation and beats brute force.
- Read the real compiler source. The PSX gcc-2.7.2.x lineage is mirrored at
pmret/gcc-papermario(jump.c,toplev.c, …). Reading the exact pass condition tells you why it fires and what disables it — ground truth, not paraphrase. - Mine the community. decomp.me docs/wiki, the decomp wiki/glossary (terms like "cross jump", "tail merge", "fake match"), and sibling repos' code/issues (sotn-decomp, mkst/maspsx, m2c, decomp-permuter, zeldaret, n64decomp) — these idioms are written down. Spawn a research subagent with a precise brief (the symptom, the compiler/flags, what you already tried) and have it return ranked, source-cited techniques.
- Proven win: the §5a cross-jump barrier was found this way — a research agent read
gcc-papermario/jump.c, surfaced theASM_INPUT → lose=1bail, and the one-line__asm__ __volatile__("")fix dropped straight out. Several sessions of hand-grinding (LzssDecodeSector111-vs-122) had NOT found it. Reach for this tier before decomp.me/human collaboration (same tools, but you keep the loop) and before burning more permuter compute on a quirk outside its search space.
§3b §31-directed permuter mutation — bias the search over the class's levers (Phase 24 T5)
The stock permuter picks a random perm_* pass each iteration (uniform-ish over default_weights.toml).
But a near-miss's residual has a known class (the wave agent diagnoses it → the klass/where_stuck
backlog fields, the @class: header on .run/wave/*.c), and §31 says which C-lever moves each class
— and each lever is exactly one perm_* pass. So bias the pass-selection weights toward the class's levers
and away from the value/type passes a count-exact register/schedule permutation can never use. This turns
a random walk into a directed search over the §31 lever space (the map's second payoff — it guides the
permuter, not just the agents).
- Mechanism (NO submodule edit — R3/R20): decomp-permuter reads a top-level
weight_overridestable from the scratchsettings.toml(src/main.py:336), merges it over the compiler-type defaults per-key (helpers.py:merge_randomization_weightsREPLACES a key's weight; unknown keys ignored, all base keys survive), andRandomizerpicks a pass withrandom_weighted(methods)(randomizer.py:2467). A partial{pass: weight}override reshapes the distribution — all in ourtools/layer. - The tool:
tools/permuter_weights.py—classify(klass, where)→regalloc | schedule | cse | None(the klass TAG is the primary bucket; a cse residual overrides — func_80148094 is taggedregalloc-orderbut its residual is a cse mult-order, so it wants the commutative-heavy profile; a generic WAVE/GIANT tag falls back to thewheretext).render_settings_toml()emits the[weight_overrides]block.p16_permute.setup(fn, draft, asm_subdir, klass=…, where=…)writes it;grinder.pyauto-threadsklass/where_stuckfrom the backlog record.klass=None→ no table → the plain gcc defaults (identical to the pre-T5 undirected search: a safe superset). - The three profiles → §31 levers (keys are the exact
perm_*names): regalloc (RC-1/2/3, S7, S11 register-permutation) up-weightsperm_reorder_decls(40, RC-1 slot / RC-3 tie-order) ·perm_reorder_stmts(40, RC-2 range / S11 LUID) ·perm_temp_for_expr(60, S2 boost) ·perm_split_assignment/perm_duplicate_assignment(set-count → RC-2 / defeat RC-7 equiv); schedule (S1–S5, D1–D4) leads withperm_reorder_stmts(60, LUID) +perm_temp_for_expr(60, S2) +perm_ins_block/perm_empty_stmt(S4 filler); cse leads withperm_commutative(40, operand order) +perm_expand_expr/perm_split_assignment(re-decompose). All three push the value/type noise (perm_add_mask/xor_zero/mult_zero/randomize_*_type/…) to ~0.1. - Validated: on
func_8014E048(S11 LUID⊗alloc, 143-ins, base masked-36) the regalloc profile found a better score in <30 s (36→34→33) where the undirected search had stalled — proof the biased distribution explores the class's territory. The whole-binary byte-gate (harvest_verify) stays the sole arbiter (G3/P9): a permuteroutput-0-*is a strong CANDIDATE to gate, never a bank. - The grinder's companion fix (T5): the old idle path did a blind
tried.clear()→ re-permuted every floor-victim on every idle tick (churn, R14). Replaced with input-changed gating (grinder.py draft_sig=(best_draft mtime, closeness)): a fn is re-opened only when the worker actually improved its draft; the permuter is deterministic givenbase.c+target.o, so an unchanged input can never newly win. - Scope note: the four flagship count-exact seeds (
func_8014E04835 ·func_80176D9452 ·func_8014809472 ·func_801412A8110) are the project's worst-case intrinsic walls (RC-6/S11, "the C-space around the target is discontinuous" — §31 regalloc RC-6). Directed mutation is the right tool but the map's "two probes, don't grind" applies. The real payoff is the broader reach-134 near-miss tail (T8: ~30 schedule / ~84 regalloc), most of which is far less extreme — there the directed profiles raise the per-batch close-rate.
§4 Flag/toolchain gotchas
--expand-divis required for any div/rem (§I4). Pinned globally inMASPSX_FLAGS.-O2 -G0(no$gp-relative).-G0confirmed (zero%gp_relin the disassembly).- psx-vs-cdk cc1 and aspsx 2.56-vs-2.67 are byte-equivalent on functions without the
discriminating idioms — don't expect a probe to distinguish them unless it uses
sltu+$ator div-expansion specifics. The evidence-based pin is psx + 2.56 (PsyQ 4.0 stamps). - Per-module mixing (§5.5) is expected: if a whole module's div is bare (no
--expand-div) or flags differ, that's a per-file override — record it here and add the Makefile mechanism.
§5 Known hard-residual classes (instruction-identical, one byte-exact blocker)
These are functions where every instruction matches but a final whole-function artifact blocks score 0. They still confirm the compiler (instruction selection + regalloc match); defer them as decomp-permuter candidates rather than hand-grinding.
- Phantom empty stack frame — gcc 2.7.2 -O2 sometimes wraps a frameless leaf in an unused
16-byte frame (
addiu sp,sp,-0x10/+0x10, no saves/spills); the target is frameless. The one extra instruction shifts the whole function → large score from a trivial cause. Structural variation / the permuter can flip frame allocation. Example:func_80016714(bzero). - Hoisted-invariant vs IV-init ordering — a loop-invariant load scheduled before/after the
counter init; not reachable by C-source changes (permuter stuck at base). Needs
PERM_*or insight. Example:func_80015A74(uint→BCD). See §3.
§5a Cross-jump tail-merge — gcc collapses two byte-identical blocks the original kept separate (FIX FOUND)
Symptom: your function is N instructions SHORTER than the target, because the original binary has two
(or more) byte-identical tail blocks (classically a "save K globals then return c" epilogue reached from
different states) but gcc merges them into one. asm-differ shows a big cascade; the instruction COUNT is
short by exactly one copy of the tail. Example: LzssDecodeSector — the original keeps block_14 (the
state-3/4 save, ending j epilogue) SEPARATE from the state-2 reload save (which falls through to the
epilogue); gcc merged them → 111 vs the original 122 instructions.
Root cause (ground-truthed against gcc-2.7.2.3 jump.c): the find_cross_jump/do_cross_jump pass
walks two blocks backward and merges them while the instruction suffix is identical (rtx_renumbered_equal_p).
It is hardcoded ON at any optimize > 0 (fires at -O1 too; no -fno-crossjumping exists before gcc 3.3),
and do_cross_jump explicitly rewrites RETURN insns — so identical save/return epilogues are exactly what
it targets. Shared-goto, explicit-epilogue, and three-inline-copy C forms all produce RTL-identical tails →
gcc re-merges every time. cdk cc1 merges too. The permuter's default randomization does NOT defeat it.
THE FIX — a zero-byte volatile-asm barrier. find_cross_jump sets lose = 1 (bails) on ANY volatile asm
node (ASM_INPUT/MEM_VOLATILE_P). Put one empty volatile asm in ONE of the twin blocks (after the last
store, before the return):
/* ...the K stores... */
__asm__ __volatile__("" ::: "memory"); /* zero-byte cross-jump barrier */
return c;
It emits no machine code but makes the block's RTL non-identical to its twin, so gcc keeps BOTH copies →
correct instruction count. Document it as load-bearing (a future reader will "clean it up" and lose 11 bytes).
This is a standard decomp idiom (sotn writes duplicate funcs explicitly; the "" ::: "memory" clobber also
pins store ordering — drop the clobber to plain __asm__ __volatile__("") if it perturbs scheduling).
Permuter caveat: pycparser rejects __asm__ __volatile__(... ::: ...). To still permute the residual
regalloc, put a placeholder call (CJBARRIER(); + an extern void CJBARRIER(void);) in base.c and have the
per-function compile.sh sed it to the real asm before compiling. Note the asm-differ object-mode score then
floats on a cosmetic .rodata-vs-jtbl_<addr> symbol floor (the migrated jump table links identically), so
verify candidates with the linked make check, not the permuter score.
The full close of this exact function — the ~4 regalloc/scheduling slots this barrier leaves behind, and the
floor-free .text metric that finally measured them — is §10 (LZSS matched byte-for-byte, Phase 7 session F).
§6 Per-module optimization mixing — the -O0 boot module (Phase 7)
Finding (2026-06-14): the EXE mixes optimization levels per original translation unit (the
§5.5 Xenogears-style mixing, now concrete). The boot/main/game-mode-dispatch module — a clean
contiguous block at vram 0x80010000–0x800123F0 (~50 funcs: start, main, GameModeDispatch,
DebugMenuHandler, the game-mode handlers) — was compiled at -O0. Everything from 0x800123F0
onward (every match so far + the file-loader cluster) is -O2. Always opt-fingerprint a new
function before writing C — the pinned -O2 is NOT global.
Detecting the opt level (do this first)
gcc 2.7.2 -O0 keeps a frame pointer: addu $fp,$sp,$zero (21F0A003) in the prologue +
addu $sp,$fp,$zero in the epilogue; -O2 omits it. Grep the target .s:
grep -l 21F0A003 asm/nonmatchings/<seg>/<fn>.s → hit = -O0, miss = -O2. Module scan: classify
every .s by that signature, sort by address; the contiguous -O0 run is the module (the boot block
is the one early -O0 run). Other -O0 tells: redundant move/addu rd,rs,$zero copies; a nop after
every load (no load-delay scheduling); single-use values parked in callee-saved s0..; large
constant member offsets left unfolded (la $reg,sym + lhu off($reg)), where -O2 folds
sym+off into one load.
-O0 idiom — far struct member via a register base pointer
Target: lui s0,%hi(BASE); addiu s0,s0,%lo(BASE); lui at,1; addu at,s0,at; lhu v0,-0x5c52(at) =
load a u16 at BASE + 0xA3AE. The lui 1 / addu / -0x5c52 is just as expanding a register-relative
load whose offset (0xA3AE) exceeds 0x7FFF (%hi=1, %lo=-0x5C52). C — a register-qualified pointer
to the base, then offset-deref:
extern u8 BASE[];
register u8 *p = BASE;
... *(u16 *)(p + 0xA3AE) ... /* base stays in a callee-saved reg; offset left unfolded */
register is load-bearing: drop it and -O0 spills the pointer (extra sw/lw, bigger frame); and
-O1/-O2 fold it all back to lhu sym+off (no base reg, no frame pointer). Plain BASE[idx] or
((struct*)BASE)->m also fold at -O0 → wrong. Example: GameModeDispatch (0x80010B40) =
gameModeHandlerTable[*(u16*)(p+0xA3AE)]() — byte-exact (asm-differ 0).
-O0 idiom — a reserved (unstored) local sets the frame size
A named local the original declares but our toolchain wouldn't store (e.g. a call result the
original checks directly) still reserves its 8-byte stack slot at -O0, enlarging the frame.
If a near-match differs only by frame size + a uniform save-offset shift (every instruction
identical), add the missing local as a declaration-only int x; + (void)x; — no store, no
load, no -Wall noise, no code, just the slot. (Assigning the result to the local instead emits a
sw/lw pair the target lacks.) Example: DebugMenuHandler (0x80011144) — if (CdReadRequest(...) != 0) with a reserved int iVar1; → frame 0x20 (score 42 → 0).
Build mechanism — per-file opt override (splat resegmentation)
One original .c = one opt level; you can't mix within a compile unit, and gcc 2.7.2 has no per-function optimize pragma. Split the module into its own splat c-subsegment and give that object its flags:
config/splat.us.exe.yaml: split the text subseg at the module boundary (a function start; file off = vram − 0x8000F800). Boot module =[0x800, c, boot] → src/boot.c; the rest stays[0x2BF0, c, 800] → src/800.c(name kept to avoid migrating matched C).Makefile: target-specific override —build/src/boot.o: CC1FLAGS := …-O0…(the pattern recipe reads$(CC1FLAGS), so this overrides just that object).- Regression gate: the split must rebuild byte-identical at 100% INCLUDE_ASM before any -O0 C
is added (opt level only affects matched C, not stubs). Verified for the boot split.
Reuse this hook for any future module whose flags differ (another opt level, bare
divu, etc.).
§7 PsyQ SDK types & symbols (the library-call prerequisite)
A function that calls PsyQ library routines needs both the SDK types and the library symbols:
- Types: pull the EXACT layout from Ghidra's imported
.gdt(mcp__ghidra__types get <Name>, category e.g./LIBCD.H) — never guess offsets (G1). Declare ininclude/psyq/<lib>.h(it#include "common.h"for u8/u32, guard-safe). Verify sizes with a compile-time assert:typedef char a[sizeof(T)==N ? 1 : -1];. - Symbols: PsyQ fns are already named in the Ghidra DB but absent from our exported
config/symbols.us.txt, so the build shows them asfunc_<addr>. AddName = 0xADDR; // functosymbols.us.txt(R15) AND rename the matchingINCLUDE_ASM("…", func_<addr>)stub(s) insrc/to the canonical name (splat won't rewrite a committed.c). Re-extract → byte-identical (label-only change). No matched C may already reference the old name. - Done Phase 7:
include/psyq/libcd.h(CdlLOC 4B, CdlFILE 24B + CdSearchFile/CdPosToInt/CdIntToPos protos) + the 4 libcd/libetc symbols — unlocks the file-loader cluster. Same pattern for libgpu/libgte/libspu as they come up.
§8 rodata island (compiler jump tables) — the .data→.rodata→.data sandwich (Phase 7)
GCC emits each switch jump table into .rodata; in this EXE all compiler rodata is ONE island at
0x80072A38–0x80074750, sitting BETWEEN the front .data (globals @0x800629DC) and the tail .data
(@0x80074750). No single splat section_order expresses data→rodata→data. Proven mechanism (session C):
- Migrate, don't standalone. A jtbl
.words reference function-internal.L/jlabeltargets, so a separate rodata object can't link — the table MUST co-locate in its function's object. Use a dotted.rodatasubseg whose NAME matches the code subseg ([<off>, .rodata, 800]):extract=False, spimdisasm migrates each single-ref jtbl/const intoasm/nonmatchings/<seg>/<fn>.sas.section .rodata. The INCLUDE_ASM stub already.includes that.s, so it flows into the object for free. Multi-ref rodata can't migrate → splat emitsINCLUDE_RODATA(...)lines (in a FRESH.c). H5: don't regen-fresh the curated.c(drops comments) — surgically INSERT just the INCLUDE_RODATA lines. - Place explicitly. splat is section-major (floats all
.rodatato the front).tools/ld_interleave.py(wired intomake extract) rewrites the.main {}body to text → front-.data→.rodata→ tail-.data→ bss, splitting front/tail by object basename. Sizes then land byte-exact. - Carve data-in-text. A trailing non-code table inside the text range (here 0x80062998–0x800629DC) must be
its own
datasubseg, or jumptable analysis mis-extends the last function across it (the +24main_TEXT_ENDoverrun's first cause). - The
.align 3file-split trap: GCC 8-aligns jtbls; concatenating many functions into one object injects padding nops the original (separate TUs) lacked → image grows. spimdisasm PRINTS file-split suggestions at the misaligned jtbls. Fix = per-file split at those boundaries (sotn-style) — OR link the real library object (§9) when the owning function is SDK code.
§8a rodata island in a flat OVERLAY — the tail sandwich, per matched jr-function (Phase 26 — PoC PROVEN)
The EXE's §8 was one central island. The overlays are different: gcc's switch jtbls sit in ONE contiguous
.rodata block at the TAIL of the flat blob — between the .data globals and a tiny .data remnant
(ov_SC01_077: island vram 0x801D7F9C..~0x801D9460, right after .data global D_801D7F94; layout = text →
data-globals → rodata-jtbls → data-tiny). While every jr-function is INCLUDE_ASM the jtbls emit as in-place
.data and the build is byte-fine. The moment you MATCH a jr-function, its C emits the jtbl into .rodata
(which the overlay section_order:[.rodata,.text,.data,.bss] floats to the FRONT @0x80128158) and the raw
copy is still in the data tail → duplicate + wrong address. The proven fix (byte-identical on func_8012ACE0,
a 25-ins single-jtbl jr-function in ov_SC01_077):
- Carve per matched fn. Split the
[…, data, tail]subseg around that function's jtbl(s) into[…, data, tail](globals + pre-carve jtbls, still raw.data) +[<off>, .rodata, <code-subseg-name>](the fn's jtbl → migrates into itsasm/nonmatchings/<subseg>/<fn>.sas.section .rodata; the name MUST match the code subseg the fn lives in, e.g.ov_SC01_077_a) +[<off2>, data, tail2](post-carve jtbls + tail, raw). Other functions' jtbls STAY raw.datauntil they too are matched (per-fn carve, not whole-island). - Place via the parameterized
ld_interleave(Phase-26: added--section .<binary>→ derives the<binary>_TEXT/DATA/RODATA/DATA2/BSSsymbol prefix; default.main= the EXE, byte-identical): it rewrites the overlay's output section to text → data(tail,--front tail.data.o) → rodata → data(tail2+trailing,--tail tail2.data.o --tail trailing.o) → bss. Wired intomake extractvia a per-binary<bin>_JTBL_INTERLEAVEvar inconfig/overlays.mk(holds the--front/--tailbasenames) + anifneq ($(strip $(JTBL_INTERLEAVE)),)branch. GOTCHA: put NO trailing#commenton theJTBL_INTERLEAVE :=line and$(strip)it — a trailing comment leaves whitespace → non-empty → the branch misfires on EVERY binary (ld_interleave then runs with EXE defaults → "front data object not found" on resident). - The C body needs
canon_sig_reconcilebefore it will compile in the real TU (the raw draft hitsconflicting types for <fn>vs the TU's forward decl +conflicting types for <typedef>vs a sibling; reconcile rewrites the def sig to canonical + uniquifies the draft's typedefs + block-scopes externs). Placement is orthogonal — reconcile first, then the carved jtbl lands byte-exact. - Alignment: gcc emits the jtbl
.rdata .align 3(8-byte). If the original jtbl address is 8-aligned (jtbl_801D8078, 0x…078) there is no pad and it lands exact. A 4-aligned original address (jtbl_801D8AFC) would force a 4-byte align pad → HANDLED (Phase 29): the §8eJTBL_PADSpad-spec filter (hit for real byjtbl_801D8144in the func_80131340 bank). - rtu_match is NOT a whole-binary gate for jr-functions — it masks relocs AND excludes the §8 jtbl rodata, so it MATCHes a body whose switch is subtly wrong (e.g. func_80159C84's 2nd jtbl was 5 words vs the real 6 — a false-MATCH). Always confirm jr-function cracks with the whole-binary gate (which now works, via this carve).
- ×134 automation (NEXT): each overlay sibling has the SAME jr-function at a per-overlay address with its own
jtbl in its own tail → the carve config + the
<bin>_JTBL_INTERLEAVEvar must be generated per overlay from the sibling's jtbl address (a tool overfamily_sweep), then reconcile+template the body per sibling. The PoC proves the per-binary mechanism; the fleet rollout is the mechanical generator.
§8a-pad — a trailing .word 0x00000000 under a jtbl dlabel is .align PAD, not an entry (Phase 26 session 6, byte-proven)
⚠ CORRECTED by §8e (Phase 29): the "maspsx drops all
.align" rationale below is FALSE (thatcontinueis in an inventory-only pass; the output path passes.alignverbatim). The TRIM itself remains correct — the pad belongs to the NEXT table's.align 3, absent when that owner isn't compiled in the same object. Multi-table spans now reproduce interior pads via the §8eJTBL_PADSspec filter.
This retroactively explains the §8a func_80159C84 "5 words vs the real 6" false-MATCH.
The raw dlabel jtbl_XXXXXXXX in asm/<ov>/data/*.data.s can span one word MORE than the switch has
cases. That last .word 0x00000000 is the ORIGINAL TU's intra-rdata .align 3 padding — emitted when a
jump table's entries end ≡4 mod 8 and another jtbl of the same TU follows. It cannot be a table entry:
0x00000000 is not a jump target.
- The true entry count is the function's
sltiu <n>range check, not the dlabel span. Byte-confirmed:func_8015AE2C→sltiu $v0, $v1, 0x7= 7 entries, yet its raw dlabel spans 8 words. - maspsx drops all
.align(maspsx.py:435), so a C-emitted jump table can NEVER reproduce the pad. - Therefore
jtbl_carvemust TRIM trailing zero words from the carve range, leaving the pad in the raw post-carvedatapiece. Carving to the next dlabel reserves 8 words while the compiled object supplies only 7 → the.rodatapiece under-fills by 4 bytes → every later symbol shifts +4 (the same image corruption class as §41d: ~271k differing bytes from one missing word). Trimming is always safe. - Existing carves are parsed from the CONFIG (their
end= the next piece's offset), not re-derived from the data asm, so the trim only affects NEW carves — committed banks are unaffected.
§8b MULTI-jtbl per overlay — the ld_interleave --order sandwich + the same-subseg cases (Phase 26 session 4)
Once ONE jr-function is banked in an overlay, banking a SECOND makes it multi-jtbl (§8a's single-carve breaks:
jtbl_family_bank.revert() restores the committed config = already has carve #1). The generalization:
ld_interleave.py --order <leaf1,leaf2,…>— an explicit, ADDRESS-ORDERED list of the data-region pieces forming the sandwich (text → [these] → bss). A*.data.o/trailing.oleaf contributes its.data; any other (code) object leaf contributes its.rodatacarve. Unlisted.data/.rodatalines must be empty code-object sections → parked byte-neutrally with.text. Generalises the 3-piece single sandwich to N pieces. Legacy--front/--tailpath is byte-untouched (main EXE + single-carve siblings unaffected).jtbl_carve.pyis additive / regenerate-from-config — parse the tail data-region + the existing.rodatacarves, add the new fn's jtbl (split its containing data piece), re-emit the address-ordered pieces + the--orderarg. Idempotent. BOUND-FIX (subtle, cost a false "non-contiguous"): a new jtbl's end is bounded by the next raw dlabel OR the next EXISTING carve start — an already-carved adjacent jtbl is GONE from the data asm, so the raw dlabels alone over-extend the new jtbl past it.jtbl_family_bank.bank()mustmake extractBEFORE the carve (the sibling asm must match the reverted committed config so the carve finds the new fn's RAW jtbl; the old error-string retry was fragile).- PROVEN cross-subseg (fleet-scale): func_801734BC (34-ins PURE jr, clean shared-tail switch
case N:t=-N;break; default:goto after;) inov_SC01_077_after+ func_8012ACE0 in_a= 2 carves / 2 subsegs → banked ×134, R22 136/136 byte-identical. - A code object emits its jtbls CONTIGUOUS (gcc source order), so two matched jr-fns in ONE subseg are
byte-correct only if their jtbls are ADJACENT in the island. Two flavors:
- (a) adjacent → MERGE into one spanning
.rodatacarve (jtbl_carvedoes this; config-proven on func_80171B4C801D8C48+ func_801734BC801D8C68). Byte-proof needs a matched adjacent pair. - (b) non-adjacent (unmatched jtbl between) → ISOLATE one fn into its own code subseg (whale
_o0bprecedent) so each object holds ONE contiguous rodata run.jtbl_carvederives the carve subseg fromfunc_subseg; isolating F preserves carves BELOW F (trim keeps<F) → bank same-subseg families ASCENDING. RESOLVED (session 6, byte-proven):tools/overlay_src_split.py(overlay-.c-aware partition, 404/404 round-trip) +tools/jr_isolate_all.py(multi-cut resegment) + the declaration-environment reconstruction of §8c. Full 54-jr isolate-all on ov_SC01_077 →d19c9580byte-identical, R22 clean-fleet 136/136. Applied LAZILY (isolate only the cores we actually bank — upfront-×134 would add ~7,200 region files):jtbl_family_bankcatchesjtbl_carve'sNON-CONTIGUOUSfail-loud →jr_isolate_all --only <core>→ re-extract → re-carve. Proven onfunc_80178D40(the 890×134 heaviest core): carve blocked → isolated (byte-neutrald19c9580) → carve lands in its own subseg.
- (a) adjacent → MERGE into one spanning
§8c Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6)
The §8b isolation wall. A mechanical source split is not mechanical: this C is written against gcc-2.7.2's lenient scoping, and a cut silently strands declarations. Four file-scope decl sources must be carried forward into each new region (regions are address-ordered and file order == address order, so ambient flows strictly FORWARD — every carried decl already preceded every item of the receiving region in the original file):
- col-0 decls in the
.c— the obvious one (the only one the first attempt handled). DEFINE_func_*macro LEADING externs. The macro expands at file scope toextern <type> <sym>; … <definition>, so its externs ARE part of the invoking TU's file-scope environment — but they live inengine_core.h, so no col-0 text scan of the.ccan ever see them (1,377 macros / 3,929 extern lines / 1,462 symbols). This strandedfunc_801734BCfromextern s16 D_80126B3E;. (Correcting the session-5 hypothesis: this is NOT a "block-scope extern persists to file scope" gcc quirk — the externs are genuinely file-scope, just textually invisible. The 148 externs INSIDE macro bodies are real block-scope shadows and must never be hoisted.)- A function DEFINITION is itself a declaration for everything below it in its TU. Cut the definition into
an earlier region and every later caller that took its address breaks (
func_8012B2CC undeclared). Synthesize its prototype — and a K&R definition declares an unprototyped function, so it must renderextern T f();, neverf(void)or the K&R param names. - File-local typedefs used by a carried prototype (
extern s32 f(Vec3s *a0)→parse error before '*'). Legal to re-emit because each region becomes its OWN TU. Emit types before decls.
Why NOT "declare every used symbol from a global symbol→type map" (the intuitive design): this codebase is
loosely typed, so a symbol legally carries contradictory decls — func_80173544 is defined at file scope as
s32 f(void *) yet declared extern void f(void); inside func_801734BC's body. Hoisting "every used
symbol" lifts that block-scope shadow to file scope, where it collides with the definition — so the design then
needs a heuristic "type-shadowed set" to dodge a problem it created. Carrying forward only what was already
file-scope is conflict-free by construction: every carried decl already coexisted with every definition in
the one original TU, and decl compatibility is order-symmetric. Shadows stay in bodies and travel with them.
- Dedup by exact decl TEXT, not by symbol. One symbol legitimately has several distinct file-scope decls
(the baseline build emits 87
type mismatch with previous external declwarnings and is byte-identical). Collapsing to the first drops a decl the original had. - Baseline-parity is the warning oracle: diff the isolated build's warnings against the baseline's. New classes mean you changed decl visibility; identical classes mean you reproduced it.
- TRAP —
func_subsegfrom the asm tree is stale-prone.make extractdoes not prune stale subseg dirs, so after an isolation BOTHnonmatchings/<ov>_after/<f>.sandnonmatchings/<ov>_jr_<A>/<f>.sexist; anos.listdirscan returns the STALE owner and silently re-creates the very collision the isolation removed. Derive the owning subseg from the CONFIG (address → containing code piece). - TRAP — a sweep's revert must restore, not delete.
overlays.mkis SHARED by all 134 overlays and every one now has a committed<ov>_JTBL_INTERLEAVE; the old revert dropped the line unconditionally, destroying a banked carve on any failed sibling. Restore it to its committed value (git show HEAD:), splice per-overlay (nevergit checkoutthe shared file mid-sweep), and delete only the region files this attempt created.jtbl_family_banknow refuses to start on a dirtyconfig/+src/(an uncommitted prior family would be silently reverted) — commit each family before sweeping the next.
§8d Templating a body INTO a TU must not CHANGE its declaration environment — demote the carried data externs (Phase 26 session 8, byte-proven on func_8015AE2C ×133)
The mirror of §8c. There, splitting a TU meant carrying its decl environment forward. Here, templating a cracked body into a sibling TU means not disturbing the environment that is already there — and the ×N family sweep was doing exactly that, silently.
family_remap.gather_externs carries the exemplar's decl for every symbol the body references and prepends
them at FILE scope. For a per-location DATA symbol that the sibling declares only at BLOCK scope —
inside its own later functions, loosely typed — that carried decl establishes a global declaration the TU
never had, and every later block-scope extern of that symbol must now agree with it. In loosely-typed
engine code they never do. The whole-binary gate proved both halves:
BLOCK(int) -> BLOCK(struct Ent *) -> FILE(void *) ... builds [ OK ] (the region, stub state)
FILE(void *) -> BLOCK(int) -> ... conflicting types for `D_801812A4' (ERROR)
D_801812A4 (ov_SC01_000's entity dispatch table) is declared four incompatible ways in one region —
(int) and (struct Ent_8015CD20 *) at block scope inside func_8015C128 / func_8015CD20, then (void *)
at file scope ×12. That compiles. Prepend the body's extern void (*D_801812A4[])(void *); above them and it
does not. It was the only hard error in the build; all 27 carried function externs were fine raw.
THE FIX (tools/scope_data_externs.py, a pure draft-text transform): emit a carried D_ extern at
block scope inside the function body whenever the target TU has no file-scope decl of that symbol above
the insertion point. It then declares no global, nothing below it can conflict, and the TU's environment is
preserved exactly. Byte-neutral — an extern emits no code, and moving it changes neither the symbol nor
the declared type, so every access keeps its opcode. It also restores fidelity: the original source declares
these symbols at block scope in precisely this way (m2c/Ghidra emit per-function externs there).
Wired into jtbl_family_bank as the scoped stage (raw → scoped → recovered → reconciled) and used as
the base for the later recovery stages. First sibling byte-identical; 562-ins core ×133.
- Never worse than raw, which is why it can be applied unconditionally: a symbol that does have a file-scope decl above is left alone (an identical spelling is a legal duplicate; a differing one is the §41 reconcile class and errors at file scope either way, so demoting could not have saved it).
reconcile_declsis the WRONG instrument for this class, twice over. (a) Its oracle is fleet-majority (engine_core.h first-seen, else a plurality vote across all overlays) — but the question is not "what does the fleet call this symbol", it is "what can this TU see". (b) ItsDATA_DECL_LINE_REcannot even parse the fn-ptr-array formextern void (*D_x[])(void *);, so it silently skipped the very symbols that were failing. A tool that no-ops on the failing input reads exactly like a tool that had nothing to fix.- TRIAGE RULE (R17 boundary, Drew 2026-07-13). "The compiler produced the wrong BYTES" → read the gcc
source (regalloc / sched / cross-jump / CSE — things no C change reaches). "The compiler refused to
compile" → read our Python. This was
conflicting types: a C front-end diagnostic, gcc correctly rejecting plain C89. Readingcse.c/global.cwould have taught nothing; the bug was ours. - Diagnostics gotcha: gcc-2.7.2 does not prefix errors with
error:— grepping a build log forerrorfinds only make'sError 33. Grep for the diagnostic text (conflicting types,undeclared,parse error,redefinition) instead, and rememberwarning: conflicting types for built-in function 'memcpy'is benign.
§8e The jtbl ALIGNMENT LAW + the pad-spec filter — multi-table .rodata spans (Phase 29, byte-proven; .run/probe_jtbl/verdict.md)
Two prior cookbook claims are CORRECTED here (both were instrument errors, R35):
- §8a-pad's "maspsx drops all
.align" is FALSE. Thecontinueatmaspsx/__init__.py:435is inpreprocess_lines— an inventory-only pass (sbss/bss/sdata dicts) whose skips produce no output; the real output path (process_line, L872-873 catch-all) re-emits.alignverbatim. The §8a-pad trim is still right, for a different reason: the trailing pad word belongs to the NEXT table's.align 3, which is only emitted when that next owner is compiled in the same object. - The Phase-29 session-2 "half-pin" ("cc1 AND maspsx emit the jtbl
.align 2") was inverted, measured off VACUOUS probes (an emptyj $31fn — no jtbl in them at all; the only.align 2was the function-entry.textalign). Lesson: a probe whose output contains no instance of the thing being probed pins nothing.
The law (each link byte-verified in .run/probe_jtbl/):
- cc1 (Sony gcc 2.7.2) emits
.rdata+.align 3+ label before every switch jump table (probe: 2 tables in one TU → 2×.align 3; the clean single-table object's.rodatash_addralign=8). - maspsx passes
.alignthrough;asbakes the pad into the section SECTION-RELATIVE — the linker can never remove an intra-object pad. (ascontrol: bare.wordsection → Al=4; with.align 3→ Al=8.) - Link placement is always TIGHT:
SUBALIGN(2)(splatsubalign: 2, fleet-wide) +ld_interleave's. = ALIGN(., 4)override input-section alignment — a 4-mod-8 carve start places exactly (the banked0xb07dccarve is the byte proof). So only intra-object pads can diverge from the original. - ORIGINAL layout semantics: originally-separate TUs pack TIGHT (PSX linker, 4-aligned placement:
jtbl_801D8078's 51 entries end 0x801D8144 exactly where the next TU's table begins, %8==4); intra-TU
consecutive tables carry a REAL zero-word pad wherever the previous table ends ≡4 mod 8
(tail2.data.s: jtbl_801D8158/8170/8188/81A0 = 5 entries + one
.word 0each). - ⇒ Merging originally-separate TUs into one decomp TU makes cc1's intra-TU
.align 3fire where the original had a tight TU boundary: a non-first table at original vram ≡4 mod 8 gains a +4 interior pad → every downstream data symbol shifts →%lorelocs break image-wide (func_80131340: pad at rodata 0xCC, SHA1 fail from a clean build). Conversely a genuine intra-TU pad must be REPRODUCED. - Isolation does NOT fix this in general:
.alignis section-relative, so an isolated object whose first table starts at vram ≡4 mod 8 flips the parity of every INTERNAL align — a multi-table function with a 4-mod-8 first table would mis-pad inside its own object. Only explicit pad control is general.
The mechanism (tools/jtbl_rodata_pads.py + jtbl_carve + Makefile JTBL_PADS): for a multi-table
span, jtbl_carve derives each boundary's pad by interval arithmetic (pad[K] = start[K] − end[K−1]
∈ {0,4}; a 4-gap must be a verifiably-zero payload word, else NON-CONTIGUOUS→isolate) and writes a
per-object build/src/<ov>/<sub>.o: JTBL_PADS := 0,4,… target var into config/overlays.mk; the Makefile
pipes that object through the filter, which REPLACES each rodata .align with the spec'd pad bytes
(.word 0 or nothing). Per-sibling self-adapting (each overlay's own addresses), fail-loud on drift
(spec count ≠ align count; non-.align 3; non-jtbl rodata content). Committed spec values are CARRIED,
never re-derived (a committed span's interior boundaries are unrecoverable from its interval).
Single-table carves get no var — their pipeline is byte-identical to pre-§8e (their lone .align 3 at
section offset 0 pads nothing and SUBALIGN neutralizes the sh_addralign).
- Object-layer proof before image-layer claims: verbatim vs filtered objdump — 0xE4/pad-at-0xCC vs 0xE0/tight — settled the mechanism before any carve landed. Cheap, decisive, reusable probe shape.
§8e-2 — the spec-derivation law (learned banking the 4 giants, Phase 29):
- The ZERO-WORD rule is the whole derivation: pad before table K ⟺ payload word at
start[K]−4is0x00000000(a zero can never be a table ENTRY — the §8a-pad axiom). No entry counts, no interval bookkeeping.spec_from_startsneeds only the span's TABLE STARTS + the payload. - Table starts are PERISHABLE — persist them.
make extractprunes a matched owner's stub.s, so a span's interior structure becomes unrecoverable (thefunc_8013F350lesson: the existing0xb0740carve was a Phase-26 merged double (8+5 tight) that the red-team's "all single-table" sample missed — R14; its structure had to be re-derived from the compiled stream + span length). TheJTBL_PADSline now persists starts as span-relative offsets (tables=+0x0,+0x20,…); source priority per span: untouched+line → reuse verbatim · untouched+no-line → skip (its natural.align 3s are committed-green) · touched → union{new fn's.s, surviving stub.s, linetables=rebased,--span-tablesoverride,--like <exemplar>role-transfer} → zero-word rule. Sibling sweeps get structure via--like(same family ⇒ same span shape; pads still derived from the LOCAL payload). - 4-mod-8 FIRST tables are placement-only (SUBALIGN packs them tight —
jtbl_801D836C/0xb07dcproofs); only NON-first tables need the spec. A single-table carve never needs a line. - Splice-reconcile carry-overs (the 59C84/F350 banks; all §56-class, byte-neutral): the draft's
standalone scalar typedefs must be STRIPPED in-TU; a conflicting draft decl is replaced by the
macro-canonical redecl (NOT just dropped — C89 rejects file-scope use-before-declaration, so the
decl must exist above the use) + §18 width-preserving store casts (
*(u16 *)&D_x = …under a canonicalu8) + §17a-1 fn-ptr call casts where the canonical prototype differs from the matched call shape (((s32 (*)(s32))func_801416D4)(…)under a(s16)def; a prototyped(void)canon needs it too). §30#2 def-side widen (extern void→s32in every DISCARDING caller decl, incl. engine_core macros). - Crash-ordering gotcha:
jtbl_carve.apply()writes the splat yaml BEFORE the overlays.mk vars — a crash between the two leaves a half-applied config vs stale asm (symptom: phantom trim counts / shifted NON-CONTIGUOUS offsets). Recover:git checkoutboth configs + re-extract, then re-carve. - The CLEAN-DRAFT law (the 59C84 sweep lesson, 3/8→100%): a family sweep's
--rawdraft must carry the exemplar's reconciles as canonical-form decls + call casts IN THE DRAFT TEXT (typedefs stripped; decls in the fleet-canonical spelling; §17a-1/§18 casts baked in). The per-sibling ladder absorbs some of a dirty draft on some siblings (3/8), which disguises the class as random — diagnose by re-running ONE failed sibling with the cleaned draft (BANKED ⇒ the class is the draft, not the siblings). Symbol names remap per sibling, so textual canonical fixes transfer. - Named deferral classes from the 4-giant campaign (fail-loud, zero corruption, burn-down items):
(i) the 4 SC07
files=1overlays compose a new span against a PRE-EXISTING_o2bcarve whose object emits no tables at raw stage (the count-guard refuses — needs a per-overlay look); (ii) an -O0 family member cannot bank until its overlay has the -O0 cluster carve (rollout_o0_cluster, the Arm-A splat wall) — func_8013C414 banked ×1 (ov_SC01_077 has_o0) but its 137 siblings uniformly gate-fail at -O2; the family rides the -O0-rollout dependency, not a codegen wall.
§9 Link real PsyQ library objects byte-exact (Phase 7 — GO proven)
~350 of BFM's functions are unmodified PsyQ 4.0 SDK code. They are byte-identical to the real PsyQ library
objects, so link them directly instead of hand-decompiling — and each library .o brings its own correct
alignment (dissolving the library-half of §8's .align 3 problem). Validated: CdPosToInt/CdIntToPos EXACT
vs PsyQ libcd; PRESET_OBJ_* ∈ LIBGS.LIB. Workflow (the decomp-standard psyq-obj-parser path):
- Tools (gitignored
tools/psyq/):psyq-obj-parser(decompme prebuilt —.OBJ→ELF; rejects.LIB),lib40/*.LIB= PsyQ 4.0 USA libraries (DTL-S2002 R2.0 = BFM's version; extracted from the redump ISO viatools/bfm_extract/iso9660.py). Identify a function's library by searching the.LIBfor a NON-relocated instruction run from its EXE bytes (relocated runs false-negative — use leaves or interior runs). - Integration: split
.LIB(LIB\x01 archive) →.OBJ→psyq-obj-parser→arper lib → link the.ofor each library function and drop its INCLUDE_ASM. BFM mixes 4.0+4.2 library stamps, so a few objects may need 4.2/4.3 libs — determine per-object by the byte test. - Proven full-object link recipe (SYS.o byte-identical to BFM, Phase 7):
- Placement —
tools/psyq_identify.py <elf_dir>: relocation-masked search finds each object's.textvram in the EXE. Per library the used objects are CONTIGUOUS in object order → place the first at the region base, link the rest in order. - Recover externals — symbols the object references but doesn't define are usually absent from
symbols.us.txt; read them straight out of the EXE's RESOLVED relocations: for each reloc,R_MIPS_26→target = ((word&0x3FFFFFF)<<2)|(pc&0xF0000000); anHI16+LO16pair →(hi<<16)+signext(lo). Feed asld --defsym NAME=0xADDR. - Alignment — psyq-obj-parser emits
.text/.rdata/.dataat align 2**3; the original is 4-aligned, so an 8-align bumps the section +4 (the tell: everyLO16to that section is off by +4). Fix:objcopy --set-section-alignment '.rdata=4' --set-section-alignment '.data=4' obj.o obj_a.obefore linking. - Link + verify —
ld -T <SECTIONS: . = <text vram>; .text:{*(.text)} . = <island>; .rdata:{*(.rodata) *(.rdata)} . = <data vram>; .data:{*(.data)}> --defsym … obj_a.o→objcopy -O binary --only-section .text→ byte-compare to the EXE..rdata/.datavrams are found by searching the EXE for the section bytes (objcopy --only-section). Tools:tools/psyq_lib_split.py,tools/psyq_build_libs.sh,tools/psyq_identify.py.
- Placement —
§9.1 Generalised per-object linker — tools/psyq_link.py (+ psyq_link_lib.py), 18/18 libcd byte-exact
Session-D generalised the SYS.o recipe into a tool that links every used object of a library byte-identical. Two gotchas the one-object recipe didn't surface, both now handled:
- psyq-obj-parser MISLABELS common-style globals. Uninitialised globals (PSYLINK
.comm) get packed into each object's.bsswith sequentialst_values, but the original linker SCATTERED them (e.g. libcd CDROM'sStRingAddr→0x800c7c94andStRingSize→0x800c7f00are 0x26c apart, the ELF claims 8). Trust nost_valuefor placement. - Robust model = recover-everything + selective override. Place
.textat its vram and the real initialised sections at their bases (byte-search; or, for.bss/reloc-bearing.data, the address the section symbol itself resolves to in the EXE). Resolve every symbol the.textreferences by the address read out of the EXE's already- linked relocations (R_MIPS_26jump field;HI16+LO16immediates; object addend subtracted, but PsyQ addends are 0). A symbol that is a genuine member of a placed section (recovered == base+st_value) is left told; a mislabelled one is--weaken-symbol'd then--defsym'd to its recovered address (a strong defsym beats the weak section def —--strip-symbolis refused on reloc-referenced symbols, weaken isn't)..textbyte-compare is the check (G3). - Tells:
ld: 'X' referenced … defined in discarded section= you discarded a section whose section-symbol the.textneeds → place it instead. A 1–4 word residual inlui/lw/swimmediates (3c0480xx) = a mislabelled.bsscommon → weaken+defsym it. - Externals split intra/extra-library. Per-object "externals" (UND) include symbols defined in sibling objects
(resolve internally in a whole-library link) vs truly external ones (other libs' funcs like
VSync/memcpy, and module data globals likeSt*) — the latter feed--defsym/symbols.us.txt(R15). libcd: 82 union = 48 intra + 34 extra.tools/psyq_link_lib.py <elf_dir>links all located objects, flags address conflicts, writes.run/psyq_link.<lib>.json. Same tooling will serve libgs/libspu/… (the +24 culprits).
§9.2 Wire a library region into the build with NOLOAD — no data carving (tools/psyq_link_region.py)
To replace the asm stubs of a library's functions with the real objects in the byte-identical build WITHOUT
carving the flat data subsegment:
- Place
.textLOADED at each object's exact vram; place.data/.rdata/.bssas NOLOAD at their vrams. A NOLOAD section contributes its symbol addresses but zero bytes toobjcopy -O binary, so the build's existing flat data subsegment still emits those bytes (no double-emit, no carve) while the hundreds of section-relative.textrefs resolve via the NOLOAD placement. Set.data/.rdata/.bssalign=4 first or a 4-but-not-8-aligned vram bumps +4 (same tell as §9). Needsld --no-check-sections(NOLOAD overlaps the loaded flat blob's VMA). - Weaken every
.bss/.sbss-defined named symbol, then--defsymit to its recovered address: the common-style globals are scattered (genuineCD_*and mislabelledSt*alike, and a.bsssymbol of object A may be referenced by object B), so a uniform strong-defsym-beats-weak-def resolves them all. Truly-undefined externals (other libs' funcs) surface from a probe link'sundefined referencelines → defsym from the recovered map. - Place each object at its EXACT vram, not by concatenation — a library's objects are mostly contiguous but a
non-library function can sit between them (libcd: a 76-B gap of non-libcd code between C_003 and C_004), so naive
*(.text)concatenation drifts past the gap. The gap stays an asm stub in the build (split the splat code subseg into [pre][lib block 1][gap stub][lib block 2][post]).tools/psyq_link_region.py <elf_dir> --emit <p>verifies the region byte-identical per-object and emits<p>.ld(text + NOLOAD lines) +<p>.syms. libcd: 18 objects byte-exact, 36 externals.
§9.3 Make it the build: resegment + swap + resolve (tools/psyq_integrate.py, libcd DONE)
Wiring a library region into make build byte-identical (libcd: 58 SDK funcs, full pipeline green):
- Resegment the splat text subseg into [pre][block1][gap][block2…][post] at the library blocks
(one
csubseg per block + per non-library gap; vram→file = −0x8000F800).make extractregenerates the gap/post stubs;tools/split_src_region.py trimrewrites the curated pre-file (keeps items <lo, preserving real C and#ifdef NON_MATCHINGblocks by brace/#endifmatching) — splat will NOT overwrite an existing.c, so a stale one mis-places everything. splat-auto-empties (void f(void){}) ≥hi regenerate identically — no move needed. psyq_integrate.py(run in the$(OUT)recipe, after objects compile — the externals discovery trial-links the whole image): (1) prep objects (align=4 + weaken every.bsssymbol) →build/psyq/<lib>/; (2) rewrite the splat.ld— replace eachbuild/src/<stub>.o(.text);with the block's real<obj>.o(.text);(concatenation places them at their vrams since the pre-file ends exactly at the block start) AND delete the stub object's other(.rodata/.data/.bss)lines (else its stub symbols multiply-define the real ones); add per-object NOLOAD data sections sorted by vram (unsorted → "dot moved backwards"); (3) resolve externals via a full trial link (symbols still defined elsewhere never appear, so no double-def — no blanket exclude needed).- External resolution, in order:
func_<addr>(external code/data calls a libcd fn by its splat address-name; the real object exports a PsyQ name) → that address; asymbols.us.txtname (jump-table / dispatch pointer in the flat.data, e.g.BIOS_OBJ_3B8) → itssymbols.us.txtaddress; a recovered data/extern global (St*/CD_*) → recovered. Always also defsym EVERY weakened.bsscommon — the ones whose object's.bssis NOLOAD-placed resolve to that weak placement and never show as undefined (the 8-wordStModemiss). Capture BOTHundefined referenceANDdefined in discarded sectionfrom the trial link. - Byte-identical with OR without the SDK objects (the stubs reproduce the same bytes), so gate the
whole thing on
[ -d <elf_dir> ]— a fresh clone withouttools/psyq/builds via stubs. Idempotent (build/psyq/<lib>in the.ld⇒ re-derive syms only).
§9.4 Integrating a SECOND library (libgs block 6 after libcd) — multi-library gotchas
Wiring a 2nd psyq_integrate call into the same build (libcd, then a libgs block) surfaced bugs the
single-library path never hit. All fixed in tools/psyq_integrate.py; reuse for libspu/libsnd/…:
- Namespace the NOLOAD section names per library (
.nl_<objdir-basename>_<i>, e.g..nl_libgs6_0). The idempotency guard was a globalif ".nl_0" not in ld— so the 2nd integration saw.nl_0(from libcd) and SKIPPED adding its own NOLOAD lines, discarding that library's.data/.rdata/.bss(symptom:'.bss'/'.rdata' referenced … defined in discarded sectionfor the 2nd lib's objects, §9.1 tell). - Globally re-sort ALL
.nl_*NOLOAD lines by vram across libraries at the end of each integrate. Two separately-sorted groups whose vram ranges interleave make ld's location counter jump back (warning: dot moved backwards) — harmless (NOLOAD emits no bytes) but noisy. Pure reordering. - Pass already-emitted sibling
*_externals.ldto the trial link. The 2nd library's trial link sees the FIRST library's real objects (already in the.ld) referencing symbols defined only in the first library's externals (e.g. libcd objects callDMACallback/DeliverEvent); without the sibling syms the trial reports them as spurious!! UNRESOLVED. Globbuild/psyq/*_externals.ldminus the current one. - A resegment can shift spimdisasm's auto-detected function/data boundaries in the UNCHANGED regions
around the new subseg (shrinking 800b re-merged
func_80052FCC/053050and re-typed theD_80062998data table as afunc_). Two deterministic fixes, both from the real artifacts (G1): declare the affected REAL functions insymbols.us.txt(verify via the PsyQ object's symbol table — these wereGsMulCoord2/3in MATRIX.o), and carve any trailing data-in-text table as its owndatasubseg (§8; here extend the front-data subseg back to the table start, updateld_interleaveFRONT_DATA basename). Always regenerate the splitsrc/*.cfrom the fresh extract so stubs match the generated.s. - Block selection is by EXE-placement + disambiguation, not the full library. A library's objects span
several non-contiguous blocks (libgs: 6 blocks + gaps incl. the GS_001 scattered-
.bssgap, §9.1).psyq_identify <full-lib-dir> <lo> <hi>over ONE block's range reports the right objects PLUS byte- identical-.textaliases (GS_131≡RVWUNIT, PRESET2≡PRESET3, OBJT2≡OBJT3); keep the one that also matches.data/.rdata(psyq_link_region --verifyconfirms per-object). Hardcode the disambiguated object list in a committed regen script (tools/make_libgs.sh) — SDK-derived dir, gitignored.
§9.5 Integrating a WHOLE multi-block library in one call (full libgs — Phase 7 session G)
Block 6 alone (§9.4) proved the pattern; the full library is then ONE psyq_integrate call over all of
its used objects — no separate integration per block. integrate auto-splits the elf dir's vram-ordered
objects into contiguous runs (contiguous_blocks) and maps the i-th run to the i-th stub: pass one
block stub per contiguous run, and a plain asm stub per non-library gap (gaps keep their own subseg).
- Derive the block/gap structure empirically (G1), don't trust the notes.
psyq_identify <used-dir>prints each object's vram; a new block starts wherever the next object's vram ≠ the previous object's end (vram + nins*4). libgs = 6 blocks, gaps 80/48/1536/48/304 B (the 1536 is the excluded GS_001). Resegment the splat code subseg into[game-pre][block1][gap1]…[blockN][gapN]…(vram→file = −0x8000F800); block subsegs become thelibgsNintegrate stubs, gap subsegs staygsgapNasm stubs (incl. the GS_001 gap). Onesplit_src_region.py trimdrops the old single-subseg's stubs from the pre-file — but verify FIRST that no matched C lives in [lo,hi) (trim drops that range unconditionally; here all matched fns were ≤0x8002Axxx, far below the 0x80051804 libgs base). - Short objects need the placement WINDOW (new
psyq_integrate <stubs> <lo> <hi>arg). An object whose.textis too short to anchor uniquely over the whole EXE — libgs GS_106 (8 ins; its pattern recurs in game code) — isambiguousin the default 0x80010000..0x800629DC scan, so it drops from the placement map, its block splits, and the block↔stub count breaks. Pass the library's text window (wired in the Makefile integrate call:… libgs1,…,libgs6 0x80051804 0x80057928); the object anchors uniquely there.tools/make_libgs.sh --verify(psyq_link_region over the same window) is the preflight. - Result + checks: 31 libgs objects / 6 blocks linked byte-identical in one integrate (≈49 named SDK functions — count OBJECTS, not stubs: splat over-segments library code into ~5× more INCLUDE_ASM stubs than real functions). The finer resegmentation (10 new subsegs) was split-deterministic across two clean extracts (no new §9.4 boundary re-detection — the GsMulCoord2/3 declarations + the 53198 data carve from §9.4 already cover it) and byte-identical WITH or WITHOUT the SDK objects (stub fallback, fresh-clone-safe). The unified call cleanly supersedes the §9.4 block-6-only integration.
§9.6 Scaling library linking to the whole EXE (Phase 8 — 8 libs linked, 20%→50% byte-identical)
Phase 8 linked the remaining footprint libraries (libetc/libgpu/libmcrd/libc2/libgte/libspu/libsnd/libapi/ libcard). New patterns + tools that make a fragmented, multi-library EXE tractable:
- Survey first (
psyq_identifyover every built.run/obj40/*). Produces the byte-confirmed footprint map (docs/psyq-worklist.md): which libs place, how many blocks, regions, aliases. 3 libs (libmath/libc/ libsn) place 0 objects — BFM links libc2 not libc. Replaces guesswork (R14). tools/gen_lib_subsegs.pyautomates the multi-block resegment. A library scatters across a game-code region in many contiguous blocks (libgte = 22, libsnd/libspu combined = 9). The tool places the objects, groups blocks, and emits the splat subseg lines (libNblocks + game-code gap frags) + the integrate stub list. Boundary gotcha (was a real bug): a block's end = the last object's.textSECTION size (readelf, 8-aligned), NOTpsyq_identify's instruction count × 4 — the count omits trailing align pad (libc2 SETJMP.o: 30 ins = 0x78, but.text= 0x80). A too-low boundary overlaps the object's padded tail and the relink inserts +N padding, shifting the WHOLE downstream image (pervasive 1-byte reloc diffs + a grown file). The tool bakes the section-size rule in.- Pass the stub list via a make var (
LIBGTE_STUBS := libgte1,…) for the long ones;progress.py'slinked_subsegs()resolves$(VAR)from the Makefile's:=defs so LINKED still counts them. - Interleaved libraries → ONE combined region. libspu+libsnd interleave object-by-object in 0x3A444..
0x4239C, so two independent passes tangle (each lib's objects span the other's gaps). Instead build a
combined curated dir (
tools/make_snd_used.py): merge both libs' objects by vram, and for an aliased address (>1 object, same masked.text) pick the one whose linked.textbyte-matches the EXE (psyq_link.link_object). libapi+libcard share the same trick (make_apicard_used.py, C112 dedup). - Scattered-
.bssexclusion = the GS_001 class, now also cross-object. An object whose.bsscommons the original linker scattered (referenced as.bss+offset via one section symbol, but resolving to >1 base in the EXE) can't be reproduced by a single NOLOAD base → EXCLUDE it (stays a byte-identical stub). Detect:psyq_link_regionshows N words differ in that object andconflicts>0. Exclude by address (the alias twin fails identically). Cases: libgpu SYS.o, sound S_R/S_GRMDT/VM_F; plus a false placement (S_IH @0x3D94C is inside libsnd SSSTART.o). Excluding a few objects banks the other 60. - CLEAN-REBUILD gotcha (verification).
psyq_integraterewrites the.ldin place; after a src change an incrementalmake buildcan re-run integrate on an already-rewritten.ldand transiently mis-resolve a sibling library's externals (a harvest falsely diffed in libmcrd). The canonical verify is alwaysmake clean && make extract && make build— never trust an incremental build for a byte check. - Honest metric: linked objects stay INCLUDE_ASM stubs in their
.c(the fresh-clone fallback), soprogress.pycounts them in a distinct LINKED bucket (not REAL, not stub). REAL = hand-written C only.
§9.7 Binary-agnostic toolchain refactor (Phase 9) — the reusable pattern for Gen2
Gen2 builds many binaries (resident blob, location overlays) with these same tools. The refactor that got there, and the technique to land it safely without ever breaking the byte-locked EXE:
- Required params, no defaults. Every binary-specific value —
--vram-base(the fileoff→vram delta),--exe,--symbols— is a REQUIRED argparse/function parameter. No module-levelEXE/VRAM_BASEdefault an overlay could silently inherit; a miss fails loud (argparse error / NameError). The roadmap's #1 risk was a hidden EXE default surfacing as a wrong overlay address only at Phase 10 — required params make that impossible by construction. --vram-baseis a single scalar (EXE0x8000F800=0x80010000 − 0x800), sufficient for any flat-loaded PS1 image. The text-scan window[lo hi]is a SEPARATE, orthogonal param (scan-narrowing only, for short/ambiguous objects) — never conflate the two.- Transitional-default technique (keeps every per-tool commit green despite in-process coupling).
psyq_integrate/psyq_link_regionimportrecover_sym_addrs/VRAM_BASEfrompsyq_linkIN-PROCESS, so deleting the global in one tool's commit breaks the build mid-sequence. Instead: each tool first gains the param DEFAULTING to the kept EXE global (build stays byte-identical as callers are updated one commit at a time); a FINAL commit removes the globals + all defaults → required, once every caller passes explicitly. Refactor leaf-first (psyq_link→psyq_identify→psyq_link_region→psyq_integrate) so a missed call site fails loud, not silently on a stale global that happens to hold the EXE value. - Negative control proves threading. A pure no-op (
143dbb89…unchanged) can pass for the wrong reason (param accepted-but-ignored). Always ALSO pass a deliberately wrong--vram-baseand confirm the build/link DIVERGES (cae22f7e…≠ target) — that proves the value is load-bearing. Per-tool:psyq_link.py … --vram-base 0x8000F900must FAIL where0x8000F800PASSes. - Makefile shape.
BINARIES := main(alias keys) +main_*vars +$(BINARY)-resolved aliases (OUT/LD_SCRIPT/VRAM_BASE/…). EXE artifact paths preserved verbatim (no rename churn against the oracle). EXE-only SDK-integration +ld_interleaveblocks gated underifeq ($(BINARY),main). Lockstep gotcha:progress.pyparses the Makefile'spsyq_integratecalls for the LINKED subseg list, so when you add leading--flag valuepairs to those calls, its stub-list regex must consume them (psyq_integrate\.py(?:\s+--\S+\s+\S+)*\s+\S+\s+\S+\s+\S+\s+\S+\s+(\S+)) — land the regex change in the SAME commit, gated onmake reportreproducing the LINKED count.
§10 Closing the regalloc/scheduling hard tail by hand (LZSS, Phase 7 session F — the full close)
LzssDecodeSector (0x80018730) was the last-mile case the §5a barrier set up but did not finish: with the
cross-jump barrier the instruction COUNT was correct (122) but ~4 register-allocation / scheduling slots were
wrong. The decomp-permuter could NOT measure progress (its object score floats on the .rodata-vs-jtbl
floor — §5a caveat), and its random search diverged. Hand-solving with the §3a research tier won — every
fix below was ground-truthed against the pinned gcc-2.7.2 source (reorg.c, jump.c, local-alloc.c).
These idioms are general; reach for them whenever a function is instruction-count-correct but off by a few
regalloc/schedule slots and the permuter can't score it.
The clean object-level metric (use this, not the permuter score, for jtbl/rodata functions)
The permuter/asm-differ object score is polluted by the migrated-jtbl symbol name (.rodata vs
jtbl_<addr>), so it can't see real .text progress. Two floor-free checks (no link needed):
- Normalized instruction diff —
objdump -dr --no-show-raw-insn -j .text, strip theR_MIPS_*lines, the<sym>operands and the branch-target hex, thendiffcandidate vstarget.o. Shows ONLY real register/opcode/order differences. (.run/permuter/LzssDecodeSector/try.shis the reference impl.) - Raw
.textbyte compare —objcopy -O binary --only-section=.text cand.o c.textand… target.o t.text, thencmp -l. In a relocatable object the%hi/%loimmediates of an unresolved symbol are BOTH 0 (the reloc fills them at link), so a migrated jtbl reference is byte-identical here and does NOT show — the only diffs that remain are genuine (e.g. a wrong local branch offset). This is the fast, authoritative iterate-on-.textoracle; finish with the linkedmake check(G3) for the whole-binary truth.
Residual A — commutative |/&/+ result lands in the wrong source-operand register
local-alloc.c combine_regs (≈line 1855) ties a commutative op's result to the first RTL operand that
dies at the insn (RTL operand order = source order; gcc 2.7.2 has NO swap_commutative_operands, so the
source order survives). Target or $v0,$v0,$v1 ⇒ result tied to the code&0xFF operand ($v0).
- Fix A1 — operand order: write the operand whose register you want the result in FIRST:
code = (code & 0xFF) | (nh << 8);(not(nh<<8)|(code&0xFF)). - Fix A1-companion — DECOUPLE shared inputs (load-bearing): if a variable feeds two expressions
(
nbfed both the low- and high-byte ORs), reshaping one OR re-allocates that variable in BOTH paths (it jumped$v1→$a0). Give the second use its own variable (nh) so the operand-order change is local. This was the unlock — A1 alone "didn't work" only because of the coupling.
Residual B — a return <const> materialised late / merged instead of distributed per-site
reorg.c fill_simple_delay_slots backward-scan (≈line 2907) pulls the common li $v0,K out of the
predecessors into the shared epilogue's branch-delay slot, and redundant_insn then collapses the other
copies — so one li $v0,1 ends up in the j <epilogue> slot instead of one per return site. Three levers,
applied where each fits:
- B-distribute (shared tail reached by ≥2 predecessors): carry the value in a plain local set in each
PREDECESSOR block (
result = 1; newState = N; goto save;…save: …stores…; return result;). Because the value is live-in from two defs, gcc emits a distinctli $v0,1per predecessor and leaves the tail's delay slotnop. (A barrier is NOT needed for this half; the predecessor structure is.) - B-schedule-early (single-path block whose
li $v0,K's only use is the sharedjr ra): the sched list-scheduler gives an independentli $v0,Kpriority 0 (its use is in another block) and the tie-break drops it to just beforejr ra; the target schedules it first. Force it with an explicit$v0register local pinned by a read-only input-asm BEFORE the stores:A plainregister s32 r __asm__("$2"); r = 1; __asm__ __volatile__("" : : "r"(r)); /* materialise li $v0,1 here, ahead of the stores */ …stores… __asm__ __volatile__("" ::: "memory"); /* the §5a cross-jump barrier, still required */ return r;resultlocal does NOT work here (gcc rematerialises the constant at the return); the+r/"=r":"0"read-write pins put it in$v1; only the explicit-$2local + early read-only input pin landsli $v0,1first, in$v0. (Scoperto the one block so$v0stays free as scratch elsewhere.) - B3-reuse-the-compare (a
return 0whose 0 already sits in a reg): aswitch(x){case…}range check issltiu $v0,x,N; beqz $v0,<dft>; on the out-of-range path$v0==0already equals the wantedreturn 0. Give gcc nothing else to do: NOdefault:and NO statement after the switch → thebeqzthreads straight tojr ra, reusing thesltiuresult (targetbeqz $v0,.epilogue). An explicitif(x>=N)return 0;, adefault: return 0;, or a trailingreturn 0;each forces a separatemove $v0,$zero(+1 insn / wrong branch target). Falling off the end of the non-void function is deliberate here and matches the original (gcc warns under-Wall; harmless). Cross-refs cookbook §3 (T4 branch-polarity / fall-through) — same family.
Method note (reinforces §3a + R16)
The permuter is the wrong tool when (a) its score can't see the residual (rodata/jtbl floor) or (b) the
residual is a specific compiler-internal placement rather than a randomizable C perturbation. For those,
web-research the exact pinned compiler source (§3a) to name the pass and its bail/tie condition, then
express the lever in C. Here a research agent reading reorg.c/jump.c/local-alloc.c produced all four
levers directly; hand-iteration with the clean .text metric closed it in a few compiles. Pin every such
construct with a LOAD-BEARING comment naming the pass — a future reader WILL try to "simplify" them.
§11 Cross-binary dedup & code-sharing (Phase 11 — "one match unlocks many")
BFM is overlay-heavy: 134 location overlays all load to the SAME vram 0x80128158 and run on the same engine,
so they share enormous amounts of code (a 770-instruction engine fn is byte-identical in all 134). Match a
shared fn ONCE, credit every binary it lives in. The pipeline (all Ghidra-free except the EXE/resident sigs):
1. Sign every binary → .run/sig.<bin>.jsonl. make sig-refresh (Ghidra, EXE/resident) + make sig-overlays (the 134 0.4.dec via tools/sig_image.py, no Ghidra). Each fn gets h_exact (SHA1 of raw
instruction bytes), h_norm (structural), h_seq, nins, calls.
2. Group across binaries → docs/duplicates.cross.md. tools/dup_report.py --cross (run by make report,
gated BINARY=main) buckets ALL sigs by h_exact then h_norm, splits cross-binary (members in >1 binary —
the Phase-12/13 work queue) vs intra-binary, ranks by collapsible bytes (count−1)×nins×4, top-200 capped.
3. Register a share → config/dedup.us.yaml. group → {id, tier, hash, source, func, members:[{binary, vram, name}]}. tools/dedup_integrate.py --check is the byte-honesty gate (fail-closed if a member's live
sig hash drifts from the recorded hash); wired into make report so a stale share fails the report (P9).
The mechanism: game-code dedup is SOURCE-LEVEL, not an object swap (R-D1, the key lesson)
psyq_integrate's stub-object swap works only for separate library subsegments. Game-code functions are
interior to one compiled object per binary (build/src/800.o, build/resident/resident.o, each overlay's
one object) — the linker can't excise interior bytes. So you share at the SOURCE level: author the matched body
ONCE as a macro in src/shared/<fn>.h and instantiate it at each member site in each binary's .c:
// src/shared/clearTbl40.h
#define CLEAR_TBL40(name) void name(void) { s32 i; for (i=0x40; i>=0; i-=0x10) (&D_80076251)[i]=0; }
// src/800.c: CLEAR_TBL40(func_80037004) ... CLEAR_TBL40(func_80037334)
Same bytes land at each vram. The byte-gate is the existing per-binary make check — the image is identical
or it is not. h_exact shares are risk-free; h_norm shares are CANDIDATES, accepted only if every claiming
binary stays byte-identical (a wrong h_norm group wastes a build, never poisons an image). A shared .h is
skipped by the find src -name '*.c' OBJS glob automatically (no exclusion needed). progress.py counts dedup
members as REAL via the registry (the macro form isn't a parseable function def).
sig_image.py (Ghidra-free signer) — notes for reuse on overlays
h_exactis the workhorse: SHA1 of raw bytes → format-independent → byte-matches the Ghidra dumper with no normalization. Validated 100% on the resident's contiguous/non-GTE functions. Use it as the cross-tool tier.h_normis self-consistent, NOT Ghidra-byte-exact (R-D2): masks j/jal targets, lui highs, hi/lo-paired address-los (a consistent lui→reg tracker); keeps registers / true constants / PC-relative branch offsets. Uniform within the overlay fleet (catches different-offset structural dups); does not cross-compare with the Ghidra-signed EXE/residenth_norm(low value — overlays call, don't embed, the resident). Full normToken byte-match is a deferred refinement.- Boundary detection: (a) seeded (pass
--seeds <sig.jsonl>when boundaries are known, e.g. the resident); (b)--bootstrapfor overlays = linear partition (split contiguous code at the firstjr $ra(+delay) that lies at/after all forward branch targets — handles early-return + double-epilogue) bounded bydetect_code_end(first run of ≥3 invalid instrs = the code→data transition; overlay code decodes ~100% valid). Call-graph BFS FAILS on overlays (they dispatch via function-pointer tables, notjal). Residual: jump-table-only fns + non-contiguous Ghidra bodies (D5) are missed — conservative, fixed when splat configs land (Phase 13).
Per-binary toolchain provenance (R24)
Verify the toolchain per binary before linking its library code: the EXE is PsyQ 4.0, the resident is 4.7
(tools/psyq/conv47/, sha-recorded in tools/psyq/CHECKSUMS.sha256). Never assume one binary's SDK applies to
another — the 4.0 libs won't byte-match the resident's 4.7 objects.
§12 Ultracode harvest — parallel-draft + byte-gate at scale (Phase 12, resident: 1.4%→71.7% in one session)
When a binary has many independent small/medium functions to hand-match (the resident: ~143 game-code fns, the overlays later), fan the drafting out to a swarm of agents and let an incorruptible byte-gate filter — a wrong match is structurally impossible to accept (G3/P9), so blind/semi-blind drafting is safe to mass-produce. This drove REAL 1→102/145 (71.7% byte-identical) on the resident in one Ultracode session (5 passes). Reusable verbatim for the Phase-13 overlays.
The loop (each pass = a Workflow + a deterministic gate; loop-until-dry):
- Draft (parallel, Workflow). N agents (round-robin a size-sorted fn list into ~13–16 batches), each reads
asm/<bin>/nonmatchings/.../<fn>.s+ this cookbook + the already-matched fns (the style/extern conventions) and writes ONE self-contained.cper fn to.run/drafts/<fn>.c(externs + body) + a.conf(high/med/low). No builds, no Ghidra inside the draft pass (the asm IS the target; Ghidra contention flakes under fan-out). Distinct files per fn → no write races (no worktree isolation needed). - Byte-gate (deterministic,
tools/harvest_verify.py). Substitutes each draft for itsINCLUDE_ASMstub,make build BINARY=<bin>, keeps it ONLY if the image stays byte-identical, else reverts to the stub. Chunk-with-bisection (apply K, build; if the SHA holds keep all, else isolate one-at-a-time). The build is the sole arbiter — agent over-claims cost nothing. (src/<bin>/*.cis git-committed → alwaysgit checkout-able.) - Loop (redraft passes). Re-run the Workflow on the residual stubs, each agent seeded by its prior failed draft(s) + a debugging checklist (the high-yield miss-modes — see below). Gate again. Resident yield per pass: +62, +8, +9, +2.
- Iterate pass (the strongest —
tools/match_one.py). Gives each agent a real per-function asm-differ loop: compile ONE fn's C standalone (the pinned triple), mask relocations (jal/HI16/LO16, exactlypsyq_identify's mask), compare to the target bytes in its.s→MATCHor a per-instructionidx | MINE | TARGETdiff. Fully isolated (own.run/match/<fn>/temp dir) → parallel-safe. Agentswrite C → run match_one → read diff → fixuntil MATCH. This cracks scheduling/regalloc near-misses blind drafting can't (resident +13 on the hard tail).
Workflow resilience: wrap the per-batch agent() in retry waves — a transient server 500/"rate limited"
returns null; re-run only the null batches up to 3× (pending/okResults pattern). One un-retried pass lost
10/14 agents to a server throttle; the retry-wave pass recovered all 13.
Under a SUSTAINED server-side throttle, throttle the FAN-OUT, not just retry it — process in SEQUENTIAL
WAVES of ≤10 agents. A big concurrent burst (40+ batches submitted at once) hammers the shared
Server is temporarily limiting requests (not your usage limit) rate limit, and even retry waves keep
failing because every wave re-bursts. Phase-15 T6 v3: a 48-batch burst harvest crawled at ~1 draft / 45 s
and finished with 33 dead batches; re-run as for (g of chunks(batches, 10)) await parallel(waveOf10)
(with intra-wave retry), the SAME 375 targets drafted at ~42 drafts / 60 s, 0 dead batches. Keeping
≤10 requests in flight stays under the per-window limit; sequential waves space the load so each window
resets between them. Pair with leaf-first / easy-first ordering so the early waves bank the high-yield
functions even if later waves get throttled. This is the rate-limit-gentle default for large harvests.
MANDATORY GAP-FILL after every multi-agent run you expect to be complete (the retry-wave is NOT enough).
Retry waves only re-run agents that returned null. But an "API Error: Connection closed mid-response. The response above may be incomplete." failure returns a truncated-but-non-null result — the workflow reports
dead_batches: 0 and a clean exit, yet that batch's agent silently wrote only SOME of its assigned drafts (or
none). The summary count looks fine; work is missing. So the gate alone would silently skip those functions
(a leftover stub is itself byte-identical — the byte-gate can't see an un-attempted target). Always
reconcile the produced artifacts against the expected work-list before gating (this generalizes to any
fan-out, not just harvests — diff produced-vs-expected whenever "expected complete"):
- Missing: every manifest target with no draft file on disk.
- Truncated/malformed: every present draft that is brace-imbalanced, has no function def, or doesn't close — a partial write. (These would fail the gate harmlessly, but re-drafting recovers them.)
- Re-draft the union with a focused gap-fill agent (same prompt, the gap names), THEN gate the full set. This is the byte-honest closure of "exhaustive" (P9/R14): the disk, not the workflow's success summary, is ground truth. Phase-15 T6 v2: workflow reported 585 drafted / 0 dead after the retry-wave re-ran 2 connection-closed batches, but a produced-vs-manifest diff still found 6 un-drafted targets (individual skips inside completed batches) — re-drafted before gating, none lost.
Two TU-level gotchas (both bit, both have a fix):
- Inline scalar-typedef redefinition. Agents told "self-contained" sometimes inline
typedef unsigned char u8;— in a.cthat already#includescommon.h, gcc-2.7.2 (C89) errors on the dup → a compile fail, NOT a byte miss.harvest_verify.py/match_one.pySTRIP^\s*typedef\b.*\b(u8|u16|…|f64)\s*;lines (common.h provides them). (Recovers false-failures: re-gate after the strip.) match_oneMATCH but whole-build FAIL = extern-type conflict.match_onecompiles standalone (one fn's externs); the real build is ONE TU (resident.c) where all fns coexist. A caller that declares a shared symbol to suit ITS call site (e.g.extern s32 func_800D1714(void);to drop anandi v0,0xffff) conflicts with that symbol's decl/def elsewhere (u16) →conflicting types for …→ gate fails. Resident pass-4: 34 standalone MATCH → 13 whole-build (21 conflict casualties on ~25 shared symbols). Fix: unify the extern types in the.c— usually widen the definition's return type where byte-identical (u16 f(){return u16g;}↔s32 f(){…}are the samelhu;jr), so all callers agree. NOT separate.cper fn: splat places the binary as ONE address-ordered object, so a second TU's.oisn't interleaved at the right vram.
Honest tail (P9). What survives the iterate pass is real compiler-internal residual — cross-jump tail-merge /
block-reorder / regalloc that no C-source shape steers (the agents document each in the draft header). Those go to
decomp-permuter (tools/permuter/) + the §3a/§5a/§10 research tier, or stay honest stubs — never forced.
Idioms the swarm surfaced (fold the asm patterns into §1/§2/§10):
- A "void-looking" dispatcher that ends in
jalrwith NO trailingmove v0,zerobut HASmove v0,zeroin its early-return delay slots is actuallys32-returning withreturn 0;early-exits andreturn fp(...)at the tail (the jalr'sv0is the return) — declaring itvoidmis-schedules the constant. - Local = global for a small fixed table: a
lw 0/4/8(base); sw …prologue copying a 3-word global into a stack array is a struct copy (Foo local = D_global;), not element-wise assignment (forced when a runtime index makes gcc materialise the whole table on the stack). *10(and small-const multiplies) decompose as(x<<1)+(x<<3), not((x<<2)+x)<<1; if a div/mul-by-const diffs by one shift/add arrangement, hand-write the explicitx*2 + x*8form.- Masked compare
andisurvives only if the value's range is unprovable.(a0 & 0xff) == kkeeps itsandi v1,a0,0xffonly when gcc can't provea0's range; a cleanu8load lets gcc-2.7.2 provea0∈[0,255]and DROP the andi (shifting the whole tail). An(s8)/(u8)cast on a wider load is the lever to restore-or-drop it. - Callee return type forces the cast at the call site:
jal f; andi v0,0xffffmeansfis declared returning a type WIDER than u16 (so the(u16)cast emits theandi); declaringfasu16lets gcc trust it and drop the andi.
§13 Add a location overlay — the canonical runbook (Phase 13; the Phase-15 fleet recipe)
All ~134 location overlays are flat LZSS-decompressed 0.4.dec payloads that stream into the SAME slot
vram 0x80128158 (position-locked, Phase 3) and chain into the resident engine. Each is its own build
binary ov_<SCxx>_<nnn>. The whole pipeline is one command + the §12 harvest; this is the reusable recipe.
One-command onboarding — tools/new_overlay.sh <SCxx> <FILE_nnn>
Computes sha1 / size / code_end (from sig_image.py --bootstrap, the last function's end), instantiates
config/splat.<ov>.yaml from config/splat.us.overlay.template.yaml, writes config/check.<ov>.sha + an empty
config/symbols.<ov>.txt, appends the <ov>_* block to the generated config/overlays.mk (the Makefile
-includes it, so the hand-maintained Makefile body is NEVER edited) + the alias to OVERLAY_BINARIES,
sentinel-inserts the entry into the 4 report/diff dicts (diff_settings.py, tools/{progress,difficulty, dup_report}.py — grep-guarded + ast.parse syntax-checked), then make extract && make build to byte-verify
at 100% INCLUDE_ASM. Idempotent (re-run = clean no-op). Proven on SC01/005, /006, SC03/001.
The flat-blob overlay config (what the template encodes)
- No header, no
gp_value(-G0), singlecodesegment @vram 0x80128158;build_path: build; per-binary nestedasm/<ov>+src/<ov>+build/<ov>;asset_path: assets/<ov>; stacked symbols[symbols.us.txt, symbols.resident.txt, symbols.<ov>.txt](overlays call the resident engine). - Overlays open with code at file 0x0 (a prologue), UNLIKE the resident's leading data word — first subseg
[0x0, c, <ov>], no leading-rodata trick. Subsegs:[0x0, c]+[<code_end>, data, tail];code_end= the last sig_image function's end (file offset). Byte-match is robust to the exact split (splat round-trips bytes).
THE NON-4-ALIGNED-OVERLAY GOTCHA (≈75% of the fleet; fixed in the template + Makefile, automatic)
A 0.4.dec whose size isn't a multiple of 4 (SC01/077 = 0xB29D7, mod 4 = 3) loses its final 1–3 bytes three ways:
- spimdisasm drops the trailing partial word (won't emit < 4 leftover bytes; a
datacarve of those emits nothing). → Carve them as abinsubseg[<word_floor>, bin, trailing](raw.incbin).new_overlay.shinjects this whensize % 4 != 0. - splat's
binasset needs a build rule —.ldreferencesbuild/assets/<ov>/trailing.o. The Makefilebuild/assets/%.o: assets/%.binrule assembles a one-line.incbinstub +objcopy --set-section-alignment .data=1(elseasdefaults.datato 16-align → ld pads the image, adds a stray byte). - splat's
.lddoes. = ALIGN(., 4)at the segment end → up to 3 zero pad bytes. The Makefile objcopy step TRIMs it: shrink-only, capped at 3 bytes, gated onsize(build) > size($(EXE)) && delta ≤ 3— can never hide a shortfall or touch the 4-aligned EXE/resident (which never hit any of this).
The A→B→C per-overlay workflow
- A. Onboard + all-asm byte-match (
new_overlay.sh) →make check BINARY=<ov>byte-identical. The milestone bar (splat round-trips bytes regardless of carve quality). - B. Seed boundaries + Ghidra import (to draft harder fns): splat carves ~all functions itself (SC01/077:
2504 cleanly), so seeding
symbols.<ov>.txtonly fixes conservative carves (§8/§11) + attaches names — the byte-match doesn't depend on it. Ghidra:ghidra_import_raw.sh <0.4.dec> 0x80128158 <ov>(MCP stopped, R23) +DefineFunctions.javaover splat's func list (auto-analysis finds only thejal-reachable subset — overlays dispatch via fn-pointer tables; SC01/077: 1237 → 2661). The overlay Ghidra DB is script-reproducible → DB-commit optional (skip to avoid ~14 MB bloat unless doing manual RE). Restarting MCP to serve<ov>drops the client SSE → pause + ask Drew to run/mcp(memorymcp-reconnect-after-restart). - C. Dedup-credit FIRST, then the §12 harvest. Credit the high-leverage shared engine functions (top
docs/duplicates.cross.mdgroups — byte-identical at fixed vrams in all 134 overlays) so they leave the queue; thendifficulty.py --binary <ov>ranks the unique remainder for the §12 parallel-draft + byte-gate. Use Ultracode (T7 evidence: xHigh agents = Max-agent yield on blind drafting — no reason to spend Max depth; seedocs/effort-map.md). SC01/077: 704 matched (yield 92%→79%→54% as difficulty rose; the hard tail → permuter/§3a or honest stubs).
Dedup-credit (§11) for overlays — two shapes
- Per-function share (general/fleet shape): a matched body lives once as a macro in
src/shared/<h>.h, instantiated in place (preserve address order — overlays link functions in source order!) at each member site, registered inconfig/dedup.us.yaml(members in >1 binary), byte-gated bydedup_integrate --check(the recordedh_exactmust equal each member'ssig_imagehash, keyed by addr-int — sig_image names lowercase, splat uppercase, validator case-moot). The nested overlay.cresolves quoted includes relative to ITS dir → use#include "../shared/<h>.h". Proven: SC01/005 ≡ 006 share 3 accessors fromsrc/shared/ov_setters.h(SETTER/RETCONST), both byte-identical from one source.progress.pycredits shared members as REAL. - Whole-overlay collapse (special case for byte-identical pairs): for two overlays with the SAME
0.4.decsha1,src/ov_B/ov_B.ccan#include "../ov_A/ov_A.c"— B inherits ALL of A's matches from one source. Maximal but doesn't generalize to partial sharing, so the per-function share is the fleet default.
Fleet build — make build-all / make check-all
Recursive $(MAKE) BINARY=<b> over $(BINARIES) (NOT foreach — the OBJS glob is parse-time per $(BINARY)),
one PASS/FAIL + per-binary .run/check.<b>.log. Serial across binaries (shared build/asm|src/** outputs make
binary-level -j racy). Day-to-day incremental; milestone fleet proof = a CLEAN run (R22): make clean && for b in $(BINARIES); do make extract BINARY=$$b; done && make check-all.
§14 Propagate a matched function across the fleet — tools/dedup_propagate.py (Phase 15)
The economics that drives Phase 15. Overlays are position-locked at 0x80128158, so a shared engine
function has the SAME vaddr (hence the SAME func_<ADDR> symbol) and a BYTE-IDENTICAL body in every overlay
that contains it. Measured on the fleet: 577 of ov_SC01_077's 785 matched functions are h_exact-identical
across ALL 134 overlays (~2.19 MB collapsible) — already matched, just needing propagation. So the rule is
match once → propagate, do NOT re-harvest each overlay. (tools/dedup_propagate.py --auto-from ov_SC01_077
enumerates exactly this set: matched-as-an-inline-def in the source AND h_exact-shared across ≥--min-reach.)
The tool. tools/dedup_propagate.py --addr 0x..[,..] --source-overlay <ov> (or --auto-from <ov> for the
whole shared set; --check-only for a dry-run plan):
- extract the matched body (preceding
externs + the def, brace-matched) from the source overlay's.c; - author it ONCE as a tool-generated
#define DEFINE_func_<ADDR>() \-continued macro insrc/shared/engine_core.h(idempotent; refuses//line comments — they break line-splicing; block/* */is fine); - at every onboarded overlay whose sig shows that
h_exact(lead withh_exact— guaranteed byte-identity), replace that function'sINCLUDE_ASMstub (or, in the source overlay, its inline def) in place withDEFINE_func_<ADDR>()— address order preserved;#include "../shared/engine_core.h"added once aftercommon.h; - byte-gate each touched overlay (
make build BINARY=<ov>== itscheck.sha); on ANY miss, restore EVERY file from an in-memory snapshot and abort (fail-closed; nothing wrong lands); - register the group in
config/dedup.us.yaml, validated bydedup_integrate --check.
Key gotchas (each cost a real bug or false pass during the Phase-15 proof):
- Key by addr-int, never the string.
sig_imagewrites lowercase hex (0x80144b9c); splat's symbol is uppercase (func_80144B9C). Compareint(addr,16); render the symbol asfunc_%08X. - Accumulate edits from the on-disk text, not a snapshot cache. When a batch propagates several functions into the same overlay, re-read the file before each edit (the snapshot dict holds the ORIGINAL for restore, not the running state) — else each function's edit clobbers the previous and only the last lands.
- The byte-gate can't catch under-application. A leftover
INCLUDE_ASMstub is itself byte-identical (it just uses the asm), somake checkpasses even if a function wasn't actually converted. Add a STRUCTURAL self-check: after editing, assert each member's.ccontainsDEFINE_func_<ADDR>()and NO leftover stub line. - Header-dependency tracking is mandatory once shared headers are build inputs (R22). The Makefile C rule
originally made
build/src/%.odepend only on the.c, so editingengine_core.h/common.hdid NOT trigger a recompile → an incrementalmake checkafter a header-only edit was STALE (a wrong shared body falsely passed). Fixed: thecppstage now emits a.d(-MMD -MP -MT $@ -MF $(@:.o=.d)) and the Makefile-includes$(C_SRCS:%.c=build/%.d). Side-effect only — output bytes unchanged. With it, the negative test (corrupt a macro body →make checkrebuilds via the.d→ SHA mismatch → fail) behaves correctly. - Idempotent + resumable. A propagated source function becomes a
DEFINE_…macro (no longer an inline def), so--auto-fromre-runs skip the done ones;--addrre-runs no-op (source is now a macro, plan is empty). - Scale note (deferred until it bites):
dedup.us.yamlmembers are listed verbose ({binary,vram,name}). For the full 134-overlay × hundreds-of-functions bulk, switch to avram + binaries:[...]shorthand (expanded bydedup_integrate/progress.py) before it becomes a 5-figure-line file.
§14a The fleet bulk run (Phase 15 — --auto-from, 947 REAL → 74,527; 3.82% → 22.14% in one pass)
A measured 577 of ov_SC01_077's matches are h_exact across all 134 overlays; dedup_propagate --auto-from ov_SC01_077 propagated 553 of them fleet-wide (each overlay byte-gated) in ~2 min. The 5 things that made
the bulk work (each cost a real failure first):
- Only self-contained bodies are mechanically liftable. A 077 match whose body names an overlay-LOCAL struct
type (
SrcB964 *a0— a harvest-invented type defined in 077.c, not common.h) compiles in 077 but FAILS in every other overlay (parse error before '*',a0 undeclared). The first bulk attempt died on one and reverted all 134 (fail-closed, correct but wasteful). Fix: a compile pre-filter —compiles_standalone(body)builds the body withcommon.honly (cpp→cc1); skip if it fails. 9/562 were local-typed → skipped honestly (P9). These need their types shared too (a future enhancement); they are NOT a byte regression, just deferred. - The registry shorthand is mandatory at this scale. 553 groups × ~134 members verbose ≈ 77k lines; the
vram + binaries:[...]shorthand keepsdedup.us.yamlat ~4k.group_members()(indedup_integrate) is the single expander used bydedup_integrate --checkandprogress.py. - Per-overlay apply, not per-(function,overlay). Group targets by overlay → one read/write per file; stub lines replace 1:1 (no shift); the source overlay's inline defs splice by range in REVERSE line order. (Naive per-pair editing is 77k file-ops and minutes slower.)
- Skip already-registered + gate-only-changed.
--auto-fromexcludes addrs already in the registry (additive, resumable, never collides with an existing SETTER/engine_core share); only overlays that actually changed are rebuilt. Re-running tops up as more overlays onboard. progress.py --fleetparse cache.dedup_membersparsed the (now-large) registry once per binary (136×) — cache it once (_DEDUP_CACHE): fleet report 6m+ → ~7s.
§14b Harvesting the UNMATCHED shared core — the match_one wall (Phase 15)
After the bulk (already-matched) propagation, the remaining shared functions are the hard residual the
original per-overlay harvest already failed on. A 25-agent Ultracode pass (§12) drafted + match_one-verified
300 of the smallest unmatched-shared functions (8–16 ins); the whole-binary byte-gate (harvest_verify in
ov_SC01_077) verified only 49 (16%) despite 292 agent-claimed "high". The gap is structural, not agent error:
match_onemasks relocations (jal 26-bit, HI16/LO16), so it CANNOT verify call/data targets. A draft that calls the wrong function — or the right function with the wrong extern signature — still printsMATCH. It is a true gate only for leaf functions (no calls, no global refs). The 49 that passed were essentially the leaves.- The real misses are the extern-type-conflict wall (§12's gotcha, now the dominant failure): a wrapper
func_Acallsfunc_B; the draft declaresextern void func_B(s32), butfunc_Bis already defined IN THE SAME TU (anengine_core.hmacro from the bulk, or an inline def) with a different signature →conflicting types→ compile fail → the byte-gate reverts it.match_one(standalone, no other defs) never sees the conflict. - Implications for future passes (the open-ended tail): (1) leaf-first — filter targets to functions whose
.shas nojal/%hi/%lofor a high-yield pass; (2) callee-signature-aware drafting — when a draft callsfunc_Xthat's already C-defined (grepengine_core.h/ the overlay.c), it must reusefunc_X's EXACT signature; a STUB callee (stillINCLUDE_ASM) takes any consistent extern (asm provides the symbol, no conflict); (3) a real per-function gate would need to link (resolve relocations), i.e. the whole-binaryharvest_verify, not the maskedmatch_one; (4) the call-heavy residual is genuine decomp-permuter / hand-iteration work — the open-ended Phase 15 continuation, not a milestone gate.
§14c Callee-signature-aware harvest — breaking the extern-type-conflict wall (Phase 15, T6)
§14b named the wall; this is how it falls. The call-heavy shared residual fails the whole-binary
byte-gate (NOT match_one, which compiles standalone and masks jal) because a draft declares a
callee or data symbol with a type that conflicts with that symbol's canonical declaration
elsewhere in the single overlay TU (an engine_core.h DEFINE macro, another banked function's
inline extern). The fix is to stop letting agents guess: pre-resolve every callee's EXACT signature
deterministically and hand it to the agent to reuse verbatim.
tools/gen_harvest_targets.py builds the callee-sig-aware target manifest (.run/t6_*.json).
For each still-INCLUDE_ASM shared function it records {name, addr, nins, reach, callees:[{sym, status, signature}]}, resolving each callee three ways (priority order):
- defined — a body exists:
engine_core.hDEFINE_func_Xmacro body, or an inline def in the overlay.c. The signature is authoritative; the draft MUST reuse it verbatim. - declared — no body, but the symbol is already
extern-declared somewhere (another macro/draft). That extern is authoritative too (a stub callee with an established signature — reuse it). - stub / extern — undeclared (a bare overlay stub, or a resident/EXE symbol). The agent infers a
minimal consistent extern from the
.sarg-setup; resident/EXE callees are conflict-free (not defined in this TU).
Result on the small band (nins 8-30, 616 fns): 855/1019 callees resolved to exact sigs, and the
whole-binary gate yield jumped 16% → 60-67%. The agents draft callee-sig-aware C (reuse exact sigs
for defined/declared callees; infer for stubs), self-check with match_one, and the whole-binary
harvest_verify is the sole arbiter (match_one cannot see TU-level conflicts).
The four conflict flavors (all real this phase):
- (a) defined-callee return/param mismatch — draft guessed
s32/wrong params; canonical body says otherwise. (func_8012A100isvoid f(s8), not(s32).) → reuse the resolved sig. - (b) declared-stub-callee — a still-
INCLUDE_ASMcallee already extern-declaredint func_X(int, int,int)by other macros; a draft declaring itvoidconflicts. (func_80150BA4→func_80151184.) → harvest the extern declarations too, reuse them. - (c) DATA-symbol type conflict — the dominant residual after (a)/(b):
D_XXXXglobals declared with different types by different drafts (u8 D[]array vss32 Dscalar) →conflicting types. The manifest resolves FUNCTION sigs but not yet DATA-symbol types; this is the next yield-limiter. Structural fix for a future pass: a single canonical decls header (all data symbols asextern u8 D_X[];byte-arrays, used via explicit casts) that the overlay.cincludes and drafts never redeclare → zero possible conflict. - (d) narrow-return-widening — a callee defined returning a NARROW type (
s8/s16/u8/u16, bodyreturn <load>) forces asll/sra(orandi) sign/zero-extension at EVERY call site; if the target asm lacks it, the original declared that functionint/s32. Widen the DEFINITION's return type tos32/u32— byte-identical because the body'slb/lbu/lh/lhualready extends to 32 bits — and all call sites match. (func_8017AE08→func_80174764.)
Operational gotcha (cost real work): NEVER git checkout src/<overlay>.c during an active harvest.
It silently reverts banked matches while the propagation artifacts (engine_core.h macros, other
overlays' DEFINE_func_X() instantiations, dedup.us.yaml groups) survive — an inconsistent (though
byte-recoverable: the next gate re-verifies) state. Use a .run/_bak.c copy for diagnostic
substitute/build/revert, never git checkout.
Resilient incremental loop (proven under server-side API rate limiting): the draft Workflow is the
only rate-limited part. Gate whatever drafts have landed (harvest_verify), dedup_propagate --auto-from (deterministic, immune to throttling), make report, repeat as more land. Quarantine
gate-failures to .run/drafts-<x>-fail/ after each round so re-gates stay fast. Partial harvests bank
cleanly; the residual resumes next pass (loop-until-dry). T6 banked 123 fns / +4.6% fleet this way with
the Workflow only ~40% through its batches.
§14d Deterministic recovery beats agent waves on the hard tail (Phase 15, the final-session finding)
Once the EASY shared core is banked, the residual is permuter-class and agent harvesting is poor ROI: a
50-agent Ultracode wave on 300 hard-tail stubs verified only 27 (~9%) for ~4.1M tokens (+0.36% fleet), while
a deterministic recovery pass added +2.67% for ~0 agent tokens the same session. Reach for these BEFORE
mass-drafting the hard tail (this is a cost rule, codified in docs/effort-map.md):
tools/sig_unify.py— unify the draft's OWN definition signature, not just callee externs. The dominant hard-tail gate failure is a TU-level signature conflict on a draft whose body is already byte-correct (it passesmatch_one). A probe is the tell: substitute each gate-failing standalone-MATCH draft alone, build, and classify the error — 30/30 sampled wereconflicting types, 0 false-positives (built-but-SHA-diff) → they are byte-correct C blocked only by declaration unification.canon_draft_declsrewrites callee externs but never the function's own def signature (definedvoid/s16here, extern-declareds32by banked callers) —sig_unifyadds that (canonical return + param types, draft's param names kept; arity-mismatch → return-only fallback). The byte-gate stays the arbiter. 191 conflict-blocked → 32 recovered deterministically.- The silent-under-propagation class (
find_sitebrace bug). A GREEN byte-gate proves what landed is correct; it does NOT prove everything that should have propagated did.dedup_propagate's inline-def detector required the opening brace on the same line as the signature, so every next-line-brace def (whichsig_unify/permuter emit) was silently skipped from propagation — a whole session's matches capped invisibly. Always sanity-check the OUTCOME metric (make reportfleet %, or--auto-from --check-onlyplan size), not just the gate. Fix:find_siteaccepts brace on the same OR the next non-blank line. (One fix unlocked a 61-function backlog.) Generalizes: any "match→register" detector must accept all draft formats. build_engine_types.pymust be ADDITIVE. A 2nd--striprun regenerating the shared types header from the source's current inline defs DROPS the already-migrated types (they were stripped last run) → build breaks. Merge with the existing header; the header is the cumulative record.- Probe-before-investing. Size every recovery lever on a ~20–30 sample (build-classify the failures) before building the full pass — it told us sig_unify was worth it (30/30 conflicts) AND that the next tier was a dead-end (below) before we spent on it.
§14e Two hard-tail dead-ends (Phase 15 — documented so they aren't re-attempted)
()no-prototype externs DON'T resolve arity/param conflicts here. The textbook PS1-decomp escape (declare inter-function externs param-less) FAILS under gcc-2.7.2: "An argument type that has a default promotion can't match an empty parameter name list declaration" — C forbids()matching a prototype with a default-promotion param (s8/s16/u8/u16/float), which these engine fns have. The remaining ~159 arity conflicts (def needs N params, callers declare M) have no clean deterministic fix.- m2c/Ghidra output is a scaffold, NOT byte-matching C — the struct is the wall. On the remaining shared band,
20/20 sampled m2c outputs are struct-heavy (
arg0->unkXXXinferred field accesses) and won't compile without the engine struct defined, plus?-typed values. A decompiler gives structure + offsets for free; it does not give the struct definition or the byte-match. The path forward (Phase 16): infer the one engine actor struct layout from the union of m2c field-accesses (offsets + widths) → feed as m2c--context→ compilable C →sig_unify→ decomp-permuter brute-force (compute-bound, low-token — the "set-it-and-go" pipeline). Validate on a 10-fn medium sample before scaling.
§15 Struct-heavy shared-core pipeline (Phase 16) — empirical determinations (S0)
Established by running m2c on real ov_SC01_077 stubs (R14) — it corrects §14e's framing on two key points and the corrections are favorable.
(1) m2c output COMPILES via tools/m2c/m2c_macros.h — the struct is NOT a hard compile prerequisite.
--valid-syntax emits M2C_FIELD(p, type, off) ≡ *(type)((s8 *)p + off) (defined in m2c_macros.h) — a byte-faithful cast (same lw/sw/lh/sh as p->field). §14e's "won't compile without the struct" was without m2c_macros.h. 30/30 sampled m2c-targets use only byte-faithful macros (M2C_FIELD/M2C_BITWISE/M2C_UNK) → they compile. The non-faithful macros — M2C_ERROR/M2C_BREAK/MULT_HI/MULTU_HI/CLZ/GLUE_F64/BSWAP/M2C_TRAP — emit (0) (discard the real op) → a function using any of them cannot byte-match (GTE/handwritten/special). Their presence = "defer, not m2c-matchable."
(2) Compiling ≠ byte-matching; the residual is regalloc/scheduling → decomp-permuter is the byte-closer (essential, not optional). Pure-leaf example func_8012CB64: macro-compiled output = 16/16 ins, identical control flow, only v0↔v1 regalloc + a trailing move/nop. That is permuter-class. Some functions m2c gets structurally wrong (e.g. 8 vs 18 ins) → permuter cannot fix → struct types / hand / defer.
The byte-match path: m2c --valid-syntax (+ macros + context) → sig_unify → decomp-permuter (regalloc/schedule) → harvest_verify whole-binary byte-gate → dedup_propagate. Struct types (struct_infer, S1) are an ENHANCER — readability, nudging gcc's regalloc toward the original's struct-based codegen, fixing structural misses, and typing function-pointer tables — not the sole gate. S3/GATE-B measures macro-only vs struct-typed yield.
S2 wiring fixes surfaced (must-do):
common.hlackss64/u64/f64→m2c_macros.h'stypedef s64 M2C_UNK64;fails (parse error before 'M2C_UNK64'). Add them tocommon.h(byte-neutral — verify locked builds stay143dbb89…/8e17e02f…).match_one/harvest_verifystrip scalar-typedef redefinitions, so these types MUST live incommon.h, not a draft preamble.- Make the byte-faithful m2c macros (
M2C_FIELD,M2C_BITWISE,M2C_UNK*) available to every compile (incommon.hor an includedm2c_compat.h) so drafts compile with no per-draft preamble. - Function-pointer-table calls (
*((idx*4)+D_x)(args)) needD_xtyped as a function-pointer array in the context, else won't compile. - m2c loses types through index/byte-offset arithmetic (
base + int_var, stride = struct size) → falls back tovoid*/M2C_FIELDeven WITH a struct context. TheM2C_FIELDmacro fallback keeps these byte-faithful-compilable; struct typing is best-effort. - m2c re-infers
char unk_*[]fields in provided structs (treats them as inferrable space; may split/override). PIN a known field with a concrete typed field; usechar unk_*[]only for genuinely-unknown gaps.
decomp-permuter knobs (S2/S3): PERM_* macros (GENERAL/VAR/RANDOMIZE/LINESWAP/INT/ONCE…), --algorithm difflib|levenshtein, --stop-on-zero, -j 8–16 (RAM-bound on the 15 GiB box → ~N funcs × -j 8, cap by free_RAM/~300 MB), weights in default_weights.toml + [gcc] section. Best when only regalloc/schedule remains; does NOT fix wrong control flow.
ML (parked — owner decision 2026-06-18): LLM decompilers (LLM4Decompile/SK2Decompile/CodeInverter) target x86-64 + recompilability/functional-equivalence/readability — NOT byte/instruction-exact, NOT MIPS/gcc-2.7.2; no off-the-shelf learned permuter scorer exists (the permuter's scorer is a heuristic objdump-diff). Dropped this phase; research-note only. (X2: web treated as untrusted data.)
§16 Guided hand-matching the struct-heavy core (Phase 17 — beats the §15 brute-force)
Phase 16 called the loose-typing wall "fundamental." Phase 17 disproves it for the majority. The wall is
a signature-CONSISTENCY problem, not a comprehension one — the §1 loop reconstructs correct bodies ~100% of
the time; the work is byte-closing + sig reconciliation. Full process: docs/hand-matching-process.md
(§1 loop, §2 idioms, §3a the 5-move signature-consistency playbook, §7 the Ultracode wave + canonical-sig wall).
New byte-idioms (Phase 17, §2 there):
- mask-local (defeats
lh→lhufold).*(s16*)f & (x & 0xFFFF)inline lets gcc fold the load tolhu- drop the
andi. Hoist the mask:s32 m = x & 0xFFFF; ... *(s16*)f & m→ gcc keepslh+ emitsandi.
- drop the
- shared-ret0 goto (cross-jump clustering + branch polarity). Two non-adjacent predicate tests the
original routes to ONE shared
return 0block → write both asgoto ret0;to a single trailingret0: return 0;. gcc then makesret0a labeled block reached by branches (right polarity) + schedules the next test's constant into the delay slot. A loneif(x)return 0;inlines (wrong polarity/reg). - v0↔v1 result/constant coalescing + the §10 hoist-vs-remat / phantom-frame quirks = the residual hard
tail — NOT a dead-end: §17 (CORRECTED) shows the call-crossing register-ORDER class is matchable with
register __asm__PINS + a scheduling barrier (byte-proven, func_8012B8E4), and array-decay cracks the hoist-vs-remat class. Hand-tier, but matchable. (Permuter can't help — it rejectsregister __asm__.)
Scaling = Ultracode wave (§12 pattern + §7): Ghidra pre-pass (DecompileFunctions.java, headless batch,
no /mcp) → parallel draft agents (m2c+Ghidra-C+asm+actor-struct+§3a, self-validate match_one) → whole-binary
gate (harvest_verify --chunk 1) → sig_unify recover → dedup_propagate --auto-from. Calibration (top-30):
60% match_one MATCH, 33% whole-binary (+0.47% fleet), 136/136.
THE CANONICAL-SIG LAYER (built Phase-17 session-4; NOT the ~2× lever it first looked like). The
match_one→whole-binary gap on the calibration's top-30 was SIG CONFLICTS (parallel agents declare shared
callees inconsistently → conflicting types in the one-big-TU; 100% compile-errors, 0 codegen). Fix = a
SURGICAL per-callee canonical-sig layer: tools/census_conflict_callees.py (the conflict predicate:
undeclared-stub callee with decl_sources = n_callers + is_target >= 2) + tools/derive_canonical_sigs.py
(byte-neutral s32 func_X(s32...), arity from Ghidra-C + asm read-before-write $a0-$a3) → a 20-extern
block at the TOP of ov_SC01_077.c (LOCAL, not engine_core.h — reach-1 names differ across overlays). (Both census_conflict_callees.py and derive_canonical_sigs.py were DELETED in Phase 26-A — R33; the fleet-canonical-sig approach was superseded by reconcile_tu's per-TU oracle. Historical record.)
gen_harvest_targets + sig_unify auto-read it; the gate pipeline is now draft → sig_unify (MANDATORY)
→ harvest_verify --chunk 1 (the accumulating baseline now carries the file-top block, so a raw draft's
guessed extern would clash without sig_unify). SIZING CORRECTION (R14): for the remaining 270, the
conflict wall is only 20 callees / 24 targets / 7% of wave reach — the "~2×" was the top-30's in-flight
conflicts, since resolved by banking. The real wall is the gcc-quirk tail, not sig conflicts — the 4
highest-reach circular targets are ALL §10-hoist / regalloc / layout-bound (0 closed by hand or permuter).
The layer makes a wave sig-clean; it does NOT unlock the quirk tail. → The match-% lever is understanding
gcc-2.7.2 (R17 compiler-source research, Phase 18), not more brute waves. Wave deferred; infra staged
(.run/harvest_wave_s4.js, 40 tractable reach-134 targets). See docs/hand-matching-process.md §8.
§17 (CORRECTED) answers it: the call-crossing register-ORDER class IS matchable — with register __asm__
pins + a scheduling barrier (byte-proven). The "brute waves won't help" point stands; HAND levers (pins) do.
§17 The compiler-quirk wall — the matching TOOLKIT (Phase 18; gcc-2.7.2 source + byte-gated)
Phase 18 read the real gcc-2.7.2 source (tools/reference/gcc-papermario, SETUP §5.6) + mined Xenogears (our
EXACT compiler). Verdict (CORRECTED — an earlier draft of this section wrongly called the register-order
class "unsteerable"; it is NOT): every quirk class met so far is matchable from C — the register-order tail
needs register __asm__ PINS, which I'd skipped. The wall was a missing lever, not an impossibility. Triage
each residual with match_one (the floor-free oracle — NEVER the permuter score on jtbl/rodata fns, §10), then
pick the tool. R14 caveat: don't conclude "unsteerable" until you've tried the PINS.
Register-allocation ORDER (call-crossing $s0/$s1 swap) → FORCE it with register pins (byte-proven)
Mechanism (why the swap happens): two pseudos live across a call → both are global allocnos (NOT
local-alloc) competing for callee-saved $s0/$s1; global.c:allocno_compare sorts by density
floor_log2(n_refs)*n_refs/live_length so the short-lived value wins $s0 and the whole-function value gets $s1
— often the reverse of the original. Clean-C reshapes / flags / cc1-swaps do NOT flip it (all tested).
The lever (DON'T skip this): pin each call-crossing value to the register the TARGET uses —
register s32 d __asm__("$16"); /* $s0 */ register s32 s1ang __asm__("$17"); /* $s1 */
gcc honors the pin and forces the allocation. Read the target .s, map each call-surviving value → its
callee-saved reg ($s0=$16, $s1=$17, $s2=$18 …), pin it. Then the residue is usually small: fix it with the
branch-polarity invert (§3-T4), explicit temps for any reassociation (t = u6+0x1000; iVar4 = u5-t;),
and a scheduling barrier for a last stuck instruction (__asm__ __volatile__("" : : "r"(u5)); emits zero
code, anchors u5 ahead of the next op). WORKED EXAMPLE — func_8012B8E4 (the flagship "unsteerable" fn):
21 → MATCH via pins + branch-polarity (24→21) + clamp temps (7→3) + the u5 barrier (3→MATCH); byte-gated +
propagated ×134. NOTE the permuter can't help here — pycparser rejects register __asm__/__asm__ (§5a), so
this residue is a HAND lever, not a permuter job. Labor: ~5-10 min/fn, but each circular fn is reach-134 → ×134.
(Xenogears ships this class as INCLUDE_ASM only because they hadn't found the pin lever — not because it's
impossible. We did.)
The STEERABLE idioms (byte-confirmed this phase)
- array-decay forces rematerialization (NEW). A stack buffer passed to a callee as
&struct/mtx.w/*(T*)arr(any address-taken form) is HOISTED into a callee-saved reg (needs an extra callee-saved → bigger frame, more spills). Declare it a local arrayT buf[N]and pass it asbuf(array-decay, never address-taken) → gcc rematerializesaddiu $reg,$sp,offper call instead (matches the original, frees the reg). func_8012B4B8: 88→52 (the hard regalloc+remat half fixed). CAVEAT: an array can't take a struct block-copy (arr = STRUCTneeds a struct; element-copy constant-folds each global addr to its ownlui, +ins), so a fn that ALSO needs a load-base-once struct-copy has an unavoidable tension. - for-loop vs do-while controls delay-slot scheduling. A counted scan as a
for(init/cond/update) lets gcc schedule the branch-taken return value into the loop test's delay slot; ado-whilewith increments in the body fills that slot with an increment instead (+1 ins, wrong schedule). func_801399A8: do-while 7 mismatch → for-loop 2 → MATCH. - statement order in the for-update = instruction order (§2-T2 extended). Independent updates in
for(...; ...; A, B)emit in source order; swap to match. (func_801399A8 final 2.) - (existing, reconfirmed) §3-T4 branch-polarity invert (func_8012B8E4 24→21), §16 mask-local, §16 shared-ret0 goto.
The pipeline gotcha — match_one ≠ the gate; sig_unify is MANDATORY
match_one masks relocations → it MATCHES even when the draft's own def-signature or a data-extern TYPE
conflicts with the canonical decl in engine_core.h (u8 *func(void) vs canonical s32 func(void);
extern u8 D_x vs extern s32 D_x). The whole-binary gate then fails conflicting types. ALWAYS retype the
draft to the canonical set (return + data-extern types; use integer address arithmetic (s32)&sym,
codegen-neutral): draft → sig_unify/canonical-retype → harvest_verify.
The LOOSE-TYPING wall is real for narrow params (Phase 16, reconfirmed)
Some STRUCTURAL_MISS fns are blocked by it: func_80146A6C needs an incoming arg as lhu (s16), but the shared
canonical sig declares it s32 (→ lw); the byte-match needs s16, another call site needs s32, no single C
type satisfies both. No clean fix (the documented narrow-param dead-end). Stub it. (Reconciles §16's
"Phase 17 disproves the loose-typing wall": disproven for pure-structure fns like func_801399A8; REAL for
narrow-param fns like func_80146A6C and inseparable from the regalloc tail.)
Strategic conclusion — the match-% lever post-research
The high-reach circular regalloc-order tail IS matchable — by register pins + barriers (above), labor-
intensive but ×134 per match. So the lever is BOTH: (a) the non-walled STRUCTURAL_MISS fns (clean
reconstruction + the structural idioms + mandatory sig_unify — proven on func_801399A8), the cheap bulk; and
(b) the circular tail hand-matched with pins (proven on func_8012B8E4), the high-value-per-fn work. Triage
each with match_one: pure structure → reconstruct; stack-buffer-to-callee → array-decay; call-crossing
register swap → PINS; last-instruction schedule → barrier. The ONLY genuine dead-end left is the narrow-param
loose-typing conflict (func_80146A6C: an arg that must be s16 here and s32 at another call site — no single C
type) → stub THAT and move on; everything else is matchable with enough hand effort.
§17a The TOOLKIT at WAVE scale (Phase-18 Step-3b/Step-1 — measured) + the pipeline-integration gotchas
The §17 toolkit was taught to a parallel Ultracode harvest wave (agent prompt = the triage above + the two worked-example templates). Measured on ov_SC01_077 tractable reach-134 residuals:
- Step-3b calibration (16 targets, prompt v1):
match_one12/16, whole-binary 9/16 = 56% close-rate (vs the Phase-17 prompt's 33%). The pins/array-decay moves landed real matches the old "stub the quirk" prompt would have lost. - Step-1 (31 targets, prompt v2 = +embedded canonical callee sigs +call-site-cast/re-validate):
match_one28/31 = 90%, whole-binary 22 verified. The big lift was embedding each callee's canonical signature per target (fromgen_harvest_targets.py) so agents declare callees right instead of guessing.
THE match_one→gate GAP is mostly DECLARATION plumbing, not codegen (so it's cheap to recover):
-
Call-site casts, NOT redeclaration (the #1 recurring miss).
match_onemasks jal/%hi/%lo, so a draft that declares a callee with the WRONG arity/return still "MATCHES" — thensig_unifyrewrites that extern to the SHARED canonical sig (fewer args /void), and the whole-binary gate fails (too many arguments/void value not ignored). FIX = keep the canonical extern, cast at the call site (codegen-neutral):((void(*)(s32,s32,s32))func_X)(a,b,c)for an over-arity call;x = ((s32(*)(s32,s32))func_X)(a,b)when avoid-canonical callee's$v0is used;(u16)/(s16)at the use site per the asmlhu/lh. Then RE-RUNmatch_oneon the canonical-typed draft — applying canonical sigs can change codegen, so gate the draft that still MATCHes WITH the canonical decls. (sig_unify can also, rarely, regress a match by forcing a canonical that's wrong for the byte-match — e.g.void/s32over a neededs32/u32def-sig; that's the narrow-param wall on the def itself → stub.) -
Stale sibling forward-decl (
M2C_UNK func_X();). A function you're matching is sometimes forward-declared by an already-matched SIBLING in the same overlay.c(m2c scaffolding:M2C_UNK func_X(); /* extern */), which conflicts with your real def. FIX = reconcile that one line to the real sig (void func_X(void);). (Proven: func_8012A418.) A standalonematch_oneMATCH that fails the whole-binary gate withconflicting types for func_X / previous declarationis this class — grep the overlay.cforfunc_X. -
Implicit-int caller plumbing→ CORRECTED (Phase 19 / T2, R14): two distinct real classes, NOT implicit-int, and propagate-first does NOT fix them. The 3 "deferred" fns (func_80147514, func_80168F40, func_8017209C) were reproduced through the gate to find the ACTUAL blocker (R14 — verify the framing against the bytes). Neither is implicit-int;dedup_propagate(def →engine_core.h) would NOT have resolved either (the def still lands at its address-order site, after any file-scope caller extern). The two classes:- (3a) Resident-callee LINK-miss (func_8017209C). The draft calls a resident EXE/engine function by
func_<ADDR>, but that address carries a curated name inconfig/symbols.us.txt(e.g.0x8004CFEC = ratan2). The linker resolves the curated name, notfunc_<ADDR>→ the draft compiles but fails to link (undefined reference to func_8004CFEC) → the gate (correctly) reverts. FIX =tools/canon_resident_calls.pyrewrites everyfunc_<ADDR>whose address has a curated// funcname to that name (extern + call), a pure draft-text transform (body bytes unchanged) → run it FIRST in the recovery pipeline. Proven: func_8017209C byte-identical afterfunc_8004CFEC→ratan2. - (3b) Shared-caller ARITY conflict (func_80147514 =
s32, func_80168F40 =void *). An already-banked shared caller macro inengine_core.h(e.g.DEFINE_func_80147478, instantiated at a lower address) declares the callee file-scope asextern void func_X(void);(void = no args) then callsfunc_X();— but func_X's real def takes an argument →conflicting types for func_X / previous declaration(NOT implicit-int; it's an explicit(void)proto). FIX = change that caller macro's extern to no-prototypeextern void func_X();(K&R): byte-neutral for the caller (the empty call is identical), and a no-proto decl is compatible with a def whose params are default-promotion-safe (int/s32/long/pointer — NOTchar/short/float; the narrow-param wall §3 of the toolkit still applies there). The macro lives in the shared header, so fleet-re-gate (make check-all). Proven: func_80147514 + func_80168F40, all 136/136 byte-identical, byte-neutral on every overlay (only ov_SC01_077 carries the def; the other 133 keep the stub, where a declaration-only change emits no code).
The recovery pipeline is therefore:
draft → canon_resident_calls → sig_unify → harvest_verify --chunk 1(canon_resident_calls first so link-miss names are fixed before any signature unification; sig_unify still mandatory for def/callee-sig canonicalization). The 3b no-proto move is a separate, one-time edit per caller/callee pair (it touches the shared header) — apply it when the gate reportsconflicting types … (void)from a shared-caller macro and the def's params are promotion-safe. - (3a) Resident-callee LINK-miss (func_8017209C). The draft calls a resident EXE/engine function by
Two NEW residual classes found at scale (beyond the §17 quirks):
- Per-file
-O0class. ~18 functions in ov_SC01_077 were built-O0(prologue sig21F0A003=addu $fp,$sp,$zero, args spilled to frame, load-delay nops, redundantaddu rd,rs,$zero). The correct C is byte-exact at -O0 but the overlay TU compiles-O2, and gcc-2.7.2 has no per-function optimize pragma (opt is per-file, Makefile). So these need their own-O0split file (thesrc/boot.cprecedent, per-fileCC1FLAGS := -O0). HIGH ROI: ~18 fns × reach-134. Members incl. func_8013C360, func_8013B568/B598/B6A0/B7AC/B7F4/B83C/BC7C/BCDC/BD34/BD74, func_8013C08C/C0F8/C360/C414/C938/C964, func_80144B9C, func_801457A4. (→ Phase 19 build-infra task.) - gcc-2.7.2 loop-guard (func_8012C2D0). [Phase-20 R14 CORRECTION — see §20] the real residual is gcc
STRENGTH-REDUCTION / IV-final-value ADDRESSING, NOT operand-order: the loop end is formed as
&D_80120194(the array base)+ 0x658C— the base materialized SEPARATELY then offset (an induction-variable final value), while the start is its own symbolD_801202A0. The right C is base-relative end +lhu/u16(gets structurally close), but gcc -O2 CONSTANT-FOLDSbase + Ninto one address (lui %hi; addiu %lo), so no clean C form (pointer-var, struct-array index) reproduces the separate base materialization. Still a genuine residual — stub; the (uncracked) lever direction is forcing the unfolded IV-final-value, not operand order. - (also: func_8014F2E0 4-off = §10 store-vs-load schedule placement, base-preservation-vs-load-order mutually exclusive — a real §10 residual.)
§18 Per-file -O0 split inside an overlay/blob (Phase 19 T1)
§18-P29 — the multi-stub cluster carve is splat-integration-fragile (defer, or use the whale's shape). P29 Arm A generalized the whale rollout (§38) to the 0x13410..0x14834
-O0cluster viatools/rollout_o0_cluster.py(3-way<ov>/<ov>_o0/<ov>_o2bcarve +O0_CLUSTER_OBJSMakefile wildcard). It banked 9/9 -O0 members on ov_SC07_010 (whole-binary, R22 140/140 — the Task-1 swing verdict is a BANKED FACT, not just masked-MATCH), but the SAME carve byte-shifts 006/007/011 (+0x20%lodata-symbol shift, 34% of bytes, from a CLEAN build; boundaries verified as real fn-starts). Root cause is splat re-disassembly: 3-way splitting a code subseg that still containsINCLUDE_ASMstubs makes spimdisasm resolve some%lorefs to a different auto-symbol — NOT the compiler, NOT a boundary bug. The whale carve (§38) never hits this because its_o0bsplit is a thin#include "../shared/<fn>.h"wrapper with no INCLUDE_ASM to re-disassemble. Lesson: a per-overlay-O0cluster rollout should route each member through a shared-per-member header (whale shape), not leave stubs in the split. Until then the fleet-O0harvest (~1,233 members / ~0.6pp) is DEFERRED;docs/decision-log.md(2026-07-16) has the full byte-evidence. This is the "-O0 cluster split infra" Phase 20 "built + reverted", now root-caused.
A cluster of functions compiled -O0 inside an otherwise--O2 binary needs its own -O0-compiled .c (the
src/boot.c precedent — gcc-2.7.2 has no per-function optimize pragma, opt is per-file via a target-specific
build/src/<path>.o: CC1FLAGS := -O0 …). Detect -O0 by the prologue 21F0A003 (addu $fp,$sp,$zero) + param
spill/reload + load-delay nops.
The mid-blob constraint (the non-obvious part). src/boot.c worked because boot is a PREFIX. When the -O0
cluster is in the MIDDLE of the address space, you CANNOT keep before+after in one .c: a single object's .text
is atomic, so the .ld references main.o(.text) twice but GNU ld consumes it on first match → the after-region
stays concatenated with before and lands at the wrong address (byte divergence appears EARLY, at the first
jal/%hi to a moved callee, not at the cluster). Fix = THREE distinct objects (before / _o0 / after), each
its own .c/subseg so each .text is independent and the .ld orders them by address. Minimize migration by
keeping the bulky side as the original name (asm paths unchanged) and moving the smaller side to a new subseg
(rewrite its INCLUDE_ASM paths with sed nonmatchings/<old>" → nonmatchings/<new>"). Keep the file-top header
(includes + canonical-sig externs) in BOTH halves. R22 clean-rebuild is a 100%-INCLUDE_ASM no-op gate before
adding bodies. NOTE: tools/split_src_region.py's naive item-parser GLOMS a file-top extern block onto a
high-address function name → mis-addresses the split; do the cut by explicit line/offset instead.
-O0 gating ≠ match_one. match_one compiles -O2, so it's WRONG for -O0 functions — gate them only via
the whole-binary build (which honors the Makefile -O0 override). harvest_verify works IF it builds via make;
direct substitute → make build → SHA + per-fn bisection is the reliable loop.
-O0 matched-idiom notes. Hoist data externs to the file top, ONE canonical type per symbol (parallel drafts
disagree: u8 vs s32 on the same D_* → conflicting types). Scalar global store (D_x = k) and
pointer-loops match cleanly.
%lo-folding indexed global — CRACKED (Phase 20, the array-of-STRUCT idiom). The residual: for indexed
global access the original FOLDS %lo into the store (lui %hi(sym); addu $at,idx; sw val,%lo(sym)($at)),
1 ins shorter than our cc1's MATERIALIZE (lui; addiu %lo; addu idx; sw 0($at)). The lever: declare the
global as extern Struct base[] where sizeof(Struct) == the array stride, and write base[index].field.
This keeps sym a symbol_ref through gcc's array-index addressing → it folds %lo(sym+field_off) into the
store, byte-matching. The forms that FAIL (and why §18 first called it irreducible): *(T*)(&sym + index*stride)
and *(T*)((char*)&sym + off) — &sym forces the symbol's full address to be MATERIALIZED as a value (lui+
addiu) before the index add, so %lo can't fold. Worked example (byte-gated, func_8013B7AC):
*(s32*)(&D_801DAA08 + a0*0x1C)=0 (materializes, FAILS) → typedef struct{s32 f0; u8 pad[0x18];} E; extern E D_801DAA08[]; D_801DAA08[a0].f0 = 0; (folds, MATCHES). Pick the struct so the accessed field's address == the
target symbol (put the field at offset 0 and base the array at the field's symbol, OR base at the real array
symbol and use the real field offset — same bytes either way; the %hi/%lo immediates encode the address).
The constant-multiply for the stride (e.g. ×0x1C → sll 3; subu; sll 2) is gcc's synth_mult, emitted even at
-O0. Gate via the WHOLE-BINARY -O0 build (match_one is -O2, wrong for -O0). NB: the -O0 cluster is
overlay-LOCAL (per the correction below) so this banks ×1 per overlay — but the idiom is reusable for ANY
-O0 (or -O2) indexed-global access, fleet-wide.
-O0 reach-134 propagation is NOT free. -O0 functions can't go through engine_core.h (it's included by
each overlay's -O2 main .c → would compile -O2 → not match). To bank the ×134, each overlay needs its OWN
-O0 split. Uniform across the fleet (all overlays share vram base 0x80128158, so the cluster offsets are
identical) → scriptable, but it's per-overlay infra + 134 gates, not the "free ×134" a dedup report implies.
Phase-20 R14 CORRECTION — the -O0 cluster is NOT reach-134 byte-identical; the rollout is INVALID.
Byte-proven: of the 134 overlays that span the cluster (vram 0x8013B568, file 0x13410), only 1 — ov_SC01_077
itself — has cluster bytes identical to the matched C (even func_8013B568's first 0x20 B match in just 1).
The cluster functions reference per-overlay data addresses (func_8013B568 stores to D_80187270/%lo 0x7270
in ov_SC01_077 but to 0x80182B04/%lo 0x2b04 in ov_SC01_005 — same instruction shape, different overlay-local
global), so each overlay's bytes differ → they are overlay-LOCAL code, not shared engine code. The
Phase-18/19 "6/16 matched (reach-134)" label conflated function-present-at-this-vram with byte-identical
(the precise R14 failure mode: trust the bytes, not the reach label). So the matched C banks ONLY ov_SC01_077
(×1); a fleet rollout via shared C is impossible. Banking these elsewhere = per-overlay RE-matching with each
overlay's own data addresses (extract %lo per overlay → template the C) — ~6 tiny fns × per-overlay, the
×1 per-overlay-unique bucket (low priority), ~0.2–0.3% for the whole-cluster, NOT the "+0.6% free ×134" the
backlog projected. The §18 split machinery (3-object before/_o0/after split + a build/src/%_o0.o: CC1FLAGS:=-O0
pattern rule + a PROVIDE-based .ld for the out-of-range data syms) was built and byte-validated to compile/
link a second overlay, then the gate exposed the per-overlay-data wall and it was reverted (the finding, not
the infra, is the deliverable). Net: skip the -O0 rollout; it's low-ROI per-overlay-unique work, not a shared win.
§19 Scaling the toolkit waves — the recovery PIPELINE + the propagation CAP (Phase 19 T3)
Two §17/§17a waves over the tractable reach-134 tail, measured. match_one close-rate is high and rising (batch-1 44/50 = 88%, batch-2 35/38 = 92% after the fixes below); the work is in the match_one→whole-binary gap (declaration plumbing) and then in propagation (not matching).
The gate PIPELINE order matters — canon-only FIRST, sig_unify FALLBACK. tools/sig_unify.py can REGRESS a
draft the agent already wrote canonically (it re-canonicalizes the def/callee sigs and occasionally forces a sig
that perturbs codegen, or mangles a line into a parse error). So gate in two stages (.run/t3_gate2.sh):
(1) canon_resident_calls → whole-binary gate ALL drafts; (2) sig_unify ONLY the stage-1 failures → gate again.
Measured: unconditional sig_unify lost ~5/batch vs canon-first.
The dominant gate failure at scale = the shared-caller ARITY class → tools/fix_arity_callers.py (automated
§17a-3b). Once garbled hints and sig_unify-regressions are removed, ~100% of the residual gate failures are:
a banked SHARED caller in engine_core.h declares the callee extern <ret> func_X(void); (or a different
arity), conflicting with the real def that takes args. FIX = rewrite the caller decl to no-prototype
extern <ret> func_X(); — byte-neutral, compatible with promotion-safe params (int/s32/u32/long/ptr; NOT
char/short/float — narrow-param wall). fix_arity_callers.py --apply --from-file <fails> --drafts <dir> does it
(skips narrow-param defs), then re-gate; --revert to undo. Batch-2: recovered 8/18 this way; the rest were
genuine loose-typing conflicts (caller sig ≠ def sig with call-site casts already in play — Phase-16 wall).
Garbled callee hints → fixed at the source. gen_harvest_targets.py INLINE_DEF_RE was matching an
indented if (func_X(...) == ...) { call-expression as a "definition" → a garbled callee "signature" agents
paste verbatim as an extern → PARSE error at the gate. Fix: column-0 + type-only prefix (^[A-Za-z_][\w \t*]*?).
0 garbled of 749 after; batch-2 close-rate 88%→92%.
THE CAP: propagation, not matching. dedup_propagate only lifts a body whose types resolve from
common.h + engine_types.h (its compiles_standalone filter) — so a matched function whose body uses a
typedef'd / anonymous / sibling local type stays ov_SC01_077-LOCAL (no ×134, no fleet %). Batch-1 propagated
25/30, batch-2 only 10/25 (struct-heavier). tools/build_engine_types.py --strip lifts NAMED structs (0
same-name-different-layout collisions historically) but NOT typedefs → ~16 banked matches still can't propagate.
Lever (Phase 20): extend the type-lift to typedefs/local types → recovers those ×134 for ~0 agent tokens AND
raises every future batch's realized yield. The harvest's bottleneck has moved from "can we match it" to "can we
share it."
§20 The wave-at-scale GATE CAP + residual-class verdicts (Phase 20)
Phase 20 closed the §19 propagation cap (T1 below) then ran the §17/§19 wave + recovery over the reach-134 ≤90-ins tail — and hit a HARDER gate cap than Phase 19's. All byte-verified.
THE CAP: at this tail the match_one→gate gap is the LOOSE-TYPING CALL-GRAPH wall (not cheap plumbing)
Batch numbers: 41/48 match_one MATCH (85%), but only 8/41 survived the whole-binary gate (~80% gap — vs
Phase 19's ~30%). The gap is NOT the §17a/§19 cheap declaration-plumbing (link-miss/arity recovered ~0 here).
Byte-verified cause: a callee (e.g. func_80153C74) is declared with CONFLICTING types at different overlay
sites, so the draft's extern hits in-TU conflicting types — a COMPILE error, not a byte miss. match_one
OVER-PREDICTS because it compiles STANDALONE with the draft's own externs and masks jal/%hi/%lo — it never
sees the overlay's conflicting decls. Every recovery lever fails at this tail (all 0): sig_unify /
canon_draft_decls impose a single "canonical" that's wrong for some sites (loose-typing); fix_arity_callers
(not arity); no-proto externs (incompatible with the overlay's NARROW existing decls — void f() can't
co-exist with void f(s16)); strip-externs (implicit-int ≠ target). The ONLY fix is the §17a-1 per-site
function-pointer cast ((ret(*)(args))func_X)(…) — it takes the callee's ADDRESS and calls with the draft's
intended sig, so there's no global decl and no conflict. Agents don't apply it reliably → AUTOMATE it: a
recovery pass that, per draft, DROPS the conflicting callee extern + CASTS the call to the draft's sig, then
gates. That is the Phase-21 cap lever (recovers a batch's lost ~33 reach-134 AND lifts every wave's gate-pass
from ~20% toward ~80%+). The bottleneck has moved again: matching ✓ (85%) → sharing ✓ (§19 type-lift) → now
in-TU declaration reconciliation under loose typing.
Diagnostic lesson: a failed in-TU build leaves a STALE .o
When you substitute a draft and the in-TU build FAILS to compile, build/src/<…>.o retains the PREVIOUS
(target/stub) bytes → objdump of that .o shows a FALSE "byte-match." Always trust the whole-binary SHA
gate, not a per-function objdump (cost a real detour: func_80153C44's stale .o looked identical while the
true failure was a conflicting types compile error).
NEW / CONFIRMED residual classes (this session)
- §10 store-vs-load scheduling (func_8014F2E0, func_80150528) — CONFIRMED unsteerable. Wave agents tried
for-init / barrier / precompute / volatile;
sched.ctie-break: the IV-init lands before the loop guard not in the preheader, and theD_x=0store schedules BETWEEN two arg-loads instead of after both — mutually exclusive with base-preservation. Stub. - §10 hoist-vs-remat regalloc tie-break (func_80149374, func_801493D0) — CONFIRMED. gcc caches a
sp+offbuffer address in a freed callee-saved reg + moves (cheaper by its count) where the target REMATERIALIZESaddiu $a,$sp,offper call. Array-decay / pins / barriers / permuter all fail. Stub. - IV-combine divergence (func_80177AD4) — NEW. Our cc1's
combine_givswon't fold a halfword RMW (lhu;sh -2(p)) into the byte biv (sb 0(p)) the way the target does (one IV atp+0x20); it spawns a dedicated 2nd IV → wrong base constant (p+0x1e). Probe-confirmed (a non-RMW*p/*(p-2)pair combines fine; the RMW spawns the 2nd IV). Genuine codegen divergence, not source-typeable. Stub. - Hoisted-invariant PROLOGUE ORDER (func_80177F84). 3 prologue insns in the wrong order — gcc emits the
pinned-
$a2pointer init before the two hoisted loop-invariant constants; the target emits the constants first. No prologue permutation is < 3-off; the permuter can't run (theregister __asm__pins are rejected by pycparser). §3/§5-class — stub. - The -O1 class (func_80161A90) — NEW build-infra (extends §18). A function built
-O1(frame 0x18,lhureload, unfolded base, load-delay nops) inside an otherwise--O2overlay.match_one(hardcoded-O2) CANNOT match it. Like the §18-O0class but-O1→ needs its own-O1split file (target-specificCC1FLAGS := -O1). Detect: prologue/scheduling between-O0(21F0A003) and-O2.
Operational gotchas (cost real time)
- Workflow
args: pass the target list as a JSON ARRAY, not a JSON string — a stringified array reaches the script as one string andargs.mapthrows (names.map is not a function). Defensive:const names = Array.isArray(args) ? args : JSON.parse(args). harvest_verify --chunk 1for wave batches. With--chunk >1, ONE draft that fails to COMPILE (a loose-typing conflict) fails the whole chunk's build and the bisection mis-attributes the innocent neighbors as failures. Gate wave drafts one at a time when the failure mode is compile-conflicts.
What WORKED — the Phase-20 reusable wins
- T1 — the typedef type-lift (
tools/build_engine_types.pyextended). Addedfind_typedefs()(brace-aware: anon-structtypedef struct{…}N;, fn-ptrtypedef r(*N)();, alias) + same-name-different-layout collision + tagged-struct-typedef overlap guards; emits typedefs in source order AFTER the named structs (deps likeA→Spreserved). Closed the §19 type-blocked propagation cap — 9 reach-134 fns ×134 for ~0 agent tokens, byte-neutral (--stripremoves the defs; type decls emit no code). - T2 — the residual router (
tools/exemplar_miner.py). Consumeswall_taxonomy.json+ per-overlay reach (dedup_propagate's computation) → routes every residual to a lever (WAVE / STRUCT / PINS / STUB) →docs/exemplar_curriculum.md+ the reach-134 wave-target list +.run/exemplar_routing.json. The "scan all residuals, pick the teachers / size the pools" router. Caveat: itsmismatchis the M2C-DRAFT mismatch, NOT the hand-match floor (a loop-guard buckets STRUCTURAL_MISS at mismatch-16 yet hand-floors to 1).
Phase-20 RESOLUTION — tools/cast_call_sites.py BUILT + the cap re-diagnosed (R14, byte-proven)
The §17a-1 per-site cast is now AUTOMATED. tools/cast_call_sites.py (pure --in/--out transform, sibling
of sig_unify/canon_resident_calls): per draft, for every callee whose canonical TU sig differs from the
draft's intended sig (exactly the conflict set), it (1) rewrites the callee's decl line → the canonical
declaration (kills the in-TU conflicting types, keeps the symbol in scope) and (2) casts every call site to
the draft's INTENDED sig ((ret(*)(args))func_X)(args) (decl lines never cast). The cast is codegen-neutral —
confirmed: gcc-2.7.2 folds (cast)func_symbol back to a direct jal func_X with the draft's calling
convention. Pipeline: canon_resident_calls → cast_call_sites → sig_unify (def-sig) → harvest_verify --chunk 1.
RESULT on T6 batch-1's 33 gate-fails: the callee-cast recovered 6 byte-identical (func_80153C44/ 8015CF58/801711FC/80161BE0/801683D8/8015F948); 5 propagated ×134 + 1 local; fleet 58.63%→58.82%, 136/136.
THE §20 "~33" PROJECTION WAS WRONG (R14 — the earlier diagnosis was incomplete). Byte-classifying all 33
gate-fails: the batch is NOT dominated by the callee-conflict class. It splits into THREE directions, ALL
declaration-driven (0 pure codegen-quirk survives match_one):
- Callee-conflict (~6) — the draft CALLS a shared callee declared inconsistently →
cast_call_sites✓. - DEF-conflict (dominant, ~18) — the GENUINE loose-typing wall, byte-proven unrecoverable by text transform.
A banked caller (an
engine_core.hDEFINE macro, invoked in this overlay) declares the draft's OWN function with a sig the matchable def can't satisfy (e.g. func_80161208: callerss32 f(void)0-arg vs defvoid f(void*)1-arg; func_80146A6C: callerss32 f(s32,void*,…)vs defvoid f(short,int,…)). Both ways fail: keep the draft's def-sig →conflicting types; canonicalize the def-sig (sig_unify) → 11/26 compile clean but BYTE-MISMATCH (the body genuinely needs the draft's sig). The symmetric "caller-side cast" fix is BLOCKED:INCLUDE_ASMemits only an__asm__(".include …")block — it declares NO C symbol — so a shared macro's internalextern func_Xis the ONLY declaration of func_X in the 133 STUB overlays; dropping it (to cast the caller) breaks them, and it can't be edited per-overlay (it's in the shared header). The only path for these is RE-DRAFTING the body under the caller-canonical sig (a future wave with the canonical pinned), and the arity-mismatch subset (0-arg callers vs N-arg def) has NO compatible C sig at all → hard stub. - DATA-conflict — byte-proven MOOT on this tail (R14, do not build the data-cast for it). A data-analog
cast (
*(T*)&D_xagainst the canonical decl) sounded like the naturalcast_call_sitesextension, but the bytes say there's nothing to cast: 0 drafts declare anextern struct/uniondata conflict, and the lone apparent "DATA-conflict" (func_8016A8FC /D_800AE620) was a typedef-REDEFINITION — the draft re-definestypedef struct{s32 w[8];} Blk20;inline whileBlk20is already inengine_types.h(both sides useextern Blk20 D_800AE620— SAME type, no data conflict). Stripping the redundant inline typedef compiles but STILL byte-mismatches (the underlying blocker is the def-sig loose-typing wall, class 2). So the data-cast would be code for 0 real cases; skip it. (A general "strip inline named-type defs already in engine_types.h" pass is a 1-line harvest_verify-style cleanup if a future wave needs it — but it recovered 0 here.)
Net: the cast tool is the real, reusable cap-lever for the callee-conflict fraction of every wave (it
recovers what §17a-1 casting can, byte-gated) — but it does NOT lift gate-pass to ~80% here, because THIS tail
is dominated by the def-side loose-typing wall, not callee-conflicts. The honest bottleneck for the reach-134
residual: a matched body whose required signature is incompatible with the established caller-canonical, with no
caller-side escape (INCLUDE_ASM declares nothing). Diagnostic discipline that found this: classify EVERY
gate-fail by build-error class (callee conflicting types / DEF conflicting types for the draft's own fn /
DATA / clean-build BYTE-MISMATCH) before assuming a single cause — the §20 first pass saw one callee example and
generalized it; the bytes said otherwise.
T3a %lo-folding -O0 — CRACKED (Phase 20). Not irreducible after all: the array-of-STRUCT idiom folds
%lo (declare extern Struct base[], sizeof == stride, access base[i].field; NOT *(T*)(&sym+i*stride),
which materializes). Full write-up + worked example in §18 ("%lo-folding indexed global — CRACKED").
Banks ×1 (the cluster is overlay-local) but the idiom is reusable fleet-wide for any indexed-global access.
§21 — wave-distilled idioms (Phase 21)
Byte-gated wins from the class-grouped waves. Each is a generalizable C shape (not a one-off), tied to its byte-matched evidence fn. (Pins/array-decay/statement-order/shared-ret0/for-vs-do-while are §17–§18 — not re-listed.)
- mem→mem unaligned N-byte copy →
memcpy(dst, src, N): when the target copies a contiguous byte region withlwl/lwr+swl/swrpairs (unaligned 8-byte block, no field math), writememcpy((void*)dst,(void*)src,8)withextern void *memcpy(void*,const void*,u32);— gcc-2.7.2 inlines the small fixed-size copy to exactly that lwl/lwr/swl/swr sequence (element-wise*(T*)dst=*(T*)srcinstead pickslw/swfor the aligned case or splits wrong for unaligned). fixes inlined-unaligned-block-copy codegen; evidence func_80153800 (sibling func_80146FC4 same shape). - unaligned-SOURCE word load (then byte-read it) →
__attribute__((packed,aligned(1)))struct in a union, plain assignment: when the source is at an UNALIGNED address (e.g. an odd global) so the target reads itlwl/lwrinto an aligned stack slot (swl/swr) and then reads individual bytes off$sp(lbu 0/1/2($sp)), the memcpy form above copies it but gives no typed handle to the bytes; declaring the source a plainint/structMISSES (gcc assumes alignment →lw). Mark the source typealigned(1)and wrap it with a byte view in a union, then ASSIGN it to a stack union and index the bytes:struct W{int w;}__attribute__((packed,aligned(1))); union U{struct W w; u8 b[4];}; extern struct W G; union U t; t.w = G; … t.b[0] … t.b[1] …. Thepacked,aligned(1)makes gcc emitlwl/lwrfor the bare struct assignment (the unaligned read), theswl/swrlands it on the aligned stack union, andt.b[i]becomes the$sp-directlbu. fixes alw-vs-lwl/lwrunaligned-source load where you also need per-byte access; evidence func_80142A80 (D_801BD644readlwl 3 / lwr 0, byte-readlbu 0/1/2($sp)). - aligned 16-byte field-block copy → one struct assignment: a contiguous aligned 4-word copy between two memory
locations is
typedef struct{u32 a,b,c,d;} Blk16; *(Blk16*)(dst)=*(Blk16*)(src);→ gcc emits its a0–a3 4-register block load/store (1 ins shorter than four separatelw/sw, and fixes the v1/v0 load order + the load-delay nop). Four element-wise word copies constant-fold each base to its ownlui(+ins, wrong order). fixes word-by-word vs block-move + load-delay; evidence func_80163A94 (0x34..0x50 = two Blk16 assigns). - partially-read out-param region → ONE stack struct, not separate scalars: when a callee fills several fields of
a stack buffer via
&buf/&buf.fieldbut the caller reads only SOME of them, declaring the slots as separate locals lets gcc DCE the unread ones and overlap/shrink the frame (wrong frame size). Make the whole region ONEstruct buf;and pass(s32)&buf/(s32)&buf.field— the struct keeps every slot live at its true frame offset. fixes DCE-driven frame-size/overlap divergence; evidence func_801749C8. - (refinement of §16 mask-local — the inverse direction) raw
lhu+(s16)at each use: to force the target'slhu;sll;srainstead of a foldedlh, load the global as a RAWu16(iVar=(u16)D_x;) and apply(s16)at every USE site — this defeats gcc'slhu+sext→lhcombine fold (the opposite goal to §16's mask-hoist, which keepslh+andi). evidence func_801749C8. - (refinement of the §5a/§17 zero-byte barrier) in-place re-tie variant:
__asm__ __volatile__("":"=r"(x):"0"(x))(output tied to input via constraint"0") forces x to be re-materialized into a register at that point, pinning where a following store schedules — distinct from the input-only anchor__asm__ __volatile__("":: "r"(x))(which only anchors x ahead of the next op). Use when interleaved stores need a value freshly re-tied mid-sequence. evidence func_80165CA0 (SHB(x)macro, combined with$v0/$v1pins). - disjoint-bits
x + CONSTemitsori, notaddiu→ break it with a re-tie barrier: when x's low bits are provably zero where CONST has bits (e.g.x = (v & 0x7F00) >> 5;thenx + 0xC00— masked-then-shifted value can't overlap 0xC00), gcc-2.7.2 proves the add is disjoint and folds it toori x,x,CONST; but the target usedaddiu. Insert the §21 re-tie barrier__asm__ __volatile__("":"=r"(x):"0"(x));BETWEEN the mask/shift and the+ CONST— re-materializing x there erases the known-zero-bits range, so gcc can no longer prove disjointness and emitsaddiu x,x,CONST. (Same barrier syntax as the bullet above, but the effect here is ARITHMETIC OPCODE SELECTION, not store scheduling.) fixes theadd→oridisjoint-bit fold; evidence func_80169058 (the+0xC00no-bit-overlap add). - steer WHICH giv becomes the loop IV anchor → make that store LAST in source order: when a loop writes several
fields off one moving record pointer (
*(T*)(p+k)=…for several k, thenp += stride), gcc-2.7.2 combines those givs into ONE induction reg, andloop.c:record_givPREPENDS each new giv tobl->givsocombine_givspicks the LAST-recorded (= last in PROGRAM ORDER) store asgiv_array[0], the anchor base. The target picks a particular offset as anchor (visible in the.sas the IV reg =addiu $iv,$base,Kand every field store reaching off it with displacements of one sign, e.g. all<=0). To match, write the field whose offset is that anchor (K) as the LAST store before the pointer bump; the others (earlier in source) then reach it with the matching-sign displacements. (This is the RECOVERABLE counterpart to §20's IV-combine failure class — there combine refuses to fold a halfword RMW and you stub; here combine DOES fold and you steer the anchor by store order.) fixes wrong IV base-constant / all-positive-vs-all-negative displacement set; evidence func_80178298 (anchor at p+0x12, all stores<=0off it; the +0x12 store moved last took it 15-mismatch→3). - register-resident
shorttruthiness test →sll rX,16+ branch (NOT a barebnez): when a flag/counter the target keeps in a register (never spilled) is tested for!= 0viasll $v0,$reg,16; beqz/bnez $v0instead of branching on the value directly, the original local was ashort/s16— gcc-2.7.2 lowers ashortrvalue's truthiness by left-shifting 16 (dropping the upper half) then branching. Reproduce it on anintlocal by writing the test explicitly asif ((flag << 0x10) != 0)(the shift is computed without storing back, so the counter stays live in its reg across the increments); equivalently declare the locals16. A plainif (flag != 0)on anintemitsbnez $regwith noslland the diff won't close. fixes the missingsll _,16before a truthiness branch; evidence func_8013F244 ((iVar2 << 0x10) != 0, the twosll $v0,$v1,16merge sites). - force a global RE-LOAD across a store WITHOUT pinning the frame/prologue → NON-volatile
__asm__("":::"memory")(NOT volatile): when the target reads a global N times with intervening pointer-stores (each section gets its ownlhu sym/reload), gcc-2.7.2 -O2 instead CSEs the derived index (e.g.idx*4survives in a reg across the store, so no reload). A memory clobber forces the reload — but the volatile form (__asm__ __volatile__("":::"memory")) is a hard scheduling fence: it (a) pins the prologue stack-adjust (addiu sp,-N) to the top of the function (cc1 emitssubu spbefore the asm because the clobber may touch the stack), so a target whose frame alloc is scheduled mid-function (into a load-delay slot) can't match, and (b) co-schedules large-constant builds (li reg,HI; ori reg,LO) as an adjacent unit. Dropping__volatile__keeps the memory clobber (→ the reload still fires) but makes it a soft dependency the scheduler reorders around: thesubu spthen floats down into the load-delay slot, AND cc1 splits the constant build around an independent address materialization (li t0; la a2; ori t0instead ofli t0; ori t0; la a2). So: use the non-volatile memory clobber when you need the reload but the target has a scheduler-mobile frame and/or a split constant build; use the volatile form only when you also want the hard fence. (The §5a cross-jump barrier still needs volatile — that's a correctness fence, different goal.) fixes simultaneous {global-reload, mid-function phantom-frame placement, split %hi/%lo-constant prologue order}; evidence func_801758FC (3 sections each reloadlhu D_800B9A02; phantom 0x10 frame alloc lands in the F7D0 pointer-load's delay slot; closed 4-/3-mismatch tensions → MATCH 55 ins). - induce a phantom (unused, no-spill) stack frame the target has but your codegen omits → an address-taken local
array
s32 frame_pad[N]; (void)&frame_pad;: when the target reservesaddiu sp,-0x10/+0x10with NO register saves and NO spills (a leaf whose original had a stack local the optimizer later kept in regs), gcc'scompute_frame_sizeroundsget_frame_size()up to the next 8 → declare a local whose address is taken (so it's not DCE'd and reservesvar_size) but never stored through.(void)&frame_pad;escapes the address with zero emitted code at -O2. Size N×4 picks the frame: 1–2 words → 8, 3–4 → 0x10. (This is the INDUCE direction; §5's phantom-frame note is the REMOVE direction where gcc adds a frame the target lacks.) evidence func_801758FC (0x10). - store a high-bit (≥0x8000) 16-bit constant to a halfword →
unsigned short*/u16*, NOTshort*/s16*: the STORE issheither way, but the constant materialization differs by signedness of the pointee. Au16store zero-extends →ori reg,$zero,K(opcode0x34, e.g.3403c040=ori v1,$zero,0xc040); ans16store sign-extends →addiu reg,$zero,K(opcode0x24, e.g.2403c040=addiu v1,$zero,-16320, value 0xFFFFC040). objdump pretty-prints BOTH asli v1,0xc040/li v1,-16320, so read the opcode (34xx=ori vs24xx=addiu), not the mnemonic. Only matters when bit 15 of the constant is set (0x8000–0xFFFF: 0xc040, 0xaa10, …); for K<0x8000 both forms emit the sameori. Picku16*to getori,s16*to get the sign-extendingaddiu. fixes theaddiu(sign-ext)↔ori(zero-ext) constant-build before a halfword store; evidence func_8017E924 + func_8017E974 (0xc040), func_80182E30 (0xaa10). $sp-manipulating scratchpad-stack-switch trampoline → a FULL__asm__ __volatile__body (clobber"memory"), copied from the byte-proven sibling with ONLY thejaltarget + theD_801D961xsymbol swapped — plus three maspsx rules that are byte-load-bearing. This 22-ins idiom (repoint$spinto the D-cache scratchpad stack at*(0x1F8003FC), call one engine fn, stash$v0throughD_801D961x, restore$sp) is NOT expressible in C (it rewrites$sparound a call), so write the whole body as inline asm with.set noreorder. It is a 15-member reach-134 duplicate family (canonicalfunc_8014CCB4), so each match is high-value. The three rules the wave proved (each a +1-ins byte-miss if violated): (1) do NOT write an explicitnopafterjal— maspsx--aspsx-version=2.56auto-fills the delay slot; putlui $at,%%hi(D_801D961x)directly after thejaland its auto-nop becomes the slot (→jal/nop/luiexactly); (2) no trailing.set reorder— it emits a stray epilogue nop; (3) escape%hi/%loas%%hi/%%loinside the__asm__string (a bare%is read as an operand placeholder → assembler error). (General beyond this trampoline: rules 1–3 apply to ANY hand-written inline- asm body carrying ajal+%hi/%lorelocations.) fixes double-delay-nop / stray-epilogue-nop /%-placeholder byte-misses in a hand-asm$sp-switch wrapper; evidence func_8014D04C, func_8014D738, func_8014DF3C, func_8014E434, func_8014E6A0, func_8014E934, func_8014ED28, func_8014F1F4, func_8014F468, func_8014F6F4, func_8014FCFC, func_80150480 (12 banked this wave, family of func_8014CCB4).- byte/
u8in-place pre-decrement that tests the OLD value → write the subtract as+ 0xFF, NOT- 1: for au8field decremented in place where the original tests the pre-decrement value (c = p[i]; p[i] = c-1; if (c==0)…, emittinglbu;addiu $v0,$v1,0xFF;bnez $v1;sb $v0), gcc-2.7.2 does NOT canonicalizec + 0xFFandc - 1to the same immediate even though they are equal mod 256 and thesbtruncates either way — it materializes whatever signed-representable literal you wrote. Empirically (this toolchain):c + 0xFF→addiu …,0xFF(2462…00FF, matches),c - 1(andp[i]-1) →addiu …,-1(2462…FFFF, MISSES); the distinction holds even on a plainint. So when the target's decrement immediate is0xFF(or any positive wrap-literal) rather than-1, write the wrapping formp[i] = c + 0xFF;explicitly. fixes theaddiu …,-1↔addiu …,0xFFimmediate-literal mismatch on a byte predecrement; evidence func_8016EBA8 (param_1[2] = c + 0xFFwith pre-decrementif (c==0)). - store a call result AND test/reuse it in one expression → combined assignment
*(T*)(p+k) = local = f();(NOT a store of a re-read local): when the target doesjal f; sw $v0,k(base)(the store in the call's delay slot, or immediately after) then BRANCHES on$v0(beqz/bnez $v0) and/or copies it to a callee-saved reg (addu $sX,$v0,$zero) for use in the success arm, write the store, the test, and the reuse off ONE local that is assigned the call result in the SAME statement as the store:*(int*)(p+0x20) = v0 = f(); if (v0==0){…} else {…use v0…}. gcc-2.7.2 then keeps the result in$v0so thesw $v0(delay slot) and thebeqz $v0both read it directly, and emits theaddu $sX,$v0copy ONLY because the success arm reusesv0across later calls. The naive two-statement form (v0 = f(); *(int*)(p+0x20) = v0;) instead emits the copy-to-$sXBEFORE the store and loses the baresw $v0/delay-slot schedule. (General: any "store an allocator/constructor result to a struct field, null-check it, then use it" pattern.) fixes the call-result store/test landing in the wrong register + lost delay-slot store; evidence func_80142DC4 (*(int*)(param_1+0x20)=v0=func_8012C1B8(); if(v0==0)… else …v0…). - force a memory-operand RELOAD (two identical
lws) with NO barrier → place the intervening store between the two reads in SOURCE ORDER: when the target reads the same field twice (lw $v0,K(base)… thenlw $v0,K(base)again) with a store to a different field in between — i.e. gcc-2.7.2 -O2 did NOT keep*(p+K)cached across that store — reproduce the reload purely by statement order: write the dependent store as a statement that uses*(p+K)and place it BEFORE the second use, so an unrelated write (*(p+J)=…) sits between the two reads and defeats the load-CSE. No__asm__clobber and no pin needed (this is the barrier-free counterpart of the §21 memory-clobber reload bullet, which is for a derived-index CSE; here the CSE'd thing is the memory load itself). Hoisting the field into one local (int t = *(p+K); …; use t;) instead keeps it in a reg → a singlelw, misses. fixes a missing 2ndlw <field>reload across an unrelated field store; evidence func_80168430 (*(p+0x30)=D[*(p+0x2C)];placed beforeiVar1=(*(p+0x2C)<<17)>>16;→ thesw 0x30between the twolw 0x2Cforces the reload, 33/33 ins). - EXPLOIT cross-jumping (the inverse of the §5a barrier) — duplicate the SAME call into BOTH if/else arms to merge
the
jalwhile keeping per-arm operand setup: when anif/elseends by calling ONE function with the same argument-register layout but DIFFERENT constant/value args per arm, and the target emits ONE sharedjalsite reached by ajfrom one arm + fall-through from the other (each arm doing its ownaddu $a0,$base+addiu $a1,Kbeforehand, with anopjoin delay slot), write the call LITERALLY IN BOTH ARMS:if(c){ g(p,5); h(p,0x1C); } else { g(p,4); h(p,0x11); }. gcc-2.7.2's cross-jump pass (§5a) finds the twojal hinsns RTL-identical → merges them into one shared site, but the per-arm arg setup differs → stays duplicated → exactly the[arm1:setup; j join][arm2:setup; fall][join: jal; nop]layout. The naive single-call form (compute K in theif, thenh(p,K)once after the join) instead schedules the arg setup differently and loses the dual-setup/j/nopshape. (This is the deliberate-MERGE direction; §5a's__asm__ __volatile__("")barrier is the deliberate-KEEP-SEPARATE direction — same pass, opposite goal. Combine with the §3-T4 branch-polarity invert to pick which arm is thebeqzfall-through.) fixes a duplicated-vs-sharedjal+ missing per-arm arg setup / joinnop; evidence func_80159BE4 (func_80154A74(arg0,0x1C)in the if-arm +func_80154A74(arg0,0x11)in the else-arm → one mergedjalat the join, 40 ins). - read-modify-write of a SCALAR global where the target materializes the address ONCE and reuses it for both the
lwand thesw→ access it through a pointer VARIABLE (T *p = &D_x; *p += 1; if (*p >= K)), NOT the bare global (D_x += 1): when the target doeslui $r,%hi(sym); addiu $r,$r,%lo(sym); lw $v,0($r); … sw $v,0($r)(one address reg,0($r)displacement on BOTH accesses), gcc-2.7.2 -O2 reaches that by CSE-ing the&D_xaddress expression across the read and the write. Writing the bare global RMW (D_x += 1) instead lets gcc fold%lointo each access INDEPENDENTLY → two separatelui … ; lw/sw %lo(sym)($at)materializations (the §18 single-store fold, applied twice; wrong reg/ins for an RMW). A pointer var to the global forces the shared base reg. (This is the GLOBAL-RMW counterpart of §18's indexed-global fold and §17's stack-buffer remat — distinct trigger: a scalar global read AND written in the same region.) NB unrelated to the same fn: read a+0x34halfword field withlhu(u16), notlh, when the target zero-extends. fixes two-separate-%lo-folds vs one-materialized-base-reg on a scalar global RMW; evidence func_80186938 (s32 *p=&D_801270C8; *p+=1; if(*p>=4)…→lui;addiu %lo;lw 0($v1);…;sw 0($v1), byte-gated). - force a RELOAD between two CONSECUTIVE stores to the SAME global (store a const, then read it back to OR/RMW) →
qualify the global
volatile, NOT an__asm__barrier: when the target doessh K,%lo(sym); lhu %lo(sym); ori …; sh %lo(sym)— i.e. it stores a known constant to a global and then re-reads it from memory before the next store — gcc-2.7.2 -O2 normally store-to-load-FORWARDS the just-stored constant and CONSTANT-FOLDS the two writes into ONE (D_x = K | 0x4000;→ a singleshwithli K|0x4000), so the reloadlhuand the secondshvanish. Declaring the globalvolatile(extern volatile u16 D_x;) forbids the forward/fold: eachD_x = …is emitted verbatim and the read between them becomes thelhureload, reproducing both stores. This is the TYPE-QUALIFIER reload lever — distinct from the §21__asm__memory-clobber (derived-index CSE across pointer stores) and the statement-order reload (two identical loads across an unrelated field store): here the value is a constant gcc KNOWS, so onlyvolatiledefeats the store-forwarding; statement order and clobbers do not. Use it whenever the target re-reads a global it just stored a literal into. fixes a folded-awaylhureload + merged doubleshto one global; evidence func_801806D8 (volatile u16 D_80126B96; D_x=2; D_x=D_x|0x4000;→sh 2; lhu; ori 0x4000; sh, byte-gated 56 ins). - force ONE specific field/pointer load to RELOAD (target re-emits
lw K(base)where gcc -O2 CSEs it across intervening calls/non-field stores) → cast THAT access*(volatile T*), NOT a global__asm__fence: when the target reads a struct field through a pointer twice (lw 0x64($s0)… other code (calls, stores to unrelated addresses) …lw 0x64($s0)again) but gcc -O2 keeps the first load cached in a register (a singlelw, the field reused via regalloc — so neither statement-order nor a constant store-forward defeats it), wrap each reload site in a volatile pointer cast:iVar = *(volatile int *)(param_1 + 0x32);(and reuse it for the follow-on field reads offiVar). Thevolatilemakes that ONE access a non-CSE-able memory reference, so gcc re-emitslw 0x64($s0)there, while leaving all other scheduling untouched. This is the SURGICAL reload lever — distinct from the §21 non-volatile__asm__("":::"memory")clobber (a function-wide fence, for a derived-index CSE), the statement-order reload (needs an unrelated field store to fall between the two reads), and the §22volatile-qualified GLOBAL (defeats constant store-forwarding of a known literal): here the CSE'd thing is a runtime field load reused via register, the intervening ops are calls/foreign stores (not a usable field-store), and you must NOT add a global barrier (it perturbs the rest of the schedule). fixes a folded-away 2nd/3rdlw <field>reload of a pointer-reached struct field across intervening calls; evidence func_801424E4 (*(volatile int*)(param_1+0x32)×2 →lw 0x64($s0)re-emitted at each use, byte-gated 58 ins). - fill a load-delay slot in a tail-store sequence → read the field into an explicit TEMP one statement EARLY
(refinement of §2-T2): when a tail reads several fields from an out-param/struct and then writes several
constant stores, and the target schedules ONE of those field loads early so its load-delay slot is filled by a
following
li(the value held in a reg and stored LAST), pull that load up with a named temp:c = out.c; *(int*)(p+0x1c)=0x5a; *(short*)(p+0x34)=1; *(short*)(p+0xe)=c;→ gcc emits thelhuforout.cahead of theli 0x5a(delay-slot filled) and thesh clast from the held register. Leaving it inline as the last store (*(short*)(p+0xe)=out.c;) instead lets gcc defer a different field's store and the schedule diverges. The temp only relocates the load in source order; §2-T2 names the mechanism but punts the load-delay case to the permuter — this is the concrete C lever for it. fixes a deferred-field-store / unfilled load-delay schedule in a const-store tail; evidence func_801856F8 (c = out.c;hoisted before the0x5a/1stores, byte-gated 71 ins). -
- **handwritten GTE
sqrbody: WRITE the two cop2-latencynops and VERIFY ON RAW BYTES — the assembler keeps - them; objdump (and
match_one) only HIDE them (false "stripped" diff):** for a handwritten GTE squared-distance - fn (
lwc2 $9/$10/$11, two latencynops,sqr 0,swc2 $25/$26/$27), write the whole GTE op as an __asm__ __volatile__body with the"nop\n" "nop\n"literally between the lastlwc2andsqr(`: : "r"(&in[0])- "$9","$10","$11","memory"
, store block pinning$2).mipsel-asdoes **NOT** strip those cop2 nops — they are in the object. The trap that made a prior session wrongly mark this whole family "not C-source reachable": **objdump ELIDES runs of zero words** (prints\t...for the two00000000nops), andmatch_onediffs via objdump → it reports a spurious mismatch (e.g. "22 mismatched") for bytes that are actually identical. **VERIFY GTE/cop2 fns on RAW bytes** (objcopy -O binary --only-section=.textthen compare words), never on the objdump listing. (Independently byte-confirmed here:+0x054/+0x058=00000000/00000000precede4AA00428/sqr 0in the built.o.) This unblocks the entire handwritten GTE-sqr` family (siblings func_8013E064/_8013E0FC/_8013E194/_8013E22C/_8013E298/ _8013E370/_8013E410). fixes the false "cop2 nops stripped" residual (an objdump zero-run-elision artifact, not a codegen miss); evidence func_8013E2C4 (43/43 raw bytes, banked + propagated ×13).
- **handwritten GTE
- store ONE value to several memory locations → write them as a CHAINED assignment
*a=*b=*c=v;, NOT separate statements: when the target copies a (callee-saved) value into a SCRATCH reg once (addu $v0,$sX,$zero) and thensh/sw $v0to several locations (often one store in a delay slot), the original wrote the stores chained — the chain's intermediate rvalue becomes that single reused scratch temp. Separate statements (*a=v; *b=v; *c=v;) instead store the source reg directly (nomove) and miss by ≥1 ins. (Distinct from §21'sfield=local=f(), which keeps a call result live for a test; here it's one value fanned out to N stores.) fixes a missingmove $v0,$sX+from-scratch store run; evidence func_80143D28 (*(s16*)(iVar3+0x18)=*(s16*)(iVar3+0x1a)=*(s16*)(iVar3+0x1c)=sVar4).
⚠ CANDIDATE (unverified) — the next two bullets are from
func_801775E0, a NEAR-MISS (closeness 2, NOT byte-banked), appended directly by a wave-13 drafter (that drafter→cookbook path is now blocked,commit:0219). They are plausible gcc-2.7.2 observations but were NOT confirmed by a byte-match. A drafter MAY try them — the byte-gate (G3/P9) is the sole arbiter, so a wrong idiom can never bank — but VERIFY before trusting. Drew to keep / refine / drop.
lbuvalue with SIGNED compares (bltz+slti, NOTsltu) → write each range bound as a SEPARATEif (…) gotostatement, never a chained||: when the target loads au8global withlbu(zero-extend → known 0..255) yet the comparisons are signed (bltz $v1;slti $v0,$v1,K) — including a provably-deadbltzon a 0..255 value — gcc-2.7.2 has emitted the conditions as INDEPENDENT signedsltbranches. A chainedif (v<0 || v>0xf6 || v<0xf3)CANONICALIZES: gcc provesv<0impossible (drops thebltz) and foldsv>0xf6 || v<0xf3into the unsigned range tricksltu $2,$v1,247/addiu -0xf3; sltiu 4(wrong:sltu, nobltz, fewer ins). Splitting intoif (v==0) goto a; if (v<0) goto b; if (v>=0xf7) goto b; if (v<0xf3) goto c;keeps each as its own signedslt+branch (thev<0becomes a realbltz, the bounds becomeslti), reproducing the target exactly. The launder/barrier tricks do NOT help (gcc re-derives the u8 range across an__asm__move); only the per-conditionif-gotosplit does. fixes the unsigned-range-collapse → signed-separate-compare divergence; evidence func_801775E0 (bltz;slti 0xF7;slti 0xF3).- shared join-block placement (call/store block reached by ≥2 paths) is steered by which exit the LAST range test
BRANCHES to vs FALLS THROUGH: gcc-2.7.2's jump pass lays the join block (e.g. a
jalreached by both an earlybeqand the range-chain fall-through) right after whichever predecessor it processes to fall through. Writing the chain's terminal asif (v>=K) goto join;(branch TO the join) puts the join AFTER the sibling block (matches a target whose join sits between the early-eq block and the tail); writingif (v<K) goto other; goto join;(branch AWAY, fall to join) FLIPS the entire layout (the early-eq test invertsbeqz↔bneztoo). The two are COUPLED — you cannot independently pick the terminal branch polarity AND the join placement; pick the form whose join placement matches and accept the terminal-branch polarity it implies (a residual the permuter can't touch underregister __asm__pins). evidence func_801775E0 (theif(v>=0xf3)goto callform gives correct [chain][eq-block][call][else] layout but leaves the terminal asbeqz→callwhere the target hasbnez→.L674; the negative form flips to a 50-mismatch layout).
§22 — DEF-side loose-typing recovery + grinder blacklist (Phase 21)
The wide self-MATCH → whole-binary-gate gap (a draft that match_one-MATCHes but the gate rejects) is dominated
by the DEF-side loose-typing wall, NOT cheap data plumbing (Phase-20 was right that data-cast is mostly moot —
the lone data exception is a D_x whose draft extern type disagrees with engine_core.h, fixed by rewriting the
draft extern to canonical and assigning through it). The DEF-side wall: the draft's byte-correct definition has
FEWER params than the canonical cross-overlay decl in engine_core.h (e.g. def void f(void) vs decl
void f(s32,s32,s32) because callers across overlays pass args the body ignores) → conflicting types for f in the
full TU → gate reject. Diagnose before building (R14): reproduce the whole-binary error (substitute the draft,
make build, grep error:/conflicting types) — harvest_verify discards it. The warnings at fixed line numbers
(incompatible pointer / built-in memcpy) are pre-existing and harmless; the real error is the conflicting types.
- Recovery = adopt the canonical param list on arity mismatch (
sig_unify.rewrite_def, Phase-21 fix). Previously it rewrote params only when arity MATCHED (return-type-only fallback otherwise — leaving the conflict). Now: when the canonical decl has MORE params, take the canonical param TYPES + COUNT, keep the draft's names where they exist, and synth_argNfor the unused extras. Unused params sit in$a0–$a3→ free at -O2 → byte-identical, conflict gone. Sincesig_unifyis in thegate_stagepipeline, this auto-recovers DEF-side near-misses on every wave + the grinder (compounding, like the Phase-20 type-lift). evidence: func_8016EDEC, func_8016EE40 banked via this. Still-hard residual (diminishing returns): narrow params (s8/s16/u8/u16/float) where gcc's default-promotion rule blocks the no-proto/wider escape (§15/§18), and multi-way loose typing (the same fn called with contradictory arities → no single ANSI prototype). - Grinder blacklist (
tools/grinder.py): a permuter "win" the whole-binary gate STILL rejects is plumbing-bound (the masked-0 doesn't survive the real link) — re-permuting can NEVER bank it. Record won-but-gate-rejected fns to a persistent.run/auto/grinder_blacklist.jsonand skip them forever (the daemon'stried.clear()-after-idle would otherwise churn the lowest-closeplumbing fns endlessly — it banked 0 in ~8h doing exactly that). Frees the permuter for genuine regalloc/schedule near-misses (the only class it can actually close).
§23 — Giant func_80153E00 cracked (scalar-data CAST); BUT most giant drafts are STALE+INCOMPLETE, and a diag MUST remove artifacts (Phase 21 — byte-proven + a self-correction)
func_80153E00 (195 ins, ×134) cracked — a clean 1-instruction near-miss. Substituting its draft compiles
AND links CLEAN and was 1/195 off. match_one over-predicted (compiles STANDALONE with the draft's own externs
AND masks jal/%hi/%lo) → it never saw the residual. Banked ×134 (fleet 62.27→62.31%).
⚠ DO NOT GENERALIZE "giants are near-misses" (a same-session R14 self-correction). After func_80153E00, I
diagnosed the other 14 MAIN-file giant drafts and a buggy diag reported 10 byte-MATCHes — a STALE-ARTIFACT
MIRAGE. The real byte-gate (harvest_verify, which os.removes the output first) banked 0 of 10: every
draft FAILS TO COMPILE. They are prior-wave drafts gone stale + incomplete vs the grown engine_core.h —
conflicting types for D_x (the draft declares u8 D_80126B58 vs canonical s32, u8 D_80126948 vs u8[]),
undeclared data symbols the draft never externs (D_800A5E8C, D_800B9A08 → link undefined reference),
and incomplete types. These are NOT bankable near-misses — they need RE-DRAFTING fresh against the current
canonical (a worker wave), not a cheap fix. func_80153E00 banked only because its draft was complete + clean.
⚠ THE STALE-.o TRAP BIT THE DIAGNOSTIC ITSELF (§20, the hard way). A per-function objdump diff that
builds WITHOUT removing build/src/<ov>.o + <ov>.elf first shows a FALSE byte-match when the spliced draft
fails to compile: make leaves the prior (stub) artifacts, so objdump disassembles the TARGET and reports
"0 diffs." This faked 10 giant MATCHes. A diag MUST os.remove the .o/.elf/binary before each build (then
a compile failure → empty disasm → honest "BUILD-FAIL", not a false match). .run/diag_funcdiff.py was fixed to
do this. The whole-binary SHA gate (harvest_verify) is the SOLE arbiter (G3/P9) — it removes the output, so
it was right while the diag lied. When a diag and the gate disagree, the gate wins; suspect a stale artifact.
THE DIAGNOSTIC (right tool, artifact-safe): .run/diag_funcdiff.py <fn> <draftdir> — build STUB →
objdump -d the fn (target); splice draft + build → objdump -d (candidate); diff, address column normalized
(s/80[0-9a-f]{6}/ADDR/). NOW removes artifacts before each build (else false match). Shows relocation vs
codegen vs type residual in seconds — but a 0-diff here is only trustworthy because of the artifact-removal +
a confirming harvest_verify. Run ONLY when no ov_SC01_077 build is in flight (concurrent builds clobber build/).
THE FIX for func_80153E00 — the scalar-data-signedness CAST (extends §22, corrects §20's "data-cast moot").
The 1 diff: target lhu D_8011DB0C (unsigned halfword) vs candidate lh (signed). The global D_8011DB0C is
declared extern s16 canonically (in engine_core.h + a banked DEFINE_* macro that writes it), but
func_80153E00 needs a u16 read (lhu). Two non-fixes: declaring the draft extern u16 D_8011DB0C; →
sig_unify reverts it to the canonical s16 (canonical wins, by design) → lh again; §22's "assign through
the canonical decl" → also lh (the canonical type IS s16). The fix is the DATA analog of cast_call_sites's
per-site cast: keep the canonical extern s16 decl, cast the READ site —
db0c = *(u16 *)&D_8011DB0C; /* canonical decl stays `extern s16`; gcc folds &sym+deref → a single lhu */
gcc-2.7.2 folds *(u16*)&sym to one lhu sym (no extra address insn), identical to declaring it u16 — but with
NO decl conflict, so it survives the gate pipeline (sig_unify leaves the canonical decl alone). Writes
(D_x = 0 → sh) are signedness-agnostic, so only READ sites need casting. This refutes §20's "data-cast is
MOOT" — that finding only checked extern struct/union conflicts; scalar signedness/width conflicts
(s16↔u16, and by extension s8↔u8 lb/lbu, s16↔s32 lh/lw) ARE real and recoverable by this
read-site cast. The same class is hinted across the other giants (e.g. func_80129CF8's note: "D_80126DB8
fields s32-store/(s16)-read, lh vs lw"). evidence: func_80153E00 — 1/195 → 0/195, banked ×134, fleet 62.27→62.31%.
TOOLING LEVER (probe-before-build, R14): if this scalar-data-conflict class recurs across the remaining
giants, build cast_data_sites.py (the data sibling of cast_call_sites.py): for each D_x read whose required
load width/signedness differs from the canonical decl, rewrite the read site → *(T*)&D_x (leave the decl
canonical; never cast a write). Folds into gate_stage → auto-recovers the class on every wave for ~0 tokens.
Gather evidence on ≥1–2 more giants first (func_80153E00 is N=1).
§24 — The ov_SC01_077_a.c split-file vein: split-aware propagation, but loose-typing-gated (Phase 21)
The opportunity (real): the Phase-19 -O0 work split ov_SC01_077 into 3 TUs (ov_SC01_077.c + _a.c +
_o0.c). The harvest pipeline defaults to src/{ov}/{ov}.c, so the 66+ fresh, cached, reach-134 fns in
ov_SC01_077_a.c were never waved — a tooling gap, not a difficulty wall. The main-.c reach-134 fuel is
byte-exhausted (1 fresh fn); _a.c is where the fresh fuel is. (tools/build_fuel_manifest.py + cross-ref the
3 split files' stub sets to enumerate it; .run/diag_a.py is the artifact-safe per-fn diff for _a.c.)
What's BUILT (validated):
dedup_propagate.pyis split-aware —overlay_files(ov)returns[(main.c, ov), (_a.c, ov_a), (_o0.c, ov_o0)]; source def-finding scans all (source_text), the member loop edits whichever split file holds each target's stub/def (per-file asm-subdir regex), the structural check spans all files. Single-file overlays are unchanged (default path). Validated: source-find locates_a.cdefs (--check-only→ 134 members); fail-path reverts correctly. (Success-path ×134 of an_afn still pending a clean propagatable+matchable_afn — the wave.)cast_call_sites.py --src-file <file>— canonicalize callee decls against the file the draft LANDS in, not main.c. Why it's needed: a callee can be declared differently in main vs_a.c(cross-file loose typing, e.g.RotTransSV), so canonicalizing against main injects a decl that conflicts with_a.c's. Pass--src-file src/ov_SC01_077/ov_SC01_077_a.cfor_adrafts. (Default = main, unchanged.)
The WALL (honest — the _a vein is NOT a clean win): propagation/matching of _a fns hits the SAME §16/§20
loose-typing wall as everywhere, in fresh form:
- Cross-overlay def-conflict → ×1, not ×134.
func_8012C098matched in_a.c(cast-recovered: a calleefunc_8012C218(void*)called 0-arg via((void(*)(void))…)()), but a banked caller in another overlay (engine_core.h DEFINE) declares itextern void func_8012C098(void)while the body USESparam_1→ the propagated macro's def conflicts there →dedup_propagate(all-or-nothing) reverts. Irreducible (no single C sig fits a param-using body + a 0-arg caller decl). Banks ×1 only. - Within-
_aloose-typed callee.func_8012F274(decls=0, would propagate freely) won't even match:RotTransSVis declared inconsistently within_a.c→ any single extern conflicts; needs the per-site cast, whichcast_call_sitesonly applies when the canonical differs from the draft's intent (here it picked one and it still clashed with another site). A genuine multi-sig callee. sig_unifystill drops_afns (itscur_stubs/decls read main.conly) → for_awaves either give it the same--src-file/split-awareness or runcanon → cast --src-file → gate(skip sig_unify; the cast class banks, the def-side class is a wall anyway). NOT yet done.
Net / next: the enabler tooling is built + safe (fail-closed); the _a vein is matchable but its ×134 yield
is loose-typing-limited (unknown fraction are ×1 walls). The proper measurement is a worker wave over the
_a pool (parallel agents match bodies; the byte-gate + split-aware propagate sort ×134 vs ×1) — needs
wave_targets _a support + sig_unify _a-awareness (or skip it). Decide whether the uncertain yield
justifies the wave vs pivoting (the wall is the same as the main vein's).
UPDATE (cont.6): sig_unify --src-file is now BUILT (closes the open item above). sig_unify.py --src-file src/ov_SC01_077/ov_SC01_077_a.c reads cur_stubs + inline/extern canonical sigs from the split file (not the
main .c), so _a/_o0 drafts are no longer dropped and DO get the def-side arity-adopt recovery. Validated:
21 _a drafts unified WITH the flag vs 0 without; the _a close=0 recovery wave then banked func_8012F568 ×134
(commit:0277). gate_stage passes --src-file to BOTH cast_call_sites and sig_unify now. See §25 for the
canon-first two-stage gate that makes this safe (sig_unify must be a FALLBACK, not unconditional).
§25 — The "schedule" class is mostly COALESCING (pin-crackable), not scheduling; + the gate two-stage + h_exact over-counts ×134 (Phase 21, cont.6)
idiom_loop.py --assess named schedule (50 "reach-134", median 15 ins off) as the next idiom. Cracking its
lowest-closeness reach-134 exemplar func_80128ED8 (close=3) surfaced three durable lessons.
The crack: the residual was gcc-2.7.2 COPY-COALESCING, not the instruction scheduler
match_one diff (3-off): MINE move $a3,$v1 / sll $v0,$a3,3 / addu $v0,$v0,$t0 vs TARGET sll $v0,$v1,3 / addu $v0,$v0,$t0 / addu $a3,$v1,$zero. The "schedule" label was misleading: gcc COALESCED the index-preserve
copy (idc = idx) into the multiply operand (sll on the copy $a3) instead of multiplying $v1 directly,
and routed idc+1 back through the copy's reg. Two §17 register pins fixed it (byte-gated, match_one
MATCH 53/53):
- Pin the preserve-copy to its TARGET register (
register s32 idc __asm__("$7")=$a3): gcc can no longer fold it into the multiply operand → the multiply uses the original$v1directly and the copy emits SEPARATELY after it. (3-off → 2-off.) - Route the copy's dependent arithmetic through a dead, target-scratch-pinned temp (reuse an already-dead
register __asm__("$2")=$v0pseudo:cnt = idc + 1; store cnt;) → the result lands in$v0(the target's scratch), not back in the copy's reg. (2-off → MATCH.) Triage rule: on a small "schedule" residual, read thematch_onediff FIRST. A copy emitted before its source's other use, with that use reading the copy's register = COALESCING → pins (cookbook §17), crackable. Don't assume the scheduler.
The genuine scheduler — rank_for_schedule (sched.c), for when it IS scheduling
gcc-2.7.2's ready-list tie-break order (sched.c, byte-read R17): (1) PRIORITY = dependency-chain height to
end-of-bb (longest chain first); (2) CLASS vs last_scheduled_insn — prefer class 3 (independent / latency-1)
over class 1 (data-dependent on the last insn) — i.e. gcc fills an address-gen→load gap with an INDEPENDENT insn
(this EXPLAINS the "copy fills the slot before the lw" schedules the targets show); (3) LUID = original
SOURCE ORDER (the stable final tie-break). LEVER for the genuine equal-priority case: reorder the SOURCE
statements (the LUID tie-break — same family as §10's operand/statement-order idioms). When priorities differ or
coalescing intervenes, source-reorder alone won't flip it → use the pins above.
The genuine schedule WALLS (do NOT re-grind — stub)
- §10 cross-jump / delay-slot merge (func_8014FD54, close=2): two
return 0paths — the target keeps them SEPARATE (one fills the innerbeqzdelay slot withmove $v0,0, one is a standalone zero block); our cc1 cross-jump-MERGES them → the innerbeqzgets a NOP delay slot. The §5a:::"memory"barrier breaks the merge but overshoots +1 (41 vs 40). No C-source form reaches the merged-with-delay-slot-fill schedule. (§10 Residual-B.) - store-vs-load placement (func_8014F2E0/func_80150528, §20-confirmed): the store schedules between two arg-loads, mutually exclusive with base-preservation. Stub.
h_exact OVER-COUNTS ×134 — verify shareability before crediting a class's "reach-134" (R14)
func_80128ED8's crack is byte-identical in ov_SC01_077 but does NOT propagate ×134 — dedup_propagate --addr 0x80128ED8 (alone, no stragglers) still [drop]s it at ov_SC01_000: a cross-overlay byte-gate reject. So it
banks ×1, despite sig_image h_exact reporting members=134. Why: h_exact is RELOCATION-MASKED (the
%hi/%lo of unresolved syms are 0 in the object), so it matches across overlays that the shared-C macro then
can't reproduce byte-identically (overlay-local data/decl differences — the §24/§20 wall). Consequence for the
idiom-loop model: --assess's per-class "reach-134" count (from the backlog's reach field = h_exact) is
OPTIMISTIC; a cracked fn's real leverage can be ×1. Before committing a token-heavy wave to a class, probe
×134-shareability on the cracked exemplar (dedup_propagate --addr <fn>), not just the h_exact count. The
"schedule" class is therefore NOT a confirmed ×134 vein — its closest reach-134 exemplars are one ×1-coalescing
crack + one §10 wall.
The gate two-stage — sig_unify is a FALLBACK, not unconditional (gate_stage.py, §19 folded in)
gate_stage ran canon→cast→sig_unify in ONE pass. sig_unify REGRESSED the func_80128ED8 crack: it rewrote the
byte-correct def s32 f(s32,s32*) → a banked caller's canonical void* f(void*,void*) → gate reject (the raw
draft banked fine via harvest_verify). Fix (the §19 "canon-first" design, now IN gate_stage): stage 1 =
canon+cast → byte-gate (already-correct drafts, incl. hand-pinned cracks, bank here); stage 2 = sig_unify
ONLY the stage-1 failures → re-gate (def-side near-misses recover) — never regressing a stage-1 winner.
harvest_verify reads the CURRENT src as baseline, so verified fns ACCUMULATE across the two gate calls (a
stage-1 winner is no longer a stub for stage 2). This is mandatory now that hand-pinned self-contained cracks
flow through the same gate as recovery drafts.
§26 — The cheap close=0 recovery lever is EXHAUSTED; idiom_loop --assess was DOUBLY inflated (Phase 21, cont.7)
cont.6 left two "do this FIRST each cycle" cheap levers: the _a close=0 recovery (validated on one fn,
func_8012F568 ×134) and idiom_loop --assess's "51 close=0 reach-134 fns → ×134 for ~0 tokens." cont.7 ran the
_a lever to completion and probed the assess. Both were over-promises; the cheap recovery fuel is dry.
The _a close=0 recovery banks 0/20 — same def-side wall as MAIN (0/40)
Re-gated the 20 still-stubbed _a/o0 close=0 reach-134 fns through the cont.6 fixed pipeline (gate_stage --src-file, canon-first two-stage; sig_unify --src-file fired on all 20). banked 0. match_one calls them all
"MATCH" (close=0) but the whole-binary gate rejects every one — the §16/§20 DEF-side multi-way loose-typing
wall: the draft's byte-correct def needs a C type that conflicts with a banked caller's canonical decl, and no
single C sig fits both (sig_unify's arity-adopt only fixes the simple-arity case — cont.6's func_8012F568 was
that lone case; the rest are genuine multi-type conflicts). Caller-side fix is blocked (INCLUDE_ASM declares no
symbol, §20). Conclusion: close=0 + already-recovery-gated = a WALL, not fuel — do NOT re-run recovery on it.
--assess was inflated TWO ways (now fixed) — verify reach AND closeness before crediting a lever
The assess named "51 close=0 reach-134 → ×134 for ~0 tokens." Two byte-proven inflations (R14/§25 family):
reach= maskedh_exactdistinct-overlay count → OVER-counts the real ×134 (§25: func_80128ED8's--check-onlyplan saysmembers=134but real propagation is ×1). The "reach-134" label is a CEILING.load_backlogdidn't drop banked-since-logged fns (the ledger keeps stale statusnearfor a fn matched in a later session — e.g. func_8016EDEC/EE40 banked in cont.6's Option-C still counted). And the close=0 "lever" counted fns already recovery-gated-and-failed (the §16/§20 wall) as if fresh. Fix (tools/idiom_loop.py, cont.7 — R16 flywheel, so the next session doesn't re-burn the lever):
load_backlognow intersects the ledger with the liveINCLUDE_ASMstub set (_open_stubs(), mirrorsbacklog._matched_now) → drops every banked-since-logged fn (the same drop-now-matched P9 honestybacklog.renderapplies). This alone removed the bogus "unknown 14" class + ~10/class of stale-matched inflation.- The DETERMINISTIC-RECOVERY line now splits FRESH (never recovery-gated
recover-*source = genuine ~0-token fuel) vs WALLED (already recovery-failed = the def-side wall, "do NOT re-run"). Post-fix: 0 fresh, 46 walled. The cheap lever is genuinely empty. RULE: before trusting an assess class/lever, the live numbers are: real reach =dedup_propagate --addr --check-onlyis still h_exact (a ceiling) → the only truth is the gate; real closeness = re-measured through the gate's canon/cast/sig_unify transforms (the stored ledger closeness is optimistic — stale records read 0 where the live residual is 30–41). h_exact over-counts the numerator, stale-ledger under-counts the denominator.
Where this leaves the reach-134 tail (cont.6 option-3, now CONFIRMED byte-backed)
The cached reach-134 cheap fuel is dry: close=0 recovery = 0 fresh (46 walls); the codegen classes that look
tractable are h_exact-inflated (schedule "39 reach-134" but the cracked-exemplar ×134 fraction is ~50% on n=2 —
func_8012F568 ×134 / func_80128ED8 ×1 — at median 15 ins off = expensive per-fn pin work, poor ROI; do NOT wave
it on the inflated count, §20 "don't wave a wall"). The remaining levers are all token-heavy fresh-session
work: (a) the GIANTS (8 reach-134 >150 ins — the byte-weight lever, ~3% auto-yield so mostly hand-finish/backlog
fuel, Phase-16); (b) per-fn pin-cracking the genuine codegen near-misses (schedule-coalescing §25 / regalloc-order).
REFUTED — the "resolved-reach probe" (cont.7b, R14 self-correction): I proposed comparing per-overlay
linked .text to get a real ×134 count. It doesn't work, because sig_image's h_exact is ALREADY
SHA1(raw image bytes) — UNMASKED (sig_image.py:170-171; h_norm is the masked tier). So reach is the
ACCURATE shipped-byte reach; a resolved-byte probe would just reproduce it. The §25 "h_exact over-counts" wording
conflated two different hashes — the IMAGE h_exact (accurate) vs the match_one/draft OBJECT masking. The real
gap is reach (shipped bytes identical) ≥ realizable-×134 (shared-C macro reproduces all N): func_80128ED8's
shipped bytes ARE identical ×134, but its matched C can't propagate (a callee/decl/loose-typing wall in the
shared-C representation, §16/§20/§24). That gap is measurable ONLY by dedup_propagate's byte-gate on an
ALREADY-MATCHED fn — there is no cheap static probe. Don't build the resolved-reach tool.
The close-N tail can be PERMUTER-achieved, not source-close (cont.7b, R14): the backlog closeness is the
BEST achieved (often by the grinder's randomized regalloc/scheduling permutation), NOT the saved draft's
source-compile distance. func_801775E0 logs close=1 but its saved best_draft compiles 31-off from source
(prologue save-SCHEDULING, branch-sense, block-order all differ — the §20 regalloc/schedule wall); the close=1 was
a permutation the source doesn't capture. RULE: before assuming a cheap source crack on a close-1..4 backlog fn,
match_one its saved draft to get the SOURCE closeness — if it's far-off, it's a permuter-class wall (grinder
territory), not a pin target. The genuinely pin-crackable ones are source-close + a single coalescing/CSE residual
(§25 func_80128ED8); the regalloc/save-schedule/cross-jump ones are the confirmed walls (§20/§25).
§27 — Giant matching recipe (Phase 21 cont.7b — validated on func_80176D94, 152 ins)
Giants (reach-134 >150 ins) are the byte-weight lever. Validated approach (func_80176D94 → structurally matched, calls/constants/GPU-packet all byte-correct, residual = pure regalloc):
- Start from the cached Ghidra-C (
.run/ghidra_c/<fn>.c) — giants are often CLEAN (straight-line + many calls, no deep control flow), so the body structure comes nearly free. (cont.2's "stale+incomplete drafts" were the OLD m2c drafts vs the grown header; the fresh Ghidra-C + canonical context is the fix.) - Arg-arity is the #1 giant blocker (the manifest's
MCOMPILE_arg-aritybucket). Giants call many helpers; m2c/Ghidra miscount args. Declare each callee to match the ACTUAL call site — count the$a0–$a3(+ stack) set before eachjal, NOT the canonical sig. e.g.func_80177784canonical is 4-arg but called 3-arg here ($a3untouched) → declarefunc_80177784(void*, s32, s32). (The shared-header canonical conflict is the gate's job —cast_call_sites; it may bank ×1 if irreducible, §24.) - Sibling templates. Giants cluster in families (GPU-packet builders, coord transforms). Find an
already-matched sibling
DEFINE_func_*inengine_core.hwith the same idiom and mirror its PROVEN C form — e.g. the GPU-coord strength-reduce((s32)(D * 10355) << 1) >> 16(sibling func_80176FF4) and the GPU linked-list pointer((u32)addr & 0xFFFFFF) | 0x3000000. - §17 register pins for the regalloc-SHIFT. Giants use 5–7 callee regs; gcc's mapping often shifts whole-hog
(param→$s0 where target uses $s2, etc.). Pin the long-lived vars to their target regs
(
register T v __asm__("$NN")). CAVEAT (cont.7b, byte-proven): do NOT pin a var whose register the target REUSES for a later spill — pinning reserves the reg for that var's whole scope → blocks the reuse → gcc grabs a FRESH callee reg (+1 reg, +2 prologue ins). (func_80176D94: pinning uVar5→$s0 blocked the accumulator from reusing dead $s0.) - The giant residual class — accumulator-spill coalescing. A value chained through
$v0→$a0across calls, then needing to survive a LATER call, must spill to a callee reg. The target reuses a now-DEAD callee reg; gcc from natural C grabs a FRESH one (+1 reg). Hard to force from C (it's gcc's coalescing graph) → permuter fuel (the grinder randomizes allocation and may find the reuse). func_80176D94's saved draft (.run/backlog_drafts/) is exactly this — structurally done, 1–2 callee regs from byte-perfect. Net: giants reach STRUCTURALLY-MATCHED fast (steps 1-3); the last mile is regalloc-coalescing (step 5) — so giants are ISOLATED-AGENT + permuter work (perbreadth-isolated-agents-not-serial), not main-loop serial grind.
WAVE RESULT (cont.7d — the honest yield, R14/P9): a 6-giant worker_wave banked 0. The agents reached the SHAPE (the §27 recipe works — bodies/calls/constants/control-flow right) but the whole-binary byte-gate rejected all 6: 1 was close=0 match_one-MATCH yet gate-REJECTED (func_8014F74C — no callee/data conflict, so match_one OVER-predicted: the masked standalone match hid a real residual, §25), 1 wouldn't compile whole-binary (func_80144090, agent-claimed "154/154" but a decl the gate couldn't reconcile), and 4 were genuinely far (close 53–164 — the agents' "structural match" self-claims were match_one-optimistic). This confirms cont.2 / Phase-20 / Phase-16 (~3% wall): giants do NOT auto-bank — even with the §27 recipe + isolated agents, the whole-binary gate (masked-residual + plumbing + large regalloc) blocks them. Giants are HAND-FINISH / backlog fuel (the wave's deliverable is 6 RANKED near-miss drafts for human sessions, NOT banks). Do not scale giant auto-waves expecting %; the automated reach-134 harvest is COMPLETE at this fleet level (every automated lever — cheap recovery, permuter, giant auto-wave — banks ~0). Forward % = hand-finishing (Drew) or closing the phase.
Frame-pressure-locked residuals are PERMUTER-ONLY (cont.7d, byte-proven on func_8014EA4C, close=6). When a
giant's matched body needs a frame-forcing HACK to hit the right stack size — e.g. a DEAD u8 buf[16]; memcpy(buf+16, src, 8) (an out-of-bounds copy whose only job is to force frame 0x60) — its remaining schedule/regalloc residuals
become PRESSURE-LOCKED: any C edit that would steer them (materialize a temp to reorder a store/load; a ternary or
s32 retype of an abs; a register pin) shifts register pressure → gcc DROPS the dead buffer → frame shrinks
(0x60→0x58) → whole-function offset cascade (close 6 → 23–123). Two hand attempts both cascaded exactly as the
drafter predicted. The fix is NOT C-steering — it's the decomp-permuter (the grinder): it randomizes
regalloc/scheduling via semantics-preserving C perturbations the byte-gate scores, exploring the frame-PRESERVING
space the hand-edits can't. Re-log such giants with their true closeness + raw draft (source=giant-raw) so the
grinder (close≤30) picks them up; do NOT hand-grind them.
Deep frame-RE does NOT crack a scheduler-walled giant (cont.7d, byte-proven on func_8014EA4C). Drew chose
"deep re-RE the real frame so regalloc resolves." The RE finding: the frame is a dead aggregate copy — 8 bytes
of a param unaligned-copied (lwl/lwr→swl/swr, char-aligned dest) to a stack local at sp+0x20 that's NEVER read
(gcc-2.7.2 keeps it — no DSE for aggregates). The drafter's u8 buf[16]; memcpy(buf+16, a2, 8) (copy PAST the
array) is a PRECISE reproduction of the exact gcc stack layout — local[0x38] or any "robust" remodel produces a
DIFFERENT frame (0x88, copy at 0x30). So the frame is already correct at close=6; the residuals are NOT
frame-caused — they're loop-body scheduler (a global store vs a call-arg load order) + regalloc (abs in $v1 vs
in-place $v0), the irreducible §20/§25 wall. Lesson: when a giant's residual is loop-body schedule/regalloc,
deep frame-RE is a dead end — only the permuter explores that space. Net for Phase 21: 5 levers
(cheap-recovery, permuter, giant-wave, hand-finish, deep-RE) are byte-proven exhausted at fleet 63.17% for the
reach-134 tail; the residual is the gcc-2.7.2 scheduler/regalloc wall, addressable only by the (low-yield) permuter
or by accepting it as the matching ceiling at this fleet level.
§28 — Banking a "close=0 gate-rejected" giant: the canonical-extern recovery (Phase 22 T1, byte-proven on func_8015126C ×134)
Phase-21 (§25/§27 cont.7d) logged a class of giants as "close=0 (match_one MATCH) but gate-REJECTED" and treated them ALL as masked-residual/permuter fuel. That was incomplete (R14). A close=0-gate-rejected giant is one of FOUR distinct walls — and one is deterministically bankable BY HAND:
- PURE-EXTERN PLUMBING (bankable — the NEW lever). The draft's self-contained file-scope externs conflict with engine_core.h's canonical decls (the giant calls already-matched engine fns; the draft guessed their sigs, e.g.
extern void func_8015173C(void*)vs canonicalvoid func_8015173C(s32*)). match_one's isolated compile (own externs) MATCHes; the whole-binary TU fails to compile (conflicting types for func_X).sig_unify/cast_call_sitesdo NOT canonicalize these (the gap that stranded them). Fix =tools/recover_giant.py: for each callee with aDEFINE_func_X()in engine_core.h, rewrite the draft's extern to that macro's exact def-sig; then move ALL externs (callee +D_data) block-scope (inside the body, after{).find_site/compiles_standalone/dedup_propagatelift the body + its internal externs as one unit — file-scope externs are excluded from the lifted body → false "not self-contained" skip. Forward-refs (higher-addr callees) MUST stay declared. Thenharvest_verify→dedup_propagate×134. - MASKED RESIDUAL (permuter fuel). Compiles whole-binary but bytes differ — the relocation-mask hid a real codegen diff (func_8014F74C, §25). The recovery won't help → grinder.
- STRUCT-WALLED (type reconciliation). The draft uses a local named
struct S8/B8(the array-of-struct %lo idiom, §18) that collides with the TU's other defs (conflicting types for S8) — func_80156B74/func_8014F74C/func_80163C2C. Needs the type lifted toengine_types.h(named once) or rewritten anonymous/raw-cast. Not near-free. - REGRESSED DRAFT. The saved best_draft was clobbered by a later worse attempt (func_80178004: worklist close=0 but saved draft is DIFF 91). Re-derive.
Triage: recover_giant.py → match_one (DIFF → #4) → whole-binary build. byte-identical → #1 banked · conflicting types for func_X → was-#1, recovery fixes · conflicting types for <Type> → #3 · compiles but bytes differ → #2. Honest yield: of the 5 close=0 giants, only func_8015126C was #1 (pure-extern); the rest are #2/#3/#4 — the canonical-extern lever is real but the close=0-giant group is NOT uniformly near-free.
The coalescing pin (extends §25 — func_8015126C's last mile). The (s16)p[0x79] != 1000 compare temp wanted $a0 (coalesced with the soon-to-be-angle arg); gcc gave $v1. An eager named temp forces $a0 but HOISTS the load (+95 ins). Fix — pin AND keep the load lazy inside the &&:
register s32 cmp __asm__("$4"); /* $a0 */
if (cond1 && ((cmp = *(s16*)(p + 0x79)) != 1000)) { ... }
The in-&& assignment stays inside the short-circuit (not hoisted) yet lands in the pinned reg. Reusable for any coalesced-compare-temp residual.
Op gotchas (Phase 22 T1): run dedup_propagate in the BACKGROUND (134 builds > the 2-min foreground cap; a SIGTERM/interrupt leaves a non-atomic partial state — macro+instantiations applied, registry unwritten). And git checkout src/ does NOT revert config/dedup.us.yaml, so on a redo reset BOTH (git checkout src/ config/dedup.us.yaml) or registered_addrs() stays dirty and re-skips the function as "already shared."
§28a — decomp.wiki GCC patterns worth trying on BFM giants (decomp.wiki/compilers/GCC, raw at decompals/decompedia; PS1-applicable subset)
- Negative struct offsets in loops —
for (i=…; …; i++, p++)makes gcc advance the pointer + use negative member offsets instead of offset-folding. Directly targets func_801412A8's "OFFSET-FOLDS the 4× prim stores instead of ADVANCING $t6" residual. - Branch-invariant code duplication — when gcc hoists a shared tail (a call) out of two branches and swaps regs, duplicating that tail inside both branches fixes the regalloc. An alternative to §17 pins for the call-crossing swap class.
- Load coalescing — adjacent struct fields compared together (
if (t->a || t->b)) fold into onelw(+lui/ori/and mask when <4 bytes). Recognition aid for engine code. - div-by-constant magic table (0x66666667→/10, 0x55555556→/3, …) + s16/s8 div-by-2 sign-extension forms; gcc-2.7.2.x
slti …,0⇒(x & (1<<31)) != 0. - N/A to PS1 (don't chase):
bnel/likely branches (MIPS II+; R3000 has none),.lit4float-literal NOPs (PS2), C++boolload/store (BFM is C).
§28c — The close=0 recovery is NOT fully exhausted (§26 corrected, R14); + the dedup_propagate registry-skip recovery (Phase 22 T2)
§26 said "the cheap close=0 recovery lever is EXHAUSTED." That was over-broad (R14). Re-running recover_giant.py (canonical-extern + block-scope) + the whole-binary gate over the still-live close=0 reach-134 set (match_one MATCH, gate-rejected) banks a real ~15-20% tail the prior waves left — Phase 22 T2 banked 7 this way (func_80156ECC, func_80147E44, func_8015ADB0, func_801661CC, func_80166054, func_8012CFA8, func_8012A62C) out of ~43 candidates. The rest (~33) ARE the genuine DEF-side loose-typing / masked-residual wall (§20/§26 holds for THEM). Two enablers this session: (a) the §28b type-lift put more shared types in engine_types.h (so compiles_standalone passes for more bodies); (b) dedup_propagate --auto-from then propagates not just the new banks but pre-existing inline-matched-but-never-propagated functions for free (7 bonus this session — always run --auto-from after a harvest_verify batch to sweep them). Recipe: survey live close=0 reach-134 (fuel_manifest.targets ∩ backlog closeness==0) → recover_giant each (strip local struct typedefs first; lift shared ones per §28b) → batch harvest_verify --chunk N per region (main / _a, distinct --src/--asm-subdir) → dedup_propagate --auto-from → make check-all (the sole arbiter).
The dedup_propagate registry-skip + its recovery (byte-proven, recurring). A full dedup_propagate run (long byte-gate loop = 134 builds) sometimes ends with the source propagated (DEFINE macro + ×134 instantiations) but config/dedup.us.yaml partially/un-written (the long loop appears to get killed before/within the register step; the run still reports exit 0). Symptoms: check-all 136/136 (bytes correct) but dedup-check/progress.py under-count (group absent). Recovery (do NOT revert — the bytes are right): the tool can't re-register an already-propagated fn (--auto-from needs inline-def; --addr needs a non-macro site), so register directly via its own helpers — for each addr: h = load_sig('ov_SC01_077')[addr]['h_exact'] (stable: bytes unchanged), members = [ov for ov in onboarded_overlays() if load_sig(ov).get(addr,{}).get('h_exact')==h], then append_groups([dict(id=f"E_{sym(addr)}", tier='h_exact', hash=h, source='src/shared/engine_core.h', addr=addr, members=[{'binary':ov} for ov in members])]) (idempotent by id — safe to re-run). Verify with dedup_integrate.py --check. Prevention TODO: make dedup_propagate write the registry BEFORE the byte-gate loop (or add a --register-only mode for already-propagated addrs).
§28b — The struct-walled close=0 giant (§28 case #3) IS bankable: the engine_types.h type-lift (Phase 22 T2, byte-proven on func_80156B74 ×134)
§28 case #3 ("STRUCT-WALLED — needs the type lifted… Not near-free") was too pessimistic — it IS near-free once you do the lift, and the lift unblocks the whole class (every giant sharing those types, including already-banked-but-LOCAL ones via the §19 propagation cap). Recipe (byte-proven on func_80156B74, 214 ins, ×134, fleet 63.22→63.25%):
- Identify the draft's named struct/typedef types (e.g.
typedef struct {s16 a,b,c,d;} S8;,B8,Blk16,Buf32). These are why the draft was struct-walled: a close=0 draft that re-typedefs a name already in its TU = C89 duplicate-typedef →conflicting types; and a body containing atypedef/named-struct can't be lifted (dedup_propagateskips it, line ~366). - Lift the typedefs to
src/shared/engine_types.h(defined ONCE; reaches all 136 binaries viaengine_core.h's include). Gotchas: a typedef NAME (S8, ordinary-identifier namespace) legally coexists with an unrelated struct TAG (struct S8, tag namespace) — gcc-2.7.2 accepts both; check for a layout-identical existing type first (e.g.B8≡ the pre-existingBlk8) and keep the draft's name to avoid churning already-matched siblings. Remove the now-duplicate file-scope typedefs fromov_SC01_077.c(the per-function prelude block) or the lifted def collides. - Verify the lift is BYTE-NEUTRAL before touching the giant (R22): rebuild the binaries whose already-matched fns use those types (the prelude's owner fn, e.g. func_80156848) + 1 overlay + resident → all must stay byte-identical. Typedefs emit no code; only a name collision can bite.
- Strip the typedefs (+ any
#include) from the draft →tools/recover_giant.py(it canonicalizes the engine_core.h-callee externs + block-scopes ALL externs, so the body is self-contained for ×134 with its types coming from the shared header). recover_giant strips only SCALAR typedefs — remove the struct ones yourself first. - match_one caveat: match_one compiles standalone WITHOUT engine_types.h, so the recovered (typedef-free) draft won't compile there — that's a harness limitation, not a defect. Confirm the MATCH on the ORIGINAL self-contained draft (with its own typedefs); gate the recovered draft with the whole-binary
harvest_verify(which includes engine_types.h via engine_core.h — the real arbiter, G3/P9). Thendedup_propagate --addr 0x… --source-overlay ov_SC01_077(BACKGROUND, §28). - Op gotcha (Phase 22 T2, byte-proven): do NOT run a second
make-invoking job (make check-all,make report) CONCURRENTLY withdedup_propagate— parallel make corrupted a partial.o(file format not recognizedon ov_SC04_016, caught by check-all + cleared by a clean rebuild). Serialize all make jobs. NOTE (corrected — §28c): the propagation's registry-skip (source propagated butconfig/dedup.us.yamlgroup unwritten) is a SEPARATE recurring issue independent of concurrency (it recurred with NO concurrent make on func_80163C2C) — see §28c for the byte-proven recovery; re-runningdedup_propagatedoes NOT re-register an already-propagated fn.
§28d — The "macro-extern-injection" lever: freeing reach-134 inline matches dedup_propagate skips as "not self-contained" (Phase 23, tools/inject_capped_externs.py)
The Phase-20 backlog flagged a set of reach-134 functions matched INLINE in ov_SC01_077 but never propagated (dedup_propagate --auto-from reports "N not self-contained (local types)") — a pure propagation cap, not a codegen wall (their bytes are already byte-correct in 077). Phase 23 surveyed them: 29 such reach-134 fns in 077 (main + _a). Root cause (byte-diagnosed by running each lifted body through compiles_standalone = common.h + engine_types.h): find_site lifts only the function DEF (+ contiguous preceding externs), but 077.c declares the callees/data at FILE scope, so the lifted macro body has undeclared func_X/D_X in every other overlay. The classes (run the diagnosis — they are NOT uniform, R14):
- callee/data extern-undeclared (the clean majority) — inject the EXACT file-scope
extern …;the overlay already declares for each referenced symbol, BLOCK-scope (just inside{). Byte-neutral (block vs file scope = same codegen; the gate proves 077 staysd19c9580…).compiles_standalonethen passes →dedup_propagate --auto-fromlifts it ×reach.tools/inject_capped_externs.pydoes exactly this, fail-safe (only rewrites a fn if the injected body thencompiles_standalone; never churns one that still won't propagate). Phase-23 result: 8 freed fromov_SC01_077.cmain (each reach-134), 077 byte-identical, propagated ×134. - split-file (
_a.c/_o0.c) — same fix, run the tool with--src-file src/ov_SC01_077/ov_SC01_077_a.c(10 of the 29 live there). (_o0.cdefs are overlay-LOCAL, §18/§20 — never auto-propagate.) - missing-extern — the referenced callee has NO file-scope extern in 077.c (077 builds via gcc's implicit declaration); inject a no-proto
extern <ret> func_X();(the call-site cast gives the shape) — a tool extension, lower yield. - type-walled — the body references a struct/typedef not in
engine_types.h(Buf,Vec3,Loc, a PsyQMATRIX); needs the §28bbuild_engine_types.pytype-lift first (and the type must actually be inline-defined in 077.c — a PsyQ type needs its header, not a lift). Pipeline:inject_capped_externs --apply→make build BINARY=ov_SC01_077(gated19c9580…, else revert) →dedup_propagate --auto-from ov_SC01_077 --min-reach 2(BACKGROUND, serialize make, §28). The whole-binary byte-gate is the sole arbiter (G3/P9): a wrong injection can't pass. The lever compounds (R16): every future LLM/agent inline bank of a shared fn that references file-scope symbols is freed the same way — run the injector before propagating.
§29 — Reasoning-model (GLM5.2) DEF-side reconciliation idioms + the wall's hard limit (Phase 23 T10.7, tools/glm_reconcile.py)
The T10.7 OpenRouter A/B found GLM5.2 is ~10× better than v3 at hard-band CODEGEN (10/18 vs 1/18 match_one on 16–22-ins ov_SC01_077 struct-core fns) — but the DEF-side loose-typing wall (§20, Phase 16/20) caps banking at 4/18 regardless of drafter, because the conflict is overlay-forward-decl-vs-the-fn's-true-signature, independent of who writes the body. Aiming GLM's REASONING directly at the wall (glm_reconcile.py: body + the conflicting TU decls + this toolkit → a consistent byte-identical decl set; reasoning captured to .run/glm_reason/ for R16) banked only 1/7 stranded; mechanical fix_arity_callers --any-proto 0/7. Verdict: the wall is INTRINSIC — a frontier reasoning model with the full toolkit cracks ~1/7. GLM's reasoning is nonetheless expert-grade and the durable value; the byte-neutral reconciliation idioms it articulated (validated by the gate):
- Match a pointer PARAM's type to the TU's existing forward-decl (
void*def-param →s32*when the TU declaresfunc_X(s32*)): both 32-bit, body casts work identically, byte-neutral — resolvesconflicting typeswithout relaxing anything (bankedfunc_80175184). - Call-site cast for a value mismatch: passing an
s32*to afunc_Y(s32)→func_Y((s32)arg0)— same 32-bit value in$a0, gcc emits NO conversion, byte-neutral. - Cast a callee that is a DEFINITION (not a forward-decl, so un-relaxable):
((R(*)(A))func_Z)(args)— the §17a-1 cast idiom, which GLM derived independently. - Match a DATA extern's type to the TU's (
intvsu32at the same width → identical loads/stores; pick the declared one). - The HARD limit (why 6/7 fail): a def with a narrow-scalar param by value (
u16/s16/u8/s8/char/short/float) can't be no-proto-relaxed (K&R default-promotion changes the ABI) AND often can't match the TU's incompatible prototype — this is the irreducible narrow-param wall (§17-stop); GLM correctly diagnoses it (u16params emitsh; widening tos32would emitsw) but cannot dissolve it. Strategic read: GLM's role is (a) a $0.03–0.08/fn direct drafter for the def-conflict-FREE hard band (~22%, which v3 can't touch), and (b) an idiom TEACHER (capturereasoning, distill here + into corpus) — NOT a wall-breaker. The real lever past the wall is community labor (the public flip, Fable5 review §4.3), not a bigger model. ← §30 CORRECTS THIS for a model that reads the gcc SOURCE.
§30 — Fable5Max cracks §20/§10 "unsteerable" from the gcc SOURCE: store-vs-load is a /s aliasing flag, the def-side wall has a macro escape, + the birthing-boost (Phase 23, byte-proven on giant func_8014EE14 248 ins ×134)
A single Fable5Max agent (an Agent with model: fable, given the target asm + tools/ghidra_c/ + this cookbook + the match_one→gate_stage loop) matched a 248-ins reach-134 GIANT on the §20/§10 store-vs-load wall — the class 22 phases of Opus/GLM called "CONFIRMED unsteerable" — by reading the actual gcc-2.7.2 source (tools/reference/gcc-papermario) with -da RTL dumps. It banked ×134 (leaf MATCH (248 ins); whole-binary banked:1; check-all 136/136; dedup-check 1780/0). The §29 "not a bigger model" verdict is corrected: a frontier model that goes to the compiler INTERNALS is a wall-breaker for the codegen classes. Three byte-proven idioms:
- STORE-vs-LOAD IS A DETERMINISTIC ALIASING FLAG, NOT A SCHEDULER TIE-BREAK (revises §10/§20 "unsteerable"). gcc-2.7.2
expr.csetsMEM_IN_STRUCT_P(/s) only when a load's address is a member/aggregate ref or was "computed by addition" (PLUS_EXPR). The front end foldsp[0]→*p, so zero-offset / bare-deref loads never get/s→ they carry a hard true-dependence on an aliasing fixed-symbol store (D_xxx = 0) and get stuck below it; offset/member loads have/sand hoist freely. So the store-vs-load "coin-flip" is a binary flag you SET from C:- Grant
/s(make the load hoist over the store): write it as a struct-member ref →((struct { s32 field; } *)p)->field. Use an ANONYMOUS struct in the cast —dedup_propagate(line ~366) rejects inline named structs, so anonymous keeps the/sflag AND stays propagatable ×134. - Deny
/s(keep the load below the store, e.g. a separate reload the target shows): keep it a bare*p/p[0]. (CSE's fixed-scalar-store invalidation only kills non-/sentries — that's what forces the target's separate reload.) - The tell in any diff: a zero-offset pointer load stuck on one side of a fixed-symbol store while offset loads float. Re-test candidates:
func_8014F2E0,func_80150528,func_8014EA4C(close=6), and every §10/§20 "store-vs-load unsteerable" backlog verdict.
- Grant
- THE DEF-SIDE RETURN-TYPE WALL HAS A MACRO ESCAPE (extends §29). When a matching def must return
s32(avoidreturn DCEs a computed local → frame shrinks → no match) but the only conflicting caller-decl is a sharedDEFINE_func_*macro that DISCARDS the return: widen that macro'sextern void func_X(...)→extern s32 func_X(...). It's byte-neutral for the caller (the return is discarded — verifiedcheck-all 136/136fleet-wide). §29 said "no escape" because a bareINCLUDE_ASMstub declares no C symbol — but a macro DOES declare the symbol via itsexternline, so there is one. (The narrow-scalar-param-by-value wall from §29 still stands — this dissolves only the return-type conflict.) - THE "BIRTHING-BOOST" PROLOGUE-ORDER LEVER.
sched.c:adjust_priority(pre-reload only) boosts to max priority any insn whose dest reg is set exactly once in the fn (birthing_insn_p:REG_N_SETS==1); sched1 schedules each bb backward, so a boosted insn is picked early = placed late. Single-set param copies (s0=a0) then sink below multi-set const inits (s6=0/s5=8, reassigned in a switch → never boosted). Fix: one zero-byte NON-volatile re-tie__asm__("" : "=r"(x) : "0"(x))on the param, placed in a LATER basic block (after the switch). It counts as a 2nd SET (boost dead), emits nothing, adds no bb0 edges → all inits tie and the LUID/source-order tie-break restores params-first. Generalizes to any wrong prologue/init ORDER between single-set and multi-set defs.
Meta-lesson (feeds effort-map / R17): the "idiom well is dry / wall is intrinsic" verdict (T10.8/T10.9, §29) was model-relative — true for GLM and the local 7B, false for a frontier model that reads the gcc-2.7.2 source. The wall-breaker recipe: Agent(model:fable) + tools/reference/gcc-papermario (RTL -da dumps) + this toolkit + the match_one/gate_stage loop, on ONE giant at a time. Cost ≈ 375k agent-tokens / giant across the leaf-crack + whole-binary integration (2 rounds). NB: match_one (isolated) masks in-TU declaration conflicts — always finish on the whole-binary gate_stage (the §20 stale-.o trap can fake a pass; force a clean compile).
§30a — §30 generalizes to the FULL near-miss backlog (via STANDARD Opus agents, not just Fable5) + 2 more steer levers + the mechanical-integration throughput unlock (Phase 23 (a))
A toolkit pass re-tested 3 backlog "store-vs-load / CONFIRMED unsteerable" verdicts with §30, run by standard Opus agents (~80–130k tokens each) APPLYING the documented idiom (not Fable5 discovering it). Result: 2 banked ×134 (func_8014F2E0 66 ins, func_80150528 53 ins; fleet 64.74→64.82%), 1 partial (func_8014EA4C 6→3 — its store-vs-load part dissolved by §30, but an orthogonal abs-coalescing/frame-fragility wall survives = genuine permuter/Fable5 territory). The "unsteerable" backlog is largely MIS-VERDICTED — these near-misses ARE matchable, and standard agents applying a GROWING toolkit crack them cheaply. Three byte-proven additions:
- §30
/sREFINEMENT — a cast-wrapped PLUS does NOT get/s.expr.c:5535expr.c:4577(guard 4570-4576, insidecase INDIRECT_REF:@4540, inexpand_expr@4026 — citation corrected 2026-07-28; 5535 was a gcc-2.8.1 line and points atMIN_EXPR/MAX_EXPRoptab code in our 2.7.2) grants/swhen theINDIRECT_REFoperand is a top-level PLUS_EXPR;*(T*)(p + k)puts a NOP_EXPR (the cast) on top of the PLUS → no/s. Precision (same re-derivation): the guard is a 4-way OR, not a single test —/sis ALSO granted when the operand is aSAVE_EXPRwrapping a PLUS, when the dereferenced type is itself an aggregate (AGGREGATE_TYPE_P (TREE_TYPE (exp))), or when the pointer is anADDR_EXPRof an aggregate. So the cast-defeats-/srule holds for a scalar-typed deref through a plain pointer, which is the case this idiom is about — do not read it as an absolute. Only a bare-typed PLUS (q[k],qa typedT*) or a COMPONENT_REF (expr.c:5891expr.c:4888,case COMPONENT_REF:@4748 — genuinely unconditional: theif/elseabove it only chooses how the address is formed, both arms fall through to the same unguarded set; NB the otherMEM_IN_STRUCT_P (op0) = 1at 4873 IS conditional (BLKmode bitfield path, returns early at 4876) and is not this one) gets it. The universal grant is the anonymous-struct member-ref((struct{s32 f;}*)p)->f(COMPONENT_REF path), NOT*(s32*)(p+off). (byte-proven func_8014F2E0, func_8014EA4C). - IV-COMBINE: keep gcc's
combine_givsat N induction vars with a SINGLE base pointer. Far-field accesses off a separateq = base + offpointer make cc1 strength-reduce a spurious extra IV. Collapse to ONE base pointer with all fields as plain byte-offsets → cc1 re-combines to the target's IV count. NB: CONFLICTS with the §30 anon-struct read (which introduces a separate pointer → re-triggers the split) — read the diff and pick the lever the target's IV structure demands. (byte-proven func_80150528). - INLINE LOOP-LIMIT → preheader-hoist. An invariant loop limit written as an inline expression
D_x + Kin BOTH the entry guard and the loop condition (NOT a cachedendlocal) makes gcc's loop optimizer hoist it into the preheader — producing the preheader copy insn AND the target's callee-saved regalloc. A cachedu8 *endlocal pins it in one reg = wrong. (byte-proven func_80150528).
INTEGRATION IS MECHANICAL — the throughput unlock. All 3 giants banked this session (func_8014EE14, func_8014F2E0, func_80150528) hit the SAME whole-binary near-1 after a perfect leaf match_one: a DEFINE_func_* caller macro declares the fn extern void but the matching def needs s32 (a void return DCEs a live local → frame shrinks → no match). Fix = §30 #2: widen that caller macro's extern void→s32 — byte-neutral (caller discards the return; check-all 136/136 confirms). This is TOOLABLE — a gate_stage recovery pass that, on a leaf-MATCH-but-near-1 fn, tries widening a discarding-caller-macro's extern to the def's return type would let every agent's leaf-MATCH auto-bank. Campaign implication: leaf-matching the near-miss backlog is now cheap (Opus agents + the toolkit); the bottleneck is the mechanical integration step → build the tool, then the ~95 "schedule"/80 "regalloc"/31 "hoist" backlog verdicts (many mis-labeled) become an agent-wave harvest, with Fable5 reserved for the genuinely-novel residuals (frame-fragility/coalescing walls).
§31 — THE gcc-2.7.2 CODEGEN MAP: pass → residual → C-lever catalog (Phase 23; 4 Fable5 agents read the compiler source)
Instead of reverse-engineering the compiler function-by-function (Fable5 ≈ 375–475k tokens/giant), 4 Fable5 agents read the gcc-2.7.2 passes directly and produced source-cited, byte-proven, rerunnable catalogs. The full detail is in docs/gcc-2.7.2-map/{sched,regalloc,loop,cse_expr}.md — consult those for the exact lever + exemplar + file:line per class. This §31 is the INDEX + triage; apply it, drop to the source only for a class not covered here (then add it, R16).
⚠️ SOURCE-VERSION CORRECTION (propagate everywhere): tools/reference/gcc-papermario is gcc 2.8.1, NOT 2.7.2 — a behavioral difference (2.8.1 &&0-disables biv-elimination paths that are ENABLED in the real 2.7.2 cc1). The vanilla gcc-2.7.2 source is now at tools/reference/gcc-2.7.2/ (SETUP §5.6) — cite it. Every byte-proven lever we have still stands (all validated via match_one against the actual pinned 2.7.2 cc1, the ground truth); only the source citations in §17/§30 were on 2.8.1.
TRIAGE TABLE — route a residual by class → {STEERABLE (lever in the catalog) | INTRINSIC → permuter}:
| pass-group (catalog) | STEERABLE (byte-proven levers) | INTRINSIC → permuter |
|---|---|---|
scheduling sched.md |
S1 LUID=source-order (transcribe target order); S2 birthing-boost — both directions (fresh single-set local to create a sink; §30#3 re-tie to kill it); S4/5/6 load-gap-filler / mem-unit spacing / hazard-front-move (a 4th rank rule — §25 corrected); S7 prologue saves; D1 delay-slot content; D2 eligibility; D3 eager-steal | S3 load/mul chain-priority sink; S11 the LUID⊗alloc coupling knife-edge (func_801571C4) — ⚠ DOWNGRADED (Phase-24 T5b): try S12+S13 first (reused-s32-temp fence · body-local param copies · asm-copy · dead-read fence — sched.md §6); func_8014E048 (35-off "intrinsic") MATCHED+banked this way |
regalloc/reload regalloc.md |
RC-1 spill-slot shape (= DECLARATION order, not first-assignment); RC-2 wrong-value-spilled (allocno density — def/use placement); RC-3 $s-order; RC-4 coalescing un-tie; RC-5 pin side-effects (4 channels); RC-7 remat-vs-spill; RC-8 reload artifacts |
RC-6 pressure-lock (func_801770E0); RC-9 cross-block copy-fold when frame-fragile |
loop.c loop.md |
L1 IV-count/anchor (single-base-ptr); L2 index-biv elimination; L3 loop reversal; L4 hoisting/preheader (inline-limit); L7 invalid-loop; L8 increment-pos | L5 final-value compensation; L6 giv-add order |
cse/expr cse_expr.md |
cross-call ADDRESS-caching = hoist-vs-remat — STEERABLE (nested-block ptr + post-call volatile output re-set); cross-call VALUE-CSE (phantom $s, §30); /s aliasing — full 4-arm model + store-side flush lever; stack layout |
cse.c 1000-insn table flush (giants); QImode never gets /s |
WALLS BROKEN this study (classes long "CONFIRMED unsteerable", now byte-proven steerable): §10/§20 hoist-vs-remat (the biggest — blocks the 400-ins func_80132784); store-vs-load (/s flag, §30); dbr delay-slot (D1 + S2 fresh-local — func_801770E0 53→49, correcting its own agent's "unsteerable" verdict); the birthing-boost sink both directions. Incidental banks from the study: func_80149374 ×134 (+ func_801493D0 leaf-matched, whole-binary-deferred). Genuine remaining walls (route to permuter, don't hand-grind): S3 chain-priority sink, RC-6 pressure-lock (true form: every edit explodes 20+ insns), the cse mega-flush. S11 was downgraded in Phase-24 T5b — func_8014E048 (the canonical "S11 intrinsic" seed, 28-off even after the directed permuter) fell to the S12 reused-s32-temp fence + S13 head-skip escape + RC-10 preference steering (sched.md §6 / regalloc.md §F); audit pins + try those before any S11 verdict.
DIAGNOSTIC TELLS (pick the class in seconds): $t0 in a spill slot = reload artifact (spilled pseudo, not a source MEM); "every small edit moves 20+ insns" = RC-6 pressure-lock → permuter; a load stuck below a D_x=0 store while offset loads float = /s-flag (§30); a phantom 7th $s-reg held across the fn = cross-call CSE; a prologue init in the wrong order = birthing-boost (S2). The 16-row full tells table is in each catalog.
HOW THE CHEAP TIER USES THIS: given a near-miss diff → read the tell → look up the class's lever in the catalog → apply → gate. This is the artifact that lets Opus agents and the local model apply compiler-internal levers without reading 80k lines of source — the permanent-knowledge payoff.
§32 — The region-a CAMERA-GIANT idiom set: struct-base hoisting + 4 sibling levers (Phase 24 T7, Fable5-cracked on func_80129CF8 191 ins, match_one MATCH; transferable to the 6 sibling giants)
The region-a (ov_SC01_077_a.c) giants Ghidra flattens into per-global lui/%lo — but the target holds global base ARRAYS in callee-saved regs across the whole function. Fable5 read the gcc-2.7.2 source and cracked func_80129CF8 in 3 edits (202→143→10→0), no pins, no permuter. The idiom set (each transferable — .run/t7/func_80129CF8.c is the worked example, .run/t7/GIANTS_SURVEY.md the target list):
- Hoisted
$sNbase = an explicit POINTER LOCAL, assigned AFTER the first call, in the target'sluiorder (p2 = D_800AF630; p1 = D_80126DB8;right after the firstjal). gcc-2.7.2 has no cross-bb CSE, so a base living in a callee-saved reg across calls/branches can ONLY come from a source local — you cannot get it from a bare global access. Noregister __asm__pins needed: allocation order = density priorityfloor_log2(n_refs)*n_refs/live_length(global.c:594 allocno_compare) then first-fit regno (mips.h has no REG_ALLOC_ORDER) → most-referenced ptr→$s0, next→$s1, next→$s2. The init placement survives because sched1's ascending-LUID tie-break (sched.c:2414 rank_for_schedule) keeps zero-dep sets in order. A single-set/single-use base local is SAFE from the RC-7 init-sink (its hi/lo SET_SRC is a LO_SUM → failsrtx_equal_patlocal-alloc.c:1169-1172). - Branch polarity is READ OFF THE TARGET OPCODE (not Ghidra's
if): targetbeqz $v0,.Lcopywith the other arm as fall-through ⇒ writeif (sel != 0) {fallthrough-arm} else {copy}. On func_80129CF8 this single inversion fixed 133 of 143 mismatches. lw/lw/nop/addu/swper-element ladders = the S12 reused-s32-temp fence —a = p1[i]; b = cam[j]; cam[k] = a+b;reusing ONE(a,b)pair across all elements. But leave the LAST element in Ghidra's fresh-temp shape (t = p1[last] + cam[..]; cam[..] = ..; cam[..] = t;) — its unfenced load is what the scheduler hoists into the previous element's load-delay slot, and a following call-arg&cam[..]fills the gap before it, both automatically.- A grouped 3-loads/3-stores copy fed by
la $reg,SRC(split lui/addiu) = a 32-byte STRUCT ASSIGNMENT, never scalar copies:*(RView*)(p2 + 6) = *(RView*)D_800AE688;(RView = the GsRVIEW2-shaped 32-byte struct, now inengine_types.h).expand_block_move(mips.c:2361) emits onemovstrsi_internalfor ≤2*MAX_MOVE_BYTES(32) with 4 scratches;output_block_moveburns the last scratch onla a1,SRC→ 3 data regs → the 3+3 grouping. The dest must go through the pointer local (p2+6, folded to0x18($s2)); a bare global dest is CONSTANT_P and degrades to 2 regs. Scalara=src[i]; dst[i]=a;copies schedule as lw/lw/sw/sw pairs — WRONG shape. - Frame bigger than args(16)+saves by a round chunk = a DEAD LOCAL AGGREGATE: add an unused
RView view;(32 B) to reserve the missing 0x20 — gcc-2.7.2 assigns stack slots to local structs/arrays at expand time regardless of use, and -O2 never deletes them. Suspect this whenever a sibling's frame is 0x20/0x28 over-accountable. - This class needs NO pins, NO asm fences, NO permuter — the interp-loop software-pipelining (a2/sll/lh-next/sra/sw + a3-in-jal-slot) is deterministic sched2+dbr output once the bases sit in $s0/$s1/$s2.
BANKING (Phase-24 T7b — RESOLVED, see §33): a freshly-matched giant MATCHes standalone (match_one) but its loose data decls (struct BigCopy / s32 / s32[] / u8[] / s8-vs-u8) collide in-TU with engine_core.h when you try to bank it. The fix is a decl-reconcile — declare each symbol its canonical type and cast byte-neutrally at the access site (cam = (s32*)D_80126948;, p1 = (s32*)&D_80126DB8; for a struct BigCopy base, D_801151D4 = (s32)cam;, *(u8*)&D_801150D6 to force lbu under s8) — done by HAND for func_80129CF8, now automated by tools/reconcile_decls.py (§33). R14 correction: the T7 note that "banking ×134 hits the wall" was a MISDIAGNOSIS — once the ×1 bank is reconciled, dedup_propagate --recover propagates it to all 134 overlays byte-identical for free (proven: func_80129CF8 ×134, clean fleet 136/136). The wall was only ever the ×1 reconcile of the loose draft, not the propagation.
§33 — Automating the giant decl-reconcile: tools/reconcile_decls.py (the DATA analog of §20's cast_call_sites) + a fleet-majority type oracle (Phase 24 T7b, byte-proven on func_80129CF8)
The gap. A freshly-matched giant/wave draft byte-matches STANDALONE with its own guessed decls, but to bank ×1 (and then propagate ×134 via a shared DEFINE_func_* macro that EMBEDS its externs) its callee/data externs must be FLEET-CANONICAL, else conflicting types in-TU. cast_call_sites (§20) does this for callee FUNCTIONS; reconcile_decls does it for DATA symbols D_XXXX — the class the giants hit (they hoist global base ARRAYS, §32). It was the last manual step in the giant pipeline.
The oracle (fleet-majority type picker) — new; none existed (gen_harvest_targets read only engine_core.h + ONE overlay, first-seen-wins, never saw a fleet disagreement). reconcile_decls.canonical_data_map(): an engine_core.h macro decl is AUTHORITATIVE (the shared/propagated set every overlay co-instantiates — conform to it and the macro can't conflict per-overlay); else the plurality across ALL overlays + resident.c (tie → lexicographically-first spelling, deterministic). Verified 8/8 on func_80129CF8's symbols (--print-canon D_XXXX inspects one).
Byte-neutral cast taxonomy (the DRAFT's decl type = the intended access; cast every use to reproduce it under the CANONICAL storage decl → gcc folds the compile-time cast → identical opcode; the whole-binary byte-gate is the sole arbiter, fail-closed):
| draft decl | canonical | decl → | access → |
|---|---|---|---|
array Ed[] |
array Ec[] (Ed≠Ec) |
Ec[] |
D_x → (Ed*)D_x; D_x[i] → ((Ed*)D_x)[i] |
array Ed[] |
struct / scalar | canon | D_x → (Ed*)&D_x (address-of the object) |
scalar Td |
scalar Tc (signed/width) |
Tc |
D_x → *(Td*)&D_x (forces the Td opcode, e.g. lbu under s8) |
ptr P* D_x |
scalar Tc |
Tc |
D_x → (*(P **)&D_x) (cast-lvalue: store/read a pointer in the scalar slot) |
Implementation: ONE re.sub per symbol with a replacement fn (captures the optional leading & / following [), single-pass so a symbol in several forms on one line can't double-wrap (the first draft double-wrapped the ptr-store — the two-sub read+write templates re-matched; collapsed to one pass). |
Pipeline placement: canon_resident_calls → cast_call_sites → reconcile_decls → sig_unify → harvest_verify --chunk 1 (wired into gate_stage; idempotent / no-op on drafts without a data conflict → can't regress the wave). Standalone giant use: tools/reconcile_decls.py --overlay <ov> --src-file src/<ov>/<ov>_a.c --in <drafts> --out <drafts>-rc.
Byte-proof (T7b, no Fable5 needed): the full loose func_80129CF8 (intended decls s32 D_80126948[] … , un-cast accesses) → reconcile_decls (5 symbols reconciled) → swapped into the DEFINE_ macro → make build BINARY=ov_SC01_077 = BYTE-IDENTICAL d19c9580.
R14 — the tool is an AUTOMATION, not an "unlock": func_80129CF8 banks ×134 FREE via existing dedup_propagate --recover once reconciled (the "×134 wall" was a stale-asm misdiagnosis, §32). reconcile_decls' value is removing the manual ×1-reconcile step for every future giant / loose draft (the 6 sibling giants + the wave tail), not unblocking propagation.
§34 — The func_80138ED0 giant crack: gcc-2.7.2's 3-qty sort bug + the zero-byte asm allocation toolkit + the giv-init fence (Phase 24 T5; Opus→close=21, Fable5→MATCH ×134)
The 2nd region-a giant (159 ins, bit-unpack/tilemap; 2 giant-local data bases, callees func_8013914C/func_800599B8). Opus applying §32 reached close=21 (all semantics/control-flow/constants exact); Fable5 reading the vanilla gcc-2.7.2 source cracked the pure regalloc/schedule residual — every class C-reachable, no permuter. Each lever byte-verified via match_one + gdb-on-cc1. The banking then took the standard pipeline (cast_call_sites reconciled func_8013914C (u8*,u16*)→canonical (s32,s32)+call-site cast; reconcile_decls a NO-OP — the 2 data bases are giant-local, no fleet conflict) → ×1 → dedup_propagate --recover → ×134 byte-identical (pins/asm body propagates fine; func_800599B8's lone (s32,s32) decl is in ov_SC01_077.c, a different TU from the _a.c bank, so no per-member conflict).
THE HEADLINE — gcc-2.7.2's 3-qty local-alloc SORT BUG (local-alloc.c:1441-1463/:1494-1516). For a block with ≤3 local register quantities, the unrolled comparison switch compares fixed qty numbers (qty_compare(0,1),(1,2),(0,1)) but exchanges order slots — when pri(q1) is highest the third compare re-fires and undoes the first swap, so 3-qty blocks allocate in qty-CREATION order, not density order (≥4 qtys go through qsort, correct). Symptom: a low-density local grabs a reg it shouldn't. Fix: a zero-instruction DECOY qty (asm("":"=r"(decoy):"r"(x)); asm("" :: "r"(decoy));) bumps the block to 4 qtys → the qsort path → correct density-order first-fit (find_reg, global.c:904; regs_used_so_far pre-seeded with the call-used regs, global.c:352-355). A real compiler bug, now a reusable lever.
THE ZERO-BYTE ASM TOOLKIT (allocation/schedule dials that emit NOTHING — the byte-gate certifies the induced codegen):
- input-only dummy
asm("" :: "r"(v))— floats tov's def; a ref-count / density dial (raise a pseudo's priority / extend a live range one way). - multi-input dummy
asm("" :: "r"(a),"r"(b))— anchors at the LATEST def; a lifetime-extender / joint-release (releases a,b together in sched1's backward pass →rank_for_schedule's class rulesched.c:2385, cost-1 dep beats cost-2 load, orders the emit). - def+use pair
asm("":"=r"(d):"r"(s)); asm("" :: "r"(d))— mints a decoy qty (the 3-qty-bug fix). - giv-init fence
asm("":"=r"(ba):"0"(ba)); dst = ba;— forcesemit_iv_add_mult's giv-init MOVE (loop.c:5556 if (reg != result) emit_move_insn) =addu dst,ba,$zero; the general fix for the "gcc coalesced the giv init, dropping one instruction → full count mismatch" class on any giant with a counter-derived pointer. (Found by the Opus pass; kept.)
Statement-position / type levers (no asm): a leading pb = param_3; rides sched.c:3191-3215's "don't delay getting parameters" pin so combine folds the parm-save into it (prologue order); an explicit u32 pv = uVar1; before p = base; replaces loop.c's move_movables hoist-at-loop_start (loop.c:1652/1810) with source order; a HImode % 3 (a u16 var) defeats gcc's x%3==0 → beq (x/3)*3,x fold.
gdb-on-cc1 (the method that settled it): the shipped tools/bin/gcc-2.7.2-psx/cc1 is i386-static WITH symbols — breakpoint find_free_reg/post_mark_life (.run/t7/fable/gdbtrace.gdb) to dump the real qty order + register grants when hand-modeling stalls. Cite the vanilla tools/reference/gcc-2.7.2/ tree (now complete — global.c/local-alloc.c/reload1.c/toplev.c/function.c/flow.c/… from the FSF tarball; sched.c/loop.c/mips.c verified byte-identical to vanilla). Pass order (why upstream fixes reach the prologue): sched1 → local_alloc → global_alloc/reload → prologue threading (toplev.c:3103) → sched2 (:3117) → jump2 → dbr; prologue saves are sched2-scheduled, everything upstream tunes sched2's LUID tie-breaks.
Gotcha: parallel match_one runs on the SAME function share .run/match/<fn> — pass a unique --work dir or the scores are garbage.
§35 — The region-a sibling-giant harvest: difficulty ≠ $s-reg count (it's global-array hoisting) + the banking recipe + new loop idioms (Phase 24 T7, 5 parallel Opus-Max agents)
Cracking the 5 remaining region-a giants via parallel Opus agents applying §32/§34 surfaced a ranking law + a reusable banking recipe.
THE RANKING LAW (retires the "8-$s-reg = hardest" heuristic): a giant's difficulty is set by whether it hoists GLOBAL base ARRAYS into callee-saved regs, NOT by $s-reg count. gcc-2.7.2 has no cross-bb CSE, so a hoisted global base can only come from a source-local pointer in the exact lui order (§32#1) — the genuinely hard part, needing Fable5 for the last-mile regalloc. Giants whose $s pressure is param / local / output-buffer derived (even all 8 $s0–$s7 live) one-shot with Opus alone (density-order first-fit). Byte-proven: func_8012D098 (189, all-8-$s) + func_8012EC04 (178, sibling+GTE-tail) MATCHED one-shot; the 3 residual monsters stall only on tiny (2–15) compiler-internal walls, not the $s allocation. Triage a giant by its %hi data bases, not its $s count.
THE BANKING RECIPE (matched giant draft → ×134; per-giant integration is mechanical, §30a):
fix_arity_callers --apply --any-proto --from-file <fn> --drafts <dir>— no-proto the caller extern conflicting with the matched def-sig (the DEF-side wall: a shared macro in engine_core.h, or a sibling in the overlay, forward-declares the giant with a different sig →conflicting types). Byte-neutral (the call's arg bits are unchanged).sig_unify— canonicalize the giant's own CALLEE externs (no-protovoid f()→ the fleet's full proto; else args promote differently standalone vs whole-binary).- Strip draft-local typedefs already in
engine_types.h(a redundanttypedef …Blk16;=conflicting typesin the shared TU) and anonymize named local typedefs (typedef {…} Buf; Buf b;→struct {…} b;, else it collides with a sibling'sBuf). harvest_verify --chunk 1→ ×1, thendedup_propagate --addr <A> --recover→ ×134. Do NOT runcast_call_siteson these — it mis-casts a no-protovoid f()tovoid(*)(void)(0 params) and breaks the call (fix candidate: treat()as compatible-with-any).
New loop idioms (byte-verified; §31/loop.md fodder): dest-off-base giv — write every primitive store *(u16*)((s32)p + k) off ONE biv so they combine into a single dest-addr giv (kills giv-split); index-form source param_3[j+k] with j+=2 (not a walked pointer) reduces to one address biv; sltiu-outer/slti-inner without CSE — a redundant SIGNED range-guard that must stay slt/slti needs a separate signed-int copy (int s = u;), since only CSE-reuse canonicalizes signed→unsigned; div2 range-extension — a zero-byte asm("":: "r"(x)) after the last flag-test extends a pinned reg's live range so a trailing andi lands in the target reg.
The 3 deferred residuals (Fable5 batch — close=2/10/15; precise root-cause in .run/t7/<fn>.c headers + docs/backlog.md): func_801392FC (2: combine folds (s16)load → atomic lh before sched2, so the pipelineable split-load never exists; permuter can't reach) · func_8013A530 (10 → MATCHED ×1, §36 below — the "RC-6 reload-pressure" verdict was wrong: it was the $2/$3 pins themselves (reload-retry poison + sched1 load-hoist collision); cracked pin-free via the $0-add opaque copy + condition-operand flip + else-arm density dummy, regalloc.md §G) · func_8013AF20 (15: loop-invariant constant-hoist ORDER — the two AND-masks materialize opposite, coupled to the AND-operand order; permuter plateaued at 15). All single-class, high byte-weight (×134), ideal Fable5 targets.
§36 — Fable5 giant-crack levers (Phase 24 T7 Fable5 batch; accumulates as each lands — full RTL dumps in .run/t7/fable/)
func_801392FC (loop count-load, close=2 → MATCH ×134):
- CROSS-BB COMBINE LAW. combine never spans basic blocks (
flow.c:2087—LOG_LINKS(y)created only whenBLOCK_NUM(y)==blocknum). So a value crossing a loop BACKEDGE — au16 cntreloaded at the loop tail after a call,(s16)cnt+1at the top — is a VARIABLE, not a foldable(sign_extend(mem))→lh. The "split-and-schedulable" count sign-extend the target shows is cross-BB dataflow, not a fold-defeat;reorg.c fill_slots_from_threadsteals the top-BBsllinto the loop-backbnezdelay slot + redirects the label tosra→ the duplicatedsll(preheader fall-in + delay-slot copy). Don't hunt for an un-fold; make the value cross the backedge. - VOLATILE-FRAME-PARITY. Make the loop-carried count-load
volatile. A non-volatilecntgets cse-commoned with the same-address condition read → combine's keep-load fold (newi2pat;combine.c:2089 elim_i2) keeps a dead ashift-temp'sREG_DEADalive → a(use reg)planted at the nearest label (combine.c:10835-10847) → a stale allocno grabs an extra 8-byte reload slot (frame 0x90 vs target 0x80).volatilemems are never cse-hashed (clean condition fold) yetsched.c:811 read_dependence(both mems volatile) still lets thelhhoist above the volatilelhu.volatilematches BOTH frame size and schedule — suspect it when a loop-carried count is 8 bytes frame-over. - Secondary (byte-verified): route
buf+0xFbyte-store sums throughs32temps (defeats the C-frontend QImode-plus operand-swapaddu v1,v1,v0vs the target's SImodeaddu v0,v0,v1); statement position drives sched2's BACKWARD list scheduler (INSN_LUID tie-break,rank_for_schedule) — puti++/acc+=AFTER the call to fill thejaldelay; aregister __asm__("$5")re-arm before an arm's 2nd call limits post-reload cross-jump merge depth (jump2 sits between sched2 and dbr).
func_8013A530 (the LARGEST giant, 204 ins, clamp double register-split, close=10 → MATCH ×1; dumps .run/t7/fable/a530/):
- THE $0-ADD OPAQUE COPY (new tool — the un-reversible, un-cse-able
move).register int zr __asm__("$0"); iVar7 = fc + zr;emits the byte-identicaladdu $rd,$rs,$zerobut as RTL(plus reg $0), NOT(set reg reg): cse'smake_regs_eqvnever links the two (no canon poisoning in either direction —canon_regcse.c:2545 also never rewrites the hard-reg side), and combine cannot absorb the source's load into the dest (no extend+plus pattern). A plainint iVar7 = fc;(fc pinned) gets REVERSED by combine — load lands in the pseudo, the pin becomes the copy dest, 1 insn shorter. Use it whenever the target keeps load+copy as two live registers with compares reading the LOAD and arms reading the COPY. $0 is fixed → zero RC-5 side channels. - PIN → RELOAD-RETRY POISON (RC-5 ch.2 extends to retry_global_alloc). The old
iVar7 $v1/t $v0pins put $2/$3 inbad_spill_regs(reload1.c:3900-15regs_explicitly_used); the div-magic LO/MD "Need" spill forced CASE1's 2nd-product allocno throughretry_global_alloc, which skips bad_spill_regs →mflo $t2. Unpinned:.greg "Register 177 now in 3"→mflo $v1. A pin can move a register FUNCTION-WIDE through the retry path even where the pinned var is dead. - CONDITION-OPERAND ORDER = LOAD PLACEMENT (sched1 backward + mem-unit hazard).
(int)mem < t-extvst-ext > (int)memproduce the SAME canonical slt but mirrored expansion uids; sched1's backward scheduler (boosted-group ties break toward higher uid;blocking insn N for 1 cycleswhen the sh occupies the mem unit) hoists the mem-first spelling'slhto the block TOP — birthing the reload INTO the pin/pseudo's live window (the RC-6 "reload-pressure" was really this). The>spelling keeps the lh below the addiu that kills iVar7 → reload (local, first pick) and iVar7 (global) sit in $v1 disjointly. - DENSITY DUMMY PLACEMENT vs maspsx (#APP blocks the ASPSX slot-hop). cc1 emits
[lh;lh;addu;slt;beqz]; maspsx/ASPSX-2.56 HOPS the addu over the slt into the branch delay slot (and re-inserts the load-delay nop) — but NOT across#APP/#NO_APP. A zero-byte dummy between the addu and slt kills the hop (that was the last 1-insn diff). Park density dummies inside an ARM as a 2-inputasm("" :: "r"(v),"r"(t))anchored at t's def — +1 ref lifted iVar7'sallocno_comparepriority (3/10→8/11) past the fe-load's 3/5 → iVar7 allocates first → $v1, fe → $a0. - KEEPALIVE KILLS THE DYING-HARD-REG SUGGESTION. The else-compare's slt-result temp grabbed $a1 via
qty_phys_sugg(fc/$a1 dying in that slt; suggested qtys allocate first).__asm__("" :: "r"(fc));after the clamp keeps $a1 alive there → no suggestion → plain first-fit $v0 (target). - The 2nd split is NATURAL (promoted-HI store-copy).
if ((cmp)) *(s16*)(p+0xc) = *(s16*)(p+0xe);— the condition's(int)memexpands as HI-load+sll/sra, the body's re-load cse-folds onto the HI pseudo, combine merges the extend into onelhand re-emits the HI pseudo as a subreg copy →lh $v1; addu $a0,$v1; slt ..$v1..; sh $a0for free. Never pin for this shape.
func_8013AF20 (185 ins, 3 addPrim GPU-builder loops, close=15 "const-order⊗AND-order coupling" → MATCH, permuter had plateaued 40k iters; dumps .run/t7/fable/dumps_C/):
- BITFIELD STORE = THE MASK-ORDER DECOUPLER (the libgpu addPrim idiom). A target preheader that materializes
0x00ffffff(lui+ori) BEFORE0xff000000(lui) while the body still computes*dest & 0xff000000FIRST cannot come from user-mask C (*p = *p & 0xff000000 | *ot & 0xffffffmaterializes in expression order, and every operand reorder flips the AND/OR shape with it — the byte-verified 19/182-diff coupling). It comes from the SDK'ssetaddr24-bit BITFIELD store:store_fixed_bit_field(expmed.c:556) expands value∧mask_rtx(mode,0,bitsize,0)=0x00ffffff FIRST (must_and:667 → :679-681), THEN dest∧mask_rtx(mode,bitpos,bitsize,1)=0xff000000 (:694-696), THENior(destmasked,value)(:706 — dest chain stays op0). scan_loop then finds the mask movables in that insn order → move_movables emits the preheader consts […, 0x00ffffff, 0xff000000] with the body bytes unchanged. TranscribeaddPrim(ot,p)literally:((P_TAG*)p)->addr = ((P_TAG*)ot)->addr; ((P_TAG*)ot)->addr = (u32)p;with the realP_TAG {u32 addr:24; u32 len:8; u8 r0,g0,b0,code;}(a bitfield STORE with user-mask RHS also byte-matches; the read is optional). 15→8. Suspect this lever on ANY OT/linked-list 24-bit-pointer-field target. - 2-INSTRUCTION CONSTANTS DODGE THE EQUIV LL-DOUBLING → they WIN the low scratch regs (gdb-proven). sched1 splits every insn pre-reload (sched.c:4830
try_split) → mips.md:3208large_intdefine_split turnsli 0xffffffinto lui+ori = TWO SETS →reg_n_sets==2failsupdate_equiv_regs' single-set gate (local-alloc.c:1021) → the mask ESCAPES thereg_live_length *= 2const-penalty (local-alloc.c:1064) that hits every 1-instruction const (addiu-able 3/0x40/0x3d, lui-only 0xff000000). Priorities: mask fl2(7)*7/35 = 4000 ≫ 0x3d fl2(7)*7/72 = 1944 → the mask allocates FIRST (allocno_compare) → first-fit $a2 (loops 1/2) / $a1 (loop 3). gdb ground truth: n_sets {61:1, 0xffffff:2, 0xff000000:1}; LL pre→post update_equiv_regs 36/35/33 → 72/35/66. A preheader const-register contest that looks "impossible by density" is usually this. - A do{}while(0) barrier is a LOOP-DEPTH REF INFLATOR (flow.c), not just a LUID shifter. flow weights every mention by loop_depth (flow.c:2067/2315/2501/2711
reg_n_refs += loop_depth; here: preheader set=1, body mention=2, +1 more per extra NOTE_INSN_LOOP nest). The Opus draft's barrier wrapped the three0x3dbyte-stores → refs 7→10 → pri 4166 > 4000 → 0x3d stole $a2 from the mask (the final 8 diffs). Deleting it (obsolete once the bitfield form reshaped the body) restored the refs tie → mask-first allocation → MATCH. AUDIT inherited barriers whenever a preheader-const register identity is off by one.
§37 — The T7 §G giant endgame: 6/8 cracked, meta-laws + transferable levers (Phase 24, 2026-07-07; FULL byte-verified detail + gcc-2.7.2 line cites in docs/gcc-2.7.2-map/t7g-giant-harvest.md)
Cracked 6 of 8 reach-134 §G giants — func_801571C4 (permuter), func_8014EA4C/func_801372B0/func_801770E0/func_80176D94/func_80148094 (Fable5) — plus 2 genuine walls: func_80178004 (biv-init emit-order, close=7) and func_801412A8 (allocation placement-knot, close=29), both stay INCLUDE_ASM. 4 propagated ×134 (fleet 65.75→65.91%, clean 136/136); 372B0/770E0 banked ×1 (blocked ×134 by the §A pin/asm + local-type self-containment gaps).
META-LAWS (audit BEFORE deep work):
- SIBLING-ISOMORPHISM (mandatory step-0): GPU-packet/addPrim builders come in FAMILIES. Diff your target
.sMNEMONIC sequence vs already-MATCHED siblings (grep -oP '\*/\s+\K\S+'); an immediates-only diff ⇒ PORT the banked sibling verbatim + swap constants = one-shot (76D94 fell from 770E0 in ~15 min). - "MUTUAL-EXCLUSION / RC-6 unsteerable" IS A TELL, NOT A VERDICT: "X must be early AND late, coupled through one priority number / every small edit moves 20+ insns" = a MISSING dep edge or an allocno TIE — real only in the broken graph. 770E0 (the named RC-6 exemplar) fell fully source-reachable.
- RTL DUMPS ARE NOT STRIPPED (corrects §34): the shipped cc1 with
-dr -dj -dc -dl -dgemits full.rtl/.jump/.combine/.lreg/.greg— readable ground truth before gdb (.lregrefs/length,.gregalloc-order/conflicts/preferences,(use (reg:SIin.combine= phantom stack slots).
TRANSFERABLE LEVERS (byte-verified):
- The
/s-DEP LATTICE (load+store dual): forceMEM_IN_STRUCT_Pvia a struct-member-at-offset-0 access —((struct{u16 h;}*)&D_global)->h(load, 770E0) /((struct{u32 w;}*)p)->w=…(store, 76D94) — to restore the missing store↔load dep edge (sched.c:820drop clause needs one side /s+varying, the other non-/s+FIXED-address). SCOPE: inapplicable when all mem ops are register-addressed (412A8/48094). - Allocno-priority ref-boost (48094):
__asm__("" :: "r"(v));at the TOP of a block where v is already live-through → +1 flow-ref, 0 live-range, 0 bytes; crosses thefloor_log2(refs)step inallocno_compare(global.c:588) → v wins the reg over a short block temp. - Coalescable-copy insn_count bump (78004): whole reg-file shifted by one hoisted loop const ⇒
move_movables' threshold at its>=boundary — a coalescable copy{u32 m=v; store=m|…;}bumps insn_count +1 → hoist refused; combine coalesces m away = 0 bytes. BANKABLE (vs a TU-wide global-register-var). - Const-register PIN cascades store order (770E0): store order is downstream of a const's register via sched2 anti-webs → pin the const (
register u32 c __asm__("$6")=…), stores re-place free. - S2-kill re-tie (372B0):
__asm__("":"=r"(v):"0"(v))after last use → reg_n_sets==2 → no birthing boost → source order (single-set pins, INCL. hard-reg, DO boost — corrects §34). + S2 fire-tick via consumer store order. - qty_compare-TIE audit (412A8): dozens of "schedule" diffs often trace to ONE equal-priority allocno tie (
find_free_reg, gdb) broken by qty/block-scan order — a window-temp reuse flips it. + multi-death block-var law (local-alloc.c:472: reg_n_deaths≠1 → global allocno → chaos). - CSE-dodge without a barrier:
(u16)x(zero_extend) head vsx&0xffff(AND) tail hash differently → no cross-call CSE → no extra callee-save. Prefer overvolatile-asm re-ties (a FULL sched barrier per sched.c ASM_OPERANDS).
BANKING PATTERNS (main + _a split): caller-extern reconcile (void→canonical return, 571C4); asm-label alias for sibling-decl signedness/proto conflicts (extern u16 X __asm__("D_x"), 372B0) — beats reconcile's *(u16*)&D_x cast whose address-of PERTURBS regalloc; canonical call-cast + anonymize the shared /s struct for isomorphic-sibling ports (76D94). ×134 blockers (§A gaps → tool fixes): register-asm pins + overlay-local named types fail dedup_propagate.compiles_standalone → bank ×1 (needs a pin/asm self-containment shim + a uniquely-renamed type-lift).
TOOLING (flywheel): fixed a silent permuter bug — tools/p16_permute.py header comments broke cpp → decomp-permuter no-op'd (0s) on EVERY commented draft (strip_c_comments); a SECOND silent no-op of the same class (Phase 26 session 6): a draft whose GTE ops are #defines CONTAINING __asm__ (the PsyQ inline_c.h convention — i.e. most renderer code) got its macro DEFINITIONS chewed up by hide_asm (which is built for __asm__ statements / register pins and scans to the nearest ;{}), swallowing the function itself → pycparser Function <fn> not found in base.c → permuter no-op (0s). Fix: cpp_expand_macros() pre-expands with cpp -P so each GTE op becomes an inline __asm__ statement hide_asm can carry — applied ONLY when a #define ... __asm__ is present, so macro-free drafts are byte-untouched. LESSON: the permuter reporting no match (0s) is a TOOLING failure signature, never a real search result — always confirm workers actually ran. p16_permute also gained --asm-subdir (it was hardcoded to ov_SC01_077's main object, so no core in another overlay/split object could be permuted at all); NEW tools/permuter_ils.py (warm-restart iterated-local-search — descends where a cold run plateaus: 48094 72→29). Escalation: model to close=2 by hand, THEN directed-permuter the residual (372B0's last lever fell at permuter iter 291; permuter is low-ROI at close=8).
§38 — The WHALE func_80144B9C (770 ins): the -O0 struct-assign memcpy idiom + the -O0 reach-134 ×134 rollout (Phase 24 T7 §G, cheap Opus — no Fable5, no calls.c)
The single biggest byte-weight lever in the fleet (770 ins × reach-134 ≈ +1.6%). It sat at close=2 for a whole session, tagged "needs gcc-2.7.2 calls.c + Fable5" — but a cheap Opus one-liner cracked it. Three durable lessons:
(1) THE CRACK — a STRUCT ASSIGNMENT, not an explicit memcpy() call, for the -O0 block-move. The whale is an -O0 function (prologue 21F0A003; match_one at -O2 reads only 458/770 — compile it -O0). Its 2-insn residual was a memcpy of a 0x24-byte struct. The tell: the target marshals the memcpy args through temp pseudos — lw $v0=src ; lui/addiu $v1=&dst ; addu $a0,$v1,$zero ; addu $a1,$v0,$zero — which an explicit memcpy(&dst, src, 0x24) call does NOT emit (it loads $a0/$a1 directly). That precompute is the signature of gcc's emit_block_move → emit_library_call(memcpy): the original C was a struct assignment dst = *src_ptr; where sizeof(struct)==0x24. gcc-2.7.2 -O0 expands a > MOVE_RATIO-word struct copy to a memcpy library call whose args go through copy_to_mode_reg (pseudos) then addu into the arg regs = the exact 2 extra moves. So: an -O0 memcpy whose target precomputes dst/src into pseudos + addus them into $a0/$a1 (vs a direct load) ⇒ write a struct-assign, not an explicit memcpy(). (The prior session tried every call form — casts, K&R, builtin ±-fno-builtin — but never the struct-assign; no calls.c was needed.)
(2) THE -O0 REACH-134 ×134 ROLLOUT — per-overlay -O0 split + a shared HEADER (not a DEFINE_ macro). An -O0 function that is reach-134 (byte-identical in all 134 overlays because it refs only SHARED globals, not per-overlay data — unlike the Phase-20 overlay-local -O0 cluster) still propagates ×134, but NOT via a DEFINE_func_X() in an -O2 TU: it only matches at -O0 and gcc-2.7.2 has no per-fn -O0 pragma. Mechanism (tools/rollout_whale_o0.py, tools/split_whale.py): each overlay's single .c is splat-emitted in vram order, so a line-based split at the whale's INCLUDE_ASM line carves it with zero item-parsing (before → keeps the <ov> name + nonmatchings/<ov> asm paths; whale → its own -O0 object <ov>_o0b; after → <ov>_after with asm paths rewritten). The whale's C lives ONCE in a shared header src/shared/func_80144B9C.h (NOT a DEFINE_ macro — its 7 local typedefs make a 200-line \-continued macro fragile; a plain header is clean, and the typedefs stay TU-local because <ov>_o0b.c includes only common.h + this header, never engine_core.h). One Makefile wildcard rule $(WHALE_O0B_OBJS): CC1FLAGS := -O0 compiles every <ov>_o0b.o at -O0. Registered as an h_exact dedup group whose source is the header — this works because dedup_integrate.group_members keys only on (binary, vram); a header-share is as valid as a macro-share. Reusable for any -O0 reach-134 giant (verify h_exact reach FIRST, R14 — an -O0 fn is only ×134 if it refs shared globals; the Phase-20 -O0 cluster was overlay-local ×1).
(3) THE memcpy SYMBOL — an __asm__ label, not a shared rename, and never a two-symbol alias. The struct-assign emits jal memcpy (gcc hardcodes the libfunc name). 0x8005C324 IS memcpy (libc2/MEMCPY.o) but the overlays auto-name it func_8005C324. Resolve it in the overlay-only symbol file symbols.resident.txt (memcpy = 0x8005C324) — NOT symbols.us.txt, because main DEFINES memcpy via MEMCPY.o and a shared symbol there would multiple-define in main's link. For an EXPLICIT same-address call elsewhere (the engine_core.h block-copy macro's variable-size memcpys, which can't be struct-assigns), keep the non-builtin C name func_8005C324 but give its extern an __asm__("memcpy") label: the literal identifier memcpy triggers gcc's built-in-memcpy codegen → byte mismatch, whereas the asm-label emits the same jal memcpy under a non-builtin identifier (only a benign "conflicting types for built-in memcpy" warning). splat REJECTS two symbols at one address (func_8005C324 + memcpy in one symbol file → error reading … at extract), so the single-symbol + asm-label is the fix, never an alias pair.
§39 — The ×1→×134 giant-endgame: propagate a matched -O2 giant via the NATIVE DEFINE-macro path (Phase 24 T7 §G close, 2026-07-08)
After the whale, two matched-but-×1 reach-134 giants remained — func_801770E0 (152) and func_801372B0 (207). The whale-session handoff proposed a hand-rolled src/shared/<fn>.h shared header per giant (the §38 whale mechanism). That was an -O0 necessity, not the -O2 path — both banked via the ordinary DEFINE_func_X() (engine_core.h) macro + dedup_propagate --recover. Five durable lessons:
(1) R14 — verify each "×1" claim against the bytes BEFORE acting. The handoff named THREE ×1 giants; func_8014E048 was ALREADY ×134 (a 134-member group + DEFINE_func_8014E048() in every overlay, done an earlier task — the handoff conflated its earlier ×1 state). Confirm the current member count (awk the group's binaries: list) + a representative other overlay's site (INCLUDE_ASM stub vs DEFINE_) before "finishing" any function. A stale handoff line is a claim, not ground truth.
(2) The DEFINE-macro path HANDLES pins/asm/anon-structs — the whale's shared header is -O0-ONLY. func_8014E048 (register-__asm__ pins + volatile-asm barriers), func_801770E0 (pins + anon-struct /s casts), and func_801372B0 (pins + asm-label externs + zero-byte barriers) ALL propagate ×134 as plain DEFINE_func_X() macros — compiles_standalone (real cc1, -O2) accepts every one (the handoff's "compiles_standalone REJECTS pin/asm bodies" was WRONG; the real blockers are (3)–(5)). Reach for a whale-style separate-object shared header ONLY when the fn is -O0 inside an -O2 TU (gcc-2.7.2 has no per-fn -O0 pragma → it needs its own object). An -O2 giant = the native macro path, full stop.
(3) THE overlay_files POST-WHALE-SPLIT GAP (the reusable fix). The whale rollout (§38) split every overlay's <ov>.c into <ov>.c/<ov>_o0b.c/<ov>_after.c, but dedup_propagate.overlay_files only scanned _a/_o0 — so every function in the post-whale _after region was invisible to dedup_propagate, for BOTH source-def-find (func_801770E0 → "no source overlay has it matched") AND stub-replacement in the 133 members. Fix = add _o0b/_after to the suffix list. Rule: any new per-file split suffix must be added to overlay_files or its functions silently can't propagate (a green per-overlay gate never fires because the site is never found — verify via the fleet dedup-check member count, not just the build).
(4) find_site's backward extern scan STOPS at a non-extern line — keep the extern block contiguous. func_801770E0's 5 externs had a /* comment */ on its own line between them; find_site collected only the externs below the comment and dropped the 4 above → compiles_standalone failed on undeclared callees. Relocate any standalone comment ABOVE the extern block (byte-neutral). (A trailing extern …; /* … */ comment IS tolerated; a whole comment LINE mid-block is not.)
(5) LOCAL-TYPEDEF giant → UNIQUE-rename + lift to engine_types.h, NEVER the bare name. func_801372B0 declared local SVEC/GLINE typedefs (above the fn, not in the extracted body) → compiles_standalone (common.h + engine_types.h only) failed on the undefined types. Critically, a DIFFERENT SVEC (u16 fields — a different fn) lives in _after.c, so lifting the bare name would double-define with a conflicting layout fleet-wide. Fix: rename to Svec_801372B0/Gline_801372B0 (all occurrences confirmed scoped to the fn's region first), lift the renamed types to engine_types.h (the plan-builder explicitly allows bodies USING engine_types.h types — only inline typedef/named-struct{ DEFS are rejected), byte-verify the source overlay unchanged (d19c9580), then --recover. The rename is byte-neutral (pointer-passing + struct-member codegen is layout-driven, not name-driven).
Result: func_801770E0 + func_801372B0 ×134, clean fleet 136/136, dedup 1811→1813/0, fleet 65.95→66.02% (byte-weighted gain larger — 152+207-ins giants). The giant endgame's matched set is now fully ×134; the only remaining reach-134 ×1 fns are small (3 propagatable stragglers 0x80174650/0x8012A018/0x80165CA0 + 17 local-type-blocked) = the T8 tail (Drew's focus directive: giants only, no small sweeps).
§40 — Structural families: the MECHANICAL symbol-remap (crack one exemplar → remap the rest, ~0 tokens) (Phase 25 T3/T7, 2026-07-08)
The reframe: regroup the UNMATCHED frontier by STRUCTURE (h_norm) not bytes (h_exact). A multi-member
h_norm family = the SAME engine fn recurring at the SAME address across overlays, byte-shattered only because each
member references PER-OVERLAY symbols (its level's data/code addresses). tools/family_manifest.py regroups + ranks
them (Phase-25: 2,764 families / 11.1 MB; levers by ov_SC01_077 membership: draftable / matched-free / absent).
h_norm families are TEMPLATES, not free dedup (byte-proven, R14): dedup_propagate --tier h_norm on a matched
exemplar banks 0/133 siblings — one C body can't name 134 overlays' different symbols (D_80187xxx in ov077 vs
D_8017Fxxx in ov000). h_norm masks the reloc fields, so h_norm-identical ⟹ diffs are RELOC-ONLY, but the reloc
TARGETS are per-overlay → not shareable by a single body.
TRAP — extract_unit mistook a DECLARATION for a DEFINITION (Phase 26 session 8, R14). Its guard was
not ln.rstrip().endswith(";"), but m2c writes declarations with a trailing comment —
M2C_UNK func_80178D40(s32, s32); /* extern */ — so the raw line ends in */ and sailed through. The forward
brace-scan then ran past the decl and swallowed the NEXT function's body, handing remap_hseq a garbage unit.
Measured: 15 of 35 substantial-family exemplars were phantom "matches" (all still INCLUDE_ASM stubs,
including func_80178D40 and the carried-queue func_801670E4), and 3 more anchored on the Phase-17
canonical-sig layer's extern … /* match-first, arity N */ decls and templated garbage — so those families were
silently unbankable. The whole-binary byte-gate rejected every one, so no wrong match was ever banked
(G3/P9 held) — but the engine burned a build per sibling on them, and any extract_unit-based readiness analysis
was wrong. Fix: strip trailing comments before the ; test. The general lesson (the phase's FOURTH silent-skip
bug, after find_site's braces, overlay_files' splits, and reconcile_decls' fn-ptr regex): a tool that
silently no-ops on input it cannot parse is indistinguishable from a tool that had nothing to do. Prefer
fail-loud on unparsed input, and regression-gate any change to a "proven" text scanner by snapshotting its output
over the whole corpus before/after — that is what caught this.
The lever — mechanical per-overlay symbol remap (tools/family_remap.py): two h_norm-identical members have
identical instruction streams except in the masked reloc fields. So disassemble both overlay images at ADDR
(extracted/retail/<SC>.CD.dir/FILE_<nnn>.dir/0.4.dec, vram 0x80128158), positionally pair the resolved reloc
targets (jal target; lui/lo combined addr via hi-register tracking — VERIFIED 22/22 vs splat .s), and substitute
the exemplar C's per-overlay symbol NAMES (D_<ADDR>/func_<ADDR>, UPPERCASE hex) with the sibling's. Shared
EXE/resident symbols (0x8002xxxx) map to themselves. Result = the sibling's C, generated for ~0 agent tokens.
The fleet sweep (tools/family_sweep.py): for each matched ov077 fn with unmatched same-address h_norm-siblings,
remap → gate into each sibling. Two-phase (stage all remaps grouped by (overlay,split) → gate each group ONCE) so
it's ~a few hundred builds, not 156×134. GOTCHAS: (1) gate remapped drafts with plain harvest_verify, NOT
gate_stage's canon/cast/sig_unify transforms — they perturb an already-correct remap → 0-bank. (2) match_one
pre-classify first: type-using families (LOCAL types like MatEntry, defined in ov077.c not engine_types.h)
CC1-FAIL in isolation → defer them, else harvest_verify bisection explodes (~30k extra builds); recover with
build_engine_types.py --source ov_SC01_077 --strip (lift local types → engine_types.h, byte-neutral) then re-sweep.
(3) func_ names are UPPERCASE-hex in src + .s filenames.
Result (matched-free harvest): +16,512 member-matches in ONE deterministic ~0-agent-token pass, R22 clean-fleet
136/136, fleet 66.02% → 70.82%. The 11.1 MB structural-family frontier is CHEAPLY recoverable: crack ONE
exemplar per family (the only real work — agent/wave), then family_sweep fills the ~133 members free. Transferable
to ANY overlay/bank-based decomp where a fn recurs per-region with per-region symbols (→ cross-project idea).
§40a — The DECL-RECONCILE pass (type-lift so remapped drafts compile in the sibling TU) (Phase 25 T7.2, 2026-07-08)
When the matched exemplar's body references a type defined INLINE in the source overlay's .c, the mechanical remap
(§40) reproduces that reference — but the type isn't in the SIBLING overlay's TU, so the sibling draft won't compile
(the sweep's match_one pre-classify tags it type/decl). Fix = lift the type into the shared header every overlay
includes (src/shared/engine_types.h via engine_core.h), then re-sweep. +1,729 members this way (fleet
70.82→71.32%, clean-fleet 136/136). Three load-bearing gotchas:
-
The pre-classify is a FALSE-NEGATIVE for lifted types.
family_sweep'smatch_onepre-classify compiles the draft in ISOLATION (mipsel-cpp -Iinclude+ a prependedcommon.h) — it does not seesrc/shared/engine_types.h(that's pulled only by the real overlay TU, via../shared/engine_core.h). So a type-lifted family still pre-classifies astype/decleven though it compiles + byte-matches in the REAL TU. Verify against the real gate, not the proxy (R14): a singleharvest_verifyof the remap into one sibling = byte-identical. Usefamily_sweep --no-preclassifyto route every remappable exemplar straight to the real-TU byte-gate (still the sole arbiter — a wrong draft is compile-failed/reverted with bounded bisection cost, since the lift makes most compile). -
Overlay SPLIT files (
_a/_after/_o0) are SEPARATE .o TUs — a blind "lift all splits" is UNSAFE. Each split has its OWN local type namespace, so two splits can define the same-named type with a DIFFERENT layout and never clash (different TUs). Lifting such a type fleet-wideconflicting types for <T>in the other split's TU. Also, a split may locallytypedef … MATRIX/VECTOR(PsyQ SDK names) — lifting those fleet-wide SHADOWS the real SDK types. So:build_engine_types --file src/<ov>/<ov>_after.c --exclude <colliding names> --striplifts one split's types, leaving conflicting/shadowing names TU-local. Detect collisions FIRST (the tool's ownfind_defs/find_typedefsacross all splits + the existing header; a same-name-DIFFERENT-body pair is the landmine). ov077's only cross-split collision wasBuf(_a≠_after);_a'sMATRIX/VECTORwere the PsyQ shadows → both deferred. Safe mechanical ceiling = base types + one non-shadowing split minus its colliders. The rest (cross-TU renames, PsyQ-layout verification, -O0 clusters) is genuine per-type reconciliation, NOT mechanical → backlog. -
Every lift must be byte-NEUTRAL. Type defs emit no code, so
--strip(move def → header) leaves the source binary identical — but VERIFY: rebuild the WHOLE source overlay (all splits) and confirm its locked SHA (ov077d19c9580) before sweeping. A broken strip / mis-ordered header surfaces as a compile failure or SHA drift at this cheap ~30s gate, before any expensive sweep. Then the full R22 clean-fleet 136/136 confirms no fleet-wide header collision. Tools:build_engine_types.py --file/--exclude,family_sweep.py --no-preclassify.
§40b — The reloc-tracker blind spot: the indexed-global idiom that hid the "reach-1 tail" (Phase 26 Task 1, 2026-07-11, byte-verified V0/V1)
The discovery (why the "36k unique tail" was largely a measurement artifact). Both norm_stream
(sig_image.py) and reloc_targets (family_remap.py) tracked lui-hi/lo pairs but popped the pending hi on
ANY R-type write (pend.pop(rd)). gcc-2.7.2's indexed-global access D[i] compiles to
lui $at,%hi(D); addu $at,$at,$idx; lw $v1,%lo(D)($at) — the addu PRESERVES the hi anchor (the index shifts
the runtime value, not the symbol). So every function that indexes a per-overlay global array left its %lo fields
raw in h_norm → the function normalized DIFFERENTLY per overlay → it looked fleet-unique (h_norm reach-1)
when it is actually a per-location family, AND family_remap's symbol map dropped those indexed D_ symbols →
the sibling draft kept the exemplar's array name → byte-gate fail (an earlier "unremappable" wall).
The fix (≤15 LOC, reloc_targets R-type branch only — NOT norm_stream). On add/addu (funct 0x20/0x21),
propagate the pending hi to rd when a source reg holds one, else pop:
elif op == 0: # R-type
funct = w & 0x3F; rd = (w >> 11) & 0x1F
if funct in (0x20, 0x21): # add/addu: address arithmetic preserves the hi anchor
rs, rt = (w>>21)&0x1F, (w>>16)&0x1F
if rs in pend: pend[rd] = pend[rs]
elif rt in pend: pend[rd] = pend[rt]
else: pend.pop(rd, None)
else: pend.pop(rd, None)
The %lo resolution is unchanged (pend[rs] + signext(lo) = the symbol; the index is a runtime reg). Leave
sig_image.norm_stream / h_norm UNTOUCHED — the fleet metrics, the proven h_norm sweep, and the manifest all
depend on its stable (blind) hashing; the h_seq family key (mnemonic skeleton) is unaffected by the tracker, so
families still cluster correctly, and the fix only makes the SYMBOL PAIRING correct so the remapped body byte-matches.
Verified (build-free, V0/V1 2026-07-11): func_801407F4 resolves 15/15 relocs vs splat .s (pre-fix: 10),
recovering the indexed arrays D_80187B88/90/B0; func_80141100 (no idiom) stays 22/22 identical (zero regression);
across 160 real h_norm sibling pairs the new remap output is byte-identical to the committed pre-fix output (96 SAME,
0 lost), differing only where it strictly recovers indexed relocs. Reproduce: .run/v0_reloc.py, .run/v1_regression.py.
Companion fix — single-pass simultaneous substitution. The old remap applied renames sequentially
(for src,dst: re.sub), which corrupts a chained/permuted map (D_A→D_B then D_B→D_C, or an immediate value
permutation 0x10→0xA & 0x4→0x10). Never tripped on h_norm data (disjoint exemplar/sibling address spaces) but the
h_seq imm engine (T2a imm_map_tier1/remap_hseq) needs it: build ONE \b(alt|…)\b regex over the full table (symbols ∪ self-rename ∪
immediates), replace via a dict lookup on the match — each source token is matched once against the ORIGINAL text.
This is also where the T2b cross-address self-rename (func_<FROM>→func_<TO>, definition + recursion) and the
T2a immediate imm_map merge into one pass. remap(addr, from_ov, to_ov, to_addr=None, imm_map=None) — backward
compatible (to_addr defaults to addr; the pre-26 same-address callers are byte-unchanged).
§40c — The h_seq per-sibling reconcile: templating a reconcile-class crack ×134 (Phase 26 Task 8, 2026-07-12, byte-proven)
A cracked exemplar whose body needs canon_sig_reconcile to bank (type-using / Ghidra-sig — the §41 def-side
wall) cannot template plainly: the reconciled ov077 body is TU-SPECIFIC (its canonical-sig casts +
Name_<addr> collision-renames fit ov077, not the sibling TUs). Plain remap_hseq of the reconciled body
re-hits the wall in every sibling → 0/134. Proven (Task-8 validation slice): the 23 triage isolation-cracks
gate 0/23 raw, reconcile 4/15 into ov077, but the 4 then template 0/4 plainly.
The fix — per-sibling re-reconcile from the RAW draft (the h_seq port of §41c's h_norm M2 path):
family_sweep --hseq --reconcile-raw <RAWDIR> → for each sibling, family_remap.remap_hseq_body h_seq-remaps
the RAW crack draft (symbol §40b + immediate T2a + cross-address self-rename T2b) then canon_sig_reconcile
re-reconciles against THAT sibling's own TU. The whole-binary byte-gate is the sole arbiter. Byte-proven:
the 4 triage cracks templated 463/0 ×~133 this way (0 failures). This is the pipeline every reconcile-class
crack (the type-using triage cracks AND the Fable5 cores) flows through to reach ×134. PURE cracks (clean
bodies, no reconcile) still template via plain --hseq (Task 5: 399 banked). Key: reconcile PER SIBLING from
the RAW draft, never remap the ov077-reconciled body.
§31-triage — R17 applies to CODEGEN residuals, never to a compile ERROR (Phase 26 session 7, Drew asked)
Before reaching for tools/reference/gcc-2.7.2/ or the docs/gcc-2.7.2-map/, classify the failure:
| Symptom | Cause | Tool |
|---|---|---|
| The build SUCCEEDS but the bytes differ | a codegen decision (regalloc / sched / cross-jump / CSE / loop) | R17 — read the pass, or gdb-on-cc1 (§45-B) |
The build FAILS to compile (conflicting types, undeclared, parse error) |
plain C89 semantics — gcc is CORRECT | our tooling. Reading gcc source tells you nothing |
Byte-example: the func_8015AE2C ×133 sweep blocker is conflicting types for D_801812A4 — two incompatible
file-scope decls of one identifier in one TU. That is not a gcc quirk; reconcile_decls was picking a
fleet-majority canonical type instead of the type the TU can actually SEE (its §8b carried decl layer).
A loop.c/global.c read would have been pure waste. Contrast func_8017BEBC (close=2, an allocno-priority
tie in global.c): compiles fine, wrong bytes → exactly an R17/§45-B target.
Rule of thumb: "wrong BYTES" → read the compiler. "won't COMPILE" → read our Python.
§41 — The DEF-SIDE canonical-sig wall: mechanically banking a drafted giant past conflicting types (Phase 25 T5b batch-2, 2026-07-09; tools/canon_sig_reconcile.py, byte-proven on func_8013B274)
The wall (dominant for GIANTS — ~universal, vs ~35% clean-bank for small fns): a drafter writes an
isolation-MATCH giant body (match_one c=0) with Ghidra-derived TYPED params — void func(u32 *a0, s16 *a2).
Placed in the real overlay TU it fails the whole-binary gate on conflicting types for func_X (a declaration
conflict, NOT a byte diff). The gate's sig_unify/cast_call_sites can't fix it and it banks 0/16. Two sources:
- The TU's callers reference the fn through the CANONICAL signature declared in
src/shared/engine_core.h(extern void func_8013B274(s32 a0, s32 a1, void *a2);) — but that decl lives inside aDEFINE_func_*macro, sosig_unify(which rewrites file-scope externs) never sees it. The draft's typed sig conflicts with it. - Absent an engine_core.h decl, a caller above the definition gives gcc-2.7.2 an implicit K&R
int func_X(); the draft'svoid/typed-param def conflicts with that. (This is why the overlay's "Phase-17 canonical-sig layer" at each_a.ctop usess32 func(s32,…)— it's K&R-int-compatible AND byte-neutral.)
The crack — reconcile the DEF to the canonical sig, BYTE-NEUTRALLY (3 steps, all proven on func_8013B274 → banked
d19c9580 byte-identical):
- Strip the draft's redefinitions of ambient symbols. A
typedef … P_TAG;identical toengine_types.h's is a redefinition error in gcc-2.7.2/C89 (not "compatible" like C11). Strip identical-def typedefs; stripexterndecls (func / dataD_*/memcpy) the TU or engine headers already declare — the draft's Ghidra-typed re-declaration is a conflict source. (memcpyalways: a mismatched prototype tripsconflicting types for built-in memcpy; the TU macros / builtin provide it.) - Rewrite the def signature to the canonical (
engine_core.hdecl if present, else the implicit-int-compatibles32 func(s32,…)at the draft's arity; arity-grow adds unused params so an N-arg implicit caller still matches). - Cast each type-changed param AT ITS USES — NEVER via an intermediate local. THE load-bearing insight:
u32 *a0 = (u32*)arg0;at the top introduces a fresh pseudo → gcc allocates it a different reg → regalloc shifts → byte diff (measured: cast-locals gave70ff4748, wrong). Casting the param in place —((s16*)a2)[i],((s16*)param + 1)(preserves stride!),*(T*)p— adds no pseudo, is free, and preserves the isolation-match codegen.tools/canon_sig_reconcile.pyblanket-wraps every use of a changed param in((origtype)name)(correct for index/deref/arith/member/already-cast alike). Give it--tu <split.c>so it treats that TU's already-declared symbols as ambient (strips their redundant draft externs too).
Result: 5/16 batch-2 giants banked purely mechanically (func_8013B274 80130D48 80167DBC 8016DC20 8018514C);
3 then family_sweep'd ×134. This is the phase's #1 def-side lever, now partly automated — reusable across the
whole giant tier AND the batch-1 backlog of "match_one-MATCH but gate-rejected" near-misses (the dominant gate-failure).
The residual walls (the other 11 — genuine per-fn T7, NOT this mechanical pass; backlogged with cause):
(a) non-identical ambient types — draft's SVEC/ApplyMatrixSV differ in layout from engine_types.h's (can't
strip: not identical; can't keep: conflicts) → needs a rename or a real layout reconcile. (b) data symbols declared
inside DEFINE_ macros (D_80078EB0) — macro-local, not file-scope, so stripping the draft's extern leaves the body
referencing an undeclared symbol, and keeping it conflicts → reconcile_decls.py/§33 byte-neutral-access-cast territory.
(c) primitive-typedef redefs the reconcile missed; (d) genuine byte-diff (reconcile compiles but codegen differs —
back to permuter/hand). Sweep fragility: a reconciled body carries ov_SC01_077-specific canonical sigs/casts, so
family_sweep to sibling overlays (with their OWN engine_core.h decls) byte-matches only some siblings (func_8016DC20
= 133 siblings failed → exemplar-only). A robust sweep of reconciled giants must re-reconcile per sibling TU (T7 follow-up).
§41a — v3.1: the def-side wall was ~71% TOOL-shaped — the five measured defects + the laws that dissolve them (Phase 25 T6, Fable5, 2026-07-09)
The T6 curriculum session R14-re-verified ALL 95 draftable-exemplar stubs (every draft in every
.run/drafts-t5* dir, best-of, match_one): 62/95 are genuine isolation-MATCH — the frontier's
"match" statuses were honest, and all 11 batch-2 "walls" + the 3 _o0 giants have byte-correct bodies.
A 6-iteration probe program (reconcile → splice into the REAL TU → full cpp|cc1|maspsx|as →
relocation-masked byte-compare of the fn inside the TU object, .run/t6_reconcile_probe*.json) then
decomposed the §41 wall into five mechanical defects of the v1 reconcile itself — fixed in
tools/canon_sig_reconcile.py v3.1, taking the mechanically-bankable set 10 → 44 of 62 (4,254 ins,
13 giants ≥145; func_8013DD68 gate-validated byte-identical through make build):
- Scalar-typedef dups (
typedef … u8;) must be stripped (C89 redef error) — same set match_one strips. Was 11 fns of "redef:u8/u16/s16/s8". - Canonical truth = the PREPROCESSED TU's file scope (cpp + brace-depth-0 scan), not a token-scan:
token-scanning counted macro/block-scope names as ambient (over-strip →
undeclared) and missed the TU's own decl of the fn (self-conflicting types). NB: aDEFINE_macro's extern lands at FILE scope when the macro is instantiated at file scope — and gcc-2.7.2 REMEMBERS block-scope extern types TU-wide, so both kinds bind later defs. - Never strip a draft extern — BLOCK-SCOPE-MOVE it (types verbatim) when no decl is visible above
the splice point. The draft's extern types are LOAD-BEARING (%lo-folding, access width, alignment:
an ambient-type rewrite byte-drifted 18/62 — e.g.
((s16**)&u8_sym)derefs at alignment 1 → lwl/lwr). Block-scope decls are private and legal even when a DIFFERENT file-scope decl exists below (recover_giant's idiom, generalized). Visible-above + identical → drop; visible-above + different → ambient + cast-at-use (fn callees per §17a-1; data via access-casts — alignment caveat above routes the narrower-object cases to §33/reconcile_decls TU-retype instead). - Colliding typedefs are RENAMED (
Vec3 → Vec3_<addr>, attribute-tolerant), never layout-reconciled: type names emit no code, so the def side NEVER has a real layout problem (SVEC/Vec3/Prim/S8/ApplyMatrixSV walls all fell to the rename — 2 giants banked). - Blanket use-site substitutions must skip decl lines ("decl lines are never cast" —
cast_call_sites' rule; violating it emits
extern void ((void(*)(…))f)(…)parse errors). Also: find the def on a COMMENT-MASKED copy (drafts' @stuck headers quote the sig and mis-anchor the rewrite).
The residue is three real classes (T6 curriculum tiers M3/M4/F): (a) arity conflicts with a
visible typed prototype (a banked caller's macro declares void f(void*) arity-1, the byte-true def
needs 3 params) — no draft transform can fix; the cure is the fix_arity_callers-class no-proto rewrite
of the engine_core.h macro extern (extern s32 f();, byte-neutral for the loose callers), R22-gated —
6 fns. (b) stale TU decl types from earlier banked drafts (u8 vs s16* on D_801870B0 etc.) →
reconcile_decls/§33 fleet-majority retype, then the drafts bank verbatim — 8 fns. (c) genuine per-fn
residue — 4 fns (incl. func_80166994's real mixed-arity loose-typing entanglement).
The ×134 sweep law (Q5, 6/6 proven): sweep the RAW draft via family_remap.symbol_map, then
re-run canon_sig_reconcile against EACH SIBLING's own TU, then gate. §41's "sweep fragility" was
exactly the missing per-sibling re-reconcile (the ov077-reconciled text carries ov077-specific
decisions). Sibling stubs live in the SAME split-file name fleet-wide (_after etc. — the whale-rollout
structure); read the asm subdir off the sibling's INCLUDE_ASM line.
Refuted: the batch-3 "-O0 in-context byte-diff needs an -O0-specific reconcile" — all 3 _o0
giants (329/198/154 ins) probe BANKABLE at -O0 under v3.1 unchanged; the old diagnosis was v1's
declaration perturbation, not a -O0 return-type law.
Probe-method notes (reusable): the in-TU masked byte-compare (insns_from_object(tu_o, fn) vs
insns_from_s(splat_s)) is a fast, link-free gate proxy — but it is jal-symbol-blind (mask eats
the target field), so harvest_verify stays the arbiter (G3/P9). And never hand-type a SHA: a
mistyped --good-sha made a byte-perfect gate run report MISMATCH — read it from config/check.*.sha.
§41b — T7 execution: the object-only probe OVER-counts BANKABLE by two link/rodata classes (Phase 25 T7-M1, 2026-07-10)
Executing the §41a curriculum banked 37 of the 40 non-jumptable M1 exemplars byte-identical through the
whole-binary gate (tools/t7_bank.py: reconcile-at-bank-time + harvest_verify, chunk-bet with per-round
re-reconcile for cross-fn ambient mutation). The 7-fn gap between the T6 probe's "44 BANKABLE" and reality is
two integration classes the T6 in-TU masked object-compare could not see — a sharper statement of "probe ≠
gate" (R14): the probe compiles to an OBJECT and masks jal/%hi/%lo, so it is blind to both rodata and link.
-
Switch jump tables in rodata (4 fns: the 3
_o0giants +func_8012ACE0). Their.textis byte-perfect — the unmasked diff is 100%j .L…/lui/addiu %hi/%lo(jtbl_…), all masked-EQ, zero real.textdiffs — but the switch emits a jump table in rodata (jtbl_801D836C…) that the object-only compare never looked at, and the whole-binary SHA diverges there. This REFUTES the T6 curriculum's "Q3 -O0 reconcile REFUTED" claim (the probe said the_o0giants bank; the gate says no) — the batch-3 "in-context byte-diff" finding STANDS. Route: the jump-table-in-rodata workflow (cookbook §8, the LZSS/§5a precedent — carve/match thejtbl_*rodata), F-band, NOT mechanical M1. -
Last-referencer link-wall (3 fns:
func_8016D688/D_801D9C20,func_8016D1D8/D_801D9C20+D_801D9C60,func_80165240/D_8018977C). The fn is the ONLY asm referencer of a scratch data symbol; splat auto-generates that symbol intoundefined_syms_auto.txtfrom the disassembly, so C-ifying the last referencer drops the symbol →ld: undefined reference to D_801D9C20. Compiles clean, fails at LINK (the object-only probe never links). Fix (M-linkwall tier): declare the symbol so ld resolves it — a manual undefined-syms entry or a splat data-symbol carve at that address (the bytes already live in the overlay image). Deferred pending the splat symbol-provisioning mechanics (don't guess an address into the byte-locked build). Reusable class: ANY bank that removes the last asm reference to an overlay-local data/scratch symbol.
Method upgrade for future curricula: an in-TU object probe is a necessary filter but NOT the gate — it
misses rodata (jump tables, float/string pools) and all link-time resolution. Size a "mechanical" tier from
the WHOLE-BINARY gate on a sample, or expect a ~15% object-probe over-count and treat the surplus as the two
classes above. tools/t7_bank.py (reconcile-per-round + chunk-bisection) is the reusable M1 driver.
§41c — T7-M2: the ×134 def-side-wall sweep via per-sibling RE-reconcile (Phase 25, 2026-07-10; 4,389 banks, ~0 agent tokens)
§40's mechanical family_remap (symbol-remap a matched exemplar → sibling) banks 0 for the def-side-wall
giants: the ov077-reconciled body carries ov077-specific block-scope-vs-ambient decisions, and each sibling's
DIFFERENT decompile state (different fns banked above the splice) needs those decisions RE-COMPUTED. The
Q5-proven fix, now tools/family_sweep.py --reconcile <rawdir>: per (exemplar, sibling), symbol-remap the
RAW draft (source→sibling via family_remap.symbol_map) then re-run canon_sig_reconcile v3.2 against
THAT sibling's TU, then the plain whole-binary byte-gate. engine_core.h is SHARED so the canonical sig is
identical fleet-wide; only the per-overlay symbol names + the sibling's visible-above set change.
Result: 4,389 of 4,655 member-remaps banked (94%) across 133 overlays for the 35 M1 exemplars — fleet
72.29% → 73.58% (+1.29%), R22 clean-fleet 136/136, dedup-check 1813/0. The 266 misses are per-sibling
loose-typing walls (the sibling banked a conflicting-type neighbor) → backlog. Cost: local cpp+build only,
zero agent tokens. Cost note: the per-member re-reconcile runs cpp on the sibling TU prefix for
visible_above — ~2 s/member (≈45 min staging for 4,655), then the group gate. _AMBIENT_CACHE/_VISIBLE_CACHE
are cleared per sibling-TU inside the sweep (the sibling source is static during phase-1 staging, so the caches
stay valid across exemplars for one TU). This is the endgame's economic engine for the def-side-wall giants:
crack + reconcile ONE exemplar, sweep it ×134 mechanically.
§41b addendum — M4 "reconcile_decls" tier is ALSO a probe over-count: 0/8 mechanical (Phase 25 T7-M4, 2026-07-10)
The T6 curriculum's third "mechanical" sub-tier (M4: 8 object-probe BYTEDRIFT fns projected to bank via a
reconcile_decls §33 TU-retype) is REFUTED by the whole-binary gate: 0/8 bank (reconcile_decls → canon_sig_reconcile → gate).
Root cause, same R14 pattern as §41b's jumptable/linkwall: the T6 object-probe's "BYTEDRIFT" does NOT imply a
data-type conflict. 4 of the 8 have reconcile_decls "touched 0" — no data decl even differs from the
fleet-canonical — so their in-TU drift is pure codegen (scheduling ORDER: e.g. func_8013E83C reads
D_80115118 before the prologue in the target; volatile-loss: func_801418F8's D_8011511A read-back;
callee interactions). The other 4 have real data-decl differences but reconcile_decls' byte-neutral cast
still perturbs the schedule. These 8 are F-band (byte-correct in ISOLATION — match_one c=0 — but drift
8–69 in the real TU) → permuter-ILS / §31, not a mechanical tier. reconcile_decls remains valid ONLY for a
genuine data-TYPE conflict where the cast is schedule-invariant (its §33 giant proofs); it is not a driftfix.
Net honest tally of the T6 "mechanical" projection (58 fns / ~2.5 MB): truly mechanical = M1 37 + M3-clean 2 = 39 exemplars (banked + swept ×134 = ~4,694 fleet fns, fleet 72.29→73.66%). The other 19 were probe over-counts → F-band/specialist: 8 M4 (codegen drift) + 4 jumptable (rodata) + 3 linkwall (undefined-sym) + 4 M3-residue (arity/loose-typing). Lesson (reinforces §41b): size a "mechanical" tier from the WHOLE-BINARY gate on a full sample, never from an object-only probe — it can't see rodata, link, OR in-TU codegen perturbation. Expect ~⅓ of an object-probe "BYTEDRIFT/COMPILE-FAIL" bucket to be genuine per-fn work.
§41d — void→s32 is NOT always byte-neutral: gate the RAW draft FIRST (Phase 26 session 6, byte-proven)
R14 correction to the Phase-17 canonical convention. The canonical-sig form ("s32 return — void→s32
is byte-neutral, §3a-1") is false for a void body with no return statement: promoting the return type
makes gcc-2.7.2 emit one extra instruction. Byte-proven on func_80182268 (31-ins jr, ov_SC01_077):
| draft | result |
|---|---|
void func_80182268(void *a0) (raw) |
MATCH, 31 ins |
s32 func_80182268(void *a0) (return type alone) |
DIFF, 32 ins |
s32 func_80182268(s32 a0) (what canon_sig_reconcile emits) |
DIFF, 32 ins |
The extra word is invisible in a leaf diff but lethal whole-binary: it pushed the isolated object's .text
4 bytes long, shifting every data symbol +4 → ~271,000 differing bytes and a 5-byte-longer image. match_one
said MATCH; only the whole-binary gate caught it (G3/P9 again).
The rule (generalizing §19's sig_unify lesson): every recovery pass is a FALLBACK, never unconditional.
canon_sig_reconcile exists to break the §41 def-side wall — it must not run on a draft that already compiles.
jtbl_family_bank now gates raw → (on failure) reconciled, and canon_sig_reconcile only promotes the
return type when a canonical extern actually demands it. Corollary: a function with no canonical decl
anywhere (grep engine_core.h + the overlay .c) should be banked exactly as drafted.
§42 — The F-band ≤28 regalloc crack wave: register-pin/DENSITY levers beat the permuter (Phase 25 T7 F-band, 2026-07-10; Ultracode 9-worker wave, 4/9 banked byte-identical, 266 swept ×134)
The F-near ≤28 band (14 fns / ~1,393 ins, one ov_SC01_077 h_norm exemplar each) is regalloc-order-DOMINATED
(9 of 14 = saved-register $sN allocation/ordering swaps). The permuter is structurally blind to this class:
it mutates C source, and pycparser rejects register __asm__ (§5a/§17), so a pure $sN-allocation swap has no
source-mutation reachable. Empirically proven this wave: permuter-ILS (regalloc-directed _REGALLOC weights,
8×120 s warm-restart) plateaued at base on EVERY regalloc fn (func_80134C20 stuck@3; schedule-class func_8017EF50
stuck@4, func_80168828 8→5) — 0 closed. A 9-worker Ultracode wave applying MANUAL §17/§31 levers cracked
7/9 to byte-0 in isolation, of which 4 banked byte-identical through the whole-binary gate.
The winning levers (all zero-runtime-code, semantics-preserving; full RTL in subagents/workflows/wf_0329d3c2-75c/):
- The §31 DENSITY lever (the workhorse for
$sNraces). To win a razor-thin saved-reg allocation, ADD a zero-byte dead-read__asm__ __volatile__("" :: "r"(v));on the pseudo you want gcc to prefer — it bumpsv's ref-count so the local-alloc density heuristic gives it the contested$sN. Calibrate the COUNT exactly (func_80134C20: ONE dead-read of the master reclaims$s5; TWO over-boost it into$s4→ 13-off). Proven: func_80134C20 (230, MATCH), func_801365B8 (155, 11→2). - The opaque asm-COPY for a param live-range split.
__asm__("addu %0,%1,$zero" : "=r"(copy) : "r"(orig));(or the §17 in-place re-tie__asm__("" : "=r"(p) : "0"((T)p));) forces gcc to keeporigin its incoming arg reg for early reads whilecopycarries the later reg — reproducing the target's single-pseudo live-range split. Proven: func_8017B614 (RC-9 hoist-vs-remat, MATCH). CAVEAT: reorg.c forbids__asm__in a delay slot, so an asm-copy that must fall in one lands a slot early (func_801365B8's irreducible 2-off). - Frame-pad induction:
s32 pad[2]; (void)&pad;— address-taken-then-discarded local defeats -O2 DCE, reserves 8 unused var_size bytes to match a target frame (0x20 vs 0x18), shifting every save offset;(void)&pademits zero code. Proven: func_80141A60 (MATCH). CAVEAT: frame-pad is ov077-specific — its 133 h_norm siblings ALL byte-drift on remap (each sibling's natural frame differs) → frame-pad families are EXEMPLAR-ONLY, NOT ×134-sweepable. - Array-initializer LUID shift:
s32 a[2] = {x, y};vs twoa[0]=x; a[1]=y;reorders the const-materialization LUIDs → sched2 emits the callee-save stores before the const chain (matches a target prologue-weave, S7). Proven: func_80180F10 (MATCH). - u16 zero-extend for a high-bit halfword store constant:* storing 0x8000+ through
unsigned short *zero-extends →ori $r,$zero,0xFFF8(opcode 0x34) vsshort *'s sign-extendaddiu/li -8(0x24). func_80141A60.
DIRECT register T v __asm__("$21") pins OFTEN BACKFIRE on giants — they wreck the prologue save-birthing order
and clobber the dead pinned regs (func_80134C20: direct pins = 97-off vs density = MATCH; func_80180F10: pin = 37-off
vs array-init = MATCH). Reach for the DENSITY lever first; use hard pins only when the residual is a clean,
uncontested-reg home (the dont-conclude-unsteerable memory still holds: try SOMETHING before declaring a wall,
but density > pins on the giants).
Attrition — isolation-MATCH ≠ real-TU bank (reinforces §41b): 7 iso-MATCH → 4 banked, 3 real-TU byte-drift
(compile OK, byte-differs). func_8017B614's drift = the T1 memcpy-builtin→call class (the sibling TU's
extern memcpy disables the builtin, so the worker's inlined lwl/lwr block-move lowers to a CALL) → re-crack with
field-by-field or explicit memcpy(x,y,8). func_801365B8 = a GENUINE irreducible cse-representative conflict → G4/INCLUDE_ASM candidate.
Tooling gotchas (each cost a false-fail cycle): (a) harvest_verify.py for a NON-resident binary MUST pass
--out build/<bin>/<bin> — its build() removes+sha1s --out (default build/resident/resident), so an overlay run
without it reports "final SHA None"/fail for EVERY draft even when byte-identical. (b) canon_sig_reconcile can't
extract a def whose body has a fn-pointer cast ((s32(*)(...))func) — such a draft banks RAW (no reconcile) if its
sig is already canonical (func_80180F10). (c) R22 clean-fleet: make clean nukes the WHOLE splat tree (asm/); make extract re-splits only the DEFAULT binary — you must make extract BINARY=$b for ALL 136, else 135 fail "can't open .s"
(a build-infra false-fail, not a byte mismatch).
Wave economics: 9 xHigh workers ≈ 1.66 M subagent tokens → 4 banked + 266 swept ×134 = ~270 fleet fns. The ≤28 regalloc band is genuine frontier — budget ~40-50% bank-rate per wave, NOT the mechanical tiers' ~94%.
§42a addendum — wave 2 (residuals + 29-100 band): iso-MATCH ≠ real-TU bank, the memcpy→struct-assign fix, +5 levers (2026-07-10b)
THE #1 LESSON — a crack worker must verify against the RECONCILED REAL TU, not isolation. Wave 2's 14 workers
produced 9 iso-MATCHes but only 4 banked — 5 iso-MATCHes DRIFTED in the real overlay TU (func_80136824/
80164930/8014DD8C/8016C188/80168828). The ONE iso-drift fn that banked (func_8017B614) did so because its worker
embedded the def into a scratch copy of the real split .c, compiled the WHOLE TU (builtins ON = the real
condition), and objdump-compared to the isolation MATCH — catching the drift cause and fixing it. isolation
match_one uses -Iinclude+prepended common.h; the real TU adds engine_core.h types, a memcpy decl, and the
reconciled sig — any of which shifts codegen. Wave-3+ crack prompt MUST require: after iso-MATCH, splice into a
scratch copy of src/ov_SC01_077/<split>.c, cc1 the TU, and confirm the target fn's bytes are identical modulo
link relocation — THEN report MATCH. (Cheap: one extra TU compile per worker; converts ~50% real-TU attrition to near-0.)
The memcpy-builtin→CALL fix (extends the T1 class, byte-proven func_8017B614): a small fixed-size mem-copy written
as memcpy(x,y,8) inlines to lwl/lwr/swl/swr in ISOLATION but lowers to a jal memcpy CALL in any TU that declares
extern memcpy (a sibling triggers conflicting types for built-in function memcpy, disabling the builtin TU-wide) →
byte-drift. FIX: typedef struct { u8 b[8]; } Blk8; *(Blk8*)dst = *(Blk8*)src; — struct-assign routes through
emit_block_move (identical lwl/lwr/swl/swr bytes) but references NO memcpy SYMBOL, so it is immune to the
builtin-disable. Mirrors the codebase's own family idiom (matched sibling func_8017B368 uses (*(SV4*)&D_x)=loc;).
Verify with cc1 -fno-builtin: struct-assign still emits lwl/lwr; the memcpy draft emits jal memcpy.
Five lever refinements (wave-2 journal wf_dbadb86a-6b7):
register intNOTregister shortfor a pin whose value is already sign-extended (anlhresult) —register int g __asm__("$6"); g = *(short*)p;pins to $a2 with nosll/srapenalty;register shortre-adds the extend (func_8017EF50).- Never density-dead-read a pseudo that is LIVE ACROSS A BLOCK — the
__asm__("":: "r"(v))adds a real instruction (count+1) and backfires; instead RESTRUCTURE the pseudo away (compute fresh at each use) (func_80136824). - When density fails, use STATEMENT-BLOCK reordering for
$v0/$v1birth order — group the var you want in $v0 so it is first-born + dense; density dead-reads that must keep a var live past its consumingsllproduce the wrong schedule (func_80164930). - Birthing-boost coupling: a single-set const-load (
li $v1,0x40) sinks to just before its EARLIEST-scheduled consumer, not to its C statement position — to move the load, reorder the CONSUMER store-block, not the assignment (func_80168828). - for-init LUID ordering controls the delay slot —
for (i=0, lim=0x19, p=P; i<N; i++)makesi=0win the beqz delay slot and emitslimbefore the pointerlui/addiu; a plain pre-loopint lim=…;captures the delay slot instead (func_80164930).
Wave-2 economics: 14 workers ≈ 2.47 M tok → 4 banked + 399 swept = ~403 fleet fns. Bank-rate 4/9 iso-MATCH — LOWER than wave 1 (real-TU attrition), fixable by the real-TU-verify rule above. The 5 nears (func_80134A74 71→16, func_80133AB0 →28 aligned, func_8012FCC4 beqz/jal delay-swap, func_80185BA4 65, func_801670E4 70 "irreducible") are permuter-ILS fuel / G4 candidates.
§42b addendum — wave 3 (Max, 2026-07-10c): THE STALE-OBJECT GATE TRAP + the read-global &-cast drift + fix
THE #1 METHODOLOGY BUG (invalidated wave-2's "iso-drift" labels; fix ALL gates). A per-function real-TU
check that does make build BINARY=<ov> >/dev/null 2>&1 and then runs asm-differ -o <fn> without checking
the build exit code and without removing the split .o first will diff a STALE object whenever the build
FAILS — reporting a phantom score 0 / "MATCH" for a draft that never compiled. Measured this wave: three
wave-2 "iso-drift" fns (func_8016C188, func_80168828, func_80136824) read as score-0 on the first pass, then
NOCOMPILE on a forced-clean pass (rm build/src/<ov>/<split>.o + exit-code check). Root cause of the false
score: the stale .o from a prior good build survives the failed compile, and asm-differ -o happily diffs it.
This is almost certainly why wave 2 mis-classified 5 fns as "iso-MATCH → drift" — several likely never
compiled in the real TU at all. MANDATORY gate shape (now in .run/crack3/diff.sh): git checkout <split> →
splice → rm build/src/<ov>/<split>.o → make build BINARY=<ov> and assert exit 0 → sha1sum the built
binary vs config/check.<ov>.sha (the real whole-binary arbiter) → only THEN asm-differ -o for the diff view.
Never trust a piped make build you didn't exit-check. (Compounds with the §42a --out gotcha — both produce
false PASS/FAIL on overlays.)
The *(T*)&D_sym read-global drift (a canon_sig_reconcile defect) + the fix — byte-proven on func_80164930.
canon_sig_reconcile rewrites an ambient-conflicting global access as *(u16*)&D_sym (cast-at-use, to dodge a
type conflict). For a write-only global this is byte-neutral (lui at,%hi; sh v,%lo(at) — direct addressing).
For a read (esp. read-modify-write) global it DRIFTS: &D_sym forces gcc to materialize the FULL address
into a held register (lui a0,%hi; addiu a0,a0,%lo; lhu v0,0(a0)) instead of the target's direct
lui v0,%hi; lhu v0,%lo(D_sym)(v0) — and it reuses that held reg for the store, shifting the whole schedule.
The wall: the target read needs lhu (u16) but the ambient TU decl is s16; a block-scoped
extern unsigned short D_sym inside the fn is a hard conflicting types ERROR in gcc-2.7.2 (cc1 exit 33,
NOT a warning — signed/unsigned short mismatch). The fix: flip the file-scope decl to the exact type
(extern s16 D_8018971C; → extern u16 D_8018971C;) — byte-neutral when the only other referencer is store-only
(func_801647A4 stores = 0x80 → sh either way) — and reference the global directly (no *(T*)&). Result:
whole-overlay d19c9580 BYTE-IDENTICAL, func_801647A4 unaffected. General rule for drafters/reconcile: a
read global that needs a specific load width (lhu/lh) must be a direct-typed lvalue at file scope, never
*(T*)&sym; align the whole TU on one type rather than casting at use. Sweep caveat: the file-scope-decl
flip is per-TU, so family_sweep --reconcile must also flip each sibling's decl (or the sibling's caller must be
an unmatched stub with no conflicting decl) — else siblings NOCOMPILE like the frame-pad class (§42 lever 3).
Wave-3 consequence: the wave-2 uc2_gate_* drafts are not reliable seeds — several NOCOMPILE (unreconciled
callee externs conflicting with the TU canonical-sig layer, e.g. conflicting types for func_80015954) and the
"iso-MATCH" labels were stale-object phantoms. Wave-3 targets must be re-reconciled + rigorously rebuilt per fn
(the .run/crack3/ harness), not gated from the wave-2 artifacts. Confirmed banks this wave: func_80164930
(the read-global fix above).
§42c addendum — wave 3 (Max orchestrator + CORRECTED Ultracode fan-out, 2026-07-10c): the real-TU-faithful parallel harness (rtu_match) + 7/9 crack, ZERO iso-drift
THE TOOL that makes a reliable crack fan-out possible — tools/rtu_match.py (real-TU-faithful, parallel-safe).
Wave-2 workers self-checked in ISOLATION (match_one), blind to in-TU decl/global-type/memcpy-builtin drift, so their
iso-MATCHes drifted at the whole-binary gate (~50% attrition). FIX: compile the WHOLE split .c with the candidate
spliced and INCLUDE_ASM neutralized (-DINCLUDE_ASM(a,b)= + -Isrc/<source> for the relative ../shared include)
→ masked-diff the fn. No asm/, no shared overlay build → many workers run in PARALLEL in per-fn temp dirs. Because
gcc-2.7.2 -O2 compiles each global fn independently, the neutralized whole-TU compile reproduces the exact ambient
context, so a rtu_match MATCH HOLDS at the whole-binary gate. Measured: 7 real-TU MATCHes → 7/7 banked
byte-identical (individually + combined d19c9580), ZERO drift (vs wave-2's ~50%). Corrected fan-out = 9 xHigh
workers ~1.27 M tok → 7 MATCH + 2 DIFF(→permuter). This is the reusable engine for the phase tail: reconcile-first
- rtu_match-gated + the levers below. Supports
//@EDIT old||newfile-scope pre-edits.
DURABLE LEVERS from the 7 cracks (all rtu_match-byte-gated):
- Callee-ARITY unblocks a delay-slot "steal" (func_8012FCC4 — the "irreducible" that wasn't). A spurious extra
register arg on a callee that is LIVE ACROSS the call blocks gcc reorg
fill_slots_from_threadfrom sharing a downstream constant into a branch delay slot (reads as an irreducible ~3-off beqz/jal delay swap). Before conceding a delay-slot residual as irreducible, RE-DERIVE THE CALLEE ARITY FROM THE ASM: drop the bogus arg → the target schedule falls out of stock reorg, no barrier/pin/mutation. - Pointer-holding global via
*(T**)&sym→lui;lw %lo(load ptr)+lh off(ptr)(deref) (func_80136824). A file-scopeextern u8 D_xthat actually HOLDS a pointer: read as(*(s16**)&D_x)[i]. Byte-neutral vs the u8 decl. - Array-decay CSE (func_80136824, the load-bearing extra): reading
extern s32 D_x[](ARRAY) as*(s16**)&D_xorD_x[0]makes gcc CSE the decayed BASE addr into a held reg (lui;addiu;lw 0(reg)reused) vs the target's per-use directlui;lw %lo(sym). FIX://@EDIT extern s32 D_x[];||extern s16 *D_x;(flip to a SCALAR POINTER). (Scalar u8 symbols fold %lo fine; only the array decays.) - §17 zero-reg-copy
x + zrfor a delay-slot-SAFE live-range copy (func_80134A74):register u32 zr __asm__("$0"); y = x + zr;copies a pseudo with NO__asm__op, so it CAN land in a branch delay slot (an__asm__volatile copy cannot, and disrupts delay-fill → +1 ins). Use to hoist a masked value into a bnez delay slot / before a range-check. - void→s32 flip for a discarded-return callee decl (func_8014DD8C): when a fn truly returns a value (
addiu $v0,1) but a sharedDEFINE_func_*macro in engine_core.h declares itextern voidand the caller DISCARDS the return, flip that macro-internal externvoid→s32(byte-neutral fleet-wide; stops the void-decl DCE'ing the return). R22-confirm fleet neutrality. Precedent: §20 func_8014EE14. - register-arg capture into a NORMAL pseudo for a callee-saved param (func_80168828, SWEEP-SAFE, no //@EDIT):
to force incoming
$a0into a callee-saved reg (targetaddu $s1,$a0,$zero): declare the fn(void), thenregister s32 a0v __asm__("$4"); s32 param_1 = a0v;. The copy into a normal pseudo (live across calls) gets a callee-saved home. A directregister ... __asm__("$4")leaves it in call-clobbered $a0 (wrong frame → 100-off). - Free-floating load temp for a scheduler hoist (func_8016C188): extracting an arg-load into its own statement
(
s32 t34 = *(s32*)(s1+0x34);) lets the scheduler hoist it early to fill a load-delay slot (vs pinned late by the call) — closed 63 mismatches at once.
block-extern-vs-definition is an ERROR, not a warning (func_80133AB0/8014DD8C): in gcc-2.7.2 a block-scope
extern whose sig conflicts with the function's own DEFINITION hard-errors (cc1 exit 33). A TU that forward-decls the
fn with a wrong/loose sig must be reconciled (match the def's sig; //@EDIT the caller decl when it discards the
return or the arg is already the right width in-register). The dominant "reconcile-first" wall for the F-band exemplars.
The 2 DIFFs (permuter tier), seeds in .run/crack3/wave3/: func_801670E4 (70→48; block birth-order levers landed,
"assign p/i late" shape from sibling func_8016A290) and func_80185BA4 (structurally 177/177, pure scheduler +
caller-saved temp-numbering residual, no responsive C lever) — decomp-permuter fuel.
§42d addendum — wave 4 (rtu_match fan-out over the mapped frontier, 2026-07-10c): 24/26 MATCH, +5 durable levers
META-YIELD (validates the frontier-map "reconcile-first" bucket): a 26-worker rtu_match fan-out over the
tractable-band draftable exemplars (the frontier map, docs/phase25-frontier-map.md) landed 24/26 MATCH
(20 banked byte-identical, 2 permuter, 4 needing per-fn link/drift fixes). Confirmed: for the F-band exemplars,
reconcile-first is often the WHOLE fix — several (func_80131B14) were byte-correct in the body and only their
TU-canonical decl layer conflicted; strip/align the decls → MATCH with no schedule/regalloc grind. The engine =
reconcile-first + rtu_match-gated + the §42/§42c/§42d levers.
NEW / generalized durable levers:
- Return-type flip goes BOTH ways (generalizes §42c #5). If a fn genuinely RETURNS a value but a discarding
caller's decl says
void, flip the declvoid→s32/short(func_8014FE60, func_8016CF04) — the void decl DCE's the return computation. INVERSELY (func_8016DF5C): if a fn is effectively VOID (barereturn;) but the draft declares its32, flips32→void— an s32 return keeps$v0LIVE at the epilogue, blocking reorg's eager fall-through delay-slot steal (a single-instruction cascade). Read the asm: does$v0carry a value out? - Address-recompute-vs-CACHE — the unifying read-global rule (subsumes §42b read-global + §42c array-decay CSE).
Taking
&D_sym(via*(T*)&symor a cached local ptr) makes gcc materialize the symbol address into ONE reg (lui;addiu) and CSE it across all uses → FEWERluithan a target that recomputes%hi/%loper reference (direct global access). When the target shows a freshlui $scratch,%hi; op %lo(sym)at EACH use, declare the global directly at the right type/scope (extern volatile unsigned short D_x;etc.) and reference it plainly — never&sym. When the target instead HOLDS the address in a reg across uses, cache it (T* p = ...;). Same root cause behind func_80164930, func_80136824, func_801418F8, func_80136334. - The full-inline-asm TRAMPOLINE idiom (func_8014FBC0, the 22×1996 family). The scratchpad-stack-switch
trampolines (func_8014F468/F6F4/FA04/FCFC/…) are hand-asm: the callee symbol AND the global live INSIDE the
__asm__string (%hi/%loescaped as%%), so ZERO C externs are declared → nothing to reconcile. maspsx 2.56 auto-fills thejaldelay slot with a nop (do NOT write an explicit post-jal nop). family_remap must substitute the callee/global symbols INSIDE the inline-asm string, not as C extern lines (the x134 sweep of an inline-asm family needs this — else siblings drop). - memcpy→struct-assign, re-confirmed at scale (func_8017B238, §42a): the TU's file-scope
extern memcpydisables the builtin → 8-byte moves lower to CALLs; model on the matched sibling's align-1typedef struct{u8 b[8];}struct-assign (routes emit_block_move, zero memcpy ref). Pair with theregister u8* __asm__("$16")+ in-place re-tie pin to hold the src pointer across the moves. - phantom-frame induction (func_80136334, §42-refined): a value live across BOTH arms of a branch makes gcc
reserve a spill slot the no-frame twin lacks — induce the frame with
s32 frame_pad[2]; (void)&frame_pad;.
Wave-4 economics: 26 workers ~2.36 M tok → 24 MATCH → 20 banked + swept ×134. Bank-rate 20/24 at the
whole-binary gate (4 hit rtu-blind link-walls / drift — rtu_match is .text-only, §41b/§42b caveat; those need the
whole-binary/link gate). The 2 permuter DIFFs: func_8012E364 (c=4), func_801549F8 (c=3, jtbl delay-slot).
§42e — propagating a CRACK ×134: the def-finder bug + the byte-drift residual (the "remap-fail" misdiagnosis)
Cracked F-band exemplars don't all propagate ×134 through family_sweep --reconcile — waves 3/4 dropped ~1,200
siblings. Diagnosis (a two-layer story; both matter for future sweeps):
- THE def-finder BUG (
canon_sig_reconcile, fixed) — mislabeled "remap-fail".family_sweep'sreconcile_remapreturns None on ANY failure and the caller counts it as "remap-fail", butfamily_remapitself SUCCEEDS (verify withtools/family_remap.py --addr … --from … --to …— it pairs the symbols fine). The real None came fromcanon_sig_reconcile.reconcileraising "no definition of func_X found in draft": its def-finder regex required a leading\n(\n(<type> fn(...)){), but a raw draft whose//@EDITheader lines were stripped has the fn definition on line 1 → no match. FIX:\n→(?:^|\n)(also match a def at draft start). This alone fully recovered func_8014FE60 (133/133 siblings) once paired with its shared-header return-type flip. - THE byte-drift residual (the genuine
--edit-remapwork). Families cracked with a file-scope//@EDIT(the array-decay pointer flip §42c#3, the no-proto flip) or a shared-header return-type flip (§42d#1) reconcile per sibling but BYTE-DRIFT, because those edits live OUTSIDE the function body thatfamily_sweepremaps: the pointer/ no-proto//@EDITtargets per-overlay decls (must be symbol-remapped + applied per sibling), and the return-type flip targets the ONE shared engine_core.h macro (apply once, globally — like func_8016CF04/8014FE60).family_sweepcarries neither. So a--edit-remap= {per-sibling: remap the exemplar's//@EDITsymbols and apply to the sibling split; once: apply any shared-header flip globally} recovers this class. func_8016DF5C/80136334/8013D9B0/80156044 are the backlog exemplars.
Forward rule (frontier-map leverage realism): a crack's ×134 is only free if its body is self-contained (no
//@EDIT, no shared-header flip). Before counting a cracked family's ×134, note whether it carries out-of-body edits;
if so it's exemplar+--edit-remap, not exemplar×134-free. LESSON (R14): trace a tool's real exception, not its
summary label — "remap-fail" was a swallowed reconcile-throw two layers down.
BUILT + measured (Phase-25 task B, 2026-07-10): family_sweep --edit-remap MANIFEST (JSON: per family, edits
= split-scope //@EDIT old||new in EXEMPLAR symbols, symbol-remapped per sibling via family_remap.symbol_map;
ec_edits = once-global engine_core.h flips, byte-neutral). Per sibling it applies the remapped edits to the split
- stages the
family_remapbody + gates the (overlay,split) group via plainharvest_verify. Orphaned edits from a failed sibling are byte-neutral (R22-checked). Manifest at.run/edit_remap_manifest.json.
THE CC1-CRASH WALL (the decisive R14 finding — only 2 of the 6 backlog families recovered): the whole-binary byte-gate is the sole arbiter, and it revealed that out-of-body-edit families split into two classes:
- array-decay pointer-flip (
extern s32 D_x[];→extern s16 *D_x;, a per-overlay symbol) — recovers cleanly ×134.func_80136824+func_80136334→ 266/266 siblings banked byte-identical, 0 failed (2×133). Light register pressure;family_remapbody + the remapped split-edit is sufficient (no reconcile, no extern injection). - register-pin-heavy (GTE 20-pin bodies
func_8013D9B0/func_8016DF5C; an exoticregister int zr __asm__("$0")zero-register pinfunc_80133AB0; the inline-asm trampolinefunc_80156044) — the original verdict here was "cc1-2.7.2 SIGABRTs compiling the sibling TU… ov077-TU-context-specific… NOT mechanically ×134-recoverable, stay exemplar-only (×1)." ⚠️ REFUTED — Phase-27 (Fable5 characterization,.run/giants/pin_crash_sigabrt.md). See the corrected verdict below; the pin-×1 ceiling was a STAGING-TOOL artefact, not a compiler wall, and it is fixed.
§42e-CORRECTION — the "pin-crash wall" is the extract_unit macro-drop, not the pins (Phase-27 T5 + SIGABRT
characterization, 2026-07-15). The SIGABRT is real and now exactly located — gcc-2.7.2/sched.c:2725,
create_reg_dead_note(): if (dead_notes == 0) abort();, a sched1 REG_DEAD-note conservation bug (flow places the
pinned reg's death on the fall-through path; sched1's clobber-aware per-block recount demands a death note for a
use-after-call in the CALL's block, whose harvested note-pool is empty → abort; backtrace abort ← create_reg_dead_note ← attach_deaths ← attach_deaths_insn ← schedule_block). But it was TRIGGERED by
family_remap.extract_unit dropping the body's file-scope #define dependencies, not by any TU context:
- Of the 4 "crash-walled" families only
func_8013D9B0ever genuinely SIGABRTed — and only because the droppedgte_*macros became implicit-declaration CALLS, putting its caller-saved pins into the fatal shape. The other three were exit-33 plumbing (a dropped multi-line typedeffunc_80133AB0; a dropped single-line typedeffunc_8016DF5C; the one-line-wrapper false-positivefunc_80156044) misfiled as crashes because the era one-big-split gate shared a TU compile with d9b0 and reported its Error-134 for all of them (the R14 lesson, recursed: one exit code folded three distinct failures into a phantom "universal SIGABRT"). - Properly staged, all four compile CLEAN in sibling TUs:
func_80133AB0133/133 (today AND at the era commit),func_8013D9B0133/133 (today, fleet-swept), df5c + x6044 spot-proven. T5's_carry_macrosfixes (a) the#definedrop; (b) multi-line typedefs route through theengine_types.hlift; (c) the one-line-wrapper false-positive is already fixed by the current comment-strip guard; (d) per-sibling decl flips are--edit-remap. - The fatal-pin predicate (checkable at DRAFT time, probe-matrix-proven): FATAL = a
register T x __asm__("$N")pin where$Nis caller-saved ($2–$15, $24, $25), the value is used after a CALL_INSN, and the post-call use has a branch-dependent use-then-conditionally-set shape. SAFE = callee-saved pins ($16–$23, $30) in any shape; caller-saved pins whose live range never crosses a call; use-only or single-level-conditional shapes;$0pins. (12-line minimal repro + probe matrix inpin_crash_sigabrt.md;-fno-schedule-insns/-O1 suppresses it — a safe "is this the dead-notes bug?" probe, useless for matching.) So ov077 banked these pins precisely because, in its TU (macros present), no pin crossed a real call. - DIAGNOSTIC (R14, corrected): exit 134 + the
create_reg_dead_notebacktrace = this bug, always; exit 33 = ordinary decl/typedef plumbing. Distinguish them (T4 surfaces cc1 stderr; don't fold both into "cc1-crash"). - Takeaway: the pin-×1 ceiling does NOT exist — P31's pin-propagation route is OPEN. Route pin-heavy families
back to the mechanical
family_sweepharvest (macros now carried); byte-identity per sibling is the byte-gate's question, but cc1-crash is no longer a barrier. (Array-decay pointer-flip families were never affected and still recover cleanly ×134.)
§43 — The K&R s16-param definition DISSOLVES the "narrow-param wall" for by-value register args (Phase 25 task A, Fable5 crack of the 369-ins giant func_80166994 ×134, 2026-07-11)
§17/§29 called a def with narrow-scalar by-value params an irreducible wall: it can't be no-proto-relaxed
(K&R default-promotion "changes the ABI") and often can't match the TU's s32 canon-sig prototype → stub it.
A Fable5 giant crack byte-proves that verdict is too broad for s16 (and s32) by-value params.
The refinement — use a K&R definition:
s32 func_X(param_1, param_2, param_3, param_4)
s32 param_1; s16 param_2; s16 param_3; s16 param_4; /* K&R: params declared narrow */
{ ... }
On MIPS all four args arrive in $a0–$a3 as 32-bit words. K&R promotes the s16 params to int for the
PROTOTYPE — ABI-identical to the canon-sig s32(s32,s32,s32,s32) (so no conflicting types, no //@EDIT
for the param types) — while the BODY still treats them as s16, producing the target's lazy per-use
in-place narrow/extend: sll aN,aN,16 ; sra aN,aN,16 on the arg register itself, with the raw values
stashed to callee-saved pseudos first (s3←a1 …) and re-extended per use after calls. The (s16)param_of_s32
cast form CANNOT reproduce this — it extends into fresh v0/v1 temps instead.
Triage tell (read it off the diff): target does sll aN,aN,16 in place on an arg reg + copies the raw
aN elsewhere first ⇒ true s16 param ⇒ K&R form. Extends into v0/v1 temps ⇒ it's a cast-of-s32, keep s32.
The return-type flip pair (void-return value-drop): if the def returns s32 but the ambient decls say
void, gcc-2.7.2 discards return expr; in a void fn (pedwarn) → you lose the target's exit
materializations (addiu v0,zero,1 / addu v0,zero,zero). Fix = flip void→s32 at BOTH:
- (a) the split canon-sig decl (
//@EDIT void func_X(...);||s32 func_X(...);) — a self-fn decl, EXEMPLAR- SPECIFIC (the canon-sig layer put it in ov077; siblings usually have 0 of these → make the split-edit OPTIONAL infamily_sweep --edit-remap: apply where present, never skip — the byte-gate is the arbiter), and - (b) the engine_core.h
DEFINE_func_*externs (ec_edit, once-global, byte-neutral because every caller discards the result — the func_80156044 trampoline precedent, now for a real returning fn).
Zero-footprint body ⇒ ×134-clean: put ALL typedefs + externs block-scope inside the function (a
conflicting file-scope typed extern is a hard cc1 error, exit 33, not a warning). Access a global as
&((Struct *)D_xxx)[i] over an ambient-compatible extern u8 D_xxx[];. No register __asm__ pins → it
propagates ×134 via family_sweep --edit-remap with no cc1-crash (contrast the pin-heavy §42e families
that SIGABRT in sibling TUs — structural cracks are the ×134-safe ones).
Scope (byte-tested = s16 only): proven for s16 by-value register params. u16/u8/s8/float
by-value, and any narrow param accessed via memory (sh/sw width differences), remain §29 walls until
byte-tested. So §29's blanket "narrow-param wall" narrows to "narrow params that aren't s16/s32-by-value in
an arg register."
Flywheel (R16): this idiom is now cheap-Opus-applicable — no Fable5 needed — for any giant whose diff
shows the in-place-sll triage tell. Check each remaining giant for the s16-param class before spending the
Fable5 tier. (Also caught: the prior wave's @stuck: none — MATCH note on func_80166994 was stale/false
— match_one re-ran DIFF 366/369; verify a "MATCH" claim against the bytes, R14, never trust a stale note.)
§44 — The Phase-25 cheap-Opus giant batch: 5 structural levers + the §43 extension (2026-07-11, 6 crackers over the frontier giants)
A 6-agent cheap-Opus batch (each applying §43 + §31 + the giant recipe, escalate-if-new-class) over the 6
frontier giants (209–399 ins, all reach-134): 3 banked ×134 (func_80166994 §43; func_80135480;
func_80163EC8), 4 pin-free/light-pin Fable5 seeds (the intrinsic wall), and 5 reusable levers. Meta-
lesson: cheap-Opus-first was right — 3 giants + 5 levers + clean seeds for far less than 6× Fable5 — and
§43 does NOT universally transfer: only 1 of 6 was a K&R-s16 case; each giant is its own class.
Lever 1 — §43 EXTENSION (widen the triage). The §43 tell "in-place sll aN,aN,16 on an arg reg" is too
narrow. If the target holds arg0 in two callee regs (a non-coalesced duplicate, e.g. move s6,a0; move s7,s6 — one for a sign-test, one for a mask-test), the K&R s16 arg0; def reproduces that duplication with
zero pins even when the sign-extend lands on the callee stash (sll $sN,16), not on $aN. (s16)cast
collapses it to one reg. Rule: try the K&R s16 form whenever the target shows a duplicate-arg0 pattern,
not only the in-place-$aN tell. (func_80133CD4.)
Lever 2 — pointer-var decl (avoid the &sym CSE-hoist). A held global pointer the target reloads per use:
declare it extern u16 *D_xxx and access it directly (D_xxx[i]), NOT via (*(u16 **)&D_xxx)[i]. The
&D_xxx form CSE-hoists the address into a callee reg (one lui;addiu, reused); the direct pointer-var form
emits a fresh lui %hi; lw %lo per use — matching the target's reload pattern. (func_80133CD4,
func_80135480.)
Lever 3 — block-scoped-pointer-split (local-alloc a reused output pointer). A single pointer reused to
write multiple output-store groups across separate return tails becomes a global allocno pinned to one
register, so it can't match a target that uses a different reg per tail. Split each store-group into its
OWN block-scoped set-once / used-N / dies-once pointer → each becomes a local-alloc pseudo that picks
the per-window lowest-free scratch, reproducing the target's per-tail allocation AND un-sticking coupled
delay-slot fills elsewhere in the schedule. Pin-free. (func_80135480, 258 ins; §31 RC-4 extension.)
Lever 4 — cross-jump the duplicated tail (steer a "permuter-only" dbr class). For a shared reset/exit tail
whose target shows call-arg-hoist into a branch delay slot + per-predecessor const-rematerialization: write
the tail duplicated inline in BOTH predecessors, NOT as one shared goto block. gcc-2.7.2 jump.c
cross-jumps the two copies, reproducing the exact dbr schedule (the call arg hoisted into the bnez delay
slot serving both paths; the mask re-materialized per-predecessor in the j/beq delay slots, sharing a reg
with the neighbouring lh). This cracks a residual §31 files under D1/D2 as permuter-only — it is
steerable. (func_80163EC8, 234 ins; one benign $v0 pin.)
Lever 5 — the "intrinsic wall" (what cheap-Opus canNOT do → Fable5/permuter). The §37 allocno-tie /
RC-6 pressure-lock / scheduling-position class: a pin-free structural seed floats at close 30–67 but the
residual is a whole-function register permutation or a schedule-position tie-break that no C-lever reaches
at the Opus tier — a caller-vs-callee allocno heuristic choice (func_80133CD4 s0v→$v0-vs-$s0). The cheap
tier's job here is to produce a pin-free, structurally-complete seed (correct body + count, zero file-scope
footprint) and hand off honestly (no forced/pinned false match). Escalation: Fable5 with
tools/reference/gcc-2.7.2/ and the §34 gdb-on-cc1 find_reg/post_mark_life method (it reads the
allocator's actual decision), or the pin-free seed → decomp-permuter.
⚠️ Phase-27 reclassification (regalloc-map §H,
.run/giants/*.fable.md): the three functions this lever once cited as intrinsic —func_8014D820"RC-6 pressure-lock",func_8016CBC0"coalescing knife-edge",func_801670E4"i=0/p co-location" — were each oracle-refuted:func_8016CBC0's callee-saved swap CRACKED byte-zero (afloor_log2density gap, and gcc-2.7.2 has no coalescing so "knife-edge" was never the class),func_8014D820's block-0 cracked pin-free 261→110 (reused-load-temp serialization; the sched.c:3199 pin was a red herring), andfunc_801670E4's dominant residual is RC-6 register allocation, not S3 scheduling (proven by the reg_renumber-swap oracle). The pattern (continuing map §F/§G): an "RC-6 intrinsic" verdict is usually map-incompleteness — audit for a density/lifetime/merge lever before declaring it. And the old "NEVER ship the pinned variant — it SIGABRTs sibling TUs (§42e)" is corrected (§42e-CORRECTION): the SIGABRT was a staging macro-drop, now fixed; a pin whose live range does not cross a call is safe to propagate. Prefer pin-free still (fewer failure modes), but the pinned-×1 ceiling is not real.
§45 — The flagship func_80133CD4 crack (399 ins ×134): the merged-variable permutation-breaker + the 1-death local-alloc gate (Phase 25 task A giant escalation, Fable5 gdb-on-cc1, 2026-07-11)
The 399-ins flagship — a "whole-function register permutation" that walled the directed permuter (masked-172) and had been tagged intrinsic for ~22 phases — fell PIN-FREE (×134-clean) to a Fable5 gdb-on-cc1 crack (whole-binary byte-gate BYTE-IDENTICAL d19c9580, banked ×134). Four reusable, byte-proven levers (worked example .run/giants/func_80133CD4.fable.c; dumps + gdb oracle in .run/giants/fable_cd4/):
Lever A — MERGED ACCUMULATOR VARIABLES break a "whole-function permutation" (the headline: 378→147 mismatches). When the target holds ONE $sN across disjoint value-regions (e.g. $s0 = {call-3 result → denominator → loop-accumulator}), gcc-2.7.2 global-alloc has no coalescing (K8), so one hard reg spanning disjoint regions can only come from one reused source variable. Merge the disjoint C variables into one → the allocno becomes call-crossing (K4, global.c:917) with a high merged ref-count → top density (K2, global.c:594 allocno_compare) → it allocates FIRST → plain regno first-fit (K3) reproduces the ENTIRE callee-saved permutation (the arg0→$s7/$fp end is §43's K&R double-copy). AUDIT for reused-variable chains BEFORE calling a whole-function permutation "unsteerable" — it is the original C reusing one variable per accumulator chain, not a compiler mystery. Retires the "N-callee vs N−1-callee permutation" giant-wall class.
Lever B — the 1-death local-alloc gate + the in-out-asm fix (67→13; found by a gdb ORACLE). A shared read-temp serialized through one register (target: lh; lh into the same reg separated by a byte-visible nop) is a 2-SET variable, which local-alloc REJECTS: reg_n_deaths != 1 (local-alloc.c:472) forces it to a GLOBAL allocno, allocated after every block-local qty → it loses the low-scratch first-fit and the whole caller-saved block permutes. No pure-C spelling yields 2-sets/1-death (flow emits REG_DEAD per region flow.c:2533; combine's 2-insn merges undo, its split path needs i1 = 3-insn combos only combine.c:1737; cse dissolves every 1-set spelling — all byte-tested). The escape (flow.c:2511): no REG_DEAD when a reg is SET in the same insn it last USES — expressible ONLY as an in-out asm __asm__("lh %0, off(%2)" : "=r"(h) : "0"(h), "r"(p) : "memory") (the "0"(h) input-tie makes read-2's lh use+set h in one insn) → 1 death → LOCAL qty → wins $v0 by qty-birth tie-break → the rest cascade by first-fit. PIN-FREE / ×134-safe (generic constraints, real opcode, no hard-reg names — NOT a register __asm__("$N") pin → no §42e sibling-TU SIGABRT). The "memory" clobber doubles as a delay-slot fence.
- THE METHOD — the gdb ORACLE (§34 flywheel). When a hypothesis reduces to ONE compiler-internal quantity, patch it mid-compile and diff the output (
break *local_alloc; set reg_n_deaths[h]=1). One run turns "plausible root cause" into "proven," licensing the (expensive) hunt for the C form that induces it.-dS/-dRdump sched1/sched2 with per-insn dependence lists on reload-born insns — read those before hand-modeling. In the shipped i386 cc1,qty_first_reglives at0x82c5404(theinfo addresssymbol is stale for this binary).
Lever C — offset-0 /s store asymmetry (last 5 diffs). p[0] = x expands non-/s (mem (reg)) while p[k≥1] are mem/s → a fixed-address (reload-born) load keeps its true-dep ONLY against the offset-0 store (sched.c:820 drop-clause needs /s+varying on one side, non-/s+fixed on the other). ((struct { s32 w; } *)p)->w = x; /s-ifies the offset-0 store → dep dropped → the load floats to the earlier delay gap. Store-side twin of §37's load-side /s lever.
Lever D — goto-shared-return isolates the exit li (tail). A common return 1 reached by goto ret1: gets its OWN basic block → stops sched1 hoisting the exit li v0,1 into a last-element load-delay slot cross-BB (freeing $v0 for a trailing temp); dbr still steals the li into the branch delay slot. Use when a return-constant materializes one instruction too early.
Transfer caveat (the §44 meta-lesson holds): each giant is its own class — Levers A/B are regalloc-permutation tools; apply them to a walled giant only when its residual IS a merged-variable or 2-set-temp permutation (read the .greg/.lreg tell first). The Phase-25 flywheel applies A–D via cheap-Opus to the sibling walled giants (func_8014D820 RC-6, func_801670E4, func_8016CBC0), escalating to Fable5 only for a genuinely new class.
§46 — The func_80178D40 crack (890 ins ×134, the heaviest core in the game): four LOOP-STRUCTURE levers cheap-Opus found by reading loop.c/jump.c/cse.c (Phase 26 session 8, 2026-07-13)
The heaviest jr core (890 ins, reach 134 = 477 KB) sat at close=39 with every case byte-exact but one. All 39 residuals lived in a single 44-instruction case body. No pins, no permuter — every residual was structural, and the permuter could not have reached any of them. Cheap-Opus + the §31 map closed it to MATCH 890/890. These four levers are new and general; the classes recur in every loop-bearing overlay function.
L1 — A loop's break must NOT land on the loop's own fall-through label (the PEEL lever).
When a break target coincides with the loop's natural fall-through exit, the RTL leaves NOTE_INSN_LOOP_BEG
followed by an unconditional jump — which fires duplicate_loop_exit_test (jump.c:2131, called from
jump.c:599). gcc rotates the loop and peels iteration 1; if the induction variable is provably 0 the peeled
i++ const-folds (li $a2,1) and drags a whole lui/addiu/lw address re-materialization block with it.
Fix: write goto <label>; instead of break; — the same destination, a different construct. It stays a plain
do-while and the peel vanishes. Tell: an extra address-materialization block and a li reg,1 at the loop head.
L2 — To make a reg-reg COPY survive, split its def and uses across extended basic blocks.
A source-level fp = q; always dies: cse's canon_reg rewrites later uses back to q (qty_first_reg keeps
the older register) and flow deletes the dead set. Every "just write the copy in C" variant collapses. But cse
resets its hash table at a label with >1 predecessor (a loop top), so a copy defined in a guard block and
used only inside the loop cannot be propagated away:
if (arg1->a.w != 0) { fp = arg1->a.w; ... do { ... fp ... } while (...); } /* addu $v1,$v0,$zero survives */
Test the memory, assign inside the branch. (Hoisting the same load into the loop preheader instead puts the
copy AFTER the lui/addiu — right copy, wrong place.)
L3 — A store merged into a shared tail must be written INSIDE the branch that reaches it.
if (i != 10) { D_801DAB2C = 3; break; } lets jump2 tail-merge the lui/sh into the shared tail label and lets
reorg steal the li 3 into the bne delay slot. Storing unconditionally before the if blocks the merge and
costs 3 inline instructions. (Same family as §8's cross-jump-merge-of-direct-stores.)
L4 — The un-coalesced loop copy = a NON-REPLACEABLE giv, and it needs all THREE parts.
This copy cannot come from C (see L2); it must come out of loop.c as a reduced giv whose move survives:
(a) an index giv — p = &D_801DA764[i]; (not a hand-rolled pointer walk);
(b) the giv is used outside the loop (e.g. the "found" body after the loop stores through p) → record_giv
(loop.c:4437) marks it non-replaceable → emit_insn_after (move dest, new_reg) at loop.c:3945;
(c) the biv increment is LAST in the body — loop.c inserts the reduced giv's addiu immediately before
the biv increment, so i++ at the bottom puts addiu $a0,$v1,4 into the loop-back delay slot. With
i++ at the top the giv-add lands at the top, nothing fills the delay slot, and reorg steals the loop-top
move into it — duplicating it, +1 instruction (891 vs 890).
L5 — Two structurally identical loops must differ in a REGISTER, or cross_jump merges their tails.
Loop3 reusing loop1's pointer pseudo put both "found" bodies in $a0, made them textually identical, and jump2
cross-jumped them into one block (−2 ins). Giving loop3 its own pseudo frees it into $v1, so
sw $zero,0($v1) ≠ sw $zero,0($a0) and the blocks stay distinct. Corollary of §8's cross-jump lever, inverted:
when you need two tails to STAY separate, separate their registers.
Meta (confirms the §44 law + Phase-23's finding). Every one of these was found by reading the actual
gcc-2.7.2 passes (jump.c, loop.c, cse.c in tools/reference/gcc-2.7.2/) and none by search: the residual
class was "the compiler produced the wrong BYTES", so R17 applies and the map/source is the lever. And the
tier held — cheap-Opus applying the documented map cracked the game's heaviest core; Fable5 was not needed.
§47 — The live-length SLIDER: splitting a global.c allocno-priority TIE with one zero-byte asm (Phase 26 session 8, Fable5 Max, byte-proven on func_8017BEBC 952 ins ×113)
The close=2 endgame class: allocation order and emission order are COUPLED (both follow creation/LUID
order), but the target needs them to DIFFER. func_8017BEBC's two hoisted invariant addresses (&g.sz1,
&g.sz2) tie in allocno_compare priority; the tie-break is creation order, so natural operand order gives
correct emission + swapped allocation (close=10), and permuting the asm operand list gives correct allocation +
transposed preheader emission (close=2). The permuter provably cannot reach it (not statement-permutable; 25 min,
no close). The fix decouples them: make the priority difference REAL so the tie-break never fires.
The method (no gdb needed — the dumps are the oracle):
- Compile with the pinned cc1 +
-dl -dg(a file input, sot.i.lreg/t.i.gregappear). Find the two preheaderaddius in the.gregRTL by theirconst_int(the sp offsets), take their insn UIDs, find the same UIDs in the.lregRTL to get the PSEUDO numbers, then read each pseudo's line:Register 228 used 13 times across 783 insns/Register 230 … across 782 insns. - Compute
pri = (int)((double)(floor_log2(n_refs) · n_refs) / live_length · 10000 · size)(global.c:594). Here: int(390000/783) = 498 = int(390000/782) — an exact int-truncation tie. - Find the boundary: the tie splits when the pair straddles an integer of
numerator/L. Here +1 on both (L = 784/783) gives 497 vs 498 — split. (−1 would also split; you can only ADD insns.) - The slider:
__asm__ volatile ("");placed BETWEEN TWO EXISTING GTE volatile asms inside the common live range. Adjacent to an existing volatile asm it adds NO new cse/sched barrier (one is already there) — it is purely +1 static insn at global-alloc time, emitting only#APP/#NO_APP(zero bytes).
Why the split can only go the right way: the later-created pseudo is defined one insn later in the
preheader and dies at the same last use → it ALWAYS has the shorter live range → pri(later) ≥ pri(earlier),
with equality only on a quantization plateau. Sliding the window off the plateau therefore always hands the
later-created pseudo the earlier allocation — which is exactly the "allocation ≠ creation" the target needs.
(If the target needed the OTHER direction, it would be unreachable by this dial — creation-order permutation
covers that case instead, §45-A.)
- Placement rule: next to an existing volatile asm (GTE-heavy functions are full of them). A bare
asm("")elsewhere is a cse table-flush + sched barrier + a maspsx#APPhop-killer (§42) — the classic perturbation trap. Between two volatile asms all three are already blocked. - The slider adds +1 live-length to EVERY pseudo spanning the insertion point — any OTHER exact-tie pair
straddling a boundary could flip.
match_oneverdicts the collateral instantly (here: none; MATCH first try). - ×N template-safe: the slider is body-local, pin-free, and travels with the template.
- Banked through the §8 whole-binary gate (jr function — match_one alone is NOT the arbiter, §8a): lazy
isolation → carve (9-piece interleave) → splice → BYTE-IDENTICAL. One TU-visible decl reconcile was
needed on the way (
D_800B9A02— declare the TU'sshort, force the unsigned access at use(*(u16 *)&D_800B9A02), the §8d sub-class (b) hand-move).
§48 — The 12-core jr crack wave: the ALLOCNO-PRICING dials and the EBB rule (Phase 26 session 8, Ultracode, 9/12 MATCH first pass)
Twelve heaviest unmatched jr cores, one agent each, §31/§46/§47 in the prompt: 9 byte-exact MATCH,
3 near (close=2/2/21), 0 dead ends. Every crack came from READING the pass (loop.c, jump.c,
cse.c, global.c, local-alloc.c, mips.c in tools/reference/gcc-2.7.2/), none from search.
The levers cluster into three families — and the first family is the important one, because it turns
register allocation into something you can steer from C without changing a byte.
A. ALLOCNO-PRICING DIALS — move a value into the register you want, byte-neutrally
All three exploit global.c:594 pri = (int)((double)(floor_log2(n_refs) * n_refs) / live_length * 10000 * size).
Higher density allocates first (and first-fit takes the lowest free reg). So any C edit that changes a
pseudo's refs or live-length while leaving the emitted insns identical is a free register dial.
- §48-A1 — SINK THE INIT INTO THE IF/ELSE ARMS (the biggest of the three;
func_8015A3C8).Byte-identical: jump2/cross_jump runs AFTER regalloc and merges the two identicalif (c) { min=A; grav=B; } → if (c) { min=A; grav=B; hi=0; } else { min=C; grav=D; } else { min=C; grav=D; hi=0; } hi = 0; /* nothing at the join */move rD,zerotails back into the single insn at the join. But at ALLOCATION time the pseudo was re-priced: measured 5 refs/318 live → 6 refs/162 live, priority 314 → 740 — enough to jump two other allocnos and take$s0. An init at a merge point is live across every path into the merge; the same init duplicated into the arms is not. Rule: to RAISE a local's priority, sink its initializer into the arms of a preceding if/else. To LOWER it, hoist the init to the join. (Companion to §47's live-length slider: that one shifts a length by ±1 to split a tie; this one collapses a length outright.) - §48-A2 — the LOCAL-ALLOC
$s0OCCUPANT (func_8015A3C8). A temp that is (a) referenced in ONE basic block and (b) crosses a call gets a callee-saved reg from local-alloc, before global-alloc runs. It lands in$s0and entersregs_used_so_far, which forces the highest-priority global allocno (thearg0copy — always first) off$s0onto$s1. If the target has arg0 in$s1, look for a call-crossing block-local temp and give it its own variable. m2c will happily merge it with a same-register variable; that merge destroys the occupant and rotates every$sreg. - §48-A3 — BLOCK-SCOPED PER-CASE TEMPS ARE A local-alloc TIE GATE (§44-L3, now source-cited;
func_8017A4AC). A function-scope scratch shared by 9 switch arms is a MULTI-BLOCK pseudo, andlocal-alloc.c:1765refuses to tie it — solhu/sll/sragets three different hard regs. Declare the temp INSIDE the case and local-alloc ties operand 0 to the dying input, collapsing the chain into one register. In a jr-switch dispatcher, NEVER share a scratch across arms.
A4 — SINK THE CONSUMER CALL INTO THE ARMS (the inverse of A1; func_8016AB6C, byte-proven)
A1 sinks an init to SHORTEN a live range. This sinks the consumer to DELETE the allocno outright.
if (c) { s3 = f(A1,P1)+3; p = P1; } → if (c) { s3 = f(A1,P1)+3; g(obj,P1); }
else { s3 = f(A2,P2)+9; p = P2; } else { s3 = f(A2,P2)+9; g(obj,P2); }
g(obj, p); /* `p` no longer exists */
The mechanism (why the join-copy is poison). A value defined in both arms and consumed only by a call in
the join becomes a cross-block global allocno whose copy-preferences include the ARG register. And
find_reg's copy-preference override scans for (i = 0; i < FIRST_PSEUDO_REGISTER; i++) — plain ascending
regno, NOT reg_alloc_order — so $5 (=$a1) deterministically beats $16 (=$s0). The only escape is
allocno_calls_crossed > 0, which makes find_reg set used1 = call_used_reg_set (global.c:906) and strip
the caller-saved prefs — but a pseudo defined after one call and dead before the next crosses ZERO calls,
so the arg reg wins and your value lands in $a1 instead of the target's $s0.
Duplicating the call into both arms deletes the pseudo: the pointer demotes to a block-local that crosses
a call, so local-alloc parks it in a callee-saved reg — and that also PRESERVES the §48-A2 $s0 occupant
that pushes arg0 onto $s1. The duplicated [addu $a0][addu $a1][jal] tail is identical in both arms after
reload, so jump2's cross_jump re-merges it into one join block — zero extra bytes — and dbr fills the jal's
delay slot with the second move.
Rule: if a value is defined in both arms and the target keeps it CALLEE-SAVED, duplicate its consumer call into the arms. cross_jump refunds the bytes. (Also note
set_preference,global.c:1535, appliesreg_renumber[]— so a locally-allocated pseudo appears in the pref set as its hard reg. That is why$16was even a contender.)
B. THE EBB RULE — the general form of §46-L2
cse resets its hash table at a label with >1 predecessor. So anything you need to survive cse must have its def and its uses in different extended basic blocks. Three instances, one rule:
- a reg-reg copy (§46-L2):
fp = q;always dies — unless defined in a guard block and used in the loop. - a held global address (
func_8015B950):la $sN,&Gin a loop preheader. gcc will NEVER emit this from plain global refs (a SYMBOL_REF is already a legal MIPS address), ands16 *p = &G;is constant-folded straight back by cse'sfind_best_addr. Fix: definep = &Gat the top of the loop body and use it only inside a case body reached through the jump table — cse starts a fresh table at the jtbl target label, cannot seep == &Gthere, thelasurvives, andloop.c move_movableshoists it to the preheader where global-alloc gives it a callee-saved reg. - a pointer-to-global held across calls (
func_8017A4ACL2):struct X *w = &D_SYM;at function top, used deep in a switch arm → survives, spans calls, gets$s0.
Corollary (
func_8013F350L2): a pointer-to-global survives only if every use is at offset 0. Withp[k], k≠0, cse'sfold_rtxfolds the SYMBOL_REF intoCONST(sym+k)— a legal address — and thelaloses its last user and is rematerialized away. For offset uses you need a struct (below).
C. TYPE- AND SHAPE-DRIVEN CODEGEN (the C type literally selects the addressing mode)
- §48-C1 — STRUCT vs SCALAR GLOBAL (
func_8013F350L1 — verified with 14 micro-probes). A scalar-typed global always folds to the direct macro (lui %hi; lhu %lo). A struct-typed global accessed by field always materializes a base (la $b,SYM+off($b)) once there are 2+ struct MEMs in the block. So: target showsla+ nonzero offsets → declare a struct. Target shows plainlui/%lo→ declare a scalar. A scalar and a struct at the same address may coexist; declare whichever each site needs. - §48-C2 —
lwl/lwr/swl/swrblock copy == a plain struct assign of a 2-BYTE-ALIGNED struct (func_80131340L1;mips.c:output_block_move, vanilla line 2580). Thelw/swarm is taken ONLY whenbytes>=4 && align>=4; align 1 AND align 2 both fall through to the unaligned pair. Sotypedef struct { u16 x,y,z,w; } V; a = b;emits lwl/lwr+swl/swr even between two 4-aligned stack slots. m2c's(unaligned s32)on an 8-byte object means "declare a 4×u16 struct and assign it" — not "hand-roll a byte copy". No memcpy, no packed attribute. - §48-C3 — DEAD-SIBLING-SCALAR TRAP (
func_8017A4ACL1 — cost 108 instructions). An arm that fills a param block and passes its address MUST use a real array. Declareu16 sp18, sp1A, sp1Cand take only&sp18, and gcc sees the siblings as never-address-taken → their stores are DEAD → flow deletes the stores and the loads feeding them. Diagnostic signature: the index chain repeated N times but only ONE load. - §48-C4 — CONST BEFORE LOAD (
func_8015A3C8L-B): to getluiinto a branch delay slot, give the compare constant a source temp one statement EARLIER, so its def precedes the operand load (lo = -0x94000; t = *p; if (t < lo)).gen_int_relationalforce_reg's a large compare constant AT the compare — i.e. after the load — and sched2 ties on LUID. Dead end: writingif (-0x94000 > t)does NOT work;compare_from_rtxcanonicalizes a CONST_INT op0 back to op1.
D. THE CROSS-JUMP RATCHET (the sharpest new trap — func_80131340 L-C)
Two cases needing opposite branch senses on the same test cannot be written as a mirrored if/else.
It looks right and even emits the right blez — but after cross_jump collapses both bodies to jumps,
jump.c's "invert a cond-jump that jumps over an uncond-jump" fires, flipping blez→bgtz; the two
cases are now byte-identical, so the NEXT cross_jump round swallows the compare entirely (−4 ins).
cross_jump + jump.c-invert together are a RATCHET toward collapse. Break it with explicit gotos into
labels living inside the other case's body: the branch targets become FAR, the invert-over-jump has no
adjacent label to fire on, and only the intended tails merge. (Same family as §46-L5: when two sites must
stay distinct, make them structurally distinct — separate registers, or separate branch targets.)
E. Meta
- 9/12 first-pass MATCH with cheap agents. The map is doing the work: every core was cracked by an ordinary Opus agent applying documented idioms + reading the pass. The Phase-23 tier doctrine holds — Fable5 DISCOVERS a class; everyone else APPLIES it.
- The
jr§8a check is mandatory and it caught nothing this time — because it was in the prompt. Every agent verified its.rodatatable against the target jtbl and reported the evidence. Bake the trap into the prompt, not into the post-mortem. - The 3 near-misses are all pure allocation/emission-order residuals (close=2, 2, 21) — §47-slider class.
§49 — The LUID DIAL: a zero-byte SCHEDULING dial (the sched.c analogue of §47) — func_8017A4AC (536 ins ×134), Phase 26 session 8
§47 splits a global.c allocno-priority tie by shifting a live-length. This splits a sched.c
rank_for_schedule tie by shifting an insn's position in the expand stream. Same philosophy: when a tie is
broken by an accident of ordering, change the ordering — without changing a single emitted instruction.
The residual. Two adjacent instructions transposed, registers already identical — a pure emission-order
residual. Not §47: global.c was innocent.
The mechanism (two passes, and the proximate cause is not the root cause).
- sched2 (proximate). gcc-2.7.2 schedules BACKWARD (
.sched2printsT-1= the last insn). At the tie point both candidates measuredpriority = 2and the same class vslast_scheduled_insn, sorank_for_schedulefell through to its final tiebreak —return INSN_LUID (tmp) - INSN_LUID (tmp2);— i.e. the tie is decided purely by position in the.gregstream. - sched1 (root).
adjust_priority→birthing_insn_p(bb_live_regs[dest] && reg_n_sets[dest]==1) hands every register-DEFINING insnLAUNCH_PRIORITY = 0x7f000001(max_priority, sched.c:2574). That boost lets the load chain seize the early backward cycles and sinks the un-boosted insn past its rivals — so the.gregLUID order comes out inverted and sched2's tiebreak then picks the wrong one. (A dead-end store —(set (mem) …), never "birthing", priority 2 — is starved and always floats to the front of its block.)
THE DIAL — materialize a call argument's sign-extension into an explicit s32 temp, placed AFTER the
intervening statement.
case 19: {
s16 a, b; s32 ea, eb;
a = ring[i]; i = (i+1) & 0x1FF; ea = a; /* NOT next to the load — see below */
b = ring[i]; i = (i+1) & 0x1FF; eb = b;
f(ea, eb); /* prototype widened to (s32, s32) */
}
It moves the sll/sra 16 pair EARLIER in the expand stream (lowering its INSN_LUID) while emitting exactly
the same instructions. Two placement rules are load-bearing:
- The prototype must be
(s32, s32)so the call itself adds no conversion. ea = a;must sit AFTER the store, not next to the load. Adjacent to the load,combinefuseslhu+sll+srainto a singlelhand you LOSE 3 instructions. The intervening store blocks that fusion, so thesll/srapair survives — identical bytes, earlier LUID.
The zero-byte dial family is now three (all emit nothing; all steer a tie):
| dial | pass | what it shifts | § |
|---|---|---|---|
live-length slider (asm("") between two volatile asms) |
global.c allocno priority |
live_length ±1 | §47 |
| sink-the-init / sink-the-consumer-call into the arms | global.c / local-alloc |
refs + live-range, or deletes the allocno | §48-A1/A4 |
| LUID dial (materialize a temp, placement-controlled) | sched.c rank_for_schedule |
INSN_LUID (expand-stream position) | §49 |
Method note (this is how it was measured, and it is reusable): -dS -dR on cc1 emits the .sched/.sched2
traces — the ready lists, the computed priorities, and the chosen order. When a residual is "two instructions
swapped, same registers", dump the schedule and read the tie: if the priorities are equal, you are on a LUID
tiebreak and the fix is a placement change, not a register change. Harness: .run/a4ac/dump2.sh.
§50 — Refinements that BOUND §47/§48 (from the func_80135EB0 wall, 21→6; Phase 26 session 8)
The one wave core that did NOT close still paid for itself: it produced the exact encoding of the §47 priority formula, a hard limit on the "cross_jump refunds the bytes" claim in §48-A1/A4, and a maspsx gotcha that explains a layout choice in the original we had never understood.
§50-A — THE PRIORITY ENCODING (use this; do not re-derive it).
pri = floor_log2(refs) * refs * size / (death − birth), where birth/death are 2 * insn_number — and
death is 2*M, not 2*M+1 (discriminated experimentally by the pre/post behaviour of a probe pseudo).
Ties break by ascending qty number = BIRTH ORDER. Worked: hoisting one statement above another pushed a
pseudo's birth one insn later, shrinking its qty range 11→10 → pri 0.4545 → 0.5, exactly tying a rival —
and because its qty is numbered first, it won the tie and took $v1. A tie you can compute is a tie you can
break: shift a birth, or shift a death.
§50-B — ⚠ THE CROSS-JUMP REFUND HAS A FLOOR (this BOUNDS §48-A1 and §48-A4).
§48-A1/A4 say "duplicate the code into both arms; cross_jump re-merges the identical tails after regalloc, so
it costs zero bytes." That is only true when the tails are ≥ 2 instructions, or when one path FALLS THROUGH
into the merged block. jump.c:1993 calls find_cross_jump(..., minimum=2) and does not count the jumps
themselves — so two js with a 1-instruction common tail will NOT merge. Only the minimum=1 path (a
jump compared against the code before its own target label) merges a single instruction.
Before using A1/A4, check the tail length. A 1-insn tail reached by two jumps costs you a real instruction — the refund does not arrive. (This is what turned a correct-registers attempt into 290 ins.)
§50-C — s16 PARAM + x | 1 MANUFACTURES A POISON TEMP; s32 DOES NOT.
With s16 arg1, v = arg1 | 1; expands to ior→T; sll; sra (REG_EQUAL sign_extend) — the ior can never
write v, so a temp is born, and its hard reg leaks into the allocator as a plain preference (see §50-D).
With s32 arg1 it is ONE insn writing v: combine's split reuses i2dest (combine.c:1818-1836, gated on
!reg_referenced_p(i2dest, newpat)) and no temp exists at all. Byte-neutral when the target's prologue does
no truncation (addu $s3,$a1,$zero). Widening a parameter can delete an allocno.
§50-D — COPY PREFERENCES BEAT PLAIN PREFERENCES, AND THEY NEED A BLOCK BOUNDARY.
set_preference (global.c:1535) runs a pseudo through reg_renumber[], so a block-local temp leaks its
hard reg as a plain preference — and find_reg scans plain prefs in ascending regno, so $v0(2) beats
$a1(5) deterministically. To override it you need a copy preference (find_reg checks those FIRST,
global.c:1000-1030), which requires the arg setup to be a bare (set (reg $aN) (reg P)) — and that means the
copy must live in a different basic block from P's last def, because combine's LOG_LINKS never cross
blocks. Route a second call through the same call site (a goto into a label at the shared jal) to give
that block ≥2 predecessors; reload then deletes the now-no-op copy — zero bytes.
§50-E — maspsx/gas MERGES lui $at FOR TWO STORES TO THE SAME 64 KB PAGE.
Reordering global stores to lengthen a live range loses an instruction: when two stores to the same 64K page
become adjacent, the assembler merges their lui $at. This explains a layout we had never understood — the
original interleaves D_801152AA / D_80126720 / D_801152A8 / D_80126724 / D_801152AC / D_80126722 precisely to
keep same-page stores apart. Never "tidy up" the store order of a matched function.
§50-F — the documented wall (honest defer). The residual 6 are a local-alloc.c:1568 (qty_compare_1 /
block_alloc) priority race needing the opposite winner from §50-A's: Q_x2 = 5 refs/range 18 → 0.5556 beats
Q_unkE = 2 refs/range 4 → 0.5. Flipping it needs either x2's range ≥ 11 (pri ≤ 0.4545) or unkE's range = 1 —
and both are blocked by sched.c, which always fills the load-use stall between lh and its consumer, and
places byte-free (reload-deleted) copies only in such stalls. The one productive angle left: a lever that
injects a reload-deleted no-op reg copy inside [lhu 4($a2) … sh %lo(D_801152AC)]. That is the whole delta.
§51 — TOOLING INTEGRITY: the silent skip, and how to hunt it
Phase 26-A (the inserted tooling-integrity audit). 68 findings across ~36 tools. This section is the METHOD and the LAWS; the findings themselves are in
docs/tooling-audit.md, the strategic why indocs/decision-log.md. Read this before writing any tool that scans the corpus.
§51a — The bug class
A scanner extracts N items from a corpus. The true count is M > N. Nobody ever compared N to M.
That is the whole class. It is not a typo, it is a structural blind spot, and it produced every one of the seven bugs found in Phase-26 session 8 and the twenty-eight found in the audit. Its signature:
- the tool reports success;
- the number it reports is smaller than reality and self-consistent;
- nothing downstream can tell, because a target that is never nominated produces silence, not an error.
Measured consequences in this project: 91.6% of all remaining work was invisible to target selection (a 3-file allowlist against a 14-file tree); the byte-gate could reach 4.9% of the canonical overlay; 62% of the endgame plan's byte-weight was already-matched phantom targets; and one 10% hole in a callee-signature oracle made nine byte-exact functions look like an intrinsic compiler wall.
§51b — Why the byte-gate cannot save you
The whole-binary byte-gate is a perfect CORRECTNESS oracle and a NULL COVERAGE oracle. It has never
once accepted a wrong match. It is also blind by construction to work never attempted: it has been
green since Phase 5, when 0% was decompiled, because INCLUDE_ASM pastes the ORIGINAL assembly.
A green byte-gate is compatible with ANY decomp percentage.
So the instrument the project trusts absolutely cannot see this class at all. Do not reach for it here.
§51c — THE METHOD (do not audit by reading the regex)
Reading regexes is the failure mode that wrote these bugs. For every scanner:
- Build a deliberately OVER-APPROXIMATING candidate detector for what it is supposed to find.
- Run both the detector and the real scanner over the real corpus (production data, not a toy).
gap = candidates − parsed.- Classify EVERY item in the gap — real silent skip, or justified exclusion. "I sampled a few" is not acceptable; if the gap is large, classify by shape and count each shape.
- Measure the blast radius against the corpus. Not "this could affect X" — go count how many functions/members/banks are actually affected today. Distinguish LIVE from LATENT (armed but not firing). Both are real; conflating them is not.
- Pair each finding with an adversarial skeptic told to REFUTE it. In this audit the skeptics killed 4 of 32 findings outright and corrected magnitudes in both directions.
§51d — THE LAWS
LAW 1 — Derive, don't re-derive (R33). Where a proven invariant answers the question, derive the answer from it rather than re-parsing the source.
- The invariant here: the fleet builds byte-identical, and
INCLUDE_ASMpastes the ORIGINAL assembly, therefore a function NOT wrapped inINCLUDE_ASMis byte-exact. progress.pyis the proof, in one file:weighted_metrics()derived from the invariant and was correct;classify()re-parsed C and inherited a bug. Same question, two tools, and the one that refused to re-derive was the one that was right.- The best outcome of an audit is a DELETED SCANNER, not a fixed regex. 28 findings collapsed to one
defect — a hand-maintained model of the corpus layout sitting on top of a filesystem that already
answers the question — and the fix was one derived oracle (
tools/corpus.py) and ~10 deleted scanners. A dict literal is strictly worse than the filesystem AND it fails OPEN (silently yields a plausible wrong answer) instead of closed. - Corollary: a derived fact cannot rot; a hand-maintained copy of it is a liability that grows with
every structural change.
.run/fuel_manifest.jsonrecorded 130 live stubs on 2026-07-08; the same code returned 30 six days later, because a TU split moved ~100 stubs out from under a dict literal.
LAW 2 — Assert your COVERAGE, not merely your correctness (R32). A tool that scans the corpus must compare what it found against an over-approximating candidate set, and fail on the gap.
⚠️ The sharpest lesson of the audit, and a correction to the first draft of R32:
build_engine_typeswas never silent. It printed[overlap] … handle manuallyevery single time, for four phases, while hard-exiting on 81% of its own corpus — the type-heavy tail's only sanctioned unblocker, unable to run on the corpus that tail lives in. It went unfixed because the message reads like a rare edge case rather than a four-fifths coverage failure, so nobody ever counted it. A loud failure that nobody counts is exactly as invisible as a silent one. "Fail loud" is not the rule. "Assert your coverage" is the rule.
LAW 3 — When an oracle is structurally blind to a class of error, add a SECOND ORACLE THAT CAN DISAGREE WITH IT (R34). Not a better assertion inside the first one.
config/symbols.us.txtdeclared a main-EXE RAM symbol at an address that is live code in every overlay. splat cut 97 real functions in half and invented 96 phantoms — 193 slices unmatchable by construction (one phantom's.sliterally beginslw $ra,0x10($sp)/addiu $sp,$sp,0x18/jr $ra: splat cut a function immediately before its epilogue and called the epilogue a function).- The byte-gate stayed green throughout and always would have, because the
.shalves are pasted back verbatim in original order. One phantom even got banked as a real match. - What exposed it:
sig_imagecomputes function boundaries from the ORIGINAL bytes without splat, and disagreed.make audit-corpusis now that second oracle, standing. - We had both oracles all along and never made them argue. Redundancy is only worth what you spend comparing it.
- ⚠️ Scope a cross-oracle check to the domain where the second oracle is genuinely independent. Run naively over all 136 binaries the same check reports 914 slices; the truth is 193. main/resident are signed by the Ghidra dumper, whose boundaries are shorter by design — so the comparison measures Ghidra's limits, not splat's errors. A check applied outside its valid domain does not become more thorough; it becomes noise.
LAW 4 — A rule that needs a human to remember it is not a gate. Make it structural.
.o ← .sis not a dependencymakecan see: assembly arrives viaINCLUDE_ASM, expanded to a.includeconsumed by maspsx/as after cpp, while-MMDtracks headers only. Re-extract, build incrementally, and make links a stale object.- This is not merely slow.
INCLUDE_ASMpastes the ORIGINAL bytes, so a stale object still yields the original image: SHA1 goes GREEN while the split you just changed is never exercised. A brokenconfig/change can be "verified" by an incremental build. - R22/H3 already legislate this ("clean rebuild"; "
make cleanafter anyconfig/change"). They are right, and they were broken anyway — by me, mid-audit. Soextractnow deletes the objects that include what it just rewrote. Structural, not advisory.
§51e — The false-wall pipeline (why this is not just hygiene)
A silent skip does not stay quiet. It compounds into a false wall:
wave_targetshands a drafter an asm path that does not exist (78 of 87 targets).- The drafter drafts against nothing and fails.
- The failure is booked into the backlog as a matching failure.
reserved_walls()reads the backlog and permanently blacklists a function that was never attempted.
Same shape with a lying closeness oracle: masked_diff left R_MIPS_PC16 unmasked, so 155 functions
scored a phantom non-zero against an unresolved placeholder that can never compare equal. An agent
grinds forever at a wall that is not there, and the result is filed as an intrinsic compiler residual.
Before you write up a wall as intrinsic, prove your instruments could have seen the alternative. How many of the walls "byte-proven" across 26 phases were lookup misses wearing a wall's clothes?
§51f — Checklist for any new corpus-scanning tool
- Does the filesystem already answer this? Then glob it — never keep a second copy (LAW 1).
- Does a proven invariant already answer this? Then derive it — never re-parse (LAW 1).
- Over-approximating candidate set +
assert found == candidates, failing with the unparsed items (LAW 2). - Does it print a count nobody checks? Then it is not asserted — it is decoration (LAW 2).
- Symbol regexes: any C identifier, not
func_[0-9A-Fa-f]{8}— curated names exist, and curated naming increases as RE quality improves, so afunc_-only oracle rots by design. - File lists: glob, never a suffix allowlist — the next split kind re-opens the hole.
- Definition detectors: handle K&R (
f(a)/int a;/{), multi-line signatures, and single-line bodies. K&R is this project's house style for exactly the biggest, highest-reach functions. And find the signature's closing paren with a real paren-walk —line.count('(')andsplit(')')[-1]both land on the wrong paren for a one-line body containing a call.
§51g — When the thing you are scanning has a GRAMMAR, parse the grammar (tools/cdecl.py)
The
cdeclbuild (Phase 26-A). Fifteen tools carried their own regex model of "what is a C declaration". They disagreed — two tools in ONE pipeline disagreed about whetherextern s32 D_a, D_b;is a declaration at all — and all fifteen shared one character class:extern\s+([A-Za-z_][\w\s\*]*?\bD_[0-9A-Fa-f]+\s*(?:\[\s*\])?)\s*;which cannot hold(,,, or a non-empty[N]. So three whole shapes were invisible to every one of them: fn-ptr / jump-table arrays (extern void (*D_8018E858[])(void);), sized arrays (extern s32 D_80127530[4];— one unparsed[4]blockedfunc_801387B8in 134 TUs), and multi-declarators (where the whole line is dropped, not just declarator 2..N).
THE LESSON. Do not enumerate shapes — parse the grammar. The audit's own prescription was a shape-aware alternation per tool. That is N more chances to diverge, and it only ever covers the shapes someone remembered. C's declarator grammar is small, closed, and total: it describes fn-ptr arrays, 2-D arrays, multi-declarators, fn-ptr parameters and K&R identifier-lists without being told they exist. A ~250-line recursive-descent parser is less code than the fifteen regexes it deletes, and it is total by construction rather than by anyone's memory.
Measured, whole corpus: 2,952,246 depth-0 statements → 2,731,521 declarators, 0 parser defects; 50,405 distinct declarations round-tripped through the real cross-gcc, 0 rejected.
LAW 4 — The candidate set can be DERIVED too (R33 applied to R32). Every other tool here hand-maintains an over-approximating candidate regex in order to measure its own coverage. It does not need one: at file scope, C admits nothing but declarations. So the candidate set is every depth-0 statement, taken from the grammar itself — the strongest assertion available, and one that cannot rot, because there is no second model to drift.
LAW 5 — Let the compiler adjudicate your own coverage gap. When 40 statements would not parse, the
temptation is to decide for yourself which "don't count" — which is grading your own homework, the exact
habit that produced the fifteen bugs. Instead, hand each one to gcc: a statement gcc also rejects is
not C, so rejecting it is correct and the INPUT is corrupt; a statement gcc accepts and your
parser does not is your defect. The exclusion set becomes a verdict from the C front end rather than an
opinion. (Outcome here: 33 residual, all 33 adjudicated NOT-C by gcc, all in dead .run/drafts*
scratch, none in src/. And an honest R14 near-miss: they were written by a recovery tool that
prepended extern to an if statement — but the current oracle emits 0 garbage over 300 signatures,
so the bug was already fixed in Phase 19. Mechanism confirmed, consequence nil: verify the blast radius,
not just the defect.)
LAW 6 — Column 0 is not file scope. m2c emits goto labels (done:, block_13:) at column 0 inside
function bodies. Every tool that equates "starts at column 0" with "is at file scope"
(reconcile_tu.tu_visible, scope_data_externs, jr_isolate_all) rests on a heuristic the corpus
violates. Track brace depth; it is ten lines.
LAW 7 — A raw text scan cannot see a TU's declarations, and cpp can. src/shared/engine_core.h is
23,546 backslash-continued lines inside 1,801 #define macro bodies. A declaration in a macro body
declares nothing — it becomes a declaration only where the macro is invoked, above the invocation
point (the §8c law). So a raw scan is wrong in both directions: skip the #defines and you miss all
1,801; read them and you invent ambient decls the TU never had. cpp answers it exactly, in 54 ms on
the largest TU (~20 s for the whole 678-TU fleet, cacheable) — which is why
canon_sig_reconcile._file_scope_statements was the one scanner the audit measured CLEAN, and why any
tool carrying its own model of macro expansion (reconcile_tu._macro_externs(), .rstrip('\\') and all)
is re-deriving what the build already guarantees. Corollary: reconcile_decls.DATA_DECL_LINE_RE is
line-anchored, and every decl in engine_core.h ends in a \ — so its "authoritative tier" over the
shared header finds zero declarations. It has always been empty.
LAW 8 — A block-scope extern is not a file-scope canonical. gen_harvest_targets and sig_unify
both count externs declared inside a function body (6 of them in engine_core.h) as authoritative
file-scope declarations. A block-scope decl is private to its function and expires at its }; promoting
one to ambient truth is precisely the confusion behind the §8d conflicting types for D_801812A4 wall.
A depth-aware parser excludes them for free.
And the dividend: a real parser lets you assert things that were previously unaskable. With cdecl
in hand, "every canonical signature the callee oracle emits must PARSE as a C declaration" becomes a
one-line check. Before it, nothing in the repo could tell a signature from garbage — which is how
extern if ((func_80029178(0x119) & 0xFF) != 0); got written into a draft and then read, downstream, as
a compiler wall.
Gate: make audit-cdecl (coverage + gcc). Standing, because a loud failure nobody counts is exactly
as invisible as a silent one (LAW 2).
LAW 9 — THE ADJUDICATOR MUST BE THE COMPILER THAT COMPILES YOUR CODE. Not the C standard, and not
whatever gcc is on the PATH. Building cdecl.compatible() ("will this declaration coexist with that
one?" — the question every recovery pass actually asks) against modern mipsel-linux-gnu-gcc and
against the real gcc-2.7.2 cc1 gives three different answers, and only one of them is the truth:
| declarations in one TU | C standard | modern gcc (C11) | gcc-2.7.2 cc1 |
|---|---|---|---|
typedef int X; twice |
error | accepts | ERROR — redefinition of 'X' |
extern u16 X; + extern volatile u16 X; |
error | error | ACCEPTS |
void X(s16); then void X(); |
error | error | ACCEPTS |
void X(); then void X(s16); |
error | error | ERROR |
Validate against modern gcc and you encode rules cc1 rejects and miss rules cc1 accepts — a recovery pass
that "approves" declarations the real front end refuses is exactly the failure this whole audit is about.
cdecl --compat therefore adjudicates with tools/bin/gcc-2.7.2-psx/cc1, and now agrees with it on
1,485/1,485 live corpus pairs. Two rules in that table were refuted by the oracle after I had
already written them from the standard.
🏆 AND THE LAST ROW IS A WALL COMING DOWN. The no-prototype rule is ORDER-DEPENDENT: only
()-then-narrow-prototype fails; prototype-then-()compiles fine. Phase 15 wrote this class up as "the 159 arity/narrow-param conflicts — no clean deterministic fix" and closed it. The stated cause does not hold. Whether the resulting codegen matches is a separate question the byte-gate answers — but the door was never locked. It took four three-line probes and 90 seconds to find out. Probe the compiler for FACTS; read its source only for LEVERS; byte-validate both (we readgcc-papermariofor five phases believing it was 2.7.2 — it was 2.8.1).
LAW 10 — DERIVE WHICH TU, TOO — not just what is in it. cast_call_sites / sig_unify /
reconcile_decls take --src-file, an optional, hand-passed flag naming the TU to canonicalize
against; unset, it defaults to src/<ov>/<ov>.c. No caller knows about the Phase-26 _jr_<ADDR> carves.
Measured on ov_SC01_077: 263 open stubs across 12 TUs — only 13 in the main .c. 95.1% of drafts were
being reconciled against a translation unit that would never compile them, while harvest_verify
(fixed in A3) correctly spliced them into the right one. corpus.stubs() already knows the answer — the
INCLUDE_ASM line is self-describing. Ask, don't assume. Fixing it took the callee-conflict repair
from 8 → 58 of 196 drafts (7× reach).
AND THE HONEST OTHER HALF (P9/R14): those 58 produced ZERO new banks. The historical draft tail fails on codegen, not plumbing —
func_801387B8, which the audit blamed on a single unparsed[4], is really 67/100 instructions off with a$s0/$s1swap. What the fix did buy is real but narrower: 52 drafts moved from "won't compile" to "compiles, N instructions off." That is not a bank — it is the difference between an invisible failure that reads as a compiler wall and a scored near-miss the permuter and the §47/§48 dials can act on. Which is the audit's thesis exactly. Do not sell it as more than it is; three times in one session a confirmed mechanism produced a null consequence.
LAW 11 — A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT. The single worst defect in the audit was not in a scanner. It was one default argument in the caller of a scanner we had already fixed:
# tools/gate_stage.py:315
summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...) # <- ALWAYS the main .c
src restricts the byte-gate to ONE translation unit. A3 had just taught harvest_verify to derive
each draft's home TU when --src is omitted, lifting the byte-gate's reach from 4.9% to 100% — and
gate_stage never omits it. So the primary banking path (every wave, the grinder, the orchestrator,
bulk_harvest) stayed structurally incapable of banking 250 of 263 stubs, after the fix, because of
its own caller's default.
AND HERE IS WHY IT SURVIVED 26 PHASES.
harvest_verifycannot splice a draft whose stub is not in the TU it was pointed at, so the draft simply never verifies — and is then logged asnear/failed, i.e. as a matching problem. The wave reports a low close-rate and the function goes to the backlog as a residual. A tool that CANNOT bank a function is indistinguishable, in every log this project keeps, from a function that CANNOT BE banked. Proof, same draft, same gate, same second:gate_stagerejectedfunc_80129C40;harvest_verifyrun directly (no--src) verified it byte-identical and banked it.
And a counting bug that hid the hiding: when match_one says MATCH but the whole-binary gate rejects,
gate_stage logs status="near" and never increments the counter. A run of 63 such drafts printed
banked 0, near 0, failed 0 — three zeros that do not sum to 63 — for phases. Nobody added them up.
(LAW 2 again, and note the shape: the number was not wrong, it was ABSENT.)
Checklist item, promoted to the top: after fixing a scanner, grep every call site and ask does any
caller pass a default that re-disables this? An audit that stops at the callee is half an audit.
§52 — The WALKER-FAMILY skeleton: 6 regalloc-order levers + a deeply-characterized intrinsic wall (func_80178004, 165 ins ×134; Phase 26, Fable5, 2026-07-15)
func_80178004 is the exemplar of the regalloc-order residual class (12 reach-134 siblings share it: the
9 close=0 + 3 close-21+ cores). A ~93-min / 477k-token Fable5 pass drove a PIN-FREE draft to
structure-exact (163 vs 165 ins); residual = pure register identity — byte-verified (match_one 126/165
masked; the 126 is dominated by one $s0↔$s2 swap rippling ~90 lines). It did not byte-match: the wall
reduces to three compiler-internal integers and is (probably) intrinsic to vanilla gcc-2.7.2.
Correction (R14): the historic "pinned MATCH" was a myth — the pinned seed was NEVER a match (best
historic permuter score 5, pinned); this is the deepest state the function has reached. Artifacts + gdb
oracles under .run/fable_80178004/ (repro runorc.sh); Fable5-derived, headline byte-verified.
The 6 levers (the "walker-family skeleton" — apply to the 12 siblings; levers 1-5 retire ~half the residual)
- Mutated-parameter pointer. Declare the walking pointer AS the mutated parameter (
u32 *p=arg0, thenp += …). Anyp = (u32*)a0COPY leaves the initial value live in the param pseudo → cse1 rebases every store block onto constant offsets and DELETES the pointer-walk. The single biggest lever; transfers to every walker-style function. - No derived-base variable. Never declare the second pointer — write
p[1..3]directly and let loop.c mint a combined DEST_ADDR giv of the biv (anchor = last-recorded giv → the -8/-4/0 offsets; preheader init reads the biv reg). A source-levelqbiv makes loop.c reduce the (q-4,q-8) pair into an EXTRA pointer (loop.c:3824 worthwhile test). - In-loop constant remat. A constant the target recomputes in-loop must be a user variable assigned in
BOTH if/else arms (
n_times_set==2fails scan_loop's movable gate, loop.c:698-712 → stays in-loop). A bare literal is hoisted. - Split the OR across two statements.
t = …|0x4000; p[3] = t|0x6d160000;beats tree-level constant reassociation → in-loopori+ a hoisted lui-only temp. - Break biv-recognition. Interpose one statement between
n = i+1andi = n(basic_induction_var follows a copy only to the immediately-previous insn, loop.c:4862) → the target's literal counter survives instead of a<<16giv. - Diagnose allocation walls with gdb-on-cc1 FIRST.
peek2.gdb(allocno_order/find_reg) + a 3-integer oracle (reg_n_deaths/reg_n_refs) tells you in minutes whether a register identity is even reachable — before grinding C.
Why the wall is (probably) intrinsic
The walker is a block-local 1-death qty → local-alloc runs first and hands it the first free callee-saved
$s0 (local-alloc.c:472/2103; nothing can pre-occupy s0/s1 for a call-crossing qty). Force it global
(deaths≥2) and its priority floor_log2(44)*44/103 ≈ 21000 dominates → allocated first → pass-1 first-fit
$s0 again (global.c:924, callee prefs stripped for call-crossers). The target needs pri < 2400 ⇒ effective
refs ≤ ~10-12, but REG_N_REFS counts RTL mentions, fixed at 44 by the bytes, and every legal construct
only pushes it UP (each ruled out with file:line). One untested lever: instrument qty_n_refs (local-alloc
SUMS at tying, local-alloc.c:1869) vs flow's per-reg REG_N_REFS with peek2.gdb on tied-copy chains — if a
tie shape yields a low-ref global view of an s2-window value, the wall falls; else it's the ×1-pinned +
whole-binary-gate route.
Flywheel note (R16): even walled, this pass paid off — a cheap-Opus wave applying levers 1-5 should crack the siblings that are NOT at the intrinsic wall. Fable5 DISCOVERS the skeleton; cheap-Opus APPLIES it.
§52a — The regalloc sibling wave: new levers + two new wall classes (cheap-Opus applying §52, 2026-07-15)
Ran §52 as a 6-agent Opus wave over the regalloc-order reach-134 cluster: 2/6 banked ×134 (func_80171FFC,
func_801775E0 → 268 instances, R22 136/136), the other 4 = precisely-characterized walls. Banks and walls
each yielded a byte-verified, reusable lever:
New banking levers (each verified by a whole-binary bank):
- Pass the callee its real arguments —
f((s32)a0,(s32)a1), NOT the void-cast no-arg trick((T(*)(void))f)(). When params are saved to callee-saved regs before the first call, the void-cast form STRIPS the param pseudos' arg refs → shuffles the$s0/$s1/$s2order (RC-10 density). Passing the args naturally BOTH suppresses arg-setup moves (params already in place) AND preserves the density order. (func_80171FFC.) - Recompute a store-base in BOTH if/else arms (the §52-lever-3 analogue for a base pointer): keeps the
recomputed base live across the branch-merge so subsequent
p[k]address off it, not gcc address-CSE onto a still-live keeper base. (func_801775E0.) - Copy-chain direction: use the incoming param DIRECTLY as the persistent keeper (
keep = param+off) with a working copyw = param; gcc savesparam→$sK, chains$sJ=$sK. An explicit keeper var reverses it. (func_801775E0.) pp-declaration position drives the prologue schedule (RC-1/K1): declare a param-copy pseudo AFTER the one you want saved first — sched1 emits in pseudo-creation order. (func_801775E0.)- A
const/RTX_UNCHANGING_Ppre-call load frees a sched2 prologue save-order tie (RC-3):f(*(const s32*)(a0+off))— sched.c:true_dependence drops the load↔store deps sosw $s0/sw $rastop readying simultaneously → the descending-regno save tie staggers right. BOUNDARY: it can OVER-free (the load then hoists above the saves), so it fixes save-order but not always the whole function. (func_80131A34.)
Two NEW intrinsic-wall classes (byte-characterized, P9 — distinct from §52's flagship $s0 local-alloc wall):
- Caller-saved priority-first-fit wall. A long-lived block-local value (
$v0/$v1/$a0scratch) stored many times has LOWqty_comparepriority (floor_log2(refs)·refs·size/(death−birth)) → loses the low-reg first-fit to its SHORT-lived competitor. §31-B in-out-asm and §47 live-length split TIES only, never a priority GAP, and the competitor can't be lengthened without deleting an instruction the target keeps. Needs pins → ×1-only. (func_80169228, bounded to 3 register identities.) - Non-coalescing delay-slot copy wall. A value computed in
$vX, tested, then copied to$vYto survive a clobber in the branch delay slot needs two non-coalescing equal-valued pseudos. Every pure-Cy=xis destroyed by cse.ccanon_reg/make_regs_eqvhead-promotion or global-alloc coalescing; the only preserving forms are#APPasm (blocks reorg's delay-slot fill) or a$0-add (SIGABRTs cc1 in sibling TUs, §42e). Intrinsic pin-free/sweep-safe wall. (func_80177AD4, 1 instruction.)
Wave economics: ~⅓ of a fully-walled cluster cracks pin-free by applying the idiom; the rest wall on a small set of distinct, now-named mechanisms. Cracks bank ×134; walls become permuter seeds or documented dead-ends.
§52b — Sibling wave 2: more de-pin levers, a third wall class, and the match_one→whole-binary gap at scale (2026-07-15)
A second 6-agent Opus wave (armed with §52a) over the close=0 regalloc cluster: 3/6 banked ×134
(func_801379FC, func_801497A8, func_801495C4), 2 whole-binary-near, 1 new wall. Additional VERIFIED levers
(each proven by a whole-binary ×1 bank):
- Per-loop pseudos for a register role-swap — when the target uses pointer=$s0/index=$s2 in one loop and the
swapped roles in another, a single shared C var can't (one hard reg each). Declare SEPARATE per-loop locals
(
s32 p; s32 idx;inside each block); the swap falls out of K2 density (the ref-heavier value wins the low callee-saved per block). (func_801379FC.) - The RC-7 "second-set" dial —
u8 *s = SYM; __asm__("" : "=r"(s) : "0"(s));makesreg_n_sets(s)==2, failingupdate_equiv_regs' single-set gate → NOREG_EQUIV→ the value is NOT rematerialized at its use → it must hold a callee-saved reg across calls (matching a target that keeps a base in$s1). Zero bytes, sweep-safe (generic"=r"/"0", no$N— distinct from the §42e$0-add). (func_801497A8.) - Value-barriers dissolve the CSE-stack-address-common wall — when correct stack-slot order (RC-1 decl order)
forces a
&buf(sp+off) to be CSE-commoned across two calls into a call-crossing pseudo that steals a callee-saved reg, wrap each&bufuse in__asm__("" : "=r"(m) : "0"(&buf))so cse can't fold them → each rematerialized fresh at its call. STEERABLE (banked), not intrinsic. (func_801495C4; itsfunc_8014964Ctemplate used a register pin + 5 barriers — the pin was superfluous.)
A third intrinsic wall class (byte-characterized, P9): the symbol-address-base "wins-low-needs-high" wall
(dual of §52a's caller-saved-priority wall). A 2-instruction symbol-address base feeding its own N loads is the
densest block-local pseudo → first-fits the LOW reg ($v0), but the target needs it HIGH ($a1); making it
low-priority is structurally impossible (a base can't out-rank the loads it feeds), and the movstri form that
would place it high triggers the §52-flagship (plus $fp const) local-alloc theft (update_equiv_regs can't
rematerialize a non-CONSTANT_P source). Only a register pin resolves both → ×1-only. (func_8012B4B8, close=15.)
Process finding — the match_one→whole-binary gap at wave scale: ~half of the agents' match_one close=0 drafts do NOT bank whole-binary (isolated reloc-masked compile overstates; A10). The whole-binary gate is the sole arbiter — a match_one MATCH is a CANDIDATE, not a bank; budget the gate cycles.
§53 — SWEEP A FAMILY WITH THE TOOL ITS EXEMPLAR NEEDED: the jr/switch carve, and how omitting it manufactured the "families don't template" doctrine (Phase 28 T1, 2026-07-15)
The law (one line): a family sweep must reproduce every build step the exemplar's own bank required. Omit one and the gate rejects every sibling — a result that reads exactly like an intrinsic wall, at any scale.
The case
0x8017BEBC (952 ins, the largest unmatched core in the game; 115 unmatched members across 21 addresses,
addr_tag: scattered, PURE 109 / IMM 6) was the roadmap's B2 — "possibly the largest cheap win left."
Phase-27 T5 swept it with family_sweep, banked 0 of 8, recorded "all genuine byte-DIFF", and
generalized it to "h_seq/h_norm structural families do not mechanically template (≈0%)" — which
rewrote the endgame's arithmetic to "(cores cracked) × (reach), NOT (families) × 120"
(PhaseEnd_Phase27 Roadmap delta; docs/calibration.md called it "the decisive P28/P29 input").
Re-run through the carve path: 8 of 8 BANKED (4 same-address + 4 cross-address via to_addr),
make clean + extract-all + check-all → 140/140 byte-identical.
Why 0/8 was structural, and predictable from two words
0x8017BEBC is a jr/switch core. §47 banked its exemplar as "lazy isolation → carve (9-piece
interleave) → splice → BYTE-IDENTICAL" and explicitly noted the fix is "×N template-safe."
family_sweep.hseq_sweep stages C and gates — it has no carve step. So gcc's generated jump table is
never placed at the sibling's address, and the residual is exactly:
classify_member(ov_SC01_000 -> ov_SC01_001) = PURE, ndiff=2 # positions 343, 345 — TWO WORDS
idx 343: 3c01801a vs 3c01801f lui $at,%hi(jtbl_801EC44C)
idx 345: 8c224374 vs 8c22c44c lw $v0,%lo(jtbl_801EC44C)($at)
config/overlays.mk:112 ov_SC01_000_JTBL_INTERLEAVE := ...,ov_SC01_000_jr_8017BEBC.o,tail12.data.o,...
config/overlays.mk:134 ov_SC01_001_JTBL_INTERLEAVE := ... <- no jr_8017BEBC entry. The table is unplaced.
A PURE, ndiff=2 family is the closest thing to templatable that exists. It failed on a build-config gap.
The rule
- A family with
has_mid_jr: trueMUST be swept withtools/jtbl_family_bank.py, neverfamily_sweep. It carves per sibling (jtbl_carve→make extract(+ld_interleavesandwich) → remap → whole-binary gate, revert-on-fail).family_sweepis correct only for non-jr families. --rawthe standalone crack, not the banked TU.extract_uniton a banked jr body hands the sweep the TU's file-scope decl layer (here twoD_800B9A02externs) = §41c pollution. The preserved crack (.run/phase26-cracks/<fn>.c) carries the fix and its reconcile at block scope, so it travels.- Trust the file, not its header.
func_8017BEBC.mdstill says "close=2 of 952" — the state BEFORE §47's slider closed it. The.cwas updated; the.mdwas not. Templating from a body you believe is a near-miss, or reading a stale header as current, produces zeros you will misread as a wall. symbol_mapdoes NOT need ajtbl_prefix. Tempting and wrong: a compiler-generated switch table is never named in C, so there is no token to substitute (grep jtbl src/ov_SC01_000/…_jr_8017BEBC.c→ nothing). The fix is placement (carve + interleave), not substitution. (This was a real mid-plan error: the diagnosis "symbol_map cannot generate jtbl_" is TRUE and the fix derived from it is FALSE.)
The meta-lesson (R35, and why this one is expensive)
The 0/8 was cited as the decisive input for two phases of planning. Three compounding failures made it:
- Wrong tool for the class — swept a jr family with a carve-less sweeper.
- n=1, least-representative —
has_mid_jris 3 of 163 matched-exemplar families; the rarest class was generalized to the whole frontier. - The corroborating evidence was pre-fix — the three Phase-26 exhaustion probes (tiny-IMM 0/241,
PURE 0/134, pinned 0/133) all predate
_carry_macros(P27 T5,commit:0637). P27's own decision-log calls its re-probe "a fourth phantom exhaustion proof" — naming the mechanism that would have faked the first three, and never re-running them.
Before a 0% retires a lever, ask: did I run the steps the exemplar's own bank required? is this family representative of the class I'm generalizing to? was the corroborating evidence taken through the same broken tool? A 0% from a broken tool and a 0% from a working one are the same number and opposite facts.
§55 — Core-crack wave levers + the GATE-ORCHESTRATION law (Phase 29 T3, 13-agent ultracode wave, 2026-07-17)
A 13-agent worker_wave over 6 fresh cores (260–371 ins) + 7 B3 near-misses (100–141) returned 7 MATCH /
6 near; 5 banked whole-binary. The durable yield is the levers + the orchestration law.
§55a — New byte-proven levers (each from a banked or near draft)
- §49-variant — suppress sched1's
birthing_insn_pLAUNCH_PRIORITY boost byreg_n_sets1→2 (func_801325B8, 113 ins, BANKED, reproduced twice). Route the load through a temp assigned in BOTH halves of a branch: the pseudo now hasreg_n_sets==2, so it is no longer "birthing", the priority boost disappears, and two transposed loads un-invert at zero byte cost. Companion to §30's birthing-boost. birthing_insn_pgoverns sched1 PLACEMENT, steerable both ways (func_80177940, 101 ins, close=5): an in-place single-variable update makes a chain non-birthing → sched1 stops sinking it (18→10); a fresh pseudo forc<<16makes it birthing (10→5). Also:__asm__("" : "=r"(v) : "0"(v))is a zero-code CSE fence that blocks the cse2 fold of a duplicated chain.cc1 -dLprints loop.cmove_movablesdecisions (moved / not-desirable, per movable) — the threshold (−3/movable) is steerable by statement order. A dead second set (col = 0;) makesn_times_set != 1, so a constant is not a movable and stays in-loop. (func_80177940.)- Switch TREE vs jump table —
CASE_VALUES_THRESHOLDis 5 (func_801387B8, 100 ins, BANKED): 4 cases{1,7,10,23}< 5 ⇒ gcc emits a branch tree (forward-beq-to-body + lone medianslti), NOT a jtbl — so a guard likeif (cmd != 0)must stay OUTSIDE the switch or a 5th case forces a jump table.u8 cmdbuys a signedsltion the split AND an unsignedsltiuoncmd >= 0x20for free; case bodies emit in SOURCE order. Ghidra's if-chain for a switch is a decompiler artifact (floors at 49-off) — recognize the dispatch tree. - Block-scope beats
*(T*)&symfor a conflicting extern (func_8014ADE0, BANKED): when aDEFINE_func_*macro declaresextern s16 D_Xand you needs32, a block-scopeextern s32 D_Xis a warning (not an error) — but it is order-dependent: block-scope must precede the file-scope decl (reverse = hard error, cc1 exit 33).*(s32*)&D_XFAILS (~100-ins shift — the §18&symmaterialize + CSE trap).
§55b — THE GATE-ORCHESTRATION LAW (3 traps, ~3.5h lost; all recovered, 0 data lost)
gate_stage's propagate step is FLEET-WIDE (dedup_propagate --auto-from), NOT scoped to the drafts you gated. Runninggate_stageonce per src-file group therefore re-runs the whole-fleet scan N times; each exceeds the 3600s timeout and dies mid-mutation → partial propagate damage (measured: 90/140 overlays broken, 887 files, engine_core.h +561). LAW:--no-propagateon every per-group gate, then ONE targeteddedup_propagate --addr <banked addrs>at the end.- COMMIT the cheap verified banks BEFORE the expensive propagate. The byte-gate is minutes; the propagate is ~2h and mutates 300+ files. Gating and propagating in one motion means every propagate failure takes the (already-verified) banks down with it. Commit, then propagate as a standalone revertable step.
- A reverted
src/needs a RE-EXTRACT (the R22 corollary, again):git checkout -- srcrestores theINCLUDE_ASMstubs, butasm/still reflects the BANKED state (splat emits no.sfor a matched fn) ⇒corpus.CorpusError: N stub(s) have NO .s on disk. Recovery =git checkout -- src docs .run+make extract BINARY=<ov>. (R34's second oracle caught this loudly — working as designed.) gate_stage's default.run/harvest_verified.txtACCUMULATES across runs and its CLI exposes no--verified-out⇒ after a revert it reports a phantombanked: Nfor functions still stubbed insrc(stale residue). Trust the SOURCE (grep INCLUDE_ASM), never the report — R32/R35 class, and still armed: either expose--verified-outon the CLI or unlink the default before each gate.
§55c — Sizing the propagate: a TARGETED propagate is ~4 min/core, and "it's slow" was a BROKEN-TREE ARTIFACT
--check-only prints the real plan first — always use it (it also aborts free when a group is undroppable).
The real cost, measured on a HEALTHY tree: --addr <1 core> = 233s for 138 members → 3 cores ≈ 12 min.
[ OK ] 138 overlays byte-identical after propagation.
The trap (and a live R14/R35 self-correction worth remembering): the same 3-core targeted propagate had
timed out at 3000s earlier in this task, and I wrote "needs ~2h+" into the commit + this cookbook as a
measurement. It was nothing of the kind — it was propagating into a tree still carrying the partial damage
of a previous killed --auto-from (90/140 overlays broken), so every member-gate was failing/retrying. On a
clean tree the identical command is ~20× faster. A timing taken on a broken tree is not a measurement of
the tool — it is a measurement of the breakage. (Same shape as the §53 carve-law and the §54 def-sig
findings: the number was real, the attribution was wrong.) Recover the tree FIRST (revert + re-extract),
THEN measure.
- Only
--auto-fromis genuinely fleet-slow (it scans every matched fn) — that is the one to avoid, not the targeted--addrpath. h_exactshare is all-or-nothing: one straggler overlay (ov_SC03_093) drops the whole group unless--recover(per-overlay exclude) is passed.- Local-type bodies are skipped ("not self-contained") until
build_engine_typeslifts their types (§19/§20 propagation cap) — 2 of this wave's 5 banked cores were blocked this way.
§54 — --fix-def-sig: the member's CANONICAL DECLARATION is a build step too (tiny-IMM mega-pools, +4,801, Phase 29 T6, 2026-07-16)
The §53 law — sweep a family with the tool its exemplar needed — has a fourth instance beyond the jr-carve
(§53) and the -O0 flag (Task 1): the member's shared-header declaration. The two tiny-IMM mega-pools
(0x80131eec, 0x80130d0c; ~15-ins jump-table dispatchers D_x[*(u16*)(a0+2)](), ~2600 members each) banked
1/4966 (0.0%) via plain family_sweep --hseq AND 0/2470 via --reconcile (canon_sig_reconcile).
Root cause (byte-proven, after 3 masked-metric mis-reads — see decision-log 2026-07-16): family_remap
copies the EXEMPLAR's def signature onto each member. When a member is forward-declared in
src/shared/engine_core.h with a caller-derived signature (a shared engine fn calls it:
extern void func_8015FAAC(s32 *a0);) that differs from the exemplar's (void *a0), the member TU throws
conflicting types for func_X (cc1 exit 33) and never compiles. The exemplar itself has no such header decl
(that asymmetry is why the family templates in ov_SC01_077 but not its members). This is INVISIBLE to
diff_regions/match_one/masked_diff (they compile the draft STANDALONE, so no header decl to conflict
with → they report O2:MATCH(0)), and to --reconcile (it reconciles against the sibling TU's local decls,
not the included header).
The fix — byte-neutral, gate-arbitrated. family_sweep --fix-def-sig (header_sig_map parses the 1005
extern … func_X(…) decls in engine_core.h/engine_types.h; reconcile_def_sig rewrites the member draft's
DEF sig to that canonical). A pointer-type param diff (s32* vs void*, (s32)a0 identical) doesn't change
codegen; the whole-binary gate rejects anything that does (G3/P9). Result: pool 1 94%, pool 2 99% =
4,801 banked. Consider making --fix-def-sig default-on for the h_seq path. Meta: a masked/standalone MATCH
is a candidate, never a diagnosis — reproduce the real member TU and read the real cc1 error (R35).
§56 — Banking a hand-drafted GIANT into its exemplar TU: self-contained draft vs live-TU decls, 4 reconciliation tactics (func_8013FAF8, 312 ins ×138, Phase 29 T4, 2026-07-17)
A hand-crafted giant draft is self-contained (its own typedefs + externs) so match_one can compile it
standalone. Splicing it into its live exemplar TU surfaces decl conflicts the standalone compile is blind to
(same blind spot as §54, but a GIANT touches dozens of symbols). harvest_verify strips the typedefs the TU
already provides (per-TU strip-set) but NOT the externs — those you reconcile by hand. Every fix below is
byte-neutral and re-verified two ways: match_one must stay green AND the whole-binary gate must stay
byte-identical (G3/P9). The loop: harvest_verify --binary <exemplar> --drafts <isolated-dir> → it prints ONE
PLUMBING: <cc1 line> per fail. That classifier line is often a red herring — it grabs the FIRST
conflicting types in stderr, which may be a pre-existing benign warning (conflicting types for built-in function memcpy, present in the 140/140 baseline). Get the FULL cc1 stderr (splice manually, make build BINARY=<ov> 2>&1 | grep error) and fix the REAL errors bottom-up.
The four conflict classes and their byte-neutral fixes (all proven on func_8013FAF8: s16/s16 def, 3 data + 2 fn conflicts, ~5 gate iterations):
-
Def-sig conflict (draft
void func_X(s16,s16)vs the fleet-canonicalextern void func_X(s32,s32), 404 decls). NARROW the extern fleet-wide s32→s16 — NOT--fix-def-sig.--fix-def-sigrewrites the DRAFT to the canon; here the s32 def diverges from the target at insn 22 (thefor(i=arg0;i<arg1)compare promotes differently). Direction test (R35): the correct direction is whichever keepsmatch_onegreen — test the canon-typed variant explicitly before assuming--fix-def-sig. Byte-neutral iff EVERY fleet caller passes cast/small-const args — verify (grep -rh 'func_X('): here all were(s16)-cast or0/5/7/8. -
Data-symbol conflict, file-scope decl BEFORE the splice point (block-scope is §55a-BLOCKED — a block-scope extern after the file-scope one is cc1 exit 33). Use the TU's established §18 cast-at-use-site convention:
*(s16*)&D_80115128(forceslh),*(u16*)&D_800B9A02(forceslhu),((s32*)&D_80187AC0)[i]— the cast fixes the load width independent of the decl's signedness, so you keep the TU's existing decl untouched and there is no conflict at all. This is also propagation-safe: unlike block-scope, it carries no macro-slot-ordering dependency across the 137 member TUs. Caveat (§18):&symcan CSE-hoist across many uses in a tight loop — verify it didn't (func_8013FAF8: 5 uses of&D_800B9A02, no hoist; giants with few uses/symbol are safe, re-test if a symbol is hammered in an inner loop). -
Data-symbol conflict, file-scope decl AFTER the splice — block-scope extern is legal here (§55a), but prefer the tactic-2 cast form anyway for propagation safety (the member TUs won't share the ordering).
-
Function-extern conflict (draft
extern s32 func_Y(s32,…)vs the TU's own DEFfunc_Y(s16,…)/int func_Z(int)): reconcile the DRAFT's decl to the TU's type + a byte-neutral call-site cast, e.g.ot = (s32*)func_80141100((int)ot). Pointer↔int and s16↔s32-of-an-already-sign-extended-value are free.
Then R22 clean-fleet (mandatory): the tactic-1 narrowing edits engine_core.h (404 decls, 266 files) —
harvest_verify gates only the exemplar binary, so the fleet-wide byte-neutrality of the narrowing is unproven
until make clean && extract-all && check-all = 140/140. func_8013FAF8: 140/140, +312 ins ×1 (propagate ×137
is the separate §55b step). The README's "pure def-sig plumbing" undersold it — a giant is a multi-symbol
reconciliation; budget ~5 gate iterations, not one edit.
§56b — PROPAGATING an h_seq giant: the exemplar's externs MUST be fleet-canonical, not the draft's types
An h_seq family (masked-identical body, per-overlay symbol names/reloc targets) propagates via
family_sweep --hseq --only <addr> --source <exemplar> --allow-pins — it copies the exemplar's matched C
into each member and remaps the per-overlay data symbols (the undefined-D_xxxxxxxx-reference the raw
exemplar throws in a member is exactly what the remap resolves; a manual harvest_verify of the raw exemplar
into a member is NOT a valid family test — only family_sweep remaps). But it copies the exemplar's extern
block VERBATIM (only the data symbols are remapped, not the callee-extern types). So if your hand-crafted
exemplar declared a callee with a type that diverges from the fleet-canonical (the member consensus /
engine_core.h), every member TU throws conflicting types for func_Y and the whole sweep banks 0/137.
The fix (byte-neutral, unblocks the entire family in one edit): before sweeping, audit every callee extern
in the banked exemplar against the fleet-canonical —
grep -rh 'extern.*\bfunc_Y\b' src/ov_*/ | sort | uniq -c | sort -rn | head -1 — and rewrite the exemplar's
decl to the high-count consensus form. Return-type and pointer↔int param diffs are byte-neutral (gcc-2.7.2
warns, doesn't error; the call's arg values are unchanged, and a discarded/(cast)-assigned return emits
identically). func_8013FAF8 had 4 divergent externs (func_8005A600 void→s32, func_80024054
s32→void*, func_80137D08 s32*→int, func_8013AB54 s32*→s32); aligning them in the committed exemplar
.c (family_sweep reads the tree, not your .run/ draft) took the sweep from 0/137 → 137/137 banked,
+312 ins ×137 ≈ +42.7k ins, R22 clean-fleet 140/140. Meta: a hand-authored exemplar carries the drafter's
type guesses; the fleet already voted on the canonical — make the exemplar agree before it becomes the template.
(Contrast the h_exact path — dedup_propagate --recover — which auto-reconciles conflicting caller externs;
family_sweep does not, so you pre-align the callee externs by hand.)
§57 — The SELF-decl normalize: the sibling's OWN caller declares the templated fn divergently (the third §17a-1 direction; tools/normalize_self_decls.py, func_801670E4 ×137, Phase 29, 2026-07-18)
§17a-1 (fold a compile-time fn-ptr cast of a known symbol back to a bare jal, byte-neutrally) has three
directions, one per "who declares the conflicting signature":
| tool | the conflicting decl is of… | lives in |
|---|---|---|
cast_call_sites (§20) |
a callee the draft calls | the DRAFT's own forward-decls |
reconcile_tu (§8d/§33) |
a data symbol D_xxxx |
the DRAFT's own externs |
normalize_self_decls (this) |
the templated fn F ITSELF | the SIBLING TU's other already-banked functions |
The blocker (byte-proven, func_801670E4 ×137). family_sweep --hseq templates F's DEF into each sibling TU.
But that sibling's OWN already-matched caller of F may carry a block-scope decl of F left when that overlay's
caller was matched — in a different C form than the exemplar's caller used. ov_SC01_004, ov_SC01_004_jr_8015AE2C.c:
the stub of F (→ where the def lands) and func_80167540's local extern void func_801670E4(struct Entity_80167540 *, s32, s32, s32); sit in one TU; splice F's def (s32 func_801670E4(s32,s32,s32,s32)) and cc1 exit-33s with
conflicting types for func_801670E4 — twice (once vs the def, once vs the canonical decl the DEFINE_func_* macro
injects at file scope; the second is error-recovery cascade from the same one decl). The EXEMPLAR never hit this:
ov_SC01_077's copy of that same caller used the fn-ptr CAST form instead of a decl, so the conflict is per-sibling
and invisible in the exemplar (like §54's header-decl asymmetry and §56b's exemplar-externs, a third "the two overlays
matched the same code in different C forms" trap). All 133 still-stubbed siblings carried the identical block-scope decl.
The fix (byte-neutral, gate-arbitrated). For each decl of F in the sibling TU incompatible with F's def
(cdecl.compatible, the cc1-validated oracle — a no-prototype void f() never conflicts in either TU order, §51g,
so skip it): DROP the decl (F's def is now the declaration cc1 sees) and CAST every call of F in that decl's
scope to the decl's ORIGINAL sig — func_F(a) → ((void(*)(struct Entity_80167540 *,s32,s32,s32))func_F)(a). This
is the byte-faithful move (preserves the caller's exact ABI, the form the exemplar's caller already used); the
sig_unify "rewrite the decl to canonical, keep the bare call" alternative reconverts the args to the canonical
param widths and drifts codegen (§20). Manually proven byte-identical on ov_SC01_004 (remove the stale output
first — a failed compile leaves a passing STALE binary that false-passes SHA, §42b) before the tool existed.
Wired. tools/normalize_self_decls.py (fix(tu_text, fn, ref_decl) — scope via cdecl._masked brace-depth,
block scope → the enclosing top-level fn body, file scope → to EOF; idempotent no-op when no divergent decl exists).
family_sweep --hseq --normalize-self-decls: a new per-sibling stage after reconcile_def_sig that edits the
sibling TU FILE (F's callers live in F's own TU = harvest_verify's baseline, like edit_remap_sweep), with a
snapshot + final-SHA-MISMATCH revert backstop (a MISMATCH ⇒ a transform bug, since harvest_verify always
reverts a wrong DRAFT — so a wrong draft leaves the binary byte-identical, only a non-neutral TU edit can MISMATCH).
Result: func_801670E4 133/133 banked, 0 failed (family 4→137/137), 0 backstop fires. Do NOT add --fix-def-sig
for this family: the raw draft def s32 func_801670E4(s32 arg0,…) is already type-compatible with canonical
(param NAMES are irrelevant to a C prototype); --fix-def-sig renames the params to a0..a3 while the body still
says arg0..arg3 → arg0 undeclared (the "rare name mismatch" its own docstring warns of). Diagnose the sweep's
ACTUAL blocker (splice one member, read cc1) before stacking plumbing flags (R35).
Not every type-lifted family needs this: func_8016CBC0 (also 137-member, also blocked) has NO divergent
self-decl (survey the members: grep 'func_X(' the-member-TUs) — its blocker is a local-typedef lift (§ type-lift,
like func_8012956C), a different lever. Route by the real cc1 error, not by "it's a stuck 137-family."
§57a — Two NSD corrections + the SURGICAL-ONLY law + the honest broad-sweep yield (Phase 29, 2026-07-18)
Three findings from applying §57 at scale, all byte-grounded:
(1) REWRITE the divergent decl to canonical — do NOT drop it (the def-after-caller trap). The first §57
build DROPPED the divergent decl. That is byte-neutral only when F's def sits ABOVE the caller in the TU (so the
def itself provides the caller's forward visibility — func_801670E4, def-before-caller). func_8013D53C is
def-AFTER-caller: a FILE-scope void f(void) forward decl, F's def spliced BELOW it — drop the decl and the
caller gets func_8013D53C undeclared. The fix is exactly what cast_call_sites does: rewrite the decl to
the def's canonical sig (matches the def → no conflict, AND keeps forward visibility), then cast the calls to the
decl's ORIGINAL sig (byte-exact). Works both ways (a canonical forward decl above/below the def is compatible).
(2) (void) is NOT no-proto — do not skip it. The first build skipped params in ('', 'void'). But void f(void) is a 0-param PROTOTYPE that genuinely conflicts with a >0-param def (cdecl.compatible(void f(void*), void f(void)) == False), while a true no-proto void f() is compatible in either order (§51g). Skip ONLY the
literal empty () (also the §32 no-proto-mis-cast guard); let cdecl.compatible judge (void). This was why the
BUILD SPEC filed D53C's (void)/(T) arity class as "not this pass" — it was a tool gap, now closed.
(3) NSD is SURGICAL-ONLY; --fix-def-sig is broad-safe — the blast-radius asymmetry. --normalize-self-decls
edits the TU file (F's callers live in F's own TU), so a non-neutral edit poisons the ENTIRE (overlay,split)
group — and unlike a bad DRAFT (which harvest_verify bisects away per-member), a bad TU edit can't be isolated;
the whole group's build MISMATCHes and the backstop reverts all of it. A broad --band substantial --normalize-self-decls (esp. combined with --fix-def-sig, whose canonical-rewrite changes NSD's reference sig)
banked 7 with ~752 groups backstop-reverted. --fix-def-sig edits DRAFTS (bisect-safe per member), so it IS
broad-safe. Law: apply NSD per-family (--only, on a surveyed self-decl blocker); apply --fix-def-sig broadly.
The backstop makes broad NSD SAFE (0 false banks) but useless.
(4) The honest broad-sweep yield (R14/R35): staging ≠ banking. The 60 substantial matched-ov077 families with
2,163 stubbed members STAGE 2,169 drafts, but a broad --fix-def-sig sweep banks only 137 (one def-sig family +
stragglers) — 2,032 fail. The substantial-family frontier is NOT broadly mechanical; each family carries its own
blocker (self-decl / type-lift / def-sig+caller / genuine codegen). Gate-probe a SAMPLE before scaling a yield
estimate off the STAGED count — the phase's own invariant, which I violated by projecting ~1,500 from 2,169 staged.
func_8013D53C is the archetype: NSD + a Cmd_8013D53C type-lift clear its plumbing (it now COMPILES), and the
sweep banks 14/137 — the h_seq members whose bodies happen to match the exemplar — while 123 carry a genuine
per-member codegen DIFF (the hard cse.c-wall crack does not fully template). So it is a PARTLY-mechanical family:
the plumbing levers harvest the easy fraction ×0 tokens, the residual 123 are permuter/Fable fuel. NB the NSD
edits are byte-neutral even where the member does NOT bank (the caller's canonical decl + cast matches with F still
a stub), so they persist on all 137 TUs — revert the ones that didn't bank (still carry the stub) so you commit
banks, not churn on matched code. The classifier's PLUMBING count is a §56 memcpy red-herring; read full cc1 stderr.
§58 — match_one MATCH ≠ BANK: the four blind spots + the crack-wave reconcile-before-bank law (Phase 29 crack-wave, 2026-07-18)
A crack-wave drafter iterates a C body against tools/match_one.py (standalone compile of ONE function
with the DRAFT's own externs + relocation masking). That is the right per-function oracle, but it is
STRUCTURALLY BLIND to everything that only surfaces when the def is spliced into its real TU + the whole
binary is linked. An 11-core wave produced 9 match_one MATCHes; all 9 gate-failed the whole-binary bank
with ZERO codegen problems — every failure was one of four integration classes match_one cannot see:
(a) Ghidra symbol names. Drafters paste DAT_801d9c20 / PTR_DAT_80186ad0 (Ghidra) instead of the
splat D_801D9C20. match_one links the draft's own extern, so it never notices; the whole-binary
link fails undefined reference. FIX: rename DAT_<hex>/PTR_DAT_<hex> → D_<UPPERHEX> (verify the
splat label exists: grep -rn D_<HEX> asm/<ov>/data/).
(b) Def-sig conflict vs the fleet. The draft's def sig (from m2c/Ghidra) diverges from engine_core.h
or the TU's own caller-decls. The draft sig is byte-TRUTH (it MATCHed); the header/caller decl is
often a stale stub-era guess (void func_80164E40(void) where the truth is s32 — canon_sig_reconcile
forcing the draft to void produced a DIFF, i.e. dropped the return computation). Conform the DECLS to
the draft, byte-neutrally: narrow the engine_core.h extern void→s32 (callers that ignore the return
are unaffected — R22 clean-fleet confirms neutrality), or tools/normalize_self_decls.py --tu --fn --canon "<draft sig>" when the TU's own caller declares F divergently.
(c) Callee-decl conflict. The draft declares a callee with a sig conflicting with the TU's canonical →
tools/cast_call_sites.py (fn-ptr-cast the calls, drop the divergent extern).
(d) Opt-level. A function in an -O2 segment can actually be -O0 (frame pointer, per-case stack
reloads). match_one masks this if the agent tried --o0; the whole-binary -O2 build then DIFFs. FIX:
relocate the def into the overlay's _o0/_o0b object (whose .text covers that addr). A "DIFF" verdict
is often THIS, not a codegen miss (func_8013C0F8 was mis-filed as a real -O0 DIFF; it was §8 jtbl).
LAW: a crack-wave's match_one MATCHes are CANDIDATES; budget a reconcile pass before banking. Best is a
pre-bank auto-reconcile (rename → cast_call_sites → reconcile_tu → normalize_self_decls / narrow-extern), then
the whole-binary gate. And feed it upstream: tell the drafters to use splat D_ names + the engine_core.h
canonical callee sigs, which removes (a) and (c) at the source. harvest_verify's per-draft failure LABEL is
a first-diagnostic red-herring (it reported a shared built-in memcpy @4017 for 7 unrelated drafts) — always
read the REAL error by splicing ONE draft and reading full cc1/ld stderr (and beware the §42b stale-image
false-pass: on a build FAIL the old image lingers, so confirm rc==0 before trusting a sha). 6/9 banked this way.
§59 — Three h_seq sweep-residual classes match_one/the-exemplar-bank don't reveal (Phase 29 crack-wave close, 2026-07-18)
After a core banks ×1 in ov_SC01_077 and swaps to matched-ov077, family_sweep --hseq --only can still bank
0/137 for reasons invisible at the exemplar (they're per-SIBLING, or per-overlay data facts). Three seen in one batch:
- (1) The exemplar's LOCAL struct type is dropped by the remap (func_80165240, 0→137/137). The exemplar TU
defines
struct W4 { u32 w; } __attribute__((packed,aligned(1)))inline;remap_hseqtemplates the DEF but not that local type, so each sibling's*(struct W4*)dst = *(struct W4*)srclowers to a DIFFERENT-sized memcpy → byte-DIFF (compiles clean — it's a DIFF, not a compile error, so match_one/standalone never sees it). FIX: prepend the local struct decl to each member draft (the tiny-inline-type analog of the §57/func_8016CBC0 engine_types.h lift — but too small/local to lift; carry it in the template). Symptom is "compiles, 0/137 DIFF." - (2) h_seq data is per-overlay RELOCATED — there is NO fleet-fixed data address (func_8016D1D8/D688,
0→274/274). The exemplar body reads
D_801D9C20(a tail work-buffer); the WRONG assumption is that 0x801D9C20 is the same in every overlay. It is NOT — each overlay's buffer sits at a different base (ov_000 → 0x801A4B78).remap_hseq'ssymbol_mapkeys the byte-OFFSET labels (D_801D9C21…) not the base symbol the C uses, so the base is left unresolved →undefined reference. FIX, scripted per sibling:base = symbol_map["D_<off>"] − off; declareD_<base> = 0x<base>; // type:u8inconfig/symbols.<ov>.txt(in-overlay → re-extract emits the linker def, byte-neutral); remap the exemplar's base symbol →D_<base>. (A few SC07 stragglers also needed carried externs likeApplyMatrixSVdropped — their TU declares them with a conflicting sig, §20/§58c.) - (3) jtbl carve isolation/table-drift walls (2 cores blocked, genuine tooling).
jtbl_carverefuses a NON-CONTIGUOUS same-subseg table (another matched fn's carve already occupies the subseg); isolating requiresjr_isolate/split_src_region, which cannot partition the Phase-17 canonical-sig-layer TU (trim: cannot resolve address of item). And a merged two-table span (§8e-2) tripsjtbl_rodata_pads: more rodata .align than pad specswhen the -O0 code object holds more jump tables than the span's derived pad spec ("one contiguous .rodata run per object"). Both are documented walls — bank the contiguous/last-table cases, report the rest.
Meta: a family_sweep --hseq "0/137 banked" is a per-sibling INTEGRATION signal, not a codegen verdict — read one
sibling's real gate result (COMPILE-fail vs byte-DIFF) before concluding. Extends §58 (match_one blind spots) to the
sweep stage.
§60 — Classify the residual, don't rank it: the deterministic residual→class classifier and what it measured about the backlog (Phase 29 Task-13A, 2026-07-21)
The instrument. tools/residual_class.py decides a near-miss's class FROM THE BYTES. It decodes each
mismatching MIPS word into (operation-skeleton, register-fields, immediate) and runs a decision tree:
drift first (a single inserted instruction desynchronises the tail and inflates closeness by the tail's
length — a 1-instruction structural delta wearing a 200-mismatch costume), then a consistent-injective
register map (⇒ REGALLOC-PERM, the §31 S11/RC-3 class), same-multiset-different-order (⇒
SCHEDULE-REORDER), nop-vs-instruction (DELAY-SLOT), then operation-family splits (WIDTH lw↔lh =
the §18/§43 idioms, BRANCH-POLARITY, STRENGTH, ADDRESSING), then immediate-only (IMM-OFFSET
constant delta = a frame/struct-layout shift, IMM-VALUE). Every path ends in a NAMED class; an opcode the
decoder does not cover is UNKNOWN and COUNTED (R32). Each class routes to a bucket — permuter /
structural / integration / redraft — which says WHICH TOOL the failure wants.
tools/autopsy.py collect materialises the corpus by recompiling every open backlog draft through the
EXISTING match_one path (R33 — never a second copy of the pipeline), deriving the two silent-artefact
inputs rather than guessing them: the asm subdir (from the stub's self-describing INCLUDE_ASM line) and
the -O0 flag (corpus.is_o0, parsed from the Makefile's own -O0 rules with a coverage assertion). 1,752
drafts in ~21 s at -j12.
Cross-check (R34). The classifier's closeness is computed by a different route than
masked_diff.structured_diff's; collect asserts equality on every row and refuses the corpus on any
disagreement. 1,673/1,673 agreed, 0 classifier errors — so the MIPS decoder covered every opcode in the
real corpus.
What it measured — the whole open backlog, byte-grounded
| bucket | fns | reach-wtd | meaning |
|---|---|---|---|
redraft |
699 | 2,162 | the stored draft is not this function (a 15-ins body vs a 132-ins target) |
structural |
578 | 7,761 | local mutation cannot introduce it — wants a C idiom, not CPU |
integration |
306 | 2,303 | byte-correct standalone; blocked on plumbing (§58/§59) |
permuter |
75 | 740 | a search-closer can actually reach it |
unknown |
2 | 2 | the LLM tier's residue |
THE FINDING: of the 972 records the grinder's own filter admits, 75 (7.7%) are permuter-shaped. 547 are
structural and 348 are junk drafts. The daemon has been spending ~92% of its CPU where the permuter provably
cannot win — which is the byte-grounded explanation of "7 banks all-time, all in Phase 21, 0 since"
(Phase-22 audit). It was never a missing transform; it was targeting. grinder.candidates() now filters
on the measured bucket (1,303 → 78 candidates) and takes its directed permuter_weights profile from the
measured class instead of the logged label — 91% of records carry NO label, so classify() returned None and
the search ran on gcc defaults. Degrades to the old undirected behaviour if the corpus is absent, and says
which mode it is in (--no-targeting A/Bs it).
Two corollaries worth remembering
- A large
closenessand a hard function are different things. 699 records rank as "near-misses" with closeness up to 278 purely because a stub-sized draft was scored against a large target. Ranked by closeness they look like a wall of nearly-done work; they are un-attempted work misfiled as near-misses — fresh crack fuel, not a backlog of hard functions. Hence the separateredraftbucket: the routing is opposite (re-draft vs seed-tweak). match_oneMATCH still ≠ bank (§58), measured. A 12-draft gate probe of theintegrationbucket (reach-134, ov_SC01_077) banked 1 of 12; the other 11 failed PLUMBING. So the 306 is a pool of integration candidates whose conversion depends on the reconcile ladder — it prices Task 14, it is not 306 free banks. (Note the harvest_verify failure LABEL is the §58 red-herring: 10 of the 11 reported the sameconflicting types for built-in functionline from an unrelated TU position.)
The parallel-probe race this surfaced
masked_diff._common_typedefs() wrote, read and deleted ONE shared path src/.masked_diff_probe.c. Under N
concurrent match_one/permuter processes, whoever unlinked first made another's open/parse fail, and that
process died with a traceback instead of a verdict: 14 of 1,752 drafts lost in a single 12-way run (0.8%)
— and every parallel wave has paid it invisibly, because a drafter that crashes on its self-check merely
looks like a drafter that failed. Now per-PID. Same defect class, one level down, as the Phase-28
match_one --work shared scratch whose docstring promised the isolation its default contradicted.
§60a — What the first DIRECTED grinder run exposed (Phase 29 Task-13B, 2026-07-21)
Turning the targeting on and running the grinder bounded (--once --batch 8 --permute-secs 90) produced a win
on the FIRST candidate — func_80181F78, close=1, classified DELAY-SLOT/schedule, banked in ~6 min — and
then immediately surfaced three latent defects that had been unreachable because the daemon had not banked
anything since Phase 21. All three are the same shape: a step whose REPORT and whose WORK had quietly
diverged.
gate_stage's commit path crashed onsrc=None.srcis deliberately never defaulted (the Phase 26-A audit: a default would silently PIN the gate to the main.c), but the commit didgit add src …unconditionally. So every caller that omitssrc— grinder, orchestrator, idiom_hunt — crashes the moment it banks. Fix:git add -u src/(every modified tracked file under src/), which also retires thesrc/ov_*/*.cfilename glob that once omitted 4 R22-verified banks from a commit because a family's members do not all live in the same-named split. COMPLEMENTARY HOLE (2026-07-22):git add -umisses the NEW files an ISOLATION creates. A jtbl sweep cuts a fresh region file per sibling (src/<ov>/<ov>_jr_<ADDR>.c), which is UNTRACKED — so-ucommits the modified TU and drops the file holding the banked body, i.e. a tree that cannot clean-rebuild. For any carve/isolation bank usegit add -A src/ config/.jtbl_family_bankalready refuses to sweep on an uncommittedconfig/+src/(its per-sibling revert restores from HEAD), and that guard is what caught this — a fail-loud precondition doing exactly its job.- The
_xformladder dirs accumulate.<drafts>-cn/-cast/-rc/-uniare reused across runs and the transform tools only write the drafts they are handed, so every stale draft from every previous run survives and is re-submitted to the byte-gate. Measured: the grinder submitted 1 draft, the gate processed 34 and banked 2. Nothing wrong entered the tree (G3/P9: the gate banks only byte-identical output) — but a run banked a function it was never asked to try, and would have committed it under a message naming a different one. Fix: clear the out dir per run. Note the symmetry with R32: a scanner that silently NARROWS its input hides work; a stage that silently WIDENS it fabricates provenance. - The grinder fired the fleet-wide propagate from inside the gate.
gate_stage(propagate=True)runsdedup_propagate --auto-from— the §55b path that timed out at 3600 s and left 90/140 overlays broken, and which, being inside the gate, takes the banks down with it when it fails. The unattended caller must never fire it: bank withpropagate=False, commit the cheap verified banks, then run ONE targeteddedup_propagate --addras its own batch.
The transferable point: a tool that has been failing for a long time accretes latent bugs on its success
path, because nothing exercises it. Before trusting an unattended fix-and-run, budget for the first success
to fail — and check the tree state, not the exit code (here the durable Task-12 winner save in
.run/permuter-winners/ is what made the crash a non-event).
§60b — The plateau autopsy's verdict: a partial drift is a WRONG DRAFT, not a missing transform (Phase 29 Task-13B close, 2026-07-21)
hindsight-study §7 predicts that a search-closer's plateaus decompose into missing-transform (extend the mutation set — "the highest-value bucket and the whole point"), seed-structural, and genuine-wall. Run against real plateaus, this class produced ZERO missing-transforms. The autopsy is worth recording because the answer was legible in the bytes and needed no LLM at all.
The measurement. A 20-target probe of the length profile: tail drifts (a single shift point explains
the whole tail) converted 1/6; partial drifts (length differs AND other positions differ) converted
0/12. Reading three partial plateaus directly:
func_8017F0C0,func_801806C8— target containssltiu $v0,$v0,0x1. That is gcc's codegen for!x/x == 0. The drafts wrote(u32)(D_x ^ 1), which emitsxori. No local mutation rewritesxoriintosltiu: it is a different operation, chosen by the front end from a different C expression.func_8017FF90— the draft stores toarg0 + 8; the target stores to a global (lui $at,%hi(D_…)/sw $zero,%lo(D_…)($at)). Not the same function at all.
Two permanent fixes, both NARROWING what the offline tool is allowed to attempt:
_drift_routenow admits a drift to the permuter only when|Δ|<=2ANDexplains == "tail". Length-profile pool 339 → 34; permuter bucket 389 → 84. Apartialdrift is seed-structural by construction — the count is wrong and other positions are wrong, which is not one local edit.SIZE-MISMATCHgained a PROPORTIONAL test (|Δ| >= 0.5*nt) alongside the absolute one.max(2, 0.15*nt)is far too permissive on a tiny target: a 2-instruction draft against a 4-instruction target is|Δ|=2and read as a near-miss when it is a wholesale mismatch.
The transferable lesson. The §7 taxonomy tacitly assumes the plateaus are near. Ours mostly were not —
they were bad drafts wearing a small closeness. So the highest-value autopsy outcome was not a new
transform but a tighter admission rule: the way to raise a search-closer's yield is at least as often to
stop feeding it unreachable work as to widen its mutation set. Same knife as Task-13A's targeting fix, one
cut finer.
Cookbook idiom for drafters (recurring): sltiu rd, rs, 1 ⇒ the C is !x / x == 0, NOT x ^ 1.
The XOR form emits xori and can never match.
§61 — Task 14: the gate ladder's missing stage is the ARITY pre-pass, and it is TU-side not draft-side (Phase 29, 2026-07-21)
gate_stage's recovery ladder was four DRAFT rewrites (canon_resident_calls → cast_call_sites →
reconcile_tu → gate → sig_unify → gate). The dominant residual blocker is not in the draft at all.
Diagnosis (not assumption). The 12-draft integration probe banked 1/12 and reported the SAME failure
label for 10 of the 11 failures: warning: conflicting types for built-in function 'memcpy'. That label is
the §58 red-herring — it is a WARNING, from an unrelated TU position, and it is not the failure. Splicing
three of the highest-reach failures individually and reading real cc1 stderr gave the actual cause:
src/…_jr_8016AB6C.c:3839: conflicting types for `func_8016EFC8' <- 3 of 3
src/…_jr_8012ACE0.c:879: redefinition of `struct V8' <- a SECOND class
The first is the loose-typing arity conflict: an already-banked shared caller macro in
src/shared/engine_core.h declares the function with FEWER parameters than its byte-true definition takes
(extern s32 func_8016EFC8(s32); and it calls with one arg, while the def takes two — the original calls
K&R-style with fewer args than the callee reads). A C89 prototype makes that a hard error.
The fix already existed and was simply not wired in. tools/fix_arity_callers.py --any-proto rewrites
those caller decls to the no-prototype K&R form (byte-neutral: an empty/short call emits identical code, and
a no-proto decl is compatible with a definition whose params are default-promotion-safe). Byte-probe:
func_8016EFC8 (reach-138) went gate-REJECTED → BANKED byte-identical after 7 caller decls were rewritten.
Wiring notes that cost real time:
- It is a TU-side pre-pass, not an
_xform: it edits shared state (engine_core.h+ the overlay's own inline caller decls), so it is scoped to the drafts in play and reverted for every function the gate then rejects — a bank that SUCCEEDED must keep its loosened decl or the tree stops building. --funcsis REQUIRED;--draftsis only the narrow-param filter. Wiring it with--draftsalone made the stage exitno funcs given— and becausesh()does not raise on a non-zero exit, the surrounding try/except never saw it. The stage silently did nothing and the gate reported 0/6 as if diagnosed. Hence the explicitreturncodecheck now in the ladder: a pre-pass that quietly no-ops is indistinguishable from one that found nothing to do, which is the exact failure this ladder exists to remove (R32). Author's note: this was committed roughly an hour after writing §60b about silent skips.
Measured: on the 7 highest-reach-weighted ov_SC01_077 integration candidates, the enriched ladder banks
2 (func_8016EFC8, func_80164418) against a 1/12 baseline for the old ladder.
THE INCIDENT THIS STAGE CAUSED, and the constraint it establishes. Pairing --apply --any-proto
with --revert for the drafts that did NOT bank broke 138 of 140 binaries. --revert rewrites
() -> (void), which inverts a PLAIN apply but not --any-proto (which relaxes ANY prototype), so the
round-trip turned an unbanked function's real decl extern void func_801708B0(void *a0); into (void) —
a DIFFERENT signature — in engine_core.h and in 6 places in the overlay's own sources.
A SINGLE-BINARY GATE CANNOT VALIDATE A FLEET-WIDE EDIT. harvest_verify --binary ov_SC01_077 reported
byte-identical and was RIGHT — about that one binary. The other 137 were broken and structurally invisible
to it, because the edit lands in a header all 138 overlays include. Only the standing R22 clean-fleet sweep
saw it. This is the same shape as §55b's propagation law, one level down, and it yields a hard constraint:
Any ladder stage that mutates SHARED state (
engine_core.h,engine_types.h, another binary's TU) must be undone by SNAPSHOT RESTORE, never by an inverse transform, and must be validated fleet-wide (R22) rather than by the per-binary gate that authorised it.
gate_stage now snapshots every file the pre-pass touches and undoes by restore + re-apply-for-the-banked-
set-only: exact by construction, and incapable of inventing a signature. The planned type-lift stage edits
engine_types.h — also shared — so it inherits this constraint by default.
The residual class, named for the next stage: redefinition of 'struct <T>' — the draft defines a local
struct the TU already defines. That is the type-lift / local-typedef-uniquify class (§19/§57a/§59), NOT the
arity class, and three of the five remaining failures carry a (void) header decl that the arity pass alone
does not clear. Wire that next, and validate it the same way: splice one, read real stderr, probe, then wire.
§61a — The Task-5 wave: 11/12 MATCH, 0 banked — three DISTINCT integration walls, each now named (Phase 29, 2026-07-21)
A 12-agent Ultracode wave over freshly-prefetched ov_SC06_018 exemplars returned 11 MATCH / 1 near
(~2M agent tokens), including all three giants (710 / 673 / 478 ins). The whole-binary gate banked ZERO.
This is §58's law at its sharpest — and splicing each class individually gave three different blockers,
none of which the ladder currently clears:
-
jtbl NON-CONTIGUOUS CARVE — 10 of 12 drafts. (Corrected: I first filed this as "§8e-2 table-count drift". That is the SYMPTOM the filter reports; it is not the wall, and the fix is NOT a jtbl_carve code change.)
jtbl_rodata_pads: more rodata .align directives than pad specs (2) — table-count drift vs the carve. The draft introduces a switch/jump table into a TU whose carve has a FIXED pad spec, sojtbl_rodata_padsfires. But re-runningjtbl_carve --func <fn>to re-derive the spec REFUSES with the real reason: "subseg would host NON-CONTIGUOUS .rodata carves (0xaa810 and 0xaa920) — a single object can't leave a gap for the unmatched jtbl between them." The newly-banked function's table is separated from the TU's existing carve by an UNMATCHED function's table, and one object cannot straddle that gap.THE RECIPE (byte-proven on
func_80135A4C, 181 ins / 138 members):tools/jr_isolate_all.py <ov> --only <fn> # give the fn its OWN code subseg make extract BINARY=<ov> && make build # isolation is BYTE-NEUTRAL by construction — verify <splice the draft> tools/jtbl_carve.py <ov> --func <fn> # now the table carves contiguously in its own object make extract BINARY=<ov> && make build # -> BYTE-IDENTICALThe tool names its own remedy in the refusal message, and
jtbl_family_bankalready auto-isolates on this class (Phase-29 Task-8) — butgate_stage/harvest_verifydo NOT, which is why a wave that banks through the ordinary gate reports a flat 0 and looks like a compiler wall. A config change needsmake extract, not justmake build(the R22 corollary) — both steps above. The structural finding: fresh crack fuel in a well-matched overlay CONCENTRATES in jtbl-carved TUs (10 of 12 here), because the non-carved TUs were harvested first. So §8e-2 is not a rare straggler — it is the gate on the next tranche of substantial cracking. -
§57 self-decl / prototype conflict — the 2 plain-TU drafts.
argument 'arg2' doesn't match prototype(def at :595 vs the TU's own decl at :447).tools/normalize_self_decls.pyexists for exactly this and is wired intofamily_sweepbut NOT intogate_stage— the same gap the arity pre-pass had. -
Local-type redefinition (
redefinition of 'struct V8') — seen in the Task-14 diagnosis set; wants the type-lift.
So gate_stage's ladder needs three stages, not one, and today only the arity pre-pass landed. Ranked by
what they unblock HERE: jtbl-drift (10/12) > self-decl (2/12) > type-lift.
Method note that made this cheap: the gate's own per-draft label is useless for this (§58's memcpy
red-herring), and make build … | grep -i error MISSED the real failure twice — once because the true error
was a jtbl_rodata_pads line containing no "error" token, once because the build failed at a later stage
than the warnings I was reading. Check rc, and read the tail unfiltered. A filtered build log is a
selection tool, and every selection tool in this project has eventually lied (R32/R35).
Preserved: all 12 drafts at .run/giants/t5wave_* (R20) — they are genuine cracks with per-function
lever notes, recoverable the moment the three ladder stages exist. Do NOT re-draft them.
§61b — The jtbl gate stage: built, and the ORDERING law it exposed (Phase 29 Task-14 stage 4, 2026-07-21)
gate_stage now carries a jtbl stage (_jtbl_prepare): for every draft whose function references a
jtbl_, carve its table into a contiguous object, auto-isolating (jr_isolate_all --only <fn>) on the
§8b walls — the logic lifted from jtbl_family_bank rather than re-implemented (R33). It is wired, it
runs, and it does not yet bank, for a reason worth writing down:
THE CARVE MUST FOLLOW THE SPLICE. The non-contiguity that requires isolation is only detectable once the function's body is in the object. While it is still
INCLUDE_ASM,jtbl_carvereports SUCCESS and produces a spec that does not hold once the body lands.
Byte-witnessed both ways on func_80135A4C: carving the spliced function → NON-CONTIGUOUS … 0xaa810 and 0xaa920; carving the unspliced one → prepared 1/1, no isolation, and the draft then
gates as a byte-DIFF. The MANUAL order banks it byte-identical:
jr_isolate_all --only <fn> ; make extract ; <splice> ; jtbl_carve --func <fn> ; make extract ; build
gate_stage runs the stage before _gate1, but harvest_verify owns the splice — so the fix is a
per-draft prep INSIDE the splice loop (harvest_verify), not a batch pre-pass in gate_stage. That is the
next increment; the stage's carve/isolate/undo machinery is correct and reusable as-is.
RESOLVED (2026-07-21, same session): the prep belongs in harvest_verify, and it BANKS there.
harvest_verify._jtbl_prep() splices each table-bearing draft TEMPORARILY, asks jtbl_carve,
isolates on the §8b walls, un-splices, re-extracts, and re-derives the stub map + baseline (isolation
MOVES a stub's TU, so both are keyed on stale paths otherwise). Byte-proven: func_80135A4C goes
[jtbl] carved 1/1 -> + chunk(1) -> BYTE-IDENTICAL, fully automated.
BUT THE BATCH STILL FAILS, AND THE REASON IS A NEW, PRECISE TOOLING GAP:
Banking a jtbl core makes its own carve UNOWNED to
jr_inventory, which then refuses every subsequent isolation in that overlay —committed .rodata carve ownership is not 1:1 (R32/R33) — a stranded/duplicated carve (§8b func_801734BC class): [('UNOWNED', '0x801d288c')].
Byte-proven both ways: on the COMMITTED tree jr_isolate_all --only func_80135260 --dry-run succeeds;
with func_80135A4C banked it fails the ownership assertion. The assertion is right — a banked
function's stub .s is pruned, so the owner lookup finds nobody — but its conclusion is wrong: the
carve IS owned, by C rather than by a stub. So today jtbl cores bank ONE PER OVERLAY.
Measured on a 10-draft batch: 6 table-bearing, 1 carved, 4 isolate-FAILED on this assertion, 1
stale-asm carve failure.
NEXT INCREMENT (precise): teach jr_inventory's ownership check to attribute a carve to a
BANKED (C) function — i.e. resolve owners from corpus.matched ∪ stubs, not stubs alone (R33: the
same derive-don't-reparse move that fixed the corpus oracle). That unblocks batch jtbl banking and
the 9 preserved cracks.
Two sub-findings, both paid for:
- A wholesale
git checkout -- config/…undo is WRONG in a batch gate.jfb.revertis right forjtbl_family_bank's one-function-at-a-time flow, but here it discarded a PREVIOUSLY-banked-but- uncommitted carve in the same overlay, leaving that bank's source with no subseg →undefined reference to func_80136C90at link. An inverse/wholesale undo cannot know what it did not do. Now a SNAPSHOT-RESTORE ofconfig/splat.<ov>.yaml+config/overlays.mk, plus removal of only the region files THIS run created (§61's constraint, applied where I had first ignored my own rule). - Being in a
_jr_*TU ≠ having a table. Only 4 of 8 wave drafts in jtbl-carved TUs actually reference ajtbl_; the stage correctly prepares only those. The other 4 fail for other classes.
§61c — The jtbl bank is INCREMENTALLY valid and CLEAN-INVALID (Phase 29, 2026-07-21) — the blocking finding
func_80135A4C banks through the automated jtbl path every time: [jtbl] carved → + chunk(1) →
verified 1 / failed 0 BYTE-IDENTICAL. And it fails a clean rebuild, twice, identically:
incremental (harvest_verify's own gate) : BYTE-IDENTICAL
make clean && extract-all && check-all : 139 passed, 1 failed ([FAIL] ov_SC06_018)
So the carve+isolation path yields a state that is not reproducible from committed config + source — the incremental tree carries something the clean pipeline does not reconstruct (extraction order, or asm that only exists mid-flow). This is the §42b stale-incremental false pass in its most expensive form: the gate that authorises the bank cannot see the defect, because the gate IS the incremental build.
Until that reproducibility gap is closed, NO jtbl core can be banked — not by hand, not by the
ladder. The correct next step is to diagnose the divergence itself (diff the incremental vs clean
build/ov_SC06_018/** object set and the generated .ld/asm for the carved subseg), NOT to bank more.
⛔ §61c IS REFUTED — the blocker does not exist (2026-07-22, R35/R14)
The diagnosis above was run, and it never got as far as diffing objects, because the failure does not reproduce. On a tree carrying ONLY this bank, applied through the single-function automated path (
harvest_verify --chunk 1,[jtbl] carved func_80135A4C→+ chunk(1)→ BYTE-IDENTICAL):per-binary clean (rm asm+build for the ov; extract; build) : BYTE-IDENTICAL cbbc4f44… make clean && extract-all && check-all (run 1) : 140 passed, 0 failed of 140 make clean && extract-all && check-all (run 2, independent) : 140 passed, 0 failed of 140So the carve+isolation path IS reproducible from committed config + source. There is no extraction-order effect and no mid-flow asm: the state the incremental gate blesses is the state a clean pipeline reconstructs.
What the 139/140 actually was. The failing R22 runs were taken on the tree left by the batch
_jtbl_prep— the same run that ended6 table-bearing → 1 carved, 4 isolate-FAILED, 1 stale-asm carve fail. That tree carried the residue of five failed preps (stranded carves and half-applied isolations); the per-function snapshot-restore that removes exactly that residue landed after those runs, in the same commit that named the blocker (commit:0803). The measurement was real; its attribution was to the wrong cause. The failing tree is not recoverable, so this is stated as the best-supported explanation, not a byte-proof — but the claim that matters (the path is clean-invalid) is byte-refuted twice, and that is the claim that was blocking the work.The transferable lesson is R35 pointed at ourselves: a fault observed on a tree that is known to be polluted must be re-observed on a clean one before it is written down as a property of the mechanism. "Twice, identically" felt like replication; it was two reads of the same contaminated state, which is one observation. A replication has to re-create the state, not re-run the check.
Faults 1 and 2 below are unaffected — they are real, they are what polluted the tree, and their fixes are what makes the single-function path reproducible. Constraint that stands: jtbl drafts are processed one per
harvest_verifyinvocation until the undo is region-aware.
Two design faults found on the way, both real and both fixed in harvest_verify:
- A stranded carve poisons the overlay.
_jtbl_prepcarved a draft the gate then REJECTED; the carve stayed with no owner (the fn is stillINCLUDE_ASM), andjr_inventory's 1:1 ownership assertion then refused EVERY later isolation in that overlay ([('UNOWNED','0x801d288c')]= func_801299C8's table). The assertion was RIGHT and caught it — R32/R33 working exactly as designed. Fix: per-function carve with snapshot-restore on gate rejection. - Per-function undo is unsound in a BATCH. Isolation REPARTITIONS shared source, so restoring one
draft's snapshot deletes region files that now host OTHER pending drafts — their stubs vanish
(
KeyErrorin render). jtbl drafts must therefore be processed one perharvest_verifyinvocation, or the undo must be region-aware.
Measured, so the next session does not re-derive it: of 11 preserved wave cracks, exactly ONE
(func_80135A4C) reaches byte-identical through the carve path; the other four table-bearing ones
fail one-at-a-time too, on the PLUMBING classes (§57 self-decl et al), not on the carve.
§61d — The undo was eating the tree: two tools, one defect, invisible to the byte-gate (Phase 29, 2026-07-22)
The §61c blocker turned out not to exist (see the REFUTED block above), and chasing why it had ever been observed found the mechanism — in two tools, both times invisible to the gate that caused it.
jr_isolate_allrepartitions a code object by writing region 0 back over the ORIGINALsrc/<ov>/<nm>.c, TRUNCATED to just that region, and emitting the remainder as new_jr_<lo>.cfiles. An undo that restores onlyconfig/and deletes the new region files therefore leaves the original TU permanently truncated — its stubs are gone, and nothing regenerates them (splat does not rewrite a committed overlay.c).
Both harvest_verify._jtbl_restore and gate_stage._jtbl_prepare snapshotted only the two config files.
Measured, twice, on live re-probe runs: live stubs 419 → 414 → 406 → 395, five orphan region files, and
undefined reference to func_80192F64 at link.
Why it survived so long: it is invisible to the byte-gate. The incremental build keeps linking the stale
objects (§42b), so make build stays GREEN while a clean rebuild fails — the gate that authorises the bank
is the same incremental build that hides the damage. This is the §42b blind spot doing real, cumulative
damage, and it is what manufactured the "139/140, twice" reading that became the §61c blocker.
Fixes. harvest_verify: snapshot every src/<binary>/*.c, restore on rejection, and delete exactly the
files the attempt created — derived from the snapshot's file set, not re-guessed from the _jr_* name shape
(R33). gate_stage._jtbl_prepare: DELETED, not patched (R33 — the best outcome is a deleted stage). It
was wrong on two independent axes: §61b had already byte-proved the carve must FOLLOW the splice, and
harvest_verify now does the correct per-draft prep one layer down. Two implementations of one capability,
the outer one both ineffective and destructive.
A label that is constant carries no information (and is worse than none). classify_fail searched the
whole build stderr, so warning: conflicting types for built-in function 'memcpy' — benign, from an
unrelated TU position, present on essentially every overlay build — won the match on 8 of 8 failures
spanning four genuinely different causes. That is the §58 red-herring, and the cookbook had been recording
"the gate label is useless here, splice individually and read real cc1 stderr" as a manual workaround for a
one-line bug. Fix: classify on NON-warning lines; fall through to CC1-FAIL:<last error line>. The same
failure instantly became ov_SC06_018.c:447: prototype declaration.
What the honest re-probe then measured (11 preserved t5wave cracks, one invocation each, clean tree):
func_8018F694(478 ins) BANKED — a giant previously inside "the gate banked ZERO".- The other 10: 4 data-decl · 3 callee-decl · 3 self-decl conflicts. ZERO jtbl-drift, ZERO local-type redefinition, ZERO codegen DIFF. So §61a's "§8e-2 jtbl drift blocks 10 of 12" does not survive the carve-follows-splice prep — the carve succeeds; what is left is ordinary decl plumbing.
- Through
gate_stage's ladder: 0/10 bank, but 9/10 now COMPILE and land as whole-binary byte-DIFF.match_oneclose=0 on several andrtu_matchMATCH-in-real-TU forfunc_80135888, whilefunc_801299C8's transformed draft does not compile in its real TU at all — the residual is MIXED, and at least one is an IMAGE-level effect rather than the draft or its TU context (prime suspect: jtbl/rodata carve placement). Deliberately not generalized from one data point.
The pricing this yields: the existing ladder converts 0 of 10 of these residuals. Task 14 stages 2-3
are therefore NOT "wire in normalize_self_decls + the type-lift and collect ten banks" — a measurement, not
a projection, which is the error §57a already caught once this phase.
The rule, stated generally: any stage that mutates shared state must undo by SNAPSHOT-RESTORE over the complete set of files it can touch — config AND source — and a stage whose undo scope is narrower than its write scope will silently destroy work that no byte-gate can see. §61's law, one level deeper.
§62 — The jtbl RECONCILE must also follow the carve: the post-carve draft reconcile (harvest_verify._jtbl_reconcile, Phase 29 SESSION-11, 2026-07-22)
§61b's law was "THE CARVE MUST FOLLOW THE SPLICE." This is its sibling: the decl RECONCILE must follow the carve too, and run against the CARVED TU.
The wall. A loose-typed jtbl function (calls/references shared symbols declared incompatibly across the
engine) drafts to a clean match_one MATCH, but the whole-binary gate fails to COMPILE — conflicting types for D_XXXX / func_XXXX — not a byte miss. jr_isolate_all accumulates each earlier region's file-scope
decls as the new split region's §8b carried-decl layer (the canonical externs for the data/callees the
body touches). The draft declares those SAME symbols its own way for the byte-match, and where the two
disagree, cc1 rejects the TU. A batch of these read as "carves, 0 banks" and looked like a codegen wall.
Why the existing reconcile ladder misses it (the ordering bug). cast_call_sites (§17a-1 callee cast) +
reconcile_tu (§8d data, conform the draft to what THIS TU declares) already fix exactly this class and take
--src-file. But gate_stage runs them pre-carve and against the default src/<ov>/<ov>.c — never the
jtbl fn's real TU, because for a jtbl fn that TU is the split file, which does not exist until harvest_verify
carves (gate_stage.py:271 deleted its batch jtbl stage for this reason: "harvest_verify owns the splice, so
the prep belongs there").
The fix. harvest_verify._jtbl_reconcile(fn): after _jtbl_prep_one establishes the carve, run
cast_call_sites then reconcile_tu with --src-file <the carved TU> and update drafts[fn]['c'] in place,
before the gate re-splices. A draft-only rewrite (no shared-state edit → no §61 undo needed); the whole-binary
gate stays the sole arbiter (G3/P9). Guarded by _jsnap is not None, so it runs ONLY when a carve happened.
Validated: func_80135260 (callee func_80134A74) and func_80191C50 (data D_801152A8) both went
conflicting types → a genuine codegen DIFF — the reconcile dissolved the plumbing.
The honest finding it exposed (R14/R31). Dissolving the plumbing does NOT bank these families — it reveals
what the plumbing hid. All four ov_SC06_018 jtbl drafts had a DEEPER issue: func_80135260/func_80191C50 a
real codegen residual (a %hi-sharing regalloc the agents' match_one MATCH over-claimed — reloc-masked
match_one cannot see it); func_8012AAAC a def-side-arity conflict AND it is in engine_core.h (fleet-shared)
AND it still DIFFs after the arity fix (a def-side register-threading wall); func_80135EB0 an [jtbl] isolate FAILED. So the reach-138 jtbl families are genuine near-misses/walls, not plumbing-only wins — the projected
"+0.58pp from 3 jtbl families" is refuted. The fix's value is (a) it BANKS any jtbl family that is
plumbing-only-blocked with a true MATCH, and (b) it makes the jtbl gate HONEST — it attributes the blocker
(plumbing vs codegen) instead of reporting every loose-typed jtbl fn as an unbankable wall.
Two traps re-confirmed while building this (both §61-class). (1) Gate jtbl functions ONE AT A TIME — a
single [jtbl] isolate FAILED in a multi-fn --chunk 1 batch corrupts the tree for the whole batch (final SHA None); the per-function restore does not contain a mid-batch isolation failure's damage. (2) The
fleet-shared-state restore trap: fix_arity_callers --funcs <fn in engine_core.h> edits src/shared/ engine_core.h fleet-wide; a git checkout HEAD -- src/<ov>/ restore MISSES it (wrong directory), and the
per-overlay build stays byte-identical so nothing flags the leak. Restore src/shared/ too, and R22
clean-fleet after any fix_arity probe (this bit me exactly as §61 warns — caught by a full git status).
§63 — The fresh-138 DEF-SIDE blocker: fix the HEADER decl, not the draft (fix_header_decl.py, Phase 29 SESSION-13, 2026-07-23)
The blocker. A genuinely fresh reach-138 family (live in ~138 overlays, matched nowhere, no
DEFINE_func_* macro) with a byte-perfect universal draft still fails the gate as
conflicting types for func_X (cc1 exit 33). Cause: a shared caller macro in src/shared/engine_core.h
forward-declares it with a SIMPLIFIED, caller-derived signature — extern void func_X(s32 a0, void *a1, void *a2); — that conflicts with the byte-true def int func_X(s32, u16*, u16*) the moment the def is
spliced into an overlay TU that #includes the header. gate_stage's §61 arity pre-pass is
param-COUNT-only (misses return type + pointer element type); §54 reconcile_def_sig rewrites the DRAFT
to match the header (wrong direction — the header is the simplified one, matching it can DCE the return).
The fix — proven ×138 (func_8014CD80). Rewrite the HEADER decl to the byte-true def. It is byte-neutral:
(a) return void↔intN — the true fn always sets $v0; a caller that declared it void never read $v0
(unchanged), and no caller USES the return; (b) pointer element type void*↔T* — register-passed
regardless. One edit → harvest_verify banks ×1 BYTE-IDENTICAL → dedup_propagate --addr fills 138/138
byte-identical → R22 140/140. Fleet +0.1pp instr off ONE family.
tools/fix_header_decl.py automates it: --fn --draft [--check|--apply]. Parses the byte-true def sig,
canon-compares (typedef-aware: int≡s32, unsigned short*≡u16*) to skip ALREADY-OK decls, REFUSES any
rewrite that changes param COUNT / flips pointer↔scalar / changes scalar class (ABI change — left to a
human), and preserves the macro line's trailing \ continuation (dropping it corrupts the DEFINE macro
fleet-wide — a self-test caught this). Snapshots + prints the git checkout restore (§61: undo = restore,
never inverse; validate FLEET-WIDE by R22, never the per-binary gate).
Market (SESSION-13 scan): of the 75 fresh (≥100-live) families, 46 carry an engine_core.h caller
forward-decl, 38 SIMPLIFIED = this pattern → each a candidate ×138 (≈+1.5–2.8pp instr). RESIDUAL RISK: the
header is only the INTEGRATION half — each family still needs a byte-true UNIVERSAL body from a wave; a
non-universal body (overlay-local D_* refs) is a genuine per-member wall regardless (func_80165CA0,
0/135). Distinguish: h_exact=1 does NOT — the BODY's universality does. Pipeline: wave (build_wave_args.py --rank live --min-live 100) → fix_header_decl --apply → harvest_verify → dedup_propagate --addr → R22.
§64 — The §20 type-lift's three laws: fold the tagged typedef, check VISIBILITY, and strip only what is TEXTUALLY IDENTICAL (lift_types.py, Phase 29 SESSION-14, 2026-07-23)
The setup. The broad §20 lift (154 fleet-local types → engine_types.h, 2,958 files stripped) had been
carried three sessions as "needs collision-vetting + -O0 strip precision." Both framings were wrong, and the
one real collision class was misdiagnosed. Fixing the tool first (R35) changed all three answers.
Law 1 — a tagged typedef is ONE entity, not two. typedef struct Tag {...} Alias; is matched by BOTH
find_defs (as Tag) and find_typedefs (as Alias), with the struct span CONTAINED in the typedef span.
Treating them as independent does three wrong things at once: it lifts the inner span, which starts at
struct, so the header gets struct Tag {...} Alias; — a variable definition of Alias in every TU; it
then lifts the typedef too (Alias redeclared as different kind of symbol); and it strips both spans
highest-first, so the outer span's end offset is stale by the length of the inner one and the second
delete removes that many EXTRA characters past its intended end. MEASURED: 13 tag/alias pairs, 6,142
occurrences, and 0 of those tags is ever defined standalone — folding is always the correct read.
build_engine_types.resolve_type_defs() is now the ONE model both tools call (R33), and it guarantees
defs/tdefs spans are pairwise disjoint; assert_disjoint() enforces that at every mutation (R32).
The Phase-29 "case-variant name collision" (
actor4cvsActor4C) was this, not a naming problem. A single-def filter accidentally hid 12 of the 13 pairs and leaked exactly the one whose tag and alias differ in case, which made a case-insensitive exclude look like the fix. It is not: it would also wrongly drop the legitimate, non-collidingObj/objandVec/vecpairs. Key by (kind, name) — the C namespace.
Law 2 — never strip a definition out of a TU that cannot SEE the replacement. MEASURED: exactly 1 TU
of 3,226 (ov_SC01_077_o0.c, the -O0 split) deliberately omits engine_core.h — for the documented reason
that its -O0 functions cannot propagate through a header other overlays compile at -O2. Stripping its types
does not consolidate them, it DELETES them, and the symptom surfaces three steps downstream: type undeclared
→ parse error on the next declaration → "data definition has no type or storage class" → gcc falls back
to implicit int → that TENTATIVE definition collides at LINK as multiple definition of D_801DAA08 — a
link error naming a data symbol nobody touched. That is the whole of the "-O0 strip precision" mystery.
bet.type_visible(path) derives the visible-header set from the src/shared include graph (R33) and the
lift keeps such defs local, naming them.
Law 3 (the expensive one) — strip a local def ONLY if what becomes visible is TEXTUALLY IDENTICAL to it.
--candidates classifies per ENTITY and defers VARIANTs, but --types takes NAMES: struct Prim was
LIFTABLE while typedef Prim was a deferred VARIANT (header ≠ the 103 overlay copies), and passing the
shared name dragged the deferred entity back in. Result: 103 overlays silently repointed at a different
Prim layout. It compiled clean, the per-binary pre-filter passed, and R22 failed 37/140 — the 103
failures were EXACTLY the 103 Prim-stripped overlays (set equality, byte-verified). The guard now lives at
the mutation, not in the selector, so a selector bug cannot reach the source; divergent copies are reported
as the per-camp reconcile work they are.
Sequencing that paid for itself. make build BINARY=ov_SC01_077 reproduced the -O0 failure in 0.26 s
with a compile error naming the five affected types, where the fleet cycle had produced a link error naming
an unrelated symbol. But a pre-filter is only evidence about what it filtered: ov_SC01_077 passed the
Prim-broken run too. Pre-filter on a binary that FAILED, not one that passed (§61 one level down).
Result. 154 types lifted, 2,958 files stripped, R22 140/140 byte-identical, engine_types.h +510
lines. Deferred and named, not silently dropped: 8 VARIANT entities (MATRIX/Buf/Vec8/Prim/Handler/…) awaiting
the per-camp field-access reconcile, 14 carried tags, and 5 types kept local in the -O0 TU.
§64a — VARIANT types: UNIQUIFY the camps, do not reconcile them (uniquify_type.py, Phase 29 SESSION-14)
The trap. §64 defers any type with 2+ distinct definitions as a VARIANT, and the obvious next move reads as "pick the canonical layout and reconcile the other camps' field access." For most of these names that is actively WRONG. Measured on this fleet:
Vec8 = { s32 w[8] } 180 files (32 bytes)
Vec8 = { s16 unk0,unk2,unk4,unk6 } 139 files ( 8 bytes)
MATRIX = { s32 m[3][3]; s32 t[3] } 578 files | { short m[3][3]; long t[3] } 71 files
Buf = { s16 h[8] } 578 files | a 0x20+ struct 6 files | { DrawEnv env; … } 1
These are not one type with two spellings — they are different types that happen to share an
identifier in different TUs of the same overlay (each TU carried its own local guess, which is
exactly why they diverged). "Reconciling" them MERGES two layouts and silently repoints the minority
camp's TUs at the wrong struct — the same failure mode that broke 103 binaries on Prim (§64 Law 3).
The correct operation is the opposite: keep both layouts, give them distinct NAMES.
reconcile -> merges two layouts -> breaks the minority camp WRONG
uniquify -> preserves both layouts -> every camp becomes liftable RIGHT
Renaming a type is byte-neutral (a type name emits no code) and TU-local by construction: the
definition and all its uses live in the same file. Once each camp is uniquely named it has exactly ONE
definition fleet-wide, so lift_types.py lifts them all by its ordinary rules and the §20 propagation
cap lifts with them. tools/uniquify_type.py --type <T> [--apply] does it: deterministic camp ordering
(file-count desc, then normalized text, so re-runs assign the same suffixes), majority keeps the name,
camp n becomes <T>_c<n>, and it rewrites ONLY files that DEFINE that camp — a file that merely uses
the name is getting the type from elsewhere and must keep referring to it (\bT\b word boundaries also
keep Buf from matching Buf80153978).
Validated on Buf (the cheapest camp: 578 / 6 / 1 files). 11 identifiers rewritten across 7 files →
3 camps all LIFTABLE → lifted (585 local copies stripped) → R22 140/140 byte-identical → the blocked
core queue fell 13 → 11 (0x8012ea90, 0x801749c8 freed, both Buf-blocked). Recipe:
uniquify_type --apply → lift_types --candidates → --types <camps> --apply → pre-filter build →
R22 → dedup_propagate --addr. Remaining camps by cost: MATRIX (578/71/6), Vec8 (180/139 — no
clear minority, so expect to name BOTH camps), then Handler / Blk8 / V8 / Prim / Prim_8016E7C8.
Also fixed here (R32): dedup_propagate's skip line printed a COUNT and no names, and aggregated
three unrelated causes into n_local — a body skipped merely for containing a // comment or a line
continuation (macro-unsafe, a one-line fix) was reported identically to one genuinely using an
overlay-local type. The queue is now named and split by cause, so the work it represents is visible.
§63 UPDATE (Phase 29 SESSION-14) — fix_header_decl's "SAFE" verdict is FLEET-BLIND; it MUST be R22-validated
The §63 header-decl reconcile (void→s32 widening of a shared engine_core.h caller decl) banked 3 of
3 fresh cracks under the PER-BINARY gate (gate_stage --no-propagate on ov_SC07_006) — then R22
clean-fleet FAILED (139/140): ov_SC01_077 broke. Reverting the biggest-fanout decl (func_8012CC88, 12
refs) did NOT fix it — at least one of the 2-ref void→s32 widenings (func_8014D12C/func_8014CF04)
ALSO perturbs ov_SC01_077, where those functions are already matched and a caller's codegen shifts under
the widened (even no-proto ()) shared decl. fix_header_decl --check's [SAFE] verdict only inspects the
ONE caller's return-use; it is structurally blind to the other ~137 overlays the shared decl reaches.
RULE: a fix_header_decl (or any engine_core.h decl) edit is a §61 shared-state mutation — validate it
with a full R22 clean-fleet, NEVER the per-binary gate that authorised it. The per-binary gate is a
NECESSARY-not-sufficient filter here. The byte-gate caught it; the whole recovery pass was reverted, 0
broken landed. The 3 drafts are byte-correct in isolation — they need a per-overlay-local decl or an
alternate integration path (not a fleet-shared header widen), logged to the backlog.
§65 — The stranded-draft recovery: BLAST-RADIUS TIERS, and the per-overlay de-macroize that refutes §20's DEF-conflict wall (tools/demacroize.py + tools/blocker_probe.py, Phase 29 SESSION-16, 2026-07-24)
The population. A crack wave banks ~27% of its drafts; the rest are stranded. Measured over the
s14+s15 residue (36 drafts, ov_SC07_006): 24 are match_one MATCH, 11 are near, 1 ERR. So the
audit's "~92% byte-correct" is a WHOLE-WAVE figure; among the stranded residue it is 67%. A third
of the "we strand paid-for correct functions" premise was never correct — and the near ones are
exactly the drafts that compile in their real TU and DIFF. Measure the residue separately from the
wave before pricing a recovery pass off it (R14).
Blockers STACK, and cc1 reveals only the FIRST. Per-blocker counts over those 36: self_decl_hdr
21 · callee_decl 19 · data_decl 16 · self_decl_tu 5 · local_type 5 — on 36 functions. So a
per-function verdict read off one cc1 run is always an underestimate; route on a static oracle's
COMPLETE list and use cc1 to confirm the class, not to enumerate it. A function's recovery tier is the
MAX over its blockers, never the first one's.
§65a — The blast-radius taxonomy (makes §61's law structural instead of remembered)
| tier | write set | validator | why |
|---|---|---|---|
| T0 draft-only | the draft file | per-binary gate | nothing else changed |
| T1 binary-local | src/<binary>/** |
per-binary gate — SUFFICIENT | the write set cannot reach another binary |
| T2 fleet-shared | src/shared/**, config/** |
R22 clean-fleet, MANDATORY | one edit reaches ~140 binaries (§63 UPDATE) |
The §63 disaster was not "a per-binary gate is untrustworthy" — it was a T2 edit validated by a T1
validator. Stating the tier makes the right validator mechanical. Every stage should DECLARE its tier
and the driver should MEASURE the write set (git status --porcelain before/after) and assert
containment: a stage whose undo scope is narrower than its write scope destroys work no byte-gate can
see (§61d), and one that lies about its tier should abort, not be trusted. Validated empirically
here: a src/<binary>/**-confined batch of 14 banks passed R22 140/140 three times.
§65b — The escape: de-macroize the instantiation, don't touch the shared header
The blocker (the largest single class, 21 of 36). A DEFINE_func_* macro in engine_core.h
forward-declares the draft's own function with a caller-derived signature the byte-true definition
cannot satisfy (typically void where the real return is live). Both obvious fixes are byte-proven
wrong: rewriting the DRAFT to the header sig compiles but MISMATCHes (§20/§58b — the body needs its own
sig), and rewriting the SHARED decl is fleet-blind (fix_header_decl banked 3/3 per-binary then failed
R22 139/140, §63 UPDATE).
The key structural fact: the conflicting extern lives INSIDE the macro BODY, so it exists only
where the macro is INSTANTIATED — and for a given overlay that is a handful of sites in that overlay's
own TU. Replacing those instantiations with the macro's own expansion, correcting ONLY the conflicting
declaration to the draft's byte-true signature (never DROPPING it — §57a-1's def-after-caller
trap), dissolves the conflict as a T1 edit. Every other overlay is textually untouched, so no R22
risk is created by construction. This is the "per-overlay-local decl" §63's own note named as the
unexplored alternative.
⇒ §20's "the DEF-conflict class is byte-proven unrecoverable by text transform" is REFUTED for the per-overlay case. §20's reasoning was sound for the mechanism it considered (the shared macro's
externis the only declaration in the 137 STUB overlays, so it cannot be dropped, and it "can't be edited per-overlay — it's in the shared header"). The missed move is that you do not have to edit the header to change what ONE overlay sees: you expand the macro there. Measured: 13/14 of the clean candidates MATCH in their real TU; 14/14 of the whole-binary attempts banked.
Generalizes to callee conflicts. The same transform, applied to any decl in the body that the DRAFT
declares incompatibly (not just the draft's own function), reaches the callee-conflict variant — e.g. a
macro declaring RotTransPers differently than the draft does. The draft is byte-truth for every type
it was compiled against; the macro body is what must yield.
The price, stated before the work, not after. A de-macroized function can no longer be propagated
×138 from the shared macro (dedup_propagate would re-macroize it). Such a bank is ×1: it credits
the FULL distinct-code unit (progress.py marks an h_exact class matched if ANY instance is —
matched_cls.add(hx)) but only ~1/138 of the instruction-weighted headline. Measured: 14 banks →
distinct-code +14 unique fns, instr-weighted and fn-count ~flat. For the 0-stubs completion contract
that is real progress; for the decomp.dev display number it is not. Say which one you are buying.
§65c — rtu_match MATCH → bank held 13/13 on self-decl, but broke on the FIRST callee-decl case
rtu_match is far better than match_one here (it compiles the real TU, so it sees decl conflicts),
and its real-TU MATCH predicted the whole-binary bank 13/13 for the self-decl class. It then failed
on func_8012F49C — a callee-decl case. That is its documented blind spot doing exactly what it
says on the tin: it is relocation-masked, so a wrong call TARGET is invisible to it and fatal to the
real link. Trust rtu MATCH more when the correction is to the function's own declaration; trust it
less when the correction is to something it CALLS. The whole-binary gate remains the only arbiter
(G3/P9), and it cost one build to find this.
§65d — Existing-ladder baseline, measured (do this before building a recovery stage)
Running the existing draft-side ladder (cast_call_sites + reconcile_tu) over all 36 clears exactly
what it targets — callee_decl 19→3, data_decl 16→0 — and converts 1 of 36 from CC1-FAIL to
compiling, which then DIFFs. §61d verbatim: dissolving the plumbing reveals what the plumbing hid.
The conclusion is not "the ladder is broken" — it is that the ladder's classes were not this
population's blocker. A recovery pass that had been built on the assumption it was would have measured
~0 and been abandoned as "the wall". Baseline the existing tooling against the actual residue first; it
costs seconds and it re-prices the whole task.
§65e — Two oracles, and the disagreement is the finding (R34 in practice)
tools/blocker_probe.py runs a STATIC oracle (cdecl.parse + cdecl.compatible — cc1's own
acceptance question) beside the REAL cc1 (via rtu_match), and reports where they disagree. Two things
this bought immediately:
- Text equality is never the question. The deleted
.run/diag_plumbing.pycompared declaration TEXT, soextern u8 D_X;vsextern unsigned char D_X;read as a conflict althoughcommon.htypedefs make them the same type and cc1 accepts both silently. That artifact alone accounted for the pre-probe "~10 data-extern co-blockers" estimate. - The static oracle's own blind spot, named by the other oracle. A bare
struct Tag {...}redefinition is not aDeclaratorconflict, so the static side reported "no blocker" forfunc_80173A60while cc1 saidredefinition of 'struct S80126B38'. The CC1-ONLY column is where a single-oracle design would have silently mis-routed the fix. Also:cdecl.tu_scoperuns REAL cpp (tu_statements, the §8c law), so it already expands instantiated macros and is authoritative about whether a conflict EXISTS. A separate macro scan is still needed, but only for attribution — TU text vs shared-header macro body are the same tier and different edits.
§65f — The de-macroize lever's BOUNDARY, measured: byte-neutral 14 times of 15, and the 15th is why the gate exists
De-macroizing corrects a declaration that an already-matched function (the macro's own body) is compiled against. Usually that is byte-neutral — but not always, and the failure is not a compile error:
| outcome | n | signature in the gate |
|---|---|---|
| banked BYTE-IDENTICAL | 14 | verified N / failed 0, final SHA == locked |
| edit shifted the macro's own matched function | 1 | verified 0 / failed 1, final SHA is a REAL hash ≠ locked |
func_8014C4AC was rtu_match MATCH and still failed, because the baseline (draft reverted, the
de-macroize edits still in place) no longer built byte-identical. Read the final SHA, not just the
verified count: final SHA None = the build produced no image (a compile/link break, e.g. §65g);
final SHA <hash> ≠ locked = the TU edit itself moved bytes. Those are different faults with different
fixes, and the count alone does not distinguish them.
So the lever's precondition is not "the draft is byte-correct" — it is "the draft is byte-correct AND correcting the decl does not perturb the macro's own function." The second half is unknowable in advance and costs one build to test. That is exactly the §63 failure mode, relocated from the fleet (where it broke 139/140 invisibly) into a place the per-binary gate can see and reject for free.
§65g — Where the cheap levers STOP: the local-type and in-TU-self-decl classes did not yield
Measured, both reverted with nothing landed:
local_type(4 fns) — 0 banked.canon_sig_reconcile._uniquify_draft_typesrenames the draft's colliding type (byte-neutral: a type name emits no code), but that renames the type in the draft's ownextern <T> D_x;declarations too, so it trades aredefinition of struct Tfor aconflicting types for D_x. Re-ordering (uniquify BEFOREcast_call_sites/reconcile_tuinstead of after) fixes the ordering bug and lets one compile — which then DIFFs 48/53, because the data reconcile's cast-at-use changes real codegen when the body's semantics depend on its own struct layout. A struct-TYPED data extern is the casereconcile_tucannot cast.self_decl_tu(3 fns) — 0 banked.normalize_self_declsskips a literal()unconditionally (blind to a RETURN-type conflict, sofunc_801376E8normalizes 0), and its narrow-param rewrite on the other two broke the build outright (final SHA None). §57a already classes it SURGICAL-ONLY — it edits the TU file, so a bad edit poisons the whole group and cannot be bisected per-member.
⇒ The recovery pass's measured yield is 14 of 36 (39%), and that is where the EXISTING transforms stop. The remaining ~8 blocked functions are not a matter of running one more tool; each needs a transform that does not exist yet. Do not re-run these two classes expecting a different number — the negative is byte-recorded here precisely so the next session does not re-buy it.
§66 — Exercise a banking driver's SUCCESS path before pointing it at a wave: the free re-bank test (Phase 29 SESSION-17, 2026-07-24)
A driver can be "built, proven, and encoded" and still have never run its own success path. SESSION-16
ended exactly there: recover_integration.py's end-to-end run banked 0, so pass 2 (re-stage winners
→ propagate), --commit, the r22() helper and --report had never executed. A driver that mutates
src/ and can commit is the wrong place to discover that.
THE TEST, and it is free. Revert ONE already-banked function to its INCLUDE_ASM stub, then re-bank
it through the driver on its saved draft, with --commit --r22. Cost: one build. It exercises
pass1 → restore → pass2 → commit → R22 → report on a case whose answer is already known.
Two properties make it a strong test rather than a smoke test:
- A byte-checked baseline for free. The reverted (stub) state must rebuild byte-identical — the
stub pastes the original asm — so a faithful revert proves itself before the driver runs. Reverting a
banked def needs
make extractfirst: splat only emitsasm/nonmatchings/**/<fn>.sfor functions not defined in source, so the.sthe restored stub includes does not exist until you re-extract (the R22 corollary, in its source-reverted form:can't open …/func_X.s for reading). - An exact equivalence check on the output.
git diff <pre-revert-commit> -- src/must come back empty: the driver has to reproduce the known-good banked state character-for-character. Anything else is a real behavioural difference, not a judgement call. (Ours: one stray blank line, from the hand revert, not the driver.)
§66a — The widest write in a pipeline is the one most likely to be UNDECLARED
The §65a blast-radius taxonomy was applied to the stages the driver author wrote (arity = fleet,
demacroize = binary) and not to the pipeline's own inherited default: run_gate(propagate=True)
shells out to dedup_propagate --auto-from, which writes src/shared/engine_core.h and up to 138
overlay .c files. So --max-tier binary — the default, chosen to mean "no shared-state edits" —
still performed the widest write in the toolchain, and --no-propagate was a thing you had to
remember (the checkpoint carried it as a "STANDING HAZARD", which is the tell).
A measurement-based guard cannot save you here. assert_write_set diffs git status --porcelain,
and under --commit the writes are already committed by the time it could look — it sees a clean tree
and passes. Containment by measurement must run before anything commits; otherwise refuse up front,
the way stage tiers already are. Now: propagation requires --max-tier fleet AND --r22, and is
refused outright after a demacroize stage (whose banks are ×1 by construction, and which
--auto-from would re-macroize and undo). Both refusals negative-control-tested, exit 1.
Generalize: when you add a blast-radius taxonomy to an existing pipeline, enumerate what the pipeline already did — inherited defaults are exactly the writes nobody re-reads.
§66b — A metric parsed out of another tool's prose goes NULL silently when the label changes
gate_stage scraped the fleet % from progress.py --fleet with a regex for byte-identical :.
progress.py later replaced that single line with the three-metric block (FLEET fn-count byte-ident:
/ FLEET instr-weighted : / FLEET distinct-code(uniq):). The regex matched nothing from that day
on, fp became None, and 50 automated bank commits recorded fleet None% in their messages
before anyone added up the zeros (R32 — a silently-nulled number is a defect, not a no-op). Fixed to
read FLEET instr-weighted with the legacy label as fallback and a loud stderr warning when neither
matches, so the next rename cannot go quiet.
Same shape, one level up: a digest regenerated while an experiment's edits are still on disk gets
committed as if it described the committed tree. docs/progress.fleet.md at HEAD disagreed with HEAD's
own source by 45 in the dedup-shared column — it was generated during the §65g local_type trial and the
edits were reverted afterwards. Provable in one command: regenerate from committed source and compare
(two independent runs, in-gate and standalone, agreed). Regenerate digests on a clean tree, or the
digest describes a state that no longer exists.
§66c — Before a wave, verify the FUEL exists; an "already attempted" set built from the wrong directory lies in BOTH directions
build_wave_args.py ranks candidates by live-sibling count but has no notion of already attempted —
so its "FRESH (>=100 live)" line means "fresh by leverage", not "never drafted". Filtering it is on you,
and the filter is where the traps are. Both of these happened in one afternoon:
- Exclusion set too NARROW → false work. Built from the backlog alone, it missed
.run/giants/, so the top three "fresh" targets werefunc_80176734(close=76),func_80176218(close=271) andfunc_80140958(close=116) — ~2.6M agent tokens of preserved, characterized, permuter-only drafts a wave would have paid to re-derive. Preserved drafts do not all live in the backlog (.run/giants/,.run/wt_uni/,.run/drafts_wave4/, the per-wave.run/drafts-*dirs); 30 of 141 high-reach cached targets have a draft and no backlog record at all. - Exclusion set too WIDE → false exhaustion. Corrected to scan
.run/**/func_*.c, it swept in.run/ghidra_c/func_*.c— which is the Ghidra decompile INPUT, not an attempt — and reported 5,488 attempts, making 100% of every pool read "already tried". A 0 from that filter and a 0 from a correct one are the same number and opposite facts (R35, one level up from tools onto queries).
The query that actually answers "is there fuel?" — over all binaries, per candidate: (cached
Ghidra-C?) ∧ (still an INCLUDE_ASM stub in ≥N binaries — the TRUE lever, recomputed from
corpus.stubs, never a manifest's stale reach) ∧ (no backlog record) ∧ (no preserved draft
anywhere outside .run/ghidra_c/). Answer at this checkpoint: 141 cached high-reach targets → 111
gated, 30 draft-only, 0 never attempted; the only never-attempted cached fuel fleet-wide is 40
functions at live 1–4.
The structural point: wave fuel at high reach is created by a per-overlay Ghidra-C prefetch, not found — the cached pool is a consumable, and once a wave drains it the next wave's cost/benefit is computed against a pool that no longer exists. Re-measure the fuel before each wave, not the leverage.
§66d — The permuter⇄reader loop: alternate a random search with a byte-verified idiom, and let residual_class decide whose turn it is (Phase 29 SESSION-17)
func_80177940 (101 ins, reach-138) was preserved as close=5 with the note "equal-priority
birthing_insn_p ties broken by INSN_LUID, not steerable from source order — swept all 6 assign
orders/pin combos". Both halves of that note were right, and it still banked, because the two halves
belong to different tools:
| residual | who closes it | why the other cannot |
|---|---|---|
| 4-ins scheduler tie (INSN_LUID order) | the permuter (5 → 1, 900 s) | a hand sweep enumerates source orders; the tie is broken by internal LUIDs, which randomized decl/statement churn moves and a human cannot address |
andi $a2,$v0,0xf vs addu $a2,$v0,$zero |
the reader | the permuter's mutation set adds masks/casts; it does not invent "delete this mask AND hard-pin the destination" |
| the register/schedule fallout of that fix (6 left) | the permuter (6 → 0, 1800 s) | ditto the first row |
The handoff signal is free and already computed. tools/match_one.py prints residual_class's
bucket on every run: [permuter] → hand it back to the search; [structural] → read it. Here it went
structural(OPCODE-MIXED) → after the idiom fix ADDRESSING [permuter] profile=cse
→ MATCH. Feed that profile straight to p16_permute --klass (permuter_weights.classify accepts a
profile name verbatim).
⚠️ Correction, same session, before this rule could mislead anyone: [structural] is a HINT, not a
veto. func_80177940 was structural(OPCODE-MIXED) at close=5 and the permuter still took it
5 → 1 — at low closeness a randomized search reaches ties the class label says nothing about, since
the label describes only the dominant residual. Read [structural] as "a search alone probably will
not finish this", not "do not run the search". Cheap policy: at LOW closeness run the search anyway (it
costs CPU, not tokens); at HIGH closeness read first, because a search that must invent a semantic
change will burn hours to plateau.
Diagnose from a byte-verified SIBLING, never from first principles. The fix came from
func_801778A8 — same family, same nibble walk, already banked byte-identical — which writes
nib = uVar1; (a plain copy) between two hard-pinned variables after the identical (x << 16) >> 28
shift pair. And test the naive reading first: deleting the redundant & 0xf alone collapsed the
copy (100 vs 101 ins, 52 mismatched), because gcc then reused one register. That failure is what proved
the target needs a distinct pinned register — the mask was never the point. A residual of one
instruction can still be a two-part fix.
§66d-1 — What transfers between giants is the LOOP, not the PIN
Applied to func_8014D820 (304 ins, close=33), whose residual is the mirror image (target holds the
lhu results in $v1, keeps $a0 live to fill the load-delay slot), the same move — pin the
reusable temp to the register the target uses — went 303 vs 304 ins, 285 mismatched: $v1 is needed
elsewhere in that function. §44's "each giant is its own class" holds at the level of the specific pin.
Its permuter run improved 33 → 27 and plateaued (seed kept for an ILS warm restart). Budget the loop
per giant; do not budget a pin.
§66d-2 — Two operational sharp edges
p16_permute.setupwipes.run/permuter/<fn>/. Re-running a target destroys the previous run'soutput-<n>-*dirs, best score included. Copy the bestsource.cout before re-running — the score-1 candidate here survived only because it had been copied to a run-local path first.run_permuter's cleanup used a globalpkill -f permuter/run_masked.py, which matches every concurrent run — so two permuters on a multi-core box silently killed each other the moment the first timed out, with no error anywhere. Now scoped to the run's own scratch dir (it is in argv). This is what makes grinding several giants at once safe, and it is likely why the giant queue had only ever been run one at a time.
§66d-3 — Read the ILS per-cycle SERIES, not its final best: a repeated score and a still-falling one look identical in a summary line and mean opposite things
Two permuter_ils runs, same session, same machine, same 8×240 s budget:
| function | per-cycle best | verdict |
|---|---|---|
func_8014D820 |
25, 25, 25, 25, 25, 25, 25, 25 | DONE. The warm restart has nothing left to find; more CPU is waste. Escalate to a reader / Fable5, or accept the stub. |
func_80140958 |
80, 75, 72, 69, 64, 62, 60, 59 | BUDGET-LIMITED. Monotone, never repeating, still dropping on the last cycle → buy more cycles. It is CPU, not tokens. |
ILS done: best=NN is the same line in both cases, and choosing on it alone gets both decisions wrong
— you stop the one that was still descending and keep paying for the one that had finished. The series
is already printed per cycle; the rule is: a repeat means stop, a fall means continue. (A first-cycle
drop followed by repeats — func_8014D820's 27→25 then ×7 — is the "one easy waypoint then done" shape,
which still means stop.)
§67 — The arg-copy PLACEMENT lever: launder a parameter into a fresh pseudo AT the statement where the target's copy lands (Phase 29 SESSION-18, func_8014D820 25 → 16)
func_8014D820 (304 ins, reach-138) sat at the permuter floor (ILS 27→25, then 25 ×7) with the
SESSION-17 note "target births $s3←a1 FIRST, $s4←a0 twelfth; mine the reverse … §17 register-ORDER
class". The diagnosis was right and the class label was wrong: this is not a register-ORDER problem you
solve by pinning registers (four pin/barrier/staging variants were byte-measured inert or worse). It is
an instruction-PLACEMENT problem — where gcc materializes the entry copy addu $sN,$aX,$zero.
The cascade, and why it looks like three unrelated bugs. gcc-2.7.2 schedules the arg→pseudo entry copies as ordinary in-block insns. An unconstrained copy gets hoisted to the earliest ready slot, and whichever parameter's copy goes first frees its argument register, which then gets used as the early load temp. So one root cause presents as:
| symptom | actually |
|---|---|
sw $s4 / move $s4,$a0 first, target has sw $s3 / addu $s3,$a1 |
the copy raced to slot 2 |
my load temp is $a0, target's is $v1 |
consequence — a0 left $a0 early, so $a0 was free |
I am +1 ins with an unfilled load-delay nop |
the target fills that slot with the deferred copy |
a later scratch is $v1 where the target uses $a0 |
downstream of the same free-register difference |
Do not chase these separately. Fix the placement and they collapse together (here 25 → 16 in one move, with the whole prologue going exact and the instruction count landing on 304).
The lever. Declare a plain, unpinned local and launder the parameter into it at the statement where the target's copy sits, then rewrite every later use of the parameter to the new name:
s32 a0v; /* NOT a register pin -- see below */
...
__asm__ __volatile__("" : "=r"(a0v) : "0"(a0)); /* zero instructions; forces the copy HERE */
ent = *((Ent **) (a0v + 0x170));
"0"(a0) ties the input to output 0, so gcc emits exactly the copy it was going to emit anyway — the
asm only fixes when. It costs no instruction.
Placement is the tuning knob, and it is NOT linear — sweep it. Measured for func_8014D820, whose
target copy sits at idx 12 (between the a1[0] load and the dx subtract):
| launder placed before … | copy lands at | result |
|---|---|---|
dx = t - u; |
idx 10 (hoisted above both loads) | 305 ins, worse |
t = a2[2]; / u = a1[2]; / dz = t - u; |
idx 12 ✅ | 304 ins, 16 mismatched |
the if (…) goto fail; |
idx 16 (the other delay slot) | 304 ins, 18 mismatched |
Three adjacent statements all give the same correct slot, so the target is a plateau, not a knife-edge —
sweep 3–4 anchors and take the best. Adding an artificial input dependency ("r"(t)) to force a slot
did nothing: gcc still hoisted the copy above the load. Placement, not dependency, is the control.
Two hard rules, both byte-measured:
- Do not pin the laundered variable to a hard register. Pinning
a0vto$s4(__asm__("$20")) made gcc pre-stage through$t0— an extramove t0,a0at idx 1 and 305 ins. Unpinned is correct; gcc picks the same callee-saved register by itself. (Same failure shape as pinning a reused temp: pinning the load temptto$3globally gave 287 mismatched / 303 ins.) - Launder inside the block that owns the delay slot. Placing it after the
beqzthat ends block 1 forces a second materialization (the value must already survive the branch), so you pay an extra insn and gain nothing.
Prerequisite — collapse redundant pointer aliases first (the two-pseudo law). The draft carried
new_var2 = a1; and used both names. That splits one pointer into two pseudos: gcc served the first use
out of the incoming arg register and deferred the callee-saved copy, which is exactly the defect. One
sed collapsing new_var2 → a1 made a1[0] load from $s3 like the target and removed idx 11 from
the diff. Do this before reasoning about copy placement, or you are debugging two mechanisms at once.
Re-test your older pins after the fix — they may be crutches. A register s32 t __asm__("$3") pin
was load-bearing before the launder (it forced the $v1 load temp) and provably redundant after
(identical 16 either way), because once a0 stays live through the early loads the temp must be $v1.
Drop it: same bytes, simpler C, one less thing to explain to the next session.
This is a PATTERN, not a one-off — it fired twice on the same function. The t pin ($3) went
redundant after the launder, and later the u pin ($2) went redundant too (identical 9 either way),
leaving only the a2 $7 pin, which stays genuinely load-bearing (dropping it costs 9 → 29). A register pin is a crutch for a mis-scheduled value; once the
value is scheduled correctly the pin is dead weight — and a stale pin actively costs you, because it
reserves a hard register the allocator then cannot use where the target does. After any structural
fix, re-test every pin you inherited and drop the ones that are inert. (Corollary, byte-measured:
adding a new pin to chase a register choice is usually worse — pinning y0 to $v0 to force the
target's early pos.y store went 9 → 12.)
A refuted hypothesis, recorded so it is not re-bought. SESSION-17 left an "untried, cheap lever":
maybe birth order follows first-use order (§31 regalloc RC-1/RC-2/RC-3 — declaration/use order drives
allocno_compare density). It does not, for this class. func_8014D820's use order already matched
the target (a1 first used at body-line 28, a0 at body-line 41 — a1 before a0, exactly the target's
birth order) while the birth order was inverted. Declaration order is likewise inert (moving ent/p to
the end of the decl block: byte-identical result). What moves the copy is its schedulable position,
and the launder is how you set it.
Where this applies. Any draft that is +1 ins with an unfilled load-delay nop and shows mirrored
sw $sN / move $sN,$aX prologue pairs. That signature is the tell — count the instructions first: a
draft one over with a nop the target fills is a placement bug, not a regalloc wall.
§67a — Run the symbol-set guard BEFORE you pay for a gate (tools/symcheck.py, Phase 29 SESSION-18)
SESSION-17 ended func_801463A0 with an open TODO: "a cheap guard worth building: diff a draft's
referenced symbol set against the target .s's %hi/%lo/jal set before gating." Built, negative-control
proven, and it belongs in front of every gate cycle.
Why nothing we already had can do this. All three of our fast oracles are blind to it by construction, not by accident:
| oracle | blindness |
|---|---|
match_one / masked_diff |
compares relocation-masked words; object-vs-.s mode is symbol-agnostic, so a reloc against the wrong symbol matches at that position |
rtu_match |
compiles but never links — an extern no symbol table defines is invisible to it |
| the whole-binary byte-gate | correct, but it tells you rejected, not why, and costs a full cycle |
The measurement that proves the point (func_8014D820, one data extern renamed to an invented
_s alias):
| correct draft | draft with an invented symbol | |
|---|---|---|
match_one |
14 mismatched | 14 mismatched — identical |
symcheck |
SYMS-OK 12 symbols agree |
SYMS-DIFF + names MISSING and INVENTED, exit 1 |
Two directions, both reported: MISSING (target references it, draft does not — the invented-alias signature, since the alias silently stands in for the real symbol) and INVENTED (draft references something the original never touched).
Where it goes: immediately before harvest_verify, and immediately after any transform that
rewrites declarations (sig_unify, canon_resident_calls, cast_call_sites, canon_sig_reconcile) —
those rewrite extern names, which is precisely how an alias gets invented. It is a necessary
condition, not a match oracle: SYMS-OK means "no link-level defect of this class", nothing more.
Finish on the byte-gate (G3/P9).
§66d-4 — "ILS converged" means converged FOR THAT WEIGHT PROFILE, not a floor (amends §66d-3; Phase 29 SESSION-18)
§66d-3 gave the read-the-series rule: a repeat means stop, a fall means continue. That rule is right
about when to stop the current run and was over-applied to mean the function is at its floor.
SESSION-17 used it to record all four remaining giants as "at their measured permuter floor (ILS
converged)". func_8014D820 then fell from 16 to 10 under nothing but weight changes.
The measurement (func_8014D820, each pass warm-started from the previous pass's best waypoint):
| pass | profile | series | best |
|---|---|---|---|
| 1 | --klass REGALLOC |
14, then ×9 unchanged | 14 |
| 2 | --klass SCHEDULE |
12, then ×7 unchanged | 12 |
| 3 | --klass cse |
10, then ×9 unchanged | 10 |
Three profiles, three identical shapes: one drop in cycle 1, then dead flat. The flatness is real —
continuing that run is waste, exactly as §66d-3 says. But it is a statement about the mutation
distribution, not about the function: permuter_weights.classify biases decomp-permuter's pass
selection toward one class's levers (regalloc.md RC-, sched.md S-, the CSE address-fold levers), so a
converged run means this profile's neighbourhood is exhausted around this seed — and the seed has
just changed, because the pass rewrote it.
The rule, corrected:
- A flat series ⇒ stop this run. (§66d-3, unchanged.)
- Before calling a floor, re-run with a different
--klassfrom the new best waypoint. Only after all three profiles come back flat from the same seed have you measured a floor. - Cheap, unattended, ~0 tokens — so it is the first thing to try on any "converged" giant, ahead of reader time and far ahead of Fable5.
And check every pass's diff for operand-order regressions. A random search cannot tell that one of
its own edits made a local position worse while the total improved. The cse pass here swapped
if (p == ent) → if (ent == p), which cost idx 110 (beq $s2,$s1 vs the target's beq $s1,$s2);
reverting just that operand order, keeping every other gain, took 10 → 9 for one compile. Same for
x >= k ↔ k <= x. These are free points and they are invisible to the scorer's total.
⚠️ QUALIFY THIS: profile diversity is worth TRYING, but it is NOT reliable. Everything above is
measured on ONE function. The same session ran the identical experiment on func_80140958 — base 56,
prior profile regalloc, fresh --klass cse pass — and got best=56, zero improvement across all 8
cycles. So the honest rule is "before declaring a floor, spend one cheap unattended pass per unused
profile", not "each profile is worth ~2 points". Score so far: 3-for-3 on func_8014D820,
0-for-1 on func_80140958. The test is cheap enough (~0 tokens, unattended CPU) that it stays worth
running — but budget it as a lottery ticket, not as expected yield, and do not plan a session around it.
MEASURED: a repeated profile does NOT yield again. REGALLOC round-2, warm-started from the close=9 seed (a seed it had never seen — SCHEDULE, cse and a reader fix had rewritten it since), came back flat for all 10 cycles, 0 gain. So the lever is profile diversity, not seed novelty: each of the three profiles is worth about one drop, and re-spending one buys nothing. Budget accordingly — three passes per giant, then stop.
That also makes rule 2 above operational: a floor is measured when all three profiles come back flat
from the SAME seed. For func_8014D820 at close=9, REGALLOC is flat; SCHEDULE and cse from that seed
are the remaining evidence needed before calling 9 a floor.
§66d-5 — residual_class's "structural ⇒ permuter CPU is waste" is WRONG for schedule permutations (measured, Phase 29 SESSION-18)
tools/residual_class.py documents its buckets (lines 55–58) as:
structural — local mutation CANNOT introduce it (a different access width, a flipped branch, a multi-instruction shape change). Spending permuter CPU here is waste; it wants an idiom.
Byte-measured counterexample. func_8014D820 was classed OPCODE-MIXED [structural], and later
WIDTH [structural] sig=WIDTH/lhu!=sh, at every waypoint — and the permuter moved it 16 → 14 → 12
→ 10 across three weight profiles. Six points of gain inside a bucket whose documented guidance is
"don't run the permuter." Meanwhile the guidance the bucket does give ("read it") was exhausted:
eleven source-shape attempts on the same window, all inert or worse.
Why the classifier is fooled. It reasons position-by-position. A pure schedule permutation
changes what lands at every index in the affected window — so a block whose instructions are all
present but reordered presents as "different operations, no single family" (OPCODE-MIXED) or as a
width flip (WIDTH), because index i now holds a sh where the target holds an lhu. The signature
of a reorder is indistinguishable, per-position, from a genuine shape change. The classifier's
SHIFT-DRIFT rule catches this only when ONE shift point re-aligns the tail; a permutation within a
window with matching endpoints re-aligns nowhere.
The corrected routing rule:
[permuter]⇒ search, as before.[structural]⇒ read first — the idiom is the cheap win when there is one.- But
structuralis NOT a permuter veto. If reading fails 2–3 times AND the instruction COUNTS match AND the same multiset of operations appears in the window in a different order, treat it as a schedule permutation and run the three profiles (§66d-4). Instruction-count equality is the tell: a genuine shape change usually changes the count; a permutation never does.
Cost of the old reading: this is a project-wide mis-route, not a one-off — every schedule-permuted residual that ever landed in OPCODE-MIXED/WIDTH was steered away from the one tool that moves it. Worth a targeted re-check of backlog entries carrying those two class tags with equal instruction counts.
§68 — A comment-only line halted the extern scan, and the skip label blamed the type cap (Phase 29 SESSION-18)
dedup_propagate.find_site() extracts a matched body as "preceding contiguous externs .. closing
brace". Its backward walk skipped blank lines but not comment-only lines. So a body written
like this — which is ordinary house style — silently lost its first extern group:
extern s16 D_80126940; /* <- DROPPED: everything above the comment */
extern s16 D_80126942;
/* ------------------------------------------------ */ /* <- the walk stops HERE */
extern s32 func_80012ABC(s32, s32, s32); /* <- only these were carried */
s32 func_80174CB0(s32 param_1, s32 param_2) { … }
The truncated body then failed compiles_standalone() on the now-undeclared symbols — and the caller
filed every such failure under "overlay-local TYPE (the real cap)".
That label is why the class went unfixed for ~4 phases. The Phase-21 backlog already carried the right answer — "a macro-extern-injection (or canonical-callee-sig embed) frees them ×134 (~+0.3%)" — but every session after that read the skip line, saw "the real cap", and treated it as the known-hard overlay-local-type wall. A diagnostic that discards its cause is invisible work (R32); one that asserts the WRONG cause actively redirects everyone who reads it.
Two fixes, both small:
compiles_standalone()returns(ok, stderr); the caller classifies by actual cc1 output —missing file-scope extern (CARRY-FIXABLE): <names>vsoverlay-local TYPE (the real cap).- the backward walk treats comment-only lines like blanks, and the emitted body filters them out so
make_macronever meets a//.
Result: func_80174CB0 went from "not self-contained (local types)" to a 138-member plan.
⚠️ "CARRY-FIXABLE" is NOT one cause — measured. After fix (2), the 7 skipped functions in
ov_SC01_077 (0x8016A73C 0x80155800 0x80167540 0x801535F4 0x8016F0AC 0x80142B2C 0x8014FE60) are
still blocked, so at least one more sub-class exists: their missing symbols are not adjacent to the
def at all (e.g. D_801152A8 appears 26× in engine_core.h but only ever INSIDE DEFINE_func_*
macro bodies, never at file scope). Those need the genuine carry mechanism — collect the referenced
file-scope decls wherever they live (cdecl.scope() is the right oracle, R33) and emit them inside
the lifted macro. Do not assume fix (2) cleared the queue: re-run
--auto-from <ov> --check-only and read the per-class counts.
The general lesson: when a tool refuses, make it print what the compiler said, not what the
author guessed the compiler meant. Every wrong-label bug in this project (§53's carve, the 193
listCdBuffer slices, masked_diff's 150 closeness lies, this one) had the true cause available and
threw it away.
§69 — How to attack a behemoth: map it, don't draft it (Phase 29 SESSION-18, func_80183814, 5,122 ins)
First attempt ever on the game's largest unmatched function. It did not match, and the useful
output was never going to be a match — it was the map. Every claim below is byte-verified against the
target .s.
What it is. Not a straight-line giant: an actor state machine. ~48-ins preamble → switch (*(u16 *)(a0 + 0x34)) over 21 cases via jtbl_801F4CE4 (sltiu $v0,$v1,0x15) → 19-ins shared
tail. 359 jals to only 37 distinct callees (verified). A cutscene/boss-script driver.
THE FINDING — it is not 5,122 unique instructions, it is a few templates repeated:
| sub-shape | instances | notes |
|---|---|---|
| "spawn-effect" packet | 35 | ~60% of cases 11–20 (~1,400 ins). Body is DEFINE_func_80142454's s16 st[10] packet verbatim. Crack one → 34 free. |
| "wait/countdown" | 7 | if (a0->0x1C < 7) …; if (--a0->0x1C != 0) break; — already reproduced at 0 skeleton diffs (cases 2/5/7) |
| "6-slot HUD/text" | 12 | cases 0/1/3/4; and case 0 ≈ case 3, case 1 ≈ case 4 (identical call histograms, 363/365 and 280/281 ins) — near-free twins |
Coupling is light: only 3 cross-jump edges (cases 5, 10, 12 all j into case 14's merged tail), |
||
| so those four must be written together and everything else is independent. |
TWO LAWS FOR GIANTS, both measured here:
- Register pressure is GLOBAL, so a partial draft can never show a matching prefix. A truncated body (15 of 21 cases stubbed) was allocated 10 callee-saved regs instead of 8, and from case-0 idx 278 gcc CSE'd address constants into callee regs that the real function cannot afford and rematerialises. ⇒ Do not aim for a contiguous matching prefix on a giant. Write ALL cases coarsely first to restore the true pressure, then refine. A "first N instructions match" partition is structurally unavailable until the whole body exists.
match_one's global number is meaningless on a partial giant (here:mine=666 target=5122,SIZE-MISMATCH). Measure region-aligned instead — per case, registers masked, relocs masked. Tool:.run/giants/s18_regions_comparator.py(reusable for any giant). It gave PROLOGUE+PREAMBLE 0 diffs, cases 2/5/7 0 diffs, 139 real skeleton diffs over 638 decoded ins (78% skeleton-identical). Caveat that must travel with the metric: it masks register NUMBERS andj/jaltargets, so it proves sequence/opcodes/constants/offsets — structural correctness — and is never a closeness score. Finish on the whole-binary gate (G3/P9).
Two idioms cracked in passing:
D_8018E034[t + K]folds the+Kinto thelw(lw a1,%lo(base+4K)). The target keeps a runtimeaddiu, so the source needs a separate index variable and a separateidx = t + K;statement. Worth 246 → 83 skeleton diffs on case 0 by itself.- A
u16field read as(s16)*(u16 *)p(lhu+sll 16+sra 16), and/455via magic0x90090091with thehi+nadd-back form.
Verdict: tractable, but a ~2,000-line WRITE, not a hard puzzle. No scheduler wall, no unsteerable
regalloc, no exotic idiom — every construct decoded on the first or second try. The blockers are
volume plus the two couplings above. Recipe for the next attempt: all 21 cases coarsely → fix the
frame/8-reg allocation → crack ONE spawn-effect instance and paste it 35× → the twin pairs (0↔3, 1↔4)
→ the 7 wait instances (already solved, reuse verbatim) → cases 17/18/20 (560/484/525 ins) last.
Banking will need §27-step-2 / §28 plumbing recovery: func_80178970/func_80178D18/func_800599B8/
func_8017D8A4 are all called at arities that disagree with their canonical externs.
§70 — The giv-init base register: walk the PARAMETER, not a copy of it (Phase 29 SESSION-18, func_801777BC)
func_801777BC (59 ins, reach 138) came down to one instruction: mine emitted
addiu $t0,$t1,0xC, the target addiu $t0,$a0,0xC. A general-induction-variable based on the wrong
register. This is the REGALLOC-PERM/$tN>$a0 signature and no pin or permuter pass reaches it —
it is decided in loop.c before allocation.
Why the natural C can NEVER produce the target (read out of tools/reference/gcc-2.7.2/):
with the obvious form —
u32 *q = (u32 *)a0; /* … */ q += 5; /* … */ return q;
cse.c:make_regs_eqv makes q the canonical register of the equivalence class, because q
out-lives a0; and loop.c:update_reg_last_use then refuses to extend a0's last-use, because the
giv-init insn's UID is ≥ max_uid_for_loop. So every giv gets based on the copy, forever.
The fix — make the loop pointer the parameter itself:
a0 = (void *)((u32 *)a0 + 5); /* … */ return a0;
Now loop.c:record_initial finds the biv's initial value is the hard register (reg:SI 4),
valid_initial_value_p accepts it (precondition: no calls in the function), and
emit_iv_add_mult bases the giv on $a0 — which yields both addu $t1,$a0,$zero (index 0) and
addiu $t0,$a0,0xC (index 16) for free.
Look for it when: a pointer-walking loop returns the walked pointer, and the only residual is a
giv/base register differing between a $t/$s temp and an argument register.
Two supporting levers from the same function:
- Hoist
i = 0;OUTSIDE the guard. Left inside,reorgsteals thelui $t4constant into theblezdelay slot instead of the target'smove $11,$0. - Reuse one variable sequentially for two constants (
mask = 0xffffff; t = … & mask; mask = 0x3000000;) so both share$v1instead of pinning two registers simultaneously.
The meta-point, and it is a tier shift. Phase 23 established that reading the gcc source to break a class was the Fable5 tier — the expensive wall-breaker. This was done by an ordinary Opus 5 drafting agent, unprompted, as part of finishing one function (270k tokens). The §31 map plus a local copy of the real compiler source appears to have moved compiler-internals reasoning down a tier. Do not read this as "Fable5 is unnecessary" on one data point — but DO give routine drafting agents the gcc source path and expect them to use it.
§71 — Before mapping a giant, look for an already-matched SIBLING beside it (Phase 29 SESSION-18, func_8017D960, 3,338 ins)
Behemoth #2. Result: 3,334 of 3,338 instructions, 98.8% register-masked-identical, 88.3% byte-aligned, byte-exact prologue AND epilogue, exact 0x320 frame, the same 10 saved registers at the same offsets, and the identical ~110 stack slots — on a 3,338-instruction function, in one session. Not a match (G3), but an order of magnitude closer than behemoth #1.
THE LEVER, and it is embarrassingly cheap. func_8017D960 is the lit variant of
func_8017CA80 — the 952-instruction renderer immediately above it in the same source file
(ov_SC03_090_jr_8017CA80.c, the file is named after it), already matched. Diffing against that
sibling handed over ~90% of the C for free and produced a 3,334/3,338-instruction draft on the FIRST
compile. No mapping phase was needed to get there.
Do this first on every giant: check whether an already-matched function elsewhere is the same ROUTINE. It costs one
grepand can replace days of structural analysis.
HOW to run the check — fingerprint by CALLEE SET, not by h_seq or adjacency (corrected same
session). Two wrong versions were tried first:
- Adjacency found
func_8017CA80only because it happened to sit next door. Running it over the six remaining behemoths: 0 hits — their neighbours are all 2–128-ins helpers. h_seqfleet-wide also returned 0 hits, and is structurally incapable of finding these:h_seqis a mnemonic skeleton, so a 952-ins base renderer and its 1,511- and 3,338-ins variants never share one. Worse, it produces FALSE families —func_8017F510andfunc_8017F5B4share anh_seqyet call completely different functions (jal Xandjal Yboth normalise tojal).- The CALLEE SET works.
func_8017CA80(952, matched),func_8017D960(3,338) andfunc_8017F510(1,511) all call exactly{func_800491EC, func_80052E38, func_800547D8}and nothing else — three variants of one renderer, found in one grep:grep -oE 'jal[[:space:]]+[A-Za-z_]\w*' <fn>.s | awk '{print $2}' | sort -uA small, distinctive callee set is a far better routine-identity fingerprint than any hash we have, because it survives the size differences that variants are made of.
§69 IS PARTLY REFUTED — corrected here, do not follow its headline blindly.
| §69 claim | verdict on a non-dispatcher |
|---|---|
| Law 1 — write the whole body coarsely first; a partial draft gets the wrong callee-saved set, so no matching prefix exists | CONFIRMED and decisive. Doing this got the exact frame + saved-reg set on the first compile. |
Law 2 — match_one's global number is meaningless; measure region-aligned |
CONFIRMED and essential (its 1806 is noise; a 4-instruction length drift destroys positional comparison) — but the tool did not transfer. s18_regions_comparator.py is per-switch-case. |
| Headline — "the deliverable is the map, not a match" | REFUTED for this shape. §69 was derived from a 359-call dispatcher with no sibling. Here mapping was not the lever at all; the sibling was. |
Tooling supersession: .run/giants/b2_mask.py + b2_full.py are a shape-agnostic word-level
masked sequence aligner (structural number and byte number, no switch assumption). They replace
s18_regions_comparator.py — use them for any giant.
What it is (useful for the family): a 3-source volumetric-light mesh renderer. Same skeleton as
the matched sibling — 3-call prologue, Part[] outer loop (stride 0x14, 8-corner AABB rtpt/rtps
- screen-bbox reject),
Prim[]inner loop (stride 0xC,rtpt/nclip/stopz), OT insertion. The added work is three axis-aligned light boxes ({s32 enable; u16 cx,cy,cz; s32 range}, stride 0x1C): per vertex, a separable per-axis linear falloff over the outer 0x80 of the range, axes visited x, z, y — that ordering is load-bearing, summed and clamped to 0x80 into a grey gouraud colour. Lit →POLY_GT3/GT4; unlit →POLY_FT3/FT4withrgbc = tp[0] & 0xFF000000.
Two shape facts worth generalising: only 2 back edges in 3,338 instructions (it is two nested loops, not a maze); and the 8-byte stack stride that looks like an exotic aggregate is just gcc's spill-slot granularity — every non-array local is spilled. Do not invent a struct to explain it.
Residual (4 ins short, ~40 divergent): the only genuine structural error is gcc fold
reassociating the colour OR-chain; a rewrite fixes structure to 99.1% but cascades a live range and
drops byte-alignment to 76% (s18_func_8017D960_b2_rgbchain.c — an unresolved trade, do not re-buy
blindly). The rest is register naming. Family: func_8017CD9C (ov_SC03_102) and func_8017E778
(ov_SC03_091) are the same 3,338-instruction function with only the 3 light-descriptor symbols
changed — one crack templates ×3.
⚠️ A PROFILING ERROR OF MINE, recorded (R14/R35): I briefed this agent that the function had no
switch, from grepping sltiu jump-table bounds. Wrong — the switch is compiled as a comparison
tree (23 slti). A jump-table grep is not a switch detector. The agent caught it; a less careful
one would have inherited my false premise.
§72 — A register __asm__ pin is a PREFERENCE, not a reservation (Phase 29 SESSION-18, func_8017F510)
Behemoth #3, and the best behemoth result so far: 1,511 of 1,511 instructions (exact length),
frame 0x258 exact, byte-exact prologue AND epilogue, identical sp-slot set, 99.5%
register-masked structural / 93.3% byte-aligned, SYMS-OK, 97 divergent. Not a match (G3) — but ~93
of the 97 trace to a single register decision (below). §71's callee-set lever delivered: diffing the
two references gave ~90% of the C and a first compile of 1528/1511.
⚠️ THE CORRECTNESS FINDING — this qualifies §17
Pinning a local with register s32 amb __asm__("$30") produced a seductive 1,511 ins / 98.9% —
and was a MISCOMPILE. gcc-2.7.2 also allocated $s8 to an unrelated live value (hi_z): one
hard register holding two live values at once.
A local
register … __asm__("$N")declaration is a hint to the allocator, NOT a reservation. gcc-2.7.2 will still hand that hard register to another pseudo. Never ship a pin without inspecting the pinned register's defs in the output.
§17 (pins + a scheduling barrier) remains valid — it is how func_8012B8E4 and others were cracked —
but it now carries this obligation. Anything already BANKED is safe by construction (the
whole-binary byte-gate would have rejected a miscompile); the exposure is un-gated drafts.
ACTION: .run/giants/s18_func_8017D960_b2.c (behemoth #2, 3,334/3,338) carries FIVE pins
($25 $17 $19 $20 $21) and must be re-checked before anyone builds on it.
The honest fix was source-level and cheap
The +17-instruction drift was live-range stretching: reusing w/wz (which already carry
prim->w1/w2 and part->zz) for the vertex-word reads stretched two live ranges enough to spill
amb, costing 7 lw+nop pairs. Dedicated temps for the vertex loads (vw/vzw) →
1528→1511 ins, 88%→99.6% structural, no pin.
Generalisable: when a draft is long by a handful of
lw+noppairs, look for a REUSED local whose live range now spans a region it did not before. A fresh temp is cheaper than a pin and cannot miscompile.
Giv record order (the §70 family)
part->prim must be read BEFORE part->nprim. loop.c:combine_givs walks bl->giv in reverse
record order, so the last-recorded giv becomes the combined base: prim-then-nprim yields base
part+0xC (the target); the other order yields part+0x10.
What is left, and what is byte-recorded as SPENT
Residual 97 = 2 ins (box-build emission/allocation transposition — source order sets both, via
sched.c UID ties and global.c allocno ties; needs a third form) + 3×2 ins (unlit-tail OT-tag
scheduling) + ~93 ins of pure register naming off ONE seed: c3 (4th quad vertex colour) is
$a2 in the target and $a3 in the draft; tp is one pseudo across all four emit tails and takes
the other of the pair, renaming the whole block. Fix c3 → $a2 and ~93 should fall together.
Do not re-buy: decl-order permutations (gcc numbers pseudos by first use, not declaration) ·
block-scoping the emit temps (1509 ins) · splitting or inlining tp · reusing f0 as c3 · vertex
x/z/y orderings · decomp-permuter, 4,724 candidates at -j 10, base 97, ZERO improvement — a §3
hard tail outside the C-randomisation search space. The remaining move is to reason the c3
allocation out of global.c, not to spend more compute.
Tooling: .run/giants/b3_align.py (shape-agnostic structural+byte aligner) and b3_pos.py
(positional diff for equal-length drafts) supersede the b2_* set.
§73 — A def-side self-decl conflict has TWO axes: RETURN (fleet widen, T2) and PARAMS (casts at each use, T0). Diagnose which before paying for the expensive one (Phase 29 SESSION-19, func_8014F3E8 + func_8014D4C0)
§30 #2 established the return-type escape: when the byte-true def must return s32 but the fleet
canon declares extern void func_X(...) inside a DEFINE_func_* macro, widen the macro's extern
— byte-neutral wherever the caller discards the return. What the recipe did not say, because the
function it was derived from had a single scalar param, is that this is only one of two independent
axes of the same conflicting types for 'func_X' error.
The batch that separated them. Both functions were byte-correct, real-TU verified, and blocked on
the identical cc1 message. One fleet-wide sed -E 's/extern void (func_8014F3E8|func_8014D4C0)/extern s32 \1/g' over src/** (16 decls in engine_core.h + 5,079 across 3,459 overlay .c files) was
proven byte-neutral in isolation — then the gate banked func_8014F3E8 (32 ins) and still refused
func_8014D4C0 with the same error. The residue was the parameter half: canon
(s32, void *, void *) vs the draft's (s32, u16 *, u16 *).
| axis | symptom | fix | blast radius |
|---|---|---|---|
| return | def needs s32, canon says void; a void def DCEs the value-set (delay slot → nop, or a live local dies → frame shrinks) |
widen extern void→extern s32 in every spelling together (macro + each overlay's carried decl layer) |
T2 fleet-shared ⇒ R22 mandatory |
| params | def wants a narrower/typed pointer than the canon's void * (or vice-versa) |
keep the canonical param types in the signature and cast at each use — ((u16 *)a1)[1], *(u16 *)a1 |
T0 draft-only — no fleet edit at all |
The param fix is §17a-1 applied to the definition's own signature (the same move func_80174CB0
needed in SESSION-18, where the wall was the fn's own decl rather than a callee's). Casts are
compile-time; the emitted code is unchanged; nothing outside the draft is touched.
Practice:
- Read the cc1 error's two halves before acting.
conflicting typesdoes not tell you which axis; diff the canon decl against the byte-true signature term by term (return, then each param). - If only the params disagree, do NOT reach for the fleet widen. It costs an R22 cycle and still fails — exactly what happened here (the widen was independently justified by the other function).
- If both disagree, fix the params in the draft first — then the widen batch only has to carry the return axis, and the fleet edit stays as narrow as possible.
- Batch the return-axis widens (one R22 for N functions, never one per function, §63) and isolate the shared edit from the drafts: build 1–2 representative binaries after the widen ALONE before splicing anything. Here that step is what made the single failure instantly attributable to the param axis rather than to the widen — one cheap build replaced a diagnosis.
- Pick the isolation probe deliberately: an overlay that instantiates the return-casting macros
(
((s32 (*)(...))func_X)(...), §17a-1) is the only place a decl's return type could plausibly touch codegen.ov_SC01_000served that role forfunc_8014F3E8.
§74 — Auditing a pinned draft: the §72 hazard is CALLER-SAVED pins spanning a call, and only the disassembly can tell you (Phase 29 SESSION-19, func_8017D960 b2, 5 pins)
§72 established that a register __asm__("$N") pin is a preference, not a reservation — gcc will
happily put another value in $N while your variable is notionally live. The banked corpus is safe
from this by construction (the whole-binary byte-gate rejects any miscompile), but un-gated drafts
are not, and a behemoth draft can carry pins for days before it ever reaches a gate. This is the
cheap audit.
Two distinct failure modes, only one of which is real most of the time:
- CALLER-SAVED PIN SPANNING A CALL — the one that silently corrupts.
$25($t9) and the$t0–$t9range are call-clobbered. gcc-2.7.2 does not save/restore an explicit-register variable across a call, so if the pinned variable's live range crosses ajal, the value is destroyed with no diagnostic. Callee-saved pins ($16–$23=$s0–$s7) are immune — the prologue/epilogue save/restore covers them. - SCRATCH REUSE OF THE PINNED REGISTER — usually benign. gcc will use
$Nas a temporary for an unrelated value before the pinned variable's own value lands there. Observed here:lui s4,..; lw s4,0(s4); addiu s4,s4,-128—$20(pinnedr1lo) carried the raw global for two insns, andaddu t9,s4,zerocopied that value out to$25(pinnedr1) on the way. Self-consistent; nothing live was clobbered.
The audit (no recompile needed if a match_one object survives — .run/match/<fn>.<pid>/<fn>/t.o;
confirm it is the draft you think it is with cmp t.c <draft>):
mipsel-linux-gnu-objdump -d <t.o> > dis.txt
# (a) does any call exist after the first pin write?
grep -nE '\bjalr?\b' dis.txt # compare addresses against the pin-write addresses
# (b) how many times is each pinned reg WRITTEN? (first operand, excluding sw/branch/jal/mult-class)
Expect writes == (assignments in the C) + 1 epilogue lw for each callee-saved pin. Anything above
that is scratch reuse — read those sites before assuming they are benign.
Worked verdict (.run/giants/s18_func_8017D960_b2.c, pins $25 $17 $19 $20 $21): 3 jals total,
all at 0x2c–0x50, and the first pin write is at 0x58 — no call after the pins are
established, so the caller-saved $25 pin never spans one and mode (1) does not arise. Modes (2)
sightings on $20/$21/$17 are the benign scratch pattern above. Draft is safe to keep building
on. (The C corroborates: everything after the three prologue calls is a macro — gte_*, BOXTEST,
ATTEN, CLAMP80 — not a function call. Verify that with a token census, not by eye: a 636-line
behemoth hides a jal easily.)
Standing rule of thumb: prefer a callee-saved register for any pin whose variable outlives a
call; if the target's register really is caller-saved and the value really does span a jal, the pin
cannot express it — that is a genuine wall verdict, not a drafting slip.
§75 — A propagation cap is usually a MINORITY-SPELLING SOURCE OVERLAY, not a wall: census the carried extern before believing the exclusion message (Phase 29 SESSION-19, func_8014F3E8 ×4 → ×138)
dedup_propagate authors the shared macro from ONE source overlay and carries that overlay's
file-scope extern lines into the macro verbatim. Those externs are then instantiated in every
member TU. So whatever spelling the source overlay happens to use becomes the fleet's spelling —
and if the source is an outlier, the macro conflicts everywhere else and the group silently caps its
reach at the outlier's island. The exclusion message calls that byte-diverge / irreconcilable,
which is the wrong cause and reads like a wall.
The tell: the bytes cannot be diverging. Members are selected by h_exact — the SHA1 of raw
instruction bytes, relocs included. If a group has N members it is because all N are byte-identical.
So an exclusion is a compile conflict, never a byte one. Any message saying otherwise is lying
(cf. §68's mislabel — same tool, same defect family).
The census that names the real cause (30 seconds, read-only). Take the symbol the carried extern declares and count every spelling of it across the tree:
grep -rh '<sym>' src --include=*.c --include=*.h \
| grep -E 'extern|^\s*(void|s32|int|u32)\s+<sym>\s*\(' \
| sed -E 's/^\s+//; s/\s+/ /g; s/ \\$//' | sort | uniq -c | sort -rn
Worked case — func_8014F468, carried by the func_8014F3E8 body:
| spelling | count | |
|---|---|---|
extern s32 func_8014F468(void); |
1,710 | fleet canon |
s32 func_8014F468(void) (def) |
134 | fleet canon |
extern void func_8014F468(void); |
20 | the outlier |
void func_8014F468(void) (def) |
4 | the outlier — and all 4 are ov_SC07_{006,007,010,011} |
The source overlay I banked from was one of the four. The macro inherited extern void, and the 134
overlays that define the symbol s32 rejected it — propagation landed on exactly the 4-overlay
island. Nothing about the code was hard.
The fix is NORMALIZATION, not a reconciliation engine. Flip the minority spelling to the fleet
canon (here 24 occurrences: 4 definitions + 19 overlay externs + 1 line in the freshly-authored
macro), byte-gate the affected binaries, then extend the group. func_8014F468 is a pure inline-asm
$sp-switch trampoline, so the return type carries no C-level value flow — and 134 overlays already
proved s32 is byte-correct for the identical function. All 4 flipped binaries stayed
byte-identical.
Why a reconciliation engine is the WRONG shape here. One macro text is instantiated in 138 TUs;
it cannot carry a per-overlay extern. If the members genuinely disagree, no per-member rewrite of a
shared body can satisfy them all — you must make them agree first. (reconcile_tu.fix() already
does "TU wins + cast at use" for DATA decls and deliberately skips d.kind == 'func'; extending it
would not have helped, because the conflicting text lives in the shared header, not in a draft.)
Practice:
- Prefer a majority-spelling source overlay. Before propagating from overlay X, census X's decls of every symbol the body carries. A minority source caps the group by construction.
- Always pass
--recoveron a targeted--addrrun. Without it, the first culprit overlay triggers the historical all-or-nothing drop and the group banks ×0 instead of ×(N−1). This session lost a whole group to that flag before re-running. - After normalizing, the body is already a macro — so the follow-up is
dedup_extend(--binaries <the excluded set>), notdedup_propagate --addr, which can only author from an inline def and will report "no source overlay has it matched". - Re-check any group historically stuck at a small reach for the same cause —
func_80174CB0(×3 since SESSION-18) was carrying the identical class.
§75a — The exclusion classes, enumerated with named causes (Phase 29 SESSION-19, the 134-binary dedup_extend sweep)
After the §75 normalization, one dedup_extend --binaries <134> sweep banked 134/400 planned and
the two residual groups failed in every binary — with harvest_verify's classifier naming a
different cause for each. That enumeration is the useful artifact: "propagation-capped" is not one
class, it is at least three, and only the first is cheap.
| class | cc1/ld says | example | remedy | cost |
|---|---|---|---|---|
| A — minority spelling (§75) | conflicting types for <callee> where one spelling dominates the census |
func_8014F468: 1,710 s32 vs 4 void defs |
normalize the minority, byte-gate, dedup_extend |
cheap, byte-neutral |
| B — genuine arity/type split | same message, but the census shows two real populations | func_8012F14C: 1,944 (s32) vs 968 (s32,s32,s32) |
NOT a typo — two live call conventions. The §29 loose-typing wall. A K&R () in the macro may be compatible with both (compatible(): () first + prototype second is ACCEPTED when no param default-promotes, and s32 does not) — but it is order-dependent, so it only works if the macro's decl precedes the TU's. Measure before moving. |
unknown — probe first |
| C — missing carried extern | undefined reference to '<sym>' (a LINK error, not a type error) |
func_80165CA0: undefined reference to 'SHB' |
the SESSION-18 CARRY-FIXABLE class — the body references a file-scope decl the extraction did not carry | cheap once carried |
The discriminator is one grep, and it decides the whole remedy: census every spelling of the symbol cc1 named. A lopsided count (≥95/5) is class A — normalize. Two substantial populations is class B — do not normalize on a guess; an arity change is not byte-neutral by inspection (§29's narrow-param wall is exactly this). A link error is class C and has nothing to do with types.
Do not generalize from one member's error. The same blocked function reported different callees
in different overlays (func_80012ABC at ov_SC01_000, func_8012F14C at ov_SC01_001) — so one
sample names one blocker, not the blocker set. Collect the classifier's line across the whole sweep
before scoping the fix.
§75b — A body's preamble can carry #defines, not just externs; extraction lifts only the externs (Phase 29 SESSION-19, func_80165CA0 ×3 → fleet)
dedup_propagate's extract_unit walks BACKWARD from a definition collecting contiguous
extern …; lines into the macro. It does not collect #defines — so a body matched with a
file-scope macro in its preamble loses that macro when it is lifted into engine_core.h, and the
shared body then compiles only where the source overlay's #define happens to be in scope above
the splice point. This is the dedup_propagate counterpart of the family_remap._carry_macros
gap Phase 27 closed.
The signature of this class is a LINK error, not a type error. undefined reference to 'SHB' —
because an unexpanded SHB(x) is parsed as a call to an undeclared function (implicit int), which
compiles cleanly and dies at link. Anything reporting conflicting types is a different class
(§75/§75a). One grep tells them apart: is the name a #define or a symbol?
grep -rh '\b<name>\b' src --include=*.c --include=*.h | sort | uniq -c | sort -rn | head
If the top hit is #define <name>(x) …, it is this class.
Worked case. func_80165CA0 sat at ×3 for a phase. Its preamble in the source overlay is:
extern s32 D_8011D030;
extern s32 D_80126728;
#define SHB(x) __asm__ __volatile__("" : "=r"(x) : "0"(x)) /* <- NOT carried */
DEFINE_func_80165CA0()
The two externs were lifted; the #define between them was not. The other 132 overlays do define
SHB — about 300 lines further down (ov_SC01_001: stub @4462, #define @4781), i.e. below the
splice point. Pure ordering. And it explains the membership exactly: the 3 members are precisely
the 3 files carrying the __volatile__ spelling of SHB — that define is the function's own
preamble, still sitting above its instantiation.
Fix — the shared header OWNS the macro, under a distinct name:
#ifndef ENGINE_SHB
#define ENGINE_SHB(x) __asm__ __volatile__("" : "=r"(x) : "0"(x))
#endif
and rewrite the body's uses. Do not reuse the original name: the overlays define SHB
themselves in two different spellings (volatile in 3 files, non-volatile in 132), and a shared
#define SHB with a different replacement list is a hard redefinition error. A distinct name cannot
collide with either. Pick the spelling the currently-banked members actually compile with (here
volatile), not the one a stale body comment claims — then let the byte-gate confirm on those members
before extending.
Generalises: any preamble construct that is not an extern — #define, a file-scope typedef,
a static helper — is silently dropped by extraction and will cap the body's reach at whatever
subset happens to supply it. When a group's reach is stuck at a suspiciously small number, diff the
source overlay's preamble against what the macro actually carries.
§75c — Class-B's remedy is the FULL §17a-1 PAIR (decl and call-site cast); a decl-only fix moves the error and looks like a new wall (Phase 29 SESSION-19, func_8012F14C)
§75a's class B is a genuine arity split — the same callee declared with two different arities across
the fleet (func_8012F14C: 1,944 (s32) vs 968 (s32,s32,s32)). The obvious move is to make the
shared macro's carried decl compatible with both, using the no-prototype () form that
cdecl.compatible() measures as accepted in either order when no parameter default-promotes. That
is half a fix, and the half that does not work.
What actually happens. () really does dissolve the declaration conflict — the byte-gate's
classifier moved the failure from PLUMBING to CC1-FAIL, which is the tell that the first wall fell
and a second appeared. Reading real cc1 stderr (hand-splice the macro into one member, build that
object; do not trust a bare make … Error 33):
ov_SC01_001_jr_801734BC.c:2616: too many arguments to function `func_8012F14C'
C's composite-type rule: after void f(s32); then void f();, the composite is still void f(s32) — the earlier prototype wins. So a 3-argument call is a hard error no declaration spelling
can rescue.
The remedy is the pair §17a-1 always specified (and what cast_call_sites.py implements):
extern void func_8012F14C(); /* conflict-free in either order */
((void (*)(s32, s32, s32))func_8012F14C)((s32)&mtx, (s32)&vec, (s32)&out); /* call bypasses the prototype */
gcc-2.7.2 folds a cast of a known function symbol back to a direct jal, so the body's bytes are
unchanged (byte-gated on all 3 existing members: 7ca772be / b3b95547 / 9885af74).
Rule: for class B, change the decl and the call together. A decl-only change is not a smaller version of the fix — it relocates the diagnostic, and a session reading only the failure class will record a fresh wall where there is none.
§76 — The allocno CLASS (local vs global) is the dominant regalloc lever, and C reaches it ONLY through declaration scope and variable reuse (Phase 29 SESSION-19, behemoth #3 func_8017F510 1,511 ins, 97 → MATCH, pin-free)
Behemoth #3 sat at 97/1511 mismatched after a full pass at a lower reasoning tier that had already localized every mismatch, exhausted ~10 C-shape levers, and run 3,663 permuter candidates at base 97 with zero improvement. The residual was not in the C-randomisation search space — and the reason is the reusable lesson: four of the five decisions were local-vs-global allocno CLASS choices, which C expresses only through declaration scope and variable reuse. Statement order, expression shape, register pins and random search cannot reach them.
The mechanism, with citations
local-alloc.c:472admits a pseudo to LOCAL allocation only whenREG_BASIC_BLOCK >= 0 && REG_N_DEATHS == 1. A function-scope variable used in four emit arms has 4 deaths ⇒ it becomes a global allocno ⇒combine_regs(local-alloc.c:1825) can no longer tie its producer chain into it (global-alloc has no coalescing), so an in-placesra/sll/adduchain on that register is unreachable. Declaring the same variable inside each arm makes it four 1-death local pseudos and the chain ties.global.c:668-671re-marks every pseudo that local-alloc already placed (reg_renumber[i] >= 0) as a hard register for global-alloc's conflict scan. So a local pseudo's placement removes a register from the global pool, and that shows up as a hard-reg number in the;; N conflicts:tail of the.gregdump. In this function a globalotpin$a2pushed a per-tail constant into$a0, which made the target'sotp = $a0structurally impossible — not merely lower-priority. Read the;; N conflicts:hard-reg tail before reasoning about priorities.global.c:594 allocno_compareranks byfloor_log2(n_refs)*n_refs/live_length, with refs counted × loop depth. Reusing an existing variable as a second temp (§45-A / RC-14 MERGE) raises its ref count — herecb27 → 39 refs — and flips the grant ORDER of a 3-way colouring (cb→$a1, c3→$a2, tp→$a3instead oftp→$a2, cb→$a1, c3→$a3). Recomputing that formula off the.lregdump reproduces the;; N regs to allocate:order exactly, so you can PREDICT a flip instead of searching for one.
The attribution primitive (use this before calling anything a scheduling residual)
Compile the draft with -fno-schedule-insns and again with -fno-schedule-insns2. If a
transposed instruction pair keeps source order under both, the ordering was fixed at RTL
expansion, not by either scheduler — so the lever is source statement order, and no amount of
sched.c reasoning applies. That test converted this function's residual A from "needs a third form of
a scheduling tie-break" into a one-line source swap.
Two diagnosis traps this function proved
- A "scheduling" diff can be a register grant in disguise. Residual B (an OT tag materialised
before a two-insn constant) was never a sched.c choice: once the unlit colour temp was granted
$a1, the0xFFFFFFconstant — also wanting$a1— could not be materialised until that temp died, so sched2 slid it below the tag by itself. Fixing the allocation fixed the "schedule". - A "single seed cascade" model can be wrong even when the cascade is real. The prior dossier
attributed ~93 of the 97 to one seed (
c3must win$a2).c3turned out to have no lever of its own — it moves only as a side-effect ofcbout-rankingtp. The cascade was real; the seed was not the thing you can steer. Steer the ranked variable, not the symptom variable.
Practice
- When a residual survives a full permuter sweep at its measured base, stop searching and ask which allocno CLASS decisions the C is failing to express. Search cannot reach a class choice.
- Sweep declaration scope at three granularities — function / innermost-
if/ per-ARM — for every temp in the divergent block. Per-arm is a distinct granularity from per-case and is frequently the one that matters (here: per-case = neutral, per-arm = the crack). - Then sweep variable REUSE (merge two temps into one) to move
allocno_comparepriority. - Keep the measured result of every lever; behemoth #3's session produced a ~50-row do-not-re-buy
table (
.run/giants/s19_f510_report.md) that is worth more than the match itself.
§77 — Every extraction tool carries a NARROW hard-coded preamble set; anything outside it silently caps the body's reach. Diff the preamble before you gate. (Phase 29 SESSION-19 — three variants in one session, two different tools)
§75b found that dedup_propagate's extract_unit lifts extern lines but not file-scope
#defines, and predicted the generalisation: "any preamble construct that is not an extern —
#define, a file-scope typedef, a static helper — is silently dropped." That prediction was
confirmed three more times the same day, in a second tool, while templating behemoth #3's crack
onto its sibling with family_remap:
| # | construct dropped | tool | how it surfaced |
|---|---|---|---|
| 1 | file-scope #define (SHB) |
dedup_propagate |
undefined reference to 'SHB' — a LINK error (§75b) |
| 2 | multi-line typedef struct {…} T; |
family_remap |
'PolyGT4' undeclared + a cascade of parse error before ')' |
| 3 | file-scope extern block sitting above a #define block |
family_remap |
'D_801B79E8' undeclared |
| 4 | the exemplar's own #include lines |
family_remap |
'PolyFT4' undeclared (it lives in engine_types.h) |
| 5 | a static inline helper |
family_remap |
LENGTH-DRIFT/-56 — no compile error at all, just a short body (§82's inlined-helper case, func_8017C730 @ ov_SC03_013) |
Variant 5 is the nastiest, and this section PREDICTED it before measuring it (the closing line
below named static helpers). It is the only variant that produces no diagnostic whatsoever — the
draft compiles clean and is simply ~56 instructions short, which reads as a codegen residual rather
than a missing construct. If a mechanically-remapped sibling shows a NEGATIVE length drift and no
compile error, look for an uncarried static/inline helper before touching a single lever.
⚠️ COROLLARY (measured, and it cost a bank): carry the MINIMAL CLOSURE, not the whole file
Fixing variant 5 by carrying the exemplar's entire region file as preamble produced a clean
standalone match_one MATCH — and then failed the whole-binary gate on PLUMBING, because
2,993 lines of unrelated declarations collide wholesale in the target TU. Over-carrying does not
"include a bit extra"; it trades a match_one failure for an in-TU collision.
The correct carry is the minimal transitive closure of what the body actually references: here the
bandsetup helper + its 5 externs + the 18 gte_* macros of that region file (helper + externs alone
still left -34). Measured ladder on one function: -56 (nothing carried) → -34 (helper + externs)
→ MATCH-but-uncommittable (whole file) → the minimal set is the only bankable point.
CLOSED AND BANKED (Phase 29 SESSION-20). The final rung is no longer a prediction: the minimal
closure — 18 gte_* macros + the 5 externs + the static inline helper, 519 lines instead of
2,993 — gave match_one MATCH (1061 ins) and went through the §81 carve chain and the
whole-binary gate first try (harvest_verify verified 1 / failed 0, R22 clean-fleet 140/140).
Full ladder, all four rungs measured on func_8017C730 @ ov_SC03_013:
-56 → -34 → MATCH-but-uncommittable → MATCH + BANKED. Cost: minutes, no drafting, no agent.
The CANDIDATE gate and the REAL gate need DIFFERENT preambles — keep the difference out of the bank
match_one compiles the draft standalone (cpp -Iinclude, no engine_core.h), so a body using
shared types (PolyFT4, SVECTOR2, MATRIX2) hits 'PolyFT4' undeclared + a cascade of
parse error before ')' — which looks exactly like a broken draft and is not one. The real TU has
those types in scope for free: every region file opens #include "../shared/engine_core.h", which
includes the guarded engine_types.h.
So the types header belongs in a throwaway PROBE COPY, never in the draft you bank:
{ echo '#include "common.h"'; echo '#include "../src/shared/engine_types.h"'; \
cat .run/drafts-X/<fn>.c; } > $SCRATCH/probe.c # ../src/… resolves via -Iinclude
python3 tools/match_one.py <fn> --c $SCRATCH/probe.c --asm-subdir asm/<ov>/nonmatchings/<subseg>
Why this matters beyond convenience: the shortcut of pasting the include into the draft itself has
already leaked an absolute path (#include "/home/musashi/bfm-decomp/src/shared/engine_types.h")
into 21 git-tracked source files / 23 lines. Every one is a guarded no-op semantically (all 21
have engine_core.h at line 2, verified), so it is byte-neutral — but cpp still has to FIND that
literal path, so those 21 TUs cannot preprocess on any clone not at /home/musashi/bfm-decomp.
No byte-gate can ever see this (the path exists on the machine that made it) — R34's null-oracle
shape again, this time aimed at portability rather than coverage.
Also: the walk-back-to-previous-} heuristic breaks on an ISOLATED REGION FILE (a _jr_<addr>.c
produced by jr_isolate_all), where the construct immediately above the function is the helper you
need — the walk stops right after it and returns a 1-line preamble. Any preamble-carry tool needs a
reference-closure rule, not a positional one.
Why #2 and #3 happen, precisely. family_remap's backward preamble walk accepts a line only if it
starts with extern / // / /* / * / typedef. A multi-line typedef ends with } PolyGT4;,
which starts with none of those, so the walk halts there — and everything above it (including a
perfectly ordinary extern block) is lost. The tool documents the typedef half of this
("Multi-line typedefs aren't carried — those functions route through the engine_types.h lift") but
the consequence is broader than the note implies: one unrecognised line truncates the whole
preamble, dropping constructs the walk would have accepted.
The recipe (cheap, and it converges in 2–3 rounds). Do NOT reason about what the tool should have carried — compile and let cc1 enumerate it:
match_onethe generated sibling. cc1 names the first missing symbol/type.- Find it in the exemplar; carry that construct across, applying the tool's own printed substitution map to any per-overlay names inside it.
- Repeat. Each round clears one construct class, and the error text tells you which.
Behemoth #3's sibling func_8017F5B4 (1,511 ins) went CC1 FAIL → CC1 FAIL → CC1 FAIL → **MATCH**
across four rounds of exactly this, for ~0 agent tokens — the remap itself was correct from the
first invocation (52 per-overlay symbols substituted); only the preamble was short.
Rule: after any mechanical template/propagate step, diff the exemplar's full file-scope preamble
against what the tool emitted before concluding anything about the body. A CC1 FAIL on a
mechanically-remapped sibling is a preamble report until proven otherwise — it says nothing about
whether the remap was right.
§78 — A LENGTH drift can be a register grant in disguise; and fold never leaves a literal first in an | chain (Phase 29 SESSION-19, behemoth #2 func_8017D960 3,338 ins, 1806 → 0, pin-free)
Behemoth #2 was carried at "3,334/3,338, one fold OR-chain error left" — a summary that reads as four
instructions from done. Measured, it was 1,806 mismatched with a LENGTH-DRIFT/-4, and the
prior diagnosis was wrong in both halves.
The drift was an allocation decision, not missing code
The 4 missing instructions were 4 emit tails × 1 nop. The target's 0xFFFFFF OT mask lives in
$a1 — the same register as tp — so it cannot be materialised until tp's last load retires, and
the first tp[] load-delay slot therefore stays a real nop. In the draft the mask sat in $a0,
free early, so maspsx hoisted it into that slot and the nop vanished. Root cause one level up:
u32 *otp; at function scope has 4 deaths ⇒ fails local-alloc.c:472 ⇒ global allocno in $a2
⇒ via global.c:668-671 that pushes tp off $a1. Declaring otp per emit arm fixed the entire
drift in one edit (3334→3338, 1806→333).
This is the second time in one session that a residual which LOOKS structural was an allocno-class
choice (the first: a "scheduling" transposition on the sibling that fell out of a register grant,
§76). Practice: before treating a length or ordering diff as structural, check whether a register
grant explains it. A nop that exists in the target and not in your draft is very often a delay
slot the target could not fill because the register it wanted was still live.
fold never leaves a literal in the first term of an | chain
Seven parenthesisations of cb | c | (c<<8) | 0x800000 were measured; all reassociate — gcc-2.7.2's
fold-const.c will not emit or acc, K, var with the constant first. So if the target's asm shows
or acc, var, K as the FIRST term of a chain, K was a VARIABLE in the original source, not a
literal. That is a direct read from asm back to source shape, and it retires an entire family of
"try another parenthesisation" sweeps.
"Make it a variable" has TWO separable effects — and the wrong choice is catastrophic
Turning a literal into a variable changes (a) opacity to fold (fixes structure) and (b) adds an
allocno (changes registers). They are independent, and you usually want only (a):
- a fresh short-lived local gets the structure right and the allocation catastrophically wrong — measured 690 mismatched / 79.6%, with damage appearing ~300 instructions away in unrelated blocks;
- reusing an already-busy or function-scope variable gets both right. When you need opacity, spend an existing variable's ref count, not a new allocno.
The economics
Nine levers, each proven individually necessary by drop-one ablation against the matching draft:
otp per arm · no pins · dedicated vw/vzw vertex-word temps · an RC-15 zero-byte mny dial in
the tri cull block only · base = D_800AF630 first · f2,f1,f0 = 0 order · shared cb colour base ·
shared rgbw result temp · s32 za, zb; per case.
Five of the nine were read straight off the MATCHED relatives (func_8017F510 1,511 and
func_8017CA80 952, same renderer family) — worth more than every expression sweep combined.
Crack the smaller family member first; it is a lever library for the larger one.
§79 — For a 0-callee giant, fingerprint by DATA symbols (§71 cannot fire); and the STACK-SLOT ORDER is a declaration-order oracle (Phase 29 SESSION-19, func_8017BF14 4,763 ins, cold start → 45/4763)
A cold-start attempt on the project's second-largest function reached 4763/4763 ins, 45 mismatched (99.06% byte-identical, 99.94% structural, exact frame, exact opcode histogram) — not a match, but it produced two levers and refuted the premise it was given.
§71 has a blind spot, and this is it
The target was briefed as "no matched relative — a genuine cold start": h_norm/h_seq family
size 1, and §71's callee-set fingerprint returned jaccard 0.00 against every matched giant. That
premise was wrong. §71 fingerprints by callee set — and this function makes zero jal calls,
so the fingerprint is empty and cannot fire by construction. Grepping the target's data symbol
D_800A5E60 landed immediately on the matched func_8017BEBC: it is the 4-light-box member of the
same volumetric-light renderer family whose 3-box sibling (func_8017D960, 3,338 ins) was matched
hours earlier.
Rule: when §71 returns an empty or zero-overlap callee set, fall back to DATA-symbol fingerprinting
(lui %hi(D_xxxxxxxx) operands in the target .s). A leaf giant has no callees to fingerprint by, but
it still touches the same globals as its family. An empty fingerprint is a "cannot answer", not a
"no relative" — do not let it become a cold-start brief.
NEW LEVER — the frame layout reads back the original declaration order
gcc-2.7.2 assigns stack slots to spilled pseudos in pseudo-number order, and pseudo numbers are
issued in order of first use ≈ declaration order. Therefore the target's frame layout is a direct
readout of its source's declaration order. Compare your draft's slot assignments against the
target's and reorder declarations until they agree — moving a single line (f0..f3 after pkt) took
73% → 84% structural and brought all 127 slots into exact correspondence. Automatable; the
session's implementation is .run/giants/bf14_slots.py.
This is the counterpart to §78's "read the asm back to source shape": there, an |-chain's first term
tells you a literal was a variable; here, the frame map tells you the declaration order.
§76 confirmed at scale, and a pin nuance
- The entire −62 length residual was ONE allocno-class decision: declaring
s32 c0..c3inside the cull blocks (1 death ⇒ local allocno ⇒global.c:668-671removes those hard regs from the global pool) spilledr1loand moved the draft 52% → 93%. An__asm__ref-dial reached the same spill and scored worse — declaration scope beat the ref dial, again. - Pins are safe on a 0-
jalfunction — §74's caller-saved-across-a-call hazard cannot arise, so the usual suspicion is unwarranted here; 4 pins took 94% → 99%. But §72 still held: pins 5 and 6 made it worse. Pins remain a preference, and past a small number they fight the allocator.
The residual, and the honest read
45 mismatches, three register-grant ties, zero structural divergence. The one §76 lever class the
session never reached is variable REUSE across c0..c3 / a0v..a3v — that is the named next move.
Artifacts: .run/giants/s19_func_8017BF14_b1.c (45/4763), a pin-free fallback at 789/4763 that is
100% structural, and s19_bf14_report.md (~40-row do-not-re-buy table + 4 refuted diagnoses).
Cold-start economics, measured: a 4,763-instruction leaf giant with a findable matched relative reached 99.06% in one pass but did not close. Budget a second pass for anything this size; the first pass buys the decode, the frame, and the length — the last ~1% is register grants.
§80 — A do-not-re-buy entry is scoped to its BASE, not to the function; and the pin's hidden cost is an unconditional qty_phys_sugg (Phase 29 SESSION-19, func_8017BF14 45 → 0)
Round 2 closed the 4,763-instruction behemoth (45 → 37 → 33 → 21 → 11 → 3 → 2 → 0, reproduced 3×
from independent work dirs, banked whole-binary BYTE-IDENTICAL). The route matters more than the win.
⚠️ THE PROCESS CORRECTION: a measured negative is relative to the draft it was measured on
Round 1 left a careful ~40-row do-not-re-buy table. Three of its entries INVERTED on round 2's base.
The same edit (qsingle23) measured 1,040 mismatched on the 45-base and 11 on the 21-base.
Re-testing the round-1 negative list cost ~20 seconds and produced three of the seven winning
levers.
So: a do-not-re-buy table is a record of (edit, base) → result, NOT edit → useless. After any
lever that moves the base materially, re-run the negative list — it is seconds with a real harness
and it is where the next levers hide. This retroactively qualifies every such table in this cookbook
(§45, §60b, §75a, §76, §78, §79 and round 1 of this function): treat them as starting hypotheses at
the base where they were taken, not as closed questions.
Corollary already seen: round 1 measured "removing the va→$t2 pin costs 4% elsewhere" and concluded
keep the pin. On a base where c0..c3 sit at function scope, removing those pins is worth 21→13
— the opposite conclusion from the same experiment.
The pin's hidden cost, with the citation
combine_regs' hard-register branch (local-alloc.c:1795, reached from :1295 with
already_dead == 0) records the pinned register in qty_phys_sugg unconditionally — there is no
death guard. So a register __asm__ pin does not merely prefer a register: it actively invites
local-alloc to tie producer chains into it, which is exactly the residual-(a) tie round 1 diagnosed
but mis-cured. Three separable cures exist; the new one is worth knowing:
- R7 — a zero-byte
__asm__ref that keeps the pinned value LIVE PAST the temp, sofind_free_regcannot honour the suggestion. That closed the final 2 instructions, and was necessary becausec1→$a0proved uniquely load-bearing (it is what spillsr1lo; every alternative pin lost 64 instructions).
The flagged "#1 move" LOST — and why the failure is informative
Variable REUSE (§45-A / RC-14 MERGE) was swept in full: every merge lost, 43–3294 across 8 merges.
It was the right lever class for the sibling func_8017F510 (97 → 10) and the wrong one here, for a
structural reason worth carrying: the TRI and QUAD grants did not differ by RANK, they differed by
IDENTITY — two independent allocno sets. Re-ranking inside one set cannot fix a two-set problem.
Diagnose whether you have a ranking problem or an identity problem before reaching for a merge.
The actual fix was s32 c0,c1,c2,c3; at function scope (33 → 21), read off the two matched
relatives (b5:310, b4:338) and confirmed against the target itself: its TRI grants are identical
to its QUAD grants.
§78's attribution primitive, run and reproduced
Under -fno-schedule-insns, -fno-schedule-insns2, and both, the draft's order was unchanged ⇒
the rgb-accumulator transposition was never a sched.c decision. A 3-statement accumulator pins the
value to one register, so no scheduler could hoist the or above the sw. Changing the grant fixed
the order for free — §78 reproduced on a second function.
Cold-start economics, now complete
A 4,763-instruction leaf giant with a findable matched relative: round 1 = decode + exact length + exact frame + 99.06%; round 2 = the last 45. Two passes, and the second was far cheaper than the first. Budget two passes at this size and do not read a 99% round-1 result as a stall.
§81 — Banking a jr (jump-table) function: the 3-step carve chain, and why match_one cannot see the problem (Phase 29 SESSION-19, func_8017C954)
match_one masks jal/HI16/LO16 relocations. A jr (switch) function therefore reports a clean
MATCH while the whole-binary gate reports DIFF — and the gate is right. Matching the C makes
gcc emit that function's jump table into .rodata (floated to the FRONT by section_order) while the
raw copy still sits in the overlay's data tail ⇒ duplicate table at the wrong address. This is the
§53 carve law, and it is the one case all session where the candidate gate and the real gate disagreed
about something that was not plumbing.
Detect it before you spend a gate cycle: grep -cE 'jr\s+\$(v0|v1|a0|t[0-9])' the target .s
(a mid-function jr on a non-$ra register), and look for a jtbl_<addr> in asm/<ov>/data/*.s.
The chain (each step byte-gated on its own, BEFORE building the next on top):
1. tools/jr_isolate_all.py <ov> --only <func> # cut the fn into its own code subseg
make extract BINARY=<ov> && make build BINARY=<ov> -> must be BYTE-IDENTICAL
2. tools/jtbl_carve.py <ov> --func <func> # carve its jtbl into a dotted .rodata subseg
make extract BINARY=<ov> && make build BINARY=<ov> -> must be BYTE-IDENTICAL
3. tools/harvest_verify.py --binary <ov> --drafts <dir> --chunk 1 # now the bank
then a FULL R22 (config changed => T2).
Use --only. jr_isolate_all <ov> bare would have resegmented 47 jr-functions across 21
objects; --only func_8017C954 touched 2 functions in 1 object. Same result, a fraction of the
blast radius. (The tool automatically pulls in already-banked jr in the same object so their existing
carves get repointed — that is why --only is safe rather than partial.)
Step 2 fails loud when the subseg would host NON-CONTIGUOUS .rodata carves — one object can
contribute at most ONE contiguous .rodata run, so two matched jr-functions in the same code subseg
with a third, unmatched jtbl between them is unsatisfiable. That refusal is the instruction to run
step 1.
The defect this exposed: a shared type that is present but invisible
Step 1 refused with "2 file-scope decls matched _HOIST_RE but could not be placed" for
extern struct PW8017E6D8 D_801E1EC4; — while struct PW8017E6D8 sits in engine_types.h:658.
_engine_types() harvested shared type names with four patterns (typedef … X;, } X;,
forward-decl struct X;, fn-ptr typedef) and a tagged definition with a body —
struct PW8017E6D8 { int w; } __attribute__((packed)); — matches none of them. Measured: 77
such tags in engine_types.h were invisible to the check. One added pattern
(^\s*(?:struct|union|enum)\s+(\w+)\s*\{) fixed it.
Why this cost twenty minutes instead of a mystery byte-diff three phases later: the Phase-26 audit
had already converted this predicate's silent drop into a loud refusal. The original bug dropped
4,040 col-0 decls, 683 of them function prototypes — and a dropped prototype is a silent
byte-changer (C89 implicit int f(), and return type drives delay-slot fill in this codebase). The
refusal named the exact symbols and the exact remedy. A loud "I cannot place this" is worth far more
than a green build.
§82 — Two source-shape oracles from behemoth #6: a duplicated addiu $aN,$sp,K across a jal means the block was INLINED, and scalar-vs-aggregate decides WHEN a stack slot is allocated (Phase 29 SESSION-19, func_8017C730 1,061 ins)
Two findings that read the ORIGINAL SOURCE SHAPE off the asm — the same class as §78's |-chain rule
and §79's frame-slot oracle, and the pair that actually cracked this function.
1. The inlined-helper signature
&X for any non-first local always creates a pseudo, and CSE always merges two of them
(expr.c:6260, ADDR_EXPR → force_operand(..., NULL); the one exception is the
virtual-stack-vars offset-0 local). So if the target re-materialises the SAME addiu $aN,$sp,K
at two call sites separated by a jal, CSE was prevented from merging them — which means those two
sites were not in the same function body. The block was an INLINED function.
17 non-inline spellings failed to reproduce the prologue; a static inline helper reproduced it
byte-for-byte on the first try.
The reusable probe that produced the hypothesis: scan the built objects for that duplicated-addiu
signature in functions that are NOT INCLUDE_ASM (i.e. already-matched code known to come from real
source) — ~1,200 objects, cheap, and it tells you which shapes the original codebase actually used.
2. Scalar vs aggregate decides when the slot is allocated
A scalar's stack slot is allocated LAZILY, at its first &; an aggregate's is allocated AT ITS
DECLARATION. So six GTE result words must be six separate longs, not a struct — only then do
they land after an inlined helper's temps (here 0x118..0x12F) and the frame comes out at the
target's 0x270. Declaring the same six as a struct puts the slot in the wrong place and no amount
of reordering recovers it.
Second-order effect worth knowing: this also flips MEM_IN_STRUCT_P (§30's /s flag). With one
of those words a fixed-address scalar, ((PolyF3*)pkt)->rgbc no longer aliases it — so a store had
to be respelled *(u32 *)(pkt + 4) to keep the target's nop. A scalar-vs-struct choice is
simultaneously a frame-layout decision and an aliasing decision.
Also reproduced on this function
§78 (reuse an already-busy variable — t32 = mid matched where a fresh temp did not) · §80(i) (a
lever went from −8 ins to exactly neutral as the base moved) · §72 (a register pin made it worse).
And the banking footnote (§75a class A, one line)
The whole-binary gate rejected the first bank with conflicting types for 'ApplyMatrixSV': the draft
declared it (MATRIX2 *, SVECTOR2 *, SVECTOR2 *), the TU and the fleet canon use (void *, void *, void *) — 2,286 of 2,835 sites. Conforming the draft's decl to the canon is byte-neutral
(pointer args pass identically) and banked first try. On a jr function, expect BOTH gates to have
something to say: the carve chain answers the jump table, and §75a answers the declarations.
§83 — The parameterised-repeat law, the spill-area trap, and why a per-case edit cannot move a per-case symptom (Phase 29 SESSION-20, func_80183814 5,122 ins, cold-ish start → 36 structural / 99.3%)
The largest unmatched function in the project, a 21-case state machine. Round 1 reached 5,127 ins vs 5,122 target, 36 structurally-unmatched instructions under a register-blind mask (99.3% exact), 17 of 21 case bodies EXACT, args+locals byte-exact at 216 bytes — not a match, and the residual is one decision, not 36 problems.
§83a — READ THE HEADLINE NUMBER CORRECTLY: a LENGTH drift makes match_one's count meaningless
match_one compares strictly index-wise. A +5 instruction delta shifts every later index, so the
tool reported 4,622 mismatched where difflib-aligned comparison shows 36. Those two numbers
describe the same draft. When the class is LENGTH-DRIFT, the mismatch count is not a progress
signal — align first (difflib) and re-read, or you will abandon a 99.3% draft as a 10% one.
(Counterpart to §78: there, a length drift was a register grant in disguise; here it inflates the
score. Both say the same thing — never read a length-drifted diff literally.)
§83b — THE LEVER: find the parameterised REPEAT before decoding case-by-case
Three callees (func_8012EC04/func_8012F14C/func_8012C51C) each appear exactly 35 times as
one 72-instruction body parameterised only by (KIND, START, BOUND) — with the invariants
member == KIND*4, array == &D_801F61C0[START], and prev.BOUND == next.START. That is 2,625 of
5,122 instructions (51%) from a SINGLE definition. Writing it once as a template and generating the
35 sites (harness: .run/giants/s20_g14_*.py — edit the template in one place, re-propagate to all
35) is the whole game on a function this size.
Practice: on any large state machine, hunt for a parameterised repeated block BEFORE decoding case
bodies one at a time. The tell is a repeated call triplet at a fixed stride with monotone constants.
(A prior handoff asserted this repeat existed; it was carried forward flagged UNVERIFIED because it
appeared nowhere in the recon. It proved TRUE — but flagging it cost nothing and the discipline stands:
an unverified premise is a hypothesis to test first, not a foundation to build on.)
Three sub-levers made the template byte-exact:
- A running POINTER walk, not array indexing — under
-G0array indexing does not strength-reduce. rand() % (u32)xto forcedivu(the signed spelling emitsdiv+ the sign-fixup dance).(s32)((u8 *)p + (X + 0xC))— the cast barrier stopscombinereassociating the offset.
§83c — TRAP: a "dead local" in a prior draft may be gcc's OWN spill area
The inherited recon modelled a 128-byte s32 pad[32] local and called it dead. It is not a local at
all — it is gcc's spill area. Declaring it explicitly adds 132 bytes and corrupts the layout;
removing it made the 216-byte args+locals area byte-exact. If a draft needs an unexplained dead
block to reach the target frame size, suspect the spill area before inventing a variable. (The frame
being 8 bytes over here is a different cause — two extra callee-saved registers, §83d.)
§83d — CSE's quantity budget is WHOLE-FUNCTION, so a local rewrite cannot fix a local symptom
The stall: in cases 0 and 3, gcc CSEs three &D_8018E27C-class address constants (+0x30) across two
call groups that share a basic block, consuming 2 callee-saved registers the target spends on real
variables — so a0 lands in $s7 instead of $s2 and every register downstream renames.
Why no rewrite of case 0 moved it: cse.c:8340 sizes the quantity table as
max(nsets*2, 500) + max_reg — gated by the whole-function pseudo count. A per-case edit does not
change max_reg, so it cannot change a CSE decision, even one whose symptom is local.
⇒ The lever for a function-global CSE fork is a function-global quantity change — here, closing the
+5 length delta is expected to move max_reg and the fork together. Diagnose the SCOPE of a
compiler decision (function-global vs block-local) before choosing where to edit.
A diagnostic a0 → $s2 pin halves the residual and makes the prologue byte-exact — which confirms
the diagnosis — but it is a hand-placed dial, not plausible source, and it does not fix the frame. It
belongs in the do-not-re-buy table, not in the deliverable (§72/§74).
§83e — §80 vindicated again, on the same day it was written
Two case residuals (6 and 11, 8 and 7 diffs) had been written off as "pure allocation". They were not:
the morph/lerp loop walks copies of its two input pointers (pa = msa; pb = msb;), not the
originals — and spelling that took both cases to zero. A residual class you assigned on an
earlier base is a hypothesis, not a verdict (§80). Re-run the negative list after the base moves.
§84 — The DERIVED-OFFSET remap bug: a hand-computed literal that encodes the DISTANCE between two per-overlay symbols, and why match_one is structurally blind to it (Phase 29 SESSION-20, func_8013D53C)
A family_remap member draft reached match_one MATCH (240 ins) and failed the whole-binary
gate. The whole binary differed by ONE BYTE.
The construct
Cracking the exemplar produced a deliberate matching idiom — reach a symbol via a different symbol
plus a literal offset, so gcc cannot CSE the two %hi/%lo pairs into one:
/* EXEMPLAR (ov_SC01_077): 0x801DA998 + 0x20 == 0x801DA9B8 ✓ */
(*(S9 *)&D_801DAA78) = *(S9 *)(&D_801DA998 + 0x20); /* same addr as &D_801DA9B8;
distinct sym defeats cse, keeps %hi/%lo */
family_remap substitutes symbol NAMES correctly — D_801DA998→D_801A5778, D_801DA9B8→D_801A5790
— and leaves the literal 0x20 alone. But 0x20 is not a constant of the algorithm: it is the
distance between two per-overlay symbols, and that distance is per-overlay.
exemplar: 0x801DA998 + 0x20 = 0x801DA9B8 ✓
member: 0x801A5778 + 0x20 = 0x801A5798 ✗ (the real symbol is 0x801A5790)
member: 0x801A5778 + 0x18 = 0x801A5790 ✓ correct offset is 0x18, not 0x20
Changing 0x20→0x18 + running the full gate_stage ladder banked it byte-identical.
Why it survived every candidate gate
match_one masks HI16/LO16, so a wrong %lo is invisible to it — it reports a clean MATCH.
Only the whole-binary gate sees the byte. This is the §81 blindness in its DATA form: §81 is
match_one blind to a jump table; §84 is match_one blind to a mis-derived data address.
A match_one MATCH that fails the whole-binary gate by ONE BYTE is a masked-field bug —
diff the built image against the payload and read the differing word before assuming plumbing.
The diagnostic that found it in one step:
python3 - <<'PY' # built vs extracted payload, byte-diff, map file offset -> vram
a=open('build/<ov>/<ov>','rb').read(); b=open('<target_path>','rb').read()
d=[i for i in range(min(len(a),len(b))) if a[i]!=b[i]]
print(len(d), [hex(BASE+i) for i in d[:8]])
PY
THE FIX IS MECHANICAL — the tool already holds the answer
The remap knows both mappings, and the exemplar's own comment even names the aliased symbol (which
the remap has already substituted: "same addr as &D_801A5790"). So:
correct_literal = mapped(aliased_sym) − mapped(base_sym)Detect&SYM + LITERALwhereSYM_exemplar + LITERALequals another mapped symbol's exemplar address, and recompute the literal from the member's own addresses. Never carry the exemplar's literal through a symbol substitution.
Scope, measured (do not over-generalise — §80)
The idiom is rare: only 5 sites across the whole matched corpus
(ov_SC01_077.c ×1, ov_SC01_077_jr_8017AE2C.c ×2, engine_core.h ×2). But one of those five
gates an entire 123-member family — 133 staged member drafts all carry the un-recomputed + 0x20
with different per-overlay base symbols (D_801F3058, D_801A5778, D_8018F9B8, …), so the single
tool fix is worth ≈ 240 ins × 123 members ≈ 29,520 ins.
It does NOT explain the sibling pool generally: the other three families probed the same day
(func_80144090, func_8012CC88, func_8014D12C) have zero derived-offset sites and fail for a
different, still-undiagnosed cause.
Two ladder lessons banked with it
- Bare
harvest_verifyis the LAST rung, not the ladder.gate_stage.pyrunscanon_resident_calls → cast_call_sites → reconcile_tu → ARITY pre-pass → sig_unify → harvest_verify. A probe that callsharvest_verifydirectly measures the un-recovered rate and will report PLUMBING for everything the ladder would have cleared. The byte fix AND the ladder were each individually insufficient here; only together did it bank. reconcile_decls.pyis RETIRED (Phase 26-A, R33) — superseded byreconcile_tu.py, because asking "what does the FLEET call this symbol?" is wrong by construction in a loosely-typed engine (548 of its answers conflicted; it was rewriting 60 of 196 live drafts). Reach forreconcile_tu, neverreconcile_decls.
§85 — The RETURN-axis fleet widen is ALL-OR-NOTHING: widening the shared header alone guarantees a conflict in the source overlay (Phase 29 SESSION-20, func_8012CC88 / func_8014D12C)
§73 says a def-side self-decl conflict has two axes — RETURN (fleet widen, T2/R22) and PARAMS (casts at each use, T0). This is the RETURN axis measured end-to-end, including the way it fails.
The conflict
A remapped member's byte-true definition needs a return value the fleet declares as void:
draft def : s32 func_8012CC88(s32, s32, s32) <- the return keeps a local alive (§30#2)
fleet decl: extern void func_8012CC88(s32, s32, s32)
-> cc1: conflicting types for `func_8012CC88'
THE FAILURE MODE — widening only engine_core.h is worse than not starting
Widening the shared header banked the member in the target overlay and broke ov_SC01_077
(R22 139/140). Reason: the source overlay carries its OWN local extern void func_X(...) decls
(they predate the shared macro), so a shared-header-only widen puts the two in direct conflict.
The per-binary gate cannot see this — it passed on ov_SC01_000 while breaking a binary it never
built. §63/§61 exactly: a T2 write set is only provable by R22.
The precondition, and how to check it in one grep
The widen is byte-neutral iff no caller consumes the return value (a call compiled against void
discards it either way, so the emitted code is identical):
grep -rhoE "[A-Za-z_]\w* *= *func_XXXX\(" src/ | wc -l # must be 0
Measured here: 0 for both functions — then the widen is safe.
Do the WHOLE axis in one edit, then R22 once
grep -rl "extern void func_XXXX(" src/ | xargs sed -i 's/extern void func_XXXX(/extern s32 func_XXXX(/g'
make clean && make extract-all && make check-all # T2 => R22 MANDATORY
Measured scale: 3,668 decl sites across 2,688 files for two functions — and 140/140
byte-identical, tools-health green, dedup 1886/0. The site count is large because every overlay
re-declares the callee locally; that is normal, not a smell.
Reading, for the next person
- A half-done axis is a guaranteed break, not a smaller win. Widen every
extern voidsite for that symbol acrosssrc/, or widen none. 0 sites remainingis the completion assertion — count before and after (R32).- The other axis (PARAMS) still uses per-use casts (§17a-1/§73); do not widen params this way.
⚠️ A verification trap that cost me a false "BYTE-IDENTICAL" report
make build BINARY=X 2>&1 | grep ... | head then echo rc=$? reports the exit status of head,
not make — it is 0 even when the build failed. The output simply had no BYTE-IDENTICAL line,
which is the thing to check. Assert on the expected SUCCESS STRING, never on $? after a pipe
(or use set -o pipefail). A green-looking rc=0 on a failed build is the same class of
self-deception R32/R35 exist to prevent — the gate was honest; my reading of it was not.
§86 — Pinned-exemplar templatability is a PER-FAMILY property, not a per-member rate; and the §42e pin guard is now over-conservative (Phase 29 SESSION-20)
The guard refuses a class that largely works
family_sweep's §42e guard skips any family whose exemplar carries register __asm__ pins. Measured
on the zero-crack pool: of 1,083 candidate members in the top 8 FREE families, 680 (63%) were
refused BEFORE any gate ran. Re-run with --allow-pins (letting the byte-gate arbitrate, G3/P9):
268 of 680 banked, and ZERO cc1 crashes across hundreds of pinned compiles.
That confirms the guard's original motivation is gone: the SIGABRT it protects against was
Phase 27's extract_unit macro-drop, not a compiler limit (§42e-CORRECTION). The guard is now
protecting against a bug that no longer exists.
THE LAW: all-or-nothing PER FAMILY
The headline "37%" from a 19-member sample was an ARTEFACT OF MIXING FAMILIES. The real distribution:
| exemplar | members | banked |
|---|---|---|
func_801749C8 |
137 | 137 (100%) |
func_8014C6F4 |
137 | 137 (100%) |
func_80133AB0 |
136 | 4 |
func_8014CF04 |
137 | 0 |
func_80143D28 |
136 | 0 |
Two families at 100%, three at ~1%. Whether a pinned exemplar templates is a property of the
FAMILY (does its pin set survive symbol substitution into a sibling TU?), not a per-member lottery.
This matches SESSION-19's func_8017A4AC, which banked ×134 with pins — a whole family, not a
fraction.
⇒ THE PROCEDURE (do this, not a blanket sweep)
for each pinned family:
probe ONE member -> banks? yes: sweep the whole family
no : SKIP IT ENTIRELY
Cost measured: the blanket sweep spent ~412 futile gate cycles (≈60% of the run) grinding three families that were never going to bank. The 1-member probe reduces that to 5 probes + 2 sweeps. A sample that straddles families reports their AVERAGE and hides the bimodality — sample per-family, never per-pool. (Counterpart to §80: a measurement is scoped to what it was taken over.)
Why the two live families differ from the three dead ones — the open question
Not yet diagnosed. The likely axis is whether the pinned registers are caller-saved across a jal
(§74's corrupting form) in the sibling's register pressure, versus pins that only fix a local
allocno. Diagnose before extending --allow-pins fleet-wide — the byte-gate makes a wrong guess
free, but a wrong PROCEDURE costs a sweep.
§87 — match_one COMPILES but never LINKS, so an unresolvable data symbol reads as MATCH; and stored drafts go STALE against the tree (Phase 29 SESSION-20)
The autopsy's "integration" bucket — 315 entries reported match_one MATCH, "byte-correct
standalone, blocked only on plumbing" — banked 0 of 27 through the full gate_stage ladder.
Two causes, neither of which is plumbing:
- UNDEFINED DATA SYMBOLS. The gate fails at LINK:
undefined reference to D_801893xx. That symbol is defined in no overlay's symbol file.match_onecompiles a single TU and never links, so anexternthat resolves nowhere is structurally invisible to it — it reports a clean MATCH. (Checked and refuted the obvious alternative: the drafts were authored for the CORRECT binary, so this is not mis-targeting.) - STALE AGAINST THE TREE.
redefinition of struct S80172C50— the struct has since been lifted intosrc/shared/engine_types.h(§20/§64 type-lift), so the draft's own copy now collides. A draft stored months ago is scored against TODAY's tree; type-lifts, shared-header growth and canon-sig changes all age it.
The blindness ladder, now complete — FOUR classes match_one cannot see
| § | class | what it masks | who catches it |
|---|---|---|---|
| §81 | jump tables | the duplicated .rodata jtbl |
whole-binary gate + carve chain |
| §84 | mis-derived %lo |
HI16/LO16 are masked | whole-binary gate (ONE byte) |
| §87 | unresolvable symbol | it compiles, never links | the LINK step |
| §87 | stale draft | the tree moved under it | cc1, at re-compile |
**match_one MATCH means "this TU compiles to the right bytes with relocations masked" — nothing |
|||
| about linking, nothing about the current tree.** Treat a stored MATCH as a CLAIM WITH A TIMESTAMP. |
Consequence for the backlog ledger
The autopsy's integration bucket is overstated as ready-to-bank work. Combined with §83's
finding that 44% of the ledger (707 redraft) are partial drafts misfiled as near-misses, the
honest read is: docs/backlog.md's headline count is not a work queue. Before planning against
any stored-draft pool, RE-GATE A SAMPLE — the recompute is one CPU-second per entry
(autopsy collect) and it is strictly better than trusting a stored verdict.
The cheap discriminator, before spending a sweep
grep -oE 'D_[0-9A-F]{8}' <draft> | sort -u | while read s; do
grep -qF "$s" config/symbols.$OV.txt config/symbols.us*.txt || echo "UNRESOLVABLE $s"
done
An entry with any UNRESOLVABLE symbol will fail at link no matter how clean the ladder run is.
§88 — cross_jump will not merge a common suffix containing a CALL; and §78 is scoped to ORDERED comparisons only (Phase 29 SESSION-20, the behemoth close-out)
Three behemoths closed the same night (func_80183814 5,122 · func_8017D2DC 1,586 · func_8017DC1C
1,518), taking the fleet to zero unmatched functions >1000 ins. Their durable output:
§88a — repeated CALL-shaped blocks are left UNMERGED; call-free tails are merged for you
Measured on func_8017D2DC: ~34 byte-identical 6-instruction jal-bearing blocks were left
unmerged, and case 66's two byte-identical 12-instruction blocks likewise — while every
call-free tail (.L8017EB5C/EB64/EB70, .L8017E9A0/E9AC/E9B4, .L8017E06C, .L8017DAF0,
.L8017E804) was merged. ⇒ Write repeated call-shaped cases LONGHAND; never hand-factor the
call-free tails — the compiler does that itself, and pre-factoring them puts you off-target.
§88b — the slti literal-position law (extends §78 to comparisons)
MIPS slti can only carry its constant on the RIGHT, so mips.c rewrites LE/GT vs CONST_INT into
LT(x, K+1). Consequence, decidable from bytes alone: if the target materialises the limit into a
REGISTER on the LEFT of slt (li K; slt K,x; beqz), K was NOT a literal in the source. All eight
literal spellings were swept (> K, >= K+1, K < x, !(x<=K), ++x > K, early-return…) and
every one folds to slti x,K+1. Only a block-local variable holding the limit
(s32 lim = K; if (lim < X)) reproduces it — and per §76 that variable's block scope matters: if
its live range spans a call it takes a callee-saved register and the li hoists.
⚠️ §88c — THE MIRROR-IMAGE FALSE POSITIVE (this one costs 25 wasted edits)
addiu $vX,$zero,1; bne looks exactly like §88b's "the constant was materialised ⇒ it was a
variable" signature. It is not. MIPS has no beqi, so equality constants are ALWAYS
materialised. §78/§88b are scoped to ORDERED comparisons only — the ones mips.c rewrites into
slti. Believing the signature on an ==/!= sends you rewriting perfectly correct literals.
§88d — BANKING ORDER: run the §81 carve chain BEFORE banking, never after
Banking func_8017DC1C first broke the build. Its draft establishes the canon for 39
previously-undeclared externs; jr_isolate_all's re-partition (overlay_src_split) then dropped
all 39 across the new split boundary — D_801C1EB0 undeclared, present in NEITHER file. That is the
§77 preamble-drop class in a third tool. The fix is ordering, not patching: carve chain first
on a clean tree (each step byte-gated), then bank. Re-sequenced, both banked first try.
§88e — a wrong diagnosis, refuted properly (the model for how to treat an inherited lever)
Round 1 handed round 2 "close the +5 length delta to move max_reg", citing cse.c:8340. Round 2
refuted it three ways: a 15-line reproducer reproduced the case-0/3 CSE exactly (so it cannot be
max_reg-gated); max_qty only gates extension ACROSS blocks and both call groups share one block;
and the target leaves $s7/$fp unused, killing the pressure story. The real discriminator came
from a second direction — case C01 has the identical two groups over the identical symbols with
zero residual, because a break puts a CODE_LABEL between them. The +5 was a SYMPTOM.
Both biggest levers were pure DECLARATION SCOPE (§45/§76), not pins. ⇒ An inherited "named
lever" is a hypothesis with a citation, not a fact. Reproduce it small before spending a round on it.
§88f — the missing rung: a RELOCATION gate between match_one and the binary
match_one masks jal/HI16/LO16 and never links (§81/§84/§87). The behemoth run produced
.run/giants/s21_g21_reloc_verify.py, which resolves every relocation — including the implicit
MIPS-REL addend objdump -r does not print (§84's trap, in tool form) — against the target:
359/359 jal callees, 60/60 internal j destinations, 406/406 %hi/%lo addresses incl. all 6
derived-offset sites. Promote this to tools/ — it closes three of the four blindness classes
before a gate cycle is ever spent.
§89 — Two throughput rules the project already had written down and was not following (Phase 29 SESSION-20)
Neither of these is a new capability. Both are cases where the tool or the rule already existed and the practice had drifted — which is worth recording precisely because that is the hardest kind of waste to notice from inside a session.
§89a — MEASURE the write set; do not assert its tier (tools/blast_radius.py)
§63's T0/T1/T2 taxonomy has existed since Phase 26 and was never enforced. Both failure directions were measured in one session:
- Over-verification: ~13 full clean-fleet R22 runs (~15 min each), most for batches that were
provably T1 (
src/<binary>/**only). Hours of serialised wall-clock proving what the written rule already guaranteed. - Under-verification — the dangerous half: the §85 return-type widen was believed contained to
one overlay and broke
ov_SC01_077(R22 139/140);gate_stage's ARITY pre-pass silently rewrote caller decls in 40 TUs. In both cases the write set was LARGER than the belief about it. A tier is a CLAIM.blast_radius.py --expect t1 --binary <b>turns it into a MEASUREMENT and exits non-zero when the tree disagrees. Negative-controlled on all four cases, including the §85 shape. Coverage asserted (R32): an unclassified path exits 2 and names itself.
§89b — the parallel gate farm existed; the family path could not reach it (tools/sweep_parallel.py)
bulk_harvest Phase B has been a ProcessPoolExecutor over DISTINCT binaries (per-binary flock,
per-worker result files, compute_fleet=False) since Phase 23 — but welded to Phase A's LLM
drafting. Family sweeps stage their drafts a different way (family_sweep --stage-only), so the farm
was unreachable from that path, and SESSION-20 gated 389 + 268 + 104 members serially
(for ov in …; do gate_stage …; done) — roughly an 8-16x throughput loss on a 32-thread box, for no
architectural reason. sweep_parallel.py is a thin adapter: same gate_stage.run_gate, same
per-binary lock, no new gate logic.
It also filters the phantom-dir bug at the source: a bare .run/sweep/*/ glob matches
gate_stage's own intermediate ladder dirs (-cn, -cn-cast, -cn-cast-rc, -s2in, -s2in-uni)
and calls them as binaries — 24 phantom PARTIAL 0/1 lines that inflated one run's notbanked from
0 to 56. Require config/splat.<bin>.yaml to exist (R33/R36: derive the binary set, never glob it).
The standing sequence
family_sweep --stage-only → sweep_parallel.py -j 12 → blast_radius.py
T2? → R22 clean-fleet mandatory T1? → the per-binary gates already ran; commit
Parallelism changes THROUGHPUT, never the verdict — the whole-binary byte-gate is still the sole arbiter (G3/P9) and still reverts a wrong draft in its own binary.
§90 — Five tool-integrity laws from one session, each of which changed an answer (Phase 29 SESSION-21, 2026-07-27)
SESSION-21 set out to run a mass wave over the family exemplars. Before the wave returned a single draft, five tools had produced a confidently wrong answer, and each fix changed the number it gave. These are the generalizable laws, not the individual bugs.
§90a — A comparison tool MUST share its reference oracle's index space, exactly
tools/reloc_verify.py (promoted this session) compares "what the target names at instruction index
i" against "what my object relocates at index i". It used objdump -dr; masked_diff has always
used -drz. Without -z, objdump ELIDES runs of identical instructions, dropping them from the
listing — func_801330E0 read 104 instructions under -dr and 110 under -drz (6 elided
nops). Every index after the first nop run therefore compared against the wrong instruction, and the
tool manufactured 2 mismatches on a draft that is clean at those sites.
A second instance in the same tool: the splat .s word field is little-endian hex TEXT
(0080033C), not the instruction integer. masked_diff byte-swaps it (struct.unpack("<I", …));
the new tool did not, and reported "word differs" on three sites that are byte-IDENTICAL
(3C038000 vs 3C038000).
The law: when you write a tool that compares against an existing oracle, do not re-derive its extraction — reuse it, or byte-for-byte replicate its flags and encodings, and cross-check on a known-clean input before trusting a single verdict. Five false alarms preceded the first true one.
§90b — "Byte-neutral" is not "wanted": undo on the SUCCESS path too
Two ladder stages (family_sweep --normalize-self-decls, gate_stage's ARITY pre-pass) had a
backstop that restored their snapshot only when the edit proved NON-neutral. A run that banked
nothing therefore left every edit in the tree: --normalize-self-decls 0/123 left 123 files /
246 insertions / 246 deletions of dead diff, and a git add -A would have committed it as noise.
The law (§61 applied to the success path): an edit that bought nothing gets reverted, regardless of whether it was byte-safe. Restore the snapshot on a 0-bank group. It cannot cost a match — there is nothing to preserve — so the only thing at risk is the noise itself.
§90c — A library-callable function must FAIL CLOSED on an unconfigured module
progress.linked_subsegs() is gated on a module global that set_binary() assigns. Imported as a
library without that call it returned an empty set — "this binary has no linked library
subsegs", which for main is wrong by 49 and silently reclassifies ~960 already-byte-identical
PsyQ-linked stubs as outstanding game-code work. The CLI path always configured first, so the defect
was invisible for as long as nobody imported it. I hit it within five minutes of importing it.
The law: a function whose correctness depends on module state must raise when that state is absent. An empty result is indistinguishable from a true negative, and that is the R32 silent-skip shape wearing a different hat.
§90d — Do not measure a live wave's drafts (§87 in real time)
Cross-checking the wave's landed drafts mid-run gave a different verdict for the same function two minutes apart — its agent had rewritten the file 12 seconds earlier. §87 says a stored draft is "a claim with a timestamp"; during a live wave that timestamp is now.
The law: draft QA happens after the wave returns, never during. A measurement of a file that is still being written describes nothing.
§90e — An agent's CONCLUSION and its EVIDENCE fail independently — re-derive the premise, not the fix
A wave agent found a genuine jtbl_carve bug (spimdisasm splits one jump table across two dlabels,
so the carve reserves 112 B for an object supplying 200 B — §84-class, invisible to match_one).
Its conclusion was correct and confirmed three ways. Its stated evidence was not: it reported the
continuation label as having "ZERO xrefs anywhere in the tree"; the label has two, and the agent's
proposed remedy — delete it — would have removed a symbol two emitted words reference.
The repair therefore uses a gate that needs no judgement call: the owning function's own
sltiu N range check, which gcc emits immediately before the indexed load, so the program
declares its own table length. Absorb a continuation only when it is adjacent, its words are all
code addresses, and absorbing lands on an exact sltiu bound — the SET of bounds, never
max(), because a multi-switch function has several and no way to say which owns this table.
Three more errors surfaced only by testing the fix: the absorption fired and the trailing-pad trim
immediately undid it (it re-trimmed against the first dlabel's words); a continuation ends at its
own last .word, not the next dlabel (224 B further on here) — which is the very assumption being
repaired; and the shortfall warning fired ~90 times across 38 tables until it was scoped to an
unambiguous single-bound pairing. A warning that fires on ambiguity is noise, not a signal.
The law: when an agent hands you a diagnosis AND a fix, re-derive the diagnosis from the bytes and design the fix from what you can prove. Verified: the split table goes 28 → 50 words, and across 38 jtbls × 6 functions = 228 combinations exactly ONE range changes. Both halves of that sentence are the deliverable — the fix, and the negative control proving its blast radius.
§91 — A structure-TRANSFER is only valid where the structure corresponds: the --like role trap (Phase 29 SESSION-21, func_8012AAAC ×137)
jtbl_family_bank sweeps a matched jr exemplar across its family by calling
jtbl_carve <sibling> --func <fn> --like <exemplar_ov>. The --like role-transfer copies the
exemplar's .rodata span STRUCTURE (its tables= starts) onto the sibling, on the premise
"same family => same span structure". That premise is a claim about the OVERLAYS' split layout,
not about the family — and it fails silently whenever the exemplar has a code-subseg split the
sibling does not.
The measured case. func_8012AAAC lives in ov_SC01_077_a (role _a) in the exemplar and in
the MAIN subseg (role ``) in all 137 siblings. Role-transfer keys on the subseg role, so it looked up
ov_SC01_077 — an unrelated seven-table span owned by entirely different functions — and stamped
those starts onto a sibling span holding one table. jtbl_rodata_pads then correctly refused:
consumed 1 rodata .align(s) but 2 pad spec(s) given — table-count drift vs the carve
Why it presented as a mystery: jtbl_family_bank deliberately excludes the table-count-drift
error from its auto-isolate retry (it is genuinely not isolate-fixable), and that path discards the
message — so all 137 siblings reported a bare gate-fail with no cause attached. The sweep read
as "this family does not template," which is exactly the verdict this project has had overturned as
tooling three separate times.
The fix: transfer only when the exemplar's subseg for this function has the sibling's role;
otherwise derive the span from the sibling's own carve, which was already computing it correctly.
Fail-open is not acceptable here — a wrong table set corrupts the image — so the guard defaults
to local derivation. Result: 0/3 → 3/3 on the probe, then 134/134 on the remainder — the family swept 137/137, zero failures. func_8012AAAC is now stubbed in ZERO overlays.
And it moved the RE-completeness number, not just the display one: distinct-code 69.3% → 69.5%. A jtbl family is byte-VARIANT (each overlay's table holds its own addresses), so every member is a genuinely new unique function — the opposite of the h_seq PURE propagation families swept earlier the same session, which added 274 members and moved distinct-code by +0.0. That is SESSION-20's routing rule reproduced twice in one session, in both directions: target byte-VARIANT families to move RE-completeness; high-reach h_exact families move only the decomp.dev display number.
The three-hypothesis trail, because two of them were wrong and the wrongness is instructive
- Call-site casts (real, fixed, NOT the blocker). Every sibling TU has the stub, then
extern void f();, then a 0-arg call later in the same file — so splicing the definition puts a prototype in scope and gcc rejects the call. 137 TUs, exactly the member count. Cast one site per TU, R22-proven byte-neutral. Still 0/3. A real defect that had to be fixed anyway, and fixing it moved nothing — "the error I can see" is not the same as "the error that is blocking me." - My own carve-alone test (a false lead I generated). §81 step 2 says the carve alone must be byte-identical. For this shape it CANNOT be: the 2-entry spec describes a table the object only emits once the body is banked, and a stub object emits one. The tool splices the body BEFORE building, so its path is valid and my simplification was not. A diagnostic that departs from the tool's real sequence tests a different program.
- Reading the tool's ACTUAL invocation — which is where
--likewas visible at all. The fix took ten minutes once the command line was read instead of imagined.
The law: any "same family ⇒ same structure" transfer must state which structural fact it assumes and CHECK it against both sides. Role-keyed transfer assumes the two overlays split their code identically; overlays differ, so verify per function, not per family. And when a tool deliberately drops an error class it cannot act on, it should still SURFACE it — a bare
gate-failrepeated 137 times cost far more than printing one line would have.
§92 — Conforming a DECLARATION: pointer changes are caller-neutral, scalar-WIDTH changes are not (Phase 29 SESSION-21, tools/conform_decls.py)
When a draft reaches match_one MATCH its signature is byte-TRUTH (§58b) and the fleet extern
is a stub-era guess, so conflicting types for func_X is fixed by moving the DECLS. That is right,
and it was the single dominant gate-failure class this session — 7 of 7 plain drafts and most of the
jtbl drafts. tools/conform_decls.py automates it: derive the signature from the draft's
DEFINITION, rewrite EVERY site, and assert completion (R32) because a half-rewritten axis is a
guaranteed break (§85), not a smaller win.
But "decls are free" is only true for some changes, and the difference is byte-measurable:
| change | caller-neutral? | evidence |
|---|---|---|
pointer type (s16*/short* → u16*) |
YES | func_80179B74: 1,600 sites / 523 files / 3 distinct forms conformed, banked, R22 140/140 |
return widen void → s32 |
only if no caller consumes it (§85) | func_8014D2A0, func_8012CC88 — precondition checked, then byte-neutral |
scalar WIDTH (s32 → u16) |
NO | func_80175DA8: decls reverted → gate PLUMBING; conform applied → gate DIFF |
arity ((void) → takes params) |
NO — breaks every 0-arg CALL | func_8015B950 by hand: ov_SC01_077 gated BYTE-IDENTICAL, 138 of 140 binaries broke |
The scalar-width case is the subtle one. Narrowing a parameter changes argument promotion at every call site, so the callers emit different code. The conform did not fix the draft — it changed the callers. A DIFF appearing only AFTER a conform is the signature of this: examine the callers (the §17a-1 pair — keep their decl compatible and cast at the call site), never the body.
The arity case is the expensive one, and the guard now names its price. func_8012AAAC needed
137 call-site casts to absorb it; func_8015B950 looked like it needed ~926 and in fact needed
ONE — its only 0-arg call is in an engine_core.h DEFINE macro body that the preprocessor
expands into all 926 TUs. Count the call SITES, not the expansions.
The law: a declaration edit is a T2 fleet-shared write set, provable only by R22 (§63/§85). The per-binary gate authorises the binary you built; it says nothing about the 139 you did not.
§93 — set -o pipefail attributes a pipeline failure to the LAST stage, not the failing one (Phase 29 SESSION-21, func_8014D820)
The c-rule is cpp | cc1 | maspsx | [jtbl_rodata_pads] | as under set -o pipefail. When cc1
exits 33, make reports the failure against the object whose recipe ends in as — so the error
reads as an assembler problem. I recorded func_8014D820 as an "assembler-stage failure" on that
basis and left it undiagnosed for hours. It was conflicting types for 'Ent' against
engine_types.h:434, fixed by moving three types to BLOCK scope (byte-neutral, and collision-proof
across all 138 member TUs).
The diagnostic, ~2 minutes: run the stages by hand and print each rc.
cpp … > t.i ; echo "cpp rc=$?"
cc1 … < t.i > t.s ; echo "cc1 rc=$?" # <- the real failure surfaces here
maspsx … < t.s ; echo "maspsx rc=$?"
as … ; echo "as rc=$?"
Then re-run the failing stage alone with stderr visible — cc1 names the file, line and symbol in
one sentence. This turned an opaque Error 33 into a one-line fix twice in one session
(func_8012AAAC's too few arguments, and this).
Corollary (§88e, earned): when handing a stuck function to another agent, pass the failure as what it IS — an undiagnosed observation — not as a named cause. I flagged this one explicitly as "never diagnosed, re-derive," and the agent found the true cause immediately. Had I written "assembler-stage failure" as established fact, it would have inherited my wrong search space.
§94 — A family sweep's 0/N is a TYPE-CARRY failure until proven otherwise: lift the exemplar's local types, ALL of them, transitively (Phase 29 SESSION-21, func_8016B6BC 0/137 → 137/137)
func_8016B6BC failed its family sweep at 0/137, reproducibly, twice — once in the 274-member
propagation batch and again after the §91 --like guard landed. It looked exactly like the §86
bimodal case ("some families template, some do not"). It was nothing of the kind.
§59(1) says a remap drops the exemplar's LOCAL struct types. It does — and the failure it produces names the wrong thing. cc1 reports:
`c' undeclared (first use this function)
`v' undeclared (first use this function)
`off' undeclared (first use this function)
Those are ordinary locals and they ARE declared in the remapped body. cc1 says "undeclared" because
it aborted the declaration block at an unknown TYPE (Prim_8016B6BC prim;) and every later
declaration in that block fell out with it. Read the first error, not the loudest one — and if a
variable you can see declared is reported undeclared, suspect its type, not its declaration.
The lift must be TRANSITIVE, and one round is not enough. Lifting the type the body names
directly (M8_8016B6BC) changed nothing: still 0/137. The full set was four, discovered by
following each definition's own references —
M8_8016B6BC → Prim_8016B6BC → Vtx_8016B6BC (named only inside Prim's body) → DVec_8016B6BC.
tools/lift_types.py --types A,B,C --apply lifts them to engine_types.h and strips the local
copies. Byte-gate the lift ALONE first (it must be neutral — it was, d19c9580… unchanged), then sweep.
Result: 0/137 → 137/137, zero failures. R22 clean-fleet 140/140.
The law: a sweep
0/Nis an INTEGRATION signal (§59), and the cheapest hypothesis is that the exemplar's body references a type only its own TU defines. Diagnose by splicing ONE sibling and reading cc1 directly (§93) — the sweep's own summary tells you nothing about cause. Then lift the transitive closure of the local types, not just the one the body mentions.Cost of not doing this: this family sat recorded as a bimodal "doesn't template" case across two sessions. The whole diagnosis, once pointed at one sibling's real stderr, took under an hour and was worth 137 members.
§95 — reconcile_tu dropped the SIBLING declarators of a multi-symbol extern line (Phase 29 SESSION-21, func_80176218)
reconcile_tu conforms a draft's DATA declarations to what the target TU can see — the right
question (§ the retired reconcile_decls asked the fleet, which has no single answer). Its rewrite
replaced the draft's declaration LINE with the TU's declaration of the conflicting symbol. But a
declaration statement can declare SEVERAL symbols:
extern u16 D_80078EB2, D_8011F82A, D_8011F82C, D_80078EB4, D_8011F8C4; /* only EB4 conflicts */
became
extern s16 D_80078EB4; /* four symbols GONE */
Why it was hard to see: the draft does not fail at the declaration. It fails later with
D_8011F82A undeclared (first use this function) — pointing at a USE, several conflicts down a
peeling chain, nowhere near the cause. I peeled four separate "next conflicts" out of this one draft
before dumping ALL cc1 errors in a single build and seeing three undeclared symbols that the tool
itself had removed.
Fix: group the plan by STATEMENT rather than by symbol, then re-emit every declarator — the TU's version for the ones that conflict, the draft's own for the rest — and note when a multi-declarator statement is touched. When the statement cannot be re-parsed, say so loudly instead of emitting only the planned symbols. After the fix the same draft reconciles 3 symbols rather than 2: the dropped declarators had been hiding a further conflict.
The law (R32, again): a transform that REPLACES a syntactic unit must account for everything that unit contained. Line-granular rewriting of C declarations is wrong by construction — the statement, not the line, is the unit, and a statement can hold N declarators.
Diagnostic worth reusing: when a draft fails in a chain, stop peeling one error per gate cycle. Splice it once and dump EVERY cc1 error — the shape of the whole set names the cause (three
undeclaredsymbols that share one original declaration line is not three problems).
§96 — The same rewrite, one shape down: reconcile_tu matched statements to lines by TEXT, so every COMMENTED declaration was silently skipped (Phase 29 SESSION-22, func_80176218 banked)
§95 fixed what the rewrite emitted (every declarator of the statement, not just the conflicting one). It did not fix how the statement was located, and that was the second half of the same bug.
cdecl.split_statements returns comment-stripped text with spans:
Stmt('extern u8 D_80078E78;' @ 4171:4193) # the line reads
# extern u8 D_80078E78; /* cur base ($s5) */
The rewrite re-found each planned statement by stmt.text.strip() == line.strip(). For any
declaration carrying a trailing comment that test is false, so:
- the declaration was left unconformed, while
- the use-cast pass (a separate loop, keyed off the same plan) still fired,
producing a draft whose uses are cast for the TU's storage against a declaration that still has the
draft's — i.e. a guaranteed conflicting types, reported by cc1 at the very declaration the tool
had just claimed to fix, with the tool exiting 0 and printing reconciled: N symbols.
Fix: rewrite by SPAN. split_statements preserves start/end precisely because drafts get
rewritten — its own docstring says so. The primitive was there; the code re-found the text instead.
And assert it landed (R32). The old code even had a dropped_check counter — incremented in two
places and never compared, which is the R32 "a loud failure nobody counts is exactly as invisible
as a silent one" shape in miniature. Now: declarators-in vs declarators-out, plus a per-symbol check
that each planned tu.declaration() is actually present in the output, each emitting a !! note
(so --strict exits non-zero).
Measured: func_80176218 went 3 → 4 data symbols reconciled and then banked whole-binary
(327 ins, ×138 = 45,126 templated instructions), after two gate cycles that had each reported a
different "next conflict".
The law, sharpened from §95: locating a syntactic unit and rewriting it are the same obligation. If you have spans, use them — re-finding a unit by its own text silently reintroduces every lexical difference (comments, whitespace, line breaks) as a miss. And when a transform can partially apply (decl not rewritten / uses rewritten), the partial state is worse than no-op, because it manufactures an error at a location the tool believes it already handled.
Corollary for the ladder:
reconcile_tuowns DATA declarations only (it skipskind == 'func'by construction). Function-decl conflicts in the same draft arecast_call_sites' axis (§20), and the two compose cleanly in either order — they touch disjoint symbol classes.func_80176218needed both: 4 data symbols conformed, then 2 callees (func_80177AD4void (int, unsigned int),func_80178298(u32*, u8*, short, short)) decl-conformed + call-site-cast to the draft's intended widths.
§97 — The gate's own tree hygiene: a refused carve, an unchecked recovery, and a snapshot that captured a dirty tree (Phase 29 SESSION-22)
A 15-draft harvest_verify --chunk 1 batch reported CC1-FAIL=4 and ended final SHA None. Three
of those four were manufactured by the harness. The chain, in order:
_okwas computed and ignored._jtbl_prep_onereturns(ok, snapshot); the caller tested onlysnapshot. Whenjtbl_carveREFUSES a table (§59(3): a non-contiguous same-subseg carve),okis False andsnapshotis None — so the draft was spliced and built without its carve, which cannot link. The resultingError 33was filed as CC1-FAIL, a codegen-flavoured verdict for a pure plumbing wall. Now classifiedCARVE-REFUSEDand skipped (also one build cheaper).attempt()does not restore on failure. It writes the candidate render and relies on the NEXT attempt's render to overwrite it — so the tree is dirty between drafts._jtbl_snapshot()snapshots the tree as it finds it. Combined with (2), a later draft's carve captured an EARLIER FAILED DRAFT'S SPLICE, and its undo then faithfully re-applied it — after the final_write(baseline)had already run. A run that verified nothing therefore ended with a body spliced on disk and unable to rebuild the binary, which reads exactly like a byte regression and is not one.- The recovery's own
make extractwas unchecked._shdoes not raise (§93's sibling), so a failed re-extract inside_jtbl_restorewould report nothing — a recovery that silently did not recover, strictly worse than not attempting one.
The invariant, one line: the gate's tree is at baseline except while a specific draft is under
test. Restore after every failed attempt (atomic branch AND bisect branch), and the snapshot can
never capture someone else's failure.
Plus an R32 coverage assertion on the cleanup itself: at 0 verified, a non-empty
git status --porcelain src/<bin> config is residue, not a result — print the files and the recovery
command rather than making the operator go looking.
Measured recovery of the false verdicts (same 4 drafts, same drafts dir, clean tree):
| draft | contaminated | true |
|---|---|---|
func_8013B83C |
CC1-FAIL | CARVE-REFUSED (§59(3) wall) |
func_801789AC |
CC1-FAIL | PLUMBING — conflicting types for func_801789AC (actionable) |
func_8017C974 |
CC1-FAIL | DIFF — corroborates its agent's global_alloc spill diagnosis |
func_80140958 |
CC1-FAIL | CC1-FAIL (genuinely its own, on a different object) |
final SHA went None → d19c9580 BYTE-IDENTICAL; tracked diff empty.
The law (R34/R35 again, pointed at the gate itself): the whole-binary byte-gate is a perfect oracle for did this draft match and a NULL oracle for what state did I leave behind. It cannot manufacture a match — no banked result in this session was affected — but it can manufacture a verdict, and verdicts are what the backlog and the roadmap are built from. A failure class is evidence about the compiler only once the harness is proven not to be the cause (§53's carve law, generalized from "sweep with the right tool" to "gate from a clean tree").
§98 — conform_decls had three defects, and only the third needed R22 to find (Phase 29 SESSION-22, func_8014CF04)
The decl-conform axis banked two drafts and gated BYTE-IDENTICAL on ov_SC01_077, then R22 came
back 139/140. Three separate defects, found in order by following the bytes:
1. The regex crossed newlines (mechanical).
rf"(extern\s+)?[A-Za-z_][\w \t\*]*?\b{fn}\s*\([^;]*\);" # [^;]* matches '\n'
Starting at a DEFINITION line it ran past the { and through the body to the first ;, matching
s32 func_8014CF04(s32 param_1, void *param_2, void *param_3) {\n register u8 *q __asm__("$17");
as one "declaration" and replacing both lines with a prototype — deleting the definition (and a
register pin) and producing undefined reference. Fix: [^;{\n]*, which makes a definition
unmatchable by construction. A multi-line prototype is then simply not matched, and the completion
assertion reports it instead of mangling it.
2. It rewrote inside COMMENTS (H5). Same runaway match. Fix: scan cdecl._mask(txt) — a
length-preserving blank-out of comments/strings whose offsets are valid in the original — and rewrite
by SPAN. (R33: that primitive already existed for exactly this.)
3. IT ASSUMED ONE SIGNATURE FITS THE FLEET — and this engine is loosely typed. The real cause,
and no amount of code-reading would have found it. ov_SC07_006 carries its OWN banked definition of
func_8014CF04 with a different byte-true signature — (s32, s32, void*) where ov_SC01_077
needs (s32, void*, void*) — beneath a local decl explicitly marked
/* de-macroized: per-overlay-local decl (byte-true sig); do NOT re-macroize */. Conforming that TU
to the fleet canonical produced conflicting types against the definition directly below it.
The rule: a TU that DEFINES the function owns its own declarations. A fleet-wide decl axis is meaningful only for TUs that CONSUME the symbol. Skip defining files whole — their definition is byte-truth there. This case grows more common as banking proceeds: every overlay that banks a function becomes a defining TU and therefore an exception.
And the assertion then cried wolf on its own by-design behaviour. With the defining TU skipped,
the R32 completion check counted it as a surviving non-canonical decl and reported
*** HALF-AXIS — DO NOT BUILD *** for a rewrite that was complete and correct. An assertion must
be exact about its DOMAIN, not just its condition — the §85 all-or-nothing invariant is over the
CONSUMING TUs. Scoped it; the axis reports complete at 1,747 sites with 1 file excluded by design.
Also hardened: PLAN → VALIDATE → WRITE. The refusal path originally aborted mid-write while claiming nothing had been modified — which is precisely the HALF-AXIS §85 calls a guaranteed break. The plan is now built and validated in full before a single file is written, so a refusal costs nothing and leaves the tree untouched.
Verified: 1,752 → 1,747 sites (the 5 differences were 1 definition, 3 comment lines, and the defining TU's own decl); definition + register pin intact; both drafts re-banked; R22 140/140.
The meta-lesson (R22's whole premise, re-earned): after fixing defect 1 I expected R22 to pass. It failed again at 139/140 for an unrelated reason, and an individual
make buildof the failing binary succeeded — because it reused objects the clean run rebuilds. An incremental pass does not refute a clean-tree failure. Every step of this diagnosis came from reading the real cc1/ld error after a genuinely clean rebuild, never from reasoning about what the tool "should" do.
§99 — The narrow-param wall is a DEF-side problem with a ZERO-blast-radius fix: convert the definition to K&R (Phase 29 SESSION-22, func_80175AB8 + func_80175DA8)
§92 said these two need "the §17a-1 caller pair, NOT a bare conform". The diagnosis was right —
conforming the fleet's void f(s32) declarations to the byte-true void f(s16) narrows the
parameter, changing argument promotion at every call site, so the callers emit different code
(§92 measured exactly that: PLUMBING before the conform, DIFF after). But the prescribed remedy
was the expensive one. The actual fix touches no declaration at all:
void func_80175AB8(param_1) /* K&R: the narrow param PROMOTES to int (C89 6.3.2.2), */
s16 param_1; /* so this is ALREADY compatible with the fleet's `s32` */
{ /* prototype — while still emitting the narrow-param codegen. */
That is §43 applied to the definition side instead of the declaration side. Cost: a draft-only
edit (T0, zero blast radius) versus a 524-site fleet conform. Both banked, d19c9580,
R22 140/140 — 57,822 templated instructions for two draft-local rewrites.
The law: when the byte-true signature has a NARROW scalar parameter and the fleet declares it wide, do NOT move the declarations. Move the DEFINITION to K&R and let C's promotion rule make the existing prototype correct. Conform only when the disagreement is a POINTER shape (caller-neutral, §85) or an arity/return change.
Two reconcile_tu bugs found underneath, one introduced while fixing the other
(a) It was blind to BLOCK-SCOPE declarations. split_statements is depth-0 by design, so for
a draft whose body is one function definition it returns exactly ONE statement and every declaration
inside is invisible — and §8d (scope_data_externs) deliberately demotes the data externs to block
scope. C still requires a block-scope extern to agree with a file-scope declaration in scope, so
the conflict is real: the tool printed reconciled: 0 draft(s), 0 data symbol(s); coverage defects: 0
for a draft cc1 then rejected with conflicting types for D_8011F7BC. Fixed by descending one level.
(b) DESCENDING INTO ANY { CORRUPTS STRUCTS — a bug I introduced with (a) and caught by diffing
the tool's own output against its input. Struct MEMBERS parse as declarations and get "conformed":
- u32 code; /* 0x04 */ -> typedef void (*code)(unsigned short*);
- p->code = *(u32 *)src; -> p->(*(u32 *)&code) = *(u32 *)src;
Guard: descend only into a FUNCTION body (a parameter list before the brace, and not a
typedef|struct|union|enum head).
(c) And it exposed a LATENT one: _cast_sub's regex matched a bare identifier, so it rewrote
member accesses as if they were the global. Broken since the tool was written; only reachable
once block-scope descent started finding such names. Guard: (?<![.\w])(?<!->).
Process note worth keeping: (b) was caught because the transform's output was diffed against its input before the result was trusted — not by a gate. The byte-gate would have reported PLUMBING and told me nothing about why, and the corrupted draft looked plausible.
§100 — Prefer the DRAFT-LOCAL fix: a type only one function uses belongs in its BODY, not in a shared header (Phase 29 SESSION-22, func_80175DA8 0/137 → 137/137)
func_80175DA8 swept 0/137. Per §94 that is a TYPE-CARRY failure until proven otherwise — and
it was: the draft defines typedef struct {…} Sp_80175DA8; at FILE scope, and remap_hseq templates
the BODY but not the type, so every sibling compiled without it.
§94's remedy is the shared engine_types.h lift (correct for func_8016B6BC, whose four types were
transitively referenced). But the cheap remedy was already visible in the same draft: it carries
typedef struct {…} S_AF634; at BLOCK scope, inside the function body, and that one templates
fine — because a type declared in the body travels WITH the body. So:
- the type is used by that function only (measured: 7 mentions, 6 inside the body, 0 elsewhere)
- → move it into the function body. Byte-neutral (
d19c9580unchanged), T0, zero blast radius - → re-sweep: 137/137, 0 failed.
versus editing a header included by 140 binaries, which then needs --strip/uniquify care (§64a),
a name that cannot collide fleet-wide, and an R22.
The law: scope the fix to the smallest unit that makes it travel. File scope in a draft is the worst of both worlds — it does not travel with the templated body, and it pollutes the TU. Lift to
engine_types.honly when a type is genuinely SHARED across functions or transitively referenced by another lifted type (§94); otherwise put it in the body (§59(1)'s "carry it in the template", which this makes concrete).
This is the same shape as §99, an hour apart: in both cases the cookbook's named remedy was the expensive fleet-wide one (524-site decl conform / shared-header lift) and the correct fix was draft-local (K&R definition / block-scope typedef). Two data points, one rule: before editing anything shared, ask what the smallest scope is that still travels with the body.
§101 — The STALE DEFAULT class: a guard whose cause was removed is a silent skip wearing a safety label (Phase 29 SESSION-22, three instances in one session)
Three separate tools were suppressing real, already-correct work — not by being wrong, but by defaulting to a protection whose cause had since been fixed, opt-out only if you remember a flag:
| tool | the guard | cause, and when it died | measured cost |
|---|---|---|---|
family_sweep |
skip any pinned exemplar (§42e cc1 SIGABRT) | the SIGABRT was extract_unit dropping file-scope macros — our bug, fixed in Phase 27 (_carry_macros) |
137 skipped, 133 of them bank |
family_sweep |
gate serially; the parallel farm is opt-in via --stage-only → sweep_parallel |
sweep_parallel.py was built in SESSION-20 after measuring the loss, and never wired to the caller |
~8-16× throughput; 3 sweeps (543 members) ran serially this session |
conform_decls |
REFUSE when 0-arg call sites exist | correct — but it printed the exact remedy and could not perform it | func_801789AC blocked two sessions; unblocked by --cast-zero-arg-calls |
The shape, every time: correct when written · the cause was later removed or the remedy became
available · the default never changed · and the tool reports the skip as a clean, quiet number
(skipped {'pinned-exemplar': 137}) that reads exactly like "nothing to do."
The law (R32/R35, pointed at defaults): a guard is a claim about the world. When the thing it guards against is fixed, the guard becomes a silent skip — and a skip is invisible in exactly the way R32 warns about, because nobody counts what a protection declined to attempt. When you fix a wall, grep for the guard that was erected against it and retire it in the same change. Phase 27's roadmap delta did say the PINS class was "back on the mechanical-harvest table" — the knowledge was recorded and the default still wasn't flipped, which is why this needs to be a mechanical habit and not a note.
Corollary — how to spot one: any run reporting a large skipped {...} bucket deserves the same
suspicion as a 0/N (§59, §94). Both are the tool declining to try, and neither is evidence about
the compiler.
§102 — A PLUMBING verdict can MASK a DIFF; and K&R is not always a codegen change (Phase 29 SESSION-22, func_8016EC0C)
Applying §99 to func_8016EC0C produced three results worth separating:
1. §99 GENERALISES — it dissolved a 1,617-site narrowing axis. conform_decls flagged
SCALAR-NARROWING (s32 → u8) across the fleet. Converting the DEFINITION to K&R promotes u8 →
s32, so the computed canonical became void func_8016EC0C(s32, s32) — the narrowing warning
disappeared entirely, leaving only a RETURN change (s32 → void) whose §85 precondition (no
caller consumes the return) was already satisfied. A caller-hazardous conform became a byte-neutral
one because of how the definition was written.
2. BUT K&R IS NOT AUTOMATICALLY A CODEGEN CHANGE — measure it, do not assume. K&R promotes, so
the CALLEE narrows (§43's in-place sll/andi tell) where ANSI assumes the caller did. That can
change bytes. Here it did not: match_one on both forms returned identical results — closeness
8, 88 ins, same residual. So the K&R rewrite was free. The rule is not "K&R always matches" nor
"K&R always differs" — it is: convert, then compare both forms against the target. (For
func_80175AB8/func_80175DA8 the K&R form is what banked; here it is byte-equal to ANSI.)
3. THE REAL FINDING: its "PLUMBING" verdict was HIDING a DIFF. The T14 census recorded
PLUMBING: conflicting types for func_8016EC0C, which reads as "recoverable, not a compiler wall".
It is not recoverable: with the decl axis clean the true verdict is SCHEDULE-REORDER, closeness
8, bucket permuter. cc1 reports the declaration conflict and never reaches the byte comparison,
so the plumbing error fires FIRST and the codegen verdict is never produced.
The law: PLUMBING is a verdict about the DECLARATIONS, never about the BODY. A census's PLUMBING bucket is therefore an upper bound on recoverable work, not a count of it — some entries are DIFFs wearing a plumbing costume, and you only learn which by clearing the plumbing and re-gating. Size a PLUMBING pool as "worth diagnosing", never as "worth banking".
Consequence for the backlog: func_8016EC0C is genuine permuter fuel (close=8,
SCHEDULE-REORDER) — unlike func_80176734/func_8017C974/func_80177B5C/func_80140958, which
measured structural and which the grinder's admission rule correctly rejects (§60/§60a).
§103 — A FILE-scope extern in a shared overlay TU is a GLOBAL constraint on every LATER function; move the DECL, not the draft (Phase 29 T48/T51, func_80135260 — the fleet-wide half)
func_80135260 needs its three per-location data symbols declared as 4-byte pointers
(extern u16 *D_x;). The canonical extern u8 D_x; + (*(u16 **)&D_x) cast makes gcc-2.7.2 CSE
&D_x into two callee-saved registers, which costs a 7th saved register and +3 instructions —
measured twice, two independent ways (reconcile_tu conform, and cast-at-use), both 139 ins against
the target's 136 with 123 mismatched. The draft could not be bent. But every host TU already
carried extern u8 D_x; at FILE scope, and a file-scope declaration constrains every later function
in the TU, so the byte-true block-scope decl was a conflicting types error.
The asymmetry is what makes this look like a compiler wall when it is a scope problem:
BLOCK(u16 *) ... then FILE(u8) -> warns, builds (the TU's own pre-existing mix)
FILE(u8) ... then BLOCK(u16 *) below it -> conflicting types (ERROR — the draft's position)
The lever: move the TU's OWN declaration down into its consumers. The engine is loosely typed (§16), so per-function views of a symbol legitimately disagree — the file-scope decl is the anomaly, not the block-scope one, and the original per-function sources declare these symbols at block scope in exactly this way. Moving it is declaration-only: every consumer keeps the identical declaration text, only its scope changes, so no access can change opcode.
Verify in two steps, always (T48's structure, and the reason the fleet application was safe):
- the decl move alone must rebuild the binary byte-identical — that proves it is declaration-only;
- only then splice the byte-true draft and gate it.
tools/scope_tu_externs.py is the tool — the TU-side complement to scope_data_externs.py
(§8d), which fixes the incoming draft. §8d has a documented give-up branch: when the TU already
declares the symbol at file scope, it drops the draft's own decl and lets the TU's type govern.
That is right when the types agree and fatal when they do not — it is precisely how 132 byte-true
siblings gate-failed while looking like a codegen wall. The two tools are halves of one lever:
| what collides | tool | move |
|---|---|---|
| the DRAFT carries a file-scope decl the TU never had | scope_data_externs.py (§8d) |
demote the draft's decl into the body |
| the TU carries a file-scope decl the byte-true draft must contradict | scope_tu_externs.py (§103) |
move the TU's decl into its consumers |
Derive the contested set, never hand-list it (R33): it is the DATA symbols the remapped draft
declares at block scope, intersected with what the target TU declares at file scope above the splice
point. --family does this per sibling in one command.
Refuse rather than skip (R32). Three conditions abort a symbol loudly: more than one file-scope
decl above the splice point; a file-scope statement below the decl that references the symbol (an
initializer has nowhere to move to); an unlocatable body brace. The rewrite then asserts its own
coverage as a delta — file-scope decls -1, block-scope decls +len(consumers) — because these
TUs already carry many legitimate block-scope decls of the same symbols, so an absolute
"at least one exists" check would pass vacuously.
Use cdecl, not a new regex. cdecl.split_statements yields depth-0 statement spans (a function
definition flushes at its closing }, so "ends with }" is a reliable is-a-definition test — a
column-0 test is not, because m2c emits goto labels at column 0 inside bodies), and cdecl._mask
blanks comments and string literals length-preservingly so offsets stay valid. Scanning raw text is
the false-positive class that made gather_externs accuse func_80135D20 on all 137 siblings and
garbled gen_harvest_targets' hints in Phase 19.
AUTOMATED (T53). The lever is now a jtbl_family_bank stage, ordered
raw → scoped → **tu-scoped** → recovered → reconciled. After the two non-invasive stages (it edits
the TU outside the spliced body) and before the two recovery stages deliberately: those bend
the DRAFT, and T48 measured both at +3 instructions for exactly this class, so they cannot succeed
here. The stage re-runs scope_data_fix against the scoped TU rather than reusing the raw body —
composition-correct, because the contested symbols no longer have a file-scope decl to be dropped
against while every other symbol is still handled normally.
The counterfactual, byte-gated on a reproduced blocker (ov_SC01_000, pre-T51 TU restored):
| stage | result |
|---|---|
raw |
compile error — conflicting types |
scoped |
compiles, fails the byte check — §8d dropped the draft's decl, so the u8 form's CSE costs the +3 |
tu-scoped |
BANKED |
That is the whole 133-sibling "wall", reproduced and dissolved in one build cycle.
Corollary — a stage that edits outside the spliced body must re-find the splice point. The stub offset is an index into the ORIGINAL TU;
tu-scoped's base is a rewritten TU, so reusing it splices at the wrong place. Each stage now carries its own base and re-searches. The pre-existing stages all passorig, where the re-search returns the identical span — so they are the same operation as before the refactor, by construction.
The law: when a byte-true draft and its host TU disagree about a symbol's type, the question is never "which type is right" — the engine has no single right type. It is "which declaration is in the wrong SCOPE." A file-scope decl in a shared overlay TU is a fleet-wide constraint that was almost certainly never in the original source; scoping it to its consumers is free, and bending the draft to it costs instructions.
§104 — Two silent-skip defects in one scan: match on MASKED text, emit from the ORIGINAL (Phase 29 T53, gather_externs)
family_remap.gather_externs decides which of the exemplar TU's file-scope externs to carry into a
templated sibling. It scanned raw text, which gave it two defects of the §96 class (matching by
TEXT rather than by what the compiler sees):
- A symbol named only in PROSE counted as a reference.
func_80135260's header comment mentions its siblingfunc_80135D20twice, so the scan demanded a declaration for it, found none, and printed "1 referenced symbol(s) have NO file-scope decl … the sibling will not compile" on all 137 members. It was wrong every time — T52 banked 132/132 against that warning. Two tasks read it as a possible cause before it was measured. - A commented-out
externcould be selected as the carried declaration and spliced into the sibling as live code. Not observed in the corpus; closed by construction.
The fix is a two-text discipline, not a smarter regex: MATCH on cdecl._masked text, EMIT by span
from the original. _mask is length-preserving, so offsets index both. A masked declaration is all
blanks — emitting from the mask would splice whitespace, which is why "just mask it" is a bug and
"mask for the search, slice the original" is the fix.
stmts = [(mask[s.start():s.end()], orig[s.start():s.end()]) # (what you search, what you emit)
for s in EXTERN_STMT.finditer(mask)]
hit = next((emit for searchable, emit in stmts if word.search(searchable)), None)
Verified as a no-op on output: 20 (exemplar, sibling) draft pairs across 4 families, old code vs new, 20 identical / 0 differing. The only behavioural change is that a false warning stopped firing.
The law: a warning with a high false-alarm rate is worse than no warning — it trains you to ignore the channel that reports real causes. This one fired on 137/137 and was right 0 times. When a diagnostic disagrees with a measurement, fix the diagnostic in the same session (R30), or the next reader pays the same tax.
§105 — A gate's revert must survive an EXCEPTION, not just a failure (Phase 29 T53, jtbl_family_bank)
jtbl_family_bank.bank() was carefully revert-on-fail: every return path restored the overlay, and
the stage loop even caught a stage that raises while producing a candidate. Nothing covered the
stages never being reached. A wrong exemplar made remap_hseq raise after the carve had rewritten
config/splat.<ov>.yaml + overlays.mk and jr_isolate had created a region file — the exception
propagated out, the revert never ran, and the tree was left with:
- a rewritten carve config (recoverable by
git checkout), and - an untracked region file, which
git checkout -- src/does not remove.
In a 132-member sweep that residue rides silently into the next member's build. Found by being bitten by it while testing something else.
The fix is a wrapper, not a bigger try inside: snapshot the region set, call the real body,
and on ANY exception revert and return a per-member exception status. Loud (it appears in the
tally), non-fatal (the sweep continues), and the tree is provably clean afterwards.
def bank(...):
keep = region_files(to_ov)
try:
return _bank(...)
except Exception as e:
revert(to_ov, keep_regions=keep)
return "exception", repr(e)[:140]
Negative-control proven: the same crashing invocation now reports {'exception': 2} and leaves
git status -- config/ src/ at 0.
The law: "revert on failure" is not the same property as "revert on every exit". Enumerate the exits — success, gate-fail, refusal, and the throw — and put the restore where all four pass through it. Same family as §97 (the gate's own tree hygiene), and the reason it matters more in a sweep than in a one-off: a one-off's residue is visible in the next
git status; a sweep's residue is consumed by the next iteration first.
§106 — Persist the MEASUREMENT, derive the POLICY: a stored route let a stale file out-vote the live table (Phase 29 T54, residual_class._ROUTE)
residual_class answers two different questions in one pass. klass is a measurement — it comes
from comparing two instruction streams and is expensive. (profile, bucket) is a policy — a table
lookup over klass that says which tool should work on it. autopsy persisted both to
.run/autopsy/residuals.jsonl, and verdicts() read both back.
That made the corpus authoritative for a decision the table owns, with two consequences:
- a route correction was inert. Editing
_ROUTEchanged nothing until someone re-ran the whole collect — so the fix and its effect were separated by an expensive step that is easy to skip. - a weeks-old row could silently contradict the live table, and nothing would ever report the disagreement (there is no oracle comparing a stored policy against the current one — R34's blind spot).
The fix: re-derive the route at read time from the stored klass + detail. The measurement stays
persisted; the policy is looked up fresh on every read.
The subtlety that makes this a technique rather than a one-liner: one route is magnitude-dependent.
LENGTH-DRIFT is permuter-shaped only when |delta| <= 2 AND explains == "tail" (§60b). A naive
re-derivation from klass alone would silently demote those rows. Both inputs are already in detail,
so route_for(klass, detail) reproduces the override exactly — verified at 1610/1610 against the
stored corpus with the table UNCHANGED, before the table was edited. Prove the derivation is faithful
first, then change the policy; otherwise a bug in the derivation is indistinguishable from the intended
change.
The route change itself (the reason this came up): ADDRESSING was routed to the permuter, which
contradicted residual_class' own bucket definition — "structural — local mutation CANNOT introduce
it … it wants a C-level idiom." The §10/§20 hoist-vs-remat shape is a multi-instruction change with a
documented deterministic recipe (gcc-2.7.2-map/cse_expr.md §2). Measured: both admitted ADDRESSING
targets plateaued under a §31-directed permuter, and the class was 32% of the entire admission pool
(18 of 56). After the fix: 56 → 38, exactly 18 rows changed, all ADDRESSING, nothing else moved.
And the bound, kept in the comment where the next reader will hit it: T31's finding 4 byte-tested
the §2 recipe on func_80132F40 across six variants and it never closed. structural here does not
promise a free fix — it means "a search over local mutations is the wrong tool; try the documented
idiom", exactly what WIDTH / BRANCH-POLARITY / IMM-OFFSET already mean.
The law: persist what was measured; derive what was decided. If a stored field can be recomputed from other stored fields plus a table, storing it converts a future correction into a silent no-op — and the staler the file, the more confidently it lies.
§107 — A lever wired into ONE gate path is a lever most families cannot reach (Phase 29 T56, func_80144090 0/136 → 136/136)
The §103 tu-scope lever was wired into jtbl_family_bank (T53) and nowhere else. But that tool only
runs for has_mid_jr families; every other family sweeps through family_sweep, which gates via
plain harvest_verify by design. So the lever was unreachable from the path most families use, and
the symptom was indistinguishable from a compiler wall: func_80144090 swept 0/136 with
conflicting types for D_800A651C.
Wired into family_sweep's staging, the same family banked 136/136, 0 failed — +20,944
instructions — with no change to the drafts at all.
Why this does not violate the sweep's plain-harvest_verify rule. That rule exists because
gate_stage's transforms perturb a correct draft (the §19/T3 finding). The tu-scope never touches
the draft — it moves a declaration in the target TU. Different object, different risk. The test
for "may this run inside the sweep?" is not "is it a transform" but "does it change the draft?"
Reuse the existing undo, do not invent one. family_sweep already snapshots TUs it edits at
staging time (--normalize-self-decls) and reverts them on two conditions: a final MISMATCH (the edit
was not byte-neutral) and a zero-bank group (§61's undo law — an edit that bought nothing gets
undone, or a git add -A commits dead diff). The tu-scope shares that dict, so it inherits both
backstops for free. A staging-time TU edit without those two reverts is how a sweep leaves residue.
Default ON, with an opt-out. It is byte-neutral by construction, a no-op when nothing collides,
and auto-reverted when it buys nothing — so gating it behind a flag only means most runs silently
miss it (the T24 --allow-pins precedent). --no-tu-scope exists to A/B it.
The law: when a lever fixes a class, ask which gate paths can reach it before declaring the class handled. A class is only handled on the paths that can apply the fix — and the paths that cannot will keep reporting it as a wall, in language identical to a real one.
Related, from the same task: scope_tu_externs refused N>1 file-scope decls of a symbol as
"ambiguous". Duplicate-identical externs are legal C, so N identical decls are one declaration
written N times — delete all N. Now it compares whitespace-collapsed forms and refuses only on a
genuine disagreement. (Measured on D_800B9A02: 3 decls in 2 different forms — so that one was
correctly refused, and the relaxation did not paper over a real conflict.)
§108 — Diagnosing a family 0/N: the four causes, and the third opt-in lever (Phase 29 T59)
A family sweep's 0/N says nothing about the code until you splice ONE member and read the compiler.
Do it like this — the shape of the output actively hides the answer:
make -j1 build/<the one .o> BINARY=<ov> # -j16 interleaves the real error out of reach
# then filter: '.c:' in line AND 'warning:' not in line
The memcpy / type mismatch with previous external decl warnings (§58) are noise from unrelated
TU positions and will dominate any naive tail. And set -o pipefail (§93) attributes the failure to
the last pipeline stage, so "Error 1"/"Error 33" names the wrong stage — read cc1's own lines.
The verdict split that matters is PLUMBING vs DIFF: a compile error is a declaration problem (recoverable, and each has a named lever); a clean compile with differing bytes is codegen. Five families diagnosed this way resolved to four distinct causes, only one of which is a wall:
| cause | signature | lever |
|---|---|---|
| shared-header signature conflict | conflicting types for func_X, "previous declaration" points at a DEFINE_func_*() macro line |
--fix-def-sig (see the bug below) |
| PsyQ/library symbol conflict | conflicting types for ApplyMatrixSV |
unresolved — the draft's carried decl vs the TU's |
| genuine codegen | compiles clean, bytes differ | permuter / §31 lookup |
| no matched exemplar | remap: no matched unit |
not a blocker — the family has no fuel |
Read the "previous declaration" line number before theorising. For func_8014D610 it pointed at
line 1727, which is not a declaration at all — it is DEFINE_func_8014D438(), a shared-macro
instantiation whose expansion forward-declares the templated function with the canonical
engine_core.h signature. cc1 reports the conflict at the macro's line. That one line identifies the
whole class.
THE --fix-def-sig BUG (why the lever did not fire). reconcile_def_sig rewrites the draft's
definition to the canonical header decl wholesale — types AND parameter names:
canonical : void func_8014D610(s32 a0, void *a1, void *a2)
draft body: ... param_1 ... param_2 ... -> `param_1' undeclared
Its docstring calls this a "rare name mismatch" that "the gate rejects, never a false bank". It is
not rare — an exemplar drafted with the param_N convention hits it every time, and the whole
family books as a compile failure. The fix is to conform the TYPES and keep the BODY's names;
both are already in hand at the call site.
The law: three times in one session a family-wide
0/Nwas a lever that was unreachable (§107), off by default, or subtly broken — never the compiler. Before diagnosing a family as hard, enumerate the levers the invocation actually enabled, then read one member's real cc1 output. A sweep's0/Nis a statement about the harness.
§109 — Conforming a definition to a shared header: fix the NAMES, then check the RETURN precondition (Phase 29 T60)
reconcile_def_sig substituted the canonical header decl wholesale — types AND parameter names —
while the body kept the exemplar's names. Its docstring called this a "rare name mismatch" the gate
would reject; both halves were wrong. An exemplar drafted with the param_N convention hits it
every time, and the outcome is not a rejected match but the whole family booking as a compile
failure, indistinguishable from a compiler wall:
canonical : void func_8014D610(s32 a0, void *a1, void *a2)
draft body: ... param_1 ... -> `param_1' undeclared (first use this function)
Fix: conform the TYPES, keep the BODY's names — both are in hand at the call site. Parse with
cdecl (base = return type, params = types, pnames = names), not a regex. Two traps in the
re-render: void* + a1 must become void *a1 (cdecl glues the stars to the type), and an empty
parameter list must be handed back verbatim — (void) and () both parse to params == [], and
they are different declarations (§99: () is the no-prototype form).
The fix is real but it is not sufficient, and the proof is that the verdicts MOVED:
| family | before | after | what is left |
|---|---|---|---|
func_8016163C |
param_1 undeclared |
DIFF | plumbing fully cleared; genuine codegen |
func_8014D610 |
param_1 undeclared |
void value not ignored as it ought to be |
the header is wrong |
func_80156044 |
unchanged | conflicting types for func_80155FF8 |
wrong lever — a CALLEE conflict |
The §85 return-axis precondition applies here too, and nothing checks it. Conforming a definition's return type to the canonical
voidis only safe when no caller consumes the return.func_8014D610's callers do — so the header'svoidcontradicts the byte truth, and conforming to it producesvoid value not ignored. The header is the thing that is wrong; changing it is fleet-shared blast radius.reconcile_def_sigshould test that precondition before promoting or demoting a return type, exactly as §85 requires ofconform_decls.
And a verdict that changes is the signal to re-route, not to push harder. One of these three is now a codegen question, one is a header-correctness question, and one was never the def signature at all. They shared a symptom, not a cause — three families, three levers.
§110 — A unit must define exactly ONE function, and "ends in ;" does not tell you which line defines it (Phase 29 T65)
extract_unit located a definition with "the line matches <type> func_<addr>( and does not end in
;". That is wrong whenever one line holds both a declaration and a definition — which the
handwritten inline-asm wrappers do:
extern void func_80156044(int, int); int func_80155FF8(int, int) { __asm__ … }
The line does not end in ;, so func_80156044 — which appears there only in the declaration —
was taken as a definition head. extract_unit lifted the neighbouring wrapper instead of the real
definition seven lines below, every sibling already defines that wrapper via its shared DEFINE_
macro, and all 137 members failed with redefinition of func_80155FF8. Read as a compiler wall.
Ask what follows the parameter list, not what ends the line. ; after the closing paren is a
declaration; {, or end-of-line (the brace-on-its-own-line form), is a definition.
Then assert it (R32): a unit that defines a function other than its target cannot template — the
sibling already has that function, so splicing can only produce redefinition of …. Refuse loudly
instead of handing the sweep a unit that fails N times.
Two traps in writing that assertion, both hit on the way:
_def_head_atalone over-fires. A call whose arguments wrap (iVar4 = func_X(a,/ newlineb);) has nothing after the(on its line, which the "end of line ⇒ definition" rule reads as a definition. It refused three families that had just banked 137/137.- The type-prefix test alone under-fires — it is what missed the wrapper in the first place, because the text before the name is a whole preceding declaration.
The predicate needs BOTH: split the prefix on its last ; and require the remainder to look like a
return type (this excludes call sites and admits the shared-line form), then check what follows the
parameter list. Regression-check any such assertion against families that are known to bank —
all five here extracted byte-identically before and after.
The law: an extraction that silently takes the wrong thing is worse than one that takes nothing, because its output looks plausible and the failure surfaces N members later wearing the compiler's clothes. Assert the shape you require; and when the assertion is about "is this a definition", remember C lets a line be two declarations and a definition at once.
§111 — The distinct-code metric is not noisy: a family pays it only if its members are byte-VARIANT (Phase 29 T66)
Seven family sweeps moved distinct-code by +125, +125, +129, and +0 four times. It read as a
metric bug for four tasks. It is not — it is the metric reporting something the instruction-weighted
number cannot.
weighted_metrics counts dedup_fns = len(matched_cls), where a class is one distinct h_exact
and is marked matched if ANY binary has it matched. So:
Δdistinct = (distinct
h_exactclasses in the family) − (classes already matched)
Exact on all seven, with no residual:
| family | classes | already matched | predicted | observed |
|---|---|---|---|---|
func_80135260 |
131 | 6 | +125 | +125 |
func_80133AB0 |
131 | 6 | +125 | +125 |
func_80156044 |
130 | 1 | +129 | +129 |
| four others | 1 | 1 | +0 | +0 |
The meaning. A family whose 138 members are byte-IDENTICAL is ONE piece of distinct code. The
exemplar's crack already reconstructed it; the other 137 banks are real (each binary now builds that
function from source instead of pasted asm, so fleet/instr-weighted pays in full) but they add
no new reverse-engineering. A byte-VARIANT family is ~130 genuinely different functions and pays
both.
So the two headline metrics rank the same work differently, and you can predict both before
spending a sweep — count the family's distinct h_exact across overlays:
cls = collections.Counter(sig[ov][addr] for ov in sigs if addr in sig[ov])
instr_yield = live_members * nins
distinct_yield = len(cls) - len(classes_already_matched)
Measured over the 49 currently-eligible non-jr families: 194,416 instructions (~1.48 pp) total, of which 13 families / 80,085 ins are byte-identical and pay ZERO distinct-code, and 36 families / 114,331 ins pay 2,962 distinct classes. Pick by which number you are trying to move.
The law: before calling a metric noisy, model it. Two behaviours with no identified variable is not noise — it is a variable you have not found. This one took a
Counterand ten minutes, after four tasks of writing "still unexplained, still not guessed at" in the log. Logging the anomaly honestly was right; leaving it unmodelled that long was not.
(And the derived-not-persisted rule from §106 applies to the table above: it is two lines of code over the sigs, so regenerate it — do not commit a ranking that rots the moment a family banks.)
§112 — A macro-scoped declaration only collides where the macro is INSTANTIATED (Phase 29 T67/T69, audit_header_sigs.py)
src/shared/engine_core.h declares, inside each DEFINE_func_M() macro body, the functions that
body calls. When such a decl disagrees with the byte-true definition, every family templating that
function fails conflicting types — ×137 members, wearing a compiler wall's clothes. Three were
found one at a time in Phase 29, each worth ~137 members. tools/audit_header_sigs.py finds them all
in one pass: parse every header decl, find every DEFINITION in src/**/*.c (via §110's
_def_head_at), and report only where NO definition agrees — one overlay disagreeing is loose
typing (§16/T49); all of them disagreeing means the header is the outlier.
Three preconditions decide whether a correction is safe, and two of them were learned by failing:
| precondition | why | how it was found |
|---|---|---|
| §85 return axis | a consumed return makes the change non-neutral | already known |
| ARITY | the macro's OWN call site passes the header's arity — correcting a (void) decl for a 1-param definition breaks it with "too few arguments" |
measured before the batch |
| VISIBLE COLLISION | a TU that carries its own incompatible decl AND instantiates the macro | a failed gate, 2/140 |
The third one is the subtle one, and getting it right took two wrong models:
- "Any disagreeing decl in
src/blocks it" — compares type SPELLINGS, sos32vsintandu32vsunsigned intcount as disagreements. Usecdecl.compatible(type IDENTITY), not==. - "Any INCOMPATIBLE decl in
src/blocks it" — still wrong, and it blocked all six corrections that had just gated 140/140 and banked 685 members.func_80161774has 1,063 TUs carrying the old spelling; correcting it was still byte-clean.
The fix is to measure the INTERSECTION, not the population. A macro-body decl is only visible where the macro is instantiated, so a collision needs a TU that does both:
colliding = [tu for tu in decls_of(fn)
if (tu.macros_instantiated & macros_declaring(fn))
and not cdecl.compatible(tu.decl, byte_truth)]
Validated against the known outcomes: the six that gated clean → 0 colliding TUs each; the one
that failed the gate (func_80147364) → 272. Perfect discrimination, and the finding count drops
61 → 32 once spelling noise is removed.
The law: a precondition that blocks work you have already proven safe is not conservative, it is wrong — and it will read as prudence forever unless you test it against known-good cases. Every gate you write deserves a control: run it against something that passed and something that failed, and require it to separate them.
§113 — An ARITY blocker only exists if the macro CALLS the function; an address-taken use has no call site (Phase 29 T72)
audit_header_sigs.py's ARITY precondition (§112) refuses to correct a header decl whose arity
differs from the definition's, on the grounds that the macro's own call site passes the header's
arity and would break with "too few arguments". True — when there is a call site.
func_80144B14 is declared void func_80144B14(void) and defined int func_80144B14(int param_1),
so the precondition blocked it. But the macro body does not call it:
*(s32 *)((s32)a0 + 0xDC) = (s32)&func_80144B14; /* address-taken, never called here */
No call site ⇒ no arity constraint ⇒ the FULL correction is available, not the §99 no-prototype
workaround. Applied: extern int func_80144B14(int param_1); — R22 clean-fleet 140/140
byte-neutral, then the family swept 137/137 with zero failures.
So the precondition must ask what the macro DOES with the symbol, not merely that both appear:
| use in the macro body | arity constraint? | correction available |
|---|---|---|
func_X(a, b) — called |
yes, the call passes the header's arity | §99 no-prototype () |
&func_X / (s32)&func_X — address taken |
no | full retype |
| declared but unused | no | full retype |
The law: a precondition derived from one usage shape will over-fire on every other shape. When a gate blocks something, check why the reason applies here before accepting it — "the call site would break" is not a fact about the declaration, it is a fact about a call site that may not exist. This one had 137 members behind it.
§114 — The THIRD decl axis: a CALLEE the draft declares differently from the target TU (Phase 29 T76/T77)
Three axes can make a templated family draft fail conflicting types, and the sweep pipeline handled
only two of them:
| axis | whose signature | lever |
|---|---|---|
| the draft's own definition | func_X itself vs the shared header |
reconcile_def_sig (§109) |
| DATA externs | D_XXXX |
scope_data_externs (§8d) + scope_tu_externs (§103) |
| a CALLEE | func_Y that the draft calls |
cast_call_sites (§17a-1/§20) — was not in the pipeline |
The third axis was the whole reason the byte-identical family tier measured 0 of 682 (T76). Three independent diagnoses had already named it and none of them was the family's own function:
func_80173A60 -> conflicting types for func_80173B4C (a callee)
func_8012F40C -> conflicting types for RotTransPers (a PsyQ LIBRARY symbol)
0x80143d28 -> conflicting types for ApplyMatrixSV (likewise)
cast_call_sites already solves exactly this — rewrite the callee's decl to the TU's canonical
signature (killing the in-TU conflict while keeping the symbol in scope) and cast each call site back
to the draft's intended signature; gcc folds the cast of a known symbol to a direct jal, so it is
codegen-neutral. It lived only in gate_stage, which the family sweep deliberately does not use
(§T3: gate_stage's transforms perturb a correct remapped draft). Wiring it into the sweep took
func_80173A60 from 0/135 to 135/135.
Two details that matter when wiring it:
- Build the canonical map from the TARGET sibling's TU, via cpp (
canonical_map(ov, src_file=tu)→cdecl.tu_scope). A raw-text scan cannot see a macro-injected declaration, andengine_core.his ~23.5k continuation lines inside ~1,800 macros — so the map would return nothing for exactly the callees that conflict (§51g LAW 7). - Read the TU after any TU-side edit is on disk, so the map reflects the environment the draft will actually meet.
The law:
conflicting types for X— read X. When X is not the function being templated, no amount of work on the definition's signature or the shared header will help, and both will look like plausible next steps. Three diagnoses named a callee before anyone checked whether the pipeline could act on one.
§115 — A func_XXXXXXXX predicate rots by design: the same name-form assumption in THREE places (Phase 29 T78)
cast_call_sites is the lever for the callee-conflict axis (§114). It could not see a named callee
at all — RotTransPers, ApplyMatrixSV, any curated symbol — because three separate places assumed
the func_XXXXXXXX form, and fixing two of them changed nothing:
| # | place | assumption | symptom when wrong |
|---|---|---|---|
| 1 | canonical_map |
re.fullmatch(r'func_[0-9A-Fa-f]{8}'), keyed by parsed address |
the callee is absent from the map → if fn not in canon: continue |
| 2 | DECL_LINE_RE |
(func_[0-9A-Fa-f]+) as the name group |
the draft's decl line does not match → never considered |
| 3 | split_sig_string |
\bfunc_[0-9A-Fa-f]+\s*\( |
returns None → if not csig: continue |
All three are silent skips that look identical to "no conflict found". With 1 and 2 fixed, the
symbol reached 3 and was dropped there — the sweep still reported 0/547, and only a trace of
transform's internals (callees cast: 0 while the canonical map clearly held
s32 RotTransPers(s32, s32, s32*, s32*)) located it. After all three: func_8012F40C 0/137 →
137/137.
Curated naming is something this project does MORE of as RE quality improves, so any
func_-only predicate is a rot-by-design defect — the same shape as stub_map's (Phase 26-A), where
a curated stub name read as "already matched".
The law: when a name-form assumption is wrong, grep for every place that encodes it before testing. A partial fix produces the identical symptom as no fix, so the negative result reads as "the hypothesis was wrong" rather than "the fix is incomplete" — and that is how a correct hypothesis gets abandoned.
§116 — Optimization level is a property of the FILE, not the function: read a family 0/N against the member's stub HOME (Phase 29 T79)
family_sweep --hseq templates the exemplar's C into each member's stub file. -O0-ness is applied
by the Makefile per object (WHALE_O0B_OBJS, O0_CLUSTER_OBJS, build/src/boot.o, …), so a
family whose exemplar was matched at -O0 banks only where the member's stub happens to live in an
-O0 object too. Nothing in the sweep, the remap, or the byte-gate reports the mismatch: the draft
compiles fine, and the gate correctly rejects 137 subtly-wrong bodies.
0x801457a4 swept 0/137 on exactly this. In ov_SC01_077 the definition sits in
ov_SC01_077_o0b.c (the whale -O0 object); in all 137 other overlays the same function's stub sits
in <ov>_after.c, which is -O2. Same C, same remap, wrong flag.
The tell, before spending a sweep: find the exemplar's home file and ask whether the Makefile
gives that object a non-default CC1FLAGS. If it does, check where the members' stubs live. This is
the same class as the Phase-29 Task-1 swing verdict (0x8013c964/0x8013c938 compiled -O2 by the
sweep and masked-MATCHing only at -O0) — that one was diagnosed at the compile-flag level; this
one shows the flag is really a file-placement question.
The fix moves the DEFINITION, not the stub — and here is why the obvious shortcut fails
A carved -O0 object whose .text ends exactly at the target function's vram can absorb that
function by appending it, so the linker places it at the same address either way. That makes
"just move the member's INCLUDE_ASM(...) line into the -O0 file" look byte-neutral by
construction. It is not even buildable, and the reason is splat, not the linker:
INCLUDE_ASM("asm/<ov>/nonmatchings/<ov>_after", func_X)resolves to a.sfile that splat only emits whilefunc_XisINCLUDE_ASM'd in that segment's own.c. Delete the line from<ov>_after.cand the nextmake extractstops generatingasm/<ov>/nonmatchings/<ov>_after/func_X.s— so the relocated reference in<ov>_o0b.cfails assembly withcan't open … func_X.s. Verified across all 133 overlays at once (Phase 29 T80).
asm/ layout follows the segment; object membership follows the .c file. Moving a stub
line changes the second while silently invalidating the first. ov_SC01_077 gets away with the
_o0b placement only because it holds a real definition there — nothing references a .s.
So the rollout is: stage the remapped body into <ov>_o0b.c and drop the INCLUDE_ASM from
<ov>_after.c in the same edit, then gate. That is a two-file atomic substitution, which
harvest_verify/family_sweep do not do (they substitute a draft for the stub in the stub's own
file), so this class needs its own small driver. Still prefer it over a splat re-carve — a re-carve
is the Arm-A wall (+0x20 data-symbol shift on 3 of 4 sampled overlays).
The law: a family-wide
0/Nwhose exemplar lives in a flag-overridden object is a build-graph statement, not a codegen one. Check the object's flags and the members' stub homes before routing it to the permuter or logging it as intrinsic.The corollary, learned the hard way: "byte-neutral by construction" is a claim about the linker. The build graph has other stages, and splat's asm generation is keyed to a different partition (segment) than the one you are editing (object). Build it before you call it neutral.
§117 — Spell the sibling's symbol from the SIBLING's address, not the exemplar's kind (Phase 29 T82)
family_remap.symbol_map builds {exemplar_symbol → sibling_symbol} by zipping the two functions'
relocation slots positionally. Phase 26-A had already fixed the exemplar side ("name the symbol by
what the address IS, not how it was loaded" — an address-taken function loads via lui/%lo and so
looks like data). The target side kept copying the exemplar's kind:
if ke == "call": m[f"func_{ae:08X}"] = f"func_{at:08X}"
else: m[f"D_{ae:08X}"] = f"D_{at:08X}"
m[f"func_{ae:08X}"] = f"func_{at:08X}" # <- target spelled from the EXEMPLAR
For a same-address family the two sides always agree, so this was invisible for 20+ phases. It only bites cross-address families, where the same slot can be a function in the exemplar and data in the member:
exemplar func_80174784 |
member func_8017CFD4 (ov_SC01_000) |
|
|---|---|---|
| callback slot | 0x801747CC — a function |
0x80182688 — data (D_80182688) |
| map emitted | func_801747CC |
func_80182688 ✗ |
The body then materialized a name for an address that is not a function, and the whole-binary gate
refused all 251 members. Fix: spell the target by what the target address is in the sibling's
overlay — func_ iff it is in that overlay's sig set (the same boundary oracle nins_of trusts,
R33). Result: 0x80174784 2/255 → 251/251, +242 distinct classes.
Why it survived so long: a MASKED oracle will MATCH a wrong symbol
rtu_match/match_one mask jal/HI16/LO16 so relocation noise does not drown the diff. A body
pointing at the wrong symbol therefore reports a clean MATCH (10 ins) while the fleet gate
rejects it. That combination — masked-MATCH + whole-binary DIFF — is the exact signature of a
compiler wall, which is how this was booked for so long.
The law: when a masked oracle says MATCH and the whole-binary gate says DIFF, suspect a symbol before suspecting codegen — masking is precisely what hides a wrong relocation target. And any positional exemplar→sibling map must derive each side's kind from its own side.
§118 — Ordinal (positional) immediate resolution: compare C tokens to the DIFFERING asm uses (Phase 29 T87)
family_remap.imm_map_tier1 substitutes a member's immediates by value over the whole body, so it
refuses (asm-ambiguous) when the exemplar uses the same literal at a position that differs AND at one
that does not — a by-value swap would corrupt the fixed one. The refusal is correct; the safety test
was too strict.
It compared the C literal's occurrences against every asm use of that value. But gcc synthesises uses that no C token names, so that comparison can never balance. The canonical case is an array index:
D_80187044[*(u16 *)((s32)a0 + 0x2)](); /* one C literal `0x2` */
emits two uses of the value 2 — the 0x2 member offset (per-member) and a fixed sll ..,2
for the 4-byte stride. One C token, two asm uses => permanently unresolvable.
The fix pairs C occurrences with asm positions in order, accepting either balance:
len(spans) == len(asm_pos)— every asm use has a C token; pair 1:1, rewrite only those indiff_idx.len(spans) == len(diff_pos)— the extras are implicit (compiler-synthesised); pair the C tokens against the differing uses only. A swap cannot corrupt what no token names.
Anything else still refuses. The order assumption (C literal order ~ emitted immediate order) is a heuristic, so the whole-binary byte-gate stays the sole arbiter (G3/P9) — a wrong pairing is rejected, never banked.
Measured: func_801599A4 0 -> 137 drafts, 137 banked (family 0x80131eec, +12 singletons =
149). Blast radius is small: only 9 of the other 144 immediate-refusals converted, so this is a
targeted lever, not a second §117 — the remaining still-zero families are blocked by something else.
The law: when a safety check counts asm occurrences against source occurrences, it must exclude the ones the compiler synthesises — otherwise the check is unsatisfiable by construction and reads as an unresolvable member forever.
§119 — Two levers on the SAME axis, opposite directions: test the off-diagonal (Phase 29 T89)
family_sweep carries two declaration-axis levers that pull opposite ways:
| flag | bends | correct when |
|---|---|---|
--fix-def-sig |
the definition -> the shared-header decl | the header is right and the draft's def is wrong |
--normalize-self-decls |
the in-TU declarations -> the definition | the definition is byte-true and a decl disagrees |
0x80161c98 needed exactly one of each: its byte-true def is (int, u32) (target emits sltiu), while
engine_core.h says (s32, s32) and a TU decl disagreed. So:
- both flags (T79):
--fix-def-sigrewrote the def tos32->slti-> byte DIFF. 0/138. - neither flag (T84/T88): correct
sltiucodegen, butconflicting types for func_80161D20from the in-TU decl -> will not compile. 0/138. --normalize-self-declsonly: def stays byte-true, the divergent decl is dropped and its calls cast. 138/138.
Three sweeps across three sessions read as a compiler wall because all three tested the diagonal of the 2x2 (both on, both off) and never the off-diagonal.
The law: when two levers act on the same axis in opposite directions, "tried it with the flags" and "tried it without" cover half the matrix. Enumerate the off-diagonal before calling the family blocked — especially when one lever's own docstring says it rewrites what the other preserves.
Measured: 0x80161c98 0 -> 138/138 (130 distinct); +23 across the remaining still-zero
families. Corollary to §118's caution: a per-family blocker can be a flag combination, not a defect.
§120 — Uniquify draft-defined TYPE names; and check which of N staging sites you actually patched (Phase 29 T93)
A remapped draft may define a local typedef struct {...} S_AF634; whose auto-generated name also
names a type at file scope in the sibling's TU. gcc-2.7.2 rejects the redefinition (C89 has no
"compatible redeclaration" escape for typedef names) and cc1 dies before codegen, so the family
reads as a wall.
canon_sig_reconcile._uniquify_draft_types is exactly this lever and lived only on the
--reconcile-raw path — the 5th "lever unreachable from THIS path" of the phase. Wired into the hseq
staging path: func_801759D8 CC1 FAIL -> MATCH (56 ins) -> 137/137 banked.
Do NOT "strip the duplicate typedef" — it breaks the extern that uses it
The obvious fix is to delete the draft's typedef since the TU already has one. That fails: the
TU's copy sits below the spliced function, so extern S_AF634 D_800AF634[]; inside the draft no
longer parses and you get D_800AF634 undeclared / used prior to declaration — which looks like a
second, deeper blocker and is really the first fix misfiring. Rename, don't remove.
The wiring trap that cost two attempts
family_sweep has three staging sites (edit-remap, hseq, plain h_norm) that share the
identical two lines:
d = os.path.join(REPO, SWEEP, ov)
os.makedirs(d, exist_ok=True)
Patching by rindex lands on the plain sweep, so an hseq run shows the draft unchanged and the
lever reads as ineffective. Anchor on something unique to the path — the hseq site's write is
func_{to_addr:08X}.c (cross-address), the others use func_{addr:08X}.c.
The law: before concluding a lever does not work, prove it ran — diff the staged artifact for the change the lever is supposed to make. A patch applied to a sibling code path is indistinguishable from a lever that does nothing.
Blast radius: 0 beyond this family (74 further families re-swept, none moved). Like §118 and unlike §117, this is a targeted lever — a path-reachability gap, not a logic defect.
§121 — Synthesise externs for macro-DEFINED callees from the macro's own definition head (Phase 29 T95)
gather_externs carries a referenced symbol's declaration by copying a file-scope extern line out
of the exemplar TU. A shared engine function defined by a DEFINE_func_X() macro has no such
line anywhere — the macro defines it. The member TU instantiates that macro too, but frequently
below the splice point, so a draft that uses the symbol as a value (a cast call site,
((void(*)(void))func_80142C84)()) fails with undeclared (first use this function).
Guessing the type is worse than not declaring it. A no-prototype extern s32 func_80142C84();
turns the error into conflicting types against the macro's real void func_80142C84(s32 a0) — a
different error, which is the useful signal that the guess (not the diagnosis) was wrong.
The fix: macro_def_sig_map() parses the #define DEFINE_func_X() \ <ret> func_X(<params>) {
heads in engine_core.h (1,878 signatures) and the sweep prepends
extern <ret> func_X(<params>); for any referenced macro-defined callee the draft does not already
declare. Byte-verified on func_80142B2C: undeclared -> (bad guess) conflicting types ->
MATCH (34 ins) -> 136/136 banked.
Note this is the complement of header_sig_map(), which reads the extern decls a macro emits for
its own callees. Two different macro-derived signature sources; a symbol in neither is a real gap.
Blast radius: 0 beyond this family — like §118 and §120, and unlike §117, a targeted lever.
§122 — GATE RAW BEFORE TRANSFORMING; the undo belongs to the WRITER, as a per-edit journal (P30 T0a, 2026-07-30)
The defect pair this closes (carried from SESSION-22, reproducible): (1) the gate_stage ladder
FAILED drafts that bare harvest_verify VERIFIED — func_8013B6A0/func_8013B598 (_o0) and
func_80138C60 (jr split), rtu_match confirming real-TU MATCHes. Every casualty lives in a
split TU; the plain-TU draft banked through the same run. Root-cause hypothesis (still open, now
harmless): the transforms take ONE batch-wide --src-file while the gate derives each draft's home
TU per-draft (Phase 26-A) — a mixed-TU batch gets decls reconciled against the wrong TU.
(2) A bare-gate workflow left fix_arity_callers --any-proto residue in 17 unrelated TUs — the
ladder's snapshot/undo existed only inside the ladder.
Law 1 — stage 0: gate the RAW drafts before ANY transform. A recovery ladder's transforms are
for drafts that FAIL as written; running them on everything lets a "recovery" regress a byte-correct
draft, and the gate then reports the regression as the draft's failure. With stage 0, the
destroyed-good-draft mode is impossible by construction — no root-cause required first (the
right sequencing under R35: neutralize, then diagnose). GATE_NO_STAGE0 restores the old order.
Law 2 — undo is the WRITER'S job, recorded per edit, not the orchestrator's file snapshot.
The snapshot needed two measured special cases (restore-shared-only on a partial bank because a full
restore reverts fresh splices, Task-14; restore-everything on a zero-bank run, §61) because a
file-level restore cannot tell the pre-pass's edits from the gate's splices. A per-edit journal
(fix_arity_callers --journal / --undo-journal --keep <banked>) round-trips each substitution's
literal text: exact by construction, immune to interleaved splices, uniform for shared+local files,
and available to EVERY workflow — ladder or bare. A decl someone else edited since is reported
MISSING loudly (R32), never silently skipped. Negative-control-proven: apply→undo → byte-identical
tree; --keep retains exactly the banked set. Bonus closed: the undo now runs AFTER stage 2, so a
stage-2 bank no longer loses its arity edit before its own gate attempt (the old latent parity gap).
Generalizes to: any pipeline where deterministic "fixers" precede a truth gate — the gate goes first on untouched input, fixers touch only failures, and every shared-state fixer journals its own writes. (Same family as §19/§25 canon-first and the §61 undo law; this entry is their composition.)
§123 — PROPAGATE A FAMILY WITH THE TOOL ITS TIER NEEDS: dedup_propagate is h_exact-only; its refusals are statements about the TOOL (P30 wave 1, 2026-07-30)
The trap, walked into and caught. Wave 1 banked 8 fresh cores in ov_SC01_077. Propagating each
via dedup_propagate --addr, four landed and four refused:
0x80133298 [skip] not self-contained — missing file-scope extern (CARRY-FIXABLE): Blk32L,m 0x80175820 [skip] not self-contained — missing file-scope extern (CARRY-FIXABLE): D_800AE7BC,D_800AF634 0x8015FBE0 [skip] 1 reach<2 0x8016E9EC [skip] 1 reach<2
Read naively that is "two need an extern carry fix, two aren't shared." Both readings are WRONG, and
the second is absurd on its face — the family map says 138 members each. The actual fact:
dedup_propagate defaults to --tier h_exact — byte-IDENTICAL bodies. All four families are
diff_class PURE/IMM: members differing by relocations/immediates (h_seq). For an h_seq family the
h_exact reach genuinely IS <2, and the liftability check genuinely does fail — because the tool is
answering a question about a tier these functions do not belong to. The right tool is
family_sweep --hseq (remap per member). The four that DID propagate had byte-identical siblings.
The law (the §53 carve-law, generalized from the CARVE axis to the TIER axis): a propagation
refusal is evidence about the tool's tier, never about the function — check diff_class in
.run/family_hseq.json BEFORE routing, and never let a refusal message's vocabulary
("not shared", "not self-contained") name the function's property. §53 said sweep a family with the
tool its exemplar needed; this says propagate a family with the tool its TIER needs. Same failure
shape, different stage, and it is the shape that manufactured the "families bank ~0%" doctrine.
Routing table (memorize this, it is the whole entry):
family diff_class |
tool |
|---|---|
| byte-identical members (h_exact) | dedup_propagate --addr |
| PURE / IMM (reloc/immediate drift) | family_sweep --hseq |
has_mid_jr: true |
jtbl_family_bank.py (§53 carve) |
| exemplar in an -O0 TU | the -O0 rollout path (§116), not either sweep |
Corollary on the CARRY-FIXABLE label: it is real for h_exact bodies (hoist the externs/typedef to file scope above the def and it lifts), but seeing it on an h_seq family means you asked the wrong tool first — fix the routing before fixing the externs.
§124 — A "not matched" verdict can mean the definition is there under a DIFFERENT C NAME: the asm-label alias blind spot (P30 SESSION-28, func_8016191C ×137)
Symptom. A family sweep reports no matched unit for func_XXXXXXXX in <ov> and skips every member —
even though corpus.stubs says the exemplar is not a stub (i.e. the invariant says it IS matched).
The two statements look contradictory; they are both true.
Cause. family_remap.extract_unit looked for a definition head literally named func_<ADDR>.
But a body whose byte-true signature conflicts with the fleet-canonical declaration on both §73 axes
at once (return AND params) is banked zero-touch by the §37 asm-label alias — the C identifier differs
and a GNU asm label binds the emitted symbol:
extern void func_8016191C(void *a0, s32 a1); /* the fleet canon, in engine_core.h */
int aF8016191C(int param_1, unsigned int param_2) __asm__("func_8016191C");
int aF8016191C(int param_1, unsigned int param_2) { ... }
extract_unit matched nothing, returned None, and every caller reads None as "not matched."
Measured (P30). That single blind spot was the ENTIRE no matched unit skip class: one exemplar ×
137 same-address members, 24 ins each = 3,288 ins, every member still an INCLUDE_ASM stub and
otherwise sweep-ready. After the fix: 137 banked / 0 failed, R22 140/140. Fourth consecutive time a
"structural" residual resolved to our own tooling (R35).
The fix, and the two traps inside it.
- Resolve
<ident>(...) __asm__("func_<ADDR>");→<ident>and accept that as the definition head. Re-derive the pattern PER FILE — one file's alias must never leak into the next file's scan. - Carry the alias DECLARATION into the unit. Without it the sibling TU emits the symbol
aF8016191Cand the body never lands atfunc_<ADDR>— it would link, build, and be wrong. Let the preceding-decl backscan walk past the alias line (so the function's own externs are carried exactly as for a plain definition), then guardstart <= alias_ln <= endagainst double-emitting it. - R32: an alias declaration with no findable definition must refuse LOUDLY. Falling through to
_macro_unitre-reports "not matched" — the exact silent skip the fix exists to delete.
Do NOT "fix" this at the source. The tempting alternative is to widen the shared header
(extern void → extern s32), delete the alias, and rename the definition back. That addresses only
§73's RETURN axis while the decl and body also disagree on PARAMS, so it still conflicts — and it is a
T2 fleet-shared edit where the alias is T0 draft-only. Fix the reader, not the source.
The general law. corpus.stubs (the invariant) and a source scanner can disagree, and when they do
the scanner is wrong — the invariant is derived from the build, the scanner is a text model of it
(R33). Any place that turns "I could not find the text" into "it is not matched" is a silent-skip
defect waiting to be measured.
§124a — a family sweep's 0 matched-exemplar families may be a FILTER, not a wall
family_sweep --hseq defaults to --band substantial. A mid/tiny family therefore returns
[hseq] 0 matched-exemplar families (band=substantial); 0 candidate members — which reads exactly like
"nothing to do here." Pass --band all (or the family's band) before concluding anything. Same
shape as §53 (the missing carve) and §116 (the wrong opt level): a 0 from the wrong invocation is not
evidence. Check the band the family map assigned before you spend a probe on the residual.
§125 — Split the CARVE from the BODY before calling a jr residue a wall — and measure it by SHA from a CLEAN tree (P30 SESSION-28; this section's first draft was WRONG and the method caught it)
Three jr targets refused the whole-binary gate. I ledgered all three as tooling walls on the strength of a body-free "carve-only" probe. Re-measured properly, two of the three verdicts were false and the third had a different cause than I recorded. The method below is sound; my instrument was not. Both halves are the lesson.
The method (keep this)
Run the carve on a sibling with no body spliced at all, then rebuild:
tools/jtbl_carve.py $OV --func $FN # carve ONLY — no draft, no remap
make extract BINARY=$OV && make build BINARY=$OV
# byte-identical -> carve is neutral; the failure is the TEMPLATED BODY
# diverged -> the failure is the CARVE; the body was never fairly tested
It separates two failures that present identically at the gate, and it costs one build.
The instrument rules that make its answer trustworthy (this is where I failed)
- Compare the built SHA against
config/check.<ov>.sha. Do NOT grep the build log for[ OK ]. A log-grep cannot distinguish "wrong bytes" from "the build did not get that far", and it silently inherits whatever stale state the tree is in. - Re-extract after EVERY config change AND after every revert.
git checkout -- config/alone leavesbuild/holding objects from the carved config — the next build then links a mixture and reports a divergence that is purely your own. (Phase-20's R22 corollary; §42b's stale-object trap. I reproduced it exactly: a reverted config with no re-extract turned a byte-identical overlay into[FAIL] got 8f28aa77 / want 38a3d919.) - A driver that aborts a target MUST revert that target before the next one. v1 of my chain
continued without reverting;config/overlays.mkis SHARED, so target 1's half-applied isolate was still in the tree while target 3 was measured. Every verdict after the first abort is suspect. - Verify the baseline against the canonical SHA too, not just "it built". "Identical to the previous build" is worthless if the previous build was already wrong.
The corrected results (each SHA-verified, from a clean tree, restore re-verified)
| target | jr_isolate_all |
jtbl_carve |
true verdict |
|---|---|---|---|
func_8018057C / ov_SC01_009 (897 ins) |
NEUTRAL | not reached | my "isolate breaks bytes" was FALSE; the original failure is not reproducible |
func_80191C50 / ov_SC06_018 (710 ins) |
NEUTRAL | DIVERGED | REAL — the carve genuinely breaks bytes here, after a neutral isolate |
func_8017BEBC / ov_SC04_004 (group B, 13 members) |
n/a | NEUTRAL | carve is fine ⇒ the failure is the BODY/template, the OPPOSITE of my first claim |
So the tidy story I wrote first — "two apparent walls are one tooling problem" — was wrong. They are two different problems, and the third target has no demonstrated problem at all.
Two further notes worth keeping
jtbl_carveon ov_SC06_018 refuses loudly when run without the isolate: "subseg would host NON-CONTIGUOUS .rodata carves (0xab9f4 and 0xaba4c) — a single object can't leave a gap for the unmatched jtbl between them." That refusal is the documented §81/§8b instruction to run step 1 first — it is the tool working, not failing. Do not confuse a loud refusal with a byte divergence.- Ledger the STAGE, not the function (
JTBL-CARVE-BREAKS-BYTES), so one instrument fix reopens every target it covers — but only after the stage is verified by rule 1–4 above. A ledger full of misattributed classes is worse than no ledger: it schedules the wrong repair.
The meta-lesson
§53 warns that a 0% from the wrong tool manufactured a doctrine that steered two phases. This is the same failure one level up: a verdict from the wrong measurement manufactures a wall just as efficiently. R35 says fix the instrument before trusting its measurement — and my own diagnostic script is an instrument, subject to the same rule as the tools it audits. The saving grace is that the method in this section is what refuted the section's own first conclusion, one build at a time.
§126 — The carve-within-a-carve: an ADDRESS RANGE is not an OPTIMIZATION REGION (P30 T2, byte-proven end-to-end)
A 4th -O0 region was found inside an -O2 jr split (0x80183CF0..0x80184920, ov_SC03_014 +
ov_SC03_015). Banking it needs the containing object sub-split into pre/-O0/post — the
"carve within a carve" the roadmap had flagged as blocked on the Arm-A splat %lo +0x20 defect.
It is not blocked. Four probes, each isolating exactly one variable, SHA vs config/check.<ov>.sha
from a clean tree:
| probe | isolated | result |
|---|---|---|
| 1 | sub-split at arbitrary addresses, everything still -O2 |
BYTE-NEUTRAL — the re-carve does not shift %lo; Arm-A does not bite |
| 2 | same split, middle region routed -O0 |
diverged (two variables changed at once — inconclusive) |
| 3 | probe-1's name, only the -O0 flag added |
diverged ⇒ the FLAG, not the subseg name |
| 4 | -O0 regions cut to EXCLUDE matched bodies |
BYTE-IDENTICAL — route proven |
The finding: opt level is per FILE, so the file's contents must be opt-HOMOGENEOUS
§116 says opt level is a property of the FILE. The corollary nobody had needed until now: when you
select a region by address range, you get everything in that range — including functions that are
already MATCHED, whose bodies expand from engine_core.h as DEFINE_func_*() instantiations and
are compiled -O2. Flipping the file recompiles them, and they stop matching. Here two matched bodies
(func_80184440, func_801848E4) sat interleaved among the 15 -O0 stubs. Cut around them —
[lo..matched), matched stays -O2, [after..hi) — and the image is byte-identical.
So the region bound is: (address range) MINUS (already-matched bodies), and a range with K
interleaved matched functions needs K+1 -O0 sub-regions, not one.
The instrument trap that hid it (and it is §124's shape again)
I first derived "15 contiguous -O0 functions, clean cut" by scanning asm/<ov>/nonmatchings/**/*.s
for the frame-pointer prologue (addu $fp,$sp,$zero / 21F0A003). A MATCHED function emits no
.s — splat writes none, because its .c carries real C. So that scan is structurally blind to
precisely the bodies that break the flip, and it reported a clean run where the range was mixed.
Derive the region's contents from the SOURCE anchors (INCLUDE_ASM stubs and DEFINE_func_*()
instantiations, in address order), never from an asm-file scan. corpus.stubs gives the stubs; the
DEFINE_func_ instantiations in the region .c give the matched ones.
The mechanics
- Cuts:
jr_isolate_all'splan()/build_new_config()already accept arbitrary cut vrams — region naming is purely positional, so nothing new is needed for the split itself. Inject the cut list and reuse its source-repartition, carve-repoint and ascending/unique validation verbatim. - The one-carve-per-region law still applies: every already-banked jr in the object must ALSO be
a cut, or two carve owners share one object and its single contiguous
.rodatamust host both. - Naming + the Makefile: name each
-O0sub-region<ov>_o0<letter>and let ONE widened rule select them — the glob is now$(wildcard src/ov_*/ov_*_o0?.c)(was_o0b). A missed-O0rule is SILENT: the region compiles-O2and every residual it produces is a pure artifact (§116).corpus.o0_sources()parses this rule and resolves?via glob, so the-O0oracle stays honest. - Verify the routing, don't assume it:
corpus.is_o0("src/<ov>/<ov>_o0c.c")must return True before you read a single residual from that region.
Method note
Probe 2 changed the name and the flag and was therefore uninterpretable. Probe 3 — same name as the proven-neutral probe 1, flag only — is what produced the answer. One variable per probe, and keep the previous probe's proven-neutral configuration as the control.
§126a — a bare except: continue around a coverage-asserting oracle re-creates the silent skip (P30 S28)
Sizing this cluster, I reported "275 open stubs across 18 overlays". The true figure is 2,184 across 138 — an 8× under-count that would have mis-scoped the whole task.
The scan was:
for ov in overlays:
try: st = corpus.stubs(ov)
except Exception: continue # <-- the defect
and it ran while make extract-all was rebuilding in the background, so corpus.stubs() hit its
R32 coverage assertion ("N stub(s) have NO .s on disk — the tree and the source disagree") for most
overlays. The bare except turned every one of those loud refusals into a silent skip, and the loop
happily reported a total over the ~18 overlays that happened to be re-extracted already.
Two rules, both already ours, both violated at once:
- A measurement taken during a rebuild is not a measurement. Earlier the same session the same
mistake was caught because
corpus.pyrefused — the assertion worked. Here I wrapped the assertion inexcept: continueand threw its answer away. - R32 lives in the CALLER too. A coverage-asserting oracle only asserts coverage if the caller
lets it raise.
except Exception: continuearound it is precisely the silent-skip class R32 exists to delete — reintroduced one level up, where no audit looks.
Practice: in any scan that will SCOPE work, let the oracle raise. If some binaries legitimately have no data, filter them by an explicit predicate you can state, and print the count you skipped and why. A total is only trustworthy if the denominator was asserted.
(It also cost credibility in the other direction: I used the bad number to call the T0(f) pin of "2,192 open members" STALE. The pin was right. Re-derivation is only worth more than a carried number if the re-derivation is sound — R35 applies to the re-measurement as much as the original.)
§127 — The -O0 regime: the CONSTANT-OFFSET FOLD, and why -O0 needs its own idiom set (P30 T3 wave, 15 targets)
The -O0 population is now large (the P30 T2 routing put 2,200+ open stubs into -O0 TUs), and
a 15-target wave against it showed the cookbook is written almost entirely for -O2. The index
fired on only 3 of 15 targets, and multiple agents INDEPENDENTLY re-derived the same idiom — the
signature that a body of knowledge has a hole (§124's lesson applied to the index itself).
The idiom they kept re-deriving: the constant-offset fold
At -O0, gcc-2.7.2 folds a constant struct offset into the load's displacement but will NOT fold
an indexed one:
p->f /* -> lbu $v0, 3($s0) — offset folded into the load */
p[i] /* -> addiu $v0,$s0,..; lw 0($v0) — address computed FIRST, then a 0-displacement load */
So when the target shows lbu 3(reg) you must write the member access; when it shows
addiu + a 0-displacement load you must write the indexed form. At -O2 these converge and the
distinction is invisible — which is why nothing in §1–§126 covers it.
The rest of the -O0 regime (write PLAIN C, and mean it)
- Frame-pointer prologue
addu $fp,$sp,$zero(21F0A003) is the detector (§6/§116); every check needs--o0or you chase a phantom SIZE-MISMATCH. - Every local is spilled to the frame and reloaded at each use. A
0x18($fp)spill/reload pair is a real named local, not a compiler temp — declare it. - Load-delay
nops and redundantaddu rd,rs,$zerocopies are normal; do not "clean" them. - Do not hand-optimize, do not add temporaries to help the compiler. At
-O0the C maps almost 1:1 to the asm; the usual-O2steering levers (pins, live-length dials, statement reordering) are mostly inert and mostly a distraction.
§127a — §71 (sibling-first) is the strongest -O0 lever, and it beats the index
Several targets fell immediately to an already-banked sibling in the same TU. func_80184868
matched off the exact shape banked hours earlier in the same _o0d file:
s32 ret; ret = func_8001ABBC(0, 0, &D_xxxxxxxx, 0, 0); return ret;
Before drafting an -O0 function, read the banked functions in its own _o0* file. An -O0 TU
is a near-uniform code regime, so a sibling's shape transfers far more reliably than at -O2.
§127b — the knowledge was in a SOURCE COMMENT, not the cookbook
Two agents reported their decisive levers came from the header comment at the top of
src/ov_SC01_077/ov_SC01_077_o0.c, not from docs/. That comment is real, hard-won knowledge sitting
where only someone already editing that file will find it. When a lever is discovered in a source
comment, promote it to the cookbook and leave a pointer — otherwise every future agent pays to
rediscover it, which is exactly what happened here across 12 of 15 targets.
§128 — A raw NUL in C source makes grep SILENTLY SKIP the file (P30 S28, 137 files)
Symptom. grep -rn func_XXXXXXXX src/ returns nothing for a function that is plainly defined
in src/. corpus.stubs says it is not a stub (so: matched). The build is byte-identical. Every
statement is true and they look contradictory.
Cause. The file contains a raw NUL byte, almost always a control character written straight
into a character literal — the source reads == '<NUL>' where it should read == '\0':
if (*(char *)(p + 4) == '\0') { ... } /* correct */
if (*(char *)(p + 4) == '<NUL>') { ... } /* compiles the same; file is now BINARY */
file(1) reports data instead of C source, and grep treats any file containing NUL as binary
and prints nothing — no warning, no error, exit 0. The file disappears from every grep-based audit
and every hand search, invisibly.
Why no existing gate catches it. The byte-gate is structurally blind here (R34): the compiled
bytes are correct, so it has nothing to say. cc1 accepts the literal. check-all stays green.
The defect lives entirely in the readability of the source to tooling — a dimension no byte oracle
measures. Same family as §124 (a scanner that cannot see something reports it is not there) and
§126a (a bare except swallowing a coverage assertion), one layer lower: in the tool everyone
reaches for first.
Scope when it was found: 137 files. Every _o0c/_o0e region created in one session — a
templated body carried the NUL, so a single defective source propagated it fleet-wide in an
afternoon. A defect that is invisible to grep is also invisible to the review that would have caught
it spreading.
The oracle: tools/audit_text_sources.py / make audit-text-sources, in tools-health,
coverage-asserted over every tracked .c/.h (R32). Fix is '<NUL>' → '\0', then re-gate to
prove byte-neutrality (it is, but prove it).
§128a — a negative control must corrupt a SCRATCH COPY, never the tracked file
Proving the new guard fires, I injected a NUL into the real tracked file and restored it through
nested shell escaping. The restore left '\\0' — an escaped backslash, a multi-character constant,
not a NUL. That IS a semantic change, and R22 duly failed 139/140. Repaired and re-verified 140/140;
nothing was ever committed.
The rule: test a guard against a throwaway copy under .run/. A negative control that mutates
the artifact it is validating can introduce the exact defect it exists to detect — and the more
convincing the control, the more dangerous the restore. Corollary, learned the same minute: a
verification pipeline ending in grep -c PATTERN exits 1 when the count is 0, so the SUCCESS
case reports failure. Read the output; an exit status is not the oracle (§125's rule 1, again).
§129 — Post-carve, rtu_match/match_one COUNT THE JUMP TABLE AS INSTRUCTIONS; and a carve must never be committed without its owner (P30 S28, func_8013BD74)
Two independent traps, both hit banking one reach-138 jr function. Neither is a compiler wall.
§129a — the target instruction count is INFLATED after a carve
jtbl_carve moves the function's jump table into a dotted .rodata subseg, and splat then emits the
function's .s with a leading .rodata section holding the table, followed by .text:
.section .rodata
dlabel jtbl_801D828C <- 28 entries
enddlabel jtbl_801D828C
.section .text
glabel func_8013BD74 <- the actual 198 instructions
rtu_match reported mine=198 ins, target=226 ins, 206 mismatched — a catastrophic-looking DIFF.
226 − 198 = 28, exactly the table's entry count: the tool counted the data words as
instructions and diffed the body against them.
So a post-carve verdict from rtu_match/match_one is meaningless. A draft that verified
cleanly before the carve will read as a total mismatch after it, and the number will look like
evidence of a deep codegen problem. Verify the body pre-carve; after the carve, let the
whole-binary byte-gate arbitrate (it always was the arbiter — §52b). This is §81's warning one
step further on: §81 says match_one masks relocations so a jr fn's MATCH is not a bank; §129a says
that after the carve its DIFF is not a diff either.
§129b — never commit a carve whose owner is still a stub (it strands the carve)
harvest_verify refuses to run on a dirty tree (§97), and the carve necessarily dirties config/.
The tempting resolution — commit the carve, then bank on a clean tree — creates a stranded carve:
a .rodata carve piece with no matched owner. jr_inventory refuses immediately (R32):
committed .rodata carve ownership is not 1:1 (R32/R33) — a stranded/duplicated carve: [('UNOWNED', '0x801d828c')]
That is a coverage oracle correctly rejecting a state the commit created, and it blocks every later jr operation on that overlay until reverted.
The route for a jr function is the INTEGRATED one: tools/jtbl_family_bank.py, which does
carve → extract → remap → whole-binary gate per sibling inside one uncommitted transaction and
reverts on failure. The §81 hand-chain is for diagnosis; it is not a banking path, because its two
constraints (carve-before-bank, clean-tree-to-bank) pull in opposite directions.
The real blocker underneath, for the record
With the carve applied, splicing the draft fails cc1 with
jtbl_rodata_pads: more rodata .align directives than pad specs (2) — table-count drift vs the carve
— §8e's pad-spec filter failing loud, as designed: the object's committed spec
(0,4 tables=+0x0,+0x70) does not account for the table the newly-matched function emits. That is
genuine §8e work (re-derive the multi-table pad spec including the new owner), not a wall — and it
was only reachable after §129a stopped the phantom 206-instruction "diff" from misdirecting the
diagnosis.
§130 — An INCREMENTAL build can report BYTE-IDENTICAL for a change the CLEAN build cannot even LINK (P30 S28, the jr pair)
R22 has always said "verify from a clean rebuild." This is the sharpest instance yet of why, and it
cost two cycles because I used a fast in-loop gate that skipped make clean.
The setup. Two jr/switch functions, func_8013B83C (jtbl_801D8254) and func_8013BD74
(jtbl_801D828C), both in the SAME -O0 object ov_SC01_077_o0. Both bodies are byte-correct —
match_one --o0 and rtu_match --o0 each report MATCH (272 and 198 ins).
What the incremental gate said. Carve both tables in one jtbl_carve call (correct — it is
additive and re-derives the full span; the object's pad spec becomes [0,4,4]), splice both drafts,
make extract && make build → BYTE-IDENTICAL. I reported both banked.
What make clean said.
ov_SC01_077_o0.c:(.text+0x10f8): undefined reference to `$L105'
ov_SC01_077_jr_801588CC.o: in function `func_801596F0':
undefined reference to `func_8013C938'
It does not link at all — and note the second error: a previously matched cluster function becomes undefined. An incremental build reused objects that still satisfied those references; a clean one has nothing to reuse and the real state surfaces.
The rule, sharpened. A byte-gate result from an incremental build is not weak evidence — it can
be actively false, and false in the most convincing direction (a green SHA). §42b named the
stale-object trap for a FALSE FAIL; this is its mirror, a FALSE PASS on a change that is not even
linkable. Anything that touches config/ (a carve, a resegment, a split) MUST be gated by
make clean && make extract-all && make check-all before it is believed, let alone reported.
The residual class this exposes: two jr functions matched in ONE object. §8b already warns that a
single object contributes at most one contiguous .rodata run; the pad-spec machinery (§8e) handles
a multi-table span in principle, but matching both owners in the same -O0 object produced
unresolved local labels ($L105) from the C-emitted tables plus a collateral undefined symbol. The
integrated per-sibling path (jtbl_family_bank) banks ONE jr function per object per transaction and
has never hit this. Ledger class: JR-PAIR-IN-ONE-O0-OBJECT. The escape, untested, is §81 step 1:
isolate one of the two into its own code subseg first so each object owns exactly one table.
The diagnostic ladder that finally located it (reusable)
The function-level tools all said MATCH, so the signal had to come from the image:
- Build with the splice, build without, diff the two binaries.
- Compare each differing byte's vram against the function's own
[lo, lo+4*nins)range. Here: 3,749 of 3,791 diffs were OUTSIDE the function, first diff near the overlay's START, and the image was 57 bytes LONGER — the §8 signature of.rodatafloating to the front, i.e. "this function emits a jump table", not "this function's code is wrong." That size-and-location fingerprint distinguishes a codegen residual from an integration/layout effect in one build, and it is what redirected the diagnosis away from three wrong guesses.
§131 — The jtbl OVER-SPAN: sltiu N is ground truth in BOTH directions, and the zero-word rule only guards one (P30 S28, func_80191C50)
The carve had a symmetric blind spot. jtbl_range computes end = the next data dlabel, then:
- extends the span when the owning function's
sltiu Ndemands more entries than the dlabel supplies (§SPLIT-TABLE REPAIR — spimdisasm can cut one table in half); - trims trailing words that are zero, on the axiom "
0x00000000cannot be a jump target"; - warns when the span is shorter than an unambiguous
sltiubound.
Nothing handled a span that is too LONG for a non-zero reason. spimdisasm attributes to a dlabel
everything up to the next dlabel, and that remainder is not always zero — it can be ordinary data.
Then no trim fires, the carve reserves more words than the table has, the object supplies only the
real entries, and the .rodata piece under-fills.
The fingerprint (this is the reusable part). Under-fill does not look like a codegen bug:
image size: −4 (×N tables), often masked to −3 by the end-align TRIM
differing bytes: hundreds, in hundreds of 1-byte runs, spread over most of the overlay
position: ~95% at byte 0 (mod 4) — the LOW BYTE of a 16-bit immediate
value: every one changes by exactly −4
That is not "the function is wrong", it is every %lo in the image pointing 4 bytes low because a
data symbol moved. Bucket the differing bytes by offset % 4 and decode a few words: if the deltas
are uniform and small and land in the immediate field, you are looking at a layout/under-fill
problem, not codegen. (Measured here: 812 of 853 at pos 0 mod 4, all −4.)
The fix, and its authorization. Clamp end down to start + 4*N when the sltiu bound is
unambiguous (exactly one — a multi-switch function cannot say which table owns which bound) AND
the surplus words are not plausible code addresses. If any surplus word is in the overlay's text
range, REFUSE and say so: it might be a real entry, and silently dropping one corrupts the image
in the opposite direction. Same standard as the extension path — act only on the program's own
statement, never on a guess.
Why it mattered: this was the single instrument failure that survived §125's retraction round — the one case where "the tool is broken" was actually true. It blocked a 710-instruction behemoth and, because a carve is per-overlay, it would have blocked every future jr family whose table happens to be followed by non-zero data. One clamp, byte-identical, behemoth banked.
§132 — The JR-PAIR-IN-ONE-O0-OBJECT "wall" was TWO instrument defects: a merged-double span the carve could not see, and a truncated object no rule deleted (P30 S29, func_8013B83C + func_8013BD74)
S28 ledgered a new residual class: two jr functions matched in ONE -O0 object produced
undefined reference to $L105 + undefined reference to func_8013C938 from a CLEAN build, while an
incremental build reported BYTE-IDENTICAL (§130). The escape recorded was §81 step 1 (isolate one into
its own code subseg). Both the class and the escape are REFUTED. Neither function is on a compiler
wall; both banked from a clean fleet with no isolation at all. Class RETIRED — the fourth
consecutive "structural wall" to resolve to our own tooling (§124/§125/§126/§131 are the others).
Defect 1 — a pre-§8e MERGED DOUBLE is not a single-table predecessor
jtbl_carve reconstructs a touched span's table starts from the union {new fn's .s, surviving stub
.s, the committed tables=, --span-tables, --like}. When a span has no JTBL_PADS line, the
tool infers "SINGLE-table predecessor, so the span start IS the table start" (§8e-2, correct for the
case it was written for). ov_SC01_077_o0's carve at 0xb01a4 predates the tables= persistence and
is a merged double — func_8013C0F8 ($L75) and func_8013C414 ($L105), the second invisible
because a matched owner's stub .s is pruned by make extract. So the tool derived 3 starts where the
object emits 4 tables, wrote JTBL_PADS := 0,4,4, and jtbl_rodata_pads refused mid-stream —
correctly, with the exact message ("more rodata .align directives than pad specs").
Fix (single choke point, spec_from_starts): assert the starts EXPLAIN the span, and recover what
is missing from the payload — every zero word INSIDE the span is an original .align 3 pad (the same
axiom the pad rule already rests on: a zero can never be an ENTRY), so the word after it STARTS a
table. Recovery is a no-op wherever the structure is already known, so committed-green spans are
untouched. Honest limit: only PAD-SEPARATED boundaries are recoverable; a tight (0-pad) interior
boundary is indistinguishable from a continuing table in the payload — but it then makes the spec
SHORT, which the filter rejects loudly at build time. Never silent in either branch.
Defect 2 — as writes a corpse and nothing deletes it
as consumes a PIPELINE. When an upstream stage dies mid-stream, as has already assembled the
prefix and written a truncated .o (its only complaint is Warning: missing .end at end of assembly). make reports Error 1 correctly — and then leaves that object on disk, NEWER than its
.c. The next build considers it up to date and links it. Measured here: 12 of 16 T func_
symbols, and undefined $L57/$L59/$L63/$L75/$L76. That is the entire mystery: the link error is one
build DOWNSTREAM of a loud, correct, attributable compile error. Fix: .DELETE_ON_ERROR: in the
Makefile (negative-control-proven: make: *** Deleting file ...). Same family as §42b/§130 — an
artifact that outlives the command that failed to produce it.
The fingerprint, and the 30-second ladder that found it
An undefined $L<n> in a LINK error is never codegen. $L labels are gcc-local: the assembler
resolves them within the object, so one can only be undefined if the stream that defined it was cut.
Read it as "an object is truncated", not "a function is wrong".
The ladder — compile the ONE TU standalone through the real pipeline and let it name the owners:
cpp … | cc1 -O0 … | maspsx … > t.s # no filter, no carve: one variable
grep -n '^\.section \.rodata' t.s # how many tables does the object ACTUALLY emit?
# for each hit, the nearest preceding `.ent` is that table's OWNER; `$L<n>:` is its label
Four blocks (13/27/27/27 entries) against a 3-start derivation, and $L105 attributed to
func_8013C414, in one command — before any build, carve or isolation. Then confirm against the
ORIGINAL payload: table starts 0x801D8254 / 0x801D828C / 0x801D82FC / 0x801D836C, each preceded by a
zero pad word, span 0xb00fc..0xb0280 = 388 B = 52+4+108+4+108+4+108 — the arithmetic closes exactly,
so the C-side table count and the payload agree and the spec is 0,4,4,4.
Result: func_8013B83C (272 ins) + func_8013BD74 (198 ins) banked in ov_SC01_077
(d19c9580), R22 clean-fleet 140/140, with the 137-sibling sweep unblocked (their _o0c spans DO
carry tables=+0x0,+0x70, so the recovery is a no-op there — the blindness was ov_SC01_077-only).
The transferable rule
A fail-loud guard is only as trustworthy as the artifact hygiene around it. A guard that refuses correctly but leaves a partial artifact behind converts its own honest error into a lie one build later — and the lie is more convincing than the truth, because it points at a different subsystem. When a loud refusal and a mystifying downstream failure appear in the same session, suspect they are the SAME event, one build apart.
§132a — --like is for a sibling with NO record; against one that HAS a record it over-derives (P30 S29, ov_SC07_010)
The 2×137 sweep banked 136/137 twice and failed on the SAME overlay both times
(consumed 3 rodata .align(s) but 6 pad spec(s) given). Cause: jtbl_carve --like <exemplar>
transfers the exemplar span's table structure and matches donor→recipient by the subseg's ROLE
NAME (the name with the overlay prefix stripped). ov_SC07_010's -O0 region is named _o0 —
the same role as the exemplar's, and the only other overlay so named; the other 136 are _o0c,
whose role never matched, which is the only reason the sweep worked at all. The exemplar had
just banked two more owners than this sibling has, so the transfer unioned its rebased 4 offsets
with the sibling's real 2 plus the new table: six starts for three emitted tables.
The rule: a sibling's own committed tables= is AUTHORITATIVE; --like exists for a span with
no record. Suppress the transfer when a record exists (jtbl_family_bank.like_arg, deriving the
subseg via jtbl_carve.func_subseg — the same derivation the carve uses, R33). The
incomplete-record case that --like used to paper over is now covered honestly by §132's payload
zero-word recovery, so nothing is lost by preferring the local record.
The shape worth remembering: the transfer was wrong for 2 of 138 overlays and inert for the
other 136 — so a sweep can be 99% green and still be running a defective rule. A per-sibling
failure that repeats on exactly the same sibling across two independent sweeps is a property of that
sibling, not noise — probe it rather than ledger it: here it cost one probe and returned 2 banks
plus a real tool defect. And note the near-miss: had ov_SC07_010 been named _o0c like its 136
peers, the defect would have stayed invisible until some later family whose exemplar shares a role
name with a recorded sibling — a silent wrong spec instead of a loud refusal.
§132b — When the span's already-matched owner is ITSELF multi-switch: --span-rel (P30 S1, func_8014032C)
§132's payload zero-word recovery closes the pad-separated case. Here is the case it cannot see, met head-on while sweeping the zero-crack tier's largest family (183 ins × 137).
The sibling gate-failed. The §132 ladder named it in one command: the object emits four tables
because both functions in it are multi-switch (8+5 entries each), while the carve derived three
starts. The missing start belongs to the already-matched func_8013FFD8, and neither oracle can
reach it — make extract prunes a matched owner's stub .s, and its second table abuts its first
with no pad (8 entries = 32 B ≡ 0 mod 8, so .align 3 emits nothing).
Note what is NOT wrong: the true pads [0,0,4,0] are exactly what natural alignment produces. The
build breaks only because a short spec gets written and then enforced.
The lever: jtbl_family_bank --span-rel d0,d1,… — table offsets relative to the FIRST NEW table,
which func_jtbls reads from the sibling's own .s. The family's layout is invariant (same code,
same entry counts; only the base moves), so one measured offset list serves every sibling. Verify
that invariance on ≥2 siblings before trusting it — one command, and it is the whole premise.
It is a per-family FLAG, not a default, and the failure is symmetric. The same sweep needed it ON for 127 siblings and OFF for 10: where the span holds only the new tables, forcing the extra starts over-specifies and the carve refuses ("table starts … do not fit the span"). Two passes — sweep with it, re-run the residue without — banked 137/137. A residue that fails the opposite way from the majority is a signal to flip the flag, not to add another lever.
§133 — The DEFAULT-FILTER class: three times in one session, a tool silently answered a narrower question than the one asked (P30 S1–S3)
Not one of these was a broken tool. Each was correct for its own purpose, and each silently scoped down a question that was asked more broadly — which is the same defect class as §124/§126/§132, one level up: not a wrong measurement, a mis-scoped one.
- My own analysis filter. Sizing the remaining ×138 work I required
nins >= 80("substantial"), reported "only 2 crackable fleet-wide families remain — the ×138 era ends", and wrote it into the phase plan as a structural signal. Re-run without the size cut: 33 families / 238,478 templatable ins, of which 30 (137,186 ins) were below my line. The conclusion was an artifact of a threshold I chose and then forgot I had chosen. worklist.mdprices byh_exactreach. A per-location PURE family shares onlyh_seq, so the worklist prices its head at ×1. Byte-proof: the day's banked pair was priced 272 and 198 ins and delivered 37,536 + 27,324;func_80176734, the single largest item in the frontier, sits at rank ~50 there. Rank family work by.run/family_hseq.jsontemplatable weight; the worklist is correct only for genuinely h_exact-reach functions.family_sweep --hseqdefaults to--band substantial. A propagation run over nine freshly banked heads — all under 80 ins — reported0 matched-exemplar familiesand banked nothing. The sweep was right; the band was the question it had been asked.--band allis the fix.
The practice. When a scan returns "nothing" or "far less than expected", the FIRST hypothesis is your own filter, not the world. State the filter out loud in the same breath as the number ("33 families with ≥100 members, any size"), and re-run once with it removed before any conclusion is allowed to shape a plan. A number that scoped a phase deserves the same instrument-check R35 demands of a probe.
§134 — MULTI-LINE BLINDNESS: one root cause, four faces, in family_remap's preamble scanner (P30 S6, 190 zero-crack families)
extract_unit builds a templatable unit by walking UP from a definition line, accepting lines that
"look like" preamble (extern / // / /* / * / typedef / blank). That scanner is line-at-a-time
over a language whose constructs span lines, and every construct that wraps was misread. Four faces,
all found in one session by probing eight zero-banked families with ONE build each (.run/s6_diag.py
— remap → splice → build that overlay → read the compiler's own error → revert):
| face | the construct | what the scanner concluded | the symptom, in the sibling |
|---|---|---|---|
| D1 | a block comment containing { — e.g. * => { u16, u16, s32 } |
the {-guard (a real T65 fix for extern void f(int); int g(){…} on one line) fired on documentation |
carry stops MID-comment; the draft opens with * … and an unmatched */ → parse error before 'the' |
| D2 | a wrapped DECLARATION — extern void func_801466F0(s32 a0, …,⏎ s32 sp8); |
_def_head_at's fallback: "param list continues past this line ⇒ ANSI definition" |
line 457 accepted as a definition head → unit is a 16-line fragment with no body (a neighbouring DEFINE_func_*(), an #include, a comment), closed by the { u16, … } inside that comment → 0/137, reads as a compiler wall |
| D4 | a wrapped asm-label alias — extern void aF…(…,⏎ s32 sp8) __asm__("func_801466F0"); |
_alias_decl_for is a single-line rx.match |
the definition is invisible → "no matched unit" |
| D5 | a multi-line typedef struct {…} T; |
the backscan meets the CLOSING line } T; first; } is not an accepted prefix, so it stops |
the type never travels → T undeclared, measured at 17 families / 24,332 templatable ins |
Why it matters more than four bugs. D2 and D5 produce a silent, uniform, whole-family failure — exactly the shape a compiler wall produces. The tell is bimodality: S6's first sweep banked 842/2,735, and the residue split 57 families all-banked / 52 zero / 8 partial. Per-member codegen residuals do not cluster like that; one per-family blocker does. Bimodal bank rates are a tooling signature — probe one member before writing the family off (R35, and the fifth consecutive "structural wall" in this project to resolve to our own instrument after §53/§124/§126/§132).
The fixes (all in tools/family_remap.py, all blast-radius-verified against the pre-fix tool by
diffing extract_unit output over every affected exemplar — 157 of 181 byte-identical, the rest
changed only in the intended direction):
_def_head_at(ln, paren_idx, more=())— the caller passes the following lines, so a wrapped list closes and the same;-vs-{test answers correctly. With lookahead supplied and no close, it returns False ("unknown" is not "definition"); with no lookahead it keeps the historical answer, so un-updated callers cannot silently change behaviour.- the
{-guard exempts comment-only lines (*,//, or an unclosed/*), plus an R32 backstop that drops-and-announces a preamble that still opens inside a comment. - the forward body scan counts braces in
cdecl._masked text (R33: one masking oracle), so a brace inside a comment or string can never close a function body. _typedef_block_start()— on a} T;closing line, walk up to the matchingtypedef(both thetypedef struct {and brace-on-its-own-line forms) and carry the WHOLE block. Only blocks that literally begin withtypedefare carried: a struct VARIABLE (struct {…} g;) closes identically and defining it in the sibling would be a duplicate global. Safe becauseharvest_verifyalready strips a typedef the sibling TU provides (cdecl.strip_provided_typedefs), so a duplicate cannot break the sibling — and the whole-binary gate remains the sole arbiter (G3/P9).
Measured payoff, one command each: D1+D2 recovered +323 members from families that had banked
zero; D5 then banked the 139-member func_8012B77C family 139/139 (8,062 ins) and took
func_80128C98 to 137/275. S6 total: 1,582 members off a population that the pre-fix tool scored
at 842. D4 was measured (1 exemplar / 3,288 ins, with a second blocker behind it) and deliberately left
UNFIXED — but it now returns None, so the sweep reports it as skipped {'no matched unit for func'}
rather than failing 137 times. A known gap that announces itself is not the same defect as a silent
one (R32).
The general practice. Any scanner that classifies C by reading one line at a time is wrong on wrapped declarations, wrapped attributes/asm-labels, multi-line typedefs, and comments containing braces — and its failures will look like walls, not bugs. Give it lookahead, mask comments/strings before counting delimiters, and when a family banks 0/N with every member failing identically, read one compiler error before believing the compiler.
§135 — Six byte-verified gcc-2.7.2 idioms from the P30 S6f-h waves (and the two-lane wave shape that found them)
Distilled from ~100 agent-drafted functions gated whole-binary across three waves. Each idiom below CLOSED a specific residual — none is a hypothesis.
The codegen idioms
-
UNSIGNED switch index ⇒ pure equality chain (no range test).
switch (*(s32 *)(p+0x30))over cases {0,1,2,3} emits a balanced tree with ansltirange split;switch (*(u32 *)(p+0x30))emitsbeq 1 / beqz / beq 2 / beq 3with no comparison at all. For an unsigned index the case-0 leaf satisfiesnode_has_low_bound(0 == TYPE_MIN), soemit_case_nodesdrops the bound test. If the target's switch has no range check, read the index UNSIGNED. (func_8017C910, 92 ins.) -
ARRAY_REFvsINDIRECT_REFchanges ALIASING, therefore scheduling. Writing a field read asa0[0x46]makes it an ARRAY_REF, which setsMEM_IN_STRUCT_P; gcc-2.7.2'strue_dependence()then DROPS the dependence between an in-struct varying load and a not-in-struct constant-address store, and the load hoists. Writing the same read as*(s16 *)((s32)a0 + 0x8C)keeps it a plain INDIRECT_REF and restores the dependence. A 4-instruction "scheduling residual" that is really a type-form choice. (func_8017CFE0.) -
A constant store whose top bit is set in the STORED width needs an UNSIGNED destination.
*(u16 *)p = 0x8C00emitsori(viaforce_fit_type, the value stays positive);*(s16 *)p = 0x8C00folds to sign-extended −29696 andliemitsaddiu. If the target materializes the constant withori, the destination is unsigned. (func_8018A0F4.) -
The list scheduler PRESERVES the relative order of disambiguable stores. A store written late in source SINKS to the end of the block rather than hoisting into load-delay slots. Two field-zero stores had to be written ABOVE a three-halfword copy to land at the target's indices — 9 mismatches → 0, no permuter. When a store lands too late, move it earlier in SOURCE. (
func_8017CAB4.) -
A
shortloop counter blocks strength reduction on array indexing.for (s16 i…) p[i].f0emits a sign-extend + multiply chain; walking explicit pointers (p++,q++) reproduces the original's biv/giv set. The 1998 source walked pointers. (func_8017C738, 96 ins.) -
Frame size off by a constant ⇒ DEAD LOCALS, not a codegen bug. A draft that is structurally exact but yields a
0x40frame against the target's0x50shows everyspdisplacement off by exactly0x10— the original declared locals ahead of the live ones. Add the padding declaration. The tell is that ALL diffs aresp-relative immediates of one constant delta. (func_8017FD58.)
The integration idioms (these decide whether a byte-correct draft BANKS)
match_oneMATCH ≠ a bank. It compiles standalone and cannot see the TU's other declarations. Measured across the three waves: 83% → 93% → 71%(+reconcile to 89%) of agent MATCHes survived the whole-binary gate. Always finish on the whole-binary gate (G3/P9).- cc1 reports only the FIRST conflict, so a draft can look one edit away and hold three more. One draft had invented prototypes for six symbols the TU declares, two BELOW the splice point. grep the whole TU for every symbol the draft names, in one pass.
- An interior address has no symbol.
D_801DA0F0does not exist — it is offset0x6CintoD_801DA084. Alui/addiupair can build an INTERIOR pointer; declaring the interior address as its own extern givesundefined referenceat LINK, not a compile error. Find the containing symbol in the data.sand index into it. (func_8017C5F0.) - Never redeclare a C-library name. A draft's own
memcpyprototype collided with the TU's — which declaresmemcpythree times with incompatible signatures. (func_8018A860.)
The wave shape that produced these
Two lanes, and the reconcile lane is the reliable one (12/12 across two waves).
- Draft lane: cheap tier (Haiku ≡ Opus at ≤~50 ins, ~4.8× cheaper —
cheap-tier-ab-validated), Opus direct ≥90 ins, Opus escalation in between. 17 of wave-1's 20 banks were Haiku. - Reconcile lane: for every gate failure, the orchestrator captures the compiler error first and embeds it. Agents cannot run the gate, so without the error a declaration conflict reads to them as a codegen wall — S29's law, re-confirmed. With it: 12/12.
- Between waves, do all three: paste args from a DERIVED manifest (never typed — a hand-transcribed path list cost wave 1 three agents' time); capture blockers; and promote wave-N's Opus discoveries into wave-N+1's cheap-tier prompt. Bank rate 83% → 93% on that alone.
- Concurrency hazard: if N reconcile targets share ONE TU, FORBID agent builds — concurrent splice-builds clobber a tracked file. Allow the single permitted splice-build only when targets are spread across TUs. (Guard the campaign, not the process — §the S27 law.)
- An agent that rejects your premise is working correctly. Told a draft was byte-correct and only
declaration-blocked, one agent re-ran
match_onefirst, found a real 1-instruction DIFF, and fixed both. Hand agents the evidence, not the conclusion.
§136 — The LOCAL-VARIABLE lever: how many C locals, at what scope (P30 wave 4a, 25 byte-verified banks)
Distilled from the wave-4a index-gap reports (33 drafted, 23 banked whole-binary first pass). The wave's dominant finding, and the reason this section exists as a class rather than a list:
In the 60–120-instruction band, most "regalloc/scheduling residuals" are decided by HOW MANY C LOCALS YOU DECLARE AND AT WHAT SCOPE — not by register pins. gcc-2.7.2 allocates one pseudo per C local;
local-alloc.c:472refuses a local allocno whoseREG_N_DEATHS > 1, promoting it to a global allocno that is ranked by density and loses the low register. So splitting one reused local into two, or merging two into one, moves whole register assignments — deterministically, at zero blast radius. Reach for the local-count lever BEFOREregister __asm__pins.
One report makes the anti-case explicit: for a redundant move $sN,$sM the pin is the wrong lever
— it acquires the register but lets gcc reuse it destructively as the sign-extension scratch. The
right lever was hoisting the assignment above the call (func_80183394).
The splitting/merging rules (each closed a residual, byte-gated)
- One local reused across N arms/repetitions ⇒ SPLIT it per arm. A function-scope pointer used
in 3 if/else arms dies in 3 places, fails the local-alloc gate, becomes a global allocno and loses
the low reg to a block-local constant. Per-arm block-scoped locals fix it in one edit. Symptom:
the same
$v0/$v1pair swapped in ONE arm only, siblings byte-correct. (func_8018C96C; same mechanismfunc_801909D8,func_80183A14.) - A compound initializer holding two values ⇒ SPLIT into two statements when you have one
callee-saved register too many.
x = *(u8*)p << k;makes a load pseudo and a shift pseudo, both live across the call ⇒ two$sregs;x = *(u8*)p; x = x << k;reuses one ⇒ one. Symptom: an extrasw $sNin the prologue, frame size otherwise identical. (func_80184FA4.) - Two variables where you wrote one ⇒ the target keeps a copy you cannot reproduce. An extra
addu $vX,$v0,$zeroright after ajalplus a later copy of the same value = the source hadv1 = f(); rnd = v1;withv1pinned. An unpinned pseudo always coalesces the pair away — verified: removing the pin merged them and cost 27 instructions of drift. (func_8017BF88.) - A local's address in a callee-saved base register ⇒ write a POINTER local, assigned before the
loop and used only inside it. Writing
local.fieldeverywhere addresses$sp-relative, allocates no register and shrinks the frame. Symptom:LENGTH-DRIFTshort by anaddiu $sN,$sp,Kplus one save/restore pair. (func_8017DAC4.) - A symbol read at a constant offset, but built into
$s1bylui/addiu⇒ cache it in a pointer local (u8 *p = D_80078E78;). A directD_xxx[0x1A]folds%loper use, loses the pin and shrinks the frame 0x20→0x18. (func_8017C61C; extends §17.) - Local stack slots are assigned in DECLARATION order, ascending from the outgoing-arg area
(0x10) — independent of use order. Frame-offset drift with correct code is a declaration-ORDER
problem. (
func_8017D2B8; complements §135-6, where the delta is a dead local.)
The type-form rules
- A real
mult $rX,$rYwith a small constant ⇒ the multiplier is a NON-CONST LOCAL, not a literal. A literalx * Kalways goes throughsynth_mult(sll/addu/subu chain). Assigns32 r = K;as its own statement before the first multiply:expand_multthen sees a REG, and CSE cannot fold it back becausemulsi3has no immediate form. Bonus — thelilands in whichever basic block the assignment is in, so its position in the.stells you where to put the statement. (func_80183B04. The inverse of the existing synth_mult entry.) - A negative addend on a narrow field coming out as
li $sN,0xfff0+addu(target:addiu $vN,$vN,-0x10) ⇒ gcc narrowed the whole expression to HImode, where the negative constant is its 16-bit unsigned image and no longer fitsaddiu. Fix: hoist the call to its own statement and put the load+subtract in a block-scopeds32temp. A signed*(s16*)load does NOT fix it (it reassociates); a function-scope temp does NOT fix it either. (func_801840FC.) lhu+sll 16+sra 16+Non a stack local an out-param call wrote ⇒ the local isu16, read as(s16)x >> N— the combiner folds the sign-extendingsra 16into the user shift. Through a PsyQSVECTOR(short vx) you getlh+sra Ninstead. A scratch vector read back sign-extended-then-shifted must beu16 v[4], NOTSVECTOR. (func_8017D2B8.)- An unexplained
addu $vX,$aY,$zerobefore a conditional branch, with the two feedinglhloads in the wrong order ⇒ the value is ans16LOCAL, nots32.LOAD_EXTEND_OPfolds the sign-extend of an already-lh-loaded HImode pseudo into a plain move, giving a second pseudo for the arithmetic while the comparison keeps the original. (func_80183EF8.) LENGTH-DRIFT +1with a narrow load of the SAME stack slot (lh 0x12($sp)besidelw 0x10($sp)) ⇒ gcc-2.7.2 narrowed a memory-operandlocal >> 16into a sign-extending halfword load at +2. Bind the local to ans32temp used twice to force onelw+sra. (func_801834BC.)andi $vN,0xffffright after ajalthat the target lacks ⇒ the TU declares the calleeu16/s16-returning and gcc re-extends the return value. Do NOT change the declaration — call through a cast. This is §135-9 applied to the RETURN axis, not the arguments. (func_8017E26C.)
The scheduling rules (refining §135-2 and §135-4)
- §135-2's
MEM_IN_STRUCT_Plever does NOT apply when the blocking store has a VARYING address.true_dependence()only drops the edge for a non-varying (constant-address) store. If the load must hoist above stores through a different register base, the only lever is source order — assign the load to a temp ABOVE the stores. This is the load-side dual of §135-4. Tell: one extranopin a load-delay slot plus constants landing in$v0instead of$v1. (func_80182F00.) memrefs_conflict_ptreats$sp-based and register-based MEMs as CONFLICTING, so a register load cannot hoist past an$spstore — but two$spstores at different constant offsets ARE disambiguable and reorder freely. Reading store order as literal source order will send you down a wrong path; the load/store base-class asymmetry is the discriminator. (func_80183B04.)- An unfilled load-delay
nopwhere the target fills it with a trailing call's argument setup ⇒ hoist a LOAD, don't chase the arg setup. Split a read-modify-write (*p = *p + 1) intov = *p + 1; … *p = v;so its load rises above an intervening pointer chase; the chase'slwthen fills the slot and the freed arg-setup instructions cascade into the earlier nops. Store order is unchanged, so it is byte-safe. (func_8019064C.) - Prologue
sw $sNstores in REGNO order where the target has DEF order ⇒ the saves are anti-dependent on each register's first def, so emission order tracks def order. Aregister __asm__pin on the incoming parameter turns itsmoveinto a schedulable body instruction that loses priority to the%hiaddress chain and reshuffles the whole prologue. Pin loop variables; NEVER pin the incoming parameter. (func_8018613C.) - An extra induction register (3 IVs where the target has 2) ⇒ do NOT write the second pointer.
Write ONE pointer and address every field as
p + const;combine_givsmanufactures the representative itself. And when the preheaderaddiu rIV,rBASE,Kbuilds the WRONG K, that is the combined-giv ANCHOR choice:record_givprepends andcombine_givstakes the list head, so the last-emitted reference in the body anchors — move the statement whose final reference sits at the target's K to the END of the body. (func_801862A8,func_8018B128; §3-Giv/§70 keyed by symptom rather than by the word "induction".)
The declaration surface (integration, not codegen)
conflicting typesfor aD_symbol whose declaration you cannot find in the split.c⇒ it lives inside aDEFINE_func_*()macro body insrc/shared/engine_core.h. Grep the macro bodies for everyD_symbol your draft names and reuse the canonical type verbatim. In particular an 8-byte-stride table declared there ass32 D_x[][2]must be indexed[i][0]/[i][1]— do NOT declare splat's interior label (D_x+4) as its own extern; that both conflicts and duplicates. Extends §135-7 to the shared-header declaration surface. (func_801854C4.)- A
lui+oripair whose halves are both small (e.g.0x8000A8) and which resolves to no symbol is a PACKED COORDINATE LITERAL, not an address. Confirm against a sibling TU's call. (func_8017CDB0.)
Wave economics (measured, for the next batch's sizing)
33 targets · 46 agents · 4.44 M tokens · 29 min → 29 claimed MATCH, 23 banked whole-binary (70%).
Bank rate by the tier that produced the FINAL draft (derived per-function from the journal + the
gate, NOT read off the workflow's by_tier, which counts claimed matches and therefore sums to 29
rather than 23 — R37):
| tier | banked / attempted |
|---|---|
| Opus direct (≥90 ins) | 10 / 14 |
| Haiku direct (≤89 ins) | 3 / 8 |
| Opus escalation after a Haiku miss | 10 / 11 |
The operative number is the escalation rescue rate: 10 of 11. On a 60–120-instruction pool the
cheap tier closes outright only ~3/8, so Haiku here is a triage stage, not a substitute — it is ≡
Opus at ≤~50 ins (cheap-tier-ab-validated), and this band is above that line. The two-lane shape
still pays because the escalation almost never fails; route ≤50 ins to Haiku and expect to pay for
an Opus pass on most of the 60–120 band.
index_hit was 13 true / 18 false — the index is now the bottleneck the cookbook itself was in
wave 1, which is why the 31 gap reports above are worth more than the matches.
§136a — Blocker capture: classify on the OUTPUT, never on the exit status
The reconcile lane only runs 12/12 because each agent is handed the compiler's own error line (§135, the S29 law). Capturing those lines needs one care point, learned the hard way this session:
make build runs check, so a draft that COMPILES PERFECTLY and merely produces different bytes
also exits non-zero. A capture tool that branches on returncode == 0 to mean "compiled fine ⇒
byte DIFF" therefore has an unreachable branch, and silently files every genuine byte-DIFF under
"unknown". Classify on what the build PRINTED:
| what the output shows | class | route |
|---|---|---|
a non-warning line matching error / conflicting types / undefined |
PLUMBING | reconcile lane — hand the agent the line verbatim |
no compiler error, but [FAIL] / got <sha> / want <sha> |
DIFF | redraft lane — the C is wrong, not the declarations |
| neither | UNKNOWN | investigate; do not route |
Note the filter must exclude warning: lines: the same conflicting types for … text appears as a
warning for built-ins (memcpy) and for benign external-decl mismatches, and those do NOT block
the bank. Only the hard-error form does.
Measured on wave 4a's 10 gate failures: 7 PLUMBING / 3 DIFF. That ratio is why the capture step is worth its ~10 builds before any reconcile fan-out.
⚠️ CORRECTION (earned the hard way — do not repeat my error). I first wrote that this meant "70% of the refusals were paperwork, not codegen." That is wrong, and a reconcile agent refuted it against the bytes. A PLUMBING verdict means only that a declaration conflict EXISTS — the conflict aborts the compile, so the byte question is never reached and the capture says NOTHING about whether the body is correct. Two of the three second-round PLUMBING drafts had a real codegen residual hiding behind the declaration conflict:
func_80188694wasDIFF/4 SCHEDULE-REORDERon the untouched draft (closed with a §21 zero-byte re-tie barrier after six other variants failed), andfunc_8018C638wasDIFF/6 ADDRESSING/cse(closed by hoisting a store above a call). Both agents ranmatch_oneon the unmodified draft FIRST, found the body defect, and said so instead of accepting my premise.So: PLUMBING ⇏ byte-correct. Route it to the reconcile lane, but tell the agent to re-verify the BODY before assuming only declarations are wrong — the wave-4b reconcile prompt's "do not rewrite the body unless you prove it is actually wrong" is the right instruction precisely because it leaves that door open. An agent that rejects your premise is working correctly (§135).
The classification was EXACTLY predictive, which is the point: all 7 PLUMBING banked through the reconcile lane; all 3 DIFF stayed stubs. Wave 4a therefore closed at 30/33 = 91% (23 first-pass
- 7 reconciled), and the reconcile lane is now 19/19 across three waves at ~13× lower token cost
than drafting (329 K for 7 fixes vs 4.44 M for the wave). Every agent found the reported conflict
PLUS a hidden second one cc1 never reached — which is the mechanical reason the "grep the whole TU
in one pass" instruction (§135-8) has to be in the reconcile prompt, not just the drafting prompt. Tool:
.run/s7_capture.py(any overlay, any draft dir; the ov_SC01_077-only.run/uc_capture.pyis its ancestor). It reverts the TU in afinally:— a killed process performs no undo (the S27 law).
Corollary — mandate a PRIVATE scratch path in the agent prompt. A wave-4a reconcile agent chose
.run/s7/scratch/spliced.c on its own initiative; a concurrent agent in the same wave overwrote it
mid-run, so its first verification compiled another agent's TU and returned a meaningless rc=0.
It caught the swap only because the emitted .s did not contain its own function. This is the
Phase-28 match_one fake-isolation defect recurring one level up — at the AGENT layer, where no
tool fix reaches it. Any parallel wave whose agents may compile must tell them to use a
process-unique scratch path ($$/pid-suffixed) and must never suggest a fixed shared one. A
shared scratch path does not produce an error; it produces a CONFIDENT WRONG VERDICT.
§136b — A prior wave's "genuine byte-DIFF" verdict is NOT reliable evidence (4 of 4 refuted)
FINAL TALLY: 8 of 8 DIFF-ledgered functions banked on redraft — wave 3's four, plus the three I classified DIFF from wave 4a's capture, plus one from wave 4b. The classifier is not wrong about what it measures ("this draft compiles clean and produces different bytes" is true and useful); it is wrong to read that as "this function resists matching." A DIFF verdict is a fact about one draft.
Phase-30 wave 3 ledgered four functions as genuine byte-DIFF — the class we treat as "real codegen residual, redraft is unlikely to help." Wave 4b re-drafted all four with fresh agents. All four banked. The recorded causes were not codegen at all:
func_801845B0— the prior draft readbeqz $v0, .L8018467Cas an inner early-exit when.L8018467Cis the epilogue, so it hoisted the whole tail out of the enclosingif. A control-flow misread. Resolve every branch TARGET LABEL to its actual instruction before trusting a prior draft's nesting: a branch to the label that beginslw $ra,K($sp)is a RETURN, not a join.func_80184A94— a D2 declaration conflict (D_801BBB78declared scalar in the draft, array in the TU at a line below the splice point). Fixed by copying an already-banked family sibling's declaration forms verbatim (§71 sibling-first).func_8017BEBC— the cached Ghidra seed was an entirely different body; the prior draft had followed it. The.swas the only usable source.func_8018480C— likewise re-derived clean.
The rule this establishes: a DIFF verdict describes the draft that was attempted, never the function's matchability. It is a statement about one attempt by one agent at one moment. So:
- Never retire a target on a DIFF verdict. Route it to the REDRAFT lane, not to a wall ledger.
- A RETRY note must be handed to the agent as a data point, explicitly labelled as one — the wave-4b prompt said "treat that as a data point, not a verdict; re-derive from the .s", and every retry agent did exactly that and refuted it.
- Re-GATING an unchanged draft is not a retry. Wave 4a's three DIFFs stayed stubs through a second gate purely because the same bytes were resubmitted; they still owe a redraft.
- Corollary for the backlog generally:
docs/backlog.mdentries carrying an old closeness/class are stale by construction (Phase-29 measured 77% of stored drafts had decayed). Re-verify before valuing one.
§136c — SIBLING-FIRST is a DERIVATION shortcut, not just a conflict fix (the fastest route in a family wave)
§71/§D1 are written as remedies for conflicting types. Wave-4b agents found their far higher-value
use: before deriving anything from the .s, grep src/shared/engine_core.h's DEFINE_func_*
macro bodies — and the target's own TU — for a byte-verified NEAR-TWIN. In a family wave the
twin usually exists, because that is what a family IS.
Measured instances this wave:
func_801859D8—DEFINE_func_80185978()inengine_core.his a near-twin: identicala0layout, identicalD_801B8748[D_801B8788[*(s16*)(a0+0x70)]][0]chain, differing only in three store values and a trailing call. Reusing its expression forms verbatim reproduced the schedule with no intervention — first-draft MATCH, and the same twin generalizes to the whole 10-member family.func_80184A94— copying an already-banked family sibling's declaration forms verbatim (extern u8 D_x[]used as(s32)D_x; the__asm__data alias) is what made it bank after a prior wave had ledgered it a genuine byte-DIFF.func_8018CB18—func_80180CC0/func_80185C6Cin the same TU fixed the whole tail shape and the u16-compare form before a line was written → first-attempt MATCH.
The rule: a byte-verified sibling is stronger evidence than the decompiler seed AND cheaper than
deriving from the .s, because its expression forms are already proven to produce the gcc-2.7.2
schedule and register assignment you need. Search order for a family target:
engine_core.h DEFINE_* near-twin → same-TU banked sibling → the .s → the Ghidra seed (last:
this session it was byte-proven to be an entirely different body twice).
§136d — Four gcc-2.7.2 levers the redraft lane found (each closed a residual no other lever moved)
These came from re-deriving four functions that had been ledgered "genuine byte-DIFF" (§136b). Each is byte-gated and none was reachable from the symptom index at the time.
-
RC-12, the
$0-add OPAQUE COPY — for a copy-pair whose COMPARE reads the wrong register. Symptom:REGALLOC-PERM, one register, on a copy pair — the target'sbeqzreads the SOURCE's register while every plain-Cb = a;spelling makes the compare read the COPY's. Cause:cse.c make_regs_eqvpromotes the longer-lived copy to canonical andcanon_regrewrites every use. Lever:register s32 zr __asm__("$0"); b = a + zr;— an opaque copy CSE cannot see through. Do NOT pin the copy's source or dest to a real hard register: pinning the source perturbs the prologue's sign-extend temp, pinning the dest lets gcc propagate the hard reg forward and delete the copy — both cost 2 instructions elsewhere. (func_8017E978, the last 1-instruction residual.) -
gcc-2.7.2
jump.cCOLLAPSES an if-then-else into a conditional overwrite.if (c) t = A; else t = B;— both arms single SETs of the SAME pseudo — becomest = B; if (c) t = A;, hoisting the else-arm's%hi/%lopair ABOVE thebeqzand shifting the whole tail (LENGTH-DRIFT -3, 20 mismatches). Symptom to look for: one arm's%hi/%lopair appears BEFORE the branch, and the target'sj-over-arm shape is missing. Lever: write the selector as TWO SEPARATE CALLS, not a ternary/select feeding one call — a call is not a simple SET so the transform cannot fire, and post-reloadcross_jumpthen merges only the common[move $a0,$s0; jal]tail, which IS the target shape. Corollary: a branch-delay slot holdingmove $a0,$s0that sits BEFORE the two arms is the fingerprint of cross_jump tail-merging, not of a hoisted argument. (func_80184494.) -
When a load hoists above a CONSTANT-ADDRESS store of a
D_global, fix the STORE, not the load.true_dependencedrops the edge between a/svarying-address load and a non-/sfixed-address store. Force/sonto the STORE via a COMPONENT_REF:((struct { s32 w; } *)&D_801E7998)->w = 1;— unconditionalMEM_IN_STRUCT_P, address stays constant, identicallui $at/sw %lo($at)codegen. REFUTED axis, recorded so nobody repeats it: §135-2's "reshape the LOAD to an INDIRECT_REF" is the wrong half of the lattice here —*(a[i] + j)still earns/s(a top-levelPLUS_EXPRgrants it) and merely re-folds the symbol into the load's%lo, going 2 → 32 mismatched. (func_80184960.) -
A branchless flag is
-(a != b) & 0xFF, never a ternary.xor / sltu $zero,x / negu / andi 0xFFisstore_flagnormalised to −1 followed by a u8 truncation. Acond ? 0xFF : 0ternary emits a BRANCH and can never reach that shape. (func_8017E978.)
Also confirmed here (§76's inverse, previously unindexed): when a narrow load lands in $v0 but
the target uses a mid scratch register, reuse an EXISTING global allocno as the destination —
but reuse one whose live range ALREADY spans the arm. Extending a SHORT allocno into the arm
lengthens its live range, drops its global.c:594 allocno_compare rank, and swaps two grants
instead of fixing one.
§136e — §136c's PRECONDITION, and two more symptom keys (wave 4b batch 3)
Sibling-first has a precondition, and an agent hit it honestly: func_801899AC's family has
all 13 members still unmatched and no DEFINE_func_801899AC in engine_core.h — so there IS no
byte-verified twin, and the search is a pure cost. Check that a banked sibling exists before
spending the greps; in an all-nonmatchings family, go straight to the .s. §136c is the fastest
route when the family has already been opened, which in a family wave is usually but not always.
Two symptom keys that had no index entry:
-
LENGTH-DRIFT -2, where MINE returns via a bare branch to the epilogue but the TARGET emitsj+addu $vX,$vY,$zero⇒ this is §136-L1 on the RETURN axis. An over-scoped function-level temp became a global allocno and swapped$v0/$v1with the returned local, so the return no longer needed a move. Scope the temp inside the loop. (func_801899AC.) -
A loop increment sitting in the loop-back DELAY SLOT plus a compensating negative
addiuon the fall-through is a SOURCE SHAPE, not areorgartefact — MIPS1 has no annulling, soreorgcannot invent the compensation. Write it asp += 2; if (t == cur) break; … p -= 2;.combine'sreg_n_sets == 1guard is what stops theaddiu -8folding into the followinglw 4($a1). The index's delay-slot entries point atreorg, which is a dead end for this one.
Also demonstrated (composition, func_8017D5F4, 46 ins): flat early-returns instead of nested
ifs to get the cross-jump layout → s32 pad[2] dead locals to sweep the frame size → s16 locals
so each load emits its lh+addu copy pair → mask-first or operand order → three register __asm__ pins on the mask temps (local-alloc otherwise takes $v0/$v1/$a3 and shifts the whole
global assignment) → two zero-byte __asm__ re-ties from the §30 toolkit → and finally
tools/permuter/run_masked.py on a pin-carrying base for the last 2. The toolkit composes; the
permuter is the LAST step on an already-pinned base, not the first.
Second correction to the capture classifier — DERIVE the class, do not pattern-match error prose.
The output-based rule above was still wrong in a third way: it decided PLUMBING by matching a regex
against cc1's diagnostic text, and cc1's vocabulary is open-ended. too many arguments to function 'func_80146C3C' — a plain arity conflict — matched none of error|conflicting|undefined|previous declaration|redeclar, so a trivially reconcilable function sat classified UNKNOWN through two
gate rounds. The closed, true fact is whether the compile produced an object:
compile_failed = bool(re.search(r'^make: \*\*\* \[.*\] Error \d+', txt, re.M)
and 'Deleting file' in txt) # .DELETE_ON_ERROR, added Phase 30 S29
Decide the class from that; keep the diagnostic lines only to hand the agent verbatim. Re-running the
fixed tool over six stubs moved the population from 5 PLUMBING / 1 UNKNOWN to 5 PLUMBING / 1 DIFF with no other change. That is three defects in one small tool in one session — an exit-status
branch that was unreachable, a regex that missed a common phrasing, and the prose-matching design
that made both possible — and each one silently mis-routed real work. R33 in one line: if an
invariant answers the question, never re-parse the output.
§136f — Two declaration sub-cases the reconcile lane surfaced (lane now 15/15 lifetime)
-
A symbol you are calling may be DEFINED — not merely declared — below your splice point.
func_8017CD9C's draft forward-declaredextern void func_8017D540(s32);, guessed from the asm (a barejalwith$a0 = 0and an unused$v0is consistent with several signatures). Butfunc_8017D540is defined in the same TU ~275 lines BELOW the splice, asint func_8017D540(int). cc1 took the draft's prototype first and rejected the later definition. D2's "grep the whole TU below the splice point" must look for DEFINITIONS, not justexternlines. The fix was to copy the definition's own signature; it was byte-neutral because the argument is a literal0and the return is discarded. -
An ARITY clash on the symbol you are DEFINING cannot be fixed by a cast — use lever (B).
func_801848DCis forward-declaredextern s32 func_801848DC(void);at three places above the splice, each used by a banked caller invoking it with no arguments, while the byte-true signature takes a pointer in$a0. Cast-at-use fixes a callee's type; it cannot change how your own definition is declared. The §37/§124 asm-label alias is the lever:s32 aF801848DC(void *a0) __asm__("func_801848DC");— define under the alias identifier, emit under the real symbol, zero blast radius, no shared header touched. In-TU precedent for this exact TU atov_SC04_018_jr_8017AE2C.c:8872(aF8018CB18). Verify the alias is byte-neutral by gating with and without it.
Lane record: 15/15 across five waves. The reconcile lane remains the most reliable stage in the pipeline and the cheapest per bank — but see §136a: it is not purely paperwork, and several of those 15 also carried a real codegen residual behind the declaration conflict.
§136g — When the index points at the WRONG lever: two byte-refuted routings (func_801863B4)
The last redraft of the session is the clearest case yet of why an agent must byte-test the index's
suggestion rather than trust it. It read the real tools/reference/gcc-2.7.2 source and refuted two
entries that the index confidently routes:
1. BRANCH-POLARITY where the return K block RELOCATES to the function tail.
The index routes BRANCH-POLARITY to §3-T4 ("invert the source condition") and §34 (the zero-byte
__asm__("") fence). Both were tried and byte-refuted here. The actual transform is
jump.c:1806 — /* Look for if (foo) bar; else break; */ — which SWAPS range1/range2 and
inverts. It runs long before reorg, so a fence instruction cannot block it. Its real
precondition is label2 = next_label(label1) being the RETURN label with
JUMP_LABEL(range1end) == label2. C lever: put ANY label between the if-join and the return
label — wrap the loop inside the guard, if (…) { loop } with ONE trailing return 0, instead of
an early-return 0 guard. The if-join label disarms the swap. (Closed the last 8 instructions.)
2. lh AND lhu of the SAME address, feeding an sll 16/sra 16 pair — not a weird cast.
MIPS LOAD_EXTEND_OP == ZERO_EXTEND (mips.h:1163), so a plain HImode local load emits lhu and
its later int use costs sll/sra, while an SImode use of the same lvalue (*p == -1) emits
lh. Source form: s16 v = *p; plus a separate *p == -1 test. combine collapses the pair
back into ONE lh unless the HImode pseudo has TWO reaching defs — so the shape only survives with
a hand-rotated guard (v = *p; if (*p != -1) { … do { …; v = *p; } while (*p != -1); }), which is
what makes both loads appear in the guard AND the loop-bottom block.
The generalizable point: the index is a starting hypothesis, not an answer. This agent tried both indexed levers, measured them at zero, went to the compiler source, and found the transform in a pass earlier than the one the index named. Record the refuted routing next to the correct one — otherwise the next agent re-runs the same two dead ends. (§136b tally: 9 for 9.)
§136h — CORRECTION: the zero-crack pool does NOT "refill with cheap work" (my error, byte-measured)
At the S7 close I recorded that the zero-crack (propagation-only) pool grew 120 → 147 families /
62,232 → 70,924 templatable ins even after ~1,400 members were propagated through it, and framed
that as a compounding cheap lever — "run the zero-crack sweep FIRST next session." I priced it off
the family map's byte_weight_templatable and did not probe a single member first.
Measured: the sweep banked 1 of 1,781. Diagnosing the top families (.run/s6_diag.py): two
compile clean and produce a byte DIFF, one fails at link (undefined reference to tail_8012F274). These are genuine per-member residuals — the remapped exemplar body does not
reproduce in the sibling.
Why the pool grows, correctly stated:
- It accumulates members that already failed earlier sweeps (S6a/S6b banked 1,582 out of this same population and left the rest).
- A fresh crack adds its family's members to the pool — but if you propagate behind every crack (which you should), those members are harvested at crack time. What accrues afterwards is the fraction that refused to propagate.
⇒ A growing zero-crack count is a residue signal, not an opportunity signal. Price this pool by
probing one member per family, never by summing byte_weight_templatable — that column counts
what could template if the bodies reproduced, which is exactly the thing in question. This is the
S28 worklist.md mis-pricing (§133) recurring on a different column: a weight column is a
prediction; the gate is the fact.
(Recorded against myself: this is R37 — probe before costing — violated in the scoping step of the very session that was correcting R37 violations elsewhere. The byte-gate cost was ~1,780 build cycles and zero tokens, and nothing wrong entered the tree; the loss was wall-clock and a wrong line in a checkpoint that a fresh session would have acted on.)
§136i — The drafter model LADDER: Haiku → Sonnet → Opus → Fable5 (Drew, 2026-08-03)
The two-tier rule from the 2026-06-29 A/B (cheap drafter ≤~50 ins, Opus for the 90+ tail) left the ~50–120-ins band unassigned, and every wave since defaulted it to Haiku-with-Opus-escalation. P30 S7 measured what that costs:
| tier that produced the FINAL draft | banked / attempted |
|---|---|
| Haiku direct (≤89 ins as routed) | 3 / 8 |
| Opus escalation after a Haiku miss | 10 / 11 |
Haiku on that band was expensive triage — a wasted draft plus a full Opus redraft — not a cheap drafter. The original A/B only proved parity ≤52 ins; everything above was extrapolation.
Route drafters by size:
| band | model: |
|---|---|
| ≤ ~50 ins | haiku (measured ≡ Opus, ~4.8× cheaper) |
| ~50–120 ins | sonnet ← the rung this section adds |
| ≥ ~120 ins, or escalation after any lower rung returns non-MATCH | opus |
| a genuinely NEW wall class nothing else cracks | fable (discovery only — never for applying known idioms) |
Never route Haiku → Opus directly, and never default a whole wave to Opus because the band "looks hard" — that is the same extrapolation in the other direction. Escalation is unchanged: any rung returning non-MATCH escalates one step up. The whole-binary byte-gate remains the sole arbiter, so a weaker drafter is a throughput risk, never a correctness risk (G3/P9).
Treat ~50 and ~120 as current best estimates, not constants — re-measure the boundaries whenever
a wave gives a clean per-tier signal (derive the split per-function from the journal + the gate, not
from the workflow's by_tier, which counts claims — §136).
§136j — The failure MIX flips with function size (measured across four bands, one session)
Blocker-capture classifications from P30 S7/S8, same tooling, same gate, four size bands:
| band | drafted | PLUMBING (declaration) | DIFF (genuine codegen) |
|---|---|---|---|
| ≤60 ins (volume lane) | 111 | majority | few |
| 60–120 ins | 33 | 7 of 10 failures | 3 |
| ≤120 aggregate, second round | — | 5 of 6 | 1 |
| 121–328 ins | 23 | 1 of 7 | 6 of 7 (86%) |
Small functions fail on paperwork; big functions fail on the compiler. The mix inverts almost completely across the range. Two operational consequences:
- Budget the lanes by band. Below ~120 ins, expect the reconcile lane to be the workhorse (it ran 15/15 lifetime and costs ~13× less than drafting). Above ~120 ins, expect the redraft lane and real gcc-source work — reconcile will have little to bite on.
- Do not read a low bank-rate on a big-function wave as a tooling problem. 16/23 (70%) on the 121–328 band with 86% of the failures being genuine byte-DIFFs is the expected shape, not a sign the pipeline is broken. The equivalent 70% on a ≤120 wave WOULD have been a tooling signal, because there the failures should be declarations.
This also re-frames §136a's correction: "a PLUMBING verdict says nothing about the body" is true everywhere, but the prior probability that a failure is paperwork at all is strongly size-dependent.
§136f addendum — the collider is often an ALREADY-BANKED SIBLING below the splice, and you can
locate it by arithmetic. func_801832A8's draft audit scanned only above its INCLUDE_ASM at
TU:4680 and concluded two callees "appear nowhere in the TU". They were declared at TU:4846-4847 —
inside the already-banked sibling func_8018389C, below the splice. The proof is arithmetic, and
it is worth doing before hunting: the draft grows the file by N lines, so a pre-splice TU line L
appears at L+N in the error output. Here N=184, and 4846+184 = 5030, 4847+184 = 5031 — exactly the
two conflicting types lines cc1 reported. If the reported line number exceeds the splice point,
subtract the growth and look there.
And a self-verification an agent can run WITHOUT the gate (stronger than match_one): splice into
a scratch copy of the real TU, run the Makefile chain cpp → cc1 → maspsx → as, then objdump the
function out of the resulting object and compare word-by-word against the target .s. The words that
differ should be exactly the unlinked relocation slots — set-compare the differing indices against
objdump -r, and require the differing-but-not-relocated set to be empty. That proves both the
declaration surface AND the codegen in real TU context (func_801832A8: 237 words, 25 differing,
all 25 relocations). It is the closest an agent can get to the whole-binary gate on its own.
§137 — REGALLOC-PERM is a TWO-COMPILE ARITHMETIC PROBLEM, not a permuter job
Symptom key: match_one reports REGALLOC-PERM — a clean swap of two registers, everything
else byte-identical ($t8/$t9, $s0/$s1, …). Historically this class went to the permuter or was
ledgered "unsteerable". It is neither: gcc-2.7.2 decides it by an arithmetic priority you can read
out of the compiler's own dumps and then target deliberately.
The mechanism. global.c:allocno_compare ranks by
pri = (int)( floor_log2(R) * R / L * 1e4 * size )
where R = times the register is used and L = the live-range length in insns. Both come out of cc1's own dumps:
cc1 -dl -dg … # t.i.lreg : "Register N used R times across L insns"
# t.i.greg : ";; Register dispositions"
The method (measured on func_801833F0, 328 ins):
- Compile with
-dl -dgand read R and L for BOTH contenders and their ranked neighbours. - Evaluate
prifor each ⇒ you get the exact admissible priority WINDOW that flips the pair. Here the two contenders were ONE unit apart —vtx1297 vs the giv 1296 — with window (1228, 1296). - R and L are both forced by the emitted code, so source reordering does not move them
(measured twice: moving a load's source position changed nothing, because
Lis recomputed post-sched1). This is why source-level levers are a dead end for this class. - Place a zero-byte
__asm__ __volatile__("" :: "r"(v))(§17/§21 primitive) at the source point that makesLland inside the window. Five probed placements gave L = 190/194/196/219/233/258; only L=219 → pri 1232 worked.
Why this matters: it converts a class we have been routing to the permuter (a random search that this session banked 0 from) into a deterministic two-compile calculation. Try it before the permuter on any clean 2-register swap.
(Companion finding, same wave, func_8017BFEC: a 5-instruction head rotation that was invariant
across every legal statement permutation — 8 head orderings × 2 store forms all identical — is the
diagnostic signature of priority / birthing-boost, not LUID order. Fix: make the pseudo
single-set by splitting a variable reused in two blocks into two locals (sched.md §1.7/§S2,
sched.c:2469/2490). An earlier agent had ledgered this exact function DIFF/7 "not steerable from
this decomposition" after ~70 source variants — refuted. Invariance under source permutation is
information: it says the lever is not in the source order.)
§137a — A gate verdict has a TIMESTAMP; re-check it against the draft's mtime
A redraft agent this wave found its handed-over "genuine byte-DIFF" was stale by 28 minutes: the
gate ran at 14:53, the draft was rewritten at 15:22 by an earlier lane whose result had been cached,
and it was never re-gated. The agent ran match_one on the file as it stood (D3), got MATCH, and
spent its budget proving the file would BANK instead of re-deriving a function that was already done.
The rule: before acting on any failure verdict — yours or a prior wave's — compare the verdict's time against the draft file's mtime. If the file is newer, re-verify before redrafting. This is the same family as the Phase-29 finding that 77% of stored drafts had decayed, but the opposite direction: a stored verdict can be stale because the draft got better, not just worse.
Two offline oracles that agent built, both worth reusing (they close the gap match_one's
relocation mask leaves open, without running make):
- Full relocation RESOLVE — resolve your
.o's relocations against the target.s(D_<addr>/func_<addr>symbol values, HI16/LO16 paired addends) and compare all words. This sees the class the mask hides: a wrongjaltarget or a wrong%hi/%losymbol or addend. Result there: 0/227 diffs including every relocation. - Collateral check — objdump the whole TU with and without the splice and require every OTHER
function to emit identical words, allowing only
.text-relativejaddends shifted by exactly your function's size (0x38C there). That catches file-scope declaration damage the target function's own bytes cannot show.
Symptom line for the index: "match_one/rtu_match say MATCH but the whole-overlay SHA still
DIFFs" has exactly three causes — a stale verdict, a wrong jal/%hi/%lo target the mask hides,
or collateral from file-scope decls — and the two oracles above discriminate all three offline.
§138 — The propagation lanes: a gate refusal is a DECLARATION, and which lever you owe depends on blast radius
Symptom key: dedup_extend or dedup_propagate plans N members and banks 0, or reports
PLUMBING: conflicting types for X / an undiagnosed DIFF. Measured this session: a 0/36 lane went
to 31/36, and every single blocker was a declaration — not one was compiler codegen.
The triage, cheapest first
Capture each failure's own compiler error (§136a — classify on the build's OUTPUT, never its exit status) and bucket by which symbol is named. The named symbols repeat: 9 failures per binary reduced to 4 distinct symbols shared across all four binaries. Then pick the lowest-blast-radius lever that is byte-neutral by construction:
| conflict | lever | radius |
|---|---|---|
macro declares (void), TU declares (ptr), use is cast |
relax the macro decl to () |
T2, 1 token |
| TU decl is unused boilerplate (zero uncast uses) | conform the TU decl to the fleet canon | T1 |
TU declares the symbol volatile, or any type the macro can't match |
asm-label alias on the DATA | T2 |
| the fn's own signature (return/arity/param) | asm-label alias on the DEFINITION | T2 |
Before relaxing to (), MEASURE the whole fleet's decl shapes for that symbol. () is illegal
against a prototype carrying a default-promotion param (s8/s16/u8/u16/char/short/float) — the
documented gcc-2.7.2 dead-end. It is legal against pointers and s32. One grep -rhoE over src/
answers it; 4,020 decls of func_80146C3C were all (void)/()/(u8*), so the relax was safe and
bought 8 of 36 for one token.
One conflict HIDES the next. A declaration conflict aborts the compile, so the error you see
says nothing about what is behind it (the S29 law). Fixing func_8012E5CC immediately revealed
func_80147364 at the same site. Re-run after every fix; do not price the lane off the first error.
Bucket by the (macro-shape, TU-shape) PAIR — NOT by the symbol. The same symbol conflicts in
BOTH directions across the fleet, and the lever differs. func_80146C3C cost this lesson twice in
one session: in the EXTEND lane the macro said (void) and the TU (u8*); in the PROPAGATE lane
the mirror — macro (u8*), TU (void). Relaxing only the (void) form (42 decls, byte-neutral,
R22-clean) fixed the first and did nothing for the second, which then banked 0/1 in all 134
overlays. One awk over the macro you are ACTUALLY fixing — not its sibling — shows the pair before
you spend a 134-build run. Read the declaration of the macro in front of you.
volatile in the host TU is a SCHEDULING BARRIER — and it looks exactly like a codegen wall
The four "undiagnosed DIFF"s in dedup_extend's own header were this. Its correctness argument says
an h_exact match guarantees byte-identity including relocs, so a DIFF should be impossible.
Both halves resolved against the bytes:
- The contract HELD —
func_80162FF4's original bytes are sha1-identical in the failing and the working overlay (af1aceb2…). Check this first: it splits "the registry is lying" from "the TU is different" in one command, with no build. - The TU differed — the host TU declared
extern volatile s32 D_80127090/94/98at FILE scope, which no working overlay's copy of that TU does. Volatile makes the macro's three stores a barrier, soaddu $a0,$s2,$zerocould not sink into thejal's delay slot: the build emitted it early plus anop, one instruction longer.
The tell: a diff that is a positional shift with a nop appearing at a delay slot is an
ordering constraint, not a wrong body. Look for a qualifier (volatile, const) on a symbol the
body touches before reaching for a codegen idiom. The fix is the data asm-label alias
(extern s32 aD_80127090 __asm__("D_80127090")): a distinct C identifier is immune to any TU's
declaration of that symbol, and is byte-neutral wherever the macro already worked.
The DEFINITION-side alias is the only escape when the fleet canon disagrees on a promoting param
func_80147364's byte-true definition is (u16, u16); 4,046 fleet decls say (u16, s32). ()
is illegal (u16 promotes) and conforming the decls would change caller codegen. Author the macro as
void aF80147364(u16, u16) __asm__("func_80147364"); + a definition of the aliased name: the real
symbol is emitted, every caller keeps its own declaration, blast radius is zero. In-tree precedent:
1,725 files already use this form. Banked ×137 first try.
Rank the lane by measured concentration, not by class count
A "45 classes / 20,837 ins" queue was really 5 classes carrying 89% of it. Re-split the ledger by
nins × n_stub before scheduling anything — and note the per-class outcomes diverge wildly
(4,110 banked ×137 · 3,288 banked ×138 · 3,973 dropped · 3,886 at 4/138 · 3,288 tool-gapped), so a
lane average predicts nothing. --recover is not a retry: the caller-extern reconcile that is
16/16 lifetime on drafts banked 4 of 138 on a propagation. Probe one excluded member's build
output before re-running any lever that already returned a bad number.
THREE carry variants hide in one "CARRY-FIXABLE" bucket — and they need different fixes
dedup_propagate reports missing file-scope extern (CARRY-FIXABLE) for all of them, but the
response differs and two are NOT tool bugs:
| variant | what extract_unit cannot carry |
response |
|---|---|---|
| a multi-line comment halts the preamble backscan | the externs above it | fix the tool (cdecl._mask) |
a draft-local struct Tag {…} in the preamble |
the type (refused by design: two macros defining one tag redefine it in a TU) | switch the exemplar to the shared engine_types.h type if the layout already exists there — byte-neutral |
a file-scope static inline helper |
the helper | hand-author the macro with the helper inlined, and exclude the source overlay |
The third is the sneakiest, because gcc-2.7.2 accepts implicit function declarations: the
extracted body passes compiles_standalone with the helper undeclared, so nothing complains until
a whole-binary byte DIFF 137 gates later. And the macro cannot be instantiated in the SOURCE overlay
— its file-scope helper is still there, so the macro's copy is a duplicate definition. The invocation
shape is --source-overlay X --binaries <all-but-X>; passing --binaries alone removes the source
from the scan pool and errors with "no source overlay has it matched".
A function defined under an asm-label alias is invisible to a name-anchored head regex. The def
is named aF<ADDR> in C and only BINDS the real symbol via __asm__("func_<ADDR>"), so any matcher
anchored on the literal func_<ADDR> returns None and every caller reads that as not matched.
family_remap._alias_decl_for resolves the form; make other tools reuse it rather than grow a
second matcher (R33). Two live consequences found together: dedup_propagate.find_site was blind to
the form entirely, and _alias_decl_for itself was blind to the wrapped (multi-line) declaration
— the §134 shape again. Measured before fixing (R37): 91 distinct alias decls fleet-wide, the
per-line matcher resolved 90, and the single miss was func_801466F0. That one function then took
all THREE fixes plus a type-lift — alias-aware find_site, wrapped-alias matching, and hoisting
its record typedef to engine_types.h — which is exactly why it survived four phases of being
written off: each blocker on its own looked sufficient to explain the failure.
Tool boundary worth knowing: once a group's members are DEFINE_func_*() sites,
dedup_propagate can no longer extend it — find_site never returns a def, so the auto-source
scan errors. dedup_extend is the tool for an already-macro-ized group. (And running
dedup_extend --check-only across ordinary overlays is a cheap census of how much wiring is
outstanding fleet-wide: measured here as exactly 1 group per overlay, i.e. no hidden backlog.)
§134 again, in a second tool — and the waiter rule corrected
dedup_propagate.find_site's preamble backscan had the SESSION-18 fix for blank / // /
single-line /* … */ lines, and still halted on a multi-line block comment (middle lines
start *; the last ends */ without starting /*). Same class S6b fixed three times in
family_remap. Decide skippability on cdecl._mask, not on line syntax — one oracle (R33),
every comment form, immune to a /* inside a string, with an R32 assertion that the mask is
length-preserving. Also: a body declaring a draft-local struct Tag {…} is unextractable by design;
if the identical layout already exists in src/shared/engine_types.h, switching the exemplar to the
shared tag is byte-neutral and unblocks propagation.
Waiter rule, corrected (three failures, one mechanism — the signal sampled is not the thing waited
for): pgrep -x make is right for ONE make and wrong for a campaign — dedup_propagate runs a
sequence of make build BINARY=<ov>, so a poll lands in the gap between two and reports a live
campaign finished. And pgrep -f <pattern> self-matches its own command line, so that waiter can
never exit. Wait on the campaign process by its real argv (ps -eo args | grep 'python3 tools/…') or
on tools/treelock.sh --status, which is a statement of intent spanning the gaps. Likewise never
wrap a campaign in nohup … & inside a backgrounded call: the harness then signals completion of the
wrapper — a fleet check "finished" at 63/140.
STEP 0 of sibling-first: grep src/ for a distinctive LITERAL from the .s
§136c's search order (engine_core.h near-twin -> same-TU banked sibling -> the .s) has a hole: both
of its first two steps are same-TU or shared-header scoped, so neither can reach a banked twin
that lives in a different overlay's TU — and the large template classes live cross-overlay by
construction. Measured: func_80188C04 (328 ins) was byte-identical to an already-banked
func_801833F0 in ov_SC02_028, and one command found it —
grep -rn "E100000A" src/ # a magic word lifted straight out of the target .s
— after which the body was reused verbatim with only the file-local type/macro suffixes renamed.
Put this ahead of engine_core.h. Pick a distinctive constant from the target: a magic word, an
unusual mask, an odd immediate. Corollary for the family map: it carries an in-family exemplar
pointer only, so a family whose twin is banked elsewhere looks un-cracked — and, separately, that
pointer can name an instance that is already banked, which hides the whole family from any
ranking built on it. Derive open sites from corpus.stubs over the member list. Measured on one
wave: ranking off the map's exemplar yielded 16,696 templatable ins; deriving from corpus.stubs
yielded 41,023 — including a 55-ins family open in 138 overlays and a 46-ins one open in 133.
Reconciling a gate-refused draft: which way you edit depends on WHERE the TU's decl is
A draft that match_one-MATCHes but the whole-binary gate refuses is declaration plumbing (the
agents cannot run the gate, so a TU-level conflict is invisible to them). Two opposite fixes, and
picking the wrong one creates the next error:
| the TU's decl is… | symptom | fix |
|---|---|---|
| ABOVE the splice point | redefinition of struct X / conflicting types for a TYPE |
DELETE the draft's duplicate — the TU already provides it (§100) |
| BELOW the splice point | 'X' undeclared (first use…) after you deleted yours |
KEEP a decl, in the TU's exact shape, and cast at the use (§17a-1 D2) |
Measured both in one session: func_8017D318 needed the deletions (the TU defines the type trio
above it — and a struct PW tag that a first pass missed, so it took two rounds), while
func_80181EE0 needed the opposite — I removed its decl, and the TU's own turned out to be ~180
lines BELOW the splice, leaving the identifier undeclared. Grep the TU for the symbol and compare
line numbers with the stub line before editing either way.
And apply §136a to your own capture tooling. My blocker-capture filtered the build log for
error|conflicting|undefined reference and so reported "no compile error" on a build that was
failing with redefinition of struct PW8017C290 and 'func_80143C74' undeclared — neither phrase
matched. A narrow keyword filter is exactly how a real error goes unseen. Keep any line naming a
source position (\.[ch]:\d+), minus the known SHB macro noise.
Symptom line for the index: "a propagation/extend lane plans N and banks 0" — read each failure's
compiler error, bucket by named symbol, and apply the lowest-radius byte-neutral alias; a volatile
in the host TU and a fleet decl carrying a promoting param are the two that masquerade as codegen.
§139 — A GATE THAT GREPS FOR VERDICTS MUST ASSERT 1:1 ACCOUNTING; and a --src filter must not survive a carve (P30 S38, wave 6: 10 of 16 drafts vanished)
The symptom was a gate run that looked ordinary: BANKED 5 / FAILED 1 — printed over 16
drafts. Six verdicts for sixteen inputs, and nothing said so. Nine of the ten missing drafts were
claiming MATCH.
The chain, both ends of which were ours:
harvest_verify._reload_corpus()exists, by its own docstring, to "re-derive the stub map after an isolation moved a stub to a new TU." It then re-applied the--srcfilter (x.path == a.src) to the reloaded map — deleting the very stub it had just followed to its new home. Line 136 of that same file already labels--src"an optional filter, not a location oracle";_reload_corpuswas the one place treating it as one. Downstream:_stubslosesfn→render()raisesKeyError→ uncaught →_jtbl_restore(snap)never runs → the carve is stranded inconfig/+src/→ and every later group in the same gate run then builds against a tree the earlier crashes mutated.- The gate driver (
.run/s6f_gate.py) captured the child's output, grepped it forVERIFIED:andFAILED :, and never looked at the returncode. "Neither line present" was booked as nothing at all, and the tally printed clean.
The fixes are both structural, not cosmetic:
- the filter never drops a draft under verification, wherever it now lives, plus an R32 loud report
if a working stub vanishes across a reload (which also repairs
_touched/baseline— a carved fn missing from_stubsleft its new TU unbaselined, so the revert path could not have restored it either); - the gate asserts
banked + failed + no-verdict == drafts, prints the child's rc and output tail for anything unaccounted, and exits non-zero — because a crashed child may have stranded a carve, so it must never look like success.
Proof it was not cosmetic: func_8017EA84 (579 ins) carves and banks BYTE-IDENTICAL under the
fixed path. The old tool reported it as nothing.
The generalisation — three corollaries worth more than the bug
(a) The byte-gate is a perfect CORRECTNESS oracle and a null COVERAGE oracle (R34, again). It cannot tell you about work it was never asked to do. Every wave-level tally is a coverage claim, and coverage claims need their own assertion. Any driver that classifies N inputs must prove it emitted N verdicts.
(b) A pipeline's exit status is the LAST command's. In the same session family_sweep … | tail -30
hid a non-zero exit (--only wants comma-separated; space-separated args were rejected). Use
set -o pipefail and read PIPESTATUS, or the tail is the error handler.
(c) A reverted CONFIG needs make extract, not just the revert. After undoing the stranded
carves I re-extracted one overlay of sixteen; the next gate run read three genuinely-banked functions
as failures because they were building against asm/ still partitioned by the old carve. This is the
Phase-20 R22 corollary, and it costs a whole gate cycle every time it is skipped. A gate result
measured against stale asm is not a measurement (R35).
And the inverse-lookup trap, same session
Propagating a just-banked head, family_hseq.json moves it out of its family's members list
and into exemplar with kind: "matched". A lookup that searches the member list for the head
therefore returns "no family" for every head — while family_sweep enumerates those same families
from the same file seconds earlier. Key on the exemplar (ov, addr) pair (two families can share
an exemplar address in different overlays). This does not contradict §138 rule 4 ("never rank off
the exemplar field"): that rule governs target selection, where an exemplar pointing at a banked
instance HIDES a family; here the freshly-banked exemplar is precisely what you are looking the
family up BY.
Symptom lines for the index: "a gate reports fewer verdicts than it had drafts" · "a tool reports NO FAMILY for a head another tool just enumerated" · "a gate result got worse after a revert" (→ re-extract, then re-measure).
§140 — A METRIC IS NOT A MEASUREMENT UNTIL IT IS REPRODUCIBLE FROM THE COMMITTED TREE (P30 S1e: a phantom regression that gated the session's best lever)
The alarm. S38's checkpoint recorded, in bold: "distinct-code FELL 89.3 → 89.2 across the last commit — UNEXPLAINED. Do NOT scale the alias lever until it is resolved. The BYTES are proven (R22); the ACCOUNTING is not." The def-side asm-label alias had just cracked a 208-conflict class 138/138 — the best-performing lever of the phase — and it was parked on that one line.
The regression never happened. True values, recomputed from each commit's own tree:
| instr | distinct | unique fns | |
|---|---|---|---|
commit:1426 true |
12,394,533 | 5,022,306 | 77,895 |
commit:1426 as committed |
12,402,412 | 5,029,324 | 78,025 |
HEAD true = committed |
12,405,402 | 5,025,082 | 77,952 |
The real delta over that span is instr +10,869, distinct +2,776 ins / +57 unique fns — everything
rose. The 843 digest was committed stale: generated from a working tree that still held work
REVERTED before the commit landed (overstating by +7,879 ins / +130 unique fns), and never
regenerated. The next honest digest was lower than the stale one, so the metric appeared to fall.
The three-line proof (do this before diagnosing any metric movement)
The weighted metrics are matched = sig − corpus.stubs. So if, between two commits, (a) the sigs are
identical (both denominators unchanged is sufficient evidence), (b) tools/ is unchanged, and
(c) git diff A B -- src/ | grep -c '^+.*INCLUDE_ASM(' is 0 — then HEAD's stub set is a strict
subset of A's, HEAD's matched set is a superset, and both numerators are mathematically forbidden
to fall. A reported fall is then a statement about the digest, not the tree. Three greps settle
it before a single hypothesis is formed; I burned two wrong mechanisms first (see below).
The two instrument defects it exposed
(1) progress.py stub_addrs swallowed the oracle's refusal. It wrapped corpus.stubs in
except Exception: return set() — and an empty stub set does not mean "no stubs", it means "the
oracle could not answer", after which matched = sig − stubs credits every function as banked.
Byte-witnessed: running the metric in a tree with no asm/ made corpus.stubs raise its correct,
coverage-asserted CorpusError for all 140 binaries, and the tool reported instr 100.00% /
distinct-code 100.00% — a complete decomp, out of a swallowed error. corpus.stubs is
deliberately fail-closed ("it refuses to answer rather than guess 'banked'"); a bare except
around a fail-closed oracle reinstates precisely the guess it refuses to make. Fixed: propagates.
(2) Nothing ever re-checked a committed digest. R34, again: the whole-binary byte-gate is a
perfect CORRECTNESS oracle and a null oracle for documents — a stale digest is byte-irrelevant,
so check-all stays 140/140 across it forever, and audit-corpus/audit-binaries assert things
about the CODE. Fixed: make audit-digest (tools/audit_digest.py, wired into tools-health
after report) recomputes the three headline metrics from the current tree and fails if the
committed digest disagrees. Negative-control-proven against the known-stale 843 digest. Note it
compares integers, not the printed percentages: the +7,879-instruction staleness rendered as
"94.4%" both before and after, so a percentage comparison would have seen nothing.
The same swallow, twice more, in the integration spine
cast_call_sites.tu_for and reconcile_tu.tu_for had the identical except Exception: pass around
corpus.stubs, falling back to the default src/<ov>/<ov>.c. cast_call_sites' own docstring, three
lines above, says the function exists because "the recovery passes were reconciling against a
DIFFERENT TRANSLATION UNIT than the one that would compile the code, and the heavy Phase-26 cores
live in exactly those jr files" — so the swallow silently reinstated the bug the function was written
to fix. A wrong-TU reconcile fails the gate, and this phase's base rate is ~24k PLUMBING vs 4,917
DIFF, so it would present as a codegen wall. Both now propagate CorpusError while keeping the
ValueError fallback for curated (non-func_ADDR) names.
Two wrong mechanisms I chased first, and why they were wrong
- The recorded lead —
progress.py:423'sSIGregex (it booksvoid aF80146A6C(…)under the alias name). Real blindness, but it feedsclassify(), which computes fn-count only. Neither weighted metric ever sees a C identifier. A lead that names a function must be checked against which metric that function actually feeds. - "The harvest reverted functions to INCLUDE_ASM" — plausible because reverting to
INCLUDE_ASMis byte-neutral (it pastes the original asm), so R22 would stay green over real coverage loss. Refuted in one grep: 483 stub lines removed, 0 added.
Symptom lines for the index: "a progress metric fell but the byte-gate is green" · "two metrics moved in opposite directions" · "a digest disagrees with the tree it describes" · "a coverage oracle reports 100%" · "a recovery pass reconciled against the wrong TU".
The law: a committed number is a claim about a tree; if it cannot be recomputed from that tree, it is not evidence — and it must never gate a lever. (R32/R34/R35; and R14 — I asserted two mechanisms before deriving either.)