diff --git a/docs/canonical-decl-reconcile-design.md b/docs/canonical-decl-reconcile-design.md new file mode 100644 index 000000000..29cbcf09d --- /dev/null +++ b/docs/canonical-decl-reconcile-design.md @@ -0,0 +1,82 @@ +# Canonical-decl reconcile tool — design input (Phase 24 T7 → follow-on; Drew chose "invest", 2026-07-03) + +> **Purpose:** the fleet-wide **×134 giant unlock**. Captured live in the session that hit the wall +> (R30) as the spec a fresh plan-mode session builds from. This is design INPUT, not a locked design +> — the plan-mode session picks the approach (§Design options) and gets Drew's approval before building. + +## The problem (byte-proven this session) +A "giant" (e.g. `func_80129CF8`, 191 ins, Fable5-cracked — cookbook §32) MATCHES standalone and banks +**×1** in `ov_SC01_077` fine, but **won't propagate ×134**: its shared callees/data are **loose-typed +across the fleet** — the same symbol is declared with incompatible types by different `engine_core.h` +macros + overlay siblings. When `dedup_propagate` instantiates the giant's macro (which carries its own +externs) in overlay X, those externs collide with X's other decls → `conflicting types` → the byte-gate +fails → the overlay is excluded → reach<2 → dropped. `--recover` only reconciles the *propagated fn's +caller* extern, NOT the *callee/data* externs inside the macro body, so it can't fix this. + +The flagship `func_80132784` propagated ×134 only because its symbols happened to be fleet-compatible. +Giants that touch loose-typed symbols (most) are ×1 without this tool → a fraction of the assumed ROI. + +## Conflict taxonomy (all seen on func_80129CF8; the tool must handle each byte-neutrally) +| class | example (func_80129CF8) | canonical | byte-neutral reconcile (PROVEN by hand this session) | +|---|---|---|---| +| **data: array elem width** | `D_80126948` = `u8[]` (4 siblings) vs my `s32[]` | `u8[]` | decl canonical `u8[]`, cast at use: `cam = (s32*)D_80126948;` | +| **data: struct vs array** | `D_80126DB8` = `struct BigCopy` (engine_core macro) vs my `s32[]` | `struct BigCopy` | decl canonical, `p1 = (s32*)&D_80126DB8;` (BigCopy = `{s32 words[41]}`, engine_types.h) | +| **data: scalar vs ptr** | `D_801151D4` = `s32` (10×) vs my `s32*` | `s32` | decl `s32`, store `D_801151D4 = (s32)cam;` | +| **data: signedness** | `D_801150D6` = `s8` (macro) vs `u8` (needed for `lbu`) | pick per-opcode | decl `s8`, access `*(u8*)&D_801150D6` to force `lbu` (the target opcode decides) | +| **callee: return type** | `func_80012F74` = `s32` (macro) vs my `s16` (target sll/sra) | `s32` | decl `s32(...)`, cast the RETURN: `x = (s16)func_80012F74(...)` reproduces the sll/sra | +| **callee: arity / param width** | (fix_arity_callers class) | — | no-proto `extern f();` where promotion-safe (existing `fix_arity_callers`) | + +**KEY INSIGHT:** the canonical decl goes in the giant's extern block (matching the fleet's majority / +the engine_core macro), and the giant's ACCESS SITE carries the cast. The cast is compile-time only → +the load/store/jal opcode is unchanged → **byte-neutral, arbitrated by the whole-binary byte-gate**. +A wrong reconcile just fails the gate (fail-closed, G3/P9) — so the tool can trial-and-error safely. + +## What "canonical" means when the fleet disagrees +The tool must PICK one type per symbol. Candidates for the picker (plan-mode decides): +- The type used by the most `engine_core.h` DEFINE_ macros (they're the shared/propagated bodies — the + fleet-consistent set), tie-broken by overall frequency. +- Never change the shared macros' type (that would re-break their matches); always conform the NEWCOMER + giant to the established canonical + cast. (If a macro itself is wrong, that's a separate migration.) +- Signedness/width where an OPCODE depends on it (lbu vs lb, lh vs lw): the *target asm* decides, applied + via an access cast under whatever the canonical storage type is. + +## Design options (for plan mode) +1. **Cast-injection into the draft/macro (lighter):** extend the reconcile pipeline (a sibling of + `sig_unify`/`cast_call_sites`/`fix_arity_callers`) to, per giant draft: resolve each callee/data + symbol to its fleet-canonical decl, rewrite the giant's extern to canonical, inject the byte-neutral + access cast at each use, re-gate. Integrates before `dedup_propagate`. **Recommended starting point** + — it's the mechanization of exactly what I did by hand, and stays per-giant (no fleet-wide churn). +2. **Shared canonical-decls header (heavier, more permanent):** one `src/shared/engine_decls.h` with ONE + canonical decl per shared symbol, included fleet-wide; migrate overlays/macros off their local decls; + giants + everything else cast against it. Bigger blast radius (touches all overlays), but kills the + loose-typing friction permanently, not just for giants. Phase-17 built a `gen_engine_decls.py` that + FAILED as a *global* header (broke existing loose matches) — revisit WHY (the fix is per-site casts, + which option 1 provides); this option only works married to universal access-casting. + +Likely answer: **build option 1 first** (unblocks giants now, low risk), keep option 2 as the eventual +consolidation once casting is universal. + +## Existing tools to extend / reuse (don't rebuild) +- `tools/sig_unify.py` — canonicalizes a draft's callee externs + its own def sig (callee-side). +- `tools/cast_call_sites.py` — §17a-1 per-site fn-ptr cast (the §20 cap tool). +- `tools/fix_arity_callers.py` — no-proto caller-extern arity (`--any-proto --binary`). +- `tools/canon_resident_calls.py` — resident-callee link-miss name rewrite. +- `tools/gen_harvest_targets.py` — resolves callee/data symbols to banked-canonical decls (scans the .s + for D_ refs). **The canonical-type oracle probably lives here or is a sibling.** Note: it defaults to + `asm//nonmatchings/` — pass `--asm-dir …/_a` for the region-a giants. +- `tools/dedup_propagate.py --recover` — where the reconciled giant then propagates ×134. +- `tools/gate_stage.py` — the pipeline that chains canon → cast → sig_unify → harvest_verify → propagate. + +## Live test case (committed) +`func_80129CF8` banked ×1 in `src/ov_SC01_077/ov_SC01_077_a.c` (byte-identical, the reconciled draft is +`.run/t7/func_80129CF8.c`). Its ×134 propagation currently fails (`dedup_propagate --addr 0x80129CF8 +--recover` → all overlays excluded → dropped). **Done = that command propagates ×134, clean fleet 136/136.** +The 6 sibling giants (`.run/t7/GIANTS_SURVEY.md`) are the scale-up target once the tool works. + +## Open questions for plan mode +- The canonical-type picker's exact heuristic + where it reads from (gen_harvest_targets? a fleet scan?). +- Per-conflict-class cast templates (the table above) — encode as a small rule set; the byte-gate validates. +- Does any conflict class have NO byte-neutral reconcile (a genuine Phase-16 "no consistent type" wall)? + If so, that giant/symbol is honestly ×1-only — detect + log, don't force. (A narrow Fable5 probe is the + only escalation, but the TOOL is Opus work — Fable5 is for stalled giant *matches*, not this plumbing.) +- Test on func_80129CF8 → then the 6 giants → then fold into the wave pipeline for all future giants. diff --git a/docs/matching-cookbook.md b/docs/matching-cookbook.md index f0a0f8acc..c3ca91d04 100644 --- a/docs/matching-cookbook.md +++ b/docs/matching-cookbook.md @@ -2112,3 +2112,15 @@ Instead of reverse-engineering the compiler function-by-function (Fable5 ≈ 375 **DIAGNOSTIC TELLS (pick the class in seconds):** `$t0` in a spill slot = reload artifact (spilled pseudo, not a source MEM); "every small edit moves 20+ insns" = RC-6 pressure-lock → permuter; a load stuck below a `D_x=0` store while offset loads float = /s-flag (§30); a phantom 7th `$s`-reg held across the fn = cross-call CSE; a prologue init in the wrong order = birthing-boost (S2). The 16-row full tells table is in each catalog. **HOW THE CHEAP TIER USES THIS:** given a near-miss diff → read the tell → look up the class's lever in the catalog → apply → gate. This is the artifact that lets Opus agents *and* the local model apply compiler-internal levers **without reading 80k lines of source** — the permanent-knowledge payoff. + +## §32 — The region-a CAMERA-GIANT idiom set: struct-base hoisting + 4 sibling levers (Phase 24 T7, Fable5-cracked on `func_80129CF8` 191 ins, match_one MATCH; transferable to the 6 sibling giants) +The region-a (`ov_SC01_077_a.c`) giants Ghidra flattens into per-global `lui/%lo` — but the target holds **global base ARRAYS in callee-saved regs across the whole function**. Fable5 read the gcc-2.7.2 source and cracked `func_80129CF8` in 3 edits (202→143→10→0), **no pins, no permuter**. The idiom set (each transferable — `.run/t7/func_80129CF8.c` is the worked example, `.run/t7/GIANTS_SURVEY.md` the target list): + +1. **Hoisted `$sN` base = an explicit POINTER LOCAL, assigned AFTER the first call, in the target's `lui` order** (`p2 = D_800AF630; p1 = D_80126DB8;` right after the first `jal`). gcc-2.7.2 has **no cross-bb CSE**, so a base living in a callee-saved reg across calls/branches can ONLY come from a *source* local — you cannot get it from a bare global access. **No `register __asm__` pins needed:** allocation order = density priority `floor_log2(n_refs)*n_refs/live_length` (`global.c:594 allocno_compare`) then first-fit regno (mips.h has no REG_ALLOC_ORDER) → most-referenced ptr→$s0, next→$s1, next→$s2. The init placement survives because sched1's ascending-LUID tie-break (`sched.c:2414 rank_for_schedule`) keeps zero-dep sets in order. A single-set/single-use base local is SAFE from the RC-7 init-sink (its hi/lo SET_SRC is a LO_SUM → fails `rtx_equal_p` at `local-alloc.c:1169-1172`). +2. **Branch polarity is READ OFF THE TARGET OPCODE** (not Ghidra's `if`): target `beqz $v0,.Lcopy` with the other arm as fall-through ⇒ write `if (sel != 0) {fallthrough-arm} else {copy}`. On func_80129CF8 this single inversion fixed **133 of 143** mismatches. +3. **`lw/lw/nop/addu/sw` per-element ladders = the S12 reused-s32-temp fence** — `a = p1[i]; b = cam[j]; cam[k] = a+b;` reusing ONE `(a,b)` pair across all elements. But **leave the LAST element in Ghidra's fresh-temp shape** (`t = p1[last] + cam[..]; cam[..] = ..; cam[..] = t;`) — its unfenced load is what the scheduler hoists into the previous element's load-delay slot, and a following call-arg `&cam[..]` fills the gap before it, both automatically. +4. **A grouped 3-loads/3-stores copy fed by `la $reg,SRC` (split lui/addiu) = a 32-byte STRUCT ASSIGNMENT, never scalar copies:** `*(RView*)(p2 + 6) = *(RView*)D_800AE688;` (RView = the GsRVIEW2-shaped 32-byte struct, now in `engine_types.h`). `expand_block_move` (`mips.c:2361`) emits one `movstrsi_internal` for ≤2*MAX_MOVE_BYTES(32) with 4 scratches; `output_block_move` burns the last scratch on `la a1,SRC` → 3 data regs → the 3+3 grouping. The **dest must go through the pointer local** (`p2+6`, folded to `0x18($s2)`); a bare global dest is CONSTANT_P and degrades to 2 regs. Scalar `a=src[i]; dst[i]=a;` copies schedule as lw/lw/sw/sw pairs — WRONG shape. +5. **Frame bigger than args(16)+saves by a round chunk = a DEAD LOCAL AGGREGATE:** add an unused `RView view;` (32 B) to reserve the missing 0x20 — gcc-2.7.2 assigns stack slots to local structs/arrays at expand time regardless of use, and -O2 never deletes them. Suspect this whenever a sibling's frame is 0x20/0x28 over-accountable. +6. **This class needs NO pins, NO asm fences, NO permuter** — the interp-loop software-pipelining (a2/sll/lh-next/sra/sw + a3-in-jal-slot) is deterministic sched2+dbr output once the bases sit in $s0/$s1/$s2. + +**⚠ BANKING WALL (Phase-24 T7, still open):** these giants MATCH standalone (match_one) but banking ×134 hits the **loose-typing data-conflict wall** — the same globals are declared as `struct BigCopy` / `s32` / `s32[]` / `u8[]` / `s8-vs-u8` across `engine_core.h` macros + `_a.c` siblings, so the giant's own decls collide in-TU. Reconcile = declare each symbol its **canonical** type (the one the shared macros use) and cast byte-neutrally at the access site (`cam = (s32*)D_80126948;`, `p1 = (s32*)&D_80126DB8;` for a `struct BigCopy` base, `D_801151D4 = (s32)cam;`, `*(u8*)&D_801150D6` to force `lbu` under an `s8` decl). This is per-symbol manual today — a `dedup_propagate`-style **data-decl reconcile tool** is the throughput unlock (cf. `fix_arity_callers` for callers). diff --git a/src/ov_SC01_077/ov_SC01_077_a.c b/src/ov_SC01_077/ov_SC01_077_a.c index 1f4703687..b12cc6a3c 100644 --- a/src/ov_SC01_077/ov_SC01_077_a.c +++ b/src/ov_SC01_077/ov_SC01_077_a.c @@ -342,7 +342,91 @@ INCLUDE_ASM("asm/ov_SC01_077/nonmatchings/ov_SC01_077_a", func_801299C8); INCLUDE_ASM("asm/ov_SC01_077/nonmatchings/ov_SC01_077_a", func_80129C40); -INCLUDE_ASM("asm/ov_SC01_077/nonmatchings/ov_SC01_077_a", func_80129CF8); +extern void func_8012A328(void); +extern void func_80053308(s32); +extern s32 func_80012F74(s32, s32, s32, s32); /* canonical s32 (engine_core); (s16)-cast the return for the sll/sra */ +extern void GsSetRefView2L(void *); +extern s8 D_801150D6; /* canonical (engine_core macro): s8 — access via *(u8*)& for lbu */ +extern u8 D_80127504; +extern s32 D_80126E60[]; +extern s32 D_80126F04[]; +extern u8 D_80126948[]; /* canonical (sibling): u8[] — cast (s32*) at use */ +extern s32 D_80126FA8[]; +extern struct BigCopy D_80126DB8;/* canonical (engine_core macro): struct BigCopy — (s32*)& at use */ +extern u8 D_800AF630[]; /* canonical (sibling): u8[] — cast (s32*) at use */ +extern s32 D_800AE688[]; +extern s32 D_801151D4; /* canonical (10 siblings): scalar s32 — store (s32)ptr */ + +void func_80129CF8(void) +{ + RView view; /* unused: reserves the extra 0x20 frame bytes the target has */ + s32 *cam; + s32 *p1; + s32 *p2; + s32 a; + s32 b; + s32 iVar2; + + func_8012A328(); + p2 = (s32 *)D_800AF630; + p1 = (s32 *)&D_80126DB8; + if (*(u8 *)&D_801150D6 == 1) { + cam = D_80126E60; + } else if (*(u8 *)&D_801150D6 == 2) { + cam = D_80126F04; + } else { + cam = (s32 *)D_80126948; + if (*(u8 *)&D_801150D6 == 3) { + cam = D_80126FA8; + } + } + if (cam[0] != 0) { + (*(void (*)(s32 *))cam[0])(cam); + } + func_80053308(cam[2]); + if (D_80127504 != 0) { + p1[0xf] = (s16)func_80012F74((s16)p1[0xf], (s16)cam[0xf], 2, 1); + p1[0x10] = (s16)func_80012F74((s16)p1[0x10], (s16)cam[0x10], 2, 1); + p1[0x11] = (s16)func_80012F74((s16)p1[0x11], (s16)cam[0x11], 2, 1); + p1[0x12] = (s16)func_80012F74((s16)p1[0x12], (s16)cam[0x12], 2, 1); + p1[0x13] = (s16)func_80012F74((s16)p1[0x13], (s16)cam[0x13], 2, 1); + p1[0x14] = (s16)func_80012F74((s16)p1[0x14], (s16)cam[0x14], 2, 1); + p1[0x15] = (s16)func_80012F74((s16)p1[0x15], (s16)cam[0x15], 2, 1); + D_80127504 = D_80127504 - 1; + } else { + p1[0xf] = cam[0xf]; + p1[0x10] = cam[0x10]; + p1[0x11] = cam[0x11]; + p1[0x12] = cam[0x12]; + p1[0x13] = cam[0x13]; + p1[0x14] = cam[0x14]; + p1[0x15] = cam[0x15]; + } + a = p1[0xf]; + b = cam[0x1f]; + cam[0x17] = a + b; + a = p1[0x10]; + b = cam[0x20]; + cam[0x18] = a + b; + a = p1[0x11]; + b = cam[0x21]; + cam[0x19] = a + b; + a = p1[0x12]; + b = cam[0x22]; + cam[0x1a] = a + b; + a = p1[0x13]; + b = cam[0x23]; + cam[0x1b] = a + b; + a = p1[0x14]; + b = cam[0x24]; + cam[0x1c] = a + b; + iVar2 = p1[0x15] + cam[0x25]; + cam[0x1e] = cam[0x16]; + cam[0x1d] = iVar2; + GsSetRefView2L(&cam[0x17]); + *(RView *)(p2 + 6) = *(RView *)D_800AE688; + D_801151D4 = (s32)cam; +} DEFINE_func_80129FF4() /* dedup: shared engine-core @0x80129FF4 (src/shared) */ diff --git a/src/shared/engine_types.h b/src/shared/engine_types.h index 6fffb0bd0..31f17c089 100644 --- a/src/shared/engine_types.h +++ b/src/shared/engine_types.h @@ -447,4 +447,8 @@ typedef struct { Blk16 lo, hi; } Buf32; typedef struct { s16 x, y, z, w; } PosT; typedef struct { u16 x, y, z, w; } MoveT; +/* Phase 24 (T7): GsRVIEW2-shaped 32-byte view record — the 8-word block copied/reserved in the + * region-a camera giants (func_80129CF8 …). Shared so the matched body propagates ×134. */ +typedef struct { s32 vpx, vpy, vpz, vrx, vry, vrz, rz, super; } RView; + #endif /* BFM_ENGINE_TYPES_H */