recover_integration split-awareness landed at commit:0470. --auto --limit 20 banked 0/16:
the closeness-0 caller-arity cohort is exhausted (Phase 24 got the cheap ones). Residuals
are per-function TU-context matching (8-byte struct-assign lowers to a memcpy CALL in a TU
that declares memcpy -> byte mismatch; use explicit memcpy()/lhu-sh, the sibling convention).
Drew: conclude T1, residuals backlogged, proceed to T2 (the structural-family lever).
--auto silently skipped the 263-stub ov_SC01_077_after cohort: the drift-check hardcoded
the main asm subdir and gate_stage was never passed src/asm/src_file. Now stub_map() reads
each stub asm subdir from its INCLUDE_ASM line, the drift-check uses the per-fn subdir, and
reconcile_and_gate groups targets by split file, gating each via run_gate(src=,asm=,src_file=).
fix_arity_callers/dedup_propagate already split-aware; harvest_verify self-filters per split.
- housekeeping: deleted stale root gccdump.lreg (gcc default -da RTL dump, dumpbase
"gccdump"; a one-off RTL-inspection run at root CWD, NOT a committed tool). Practice:
RTL dumps go to a .run/ CWD or -dumpbase .run/gccdump (R12)
- make sig-overlays (134 sigs); worklist.py --refresh -> build_fuel_manifest + backlog
render: docs/worklist.md + docs/backlog.md regenerated (stale Jun-26 ranking killed;
the banked whale func_80144B9C dropped from backlog, P9)
- fix tools/sig_image.py docstring: h_norm is the live norm_stream normalizer, not a
"T5 placeholder" (R21/R30)
- CURRENT_PHASE.md: Phase-25 plan-of-record + T0 frontier survey (R14): h_norm 2,764
multi-member families / 11.1 MB; 46% of large solo monsters (3,989/8,670) are family
members; reach-134 h_exact warm-up = 130 classes / 4.11 MB
- func_801770E0 (152 ins, reach-134) propagated ×134 via dedup_propagate --recover.
Chose the NATIVE DEFINE-macro path over a hand-rolled shared header: func_8014E048
(pins+asm) is already ×134 via a DEFINE macro, proving that path handles pin/asm -O2
giants (the whale needed a shared header only because it is -O0 -> separate object).
Clean fleet check-all 136/136 (R22), dedup-check 1811->1812/0.
- R14: func_8014E048 was ALREADY ×134 (T6 §A) — the whale-session handoff was stale;
only func_801770E0 + func_801372B0 actually remained ×1.
- tooling (reusable): dedup_propagate.overlay_files now also scans the whale-rollout
_o0b/_after splits — post-whale-region fns (func_801770E0 in _after.c) were invisible
for both source-def-find and stub-replacement. func_801770E0 extern block made
contiguous (a comment between externs made find_site drop 5 externs; byte-neutral).
The single biggest byte-weight lever (770 ins ×134 ~ +1.6% byte-weight), banked across all 134
overlays as -O0 compiled C (was ×1 in ov_SC01_077, commit:0463).
- ROLLOUT (tools/rollout_whale_o0.py): per single-file overlay, LINE-split <ov>.c at the whale
(splat emits in vram order), carve the yaml code subseg into before/<ov>_o0b(-O0)/<ov>_after,
thin <ov>_o0b.c #includes the shared src/shared/func_80144B9C.h (the whale C moved there — DRY,
TU-local typedefs, not a fragile 200-line DEFINE_ macro). Makefile WHALE_O0B_OBJS wildcard -O0.
- DEDUP: group E_func_80144B9C (134 members, h_exact 74186b97, source=the header; group_members
keys on binary+vram so a header-share validates like a macro-share).
- MILESTONE: clean fleet check-all 136/136 (R22); dedup-check 1811/0; FLEET REAL 225816->225949,
byte-identical 65.91->65.95% function-count (the gain is byte-weighted). Validated on ov_SC01_000
+ spot-checks SC02/SC03/SC07 before the full fleet.
- cookbook §38 + SETUP inventory.
- CRACK (cheap Opus, no Fable5/calls.c): the 770-ins reach-134 whale's 2-insn residual was a
STRUCT ASSIGN `D_80078E50 = *ent;` (sizeof(ENT)==0x24), NOT an explicit memcpy() call — gcc-2.7.2
-O0 expands a >MOVE_RATIO-word struct copy via emit_block_move->emit_library_call(memcpy),
precomputing dst/src into pseudos then addu into $a0/$a1 = the exact 2 missing moves.
- -O0 SPLIT: whale is a 2nd -O0 region (prologue 21F0A003) in the -O2 'after' segment; carved into
its own -O0 object o0b (splat.ov_SC01_077.yaml 3-way + Makefile CC1FLAGS=-O0; address-sorted
line-split preserves all banked C). ov_SC01_077 == d19c9580.
- memcpy: memcpy=0x8005C324 in symbols.resident.txt (overlays+resident only; main untouched vs its
MEMCPY.o) + __asm__("memcpy") on the engine_core.h block-copy macro (non-builtin C name -> no
built-in codegen, same call).
- MILESTONE: CLEAN fleet check-all 136/136 byte-identical (R22), dedup 1810/0. reach-134 -> ×134 (W9) next.
- tools/reconcile_decls.py: the DATA-symbol analog of cast_call_sites.py. A fleet-majority
canonical-type ORACLE (engine_core.h-authoritative + plurality across all overlays+resident —
the picker gen_harvest_targets lacked) + a byte-neutral access-cast TRANSFORM (array/struct/
scalar/ptr taxonomy, single-pass so no double-wrap). Mechanizes the manual giant decl-reconcile
that func_80129CF8 got by hand.
- wired into gate_stage after cast_call_sites (canon -> cast -> reconcile_decls -> byte-gate);
idempotent / no-op on drafts without a data conflict -> can't regress the wave; the whole-binary
byte-gate stays the sole arbiter (G3/P9).
- VALIDATED: oracle 8/8 correct picks for func_80129CF8's symbols; full loose giant ->
reconcile_decls -> swapped into the DEFINE_ macro -> make build BINARY=ov_SC01_077 = BYTE-IDENTICAL
d19c9580; idempotent (0 reconciled) on the already-canonical committed draft.
- R14: diagnosed the prior "×134 BLOCKED / all overlays excluded" by RUNNING it — refuted.
func_80129CF8's ×1 bank was already fleet-canonical; dedup_propagate --recover propagates it
to all 134 overlays byte-identical (no new tool needed). The "wall" was a stale-asm /
incremental-tree artifact (the T5c/T6-§A trap).
- banked ×134 via existing dedup_propagate --recover; clean fleet check-all 136/136,
fleet 64.90→65.52%, dedup 1799→1800/0, 0 NON_MATCHING (G4).
- T7b reframed: the reconcile tool automates the manual decl-reconcile for FRESHLY-matched
giant drafts (the 6 sibling giants + wave tail), not an 'unlock' of already-reconciled giants.
- CURRENT_PHASE: T1 done + the R14 finding logged; plan plan-…mossy-dawn.md (T1-T6).
CURRENT_PHASE: T7 progress-log entry (func_80129CF8 crack + the ×134 loose-typing
finding), T7b (the reconcile tool) is the next task (Opus/plan-mode/fresh), spec at
docs/canonical-decl-reconcile-design.md.
- Fable5Max cracked func_80129CF8 (191-ins region-a camera giant; match_one MATCH, 0-off): struct-base
hoisting via SOURCE-LOCAL pointers (gcc-2.7.2 has no cross-bb CSE → a callee-saved base can only come
from a local, NO pins), branch-polarity-off-the-opcode, S12 reused-temp fence, expand_block_move
struct-assignment for the grouped copy, dead-local-aggregate for the frame. Distilled → cookbook §32.
- banked ×1 in ov_SC01_077_a.c (d19c9580; clean fleet 136/136) after reconciling its loose-typed decls
(u8[]/s32[]/struct BigCopy/s8-vs-u8; func_80012F74 s16-vs-s32) to canonical + byte-neutral access casts;
RView (GsRVIEW2-shaped 32B) lifted to engine_types.h.
- FINDING (byte-proven): ×134 propagation is BLOCKED by the fleet-wide loose-typing wall — the giant's
shared callees/data are declared incompatibly across overlays, so the macro's externs conflict per
overlay (dedup_propagate --recover only fixes the propagated fn's CALLER extern, not its callee/data).
Unlike the flagship func_80132784 (compatible → ×134), loose-typed giants bank ×1, not ×134.
- Drew's call (A, invest): build the fleet-wide canonical-decl reconcile tool = the ×134 giant unlock.
Spec + conflict taxonomy + byte-neutral cast patterns: docs/canonical-decl-reconcile-design.md.
- clean fleet check-all 136/136 BYTE-IDENTICAL (R22). No new governance rules.
- find_site extern-collection: allow a trailing `/* comment */` after the `;`. The
comment-blind regex `^\s*extern\b.*;\s*$` stopped the backward scan at
`extern u8 D_801152A8[]; /* canonical TU type */`, dropping every EARLIER extern →
compiles_standalone failed on the now-undeclared callees/data (func_80135A4C,
func_80133784, D_801152A8). R14: THIS — not "pin/asm" as the backlog framed it — was
func_8014E048's real self-containment blocker.
- propagated func_8014E048 (the T5b S11 pins+barrier crack) + func_80157580 ×134;
dedup 1797→1799 groups (0 failed). CLEAN fleet check-all 136/136 BYTE-IDENTICAL (R22).
Fleet byte-identical 65.40% → 65.48%.
find_site + dedup_propagate --recover + build_engine_types fixes, then re-bank
the 13 recover_integration leaf-MATCHes and propagate each across all 134 overlays.
- find_site: match INDENTED inline defs (was column-0 only, silently dropping every
recover_integration-banked def from propagation — T6 blocker 1). Unit-tested:
indented defs match; indented call-exprs (if/assign/bare/return) correctly rejected.
- dedup_propagate --recover: on a straggler byte-gate failure, FIRST no-proto that
overlay's conflicting caller extern + re-gate (Part B, byte-neutral, same lever as
fix_arity_callers --any-proto); else EXCLUDE only that overlay (Part A, ×N-1) rather
than the historical all-or-nothing drop. Wired into gate_stage.
- build_engine_types: comment-aware find_defs/find_typedefs (blank_comments). The
generated header's own "...typedef lift" comment was captured as a bogus `typedef
vec`, self-colliding and blocking every --strip. NOTE: full --strip still conflicts
with the _a/_o0 split files, so split-file overlays need a TARGETED lift.
- banked + propagated 13 fns ×134: func_8014F74C 801542A4 8015BE94 8015F380 80160F00
801653B8 80166244 8016E778 801732C4 8017331C 80173374 80174554 801745AC.
func_8014F74C needed PosT/MoveT lifted to engine_types.h (targeted, byte-neutral).
- dedup 1784→1797 groups (0 failed). CLEAN fleet check-all 136/136 BYTE-IDENTICAL (R22:
make clean && extract-all && check-all). Fleet REAL 224073, byte-identical 65.40%.
R14: the confounding stale-asm/ tree (13 missing .s) that masked the first --recover
test is fixed by re-extract; the 3 pre-existing --auto-from stragglers (0x80174650/
8012A018/80165CA0) are pin/asm + uncaptured-local-macro (SHB) bodies, correctly dropped.
The integration-recovery tool for leaf-MATCH-but-whole-binary-gate-rejected fns ("declaration/TU
plumbing" — the dominant residual gate_stage's canon/cast/sig_unify pipeline doesn't reach).
- tools/recover_integration.py (NEW): batch recovery — gather leaf-MATCH candidates (--auto from
the backlog, drift-checked R14; or --funcs/--from-file) → no-proto their conflicting caller decls
→ gate_stage (byte-gate + log). 2-PASS snapshot/restore: pass 1 finds the bankable set, pass 2
re-banks ONLY winners from the clean snapshot (so non-banks are never corrupted).
- tools/fix_arity_callers.py: extended with --binary — scan+rewrite the overlay's OWN inline caller
decls (src/<bin>/<bin>*.c), not just engine_core.h. That was THE gap: a conflicting caller extern
is often inline in the overlay src (e.g. func_8016E778's `extern void f(void)` vs def `f(int)`),
which fix_arity_callers never saw -> the fn stayed unbanked.
- VALIDATED: banked 13 leaf-MATCH fns (func_8014F74C/801542A4/8015BE94/8015F380/80160F00/801653B8/
80166244/8016E778/801732C4/8017331C/80173374/80174554/801745AC), CLEAN-verified together
(ov_SC01_077 d19c9580). The banks themselves are reverted here (they re-bank via the tool and
will land ×134 once propagation-recovery lands — cleaner than committing ×1).
- R14 lesson (clean-verify caught it, R22): fix_arity_callers --revert is LOSSY for --any-proto
(()->(void), not back to the original args) -> corrupted non-banks; fixed with the 2-pass snapshot.
- REMAINING T6 (×134 propagation-recovery, 3 diagnosed blockers): (1) dedup_propagate find_site
misses INDENTED inline defs (Phase-15 class); (2) overlay-local-type lift; (3) auto-reconcile the
straggler's conflicting caller externs (the flagship func_80132784 / ov_SC02_005 class, done by
hand — needs automating). See CURRENT_PHASE.
The 400-ins giant func_80132784 (S11 prologue-weave; cracked T4, banked ×1) is now shared across
ALL 134 overlays byte-identical — the highest single byte-weight lever in the project.
- Blocker (T4-deferred): dedup_propagate's single-`--addr` plan is all-or-nothing; ONE straggler
overlay (ov_SC02_005) failed to compile the shared macro -> the whole fn dropped, kept ×1.
- Root cause: ov_SC02_005 declared a CONFLICTING caller extern `void func_80132784(s32,s32,s32)`
while the def is `(s32,s32,u32)` (3rd param s32 vs u32) -> `conflicting types`. The call site
casts `((void(*)(u8*,u8*,u16))func_80132784)(...)` so the extern type is codegen-irrelevant.
- Fix (T6 integration-recovery pattern, proven): reconcile the straggler's caller extern to the
def's canonical signature (s32->u32, byte-neutral), then propagate. dedup_propagate then rebuilt
all 134 overlays BYTE-IDENTICAL; registered group E_func_80132784 (134 members).
- dedup-check: 1784 validated, 0 failed. Verified by a clean R22 fleet check-all (136/136).
- This is the concrete spec for the T6 tool: detect a straggler's conflicting caller decl,
reconcile to the def's canonical sig, re-gate, propagate full (vs. the all-or-nothing drop).
Pre-existing latent breakage (Phase-21 close commit:0292, "+62 PsyQ names"), found during T5b's
R22 fleet verify. A genuinely-clean `make clean` + re-extract + build of `main` failed with
dozens of `can't open asm/nonmatchings/800c3/func_*.s` — clean-rebuild broken since Phase 21,
masked all along by incremental builds reusing stale pre-rename .s/.o (the exact R22 failure mode).
- Root cause (byte-proven): Phase-21 xdedup renamed 62 PsyQ library functions to their proper
names (InitHeap, FlushCache, GetTPage, SysEnqIntRP, SpuWrite, CdMix, __main …) in
symbols.us.txt, but never regenerated/renamed the committed src/*.c stub refs that call them by
the OLD func_<ADDR> name. splat's FRESH regeneration of a stub .c uses the CURATED names (move
src/800c3.c aside -> splat writes INCLUDE_ASM(InitHeap) + emits InitHeap.s); the committed stubs
were simply stale.
- Fix: rename all 62 INCLUDE_ASM(func_<ADDR>) -> the curated name across 12 files (800c.c 1,
800c3.c 22, apicard1/2/4 4/6/2, boot.c 1, libcd1.c 8, libetc.c 6, libgpu.c 3, sgap.c 1,
snd1.c 7, snd2.c 1). Pure rename to match splat's canonical output; byte-neutral.
- VERIFIED: main clean-builds 143dbb89 from a fully clean tree; full clean fleet check-all 136/136.
- Lesson (cookbook): a symbols.us.txt rename must be propagated to (a) shared-macro bodies
(engine_core.h — the T5b GetTPage fix) AND (b) INCLUDE_ASM stub refs (this), AND verified by a
genuinely-clean check-all (make clean + full re-extract), never incremental. Added a lint check.
Both concerns byte-verified from a fully-clean tree (ov_SC01_077 d19c9580, ov_SC07_009 2a6499b6
+ 133 overlays). Fleet is 135/136 — the 1 remaining failure is `main`, a SEPARATE pre-existing
Phase-21 breakage (62 dangling INCLUDE_ASM refs, tracked in CURRENT_PHASE, fix in progress),
NOT introduced by this commit.
[FIX (partial) — pre-existing Phase-21 latent breakage, found during T5b's fleet verify]
- A genuinely-clean `make check-all` failed 135/136 (overlays: `undefined reference to func_80058B40`;
main: dozens of `can't open asm/nonmatchings/800c3/func_*.s`). This commit fixes the OVERLAY side.
- Root cause: Phase-21 xdedup renamed func_80058B40 -> GetTPage in symbols.us.txt (one of the
+62 PsyQ names) but did NOT update the two shared engine_core.h macros that CALL it
(DEFINE_func_80139680, DEFINE_func_8012E28C — in all 134 overlays + ov_SC01_077_a). Since
Phase 21 a clean rebuild couldn't link; incremental builds reused stale .o's and masked it
(the R22 failure mode) — every "check-all 136/136" Phase 21->23 was incrementally-stale.
- Scope = exactly 1 symbol (static scan of all func_/D_ refs in the shared headers vs symbols).
- Fix: rename the 4 occurrences -> GetTPage (byte-neutral, same addr 0x80058b40 -> identical jal;
G6 curated-name). ov_SC07_009 link-fail -> byte-identical 2a6499b6.
- Lesson: a symbols.us.txt rename must be propagated to shared-macro bodies AND verified by a
genuinely clean (make clean + full re-extract) check-all, never incremental.
[FEAT — T5b: the S11 class crack, Fable5 spike]
- The S11 LUID(x)alloc "intrinsic" verdict was MAP-INCOMPLETENESS. func_8014E048 (143 ins,
reach-134; "not source-steerable" since the map wave; 28-off even after T5's directed permuter)
-> MATCH (143/143) -> whole-binary BANKED (ov_SC01_077 d19c9580). Derived by reading gcc-2.7.2
source + RTL dumps (12 experiments .run/gccmap/exp/e1a..e1k.c).
- NEW LEVERS (byte-proven): S12 reused-s32-temp fence (u16 temps DON'T work — combine folds the
unpromoted-HI zext temps away); S13 head-skip escape (body-local param copies conflict-steer the
scratch contest; volatile-asm dead-read wedge fence; multi-input dead-read K2 rebalance);
cse-opaque asm-copy; RC-4b pinned store-temp; RC-10 preference-cascade mechanics.
- Integration reconciles (both T6 classes): engine_core.h caller decl s16*->u16* (codegen-neutral)
+ canonical data decls w/ *(u16*) casts (D_801152A8 u8[] / D_801152AC s16). x134 lift blocked by
dedup_propagate self-containment (pins/asm) -> T6 target (joins func_80132784).
- Distilled (R30/R16): sched.md §6 (S12/S13) + regalloc.md §F (RC-10 + RC-6/S11 downgrade) +
cookbook §31 triage update; backlog re-logged (capped); memory updated.
- p16_permute.hide_asm: b64literal-pragma carrier for register pins + GTE __asm__ blocks.
pycparser parses the pragma; decomp-permuter's process_pragmas decodes it back so cc1 sees
the real pins/asm (regalloc steered, mvmva compiles). No submodule edit (reuses its own carrier).
- drop_preproc_and_scalar_typedefs: keep #define + custom struct/typedefs, drop only #include
(fixes func_801412A8 'OTLINK undeclared'); +f32 typedef gap; make_base_c hides asm.
- compile.sh/compile_o0.sh: prepend .include "macro.inc" so GTE mvmva assembles (the real build
gets it via include_asm.h, which base.c omits + -DPERMUTER disables). Byte-neutral for non-GTE.
compile_o0.sh (-O0) auto-selected for _o0 targets.
- run_masked.py: expr_type->int fallback for hidden-pin vars -> 0 internal-permuter-failures
(perm_split_assignment/perm_temp_for_expr no longer KeyError on pinned drafts).
- ALL 5 seeds parse+compile (base 4/110/36/52/77). FLAGSHIP func_80132784 (4/400) CLOSED to a
masked-0 that match_one confirms MATCH (400 ins) -> .run/wave/func_80132784.win.c for T4 gate.
- no build-input changed (136/136 untouched)
Drew T