recover_integration split-awareness landed at commit:0470. --auto --limit 20 banked 0/16:
the closeness-0 caller-arity cohort is exhausted (Phase 24 got the cheap ones). Residuals
are per-function TU-context matching (8-byte struct-assign lowers to a memcpy CALL in a TU
that declares memcpy -> byte mismatch; use explicit memcpy()/lhu-sh, the sibling convention).
Drew: conclude T1, residuals backlogged, proceed to T2 (the structural-family lever).
- housekeeping: deleted stale root gccdump.lreg (gcc default -da RTL dump, dumpbase
"gccdump"; a one-off RTL-inspection run at root CWD, NOT a committed tool). Practice:
RTL dumps go to a .run/ CWD or -dumpbase .run/gccdump (R12)
- make sig-overlays (134 sigs); worklist.py --refresh -> build_fuel_manifest + backlog
render: docs/worklist.md + docs/backlog.md regenerated (stale Jun-26 ranking killed;
the banked whale func_80144B9C dropped from backlog, P9)
- fix tools/sig_image.py docstring: h_norm is the live norm_stream normalizer, not a
"T5 placeholder" (R21/R30)
- CURRENT_PHASE.md: Phase-25 plan-of-record + T0 frontier survey (R14): h_norm 2,764
multi-member families / 11.1 MB; 46% of large solo monsters (3,989/8,670) are family
members; reach-134 h_exact warm-up = 130 classes / 4.11 MB
- func_801770E0 (152 ins, reach-134) propagated ×134 via dedup_propagate --recover.
Chose the NATIVE DEFINE-macro path over a hand-rolled shared header: func_8014E048
(pins+asm) is already ×134 via a DEFINE macro, proving that path handles pin/asm -O2
giants (the whale needed a shared header only because it is -O0 -> separate object).
Clean fleet check-all 136/136 (R22), dedup-check 1811->1812/0.
- R14: func_8014E048 was ALREADY ×134 (T6 §A) — the whale-session handoff was stale;
only func_801770E0 + func_801372B0 actually remained ×1.
- tooling (reusable): dedup_propagate.overlay_files now also scans the whale-rollout
_o0b/_after splits — post-whale-region fns (func_801770E0 in _after.c) were invisible
for both source-def-find and stub-replacement. func_801770E0 extern block made
contiguous (a comment between externs made find_site drop 5 externs; byte-neutral).
The single biggest byte-weight lever (770 ins ×134 ~ +1.6% byte-weight), banked across all 134
overlays as -O0 compiled C (was ×1 in ov_SC01_077, commit:0463).
- ROLLOUT (tools/rollout_whale_o0.py): per single-file overlay, LINE-split <ov>.c at the whale
(splat emits in vram order), carve the yaml code subseg into before/<ov>_o0b(-O0)/<ov>_after,
thin <ov>_o0b.c #includes the shared src/shared/func_80144B9C.h (the whale C moved there — DRY,
TU-local typedefs, not a fragile 200-line DEFINE_ macro). Makefile WHALE_O0B_OBJS wildcard -O0.
- DEDUP: group E_func_80144B9C (134 members, h_exact 74186b97, source=the header; group_members
keys on binary+vram so a header-share validates like a macro-share).
- MILESTONE: clean fleet check-all 136/136 (R22); dedup-check 1811/0; FLEET REAL 225816->225949,
byte-identical 65.91->65.95% function-count (the gain is byte-weighted). Validated on ov_SC01_000
+ spot-checks SC02/SC03/SC07 before the full fleet.
- cookbook §38 + SETUP inventory.
- CRACK (cheap Opus, no Fable5/calls.c): the 770-ins reach-134 whale's 2-insn residual was a
STRUCT ASSIGN `D_80078E50 = *ent;` (sizeof(ENT)==0x24), NOT an explicit memcpy() call — gcc-2.7.2
-O0 expands a >MOVE_RATIO-word struct copy via emit_block_move->emit_library_call(memcpy),
precomputing dst/src into pseudos then addu into $a0/$a1 = the exact 2 missing moves.
- -O0 SPLIT: whale is a 2nd -O0 region (prologue 21F0A003) in the -O2 'after' segment; carved into
its own -O0 object o0b (splat.ov_SC01_077.yaml 3-way + Makefile CC1FLAGS=-O0; address-sorted
line-split preserves all banked C). ov_SC01_077 == d19c9580.
- memcpy: memcpy=0x8005C324 in symbols.resident.txt (overlays+resident only; main untouched vs its
MEMCPY.o) + __asm__("memcpy") on the engine_core.h block-copy macro (non-builtin C name -> no
built-in codegen, same call).
- MILESTONE: CLEAN fleet check-all 136/136 byte-identical (R22), dedup 1810/0. reach-134 -> ×134 (W9) next.
- R14: diagnosed the prior "×134 BLOCKED / all overlays excluded" by RUNNING it — refuted.
func_80129CF8's ×1 bank was already fleet-canonical; dedup_propagate --recover propagates it
to all 134 overlays byte-identical (no new tool needed). The "wall" was a stale-asm /
incremental-tree artifact (the T5c/T6-§A trap).
- banked ×134 via existing dedup_propagate --recover; clean fleet check-all 136/136,
fleet 64.90→65.52%, dedup 1799→1800/0, 0 NON_MATCHING (G4).
- T7b reframed: the reconcile tool automates the manual decl-reconcile for FRESHLY-matched
giant drafts (the 6 sibling giants + wave tail), not an 'unlock' of already-reconciled giants.
- CURRENT_PHASE: T1 done + the R14 finding logged; plan plan-…mossy-dawn.md (T1-T6).
CURRENT_PHASE: T7 progress-log entry (func_80129CF8 crack + the ×134 loose-typing
finding), T7b (the reconcile tool) is the next task (Opus/plan-mode/fresh), spec at
docs/canonical-decl-reconcile-design.md.
The integration-recovery tool for leaf-MATCH-but-whole-binary-gate-rejected fns ("declaration/TU
plumbing" — the dominant residual gate_stage's canon/cast/sig_unify pipeline doesn't reach).
- tools/recover_integration.py (NEW): batch recovery — gather leaf-MATCH candidates (--auto from
the backlog, drift-checked R14; or --funcs/--from-file) → no-proto their conflicting caller decls
→ gate_stage (byte-gate + log). 2-PASS snapshot/restore: pass 1 finds the bankable set, pass 2
re-banks ONLY winners from the clean snapshot (so non-banks are never corrupted).
- tools/fix_arity_callers.py: extended with --binary — scan+rewrite the overlay's OWN inline caller
decls (src/<bin>/<bin>*.c), not just engine_core.h. That was THE gap: a conflicting caller extern
is often inline in the overlay src (e.g. func_8016E778's `extern void f(void)` vs def `f(int)`),
which fix_arity_callers never saw -> the fn stayed unbanked.
- VALIDATED: banked 13 leaf-MATCH fns (func_8014F74C/801542A4/8015BE94/8015F380/80160F00/801653B8/
80166244/8016E778/801732C4/8017331C/80173374/80174554/801745AC), CLEAN-verified together
(ov_SC01_077 d19c9580). The banks themselves are reverted here (they re-bank via the tool and
will land ×134 once propagation-recovery lands — cleaner than committing ×1).
- R14 lesson (clean-verify caught it, R22): fix_arity_callers --revert is LOSSY for --any-proto
(()->(void), not back to the original args) -> corrupted non-banks; fixed with the 2-pass snapshot.
- REMAINING T6 (×134 propagation-recovery, 3 diagnosed blockers): (1) dedup_propagate find_site
misses INDENTED inline defs (Phase-15 class); (2) overlay-local-type lift; (3) auto-reconcile the
straggler's conflicting caller externs (the flagship func_80132784 / ov_SC02_005 class, done by
hand — needs automating). See CURRENT_PHASE.
Pre-existing latent breakage (Phase-21 close commit:0292, "+62 PsyQ names"), found during T5b's
R22 fleet verify. A genuinely-clean `make clean` + re-extract + build of `main` failed with
dozens of `can't open asm/nonmatchings/800c3/func_*.s` — clean-rebuild broken since Phase 21,
masked all along by incremental builds reusing stale pre-rename .s/.o (the exact R22 failure mode).
- Root cause (byte-proven): Phase-21 xdedup renamed 62 PsyQ library functions to their proper
names (InitHeap, FlushCache, GetTPage, SysEnqIntRP, SpuWrite, CdMix, __main …) in
symbols.us.txt, but never regenerated/renamed the committed src/*.c stub refs that call them by
the OLD func_<ADDR> name. splat's FRESH regeneration of a stub .c uses the CURATED names (move
src/800c3.c aside -> splat writes INCLUDE_ASM(InitHeap) + emits InitHeap.s); the committed stubs
were simply stale.
- Fix: rename all 62 INCLUDE_ASM(func_<ADDR>) -> the curated name across 12 files (800c.c 1,
800c3.c 22, apicard1/2/4 4/6/2, boot.c 1, libcd1.c 8, libetc.c 6, libgpu.c 3, sgap.c 1,
snd1.c 7, snd2.c 1). Pure rename to match splat's canonical output; byte-neutral.
- VERIFIED: main clean-builds 143dbb89 from a fully clean tree; full clean fleet check-all 136/136.
- Lesson (cookbook): a symbols.us.txt rename must be propagated to (a) shared-macro bodies
(engine_core.h — the T5b GetTPage fix) AND (b) INCLUDE_ASM stub refs (this), AND verified by a
genuinely-clean check-all (make clean + full re-extract), never incremental. Added a lint check.
Both concerns byte-verified from a fully-clean tree (ov_SC01_077 d19c9580, ov_SC07_009 2a6499b6
+ 133 overlays). Fleet is 135/136 — the 1 remaining failure is `main`, a SEPARATE pre-existing
Phase-21 breakage (62 dangling INCLUDE_ASM refs, tracked in CURRENT_PHASE, fix in progress),
NOT introduced by this commit.
[FIX (partial) — pre-existing Phase-21 latent breakage, found during T5b's fleet verify]
- A genuinely-clean `make check-all` failed 135/136 (overlays: `undefined reference to func_80058B40`;
main: dozens of `can't open asm/nonmatchings/800c3/func_*.s`). This commit fixes the OVERLAY side.
- Root cause: Phase-21 xdedup renamed func_80058B40 -> GetTPage in symbols.us.txt (one of the
+62 PsyQ names) but did NOT update the two shared engine_core.h macros that CALL it
(DEFINE_func_80139680, DEFINE_func_8012E28C — in all 134 overlays + ov_SC01_077_a). Since
Phase 21 a clean rebuild couldn't link; incremental builds reused stale .o's and masked it
(the R22 failure mode) — every "check-all 136/136" Phase 21->23 was incrementally-stale.
- Scope = exactly 1 symbol (static scan of all func_/D_ refs in the shared headers vs symbols).
- Fix: rename the 4 occurrences -> GetTPage (byte-neutral, same addr 0x80058b40 -> identical jal;
G6 curated-name). ov_SC07_009 link-fail -> byte-identical 2a6499b6.
- Lesson: a symbols.us.txt rename must be propagated to shared-macro bodies AND verified by a
genuinely clean (make clean + full re-extract) check-all, never incremental.
[FEAT — T5b: the S11 class crack, Fable5 spike]
- The S11 LUID(x)alloc "intrinsic" verdict was MAP-INCOMPLETENESS. func_8014E048 (143 ins,
reach-134; "not source-steerable" since the map wave; 28-off even after T5's directed permuter)
-> MATCH (143/143) -> whole-binary BANKED (ov_SC01_077 d19c9580). Derived by reading gcc-2.7.2
source + RTL dumps (12 experiments .run/gccmap/exp/e1a..e1k.c).
- NEW LEVERS (byte-proven): S12 reused-s32-temp fence (u16 temps DON'T work — combine folds the
unpromoted-HI zext temps away); S13 head-skip escape (body-local param copies conflict-steer the
scratch contest; volatile-asm dead-read wedge fence; multi-input dead-read K2 rebalance);
cse-opaque asm-copy; RC-4b pinned store-temp; RC-10 preference-cascade mechanics.
- Integration reconciles (both T6 classes): engine_core.h caller decl s16*->u16* (codegen-neutral)
+ canonical data decls w/ *(u16*) casts (D_801152A8 u8[] / D_801152AC s16). x134 lift blocked by
dedup_propagate self-containment (pins/asm) -> T6 target (joins func_80132784).
- Distilled (R30/R16): sched.md §6 (S12/S13) + regalloc.md §F (RC-10 + RC-6/S11 downgrade) +
cookbook §31 triage update; backlog re-logged (capped); memory updated.
- p16_permute.hide_asm: b64literal-pragma carrier for register pins + GTE __asm__ blocks.
pycparser parses the pragma; decomp-permuter's process_pragmas decodes it back so cc1 sees
the real pins/asm (regalloc steered, mvmva compiles). No submodule edit (reuses its own carrier).
- drop_preproc_and_scalar_typedefs: keep #define + custom struct/typedefs, drop only #include
(fixes func_801412A8 'OTLINK undeclared'); +f32 typedef gap; make_base_c hides asm.
- compile.sh/compile_o0.sh: prepend .include "macro.inc" so GTE mvmva assembles (the real build
gets it via include_asm.h, which base.c omits + -DPERMUTER disables). Byte-neutral for non-GTE.
compile_o0.sh (-O0) auto-selected for _o0 targets.
- run_masked.py: expr_type->int fallback for hidden-pin vars -> 0 internal-permuter-failures
(perm_split_assignment/perm_temp_for_expr no longer KeyError on pinned drafts).
- ALL 5 seeds parse+compile (base 4/110/36/52/77). FLAGSHIP func_80132784 (4/400) CLOSED to a
masked-0 that match_one confirms MATCH (400 ins) -> .run/wave/func_80132784.win.c for T4 gate.
- no build-input changed (136/136 untouched)
- cookbook §29: reasoning-model reconciliation idioms (match-pointer-type-to-TU-decl, call-site cast
for value mismatch, cast-a-callee-definition, data-type match) + the narrow-param hard limit
- gen2-mips-matching-model.md + CURRENT_PHASE: Option-3 verdict (GLM reasons the wall expertly but
banks 1/7; wall INTRINSIC, Fable5 §3c triple-confirmed); GLM role = $0.03-0.08/fn hard-band drafter
+ idiom teacher; real lever past the wall = public flip, not a bigger model
- fleet 64.19%, 136/136 byte-clean; injection lever +0.34% banked this session
- ★ NEXT TASK rewritten: build phase-separated + vLLM batched harvester (Drew option c)
with the full fuel map (~4,597 unique ≤15 funcs, SC03=12,497), the throughput finding
(~48s/func + GPU-idle-during-gate), the applied fixes, and the parked items
- supersedes the stale reach≥2/14B plan; 14B-for-giants now AFTER ≤15 saturation
- the model rambled to 4096 tokens on no-stop-token drafts (~80-130s each); a
≤15-ins fn's C is ~100-300 toks, so 512 truncates the ramblers (gate-rejected anyway)
- CURRENT_PHASE: ≤15 band NOT saturated (~4,597 unique / 4,087 untried, SC03=12,497);
drafting ~15-18s/func serial is the throughput wall; architecture = phase-separate +
parallel-gate + vLLM batched serving (the real GPU lever); fresh bank-rate still TBD
Drew's call: the fresh session starts with Path A (lora_grind --min-reach 2 + v3 → setter banks
propagate ×134 + dedup-collapse + flywheel), then Path B (corpus-v4 + a cloud Qwen2.5-Coder-14B
for the giant band). Rationale: A is $0/immediate and its banks enrich B's training corpus.
Refresh the resume blocks for a fresh session: state (v3 current model, serve via
serve_local.py — LM Studio ejected, fleet 63.82%); T8 corpus-v3 + T9 operating-loop marked
DONE; NEXT = two paths (A: lora_grind --min-reach 2 with v3 → setter banks propagate ×134 +
dedup-collapse; B: corpus-v4 + a cloud 14B for the giant band, gate now satisfied); serve+run
command updated to serve_local.py + v3; Decisions/Guardrails/Reuse updated off LM Studio.
The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.
Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.
TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.
check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
Add a sig-based reach oracle + --min-reach N to lora_grind so the mass-run can prefer
SHARED functions (one bank propagates x reach — the fleet-% multiplier). The oracle reads
the same .run/sig.ov_*.jsonl dedup_propagate uses (validated: 0 mismatch over 60 stubs +
the func_8017CE24=2 ground truth), so a reach>=N target is exactly one dedup_propagate
will stamp x reach after the bank.
Bounded reach>=2 mass-run (ov_SC01_000, 15 shared <=15-ins stubs): 0/15 banked, vs the
reach-1-heavy spot-run's 7/15. The model is weakest exactly on reach>=2 (shared) code:
(1) the corpus skipped the shared DEFINE_func macro bodies (export_pairs reads only src/
defs -> 96.6% of the corpus is overlay-unique), and (2) the shared engine fns are the
harder regalloc/schedule residuals. So reach>=2 model-only is NOT a fleet lever by itself.
BUT the reach>=2 drafts are high-value FUEL: 5/15 are close<=3 reach-134 near-misses
(3x close=1: func_8012E27C/BF4C/AD64) -> x134 each if closed. The real lever is
reach>=2 draft -> permuter-grinder close (x134), which needs the SAME per-binary fix T7
applied to lora_grind: grinder.py calls run_gate with no binary (-> 077) and the backlog
stores no binary field. That two-part fix is the next step. byte-neutral: check-all 136/136.
- docs/gen2-mips-matching-model.md: T9 RESULT
- phase-ends/CURRENT_PHASE.md: T9 done; next = grinder per-binary fix, then corpus-v3
The 500-fn calibration banked 0/222 across the binary rotation. Root-caused (R14, by
reading the code + the run's own backlog — resolving a flat contradiction between two
scout agents) to TWO independent bugs in lora_grind's use of gate_stage.run_gate, NOT
model quality:
- Bug A: good_sha() passed the sha1sum line "<sha> <name>" vs harvest_verify's bare
sha1() -> 0 banks for EVERY binary incl. 077 (so the "0/12" was a bug artifact, not
an exhausted tail)
- Bug B: the gate call left src/asm/out at the hardcoded ov_SC01_077 defaults -> non-077
drafts dropped at the 077 stub-filter, silently (and the asm mis-resolution contaminated
the backlog near-miss classification)
Fix (tools/gate_stage.py): run_gate resolves src/asm/out/good_sha from `binary` when unset
(binary-agnostic, no silent ov_SC01_077 default an overlay inherits; good_sha bare-hash
normalized) + a loud negative-control guard (0-overlap binary/src mismatch warns, so a 0
can never again masquerade as 'nothing matched'). tools/lora_grind.good_sha fixed at source.
Byte-neutral: make check-all 136/136.
Proof: ov_SC01_000 spot-run banked 7/15 (47%) byte-identical (@commit:0322); reach-2
func_8017CE24 propagated x2. ROI finding: 6/7 banks are reach-1 (overlay-unique) -> broad
rotation is high bank-RATE / low fleet-% ROI; the fleet lever is reach>=2 targeting (T9) +
corpus-v3 (T8). Backlog now correctly classified (4x close=1 = grinder fuel).
- docs/gen2-mips-matching-model.md: T7 RESULT section
- phase-ends/CURRENT_PHASE.md: T7 done; next = T8 corpus-v3 / T9 reach>=2 selection
Drew T