The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.
Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.
TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.
check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
The 500-fn calibration banked 0/222 across the binary rotation. Root-caused (R14, by
reading the code + the run's own backlog — resolving a flat contradiction between two
scout agents) to TWO independent bugs in lora_grind's use of gate_stage.run_gate, NOT
model quality:
- Bug A: good_sha() passed the sha1sum line "<sha> <name>" vs harvest_verify's bare
sha1() -> 0 banks for EVERY binary incl. 077 (so the "0/12" was a bug artifact, not
an exhausted tail)
- Bug B: the gate call left src/asm/out at the hardcoded ov_SC01_077 defaults -> non-077
drafts dropped at the 077 stub-filter, silently (and the asm mis-resolution contaminated
the backlog near-miss classification)
Fix (tools/gate_stage.py): run_gate resolves src/asm/out/good_sha from `binary` when unset
(binary-agnostic, no silent ov_SC01_077 default an overlay inherits; good_sha bare-hash
normalized) + a loud negative-control guard (0-overlap binary/src mismatch warns, so a 0
can never again masquerade as 'nothing matched'). tools/lora_grind.good_sha fixed at source.
Byte-neutral: make check-all 136/136.
Proof: ov_SC01_000 spot-run banked 7/15 (47%) byte-identical (@commit:0322); reach-2
func_8017CE24 propagated x2. ROI finding: 6/7 banks are reach-1 (overlay-unique) -> broad
rotation is high bank-RATE / low fleet-% ROI; the fleet lever is reach>=2 targeting (T9) +
corpus-v3 (T8). Backlog now correctly classified (4x close=1 = grinder fuel).
- docs/gen2-mips-matching-model.md: T7 RESULT section
- phase-ends/CURRENT_PHASE.md: T7 done; next = T8 corpus-v3 / T9 reach>=2 selection
The fine-tuned 7B (bfm-match-7b-v2) drafted real OPEN ov_SC01_077 stubs; whole-binary gate banked 4
(func_80160B34 func_8015CC74 func_8016084C func_801705C0). Sample: 9/22 match_one proxy -> 4/22
whole-binary banked (18%; the proxy->gate gap is the TU-plumbing wall). Model is format-robust (raw .s
== normalized). api_draft: NORMALIZE_ASM bridge (unused — model handles raw .s) + ghidra_c-empty fix.
- docs/{worklist,backlog,progress.fleet}.md regenerated after the func_8015126C x134 bank
- CURRENT_PHASE.md: RESUME HERE block (T2 giant queue + levers + op gotchas) for a fresh session
- grinder stopped (STOP sentinel set); tree clean except R23 db churn
The 2nd straggler class (cont.4 wave-2): cross-overlay loose-typing COMPILE errors (not just
the -O0 byte-mismatch class). The -O0 exclusion alone was insufficient. Refactor the propagation
into a drop-straggler retry loop: extract apply_plan(subplan, restrict=); on a byte-gate failure,
isolate the culprit(s) for the failing overlay (per-fn trial), drop them (kept x1), retry the
survivors. Handles BOTH straggler classes failing ANY overlay; plan strictly shrinks (terminates);
byte-gate stays sole arbiter (a dropped fn never banks where it isn't byte-identical).
Validated end-to-end: wave-2 --auto-from auto-dropped 2 loose-typing stragglers + propagated 3
clean x134 (commit:0271); fleet 62.70 -> 62.82%. Future _a waves now auto-realize x134, no manual probe.
- worker_wave.js: derive match_one self-check --asm-subdir per-target from t.asm
(was hardcoded main subdir -> every _a agent self-check crashed on missing .s ->
blind drafting). Enabled the 16/24 close rate on the _a measurement wave.
- CURRENT_PHASE cont.4: the _a vein is x134 (10/11 propagate byte-identical), not the
cont.3/§24 x1 wall. The gate's propagated:0 was 2 bugs: dedup_propagate all-or-nothing
batch-revert poisoned by 1 stale -O0 overlay-local straggler (func_8013C360), and
gate_stage sh() swallows dedup_propagate's non-zero exit. Realized +0.39% (commit:0267).
- grinder.py: persistent blacklist for permuter-won/gate-rejected (plumbing-bound) fns;
stops the tried.clear()-after-idle churn (was 0 banks in 8h on func_8014F3E8/func_8014FE60)
- sig_unify.py rewrite_def: adopt the canonical param list on arity mismatch (DEF-side
loose-typing wall) — unused extras sit in $a0-$a3, free at -O2; banked func_8016EDEC/EE40
- cookbook §22 (DEF-side recovery + grinder blacklist); CURRENT_PHASE option-C log
- compounds: sig_unify is in the gate pipeline -> every future wave + the grinder auto-recover
- gate_stage on the 15 worker drafts saved before the reboot: banked 2 byte-matches,
propagated x134 (dedup 1508->1510); the gate process was killed pre-commit (empty log),
banks recovered from the working tree (dedup_propagate fail-closed per-overlay = verified).
Remaining 13 -> backlog near-misses.
- tools/cast_call_sites.py: per draft, callee whose canonical TU sig != draft's
intended sig -> decl->canonical (kills in-TU conflicting types, keeps symbol in
scope) + call-site cast ((ret(*)(args))func_X)(args) (codegen-neutral: gcc folds
the cast of a known symbol -> direct jal). Pure --in/--out; whole-binary gate is
the arbiter (G3/P9). Pipeline: canon_resident_calls -> cast_call_sites ->
sig_unify -> harvest_verify --chunk 1
- recovered 6 of T6 batch-1's 33 gate-fails byte-identical (func_80153C44/8015CF58/
801711FC/80161BE0/801683D8/8015F948); 5 propagated x134 + 1 local; check-all 136/136
- R14 CORRECTION of cookbook §20's '~33' projection (byte-proven): the batch is NOT
mostly callee-conflicts — ~6 callee (cast-recovered), ~18 DEF-side loose-typing wall
(def's matchable sig conflicts with the unchangeable caller-canonical; caller-side
fix BLOCKED because INCLUDE_ASM declares no symbol -> a shared macro's internal
extern is load-bearing in the 133 stub overlays), + a small DATA tail. The def-side
needs re-drafting under the caller-canonical sig (future wave), not a text transform
- cookbook §20 RESOLUTION + SETUP tool inventory (R16/R21/R30); CURRENT_PHASE log (R28)
- fleet 58.63% -> 58.82%, dedup 1501 -> 1508, 0 NON_MATCHING (G4)
- build_engine_types.py --strip: lifted 48 named types -> src/shared/engine_types.h
(byte-neutral, ov_SC01_077 d19c9580); unblocked +4 struct-using matches -> propagated ×134
(incl. func_80147514, a T2 match freed by the lift)
- check-all 136/136 BYTE-IDENTICAL (R22); fleet 57.93% -> 58.00%; phase total 56.64% -> 58.00%
- KEY FINDING: propagation (not matching) now caps the fleet-% yield — ~16 banked matches stay
ov_SC01_077-local because their bodies use typedef'd/anonymous/sibling local types that the
named-struct-only build_engine_types can't lift (dedup_propagate compiles_standalone rejects).
Phase-20 lever: extend the type-lift to typedefs -> recovers those ×134 for ~0 agent tokens
- R14 finding: the 3 Phase-18 "implicit-int / propagate-first" near-misses are NOT
implicit-int (reproduced each through the byte-gate). Real classes:
(a) resident-callee LINK-miss: draft calls func_<ADDR> but the addr has a curated
name (0x8004CFEC = ratan2) -> "undefined reference to func_8004CFEC". NEW
tools/canon_resident_calls.py rewrites func_<ADDR> -> curated name (pure draft
text, body bytes unchanged) -> run FIRST in the recovery pipeline.
(b) shared-caller ARITY conflict: an engine_core.h caller macro declares the callee
file-scope `extern void func_X(void)` then calls func_X(); the real def takes an
arg -> "conflicting types". FIX = caller extern -> no-proto `extern void func_X();`
(byte-neutral for the caller; compatible with promotion-safe params int/long/ptr).
- 3 matches banked in ov_SC01_077: func_8017209C (ratan2 rewrite), func_80147514 (s32),
func_80168F40 (void*); 2 no-proto lines in src/shared/engine_core.h
- recovery pipeline: draft -> canon_resident_calls -> sig_unify -> harvest_verify --chunk 1
- make check-all 136/136 BYTE-IDENTICAL (full extract, R22); dedup 1450 validated/0 failed;
0 NON_MATCHING in any default build (G4); fleet 56.64% (the 3 are ov_SC01_077-local,
x134 propagation rides T3/Phase-20)
- cookbook §17a-3 corrected (R14/R16); SETUP tool inventory (R21); CURRENT_PHASE T2 done
The 'unsteerable' regalloc-order class is NOT unsteerable. Drew's directive (hand-match
everything) exposed the lever I'd skipped: explicit register pinning.
- func_8012B8E4 (the flagship circular reach-134 'unsteerable' fn): 21 -> MATCH via
* register pins: register s32 d __asm__("$16"); register s32 s1ang __asm__("$17")
-> FORCES the $s0/$s1 global allocation gcc otherwise density-orders the other way
* branch-polarity invert (§3-T4): 24->21
* explicit clamp temps (t=u6+0x1000; iVar4=u5-t): fixes the reassociation, 7->3
* scheduling barrier __asm__ __volatile__("" : : "r"(u5)): anchors the last sll, 3->MATCH
- byte-gated (harvest_verify, G3/P9) + propagated x134 (per-overlay gate); 134 byte-identical
- recipe = §10-family hand technique (pins + barriers); labor-intensive (per-fn register map
from the asm) but it WORKS -> the circular tail IS hand-matchable, not asm-only
- supersedes the T1-T5 'unsteerable' verdict; cookbook §17 + PhaseEnd to be revised accordingly
- banked 42 gate-proven SIG_FIXABLE_KR matches in ov_SC01_077 (of 48 predicted; 6 honestly
reverted by the whole-binary byte-gate, P9) + propagated 13 h_exact-shared x134 via
dedup_propagate (one body -> 134 overlays, each byte-gated, fail-closed)
- fleet byte-identical-from-source 54.48% -> 55.00% (+0.52%); REAL +1,778; dedup groups
1395 -> 1408; make check-all 136/136 byte-identical (R22); dedup-check 1408 validated/0 failed
- wall probe (R14): raw drafts 0/99 + sig_unify fails -> the K&R/sig_unify avenue is exhausted;
the residual conflict-class is ARITY_WALL (the §14e param-count dead-end) or structural (T4/T5)
- FIX a T1 classifier bug surfaced by verifying the wall vs the bytes: tu_compile_error captured
the baseline's pre-existing byte-NEUTRAL warnings ("makes integer from pointer", cc1 rc=0) and
mislabeled 99 fns LOOSE_TYPING_WALL; now keeps only non-warning errors -> there is NO loose-typing
wall, the real residual wall is ARITY_WALL=74. docs/wall-taxonomy.md amended with the correction.
- 134 overlay .c files carry the propagated DEFINE_func instantiations; a one-time full-fleet
make extract was required for the per-overlay propagation gate + check-all
- tools/sig_unify.py (NEW): unifies a draft's FULL signature set to the banked-canonical
decls — both callee externs AND the draft's OWN definition signature (return + param
types, body param-names preserved). Recovery for the standalone-MATCH residual: a probe
found 30/30 sampled failures were type conflicts, 0 false-positives. Gated 191 -> 32 verified.
- tools/dedup_propagate.py find_site FIX: the inline-def detector required the opening brace
ON THE SAME LINE as the signature, silently dropping every next-line-brace def from
propagation (sig_unify/permuter outputs + ~stragglers all session). Now accepts brace on
the same OR next line -> unlocked a 61-function propagation backlog.
- propagated 61 fns fleet-wide (incl. 32 sig-unify + the permuter win func_801508F8 +
earlier-dropped next-line-brace matches), all 134 byte-identical; dedup-check 1394/0
- R22 CLEAN fleet rebuild: 136/136 byte-identical. REAL 177501 -> 185646. Zero agent tokens.
- fix tools/build_engine_types.py to be ADDITIVE: merge new source types with the
types already in engine_types.h (a re-run after a prior --strip was destructive —
it dropped the ~34 already-migrated types and broke the build; caught + reverted)
- engine_types.h 34 -> 42 named types (the 8 from wave-1's struct-using matches);
inline defs stripped from ov_SC01_077.c (byte-neutral)
- dedup_propagate --auto-from: 8 of 15 struct-blocked matches now self-contained ->
propagated across all 134 overlays (byte-gated); 7 remain (anonymous/typedef'd types)
- R22 CLEAN fleet rebuild: 136/136 byte-identical; dedup-check 1333/0
- zero agent tokens (deterministic)
- canon_draft_decls recovery on the combined 505 quarantined drafts (285 small + 220 medium),
now canonicalizing against the much-larger banked symbol set: 47 verified (no API), 42
propagated fleet-wide (struct/local-type skipped), 134 overlays byte-identical; dedup
1141 -> 1183 groups, dedup_integrate --check 1183 validated / 0 failed.
- fleet REAL substantive 151,908 -> 157,541; byte-identical 44.64% -> 46.27%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- scoped the struct-using-shared-fn lever (read-only): 34 named struct/union types in 077,
ZERO same-name-different-layout collisions -> a shared-types-header is a clean path to
propagate the ~100 struct-using fns (next).
- gen_harvest_targets.py: now also resolves canonical DATA-symbol declarations per target
(scan the .s for D_XXXX refs -> `extern <type> D_XXXX;` from banked code) in addition to
callee function sigs — the §14c(c) fix at the source, so agents declare both functions and
data correctly and avoid the dominant remaining conflict class.
- v2 harvest (callee+data-sig-aware) on the 591 small still-stub call-heavy fns + a gap-fill
agent for 6 targets dropped by a connection-closed-mid-response failure (the retry-wave
misses truncated-non-null results). Gate: 306 verified (ov_SC01_077 1554 -> 1248 stubs).
- dedup_propagate.py: skip any struct/union-touching body — agents named structs with colliding
generic names (`struct S`/`struct vec`) inline, so two macros' types redefine/conflict when
instantiated in one overlay (ov_SC01_000 abort). 34 skip; struct-using shared fns stay banked
in 077 (shared-types-header follow-up).
- 281 functions propagated fleet-wide (134 overlays each, every overlay byte-gated); dedup
795 -> 1076 groups, dedup_integrate --check 1076 validated / 0 failed.
- fleet REAL substantive 105,764 -> 143,322; byte-identical 31.22% -> 42.14%.
- R22 clean rebuild: 136/136 byte-identical, 0 extract failures.
- cookbook §12: gap-fill is now a MANDATORY post-step for every multi-agent run (reconcile
produced-vs-expected: missing + truncated drafts) before gating.
Drew T