Commit Graph

195 Commits

Author SHA1 Message Date
Drew T 0cda15c6de fix(phase-24): T5c — restore main's clean-rebuild: propagate the Phase-21 PsyQ renames to 62 stub refs
Pre-existing latent breakage (Phase-21 close commit:0292, "+62 PsyQ names"), found during T5b's
R22 fleet verify. A genuinely-clean `make clean` + re-extract + build of `main` failed with
dozens of `can't open asm/nonmatchings/800c3/func_*.s` — clean-rebuild broken since Phase 21,
masked all along by incremental builds reusing stale pre-rename .s/.o (the exact R22 failure mode).

- Root cause (byte-proven): Phase-21 xdedup renamed 62 PsyQ library functions to their proper
  names (InitHeap, FlushCache, GetTPage, SysEnqIntRP, SpuWrite, CdMix, __main …) in
  symbols.us.txt, but never regenerated/renamed the committed src/*.c stub refs that call them by
  the OLD func_<ADDR> name. splat's FRESH regeneration of a stub .c uses the CURATED names (move
  src/800c3.c aside -> splat writes INCLUDE_ASM(InitHeap) + emits InitHeap.s); the committed stubs
  were simply stale.
- Fix: rename all 62 INCLUDE_ASM(func_<ADDR>) -> the curated name across 12 files (800c.c 1,
  800c3.c 22, apicard1/2/4 4/6/2, boot.c 1, libcd1.c 8, libetc.c 6, libgpu.c 3, sgap.c 1,
  snd1.c 7, snd2.c 1). Pure rename to match splat's canonical output; byte-neutral.
- VERIFIED: main clean-builds 143dbb89 from a fully clean tree; full clean fleet check-all 136/136.
- Lesson (cookbook): a symbols.us.txt rename must be propagated to (a) shared-macro bodies
  (engine_core.h — the T5b GetTPage fix) AND (b) INCLUDE_ASM stub refs (this), AND verified by a
  genuinely-clean check-all (make clean + full re-extract), never incremental. Added a lint check.
2026-07-03 02:50:03 -06:00
Drew T 6c2fb9f04f fix+feat(phase-24): T5b — S11 CRACKED (func_8014E048 banked) + fleet GetTPage linkage fix
Both concerns byte-verified from a fully-clean tree (ov_SC01_077 d19c9580, ov_SC07_009 2a6499b6
+ 133 overlays). Fleet is 135/136 — the 1 remaining failure is `main`, a SEPARATE pre-existing
Phase-21 breakage (62 dangling INCLUDE_ASM refs, tracked in CURRENT_PHASE, fix in progress),
NOT introduced by this commit.

[FIX (partial) — pre-existing Phase-21 latent breakage, found during T5b's fleet verify]
- A genuinely-clean `make check-all` failed 135/136 (overlays: `undefined reference to func_80058B40`;
  main: dozens of `can't open asm/nonmatchings/800c3/func_*.s`). This commit fixes the OVERLAY side.
- Root cause: Phase-21 xdedup renamed func_80058B40 -> GetTPage in symbols.us.txt (one of the
  +62 PsyQ names) but did NOT update the two shared engine_core.h macros that CALL it
  (DEFINE_func_80139680, DEFINE_func_8012E28C — in all 134 overlays + ov_SC01_077_a). Since
  Phase 21 a clean rebuild couldn't link; incremental builds reused stale .o's and masked it
  (the R22 failure mode) — every "check-all 136/136" Phase 21->23 was incrementally-stale.
- Scope = exactly 1 symbol (static scan of all func_/D_ refs in the shared headers vs symbols).
- Fix: rename the 4 occurrences -> GetTPage (byte-neutral, same addr 0x80058b40 -> identical jal;
  G6 curated-name). ov_SC07_009 link-fail -> byte-identical 2a6499b6.
- Lesson: a symbols.us.txt rename must be propagated to shared-macro bodies AND verified by a
  genuinely clean (make clean + full re-extract) check-all, never incremental.

[FEAT — T5b: the S11 class crack, Fable5 spike]
- The S11 LUID(x)alloc "intrinsic" verdict was MAP-INCOMPLETENESS. func_8014E048 (143 ins,
  reach-134; "not source-steerable" since the map wave; 28-off even after T5's directed permuter)
  -> MATCH (143/143) -> whole-binary BANKED (ov_SC01_077 d19c9580). Derived by reading gcc-2.7.2
  source + RTL dumps (12 experiments .run/gccmap/exp/e1a..e1k.c).
- NEW LEVERS (byte-proven): S12 reused-s32-temp fence (u16 temps DON'T work — combine folds the
  unpromoted-HI zext temps away); S13 head-skip escape (body-local param copies conflict-steer the
  scratch contest; volatile-asm dead-read wedge fence; multi-input dead-read K2 rebalance);
  cse-opaque asm-copy; RC-4b pinned store-temp; RC-10 preference-cascade mechanics.
- Integration reconciles (both T6 classes): engine_core.h caller decl s16*->u16* (codegen-neutral)
  + canonical data decls w/ *(u16*) casts (D_801152A8 u8[] / D_801152AC s16). x134 lift blocked by
  dedup_propagate self-containment (pins/asm) -> T6 target (joins func_80132784).
- Distilled (R30/R16): sched.md §6 (S12/S13) + regalloc.md §F (RC-10 + RC-6/S11 downgrade) +
  cookbook §31 triage update; backlog re-logged (capped); memory updated.
2026-07-03 02:02:26 -06:00
Drew T 0eadf313ee feat(phase-24): T5 — §31-directed permuter mutation + grinder input-changed gating
- tools/permuter_weights.py (NEW): classify(klass,where) -> regalloc|schedule|cse|None
  + render_settings_toml emitting the [weight_overrides] table decomp-permuter merges over
  the gcc defaults (main.py:336 / helpers.py per-key replace). Biases pass-selection toward
  each class's §31 levers (perm_reorder_decls RC-1/3, perm_reorder_stmts RC-2/S1,
  perm_temp_for_expr S2, perm_commutative cse); value/type noise -> ~0.1. NO submodule edit.
- p16_permute.setup(..., klass=, where=) writes it (+--klass, backlog auto-lookup);
  grinder.py auto-threads klass/where_stuck. klass=None -> gcc defaults (undirected superset).
- grinder fix (R14): replace the blind tried.clear() idle churn with input-changed gating
  (draft_sig=(best_draft mtime, closeness) — re-open a fn only when the worker improved it).
- VALIDATED: regalloc profile drove func_8014E048 masked 36->29 (match_one -drz 35->28) where
  the undirected search stalled; re-logged (28, improved draft promoted). Flagship S11/RC-6
  count-exact seeds improve but do not gate (intrinsic, §31 "two probes dont grind") -> T5b fuel.
- distill: cookbook §3b + SETUP inventory (R16/R30). No build-input changed -> 136/136 untouched.
2026-07-03 00:14:29 -06:00
Drew T b2b1d0eeae chore(phase-24): T4 close — flagship banked x1, count-exact seeds -> T5, xN -> T6
- func_80132784 (400 ins) banked x1 via the T2+T3 permuter (commit commit:0438); xN
  propagation capped by per-overlay declaration plumbing (ov_SC02_005 straggler +
  extern/type mismatch) -> logged as a T6 integration-recovery target
- count-exact seeds (E048/76D94/48094/412A8) parse+compile+search clean but need
  T5 directed mutation (intrinsic RC-6/S11 residuals) / T5b Fable5
- regenerated digests; dedup-check 1783 validated/0 failed; fleet 64.86%; P6 rules check done
2026-07-02 23:44:38 -06:00
Drew T 5984749421 feat(phase-24): T2 — floor-free relocation-masked permuter scorer (-drz)
- tools/masked_diff.py: shared objdump -drz masking oracle (jal/j 26-bit + HI16/LO16
  immediate mask; -z keeps nop runs -> no GTE under-count). object-vs-object (permuter,
  +reloc-operand equality) and object-vs-.s (match_one) modes.
- match_one.py refactored onto masked_diff (-dr -> -drz); regression-clean on count-exact
  seeds (35/52/72/110, leaf-MATCHes), output format unchanged (gate_stage/grinder compatible)
- tools/masked_scorer.py MaskedScorer: drop-in for decomp-permuter's Scorer; scores masked
  .text closeness (bottoms out at 0) not the stock mnemonic-diff floor
- tools/permuter/run_masked.py: rebinds src.main.Scorer -> MaskedScorer (NO submodule edit,
  R3/R20); p16_permute.run_permuter wired to it
- VALIDATED: masked_self=0; masked_cand~match_one; STOCK floor 1750-2100 vs masked 36-77
  (the wander cause); live permuter base score = masked 77 (not stock 1930), descends to 74
- R14 SELF-CORRECTION: -drz confirms func_80132784 is 4/400 count-exact (T1's -dr 204 was a
  16-nop-collapse artifact); backlog re-logged close 4. The flagship IS 4 ins away.
- no build-input changed (136/136 untouched); 2 compile FAILs surfaced -> T3 (GTE asm, typedefs)
2026-07-02 23:10:54 -06:00
Drew T 394e3a81b3 chore(phase-24): T1 — re-log map-wave seeds (byte-verified) + backlog/gate_stage hygiene
- re-logged 7 map-wave seeds in .run/backlog.jsonl with match_one-verified
  closeness (dropped 19 stale/unreproducible records, .bak kept):
  func_8014E048=35 (was mis-logged 'failed'), func_80176D94=52,
  func_80148094=72 (best draft vG2, not the named file), func_801412A8=110;
  T6 leaf-MATCHes func_8014F4C0/func_80155800=0; best_draft -> .run/backlog_drafts/
- R14 FINDING: the 'func_80132784 is 4/400' premise is NOT reproducible from any
  on-disk draft (best=204-off, wrong instr count 384!=400 -- match_one -dr collapses
  ~16 nops on this GTE seed); true closeness pends T2's -drz scorer -> T4 reframed
- gate_stage.py: fixed the stale 'sig_unify SKIPPED with --src-file' doc-drift
  (code runs it; sig_unify.py:151 has --src-file support)
- wave_targets --class REGALLOC + grinder now surface the 4 count-exact permuter
  seeds with true closeness; no build-input changed (136/136 invariant untouched)
- added T5b (Fable5 S11/RC-6 map-extension spike) per Drew
2026-07-02 22:55:05 -06:00
Drew T 80b89d8a2d feat(phase-23): LLM matching tier + the Fable5 wall-breaker + the gcc-2.7.2 codegen map (v1.22.0)
- LLM TIER (T1-T10.9): tools/serve_local + api_draft + lora_grind + bulk_harvest + the LoRA
  pipeline (v3 model). Local v3 saturated <=15 (~1300 banks, $0). Frontier GLM-5.2 (~$4)
  confirmed the def-side wall INTRINSIC + the idiom well DRY at the leaf (both angles).
- THE PIVOT (07-02): Fable5Max reading the gcc-2.7.2 SOURCE matched func_8014EE14, a §20/§10
  store-vs-load giant "CONFIRMED unsteerable" for 22 phases -> the "wall is intrinsic" verdict
  is MODEL-RELATIVE. Generalized to Opus agents applying §30 (2 more giants, 3-5x cheaper).
- §31 THE CODEGEN MAP: 4 Fable5 agents read the whole gcc-2.7.2 source -> docs/gcc-2.7.2-map/
  {sched,regalloc,loop,cse_expr}.md (935 lines, byte-proven residual->lever catalogs) +
  cookbook §31 (index+triage). Broke hoist-vs-remat/delay-slot/coalescing walls. Correction:
  gcc-papermario is 2.8.1 not 2.7.2 -> tools/reference/gcc-2.7.2/ (SETUP §5.6).
- THE WAVE (8 reach-134 near-misses, Opus + §31): 3 leaf-MATCH by lookup (47-137k tok vs
  200-250k walls) + 5 tight permuter seeds (func_80132784 240->4-off!), zero dead-ends.
- 5 giants/hard-fns banked x134 (func_8014EE14/F2E0/150528/149374/144090); fleet 63.66->64.86%,
  136/136 byte-identical, 0 NON_MATCHING. cookbook §30/§30a/§31. No new governance rules.
- FINDING: matching is SOLVED by the map; bottlenecks are now the permuter (S11 seeds) +
  whole-binary integration (auto-bank leaf-MATCHes) -> Phase 24. worklog -> logs/Phase23.md.
2026-07-02 21:51:25 -06:00
Drew T 3b6f2c5f94 docs(phase-23): §31 — the gcc-2.7.2 codegen map (4 Fable5 agents read the source)
- docs/gcc-2.7.2-map/{sched,regalloc,loop,cse_expr}.md: source-cited, byte-proven
  pass -> residual -> C-lever catalogs (935 lines). cookbook §31 = the index + triage.
- WALLS BROKEN (byte-proven steerable, were "CONFIRMED unsteerable"): §10/§20
  hoist-vs-remat (cross-call address-caching), store-vs-load (/s), dbr delay-slot
  (D1+S2 fresh-local: func_801770E0 53->49), birthing-boost both directions.
  Incidental bank: func_80149374 x134 (fleet 64.86%).
- GENUINE walls -> permuter: S3 chain-priority sink, S11 LUID(x)alloc coupling,
  RC-6 pressure-lock, cse 1000-insn table flush.
- CORRECTION (loop agent): tools/reference/gcc-papermario is gcc 2.8.1 NOT 2.7.2
  (behavioral biv-elim diff). Vanilla gcc-2.7.2 -> tools/reference/gcc-2.7.2/
  (gitignored); SETUP §5.6 + §31 flag it. All banked levers stand (match_one-validated
  vs the real cc1). Also corrected: spill-slot = declaration order; §25 4th rank rule.
- the payoff: the cheap tier (Opus agents + local model) can now apply compiler-internal
  levers by triage-table lookup, without reading 80k lines of source.
2026-07-02 20:12:19 -06:00
Drew T e74dd52b19 docs(phase-23): cookbook §30a — §30 generalizes via Opus agents + 2 new levers (IV-combine, inline-limit) + mechanical macro-widen integration 2026-07-02 17:34:53 -06:00
Drew T 6678c6868a feat(phase-23): Fable5Max cracks §20 "unsteerable" giant func_8014EE14 ×134
- Fable5Max agent (Agent model=fable) matched a 248-ins reach-134 GIANT on the
  §20/§10 store-vs-load wall (22 phases "CONFIRMED unsteerable") by reading the
  gcc-2.7.2 source (tools/reference/gcc-papermario) + RTL -da dumps. Leaf
  MATCH(248 ins) -> whole-binary banked:1 -> dedup_propagate ×134. Verified:
  check-all 136/136 byte-identical, dedup-check 1780 validated/0 failed.
- 3 byte-proven idioms -> cookbook §30 (corrects §29's "not a bigger model"):
  (1) store-vs-load is a deterministic MEM_IN_STRUCT_P /s aliasing flag, not a
      scheduler tie-break; steer via ((struct{s32 f;}*)p)->f (anon struct keeps
      /s AND propagates ×134) to grant, *p to deny
  (2) def-side return-type wall has a MACRO escape: widen a discarding caller
      macro's extern void->s32 (byte-neutral, check-all-verified) -- extends §29
  (3) birthing-boost prologue-order lever: __asm__("":"=r"(x):"0"(x)) re-tie in a
      later bb kills sched.c's REG_N_SETS==1 priority boost
- tools/glm_parallel.sh: K concurrent OpenRouter/GLM cloud drafters (parallel
  api_draft), key read from .env at runtime
- §10/§20 "store-vs-load unsteerable" backlog now re-test candidates:
  func_8014F2E0, func_80150528, func_8014EA4C
2026-07-02 16:56:56 -06:00
Drew T 6127367017 docs(phase-23): T10.7 conclusion — def-side wall intrinsic (GLM 1/7); GLM = drafter+teacher not wall-breaker
- cookbook §29: reasoning-model reconciliation idioms (match-pointer-type-to-TU-decl, call-site cast
  for value mismatch, cast-a-callee-definition, data-type match) + the narrow-param hard limit
- gen2-mips-matching-model.md + CURRENT_PHASE: Option-3 verdict (GLM reasons the wall expertly but
  banks 1/7; wall INTRINSIC, Fable5 §3c triple-confirmed); GLM role = $0.03-0.08/fn hard-band drafter
  + idiom teacher; real lever past the wall = public flip, not a bigger model
2026-07-01 21:05:18 -06:00
Drew T bce95a13cd docs(phase-23): T10.7 GLM5.2 A/B result — 10x codegen edge, but def-side wall caps banks (3/18)
- GLM5.2 vs v3 on 18 hard-band fns: 10/18 vs 1/18 match_one; 3/18 vs ~1/18 whole-binary bank
- def-side loose-typing wall (Phase 16/20) caps banking for ANY drafter (Fable5 review §3c: HOLDS)
- 6 hardest beyond GLM too (0/6 at MAXTOK=16000); ~$1.02 of $25 spent
- 3 strategic options handed to Drew (direct-drafter / flywheel / wall-reconciliation)
2026-07-01 20:01:14 -06:00
Drew T 2d0d63953e docs(phase-23): Fable5 strategy review — scope-vs-reality scorecard, wall re-tests, resource map
- scorecard: original 2026-06-10 scope vs 22 phases of byte-verified reality (what held,
  what emerged beyond scope, what deviated and should be revisited)
- adversarial pass: P21 no-shortcut + giant scheduler walls HOLD; P16 loose-typing wall has
  a TIMESTAMP GAP (declared 06-19, pre-dating cast_call_sites/block-scope-externs/v3) -> re-test
- July-2026 resources: frontier-on-hard-band (T10.7 re-aim), continuous architect-tier judgment,
  RE-ELEVATE THE PUBLIC FLIP (community labor = the only lever that scales into the proven tail)
- strategic fork: posture A/B/C on the byte-match goal; recommends dual-metric (B), Drew's call
- ranked recs 1-6 + explicit endorsements of what not to change
2026-07-01 17:25:47 -06:00
Drew T 70aaad4efb chore(phase-23): regenerate backlog + fleet digests (≤15 campaign end, fleet 64.6%) 2026-07-01 14:20:12 -06:00
Drew T 8dc42f12b5 docs(gen3): parking-lot for asset export/Unity + verified mesh-state survey
- docs/gen3-parking-lot.md (NEW): town mesh/texture + Musashi->Unity export ideas, PARKED (Gen3).
  Verified survey (read-only, 2026-07-01): BFM uses the libgs TMD model path — real jal
  GsMapModelingData + GsLinkObject5 call sites (EXE 0x8001C2xx-0x8001C9xx model subsystem +
  ov_SC02_005); render = stock libgpu; no libgs sort/anim calls (custom anim -> Musashi harder);
  textures via VRAM rip + type-0 'graphics' blobs; no exporter tooling yet. Decisive next probes listed.
- CURRENT_PHASE.md: T10 done + T10.6 campaign in-flight + T10.7 OpenRouter-next notes (crash-recovery)
2026-07-01 00:52:23 -06:00
Drew T 265e930f65 chore(phase-23): regenerate backlog + fleet digests (T10 banks) 2026-06-30 23:23:04 -06:00
Drew T e9f199530e docs(phase-23): T10 harvester built + measured — 65% bank-rate, parallel gate ~75x, vLLM justified
- gen2-mips-matching-model.md: T10 RESULT (bulk_harvest phase-separation; 52/80=65% on fresh
  SC03 <=15-ins; gate 0.4s/fn/8w vs ~30s/fn serial; drafting now the sole bottleneck -> vLLM)
- SETUP.md: bulk_harvest.py inventory row + gate_stage/harvest_verify parallel-safe params (R21)
- CURRENT_PHASE.md: T10 done (built+measured), T10.5 vLLM split out as Drew-gated; progress log
2026-06-30 23:22:46 -06:00
Drew T 72dabb470c feat(phase-23): broad v3 harvest (48 banks) + final propagate sweep; docs
- bounded broad v3 harvest banked 48 small fns across SC01/SC02 overlays (mostly
  reach-1 ×1; fleet 64.16->64.18%), final propagate sweep +4 groups (1687->1691)
- check-all 136/136 byte-identical, dedup-check 0 failed
- docs/community-matching-model-plan.md (publishing the drafter — parked)
- docs/sister-decomps.md (VS cross-project plan + idiom-discovery rationale — parked)
- CURRENT_PHASE.md: 2026-07-01 session arc (injection lever +0.34%, the type-walled-14
  deferred, the saturation-campaign plan)
2026-06-30 13:54:53 -06:00
Drew T fdd3bf3c28 feat(phase-23): inject-extern lever on _a.c — +3 reach-134 fns (fleet 64.09→64.16%)
- inject_capped_externs --src-file ov_SC01_077_a.c freed 3 more reach-134 fns
  (0x8012A018/0x8012BA10/0x8012FDA8); 077 byte-identical; dedup_propagate
  registered 2 clean ×134 groups + 2 ×133 (ov_SC01_000 straggler kept ×1)
- 1685->1687 shared groups, check-all 136/136, dedup-check 0 failed
- injection lever total this session: 63.82% -> 64.16% (+0.34%, ~0 LLM tokens)
2026-06-30 11:20:13 -06:00
Drew T 205fc202e9 feat(phase-23): macro-extern-injection lever — 8 reach-134 fns freed ×134 (fleet 63.82→64.09%)
- tools/inject_capped_externs.py (new): source each propagation-capped reach≥2
  inline match's externs from ov_SC01_077's own file-scope decls, inject them
  block-scope so the lifted macro is self-contained (the Phase-20-backlog
  macro-extern-injection lever; byte-neutral, fail-safe via compiles_standalone)
- 8 reach-134 fns freed from ov_SC01_077.c, 077 byte-identical; dedup_propagate
  --auto-from registered 7 new shared groups ×134 (1678->1685), check-all 136/136
- the LLM reach≥2 measurement run already banked +7 (v3 banks reach≥2: 6/15 in 005
  vs v2's 0/15) — committed earlier (commit:0359, commit:0360)
- cookbook §28d + SETUP tooling inventory (R16/R21/R30)
- fleet 63.82% -> 64.09%, 0 NON_MATCHING, dedup-check 1685/0 failed
2026-06-30 11:15:11 -06:00
Drew T ecc95da031 chore(phase-23): regen reports after the reach≥2 measurement run 2026-06-30 11:02:19 -06:00
Drew T 1eadbc73e9 docs(phase-23): 8-hour autonomous run — corpus-v3 + v3 + local serving; fleet 63.67->63.82%
The run (Drew away): built tools/serve_local.py (Unsloth GPU serving, replaces ejected LM
Studio); the prompt fix (translate-every-instruction, 0/3->2/3 on the empty-leaf class);
corpus-v3 (mine the 1623 engine_core.h DEFINE_func macros + struct types: 1312->2891 pairs,
2534+291 trainable); v3 trained (loss 1.275->0.085) eval 23/40 (57.5%); v3 production batch
banked ~352 fns + 45 new shared groups -> fleet 63.67->63.82% (+502 byte-identical), 136/136
byte-clean, 25 auto-commits, $0. v3 banks the empty-leaf/setter class v2 couldn't.

- docs/gen2-mips-matching-model.md: corpus-v3 + serving + prompt-fix + v3 eval + harvest
- docs/SETUP.md: LLM matching tier added to the Tooling inventory (R21)
- phase-ends/CURRENT_PHASE.md: run progress + next levers (reach>=2 targeting, dedup-collapse)
2026-06-30 07:36:33 -06:00
Drew T e91859fb4a feat(phase-23): grinder per-binary fix (5-layer) — unlock non-077 near-miss grinding
The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
   propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
   of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.

Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.

TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
  target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
  empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
  draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
  add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
  local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.

check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
2026-06-30 00:58:13 -06:00
Drew T 366f7e4c7f feat(phase-23): T9 — reach>=2 targeting (lora_grind --min-reach); shared-code is the model's weak band
Add a sig-based reach oracle + --min-reach N to lora_grind so the mass-run can prefer
SHARED functions (one bank propagates x reach — the fleet-% multiplier). The oracle reads
the same .run/sig.ov_*.jsonl dedup_propagate uses (validated: 0 mismatch over 60 stubs +
the func_8017CE24=2 ground truth), so a reach>=N target is exactly one dedup_propagate
will stamp x reach after the bank.

Bounded reach>=2 mass-run (ov_SC01_000, 15 shared <=15-ins stubs): 0/15 banked, vs the
reach-1-heavy spot-run's 7/15. The model is weakest exactly on reach>=2 (shared) code:
(1) the corpus skipped the shared DEFINE_func macro bodies (export_pairs reads only src/
defs -> 96.6% of the corpus is overlay-unique), and (2) the shared engine fns are the
harder regalloc/schedule residuals. So reach>=2 model-only is NOT a fleet lever by itself.

BUT the reach>=2 drafts are high-value FUEL: 5/15 are close<=3 reach-134 near-misses
(3x close=1: func_8012E27C/BF4C/AD64) -> x134 each if closed. The real lever is
reach>=2 draft -> permuter-grinder close (x134), which needs the SAME per-binary fix T7
applied to lora_grind: grinder.py calls run_gate with no binary (-> 077) and the backlog
stores no binary field. That two-part fix is the next step. byte-neutral: check-all 136/136.

- docs/gen2-mips-matching-model.md: T9 RESULT
- phase-ends/CURRENT_PHASE.md: T9 done; next = grinder per-binary fix, then corpus-v3
2026-06-30 00:12:45 -06:00
Drew T bff03dbdb5 fix(phase-23): T7 — lora_grind broad-rotation banking (two gate bugs); ov_SC01_000 7/15
The 500-fn calibration banked 0/222 across the binary rotation. Root-caused (R14, by
reading the code + the run's own backlog — resolving a flat contradiction between two
scout agents) to TWO independent bugs in lora_grind's use of gate_stage.run_gate, NOT
model quality:
- Bug A: good_sha() passed the sha1sum line "<sha>  <name>" vs harvest_verify's bare
  sha1() -> 0 banks for EVERY binary incl. 077 (so the "0/12" was a bug artifact, not
  an exhausted tail)
- Bug B: the gate call left src/asm/out at the hardcoded ov_SC01_077 defaults -> non-077
  drafts dropped at the 077 stub-filter, silently (and the asm mis-resolution contaminated
  the backlog near-miss classification)

Fix (tools/gate_stage.py): run_gate resolves src/asm/out/good_sha from `binary` when unset
(binary-agnostic, no silent ov_SC01_077 default an overlay inherits; good_sha bare-hash
normalized) + a loud negative-control guard (0-overlap binary/src mismatch warns, so a 0
can never again masquerade as 'nothing matched'). tools/lora_grind.good_sha fixed at source.
Byte-neutral: make check-all 136/136.

Proof: ov_SC01_000 spot-run banked 7/15 (47%) byte-identical (@commit:0322); reach-2
func_8017CE24 propagated x2. ROI finding: 6/7 banks are reach-1 (overlay-unique) -> broad
rotation is high bank-RATE / low fleet-% ROI; the fleet lever is reach>=2 targeting (T9) +
corpus-v3 (T8). Backlog now correctly classified (4x close=1 = grinder fuel).

- docs/gen2-mips-matching-model.md: T7 RESULT section
- phase-ends/CURRENT_PHASE.md: T7 done; next = T8 corpus-v3 / T9 reach>=2 selection
2026-06-29 23:44:12 -06:00
Drew T b4c312a30c feat(phase-22): FIRST free-local-model banks — 4 open stubs matched by the v2 LoRA
The fine-tuned 7B (bfm-match-7b-v2) drafted real OPEN ov_SC01_077 stubs; whole-binary gate banked 4
(func_80160B34 func_8015CC74 func_8016084C func_801705C0). Sample: 9/22 match_one proxy -> 4/22
whole-binary banked (18%; the proxy->gate gap is the TU-plumbing wall). Model is format-robust (raw .s
== normalized). api_draft: NORMALIZE_ASM bridge (unused — model handles raw .s) + ghidra_c-empty fix.
2026-06-29 21:58:09 -06:00
Drew T 6a45497a1e docs(phase-22): corpus-v2 result — POSITIVE, data was the bottleneck
Extern-block capture: compile 52->92%, train 638->1111 (non-trivial 257->813). SAME 7B retrained:
6-15 ins 0%->85%, non-trivial 0->26, meaningful(>15) 0->3. A free local 7B now matches trivial+
small-medium (<=15 ins) at 85-93% (rivals Haiku at $0). Limits: >=16 ins falls off (7B capacity),
giants compile-fail (need struct types = v3). Decision gate=GO: scale to cloud dense 14-32B.
2026-06-29 21:33:21 -06:00
Drew T 0a75c2470d docs(phase-22): LoRA pilot result — NEGATIVE on meaningful fns (corpus-starved)
7B QLoRA on 638 compile-filtered pairs: trivial ≤5-ins 39/41, but ≥6 ins 0/34, >15 ins 0/24 (=stock);
near-misses far (near≈nins). Root cause: compile-filter dropped the 536 harder global/struct fns →
no non-trivial signal. Fix = corpus-v2 (externs). Cheap-cloud (Haiku/GLM) remains the working tier.
2026-06-29 19:43:59 -06:00
Drew T 671177aece docs(phase-22): capture stock-local-model floor — 0 reliable banks, motivates the LoRA specialist
Qwen3.6-35B-A3B stock drafter: fair harness fixed compiles but model stuck at fixed near-misses
(can't refine from diff); full cookbook worse+2.3x slower than curated (dilution). Gets structure,
misses gcc-2.7.2 precision — exactly what src-pair LoRA targets. Forward: fine-tune or permuter-seed.
2026-06-29 16:26:29 -06:00
Drew T ad535ad1ab feat(phase-22): gen2 matching-model design + export_pairs.py corpus miner
docs/gen2-mips-matching-model.md: the BFM/gcc-2.7.2 matching-specialist idea (LoRA on our own
gate-verified pairs — the corpus off-the-shelf RE LLMs lack). export_pairs.py mines 1307 banked
(asm<->C) pairs from build objects (asm/ is gitignored, so disasm the ROM-identical build, splat-like
format) + src defs -> datasets/match_pairs/{pairs,train,test}.jsonl (gitignored, 1174/133 split).
api_draft.py: TEMP env-tunable. .gitignore: datasets/ models/ weights.
2026-06-29 14:27:19 -06:00
Drew T f154822f7b feat(phase-22): cheap-tier A/B experiment harness + findings
ab_match.js (per-arm-model fan-out fork of worker_wave) + ab_score.py (disk-truth
match_one scorer, --measured for real $). Haiku drafters vs Opus on 20 reach1 fns:
proxy 10 vs 15 matches, Haiku ~4.8x more matches/$, parity on <=52-ins band. Whole-binary
gate banked 4 (proxy over-counts; TU-plumbing wall). backlog/fleet refreshed.
2026-06-29 12:52:39 -06:00
Drew T a126d2170d docs(phase-22): correct cookbook §28b-6 — the dedup_propagate registry-skip is independent of concurrent make (R14, per §28c) 2026-06-26 22:25:17 -06:00
Drew T 20037480cf docs(phase-22): log func_801372B0 structural near-miss (giant scheduler-wall) + session checkpoint
- from-scratch giant func_801372B0 (207): structurally cracked at xHigh (206/207 ins,
  logic 100%%) but byte-blocked by the gcc list-scheduler/regalloc last-mile (§27/§17);
  banked the structural draft to backlog as a future permuter head-start
- xHigh finding (Drew's test): giants reach structural match fast — the wall is gcc
  determinism, not reasoning depth; xHigh sufficient for this work
- session total: 16 functions ×134 (2 giants type-lift + 14 close=0), fleet 63.22%->63.66%
2026-06-26 10:01:01 -06:00
Drew T 23ba3cc677 chore(phase-22): refresh worklist after close=0 batch (441 live stubs, fleet 63.66%) 2026-06-26 04:15:54 -06:00
Drew T c1230fa6ea feat(phase-22): T2 close=0 recovery batch — 14 fns ×134 (fleet 63.29%->63.66%)
- R14 corrects cookbook §26 ('close=0 recovery exhausted'): recover_giant + the
  whole-binary gate still banks a ~15-20%% tail. Banked 7 close=0 reach-134 fns
  (func_80156ECC/80147E44/8015ADB0/801661CC/80166054/8012CFA8/8012A62C); dedup_propagate
  --auto-from swept those + 7 pre-existing inline-matched bonus fns ×134 (14 total)
- the other ~33 close=0 are the genuine DEF-side loose-typing / masked-residual wall
- func_80132784 HARD-DEFER (hoist-vs-remat + register-lifetime-reuse not C-expressible)
- §28c distilled: close=0-not-exhausted recipe + the dedup_propagate registry-skip
  recovery (register via append_groups; bytes were already check-all-correct)
- make check-all 136/136 byte-identical (R22); dedup 1619->1633 validated/0 failed;
  0 NON_MATCHING (G4)
2026-06-26 04:14:06 -06:00
Drew T d789d9c242 feat(phase-22): T2 — giant func_80163C2C banked ×134 via Blk16/Buf32 type-lift (fleet 63.25%->63.29%)
- second struct-walled close=0 giant (167 ins): lift Blk16(u32x4)+Buf32(2xBlk16) to
  src/shared/engine_types.h, remove the Blk16 dupes from ov_SC01_077.c + the -O0 split
  ov_SC01_077_a.c (byte-neutral), recover_giant, harvest_verify MATCH, dedup_propagate
  ×134 (+22378 ins byte-weighted)
- struct-walled trio done: func_80156B74 + func_80163C2C banked; func_8014F74C is a
  masked residual (grinder fuel, not type-liftable)
- make check-all 136/136 byte-identical (R22); dedup 1618->1619 validated/0 failed;
  0 NON_MATCHING (G4)
2026-06-26 03:51:11 -06:00
Drew T b5a1fe6dab feat(phase-22): T2 — giant func_80156B74 banked ×134 via engine_types.h type-lift (fleet 63.22%->63.25%)
- struct-walled close=0 giant (cookbook §28 case #3) is bankable: lift the S8/B8
  typedefs from ov_SC01_077.c's prelude to src/shared/engine_types.h (shared via
  engine_core.h; byte-neutral), recover_giant block-scopes externs, whole-binary
  harvest_verify MATCH, dedup_propagate ×134 (214 ins, +28676 ins byte-weighted)
- cookbook §28b: the type-lift recipe + the serialize-make-jobs gotcha (concurrent
  make corrupted an .o and dropped the registry write; re-ran dedup_propagate idempotent)
- make check-all 136/136 byte-identical (R22); dedup 1617->1618 validated/0 failed;
  0 NON_MATCHING (G4)
2026-06-26 03:45:35 -06:00
Drew T b234130666 chore(phase-22): session checkpoint — reports regen + resume pointer (fleet 63.22%, grinder stopped)
- docs/{worklist,backlog,progress.fleet}.md regenerated after the func_8015126C x134 bank
- CURRENT_PHASE.md: RESUME HERE block (T2 giant queue + levers + op gotchas) for a fresh session
- grinder stopped (STOP sentinel set); tree clean except R23 db churn
2026-06-26 03:16:39 -06:00
Drew T 0375cd529b docs(phase-22): T3 — distill §28 canonical-extern recovery + recover_giant.py + decomp.wiki patterns
- cookbook §28: the 4-way triage of a close=0 gate-rejected giant (pure-extern plumbing /
  masked-residual / struct-walled / regressed-draft) + the canonical-extern recovery (the NEW
  lever that banked func_8015126C, which Phase-21 left as a close=0 wall) + the coalescing pin
  (register __asm__ + lazy in-&& assign). §28a: PS1-applicable decomp.wiki GCC patterns
  (negative-offset loops, branch-duplication, load-coalescing, div-magic table, slti-0)
- tools/recover_giant.py: canonicalize a draft's engine_core.h-callee externs to def-sigs +
  move all externs block-scope (validated func_8015126C -> MATCH)
- R14: close=0-giant fan-out NOT uniformly near-free — only func_8015126C was pure-extern;
  the rest are struct-walled / masked-residual / regressed -> genuine per-giant hand-work
2026-06-26 03:06:54 -06:00
Drew T 013806b075 feat(phase-22): T0 — unified byte-weighted ranked worklist (the loop's decision spine)
- tools/worklist.py: join fuel_manifest (class/reach/nins/leverage/cached/region)
  + backlog near-misses (closeness/where_stuck/best_draft) -> docs/worklist.md +
  .run/worklist.json, ranked by byte-weighted gain (gain_ins = reach*nins),
  grouped by idiom class, with %rem + cum% columns; --refresh re-runs
  build_fuel_manifest + backlog.render for one-command loop use
- docs/worklist.md: 451 live stubs, 1.85M ins remaining gain; GIANT queue =
  23 fns = 37.2% of remaining gain; surfaces the idiom sub-groups (close=0
  plumbing-wall / won't-compile loose-typing / regalloc-coalescing / irreducible)
- phase-ends/CURRENT_PHASE.md: Phase-22 plan + per-task crash-recovery log
- no source touched -> byte-locked binaries unaffected (no build needed)
2026-06-26 02:07:03 -06:00
Drew T 1f4aac62fb docs(phase-21): cont.8 — collaboration sweep (clean verdicts) + 62 PsyQ names + SETUP -G fix
- reach-1 idiom loop: +16 ×1 matches (commit:0289, commit:0291) + chained-store-fanout idiom
  (commit:0290); giants pivot 0 auto-banks (2× = the byte-proven wall) -> 6 ranked
  hand-finish near-misses in backlog (2 close: func_8015126C close=2, func_8014EE14 close=6)
- collaboration sweep CLOSED with clean verdicts (5 scouts): parity research (we're SOTA;
  no neural/GPU/diffusion shortcut), decomp.dev survey (4 gcc-2.7.2-psx twins; VS=CC0 the
  clean pool partner), census (no Square symbols; Croc=SDK-naming only), VS corpus (76
  byte-verified pairs, low-priority asset), dedup probe (CLEAN NEGATIVE — BFM shares only
  PsyQ with sisters, AKAO refuted, 0 engine code) -> no external shortcut to the gcc wall
- config/symbols.us.txt: +62 PsyQ library/BIOS names from cross-project byte-signature
  dedup (xdedup vs VS/Xenogears); byte-safe (main still 143dbb89, R22-verified)
- docs/SETUP.md §5.3: corrected stale "-G TBD" marker -> RESOLVED Phase 5 (-G0) (R21)
- docs/backlog.md: giant near-misses logged with true worker closeness (cont.7d-safe)
- CURRENT_PHASE.md cont.8: session log + 3 parked R&D items + process lessons
- fleet 63.18%, 136/136 byte-identical, 0 NON_MATCHING
2026-06-25 23:56:12 -06:00
Drew T 2b69316963 docs(phase-21): distill — chained-store-fanout idiom from func_80143D28 2026-06-25 20:41:57 -06:00
Drew T 653d123b65 docs(phase-21): cont.7d — deep frame-RE doesn't crack a scheduler-walled giant (§27)
Drew chose deep re-RE of func_8014EA4C's real frame. Finding: the frame is a DEAD aggregate
copy (8 bytes unaligned-copied to sp+0x20, never read; gcc-2.7.2 keeps it, no DSE). The agent's
buf[16]+memcpy(buf+16) PRECISELY reproduces the exact gcc stack layout (local[0x38] -> frame
0x88, wrong). So the frame is already correct at close=6; the 6 residuals are NOT frame-caused
-- they're loop-body scheduler (global-store vs call-arg-load order) + regalloc (abs  vs
in-place ), the irreducible §20/§25 wall. Deep frame-RE is a dead end for schedule/regalloc
residuals -- only the permuter explores that space. 5 levers (cheap-recovery, permuter,
giant-wave, hand-finish, deep-RE) byte-proven exhausted at fleet 63.17% for the reach-134 tail.
2026-06-25 18:19:19 -06:00
Drew T 4284afb9e6 docs(phase-21): cont.7d — frame-pressure-locked giant residuals are PERMUTER-ONLY (§27)
Hand-finish attempt on func_8014EA4C (close=6, Drew's choice): 2 byte-proven attempts both
cascaded — q0_arg reorder swapped p/q regalloc (6->123); abs-ternary dropped the load-bearing
dead buf[16] frame hack -> frame 0x60->0x58 -> offset cascade (6->23). Confirms the agent: the
residuals (store/load schedule + abs regalloc) are pressure-locked by the frame-forcing hack;
C-steering breaks the frame. The fix is the decomp-permuter (grinder), not C — it explores the
frame-preserving regalloc/schedule space. Grinder is now auto-permuting func_8014EA4C +
func_80144090 (re-logged close=6/25, token-free). cookbook §27.
2026-06-25 17:36:17 -06:00
Drew T 4b836add6d feat(phase-21): cont.7d — 6-giant wave banked 0; giants don't auto-bank (the honest result)
- 6-giant worker_wave (1.16M tokens) BANKED 0. Agents got the SHAPE (§27 recipe works) but the
  whole-binary byte-gate (G3/P9) rejected all 6. func_8014F74C close=0 match_one MATCH yet its RAW
  draft FAILS the byte-gate (no plumbing/data conflict) -> match_one OVER-predicted (§25 masked
  residual). The rest are pure regalloc/scheduling 'not C-steerable -> permuter' (close 6/25/53/91/164).
- Confirms cont.2/Phase-20/Phase-16: giants are HAND-FINISH/permuter fuel, NOT auto-bankable.
  cookbook §27 updated. Do NOT scale giant auto-waves for % (1.16M tokens -> 0 banks).
- GATE BUG (R16, noted): canon/cast/sig_unify regressed near-misses (func_8014EA4C 6->128) and
  poisoned the backlog. Re-logged the 6 with TRUE closeness + raw drafts (source=giant-raw);
  func_8014EA4C(6)+func_80144090(25) now grinder-eligible (token-free).
- INFLECTION (P9): the automated reach-134 harvest is COMPLETE at 63.17% — cheap recovery (0 fresh),
  permuter (~0), giant wave (0/6) all exhausted. Forward % = hand-finishing or close Phase 21.
2026-06-25 17:27:46 -06:00
Drew T 8e78cd7ce8 feat(phase-21): cont.7c — giant approach VALIDATED (func_80176D94 structurally matched) + recipe §27
- Drew chose GIANT SESSION. Validated the giant-matching approach end-to-end:
  hand-matched func_80176D94 (152 ins) to STRUCTURALLY-MATCHED (calls/constants/GPU-packet
  all byte-correct) via the recipe: cached Ghidra-C + call-matching callee decls (arg-arity:
  func_80177784 called 3-arg vs canonical 4-arg) + §17 register pins (regalloc-shift) +
  sibling func_80176FF4 idioms (coord strength-reduce, GPU linked-list pointer).
- Residual = pure regalloc-coalescing (2 extra callee regs vs target caller-saved/-reuse)
  -> saved .run/backlog_drafts/func_80176D94.c as grinder/wave fuel (permuter territory).
- cookbook §27: the Giant matching recipe (5 steps + the pin-reuse caveat + the
  accumulator-coalescing residual class). Giants reach structurally-matched FAST; the last
  mile is regalloc -> isolated-agent + permuter work, not main-loop serial grind.
- Giant pool is ~18 tractable (152-284 ins) + 2 monsters, bigger than the assess's 8.
- NEXT: worker_wave over the giants (§27-seeded) -> gate + propagate ×134. fleet 63.17%.
2026-06-25 16:32:31 -06:00
Drew T 61a37534b9 docs(phase-21): cont.7b — resolved-reach probe REFUTED + close-1 tail is permuter-class (2 R14)
- R14: sig_image h_exact = SHA1(raw IMAGE bytes), UNMASKED (sig_image.py:170-171) -> reach is
  the ACCURATE shipped-byte reach; a resolved-reach probe just reproduces it. The real gap is
  reach (shipped identical) >= realizable-x134 (shared-C reproduces all N), measurable ONLY by
  dedup_propagate's byte-gate on a MATCHED fn. No cheap static probe -> cont.7 'build it' WITHDRAWN.
- R14: func_801775E0 backlog close=1 but saved draft compiles 31-off from source (prologue
  save-scheduling/branch-sense/block-order, the S20 wall) -> close=1 was permuter-achieved.
  RULE (S26): match_one a draft for SOURCE closeness before assuming a pin crack. The close-1..4
  codegen tail is permuter-class (grinder territory), not cheap pin targets.
- INFLECTION (P9): the cached reach-134 automated harvest is COMPLETE; forward %-progress needs
  a token-heavy lever (8 giants / per-fn pin-cracking) -> Drew's ROI greenlight. cookbook S26.
2026-06-25 13:52:43 -06:00
Drew T cf23808d5c feat(phase-21): cont.7 — cheap close=0 recovery EXHAUSTED (byte-proven); fix --assess double inflation
- THE FINDING (R14/P9): the _a close=0 recovery banks 0/20 (same def-side multi-way
  loose-typing wall as MAIN 0/40, cont.6). cont.6's func_8012F568 was the lone simple-arity
  win; the rest need a C type that conflicts caller-decl-vs-def with no single fitting sig.
  The cheap close=0 ×134 recovery lever is EXHAUSTED.
- TOOLING (R16, idiom_loop.py --assess was DOUBLY inflated, now fixed):
  (1) load_backlog drops banked-since-logged fns (intersect ledger w/ live INCLUDE_ASM stub
      set, mirrors backlog._matched_now) -> killed the bogus 'unknown 14' class + ~10/class
      stale-matched inflation (schedule 49->39 r134, regalloc 44->39).
  (2) DETERMINISTIC-RECOVERY splits FRESH (never recovery-gated = genuine fuel) vs WALLED
      (recovery-failed = def-side wall, do NOT re-run). Post-fix: 0 FRESH / 46 WALLED.
- schedule wave NOT fired (ROI-gate, S20/S26): ~50% x134 fraction on n=2, median 15 ins off,
  over an h_exact-inflated count. No tokens spent speculatively.
- cont.6 option-3 CONFIRMED: cached reach-134 cheap fuel is dry; the genuine levers are
  token-heavy fresh-session work (the 8 giants / per-fn pin-cracking / build a resolved-reach
  probe). cookbook S26. 0 banks; fleet 63.17% unchanged; 136/136 byte-identical; 0 NON_MATCHING.
2026-06-25 13:09:02 -06:00
Drew T 790326057a feat(phase-21): cont.6 — sig_unify --src-file + gate two-stage; schedule idiom = coalescing (§25)
- sig_unify.py --src-file: read cur_stubs/decls from the SPLIT .c (_a/_o0) so split-file
  drafts aren't dropped + get the def-side recovery (closes the §24 gap). gate_stage passes
  it to cast + sig_unify in --src-file mode. Banked func_8012F568 x134 earlier (commit:0277).
- gate_stage.py CANON-FIRST two-stage (§19 folded into the shared gate): stage1 canon+cast
  gate (already-correct/hand-pinned cracks bank), stage2 sig_unify ONLY the failures (def-side
  recovery, no regression). Fixes sig_unify regressing the func_80128ED8 crack.
- schedule idiom CRACKED (cookbook §25): func_80128ED8 = gcc copy-COALESCING, not scheduling;
  2 register pins (idc->$a3 forces the multiply onto $v1; idc+1->dead $v0 temp) -> MATCH,
  banked x1 (commit:0278). + the rank_for_schedule tie-break (priority->class->LUID source-order)
  + the genuine walls (§10 cross-jump/delay-slot, store-vs-load).
- FINDING (R14/P9): h_exact OVER-COUNTS x134 (relocation-masked) -> func_80128ED8 is x1 not
  x134; probe dedup_propagate --addr before a token-heavy wave. schedule is NOT a confirmed
  x134 vein -> the wave is ROI-deferred.
- check-all 136/136 byte-identical (R22); 0 NON_MATCHING (G4); fleet 63.13% -> 63.17%
2026-06-25 01:17:12 -06:00
Drew T 63b83d93f6 chore(phase-21): regen digests at the cont.5 handoff checkpoint (fleet 63.13%) 2026-06-24 23:17:06 -06:00