BFM-decomp/phase-ends/CURRENT_PHASE.md

CURRENT PHASE — Phase 23: The offline LLM matching tier (free local-model grinder + the data flywheel)

Generation: Gen2 (15th phase of the arc) · Started: 2026-06-29 · Effort: Max (synthesis/decisions/debug) ↔ xHigh (mechanical) · Plan: set up at PhaseEnd_Phase22 close (Drew); substance was already built across the 2026-06-29 session.

Per-task crash-recovery log (P3). This file is the committable state a fresh session resumes from. Phase 22 (giants) closed → PhaseEnd_Phase22.md + phase-ends/logs/Phase22.md. The deep design + measured results of this phase live in docs/gen2-mips-matching-model.md (+ docs/history/cheap-tier-ab-experiment.md); the working-knowledge in the cheap-tier-ab-validated memory.

Goal

Stand up a free, local, fine-tuned matching model that grinds the small/medium-function bulk and banks byte-verified matches for $0 — with (a) a data flywheel (banks grow the corpus → retrain → the model improves during the run), (b) escalation (the permuter grinder.py closes the regalloc/schedule near-misses the model leaves; bigger models / your Opus take the tail), and (c) the whole-binary byte-gate as the incorruptible sole arbiter (a weak model is a throughput risk only, never correctness). On-demand, NOT 24/7 (run only when Drew says go).

Decisions (Drew)

• On-demand runs, bounded — never a standing 24/7 daemon. Queue a large batch, run a single shot, measure, scale the next.
• Serve the model on the GPU via LM Studio (Unsloth's bundled llama.cpp is CPU-only); the WSL agent hits it over the LAN (http://192.168.1.113:1234/v1, "serve on local network" + Windows host IP).
• Measure before investing (the discipline that killed the v1 dead-end): every corpus/model change is retested free on the 7B first, and only scaled to a cloud dense model if the cheap retest pays.
• Corpus-v3 = struct types (next), then a dense Qwen2.5-Coder-14B (the "12B") on cloud for the struct giants — only if v3-on-7B lifts the struct band.

The operating loop (when running)

A. lora_grind.py (GPU) drafts open ≤N-ins stubs with the served fine-tuned model → banks via gate_stage → near-misses to the backlog. B. grinder.py (CPU permuter) closes the backlog's regalloc/schedule near-misses → banks. Run A+B concurrently (GPU vs CPU; the histogram already shows 94 schedule/regalloc nears waiting). C. Periodic propagate sweep (dedup_propagate) multiplies each bank fleet-wide. D. Periodically: export_pairs → format_finetune → train_lora → redeploy (the model learns its own new wins); read the near-by-class histogram → pick the corpus-v3 / recovery-tooling target; raise --max-nins as the band lifts.

Tasks

• T1 — Cheap-tier A/B (Opus orchestrator + cheap agents) — Haiku 4.8× more matches/$ than Opus on the bulk; tools tools/workflows/ab_match.js + tools/ab_score.py. (docs/history/cheap-tier-ab-experiment.md.)
• T2 — Stock-local floor — Qwen3.6-35B-A3B (LM Studio): structurally smart but 0 reliable byte-matches (can't refine to byte-exact); format-robust; full-cookbook context made it worse (dilution). The floor to beat.
• T3 — The LoRA specialist pipeline — export_pairs.py (mine banked asm↔C from build objects) → format_finetune.py (Qwen chat-template, compile-filtered) → train_lora.py (Unsloth QLoRA, Qwen2.5-Coder-7B, GPU on the 3080 Ti) → eval_lora.py (held-out gate-true). Corpus datasets/match_pairs/ (gitignored).
• T4 — Corpus-v2 = the extern-block fix (THE unlock) — capture the extern <type> D_xxx; block the src declares above each def (correct byte-verified types) → self-contained completions, compile 52%→92%, train 638→1111 (non-trivial 257→813). Same 7B retrained: held-out 6–15 ins 0%→85%, non-trivial 0→26, meaningful(>15) 0→3. Data was the bottleneck, confirmed.
• T5 — First REAL banks on open stubs — the v2 model drafted real open ov_SC01_077 stubs (LEAN, raw .s — model is format-robust, no bridge needed); whole-binary gate banked 4 (func_80160B34, func_8015CC74, func_8016084C, func_801705C0; @commit:0320). Sample rate: 9/22 match_one proxy → 4/22 (18%) whole-binary (the proxy→gate TU-plumbing gap).
• T6 — The mass-run driver — tools/lora_grind.py: rotates every binary (config/check.*.sha), drafts the served model, banks via gate_stage (commit), defers/periodic-propagates, writes the classified near-miss histogram (the flywheel "missing idioms" signal). STOP/heartbeat/stats like grinder.py.
• T7 — Calibration run + the 0/222 puzzle — DEBUGGED + FIXED (2026-06-30). Root cause = TWO independent harness bugs in lora_grind's use of gate_stage.run_gate (R14, by reading the code + the run's backlog — which resolved a direct contradiction between two scout agents): Bug A — good_sha() passed the whole sha1sum line "<sha> <name>" vs harvest_verify's bare sha1() → 0 banks for EVERY binary incl. 077 (the "0/12" was a bug artifact, NOT an exhausted tail — the prior ".sha files carry the filename" note was the unfollowed thread); Bug B — the gate call left src/asm/out at the hardcoded ov_SC01_077 defaults → non-077 drafts dropped at the 077 stub-filter, silently (and the asm mis-resolution contaminated the backlog near-miss classes). Fix (tools/gate_stage.py): run_gate is binary-agnostic (resolve src/asm/out/good_sha from binary; good_sha bare-hash normalized) + a loud negative-control guard; lora_grind.good_sha fixed at source; byte-neutral (check-all 136/136). Proof: ov_SC01_000 spot-run banked 7/15 (47%) byte-identical (was 0; @commit:0322). ROI finding: 6/7 banks are reach-1 (overlay-unique ×1) → broad rotation = high bank-RATE, low fleet-% ROI; the fleet lever is reach≥2 targeting (T9) + corpus-v3 (T8). Full write-up: docs/gen2-mips-matching-model.md → "T7 RESULT".
• T8 — Corpus-v3 (struct types) — emit the struct {...} definitions each fn needs (v2 did globals only; giants compile-fail on undefined structs). Build → retest on the 7B free (does the struct band lift on small/medium?) → only then a dense 14B cloud train for the struct giants.
• T9 — Wire the operating loop — concurrent lora_grind + grinder.py for on-demand runs; the periodic retrain cycle; raise --max-nins as retrains lift the band.
• T10 — Progress honesty + PhaseEnd — track fleet % + bank-rate; PhaseEnd_Phase23 at a clean checkpoint (Tier-1 Max).

▶ RESUME HERE (fresh session)

State: Phase 23 in progress (NOT a phase end). Phase 22 closed (PhaseEnd_Phase22.md, uncommitted — Drew's gate-2 commit+push). The fine-tuned model bfm-match-7b-v2 (Qwen2.5-Coder-7B QLoRA on corpus-v2) is built; served by LM Studio at http://192.168.1.113:1234/v1 (model id bfm-match-7b-v2; GGUF at models/bfm-match-7b_gguf/). Corpus datasets/match_pairs/ + training stack .venv-train (gitignored). T7 FIXED — the gate banks fleet-wide now (ov_SC01_000 7/15 byte-identical, +1 reach-2 propagated; @commit:0322 + the T7 checkpoint commit). The 0/222 was two harness bugs (good_sha format + src/asm/out 077-default), not the model.

NEXT TASK — corpus-v3 (better leaf drafts) + the dedup-collapse (stuck-local ×reach). T7 (gate), T9 (reach targeting), and the grinder per-binary fix (5-layer, validated) are DONE. The grinder fix re-characterized the reach≥2 close=1 fuel (byte-evidenced): it's MODEL semantic-misses, not permuter fuel — the 7B drafts void f(void){} for functions that are literally return 1 / trivial sw/sh setters (the corpus's overfit empty-leaf pattern); a corrected draft banks them (did: +3 byte-identical via the fixed per-binary gate, @commit:0326) — the permuter can't. AND ×reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the §19/20 stuck-local cap), so dedup_propagate --auto-from reports "nothing to propagate" → they banked ×1. So the two levers to the reach-134 ×134 payoff: (1) corpus-v3 — fix the empty-leaf overfit (leaf variety: return-const + setters) so the model drafts these right + emit struct types for the compile-fails → retest free on the 7B; (2) dedup-collapse — collapse the inline-matched-in-077 stuck-local fns into shared engine_core.h macros so they propagate ×reach. The grinder is now wired for whatever genuinely-permuter-amenable (regalloc/schedule) near-misses future runs surface.

Run a bounded mass-run (when Drew says go):

# LM Studio serving bfm-match-7b-v2 on the network first:
API_BASE=http://192.168.1.113:1234/v1 MODEL=bfm-local/bfm-match-7b-v2 GATE_PHASE=phase-23 \
  .venv/bin/python -u tools/lora_grind.py --max-nins 15 --batch 15 --max-batches N
# concurrent permuter (closes the near-misses): rm .run/auto/STOP; bash tools/auto_supervisor.sh ... (grinder.py)
# stop anytime: touch .run/auto/STOP

Verification invariant (every bank)

The whole-binary byte-gate (gate_stage/harvest_verify, G3/P9) is the sole arbiter — a wrong/weak draft can NEVER bank (it reverts to the stub). make check-all 136/136 byte-identical from a clean tree (R22); dedup-check 0 failed; the db.*.gbf churn is R23 restart-noise (do NOT stage). The fine-tuned model only affects throughput, never correctness.

Reuse (no rewrites)

New (this phase): api_draft.py (provider-agnostic LEAN/full drafter + the unused .s→objdump NORMALIZE bridge) · ab_match.js/ab_score.py (the cost A/B) · export_pairs.py / format_finetune.py / train_lora.py / eval_lora.py (the LoRA pipeline) · lora_grind.py (the mass-run). Existing: gate_stage (now GATE_PHASE-tagged) / grinder.py + auto_supervisor.sh / dedup_propagate / backlog / harvest_verify / match_one; cookbook §17–§28.

Guardrails

• On-demand only — never leave a 24/7 daemon running; bounded --max-batches, STOP-sentinel safe-exit.
• Measure before investing — retest every corpus/model change free on the 7B before any cloud spend.
• Disk — GGUF conversion writes ~30 GB intermediates onto the WSL vhdx (C:); train_lora now auto-cleans them; keep only the q4 GGUF. (A disk-full crash cost a session on 2026-06-29.)
• Serve on GPU (LM Studio) — Unsloth's llama.cpp is CPU-only; don't eval through it.
• Model size matches data size — no "massive" models on ~1–2k examples (overfit); dense > MoE for a limited-data LoRA.

Blockers

• (cleared) T7 — the 0/222 broad-rotation banking. Root-caused to two harness bugs (good_sha sha1sum-format + the src/asm/out ov_SC01_077-default) and fixed + proven (ov_SC01_000 7/15 byte-identical, check-all 136/136). No blockers; a bounded mass-run is safe to size. Open levers (not blockers): corpus-v3 struct types (T8) + reach≥2 targeting (T9).

Progress log

• 2026-06-29: Phase opened at PhaseEnd_Phase22 close (Drew). Built across this session: cheap-tier A/B (T1, Haiku 4.8×/$), stock-local floor (T2, 0), the LoRA pipeline (T3) + corpus-v2 extern-fix (T4, 6–15 ins 0%→85%), first 4 real open-stub banks (T5, @commit:0320), the lora_grind mass-run driver (T6). Calibration run (T7) launched (500 fns) — 18% on ov_SC01_077 but 0/222 broad rotation → #1 debug. gate_stage commit tag made phase-agnostic. The whole arc + measured numbers: docs/gen2-mips-matching-model.md; memory cheap-tier-ab-validated. NEXT: T7 debug, then bounded mass-runs + corpus-v3.
• 2026-06-30: T7 DEBUGGED + FIXED. 3 Explore scouts (tooling / run-evidence / corpus) + a direct code read (R14 — which resolved a flat contradiction between two scouts) found two independent bugs in lora_grind's gate path: (A) good_sha() passed "<sha> <name>" vs harvest_verify's bare sha1() → 0 banks for ALL binaries incl. 077 (so 077's "0/12" was a bug artifact); (B) src/asm/out defaulted to ov_SC01_077 → non-077 drafts dropped at the 077 stub-filter, silently. Fixed gate_stage.run_gate (binary-agnostic resolution + bare-hash normalize + a loud negative-control guard) + lora_grind.good_sha; byte-neutral (check-all 136/136). ov_SC01_000 spot-run banked 7/15 (47%) byte-identical (@commit:0322) → reach-2 func_8017CE24 propagated ×2. ROI: 6/7 reach-1 → broad rotation is high bank-rate / low fleet-% ROI; the fleet lever is reach≥2 targeting + corpus-v3. Backlog now correctly classified (4× close=1 = grinder fuel). NEXT: T8 corpus-v3 (struct types) + T9 reach≥2 selection + concurrent grinder.
• 2026-06-30 (cont.): T9 reach≥2 targeting built + measured. Added lora_grind --min-reach N (lazy sig-based reach oracle == dedup_propagate, validated 0-mismatch/60 + the func_8017CE24=2 ground truth; --min-reach 2 ranks high-reach-first, naturally restricts to overlays). Bounded reach≥2 mass-run: ov_SC01_000's 15 reach≥2 (shared) stubs banked 0/15 (vs the reach-1 spot-run's 7/15) — the model is weakest on shared code (corpus skipped the DEFINE_func bodies + it's the regalloc/schedule tail). But 5/15 are close≤3 reach-134 near-misses (3× close=1 = func_8012E27C/BF4C/AD64) → high-value permuter fuel (×134 each). FINDING: reach≥2 model-only ≠ a fleet lever; the lever is reach≥2-draft → grinder-close (×134), which needs grinder.py's per-binary fix (same class as T7) + a backlog binary field. (A foreground mass-run hit the 10-min Bash cap mid-2nd-batch; tree recovered clean via git checkout, check-all 136/136.) Details: docs/gen2-mips-matching-model.md "T9 RESULT". NEXT: the grinder per-binary fix (realize the reach-134 ×134 fuel), then corpus-v3.
• 2026-06-30 (cont.): Grinder per-binary fix (5-layer) — built + validated; reach≥2 fuel re-characterized. The grinder/backlog were ov_SC01_077-hardcoded 5 layers deep — fixed all: gate_stage records binary, backlog.FIELDS keeps it, backlog.load_best/_open_stubs fleet-aware (stuck-local fns surface via their overlay record), p16_permute.setup takes asm-subdir, grinder per-binary resolution + grouped gating + None-nins allow. Validated: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface + resolve + gate per-000. FINDINGS (byte-evidenced): (a) the reach≥2 close=1 fuel is MODEL semantic-misses (7B drafts empty void f(void){} for trivial return 1/setters), NOT permuter fuel — a corrected draft banks them (+3 byte-identical via the fixed gate, @commit:0326); (b) ×reach is propagation-capped (inline-matched in ov_SC01_077_a.c → dedup "nothing to propagate" → banked ×1). So the reach-134 ×134 payoff routes through corpus-v3 (leaf variety) + the dedup-collapse, NOT the permuter. check-all 136/136 throughout. Details: docs/gen2-mips-matching-model.md (grinder fix). NEXT: corpus-v3 + dedup-collapse.