mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-28 14:59:48 -04:00
Drew T
e91859fb4a
The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.
Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.
TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.
check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
70 lines
15 KiB
Markdown
70 lines
15 KiB
Markdown
# CURRENT PHASE — Phase 23: The offline LLM matching tier (free local-model grinder + the data flywheel)
|
||
|
||
**Generation:** Gen2 (15th phase of the arc) · **Started:** 2026-06-29 · **Effort:** Max (synthesis/decisions/debug) ↔ xHigh (mechanical) · **Plan:** set up at PhaseEnd_Phase22 close (Drew); substance was already built across the 2026-06-29 session.
|
||
|
||
> Per-task crash-recovery log (P3). This file is the committable state a fresh session resumes from. Phase 22 (giants) closed → `PhaseEnd_Phase22.md` + `phase-ends/logs/Phase22.md`. The deep design + measured results of this phase live in `docs/gen2-mips-matching-model.md` (+ `docs/history/cheap-tier-ab-experiment.md`); the working-knowledge in the `cheap-tier-ab-validated` memory.
|
||
|
||
## Goal
|
||
Stand up a **free, local, fine-tuned matching model** that grinds the small/medium-function bulk and banks byte-verified matches for **$0** — with (a) a **data flywheel** (banks grow the corpus → retrain → the model improves *during* the run), (b) **escalation** (the permuter `grinder.py` closes the regalloc/schedule near-misses the model leaves; bigger models / your Opus take the tail), and (c) the **whole-binary byte-gate as the incorruptible sole arbiter** (a weak model is a *throughput* risk only, never correctness). **On-demand, NOT 24/7** (run only when Drew says go).
|
||
|
||
## Decisions (Drew)
|
||
- **On-demand runs**, bounded — never a standing 24/7 daemon. Queue a large batch, run a single shot, measure, scale the next.
|
||
- **Serve the model on the GPU via LM Studio** (Unsloth's bundled llama.cpp is CPU-only); the WSL agent hits it over the LAN (`http://192.168.1.113:1234/v1`, "serve on local network" + Windows host IP).
|
||
- **Measure before investing** (the discipline that killed the v1 dead-end): every corpus/model change is **retested free on the 7B first**, and only scaled to a cloud dense model if the cheap retest pays.
|
||
- **Corpus-v3 = struct types** (next), then a dense **Qwen2.5-Coder-14B** (the "12B") on cloud for the struct *giants* — only if v3-on-7B lifts the struct band.
|
||
|
||
## The operating loop (when running)
|
||
A. `lora_grind.py` (GPU) drafts open ≤N-ins stubs with the served fine-tuned model → banks via `gate_stage` → near-misses to the backlog.
|
||
B. `grinder.py` (CPU permuter) closes the backlog's regalloc/schedule near-misses → banks. **Run A+B concurrently** (GPU vs CPU; the histogram already shows 94 schedule/regalloc nears waiting).
|
||
C. Periodic propagate sweep (`dedup_propagate`) multiplies each bank fleet-wide.
|
||
D. Periodically: `export_pairs → format_finetune → train_lora → redeploy` (the model learns its own new wins); read the near-by-class histogram → pick the corpus-v3 / recovery-tooling target; raise `--max-nins` as the band lifts.
|
||
|
||
## Tasks
|
||
- [x] **T1 — Cheap-tier A/B** (Opus orchestrator + cheap agents) — Haiku 4.8× more matches/$ than Opus on the bulk; tools `tools/workflows/ab_match.js` + `tools/ab_score.py`. (`docs/history/cheap-tier-ab-experiment.md`.)
|
||
- [x] **T2 — Stock-local floor** — Qwen3.6-35B-A3B (LM Studio): structurally smart but **0 reliable byte-matches** (can't refine to byte-exact); format-robust; full-cookbook context made it *worse* (dilution). The floor to beat.
|
||
- [x] **T3 — The LoRA specialist pipeline** — `export_pairs.py` (mine banked asm↔C from build objects) → `format_finetune.py` (Qwen chat-template, compile-filtered) → `train_lora.py` (Unsloth QLoRA, Qwen2.5-Coder-7B, GPU on the 3080 Ti) → `eval_lora.py` (held-out gate-true). Corpus `datasets/match_pairs/` (gitignored).
|
||
- [x] **T4 — Corpus-v2 = the extern-block fix (THE unlock)** — capture the `extern <type> D_xxx;` block the src declares above each def (correct byte-verified types) → self-contained completions, compile 52%→92%, train 638→1111 (non-trivial 257→813). Same 7B retrained: held-out **6–15 ins 0%→85%**, non-trivial 0→26, meaningful(>15) 0→3. **Data was the bottleneck, confirmed.**
|
||
- [x] **T5 — First REAL banks on open stubs** — the v2 model drafted real open ov_SC01_077 stubs (LEAN, raw `.s` — model is format-robust, no bridge needed); whole-binary gate **banked 4** (func_80160B34, func_8015CC74, func_8016084C, func_801705C0; @commit:0320). Sample rate: 9/22 match_one proxy → **4/22 (18%) whole-binary** (the proxy→gate TU-plumbing gap).
|
||
- [x] **T6 — The mass-run driver** — `tools/lora_grind.py`: rotates every binary (config/check.*.sha), drafts the served model, banks via gate_stage (commit), defers/periodic-propagates, writes the **classified near-miss histogram** (the flywheel "missing idioms" signal). STOP/heartbeat/stats like grinder.py.
|
||
- [x] **T7 — Calibration run + the 0/222 puzzle — DEBUGGED + FIXED (2026-06-30).** Root cause = TWO independent harness bugs in `lora_grind`'s use of `gate_stage.run_gate` (R14, by reading the code + the run's backlog — which resolved a direct contradiction between two scout agents): **Bug A** — `good_sha()` passed the whole sha1sum line `"<sha> <name>"` vs harvest_verify's bare `sha1()` → **0 banks for EVERY binary incl. 077** (the "0/12" was a bug artifact, NOT an exhausted tail — the prior "`.sha` files carry the filename" note was the unfollowed thread); **Bug B** — the gate call left `src/asm/out` at the hardcoded ov_SC01_077 defaults → non-077 drafts dropped at the 077 stub-filter, **silently** (and the asm mis-resolution contaminated the backlog near-miss classes). Fix (`tools/gate_stage.py`): `run_gate` is binary-agnostic (resolve src/asm/out/good_sha from `binary`; good_sha bare-hash normalized) + a **loud negative-control guard**; `lora_grind.good_sha` fixed at source; byte-neutral (check-all 136/136). **Proof:** ov_SC01_000 spot-run banked **7/15 (47%) byte-identical** (was 0; @commit:0322). **ROI finding:** 6/7 banks are reach-1 (overlay-unique ×1) → broad rotation = high bank-RATE, low fleet-% ROI; the fleet lever is reach≥2 targeting (T9) + corpus-v3 (T8). Full write-up: `docs/gen2-mips-matching-model.md` → "T7 RESULT".
|
||
- [ ] **T8 — Corpus-v3 (struct types)** — emit the `struct {...}` definitions each fn needs (v2 did globals only; giants compile-fail on undefined structs). Build → **retest on the 7B free** (does the struct band lift on small/medium?) → only then a dense **14B cloud** train for the struct giants.
|
||
- [ ] **T9 — Wire the operating loop** — concurrent `lora_grind` + `grinder.py` for on-demand runs; the periodic retrain cycle; raise `--max-nins` as retrains lift the band.
|
||
- [ ] **T10 — Progress honesty + PhaseEnd** — track fleet % + bank-rate; PhaseEnd_Phase23 at a clean checkpoint (Tier-1 Max).
|
||
|
||
## ▶ RESUME HERE (fresh session)
|
||
**State:** Phase 23 in progress (NOT a phase end). Phase 22 closed (`PhaseEnd_Phase22.md`, uncommitted — Drew's gate-2 commit+push). The fine-tuned model **`bfm-match-7b-v2`** (Qwen2.5-Coder-7B QLoRA on corpus-v2) is built; **served by LM Studio** at `http://192.168.1.113:1234/v1` (model id `bfm-match-7b-v2`; GGUF at `models/bfm-match-7b_gguf/`). Corpus `datasets/match_pairs/` + training stack `.venv-train` (gitignored). **T7 FIXED** — the gate banks fleet-wide now (ov_SC01_000 7/15 byte-identical, +1 reach-2 propagated; @commit:0322 + the T7 checkpoint commit). The 0/222 was two harness bugs (good_sha format + src/asm/out 077-default), not the model.
|
||
|
||
**NEXT TASK — corpus-v3 (better leaf drafts) + the dedup-collapse (stuck-local ×reach).** T7 (gate), T9 (reach targeting), and the grinder **per-binary fix (5-layer, validated)** are DONE. The grinder fix re-characterized the reach≥2 close=1 fuel (byte-evidenced): it's **MODEL semantic-misses, not permuter fuel** — the 7B drafts `void f(void){}` for functions that are literally `return 1` / trivial `sw`/`sh` setters (the corpus's overfit empty-leaf pattern); a **corrected draft** banks them (did: +3 byte-identical via the fixed per-binary gate, @commit:0326) — the permuter can't. AND ×reach is **propagation-capped**: the 3 are inline-matched in `ov_SC01_077_a.c` (the §19/20 stuck-local cap), so `dedup_propagate --auto-from` reports "nothing to propagate" → they banked ×1. So the two levers to the reach-134 ×134 payoff: (1) **corpus-v3** — fix the empty-leaf overfit (leaf variety: return-const + setters) so the model drafts these right + emit struct types for the compile-fails → retest free on the 7B; (2) **dedup-collapse** — collapse the inline-matched-in-077 stuck-local fns into shared `engine_core.h` macros so they propagate ×reach. The grinder is now wired for whatever genuinely-permuter-amenable (regalloc/schedule) near-misses future runs surface.
|
||
|
||
**Run a bounded mass-run (when Drew says go):**
|
||
```
|
||
# LM Studio serving bfm-match-7b-v2 on the network first:
|
||
API_BASE=http://192.168.1.113:1234/v1 MODEL=bfm-local/bfm-match-7b-v2 GATE_PHASE=phase-23 \
|
||
.venv/bin/python -u tools/lora_grind.py --max-nins 15 --batch 15 --max-batches N
|
||
# concurrent permuter (closes the near-misses): rm .run/auto/STOP; bash tools/auto_supervisor.sh ... (grinder.py)
|
||
# stop anytime: touch .run/auto/STOP
|
||
```
|
||
|
||
## Verification invariant (every bank)
|
||
The **whole-binary byte-gate** (`gate_stage`/`harvest_verify`, G3/P9) is the sole arbiter — a wrong/weak draft can NEVER bank (it reverts to the stub). `make check-all` 136/136 byte-identical from a clean tree (R22); `dedup-check` 0 failed; the `db.*.gbf` churn is R23 restart-noise (do NOT stage). The fine-tuned model only affects *throughput*, never correctness.
|
||
|
||
## Reuse (no rewrites)
|
||
*New (this phase):* `api_draft.py` (provider-agnostic LEAN/full drafter + the unused `.s`→objdump NORMALIZE bridge) · `ab_match.js`/`ab_score.py` (the cost A/B) · `export_pairs.py` / `format_finetune.py` / `train_lora.py` / `eval_lora.py` (the LoRA pipeline) · `lora_grind.py` (the mass-run).
|
||
*Existing:* `gate_stage` (now `GATE_PHASE`-tagged) / `grinder.py` + `auto_supervisor.sh` / `dedup_propagate` / `backlog` / `harvest_verify` / `match_one`; cookbook §17–§28.
|
||
|
||
## Guardrails
|
||
- **On-demand only** — never leave a 24/7 daemon running; bounded `--max-batches`, STOP-sentinel safe-exit.
|
||
- **Measure before investing** — retest every corpus/model change free on the 7B before any cloud spend.
|
||
- **Disk** — GGUF conversion writes ~30 GB intermediates onto the WSL vhdx (C:); `train_lora` now auto-cleans them; keep only the q4 GGUF. (A disk-full crash cost a session on 2026-06-29.)
|
||
- **Serve on GPU** (LM Studio) — Unsloth's llama.cpp is CPU-only; don't eval through it.
|
||
- Model size matches data size — no "massive" models on ~1–2k examples (overfit); dense > MoE for a limited-data LoRA.
|
||
|
||
## Blockers
|
||
- **(cleared) T7 — the 0/222 broad-rotation banking.** Root-caused to two harness bugs (good_sha sha1sum-format + the src/asm/out ov_SC01_077-default) and fixed + proven (ov_SC01_000 7/15 byte-identical, check-all 136/136). No blockers; a bounded mass-run is safe to size. Open levers (not blockers): corpus-v3 struct types (T8) + reach≥2 targeting (T9).
|
||
|
||
## Progress log
|
||
- 2026-06-29: **Phase opened at PhaseEnd_Phase22 close (Drew).** Built across this session: cheap-tier A/B (T1, Haiku 4.8×/$), stock-local floor (T2, 0), the LoRA pipeline (T3) + corpus-v2 extern-fix (T4, 6–15 ins 0%→85%), first 4 real open-stub banks (T5, @commit:0320), the `lora_grind` mass-run driver (T6). Calibration run (T7) launched (500 fns) — **18% on ov_SC01_077 but 0/222 broad rotation → #1 debug.** gate_stage commit tag made phase-agnostic. The whole arc + measured numbers: `docs/gen2-mips-matching-model.md`; memory `cheap-tier-ab-validated`. NEXT: T7 debug, then bounded mass-runs + corpus-v3.
|
||
- 2026-06-30: **T7 DEBUGGED + FIXED.** 3 Explore scouts (tooling / run-evidence / corpus) + a direct code read (R14 — which resolved a flat contradiction between two scouts) found **two independent bugs** in `lora_grind`'s gate path: **(A)** `good_sha()` passed `"<sha> <name>"` vs harvest_verify's bare `sha1()` → 0 banks for ALL binaries incl. 077 (so 077's "0/12" was a bug artifact); **(B)** `src/asm/out` defaulted to ov_SC01_077 → non-077 drafts dropped at the 077 stub-filter, silently. Fixed `gate_stage.run_gate` (binary-agnostic resolution + bare-hash normalize + a loud negative-control guard) + `lora_grind.good_sha`; byte-neutral (check-all 136/136). ov_SC01_000 spot-run **banked 7/15 (47%) byte-identical** (@commit:0322) → reach-2 `func_8017CE24` propagated ×2. **ROI:** 6/7 reach-1 → broad rotation is high bank-rate / low fleet-% ROI; the fleet lever is **reach≥2 targeting** + corpus-v3. Backlog now correctly classified (4× close=1 = grinder fuel). NEXT: **T8 corpus-v3** (struct types) + **T9 reach≥2 selection** + concurrent grinder.
|
||
- 2026-06-30 (cont.): **T9 reach≥2 targeting built + measured.** Added `lora_grind --min-reach N` (lazy sig-based reach oracle == `dedup_propagate`, validated 0-mismatch/60 + the func_8017CE24=2 ground truth; `--min-reach 2` ranks high-reach-first, naturally restricts to overlays). Bounded reach≥2 mass-run: ov_SC01_000's 15 reach≥2 (shared) stubs banked **0/15** (vs the reach-1 spot-run's 7/15) — the model is **weakest on shared code** (corpus skipped the `DEFINE_func` bodies + it's the regalloc/schedule tail). But **5/15 are close≤3 reach-134 near-misses** (3× close=1 = func_8012E27C/BF4C/AD64) → high-value permuter fuel (×134 each). **FINDING: reach≥2 model-only ≠ a fleet lever; the lever is reach≥2-draft → grinder-close (×134)**, which needs `grinder.py`'s per-binary fix (same class as T7) + a backlog `binary` field. (A foreground mass-run hit the 10-min Bash cap mid-2nd-batch; tree recovered clean via `git checkout`, check-all 136/136.) Details: `docs/gen2-mips-matching-model.md` "T9 RESULT". NEXT: the grinder per-binary fix (realize the reach-134 ×134 fuel), then corpus-v3.
|
||
- 2026-06-30 (cont.): **Grinder per-binary fix (5-layer) — built + validated; reach≥2 fuel re-characterized.** The grinder/backlog were ov_SC01_077-hardcoded **5 layers deep** — fixed all: `gate_stage` records `binary`, `backlog.FIELDS` keeps it, `backlog.load_best`/`_open_stubs` fleet-aware (stuck-local fns surface via their overlay record), `p16_permute.setup` takes asm-subdir, `grinder` per-binary resolution + grouped gating + None-`nins` allow. Validated: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface + resolve + gate per-000. **FINDINGS (byte-evidenced):** (a) the reach≥2 close=1 fuel is **MODEL semantic-misses** (7B drafts empty `void f(void){}` for trivial `return 1`/setters), NOT permuter fuel — a corrected draft banks them (**+3 byte-identical** via the fixed gate, @commit:0326); (b) ×reach is **propagation-capped** (inline-matched in `ov_SC01_077_a.c` → dedup "nothing to propagate" → banked ×1). So the reach-134 ×134 payoff routes through **corpus-v3** (leaf variety) + the **dedup-collapse**, NOT the permuter. check-all 136/136 throughout. Details: `docs/gen2-mips-matching-model.md` (grinder fix). NEXT: corpus-v3 + dedup-collapse.
|