Commit Graph

3 Commits

Author SHA1 Message Date
Drew T 066a2e5c8f feat(phase-23): local GPU serving + prompt-fix + corpus-v3 (macro+struct mining)
LM Studio was ejected, so serving is ours now: tools/serve_local.py serves base+LoRA via
Unsloth (.venv-train cu128) as an OpenAI endpoint — no llama.cpp build (this CPU has no
AVX-512, which SIGILLs the prebuilt llama-cpp-python CUDA wheels). api_draft/lora_grind hit
it unchanged.

PROMPT FIX (api_draft.LEAN_SYS + format_finetune.SYS, kept in sync): 'translate EVERY
instruction, never an empty body' — a prompt test took the small-leaf band 0/3 -> 2/3 MATCH
(the v2 corpus overfit an empty void f(void){} leaf pattern). Validated end-to-end: a fresh
ov_SC01_001 batch banked 3 via the local server + better prompt.

CORPUS-V3 (export_pairs + format_finetune):
- export_pairs now ALSO mines the 1623 DEFINE_func macro bodies in engine_core.h (the shared
  setters/return-const/dispatchers extract_defs never saw -> 96.6% of v2 was overlay-unique,
  the root of the empty-leaf overfit). Corpus 1312 inline -> 2891 (1312 inline + 1579 macros).
- format_finetune inlines engine_types.h structs in the compile-filter so struct-using bodies
  are KEPT not dropped: train 2534/2591 (97.8%) compile standalone (v2 was 1111 total).
2026-06-30 01:46:54 -06:00
Drew T 8537706006 feat(phase-22): corpus-v2 — capture the extern block above each def (self-contained completions)
The pilot failed because the compile-filter dropped the 536 hard (global/struct) fns: bare defs
reference undeclared D_xxxx globals -> cc1 error. The src declares them as 'extern <type> D_xxx;'
immediately ABOVE each def with correct byte-verified types. Capturing that contiguous extern block
lifts standalone-compile 52%->92% (16/40 hard fns recovered) AND teaches the model to declare globals
with the right types (a real matching skill). Fixes the trivial-bias that capped the 7B pilot.
2026-06-29 20:03:53 -06:00
Drew T ad535ad1ab feat(phase-22): gen2 matching-model design + export_pairs.py corpus miner
docs/gen2-mips-matching-model.md: the BFM/gcc-2.7.2 matching-specialist idea (LoRA on our own
gate-verified pairs — the corpus off-the-shelf RE LLMs lack). export_pairs.py mines 1307 banked
(asm<->C) pairs from build objects (asm/ is gitignored, so disasm the ROM-identical build, splat-like
format) + src defs -> datasets/match_pairs/{pairs,train,test}.jsonl (gitignored, 1174/133 split).
api_draft.py: TEMP env-tunable. .gitignore: datasets/ models/ weights.
2026-06-29 14:27:19 -06:00