mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 14:02:04 -04:00
docs(phase-23): CURRENT_PHASE — the giant campaign (Fable5 wall-break + toolkit generalization, 3 giants ×134, reframe)
This commit is contained in:
@@ -93,3 +93,9 @@ The **whole-binary byte-gate** (`gate_stage`/`harvest_verify`, G3/P9) is the sol
|
||||
- 2026-07-01 (**evening — T10.6 ≤15 campaign + T10.7/8/9 the OpenRouter/GLM5.2 frontier-model exploration**): CONTEXT CHECKPOINT (85% ctx, wind-down). **T10.6 DONE:** `bulk_harvest.py` ≤15 saturation campaign ran 23 cycles → **1,297 banks, fleet 64.6%, ~92% of the ≤15 band saturated**, 136/136, crash-graceful exit (serve_local bnb `ops.cu:81` after 14h — not our bug). **T10.7 DONE (GLM5.2 A/B, `z-ai/glm-5.2` via OpenRouter):** on 18 hard-band fns (16-22 ins), **GLM 10/18 match_one vs v3 1/18** (~10× CODEGEN edge) but **whole-binary banks 4/18** — the **DEF-side loose-typing wall (Phase 16/20) is INTRINSIC** (caps ANY drafter; it's overlay-decl-vs-true-sig, a C/ABI limit). Option-3 probe (`glm_reconcile.py`): GLM reasons the wall EXPERTLY (store-width `sh`/`sw`, K&R promotion, independently derives the §17a-1 cast idiom) but banks only 1/7 — reconciliation idioms → cookbook **§29**. **T10.8 DONE (`idiom_hunt.py`, $0.51):** batch-clustered the FAILED backlog by class → GLM re-derives our OWN idioms (pins §17, array-of-struct §18) + confirms walls, **0 new bankable idioms** (the failed residual is the worst discovery material — Drew's catch). **T10.9 IN FLIGHT (bg task `bmv1wxzdz`, `.run/glm_fresh.log`):** Drew's CORRECT reframe — the idiom engine is DEEP hand-solving of FRESH never-tried hard fns (like Opus Max), then mining the byte-correct cracks for NEW idioms (idiom lives in the correct BODY, not the bank). GLM deep-solving **15 fresh 25-118-ins fns** (`ov_SC01_077`/`SC03_001`/`SC02_005`), LEAN, MAXTOK=32k, iters=3, **reasoning captured** (`.run/glm_fresh/*.reasoning.txt`, `REASON=1`). **TOOLING BUILT this session:** `tools/bulk_harvest.py` (phase-separated parallel-gate harvester + `--cycles`), `tools/glm_reconcile.py` (Option-3 wall reconciler), `tools/idiom_hunt.py` (idiom researcher, `--budget`-capped), `api_draft` (`MAXTOK` env + `usage.cost` + `REASON=1` reasoning capture), `fix_arity_callers --any-proto`, `gate_stage.run_gate` parallel params (`lock_path`/`verified_out`/`failed_out`/`compute_fleet`), `harvest_verify --verified-out/--failed-out`. **Commits:** `commit:0379` (T10 tooling)…`commit:0410`/`commit:0412` (≤15 campaign)…`commit:0414`(MAXTOK)…`commit:0418`(glm_reconcile+§29)…`commit:0420`(idiom_hunt+T10.8). **OpenRouter setup:** key in `.env` as `open_router_key` (gitignored, `sk-or-`); GLM5.2 is a REASONING model (needs MAXTOK≥8k or it returns empty content — burns budget on reasoning tokens); ~**$23.6 left of $25** (spent ~$1.90); the key shows `"limit": 20` (possibly a $20 spend cap — raise in the OpenRouter dashboard before a bigger run); no hard concurrency cap (~6 providers serve glm-5.2). **STRATEGIC STATE:** the LLM matching tier is now THOROUGHLY explored — local v3 saturated ≤15 ($0); frontier GLM confirmed the hard-band def-side wall is INTRINSIC (triple-confirmed) + banks a modest ~22% of the conflict-free hard band at ~$0.05/fn; the new-idiom well is DRY on the failed residual (T10.8), with T10.9 testing whether FRESH hand-solving still yields NEW idioms (Drew's hypothesis). **DREW CHOSE: (A) a bounded GLM fresh-hard-band campaign** (which T10.9 pilots). Per the Fable5 strategy review (`docs/fable5-strategy-review-2026-07.md`), the real lever PAST the wall is the **public flip / community labor** (§4.3), not a bigger model. **RESUME (fresh session):** (1) read `.run/glm_fresh.log` + gate the correct bodies (bank the def-conflict-free ones; use `glm_reconcile`/`inject_capped_externs` recovery), (2) MINE `.run/glm_fresh/*.reasoning.txt` for NOVEL idioms (vs cookbook §17-§29) → distill real ones to cookbook, (3) decide the fork: scale the GLM fresh-hard campaign (money-open but modest), pivot to the public flip (Fable5 §6.3, the strategic lever), or PhaseEnd_Phase23 (the LLM tier is well-characterized — a clean Tier-1 close). **T11 (PhaseEnd) remains.** serve_local is DOWN (GPU free). Cost discipline: Drew is cost-conscious — calibrate cheap, `--budget`-cap, no surprise overnight bills.
|
||||
- 2026-07-02 (**T10.9 RESULT + STOPPING POINT — the fresh-hand-solve idiom test**): Drew's fair test of "does GLM hand-solving FRESH hard funcs teach NEW idioms?" (the mechanism that gave us pins/array-of-struct via Opus-Max). GLM deep-solved **15 fresh 25-118-ins fns** (LEAN, MAXTOK=32k, iters=3, reasoning captured `.run/glm_fresh/*.reasoning.txt`): **4/15 match_one, 2 whole-binary banks** (`func_8017DE28`, `func_8015EEE0`; @`commit:0422`, check-all 136/136), **$2.00**. **IDIOM VERDICT (the point):** GLM's 4 correct bodies reason entirely about **KNOWN gcc mechanics** — delay-slot scheduling, callee-saved `$s0` preservation, reload-after-call aliasing, switch jump tables — all already in cookbook §10/§17/jump-table. **NO new idiom emerged.** GLM is a strong reasoner *applying* mapped idioms, not finding unmapped quirks (expected — 22 phases of Opus-Max mining charted the common gcc-2.7.2 quirks). **So the new-idiom well is DRY, now confirmed from BOTH angles: failed-residual (T10.8) AND fresh-hand-solve (T10.9).** 8/15 were reasoning-truncated empties (giants overrun 32k); near-misses `func_8018119C` (near-4) / `func_80187DAC` (near-5) are grinder/reconcile fodder. **ROBUST STRATEGIC CONCLUSION (multi-angle):** the LLM matching tier is exhausted of cheap+idiom leverage — (1) local v3 saturated ≤15 ($0, ~1300 banks), (2) frontier GLM = a modest ~22% hard-band drafter at ~$0.05-0.13/fn (def-side wall caps banking; wall is INTRINSIC, triple-confirmed), (3) no new idioms from either failed-residual or fresh cracks. Total OpenRouter spend ~$4/$25. **Per the Fable5 strategy review (`docs/fable5-strategy-review-2026-07.md` §4.3/§6.3), the real lever past the wall is the PUBLIC FLIP (community hand-matching), not a bigger model.**
|
||||
**★ CLEAN STOPPING POINT (fresh session, Tier-1 Max, plan mode):** the LLM-tier exploration is comprehensively concluded. Options for the fresh session, in recommended order: **(A) PhaseEnd_Phase23** — write it (the LLM tier is fully characterized: local grinder + frontier test + idiom hunt, all byte-proven; a clean Tier-1 close), then **(B) plan the PUBLIC FLIP as the next phase** (Fable5 §6.3: R20 clean mirror, rom→decoder, CI, a naming pass on major systems, contributor on-ramp from `docs/backlog.md` — the strategic lever). A bounded GLM fresh-hard-band campaign remains available (money-open, modest fleet%) but is NOT the high-leverage move. Fleet stands at **~64.2%** (function-count) / ~30% byte-weighted, 136/136 byte-identical, 0 NON_MATCHING. All work committed (`commit:0379`…`commit:0422`); Drew pushes (R6). serve_local DOWN, GPU free. `db.*.gbf` churn is R23 restart-noise (do NOT stage).
|
||||
|
||||
- 2026-07-02 (**THE GIANT CAMPAIGN — Fable5 wall-break + toolkit generalization; SUPERSEDES the "PhaseEnd next" stopping point above**): Drew redirected from PhaseEnd to hand-solving giants + a Fable5 experiment. **RESULT: 3 giants banked ×134, fleet 64.67% → 64.82%, check-all 136/136, all committed** (`commit:0427`, `commit:0428`, `commit:0429`).
|
||||
- **Fable5Max (`Agent` model=fable) CRACKED `func_8014EE14`** (248 ins, reach-134) — a §20/§10 store-vs-load **"CONFIRMED unsteerable"** giant — by reading the **gcc-2.7.2 SOURCE** (`tools/reference/gcc-papermario`) + RTL `-da` dumps. **This REFUTES the T10.8/T10.9 "wall intrinsic / idiom well dry" verdict: it was MODEL-RELATIVE** (true for GLM/7B, false for a frontier model reading gcc internals). 3 idioms → cookbook **§30**: (1) store-vs-load is a deterministic `MEM_IN_STRUCT_P`/`/s` aliasing flag (grant via `((struct{s32 f;}*)p)->f` anon-struct member-ref, deny via `*p`), NOT a scheduler tie-break; (2) the def-side return-type wall has a **MACRO escape** (widen a *discarding* caller-macro's `extern void`→`s32`, byte-neutral); (3) the birthing-boost prologue-order re-tie.
|
||||
- **(a) toolkit pass:** 3 standard **Opus agents** *APPLIED* §30 to 3 backlog "unsteerable" store-vs-load fns → `func_8014F2E0` (66) + `func_80150528` (53) **banked ×134** (`commit:0428`); `func_8014EA4C` 6→3 (partial — abs/frame-fragility residual, permuter/Fable5 tail). +2 idioms → **§30a**: IV-combine single-base-pointer (keep `combine_givs` at N IVs); inline loop-limit → preheader-hoist. **KEY FINDINGS:** standard Opus agents *applying* documented idioms crack leaves at ~80–130k tokens (3–5× cheaper than Fable5 *discovering*); the "unsteerable" backlog is **largely MIS-VERDICTED**; the whole-binary integration is **MECHANICAL** (all 3 giants = the same `void`→`s32` macro-widen) → **TOOLABLE**.
|
||||
- Also this session: GLM parallel-batch campaign (`tools/glm_parallel.sh`, K concurrent OpenRouter drafters) over the 102 reach-134 hard band → **2 banks/102** (~$10 OpenRouter, def-side wall confirmed at scale; MAXTOK-tuning: reasoning models starve low, 96k overkill, ~48k+iters=2 sweet spot; iter-2 rescues ~29% of near-misses but not past the wall); free v3 grind **+89 fns** (fleet →64.7%) before I stopped it. `type-dependency map`: the 8 compile-blocked giants are **mechanically** unblocked (ghidra-types + externs), **NOT struct-walled**.
|
||||
- **★ CAMPAIGN REFRAME (the current plan):** bottleneck moved from "can we match" (yes — toolkit + Opus agents) to **THROUGHPUT**. Path: **(1) BUILD the §30 integration recovery tool** (auto macro-widen → every leaf-MATCH auto-banks — the throughput unlock); **(2) Opus-agent WAVE** over the near-miss backlog (~95 "schedule"/80 "regalloc"/31 "hoist" verdicts, mostly mis-labeled) applying the toolkit; **(3) Fable5** for genuinely-novel residuals + the untouched big giants (`func_80144B9C` 770, `func_80132784` 400 HARD-DEFER, region `a`) + the 8 compile-blocked (mechanical unblock + draft). **v4-retrain on the new idioms = LOW ROI** (too few examples, class modest, giants exceed 7B — TOOL the idioms, don't train). **IN FLIGHT:** a 2nd Fable5 on giant `func_801571C4` (198 ins, close=24, region main — prologue-schedule/slot-swap residual). Tasks tracked in the harness list (R28). serve_local DOWN, GPU free. NB `.gitignore`+`docs/backlog.md` had pre-session uncommitted edits (unrelated); `db.*.gbf` = R23 noise.
|
||||
|
||||
Reference in New Issue
Block a user