feat(phase-21): T2 — fuel prefetch, Ghidra-C cache complete for the ROI pool

- headless DecompileFunctions over 263 uncached ROI-pool fns (MCP stopped, R23):
  263 ok / 0 fail / 0 no-func. cache 300 -> 563. ROI-pool fuel now 325/325 cached
  (28/28 giants, 310/310 reach-134, 7/7 capped) -> the unattended run never needs MCP.
- FIX DecompileFunctions.java: key output by entry address func_<UPPERHEX>.c (Ghidra's
  raw-import default is FUN_<lowerhex>, which every consumer + the existing 300 files
  miss). Renamed the 263; the tool fix makes re-prefetch reproducible.
- build_fuel_manifest.py --emit-prefetch: reproducible uncached-ROI-pool addr-list
  (for the T7 runbook). 349 reach-1 x1-leverage fns intentionally excluded (P9).
- ghidra/ db churn is the MCP-stop no-op save (R23 restart-noise, not staged).
This commit is contained in:
Drew T
2026-06-21 12:00:47 -06:00
parent 971f95f7c8
commit 5674587bcb
3 changed files with 29 additions and 2 deletions
+9 -1
View File
@@ -21,7 +21,7 @@
## Task checklist
- [x] **T1 — Target-pool manifest + giants byte-verification** (no MCP; Max) → `.run/fuel_manifest.json`; h_exact-verify the 28 giants are genuinely ×134 (R14). **DONE.**
- [ ] **T2 — Fuel prefetch → complete Ghidra-C cache** (MCP-stop headless; Max) → `.run/ghidra_c/` cache-complete over the manifest.
- [x] **T2 — Fuel prefetch → complete Ghidra-C cache** (MCP-stop headless; Max) → `.run/ghidra_c/` cache-complete over the manifest. **DONE.**
- [ ] **T3 — Backlog ledger + shared deterministic bank/log stage** (Max) → `tools/gate_stage.py` + `tools/backlog.py` + `docs/backlog.md`; verify on the 7 capped fns (~+0.3%).
- [ ] **T4 — Worker Workflow** (Max; agents xHigh) → new Workflow, sample-validate ~8–10 fresh targets. **← P6 rules re-read after this.**
- [ ] **T5 — Grinder daemon repoint** (xHigh) → `auto_driver.py` permutes worker near-misses + full gate pipeline + backlog logging.
@@ -45,3 +45,11 @@
- **R14 -O0 nuance (byte-finding):** 8/9 -O0 stubs are overlay-local (reach-1, matches Phase-20), but **func_8013C08C is reach-134** — a genuine exception to the blanket "overlay-local" assumption (propagatable ×134 if matched).
- **T2 workload:** 253 reach-134 stubs uncached (of 310); tractable reach-134 (WAVE/PINS/STRUCT ≤150 ins) = 223, 171 uncached; giants 25/28 uncached. Ghidra-C cache currently 300.
- No MCP, no build mutation. Checkpoint commit: tool + this log.
### T2 — fuel prefetch → Ghidra-C cache complete (DONE, 2026-06-21)
- Stopped MCP (R23, clean Save). Headless `analyzeHeadless ... -process ov_SC01_077 -readOnly -postScript DecompileFunctions.java <addrs> .run/ghidra_c` over the **263 uncached ROI-pool fns** → **263 ok / 0 fail / 0 no-func**. Cache 300 → **563**.
- **Naming bug found + fixed:** Ghidra emits `FUN_<lowerhex>` (raw-import default); consumers + the 300 existing files use `func_<UPPERHEX>`. Renamed the 263, and **fixed `DecompileFunctions.java` to key output by entry address (`func_<UPPER>`)** so re-prefetch is reproducible. Added `build_fuel_manifest.py --emit-prefetch` (reproducible addr-list for the T7 runbook).
- **T2 verification (cache-complete):** ROI-pool fuel = **325 fns, ALL cached, 0 uncached** (28/28 giants, 310/310 reach-134, 7/7 capped). The unattended run will never need live MCP.
- Scope (P9): 349 reach-1 ×1-leverage fns intentionally NOT prefetched (not in the ROI pool rotation: tractable reach-134 → giants → -O0 → capped). A later prefetch can add them if ever needed.
- MCP left STOPPED (the run is cache-based; the grinder supervisor stops it anyway). `ghidra/ db.*.gbf` churn is the MCP-stop's no-op SLUS save — R23 restart-noise, NOT staged.
- Checkpoint commit: `DecompileFunctions.java` + `build_fuel_manifest.py` + this log (cache + manifest are gitignored/regenerable).
+14
View File
@@ -102,6 +102,10 @@ def main():
ap = argparse.ArgumentParser()
ap.add_argument("--source", default="ov_SC01_077")
ap.add_argument("--out", default=".run/fuel_manifest.json")
ap.add_argument("--emit-prefetch", metavar="PATH", default=None,
help="also write the UNCACHED ROI-pool addrs (hex, no 0x) here for "
"DecompileFunctions.java (reproducible T2 fuel prefetch). ROI pool = "
"reach>=2 OR class in {O0,O1} OR a capped-recovery fn.")
a = ap.parse_args()
src = a.source
@@ -195,6 +199,16 @@ def main():
outp = os.path.join(REPO, a.out)
json.dump(manifest, open(outp, "w"), indent=1)
if a.emit_prefetch:
cap_uncached = [c for c in CAPPED if c not in cached]
pool = {t["name"] for t in targets
if not t["cached"] and ((t["reach"] or 1) >= 2 or t["class"] in ("O0", "O1"))}
pool |= set(cap_uncached)
lines = sorted(n[len("func_"):] for n in pool) # hex, no 0x, for DecompileFunctions.java
with open(os.path.join(REPO, a.emit_prefetch), "w") as f:
f.write("\n".join(lines) + ("\n" if lines else ""))
print(f" prefetch addr-list -> {a.emit_prefetch} ({len(lines)} uncached ROI-pool fns)")
print(f"fuel manifest -> {a.out}")
print(f" overlays signed: {n_overlays} | live stubs: {len(targets)} | "
f"ghidra_c cached: {len(cached)} (manifest cached {cached_n}, uncached {len(targets)-cached_n})")
+6 -1
View File
@@ -45,7 +45,12 @@ public class DecompileFunctions extends GhidraScript {
out = "// DECOMPILE FAILED: " + (res != null ? res.getErrorMessage() : "null result");
fail++;
}
Files.write(Paths.get(outDir, f.getName() + ".c"), out.getBytes());
// Key the cache file by the function's ENTRY ADDRESS as func_<UPPERHEX>.c — the
// convention every consumer uses (derive_canonical_sigs.py, gen_wave.py, the
// fuel manifest). Ghidra's default raw-import name is FUN_<lowerhex>, so naming by
// f.getName() would miss the cache; keying by address is reproducible + correct.
String fname = "func_" + Long.toHexString(f.getEntryPoint().getOffset()).toUpperCase() + ".c";
Files.write(Paths.get(outDir, fname), out.getBytes());
}
di.dispose();
println("DecompileFunctions: " + ok + " ok, " + fail + " decompile-fail, " + nofunc