mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-29 07:10:32 -04:00
docs(phase-21): automation runbook + status tool (grinder running; worker = remote-driven)
- docs/automation-runbook.md: launch/monitor/stop for the grinder (live) + worker /loop; safety invariants (byte-gate sole arbiter, git crash-safe, auto_stop kill switch); remote-management note (Drew can remote in -> no bulletproof keep-alive needed). - auto_status.sh: show grinder + worker heartbeats, backlog, phase-21 checkpoints, daemons.
This commit is contained in:
@@ -0,0 +1,73 @@
|
||||
# Phase-21 Automation Runbook (the unattended grind)
|
||||
|
||||
The automation manager: a **token-free grinder** (CPU permuter) + a **token-heavy worker** (LLM
|
||||
agent waves), both banking through the **incorruptible whole-binary byte-gate** (G3/P9 — a wrong
|
||||
match can NEVER bank) and logging every near-miss to a **ranked backlog** for hand-finishing.
|
||||
|
||||
## What is running right now
|
||||
|
||||
- **Grinder** (`tools/grinder.py` under `tools/auto_supervisor.sh`) — **LAUNCHED**, token-free.
|
||||
Permutes the backlog's closest near-misses → byte-gate → banks the real matches → propagates ×134.
|
||||
Idles when the backlog is drained (waiting for worker-produced near-misses). Survives crashes
|
||||
(supervisor relaunches). Heartbeat: `.run/auto/grinder_heartbeat.json`.
|
||||
|
||||
## Monitor (read-only, from anywhere)
|
||||
|
||||
```sh
|
||||
bash tools/auto_status.sh # grinder + worker heartbeats, backlog size, recent commits
|
||||
cat .run/auto/grinder_heartbeat.json # grinder: state/current/banked/fleet
|
||||
.venv/bin/python tools/orchestrator.py status # worker ROI state (pool, waves, banked_total)
|
||||
.venv/bin/python tools/backlog.py show -n 40 # the ranked near-miss backlog (docs/backlog.md)
|
||||
git log --oneline -15 | grep phase-21 # what banked
|
||||
make report BINARY=main # fleet % (+ dedup byte-honesty check, must be 0 failed)
|
||||
```
|
||||
|
||||
## STOP everything (the kill switch)
|
||||
|
||||
```sh
|
||||
bash tools/auto_stop.sh # touches .run/auto/STOP -> grinder finishes its current step, exits;
|
||||
# the supervisor sees STOP and does not relaunch. Safe at any time.
|
||||
rm .run/auto/STOP # to allow a relaunch later
|
||||
```
|
||||
|
||||
## Launch the WORKER waves (token-heavy — the high-yield engine)
|
||||
|
||||
The worker drafts matching C with LLM agents (the §17–20 toolkit: register pins, array-of-struct
|
||||
%lo-fold, call-site casts) — it cracks the hard tail the grinder can't. It needs a **Claude session**
|
||||
(only a session can invoke the Workflow tool), so it runs as a self-paced **`/loop`**:
|
||||
|
||||
1. In a Claude Code session in this repo, run **`/loop`** with this cycle as the prompt:
|
||||
> Run one orchestrator cycle: `tools/orchestrator.py prep --n 24` → read `.run/auto/wave_batch.json`
|
||||
> → launch the `tools/workflows/worker_wave.js` Workflow with `args={draftDir:".run/drafts-wave",
|
||||
> targets:<the batch>}` → after it completes, `tools/orchestrator.py finish --drafts
|
||||
> .run/drafts-wave --commit` → report the JSON summary. Stop if `.run/auto/STOP` exists.
|
||||
2. `/loop` self-paces (one wave per tick, ~15–20 min/wave, ~275k tokens/wave of ~24 agents).
|
||||
The grinder runs alongside, draining the near-misses each wave produces (CPU vs token budget).
|
||||
3. Cost note: ~24 xHigh agents/wave. The ROI gate rotates pools (tractable → giants → o0 → capped)
|
||||
when a pool's bank-rate drops, so it doesn't grind a wall. Stop anytime with `auto_stop.sh`.
|
||||
|
||||
**Remote management (Drew has laptop + can remote into the dev box):** you don't need a bulletproof
|
||||
keep-alive — if the worker `/loop` session dies, just remote in and re-run `/loop` (the grinder daemon
|
||||
keeps running regardless, and every bank is already committed, so nothing is lost). Monitor with
|
||||
`tools/auto_status.sh`; stop with `tools/auto_stop.sh`; resume by re-launching. The byte-gate guarantees
|
||||
correctness while unattended, so the worst case of a crash is "it paused," never "it broke something."
|
||||
|
||||
## Re-prefetch fuel (only if adding fresh targets, needs Ghidra)
|
||||
|
||||
The run is **cache-based** (no live MCP needed). To add targets to the Ghidra-C cache later:
|
||||
```sh
|
||||
bash tools/ghidra_mcp_stop.sh # R23 (free the project lock)
|
||||
.venv/bin/python tools/build_fuel_manifest.py --emit-prefetch .run/prefetch_addrs.txt
|
||||
"$HOME/ghidra_12.1_PUBLIC/support/analyzeHeadless" "$HOME/bfm-decomp/ghidra" bfm \
|
||||
-process ov_SC01_077 -noanalysis -readOnly -scriptPath tools/ghidra_scripts \
|
||||
-postScript DecompileFunctions.java .run/prefetch_addrs.txt .run/ghidra_c
|
||||
```
|
||||
|
||||
## Safety invariants (why this is safe to leave running)
|
||||
|
||||
- **Byte-gate is the sole arbiter (G3/P9):** every bank is whole-binary SHA1-verified; a wrong draft
|
||||
is reverted, never banked. `make check-all` stays 136/136.
|
||||
- **git is the crash-safe state machine:** every bank is a checkpoint commit (push is manual, R6 —
|
||||
nothing leaves the machine on its own); `dedup_propagate --auto-from` is additive/resumable.
|
||||
- **`auto_stop.sh`** halts both engines at the next safe boundary.
|
||||
- **Honest measurement (P9):** only byte-matches bank; near-misses go to `docs/backlog.md`, ranked.
|
||||
+13
-6
@@ -1,9 +1,16 @@
|
||||
#!/bin/bash
|
||||
# auto_status.sh — show the unattended run's heartbeat + recent checkpoints (remote check-in).
|
||||
# auto_status.sh — show the unattended run's state (grinder + worker) for a remote check-in.
|
||||
cd "$(dirname "$0")/.."
|
||||
echo "=== heartbeat (.run/auto/heartbeat.json) ==="
|
||||
[ -f .run/auto/heartbeat.json ] && cat .run/auto/heartbeat.json || echo "(no heartbeat yet)"
|
||||
echo; echo "=== STOP requested? ==="; [ -f .run/auto/STOP ] && echo "YES (.run/auto/STOP present)" || echo "no"
|
||||
echo; echo "=== last 8 phase-16 checkpoints ==="
|
||||
git log --oneline -8 --grep='phase-16' 2>/dev/null || git log --oneline -8
|
||||
echo "=== grinder heartbeat (.run/auto/grinder_heartbeat.json) ==="
|
||||
[ -f .run/auto/grinder_heartbeat.json ] && cat .run/auto/grinder_heartbeat.json || echo "(no grinder heartbeat yet)"
|
||||
echo; echo "=== worker orchestrator state (.run/auto/orch_state.json) ==="
|
||||
[ -f .run/auto/orch_state.json ] && cat .run/auto/orch_state.json || echo "(worker loop not started)"
|
||||
echo; echo "=== legacy auto_driver heartbeat ==="
|
||||
[ -f .run/auto/heartbeat.json ] && cat .run/auto/heartbeat.json || echo "(none)"
|
||||
echo; echo "=== STOP requested? ==="; [ -f .run/auto/STOP ] && echo "YES — run is halting (rm .run/auto/STOP to resume)" || echo "no (running)"
|
||||
echo; echo "=== backlog (open near-misses, ranked) ==="
|
||||
[ -f .run/backlog.jsonl ] && .venv/bin/python tools/backlog.py show -n 12 2>/dev/null || echo "(backlog empty)"
|
||||
echo; echo "=== last 12 phase-21 checkpoints ==="
|
||||
git log --oneline -12 --grep='phase-21' 2>/dev/null || git log --oneline -12
|
||||
echo; echo "=== supervisor tail ==="; tail -6 .run/auto/supervisor.log 2>/dev/null || echo "(none)"
|
||||
echo; echo "=== running daemons ==="; pgrep -af 'grinder.py|auto_supervisor' | sed 's/ -c .*//' | head -4 || echo "(none)"
|
||||
|
||||
Reference in New Issue
Block a user