mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 13:33:34 -04:00
feat(phase-29): backlog prune — compact the append-only near-miss log + wire into make report
The near-miss ledger (.run/backlog.jsonl) is append-only, so it filled with already-banked noise: 6,867 rows, ~98% banked. load_best()/render() already filtered on READ (docs/backlog.md was correct), but the raw log drifted stale and every render re-scanned all 6,867 rows against the stub oracle. - backlog.py: new `prune` subcommand — atomic rewrite (temp + os.replace) to load_best()'s output (drop-now-matched P9 + best-per-addr collapse). Idempotent. 6,867 -> 1,704 open near-misses. - Makefile: `backlog.py prune` wired into `make report` (BINARY=main block) so the ledger tracks reality every cycle instead of drifting. - Finding (Drew's question): crack waves DO log every non-byte-match to the backlog durably (gate_stage copies best_draft -> .run/backlog_drafts/). BUT the `closeness` field is UNRELIABLE — byte-correct drafts (match_one MATCH) are logged with closeness>0 (e.g. func_8012F49C logged 29, actually MATCH). And a reach-N function's draft is overlay-SPECIFIC (per-location symbols), so the backlog is a messy recovery source vs the fresh per-wave stranded drafts. Integration-recovery should consume the fresh wave-dir strandeds, not re-derive from the backlog.
This commit is contained in:
+1687
-6850
File diff suppressed because it is too large
Load Diff
@@ -216,6 +216,10 @@ ifeq ($(BINARY),main)
|
||||
$(VENV_PY) tools/dup_report.py --cross
|
||||
# Fleet roll-up (Phase 15): deterministic per-binary table + fleet totals -> docs/progress.fleet.md.
|
||||
$(VENV_PY) tools/progress.py --fleet
|
||||
# Backlog compaction (Phase 29): the near-miss log is append-only, so it fills with already-banked
|
||||
# noise (measured 6,867 rows, 98% banked). prune rewrites .run/backlog.jsonl to the open near-misses
|
||||
# (drop-now-matched P9 + best-per-addr) so the ledger tracks reality instead of drifting stale.
|
||||
$(VENV_PY) tools/backlog.py prune
|
||||
# Rename-drift gate (Phase 26-A): fail-closed if a symbols.us.txt rename left a func_<ADDR>
|
||||
# ref dangling in committed src/ or src/shared/*.h — the R22 failure mode an incremental build
|
||||
# masks (stale .o) but a genuinely-clean rebuild fails on. The ONLY detector for it.
|
||||
|
||||
+1705
-1713
File diff suppressed because it is too large
Load Diff
@@ -150,6 +150,29 @@ def render():
|
||||
return len(recs)
|
||||
|
||||
|
||||
def prune():
|
||||
"""Compact .run/backlog.jsonl to reality: keep ONE record per addr (the best), and DROP every
|
||||
entry whose function is no longer an open stub in its binary (banked since it was logged, P9).
|
||||
|
||||
The raw log is append-only, so it grows without bound and fills with already-banked noise
|
||||
(measured 2026-07-24: 6,867 rows, ~98% already banked). `render()`/`load_best()` already filter
|
||||
on READ, so docs/backlog.md was correct — but every render re-scanned all 6,867 rows and re-hit
|
||||
the stub oracle, and the file itself misrepresented the real open count. prune rewrites the log
|
||||
to load_best()'s output so the file matches what the tools already compute.
|
||||
|
||||
Atomic (temp + os.replace) so an interrupt never truncates the log. Idempotent."""
|
||||
before = sum(1 for _ in open(JSONL)) if os.path.exists(JSONL) else 0
|
||||
kept = load_best() # already: drop-now-matched (P9) + best-per-addr
|
||||
tmp = JSONL + ".tmp"
|
||||
with open(tmp, "w") as f:
|
||||
for r in kept:
|
||||
f.write(json.dumps({k: r.get(k) for k in FIELDS}) + "\n")
|
||||
os.replace(tmp, JSONL)
|
||||
print(f"backlog prune: {before} rows -> {len(kept)} open near-misses "
|
||||
f"(dropped {before - len(kept)} banked/superseded)")
|
||||
return len(kept)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
@@ -159,6 +182,7 @@ def main():
|
||||
for fl in ("reach", "nins", "closeness"):
|
||||
lg.add_argument(f"--{fl}", type=int, default=None)
|
||||
sub.add_parser("render")
|
||||
sub.add_parser("prune")
|
||||
sh = sub.add_parser("show"); sh.add_argument("-n", type=int, default=40)
|
||||
a = ap.parse_args()
|
||||
if a.cmd == "log":
|
||||
@@ -169,6 +193,8 @@ def main():
|
||||
print(f"logged {rec.get('name') or rec.get('addr')}; backlog open={n}")
|
||||
elif a.cmd == "render":
|
||||
print(f"docs/backlog.md: {render()} open near-misses")
|
||||
elif a.cmd == "prune":
|
||||
prune(); render()
|
||||
elif a.cmd == "show":
|
||||
for i, r in enumerate(sorted(load_best(), key=_rank_key)[:a.n], 1):
|
||||
print(f"{i:3} {r.get('name'):16} reach={r.get('reach')} {r.get('klass'):7} "
|
||||
|
||||
Reference in New Issue
Block a user