mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-01 07:40:42 -04:00
feat(recover_route): route a gate DROP to the tool that applies, and wire it into gate_main
gate_main printed ONE recovery chain for every dropped draft, and it was the
SELF chain (fix_arity_callers --any-proto + cast_self_callers) regardless of
what the clashing symbol actually was. Two of the three classes are not that
chain:
CALLEE — §378 does not transfer; cast_self_callers reads the return type off
the draft and cannot cast a callee, so --any-proto runs unprotected
over every call site. S69 measured 60 decls no-protoed, binary RED.
DATA — neither tool in the printed chain touches a data extern at all.
Measured cost of the wrong route THIS session: func_8006252C was dropped on a
clash with itself; following the shape of the printed chain I reached for
scope_demote_drafts first, which aliased D_80078D08 through __asm__ and BROKE
the build. The real blocker was one --sync-decls away. Three tools, wrong
order, one destructive — because the report named a chain instead of a route.
A route is an ORDERED LADDER, not a prediction: for a DATA clash the choice
between adopting the TU's spelling and demoting to block scope depends on
whether the draft can live with the TU's type, which no classifier can know.
The byte gate remains the sole arbiter (G3/P9). Refusals come first (R43/R61a):
a verbatim draft and a NEAR are not declaration problems.
NEGATIVE CONTROL (R39): all 7 S77 drops whose winning tool was already known
route correctly — 2 SELF (cast_self_callers), 1 CALLEE (sync_tu_decls via a
definition header), 4 DATA — and the DATA ladder's order matches which rung
actually won in each case (sync for D_80072978, demote for D_80072960 and
D_80074818). Verbatim draft refused; real-C draft not refused.
Playbook §4b and SETUP updated in the same change.
This commit is contained in:
Drew T
@@ -978,6 +978,7 @@ fills fast). Nothing is leaking — but the host does not get the memory back on
|
||||
| tool | what it does | when you need it |
|
||||
|---|---|---|
|
||||
| `tools/gate_main_parallel.py` **(NEW)** | runs the REAL `gate_main` inside N git worktrees to discover which drafts pass, then hands the union to ONE authoritative `gate_main` in the real tree. Workers discover; only the final serial pass banks | a main slate large enough that serial bisection hurts. **Measured: one gate cycle is 16 s**, so MAX_STEPS=24 is ~6.4 min serial and ~90 s across four workers. Run `--negative-control` once per environment first |
|
||||
| `tools/recover_route.py` **(NEW)** | reads a gate's DROP ledger and names the tool that actually applies — SELF / CALLEE / DATA, each with its ordered ladder; refuses a verbatim draft up front. Wired into `gate_main`'s dropped report | after any gate that drops a draft. Replaces the single hardcoded chain gate_main used to print for every drop, which was the SELF chain and measured RED on a callee (S69) and inert on data. Negative-controlled against all 7 S77 drops whose winning tool was already known |
|
||||
| `tools/sync_tu_decls.py` **(NEW)** | banks a draft the gate refuses by copying the TU's OWN `extern` line for whatever symbol the gate names, re-gating, and repeating | a draft that is byte-correct but rejected on a declaration conflict. Refuses `self_decl_tu` (use `cast_self_callers --sync-decls`) and refuses a NEAR up front, since syncing declarations makes a body COMPILE, never MATCH |
|
||||
|
||||
**Oracle corrections — re-read any verdict recorded before these:**
|
||||
|
||||
@@ -362,6 +362,22 @@ tools/cast_self_callers.py --undo-journal J2 --keep <banked> # MANDATORY —
|
||||
|
||||
Or in one driver: `recover_integration.py --binary B --stages arity,self-cast --max-tier fleet --r22`.
|
||||
|
||||
> **DO NOT PICK THE TOOL BY HAND — `tools/recover_route.py` holds the table (P31 S77).** `gate_main`
|
||||
> used to print the SELF chain above for EVERY drop, whatever the clashing symbol was, and two of the
|
||||
> three classes are not that chain: a CALLEE drop run through `--any-proto` measured **60 decls
|
||||
> no-protoed and the binary RED** (S69), and a DATA drop is touched by neither tool in it. Following
|
||||
> the shape of that text is what sent S77's `func_8006252C` through `scope_demote_drafts` first,
|
||||
> which aliased a symbol through `__asm__` and BROKE the build — the real blocker was one
|
||||
> `--sync-decls` away. `gate_main` now routes each drop; run the router standalone with
|
||||
> `tools/recover_route.py .run/gate_main_dropped.json --drafts <dir>`. It refuses a verbatim draft
|
||||
> outright, and a route is the order to TRY, never a prediction — the byte gate still arbitrates.
|
||||
>
|
||||
> | drop | tool |
|
||||
> |---|---|
|
||||
> | symbol **is** the function being banked | `cast_self_callers --sync-decls` (§477, 16/16) |
|
||||
> | symbol is a **different function** | `sync_tu_decls` — adopt the TU's spelling, incl. from its DEFINITION header |
|
||||
> | symbol is **data** | `sync_tu_decls`, then `scope_demote_drafts` (§8d/§481) |
|
||||
|
||||
> **THE UNDO IS NOT OPTIONAL.** The casts go in BEFORE the gate, in preparation. A cast left behind
|
||||
> for a draft that did not bank made `ov_SC07_000` fail to COMPILE at HEAD, so every later gate
|
||||
> verdict on that binary measured a broken baseline rather than a draft (found only because two
|
||||
|
||||
+25
-7
@@ -1012,13 +1012,31 @@ def main():
|
||||
# to disk with the chain spelled out makes the recovery the obvious next command instead
|
||||
# of a paragraph someone has to remember.
|
||||
json.dump(dropped, open('.run/gate_main_dropped.json', 'w'), indent=1)
|
||||
print(f" -> .run/gate_main_dropped.json ({len(dropped)} to reconcile). The chain is:")
|
||||
print(f" tools/fix_arity_callers.py --apply --any-proto --funcs "
|
||||
f"{','.join(d['fn'] for d in dropped)} \\\n"
|
||||
f" --drafts <dir> --journal .run/<id>/arity.json")
|
||||
print(f" tools/cast_self_callers.py --binary main --funcs <same> --drafts <dir> "
|
||||
f"--apply --journal .run/<id>/cast.json")
|
||||
print(f" tools/gate_main.py <slate> --apply # the byte-gate arbitrates")
|
||||
print(f" -> .run/gate_main_dropped.json ({len(dropped)} to reconcile). Routed per drop:")
|
||||
# ONE CHAIN FOR EVERY DROP WAS WRONG FOR TWO OF THE THREE CLASSES (P31 S77). What used to
|
||||
# print here was the SELF chain (`fix_arity_callers --any-proto` + `cast_self_callers`)
|
||||
# regardless of what the clashing symbol actually is. For a CALLEE the playbook already
|
||||
# records the cost -- §378 does not transfer, `--any-proto` runs unprotected over every call
|
||||
# site, and S69 measured 60 decls no-protoed and the binary RED. For a DATA symbol neither
|
||||
# tool touches it at all. Following the shape of this text is what sent S77's func_8006252C
|
||||
# through `scope_demote_drafts` first, which aliased a symbol through __asm__ and BROKE the
|
||||
# build, when the real blocker was one `--sync-decls` away. A drop is a ROUTE (S72) -- so
|
||||
# print the route, not a chain. recover_route is the shared table; the byte gate still
|
||||
# arbitrates (G3/P9).
|
||||
try:
|
||||
import recover_route as _rr
|
||||
for _d in dropped:
|
||||
_dd = os.path.dirname(next((e['draft'] for e in slate if e['fn'] == _d['fn']), '') or '')
|
||||
_kind, _cmds, _why = _rr.route(_d['fn'], _d.get('symbol', ''), _d.get('kept', ''),
|
||||
_d.get('this', ''),
|
||||
draft=os.path.join(_dd, _d['fn'] + '.c') if _dd else None,
|
||||
binary='main', tu=_d.get('file'))
|
||||
print(f" {_d['fn']:<16} {_kind:<9} on {_d.get('symbol')} — {_why}")
|
||||
for _c in _cmds:
|
||||
print(f" {_c}")
|
||||
except Exception as _e:
|
||||
print(f" (recover_route unavailable: {type(_e).__name__}: {_e}) "
|
||||
f"— tools/recover_route.py .run/gate_main_dropped.json")
|
||||
if not a.apply:
|
||||
print("\nDRY RUN. Re-run with --apply to substitute and clean-rebuild.")
|
||||
return
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
#!/usr/bin/env python3
|
||||
"""recover_route.py — given a gate's DROP verdict, name the tool that actually applies. (P31 S77)
|
||||
|
||||
WHY THIS EXISTS, WITH THE MEASUREMENT. `gate_main` writes every dropped draft to
|
||||
`.run/gate_main_dropped.json` and then prints ONE chain for all of them:
|
||||
|
||||
tools/fix_arity_callers.py --apply --any-proto --funcs <all dropped> ...
|
||||
tools/cast_self_callers.py --binary main --funcs <same> ...
|
||||
|
||||
That is the SELF-declaration chain (§378), and it is correct for exactly one of the three drop
|
||||
classes. The playbook already records what the others cost:
|
||||
|
||||
* CALLEE (`conflicting types for func_8012AD44`, a DIFFERENT function): §378 does NOT transfer.
|
||||
`cast_self_callers` reads the return type off the draft and cannot cast a callee, so
|
||||
`--any-proto` runs UNPROTECTED and changes argument conversion at every call site.
|
||||
MEASURED (S69): 60 decls no-protoed, **binary RED**, reverted.
|
||||
* DATA (`D_800A5E60`, `D_80072960`): neither tool in the printed chain touches a data extern at
|
||||
all. The fix is to adopt the TU's spelling into the DRAFT (`sync_tu_decls`) or, when the draft
|
||||
genuinely needs an incompatible type, demote it to block scope (§8d / §481).
|
||||
|
||||
MEASURED COST OF THE WRONG ROUTE, THIS SESSION. `func_8006252C` was dropped on a clash with
|
||||
`func_8006252C` ITSELF. Following the shape of the printed chain I reached for
|
||||
`scope_demote_drafts` first: it aliased `D_80078D08` through `__asm__`, the build FAILED, and the
|
||||
draft looked broken. The actual blocker was the self-declaration, one command away. Three tools,
|
||||
wrong order, one of them destructive — because the report named a chain instead of a route.
|
||||
|
||||
WHAT THIS DOES NOT DO. It does not predict which tool will WIN. For a DATA clash there are two
|
||||
plausible fixes and the difference is whether the draft can live with the TU's type at all — not
|
||||
something a classifier can know. So a route is an ORDERED LADDER and **the byte gate remains the
|
||||
sole arbiter (G3/P9)**; the ladder just stops you running the destructive one first.
|
||||
|
||||
REFUSALS COME FIRST (R43/R61a). A verbatim draft and a NEAR are not declaration problems, and
|
||||
routing them to a declaration tool wastes a clean rebuild to learn what two cheap checks already
|
||||
know.
|
||||
|
||||
tools/recover_route.py .run/gate_main_dropped.json --drafts .run/<wave>/<arm>
|
||||
tools/recover_route.py --fn func_X --symbol D_Y --kept "..." --this "..."
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO = os.path.dirname(HERE)
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
SELF, CALLEE, DATA = 'SELF', 'CALLEE', 'DATA'
|
||||
|
||||
|
||||
def symbol_kind(fn, symbol, kept='', this=''):
|
||||
"""SELF / CALLEE / DATA for the symbol a gate named in a DROP.
|
||||
|
||||
The signature strings are `typesig` tuples rendered by gate_main, e.g.
|
||||
function : ('void', ('void',)) -> a declarator WITH a parameter list
|
||||
fn-pointer : ('s32 (*', ('* void',)) -> an OBJECT whose type is pointer-to-function
|
||||
plain data : ('s16', '[]') -> no parameter list at all
|
||||
A `(*` in the base type is what separates a pointer-to-function OBJECT from a FUNCTION, and it
|
||||
is the distinction that decides the tool: `cast_self_callers` casts CALL SITES and is
|
||||
meaningless for an object."""
|
||||
if symbol == fn:
|
||||
return SELF
|
||||
blob = '%s %s' % (kept, this)
|
||||
looks_callable = ('(' in blob and ')' in blob and '(*' not in blob) or bool(re.search(r'\(\s*\(', blob))
|
||||
if looks_callable and symbol.startswith('func_'):
|
||||
return CALLEE
|
||||
if symbol.startswith('func_') and not blob.strip():
|
||||
return CALLEE
|
||||
return DATA
|
||||
|
||||
|
||||
def refusal(fn, draft, binary='main', asm_subdir=None):
|
||||
"""A reason this drop should not be routed to a declaration tool at all, or None."""
|
||||
if not draft or not os.path.exists(draft):
|
||||
return None
|
||||
try:
|
||||
import draft_prechecks as DP
|
||||
if DP.is_verbatim_asm_draft(open(draft, errors='replace').read(), fn):
|
||||
return ("VERBATIM — the draft is the target's own asm in a file-scope __asm__ (§265/§478). "
|
||||
"It is not a decompile, no declaration fix applies, and the gate refuses it for "
|
||||
"free. REDRAFT or accept it as a §265 bank via asm_verbatim.py.")
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def route(fn, symbol, kept='', this='', draft=None, binary='main', tu=None):
|
||||
"""(kind, [ordered commands]) — the ladder to try, cheapest/safest first."""
|
||||
stop = refusal(fn, draft)
|
||||
if stop:
|
||||
return 'REFUSED', [], stop
|
||||
kind = symbol_kind(fn, symbol, kept, this)
|
||||
d = os.path.dirname(draft) if draft else '<drafts dir>'
|
||||
if kind == SELF:
|
||||
cmds = [
|
||||
"tools/cast_self_callers.py --binary %s --funcs %s --drafts %s --sync-decls --apply "
|
||||
"--journal .run/<id>/cast.json" % (binary, fn, d),
|
||||
"make check BINARY=%s # LAW 1: byte-neutral with NO draft substituted" % binary,
|
||||
"git commit # the gate checks out the TUs / pins a worktree",
|
||||
"tools/gate_main.py <slate> --apply",
|
||||
"tools/cast_self_callers.py --undo-journal .run/<id>/cast.json --keep <banked>",
|
||||
]
|
||||
why = ("the clash names the function being banked, so the CALL SITES must change too — "
|
||||
"this is the only class §378 covers, and it banked 16 of 16 in S77 (§477).")
|
||||
elif kind == CALLEE:
|
||||
cmds = [
|
||||
"tools/sync_tu_decls.py --binary %s --fn %s --draft %s/%s.c --rounds 6 --apply"
|
||||
% (binary, fn, d, fn),
|
||||
]
|
||||
why = ("the clash names a DIFFERENT function, so §378 does NOT transfer: cast_self_callers "
|
||||
"reads the return type off the draft and cannot cast a callee, and --any-proto would "
|
||||
"run unprotected over every call site (S69: 60 decls no-protoed, binary RED). Adopt "
|
||||
"the TU's spelling into the DRAFT instead — including from its DEFINITION header.")
|
||||
else:
|
||||
cmds = [
|
||||
"tools/sync_tu_decls.py --binary %s --fn %s --draft %s/%s.c --rounds 6 --apply"
|
||||
% (binary, fn, d, fn),
|
||||
"tools/scope_demote_drafts.py --overlay %s --in <dir> --out <dir> --src-file %s"
|
||||
% (binary, tu or '<tu>.c'),
|
||||
]
|
||||
why = ("a DATA symbol: neither tool in gate_main's printed chain touches one. Try adopting "
|
||||
"the TU's spelling first; if the draft genuinely needs an incompatible type, demote "
|
||||
"it to BLOCK scope — legal because `conflicting types' is a SAME-SCOPE error and "
|
||||
"across scopes it is only a warning (§481/§8d). The byte gate decides which.")
|
||||
return kind, cmds, why
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("dropped", nargs="?", default=".run/gate_main_dropped.json")
|
||||
ap.add_argument("--drafts", default=None, help="dir holding <fn>.c, so the refusal checks can run")
|
||||
ap.add_argument("--binary", default="main")
|
||||
ap.add_argument("--fn"); ap.add_argument("--symbol")
|
||||
ap.add_argument("--kept", default=""); ap.add_argument("--this", default="")
|
||||
ap.add_argument("--json", action="store_true")
|
||||
a = ap.parse_args()
|
||||
|
||||
if a.fn and a.symbol:
|
||||
rows = [{"fn": a.fn, "symbol": a.symbol, "kept": a.kept, "this": a.this}]
|
||||
else:
|
||||
p = a.dropped if os.path.isabs(a.dropped) else os.path.join(REPO, a.dropped)
|
||||
if not os.path.exists(p):
|
||||
sys.exit("recover_route: no %s — run a gate first (it writes the drop ledger)." % a.dropped)
|
||||
rows = json.load(open(p))
|
||||
|
||||
out = []
|
||||
for r in rows:
|
||||
draft = os.path.join(a.drafts, r["fn"] + ".c") if a.drafts else None
|
||||
kind, cmds, why = route(r["fn"], r.get("symbol", ""), r.get("kept", ""), r.get("this", ""),
|
||||
draft=draft, binary=a.binary, tu=r.get("file"))
|
||||
out.append({"fn": r["fn"], "symbol": r.get("symbol"), "kind": kind, "why": why, "cmds": cmds})
|
||||
|
||||
if a.json:
|
||||
print(json.dumps(out, indent=1)); return 0
|
||||
for o in out:
|
||||
print("\n%-16s %-14s clash on %s" % (o["fn"], o["kind"], o["symbol"]))
|
||||
print(" why: %s" % o["why"])
|
||||
for c in o["cmds"]:
|
||||
print(" %s" % c)
|
||||
if out:
|
||||
print("\n%d drop(s) routed. The byte gate remains the sole arbiter — a route is the order to "
|
||||
"TRY, never a prediction of which one wins." % len(out))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user