mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-04 08:21:50 -04:00
phase-36: generator R27 named_ports — the same function lever-free in another binary, ported by pairing the two original objects' relocation sequences (+ carried file-scope externs, the target's return type); wired into delever_regen and recipe_candidates; 511/1,010 residue classes have a donor, a 24-class probe closed 4; METHOD step 12 (S104's landings)
This commit is contained in:
Drew T
@@ -105,3 +105,27 @@
|
||||
scalar stores in `sched.c`/`cse.c` is the known channel). If you can test it on bytes with a body-local struct declaration, do.
|
||||
11. The free sweep already ran every generator family (R2–R26) on your class: its best candidate is named in your brief (the path
|
||||
under `.run/P36/regen/s104_all/`) — start from whichever of it and `body_free.c` scores lower.
|
||||
12. **S104's landings — all at 0, eight of nine with ZERO levers (read these before the allocation table):**
|
||||
- (d2, d6) **a lever-free SAME-NAME variant in another binary is the answer half the time**: grep the WHOLE `src/` for the
|
||||
function's other definitions (not only `related.txt`, which searches one overlay). Port it: rename the per-overlay `D_`
|
||||
symbols (pair the two objects' relocations, or the extern lists), take the return type from the target TU's own later
|
||||
`extern`, add body-local `extern`s for symbols the donor TU declared at file scope.
|
||||
- (d3, d9) **a goto chain → STRUCTURED if/else (or a `||`)** — every generator mutates the goto text, so the sweep cannot
|
||||
reach it. A label between a condjump and a constant store blocks jump.c's if-conversion (`jump.c:805`, `:1019`); one
|
||||
constant store per arm keeps cross-jump from re-merging (`jump.c:2371`).
|
||||
- (d1) `x = a; if (c) x = f(a);` → `if (!c) x = a; else x = f(a);` — the NON-simple value in the ELSE arm (jump1
|
||||
re-merges an if/else whose else value is a reg/const, `jump.c:739-741`); never a ternary.
|
||||
- (d3) **reorder an independent test before a value's definition** to flip a scheduler tie-break (`sched.c:2428`) → the
|
||||
live lengths `allocno_compare` ranks on.
|
||||
- (d5) a hoisted temp `v = *(T*)(b+K); … x = x + v;` → `x += *(T*)(b+K)`: local-alloc's THREE-quantity order is by BIRTH,
|
||||
not a sort (`local-alloc.c:1486-1507`). `tools/localalloc_sim.py` sorts fully and mispredicts this — the fix is in
|
||||
`.run/P36/agents/ov_SC05_018__func_801837E8/scratch/lsim3.py`.
|
||||
- (d7) **declare a function-scope local INSIDE each switch case / if-arm** when it is used only there: a per-arm local
|
||||
crossing a call takes `$s0` in local-alloc (`local-alloc.c:2101-2106`) before global runs; `combine_regs` can then tie
|
||||
it (`:1722`, refused at `:1773` for a shared one). The inverse of S103 c18's merge.
|
||||
- (d8) **the WIDTH of an incremented local** decides sched1's birth priority: `(*(u16 *)(p + 2))++` or a `u16` counter
|
||||
(a SUBREG destination fails `birthing_insn_p`, `sched.c:2477-2490`); `+= 1` folds back to SImode and fails.
|
||||
- (d2) `home = (short)t;` re-extending an already-short value, with `t` given a second use (`combine.c:7926-7942`).
|
||||
- (d4) a marked `do { store; } while (0)` as a LOOP-note scheduling barrier (`sched.c:2058-2074`) — allowed, marked.
|
||||
- The `.sched` ready-list trace (`blocking insn N for 1 cycles`, the uid picked next) and `.lreg`'s `Register N in 16.`
|
||||
lines settled three of these where the allocation table misled.
|
||||
|
||||
@@ -1820,3 +1820,15 @@ CLAIM, not a fact — two were refuted on bytes on 2026-09-10.**
|
||||
classes the pass has judged and not closed. An ALL-families pass (`--families R2 … R26`, `-j 26`) costs ~1.8 s per `--try`
|
||||
and 100–370 candidates per head class; it closed an R12 class in its first minute, so re-running the old families on a
|
||||
moved tree is not redundant.
|
||||
- **Generator R27 `named_ports` (`tools/delever.py`, S104; `delever_regen --families R27`)** — the SAME function already
|
||||
lever-free in another binary, ported. Donors: every definition of the name in `src/` (`named_definitions()`, one `git
|
||||
grep`, cached) with no `register`/`__asm__`/`!FAKE`, nearest line count first, ≤ 6 distinct texts. Symbol renaming by
|
||||
**`reloc_map`**: the two ORIGINAL objects' relocation sequences for the function (`objdump -dr`, the snapshot object; a
|
||||
shared header's first includer's) paired in order — a donor symbol mapping to two targets refuses the map; fallbacks pair
|
||||
the `extern` lists by position or the first occurrences, or rename nothing. **`_carry_decls`** copies the donor TU's
|
||||
file-scope `extern` for every renamed data symbol the target TU (minus the replaced body) never declares — d6's
|
||||
COMPILE-ERROR. Signature variants: the donor's, the donor's with the TARGET's return type (d2's `s32` vs the TU's later
|
||||
`extern void`), the target's whole header. Measured at birth: 511 of 1,010 residue classes (636 bodies) have a donor; a
|
||||
24-class probe closed 4 at the first candidate, all by `reloc`. Known limits: a same-name donor that is a DIFFERENT
|
||||
function scores high (the ranking is by line count only); a pair whose REL addends differ cannot match (MIPS REL: the
|
||||
addend is in the instruction, the object comparison sees the symbol name).
|
||||
|
||||
+180
-1
@@ -3096,7 +3096,183 @@ def alias_repeated_addresses(text, tu, fn, d_):
|
||||
return out
|
||||
|
||||
|
||||
ALL_FAMILIES = ("R2", "R3", "R4", "R5", "R6", "R7", "R8", "R9", "R10", "R12", "R13", "R14", "R15", "R16", "R17", "R18", "R19", "R20", "R21", "R22", "R23", "R24", "R25", "R26")
|
||||
_NAMED_DEFS = None
|
||||
_SYM = r"(?:D|g)_[0-9A-Fa-f]{8}"
|
||||
|
||||
|
||||
def named_definitions():
|
||||
"""{name: [(file, line)]} — every definition-looking line of a `func_XXXXXXXX` in src/ (.c and shared .h), cached once."""
|
||||
global _NAMED_DEFS
|
||||
if _NAMED_DEFS is None:
|
||||
r = subprocess.run(["git", "grep", "-nE", r"^[A-Za-z_][^;]*\bfunc_[0-9A-Fa-f]{8}[[:space:]]*\([^;]*$", "--", "src/*.c", "src/*.h"],
|
||||
cwd=REPO, capture_output=True, text=True, errors="surrogateescape")
|
||||
idx = collections.defaultdict(list)
|
||||
for ln in r.stdout.splitlines():
|
||||
f, n, t = ln.split(":", 2)
|
||||
for m in re.finditer(r"\b(func_[0-9A-Fa-f]{8})\s*\(", t):
|
||||
idx[m.group(1)].append((f, int(n)))
|
||||
break
|
||||
_NAMED_DEFS = dict(idx)
|
||||
return _NAMED_DEFS
|
||||
|
||||
|
||||
def _fn_text(raw, rel, fn):
|
||||
d_ = sc_body_span(raw, rel, fn)
|
||||
if not d_:
|
||||
return None
|
||||
ls = line_starts(raw)
|
||||
return raw[ls[d_["line"] - 1]:ls[d_["end"]]] if d_["end"] < len(ls) else raw[ls[d_["line"] - 1]:]
|
||||
|
||||
|
||||
def sc_body_span(text, rel, fn):
|
||||
return next((r for r in sc.scan_text(text, rel, shared_defs=None) if r["form"] == "def" and r["name"] == fn), None)
|
||||
|
||||
|
||||
def _uniq(seq):
|
||||
seen, out = set(), []
|
||||
for x in seq:
|
||||
if x not in seen:
|
||||
seen.add(x)
|
||||
out.append(x)
|
||||
return out
|
||||
|
||||
|
||||
def _obj_of(rel):
|
||||
"""The original-bytes object that carries `rel`'s code: a .c file's own object (snapshot first); a shared header's
|
||||
first includer's."""
|
||||
if rel.endswith(".h"):
|
||||
r = subprocess.run(["git", "grep", "-l", "-F", pathlib.Path(rel).name, "--", "src/*.c"], cwd=REPO,
|
||||
capture_output=True, text=True)
|
||||
inc = r.stdout.split()
|
||||
if not inc:
|
||||
return None
|
||||
rel = inc[0]
|
||||
p = oracle.baseline_path("build/" + rel[:-2] + ".o")
|
||||
return p if p.exists() else None
|
||||
|
||||
|
||||
def fn_relocs(obj, fn):
|
||||
"""[symbol] — the relocation targets of `fn` in `obj`, in address order (objdump -dr)."""
|
||||
r = subprocess.run(["mipsel-linux-gnu-objdump", "-dr", "--no-show-raw-insn", str(obj)], capture_output=True, text=True)
|
||||
out, inside = [], False
|
||||
for ln in r.stdout.splitlines():
|
||||
m = re.match(r"^[0-9a-f]+ <([^>]+)>:$", ln)
|
||||
if m:
|
||||
inside = m.group(1) == fn
|
||||
continue
|
||||
if inside:
|
||||
m = re.search(r"\bR_MIPS_\w+\s+(\S+)", ln)
|
||||
if m:
|
||||
out.append(m.group(1).split("+")[0])
|
||||
return out
|
||||
|
||||
|
||||
def reloc_map(donor_rel, tu, fn):
|
||||
"""{donor symbol: target symbol} — the two ORIGINAL objects' relocation sequences for `fn` paired in order (the same
|
||||
function at the same address in two binaries: same instructions, per-binary data symbols). None when the sequences do
|
||||
not align (different lengths) or one donor symbol would map to two targets."""
|
||||
a, b = _obj_of(donor_rel), _obj_of(tu)
|
||||
if not a or not b:
|
||||
return None
|
||||
ra, rb = fn_relocs(a, fn), fn_relocs(b, fn)
|
||||
if not ra or len(ra) != len(rb):
|
||||
return None
|
||||
m = {}
|
||||
for x, y in zip(ra, rb):
|
||||
if not re.match(r"(?:D|g|func)_[0-9A-Fa-f]{8}$", x) or not re.match(r"(?:D|g|func)_[0-9A-Fa-f]{8}$", y):
|
||||
continue
|
||||
if m.setdefault(x, y) != y:
|
||||
return None
|
||||
return {x: y for x, y in m.items() if x != y}
|
||||
|
||||
|
||||
def _carry_decls(ported, m, donor_raw, target_raw):
|
||||
"""Body-local copies of the donor TU's file-scope `extern` declarations for every renamed symbol the target TU does not
|
||||
declare anywhere (a donor symbol declared at file scope is 'undeclared' in the target — d6's COMPILE-ERROR)."""
|
||||
add = []
|
||||
for x, y in m.items():
|
||||
if not y.startswith(("D_", "g_")) or re.search(r"\b%s\b" % re.escape(y), target_raw):
|
||||
continue
|
||||
if re.search(r"extern\b[^;]*\b%s\b" % re.escape(y), ported):
|
||||
continue
|
||||
dm = re.search(r"^extern\b[^;\n]*\b%s\b[^;\n]*;" % re.escape(x), donor_raw, re.M)
|
||||
if dm:
|
||||
add.append(" " + re.sub(r"\b%s\b" % re.escape(x), y, dm.group(0)))
|
||||
if not add:
|
||||
return ported
|
||||
i = ported.index("{") + 1
|
||||
return ported[:i] + "\n" + "\n".join(add) + ported[i:]
|
||||
|
||||
|
||||
def named_ports(tu, fn, max_donors=6):
|
||||
"""[(description, candidate text)] — R27: the SAME function already lever-free in another binary, ported with its
|
||||
symbols renamed onto this binary's.
|
||||
|
||||
T7 agents d2 (func_80166F58, ov_MAIN_012 ×6 from ov_SC04_011's shared header) and d6 (func_8017B614, ov_SC07_010 ×5
|
||||
from ov_SC01_000), P36 S104: both closed on their FIRST `--try` by porting a banked lever-free variant — the overlays
|
||||
carry one engine function at one address with per-overlay data symbols. The sweep had spent 1,699 compiles on d6's
|
||||
class without getting below 13. The port needs two repairs, both mechanical: (1) the data symbols renamed — pairing
|
||||
the two bodies' `extern` declaration lists by POSITION (d6's port.py) or their first occurrences in order; (2) the
|
||||
return type taken from the TARGET, whose TU declares the function again later (`extern void …` vs a donor's `s32`:
|
||||
"conflicting types", d2). Donors: every definition of `fn` in src/ with no `register`/`__asm__`/`!FAKE`, nearest line
|
||||
count first. Every candidate is judged on the bytes; a wrong pairing just does not score."""
|
||||
raw_t = (REPO / tu).read_text(errors="surrogateescape")
|
||||
target = _fn_text(raw_t, tu, fn)
|
||||
if not target:
|
||||
return []
|
||||
t_head = target[:target.index("{")] if "{" in target else ""
|
||||
t_ext = _uniq(re.findall(r"extern\b[^;]*?\b(%s)\b" % _SYM, target))
|
||||
t_occ = _uniq(re.findall(r"\b(%s)\b" % _SYM, target))
|
||||
donors, seen = [], set()
|
||||
for f, _n in named_definitions().get(fn, ()):
|
||||
if f == tu:
|
||||
continue
|
||||
try:
|
||||
body = _fn_text((REPO / f).read_text(errors="surrogateescape"), f, fn)
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
if not body or re.search(r"__asm__|\bregister\b|!FAKE", body):
|
||||
continue
|
||||
key = re.sub(r"\s+", " ", re.sub(r"\b%s\b" % _SYM, "D", body))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
donors.append((abs(body.count("\n") - target.count("\n")), f, body))
|
||||
out = []
|
||||
for _d, f, body in sorted(donors)[:max_donors]:
|
||||
d_ext = _uniq(re.findall(r"extern\b[^;]*?\b(%s)\b" % _SYM, body))
|
||||
d_occ = _uniq(re.findall(r"\b(%s)\b" % _SYM, body))
|
||||
maps = []
|
||||
rm = reloc_map(f, tu, fn)
|
||||
if rm is not None:
|
||||
maps.append(("reloc", rm))
|
||||
maps.append(("same", {}))
|
||||
for tag, a, b in (("extern-order", d_ext, t_ext), ("first-occurrence", d_occ, t_occ)):
|
||||
if a and len(a) == len(b):
|
||||
m = {x: y for x, y in zip(a, b) if x != y}
|
||||
if len(set(m.values())) == len(m):
|
||||
maps.append((tag, m))
|
||||
donor_raw = (REPO / f).read_text(errors="surrogateescape")
|
||||
for tag, m in maps:
|
||||
ported = re.sub(r"\b(?:D|g|func)_[0-9A-Fa-f]{8}\b", lambda x: m.get(x.group(0), x.group(0)), body) if m else body
|
||||
if m: # the target's own body is replaced, so its local declarations do not count
|
||||
ported = _carry_decls(ported, m, donor_raw, raw_t.replace(target, "", 1))
|
||||
if "{" not in ported:
|
||||
continue
|
||||
d_head = ported[:ported.index("{")]
|
||||
k = d_head.find(fn)
|
||||
variants = [("", ported)]
|
||||
tk = t_head.find(fn)
|
||||
if k >= 0 and tk >= 0 and d_head[:k] != t_head[:tk]:
|
||||
variants.append((" +target-return", t_head[:tk] + ported[k:]))
|
||||
if t_head and d_head != t_head:
|
||||
variants.append((" +target-signature", t_head + ported[ported.index("{"):]))
|
||||
for vtag, cand in variants:
|
||||
out.append((f"port {f.split('/')[-1]} {tag}{vtag}", cand))
|
||||
return out
|
||||
|
||||
|
||||
ALL_FAMILIES = ("R2", "R3", "R4", "R5", "R6", "R7", "R8", "R9", "R10", "R12", "R13", "R14", "R15", "R16", "R17", "R18", "R19", "R20", "R21", "R22", "R23", "R24", "R25", "R26", "R27")
|
||||
RUNG_R_FAMILIES = ("R2", "R3", "R4", "R5", "R6", "R7") # the free sweep's set (R8/R9 are the search engine's until measured)
|
||||
|
||||
|
||||
@@ -3237,6 +3413,9 @@ def recipe_candidates(text, tu, fn, names, limit=24, rng=None, cap=40, blocks=Tr
|
||||
if "R26" in fam:
|
||||
for desc, cand in alias_repeated_addresses(text, tu, fn, d_):
|
||||
out.append(("R26", desc, cand))
|
||||
if "R27" in fam and not tu.startswith("src/fx/") and (REPO / tu).exists(): # the named port needs the real TU
|
||||
for desc, cand in named_ports(tu, fn):
|
||||
out.append(("R27", desc, cand))
|
||||
if blocks and "R7" in fam: # last: one candidate per statement, so the targeted recipes go first
|
||||
for desc, cand in block_wraps(text, tu, fn, d_):
|
||||
out.append(("R7", desc, cand))
|
||||
|
||||
+11
-2
@@ -74,9 +74,16 @@ def one(e, fams, label):
|
||||
# the census's site lines no longer fit the file — the tree moved under the pass (S103: a bank during the R26 run
|
||||
# crashed the whole pool here). One class refused loudly, never the pass (R43); rerun after a census refresh.
|
||||
return dict(e, verdict="STALE-SITES", err=str(x)[:120], tried=0)
|
||||
for sname, body in st:
|
||||
# R27 (the named port) reads the REAL translation unit and the objects — it is not a rewrite of a start text, so it
|
||||
# runs once per class, first (S104: 4 of 24 donor classes closed at the first candidate, d2/d6's move made mechanical)
|
||||
srcs = ([("port", None)] if "R27" in fams else []) + st
|
||||
for sname, body in srcs:
|
||||
try:
|
||||
cands = dl.recipe_candidates(body, "src/fx/regen.c", e["fn"], [], cap=None, families=fams)
|
||||
if sname == "port":
|
||||
cands = [("R27", d, c) for d, c in dl.named_ports(e["tu"], e["fn"])]
|
||||
else:
|
||||
cands = dl.recipe_candidates(body, "src/fx/regen.c", e["fn"], [], cap=None,
|
||||
families=tuple(f for f in fams if f != "R27"))
|
||||
except Exception as x: # a generator crash is a finding, not a silent skip (R43)
|
||||
return dict(e, verdict="GEN-ERROR", err=str(x)[:160], tried=tried)
|
||||
for rec, desc, cand in cands:
|
||||
@@ -116,6 +123,8 @@ def run(a):
|
||||
ds.sites_by_body()
|
||||
if "R19" in fams:
|
||||
dl.real_signatures()
|
||||
if "R27" in fams:
|
||||
dl.named_definitions()
|
||||
with cf.ProcessPoolExecutor(max_workers=a.jobs, mp_context=multiprocessing.get_context("fork")) as pool:
|
||||
futs = {pool.submit(one, e, fams, label): e for e in ex}
|
||||
# one line per judged class AS IT LANDS (R55): the agent lane draws only classes this pass has already judged
|
||||
|
||||
Reference in New Issue
Block a user