phase-36: T6 — the kit corpus and the tool index regenerated for delever_permute.py (tool_census --check: 0 gaps)

This commit is contained in:
Drew T
2026-09-09 10:23:35 -06:00
parent 6e1677b4da
commit 22a31bc025
8 changed files with 853 additions and 36 deletions
+3 -2
View File
@@ -8,8 +8,8 @@
> its platform SDK need the marked adaptation. The last table lists the tools that are project-only in code (their *shape* is a task;
> their code does not transfer). *TODO(platform): the MIPS and PlayStation SDK hard-codes are the ones another platform replaces first.*
>
> **Coverage:** 300 tool files in scope (submodules, vendored and downloaded code excluded), of which 300 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 26 · P6 51 · P7 20 · P8 86 · P9 27 · P10 20 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (30 pointer rows); one-offs are omitted. Table rows in all: 330 (the installer checks its copy against this figure).
> **Coverage:** 301 tool files in scope (submodules, vendored and downloaded code excluded), of which 301 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 26 · P6 51 · P7 20 · P8 86 · P9 27 · P10 21 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (30 pointer rows); one-offs are omitted. Table rows in all: 331 (the installer checks its copy against this figure).
## P1 — extraction + manifest
@@ -341,6 +341,7 @@
| `verbatim_target_s.py` | regenerate a splitter-format target disassembly for a function no longer stubbed | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | repo build/asm layout |
| `delever_cycle.sh` | run the de-lever batch cycle unattended: batch, verify by exit code, clean fleet gate, census, log, commit | The Phase-36 T3/T4 driver: per batch a clean-tree check, the oracle recalibrated when stale, delever --apply read by exit code and its final X/X line, the clean fleet run as the outer gate (218/218), the census rerun, the phase-log entry and the checkpoint headline, one commit per batch; stops on the first red with the files in place for --restore | the batch size, the aliases, the fleet count |
| `share_body_cycle.sh` | run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check | The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed the moment it is green, the clean fleet run every N batches — stops on the first red | repo paths, the phase-log format, the fleet count from config/check.*.sha |
| `delever_permute.py` | search the compiler's own shape space for a lever-free body that keeps the bytes | The Phase-36 rung D: one exemplar per residue text class from the delever ledger (largest class first — a match banks every copy), each prepared as a single-function translation unit (delever's rung-A rewrite of every removable site, every other definition reduced to a prototype, shared-header includes to their prototypes, INCLUDE_ASM and file-scope asm dropped, the build's own CPPFLAGS through cpp -P) against a target regenerated from the ROM image in both the assemblable and the splat form; every attempt calibrates itself first (the LEVERED body must be MATCH, or the harness is not measuring that function) and records the lever-free body's starting distance; decomp-permuter through permuter_ils with the weight profile chosen from the NEEDED kinds; a score-0 winner is banked only through delever --apply-body and the GTE re-fold; scratch, winners and the outcome ledger keyed by alias+fn | the ledger, the permuter harness, the target regenerator |
| `share_body.py` | share one byte-identical function class across its binaries through an include-at-site header, gated per binary | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | repo paths, the registry and signature schemas |
| `delever.py` | take register pins, asm statements, volatile and register levers out of matched C, byte-gated per site | The Phase-36 de-lever engine and campaign tool: positional rewrites per lever class (pin -> plain declaration, barrier/keep-alive deleted, a launder deleted or turned into the assignment it is, a hand-placed instruction -> its C, the zero-register variable -> 0, a macro-carried site deleted/valued/refused by its macro's shape, volatile/register dropped); per body replay -> rung A strip-all -> rung B greedy through delever_oracle; the file is the write unit and its final compile through every recipe the proof; --plan/--apply batches (TUs parallel, exemplar files first; headers serial, includers parallel), the ledger keyed by tu+fn+addr with the body's nhash before/after, !FAKE markers on class A/B survivors, --redraw for bodies refused by an older tool, --restore from inflight.json, --scrub for orphan markers, --apply-body for T6/T7, --selftest with a stub oracle, --probe (T2) | the census site records, the oracle |
| `verbatim_to_stub.py` | turn an inline-assembly body back into a stub the toolchain can reach | Turns an inline-assembly body back into an include-assembly stub | repo src/asm layout |
@@ -75,6 +75,7 @@ INFLIGHT = RUN / "inflight.json"
PROBE = REPO / ".run" / "P36" / "probe"
PRELUDE = "src/shared/engine_prelude.h"
REMOVABLE = {("A", "pin"), ("B", "barrier"), ("B", "launder"), ("B", "keepalive"), ("B", "instruction"),
("B", "gte-lever"),
("C", "cast"), ("C", "decl-body"), ("C", "param"), ("D", "register")}
FILE_SCOPE_REMOVABLE = {("C", "decl-file"), ("B", "barrier"), ("B", "launder"), ("B", "keepalive"), ("B", "instruction")}
# a file-scope asm statement is a TU-level site: a barrier/launder/keep-alive is judged like any other; a `.section` block is a rodata
@@ -343,6 +344,52 @@ def macro_shape(raw, rel, name, use_line):
return res
_GTE_CANON = None
_gte_variant_cache = {}
def gte_variant_target(raw, rel, name, use_line):
"""The canonical macro name a lever-variant macro use should point at, from the SIGNATURE of the variant's own
governing `#define` (never from its spelling: T5 named `gte_rt_m` after Sony's `gte_rt` while that signature's
canonical name is `gte_rt_alt`). None when there is no single canonical macro for it."""
key = (rel, name, use_line)
if key in _gte_variant_cache:
return _gte_variant_cache[key]
import gte_consolidate as gc # lazy: gte_consolidate imports THIS module
gte_canonical_clob("") # loads the table
body = None
for (l0, l1, n, b) in lc.define_blocks(raw):
if n == name and l1 < use_line:
body = b
inner = lc._macro_asm_inner(body) if body is not None else None
names = None
if inner is not None:
names, _ = gc.canonical_match(gc.signature(inner), _GTE_CANON)
res = names[0] if names and len(names) == 1 else None
_gte_variant_cache[key] = res
return res
def gte_canonical_clob(inner):
"""The canonical clobber list for a GTE asm statement's inner text, or None when it has none to take
(no table, unsigned template, no canonical entry, or it already carries the canonical set)."""
global _GTE_CANON
if _GTE_CANON is None:
import gte_consolidate as gc # lazy: gte_consolidate imports THIS module
_GTE_CANON = json.loads(gc.CANON.read_text()) if gc.CANON.exists() else {}
canon = _GTE_CANON.get("canonical") or {}
if not canon:
return None
import gte_consolidate as gc
sg = gc.signature(inner)
names, clob = gc.canonical_match(sg, _GTE_CANON)
if names is None or tuple(sg["clob"]) == tuple(clob):
return None
return clob
def site_edits(raw, m, ls, site, keep_register=False):
"""[(start, end, replacement)] for one site, or raise Refuse. `m` = same_len_mask(raw)."""
pos = ls[site["line"] - 1] + site["col"] - 1
@@ -381,6 +428,42 @@ def site_edits(raw, m, ls, site, keep_register=False):
return [(pos, consume_marker(raw, m, e), new)]
if cls == "B" and kind in DEFERRED_KINDS:
raise Refuse("asm-body: a whole routine in a C shell is T7's work (DEFERRED)")
if cls == "B" and kind == lc.GTE_LEVER_KIND:
# A GTE op whose clobber list exceeds its canonical signature's is a SCHEDULING STEER wearing Sony's
# coprocessor idiom (T5 named them and marked them; it judged only the macro DEFINITIONS, so a DIRECT
# statement's extra clobbers were never offered to the ladder). The rewrite is the canonical clobber
# set for that signature — not deletion: the op itself is real code. Refused when the tree has no
# canonical table, when the statement does not sign, when its signature is not canonical, or when it
# already carries the canonical set (then it is not a lever and the census is wrong about it, R43).
# The survivor keeps T5's own richer `// !FAKE:` text: gte-lever is deliberately NOT in MARK_KINDS.
name = site.get("via")
if name:
# a use of a LEVER VARIANT macro (`gte_x_m` / `gte_x_v<hash>`, kept per TU by T5): the lever is the variant's
# extra clobbers, so the rewrite points the use at the canonical macro of include/gte_inline.h. The variant's
# own `#define` is left dead for `gte_consolidate.py --sweep` (its marker for `--scrub`).
if not m.startswith(name, pos):
raise Refuse(f"token mismatch at {site['tu']}:{site['line']}: expected `{name}`")
base = gte_variant_target(raw, site["tu"], name, site["line"])
if base is None:
raise Refuse(f"GTE lever `{name}`: no single canonical macro for its signature")
if base == name:
raise Refuse(f"GTE lever `{name}` already IS its canonical macro")
return [(pos, pos + len(name), base)]
if not ASM_HEAD.match(m, pos):
raise Refuse(f"token mismatch at {site['tu']}:{site['line']}: expected an asm statement")
o = m.find("(", pos)
c = lc._paren_span(m, o)
e = stmt_end(m, pos)
if o < 0 or c < 0 or e < 0:
raise Refuse("unterminated GTE asm statement")
clob = gte_canonical_clob(m[o + 1:c]) # the MASKED inner: T5's `// !FAKE:` sits INSIDE the parens
if clob is None:
raise Refuse(f"GTE lever `{site.get('detail', '')}`: no canonical signature to take the clobbers from")
import gte_consolidate as gc # lazy: gte_consolidate imports THIS module
# the statement without its comments (the marker included — a de-levered site owns no honesty marker); set_clobbers
# parses the sections positionally and a comment between `(` and the template would derail it
clean = "".join(raw[i] for i in range(pos, e) if m[i] == raw[i])
return [(pos, consume_marker(raw, m, e), gc.set_clobbers(clean, clob))]
if cls == "B" and site.get("via"):
name = site["via"]
if not m.startswith(name, pos):
@@ -0,0 +1,714 @@
#!/usr/bin/env python3
"""delever_permute.py — rung D of the lever ladder: decomp-permuter on the RESIDUE exemplars (Phase 36 T6).
tools/delever_permute.py --plan [--limit N] [--only X ...] # the exemplar list from the ledger (no writes)
tools/delever_permute.py --prepare TU FN # one exemplar's draft.c + target .s, no permuter
tools/delever_permute.py --run [--limit K] [--workers W] [--secs S] [--cycles C] [-j J] [--only X ...]
tools/delever_permute.py --bank [--label dN] [--only X ...] [--dirty-ok]
tools/delever_permute.py --status
tools/delever_permute.py --selftest
WHY A PERMUTER RUNG AT ALL. Rungs A–C (tools/delever.py) asked "does the object stay identical without this site?" and left
12,970 bodies where the answer was no for at least one site. Those sites are not magic: each is a shape the compiler would have
produced anyway from DIFFERENT C. The permuter's randomizer moves exactly those shapes (temps, statement and declaration order,
scoping, operand order), so the residue is its natural population — and the seed is the LEVER-FREE body, which is what makes this
different from every earlier permuter run in this project: the search starts from C that has no pins and no asm at all, and any
score-0 it reaches is by construction a lever-free match.
THE PIPELINE PER EXEMPLAR (probe-proven at S98, the T6 log entry):
1. the lever-free TU text — delever's own rung-A rewrite of every REMOVABLE site of the body (a site it REFUSES makes the
exemplar UNSTRIPPABLE: recorded, skipped — never a silently narrowed scope, R43/silently-narrowed-tool-scope).
2. the ISOLATED TU — every OTHER definition replaced by its prototype, every `#include "…/shared/…/func_*.h"` replaced
by the prototypes that header defines, INCLUDE_ASM/INCLUDE_RODATA lines and file-scope asm statements dropped: the object
the permuter compiles must hold ONE .text function, because the masked scorer compares whole .text.
3. cpp -P with the BUILD's own CPPFLAGS + -I<the TU's directory> (common.h, the prelude, engine_types.h and the GTE header
expand here — p16_permute.make_base_c runs its own cpp WITHOUT includes and would lose them) -> draft.c.
4. the target .s — tools/verbatim_target_s.py regenerates it from the EXTRACTED ROM IMAGE (R34: an independent
oracle; our own source is the thing under test), into the exemplar's scratch dir, where p16_permute.setup reads it.
5. tools/permuter_ils.py — warm-restarting decomp-permuter, the weight profile chosen from the NEEDED sites' kinds
(pins -> regalloc, barriers/launders/keep-alives -> schedule, mixed -> regalloc).
6. a score-0 winner is a CANDIDATE, never a bank: --bank puts the function's definition back through
`delever.py --apply-body` (which refuses a body that still carries a class A/B lever and judges it on the object's bytes
through every recipe of the TU), then re-folds the cpp-expanded GTE asm with `gte_consolidate.py --apply --rejudge`.
The clean fleet run (R22) gates the batch, as it gates every batch of this phase.
SCRATCH IS KEYED BY ALIAS+FN (R48): `.run/P36/permuter/<alias>__<fn>/` — a function NAME is not unique across the fleet (the
overlays overlap in RAM, so two different bodies can both be `func_8013B274`), and the permuter's own scratch/winner paths are
keyed by the bare name. Every attempt is recorded in `.run/P36/permuter/outcomes.jsonl`, which is also the skip list.
NEVER RUN THE CAMPAIGN AS A HARNESS BACKGROUND TASK (the low-memory guard kills it): `setsid nohup … &` + a Monitor on the log.
"""
import argparse
import collections
import json
import os
import pathlib
import re
import shutil
import subprocess
import sys
import threading
import time
from concurrent.futures import ThreadPoolExecutor
REPO = pathlib.Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "tools"))
import delever as dl # noqa: E402
import delever_oracle as oracle # noqa: E402
import lever_census as lc # noqa: E402
import p16_permute as p16 # noqa: E402
import share_census as sc # noqa: E402
RUN = REPO / ".run" / "P36" / "permuter"
OUTCOMES = RUN / "outcomes.jsonl"
PY = str(REPO / ".venv" / "bin" / "python")
CPP = "mipsel-linux-gnu-cpp"
# the Makefile's own CPPFLAGS (line 1010) — the draft must see the same macro world the build sees
CPPFLAGS = ["-lang-c", "-Iinclude", "-undef", "-Wall", "-fno-builtin", "-Dmips", "-D__GNUC__=2",
"-D__OPTIMIZE__", "-Dpsx", "-D_PSYQ", "-D_MIPSEL", "-D_LANGUAGE_C"]
# decomp-permuter parses base.c with pycparser, which rejects `__attribute__((packed, aligned(1)))` outright
# ("Syntax error in base.c" -> the run REFUSES and permutes nothing). The draft therefore defines the keyword away.
# This is a fidelity LOSS, not a correctness one: a packed struct the exemplar actually uses will lay out differently
# in the search, so that exemplar simply cannot reach score 0 — and the bank gate is the real object oracle either
# way. The count is recorded per attempt (`attrs`) so the yield report can say whether it mattered (76 sites in the
# whole tree at T5; the class belongs to the types phase, R95).
CPP_ATTR = "-D__attribute__(x)="
INCLUDE_ASM_LINE = re.compile(r"^[ \t]*INCLUDE_(?:ASM|RODATA)\b[^\n]*\n", re.M)
INCLUDE_LINE = re.compile(r'^[ \t]*#[ \t]*include[ \t]*"([^"]+)"[^\n]*\n', re.M)
SCHED_KINDS = {"barrier", "launder", "keepalive"}
_LOCK = threading.Lock()
class Unstrippable(Exception):
"""the body cannot be made lever-free by the rewrite table — the exemplar is T7's, not rung D's."""
# ----------------------------------------------------------------------------------------------------------------------
# the population: the residue's classes, one exemplar each
# ----------------------------------------------------------------------------------------------------------------------
def load_outcomes():
rows = []
if OUTCOMES.exists():
for l in OUTCOMES.read_text().splitlines():
if l.strip():
rows.append(json.loads(l))
return rows
def outcome_append(row):
RUN.mkdir(parents=True, exist_ok=True)
with _LOCK, open(OUTCOMES, "a") as f:
f.write(json.dumps(row) + "\n")
def alias_of(tu, row=None):
"""the binary whose object holds this TU (a shared header: its first includer's). The ledger row's own aliases first."""
al = (row or {}).get("aliases")
if al:
return al[0]
by_src = oracle.recipes_by_src(oracle.load_recipes()["recipes"])
src = tu
if tu.endswith(".h"):
inc = dl.includers().get(tu) or []
src = next((t for t in inc if t in by_src), None)
if src is None:
return None
recs = by_src.get(src) or []
return recs[0]["alias"] if recs else None
def matches(ex, only):
if not only:
return True
return any(o == ex["fn"] or o == ex["tu"] or o in ex["tu"] or ex["nhash"].startswith(o) or o == ex["alias"] for o in only)
def exemplars(only=(), limit=None, include_done=False):
"""One exemplar per RESIDUE text class, largest class first (a match banks every copy), then fewest NEEDED sites."""
cur, order = {}, []
for r in dl.load_ledger():
k = (r.get("tu"), r.get("fn"))
if not k[0] or not k[1]:
continue
if k not in cur:
order.append(k)
cur[k] = r
groups = collections.OrderedDict()
for k in order:
r = cur[k]
if r.get("verdict") != "RESIDUE" or r["fn"] == dl.FILE_SCOPE_FN:
continue # a TU's file-scope `volatile` pseudo-body is not a function to permute
groups.setdefault(r.get("nhash_after") or r.get("nhash_before"), []).append((k, r))
done = set() if include_done else {o["nhash"] for o in load_outcomes() if o.get("kind") == "attempt"}
out = []
for nh, members in groups.items():
if nh in done:
continue
(tu, fn), r = members[0]
needed = [s for s in r.get("sites", []) if s.get("verdict") == "NEEDED"]
out.append(dict(nhash=nh, tu=tu, fn=fn, alias=(r.get("aliases") or [None])[0], copies=len(members),
needed=len(needed), kinds=sorted({s["kind"] for s in needed}),
members=[dict(tu=t, fn=f) for (t, f), _ in members], row=r))
out.sort(key=lambda e: (-e["copies"], e["needed"], e["tu"], e["fn"]))
out = [e for e in out if matches(e, only)]
return out[:limit] if limit else out
def klass_for(kinds):
"""the permuter_weights profile for a residue's NEEDED-site mix (the profile NAME is accepted verbatim by classify())."""
ks = set(kinds)
if ks and ks <= SCHED_KINDS:
return "schedule"
return "regalloc"
# ----------------------------------------------------------------------------------------------------------------------
# step 1-3: the lever-free, isolated, preprocessed draft
# ----------------------------------------------------------------------------------------------------------------------
_sites_cache = None
def tu_sites(tu):
global _sites_cache
if _sites_cache is None:
_sites_cache = collections.defaultdict(list)
for s in dl.load_sites():
_sites_cache[s["tu"]].append(s)
return _sites_cache[tu]
def body_sites(tu, fn):
return [s for s in tu_sites(tu) if s.get("fn") == fn]
def file_asm_sites(tu):
"""the TU's file-scope asm STATEMENTS (13 in the fleet: 7 rodata carves, 6 barriers) — asked of the census, never
re-parsed: a bare `__asm__(` scan cannot tell a statement from the asm-LABEL clause of a declaration
(`extern void func_8005C324(…) __asm__("memcpy");` — 6,423 of those), and eating one leaves a headless K&R body."""
return [s for s in tu_sites(tu) if s.get("scope") == "file" and s["cls"] == "B"
and s["kind"] not in lc.NON_LEVER_KINDS and s["kind"] != "asm-label"]
def lever_free_text(tu, raw, fn, sites, strip=True):
"""the TU's text with every REMOVABLE site of THIS body rewritten away (delever's own table) and its file-scope asm
statements deleted (the draft compares .text only); Unstrippable if any body site refuses.
strip=False keeps every lever — that is the CALIBRATION seed: the body as the tree has it, which must score MATCH
against the regenerated target or the harness (target, isolation, cpp, compile) is not measuring this function."""
m = dl.same_len_mask(raw)
ls = dl.line_starts(raw)
edits, refused = [], []
for s in file_asm_sites(tu):
pos = ls[s["line"] - 1] + s["col"] - 1
e = dl.stmt_end(m, pos)
if dl.ASM_HEAD.match(m, pos) and e > 0:
edits.append((pos, dl.consume_marker(raw, m, e), ""))
for s in (sites if strip else []):
if (s["cls"], s["kind"]) not in dl.REMOVABLE:
if s["kind"] in dl.DEFERRED_KINDS:
refused.append((s["kind"], s["line"], "asm-body: T7's work"))
continue
try:
edits += dl.site_edits(raw, m, ls, s)
except dl.Refuse as ex:
refused.append((s["kind"], s["line"], str(ex)[:120]))
if refused:
raise Unstrippable(refused)
if not edits:
return raw
try:
return dl.apply_edits(raw, edits)
except dl.Refuse as ex:
raise Unstrippable([("<combination>", 0, str(ex)[:120])])
def prototype_at(text, m, start):
"""`<head>;` for the definition that starts at index `start` — the head up to the last `)` before its opening `{`."""
o = m.find("{", start)
if o < 0:
return None
close = m.rfind(")", start, o)
if close < 0:
return None
head = " ".join(text[start:close + 1].split())
return head + ";"
def header_prototypes(path):
"""the prototypes of every function a shared header DEFINES (a static one keeps `static`)."""
p = REPO / path
if not p.exists():
return []
text = p.read_text(errors="surrogateescape")
m = dl.same_len_mask(text)
ls = dl.line_starts(text)
out = []
for r in sc.scan_text(text, path, shared_defs=None):
if r["form"] != "def":
continue
pr = prototype_at(text, m, ls[r["line"] - 1])
if pr:
out.append(pr)
return out
def isolate(tu, text, fn):
"""every OTHER definition -> its prototype; shared-header includes -> their prototypes; INCLUDE_ASM/file-scope asm dropped."""
m = dl.same_len_mask(text)
ls = dl.line_starts(text)
recs = sc.scan_text(text, tu, shared_defs=None)
defs = [r for r in recs if r["form"] == "def"]
if not any(r["name"] == fn for r in defs):
raise Unstrippable([("<isolate>", 0, f"{fn} is not defined in {tu}")])
edits = []
for r in defs:
s, e = ls[r["line"] - 1], ls[r["end"]]
if r["name"] == fn:
continue
pr = prototype_at(text, m, s)
if pr is None:
raise Unstrippable([("<isolate>", r["line"], f"no prototype for {r['name']}")])
edits.append((s, e, pr + "\n"))
for mm in INCLUDE_ASM_LINE.finditer(m):
edits.append((mm.start(), mm.end(), ""))
for mm in INCLUDE_LINE.finditer(m):
inc = mm.group(1)
rel = os.path.normpath(os.path.join(os.path.dirname(tu), inc))
if not rel.startswith("src/shared/"):
continue
pr = header_prototypes(rel)
if pr: # a macro-only header (the prelude) has none: keep the include
edits.append((mm.start(), mm.end(), "\n".join(pr) + "\n"))
return dl.apply_edits(text, sorted(edits, key=lambda x: x[0]))
def drop_file_scope_asm(text, tu):
"""delete every asm STATEMENT that is not inside a function — after cpp, `include/include_asm.h` has injected
`__asm__(".include \\"include/labels.inc\\"");` at the top of every draft, and the permuter's own compile.sh
already prepends `.include "macro.inc"`, so the assembler dies on `Macro 'glabel' was already defined`.
An asm-LABEL clause (`extern void f(…) __asm__("memcpy");`, 6,423 of them) is NOT a statement: it follows a
declarator, so the preceding non-space character is not `;` or `}`."""
m = dl.same_len_mask(text)
ls = dl.line_starts(text)
spans = [(ls[r["line"] - 1], ls[r["end"]]) for r in sc.scan_text(text, tu, shared_defs=None) if r["form"] == "def"]
edits = []
for mm in dl.ASM_HEAD.finditer(m):
p = mm.start()
if any(s <= p < e for s, e in spans):
continue
prev = m[:p].rstrip()
if prev and prev[-1] not in ";}":
continue # an asm-label clause on a declaration
e = dl.stmt_end(m, p)
if e > 0:
edits.append((p, e, ""))
return dl.apply_edits(text, edits) if edits else text
def cpp_expand(tu, text):
"""the build's cpp over the isolated text (fed on stdin, so `-I<the TU's directory>` is what resolves its relative includes)."""
p = subprocess.run([CPP, "-P"] + CPPFLAGS + [CPP_ATTR, "-I" + os.path.dirname(tu), "-"],
input=text, capture_output=True, text=True, cwd=REPO)
if p.returncode or not p.stdout.strip():
raise Unstrippable([("<cpp>", 0, (p.stderr.strip().splitlines() or ["empty output"])[-1][:160])])
return p.stdout
def scratch_of(alias, fn):
return RUN / f"{alias}__{fn}"
def prepare(ex, quiet=False):
"""(scratch dir, draft path, target path, klass) for one exemplar — the whole probe-2 pipeline."""
tu, fn = ex["tu"], ex["fn"]
alias = ex["alias"] or alias_of(tu, ex.get("row"))
if not alias:
raise Unstrippable([("<alias>", 0, f"no binary compiles {tu}")])
ex["alias"] = alias
raw = (REPO / tu).read_text(errors="surrogateescape")
sites = body_sites(tu, fn)
if not sites:
raise Unstrippable([("<census>", 0, f"the census has no site in {tu}:{fn} (stale? rerun lever_census --sites)")])
d = scratch_of(alias, fn)
if d.exists():
shutil.rmtree(d)
(d / "gas").mkdir(parents=True)
(d / "splat").mkdir()
for strip, name in ((True, "draft.c"), (False, "levered.c")):
free = lever_free_text(tu, raw, fn, sites, strip=strip)
iso = isolate(tu, free, fn)
text = drop_file_scope_asm(cpp_expand(tu, iso), tu)
if strip:
(d / "tu.c").write_text(free, errors="surrogateescape") # the lever-free TU (what rung A would have written)
(d / "iso.c").write_text(iso, errors="surrogateescape") # + one body, the rest prototypes (pre-cpp; for reading)
(d / name).write_text(text, errors="surrogateescape")
if strip:
base = p16.make_base_c(text)
if not p16.defines_fn(base, fn):
raise Unstrippable([("<base>", 0, f"base.c would lose the definition of {fn} (inspect {d}/draft.c)")])
# TWO target forms, for the two instruments. --gas is the ASSEMBLABLE one ($-registers, .L labels, symbol names) that
# `p16_permute.setup` turns into target.o; the default splat listing is the WORD oracle `match_one.py` reads. Nothing
# ever assembles the splat form — pasting it into decomp.me was P34's `invalid operands 'li a2,2'` (R98/DK-81), and
# `mipsel-as` refuses its bare `addiu sp,sp,-152` the same way.
for args, sub, name in (( ["--gas"], "gas", f"{fn}.gas.s"), ([], "splat", f"{fn}.s")):
r = subprocess.run([PY, "tools/verbatim_target_s.py", "--binary", alias, "--fn", fn] + args
+ ["--out", str(d / "tgt")], capture_output=True, text=True, cwd=REPO)
tgt = d / "tgt" / alias / name
if r.returncode or not tgt.exists():
raise Unstrippable([("<target>", 0, ((r.stderr or r.stdout).strip().splitlines() or ["no listing"])[-1][:160])])
shutil.copy(tgt, d / sub / f"{fn}.s") # p16_permute.setup / match_one.py read <dir>/<fn>.s
if not quiet:
n = len((d / "draft.c").read_text(errors="surrogateescape").splitlines())
print(f" prepared {alias}__{fn}: draft {n} lines, target {(r.stdout or '').strip().split()[-3:]}", flush=True)
return d, d / "draft.c", d / "gas" / f"{fn}.s", klass_for(ex["kinds"])
# ----------------------------------------------------------------------------------------------------------------------
# the calibration (R39/R56): the harness must agree with the tree before it may judge a candidate
# ----------------------------------------------------------------------------------------------------------------------
MISMATCH_RE = re.compile(r"(\d+) mismatched")
def closeness(d, fn, c):
"""(mismatched instructions, first line) from tools/match_one.py against the SPLAT listing — 0 == byte-identical
(relocation-masked). None when the tool could not judge (R61: not judged is not a verdict)."""
r = subprocess.run([PY, "tools/match_one.py", fn, "--c", str(c), "--asm-subdir", (d / "splat").as_posix()],
capture_output=True, text=True, cwd=REPO)
first = ((r.stdout or "") + (r.stderr or "")).strip().splitlines()
first = first[0] if first else "(no output)"
if "MATCH" in first and "mismatched" not in first:
return 0, first
mm = MISMATCH_RE.search(first)
return (int(mm.group(1)) if mm else None), first
def calibrate_one(ex, quiet=False):
"""the control: the body AS THE TREE HAS IT (levers and all) must be MATCH against the regenerated target. It proves the
whole chain — the target from the ROM image, the isolation, the cpp expansion, the pinned triple — is measuring THIS
function. Then the lever-free draft's distance is the search's real starting point, and it is worth recording (R37)."""
d, draft, tgt, klass = prepare(ex, quiet=quiet)
lev, lev_line = closeness(d, ex["fn"], d / "levered.c")
free, free_line = closeness(d, ex["fn"], draft)
ok = (lev == 0)
if not quiet:
print(f" {'OK ' if ok else 'FAIL'} {ex['alias']}__{ex['fn']}: levered {lev_line[:60]} · lever-free {free_line[:60]}",
flush=True)
return dict(ok=ok, levered=lev, free=free, levered_line=lev_line, free_line=free_line, klass=klass, d=d)
# ----------------------------------------------------------------------------------------------------------------------
# step 5: the permuter
# ----------------------------------------------------------------------------------------------------------------------
BEST_RE = re.compile(r"best score = (\d+)")
def run_one(ex, secs, cycles, j, max_start=None):
t0 = time.time()
row = dict(kind="attempt", ts=time.strftime("%Y-%m-%d %H:%M:%S"), nhash=ex["nhash"], tu=ex["tu"], fn=ex["fn"],
alias=ex["alias"], copies=ex["copies"], needed=ex["needed"], kinds=ex["kinds"],
secs=secs, cycles=cycles, j=j)
try:
d, draft, tgt, klass = prepare(ex)
except Unstrippable as ex_:
row.update(verdict="UNSTRIPPABLE", why=ex_.args[0][:6], seconds=round(time.time() - t0, 1))
outcome_append(row)
print(f" {ex['alias'] or '?'}__{ex['fn']}: UNSTRIPPABLE — {ex_.args[0][:2]}", flush=True)
return row
# EVERY attempt calibrates itself first (R56): the body as the tree has it must be MATCH against the regenerated
# target, or this harness is not measuring this function and its budget would buy a meaningless number. ~15 s
# against ~12 min of search.
lev, lev_line = closeness(d, ex["fn"], d / "levered.c")
free, free_line = closeness(d, ex["fn"], draft)
row.update(levered=lev, free=free, levered_line=lev_line[:120], free_line=free_line[:120])
if lev != 0:
row.update(verdict="UNCALIBRATED", seconds=round(time.time() - t0, 1))
outcome_append(row)
print(f" {ex['alias']}__{ex['fn']}: UNCALIBRATED — the LEVERED body is not MATCH ({lev_line[:80]})", flush=True)
return row
if max_start is not None and (free is None or free > max_start):
# TRIAGE, not a verdict about the function: a lever-free body this far from the target (usually because removing a
# hand-placed instruction changed the instruction COUNT and shifted everything after it) will not close in a
# 12-minute search. Recorded with its distance; `--include-done` draws it again when the budget or the rung grows.
row.update(verdict="FAR", seconds=round(time.time() - t0, 1))
outcome_append(row)
print(f" {ex['alias']}__{ex['fn']}: FAR start={free} > {max_start} — not searched ({ex['copies']} copies)", flush=True)
return row
rel = d.relative_to(REPO).as_posix()
log = d / "ils.log"
cmd = [PY, "tools/permuter_ils.py", ex["fn"], "--draft", (draft.relative_to(REPO)).as_posix(),
"--asm-subdir", rel + "/gas", "--klass", klass, "--cycles", str(cycles), "--secs", str(secs),
"--j", str(j), "--winners", rel, "--pd", rel + "/pd"]
row["klass"] = klass
row["cmd"] = " ".join(cmd)
rc = None
with open(log, "w") as f:
f.write(" ".join(cmd) + "\n\n")
f.flush()
try:
rc = subprocess.run(cmd, cwd=REPO, stdout=f, stderr=subprocess.STDOUT,
timeout=cycles * (secs + 45) + 300).returncode
except subprocess.TimeoutExpired:
f.write("\n[delever_permute] the ILS wrapper overran its own budget — killed\n")
out = log.read_text(errors="replace")
scores = [int(x) for x in BEST_RE.findall(out)]
win = d / f"{ex['fn']}.c"
# NOT JUDGED IS NOT A VERDICT (R61): a setup failure, a parser refusal or a wrapper that never iterated is its own
# outcome — reporting it as NO-MATCH is how a tool reports work it never did.
if win.exists():
verdict = "MATCH"
elif "setup FAILED" in out:
verdict = "SETUP-FAILED"
elif "ABORTED" in out or "REFUSED" in out:
verdict = "ABORTED"
elif "cycle 1:" not in out:
verdict = "NOT-JUDGED"
else:
verdict = "NO-MATCH"
row.update(best=(min(scores) if scores else None), seconds=round(time.time() - t0, 1), rc=rc,
winner=(win.relative_to(REPO).as_posix() if win.exists() else None), verdict=verdict)
outcome_append(row)
print(f" {ex['alias']}__{ex['fn']}: {row['verdict']} start={row['free']} best={row['best']} "
f"({row['seconds']}s, {ex['copies']} copies, {ex['needed']} needed, {klass})", flush=True)
return row
# ----------------------------------------------------------------------------------------------------------------------
# step 6: the bank (serial — every step here runs `make`)
# ----------------------------------------------------------------------------------------------------------------------
def winner_body(winner_c, fn, tu):
"""the winner's definition of fn, as the text `delever --apply-body` splices in."""
draft = p16.winner_to_draft(pathlib.Path(winner_c).read_text(errors="surrogateescape"))
ls = dl.line_starts(draft)
for r in sc.scan_text(draft, tu, shared_defs=None):
if r["form"] == "def" and r["name"] == fn:
return draft[ls[r["line"] - 1]:ls[r["end"]]]
return None
def bank(a):
outs = load_outcomes()
banked = {(o["alias"], o["fn"], o["nhash"]) for o in outs if o.get("kind") == "bank" and o.get("applied")}
todo = [o for o in outs if o.get("kind") == "attempt" and o.get("verdict") == "MATCH"
and (o["alias"], o["fn"], o["nhash"]) not in banked
and (not a.only or any(x in (o["fn"], o["tu"], o["alias"]) or o["nhash"].startswith(x) for x in a.only))]
if not todo:
print("delever_permute --bank: no unbanked winner (R68: nothing to do, loudly)")
return 1
ok, bad = 0, 0
for o in todo:
d = scratch_of(o["alias"], o["fn"])
body = winner_body(REPO / o["winner"], o["fn"], o["tu"]) if o.get("winner") else None
if not body:
print(f" {o['alias']}__{o['fn']}: the winner holds no definition of {o['fn']} — SKIPPED", flush=True)
bad += 1
continue
bf = d / "body.c"
bf.write_text(body, errors="surrogateescape")
cmd = [PY, "tools/delever.py", "--apply-body", o["tu"], o["fn"], bf.relative_to(REPO).as_posix(),
"--label", a.label, "--rung", "D"] + (["--dirty-ok"] if a.dirty_ok else [])
r = subprocess.run(cmd, cwd=REPO, capture_output=True, text=True)
print(" " + (r.stdout or r.stderr).strip().splitlines()[-1][:220], flush=True)
row = dict(kind="bank", ts=time.strftime("%Y-%m-%d %H:%M:%S"), nhash=o["nhash"], tu=o["tu"], fn=o["fn"],
alias=o["alias"], label=a.label, applied=(r.returncode == 0),
apply_out=(r.stdout or r.stderr).strip()[-400:])
if r.returncode == 0:
g = subprocess.run([PY, "tools/gte_consolidate.py", "--apply", "--only", o["tu"], "--rejudge",
"--label", a.label + "g"], cwd=REPO, capture_output=True, text=True)
row["gte_refold"] = (g.stdout or g.stderr).strip()[-300:]
ok += 1
else:
bad += 1
outcome_append(row)
print(f"delever_permute --bank: {ok} applied, {bad} rejected of {len(todo)} — "
f"now the clean fleet run (R22) and one commit per batch (R42)")
return 0 if ok else 1
# ----------------------------------------------------------------------------------------------------------------------
# the commands
# ----------------------------------------------------------------------------------------------------------------------
def cmd_plan(a):
ex = exemplars(a.only, a.limit, a.include_done)
tot = exemplars(include_done=True)
kinds = collections.Counter(k for e in ex for k in e["kinds"])
print(f"delever_permute --plan: {len(ex)} exemplars drawable of {len(tot)} residue classes "
f"({sum(e['copies'] for e in ex):,} bodies behind them); NEEDED kinds {dict(kinds)}")
for e in ex[:a.show]:
print(f" {e['copies']:4d} copies · {e['needed']:2d} needed {','.join(e['kinds']):<28} "
f"{klass_for(e['kinds']):8} {e['alias'] or alias_of(e['tu'], e['row'])} {e['tu']}:{e['fn']}")
if len(ex) > a.show:
print(f" … {len(ex) - a.show} more")
return 0
def cmd_run(a):
dl.ensure_census(a.j)
ex = exemplars(a.only, a.limit, a.include_done)
if not ex:
print("delever_permute --run: no drawable exemplar (every class attempted, or --only matched none)")
return 1
per = max(1, a.j // max(1, a.workers))
print(f"delever_permute --run: {len(ex)} exemplars, {a.workers} at a time, {a.cycles} cycles x {a.secs}s @ -j{per} each "
f"(budget ≈ {len(ex) / a.workers * a.cycles * a.secs / 3600:.1f} h)", flush=True)
t0 = time.time()
rows = []
with ThreadPoolExecutor(max_workers=a.workers) as pool:
for r in pool.map(lambda e: run_one(e, a.secs, a.cycles, per, a.max_start), ex):
rows.append(r)
won = [r for r in rows if r["verdict"] == "MATCH"]
v = collections.Counter(r["verdict"] for r in rows)
print(f"\npermuter: {len(won)} of {len(rows)} exemplars matched lever-free in {(time.time() - t0) / 3600:.2f} h "
f"({sum(r['copies'] for r in won):,} of {sum(r['copies'] for r in rows):,} bodies behind them) — "
+ " · ".join(f"{k} {n}" for k, n in v.most_common()))
return 0
def cmd_calibrate(a):
ex = exemplars(a.only, a.limit or 6, include_done=True)
print(f"delever_permute --calibrate: {len(ex)} exemplars — the levered body must be MATCH, the lever-free one is the "
f"search's starting distance", flush=True)
rows, dist = [], []
for e in ex:
try:
r = calibrate_one(e)
except Unstrippable as u:
print(f" SKIP {e['tu']}:{e['fn']} — {u.args[0][:2]}", flush=True)
continue
rows.append(r)
if r["ok"] and r["free"] is not None:
dist.append(r["free"])
good = sum(1 for r in rows if r["ok"])
print(f"\ncalibration: {good} of {len(rows)} levered bodies MATCH their regenerated target"
+ (f" · lever-free distance min {min(dist)} / median {sorted(dist)[len(dist) // 2]} / max {max(dist)} "
f"over {len(dist)} exemplars" if dist else ""))
(RUN / "calibration.json").write_text(json.dumps(
[dict(tu=e["tu"], fn=e["fn"], alias=e["alias"], ok=r["ok"], levered=r["levered"], free=r["free"],
levered_line=r["levered_line"], free_line=r["free_line"])
for e, r in zip(ex, rows)], indent=1))
return 0 if good == len(rows) and rows else 1
def cmd_status(a):
outs = load_outcomes()
att = [o for o in outs if o.get("kind") == "attempt"]
bk = [o for o in outs if o.get("kind") == "bank"]
left = exemplars()
v = collections.Counter(o.get("verdict") for o in att)
print(f"delever_permute --status: {len(att)} attempts {dict(v)} · {sum(1 for b in bk if b.get('applied'))} banked of "
f"{len(bk)} tried · {len(left)} residue classes not yet attempted "
f"({sum(e['copies'] for e in left):,} bodies)")
for o in att:
if o.get("verdict") == "MATCH":
print(f" MATCH {o['alias']}__{o['fn']} ({o['copies']} copies, best={o.get('best')}, {o.get('seconds')}s)")
return 0
FIX_TU = """#include "common.h"
extern s32 D_800A0000;
s32 helper_one(s32 a0)
{
return a0 + D_800A0000;
}
INCLUDE_ASM("asm/x/nonmatchings/x", func_dead);
s32 target_fn(s32 a0, s32 *a1)
{
register s32 s __asm__("$16") = a0;
__asm__ __volatile__("" ::: "memory");
*a1 = helper_one(s);
return s;
}
static void tail(void) { }
"""
def cmd_selftest(a):
ok = True
d = REPO / ".run" / "P36" / "permuter" / "_selftest"
if d.exists():
shutil.rmtree(d)
d.mkdir(parents=True)
tu = "src/selftest/fix.c"
sites = [dict(tu=tu, fn="target_fn", cls="A", kind="pin", detail="$16", via="", line=10, col=5,
fn_line=9, fn_end=15, zero=False),
dict(tu=tu, fn="target_fn", cls="B", kind="barrier", detail="", via="", line=11, col=5,
fn_line=9, fn_end=15, zero=False)]
free = lever_free_text(tu, FIX_TU, "target_fn", sites)
if "__asm__" in free or "register" in free:
print(f"selftest: the lever-free text still carries a lever:\n{free}")
ok = False
iso = isolate(tu, free, "target_fn")
if "helper_one(s32 a0);" not in " ".join(iso.split()):
print(f"selftest: helper_one was not reduced to a prototype:\n{iso}")
ok = False
if "INCLUDE_ASM" in iso:
print("selftest: the INCLUDE_ASM line survived isolation")
ok = False
if iso.count("{") != 1:
print(f"selftest: {iso.count('{')} bodies left in the isolated TU (want 1)")
ok = False
if "static void tail(void);" not in " ".join(iso.split()):
print("selftest: a static definition lost its `static` (or its prototype)")
ok = False
if klass_for(["pin"]) != "regalloc" or klass_for(["barrier", "launder"]) != "schedule" \
or klass_for(["pin", "barrier"]) != "regalloc" or klass_for([]) != "regalloc":
print("selftest: klass_for is wrong")
ok = False
base = p16.make_base_c(iso)
if not p16.defines_fn(base, "target_fn"):
print("selftest: base.c lost target_fn")
ok = False
shutil.rmtree(d)
print(f"delever_permute --selftest: {'OK' if ok else 'FAILED'} — lever-free + isolation + prototypes + klass + base.c")
return 0 if ok else 1
def main():
ap = argparse.ArgumentParser()
g = ap.add_mutually_exclusive_group(required=True)
g.add_argument("--plan", action="store_true")
g.add_argument("--prepare", nargs=2, metavar=("TU", "FN"))
g.add_argument("--calibrate", action="store_true",
help="the control: N exemplars' levered bodies must all be MATCH against their regenerated targets")
g.add_argument("--run", action="store_true")
g.add_argument("--bank", action="store_true")
g.add_argument("--status", action="store_true")
g.add_argument("--selftest", action="store_true")
ap.add_argument("--only", nargs="*", default=[], help="fn, TU (substring), alias or nhash prefix")
ap.add_argument("--limit", type=int)
ap.add_argument("--show", type=int, default=25)
ap.add_argument("--workers", type=int, default=4)
ap.add_argument("--secs", type=int, default=240)
ap.add_argument("--cycles", type=int, default=3)
ap.add_argument("-j", type=int, default=16)
ap.add_argument("--label", default="d1")
ap.add_argument("--dirty-ok", action="store_true")
ap.add_argument("--max-start", type=int,
help="skip (as FAR) an exemplar whose lever-free body is further than N instructions from the target")
ap.add_argument("--include-done", action="store_true", help="draw classes that already have an outcome row")
a = ap.parse_args()
os.chdir(REPO)
RUN.mkdir(parents=True, exist_ok=True)
if a.selftest:
return cmd_selftest(a)
if a.plan:
return cmd_plan(a)
if a.status:
return cmd_status(a)
if a.prepare:
tu, fn = a.prepare
ex = next((e for e in exemplars(include_done=True) if e["tu"] == tu and e["fn"] == fn), None)
if ex is None:
ex = dict(nhash="", tu=tu, fn=fn, alias=None, copies=1, needed=0, kinds=[], members=[], row={})
d, draft, tgt, klass = prepare(ex)
print(f"delever_permute --prepare: {d.relative_to(REPO)} (draft.c, {tgt.name}, klass={klass})")
return 0
if a.calibrate:
return cmd_calibrate(a)
if a.run:
return cmd_run(a)
if a.bank:
return bank(a)
if __name__ == "__main__":
sys.exit(main())
@@ -486,20 +486,16 @@ def plan_files(inv, canon, only=None, bound=None):
return files
def direct_rewrite(s, canon):
"""('call', text, name) | ('lever', variant_name, extra) | ('none', why, None) for a direct GTE statement."""
sg = s["sig"]
def canonical_match(sg, canon):
"""(names, clobbers) for a signed statement's signature — the canonical macro it IS (one entry), or the
concatenation of two canonical macros it is (a load then an op: bytes = b1 + b2, inputs the first's,
clobbers the union) — else (None, None). ONE reader of the canonical table (R33): `direct_rewrite` asks it
what the statement should look like, `delever.gte_canonical_clob` asks it what the canonical clobbers are."""
if not sg["ok"]:
return "none", "unsigned", None
key = sig_key(sg)
t = canon["canonical"].get(key)
return None, None
t = canon["canonical"].get(sig_key(sg))
if t and sg["nout"] == 0:
if tuple(sg["clob"]) == tuple(t["clob"]):
return "call", f"{t['name']}({', '.join(sg['ins'])});", t["name"]
extra = sorted(set(sg["clob"]) - set(t["clob"]))
fewer = sorted(set(t["clob"]) - set(sg["clob"]))
return "lever", (f"{t['name']}_m" if extra == ["memory"] and not fewer else f"{t['name']}_v"), extra or fewer
# a concatenation of two canonical macros (a load then an op): bytes = b1 + b2, inputs = the first's, clobbers = the union
return [t["name"]], list(t["clob"])
b = sg["bytes"]
for k1, t1 in canon["canonical"].items():
b1 = k1.split("|")[0]
@@ -509,12 +505,28 @@ def direct_rewrite(s, canon):
t2 = canon["canonical"].get(k2)
if not t2 or t1["nout"] or t1["nin"] != sg["nin"]:
continue
union = sorted(set(t1["clob"]) | set(t2["clob"]))
if union == list(sg["clob"]):
return "call", f"{t1['name']}({', '.join(sg['ins'])}); {t2['name']}();", f"{t1['name']}+{t2['name']}"
extra = sorted(set(sg["clob"]) - set(union))
return "lever", f"{t1['name']}+{t2['name']}_m" if extra == ["memory"] else f"{t1['name']}+{t2['name']}_v", extra or sorted(set(union) - set(sg["clob"]))
return "none", "no canonical signature", None
return [t1["name"], t2["name"]], sorted(set(t1["clob"]) | set(t2["clob"]))
return None, None
def direct_rewrite(s, canon):
"""('call', text, name) | ('lever', variant_name, extra) | ('none', why, None) for a direct GTE statement."""
sg = s["sig"]
if not sg["ok"]:
return "none", "unsigned", None
names, clob = canonical_match(sg, canon)
if names is None:
return "none", "no canonical signature", None
if len(names) == 1:
if tuple(sg["clob"]) == tuple(clob):
return "call", f"{names[0]}({', '.join(sg['ins'])});", names[0]
extra = sorted(set(sg["clob"]) - set(clob))
fewer = sorted(set(clob) - set(sg["clob"]))
return "lever", (f"{names[0]}_m" if extra == ["memory"] and not fewer else f"{names[0]}_v"), extra or fewer
if clob == list(sg["clob"]):
return "call", f"{names[0]}({', '.join(sg['ins'])}); {names[1]}();", f"{names[0]}+{names[1]}"
extra = sorted(set(sg["clob"]) - set(clob))
return "lever", f"{names[0]}+{names[1]}_m" if extra == ["memory"] else f"{names[0]}+{names[1]}_v", extra or sorted(set(clob) - set(sg["clob"]))
def file_edits(tu, raw, p, canon, label):
@@ -233,8 +233,11 @@ def klass_for_fn(fn):
return "", ""
def setup(fn, draft_c, asm_subdir=ASM, klass=None, where=""):
pd = os.path.join(REPO, ".run/permuter", fn)
def setup(fn, draft_c, asm_subdir=ASM, klass=None, where="", outdir=None):
# `outdir` keys the scratch dir by the CALLER's identity instead of the bare function name (R48). A name is not
# unique across the fleet — the overlays overlap in RAM, so two different bodies are both `func_8013B274` — and two
# concurrent runs on one name would share (and corrupt) this directory. Default: the historical path, unchanged.
pd = os.path.join(REPO, outdir) if outdir else os.path.join(REPO, ".run/permuter", fn)
if os.path.exists(pd):
shutil.rmtree(pd)
os.makedirs(pd)
@@ -41,10 +41,12 @@ def main():
ap.add_argument("--secs", type=int, default=180, help="per-cycle time box")
ap.add_argument("--j", type=int, default=12)
ap.add_argument("--winners", default=".run/permuter-winners")
ap.add_argument("--pd", help="scratch dir for this run (default .run/permuter/<fn>) — key it by alias+fn when "
"several runs share a function NAME (R48)")
a = ap.parse_args()
draft = open(a.draft).read()
pd = P.setup(a.fn, draft, asm_subdir=a.asm_subdir, klass=a.klass)
pd = P.setup(a.fn, draft, asm_subdir=a.asm_subdir, klass=a.klass, outdir=a.pd)
if not pd:
print("ILS setup FAILED (target .s didn't assemble?)"); sys.exit(1)
print(f"ILS {a.fn}: {a.cycles} cycles x {a.secs}s @ -j{a.j}, klass={a.klass}")
+3 -2
View File
@@ -8,8 +8,8 @@
> its platform SDK need the marked adaptation. The last table lists the tools that are project-only in code (their *shape* is a task;
> their code does not transfer). *TODO(platform): the MIPS and PlayStation SDK hard-codes are the ones another platform replaces first.*
>
> **Coverage:** 300 tool files in scope (submodules, vendored and downloaded code excluded), of which 300 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 26 · P6 51 · P7 20 · P8 86 · P9 27 · P10 20 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (30 pointer rows); one-offs are omitted. Table rows in all: 330 (the installer checks its copy against this figure).
> **Coverage:** 301 tool files in scope (submodules, vendored and downloaded code excluded), of which 301 live rows
> below; per phase: P1 2 · P2 26 · P3 17 · P4 9 · P5 26 · P6 51 · P7 20 · P8 86 · P9 27 · P10 21 · PROJECT-ONLY 16. Superseded tools appear only as pointers to their successor (30 pointer rows); one-offs are omitted. Table rows in all: 331 (the installer checks its copy against this figure).
## P1 — extraction + manifest
@@ -341,6 +341,7 @@
| `verbatim_target_s.py` | regenerate a splitter-format target disassembly for a function no longer stubbed | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | repo build/asm layout |
| `delever_cycle.sh` | run the de-lever batch cycle unattended: batch, verify by exit code, clean fleet gate, census, log, commit | The Phase-36 T3/T4 driver: per batch a clean-tree check, the oracle recalibrated when stale, delever --apply read by exit code and its final X/X line, the clean fleet run as the outer gate (218/218), the census rerun, the phase-log entry and the checkpoint headline, one commit per batch; stops on the first red with the files in place for --restore | the batch size, the aliases, the fleet count |
| `share_body_cycle.sh` | run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check | The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed the moment it is green, the clean fleet run every N batches — stops on the first red | repo paths, the phase-log format, the fleet count from config/check.*.sha |
| `delever_permute.py` | search the compiler's own shape space for a lever-free body that keeps the bytes | The Phase-36 rung D: one exemplar per residue text class from the delever ledger (largest class first — a match banks every copy), each prepared as a single-function translation unit (delever's rung-A rewrite of every removable site, every other definition reduced to a prototype, shared-header includes to their prototypes, INCLUDE_ASM and file-scope asm dropped, the build's own CPPFLAGS through cpp -P) against a target regenerated from the ROM image in both the assemblable and the splat form; every attempt calibrates itself first (the LEVERED body must be MATCH, or the harness is not measuring that function) and records the lever-free body's starting distance; decomp-permuter through permuter_ils with the weight profile chosen from the NEEDED kinds; a score-0 winner is banked only through delever --apply-body and the GTE re-fold; scratch, winners and the outcome ledger keyed by alias+fn | the ledger, the permuter harness, the target regenerator |
| `share_body.py` | share one byte-identical function class across its binaries through an include-at-site header, gated per binary | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | repo paths, the registry and signature schemas |
| `delever.py` | take register pins, asm statements, volatile and register levers out of matched C, byte-gated per site | The Phase-36 de-lever engine and campaign tool: positional rewrites per lever class (pin -> plain declaration, barrier/keep-alive deleted, a launder deleted or turned into the assignment it is, a hand-placed instruction -> its C, the zero-register variable -> 0, a macro-carried site deleted/valued/refused by its macro's shape, volatile/register dropped); per body replay -> rung A strip-all -> rung B greedy through delever_oracle; the file is the write unit and its final compile through every recipe the proof; --plan/--apply batches (TUs parallel, exemplar files first; headers serial, includers parallel), the ledger keyed by tu+fn+addr with the body's nhash before/after, !FAKE markers on class A/B survivors, --redraw for bodies refused by an older tool, --restore from inflight.json, --scrub for orphan markers, --apply-body for T6/T7, --selftest with a stub oracle, --probe (T2) | the census site records, the oracle |
| `verbatim_to_stub.py` | turn an inline-assembly body back into a stub the toolchain can reach | Turns an inline-assembly body back into an include-assembly stub | repo src/asm layout |
+12 -11
View File
@@ -5,7 +5,7 @@ freshness). The derived columns come from the tree on every run; the authored on
dictionary, whose coverage is asserted both ways. Read it by NEED: find the phrase that matches what you are trying to do, then the tool,
then what proved it. The same data generates the day-one kit's manifest and its verbatim tool corpus.*
**Coverage:** 300 tool files in scope (submodules, vendored and downloaded code excluded; `find` and `git ls-files` agree) + 38 retired under `tools/sunset/`. Classes: LIVE 241 (a runtime consumer), REFERENCED 29 (a SETUP row only), ORPHAN 30 (neither) — of 300. Portability: PORTABLE 23, ADAPT 260, PROJECT-ONLY 17.
**Coverage:** 301 tool files in scope (submodules, vendored and downloaded code excluded; `find` and `git ls-files` agree) + 38 retired under `tools/sunset/`. Classes: LIVE 241 (a runtime consumer), REFERENCED 30 (a SETUP row only), ORPHAN 30 (neither) — of 301. Portability: PORTABLE 23, ADAPT 261, PROJECT-ONLY 17.
## P1 — extraction + manifest
@@ -72,7 +72,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| When you need to… | Tool | What it does | Proven by | Adapt | Class |
|---|---|---|---|---|---|
| compare instructions with relocations masked, shared by matcher and scorer | `masked_diff.py` | Shared relocation-masked instruction comparison used by the matcher and the permuter scorer | diff_regions.py, masked_scorer.py, match_one.py, reg_renumber_swap.sh (+6) | MIPS relocation encodings | LIVE |
| compile one function standalone and compare its masked bytes to the target | `match_one.py` | Compiles one function standalone with the pinned toolchain, masks relocations, compares to target bytes | ab_score.py, api_draft.py, aprop_autodraft.py, asm_verbatim.py (+22) | compiler triple, repo build flags | LIVE |
| compile one function standalone and compare its masked bytes to the target | `match_one.py` | Compiles one function standalone with the pinned toolchain, masks relocations, compares to target bytes | ab_score.py, api_draft.py, aprop_autodraft.py, asm_verbatim.py (+23) | compiler triple, repo build flags | LIVE |
| dump every compiler pass file for a self-contained draft | `cc1_dumps.sh` | Dumps every compiler pass file for a self-contained draft into a private directory | cc1_dumps_tu.sh, ghost_census.py | compiler triple, repo scratch paths | LIVE |
| dump every compiler pass file for the spliced real translation unit | `cc1_dumps_tu.sh` | Same pass dumps for the spliced real translation unit, the faithful compile | alloc_table.py | absolute repo path, compiler triple | LIVE |
| re-judge standalone compile failures against their real translation unit before dropping | `rtu_second_chance.py` | Re-judges standalone compile-failures against their real translation unit before dropping them | maintenance.sh | repo scratch paths | LIVE |
@@ -175,10 +175,10 @@ then what proved it. The same data generates the day-one kit's manifest and its
| classify a length-drift near-miss against its own target and build its routing card | `len_tells.py` | Classifies a length-drift near-miss against its own target and builds the routing card | atlas_features.py | repo scratch paths | LIVE |
| classify a residual's shape to its documented lever, as an independent implementation | `residual_rules_b.py` | Independent second implementation of residual-shape to rule classification, for head-to-head | triage_ladder.py | repo doc paths | LIVE |
| classify where a remapped sibling's compiled bytes diverge from its target | `diff_regions.py` | Classifies where a remapped member's compiled bytes diverge from its target | — | repo build paths | ORPHAN |
| compile a permuter candidate exactly as the build rule does | `permuter/compile.sh` | The permuter's compile command, mirroring the build rule exactly so objects are build-faithful | p16_permute.py, compile_o0.sh | compiler triple, repo build flags | LIVE |
| compile a permuter candidate exactly as the build rule does | `permuter/compile.sh` | The permuter's compile command, mirroring the build rule exactly so objects are build-faithful | delever_permute.py, p16_permute.py, compile_o0.sh | compiler triple, repo build flags | LIVE |
| compile a permuter candidate unoptimized, exactly as the build rule does | `permuter/compile_o0.sh` | Unoptimized variant of the same permuter compile command | p16_permute.py | compiler triple, repo build flags | LIVE |
| decide whether a residual reorder is scheduling or a register-grant consequence | `oracle/reg_renumber_swap.sh` | Mechanized oracle separating a register-allocation swap from a scheduling reorder | — | compiler triple, repo scratch paths | ORPHAN |
| drive the permuter per function, building base, target and settings | `p16_permute.py` | Per-function permuter driver building base, target and settings, then reporting closeness | auto_driver.py, grinder.py, permuter_ils.py | repo scratch paths, permuter layout | LIVE |
| drive the permuter per function, building base, target and settings | `p16_permute.py` | Per-function permuter driver building base, target and settings, then reporting closeness | auto_driver.py, delever_permute.py, grinder.py, permuter_ils.py | repo scratch paths, permuter layout | LIVE |
| find pseudo-registers with references but no remaining occurrences, and their frame slots | `ghost_census.py` | Frame-residue oracle: finds pseudos with references but no remaining occurrences, and their slots | cc1_dumps.sh | compiler pass-dump format | LIVE |
| find which words diverge when the whole-binary gate says different but not where | `diff_autopsy.sh` | Reproduces exactly what the gate saw, then decodes the diverging words and restores the tree | — | repo src/build paths | REFERENCED |
| grind the closest near-misses with the permuter and bank through the gate | `grinder.py` | Token-free permuter daemon grinding the closest near-misses and banking through the gate | auto_status.sh, auto_supervisor.sh, autopsy.py, gate_stage.py (+4) | repo scratch paths | LIVE |
@@ -191,7 +191,7 @@ then what proved it. The same data generates the day-one kit's manifest and its
| steer permuter mutation toward the lever family the residual class implies | `permuter_weights.py` | Directs permuter mutation toward the lever family the residual class implies | autopsy.py, p16_permute.py | permuter settings format | LIVE |
| tag every compiler-source citation with the source tree it refers to | `gccmap_cites.py` | Tags every compiler-source citation in the codegen-map docs with the tree it refers to | .github/workflows/no-rom.yml, Makefile, tool_census.py | repo doc/reference paths | LIVE |
| turn a red whole-binary result into a named list of divergent symbols | `main_diff_locate.py` | Turns a red whole-binary result into a named list of divergent symbols | gate_main.py | repo build/config layout | LIVE |
| warm-restart the permuter from its best waypoint each cycle | `permuter_ils.py` | Iterated-local-search wrapper warm-restarting the permuter from the best waypoint each cycle | grinder.py, permuter_sweep.py | permuter layout, repo scratch paths | LIVE |
| warm-restart the permuter from its best waypoint each cycle | `permuter_ils.py` | Iterated-local-search wrapper warm-restarting the permuter from the best waypoint each cycle | delever_permute.py, grinder.py, permuter_sweep.py | permuter layout, repo scratch paths | LIVE |
## P8 — the campaign: cards, lanes, gates, recovery, harvest
@@ -322,23 +322,24 @@ then what proved it. The same data generates the day-one kit's manifest and its
|---|---|---|---|---|---|
| add per-site function-pointer casts so a draft can call a differently typed callee | `cast_call_sites.py` | Adds per-site function-pointer casts so a draft can call a differently typed callee | canon_sig_reconcile.py, cast_self_callers.py, family_sweep.py, gate_stage.py (+4) | repo src layout | LIVE |
| apply curated names and signatures inside the analysis tool and save | `ghidra_scripts/ApplySymbols.java` | The in-tool half of that mirror: apply curated names and signatures, save on exit | ghidra_apply_symbols.sh, ghidra_export_annotations.sh, ghidra_rebuild.sh, ExportAnnotations.java (+1) | none | LIVE |
| census every compiler-forcing construct in the C and gate the levers-off phase | `lever_census.py` | The Phase-36 census of lever sites (register pins, asm statements by kind, volatile levers, bare register, asm-label aliases, builtins, attributes; the whole-body hand-asm routines and the GTE ops set apart) derived from the shared-body scanner with a per-token coverage assertion against the raw text, four known-true controls, the !FAKE marker split, --check (0 unmarked pins/asm, 0 orphan markers) and --strict (0 pins, 0 asm) gates, --sites for the delever ledger, a fixture self-test | delever.py, delever_cycle.sh, gte_consolidate.py, progress.py (+1) | repo paths, the shared-body scanner | LIVE |
| consolidate the GTE coprocessor inline-asm macros under Sony's names in one header and sweep dead lever macros | `gte_consolidate.py` | The Phase-36 T5 tool: every asm-bearing macro definition and direct GTE statement SIGNED by the build's own maspsx→as tail (template bytes with operands bound to fixed registers, operand counts, clobbers); one canonical text per signature named by PsyQ's inline_c.h convention (Sony's file supplies names and clobber lists, never opcode words) written to include/gte_inline.h; per file the canonical duplicates deleted, private names renamed, a variant with extra clobbers tried as canonical and kept as a marked <name>_m lever only when the object differs, direct statements rewritten into canonical calls (a concatenation of two included); every file judged through every recipe; --sweep deletes dead asm-bearing macros and drops a compound macro's inner asm when byte-neutral | delever_cycle.sh, lever_census.py | Sony's inline_c.h path, the census sites, the oracle | LIVE |
| census every compiler-forcing construct in the C and gate the levers-off phase | `lever_census.py` | The Phase-36 census of lever sites (register pins, asm statements by kind, volatile levers, bare register, asm-label aliases, builtins, attributes; the whole-body hand-asm routines and the GTE ops set apart) derived from the shared-body scanner with a per-token coverage assertion against the raw text, four known-true controls, the !FAKE marker split, --check (0 unmarked pins/asm, 0 orphan markers) and --strict (0 pins, 0 asm) gates, --sites for the delever ledger, a fixture self-test | delever.py, delever_cycle.sh, delever_permute.py, gte_consolidate.py (+2) | repo paths, the shared-body scanner | LIVE |
| consolidate the GTE coprocessor inline-asm macros under Sony's names in one header and sweep dead lever macros | `gte_consolidate.py` | The Phase-36 T5 tool: every asm-bearing macro definition and direct GTE statement SIGNED by the build's own maspsx→as tail (template bytes with operands bound to fixed registers, operand counts, clobbers); one canonical text per signature named by PsyQ's inline_c.h convention (Sony's file supplies names and clobber lists, never opcode words) written to include/gte_inline.h; per file the canonical duplicates deleted, private names renamed, a variant with extra clobbers tried as canonical and kept as a marked <name>_m lever only when the object differs, direct statements rewritten into canonical calls (a concatenation of two included); every file judged through every recipe; --sweep deletes dead asm-bearing macros and drops a compound macro's inner asm when byte-neutral | delever.py, delever_cycle.sh, delever_permute.py, lever_census.py | Sony's inline_c.h path, the census sites, the oracle | LIVE |
| convert a shared-body macro header into per-function plain-C headers included at each site | `macro_to_header.py` | The Phase-35 converter: every DEFINE_ macro body becomes a plain-C header under src/shared/<space>/ included at its site, the legacy name-parameterized headers converted, the registry text-edited, the macro header deleted; --plan / --apply / --finalize / --verify | share_body.py, tool_census.py | the macro form is this project's; a kit-born project shares headers from its first bank | LIVE |
| emit a function's target assembly as an inline-assembly body | `asm_verbatim.py` | Emits the file-scope inline-assembly body form from a disassembly file | docs/wave-playbook.md, draft_prechecks.py, gate_main.py, recover_route.py | repo asm layout | LIVE |
| extract inline-defined types and typedefs from a source file into a shared header | `build_engine_types.py` | Extracts inline-defined named types and typedefs from a source file into a shared header | inject_capped_externs.py, lift_types.py, uniquify_type.py | repo shared-header path | LIVE |
| flag address-named references whose address now has a curated name | `lint_symbol_refs.py` | Flags address-named references in committed sources whose address now has a curated name | .github/workflows/no-rom.yml, Makefile | repo symbol/src paths | LIVE |
| give each conflicting camp of a same-named type its own name | `uniquify_type.py` | Gives each conflicting camp of a same-named type its own name so every camp becomes liftable | — | repo src layout | REFERENCED |
| guard that every inline-assembly body still reproduces its target bytes | `verbatim_check.py` | Regression guard that every inline-assembly body still reproduces its target bytes | .github/workflows/no-rom.yml, lever_census.py, progress.py, verbatim_target_s.py (+1) | repo src layout | LIVE |
| judge one translation-unit edit by the bytes of its object in under a second | `delever_oracle.py` | The Phase-36 fast byte oracle: every object's exact build command captured once via make -n -W (the Makefile's own recipe, pad stage and -O0 overrides included), a candidate compiled in place with -o/-MF redirected to scratch and compared with the fleet run's object; --calibrate proves 100 % equality untouched, twin == primary and a positive control before any verdict is trusted | delever.py, delever_cycle.sh, gte_consolidate.py | the Makefile's object rules, the twin rule | LIVE |
| measure duplicate function bodies across the fleet and assert one source per unique function | `share_census.py` | The census of byte-identical function classes across every binary with their source forms and verdicts, plus the S1 invariant check with its exception ledger and a fixture self-test | Makefile, audit_binaries.py, dedup_integrate.py, delever.py (+9) | repo paths, the registry and signature schemas | LIVE |
| judge one translation-unit edit by the bytes of its object in under a second | `delever_oracle.py` | The Phase-36 fast byte oracle: every object's exact build command captured once via make -n -W (the Makefile's own recipe, pad stage and -O0 overrides included), a candidate compiled in place with -o/-MF redirected to scratch and compared with the fleet run's object; --calibrate proves 100 % equality untouched, twin == primary and a positive control before any verdict is trusted | delever.py, delever_cycle.sh, delever_permute.py, gte_consolidate.py | the Makefile's object rules, the twin rule | LIVE |
| measure duplicate function bodies across the fleet and assert one source per unique function | `share_census.py` | The census of byte-identical function classes across every binary with their source forms and verdicts, plus the S1 invariant check with its exception ledger and a fixture self-test | Makefile, audit_binaries.py, dedup_integrate.py, delever.py (+10) | repo paths, the registry and signature schemas | LIVE |
| mirror the curated symbol file into the analysis database with a real save | `ghidra_apply_symbols.sh` | Mirrors the curated symbol file into the analysis program headlessly, with a real save | — | repo symbol path, project name | REFERENCED |
| refuse, from one place, the command line of a tool the project has frozen | `frozen.py` | The one refusal a FROZEN tool prints from its main(): the tool, the successor and why; imports never exit (libraries stay usable) | aprop_autodraft.py, blocker_probe.py, canon_sig_reconcile.py, conform_decls.py (+10) | nothing | LIVE |
| regenerate a splitter-format target disassembly for a function no longer stubbed | `verbatim_target_s.py` | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | decompme_replica.sh | repo build/asm layout | LIVE |
| regenerate a splitter-format target disassembly for a function no longer stubbed | `verbatim_target_s.py` | Regenerates a splitter-format target disassembly for a function that is no longer a stub; --gas emits the assemblable gas-syntax form ($-registers, .L labels, noreorder) that a web diff service accepts as a pasted target | decompme_replica.sh, delever_permute.py | repo build/asm layout | LIVE |
| run the de-lever batch cycle unattended: batch, verify by exit code, clean fleet gate, census, log, commit | `delever_cycle.sh` | The Phase-36 T3/T4 driver: per batch a clean-tree check, the oracle recalibrated when stale, delever --apply read by exit code and its final X/X line, the clean fleet run as the outer gate (218/218), the census rerun, the phase-log entry and the checkpoint headline, one commit per batch; stops on the first red with the files in place for --restore | delever.py, gte_consolidate.py | the batch size, the aliases, the fleet count | LIVE |
| run the share-body batch cycle unattended: batch, verify by exit code, log, commit, periodic clean fleet check | `share_body_cycle.sh` | The Phase-35 T5 driver for bucket new: per batch share_body --apply --bucket new --batches 1 --label new<k>, the gated N/N line read, the phase log entry appended, the bank committed the moment it is green, the clean fleet run every N batches — stops on the first red | — | repo paths, the phase-log format, the fleet count from config/check.*.sha | ORPHAN |
| search the compiler's own shape space for a lever-free body that keeps the bytes | `delever_permute.py` | The Phase-36 rung D: one exemplar per residue text class from the delever ledger (largest class first — a match banks every copy), each prepared as a single-function translation unit (delever's rung-A rewrite of every removable site, every other definition reduced to a prototype, shared-header includes to their prototypes, INCLUDE_ASM and file-scope asm dropped, the build's own CPPFLAGS through cpp -P) against a target regenerated from the ROM image in both the assemblable and the splat form; every attempt calibrates itself first (the LEVERED body must be MATCH, or the harness is not measuring that function) and records the lever-free body's starting distance; decomp-permuter through permuter_ils with the weight profile chosen from the NEEDED kinds; a score-0 winner is banked only through delever --apply-body and the GTE re-fold; scratch, winners and the outcome ledger keyed by alias+fn | — | the ledger, the permuter harness, the target regenerator | REFERENCED |
| share one byte-identical function class across its binaries through an include-at-site header, gated per binary | `share_body.py` | The Phase-35 successor of dedup_propagate + dedup_extend for the include-at-site form: exemplar by majority text, the header written once, every private copy replaced by the include at its position, per-binary byte gate with object comparison, bisect on red, the registry appended or extended by text, the exception ledger on refusal; --plan / --apply --bucket extend|new | aprop_autodraft.py, audit_binaries.py, auto_driver.py, blocker_probe.py (+18) | repo paths, the registry and signature schemas | LIVE |
| take register pins, asm statements, volatile and register levers out of matched C, byte-gated per site | `delever.py` | The Phase-36 de-lever engine and campaign tool: positional rewrites per lever class (pin -> plain declaration, barrier/keep-alive deleted, a launder deleted or turned into the assignment it is, a hand-placed instruction -> its C, the zero-register variable -> 0, a macro-carried site deleted/valued/refused by its macro's shape, volatile/register dropped); per body replay -> rung A strip-all -> rung B greedy through delever_oracle; the file is the write unit and its final compile through every recipe the proof; --plan/--apply batches (TUs parallel, exemplar files first; headers serial, includers parallel), the ledger keyed by tu+fn+addr with the body's nhash before/after, !FAKE markers on class A/B survivors, --redraw for bodies refused by an older tool, --restore from inflight.json, --scrub for orphan markers, --apply-body for T6/T7, --selftest with a stub oracle, --probe (T2) | delever_cycle.sh, gte_consolidate.py, lever_census.py | the census site records, the oracle | LIVE |
| take register pins, asm statements, volatile and register levers out of matched C, byte-gated per site | `delever.py` | The Phase-36 de-lever engine and campaign tool: positional rewrites per lever class (pin -> plain declaration, barrier/keep-alive deleted, a launder deleted or turned into the assignment it is, a hand-placed instruction -> its C, the zero-register variable -> 0, a macro-carried site deleted/valued/refused by its macro's shape, volatile/register dropped); per body replay -> rung A strip-all -> rung B greedy through delever_oracle; the file is the write unit and its final compile through every recipe the proof; --plan/--apply batches (TUs parallel, exemplar files first; headers serial, includers parallel), the ledger keyed by tu+fn+addr with the body's nhash before/after, !FAKE markers on class A/B survivors, --redraw for bodies refused by an older tool, --restore from inflight.json, --scrub for orphan markers, --apply-body for T6/T7, --selftest with a stub oracle, --probe (T2) | delever_cycle.sh, delever_permute.py, gte_consolidate.py, lever_census.py | the census site records, the oracle | LIVE |
| turn an inline-assembly body back into a stub the toolchain can reach | `verbatim_to_stub.py` | Turns an inline-assembly body back into an include-assembly stub | — | repo src/asm layout | REFERENCED |
## PROJECT-ONLY — project-only in code (the shape is a task; the code does not transfer)