phase-36: the census walker defaults to the tree's cross-file asm-macro table for every out-of-census caller (delever scrub/bank/judge) — S103's scrub false positive on an ENGINE_SHB launder reproduced and fixed; control still finds a real orphan; census unchanged 5,097 / 0 unmarked

docs/SETUP.md

@@ -785,7 +785,7 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
 | | `tools/lever_census.py` | **(P36 T1)** The census of every compiler-forcing construct ("lever") in the fleet's C — register pins, asm statements by kind (barrier / launder / keepalive / instruction / GTE / verbatim-body), volatile levers, bare `register`, plus the deferred asm-label aliases, builtins and attributes — derived from `share_census`'s scanner with a per-token coverage assertion against the raw text, four known-true controls, the `// !FAKE:` marker split, `--check` (0 unmarked pins/asm AND 0 orphan markers — a `// !FAKE:` line with no pin/asm site on it nor on the line below) and `--check --strict` (0 pins, 0 asm outside the GTE header) gates, cross-file macro names (a use of a macro defined in another file — the prelude's `ENGINE_SHB`, a unit's `SHB` inside an included header, the GTE header's names — is classed by the majority definition's kind; it was invisible before), the JSON's `head` + stat-based `src_stamp` (delever refuses a census that does not describe the tree) and a per-file walk cache keyed on the walker's own hash (a tool change invalidates it, R35), `--sites` (every site to `.run/P36/census/lever_sites.jsonl`, the delever ledger's input), `--selftest`. Evidence: `.run/P36/census/lever_census.{json,txt}`; `progress.py` publishes the `levers` block and a README sentence from it. |
 | | `tools/delever.py --recipes` | **(P36 T6, rung R)** `tools/delever.py --recipes --label L [--only …] [--limit N] [--control 8]`: the cookbook's byte-neutral SHAPE recipes tried mechanically on every RESIDUE body, seeded with that body's lever-free text — R2 the formerly-pinned declarations permuted among their own lines, R4 one of them moved to every other slot of the body's declaration run (§76/§501-R: the allocation order is the bank), R3 an initializer split into a declaration and a first assignment placed after the WHOLE declaration run (C89), deduplicated and capped at 40 candidates; the first candidate whose object is IDENTICAL replaces the body and its `// !FAKE:` markers are scrubbed within its own line span only. The control runs first (R39) on the drawn bodies: the identity permutation through the same splice code must reproduce its input text exactly (a text assertion, no compile) and the untouched file must still judge IDENTICAL. Scope, stated: only a body with a formerly-PINNED declaration has candidates. |
 | | `tools/delever_oracle.py` | **(P36 T2)** The fast byte oracle for a single-translation-unit edit: `--recipes` captures every object's exact build command through `make -n -W <src> <obj> BINARY=<alias>` (the Makefile's own pipeline — the jump-table pad stage, the per-object `-O0` overrides, the twin rule — into `.run/P36/delever/recipes.json`, regenerable in ~15 s); a candidate is compiled IN PLACE with `-o`/`-MF` redirected to scratch and its bytes compared with `build/`'s object from the fleet run (build/ is never written); `--calibrate <aliases>` proves 100 % equality untouched + twin == primary + a positive control (a nop injected at the end of a real body → DIFFERS) and writes `calibration.json` (HEAD, config stamp, per-object seconds); `--status`. Measured at T2: main 0.08–0.77 s per object, overlays ~0.14 s. |
-| | `tools/delever.py` | **(P36 T2/T3)** The de-lever engine AND the campaign tool. Rewrites per lever class on the raw text at the census's positions (a token mismatch REFUSES, never guesses): pin → plain declaration (type/qualifiers/initializer kept; the `$0` zero-register variable's uses → 0, refused if it is ever assigned), barrier / keep-alive → deleted, launder → deleted when it launders a value into itself, **an ASSIGNMENT `out = in;` when its output and input differ** (deleting those made cc1 2.7.2 abort — the T2 probe counted them NEEDED), a hand-placed instruction → its C (addu-$zero/move/la/lh/lw/addiu/sll/srl/and/andi/lui/li and the lui+addiu / lui+ori pairs), a macro-carried site → deleted for a pure launder statement macro (`SHB`), its value `((T)(p))` for a launder statement-expression, REFUSED for a compound macro (XFER/DRAW/RTP_SND: the lever is in the `#define`, T5), volatile / register → dropped. Per body: replay (a ledger exemplar with the same normalized text: 1 compile) → rung A strip-all → rung B greedy, through `delever_oracle` (verdicts IDENTICAL / DIFFERS / COMPILE-ERROR / COMPILE-CRASH). **The file is the write unit and its final compile the proof**: every body's accepted edits + the file-scope volatile edits + the `// !FAKE:` markers spliced once, compiled through every recipe of the file (a twin's object; every includer of a header, in parallel) — IDENTICAL or the file-scope edits are dropped, or the file is restored and REFUSED (COMBINATION-FAILED). `--plan` / `--apply --batch N --label L [--headers] [--only …] [-j 12]` (TUs in parallel, exemplar files before their copies; headers serial; the tree must be clean, the calibration current, the census's src stamp the tree's — it reruns the census itself), `--redraw REFUSED NOTHING-USABLE` (draw again the bodies whose latest ledger verdict is one of these — after a tool fix), `--restore` (from `inflight.json`, the only restore — R102; it also drops the in-flight batch's ledger rows into an ignored `ledger.jsonl.killed_<label>`, so a killed batch leaves no trace and its bodies are drawn again), `--status`, `--scrub [--only …]` (remove the census's ORPHAN markers — a `// !FAKE:` whose site is gone — each file judged through every recipe; a removed or rewritten site consumes its own trailing marker, so orphans arise only from older tool versions), `--apply-body TU FN FILE --label L [--rung E]` (T6/T7: a reshaped body judged on the bytes, refused if a class A/B lever remains), `--selftest` (the fixture through the census's own parser: every rewrite, the ladder against a stub oracle with a crashing site, the markers, a replay and a disagreeing replay, the ledger index, the oracle's crash forms), `--probe` (T2). The ledger `.run/P36/delever/ledger.jsonl`: one row per (tu, fn, addr, aliases) per judgement with every site's verdict REMOVED / REWRITTEN / NEEDED(oracle word) / REFUSED(why) / DEFERRED, the rung, the calibration id, the body's nhash BEFORE and AFTER — "done" = the current nhash is some row's after-hash; a body with a row's before-hash replays it. Class C/D survivors are never marked (decision 3); class A/B survivors carry `// !FAKE: <kind> <detail> — <verdict> (P36 <rung> <label>)` at the end of their line. Evidence: `apply_<label>.log`, `batch_<label>.json`. Per-batch setup: the per-file include lists are cached in the ignored `includers_cache.json` on (mtime, size) — 29 s → 1.3 s, proven equal to the uncached map. |
+| | `tools/delever.py` | **(P36 T2/T3)** The de-lever engine AND the campaign tool. Rewrites per lever class on the raw text at the census's positions (a token mismatch REFUSES, never guesses): pin → plain declaration (type/qualifiers/initializer kept; the `$0` zero-register variable's uses → 0, refused if it is ever assigned), barrier / keep-alive → deleted, launder → deleted when it launders a value into itself, **an ASSIGNMENT `out = in;` when its output and input differ** (deleting those made cc1 2.7.2 abort — the T2 probe counted them NEEDED), a hand-placed instruction → its C (addu-$zero/move/la/lh/lw/addiu/sll/srl/and/andi/lui/li and the lui+addiu / lui+ori pairs), a macro-carried site → deleted for a pure launder statement macro (`SHB`), its value `((T)(p))` for a launder statement-expression, REFUSED for a compound macro (XFER/DRAW/RTP_SND: the lever is in the `#define`, T5), volatile / register → dropped. Per body: replay (a ledger exemplar with the same normalized text: 1 compile) → rung A strip-all → rung B greedy, through `delever_oracle` (verdicts IDENTICAL / DIFFERS / COMPILE-ERROR / COMPILE-CRASH). **The file is the write unit and its final compile the proof**: every body's accepted edits + the file-scope volatile edits + the `// !FAKE:` markers spliced once, compiled through every recipe of the file (a twin's object; every includer of a header, in parallel) — IDENTICAL or the file-scope edits are dropped, or the file is restored and REFUSED (COMBINATION-FAILED). `--plan` / `--apply --batch N --label L [--headers] [--only …] [-j 12]` (TUs in parallel, exemplar files before their copies; headers serial; the tree must be clean, the calibration current, the census's src stamp the tree's — it reruns the census itself), `--redraw REFUSED NOTHING-USABLE` (draw again the bodies whose latest ledger verdict is one of these — after a tool fix), `--restore` (from `inflight.json`, the only restore — R102; it also drops the in-flight batch's ledger rows into an ignored `ledger.jsonl.killed_<label>`, so a killed batch leaves no trace and its bodies are drawn again), `--status`, `--scrub [--only …]` (remove the census's ORPHAN markers — a `// !FAKE:` whose site is gone — each file judged through every recipe; a removed or rewritten site consumes its own trailing marker, so orphans arise only from older tool versions; **S104:** every `lever_census.walk_file` call outside the census now defaults to the tree's cross-file asm-macro table, `lever_census.tree_asm_macros()` (~24 s once per process) — S103's scrub walked a shared header alone, read its prelude-defined `ENGINE_SHB` launder as no site, and deleted a live marker), `--apply-body TU FN FILE --label L [--rung E]` (T6/T7: a reshaped body judged on the bytes, refused if a class A/B lever remains), `--selftest` (the fixture through the census's own parser: every rewrite, the ladder against a stub oracle with a crashing site, the markers, a replay and a disagreeing replay, the ledger index, the oracle's crash forms), `--probe` (T2). The ledger `.run/P36/delever/ledger.jsonl`: one row per (tu, fn, addr, aliases) per judgement with every site's verdict REMOVED / REWRITTEN / NEEDED(oracle word) / REFUSED(why) / DEFERRED, the rung, the calibration id, the body's nhash BEFORE and AFTER — "done" = the current nhash is some row's after-hash; a body with a row's before-hash replays it. Class C/D survivors are never marked (decision 3); class A/B survivors carry `// !FAKE: <kind> <detail> — <verdict> (P36 <rung> <label>)` at the end of their line. Evidence: `apply_<label>.log`, `batch_<label>.json`. Per-batch setup: the per-file include lists are cached in the ignored `includers_cache.json` on (mtime, size) — 29 s → 1.3 s, proven equal to the uncached map. |
 | | `tools/gte_consolidate.py` | **(P36 T5)** The GTE consolidation. `--inventory`: every asm-bearing `#define` and every direct GTE statement signed by the build's own tail (`maspsx --aspsx-version=2.56` → `mipsel-linux-gnu-as` with `include/macro.inc` + `gte_macros.inc`, where `rtps`/`mvmva`/`sqr 0` are GAS macros): signature = (bytes with %k bound to $4–$7 / $2–$3, #outputs, #inputs); `nop;nop;rtps`, `.word 0x4a180001` and `cop2 0x0180001` are one signature. The canonical table (`.run/P36/gte/canonical.json`): one text per signature (the majority body — byte-proven in the fleet), Sony's name when the operand counts agree with `inline_c.h` (Sony's converted header is unreliable for opcode words: `gte_rtps` there is `.word 0x0000007f`), Sony's clobber set canonical even when every definition carries the steer (the body is synthesized with Sony's clobbers); a definition with the canonical bytes and OTHER clobbers is a lever variant `<name>_m` / `<name>_v<hash>`. `--header` writes `include/gte_inline.h` (50 macros at T5) and the include in `common.h`. `--apply --batch N --label L`: per file, canonical duplicates deleted, private names renamed to the canonical, each variant TRIED as canonical (the clobber dropped, judged) and kept as a marked `_m` lever only when the object differs, direct statements rewritten into canonical calls (a concatenation of two canonical macros included; a direct statement with extra clobbers is marked as a lever; an unmatched sequence is counted); the file judged through every recipe, the definition edits alone retried when the direct rewrites differ. `--sweep`: dead asm-bearing macros deleted — a use (or an `#include` of a header using the name) is counted for the GOVERNING definition, the last one above it, so a used sibling definition keeps no dead one alive; a compound macro's inner asm dropped when byte-neutral. `--remark [--dirty-ok]`: the census's UNMARKED gte-lever sites marked on their own (first) lines after a `delever --scrub`. The selftest never writes the real table (`canonical_table(write=False)`) — it once did, and the census then classified against a fixture. `tools/delever_cycle.sh … gte` runs the apply batches. The census classes a variant's uses and a direct statement with clobbers beyond the canonical's as `gte-lever` — a class-B lever INSIDE the phase's number from T5 on (2026-09-09; the headline's definition gained the clause), marked, 0 at the close — and counts per-TU asm macro definitions. |
 | | `tools/delever_cycle.sh` | **(P36 T3)** The unattended batch cycle: `[LABEL_PREFIX=…] [TASK=T4] [REDRAW="REFUSED NOTHING-USABLE"] tools/delever_cycle.sh START END [BATCH=300] [tus|headers] [ONLY="alias …"]` — per batch: the tree must be clean (a dotfile under `src/` refuses), the oracle recalibrated when stale (2–3 s; every commit stales it), `delever --apply` (exit code + its `final X/X identical` — an empty verify line is a failure, R32), the CLEAN fleet run as the outer gate (`218 passed, 0 failed of 218`, R22 — it also refreshes `build/`, the oracle's baselines), the census rerun on the new tree, the phase-log entry + the 🛑 headline (R101), one commit of `src/` + the ledger/batch record/logs + the census + the log (R42). Stops on the first red with the batch's files in place (inspect, then `delever --restore`); stops cleanly when nothing is drawable. Size a call so a batch stays under ~10 min. |
 | | `tools/verbatim_target_s.py --gas` (S99 amendment) | The gas listing is now VERIFIED before it is emitted: it is assembled, disassembled and compared word by word with the ROM image, every instruction that does not reproduce its word is replaced by `.word 0x…` with the mnemonic kept in the comment, and a listing that still disagrees is REFUSED. The class this catches: objdump prints the pseudo-instruction `move` for `addu rX,rY,$zero` and gas assembles `move` as `or` — 24 wrong words in one 234-instruction function, silently. Words carrying a relocation (jal/j, HI16/LO16) are excluded, since the linker fills those. NOTE for any consumer: the listing's %hi/%lo pairs are RESOLVED (no relocation), so an object assembled from it cannot be compared reloc-for-reloc against a compiled candidate — that is why the permuter's target is now the compiled body, not this file (see `tools/delever_permute.py`). |

tools/lever_census.py

@@ -415,11 +415,26 @@ def global_asm_macros(files):
 
 
 PRELUDE = "src/shared/engine_prelude.h"
+_TREE_NAMES = None
+
+
+def tree_asm_macros():
+    """`global_asm_macros` over the census's whole file set, cached per process — the table every caller OUTSIDE the census needs.
+    S103: `delever --scrub` walked `src/shared/ov/func_80165CA0.h` alone, read its `ENGINE_SHB(v)` launder (defined in the prelude)
+    as no site at all, and deleted a live `!FAKE` marker (the census then reported UNMARKED 1). Same table, same verdict (R34)."""
+    global _TREE_NAMES
+    if _TREE_NAMES is None:
+        aliases, dirs = sc.fleet_and_dirs()
+        tu_aliases, headers, _ = enumerate_files(aliases, dirs)
+        _TREE_NAMES = global_asm_macros(list(tu_aliases) + list(headers))
+    return _TREE_NAMES
 
 
 def walk_file(raw, rel, is_header, global_names=None):
     """One file. Returns dict(sites=[...], defs=[...], macro_defs=[...], tokens=dict(raw, live, macro_block, comment_dead, unclassified),
-    unclassified=[...])."""
+    unclassified=[...]). `global_names=None` means the TREE's cross-file macro table (`tree_asm_macros`) — never an empty one."""
+    if global_names is None:
+        global_names = tree_asm_macros()
     masked = sc.mask_text(raw)
     raw_lines = raw.split("\n")
     m_lines = masked.split("\n")