## Tooling inventory

Every script under `tools/` (plus the two report make-targets), grouped by purpose — one line each. Deep HOW-TO is **not** here: see `docs/matching-cookbook.md` (§6 per-module -O0, §8 rodata island, §9.1–§9.5 library linking) and the relevant PhaseEnd.

| Group | Member | One-line purpose |
|---|---|---|
| **MCP lifecycle** | `tools/ghidra_mcp_start.sh` | Spawn the headless MCP server detached → `.run/ghidra-mcp.log`, port 8080 (§2.8). |
| | `tools/ghidra_mcp_stop.sh` | Clean save+close via the `.run/mcp-stop.req` sentinel — the only persistence event; never SIGKILL (§2.8). |
| | `tools/ghidra_mcp_verify.sh` | Read-only persistence re-check `<addr> <name>` after a clean stop (R9). |
| | `tools/ghidra_scripts/BfmMcpServer.java` | The headless MCP server itself (holds an open transaction while serving). |
| **Ghidra headless scripts** (`tools/ghidra_scripts/`) | `ImportPsyqGdt.java` | Resolve `psyq*.gdt` types into the program DTM headlessly (§2.5 step 5). |
| | `ExportSymbols.java` | Dump curated symbols (feeds `config/symbols.us.txt`, R15). |
| | `DumpProgramInfo.java` | Dump program metadata (loader, language, ImageBase, function count). |
| | `DumpFunctionSignatures.java` | Dump function signatures (feeds `make sig-refresh`). |
| | `ImportOverlay.java` | **RETIRED (S45, R33)** — 1-overlay-era hardcoded import; `tools/ghidra_import_raw.sh` is the live path. |
| | `VerifyOverlay.java` | **RETIRED (S45, R33)** — companion of ImportOverlay.java; retired with it. |
| | `GetSymbolAt.java` | Read the symbol at a given address (scripted lookup). |
| | `DecompileAt.java` | Decompile the function at a given address (scripted scaffold). |
| | `DefineFunctions.java` | Disassemble + create functions at splat's validated entry points (`.run/<prog>_funcs.txt`) — completes a raw-blob program's function set (Phase 10). |
| | `ApplySymbols.java` + `tools/ghidra_apply_symbols.sh` | **(P31 S78) The Ghidra MIRROR of the curated symbol file (R15/G6), headless with a real save.** `tools/ghidra_apply_symbols.sh [PROG] [symbols files…]` (defaults `SLUS_007.26 config/symbols.us.txt`; MCP must be STOPPED first) reads `name = 0xADDR;` rows and sets every function/label to its curated name; a name held by another address is moved to that address's own curated name first (`firstfile`/`firstfile2`), else to `<name>__at_<addr>`. Idempotent; prints `BFMAPPLY renamed_funcs=… unchanged=…`; R9-verify with `ghidra_mcp_verify.sh`. **Use this, not MCP `rename_symbol`/`batch_rename`, for renames:** S78 observed 47 MCP renames NOT persisting through the sentinel stop ("Save succeeded", DB grew, names gone — R9 caught it; cause not yet isolated), while the postScript path persisted 73/73 on the first run. |
| **Public flip / CI** | `.github/workflows/no-rom.yml` | **(P33 B7) The ROM-free CI**: job `audits` (audit_public, audit_text_sources, verbatim_check --strict, cookbook_index --check (retired from `make tools-health` at P38 T1: its sources moved to `docs/retired/`; successor `cookbook/INDEX.md` via `tools/cookbook_add.sh`), ghidra_roster --check, work_evidence --selftest, test_lzss, lint_symbol_refs — ≈45 s of checks) + job `compile-only` (binutils-mipsel + `cpp-mipsel-linux-gnu` from apt, cc1 from the tracked tarball sha256-checked, maspsx submodule; PR scope `main resident ov_SC01_077 md_MAIN_013`; `--all` weekly Mon 06:17 UTC + `workflow_dispatch`). Byte-identity is NOT proven in CI (needs the disc) — `docs/verification.md`. |
| | `tools/timeline.py [--check]` | **(P33 F1)** The progress timeline from the repository's own committed digests: every commit touching `docs/progress.fleet.md` / `docs/progress.md` (both historical formats parsed), one row per DATE (never keyed by hash — the rewrite changed them), PhaseEnd ticks from the headers, commits per day → `docs/story-timeline.md` + `docs/story-timeline.svg` (three polylines, phase ticks, the 07-22 denominator step annotated from the rows); self-check: the last row == `docs/progress.json`; `--check` = stale detection. 72 rows, 1.5 s. **Wired P33.5 task 7:** regenerated by `make report BINARY=main` (after `progress.py --json`), asserted fresh by `make audit-digest` (it had been wired to nothing and sat stale at the Phase-33 close). |
| | `tools/mine_hindsight.py [--out …]` | **(P33 F2)** Gathers every recorded hindsight with `file:line` anchors — the decision-log's `Hindsight` bullets and `### Hindsight` sections (19 over 79 entries), the PhaseEnds' "What we believed…" sections (2), every Deviations table (237 rows / 32 PhaseEnds) → `.run/P33/hindsight.md` (scratch; `docs/retrospective.md` cites the sources, never the working set). Prints the census. |
| **Docs** | `tools/doc_links.py [--strict] [--disk] [FILE…]` | **(P33 D5; extended P33.5 task 7)** Six checks, each printed with its denominator: (1) every relative Markdown link in the public-facing docs (the DEFAULT set + every wiki page and how-to chapter) resolves; targets listed in `docs/doc_links_pending.txt` (`path<TAB>creating task`) count as PENDING, not broken — `--strict` (gate 2) refuses any pending entry; (2) nothing links into `docs/sunset/`; (3) a wiki page/chapter links into `docs/` only at a target the Reference index or the README links (the allow-list is DERIVED from those two pages; the two index pages are exempt); (4) every tracked `docs/` file outside wiki/how-to/sunset is covered the same way; (5) backticked `docs/…` / `.run/…` citations are TRACKED or UNTRACKED by `git ls-files` (never the disk) — a wiki page citing an UNTRACKED path fails, elsewhere it is counted; `--disk` adds the PRIVATE/DANGLING split for the maintainer; (6) wiki-first WARNINGS (exit 0) for a non-wiki document linking a `docs/` file whose topic has a wiki page. In `tools-health`. Controls: a broken link → rc 1; the P33.5 cookbook control (`--disk docs/matching-cookbook.md` listed the stale `.run/` citations before they were fixed). |
| | `tools/gitignore_template_check.py` | **(P33.5 task 7)** The ```` ```gitignore ```` fence of `docs/wiki/The-ROM-firewall.md` must equal `decomp-architect/templates/gitignore.decomp` byte for byte (one source, two copies; R75-shaped). rc 1 on drift, rc 2 when the template does not exist yet ("nothing to compare" — never a pass, R43); refuses a page with ≠ 1 fence. In `tools-health` behind an existence test that skips loudly until the kit lands (task 11). |
| | `tools/kit_lint.py [--selftest] [--paths …]` | **(P33.5 task 11)** The day-one decomp kit (`decomp-architect/`) stays de-specialised: (1) LEAK — no line outside a ```` ```calibration ```` fence and off a `provenance:` line matches `SLUS|Musashi|BFM|Druthulu|func_80|ov_SC|/home/musashi|/mnt/z|172\.17\.|\bR[0-9]{1,2}\b|§[0-9]+` (fence-aware: `grep -v calibration` would drop only lines containing the word); (2) PLACEHOLDERS — the `{{NAME}}` set used under the package equals the backticked set in `templates/PLACEHOLDERS.md` (the contract); (3) SYNTAX — `bash -n` / `py_compile` (no bytecode written) / JSON+YAML parse; (4) the gitignore template diff (delegated); (5) `TODO(platform)` / `TODO(phase-N)` counts; (6) coverage — zero files is a failure. rc 1 findings, rc 2 package absent (R43). `--selftest` = the R39 control (a planted leak line + a planted unlisted placeholder must be caught, fenced and provenance lines must not). In `tools-health` (selftest, then the real run). |
| | `tools/tool_census.py [--check | --manifest | --corpus | --all | --consumers FILE]` | **(P33.5 task 13.5)** The tools audit as a derived instrument: two independent enumerations of every tool file under `tools/` (`find` vs `git ls-files`, submodules/vendored/downloaded excluded — they must agree, R34); per tool the docstring line, its SETUP row, its CONSUMERS (Makefile/`.mk` targets, CI, the wave playbook, other tools by import or by name) and hence its class (LIVE · REFERENCED · ORPHAN); the AUTHORED facts live in `config/tool_dictionary.tsv` (phase · portability · the NEED the tool answers · what · what it hard-codes · the retirement verdict with its successor or product) with coverage asserted BOTH ways (R32 — a new tool without a row fails `--check`). Generates `docs/tool-index.md` (the need-keyed dictionary; KEEP-GEN), the kit's `tools/MANIFEST.md` (`--manifest`) and the three verbatim corpora `decomp-architect/corpus/tools/<phase>/` + `corpus/cookbook/` + `corpus/record/` (the how-to, the decision log, the accelerators, the retrospective, the story, the playbook, the effort map, the gen3 docs, the digest, every PhaseEnd — task 14.5) (`--corpus`; superseded tools as pointer files; sha1-equal to their sources). `--check` in `tools-health`; `make kit-corpus` = `--all`. `--consumers FILE` is the referrer census before any `git mv` of a tool. |
| | `tools/share_census.py` | **(P35 T1)** The fleet-wide census of byte-identical function classes and the S1 "one source per unique function" checker (`--check`, `--selftest`, `--scope`, `--strict-macros`, `--strict-text`); its ledger is `config/dedup_exceptions.tsv`; details in the P35 T1 section below. |
| | `tools/lever_census.py` | **(P36 T1)** The census of every compiler-forcing construct ("lever") in the fleet's C — register pins, asm statements by kind (barrier / launder / keepalive / instruction / GTE / verbatim-body), volatile levers, bare `register`, plus the deferred asm-label aliases, builtins and attributes — derived from `share_census`'s scanner with a per-token coverage assertion against the raw text, four known-true controls, the `// !FAKE:` marker split, `--check` (0 unmarked pins/asm AND 0 orphan markers — a `// !FAKE:` line with no pin/asm site on it nor on the line below) and `--check --strict` (0 pins, 0 asm outside the GTE header) gates, cross-file macro names (a use of a macro defined in another file — the prelude's `ENGINE_SHB`, a unit's `SHB` inside an included header, the GTE header's names — is classed by the majority definition's kind; it was invisible before), the JSON's `head` + stat-based `src_stamp` (delever refuses a census that does not describe the tree) and a per-file walk cache keyed on the walker's own hash (a tool change invalidates it, R35), `--sites` (every site to `.run/P36/census/lever_sites.jsonl`, the delever ledger's input), `--selftest`. Evidence: `.run/P36/census/lever_census.{json,txt}`; `progress.py` publishes the `levers` block and a README sentence from it. **(P36 T8, S105)** `--selftest` + `--check -j 16 --quiet` are a `make tools-health` rung (after `verbatim_check --strict`, before `report`): 0 UNMARKED pins/asm and 0 orphan markers on every health run; `--strict` (0 pins, 0 asm) is the STRUCTS phase's finish line by Drew's S104 amendment, not a rung. |
| | `tools/delever.py --recipes` | **(P36 T6, rung R)** `tools/delever.py --recipes --label L [--only …] [--limit N] [--control 8]`: the cookbook's byte-neutral SHAPE recipes tried mechanically on every RESIDUE body, seeded with that body's lever-free text — R2 the formerly-pinned declarations permuted among their own lines, R4 one of them moved to every other slot of the body's declaration run (§76/§501-R: the allocation order is the bank), R3 an initializer split into a declaration and a first assignment placed after the WHOLE declaration run (C89), deduplicated and capped at 40 candidates; the first candidate whose object is IDENTICAL replaces the body and its `// !FAKE:` markers are scrubbed within its own line span only. The control runs first (R39) on the drawn bodies: the identity permutation through the same splice code must reproduce its input text exactly (a text assertion, no compile) and the untouched file must still judge IDENTICAL. Scope, stated: only a body with a formerly-PINNED declaration has candidates. |
| | `tools/delever_oracle.py` | **(P36 T2)** The fast byte oracle for a single-translation-unit edit: `--recipes` captures every object's exact build command through `make -n -W <src> <obj> BINARY=<alias>` (the Makefile's own pipeline — the jump-table pad stage, the per-object `-O0` overrides, the twin rule — into `.run/P36/delever/recipes.json`, regenerable in ~15 s); a candidate is compiled IN PLACE with `-o`/`-MF` redirected to scratch and its bytes compared with `build/`'s object from the fleet run (build/ is never written); `--calibrate <aliases>` proves 100 % equality untouched + twin == primary + a positive control (a nop injected at the end of a real body → DIFFERS) and writes `calibration.json` (HEAD, config stamp, per-object seconds); `--status`. Measured at T2: main 0.08–0.77 s per object, overlays ~0.14 s. |
| | `tools/delever.py` | **(P36 T2/T3)** The de-lever engine AND the campaign tool. Rewrites per lever class on the raw text at the census's positions (a token mismatch REFUSES, never guesses): pin → plain declaration (type/qualifiers/initializer kept; the `$0` zero-register variable's uses → 0, refused if it is ever assigned), barrier / keep-alive → deleted, launder → deleted when it launders a value into itself, **an ASSIGNMENT `out = in;` when its output and input differ** (deleting those made cc1 2.7.2 abort — the T2 probe counted them NEEDED), a hand-placed instruction → its C (addu-$zero/move/la/lh/lw/addiu/sll/srl/and/andi/lui/li and the lui+addiu / lui+ori pairs), a macro-carried site → deleted for a pure launder statement macro (`SHB`), its value `((T)(p))` for a launder statement-expression, REFUSED for a compound macro (XFER/DRAW/RTP_SND: the lever is in the `#define`, T5), volatile / register → dropped. Per body: replay (a ledger exemplar with the same normalized text: 1 compile) → rung A strip-all → rung B greedy, through `delever_oracle` (verdicts IDENTICAL / DIFFERS / COMPILE-ERROR / COMPILE-CRASH). **The file is the write unit and its final compile the proof**: every body's accepted edits + the file-scope volatile edits + the `// !FAKE:` markers spliced once, compiled through every recipe of the file (a twin's object; every includer of a header, in parallel) — IDENTICAL or the file-scope edits are dropped, or the file is restored and REFUSED (COMBINATION-FAILED). `--plan` / `--apply --batch N --label L [--headers] [--only …] [-j 12]` (TUs in parallel, exemplar files before their copies; headers serial; the tree must be clean, the calibration current, the census's src stamp the tree's — it reruns the census itself), `--redraw REFUSED NOTHING-USABLE` (draw again the bodies whose latest ledger verdict is one of these — after a tool fix), `--restore` (from `inflight.json`, the only restore — R102; it also drops the in-flight batch's ledger rows into an ignored `ledger.jsonl.killed_<label>`, so a killed batch leaves no trace and its bodies are drawn again), `--status`, `--scrub [--only …]` (remove the census's ORPHAN markers — a `// !FAKE:` whose site is gone — each file judged through every recipe; a removed or rewritten site consumes its own trailing marker, so orphans arise only from older tool versions; **S104:** every `lever_census.walk_file` call outside the census now defaults to the tree's cross-file asm-macro table, `lever_census.tree_asm_macros()` (~24 s once per process) — S103's scrub walked a shared header alone, read its prelude-defined `ENGINE_SHB` launder as no site, and deleted a live marker), `--apply-body TU FN FILE --label L [--rung E]` (T6/T7: a reshaped body judged on the bytes, refused if a class A/B lever remains), `--selftest` (the fixture through the census's own parser: every rewrite, the ladder against a stub oracle with a crashing site, the markers, a replay and a disagreeing replay, the ledger index, the oracle's crash forms), `--probe` (T2). The ledger `.run/P36/delever/ledger.jsonl`: one row per (tu, fn, addr, aliases) per judgement with every site's verdict REMOVED / REWRITTEN / NEEDED(oracle word) / REFUSED(why) / DEFERRED, the rung, the calibration id, the body's nhash BEFORE and AFTER — "done" = the current nhash is some row's after-hash; a body with a row's before-hash replays it. Class C/D survivors are never marked (decision 3); class A/B survivors carry `// !FAKE: <kind> <detail> — <verdict> (P36 <rung> <label>)` at the end of their line. Evidence: `apply_<label>.log`, `batch_<label>.json`. Per-batch setup: the per-file include lists are cached in the ignored `includers_cache.json` on (mtime, size) — 29 s → 1.3 s, proven equal to the uncached map. |
| | `tools/gte_consolidate.py` | **(P36 T5)** The GTE consolidation. `--inventory`: every asm-bearing `#define` and every direct GTE statement signed by the build's own tail (`maspsx --aspsx-version=2.56` → `mipsel-linux-gnu-as` with `include/macro.inc` + `gte_macros.inc`, where `rtps`/`mvmva`/`sqr 0` are GAS macros): signature = (bytes with %k bound to $4–$7 / $2–$3, #outputs, #inputs); `nop;nop;rtps`, `.word 0x4a180001` and `cop2 0x0180001` are one signature. The canonical table (`.run/P36/gte/canonical.json`): one text per signature (the majority body — byte-proven in the fleet), Sony's name when the operand counts agree with `inline_c.h` (Sony's converted header is unreliable for opcode words: `gte_rtps` there is `.word 0x0000007f`), Sony's clobber set canonical even when every definition carries the steer (the body is synthesized with Sony's clobbers); a definition with the canonical bytes and OTHER clobbers is a lever variant `<name>_m` / `<name>_v<hash>`. `--header` writes `include/gte_inline.h` (50 macros at T5) and the include in `common.h`. `--apply --batch N --label L`: per file, canonical duplicates deleted, private names renamed to the canonical, each variant TRIED as canonical (the clobber dropped, judged) and kept as a marked `_m` lever only when the object differs, direct statements rewritten into canonical calls (a concatenation of two canonical macros included; a direct statement with extra clobbers is marked as a lever; an unmatched sequence is counted); the file judged through every recipe, the definition edits alone retried when the direct rewrites differ. `--sweep`: dead asm-bearing macros deleted — a use (or an `#include` of a header using the name) is counted for the GOVERNING definition, the last one above it, so a used sibling definition keeps no dead one alive; a compound macro's inner asm dropped when byte-neutral. `--remark [--dirty-ok]`: the census's UNMARKED gte-lever sites marked on their own (first) lines after a `delever --scrub`. The selftest never writes the real table (`canonical_table(write=False)`) — it once did, and the census then classified against a fixture. `tools/delever_cycle.sh … gte` runs the apply batches. The census classes a variant's uses and a direct statement with clobbers beyond the canonical's as `gte-lever` — a class-B lever INSIDE the phase's number from T5 on (2026-09-09; the headline's definition gained the clause), marked, 0 at the close — and counts per-TU asm macro definitions. |
| | `tools/delever_cycle.sh` | **(P36 T3)** The unattended batch cycle: `[LABEL_PREFIX=…] [TASK=T4] [REDRAW="REFUSED NOTHING-USABLE"] tools/delever_cycle.sh START END [BATCH=300] [tus|headers] [ONLY="alias …"]` — per batch: the tree must be clean (a dotfile under `src/` refuses), the oracle recalibrated when stale (2–3 s; every commit stales it), `delever --apply` (exit code + its `final X/X identical` — an empty verify line is a failure, R32), the CLEAN fleet run as the outer gate (`218 passed, 0 failed of 218`, R22 — it also refreshes `build/`, the oracle's baselines), the census rerun on the new tree, the phase-log entry + the 🛑 headline (R101), one commit of `src/` + the ledger/batch record/logs + the census + the log (R42). Stops on the first red with the batch's files in place (inspect, then `delever --restore`); stops cleanly when nothing is drawable. Size a call so a batch stays under ~10 min. |
| | `tools/verbatim_target_s.py --gas` (S99 amendment) | The gas listing is now VERIFIED before it is emitted: it is assembled, disassembled and compared word by word with the ROM image, every instruction that does not reproduce its word is replaced by `.word 0x…` with the mnemonic kept in the comment, and a listing that still disagrees is REFUSED. The class this catches: objdump prints the pseudo-instruction `move` for `addu rX,rY,$zero` and gas assembles `move` as `or` — 24 wrong words in one 234-instruction function, silently. Words carrying a relocation (jal/j, HI16/LO16) are excluded, since the linker fills those. NOTE for any consumer: the listing's %hi/%lo pairs are RESOLVED (no relocation), so an object assembled from it cannot be compared reloc-for-reloc against a compiled candidate — that is why the permuter's target is now the compiled body, not this file (see `tools/delever_permute.py`). |
| | `tools/lever_progress.py` | **(P36, the record)** The lever series behind `docs/levers.md`: `--snapshot "<task>"` appends one milestone row (the census's totals by class + the tree's HEAD) to `docs/lever-progress.tsv` and re-renders the document's generated block; `--render` rebuilds that block (the campaign half is derived from the de-lever ledger every time, scored as state TRANSITIONS so the rung that finishes a body gets the credit, not only the rung that first judged it); `--check` refuses a series whose last row is not this tree (a stale series is a wrong chart). Run it at the close of every task that changes the count, next to `lever_census --check`. |
| | `tools/delever_permute.py` | **(P36 T6, rung D)** The permuter on the residue: `--plan` lists one exemplar per RESIDUE text class of the delever ledger (copies desc, then fewest NEEDED sites — a match banks every copy); `--prepare TU FN` builds that exemplar's scratch `.run/P36/permuter/<alias>__<fn>/` (R48: a function NAME repeats across overlapping overlays) — `tu.c` the lever-free TU (delever's own rung-A rewrite of every REMOVABLE site; a REFUSED site makes the exemplar UNSTRIPPABLE), `iso.c` the same with every OTHER definition reduced to a prototype, shared-header includes replaced by the prototypes they define, `INCLUDE_ASM`/`INCLUDE_RODATA` and file-scope asm statements dropped, `draft.c` that through `cpp -P` with the Makefile's own CPPFLAGS + `-I<the TU's dir>` + `-D__attribute__(x)=` (pycparser rejects `__attribute__` and decomp-permuter then REFUSES base.c and permutes nothing), `levered.c` the same pipeline with the levers KEPT, and the target in both forms — `gas/<fn>.s` (`verbatim_target_s --gas`, the only assemblable one: `mipsel-as` refuses the splat listing's bare `addiu sp,sp,-152` exactly as decomp.me did, R98) for `p16_permute.setup`, `splat/<fn>.s` for `match_one`. `--positive-control TU FN` perturbs a MATCHING body by one commutative swap and requires the permuter to find its way back to score 0 — the control that tells a hard population from a broken scorer (S99: two campaigns returned 0 of 16 against a target assembled from a listing, whose base score for the tree's own body was 28, not 0). `--calibrate [--limit N]` and every `--run` attempt: the LEVERED body must be `match_one` MATCH against the regenerated target (R39/R56 — the harness must agree with the tree before it may judge a candidate; 12 of 12 at S99), then the lever-free body's distance is recorded as the search's starting point (min 8 / median 78 / max 276 over those 12; a removed hand-placed `instruction` changes the instruction COUNT and shifts everything after it — `--max-start N` triages those as FAR with their number). `--run [--limit K] [--workers W] [--secs S] [--cycles C] [-j J]` runs `permuter_ils.py` per exemplar (profile from the NEEDED kinds: pins → regalloc, barriers/launders/keep-alives → schedule), `--winners`/`--pd` pointed at the scratch; every attempt appended to `.run/P36/permuter/outcomes.jsonl` (also the skip list; `--include-done` redraws). `--bank [--label dN]` puts each winner's definition back through `delever --apply-body … --rung D` (which refuses a body still carrying a class A/B lever and judges the real object through every recipe) and then `gte_consolidate --apply --rejudge` to re-fold the cpp-expanded GTE asm; serial, because each step runs `make`. Verdicts distinguish MATCH / NO-MATCH / FAR / UNSTRIPPABLE / UNCALIBRATED / SETUP-FAILED / ABORTED / NOT-JUDGED (R61: a run that never iterated is not a no-match). Run the campaign DETACHED (`setsid nohup … &` + a `Monitor`), never as a harness background task. |
| | `tools/delever_search.py` | **(P36 S101, rung G — the guided search)** Rung R tests ~57 one-move candidates per body for IDENTICAL and learns nothing from a miss (0 of 300 where no shape was known, §454a); rung D discovers but reprints the source. This engine keeps rung R's generators (`delever.recipe_candidates`, now with a `families` filter and two new moves: R8 a temp introduced/hoisted — R6's inverse — and R9 two adjacent statements swapped; R7 gained its own inverse, the unwrap) and the per-TU oracle, and SCORES every candidate: the function's instructions are read from the scratch object (`objdump -drz`, 35 ms on the largest object) and compared with the same function in the fleet run's baseline object under `build/` — the tree's own bytes, carrying the candidates' relocations by construction, so no listing is assembled and nothing is isolated (the two instrument classes of §454). The score is an EDIT DISTANCE over the reloc-masked words (a positional count read one inlined temp as 43 shifted words; difflib reads the real delta), the residual is classified from the diff blocks (REG on the caller- or callee-saved bank from the register pairs, COUNT, ORDER, MIXED) and the class picks the move families, ranked round-robin (a strict family order starved the inverse of a one-move perturbation behind 64 block wraps). The search is a beam (`--beam 3 --depth 3 --cap 48 --budget 400`): a child worse than its parent is dropped, the first score-0 is verified on every recipe of the file and banked through `delever --apply-body --rung G`, siblings by `--propagate` serially after the parallel phase. `--plan [--score]` lists the residue's exemplars (largest class first) with their starting distance and class; `--run [--limit K] [-j W] [--label gN]`; `--positive-control TU FN [--moves 1\|2]` perturbs a matching body by generator moves and requires the search to return to 0 without writing the tree (S101: one move back in 23 compiles; inline+wrap back in 74 by hoist+unwrap; an inlined pointer temp under a dereference has no inverse generator yet and stalls at 10 — the measurable stall mode); `--explain TU FN [--path "m1|m2"]` reads one body's residual as mnemonic blocks (mine vs the target) after any move path — the instrument that turns a stall into a generator (S101: the two 6-distance bodies are one surviving copy; a 7 is an association order plus a negation named once; a 40 is a duplicated call tail); `--selftest` (the classifier on synthetic streams, the beam on a stub needing three composed moves, the generators on fixtures). The registry's later moves, each from lane B's map of the compiler source and each with its selftest: R10 a parameter routed through a body-local copy and the reverse (map 1a-9), R12 a local's scalar width (1c-1 — verified NOT to reach a copy whose value's known bits fit the narrow mode), R13 two terms of a `+`/`-` chain exchanged, R8's third form a repeated RHS named once; a constant-operand R5 swap is never generated (fold moves it right — verified on bytes, `.run/P36/engine/micro/`), and the `do { } while (0)` lever works through the ref weight (`.run/P36/engine/micro/dowhile/`: `.lreg` `used 5` → `used 6 times`, `$18` → `$17`). Second round (S101, from `--explain` on the thirteen small residuals g3 left unmoved): R12 now covers `u8` and the `short`/`int`/`char` spellings; **R14** a PARAMETER's declared width in the header (**the original rationale — "a `short` parameter is sign-extended in place, `sra a1,a1,16; move s4,a1`, where a cast at the use extends into the destination" — was CORRECTED ON BYTES at P36 S102 by agent a15: MIPS in gcc 2.7.2 defines only `PROMOTE_PROTOTYPES`, NOT `PROMOTE_MODE` (`config/mips/mips.h:1153`), so a narrowed parameter stays a HImode pseudo and the extension still happens at the USE — the agent compiled `func_80166F58` both ways and the K&R form too, and all three give the same 67-instruction merged form. R14's banked closes elsewhere stand on their bytes, not on this rationale; and since S102 the generator refuses any function the TU declares outside its definition, because the bank is body-only**); R8's fourth form one address local shared by every dereference of one base, stores included (lane B 2-6 and 2-12: a store through a bare pointer flushes cse's memory table and a global is re-loaded); R10's alias form through casts (`src = (u8 *)((u32)param_2)`). The bank and the propagation run IN PROCESS (`delever.apply_body_core`, `delever.propagate` returning `(banked, n, refused)`): run g3's 1,503 siblings had cost ≈40 min as a subprocess each, and a `delever.py` edit during a run could break a bank mid-run — now the running process holds its own copy. Measured runs: g1 1/16 (0.12 h), g2 1/16 (0.32 h, the wider beam), **g3 13/64 (1.02 h, 1,516 bodies; by first family R12 ×5, R7 ×3, R9 ×3, R10 ×2 — lane B's width and parameter moves half the closes)**. Fourth round: R14 rewrites the TU's prototypes with the header (else every candidate is a conflicting-types error); R8's named-once form also names a repeated depth-0 operand or parenthesised group; R12 splits a multi-declarator line to widen one name (`MULTI_DECL` also ends neither the declaration run nor is offered as a statement). Draw rules: a body whose NEEDED sites are all class C/D (a byte-needed `volatile` cast, a bare `register`) is DONE by decision 3 and is not drawn, and the seed keeps such sites; the scorer's scratch object is keyed by the TU (a per-tag name let two workers share one file — a byte-identical body read as 14,871 mismatches). g4s 3/38 (the thirteen explained names across the fleet, 133 bodies). Every attempt in `.run/P36/engine/outcomes.jsonl`, every scored candidate in `.run/P36/engine/trace/<alias>__<fn>.jsonl` — the byte record lane B's compiler-source hypotheses are checked against. A worker owns a TU; headers serial; `--dirty-ok` for a run that banks several bodies before one commit; run a campaign DETACHED with `nice`. `delever.py --repair-nhash` filled the 301 rung-R RESIDUE rows that carried no text hash (S99/S100 sweeps) from their bodies' earlier rows — the cause fixed in `recipes()`. **R15, the sink (P36 S102, the first T7 agent's crack toolified):** the statement AFTER an if/else chain pushed into every arm and the variables it consumed deleted — `if (c) { v = e1; } else { v = e2; } w = f(v);` becomes `if (c) { w = f(e1); } else { w = f(e2); }`. It is a REGISTER move, not a scheduling one: a value set in every arm and read after the merge is a CROSS-BLOCK pseudo local-alloc never gives a quantity to (`local-alloc.c:472`, `next_qty` reset per block at `:517`), so each arm holds two quantities and takes `block_alloc`'s unrolled `case 2` (`:1499-1502`, one `qty_compare` `:1578-1596`, higher density first); sinking makes it a third block-local quantity and `case 3` (`:1491-1496`) FALLS THROUGH into `case 2`, applying that comparison a second time and undoing its own exchange, so the two caller-saved colours swap — and while it is a global allocno it can inherit a copy preference from whatever the merge result is passed to (`set_preference` `global.c:1535+`, merged by `expand_preferences` `global.c:781-825`, overriding first-fit at `:1034-1067`), which sinking removes with it. Applicability is CHECKED: every consumed variable must be assigned exactly once in every arm by a simple statement, appear in the merge statement, and occur nowhere else in the function. `if_chains()` walks the arms counting a line's CLOSING braces before its opening ones — on a `} else if (…) {` line the two net to zero and a naive depth counter never closes the arm (the defect the generator's first run had). Ranked third in REG-caller and REG-mixed, third in COUNT; three selftest controls (a variable read after the merge, a variable one arm does not set, the brace walk's three arms). Its known-true check: run on `func_80156044`'s own pre-bank text it reproduces the agent's crack and scores 0. **R16 the constant holder and R17 the constant-run split (P36 S102, the second T7 agent's crack toolified):** R16 writes a local whose ONLY assignment is one integer literal at every use and deletes it — R6 stops at a temp read exactly ONCE, so a holder read four times was invisible to the search and the whole family with it; deleting it is byte-neutral by itself but removes a quantity from the block, which is what lets the next move reach the allocator. R17 splits a run of consecutive statements assigning the same integer literal by moving the nearest differently-valued literal assignment into it, at each interior split point: the decision is `find_free_reg`'s live-range scan (`local-alloc.c:2109-2110`) — while the two constants' ranges are disjoint they share one caller-saved register, and splitting the run makes the first live across the second so it takes another colour; the discriminator in the dumps is the `.lreg` line `Register N used K times across M insns`, whose M grows when the split lands. R17 is the DIRECTED form of a move R9 contains but reaches by luck (agent a2 measured 2,271 compiles for R9 in `func_80168828`; **R16 + R17 reproduce that close in ten**, the known-true check, on the seed that keeps the body's semantically-forced `$4` pin). Ranked: R16 second in REG-caller/REG-mixed and second in COUNT, R17 FIRST in ORDER. The engine selftest's caller-saved assertion is now the ORDERING INVARIANT — every targeted lever (R5, R15, R16) before every blind family (R9, R2, R4) — instead of a fixed window widened once per generator. `--propagate` inherits the exemplar's lever allowance, derived from the count of surviving `// !FAKE:` markers in its own banked text, and refuses by name any sibling whose remap would carry more (agent a2's reshape keeps one pin: 124 of 124 siblings banked once this existed). **R18 the bystander move (P36 S102, from the third T7 agent — the one that did NOT close its body):** one simple statement moved to each other position inside its own brace block, up to six statements away; R9's adjacent swap is its special case and the DISTANCE is the point. MIPS declares no `REG_ALLOC_ORDER`, so `find_free_reg` (`local-alloc.c:2158`) hands out the lowest free regno and the winner is whichever quantity `qty_compare` (`:1579`) reaches first, with `combine_regs` (`:1722`) welding an index chain into one quantity that competes with a short-lived value; moving a statement that depends on NEITHER contender into or out of the interval between the chain's birth and its first consumer changes `reg_live_length` and flips that race at ZERO instruction cost — unlike `R7 do-while`, whose LOOP notes are a full `sched1` barrier (`sched.c:2058-2074`) and always cost one displaced insn (the agent enumerated all 60 wrap ranges in its body and every one paid exactly that), which is why R18 is ranked ahead of R7 in every class. Two spellings were wrong before the known-true check passed: identifier-disjointness as a REQUIREMENT offered three candidates and none of them the agent's (it is now only an ordering preference — byte-identical output is the same program, so the oracle is the whole correctness proof, the footing R9 has always stood on), and a BLANK line counted as an obstacle, so the generator never offered the very move it was written from. It now reproduces that body exactly (`bystander @21->17`). `delever_pack.py` writes each trace candidate's residual CLASS beside its score, because a bare number hid from that agent that a move had already turned its body's residual from REG into ORDER. |
| | `tools/delever_search.py --try TU FN FILE [--body]` + `tools/delever_pack.py` | **(P36 S101, T7 one agent at a time — Drew: efficiency, not wall-clock; hone the method after each lands.)** `--try` scores a candidate translation-unit text (or, with `--body`, a function body spliced into the tree's TU) WITHOUT writing the tree: the TU's own recipe is run on a scratch copy (`-I<the TU's directory>` so its relative includes resolve; a header candidate through a shadowing include dir) and the function's instructions are compared with the fleet run's baseline object — the same score, class and mnemonic diff as `--explain`. Proven on a known-true case (the tree's text 0; one pin removed the known residual). It is the loop an agent runs on its own candidate, so any number may run beside a campaign; the bank stays the coordinator's (`delever.apply_body_core` on the real recipe, then `--propagate`). `delever_pack.py --build [--min-copies 100] [--limit N]` writes one pack per residue exemplar under `.run/P36/agents/<alias>__<fn>/` (tu.txt, body_tree.c, body_free.c = the seed, residual.txt from `--try`, sites.txt, history.txt = every engine attempt and the best-scoring moves of the last trace) and `ORDER.tsv` (best distance reached ascending, then copies); `PROMPT.md` beside them is the brief (read the residual → name the pass from `tools/reference/gcc-2.7.2/` + the map → test on bytes with `--try` → deliverables early: `body.c` + `mechanism.md` with a GENERATOR PROPOSAL). `delever --restore` now refuses loudly on an empty or torn inflight.json (a kill mid-write) and says how to reconcile (the oracle: a bank is IDENTICAL, a leftover candidate DIFFERS). |
| | `tools/kit_coverage.py` | **(P33.5 task 14.5)** The kit's DISTILLATION coverage: derives the rule population (every `- **R<n>` of DIGEST §3, asserted contiguous) and the hindsight population (every `## ` heading of `docs/accelerators.md` at numbered-item granularity — 58 at S92) and asserts each is cited by a `provenance:` line of the registry seed / the kernels OR dispositioned in `config/kit_coverage_map.tsv` (`G<id>` / `DK-<n>` / `FOLDED:G<id>` / `ENV` / `PA` / `SEED:<file>` / `KIT:<path>` / `RECORD` / `COOKBOOK` / `NOT-PORTABLE`; unknown ids refused, R43); counts with denominators (R41); rc 1 on any gap. In `tools-health` after `tool_census --check`. Its first run found 26 uncited rules and 21 uncited entries → three new kernels (DK-66–DK-68) and 41 authored dispositions. |
| | `decomp-architect/` (the day-one decomp kit) | **(P33.5 tasks 9–14)** The package a new matching-decomp project installs as Phase 0.5 on ProjectArchitect 2.0: `README.md` (the three steps), `intake.decomp.md` (ProjectArchitect's twelve items pre-answered + the phase ladder + the six readability inversions), `SETUP.md` (the installer, Step 0 contract … Step 10 verify + hard stop; `answers: <path>` for unattended runs), `decomp-architect.md` (the methodology), `templates/` (the firewall pack — `gitignore.decomp`, `firewall.txt`, `audit_public.template.py`, the planted fixture, `no-rom.template.yml` —, the READMEs, `pa-overlays.md`, `registry-E.decomp.md` G1–G67, the skeletons, `PLACEHOLDERS.md`, `layout-contract.md`), `corpus/decomp-kernels.md` (DK-1 … DK-80), the three dictionaries `corpus/tools/<phase>/` + `corpus/cookbook/` + `corpus/record/` (generated by `make kit-corpus`), `memory-seed/` (18), `tools/MANIFEST.md` (generated). **How it is checked:** `tools/kit_lint.py` (de-specialisation, placeholders, syntax, the gitignore-template diff) + `tools/tool_census.py --check` (the corpora equal their sources) + `tools/gitignore_template_check.py`, all in `tools-health`; the dry-run harness under `.run/P33.5/kit-dryrun/` (`answers.md`, `expected-manifest.txt`, `judge.py`, the install logs and verdicts of runs 1–5 — a kit change is re-verified by a resume on the last throwaway `repo/`, a fresh full run only when SETUP's steps change; the judge compares the real tree's dirty PATH SETS before/after). Wiki page: `docs/wiki/Start-a-new-decomp-project.md`. Split into its own repository after the flip. |
| **Verification** | `tools/verify_contract.sh` | **(P33 A5/C8) THE recorded contract run**: 00 tree · 01 check-env · 02 family_hseq · 03 `make clean && extract-all && check-all` · 04 sdk-dual (or a recorded SKIP) · 05 tools-health (zero `[warn]`) · 06 audit-frontier · 07 audit-disc · 08 report; one log per step ending `EXIT=<rc>`, abort on the first red (R53), every step asserted by its contract line (R49), `SUMMARY.md` generated → `.run/P33/verify/` (tracked evidence, quoted by `docs/verification.md` §2); step 00 ignores its own output dir. ≈14 min on 32 CPUs. |
| **Publishing** | `tools/progress.py --json \| --readme [--check]` | **(P33 D1/D3)** The same numbers as DATA: `--json` → `docs/progress.json` (schema 1: the four metrics with numerator/denominator/pct, the counts, 218 per-binary rows incl. instruction totals; no run date) + `docs/badges/{fleet_instr,fleet_fn,distinct,binaries}.json` (shields endpoint format; the README references `fleet_instr` + `binaries` by name); `--readme` rewrites the README's `<!-- progress:begin/end -->` block (refuses a README without the markers); `--check` asserts JSON + block + badges are fresh (in `make audit-digest`). Run by `make report BINARY=main`. |
| | `tools/wiki_render.py OUT_DIR \| --list \| --selftest` | **(P33 F3)** Render `docs/wiki/*.md` + `docs/how-to-ai-decomp/*.md` into GitHub-wiki page names with every relative link rewritten deterministically (wiki page → its name; a chapter → `How-to-AI-decomp-NN-name`; any other repo path → a `blob/main` / `tree/main` / raw URL; URLs, mailto and anchors untouched; **a dead link is an error**, R43). `--selftest` = the 12-case fixture incl. the dead-link negative control **plus (P33.5 task 7) the reachability assertion: every `docs/wiki/*.md` except `_Sidebar`/`_Footer`/`Home` is linked from `_Sidebar.md`, every chapter from `_Sidebar.md` AND `How-to-AI-decomp.md`** — a published page nobody can navigate to fails here; in `make tools-health`. |
| | `tools/wiki_sync.sh [--push] [--wiki-url URL]` | **(P33 F3)** Render into `.run/wiki/render/` → clone or fast-forward `BFM-decomp.wiki.git` under `.run/wiki/` → REPLACE its pages with the rendered set (the repo is the source of truth; a page edited on GitHub is overwritten) → `git status --short`; **`--push` is Drew's (R6)**: commit + push. Before the wiki repo exists (after the flip AND the first page created in the GitHub UI) the dry run renders and lists (exit 0) and `--push` refuses (exit 2). |
| | `tools/gccmap_cites.py [--dry-run \| --check \| --verify \| --controls \| --explain] [--retag]` | **(P33 E3)** Tag every `file.c:NNN` cite in `docs/gcc-2.7.2-map/*.md` with the source tree its line number belongs to — `[2.7.2]` (the vanilla subset), `[2.8.1 pm]` (gcc-papermario), `[repo]` — derived from the trees (quoted snippets, identifiers with function extents and nearest distance, the author's cues; a contradiction fails loudly; ties → `cite_overrides.tsv` → cue → 2.7.2); writes in place, idempotent, never silently changes an existing tag. `--check` is textual (every cite tagged, no stale override) and runs in `make tools-health` + CI; the others need both reference trees and refuse without them. |
| | `tools/xsig/xsig.py sign-s \| sign-objdump \| cross \| verify \| selftest` | **(P33 E4; Phase-21 origin)** Relocation-masked per-function signatures for CROSS-PROJECT code identification: mask `j`/`jal` targets and HI16/LO16 immediates (from `%hi`/`%lo` operands or `objdump -dr` records), keep opcodes/registers/constants/branches; `cross` joins two JSONL sets on `sig` with a coverage line; `verify` prints the instruction-by-instruction diff (opcode/register/immediate/length). Self-contained (stdlib, MIT, own README + LICENSE + `tests/` from a game-free fixture compiled at two link addresses with `--emit-relocs`); `tests/test_xsig.py` (8) in `make tools-health` + CI. Published: `https://github.com/Druthulu/xsig` (public, pushed S88 at Drew's instruction). |
| | `tools/permuter/upstream/0001-reloc-masked-scorer.patch` | **(P33 E5)** The upstream PR as a `git format-patch` (one commit against `simonlindholm/decomp-permuter` main `41bd0bfc`, 2026-09-05): `src/reloc_scorer.py` (`RelocMaskedScorer`, a `Scorer` subclass), `--score-mode {mnemonic,reloc-masked}` + the `score_mode` settings key, docs, `test/test_reloc_scorer.py` (10, no cross toolchain). Regenerate the branch: clone upstream, `git am` the patch (proven clean). Two commits (the scorer; a USAGE note on restarting from the best candidate), plain contributor style. Submitted 2026-09-07: PR simonlindholm/decomp-permuter#213 (from the fork's `reloc-masked-scorer`), issue #214 (the symbol-regex proposal; `docs/permuter-ils.md` §3). **Outcome:** the maintainer closed PR #213 and issue #214 on 2026-09-07 without merge: `field_matches_any_symbol` is intentional (it covers temporary `.text+0x…` names), a name mismatch between C and asm should be fixed by renaming the symbol in the target asm (splat's symbol files), and a nonzero optimal score is acceptable in the interactive workflow; he also said the issue text read as LLM-generated and felt disrespectful of his time. The scorer stays a local tool. |
| | `docs/matching-drafter-pipeline.md` | **(P33 E6)** The write-up of the fine-tuned local matching drafter (Phases 22–25): the pipeline (`export_pairs` → `format_finetune` → `train_lora` → `eval_lora` → `serve_local` → `api_draft` / `lora_grind` / `bulk_harvest` → the byte gate; `ab_score` + `workflows/ab_match.js`; `grinder`), every measurement in order (stock floor ~0 → v2 85% on 6–15 ins → v3 57.5% held-out and ~352 production banks → v4 negative → the GLM hard-band A/B and the def-side wall), the portable lessons, the hardware, a five-step recipe, and what is NOT published (the ROM-derived pair dataset `datasets/`, the adapter weights `models/` — both gitignored). |
| | `docs/gen3-handoff.md` | **(P33 G1)** The Gen3 seed: where Gen2 ends, the owner's next intent (casts → structs, pins off, names), the starter census with the commands that re-derive it (143 raw address casts · 61,898 `D_` · 16,335 `func_` · 44,243 register pins · 1,083 symbol-file entries · 1,232 struct defs · 2,220 dedup groups · 5 verbatim bodies), the one invariant (every edit byte-gated; shared bodies change every member; types are comprehension not bytes), the inherited levers, shiftability honestly scoped (position-locked slots, LZSS not byte-stable, the Sony regions), the parked ideas, and the governance for a new generation. |
| | `tools/verbatim_target_s.py --binary B --fn F [--out DIR] [--gas] \| --all [--gas]` | **(P31 S75)** Regenerates the splat-format target `.s` (the `/* off vaddr word */` word oracle) for a §265 verbatim body from the EXTRACTED IMAGE — a listing, never assembled by anything here. **(P34 task 2, 2026-09-08) `--gas`** writes `<fn>.gas.s`, the ASSEMBLABLE form: `$`-registers, `.L<addr>` labels for in-function branch/jump targets, symbol names (or `func_<ADDR>`) for external `j`/`jal`, `.set noat` + `.set noreorder` (the listing carries its delay-slot nops), no `.include`; the comment column is kept so the word oracle reads either form. This is what a decomp.me scratch takes as its target (the listing fails there: `invalid operands 'li a2,2'`); proven per function by `decompme_replica.sh` step D. |
| | `tools/decompme_replica.sh` | **(P33 E1)** Runs a function through decomp.me's EXACT PS1 `gcc2.7.2-psx` toolchain locally — old-gcc 0.13 (tarball sha256-checked into `.run/decompme/`) + maspsx `86ccd7d8` (from the submodule's object store) behind the image's two-line `as` wrapper, driven by the backend's own two commands — and compares the code words to the ROM-derived target (`verbatim_target_s.py`); our Makefile pipeline on the same TU is the control (exit 3 if IT fails), then cc1 and maspsx are swapped one at a time so a difference names its producer. Default probe `func_80018F20`; `--src TU --fn NAME [--binary B] [--flags "…"]`; `--upstream` compares decomp.me's current Dockerfile pins with the script's constants. Trailing gas padding is reported, never counted. Exit 0 = the preset reproduces the function byte-identically on decomp.me's toolchain. **(P34 task 2, 2026-09-08)** Two more steps: **D** proves the target in the form Drew PASTES (`verbatim_target_s.py --gas`) through decomp.me's own `as` wrapper (fed on STDIN — a file argument blocks it), linked at the function's address and compared word for word; **E** writes Drew's bundle `.run/decompme/drew_bundle/` (1–4) from the proven run. PASS now needs A AND D (exit 4 = compile matches but the paste does not round-trip). Why: the splat-format listing §5 used to name is the word ORACLE — nothing assembles it — and it failed on decomp.me with `invalid operands 'li a2,2'`. |
| | `docs/decompme-preset.md` | **(P33 E1)** The decomp.me preset (fields, name, flags), why each flag, the measured decomp.me-vs-ours toolchain table (0.13 vs 0.17 cc1; maspsx `86ccd7d8` vs `874855c5` — 4 commits, both behavioural ones gated on aspsx < 2.30), the recorded `decompme_replica.sh` PASS + its two negative controls, and Drew's post-flip browser session (scratch → 100% → the preset-request issue on decompme/decomp.me → the manual search closing ledger row 14). |
| | `docs/outreach/archipelago.md` (local only — untracked at P34 task 6, Drew 2026-09-08, after the note was sent) | **(P33 E2)** The Archipelago outreach: what AP and the BFM world are (v0.8.1, BizHawk, US+JP, 342 polled addresses, runtime-injected MIPS patches), the table of what our source says about their data (the day-of-week byte CONFIRMED and our memory map corrected; their four patch sites and the Time-Sanity hook resolved to shared engine functions; the town-ID patch retired as historic; the version-check string offset), what we ask (their unpublished US RAM notes; attribution consent), the ready-to-send GitHub issue + a Discord short form (plain style; send only after the flip), and the G5 procedure for the reply. |
| | `docs/outreach/tools-announcement.md` (local only — untracked at P34 task 6 after the posts went out) | **(P33 S89)** The post-flip announcement of xsig for the decomp community (Discord tools channel / a Decompedia tools-page row), written in a developer's voice, with the facts it rests on. |
| | `docs/wiki/Tools-from-this-project.md` | **(P33 S89)** The wiki page mirroring the README's "Tools from this project" section: xsig, the permuter driver (upstream as a pinned unmodified submodule + our class swap; offered upstream and declined), the codegen map, the decomp.me replica, the drafter write-up, the how-to. Linked from the sidebar and the Home table. |
| | `docs/gen3-standards.md` | **(P33 S89)** The charter for Gen3 and the project's public standard: §1 accuracy (met; whole-binary SHA1 every build, no NON_MATCHING, clean rebuild per batch — stricter than the field), §2 readability (sotn STYLE.md adopted as baseline + four rules of our own: names only with evidence, every pin off or `// !FAKE:`, shared bodies as C not macros, types from code / names from observation), §3 the use of AI — conduct (byte gate as the only claim; no guessed names; outward text written by a person; no automated traffic to decomp.me; target projects' AI policies followed), §4 the measured gap and the order of work + a proposed definition of done. |
| | `docs/phase34-seed.md` | **(P33 S89)** The seed for Phase 34 (re-charter, Drew 2026-09-07): Phase 33 closes on what is done (v1.32.0); the flip-gated remainder — probe, C10, E1/E2/D3/F3 outward, tool discoverability, C11, the v2.0.0 PhaseEnd = Gen2 EXIT — becomes Phase 34 with every task's actor, effort and carrying document, the full inventory of what Phase 33 delivered, and the milestone (Phase 33's original items 2–4). Gen3 opens at Phase 35. |
| | `tools/objdiff_report.py [--in docs/progress.json] [--out report.json]` | **(P33 D3)** progress.json → objdiff's report format (report.proto v2, snake_case — validated with `objdiff-cli` 3.8.1 `report changes`): one unit per binary (code = instructions × 4, functions byte-identical / matchable, metadata complete), categories `game-code` and `linked-sony-objects` (functions only). `.github/workflows/progress.yml` runs it on every push (no rebuild — the committed JSON) and uploads the artifact **`SLUS_007.26_report`** for decomp.dev (Drew registers at decomp.dev/manage/new after the flip). |
| | `tools/frogress_upload.py [--push --project bfm --version us]` | **(P33 D3)** stdlib; `--dry-run` is the default (prints the payload); `--push` POSTs `{"api_key","entries":[{git_hash,timestamp,categories:{default:{measures…}}}]}` to `progress.deco.mp/data/<project>/<version>/` with `FROGRESS_API_SECRET` from the environment (never a file). frogress projects are admin-created — Drew requests the slug + key after the flip. |
| | `tools/public_rewrite/` (P33 C1) | **The history-rewrite package** (`docs/public-flip-runbook.md` §3 is the operating table). `common.py` (shared: the purge rules, the DERIVED content-hash sets, identities from the log, the one hash regex, a persistent `cat-file --batch`) · `hash_dict.py [--write-mailmap]` (every commit OBJECT → `commit:NNNN` / twin / orphan; prefix index 7..40; asserts 0 ambiguous; records content-hash collisions as excluded; writes the scratch mailmap) · `scrub.py --test \| --sample \| --file` (THE scrub: hash tokens, addresses → noreply, trailer lines in messages; 12 known-true cases; the HEAD sample with git's own object lookup as the independent oracle) · `gate_scan.py --all\|--refs … [--worktree] [--expect-fail FIXTURE]` (paths ever touched × purge rules; every reachable blob's content sha1 × the ROM set; 5 byte signatures; 50 MiB; emits `rom_blob_ids.txt` = hits ∪ every blob ever under a purge path; the fixture `expected_offenders.txt` is the R39 negative control) · `run_filter.py [--sample]` (the git-filter-repo 2.47.0 module-API run inside the scratch bare clone; refuses elsewhere) · `verify_rewrite.py --old --new` (the pairwise proof) · `build_commit_map.py [--out]` (`docs/commit-map.tsv`, asserted free of old hashes) · `resolve_tokens.py [--check] [--map]` (tokens → shortest unique ≥9-char new abbreviations at the tip) · `absent_scan.py [--repo] [--tree]` (nothing old anywhere) · `probe_github.sh [--after-flip]` (Drew's purge probe). Scratch (`.run/public_rewrite/`, never committed): `dict.json`, `mailmap`, `rom_blob_ids.txt`, `old-to-new.tsv`, `repo.git`, the bundle. · `probe_github.sh [--after-flip]` (Drew's daily post-purge probe, C10: 33 sampled old shas via `gh api` + a fetch; **S88, R57:** the fetch runs in a throwaway bare repo under `.run/public_rewrite/` with `--filter=blob:none --depth=1`, never in the working repo — a successful fetch of an old sha imports its purged closure, which the S87/S88 runs did (5.97 GiB unreachable) — and it ends with a self-check naming any sampled old commit the working repo still holds + the gc recipe) |
| | `.venv/bin/git-filter-repo` 2.47.0 | (P33 C1) `pip install git-filter-repo==2.47.0` (in `requirements-python.txt`); used through its module API by `run_filter.py`. |
| | `tools/audit_public.py [--paths …]` | **(P33 B7; extended P33.5 task 8) The first-push gate**, four checks: (1) no tracked file under `tools/public_rewrite/purge_set.txt` (the C1 rewrite's own input, filter-repo syntax) OR under the audit-only sibling `tools/public_rewrite/untracked_after_rewrite.txt` (paths untracked after the rewrite without a second rewrite — kept out of the purge set because `gate_scan.py` reads that as the history census; both files refuse to be empty, R43); (2) none whose SHA1 is ROM-derived (DERIVED set: every `sha1` in `extracted/retail/manifest.jsonl` + `config/check.*.sha` + the redump Track-1 SHA1; zero-length files exempt — the empty-file SHA1 is also SC04/SC05 `FILE_029/1.6`'s); (3) none > 100 MiB (GitHub's hard limit; 50 MiB until 2026-09-12); (4) **CONTENT** — no tracked text file with ≥ 64 CONTIGUOUS disassembly-shaped lines (asm-differ incl. operand-less `nop`, objdump, splat `/* ADDR HEX hex */`, `glabel`) — the class-3 case path+hash cannot see. Names every offender, exits 1; prints the three longest runs. ≈2 s over 6,443 paths. Controls: S87 the purge set; P33.5 the two `fable_cd4` listings fail check 4 at 398/179 lines by `--paths`, the 40-line xsig fixtures pass at 14. |
| | `tools/compile_only.py <aliases…> \| --all [-j N] [--list]` | **(P33 B7)** cpp → cc1 → maspsx → as on every eligible TU with the Makefile's flags PARSED at run time; TUs per binary from `<alias>_SRC_DIR` with nested-binary pruning (= the Makefile's `C_SRCS`); skips main's 70 LINKED tiles (`progress._main_linked_segs_from_makefile`) and the 47 `INCLUDE_ASM(`/`INCLUDE_RODATA(` TUs (they `.include` asm/); -O0 TUs (`corpus.o0_sources`) compile at -O0. Coverage line with every denominator. Measured: PR scope 54 of 124 TUs in 1.6 s; fleet 4,170 of 4,287 in 123 s at -j32 (≈50 CPU-min). |
| | `tools/public_rewrite/purge_set.txt` | **(P33 B7)** THE purge set (Drew's decisions 3+11): the EXE at both historical paths, `glob:dumps/*.bin`, `ghidra/`, `tools/psyq/`, `session archive/`, `glob:tools/ghidra-ext/*.zip`, `tools/brave-CUE/brave.exe`. Read by audit_public now and by C1's `git filter-repo --paths-from-file` later. |
| | `ExportAnnotations.java` + `tools/ghidra_export_annotations.sh` | **(P33 B5) The read-only TEXT export of a program** — byte-stable JSONL (fixed key order, sorted, `0x%08x`): `program`/`block`/`archive` container rows, LOCAL-archive types, every function signature (params/locals/storage/sources/comment), defined data, the 5 comment kinds, bookmarks, equates, labels not in the symbol files. `tools/ghidra_export_annotations.sh [PROG…]` → `.run/ghidra_export/<prog>.jsonl` (no arg = all programs in one `-readOnly` run; 129 in 18.5 s). MCP must be STOPPED. |
| | `ImportAnnotations.java` | **(P33 B5)** Idempotent compare-before-write import of that JSONL (creates a missing function for a `func` row; refuses unknown row kinds, R43); prints `BFMANN types=… funcs=… … failed=0 rows=N`. **OSGi gotcha:** Ghidra compiles `tools/ghidra_scripts/` as ONE bundle — a compile error in ANY `.java` there breaks EVERY script ("Failed to get OSGi bundle containing script"); javac diagnostics are not shown — compile by hand: `javac -nowarn -d .run/javac_check -cp "$(find ~/ghidra_12.1_PUBLIC/Ghidra -name '*.jar' -path '*/lib/*' | tr '\n' ':')" tools/ghidra_scripts/*.java`. |
| | `tools/ghidra_annotations_delta.py` | **(P33 B5)** `live.jsonl baseline.jsonl out.jsonl [--census]` — the HAND-AUTHORED part of a program = live rows not in the fresh rebuild's baseline (container rows always kept), minus three counted analysis-drift classes: `Error`/`Analysis` bookmarks; `func` rows absent from the baseline with a DEFAULT signature, no comment and an auto name (function-set drift); `func` rows differing from the baseline only by an auto name (the DB lagging the curated symbol file, R15). Prints the census + the dropped counts. |
| | `tools/ghidra_rebuild.sh <program> [--proof] [--keep]` | **(P33 B5) Rebuild ONE program FROM TEXT + the disc** in a scratch project (`build/ghidra_rebuild/proj` — Ghidra refuses a path component starting with `.`, so not `.run/`): import (PSX loader for the 3 EXEs, raw blob at `make -s print-VRAM_BASE` otherwise) + analysis + psyq400.gdt → `DefineFunctions` from the built ELF → `ApplySymbols` (the yaml's symbol files) → baseline export → `ImportAnnotations config/ghidra/<program>.jsonl` → export → delta. `--proof`: `cmp` delta vs the committed file → `PROOF PASS`/`FAIL` (+ `.run/ghidra_rebuild/<program>.proof` marker). Without a committed file it writes `<program>.candidate.jsonl` to review. MCP must be STOPPED. ≈65 s resident/overlays, ≈200 s the EXEs. |
| | `tools/ghidra_roster.py [--check]` | **(P33 B5)** `config/ghidra/ROSTER.md` from the committed files: kind/payload/vram (from the build registry), blocks, hand-authored census, last proof marker. |
| | `DecompileFunctions.java` | **Batch**-decompile a list of addresses (arg0 = addr-per-line file, arg1 = out-dir) → `<name>.c` each. Headless harvest Ghidra-C pre-pass (Phase 17); no live MCP / `/mcp` needed. Run: stop MCP, `analyzeHeadless ghidra bfm -process <prog> -noanalysis -postScript DecompileFunctions.java <addrfile> <outdir>`. |
| | `tools/ghidra_import.sh` | Headless `analyzeHeadless` import/analysis driver (PS-X EXE; auto-detect PSX loader). |
| | `tools/ghidra_import_raw.sh` | Headless import of a RAW flat blob — `BinaryLoader` + `--loader-baseAddr <vram>` + `PSX:LE:32:default` (resident blob / Gen2 overlays; no PS-X EXE header). |
| | `tools/prefetch_fleet.py` | **The fleet Ghidra-C prefetch batch** (P30 T0.5): one representative per remaining h_seq distinct class + ALL main/resident stubs → `.run/ghidra_c/`, resumable (skips cached), serial on the project lock; auto-stops a serving MCP (R23 — R29 applies at next MCP use); imports a missing overlay program on demand (`ghidra_import_raw.sh`, blob via `family_remap.img_path`, vram via the splat yaml). `--dry-run` / `--limit-programs N` (probe-first). R32: per-program outcome report, batch continues past failures. |
| | `tools/family_cousins.py` | **(P30 S49, cookbook §168)** The COUSIN-UNIT survey — the similarity tier ABOVE h_seq. h_seq is an exact skeleton hash, so ±1 instruction (li-expansion, table-size drift) fragments same-source families into "singletons"; this tool clusters the distinct open skeletons by mnemonic-stream similarity (≥0.85, union-find) and attaches the best MATCHED-skeleton seed per unit → `.run/family_cousins.json` + `docs/family-cousins.md`. Categories A-prop / seeded / cousin-multi / cold. `--targets N --wave waveN` emits the crack_wave.js slate ranked by whole-UNIT open ins, with each seed's C body location (engine_core.h macro or inline src file) resolved. R32 both ways: fails loud on a stale family map (independent stub recount) and on a partition defect. RANKS AND SEEDS ONLY — cousins need a per-member seeded CRACK (skeleton drift ⇒ recompile), never a `family_sweep` remap; the whole-binary byte-gate stays the sole arbiter. |
| | `tools/aprop_symfix.py` | **(P30 S50, cookbook §171)** The stale-SEED-SYMBOL guard for the adapt/A-prop lanes — the second, DISAGREEING oracle (R34) for the one class `match_one` is structurally blind to. A per-location data symbol (`D_8018xxxx`) carried out of a seed body unrebased scores MATCH standalone (encodings identical; the scorer cannot see a relocation's target NAME) and dies in the host TU at link with `undefined reference`. Audits every draft's vram-suffixed symbols against the symbols the TARGET's own `.s` relocates; `--fix` rewrites the 1:1 cases and emits a `gate_lane`-shaped slate. Deterministic, no build — run it BEFORE the gate, never after. Measured on the S49 A-prop residue: 24 of 24 concentrated failures were this class, 23 banked after the rebase (A-prop conversion 57% -> 87%). Its primitives are imported by `family_cousins.py`, whose `--aprop-cards` members now carry an explicit `sym_map` of `{seed -> member}` renames. |
| | `tools/aprop_autodraft.py` | **(P30 S50, cookbook §171a)** Mechanically drafts lane-A members with NO agent in the loop — seed body + `family_remap.symbol_map` (positional reloc zip) + a MINIMAL preamble synthesized from scratch, so the seed's declaration layer never travels (that layer is `family_sweep`'s dominant failure: 331 of 458 S49 verdicts). Handles inline AND `DEFINE_<fn>()` macro seeds (take the DEFINITION; the block is only the decl source — pasting it whole scored 28% vs inline's 68%), and applies T2a's `imm_map_tier1` so a per-location LITERAL resolves like a per-location symbol (131 of 275 IMM members). REFUSES at generation time what cannot work: non-PURE members, arity conflicts, undefined data, and `.s` files carrying data or a jtbl. **256 banked at zero agent tokens.** Every draft is a candidate — gate via `gate_lane.py`. |
| | `tools/draft_prechecks.py` | **(P30 S50)** Static pre-checks shared by the draft tools: arity disagreement vs the destination's own declaration, and symbols nothing in the destination binary defines. Both are real measured build failures and both are decidable WITHOUT compiling — at ~1 min per gate group that difference is the run. Negative-controlled against all 205 banked drafts of the first run (zero false positives; catches 39 of 67 known failures); that control found two bugs in the checks themselves — C89 `f()` declares UNSPECIFIED parameters (not zero), and a member's own definition read as a call to itself. Deliberately CONSERVATIVE: flags only `decl > call` on named parameters, because a pre-check that discards good drafts is worse than one that lets a few builds fail. |
| **Disc/.CD extraction** (`tools/bfm_extract/`) | `extract.py` | Walk the disc / extract root files (`make extract`). |
| | `extract_exe.py` | Extract & verify `SLUS_007.26` (`--verify-disc`, owns `EXPECTED_EXE_SHA1`). |
| | `extract_proto_exe.py` | Extract the prototype/demo EXE for cross-checking. |
| | `cd_archive.py` | Parse the `.CD` container format. |
| | `pac.py` | Parse the PAC archive format. |
| | `lzss.py` | LZSS (de)compression for packed blobs. |
| | `manifest.py` | Build/verify the extraction SHA1 manifest. |
| | `crosscheck.py` | Cross-check extracted bytes against the runtime RAM dump. |
| **Matching harness** | `tools/decompile.py` | m2c wrapper — C scaffold for a function (§6.6). |
| | `tools/match_protos.py` | Match prototype-EXE functions against retail. |
| | `tools/permuter/` | decomp-permuter harness (PERM_ recipes/weights) for stubborn near-misses. |
| | `diff_settings.py` *(repo root)* | asm-differ config (arch `mipsel`, object mode vs `expected/`). |
| | `tools/new_overlay.sh` | One-command location-overlay onboarding: `<SCxx> <FILE_nnn> [ENTRY]` — instantiate `config/splat.<ov>.yaml` from the template (+ non-4-aligned `bin` carve), register the binary in `config/overlays.mk` + the report/diff dicts, `make extract && build` byte-check. Idempotent (Phase 13, cookbook §13). **Phase-27 T7:** the optional `ENTRY` arg (default `0.4`) reaches a non-`0.4.dec` payload — the 4 SC07 overlays put code at PAC entry 1 (`1.4`). difficulty.py dropped from the insertion set (it derives now, T6). |
| | `tools/disc_code_sweep.py` | **RETIRED (S45, R33)** — superseded by `tools/disc_audit.py` / `make audit-disc` (whole-payload, BOTH raw+LZSS layers, residue-0 partition, claimed-by derived from `config/check.<bin>.sha`). The sweep read only the RAW layer through a 4,096-word window and had no notion of a claim; its historical findings are preserved in `docs/disc-completeness.md`. |
| | `tools/new_binary.sh` | **(P30 S44/S45)** One-command onboarding for ANY flat-blob binary class: `<alias> <payload> <VRAM> [TEXT_LO]` — ov_* (overlay slot, registry `overlays.mk`) or md_* (own §S44 slot, registry `modules.mk`). Signs (TEXT_LO-aware §154), instantiates the shared template (non-zero TEXT_LO ⇒ paired-`.rodata` header carve), writes check.sha + symbols, registers in the registry + the 3 report/diff dicts (sentinel-anchored, ast-checked), extract+build byte-check. Idempotent. `new_overlay.sh` is now a thin wrapper over it. §6.7 module recipe. |
| | `tools/find_addr_refs.py` | **Register-tracked** search for code that materializes an absolute address (§155: never window-paired). `<addr> [--binaries a,b]`. **S46:** also follows the hi half through an index `addu`, so gcc's indexed global-array read (`lui`/`addu`/`lh lo(base)`) is no longer invisible — those hits are labelled `-indexed` (cookbook §155c). `scan_all()` is the one tracker other tools import (R33). |
| | `tools/idxtab_map.py` | **(P30 S46)** The master **IDXTAB → payload → owning binary → DESTPTR load address** map, fleet-wide. Controls-gated (R32/R35 — refuses to emit if the two byte-proved rows don't reproduce); process-pooled; `--controls-only` / `--binaries` / `--min-len`. Output `.run/idxtab_map.json`; **read `docs/idxtab-map.md` for what it does and does NOT prove** (it cannot show a payload is dead — byte-proved loaders are absent from this route too). |
| **PsyQ library linking** (cookbook §8/§9) | `tools/psyq_lib_split.py` | Split a PsyQ `.LIB` into per-object members. |
| | `tools/psyq_build_libs.sh` | Build the PsyQ libs from split members. |
| | `tools/psyq_identify.py` | Identify which SDK objects a region's functions belong to. |
| | `tools/psyq_link.py` | Link identified PsyQ objects into the build. **P31 S78 #4:** `link_object` first runs `psyq_bss_split.prepare_object` (the same prepare step the region verify and the build use), so its PASS/FAIL is the build's verdict for scattered-`.bss` objects; links with `--no-check-sections` like the build. |
| | `tools/psyq_bss_split.py` | **(P31 S78 #4, cookbook §489)** Splits a PsyQ object's scattered `.bss` into per-base NOBITS pieces (`.bss`, `.bss2`, …) — a pure-Python ELF32 REL rewrite: bases derived from the game bytes per HI16/LO16 pair, runs in offset order, cuts snapped to symbol starts, relocs retargeted with the addend rewritten in place, self-checked. Refuses (loud, R43) a sized symbol straddling a cut or a HI16 shared across pieces. Runs automatically inside `psyq_link` / `psyq_link_region` / `psyq_integrate` (`prepare_object`); the CLI reports a plan (`--vram --exe --vram-base`, `-o` to write). Takes SYS.o (2 pieces), VM_F.o (2), GS_001.o (6). |
| | `tools/psyq_link_lib.py` | Per-library link driver. |
| | `tools/psyq_link_region.py` | Link a specific address region from PsyQ libs. |
| | `tools/progress.py` (weighted main) | **(P31 S78)** `MAIN game-code weighted` now excludes LINKED subsegs LIVE (`_main_linked_ranges`: Makefile `psyq_integrate` stub lists → yaml ranges, R33); the 2026-08-05 sig had carried all linked-SDK instructions, under-reporting main by ~32 points (59.8% → 91.8%). |
| | `tools/psyq_integrate.py` | Integrate linked PsyQ results back into the source tree. **P31 S78:** `--yaml <splat yaml>` (every main call passes `$(main_SPLAT_YAML)`) maps stub↔objects by SUBSEG RANGE with an exact-tiling check and PRINTS the located-but-unwired objects (`~~ N located object(s) / M ins OUTSIDE the stub subsegs`) — the completion contract's SDK-residue line; and a library object's DEFINED symbol whose recovered address the curated symbol file names differently is `--redefine-sym`'d to the curated name (R15; e.g. libapi 4.0 `A66.o` `firstfile`→`firstfile2`). Without `--yaml` the old contiguity mapping runs (overlay-free libraries only). |
| | `tools/make_libgs.sh` | Build/link the `libgs` block (cookbook §9). |
| | `tools/gen_lib_subsegs.py` | **(Phase 8)** Generate splat subseg lines + integrate stub list for a multi-block library (section-size-correct boundaries; cookbook §9.6). |
| | `tools/make_snd_used.py` | **(Phase 8)** Build the combined libspu+libsnd curated dir (alias dedup by byte-match; 3 address exclusions since S78 #4 — VM_F rejoined via the `.bss` split; §9.6). |
| | `tools/make_apicard_used.py` | **(Phase 8; S79 #5)** Build the combined libapi+libcard curated dir `.run/obj42/apicard_used` — libapi **4.2** (`.run/obj42/libapi42`, the EXE's real libapi, incl. FIRST/PAD/PATCH/CHCLRPAD) + libcard 4.0 (§9.6, §490). |
| | `tools/ld_interleave.py` | Interleave linker inputs to match the original section ordering. Three forms: `--front/--tail` (the original data->rodata->data sandwich), **`--order`** (P31 S72 — an address-ordered leaf list, needed once a binary owns SEVERAL `.rodata` carves: a `*.data.o` leaf contributes its `(.data)`, a code-object leaf its `(.rodata)`; main's island is a 7-piece sandwich `--front/--tail` cannot express), and **`--pre`** (P31 S74, cookbook §440 — a piece that must land BEFORE the text). `--pre` exists for the **resident**, which opens with `- [0x0, rodata, hdr]`, a 1-word `.rodata` header ahead of its code: every `--order` piece is emitted after TEXT_START, so `hdr.rodata.o(.rodata)` would fall into the unchecked `empties` bucket, be parked after the text, and move every byte in the binary. |
| | `tools/jtbl_rodata_pads.py` | **(Phase 29, cookbook §8e)** Post-maspsx filter for multi-table `.rodata` carve spans: REPLACES each cc1 `.align 3` (which is section-relative and would mis-pad a merged/4-mod-8 span) with the ORIGINAL's exact pad bytes per the object's `JTBL_PADS` spec (written by `jtbl_carve` into `config/overlays.mk`; armed via `$(if $(JTBL_PADS),…)` in the Makefile `build/src/%.o` recipe). Fail-loud on table-count drift / non-`.align 3` / non-jtbl rodata content. Unset var ⇒ pipeline byte-identical. |
| | `tools/split_src_region.py` | Split a `src/` region file at object boundaries. |
| | `tools/rollout_whale_o0.py` | **(Phase 24 W9)** Roll out the -O0 whale `func_80144B9C` ×134: per single-file overlay, line-split `<ov>.c` at the whale, carve the yaml code subseg into before/`_o0b`(-O0)/`_after`, write a thin `<ov>_o0b.c` that `#include`s the shared `src/shared/func_80144B9C.h`. Idempotent; the `WHALE_O0B_OBJS` Makefile wildcard -O0-compiles all `_o0b.o` (cookbook §38). |
| **Reports** | `tools/progress.py` | Per-binary decomp progress (`make report`); counts dedup-shared fns as REAL via the registry (Phase 11). **`--fleet`** (Phase 15) aggregates all 136 binaries → `docs/progress.fleet.md` (deterministic, source-derived). **`--weighted`** (Phase 25) prints the two BYTE/instruction-weighted metrics from `.run/sig.*.jsonl` (executable code only): **instr-weighted** (fleet per-overlay, the decomp.dev-display number) + **distinct-code** (dedup, each unique fn once, the distinct-RE number); both also fold into `--fleet` alongside the ×134-inflated function-count %. Needs `make sig-overlays` first; degrades gracefully without sigs. |
| | `tools/difficulty.py` | Per-function difficulty scoring. |
| | `tools/dup_report.py` | Duplicate-function report; `--cross` (Phase 11) buckets all binaries → `docs/duplicates.cross.md`. **Phase 15:** ingests each overlay once (named ∪ `sig.ov_*` glob, deduped by alias) — else onboarded overlays double-count and inflate collapsible bytes ~2×. |
| **Cross-binary dedup** (Phase 11, cookbook §11) | `tools/sig_image.py` | **Ghidra-FREE** per-function signer for a flat image (overlay/resident); `h_exact` byte-matches the Ghidra dumper, self-consistent `h_norm`; linear-partition + `detect_code_end` boundaries. |
| | `tools/dedup_integrate.py` | Byte-honesty validator for `config/dedup.us.yaml` code-shares (`--check`; fail-closed on sig-hash drift). |
| | `tools/dedup_propagate.py` | **(Phase 15, cookbook §14)** Match-once → propagate-many: lift a matched body, author a `DEFINE_func_<ADDR>()` macro in `src/shared/engine_core.h`, instantiate it at every onboarded overlay sharing that `h_exact`, byte-gate each (fail-closed), register in `dedup.us.yaml`. `--addr`/`--auto-from`/`--check-only`. `find_site` accepts brace on the same OR next line (Phase-15 fix — next-line-brace defs were silently un-propagated). |
| | `tools/sig_unify.py` | **(Phase 15, cookbook §14d)** Deterministic hard-tail recovery: unify a gate-failing draft's FULL signature set — callee externs **and** the draft's OWN definition signature — to the banked-canonical decls (`--overlay`/`--in`/`--out`); the whole-binary byte-gate stays the arbiter. Recovered 32 of 191 standalone-MATCH conflict-blocked drafts, zero agent tokens. |
| | `tools/gen_harvest_targets.py` / `tools/canon_draft_decls.py` / `tools/build_engine_types.py` | Callee-sig-aware target manifest (§14b/c) / callee-extern canonicalizer (§14c) / additive shared-types-header extractor (`src/shared/engine_types.h`, §14 struct; **Phase 20: also lifts typedefs — anon-struct / fn-ptr / alias — with collision + tagged-struct-typedef-overlap guards, closing the §19 type-blocked propagation cap**). |
| | `tools/canon_resident_calls.py` | **(Phase 19 / T2, cookbook §17a-3a)** Link-miss recovery: rewrite each `func_<ADDR>` in a draft to the **curated** resident symbol name when that address has one in the stacked symbol files (`0x8004CFEC`→`ratan2`), so the linker resolves it. Pure draft-text (body bytes unchanged); **run FIRST** in the recovery pipeline `draft → canon_resident_calls → sig_unify → harvest_verify`. |
| | `tools/fix_arity_callers.py` | **(Phase 19 / T3, cookbook §17a-3b/§19)** The no-prototype recovery for the dominant gate-failure class: a banked SHARED caller in `engine_core.h` declares the callee `extern <ret> func_X(void);`, conflicting with a real def that takes args. Rewrites that caller decl to `extern <ret> func_X();` (`--apply`, byte-neutral; skips narrow-param defs via `--drafts`; `--revert` undoes). Re-gate after. |
| | `tools/cast_call_sites.py` | **(Phase 20, cookbook §20)** The §17a-1 per-site function-pointer cast recovery for the loose-typing CALLEE-conflict class: per draft, for each callee whose canonical TU sig differs from the draft's intended sig, rewrite the decl line → canonical (kills the in-TU `conflicting types`, keeps the symbol in scope) AND cast each call site → the draft's intended sig `((ret(*)(args))func_X)(args)` (decl lines never cast; gcc folds the cast of a known symbol → direct `jal`). Pure `--in`/`--out`; whole-binary gate is the arbiter. Pipeline `canon_resident_calls → cast_call_sites → sig_unify → harvest_verify --chunk 1`. Recovered 6 of T6 batch-1's 33; the rest are the def-side loose-typing wall (caller-side blocked: INCLUDE_ASM declares nothing). |
| | `tools/reconcile_decls.py` | **(Phase 24 T7b, cookbook §33)** The **DATA-symbol** analog of `cast_call_sites`: a fleet-majority canonical-type **oracle** (`engine_core.h`-authoritative + plurality across all overlays+resident; `--print-canon D_XXXX`) + a byte-neutral access-cast **transform** — rewrite each loose `extern <T> D_XXXX...;` → the canonical decl and cast every use to the intended type (`(Ed*)D_x` array, `(Ed*)&D_x` struct-base, `*(Td*)&D_x` signedness, `(*(P**)&D_x)` ptr-in-scalar). Mechanizes the manual giant decl-reconcile. Pure `--in`/`--out` (+`--overlay`/`--src-file`); wired into `gate_stage` after `cast_call_sites` (idempotent/no-op without a data conflict); whole-binary gate is the arbiter. Byte-proven: full loose `func_80129CF8` → reconcile → `make build BINARY=ov_SC01_077` = `d19c9580`. |
| | `tools/scope_data_externs.py` | **(Phase 26 session 8, cookbook §8d)** The `scoped` stage of `jtbl_family_bank` — places a templated body's **DATA** externs at the scope the target TU can accept. `family_remap.gather_externs` prepends carried decls at FILE scope; for a per-location symbol the sibling declares only at BLOCK scope inside its own later functions, that decl **establishes a global the TU never had** and every later block-scope `extern` must now agree with it (loose typing ⇒ they don't) → `conflicting types for D_801812A4`. Fix: demote the carried `D_` extern to **block scope inside the function body** when the TU has no file-scope decl of it above the insertion point. Byte-neutral (an `extern` emits no code; type + access opcodes unchanged), never worse than raw ⇒ needs no oracle, no type comparator, no fn-ptr parser. **Use this, not `reconcile_decls`, for the templating class** — that tool's oracle is fleet-majority (wrong question) and its regex cannot parse `extern void (*D_x[])(void *);` (silently skips the failing symbols). Library `fix(body, tu, pos, func)`; CLI `--body/--tu/--func[/--out]`. Byte-proven: `func_8015AE2C` (562 ins) ×133. |
| | `tools/scope_tu_externs.py` | **(Phase 29 T48/T51, cookbook §103)** The **TU-side** complement of `scope_data_externs`: move the target TU's **own** file-scope `extern` decl of a contested `D_` symbol down into every later function that references it and lacks its own block-scope decl, then delete the file-scope line. Needed because §8d's give-up branch **drops the draft's decl** when the TU already declares the symbol at file scope — right when the types agree, fatal when the byte-true draft needs a different one (a file-scope decl constrains every LATER function in the TU). Declaration-only ⇒ byte-neutral; verify in two steps (decl move alone rebuilds byte-identical, *then* splice). Contested set is **derived** (the remapped draft's block-scope `D_` externs ∩ the TU's file-scope decls above the splice point), never hand-listed. Built on `cdecl.split_statements`/`_mask` (comment-masked spans — not a 7th regex); **refuses loudly** (R32) on a duplicate file-scope decl, a file-scope reference below the decl, or an ambiguous body brace, and asserts coverage as a **delta** (file-scope −1, block-scope +consumers). Library `scope(tu_text, syms, above)`; CLI `--tu/--syms/--above` or fleet `--family/--from/--from-addr/--members [--in-place]`. **Wired into `jtbl_family_bank` (T53) as the `tu-scoped` stage** — after `raw`/`scoped` (it edits outside the spliced body), before `recovered`/`reconciled` (those bend the DRAFT, measured at +3 ins for this class). Byte-proven: `func_80135260` ×132 TUs banked 132/132, R22 clean-fleet 140/140; counterfactual on a reproduced blocker = raw compile-error / scoped byte-mismatch / **tu-scoped BANKED**. |
| | `tools/inject_capped_externs.py` | **(Phase 23, cookbook §28d)** The "macro-extern-injection" lever for reach-134 fns matched INLINE in ov_SC01_077 but skipped by `dedup_propagate` as "not self-contained": source the EXACT file-scope `extern …;` the overlay already declares for each referenced `func_`/`D_` symbol and inject it BLOCK-scope so the lifted macro body resolves in every overlay. Byte-neutral (gate proves 077 stays `d19c9580…`); fail-safe (only rewrites a fn whose injected body then `compiles_standalone`). `--overlay`/`--src-file`/`--min-reach`/`--apply`. Then `make build BINARY=ov_SC01_077` → `dedup_propagate --auto-from`. |
| | `config/dedup.us.yaml` / `src/shared/*.h` | The code-share registry + the shared bodies (one macro → N sites, byte-gated). |
| | `make report` / `make sig-refresh` / `make sig-overlays` | Convenience targets: reports (+`--cross`) / Ghidra signature-dump / Ghidra-free sign all 134 overlays. |
| **LLM matching tier** (Phase 21–23, `docs/gen2-mips-matching-model.md`) | `tools/gate_stage.py` | The shared deterministic bank/log spine: `canon_resident_calls → cast_call_sites → sig_unify → harvest_verify (byte-gate) → dedup_propagate → backlog`. **Binary-agnostic** (resolves src/asm/out/good_sha + bare-hash from `binary`; Phase-23 fix). **Phase-23 T10:** optional `lock_path` (per-binary lock) / `verified_out` / `failed_out` (per-worker scratch) / `compute_fleet` for `bulk_harvest`'s parallel gate — all default to the serial behavior. |
| | `tools/harvest_verify.py` | The whole-binary byte-gate (substitute draft → `make build` → keep iff byte-identical, else revert). Sole arbiter (G3/P9). `--verified-out`/`--failed-out` = per-worker result paths (parallel gating, Phase-23 T10). **(P31 S74)** The duplicate-typedef strip-set is computed **SCOPED**: `cdecl.typedef_names(path, above=fn)`. Unscoped, it stripped from a draft a typedef declared BELOW its splice point — the draft's own copy was the only one in scope — and the result was `parse error`, logged as PLUMBING and indistinguishable from a real declaration conflict (cookbook §441). One function was blocked by that alone. |
| | `tools/rtu_match.py` | **(Phase 25 wave-3, cookbook §42b)** REAL-TU-faithful per-fn match check that fixes match_one's blind spot: splices a candidate into a copy of the split `.c`, neutralizes `INCLUDE_ASM` (`-DINCLUDE_ASM(a,b)=` → no `asm/` needed) with `-Isrc/<source>` for the relative `../shared/` include, compiles the WHOLE TU (`cc1→maspsx→as`), masked-diffs the fn (shared `masked_diff`). Captures the in-TU decl/global-type/memcpy-builtin drift that isolation (`match_one`) misses, so a MATCH holds at the whole-binary gate; per-fn temp dir → **parallel-safe, no shared overlay build** (enables a real-TU-faithful crack fan-out). Supports `//@EDIT old||new` file-scope pre-edits (the §42b read-global s16→u16 flip). STILL finish on `make build` SHA (G3/P9). |
| | `tools/diff_autopsy.sh` | **(Phase 31 S62 T1, cookbook §301)** The gate said DIFF, this says WHERE: splices the raw draft into the real TU exactly as the gate does, builds, `cmp`s against the byte-good binary, decodes the diverging words, restores tree + build. `tools/diff_autopsy.sh <binary> <fn> <tu.c> <draft.c>`. |
| | `tools/stub_invariant_audit.py` | **(Phase 31 S62 T1, cookbook §301)** Regression test for the masked comparer: INCLUDE_ASM pastes the original bytes, so `diff_object_s(object, .s)` must be 0 for every stub; quotes the denominator (stubs with internal-`j` relocs). Run after ANY change to `masked_diff.py`. |
| | `tools/interleave_check.py` | **(Phase 31 S62 T2, cookbook §302)** Order ⇔ yaml consistency for an overlay's jtbl carve: the `_JTBL_INTERLEAVE` order must equal the yaml subseg sequence position by position. `--fix` regenerates the order from the yaml (never hand-edit it). Run after ANY restore of overlays.mk or a splat yaml (two of the five S61 reds and four of the six clean-sweep failures were exactly this). |
| | `tools/pads_audit.py` | **(Phase 31 S62 T2, cookbook §302)** DERIVES each object's `JTBL_PADS` from the bytes: compiles the TU to count its tables, walks the retail words inside the yaml carve with those sizes (pad 4 ⇔ one zero word), reports SPEC-DRIFT / CARVE-DRIFT / ok with the derived spec. Reproduced 15/15 correct specs as its negative control; replaced S61's blind value searches. |
| | `tools/claude_wave_packs.py` · `tools/workflows/claude_wave_draft.js` · `tools/wave_judge.py` | **(Phase 31 S62 T4)** The Claude-subagent wave harness that ran the model-ladder probe: identical packs from `api_agent` (SYS + user_msg + law-1c warm start + gate feedback) → a Workflow of drafting agents per (arm, fn) with CLI equivalents of grep/read_file/match_one/submit, writing only under `.run/<wave>/<arm>/` → `wave_judge.py` gates each arm with `sweep_parallel` (banked = removed INCLUDE_ASM lines in `git diff`; tree reset between arms; `--union` leaves the cheapest winning drafts in the tree). `tools/workflows/claude_wave_distill.js` = the R16 distill (extract per byte-proven transcript → verify vs the cookbook). Routing rule from T4: ≤50 Sonnet, 51–120 Sonnet, >120 Opus, Opus on residue; haiku retired; DeepSeek optional at ~$0.03/fn. |
| | `tools/t5_targets.py` · `tools/t5_cards.py` · `tools/wave_card_fuel.py` · `tools/t5_bank.sh` | **(Phase 31 S63 T5)** The wave *draw* and the wave *bank* halves around the T4 harness: `t5_targets.py --wave .run/t5x --n 48 [--residue .run/t5w] [--main-only]` draws a routed slate (open per `corpus.stubs`, ledger `.run/t5/drawn.json` keyed (binary,fn), ≤120 ins → sonnet / >120 → opus, residue → opus once, stratified by band, ≤4 per binary, one fn NAME per wave — the harness keys packs/drafts by bare name, so `claude_wave_packs`/`wave_judge` now REFUSE duplicates, R43/R48); then **`t5_cards.py --wave .run/t5x`** BUILDS the target's OWN card fuel (`tu_ref` = in-TU banked neighbours §194-E, `decl_prior` = fleet signatures §196) from `wave_card_fuel.py` — those helpers extracted VERBATIM out of `build_wave_atlas.py` (which parses argv at import and so was unusable as a library) so both consumers share ONE oracle (R33); measured 88% tu_ref / 98% decl_prior on a fresh K-class slate, against 0% correct fuel before the fix. Then `claude_wave_packs.py --cards <wave>/cards.json` → Workflow `tools/workflows/claude_wave_draft.js` `args={wave, targets}` (one agent per target, `model=target.arm`) → `tools/t5_bank.sh .run/t5x sonnet opus` (refuses if a lane/gate runs or src/ config/ is dirty; `wave_judge --union`; then a RECOVERY pass — every un-banked draft through `fix_tu_ret_decls`, which retypes a destination TU's `extern void` decl on a value-returning function and re-gates (wave t5a: 2/2 banked, both byte-exact bodies refused for that decl alone); then the R22 clean fleet sweep parsed `N passed, 0 failed of N`; commit `— N banked`). |
| | `tools/jtbl_rodata_pads.py` — S74 corrections | **(P31 S74, cookbook §436-C/§441)** Three measurement bugs in the island walk, all of which ACCUSE THE CARVE when they fire. (a) `_s_rodata_span` ignored a **trailing `.align`**: a `.s` ending `.asciz "r"` + `.align 2` occupies 4 bytes, not 2. Latent by construction — `--derive`'s `zero_gap` self-corrects a 1-3 byte undershoot whenever the next item is an anchor, and **a C jump table has no anchor**, so it fires only when someone banks a switch function into such an object. (b) `_items` matched a rodata anchor only as `D_xxxxxxxx:`, missing the `labels.inc` macro form **`dlabel D_xxxxxxxx`** that an inline `__asm__` in C produces (as expands it; cc1/maspsx pass it verbatim) — an 8-byte hole in the walk. (c) the `ctable` branch read `word(pos)` without first stepping the sub-word zero gap, so a preceding `.asciz` ending unaligned made it refuse a correct layout. Two agents found (a) independently from opposite directions. Byte-neutral controls: `md_SC07_003` + `md_MAIN_011` clean rebuilds, `gate_main --assert-baseline`, and a full `check-all` 213/213. |
| | `tools/jtbl_carve.py` — the `covered` verdict | **(P31 S74, cookbook §440)** A table already inside an existing `.rodata` carve **bound to the function's own subseg** needs NO carve work: in stub state spimdisasm migrates it into the fn's `.s` and the object fills the piece exactly, so banking swaps that block for cc1's identical one. `island_probe` used to classify such a function `tail` on the table's ADDRESS, `apply()` routed it to `build_carve`, which resolves spans out of the RAW data asm where a carved table no longer is → `not found in the raw data asm` → `harvest_verify` booked **CARVE-REFUSED**, a verdict about the route we chose rather than about the function (R43). Now: a `covered` verdict, a `covered-tpad` wall (the retail copy carries a trailing §8a pad word the matched body will not emit — bankable, needs a `0t<n>` spec entry), and a fully-covered batch is a no-op before either route. |
| | `tools/reconcile_tu.py` — S75 corrections | **(P31 S75, cookbook §442)** Three defects in the `gate_stage` `-rc` rung, found by running S74's "9/9 via `harvest_verify`, 7/9 via `gate_stage`" measurement to ground. (a) **The premise was false at BLOCK scope.** The tool conformed a draft decl to the TU's in either direction because "a decl BELOW still conflicts" — four `cdecl._cc1_accepts` probes say a block-scope `extern` vs a TU decl BELOW the splice point is **ACCEPTED** (pedwarn `type mismatch with previous external decl`), while file-scope-vs-below and block-scope-vs-above both REJECT. Conforming it is destructive, because the TU's decl names the TU's TYPE and a type declared below the splice point is not in scope at it: the emitted result gets `syntax error before 'D_x'`. Byte-witnessed on `resident:func_800D06E8` (344 ins, `match_one` closeness 0) whose block-scoped `extern Blk80078E78` became `extern Struct80078E78` — typedef 388 lines lower. It is also the construct this ladder's OWN `scope_demote_drafts` (§8d) rung emits on purpose, used by three already-banked functions in that TU: one rung undoing another. Now skipped with a note (R43). (b) **The cast pass rewrote COMMENT PROSE** — it looped over lines skipping only `extern`/`typedef`, so a 40-line header comment had the symbol rewritten 8× including inside a quoted cc1 diagnostic; now matches on `cdecl._mask` (length-preserving, mask offset = source offset) and splices into the original. (c) **`&sym` emitted `&` applied to a cast** — legal for the scalar arm, `invalid lvalue in unary '&'` for the array/fnptr/fnptr_array arms (measured); `&` now SELECTS a pointer form built from the address and consumes itself. |
| | `tools/gate_stage.py --skip-stages` | **(P31 S75, cookbook §442)** Switch ladder rungs OFF by tool name (`--skip-stages reconcile_tu,sig_unify`, or `GATE_SKIP_STAGES=`); prints what it skipped, because a silently-disabled recovery rung is the R32 defect class. Exists because stage 0 gates the RAW drafts first, so a broken rung can only cost a RECOVERY — which is exactly what makes it invisible: the function it destroys was already failing, so its DIFF reads as a fact about the function. The next bad rung costs a flag instead of a session. |
| | `tools/verify_worktree.py` step 3c + `tools/jr_isolate_all.py` scan guards | **(P31 S75, cookbook §443)** `provision()` now symlinks every `.run/sig.*.jsonl` into the worktree (main clone 259, provisioned worktree **0** before this) — the third member of the same class as `extracted/` (3) and `.run/obj40` (3b), and the SAME defect `parallel_gate.stage_generated` had fixed per-binary in S69: two worktree provisioners, no shared list, so it had to be fixed twice; they now cross-reference each other. At the other end, `jr_isolate_all`'s carve-ownership scan wrapped `reloc_targets` in `try/except Exception: continue` — correct for one unwalkable function, catastrophic when the cause is environmental: **2,603 of 2,603 functions raised, the scan found 0 owners, and the downstream R32 1:1-ownership assert reported carve CORRUPTION that did not exist** (R54 — a guard downstream of the failure is not a guard). `FileNotFoundError` now aborts at the cause naming `make sig-all`, and `_assert_scan_covered` fails loud when attempted == raised (R32: a scan where everything raised measured nothing, so its zero is an artifact). |
| | `tools/harvest_verify.py` — `_mk_block_spans` | **(P31 S75, cookbook §444)** The `overlays.mk` carve-state snapshot/restore is now **plural**. `_mk_block_span` (singular) took the FIRST `# --- <binary>` header and ran to the next `# --- `, so a binary whose carve state spans more than one block was half-snapshotted and the revert silently half-restored. Measured: a REJECTED gate of `resident:func_800D06E8` left a 4th `JTBL_PADS` entry and dropped `--pre hdr.rodata.o` (the §440 leading-rodata sandwich), so the binary would not build — `consumed 3 rodata jump table(s) but 4 pad spec(s) given` — while `src/` was perfectly clean, which is where everyone looks. `resident` is the only such binary (1 of 142 with a block, 2 blocks). Restore now rewrites blocks tail-first, collapses to the snapshot if the header count changed, and RE-READS and compares the result (the defect it replaces was a reported success, R32). NC: no-op on 142/142 binaries; the real damage undone; the old single-block path provably does NOT undo it. |
| | `tools/jtbl_carve.py` — per-table `sltiu` bound | **(P31 S75, cookbook §446)** Two silent defects that made a byte-identical `.text` gate DIFF for five sessions across four 279-ins siblings. (a) **The over-span clamp disabled itself on the functions that needed it.** spimdisasm runs an island's LAST `jtbl_` dlabel one word into the following NON-ZERO data (string bytes), so the zero-word trim cannot see it; the clamp that would have caught it was guarded by `len(sltiu_bounds)==1`, and `sltiu` is ALSO how gcc emits an unsigned range check (`(u32)(x-lo) < n`, I1) — the four siblings carry five distinct `sltiu` immediates, so the guard stood down. A 0x28 table got 0x2C reserved: image 4 bytes short, ~850 `%lo` shifted, whole-binary DIFF. Fixed with `_table_bound()` — gcc-2.7.2's dispatch is a fixed idiom, so the `sltiu` nearest ABOVE that table's own `%hi(jtbl_X)` is unambiguous per table. (b) **A carve span whose `JTBL_PADS` line has no `tables=` comment** (written by `jtbl_pads_fix`) fell through both merge branches and lost its existing table's start, refusing with "table starts do not fit the span" — which `harvest_verify` then 'repaired' with a needless `jr_isolate_all` that walked back into (a); the merge now seeds from the span start unconditionally, per the invariant the validator already asserts (every carve span begins with a table). NC over every other open table-bearing stub fleet-wide: 11 tables, 10 unchanged, 1 changed — a FIFTH latent victim (`ov_SC06_022:func_80185B80`). All five banked byte-identical. |
| | `tools/dedup_propagate.py` — `_split_masked` memo | **(P31 S75)** `find_site` re-ran `cdecl._mask` (4 regex passes, one `re.S`) + two `splitlines()` over the ENTIRE concatenated source on EVERY call, though the mask is a pure function of the text and the caller loops over every function in the binary. Measured on `ov_SC01_005` (2.50 MB, 2,503 fns): 6.5 + 38.9 + 4.6 = **54 ms per call before any searching**. Memoized via `lru_cache(maxsize=8)` — callers already hold one text object per binary and CPython caches a str's hash, so a repeat lookup is a pointer compare. **2x on that loop (58.3 -> 33.0 ms/call), NC identical results on 120 addresses.** Honest scope: that loop is only ~2.4 min of a 30-min run; the profiler puts 43% of `--check-only` in `family_remap._alias_decl_for` (107 s / 1,312 calls), which is the real target and is NOT fixed here. |
| | `tools/main_diff_locate.py` — `classify()` | **(P31 S75, cookbook §447)** The `TABLE REJECT` class was **unreachable for main** and demanded purity. (a) It summed bytes whose object string contains `(.rodata)`, but main's `section_order` is `[.rodata, .text, .data, .bss]` — its rodata sits BELOW `.text` and its jump tables live in `.data` objects, so the test could never fire on the one binary with the most jump-table functions left. Now matches `(.data)` OR `(.rodata)`. (b) It required `ro == outside`, so a few bytes of perturbed code dropped the verdict through to `PLUMBING REJECT` and its §376 declaration advice. Now **dominance-based** (≥60%), reporting the split and naming which part is the table problem and which the declaration problem. Measured on `SaveLoadRoutine` (1,165 ins, the §434 wall): body BYTE-IDENTICAL, 3,787 of 3,989 differing bytes (94.9%) in `.data` jump tables, 202 (5.1%) in `.text`, built image 4 bytes SHORT — verdict moved `PLUMBING REJECT` → `TABLE REJECT (MIXED)`. The §376 chain had been run on it twice and fixed nothing, because it addresses the 5%. NC over all five pre-existing verdict shapes: 5 of 6 unchanged. |
| | **`tools/asm_in_c.py`** (NEW) | **(P31 S75, cookbook §448)** Finds every function that is **assembly posing as C** — a §265 file-scope `__asm__` body (class A) or a C function whose body is only asm statements (class B) — while correctly EXCLUDING the §3a cross-jump barrier, which is what `__asm__` means in 3,182 of the 4,224 sources. Measured: **199 functions, 154 of them GAME CODE, 171 in main**, largest `SaveLoadRoutine` (1,165 ins). These were in NO `progress.py` bucket, so main's REAL% was overstated (45.88% → **42.15%** once counted; `progress.py` gained a `VERBATIM __asm__ bodies` line that counts them byte-identical but NEVER as REAL). Design is the point: **three independent detectors that must agree**, disagreement reported as a defect (it caught `jtbl_*` being claimed as functions); **SDK-ness derived from the 14 shipped PsyQ archives via `nm`** (2,227 symbols), not a hand list (which had mis-classified `VectorNormalSS`/`SquareRoot12`/`OuterProduct12` as game code); coverage asserted; and **`--selftest`** with a known-true case of EVERY spelling — hand counts went 116→112→108→178→199 because the sources use both `".ent\tNAME\n"` and `".ent NAME\n"`, and a bare `".ent\t"` fragment yields a phantom function called `t`. Run `--selftest` before believing the number. |
| | `tools/jtbl_carve.py` — `_merge_pre` carries `--pre` forward | **(P32 T1a, cookbook §498)** `set_overlays_var` rebuilt `<bin>_JTBL_INTERLEAVE` from the carve set alone and DROPPED the binary's `--pre <obj>` clause (the resident's §8f leading-rodata sandwich, `--pre hdr.rodata.o` — the one binary in the fleet with a `--pre`). `make extract` then refused (`ld_interleave: hdr.rodata.o … would be parked with .text`), the build linked the STALE script, and the gate booked the draft as DIFF (249,252 differing bytes of artefact). `_merge_pre(existing_line, args)` now carries an existing `--pre` into the regenerated value; idempotent; a line without `--pre` is unchanged (every overlay). Unit control on 4 shapes. |
| | `tools/split_indicator.py` — population derived, not stored | **(P32 T2c)** When `.run/S70_bins_sorted.txt` existed it WAS the population: a stored 213-name list that went stale at the first onboarding (R51) — tools-health printed "213 OK of 213" over a 218-binary fleet. The population is now derived from `config/splat.*.yaml` (+ main); the S70 file is an ORDER hint only, new names are appended, and the denominator is asserted equal to the yaml-derived set (R32). Verified 218/218 after the five module onboardings. |
| | `tools/payload_base_evidence.py` | **(P32 T2a, memory-map §S45 p7)** Static, controls-gated base evidence for a never-onboarded payload: module-id word, TEXT_LO estimate (first `addiu $sp,-N`), absolute-pointer set, `lui` hi-half histogram, and a **jal→function-start vote** (every `target − start` pair; ≥2 hits = a self-calling module — starts = prologues ∪ the word after each `jr $ra`+delay, because leaf functions have no prologue). Scores a BOUNDED candidate list (the 5 §S44 slots ∪ all 134 IDXTAB DESTPTRs ∪ the vote's top bases): STRONG (every internal jal lands on a start, ≥2) / CONSISTENT / INCONSISTENT / NO-EVIDENCE; prints an AMBIGUOUS tie set instead of picking; **OUTWARD-EXPLAINED** (T2b): a pure vote base whose internal targets are function starts of the fleet's overlays is downgraded (SC03/56's false STRONG); the requester cross-check (`req_fit`) is informational only — shared engine code makes every requester fit. **R39 `--controls`** (runs before ANY emission): the byte-proven bases of md_MAIN_008/011/013/042, md_SC03_073, md_SC02_009, md_SC07_004 must come out top-ranked from their payloads alone (7/7; every TEXT_LO estimate == the yaml's incl. the header-table modules 0x7C/0x14/0x158). **R43:** a payload with no self-reference at any candidate is REFUSED as base-independent. The byte gate (`new_binary.sh` first build) is the arbiter (P9). `--json` keeps the rows (`.run/P32/t2a/evidence.json`). |
| | `tools/interleave_check.py` — `--pre`-tolerant anchor | **(P32 T1a, cookbook §498)** The `--order` anchor was `:= --order`, so a line carrying `--pre <obj>` first (the resident) parsed as n=0 and printed a false DRIFT (rc 0 — a soft alarm). Now `:= (?:--pre \S+ )?--order`; `--fix` already preserved the prefix (it replaces only the order token). Control: ov_SC02_017 ALIGNED n=44 unchanged; resident ALIGNED n=5. |
| | `tools/harvest_verify.py` — post-carve extract rc checked | **(P32 T1a, cookbook §498)** `_jtbl_prep_one` ran `make extract` after a successful carve and IGNORED its exit code (R49/R61): a refused layout left the stale linker script in place and the whole-binary gate reported the draft as DIFF. Now a failed re-extract prints `!! extract-after-carve FAILED … CARVE refusal, NOT a draft verdict`, restores the carve snapshot, re-extracts, and returns refusal (CARVE-REFUSED class). |
| | `tools/jr_isolate_all.py` — `typedef struct Tag Alias;` keyed by the ALIAS | **(P32 T1b, cookbook §497)** The carried-type dedupe keyed every block by the names its regexes found; for a bodiless `typedef struct Rec801806C8_s Rec801806C8;` that was the TAG, so the typedef block and the tag's own `struct Rec801806C8_s {...} __attribute__((packed, aligned(1)));` definition collided under one key with different bodies and the R43 "CONFLICTING bodies — a rename is needed" refusal fired on legal C (ov_SC02_017). `_type_names` now keys such a line by the alias (`_TYPEDEF_TAG_ALIAS`), and the `carried` set learns the alias too; `typedef struct X X;` (alias == tag) keeps the old key so a second one still dedupes/refuses. Unit control on 7 block shapes; the refusing overlay's dry-run is CLEAN (2 region files) with no source rename. |
| | `tools/jr_isolate_all.py` — include-derived provided types | **(P32 T1a, cookbook §496)** The carried-type test consulted `_engine_types()` (engine_types.h + common.h) for EVERY TU, assuming each region `#include`s engine_core.h. Overlays do; the resident, the `md_*` modules and main's TUs include only `common.h`. A resident file-local typedef whose NAME engine_types.h also defines (`CdFileLoc`) was therefore NOT carried into the new regions ("the shared headers already define it") → `parse error before cdFileLocTable` in both region TUs, build rc 2 while the stale binary on disk read GREEN (R53). Now `_provided_types(header)` derives the provided set from the TU's own `#include` lines (engine_core.h ⇒ engine_types.h + common.h — engine_core's OWN typedefs live inside `DEFINE_func_*` macro bodies and are provided only where invoked; `common.h` ⇒ common.h only) and `_file_scope_decls(items, provided)` uses it at both decision points; `_engine_types()` kept for legacy callers. R39 controls: an overlay header yields exactly the legacy set (1,197 names); the resident header's set lacks `CdFileLoc`. Positive control: the resident 3-region split builds byte-identical (`8e17e02f…`). |
| | `tools/jr_isolate_all.py` — boundary derivation | **(P31 S74, cookbook §441)** `_region_emit_start()`: the yaml offset for a region is derived from the region's **CONTENT** — min of item addresses and of every `.globl`/`.ent` its text names that resolves inside the object — and taken as `min(cut, emit)`, so a boundary can only move DOWN. Reason: **a §265 verbatim `__asm__` body is not one of `parse_overlay_c`'s four addressed-anchor forms, so it attaches to the NEXT anchor as PREAMBLE — and preamble is assumed byte-neutral when it emits bytes.** A cut at `func_800D0268` would have moved 0x168 bytes of three other functions into the new object while the yaml claimed the region started higher. Where no verbatim asm is in play it equals the cut, so every existing isolate is unchanged. Also: an item-less CLOSING region used to emit a duplicate `- [off, c, …]` line (the empty-region skip covered only region 0, and `_partition`'s empty `footer` made the closing region look non-empty). |
| | `tools/jtbl_rodata_pads.py --derive` | **(Phase 31 S62 T3a, cookbook §303)** Module path of the §8e pads filter: the Makefile runs `--derive $(BINARY) --tu <tu>` for every `md_*` object **and for `main` (P31 S72)** — jump-table pads derived at build time from the retail island + the emission stream (trailing pads `0t1`, table-aware, const data passes through). No stored spec; an anchor miss fails the build with the offset. **S72:** `--derive` now serves main too — `_file0_vram` returns the code segment's `vram - start` (the PS-X EXE's 0x800 header), which makes both `raw[a - vram]` and `vram + <yaml offset>` correct for the EXE and leaves flat overlays byte-identical; `_splat_yaml` resolves main to `splat.us.exe.yaml`. |
| | `config/wave_exclude.txt` | **(P31 S72)** THE canonical wave exclude list — **tracked**, and **regenerated, never hand-edited**: `tools/exclude_audit.py config/wave_exclude.txt --write <new>`. Consumed as `draw_waves --exclude-file config/wave_exclude.txt`, which AUDITS it as a PREREQUISITE and refuses to draw on a stale one (`--exclude-stale-ok` overrides, loudly). Two classes: **CARVE-BLOCKED** (derived from `split_indicator`; disappears when the subseg is split — **EMPTY since P31 S74**, all four overlays split) and **WALL** (curated, cannot be re-derived — the `# WALL:` annotation is a PIN that survives regeneration, and its note is the refutation list to beat before reopening the entry). It replaces the nine `.run/S*_exclude.txt` snapshots and seven walls ledgers, none of which was authoritative; measured on its predecessor, 88 of 107 entries were stale one day after it was written, 46 of them open drawable work totalling 12,750 instructions. |
| | **main's TU layout (P31 S72)** | main's game code is no longer one file. `src/800.c` was split at the jtbl-span TU boundaries into **`src/800.c`** (vram 0x800123F0-0x8002B0B4, owns `.rodata` span A 0x80072A38-0x80072C70), **`src/800_b.c`** (0x8002B0B4-0x80035270, span B 0x80072E44-0x80073140) and **`src/800_c.c`** (0x80035270-0x8003A444, span C 0x800732A0-0x8007344C), plus **`src/800_shared.h`** for the 19 typedefs + 2 includes that cross. Reason: one code object contributes exactly ONE contiguous `.rodata` run, so each jump-table span needs its own object or every switch function in it double-emits its table (cookbook §426/§431). main's island is now a 7-piece sandwich and `ld_interleave` runs with `--order`, not `--front/--tail`. **Any tool that reads "main's source" must glob `corpus.src_files('main')`, never hardcode `src/800.c`** — `reconcile_slate` did, and saw 133 of 187 typedefs after the split. |
| | `tools/split_indicator.py` | **(P31 S72, cookbook §426/§431)** Which code subsegs MUST be split before their switch functions can bank: a code object contributes exactly ONE contiguous `.rodata` run, so a subseg owning raw jump tables in ≥2 non-adjacent island spans makes every switch function outside the one carveable span unbankable at any effort. **Decidable from the raw image at 0% matched** — no attempt needed. `--self-test` fires on main's pre-S72 island, stays silent on main today, and does not over-fire on a one-span subseg; linked PsyQ subsegs are excluded on principle. **(P31 S74)** Runs in `make tools-health` as a **HARD GATE** — the four violations it waited on are split, the fleet is **213 OK of 213**, and a new one now fails the build. The owner of a table is derived from the CONFIG by address, never from the stub's directory path: `make extract` does not prune a re-homed subseg's `nonmatchings/<old>/` dir, and reading it made the tool assert NEEDS SPLIT for a split that was already correct and byte-green (cookbook §436). Stale debris is now named in a `note:` line, which prints on an OK verdict too. |
| | `tools/overlay_src_split.py` `comment_open_at()` | **(P31 S74, cookbook §437)** THE comment-state oracle for the TU-split chain: per line, does it BEGIN inside a block comment. Every peeler in the chain used to ask `line.strip().startswith("/*")`, which is blind to a comment a construct OPENS MID-LINE and WRAPS — 238 such lines across 193 tracked `.c` files. The construct ends at its `;` *before* the `/*`, so the caller resumed on comment PROSE and `scan_construct` read it as code (`(s32,s32)` in the prose closes a depth-0 paren -> `seen_header` latches -> every later `;` reads as a K&R param decl -> one 'construct' swallows the whole preamble). Result: `parse_overlay_c` anchored a `def` on a pure declaration run, and `def_proto` emitted `extern #define CALL_… extern void func_…();` into the §8b carried decl layer — **`parse error before '#'`, which blocked five independently-MATCHed jr bodies in `ov_SC06_029`**. Now consulted by `parse_overlay_c`, `def_proto`, `split_src_region.parse` and `jr_isolate_all._file_scope_decls` (which also truncates a hoisted col-0 decl at an unterminated `/*` — that comment used to run on and silently eat the next carried decls). `parse_overlay_c` may now RAISE `ValueError` (R43) when a wrapped comment CLOSES with code after the `*/`, because that construct could never anchor — 0 occurrences fleet-wide, so callers see it only if someone writes one. Building the guard exposed a third defect in the same model: **`_strip` tested for `/*` before stripping `//`**, so a line comment containing `/*` (`// … src/*/*.c`, 7 lines in 5 sources) opened a phantom block comment and blanked every following line until some later `*/`; `_strip` now lexes left to right. Negative-controlled: `ov_SC06_029` byte-identical before the change and after, with the five bodies banked (sha1 `b7b0d4ae…`). |
| | `tools/exclude_audit.py` | **(P31 S72)** Classifies every wave-exclude entry by its CURRENT blocker — `BANKED` / `LINKED` / `RE-PROBE` (blocker since fixed) / `CARVE-BLOCKED` / `WALL` — and regenerates the list keeping only the still-valid classes. **`draw_waves --exclude-file` runs it as a PREREQUISITE and refuses a stale list** (`--exclude-stale-ok` overrides, loudly). Measured on `.run/S71_exclude.txt` one day after it was written: 88 of 107 entries stale, of which 46 were open drawable functions totalling 12,750 instructions — `main:SaveLoadRoutine` among them. |
| | `tools/main_diff_locate.py` | **(P31 S72, cookbook §426/§427)** Turns a RED main gate — whose whole output is two SHA1s — into a NAMED list of divergent symbols: byte-diffs `build/us/SLUS_007.26` against `extracted/retail/SLUS_007.26`, coalesces the differing bytes into runs, and attributes them PER BYTE to symbols via `build/us/SLUS_007.26.map` (file-offset mapping DERIVED from each output section's `load address`, not the 0x800 header constant). `--focus <fn>` prints the routing verdict, FOUR of them: **BODY REJECT** (divergence confined to the function) · **TABLE REJECT** (§405-A — `.text` byte-identical, all divergence in `.rodata`, i.e. its own jump table: fix the case VALUES/ORDER, do NOT respell the body and do NOT route to §376; checked FIRST because it was mislabelled PLUMBING until `func_800316F8` produced it) · **PLUMBING REJECT** (the function is byte-identical, everything differs elsewhere IN CODE → the §376/§378 chain) · **MIXED**. `--self-test <addr>` is the negative control: flips one byte at a known address and asserts the containing symbol is named, plus the identical-pair-reports-zero direction. `gate_main` calls it automatically and preserves the red image + map under `.run/gate_main_fail/` BEFORE the R40 baseline control rebuilds over them. |
| | `tools/blocker_probe.py` | **(Phase 29 SESSION-16, cookbook §65)** WHY a byte-correct draft fails the whole-binary gate. Read-only; **two oracles** (R34): STATIC (`cdecl.parse` + `cdecl.compatible` — cc1's own acceptance question, never text equality) beside the REAL cc1 (via `rtu_match`), leading with the DISAGREEMENT table. Classes `self_decl_hdr`/`self_decl_tu`/`callee_decl`/`data_decl`/`local_type`, each mapped to a blast-radius tier (T0 draft-only / T1 binary-local / T2 fleet-shared). Blockers STACK, so a function's tier is the MAX over them. 36 drafts in ~9 s. Replaced+deleted `.run/diag_plumbing.py`. |
| | `tools/demacroize.py` | **(Phase 29 SESSION-16, cookbook §65b)** The per-overlay-local escape from a shared-header decl conflict — the largest stranded class, and the one §20 called unrecoverable. The conflicting `extern` lives INSIDE a `DEFINE_func_*` body, so it exists only at instantiation sites: this expands those instantiations **in the overlay's own TU**, correcting only the conflicting decl to the draft's byte-true sig (never dropping it, §57a-1). **T1** — writes confined to `src/<binary>/**`, so the per-binary gate suffices and no R22 risk is created by construction (contrast `fix_header_decl`, fleet-blind, §63 UPDATE). `--emit-edits` (read-only, feeds `rtu_match`) / `--apply`. **Price: the function can no longer propagate ×138 — such a bank is ×1** (full distinct-code credit, ~1/138 of instr). |
| | `tools/recover_integration.py` | **(Phase 24 T6; extended Phase 29 SESSION-16/17, cookbook §65/§66)** The stranded-draft recovery DRIVER — consumes a wave dir (`--draft-dir`, repeatable; the backlog is the wrong source: unreliable `closeness`, overlay-specific drafts), runs the declared `--stages` (`demacroize` T1 / `arity` T2), then gates in TWO passes (**gate all → exact snapshot-restore → re-stage winners only**), so a non-bank never leaves an edit behind. `--run-id` puts all scratch + `verified_out`/`failed_out` under `.run/recover/<id>/` (closes §55b trap 4); bank truth is `banked_from_source()` (the stub is GONE from src), never a gate report; `stub_map` derives from `corpus.stubs` (R33). **Blast-radius tiers are ENFORCED** (`--max-tier`, write-set assertion), and **propagation is itself fleet-tier**: it needs `--max-tier fleet` AND `--r22`, and is refused outright after `demacroize` (those banks are ×1 by construction; `--auto-from` would re-macroize and undo them) — both refusals negative-control-tested, exit 1. `--probe-only` / `--report`. Success path verified end-to-end by the §66 free re-bank test. |
| | `tools/lift_types.py` · `tools/uniquify_type.py` | **(Phase 29 SESSION-14, cookbook §64/§64a)** Fleet-wide type lift into `src/shared/engine_types.h`, and the camp-uniquify that must precede it for VARIANT names (same identifier, different layouts in different TUs — reconciling them merges two layouts and breaks the minority camp). Both are **T2**: dry-run by default, and **R22 clean-fleet is the arbiter, not the per-binary gate**. **P38 T5.1:** `--census TSV` (relocate every def type_census reports outside the canonical files; `--rename` pass 1, `--exclude`/`--excluded-out`) and `--fold TSV` (one canonical name per class of type_census's `dup_gating.tsv`; `--fold-tier1`/`--fold-sdk`/`--fold-class`/`--fold-exclude`/`--fold-canon`/`--fold-report`; `--fold-dead CENSUS_JSON` deletes dead canon types). Gotcha: a layout hash must include alignment attrs. |
| | `tools/struct_twins.py` | **(P38 T5.1)** Drafts the evidence for tier-2 (all-placeholder) duplicate-layout classes from the census dir: SEPARATE / SHARED / UNBOUND per class, causes from struct_map use sites, PsyQ SDK layout as a note. `--write` regenerates `## Twins` of `docs/struct-twins.md` (which `type_census --check-structs` reads: stale lhash fails, `twins_listed=N` non-gating); `--show LHASH`. Run after every fold, as its own call after the census. |
| | `tools/backlog.py` | Near-miss ledger (`.run/backlog.jsonl` + `docs/backlog.md`); **fleet-aware** `load_best` (a 077-matched-but-stuck-local fn surfaces via its overlay record; Phase 23). | **S84 repair (2026-09-06):** `load_best` now keeps the LATEST record at equal closeness (its docstring's contract; the `(cscore, ts) <= …` compare had kept the EARLIEST, so a re-verdict at the same closeness never rendered — func_80032A74's PROVED row).
| | `tools/lora_grind.py` | Mass-run driver: rotate binaries → draft open ≤N-ins stubs with the served model → gate → propagate. `--min-reach N` (Phase 23) targets shared fns (sig-reach oracle == `dedup_propagate`). |
| | `tools/bulk_harvest.py` | **(Phase 23 / T10)** The phase-separated + parallel-gate harvester (throughput rebuild of `lora_grind`): **(A)** bulk-draft K fresh ≤N-ins stubs (GPU) → **(B)** `ProcessPoolExecutor --workers` byte-gate over DISTINCT binaries (`build/<bin>/**` isolated; `run_gate` per-binary lock + per-worker scratch, `propagate=False`/`commit=False`/`compute_fleet=False`) → **(C)** dedupe-once + ONE commit. Round-robin fuel spread; STOP-sentinel; on-demand/bounded. Measured 2026-07-01: gate **0.4s/fn** (8 workers, ~75× the serial gate) ⇒ drafting is the bottleneck (→ vLLM next). Run: `API_BASE=… MODEL=bfm-match-7b-v3 tools/bulk_harvest.py --binary-glob 'ov_SC03_*' --count 80 --workers 8 --measure`. `lora_grind` kept as the serial fallback. |
| | `tools/grinder.py` | Token-free decomp-permuter daemon on the backlog near-misses; **per-binary** (Phase-23 fix). `auto_supervisor.sh`/`auto_stop.sh` keep-alive + STOP sentinel. **Phase-24 T5:** auto-threads the residual `klass`/`where_stuck` into `p16_permute.setup` (§31-directed weights) + **input-changed idle gating** (`draft_sig`) replacing the blind `tried.clear()` churn. **Phase-29 T13A TARGETING:** `candidates()` filters on the MEASURED residual bucket from `autopsy.verdicts()` (1,303 → 78) and takes its directed profile from the measured class, not the logged label (91% of records have none, so the search silently ran on gcc defaults). Measured: of the 972 records this filter admitted, only **75 (7.7%) were permuter-shaped** — ~92% of the daemon's CPU was going where a search-closer provably cannot win, which is why it banked 0 after Phase 21. Degrades to undirected if the corpus is absent and says which mode it is in; `--no-targeting` A/Bs it. |
| | `tools/residual_class.py` + `tools/test_residual_class.py` | **(Phase 29 T13A, cookbook §60)** The DETERMINISTIC residual→class classifier. Decodes each mismatching MIPS word into `(operation-skeleton, register-fields, immediate)` and names the class from the bytes: `REGALLOC-PERM` (consistent injective register map = §31 S11/RC-3) · `SCHEDULE-REORDER` (same multiset, different order) · `DELAY-SLOT` · `LENGTH-DRIFT` / `SIZE-MISMATCH` (drift detection runs FIRST — one inserted instruction inflates `closeness` by the tail length) · `WIDTH` / `BRANCH-POLARITY` / `STRENGTH` / `ADDRESSING` / `IMM-OFFSET` / `IMM-VALUE`. Every class routes to a **bucket** (`permuter`/`structural`/`integration`/`redraft`) = WHICH TOOL the failure wants. Uncovered opcode → `UNKNOWN`, counted (R32). 16 synthetic unit tests. `classify_streams` is authoritative (full streams); `classify_residual` is the declared-degraded path for capped stored residuals. |
| | `tools/autopsy.py` | **(Phase 29 T13A)** `collect` materialises the residual corpus by recompiling every open backlog draft through the EXISTING `match_one` path (R33), deriving the asm subdir (stub's self-describing INCLUDE_ASM) and the **-O0 flag** (`corpus.is_o0`) — both silent-artefact generators if guessed. 1,752 drafts / ~21 s at -j12; asserts its closeness against `masked_diff.structured_diff` on every row (R34) and refuses the corpus on any disagreement. `report` → `docs/autopsy.md` (buckets, classes, reach-weighted, `sig` clusters, and the second-oracle cross-tab vs the recorded label). `verdicts()` is the consumer accessor. |
| | `tools/corpus.py` `o0_sources()` / `is_o0()` | **(Phase 29 T13A)** The opt-level oracle, DERIVED from the Makefile's own `-O0` `CC1FLAGS` rules (explicit targets + `$(wildcard)`-built object lists) with a coverage assertion: an unrecognised rule form raises rather than mislabelling. Scoring an -O0 target's draft at -O2 makes the whole residual an artefact — the trap this phase hit four times. |
| | `tools/permuter_weights.py` | **(Phase 24 T5, cookbook §3b)** §31-directed permuter mutation: `classify(klass, where)` → `regalloc\|schedule\|cse\|None`, `render_settings_toml()` emits the `[weight_overrides]` table decomp-permuter merges over the gcc defaults (`main.py:336` / `helpers.py:merge`). Biases pass-selection toward the class's §31 levers (`perm_reorder_decls` RC-1/3, `perm_reorder_stmts` RC-2/S1, `perm_temp_for_expr` S2, `perm_commutative` cse) and away from value/type noise. No submodule edit. `klass=None` → gcc defaults (unchanged). |
| | `tools/p16_permute.py` | Per-fn permuter driver: `setup(fn, draft, asm_subdir, klass=, where=)` builds `base.c`+`target.o`+`settings.toml` (now with the T5 `[weight_overrides]`), `run_permuter` via `run_masked.py` (T2 floor-free masked scorer). `--klass` overrides the backlog auto-lookup. |
| | `tools/lint_symbol_refs.py` | **(Phase 24 T5c)** Guard against the Phase-21-class breakage: flags every `func_<ADDR>` token in committed `src/` whose address has a CURATED name in the symbol files and no `func_<ADDR>` symbol (a rename that would break a genuinely-clean rebuild but is masked by incremental builds — R22). Comment/string-aware. Exit 1 on stale refs. **Run after any symbols rename**; candidate for `make report`. Caught the T5b (macro-call) + T5c (INCLUDE_ASM) refs. **P31 S78:** also scans the §265 verbatim `__asm__("...")` string bodies (`.ent\tfunc_X`, `.globl func_X`, `func_X:`), where the masked scan was blind and a `\b`-regex rename misses `\tfunc_X`; `__asm__`-label-bound addresses exempted (negative-controlled on the passing tree). |
| | `tools/api_draft.py` | Provider-agnostic LEAN drafter against the served model's OpenAI endpoint. `LEAN_SYS` carries the "translate every instruction, never an empty body" clause (Phase 23 — fixes the v2 empty-leaf overfit). |
| | `tools/serve_local.py` | **Serve the fine-tuned model on the GPU** (base+LoRA via Unsloth, `.venv-train`, OpenAI endpoint) — the in-repo replacement for LM Studio. Run: `LD_LIBRARY_PATH=$(ls -d .venv-train/lib/python3.12/site-packages/nvidia/*/lib \| tr '\n' :) .venv-train/bin/python tools/serve_local.py --adapter models/bfm-match-7b-v3 --name bfm-match-7b-v3 --port 1234`. (Prebuilt `llama-cpp-python` CUDA wheels SIGILL on this no-AVX-512 CPU; the Unsloth/torch path is reliable, no build.) |
| | `tools/export_pairs.py` / `format_finetune.py` / `train_lora.py` / `eval_lora.py` | The corpus→LoRA pipeline (`.venv-train`): mine (asm↔C) pairs incl. the `engine_core.h` **macro bodies** + `engine_types.h` structs (corpus-v3) → Qwen chat-template + compile-filter → Unsloth QLoRA (3080 Ti) → held-out gate-true eval. Datasets/weights gitignored (`datasets/`, `models/`, `.venv-train/`). |
| **The derived oracles** (Phase 26-A tooling audit; R33 before R32 — *the best outcome is a deleted scanner, not a fixed regex*) | `tools/corpus.py` + **`make audit-corpus`** | THE corpus oracle. Derives from the FILESYSTEM (which `.c` files make a binary; the `INCLUDE_ASM` line is self-describing — its first argument *is* the asm subdir) and from the PROVEN INVARIANT (`matched = sig − stubs`, never re-parsed from C). Killed ~10 hand-maintained layout models. `audit-corpus` is a **second oracle that can disagree**: it cross-checks splat's boundaries against `sig_image`'s independent ones (0 phantom + 0 truncated since A4; was 193 unmatchable slices). |
| | `tools/audit_digest.py` + **`make audit-digest`** | **(P30 S1e, cookbook §140)** The **scoreboard** oracle: recomputes the three headline metrics from the CURRENT tree and fails if the committed `docs/progress.fleet.md` disagrees. Wired into `tools-health` AFTER `report`. Exists because a digest generated from a working tree that later changed (work reverted before the commit landed) is **byte-invisible** — `check-all` stays 140/140 over it forever (R34: the byte-gate is a null oracle for DOCUMENTS) — and the next honest regeneration then reads as a REGRESSION that never happened. That is exactly what the `10f954627` digest did: overstated **+7,879 ins / +130 unique fns**, which parked the phase's best lever on a phantom for a session. Compares **integers, not the printed percentages** (the staleness rendered as "94.4%" on both sides). Negative-control-proven against that stale digest. Same task hardened `progress.py stub_addrs`, which wrapped the fail-closed `corpus.stubs` in a bare `except` → empty stub set → `matched = sig − stubs` credited EVERY function: byte-witnessed reporting **instr 100.00% / distinct 100.00%** in a tree with no `asm/`. The identical swallow was fixed in `cast_call_sites.tu_for` + `reconcile_tu.tu_for`, where it silently reconciled drafts against the default `<ov>.c` instead of the jr/-O0 split TU — the very bug `cast_call_sites`' docstring exists to fix. |
| | `tools/cdecl.py` + **`make audit-cdecl`** | **THE C-declaration oracle (cookbook §51g).** ONE recursive-descent parser of C's **declarator grammar**, replacing fifteen tools' private regex models — models that disagreed with each other and were, all fifteen, blind to fn-ptr/jump-table decls (`extern void (*D_X[])(void);`), sized arrays (`[4]`), and multi-declarators (where the *whole line* was dropped). Total by construction, not by shape enumeration. **Two statement paths, because the inputs differ:** `tu_statements()` derives a TU's file scope from **`cpp`** (a decl inside a `DEFINE_func_*` macro body declares nothing until invoked — §8c; 54 ms/TU), and `split_statements()` is a **span-preserving** raw split for drafts (which get rewritten). API: `parse` / `scope` / `tu_scope` / `Declarator{name,kind,type,params,pnames,is_proto,is_definition}`. Verified: **2,952,246 depth-0 statements → 2,731,521 declarators, 0 parser defects**; **50,405 distinct declarations round-tripped through the real cross-gcc, 0 rejected**; residue adjudicated NOT-C *by gcc*, not by opinion. **Phase-27 T4 — the canonical draft-typedef strip:** `typedef_names(tu_path)` (the names a TU declares as typedefs, robust `tu_statements`-based so a coverage gap can't crash the byte-gate) + `strip_provided_typedefs(draft, provided)` (drop a draft's self-contained typedefs the target already supplies, splitting multi-typedef lines and covering scalar AND struct typedefs). Replaced **six** copied scalar-name regexes with complementary holes: `harvest_verify` now strips per-TU (unblocks the 39 struct-typedef drafts `_TD` dropped) and **surfaces cc1 stderr** so a `redefinition`/`conflicting types` failure reports as **PLUMBING**, not a byte mismatch (`.run/harvest_failed.classified.txt`); `masked_diff.strip_scalar_typedefs()` (used by `match_one`/`p16_permute`) fixes the multi-typedef-line skip that discarded 42 masked-MATCH drafts over whitespace (`func_8015C030` → `MATCH (23 ins)` unedited). `canon_sig_reconcile`/`eval_lora`/`format_finetune` keep their own copies for now (migrate per-bank, byte-gated — the audit-prescribed cadence). |
| | Phase 26-A tool-hygiene close (A9d–A10) | **DELETED** (R33, dead Phase-17 chain): `tools/census_conflict_callees.py` + `tools/derive_canonical_sigs.py` — `reconcile_tu`/`cdecl` answer their question from the build. **`overlay_src_split.py`**: `scan_construct` force_decl latch fixed (no longer swallows a def sharing a line with leading externs) + `hidden_definitions()` R32 coverage oracle wired into `selftest`. **`jr_isolate_all.py` `jr_inventory`**: `banked` DERIVED FROM THE IMAGE (`family_remap.reloc_targets` owns-a-carve) not a gitignored roster (R33) + curated-name via `addr_of` + 1:1 carve-ownership assert. **`family_remap.reloc_targets`**: optional `data=` param (read the image once, pass to N calls). **`backlog.py`**: `BACKLOG_NO_RENDER` env so parallel `gate_stage` workers skip the render race (append is atomic). `reconcile_tu` confirmed live on BOTH banking paths (`gate_stage` + `jtbl_family_bank.recover`→`bank_exemplar`). |
| | **S68 tooling, round 2 — retrieval, triage, walls, liveness** (P31 S68) | **`tools/neighbor_ref.py` (NEW)** — for an OPEN stub, the already-MATCHED functions worth READING as worked examples, ranked SAME-TU first, then same binary, then shape (li-normalised skeleton / call-sequence hash / reloc-kind sequence / CFG counts / opcode cosine, all precomputed in `.run/feat.*.jsonl`), with a hard penalty for opt-level mismatch (§116). Surfaces the neighbour's HEADER COMMENT — the payload agents actually consume. Answers the question `seed_ref` cannot: seed_ref needs a hash-identical twin, this finds a near-twin. **Run it for every card.** Measured motivation: S68's cheapest large matches all came from a neighbour (555 ins/72k, 657 ins/122k first compile, 753 ins/177k) while neighbour-less main fns ran 200-350k for ~80 ins. **`tools/wall_sweep.py` (NEW)** — enumerates the §332 delay-slot macro walls (10 fns / 1,027 ins); `--emit-exclude` feeds `draw_waves --exclude`. **Run before every draw AND before every escalation** — an escalation cannot beat the toolchain. **`tools/residual_rules.py` + `residual_rules_b.py` (NEW, experimental)** — residual→cookbook-lever classifiers; `_b` is the stronger (113/113 classified, 63% certain/high). NOT yet wired into the pipeline: see `docs/next-session-triage-ladder.md`. **`tools/lane_inflight.py` (NEW)** — the RECORDED liveness ledger (`add` on launch, `done` on verdict); `list` exits non-zero while any agent is live and IS the guard. **`tools/r22_verify.sh` (NEW)** — the clean-fleet verify, EXCLUSIVE by construction: it refuses while `lane_inflight` shows live agents, because `make clean` deletes `asm/` and drafting agents READ it. Clears `.run/R22_DEBT` only on a genuinely green run. |
| | **S68 tooling — the gater lane + the -O0 route** (P31 S68) | **`tools/gater_lane.py` (NEW)** — the continuous gater: drains a wave's finished drafts into `parallel_gate`, grouped by binary, `--r22` by default. Ledger AND verdicts keyed **`binary:fn:arm`** (R48 — and an escalation ALWAYS has a prior verdict, so arm-keying is what stops an in-flight fable draft riding the opus one). `--extra BINARY:PATH` for non-wave drafts, `--skip-binary` for lanes that may be writing, and **main is routed IN-TREE via `harvest_verify`** because `parallel_gate`'s worktree cannot stage main's psyq_integrate link inputs. **`tools/workflows/escalate_fable.js` (NEW)** — warm-started escalation: passes the prior draft + its measured closeness + its ruled-out levers, and demands a reusable `new_idiom`. **`tools/o0_boundary.py` (NEW)** — the stranded-boundary -O0 sweep (141 binaries / 288 boundaries / 0 candidates: the class is EXHAUSTED, and that null is negative-controlled). |
| | **S69 tooling — the TRIAGE LADDER** (P31 S69) | **`tools/triage_ladder.py` (NEW)** — the zero-token pass that answers *"does this target need an agent at all?"* before one is spent. **The PRE/POST split is the point:** `--pre <wave>` runs the TARGET-SIDE tiers only (BANKED · WALL-332 · PARKED) — no draft, no build, milliseconds — and writes `<wave>/triage.json` + `triage_exclude.txt`; `--post` runs the full `residual_rules_b` residual routing, which needs a draft and runs `match_one`. `--escalate B:FN` exits 2 on a walled/banked target (the check S68 was missing when it escalated a §332 wall at closeness 8). `--acceptance` is the R39/R32 harness: false-skip over EVERY open stub, recall over sampled matched fns, and a wall negative control over already-banked code — all pure filesystem work, so it runs in seconds over the whole corpus. **It REFUSES on a non-quiescent tree** (`pgrep -af` rows for gate/build lanes + `lane_inflight`): a live gate makes the stub oracle transiently wrong in both directions (§377). Wired into `wave_args.py` (drops walled/parked targets at draw time, reusing `pre_classify` — one implementation, R33) and into `tools/workflows/escalate_fable.js`, which now REFUSES any target that does not carry `triage:'DRAFT'`. **Routing correction (§376):** the `INTEG-STANDALONE-MATCH` / `NOCOMPILE-UNDECLARED-*` tiers are GATE-FIRST candidates, never free banks — S69 gated that class **0/28** raw; the route is `fix_arity_callers --any-proto` then the gate. |
| | **S69 tooling — the §378 SELF-CALLER CAST** (P31 S69) | **`tools/cast_self_callers.py` (NEW)** — the mirror of `cast_call_sites.py`. That one fixes the DRAFT calling a conflicting CALLEE; this one fixes the TU's OWN already-banked code calling the function the draft is about to DEFINE, which nothing handled and which is the terminal blocker of the §376 pile. **Never run it alone** — it answers the error that `fix_arity_callers --any-proto` CREATES: no-protoing the conflicting decl makes the draft's definition the prototype in scope, so the TU's own call fails anew with `too few arguments`. Order is `arity -> self-cast -> [--sync-decls] -> gate`. Byte-neutral because gcc-2.7.2 folds a cast of a known function symbol back to a direct `jal`. `--sync-decls` covers the narrow-param case a no-proto decl CANNOT legally reach (C89 requires promotion-stable parameter types when one declaration has no prototype, so `void f(s16)` is illegal against `extern void f();` — which is exactly why `fix_arity_callers` skips it as 'narrow-param'); safe only once the call sites are cast, because a declaration then emits no code. It REFUSES a function whose return type it cannot read off the draft (R43). **Journals every edit, and `--undo-journal --keep <banked>` after the gate is MANDATORY** — a leftover cast made `ov_SC07_000` stop compiling and every later gate verdict on it measured a broken baseline. Wired into `recover_integration.py --stages arity,self-cast` (tier `binary`), prescribed by `residual_rules_b`'s decl-conflict tiers, and in the playbook §4b. Measured: **8 banked of 28** including `main/func_80036D58` at zero agent tokens; generalises to the callee the diagnostic NAMES (banked `main/func_80021D38` that way). |
| | **S69 tooling — the NEAR-TWIN BAND** (P31 S69) | **`tools/seed_ref.py --near [--max-d N] [--near-control]` (WIDENED, not a new tool — R33)** — the fleet-wide twin oracle gained an edit-distance tier over reloc-normalized instruction streams, because the exact-hash tier answers only *"is there a byte-identical copy?"* while a frontier needs *"is there anything CLOSE?"*. **Measured widening: 22 of 352 reachable open stubs had a d=0 hash twin; 75 of 352 (21%) have a banked match at d<=25 — 3.4x.** Root cause of the gap is §389: `h_norm`'s normalizer drops its pending lui-hi on an intervening R-type, so indexed-global reloc twins hash differently and are invisible to seed_ref/twin_sweep/dedup/family-maps simultaneously. **Do NOT fix `h_norm`** — every stored map and calibration keys on it; the near tier reads through the hole. Verification built in: sound prefilters that cannot lose a true pair, R32 population assertion, R34 cross-check reproducing all 22 exact twins every run, R39 controls (positive 200/200 at d=0; random-pair base rate 1.17%). Classes emitted: HASH-TWIN · RELOC-ONLY (mechanical — remap via `family_remap` and gate; **8 of 10 banked at ~0 agent tokens on first use**) · NEAR-COUSIN (seeded crack). Known remaining gap: `family_sweep.load_sigs` globs `ov_` only, so 38 md_ + 5 resident + 67 main are structurally invisible to it (a 407-ins md_SC05_026 PURE twin of banked ov_MAIN_012 code was found in the wild). Full audit: `.run/S69_fable/report.md`. |
| | **S69 tooling — the CONTAINED tier + the twin LADDER** (P31 S69) | **`tools/seed_ref.py --contained [--contained-control]` (WIDENED again, R33)** — finds an open stub that is a banked body **plus or minus whole blocks** (any gap size), the class edit-distance ranks badly. Needs branch-offset masking (unmasked offsets veto exactly the target pairs) and a min-side-25 floor (89% of raw hits were prologue/epilogue vacuity). **Ranks by (substitutions+regions, cover), NOT by d** — §390's law: a deletion is free, a substitution is thought; the near tier ranked a d=5 substitution twin ABOVE a same-C-minus-one-statement pair that banked at closeness 0. Verified: planted-deletion positive control 60/60, random-pair base rate 0/397, R32 population assert 346/346, `--near` regression reproduces the stored slice exactly (352/22/28). **Use the ladder in playbook §2a-2** (exact → RELOC-ONLY → CONTAINED → cousin → cold), filtering lookalikes at `r = d/min(nins) >= ~0.3`. **Yield: 9 usable stubs, 1 banked.** And the standing conclusion: three fleet-wide probes past RELOC-ONLY returned 0 new / 9 / 2 — **the scanner well is dry; spend integration effort (§376/§378), not scanner effort.** Audit: `.run/S69_fable2/report.md`. |
| | **S69 tooling — the carve artifact fix + `asm_verbatim`** (P31 S69, Fable-3 audit) | **`tools/parallel_gate.py` (FIXED)** — `stage_generated` now symlinks `.run/sig.<binary>.jsonl` into each worktree. It is gitignored, so no worktree had it, so `jr_isolate_all.jr_inventory` read EVERY carve as UNOWNED, R32-aborted, and every jtbl draft was booked **CARVE-REFUSED** — a verdict about the WORKTREE, not the function (§322b). Re-check any CARVE-REFUSED recorded before this fix. Absence is now reported in `missing_generated` rather than silently skipped. **`tools/asm_verbatim.py` (NEW)** — `.s` → §265 file-scope `__asm__` block: decimal immediates AND offsets, comma-no-space operands (maspsx dies on `sltu $v0, $s0, $v1`), derived `.frame`/`.mask`, R43 refusals for rodata/jtbl. Ledger **MATCH 12 / NEAR 1 / REFUSED 2** + a non-wall control. Byte-equivalent to the stub BY CONSTRUCTION — use only for genuine hand-asm; §265 accounting applies. **But prefer the real fix first:** the §332 walls are a per-OBJECT assembler mode, and a 3-line maspsx reorder-passthrough + `as -O2` is byte-inert on the whole 800c3/800c2 objects and gives 0 diffs for 6 walls that already have drafts (§332b) — that route retires `oracle_reorder.py`. Audit: `.run/S69_fable3/report.md`. |
| | **S69 tooling — the two HABIT tools** (P31 S69) | **`tools/verify_binary.py` (NEW)** — the only correct way to byte-verify one binary by hand. `make build BINARY=<b>` is NOT a verification after a gate that touched `config/`: a carve rewrites splat INPUTS, so a build without a re-extract links newly-carved C against stale state and the SHA is meaningless (§384). It ALWAYS re-extracts by default (`--no-extract` to opt out), passes `-j`, and exits non-zero only on a genuine mismatch; `--all-touched` sweeps every binary with uncommitted `src/`/`config/` changes. **Written because the rule was documented after the FIRST violation and then broken twice more by reflex** — costing a reverted 96-line match and 23 reverted decl edits, both later restored byte-identical. **`tools/twin_rescan.py` (NEW)** — run after EVERY gate that banked. The twin oracle answers *is there a BANKED body like this?*, so an open-open cluster reports 'no banked twin' for all members and that answer is stale the moment one banks (§397). It diffs the scan against the previous snapshot and prints **only what just became free**, with the ready-to-run `family_remap` command. Baseline at first run: 318 open stubs, **37 already carry a banked twin at d<=5**. |
| | **S69 fix — `parallel_gate` adopts NEW carve files** (P31 S69) | The merge-safety check compared `git show <pin>:<path>`'s **stdout** (`""` when the path is not at the pin) against `None` (absent from the main tree), so a file present in NEITHER — exactly what a jtbl carve creates when it splits a TU into `src/<bin>/<bin>_jr_<addr>.c` — read as *"main tree moved under them"* and was **refused and left untracked**. Eight accumulated in one session; nothing failed locally (file on disk, R22 green) but the naming yaml IS committed, so a fresh clone would get config without source. Now distinguishes *not at the pin* from *empty at the pin* via git show's RETURN CODE, and **prints every new adoption** rather than merging it silently (R32). Cookbook §400. **Docstring correction while here:** this tool does NOT use `git add -u src/` (that is `gate_stage`'s form) — it adds exactly the adopted paths, which is why the baseline check decides what gets committed; a stale reading of that line misdirected the first diagnosis. |
| | **S68 fixes — six instances of the overlay-layout assumption** (cookbook **§363**) | `dedup_propagate` could not even IMPORT (`os.` at module level in the one module that imports `os as _os`). `seed_ref` offered main's LINKED-subseg DEAD TEXT as bankable twins (43 of 82 hits — a draft there gates GREEN while wrong); now refuses and COUNTS the refusal. `parallel_gate.stage_generated` hard-coded `build/<b>/<b>.ld`; now asks the Makefile for `<b>_LD_SCRIPT`/`<b>_UNDEF_SYMS`/`<b>_UNDEF_FUNCS` and REFUSES when absent. `rtu_match` gained **`--tu`** (+ `blocker_probe` passes `stub.path` and `stub.asm_dir`) — it reconstructed `src/<source>/<split>.c`, which is the overlay layout; main's sources are LOOSE FILES in `src/`. `gate_stage` no longer synthesises `--out`/`--good-sha` — for main those resolved to a nonexistent path and then **ov_SC01_077's SHA** via `DEF_SHA`. **`psyq_integrate`**: the `*_externals.ld` map is now MONOTONIC — it was re-derived against the CURRENT `.ld`, so `firstfile = 0x80061FA8;` was DROPPED on every incremental relink and main was 2 bytes red before any draft was spliced (**the true identity of the 2026-08-15 'main link defect'**). |
| | **S68 — the module-binary -O0 carve route** (cookbook **§371**) | `jr_isolate_all` + `overlay_src_split` + the **Makefile -O0 glob widened to `src/md_*/md_*_o0?.c`** open carving for the single-object `md_*` binaries. Three stacked causes behind one `unaddressable content` message (interior-YAML-comment symbol-list truncation; a trailing verbatim-asm chunk with no region; bare tag forward decls), then the **spimdisasm rodata-migration trap**: migrated rodata follows its function ONLY within the same subseg, so a carve silently drops it and `INCLUDE_RODATA` cannot bring it back — rename the `.rodata` subseg to the object its emitters moved to. **The Makefile hunk MUST be committed with the carve** or a fresh clone loses -O0 on the region and every draft banked there mystery-fails. |
| | `tools/recover_rejects.py` | **(P31 S59)** Free recovery of PRE-GATE rejects, wired into the maintenance lane. Two paths exist for a draft that does not bank and only one was recorded: a gate failure gets a backlog row (closeness/class/best draft), while a draft the reloc pre-filter drops reached nothing — **569 of 1,261 drafts over eight waves, 45%**. Of the `MISMATCH?` rejects, **13% carry `shape: MATCH`** — right body, wrong symbol names, i.e. the §171 stale-seed class `aprop_symfix` rebases deterministically. Reads `.run/reloc_rejects.jsonl` (written by `ox_campaign.reloc_filter`), keeps shape-MATCH rows that are STILL open stubs, runs `aprop_symfix --fix`, and STAGES the rebased bodies into `.run/sweep_maint/<bin>/` for the lane's existing free gate. It never substitutes, gates or commits — a bad recovery can waste a build, never a bank. Tried-once is remembered in `.run/recover_rejects_seen.json`. Zero model tokens. |
| | `tools/lanes/restart_main_lane_when_idle.sh` | **(P31 S59)** Restart the main lane's SHELL at its one safe boundary — no main-lane agents alive AND no `gate_main` running, i.e. between its gate and its next draw — so an env/arg change (`MAXTOK`, `HTTP_TIMEOUT`) lands without discarding drafted work or aborting a batch. Companion to `relaunch_drafter_shell.sh` (wave boundary) and `restart_gater_when_idle.sh` (no sweep in flight); see `docs/accelerators.md` #5 for why a running lane never reads your edit. |
| | **campaign constants** (`MAXTOK` / `HTTP_TIMEOUT`) | **(P31 S59, probed against `stealth/ox-alpha`)** `--maxtok 16000` and `export HTTP_TIMEOUT=700` on both drafting lanes — **one setting, not two**. ox reports `reasoning_tokens=0` (its thinking is IN the content stream), so the output cap WAS the reasoning cap: measured over ALL turns, wave `bk` at 8k truncated **240 of 3,222 = 7.4%** and wave `bt` at 16k truncated **16 of 1,210 = 1.3%** (~6x better); an early count of **240 of 244** compared truncated turns against turns that printed a finish reason — against themselves — and wrongly read as ~100%. A truncated turn is a tax of one turn in 24, not a lost agent: the next turn emits the tool call. An uncapped hard prompt wanted **8,067** tokens. It generates at **~30 tok/s**, so 16k needs ~530 s and the old 420 s socket would have killed those turns (a timeout wastes the whole turn; truncation leaves a partial). Model ceiling is 1M context / **131,072** max completion, so 16k is our choice, not a limit. Ordering that must hold: generation < `HTTP_TIMEOUT` (700) < stallguard's wedged-agent kill (1200 s). `REASON_CAP` works on ox but shortens the ANSWER too (618-672 tokens) — a quality dial, not a truncation fix. Turn caps are NOT binding on the default lane (non-MATCH median 4 oracle calls, p90 12, of 24). |
| | `tools/main_lane.py` + `tools/lanes/main.sh` | **(P31 S59)** The EXE's own draft→gate→commit cadence, beside the overlay lanes. main is excluded from every wave draw because its gate is a clean whole-EXE rebuild that bisects; this lane runs that gate on its own clock. Parked drafts (`.run/main_queue/`) first, then `build_wave_atlas --only-bins main`; reloc pre-filter; `gate_main --apply` in batches of 40 (one rebuild verifies the slate, ~15 s measured); BISECTS a COMPILE conflict that gate_main refuses to bisect; credits a bank only when the INCLUDE_ASM line is gone AND main re-checks byte-identical; commits named paths (R42). Failures parked with a try count, capped at 2. |
| | `tools/lanes/distill.sh` + `tools/distill_scan.py` | **(P31 S59)** The flywheel's second half, beside drafting. The gater's per-wave `idiom_harvest` is EXTRACTION; this raises a READY marker (`.run/distill_ready/`) when ≥30 novel candidates or ≥2 waves accumulate, and a human + an **Opus/Sonnet** subagent (never Fable) distills and lands the sections. State is `{tag: novel-count-when-mined}`, never a done-list — a re-gated wave rewrites its candidate file under the same tag. Measured: ~82-88% of candidates are already covered, and 1 in 3 credited levers is byte-inert (§266). |
| | `tools/lanes/relaunch_drafter_shell.sh`, `restart_gater_when_idle.sh`, `bounce_drafter_on_queue.sh` | **(P31 S59)** Ship a lane change without losing work. bash parses a `while…done` body up front, so a lane-ARG change is invisible to the running shell and a python bounce re-runs the OLD command line. These wait for a safe boundary (a queued wave; an idle gate) and then restart the right thing. See `docs/accelerators.md` #5 for the code/args/draw-defaults table. |
| | `tools/integration_resolver.py` | **(P31 S61)** THE INTEGRATION-RESOLVER LANE, zero model tokens (the S60 frontier analysis §4 — archived as `docs/sunset/tool-designs/frontier-analysis-s60.md`; the finding is the decision log's 2026-08-25 entry). Intake = backlog rows at closeness 0 ∪ `reloc_rejects` rows with shape MATCH (an INDEX, re-judged against today's tree); per item `rtu_match` at the real split TU → CC1: the gate ladder's draft-side transforms + one retry → MATCH: `reloc_identity` as the disagreeing oracle (rtu masks reloc fields, so a wrong symbol NAME still reads MATCH) → MISMATCH/shape-MATCH: `aprop_symfix` → stage `.run/sweep_resolver/<bin>/<fn>.c` → `sweep_parallel` (whole-binary SHA) → commit at once (R42). Holds `.run/auto/draw.lock` for judge+gate. Ledger `.run/resolver/verdicts.jsonl` (per-candidate verdict + draft/TU hashes; unchanged rejects are not re-judged). `--intake-only` prints the stock with denominators; `--negative-control N` re-judges N banked fns first and refuses on <N/N. Lane wrapper `tools/lanes/resolver_lane.sh`. First run S61: 424 items → 245 staged in 41 s. |
| | `tools/decl_from_use.py` | **(P31 S61)** Infer a MINIMAL C89 extern for a data symbol a draft uses but its destination TU does not declare (the A-prop "no seed decl" / resolver `undeclared` class): access widths/sign/index-scale from the target `.s`, use-forms from the draft, same-binary spellings preferred; refuses STRUCT shapes, conflicting width evidence, function symbols. Negative-controlled on 4,702 banked ground-truth triples (kind agreement 97.4%; the control forced 3 instrument fixes before any case verdict was read). Importable `infer()`/`tu_compiles()` for aprop_autodraft:522 and the resolver. First measured run: 39 of 42 "undeclared" cases were ONE uncompilable TU, not draft defects — classify `undeclared` only after a TU-alone compile probe. Ledger `.run/decl_from_use/verdicts.jsonl`. |
| | `.run/baseline_red.txt` (live gate input) | **(P31 S61)** The binaries whose CLEAN build fails the locked SHA at HEAD. `gate_stage` refuses their drafts pre-build as class BASELINE-RED (a RED binary rejects every draft gated against it — 15/214 binaries were red on 08-26 00:00 and ate 174/245 of the resolver's doubly-verified drafts). Maintained by the fleet audit + repairs; the maintenance lane's lock-aware fleet sweep is the standing detector. Keep it current: a repaired binary comes OFF the list or its drafts stay unjudged. |
| | `tools/jtbl_lane.py` (+ `jtbl_carve --probe/--island-split`, `harvest_verify._ISLAND_WALLS`) | **(P31 S59, cookbook §260-A; design `docs/tool-designs/jtbl-automation-s59.md`)** The jtbl carve→draft→bank lane. The carve stays AT THE GATE (`harvest_verify._jtbl_prep_one`, §61b's proven order — the lane never pre-carves); the lane probes feasibility READ-ONLY (`jtbl_carve.island_probe`: tail / covered / island-end / island-blocked / island-pads / main-manual), drafts (api_agent or `--draft-dir`), gates via the exact `sweep_parallel` worker call (per-binary flock + exclusive shared lock) while HOLDING THE CAMPAIGN'S DRAW LOCK across gate+commit (gates never overlap), and commits the named per-binary paths on a bank (R42). The §260 md island split is automated inside the gate (`_ISLAND_WALLS`: isolate body-spliced → `--island-split` insert → re-extract → covered no-op). One jtbl target per gate invocation (§61c batch constraint, by construction). `jtbl_lane --census` = the ranked work-list (245 members probed: 181/26,445 ins reachable). Wave route: `build_wave_atlas` gained the probe filter + a one-jtbl-card-per-binary cap, both inert unless `--levers jtbl-carve`. `idiom_serial` now REFUSES the jtbl-carve lever (its pre-carve ran the refuted order). Byte-proven ×3, one per class: `b801b499e` (ov tail, §8b merge + §8e pads), `bad793c73` (md covered — §260 stage 2, the FIRST md jr bank), `f74ad7ac8` (md island-end, full split by the gate on a virgin module). |

---