THE INSTRUMENT (R40). Two campaigns returned "0 of 16" with a straight face; both were the harness. The target object had
been assembled from a DISASSEMBLY LISTING, which is a second toolchain with its own answers:
- objdump prints the pseudo-instruction `move` for `addu rX,rY,$zero`; gas assembles `move` as `or` — 24 wrong words in
one 234-instruction function, silently;
- a listing's %hi/%lo pairs come back RESOLVED with no relocation, while every candidate carries one, and the masked
scorer compares reloc operands.
The permuter therefore scored 28 for a body that IS byte-identical: score 0 was unreachable and every NO-MATCH was its own.
- tools/delever_permute.py: the target is now the tree's OWN (levered) body compiled by the build's tail into a
one-function object — the candidates' relocations by construction — and `match_one` must call that body a MATCH against
the ROM listing before the search starts (R34 keeps it from being circular). Base score for the tree's own body: 0.
- tools/p16_permute.py `setup(target_o=)` + tools/permuter_ils.py `--target-o` (defaults unchanged).
- `--positive-control TU FN`: perturb a matching body by one commutative swap, require the permuter back to 0.
- tools/verbatim_target_s.py --gas now VERIFIES itself: assemble, disassemble, compare word by word with the image,
`.word 0x…`-patch what does not reproduce (24 in that function), REFUSE what still disagrees. The listing is a public
artifact (decomp.me) and was wrong for every function containing a `move`.
THE PROFILE. The weight profile now comes from the register a needed pin names, not just the site kind: callee-saved
($16-$23) is an allocation-order residual -> regalloc; caller-saved ($2/$3/$4-$7) is not -> cse. Read from the bytes: the
residual on func_80163EC8 (`register … __asm__("$2")`) is `and v0,v1,v0` against `and v0,v0,v1` — the operand order of one
`&` — and the regalloc profile weights perm_commutative 2.0 while cse weights it 40.0.
RUNG R (tools/delever.py --recipes): the cookbook's byte-neutral shape recipes, mechanically, seeded with the body's
lever-free text — R2 the formerly-pinned declarations permuted, R4 one moved through the whole declaration run, R3 an
initializer split placed after the run (C89), R5 the operand order of one commutative operator (the caller-saved lever, and
the only recipe needing no pinned declaration). Identity control on both the splice and the oracle before any verdict;
markers scrubbed within the banked body's own span only; selftest cases on a fixture whose answers are known by hand.
Also: a threading race in the site cache published the empty dict before filling it and made a whole batch report
"no site in this TU"; the file-scope asm dropper took an asm-LABEL clause for a statement. SETUP + dictionary rows.
- tools/delever_permute.py: one exemplar per RESIDUE text class from the ledger (copies desc, needed asc) prepared as a
single-function TU (delever's rung-A rewrite; other definitions -> prototypes; shared-header includes -> their prototypes;
INCLUDE_ASM and file-scope asm dropped; the build's own CPPFLAGS through cpp -P), the target regenerated from the ROM image in
BOTH forms (--gas for target.o, splat for match_one), permuter_ils with the profile from the NEEDED kinds, a winner banked only
through delever --apply-body + the GTE re-fold. Scratch/winners keyed alias+fn (R48).
- the control (R39/R56, new): every attempt first requires the LEVERED body to be match_one MATCH against the regenerated target,
then records the lever-free body's distance. --calibrate --limit 12: 12 of 12 MATCH; starting distance min 8 / median 78 / max 276.
- delever: ("B","gte-lever") joins REMOVABLE — a direct statement's clobbers reset to its canonical set, a variant-macro use pointed
at the canonical macro whose name comes from the variant definition's SIGNATURE (gte_rt_m -> gte_rtv0tr, not Sony's gte_rt);
462 of 462 gte-lever sites now offered to the ladder, 0 before. gte_consolidate.canonical_match() is the one reader of the
canonical table (R33), direct_rewrite refactored onto it; both selftests green.
- p16_permute.setup(outdir=) + permuter_ils --pd: a scratch dir keyed by the caller, defaults unchanged.
- four harness defects found by running it: the splat listing is not assemblable (R98 in a second place); pycparser rejects
__attribute__ and the permuter then silently permutes nothing; include_asm.h injects a file-scope .include "labels.inc" that
collides with the permuter's own macro.inc; an asm-LABEL clause is not an asm statement (a bare scan ate one and left a headless
K&R body). SETUP row (R21), dictionary row (R87), .gitignore allowlist for the outcomes ledger.
Stubs 32 -> 31 after the func_80015760 bank (commit:3877); R22 fleet 213/213 (.run/S79_check_all_8.log);
main game-code 93.5% (38,854 / 41,534). Permuter ILS plateaus recorded with their residual named:
func_80015608 best 1, func_80039B20 best 7, func_80038698 pinned seed refused (11). The ILS runner
had reported "no waypoint" for 8 cycles in 20 s on a seed the permuter's C parser rejects; it now
prints [permuter] REFUSED and leaves PERMUTER_REFUSED.txt (positive-controlled on func_80038698).
- ROOT CAUSE (reproduced): make_base_c ran cpp_expand_macros BEFORE #include lines were
dropped, so `cpp -P -nostdinc -` died on `#include "common.h"` (rc=1, empty stdout) and the
`return c` fallback handed back the UNEXPANDED draft. hide_asm then ate the gte_* #define
block + the function itself -> "Function not found in base.c" -> decomp-permuter no-opped
in 0s, indistinguishable at the call site from "searched, found nothing".
- FIX: strip #include inside cpp_expand_macros (byte-neutral) + RAISE on cpp failure (R32/R35,
no silent fallback); NEW defines_fn() assertion in setup() guards the OUTPUT so it catches
every swallow cause (this, the §G comment class, future macro shapes); main() catches per-fn
so a bad draft is loud+counted but cannot abort a batch.
- VERIFIED: func_8017C6F4 base.c keeps the def, 0 gte_ macros left, 35 asm b64-carriers;
proxy validated over 388 stored drafts = 0 false alarms, 0 cpp raises (macro-free untouched);
permuter now loads at base score 65 and iterates (was a 0s no-op).
- BLAST RADIUS (14,899 drafts scanned): 63 carry `#define … __asm__` + `#include`, incl. the
behemoth renderer drafts — the permuter was silently dead on the highest-byte-weight targets.
- CONSEQUENCE (R14): §147/§148's ~40-probe floors were measured with the permuter UNAVAILABLE;
"the permuter also plateaus" was never actually tested on those functions. §148 note corrected.
- REFUTES the .run/giants README's "pycparser/permuter CANNOT ingest it as-is": p16_permute.setup's
b64-pragma pin carrier handles it (6 pins -> 6 carriers, 0 raw __asm__, target.o built, §31 profile).
Checked against the tool, not the note (R35) — the 3rd recorded wall this session to dissolve.
- Drift-check first (R14): all preserved drafts reproduce their recorded closeness exactly (5/33/76/116).
func_801670E4 (close=16) is ALREADY BANKED fleet-wide — the README is stale; it is not work.
- 900s @ -j12: 5 -> 1. The permuter closed the 4-ins INSN_LUID scheduler tie the drafting agent had
recorded as un-steerable after sweeping all 6 assign orders + pin combos by hand.
- Last instruction (andi vs addu) diagnosed from the byte-verified sibling func_801778A8, whose
"nib = uVar1;" plain-copy idiom (both vars hard-pinned) after the identical (x << 16) >> 28 shift
pair is what materializes the addu. Dropping my redundant & 0xf alone COLLAPSES the copy (100 vs
101 ins, 52 mismatched), so the target needs a distinct pinned register. Pinning n to $a2 ->
101/101 with 6 left, class ADDRESSING [permuter] -> handed back to the permuter from the
structurally-correct seed rather than hand-designed.
- FIX: run_permuter's cleanup pkill matched EVERY concurrent run (two permuters silently killed each
other); scoped to the run's own scratch dir -> concurrent giant grinding is now safe.
The plan named two defective regexes; the tree had SIX with complementary holes, each
silently recording the resulting compile failure as "not a match" — a plumbing error
wearing a compiler wall's clothes, the exact class the 26-A audit exists to end (R32).
- cdecl.py: the canonical primitive — typedef_names(tu_path) + strip_provided_typedefs
(draft, provided). Built on tu_statements (robust) NOT tu_scope (which coverage-asserts
-> would crash the byte-gate on any unrelated unparseable file-scope statement). Splits
multi-typedef lines (split_statements, depth-aware); covers scalar AND struct typedefs;
keeps draft-local types. lru_cached.
- harvest_verify.py: strips PER-TU (cdecl.typedef_names of the draft's real target TU) ->
unblocks the 39 struct-typedef drafts the scalar-only _TD dropped. And SURFACES cc1
stderr: build() stashes it; a single-draft failure is classified DIFF / PLUMBING:… /
CC1-FAIL / SKIP -> .run/harvest_failed.classified.txt. A `redefinition` is no longer
recorded byte-identically to a codegen miss.
- masked_diff.py: strip_scalar_typedefs() (common.h set derived from the header once, R33,
cached) replaces SCALAR_TYPEDEF_RE.sub for the ISOLATED compile; wired into match_one +
p16_permute. Fixes the multi-typedef-LINE skip that discarded 42 masked-MATCH drafts over
whitespace. Unblocks B4's func_8015C32C (redefinition of 's16').
- canon_sig_reconcile / eval_lora / format_finetune keep their own copies — migrate
per-bank, byte-gated (the audit-prescribed cadence, not a big-bang swap).
VERIFIED:
- HEADLINE known-answer: func_8015C030 -> MATCH (23 ins) UNEDITED via match_one (was
CC1-FAIL; the multi-line typedef split alone fixes it — a live x134-family draft that
was being discarded over whitespace).
- unit: 7/7 scalars stripped; a local struct KEPT; a TU-provided Blk16 stripped.
- classifier unit: DIFF / PLUMBING:… / CC1-FAIL / SKIP all label correctly.
- all 5 edited tools import + AST-parse clean.
- R22 clean-fleet: check-all 136/136; main clean-rebuild 143dbb89. (A mid-test c4546248
"mismatch" was a stale-incremental artifact from concurrent compiles, cleared by a clean
rebuild — the R22 lesson; edits touch only tools/, src/ stayed git-clean.)
- SAFETY: a strip bug can only fail-to-bank, never falsely bank (INCLUDE_ASM pastes the
original asm; a wrong draft always changes bytes -> always fails SHA1).
Second silent no-op of the §G class, found while permuting func_8017BEBC (close=2):
- hide_asm() is built for __asm__ STATEMENTS and `register __asm__("$sN")` pins inside a
function body (it scans back to the previous ;{} and forward to the next top-level ;).
A draft whose GTE ops are #defines CONTAINING __asm__ (the PsyQ inline_c.h convention,
i.e. most renderer code) therefore had its macro DEFINITIONS chewed up, swallowing the
function itself -> pycparser 'Function <fn> not found in base.c' -> decomp-permuter
no-op'd in 0s. base.c contained ZERO occurrences of the target function.
FIX: cpp_expand_macros() pre-expands with `cpp -P` so each GTE op becomes an inline
__asm__ statement hide_asm can carry via the b64 pragma. Applied ONLY when a
'#define ... __asm__' is present -> macro-free drafts byte-untouched.
- p16_permute was hardcoded to OV=ov_SC01_077's MAIN object, so no core in another overlay
or split object could be permuted at all. Added --asm-subdir (threaded explicitly: a
def-time default arg cannot see a mutated global).
LESSON (cookbook): permuter 'no match (0s)' is a TOOLING failure signature, never a real
search result. Verify workers actually ran.
Verified: base.c now holds the function; 16 workers searching on func_8017BEBC.
- p16_permute.hide_asm: b64literal-pragma carrier for register pins + GTE __asm__ blocks.
pycparser parses the pragma; decomp-permuter's process_pragmas decodes it back so cc1 sees
the real pins/asm (regalloc steered, mvmva compiles). No submodule edit (reuses its own carrier).
- drop_preproc_and_scalar_typedefs: keep #define + custom struct/typedefs, drop only #include
(fixes func_801412A8 'OTLINK undeclared'); +f32 typedef gap; make_base_c hides asm.
- compile.sh/compile_o0.sh: prepend .include "macro.inc" so GTE mvmva assembles (the real build
gets it via include_asm.h, which base.c omits + -DPERMUTER disables). Byte-neutral for non-GTE.
compile_o0.sh (-O0) auto-selected for _o0 targets.
- run_masked.py: expr_type->int fallback for hidden-pin vars -> 0 internal-permuter-failures
(perm_split_assignment/perm_temp_for_expr no longer KeyError on pinned drafts).
- ALL 5 seeds parse+compile (base 4/110/36/52/77). FLAGSHIP func_80132784 (4/400) CLOSED to a
masked-0 that match_one confirms MATCH (400 ins) -> .run/wave/func_80132784.win.c for T4 gate.
- no build-input changed (136/136 untouched)
The grinder/backlog pipeline was ov_SC01_077-hardcoded 5 layers deep (same class as the
T7 lora_grind bug). Fixed all so the permuter grinder can process a non-077 near-miss:
1. gate_stage.append_record stores the source "binary"
2. backlog.FIELDS keeps it (else append_record dropped it)
3. backlog.load_best/_open_stubs is fleet-aware: a fn matched in ov_SC01_077 but
propagation-stuck stays OPEN in its overlay, so it surfaces via that record instead
of being dropped as "matched" (the grinder must SEE it to grind it)
4. p16_permute.setup takes the target binary's asm-subdir (was hardcoded 077)
5. grinder resolves per-binary asm + gates grouped by binary + allows unknown nins
Backward-compatible: legacy records (no binary) default ov_SC01_077.
Validated end-to-end: the 3 fresh reach-134 close=1 ov_SC01_000 fns now surface, resolve
to ov_SC01_000's asm, and gate via ov_SC01_000.
TWO byte-evidenced findings (redirect the fuel strategy):
- the reach>=2 close=1 fuel is MODEL semantic-misses, not permuter fuel: func_8012E27C's
target is "return 1" but the 7B drafted an empty "void f(void){}" (corpus overfit
empty-leaf); func_8012BF4C/AD64 are trivial sw/sh setters drafted empty. A corrected
draft banks them (+3 byte-identical via the fixed gate, @commit:0326); the permuter cannot
add a missing return/store. Lever = corpus-v3 leaf variety, not the permuter.
- x reach is propagation-capped: the 3 are inline-matched in ov_SC01_077_a.c (the stuck-
local cap) -> dedup_propagate "nothing to propagate" -> banked x1. Lever = dedup-collapse.
check-all 136/136 throughout. docs/gen2-mips-matching-model.md + CURRENT_PHASE updated.
- docs/struct-core-pivot.md: findings + decision + new research directions. Root cause = the
original engine is LOOSELY TYPED (K&R; same fn called with int/ptr, arg/no-arg across sites),
so no single canonical signature exists -> m2c guesses inconsistently, permuter can't fix
semantics, byte-gate (correctly) rejects. Yields ~3%, not the crack. New plan: emulator-recover
the actor struct/types -> Ghidra global type propagation -> Ghidra-C -> permuter+gate.
- harness bug-fixes (REAL, kept): p16_permute output-0-only match (killed the false '42%'),
base.c keeps callee externs, winner_to_draft line-strip; sig_unify canonicalizes m2c's
no-extern prototypes; gen_engine_decls.py (documents why a global canonical header breaks
loose-typed matches).
- a few byte-gated leaf matches banked in ov_SC01_077.c.
Drew T