mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 21:36:06 -04:00
fix(phase-31): S80 #9c — the permuter could not permute a PINNED seed, and it was our instrument: hide_asm carried only the __asm__ spelling (3 S79 seeds use asm("$7")), permuter_ils warm-restarted from the DECODED waypoint (raw pins back in base.c → cycles 2..N were silent parser refusals reported "(unchanged)"), and defines_fn refused K&R-style definitions (436 stored backlog drafts kept out of the lane for four phases). Fixed + R39-controlled over 5,311 drafts (the bare word asm in INCLUDE_ASM path strings was a caught false positive): re-hide every waypoint, assert the definition survived, abort exit-2 on a refusal (R61a), flushed logs (R55). Every S79 pinned seed now iterates; ov_SC06_022:func_8017DF28 (pinned WALL, closeness 2) reached 1 in its first cycle. cookbook §493 S80 correction + §494 v1 (S79 idioms); SETUP rows (p16_permute/permuter_ils, agent_verdicts.py)
This commit is contained in:
Drew T
@@ -1004,6 +1004,8 @@ fills fast). Nothing is leaking — but the host does not get the memory back on
|
||||
| `tools/exclude_audit.py` | LINKED now outranks a pinned `# WALL` (a function in a linked PsyQ subseg is never a target, whatever compiler fact was recorded about it): dropped the four §332 walls + `func_80062144` + `PopMatrix`/`PushMatrix` (libgte3 since Phase 8) | every regeneration of `config/wave_exclude.txt` |
|
||||
| `tools/frontier_classify.py` | its LINKED subseg set is DERIVED from the Makefile (`progress._main_linked_segs_from_makefile`) instead of a hard-coded 49-name literal that missed every block wired since it was written (R51; it reported 337 "stubs" after task #5) | every census |
|
||||
| `tools/p16_permute.py` (task #8) | `run_permuter` no longer swallows run_masked's output: a C-parser refusal of the seed (`register … asm("$7")` pins) is printed as `[permuter] REFUSED <fn>: Syntax error …` and recorded in `<scratch>/PERMUTER_REFUSED.txt` instead of reading as "no improvement" (cookbook §493) | every `permuter_ils` / `permuter_sweep` run |
|
||||
| `tools/agent_verdicts.py` (S79/S80, task #9) | pulls the FINAL JSON verdict (`fn, binary, arm, status, closeness, draft_path, note`) out of Agent-tool subagent transcripts (`~/.claude/projects/<proj>/<session>/subagents/agent-*.jsonl`, symlinked from `/tmp/claude-1000/<proj>/<session>/tasks/*.output`) without reading them into a session; `--append <ledger.jsonl>`; a transcript with no verdict prints `NO-VERDICT` (R32). S80 aggregated the 11 S79 drafters this way: 2 MATCH banked, 8 NEAR ledgered | whenever Agent-tool drafters outlive the session that spawned them |
|
||||
| `tools/p16_permute.py` + `tools/permuter_ils.py` (S80, task #9c) | **Pinned seeds are permutable now** (cookbook §493 S80 correction): `hide_asm` carries `asm(`/`__asm(` as well as `__asm__(` into the b64 pragma (keyword must be followed by `(`/`volatile`, so `INCLUDE_ASM("asm/…")` path strings are not statements — R39-controlled over 5,311 drafts); `permuter_ils` RE-HIDES every warm-restart waypoint (the permuter serializes pins raw), asserts `defines_fn` survived, aborts exit-2 on a refusal (R61a), and flushes its log (R55); `defines_fn` accepts K&R-style definitions. Any pre-S80 "permuter plateau" on a pinned seed was 1 cycle + silent no-ops — re-measure before citing it | every `permuter_ils` / `permuter_sweep` run; the S79 NEAR ledger re-run |
|
||||
| `tools/agent_verdicts.py` (task #9 handoff) | extracts the FINAL JSON verdict from Agent-tool subagent transcripts (`.../tasks/<agentId>.output`, huge JSONL — never cat them) and appends them to a ledger; NO-VERDICT rows for files without one (R32) | after any batch of single drafting agents; the S79 handoff's step 2 |
|
||||
| `tools/jtbl_pads_fix.py` (task #6) | `PAD_ERR_MORE` accepts the current `jtbl_rodata_pads` wording ("more rodata jump tables than pad specs (N)") as well as the old ".align directives" one — it had reported "no pad-count drift" over a red build (cookbook §491 gap 1) | whenever a bank changes a TU's jump-table population and the build dies in `jtbl_rodata_pads` |
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1161 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1162 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -996,7 +996,7 @@
|
||||
- **§421** — ★★★ — A `la $tN` + `addiu` PAIR CAN BE A **RELOAD** ARTIFACT THAT NO C SPELLING REACHES (P31 S71; byte-proven `md_SC07_003/func_801A293C`, 313 ins, 6 → 0) <sub>L33729</sub>
|
||||
- **§446** — ★★★ — "RELOC-ONLY REMAP GATED DIFF" IS A CARVE VERDICT, NOT A CODEGEN VERDICT, UNTIL YOU DIFF THE `.text` (P31 S75; 4 siblings + 1 latent, ~1,300 ins) <sub>L34807</sub>
|
||||
|
||||
### integration / TU plumbing (85)
|
||||
### integration / TU plumbing (86)
|
||||
|
||||
- **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) <sub>L456</sub>
|
||||
- **§8d** — Templating a body INTO a TU must not CHANGE its declaration environment — demote the carried data externs (Phase 26 session 8, byte-proven on `func_8015AE2C` ×133) <sub>L502</sub>
|
||||
@@ -1083,6 +1083,7 @@
|
||||
- **§477** — ★★★ — THE `self_decl_tu` CLASS IS A SOLVED, MECHANICAL LANE: 16 DRAFTS, 16 BANKS (P31 S77) <sub>L35713</sub>
|
||||
- **§492** — ★★ — "C-PLUMBING" WAS THREE DIFFERENT THINGS (P31 S79 #7): A RAW SPLICE THE GATE'S LADDER BROKE, TWO -O0 BODIES THE CHECKER COMPILED AT -O2, AND TWO DRAFTS THAT BELONGED TO OTHER OVERLAYS <sub>L36392</sub>
|
||||
- **§493** — ★★ — THE PERMUTER ROUTE END-TO-END, AND THE THREE PLUMBING STEPS BETWEEN A SCORE-0 WINNER AND THE MAIN GATE (P31 S79 #8) <sub>L36420</sub>
|
||||
- **§494** — ★★★ — SEVEN BANKS FROM ONE-AGENT-PER-FUNCTION DRAFTING (P31 S79 #9): THE IDIOMS, THE PLUMBING, AND THE THREE WAYS AN AGENT'S "MATCH" WAS NOT ONE <sub>L36480</sub>
|
||||
|
||||
### build graph, splat & the harness (213)
|
||||
|
||||
@@ -2948,6 +2949,7 @@
|
||||
- **§491** — ★★ — THE MECHANICAL LEFTOVERS (P31 S79 #6): A PHANTOM STUB, TWO JTBL TWINS, ONE EXACT CLONE — AND THREE TOOL GAPS THE BANKS EXPOSED <sub>L36346</sub>
|
||||
- **§492** — ★★ — "C-PLUMBING" WAS THREE DIFFERENT THINGS (P31 S79 #7): A RAW SPLICE THE GATE'S LADDER BROKE, TWO -O0 BODIES THE CHECKER COMPILED AT -O2, AND TWO DRAFTS THAT BELONGED TO OTHER OVERLAYS <sub>L36392</sub>
|
||||
- **§493** — ★★ — THE PERMUTER ROUTE END-TO-END, AND THE THREE PLUMBING STEPS BETWEEN A SCORE-0 WINNER AND THE MAIN GATE (P31 S79 #8) <sub>L36420</sub>
|
||||
- **§494** — ★★★ — SEVEN BANKS FROM ONE-AGENT-PER-FUNCTION DRAFTING (P31 S79 #9): THE IDIOMS, THE PLUMBING, AND THE THREE WAYS AN AGENT'S "MATCH" WAS NOT ONE <sub>L36480</sub>
|
||||
|
||||
|
||||
---
|
||||
@@ -4121,3 +4123,4 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
|
||||
| L36346 | §491 | ★★ — THE MECHANICAL LEFTOVERS (P31 S79 #6): A PHANTOM STUB, TWO JTBL TWINS, ONE EXACT CLON |
|
||||
| L36392 | §492 | ★★ — "C-PLUMBING" WAS THREE DIFFERENT THINGS (P31 S79 #7): A RAW SPLICE THE GATE'S LADDER |
|
||||
| L36420 | §493 | ★★ — THE PERMUTER ROUTE END-TO-END, AND THE THREE PLUMBING STEPS BETWEEN A SCORE-0 WINNER |
|
||||
| L36480 | §494 | ★★★ — SEVEN BANKS FROM ONE-AGENT-PER-FUNCTION DRAFTING (P31 S79 #9): THE IDIOMS, THE PLUMB |
|
||||
|
||||
@@ -36451,6 +36451,24 @@ seed the permuter's C parser refuses (`register u8 *a3 asm("$7")` — pins are n
|
||||
(positive-controlled). And `permuter_ils` seeds from a REGISTER-PINNED best draft cannot be permuted at all
|
||||
— the pinned 17→11 gain and the permuter are mutually exclusive on that function.
|
||||
|
||||
**S80 CORRECTION — the "pinned seeds cannot be permuted" claim was the INSTRUMENT, not the permuter.** Two
|
||||
defects, both in our layer (R35/R40): (1) `p16_permute.hide_asm` carried only the `__asm__` spelling into
|
||||
the b64 pragma; `register u8 *a3 asm("$7")` (the `asm(`/`__asm(` spellings, 3 of the S79 seeds) stayed raw
|
||||
in base.c and pycparser refused it at cycle 1; (2) `permuter_ils`'s WARM RESTART copied the waypoint's
|
||||
`source.c` — which decomp-permuter serializes with the pragmas DECODED back to raw pins — straight into
|
||||
base.c, so on ANY pinned seed cycle 1 ran and cycles 2..N were parser refusals reported as "(unchanged)".
|
||||
The S79 `func_80020DA4` "8-cycle plateau at 2" was one cycle. Fixed: `hide_asm` matches all three
|
||||
spellings, followed by `(`/`volatile` (the bare word `asm` also lives inside `INCLUDE_ASM("asm/…")` path
|
||||
strings — the R39 control over 5,311 drafts caught that false positive before it shipped); the ILS re-hides
|
||||
every waypoint before restarting, asserts the definition survived, and ABORTS non-zero on a refusal
|
||||
instead of counting no-op cycles (R61a). `defines_fn` also now accepts a K&R-style definition
|
||||
(`void f(a, b) s32 a; s16 b; {` — the documented lever for an `s16` parameter's in-place promotion,
|
||||
func_80039DEC), which it had refused as "lost the definition" — the R39 control over 5,311 stored drafts found
|
||||
**436 K&R-style backlog drafts** that this check alone had kept out of the permuter lane. First re-run on the S79 seeds: every
|
||||
pinned seed iterates, and `ov_SC06_022:func_8017DF28` (pinned WALL, "closeness 2 on five RTL-verified
|
||||
attempts") reached **1** inside its first cycle. A permuter verdict on a pinned seed dated before S80 is
|
||||
NOT a measurement of that function.
|
||||
|
||||
**The ledger the class leaves (for the PhaseEnd):** `main:func_80011380` 192 — WALL, §474 PROVED (pinned
|
||||
S79). `main:func_80015608` 86 — permuter plateau at **1** (REGALLOC-PERM: target `addu $s3,$s6,$s3` where the
|
||||
best draft emits `sll $s3,$s6,1` — a copy-then-add spelling of `x*2`, agent-sized). `main:func_80039B20` 79 —
|
||||
@@ -36459,3 +36477,45 @@ cycles). `main:func_80038698` 74 — best 11 (§ pins fix the interleave; the pe
|
||||
DAG-priority + local-alloc self-coalesce). `ov_SC03_105:func_801834A4` 106 — pinned WALL (S71, §148-A/§193-F,
|
||||
closeness 6 on four attempts).
|
||||
|
||||
#### §494 ★★★ — SEVEN BANKS FROM ONE-AGENT-PER-FUNCTION DRAFTING (P31 S79 #9): THE IDIOMS, THE PLUMBING, AND THE THREE WAYS AN AGENT'S "MATCH" WAS NOT ONE
|
||||
|
||||
**Yield.** 25 packs (`claude_wave_packs`: journal history + a matched neighbour each), one Agent-tool subagent
|
||||
per function (Haiku ≤50 ins, Sonnet ≤120, Opus above), no wave. Seven banked byte-identical the same day:
|
||||
`md_MAIN_003:func_800D0174` + `func_800D1D14` (-O0 island), `main:func_80015608` + `func_8002AC98`,
|
||||
`ov_SC05_018:func_80180BE0`, `ov_SC06_010:func_801809E4`, `ov_SC05_010:func_8017FFA8` (a 6-way jtbl switch).
|
||||
|
||||
**Idioms that closed functions (each byte-proven today).**
|
||||
* **The fresh-temp lever for a commutative operand order** — target `addu $s3,$s6,$s3` where every source
|
||||
order emits `addu $s3,$s3,$s6`: gcc 2.7.2's `expand_binop` swaps operands when op1 IS the expansion target
|
||||
register; route the add through a NEW pseudo — `{ s32 xt = blockSize + x0; x0 = xt; }` — and the source order
|
||||
survives, xt coalesces into the same hard register. Two banks (`func_80015608`, `func_8002AC98`); the temp AND
|
||||
both operands must be `s32` (an `s16` temp re-coalesces with the target and the swap returns; an `s16` operand
|
||||
costs a sign-extend pair).
|
||||
* **-O0: address-of + cast beats the `%lo` fold** — `D_800D3630` (4-byte stride, low 16 bits read) as
|
||||
`s16[][2]` or a 2-field struct makes gcc fold `%lo` into the `lh` displacement through `$at` (-2 ins per site);
|
||||
`extern s32 D_800D3630[]` + `*(s16*)&D_800D3630[i]` materialises `lui`+`addiu` first and matches.
|
||||
* **An early `return` that must fall through** — the Haiku plateau's missing 2 instructions and "frame -8 vs
|
||||
-0x10" were a `return` inside a branch that should fall into the shared mask/store tail; the -0x10 no-save
|
||||
frame was two `s16` locals (§186b), not a call. Branch polarity `if (v0 >= v1)` with the arms swapped.
|
||||
* **jtbl switch:** the loop index must be `s32` (an `s16` gives the fused `lhu+sll16+sra12`, §241) and a
|
||||
table lookup goes into its own named temp BEFORE the `found=1`/zero-store statements, or the store schedules
|
||||
ahead of the load.
|
||||
* **The phantom stack frame** — an address-taken `s32 frame_pad[3]` induces the target's unexplained 16-byte
|
||||
frame (`func_80020DA4`, 8 → 2; the remaining 2 is a `mflo` destination, permuter class).
|
||||
* **`memcpy(…,12)` under a TU's `extern memcpy`** compiles to a `jal` (§48-C2/§160a); the inline
|
||||
`lwl/lwr/swl/swr` shape needs a struct assign through an align-1 12-byte typedef (the TU's own Blk8 idiom).
|
||||
|
||||
**Three ways an agent's "MATCH" was not one.** (1) A naked `__asm__` reproduction of the target — the verbatim
|
||||
class; sent back, it returned genuine C. (2) Standalone `match_one` MATCH, real-TU DIFF: the TU's typedefs and
|
||||
externs (`rtu_match` names them; §376). (3) rtu MATCH, gate NEAR: the gate's transform ladder altered the body
|
||||
(§492a) — raw splice. Rule for the coordinator: **grep the draft for `.ent`/`.word`, re-measure in the real TU,
|
||||
gate, commit — per result, never in bulk.**
|
||||
|
||||
**Ledger rows this task leaves** (drafts in `.run/S79w/<arm>/`, notes in `.run/S79w/verdicts/verdicts.jsonl`):
|
||||
`main:func_8001BC6C` 28 (sched1 birthing-boost — two mutually exclusive schedules), `main:func_8002FDE8` 35
|
||||
(local-alloc caches `&D_800A46D2` across a call where the target rematerialises, §153), `main:func_80038698` 11
|
||||
(sched1 DAG priority on the byte-pair idiom ×3; the permuter refuses the pinned seed), `main:func_80039B20` 7
|
||||
(§461 tie-break), `main:func_80020DA4` 2 (mflo destination — permuter fuel), `main:func_80015B6C` 44 (two
|
||||
documented walls, better than 3 of 6 attempts), `ov_SC06_022:func_8017DF28` 2 (delay-slot fill from
|
||||
`expand_block_move`'s cse-reused address pseudo — five RTL-verified attempts, WALL).
|
||||
|
||||
|
||||
+14
-2
@@ -99,7 +99,13 @@ def hide_asm(c):
|
||||
steered, mvmva/GTE ops still compile), while pycparser never chokes on `__asm__`. (No submodule
|
||||
edit: this reuses decomp-permuter's OWN b64literal pragma carrier.)"""
|
||||
out, i = [], 0
|
||||
pat = re.compile(r"\b__asm__\b")
|
||||
# `asm(` and `__asm(` are the SAME gcc keyword and pycparser rejects all three spellings alike;
|
||||
# the old `__asm__`-only pattern left `register u8 *a3 asm("$7")` raw in base.c -> "Syntax error in
|
||||
# base.c" -> the whole run silently no-ops (P31 S79 #8 func_80038698; fixed S80).
|
||||
# The keyword must be FOLLOWED by `(` or a volatile qualifier: the bare word `asm` also occurs
|
||||
# inside path strings -- `INCLUDE_ASM("asm/ov_SC03_028/...", fn)` -- and treating that as a
|
||||
# statement ate the line up to the function's own `{` (caught by the R39 control, S80).
|
||||
pat = re.compile(r"\b(?:__asm__|__asm|asm)\b(?=\s*(?:\(|__volatile__\b|volatile\b))")
|
||||
while True:
|
||||
m = pat.search(c, i)
|
||||
if not m:
|
||||
@@ -197,7 +203,13 @@ def defines_fn(base_c, fn):
|
||||
prep step here can swallow the function (hide_asm chewing an unexpanded multi-line macro block,
|
||||
an unterminated comment eating the rest of the file, a future cpp/typedef edge) so the check
|
||||
belongs on the OUTPUT, where it catches all of them, not on each cause."""
|
||||
return re.search(r"^[^#/\n]*?\b" + re.escape(fn) + r"\s*\([^;{]*\)\s*\{", base_c, re.M) is not None
|
||||
# A K&R-style definition -- `void fn(a0, a1, a2) s32 a0; s32 a1; s16 a2; {` -- carries a declaration
|
||||
# list between `)` and `{`. It is the documented lever for an `s16` parameter's in-place sll/sra
|
||||
# promotion (P31 S79 func_80039DEC), pycparser accepts it, and this check refused it as "lost the
|
||||
# definition" (S80). Each K&R declaration must START with an identifier so a call `fn(1) ; x = 2; {`
|
||||
# cannot chain into a false definition.
|
||||
return re.search(r"^[^#/\n]*?\b" + re.escape(fn) + r"\s*\([^;{]*\)(?:\s*[A-Za-z_][^;{}]*;)*\s*\{",
|
||||
base_c, re.M) is not None
|
||||
|
||||
|
||||
def winner_to_draft(winner_c):
|
||||
|
||||
+25
-2
@@ -11,10 +11,14 @@ intermediate waypoints can be semantically divergent (the permuter rewrites stor
|
||||
--asm-subdir asm/ov_SC01_077/nonmatchings/ov_SC01_077 --klass REGALLOC --cycles 10 --secs 180 --j 12
|
||||
On a score-0 winner -> .run/permuter-winners/<fn>.c (then winner_to_draft + gate whole-binary).
|
||||
"""
|
||||
import argparse, glob, os, re, shutil, sys
|
||||
import argparse, functools, glob, os, re, shutil, sys
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import p16_permute as P
|
||||
|
||||
# stdout is BUFFERED when redirected to a log; an 8-cycle run then shows an EMPTY log for 20 minutes
|
||||
# (S80: eight parallel runs, no evidence of progress -- R55). Always flush.
|
||||
print = functools.partial(print, flush=True)
|
||||
|
||||
|
||||
def best_waypoint(pd):
|
||||
"""(score, dir) of the lowest-score output-<score>-<n>/source.c across ALL cycles; None if none.
|
||||
@@ -46,8 +50,18 @@ def main():
|
||||
print(f"ILS {a.fn}: {a.cycles} cycles x {a.secs}s @ -j{a.j}, klass={a.klass}")
|
||||
|
||||
prev = None
|
||||
refused = os.path.join(pd, "PERMUTER_REFUSED.txt")
|
||||
for cyc in range(1, a.cycles + 1):
|
||||
if os.path.exists(refused):
|
||||
os.remove(refused)
|
||||
P.run_permuter(pd, a.secs, a.j) # writes output-*/ ; kills stragglers
|
||||
if os.path.exists(refused):
|
||||
# NOT-JUDGED IS NOT A VERDICT (R61a). A refused cycle permuted nothing; reporting it as
|
||||
# "(unchanged)" for the remaining cycles is how S79's func_80020DA4 run showed 8 cycles of
|
||||
# work that were 1 cycle + 7 no-ops. Stop, say so, exit non-zero.
|
||||
print(f"ILS ABORTED at cycle {cyc}: the permuter REFUSED base.c (see {refused}); "
|
||||
f"best so far={prev}")
|
||||
sys.exit(2)
|
||||
bw = best_waypoint(pd)
|
||||
if bw is None:
|
||||
print(f" cycle {cyc}: no waypoint (no improvement over base yet)")
|
||||
@@ -61,7 +75,16 @@ def main():
|
||||
shutil.copy(os.path.join(d, "source.c"), dst)
|
||||
print(f" WINNER score 0 -> {dst} (gate whole-binary before banking)")
|
||||
return
|
||||
shutil.copy(os.path.join(d, "source.c"), f"{pd}/base.c") # warm restart
|
||||
# WARM RESTART. The permuter DECODES the b64 pragma carrier when it serializes a candidate, so a
|
||||
# waypoint's source.c holds the raw `register … __asm__("$N")` pins / `__asm__` statements again.
|
||||
# Copying it verbatim made every cycle after the first a parser refusal on any pinned seed
|
||||
# (P31 S79: func_80020DA4 = 1 real cycle + 7 silent no-ops). Re-hide before restarting, and assert
|
||||
# the function definition survived the re-hide (R32).
|
||||
wp = P.hide_asm(open(os.path.join(d, "source.c")).read())
|
||||
if not P.defines_fn(wp, a.fn):
|
||||
print(f"ILS ABORTED at cycle {cyc}: re-hiding the waypoint lost the definition of {a.fn} "
|
||||
f"(inspect {d}/source.c)"); sys.exit(2)
|
||||
open(f"{pd}/base.c", "w").write(wp)
|
||||
prev = score
|
||||
print(f"ILS done: best={prev} (no score-0; seed for Fable5 or a longer run)")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user