mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-01 15:43:06 -04:00
feat(p31 s67): harvest §325-§331 + gate_wave.py (split jtbl/parallel, both lanes concurrent)
HARVEST — the s67o1/s67m1 wave banked 7 cookbook sections:
* §325 a shared small constant stored twice in the pre-loop block is a LOCAL-ALLOC $s-occupant that
steals the argument allocno's register — pin the ARGUMENT-derived local, not the constant
(pinning the constant reached only closeness 15). byte-proven func_80184F18.
* §326 spelling two reads of the same halfword differently (sym[i] vs *(s16*)(base+i*4+2)) yields
different address rtx and DEFEATS address-CSE, restoring separate %hi/%lo groups. func_8017FAAC.
* §327 a range test must be HImode: with s32 + a (u16) cast gcc PROVES the mask redundant and drops
the andi — a real -1 length drift that reads as a schedule. +3 levers. func_8017EC34.
* §328 NEW LAW: the volatile alias must be an aliased OBJECT; `*(volatile s32*)&sym` unfolds %lo
into a separate addiu (+1 ins). func_80181B8C.
* §329 fold-const narrows `(int)s16 & 0xFFF` onto the RAW HImode pseudo, breaking the
sign-extend/mask register tie; a zero-byte `s32 e = t;` widening temp restores it (30 rows -> 0).
* §330 the NEIGHBOUR-SHAPE lever, four independent instances in one wave — copy an already-banked
in-TU function's SPELLING before any codegen reasoning (one dissolved 18 REGALLOC-PERM rows in a
single compile). Corollary: a warm start from another binary is often worth LESS than the
neighbour 20 lines away.
* §331 OPEN GAP, recorded as unsolved: no lever eliminates an UNWANTED DUPLICATE copy at a
branch-target block head (main/func_80013154, closeness 12, ~16 iterations, 5 approaches refuted).
TOOLIFY — tools/gate_wave.py: split the batch on the per-draft jtbl predicate, run parallel_gate
and the serial jtbl lane CONCURRENTLY. Measured this session: 4 binaries in 103s wall through
parallel_gate (87/87/88/102s each) vs ~6 min serially; I had gated all 16 serially to protect ONE
jtbl draft, ~1 hour. The split precedes the run because a jtbl worker does NOT fail cleanly — it
re-extracts through the worktree's asm/ symlink and writes the MAIN tree while other workers read it.
Its own negative control found two defects in it before first use:
* listdir counted gate_stage's _xform output dirs (-cn/-cast/-rc/-sd, written as SIBLINGS inside
the drafts root) as binaries: 20 "binaries" for a 16-binary wave. Now validated against
progress.BINARIES and refused loudly (R32/R43).
* a post-hoc control over BANKED functions cannot reproduce a split (has_jtbl has no stub to read);
re-controlled against a live draft set, where it correctly routes the two functions the gate had
independently reported CARVE-REFUSED.
This commit is contained in:
Drew T
+35
-9
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 972 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 979 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -34,7 +34,7 @@
|
||||
## By symptom
|
||||
|
||||
|
||||
### delay slots & branches (58)
|
||||
### delay slots & branches (59)
|
||||
|
||||
- **§3-T4** — Branch polarity: invert the source condition to flip gcc's chosen branch <sub>L90</sub>
|
||||
- **§5a** — Cross-jump tail-merge — gcc collapses two byte-identical blocks the original kept separate (FIX FOUND) <sub>L211</sub>
|
||||
@@ -94,6 +94,7 @@
|
||||
- **Addendum** — REGALLOC-PERM store in a cross-jump shared tail defeats §137's barrier (func_8017EDE8) <sub>L30630</sub>
|
||||
- **Addendum** — LENGTH-DRIFT nop clears when offset math precedes the symbol launder (func_80039B20) <sub>L30634</sub>
|
||||
- **Addendum** — Return-0 statement order: extra j+move vs delay-slot fusion (func_8018BB50) <sub>L30721</sub>
|
||||
- **§331** — OPEN GAP: NO LEVER ELIMINATES AN *UNWANTED DUPLICATE* COPY AT A BRANCH-TARGET BLOCK HEAD (P31 S67; main/func_80013154, closeness 12, NOT solved) <sub>L31059</sub>
|
||||
|
||||
### instruction scheduling (70)
|
||||
|
||||
@@ -168,7 +169,7 @@
|
||||
- **Addendum** — Volatile zero-byte asm slider scrambles the prologue schedule (func_801814B0) <sub>L30747</sub>
|
||||
- **Addendum** — §194-A addendum — the bare/colon-less fence measured NULL and only the "memory"-clobber fo (func_8017E464) <sub>L30761</sub>
|
||||
|
||||
### register allocation & pins (115)
|
||||
### register allocation & pins (117)
|
||||
|
||||
- **§10** — Closing the regalloc/scheduling hard tail by hand (LZSS, Phase 7 session F — the full close) <sub>L854</sub>
|
||||
- **Residual** — A — commutative `|`/`&`/`+` result lands in the wrong source-operand register <sub>L875</sub>
|
||||
@@ -285,8 +286,10 @@
|
||||
- **Addendum** — LENGTH-DRIFT −1 on a coalesced-away copy: a CALLER-SAVED SOURCE pin can resurrect it, boun (func_8017D72C) <sub>L30714</sub>
|
||||
- **Addendum** — The dying-pinned-register reuse on a sign-extend chain: route every later read through a s (func_80186868) <sub>L30743</sub>
|
||||
- **Addendum** — Register-pinned helper pointer local regresses closeness; use plain local (func_80013CFC) <sub>L30757</sub>
|
||||
- **§325** — REGALLOC: A SHARED SMALL CONSTANT STORED TWICE IN THE PRE-LOOP BLOCK IS A *local-alloc* $s-OCCUPANT THAT STEALS THE ARGUMENT ALLOCNO'S REGISTER — PIN THE ARGUMENT-DERIVED LOCAL, NOT THE CONSTANT (P31 S67; byte-proven ov_SC03_119/func_80184F18, 153 ins) <sub>L30987</sub>
|
||||
- **§329** — fold-const NARROWS `(int)s16_var & 0xFFF` ONTO THE *RAW HImode PSEUDO*, BREAKING THE SIGN-EXTEND/MASK REGISTER TIE — A ZERO-BYTE WIDENING TEMP RESTORES IT (P31 S67; byte-proven ov_SC01_084/func_80183244, 157 ins) <sub>L31034</sub>
|
||||
|
||||
### CSE / redundancy / rematerialization (33)
|
||||
### CSE / redundancy / rematerialization (36)
|
||||
|
||||
- **§46** — The `func_80178D40` crack (890 ins ×134, the heaviest core in the game): four LOOP-STRUCTURE levers cheap-Opus found by reading loop.c/jump.c/cse.c (Phase 26 session 8, 2026-07-13) <sub>L3347</sub>
|
||||
- **§83d** — CSE's quantity budget is WHOLE-FUNCTION, so a local rewrite cannot fix a local symptom <sub>L6486</sub>
|
||||
@@ -321,8 +324,11 @@
|
||||
- **REFUTED** — claim — REFUTED — the `j`-slot store is an ASPSX macro-expansion hop, not a cse split; maspsx hard (func_8005DBD8) <sub>L30589</sub>
|
||||
- **§NNN** — REFUTED: A `sw %lo(SYM)($at)` IN A JUMP'S DELAY SLOT WITH ITS `lui $at` STRANDED ABOVE THE JUMP IS **ASSEMBLER** MACRO-EXPANSION, NOT A cse SPLIT — OUR PIPELINE CANNOT EMIT IT FROM ANY C, AND THE +1 IS A `maspsx` GAP (P31 S66 round2, `func_8005DBD8`, byte-probed) <sub>L30591</sub>
|
||||
- **Addendum** — Volatile zero-byte asm slider scrambles the prologue schedule (func_801814B0) <sub>L30747</sub>
|
||||
- **§326** — DEFEATING ADDRESS-CSE: SPELL TWO READS OF THE SAME HALFWORD DIFFERENTLY AND GCC CANNOT SHARE THE ADDRESS (P31 S67; byte-proven ov_SC01_077/func_8017FAAC, 154 ins) <sub>L31000</sub>
|
||||
- **§327** — A RANGE TEST MUST BE HImode: WITH `s32` + A `(u16)` CAST GCC *PROVES* THE MASK REDUNDANT AND DROPS THE `andi` (P31 S67; byte-proven ov_SC02_039/func_8017EC34, 154 ins) <sub>L31009</sub>
|
||||
- **§328** — THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INTO A SEPARATE `addiu` (P31 S67; byte-proven ov_SC07_007/func_80181B8C, 156 ins, NEW LAW) <sub>L31024</sub>
|
||||
|
||||
### loops & induction variables (33)
|
||||
### loops & induction variables (34)
|
||||
|
||||
- **§3-T1** — Loop pointer: top-of-body for `addu` induction, not constant-folded `addiu` <sub>L71</sub>
|
||||
- **§34** — The `func_80138ED0` giant crack: gcc-2.7.2's **3-qty sort bug** + the **zero-byte asm allocation toolkit** + the **giv-init fence** (Phase 24 T5; Opus→close=21, Fable5→MATCH ×134) <sub>L2473</sub>
|
||||
@@ -357,6 +363,7 @@
|
||||
- **§297** — ONE GIV SERVING MIXED-WIDTH LOADS *AND* A STORE NEEDS STRUCT-MEMBER SPELLING; CAST/INDEX SPELLINGS RE-SPLIT THE PSEUDO (P31 S61; wave m0a, `func_800347C8`, main, byte-proven 31/31) <sub>L29499</sub>
|
||||
- **§312** — A BARE RELATIONAL IN AN `if` GIVES THE `slt` NO TARGET: ASSIGN THE COMPARISON INTO THE ALREADY-ALLOCATED VARIABLE TO CHOOSE ITS DESTINATION REGISTER, AT ZERO INSTRUCTION COST (P31 S64 t5o-t5r; byte-proven func_80180DCC) <sub>L30159</sub>
|
||||
- **Addendum** — Loop sentinel wrongly hoisted out of a call-containing loop (func_801835B0) <sub>L30493</sub>
|
||||
- **§325** — REGALLOC: A SHARED SMALL CONSTANT STORED TWICE IN THE PRE-LOOP BLOCK IS A *local-alloc* $s-OCCUPANT THAT STEALS THE ARGUMENT ALLOCNO'S REGISTER — PIN THE ARGUMENT-DERIVED LOCAL, NOT THE CONSTANT (P31 S67; byte-proven ov_SC03_119/func_80184F18, 153 ins) <sub>L30987</sub>
|
||||
|
||||
### structs, block moves & memcpy (78)
|
||||
|
||||
@@ -439,7 +446,7 @@
|
||||
- **Addendum** — Masked-OR field-merge plateaus at close=10; bitfield store clears it (func_8017FD64) <sub>L30700</sub>
|
||||
- **§321** — FILE-SCOPE DUPLICATE ANONYMOUS-STRUCT TYPEDEFS ARE A HARD ERROR; THE SAME TEXT AT BLOCK SCOPE IS A WARNING (P31 S66; byte-proven func_80180728, func_8017F9F8, func_8017E07C) <sub>L30845</sub>
|
||||
|
||||
### types, signedness & load/store width (84)
|
||||
### types, signedness & load/store width (86)
|
||||
|
||||
- **§3-I1** — Unsigned range check: `(x - lo) < (hi-lo)` → `addiu`+`sltiu` <sub>L41</sub>
|
||||
- **§3-I2** — Byte mask forces `andi` even after `lbu` <sub>L47</sub>
|
||||
@@ -525,6 +532,8 @@
|
||||
- **§320** — THE §43 "RETURN-TYPE FLIP PAIR" IS **NOT** TU-EDIT-REQUIRED: THREE DRAFT-ONLY ESCAPES (P31 S66; byte-proven func_800CCBC0, func_800D30D0, func_800D2A24) <sub>L30786</sub>
|
||||
- **§321** — FILE-SCOPE DUPLICATE ANONYMOUS-STRUCT TYPEDEFS ARE A HARD ERROR; THE SAME TEXT AT BLOCK SCOPE IS A WARNING (P31 S66; byte-proven func_80180728, func_8017F9F8, func_8017E07C) <sub>L30845</sub>
|
||||
- **§323** — CARRYING FILE-SCOPE TYPES BETWEEN SPLIT TUs: DEDUPE BY NAME, SKIP HEADER-PROVIDED, AND THE GUARD TRAVELS WITH THE BLOCK (P31 S67; ov_SC02_000, two defects fixed, one open) <sub>L30896</sub>
|
||||
- **§327** — A RANGE TEST MUST BE HImode: WITH `s32` + A `(u16)` CAST GCC *PROVES* THE MASK REDUNDANT AND DROPS THE `andi` (P31 S67; byte-proven ov_SC02_039/func_8017EC34, 154 ins) <sub>L31009</sub>
|
||||
- **§328** — THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INTO A SEPARATE `addiu` (P31 S67; byte-proven ov_SC07_007/func_80181B8C, 156 ins, NEW LAW) <sub>L31024</sub>
|
||||
|
||||
### declarations, prototypes & K&R (101)
|
||||
|
||||
@@ -888,7 +897,7 @@
|
||||
- **§323** — CARRYING FILE-SCOPE TYPES BETWEEN SPLIT TUs: DEDUPE BY NAME, SKIP HEADER-PROVIDED, AND THE GUARD TRAVELS WITH THE BLOCK (P31 S67; ov_SC02_000, two defects fixed, one open) <sub>L30896</sub>
|
||||
- **§324** — THE ARITY-SELF WALL NEEDS *BOTH* HALVES: A K&R DEFINITION **AND** A NO-PROTO TU DECL; EITHER ALONE STILL FAILS (P31 S67; rtu-proven ov_SC01_005/func_8017FBCC, 14 instances measured) <sub>L30949</sub>
|
||||
|
||||
### build graph, splat & the harness (175)
|
||||
### build graph, splat & the harness (176)
|
||||
|
||||
- **§4** — Flag/toolchain gotchas <sub>L190</sub>
|
||||
- **Build** — mechanism — per-file opt override (splat resegmentation) <sub>L307</sub>
|
||||
@@ -1065,8 +1074,9 @@
|
||||
- **Addendum** — Addendum to §167-12 — the single-operand volatile keepalive, re-confirmed at a gated MATCH (func_8017F498) <sub>L30485</sub>
|
||||
- **§NNN** — REFUTED: A CODE BLOCK SHARED BY TWO *DIFFERENT* SWITCH STATEMENTS IS ORDINARY FORWARD `cross_jump`, NOT A SHAPE C CANNOT REACH — §164-11 ALREADY BANKS IT FROM PLAIN NESTED SWITCHES (P31 S66 round3, `func_8001B0D4`; ⚠ **UNPROVEN** — gate-REFUSED draft, closeness 68→45, never MATCHed) <sub>L30640</sub>
|
||||
- **§323a** — R53, TWICE IN ONE HOUR: A FAILED BUILD LEAVES THE PREVIOUS BINARY, AND `sha1sum` READS GREEN (P31 S67) <sub>L30926</sub>
|
||||
- **§328** — THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INTO A SEPARATE `addiu` (P31 S67; byte-proven ov_SC07_007/func_80181B8C, 156 ins, NEW LAW) <sub>L31024</sub>
|
||||
|
||||
### process, measurement & doctrine (116)
|
||||
### process, measurement & doctrine (117)
|
||||
|
||||
- **§8e** — The jtbl ALIGNMENT LAW + the pad-spec filter — multi-table .rodata spans (Phase 29, byte-proven; `.run/probe_jtbl/verdict.md`) <sub>L549</sub>
|
||||
- **§3-The** — mechanism: game-code dedup is SOURCE-LEVEL, not an object swap (R-D1, the key lesson) <sub>L945</sub>
|
||||
@@ -1184,8 +1194,9 @@
|
||||
- **§322** — A PROBE THAT ANSWERS A *NECESSARY BUT NOT SUFFICIENT* QUESTION WILL PRICE BLOCKED WORK AS FREE: RUN THE REAL PLANNER WHEN THE PLANNER IS PURE (P31 S67; measured, 96 of 159 open jtbl functions) <sub>L30866</sub>
|
||||
- **§323a** — R53, TWICE IN ONE HOUR: A FAILED BUILD LEAVES THE PREVIOUS BINARY, AND `sha1sum` READS GREEN (P31 S67) <sub>L30926</sub>
|
||||
- **§324** — THE ARITY-SELF WALL NEEDS *BOTH* HALVES: A K&R DEFINITION **AND** A NO-PROTO TU DECL; EITHER ALONE STILL FAILS (P31 S67; rtu-proven ov_SC01_005/func_8017FBCC, 14 instances measured) <sub>L30949</sub>
|
||||
- **§328** — THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INTO A SEPARATE `addiu` (P31 S67; byte-proven ov_SC07_007/func_80181B8C, 156 ins, NEW LAW) <sub>L31024</sub>
|
||||
|
||||
### (unbucketed — title matched no symptom vocabulary) (296)
|
||||
### (unbucketed — title matched no symptom vocabulary) (297)
|
||||
|
||||
- **§3-How** — to use this <sub>L30</sub>
|
||||
- **§1** — Idiom catalog (asm pattern → C that produces it) <sub>L39</sub>
|
||||
@@ -1483,6 +1494,7 @@
|
||||
- **Addendum** — Integer-space address arithmetic is a THIRD no-movable spelling, and a distant `SYM[0]` re (func_8017D89C) <sub>L30704</sub>
|
||||
- **RETRIEVAL** — FAILURES this round <sub>L30765</sub>
|
||||
- **§323b** — A SCRIPT THAT PARSES argv AT IMPORT CANNOT BE SHARED; EXTRACT THE PREDICATE, DO NOT COPY IT (P31 S67) <sub>L30935</sub>
|
||||
- **§330** — THE NEIGHBOUR-SHAPE LEVER IS THE CHEAPEST FIRST MOVE, AND IT IS UNDER-USED (P31 S67; four independent instances in one 20-function wave) <sub>L31042</sub>
|
||||
|
||||
|
||||
## All sections, in order
|
||||
@@ -2459,6 +2471,13 @@
|
||||
- **§323a** — R53, TWICE IN ONE HOUR: A FAILED BUILD LEAVES THE PREVIOUS BINARY, AND `sha1sum` READS GREEN (P31 S67) <sub>L30926</sub>
|
||||
- **§323b** — A SCRIPT THAT PARSES argv AT IMPORT CANNOT BE SHARED; EXTRACT THE PREDICATE, DO NOT COPY IT (P31 S67) <sub>L30935</sub>
|
||||
- **§324** — THE ARITY-SELF WALL NEEDS *BOTH* HALVES: A K&R DEFINITION **AND** A NO-PROTO TU DECL; EITHER ALONE STILL FAILS (P31 S67; rtu-proven ov_SC01_005/func_8017FBCC, 14 instances measured) <sub>L30949</sub>
|
||||
- **§325** — REGALLOC: A SHARED SMALL CONSTANT STORED TWICE IN THE PRE-LOOP BLOCK IS A *local-alloc* $s-OCCUPANT THAT STEALS THE ARGUMENT ALLOCNO'S REGISTER — PIN THE ARGUMENT-DERIVED LOCAL, NOT THE CONSTANT (P31 S67; byte-proven ov_SC03_119/func_80184F18, 153 ins) <sub>L30987</sub>
|
||||
- **§326** — DEFEATING ADDRESS-CSE: SPELL TWO READS OF THE SAME HALFWORD DIFFERENTLY AND GCC CANNOT SHARE THE ADDRESS (P31 S67; byte-proven ov_SC01_077/func_8017FAAC, 154 ins) <sub>L31000</sub>
|
||||
- **§327** — A RANGE TEST MUST BE HImode: WITH `s32` + A `(u16)` CAST GCC *PROVES* THE MASK REDUNDANT AND DROPS THE `andi` (P31 S67; byte-proven ov_SC02_039/func_8017EC34, 154 ins) <sub>L31009</sub>
|
||||
- **§328** — THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INTO A SEPARATE `addiu` (P31 S67; byte-proven ov_SC07_007/func_80181B8C, 156 ins, NEW LAW) <sub>L31024</sub>
|
||||
- **§329** — fold-const NARROWS `(int)s16_var & 0xFFF` ONTO THE *RAW HImode PSEUDO*, BREAKING THE SIGN-EXTEND/MASK REGISTER TIE — A ZERO-BYTE WIDENING TEMP RESTORES IT (P31 S67; byte-proven ov_SC01_084/func_80183244, 157 ins) <sub>L31034</sub>
|
||||
- **§330** — THE NEIGHBOUR-SHAPE LEVER IS THE CHEAPEST FIRST MOVE, AND IT IS UNDER-USED (P31 S67; four independent instances in one 20-function wave) <sub>L31042</sub>
|
||||
- **§331** — OPEN GAP: NO LEVER ELIMINATES AN *UNWANTED DUPLICATE* COPY AT A BRANCH-TARGET BLOCK HEAD (P31 S67; main/func_80013154, closeness 12, NOT solved) <sub>L31059</sub>
|
||||
|
||||
|
||||
---
|
||||
@@ -3443,3 +3462,10 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
|
||||
| L30926 | §323a | R53, TWICE IN ONE HOUR: A FAILED BUILD LEAVES THE PREVIOUS BINARY, AND `sha1sum` READS GRE |
|
||||
| L30935 | §323b | A SCRIPT THAT PARSES argv AT IMPORT CANNOT BE SHARED; EXTRACT THE PREDICATE, DO NOT COPY I |
|
||||
| L30949 | §324 | THE ARITY-SELF WALL NEEDS *BOTH* HALVES: A K&R DEFINITION **AND** A NO-PROTO TU DECL; EITH |
|
||||
| L30987 | §325 | REGALLOC: A SHARED SMALL CONSTANT STORED TWICE IN THE PRE-LOOP BLOCK IS A *local-alloc* $s |
|
||||
| L31000 | §326 | DEFEATING ADDRESS-CSE: SPELL TWO READS OF THE SAME HALFWORD DIFFERENTLY AND GCC CANNOT SHA |
|
||||
| L31009 | §327 | A RANGE TEST MUST BE HImode: WITH `s32` + A `(u16)` CAST GCC *PROVES* THE MASK REDUNDANT A |
|
||||
| L31024 | §328 | THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INT |
|
||||
| L31034 | §329 | fold-const NARROWS `(int)s16_var & 0xFFF` ONTO THE *RAW HImode PSEUDO*, BREAKING THE SIGN- |
|
||||
| L31042 | §330 | THE NEIGHBOUR-SHAPE LEVER IS THE CHEAPEST FIRST MOVE, AND IT IS UNDER-USED (P31 S67; four |
|
||||
| L31059 | §331 | OPEN GAP: NO LEVER ELIMINATES AN *UNWANTED DUPLICATE* COPY AT A BRANCH-TARGET BLOCK HEAD ( |
|
||||
|
||||
@@ -30983,3 +30983,87 @@ wall ledger, 8-20 → long-budget permuter. Do not price this rung as 14 banks.
|
||||
**BLAST RADIUS.** The TU-side half edits shared-within-binary source, so it needs the same
|
||||
journal/revert discipline as the arity pre-pass: keep the no-proto only where it bought a match,
|
||||
revert it everywhere else, and let the whole-binary gate arbitrate (G3/P9).
|
||||
|
||||
## §325 — REGALLOC: A SHARED SMALL CONSTANT STORED TWICE IN THE PRE-LOOP BLOCK IS A *local-alloc* $s-OCCUPANT THAT STEALS THE ARGUMENT ALLOCNO'S REGISTER — PIN THE ARGUMENT-DERIVED LOCAL, NOT THE CONSTANT (P31 S67; byte-proven ov_SC03_119/func_80184F18, 153 ins)
|
||||
|
||||
The twin gave the whole shape in one compile (147 -> 24 residual); the last 24 were a pure $s2/$s3
|
||||
swap that no pin-free dial reaches. WHY: a CSE'd HImode constant written twice in block 0
|
||||
(`desc.v[0].vz = 3` ×2) is a SINGLE-BLOCK, call-crossing quantity, so **local-alloc** assigns it a
|
||||
callee-saved register BEFORE global-alloc runs, and `find_free_reg` always takes the first free one —
|
||||
pushing the argument allocno from $s2 to $s3 and cascading. There is no 4th block-0 call-crossing
|
||||
local that could die before the `$a2` copy is born, so no source-level reordering reaches it.
|
||||
|
||||
**THE LEVER, AND ITS DIRECTION IS THE POINT:** pin the ARGUMENT-DERIVED local
|
||||
(`register s32 u __asm__("$18")` on the uv-base local), NOT the constant. Pinning the constant to
|
||||
$19 only reached closeness 15. §175's caller-saved-pin correctness trap does not apply to $s2.
|
||||
|
||||
## §326 — DEFEATING ADDRESS-CSE: SPELL TWO READS OF THE SAME HALFWORD DIFFERENTLY AND GCC CANNOT SHARE THE ADDRESS (P31 S67; byte-proven ov_SC01_077/func_8017FAAC, 154 ins)
|
||||
|
||||
The target keeps three separate `lui/addu/l*` $at groups where gcc wanted one shared base. Writing
|
||||
one read as `D_8018A9B2[i]` and the other as `*(s16*)(D_8018A9B0 + i*4 + 2)` yields DIFFERENT address
|
||||
rtx for the same location, so address-CSE cannot merge them and each keeps its own %hi/%lo pair.
|
||||
Same function also needed §186: moving the `0x1A +=` before the `0x16 -= 0x40` put the if-arm's last
|
||||
store in $v1 vs the else-arm's $v0, blocking the jump2 tail-merge that was costing 2 instructions
|
||||
(cross-jump runs AFTER scheduling, so only a post-schedule register difference blocks it).
|
||||
|
||||
## §327 — A RANGE TEST MUST BE HImode: WITH `s32` + A `(u16)` CAST GCC *PROVES* THE MASK REDUNDANT AND DROPS THE `andi` (P31 S67; byte-proven ov_SC02_039/func_8017EC34, 154 ins)
|
||||
|
||||
`s32 q` + `(u16)q` lets fold-const derive `q = 0x7F ± (s16 x >> 1)` and conclude the mask cannot
|
||||
change the value, so the `andi` never emits — a REAL −1 length drift that reads like a schedule
|
||||
difference and sends you hunting the scheduler. Declare `u16 q` and test `q >= 0x2F && q < 0x80`.
|
||||
Three more levers from the same function, all reusable:
|
||||
* **§194-H in-place AND.** `v1 &= 0x2000;` inside `if (v1 != 0)` — only the IN-PLACE form reuses the
|
||||
hard reg, which is what lets dbr speculate it into the guard's delay slot.
|
||||
* **§3-T2 chain order.** Put the LONG chain (`p->0xA += D_8019B5E6[i]`) BEFORE the short store
|
||||
(`p->0x34 = s1+1`); sched1 re-swaps the two `sh`s itself, and the leading `lh` forces the genuine
|
||||
nop delay slot (`may_trap_p` on a MEM blocks dbr) while freeing $a0 for the table value.
|
||||
* **Build the 2nd argument INLINE in the call** with a `(u16)` cast. As a preceding statement
|
||||
(`u16 code = ...; & 0xFFFF`) its sll/ori/or chain outranks `move $a0,$s1` in sched priority and
|
||||
steals the beqz delay slot.
|
||||
|
||||
## §328 — THE VOLATILE ALIAS MUST BE AN *OBJECT*, NOT A CAST: `*(volatile s32*)&sym` UNFOLDS %lo INTO A SEPARATE `addiu` (P31 S67; byte-proven ov_SC07_007/func_80181B8C, 156 ins, NEW LAW)
|
||||
|
||||
Four identical `lui/lw D_801C79A0; j LDDC; addiu` tails needed TWO levers together:
|
||||
(1) `register __asm__("$2")` pins on cases 0 and 3 — cc1 otherwise homes them in $v1, missing the
|
||||
depth-2 merge onto `.L80181DDC` and deep-merging case 0 into case 3 at the shared `jal`; and
|
||||
(2) an `__asm__`-aliased **volatile OBJECT** view of `D_801C79A0` for case 3's reload only, to block
|
||||
the back-merge into case 1's identical tail.
|
||||
**THE LAW:** the volatile must be a declared aliased object (`extern volatile s32 vD_x __asm__("D_801C79A0");`).
|
||||
Spelling it `*(volatile s32*)&D_801C79A0` unfolds the %lo into a separate `addiu` and costs +1 ins.
|
||||
|
||||
## §329 — fold-const NARROWS `(int)s16_var & 0xFFF` ONTO THE *RAW HImode PSEUDO*, BREAKING THE SIGN-EXTEND/MASK REGISTER TIE — A ZERO-BYTE WIDENING TEMP RESTORES IT (P31 S67; byte-proven ov_SC01_084/func_80183244, 157 ins)
|
||||
|
||||
gcc-2.7.2 narrows the AND onto the raw HImode pseudo (`andi $s2,$a1`), so the sign-extended value and
|
||||
the masked value land in different registers and the difference cascades — measured 27 register rows
|
||||
plus 3 schedule rows from this one cause. FIX: `s32 e = t;` — a named SImode temp that forces ONE
|
||||
sign_extend, so local-alloc coalesces the fraction onto $s1 (`andi $s1,$s1`) while `sh $a1,0xDC`
|
||||
keeps the raw register. 30 residual rows -> 0. (The neighbour §194-E gave the TU shape first: 70->30.)
|
||||
|
||||
## §330 — THE NEIGHBOUR-SHAPE LEVER IS THE CHEAPEST FIRST MOVE, AND IT IS UNDER-USED (P31 S67; four independent instances in one 20-function wave)
|
||||
|
||||
Before any codegen reasoning, read an ALREADY-BANKED function in the same TU and copy its SPELLING,
|
||||
not just its structure. Measured this wave:
|
||||
* `md_MAIN_031/func_800CAE0C` — the neighbour 6 lines away contained the same 3-call body verbatim
|
||||
(same symbols, same `(void*)` casts). Copied its call spelling -> MATCH on first compile.
|
||||
* `ov_SC01_084/func_8017EF28` — the neighbour's memory-to-memory house form
|
||||
(`*(u16*)(p+0xE2) = *(u16*)(p+0xE2) + 0x70`) replaced a reused `int v` local and dissolved ALL 18
|
||||
$v0/$v1 REGALLOC-PERM rows in ONE compile — no §137 arithmetic, no pins.
|
||||
* `ov_SC01_084/func_80183244` — neighbour gave the exact TU shape, 70 -> 30, nins exact.
|
||||
* `main/func_8002A088` + `func_8002A7B4` — mirroring the already-banked in-TU twin `func_8002A2D4`'s
|
||||
local shape (separate i/off locals, subtraction inlined as the call arg) produced the correct 0x18
|
||||
frame; the warm-start body had been 4-off on frame size alone.
|
||||
**The corollary that costs functions when ignored:** a warm-start body from ANOTHER binary is often
|
||||
worth less than the neighbour 20 lines away. `func_8017FAAC`'s warm start was a different function
|
||||
entirely and was discarded.
|
||||
|
||||
## §331 — OPEN GAP: NO LEVER ELIMINATES AN *UNWANTED DUPLICATE* COPY AT A BRANCH-TARGET BLOCK HEAD (P31 S67; main/func_80013154, closeness 12, NOT solved)
|
||||
|
||||
nins matches 43/43. An inline-asm-forced `sy=y` copy needed at the "ax!=ay" block head gets
|
||||
DUPLICATED by gcc into two hardregs: `$t1` in the bne's delay slot (correct) and a redundant `$t3` at
|
||||
the block's own head (wrong), cascading into idx10-16, a register-role swap with `nx` at idx25-28,
|
||||
and `$t3`-vs-`$t1` at the final subu. Tried and REFUTED: 5 sy-placement variants, register pins
|
||||
($9/$8/$10), an explicit-register asm constraint (cc1-fail), and substituting `(s16)sy` for `ay` in
|
||||
the later comparisons (regresses to 46 ins). ~16 iterations.
|
||||
§164-36/§31-lever-2 cover asm at a branch-target block head + delay-slot fill, but every listed lever
|
||||
FILLS AN EMPTY SLOT; none REMOVES A DUPLICATE. This is a genuine hole in the knowledge base — record
|
||||
it as such rather than spending another wave slot on it blind.
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
#!/usr/bin/env python3
|
||||
"""gate_wave.py — gate a whole wave the fast way: split on jtbl, run both lanes CONCURRENTLY. (P31 S67)
|
||||
|
||||
WHY THIS EXISTS (measured, S67, and it cost an hour). `parallel_gate` cannot host a jtbl-bearing
|
||||
draft: `harvest_verify`'s carve runs `make extract`, and a worker's worktree `asm/` is a SYMLINK to
|
||||
the main tree (`parallel_gate.py:77` states the invariant — "a gate never writes them, only
|
||||
`make extract` does"). Knowing that, I gated a 16-binary wave SERIALLY to protect the ONE jtbl
|
||||
function in it. Numbers from that run:
|
||||
|
||||
4 binaries through parallel_gate (4 workers) : 103 s wall (87 / 87 / 88 / 102 s each)
|
||||
the same 4 serially : ~6 min
|
||||
the full 16 serially : ~1 hour, for 1 jtbl draft
|
||||
|
||||
Applying a true rule to the wrong scope is the defect. The predicate is per-DRAFT and it is cheap:
|
||||
`jtbl_carve --probe` runs the real planner (since S67), so the split costs one probe per target.
|
||||
|
||||
WHY THE SPLIT PRECEDES THE RUN, rather than triaging failures afterwards. A jtbl draft in a
|
||||
parallel worker does NOT fail cleanly — it re-extracts through the shared `asm/` symlink and writes
|
||||
the MAIN tree while nine other workers read it. "Run everything parallel and re-run what failed"
|
||||
can therefore poison the whole batch instead of isolating one draft. Split first, then triage what
|
||||
the filter missed (R32: verify the pre-filter against the outcome; never assume it was complete).
|
||||
|
||||
tools/gate_wave.py --drafts .run/S67_ov --workers 8 --commit [--r22]
|
||||
<drafts>/<binary>/<fn>.c — the layout gate_stage and parallel_gate both take
|
||||
"""
|
||||
import argparse
|
||||
import concurrent.futures as cf
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO = os.path.dirname(HERE)
|
||||
sys.path.insert(0, HERE)
|
||||
PY = os.path.join(REPO, ".venv/bin/python")
|
||||
_JTBL_RE = re.compile(r"jtbl_[0-9A-Fa-f]{8}")
|
||||
|
||||
import corpus # noqa: E402
|
||||
import progress # noqa: E402
|
||||
|
||||
|
||||
def sh(cmd, **kw):
|
||||
return subprocess.run(cmd, cwd=REPO, capture_output=True, text=True, **kw)
|
||||
|
||||
|
||||
def has_jtbl(binary, fn):
|
||||
"""Does this function's target .s reference a jump table?
|
||||
|
||||
The SAME predicate `harvest_verify._fn_has_jtbl` uses to decide whether to carve — one
|
||||
definition of the question, so the router and the gate cannot disagree (R33/R34).
|
||||
"""
|
||||
st = next((s for s in corpus.stubs(binary).values() if s.symbol == fn), None)
|
||||
if st is None:
|
||||
return False # already banked; the gate will no-op it
|
||||
try:
|
||||
return bool(_JTBL_RE.search(open(os.path.join(REPO, st.asm_path), errors="replace").read()))
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def split(drafts_root):
|
||||
"""(parallel_plan, serial_items) — parallel is a plan.json for parallel_gate; serial is
|
||||
[(binary, [fn...])] for gate_stage."""
|
||||
par, ser = [], []
|
||||
known = set(progress.BINARIES)
|
||||
unknown = []
|
||||
for binary in sorted(os.listdir(os.path.join(REPO, drafts_root))):
|
||||
d = os.path.join(REPO, drafts_root, binary)
|
||||
if not os.path.isdir(d):
|
||||
continue
|
||||
# THE LADDER LEAVES ITS OWN OUTPUT DIRS BESIDE THE DRAFTS. `gate_stage._xform` writes
|
||||
# `<drafts>-cn`, `-cn-cast`, `-cn-cast-rc`, `-cn-cast-rc-sd` as SIBLINGS inside this root,
|
||||
# so a plain listdir reports 20 "binaries" for a 16-binary wave and would gate transformed
|
||||
# intermediates as if they were fresh work. Validate the name against the corpus and refuse
|
||||
# what is not a binary (R32: assert the denominator; R43: refuse, never mishandle).
|
||||
if binary not in known:
|
||||
unknown.append(binary)
|
||||
continue
|
||||
fns = sorted(f[:-2] for f in os.listdir(d) if f.endswith(".c"))
|
||||
if not fns:
|
||||
continue
|
||||
jt = [f for f in fns if has_jtbl(binary, f)]
|
||||
# A binary goes SERIAL if ANY of its drafts carries a table: the gate runs per binary, and
|
||||
# one carving draft is enough to make the whole worker unsafe.
|
||||
(ser if jt else par).append((binary, fns, jt))
|
||||
if unknown:
|
||||
print(" skipped %d non-binary dir(s) (ladder intermediates): %s"
|
||||
% (len(unknown), " ".join(sorted(unknown)[:6])), file=sys.stderr)
|
||||
plan = [{"binary": b, "drafts": os.path.join(REPO, drafts_root, b)} for b, _, _ in par]
|
||||
return plan, ser, par
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--drafts", required=True, help="<dir>/<binary>/<fn>.c")
|
||||
ap.add_argument("--workers", type=int, default=8)
|
||||
ap.add_argument("--commit", action="store_true")
|
||||
ap.add_argument("--r22", action="store_true")
|
||||
ap.add_argument("--source-tag", default="wave")
|
||||
ap.add_argument("--plan-only", action="store_true")
|
||||
a = ap.parse_args()
|
||||
|
||||
plan, ser, par = split(a.drafts)
|
||||
n_par = sum(len(f) for _, f, _ in par)
|
||||
n_ser = sum(len(f) for _, f, _ in ser)
|
||||
print("split: %d binaries / %d drafts PARALLEL · %d binaries / %d drafts SERIAL (jtbl)"
|
||||
% (len(plan), n_par, len(ser), n_ser))
|
||||
for b, fns, jt in ser:
|
||||
print(" serial %-16s %s (jtbl: %s)" % (b, " ".join(fns), " ".join(jt)))
|
||||
if a.plan_only:
|
||||
return 0
|
||||
if not plan and not ser:
|
||||
print("REFUSING: no drafts found under %s" % a.drafts, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
plan_path = os.path.join(REPO, ".run/gate_wave_plan.json")
|
||||
with open(plan_path, "w") as fh:
|
||||
json.dump(plan, fh, indent=1)
|
||||
|
||||
def run_parallel():
|
||||
if not plan:
|
||||
return "(no parallel lane)"
|
||||
cmd = [PY, "tools/parallel_gate.py", "--plan", ".run/gate_wave_plan.json",
|
||||
"--workers", str(a.workers)]
|
||||
if a.commit:
|
||||
cmd.append("--commit")
|
||||
r = sh(cmd, timeout=14400)
|
||||
return (r.stdout or "") + (r.stderr or "")
|
||||
|
||||
def run_serial():
|
||||
out = []
|
||||
for b, _, _ in ser:
|
||||
cmd = [PY, "tools/gate_stage.py", "--binary", b,
|
||||
"--drafts", os.path.join(a.drafts, b), "--source-tag", a.source_tag]
|
||||
if a.commit:
|
||||
cmd.append("--commit")
|
||||
r = sh(cmd, timeout=7200)
|
||||
out.append("[serial] %s rc=%d %s" % (b, r.returncode, (r.stdout or "").strip()[-200:]))
|
||||
return "\n".join(out)
|
||||
|
||||
# BOTH LANES AT ONCE (Drew, S67: no lane should idle). They are safe together — parallel_gate's
|
||||
# workers are isolated by construction and it adopts a file only if the main tree's copy still
|
||||
# matches the pinned baseline, REFUSING rather than clobbering if the serial lane moved it.
|
||||
with cf.ThreadPoolExecutor(max_workers=2) as ex:
|
||||
fp, fs = ex.submit(run_parallel), ex.submit(run_serial)
|
||||
print(fp.result())
|
||||
print(fs.result())
|
||||
|
||||
if a.r22:
|
||||
print("[gate_wave] R22 clean-fleet ...")
|
||||
r = sh(["make", "clean"], timeout=3600)
|
||||
r = sh(["make", "extract-all"], timeout=14400)
|
||||
r = sh(["make", "check-all"], timeout=14400)
|
||||
tail = ((r.stdout or "") + (r.stderr or "")).strip().splitlines()[-3:]
|
||||
print("\n".join(tail))
|
||||
if r.returncode:
|
||||
print("R22 FAILED — do not report banks as done (R22/R50)", file=sys.stderr)
|
||||
return 2
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user