diff --git a/docs/cookbook-index.md b/docs/cookbook-index.md index 213b8e91f..7e629626c 100644 --- a/docs/cookbook-index.md +++ b/docs/cookbook-index.md @@ -2,7 +2,7 @@ > **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section. > -> `docs/matching-cookbook.md` is ~716 KB / 564 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses. +> `docs/matching-cookbook.md` is ~716 KB / 585 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses. **How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win. @@ -34,7 +34,7 @@ ## By symptom -### delay slots & branches (8) +### delay slots & branches (9) - **§3-T4** — Branch polarity: invert the source condition to flip gcc's chosen branch L90 - **§5a** — Cross-jump tail-merge — gcc collapses two byte-identical blocks the original kept separate (FIX FOUND) L211 @@ -43,9 +43,10 @@ - **§162** — CROSS-JUMP DIRECTION: the surviving copy is always the LATER one, so a BACKWARD `j` into a sibling arm is a source `goto` (P30 S48) L11263 - **§162** — The cross-jump "CALL veto" is a COUNT law, not a CALL law (BOUNDS §88a; P30 S48, `func_80189540`) L11299 - **§177** — 🔴 THE EPILOGUE RETURN-DELAY SLOT IS DECIDED BY YOUR SAVED-REGISTER SET, NOT BY SCHEDULING L17156 -- **§176-A** — "SCHEDULE / DELAY-SLOT / LENGTH-DRIFT ±1" ⇒ check STATEMENT ORDER around the call first L17602 +- **§176-A** — "SCHEDULE / DELAY-SLOT / LENGTH-DRIFT ±1" ⇒ check STATEMENT ORDER around the call first L17613 +- **§186** — CROSS-JUMPING RUNS **AFTER** SCHEDULING, SO NO C-LEVEL BARRIER CAN STEER IT L18060 -### instruction scheduling (24) +### instruction scheduling (25) - **§3-T2** — Source statement order drives instruction scheduling L78 - **§3** — When a diff is pure scheduling → decomp-permuter (harness built, Phase 6) L107 @@ -68,11 +69,12 @@ - **§16Z** — SHARPENS *(sharpens §37 "the /s-DEP LATTICE", §136-13, §136-14, §16Xy, §162q, `gcc-2.7.2-map/sched.md` §64)* L14337 - **§175** — A CALLER-SAVED REGISTER PIN CAN BE A CORRECTNESS BUG, NOT JUST A SCHEDULING CHOICE (P31 wave H, 2026-08-15) L16709 - **§177** — 🔴 THE EPILOGUE RETURN-DELAY SLOT IS DECIDED BY YOUR SAVED-REGISTER SET, NOT BY SCHEDULING L17156 -- **§3-B.** — A `return ` IS A PRIORITY-1 HARD-REG SET THE SCHEDULER PLACES FIRST (func_8001BE30, 92 ins) L17230 -- **§176-A** — "SCHEDULE / DELAY-SLOT / LENGTH-DRIFT ±1" ⇒ check STATEMENT ORDER around the call first L17602 -- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17666 +- **§3-B.** — A `return ` IS A PRIORITY-1 HARD-REG SET THE SCHEDULER PLACES FIRST (func_8001BE30, 92 ins) L17241 +- **§176-A** — "SCHEDULE / DELAY-SLOT / LENGTH-DRIFT ±1" ⇒ check STATEMENT ORDER around the call first L17613 +- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17677 +- **§186** — CROSS-JUMPING RUNS **AFTER** SCHEDULING, SO NO C-LEVEL BARRIER CAN STEER IT L18060 -### register allocation & pins (51) +### register allocation & pins (54) - **§10** — Closing the regalloc/scheduling hard tail by hand (LZSS, Phase 7 session F — the full close) L835 - **Residual** — A — commutative `|`/`&`/`+` result lands in the wrong source-operand register L856 @@ -119,19 +121,22 @@ - **§175** — A CALLER-SAVED REGISTER PIN CAN BE A CORRECTNESS BUG, NOT JUST A SCHEDULING CHOICE (P31 wave H, 2026-08-15) L16709 - **§176j** — STOPPING A WAVE MID-FLIGHT COSTS THE IN-FLIGHT TAIL (and how much is recoverable) L17093 - **§177** — 🔴 THE EPILOGUE RETURN-DELAY SLOT IS DECIDED BY YOUR SAVED-REGISTER SET, NOT BY SCHEDULING L17156 -- **§179-A** — 🔴 A LOOP-WALKED POINTER **PARAMETER** HANDS ITS ARGUMENT REGISTER TO THE GIV (9 byte-proofs) L17299 -- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17495 -- **§179-G** — 🟡 A PIN CAN **CREATE** A COMBINE `LOG_LINK` AND DELETE AN `andi` (sixth RC-5 channel, n=1) L17518 -- **§176** — SEVEN LEVERS FROM THE P31 OVERNIGHT WAVES (2026-08-15): statement order, false regalloc, and the pin that fights back L17596 -- **§176-B** — "REGALLOC-PERM, 1-4 instructions off" ⇒ it is usually NOT register allocation L17626 -- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17666 +- **§179-A** — 🔴 A LOOP-WALKED POINTER **PARAMETER** HANDS ITS ARGUMENT REGISTER TO THE GIV (9 byte-proofs) L17310 +- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17506 +- **§179-G** — 🟡 A PIN CAN **CREATE** A COMBINE `LOG_LINK` AND DELETE AN `andi` (sixth RC-5 channel, n=1) L17529 +- **§176** — SEVEN LEVERS FROM THE P31 OVERNIGHT WAVES (2026-08-15): statement order, false regalloc, and the pin that fights back L17607 +- **§176-B** — "REGALLOC-PERM, 1-4 instructions off" ⇒ it is usually NOT register allocation L17637 +- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17677 +- **§3-18** — of 20 reconciled while keeping the match. The two that did not are mechanism, not effort. L17930 +- **§186** — CROSS-JUMPING RUNS **AFTER** SCHEDULING, SO NO C-LEVEL BARRIER CAN STEER IT L18060 +- **§186c** — WHERE A VALUE IS LOADED DECIDES WHICH ALLOCATOR OWNS IT, AND THEREFORE ITS REGISTER L18089 ### CSE / redundancy / rematerialization (4) - **§46** — The `func_80178D40` crack (890 ins ×134, the heaviest core in the game): four LOOP-STRUCTURE levers cheap-Opus found by reading loop.c/jump.c/cse.c (Phase 26 session 8, 2026-07-13) L3313 - **§83d** — CSE's quantity budget is WHOLE-FUNCTION, so a local rewrite cannot fix a local symptom L6452 - **§153** — THE ADDRESS-REMATERIALISATION LAUNDER: a third zero-emission asm lever (P30 S43, `func_8018D98C`, 710 ins) L10463 -- **§176-D** — CSE-class levers used in reverse (two sharpenings of §153 and cse_expr §2) L17692 +- **§176-D** — CSE-class levers used in reverse (two sharpenings of §153 and cse_expr §2) L17703 ### loops & induction variables (13) @@ -145,11 +150,11 @@ - **§145** — Three loop/combine levers from the S40 wave-2 drafters (16/16 match_one) L9924 - **§148** — The loop.c hoisting THRESHOLD is arithmetic you can compute, and the `?:` clamp that folds to MIN_EXPR (P30 S42, `func_8017C6F4`, 947 ins) L10142 - **§171a** — THE MECHANICAL A-PROP DRAFT (P30 S50): 256 members banked with no agent in the loop L16417 -- **§179-A** — 🔴 A LOOP-WALKED POINTER **PARAMETER** HANDS ITS ARGUMENT REGISTER TO THE GIV (9 byte-proofs) L17299 -- **§179-E** — A `>2*MAX_MOVE_BYTES` BLOCK COPY IS A **STRUCT ASSIGNMENT**, NOT A HAND LOOP L17469 -- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17495 +- **§179-A** — 🔴 A LOOP-WALKED POINTER **PARAMETER** HANDS ITS ARGUMENT REGISTER TO THE GIV (9 byte-proofs) L17310 +- **§179-E** — A `>2*MAX_MOVE_BYTES` BLOCK COPY IS A **STRUCT ASSIGNMENT**, NOT A HAND LOOP L17480 +- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17506 -### structs, block moves & memcpy (39) +### structs, block moves & memcpy (40) - **§3-T2** — Source statement order drives instruction scheduling L78 - **§5** — Known hard-residual classes (instruction-identical, one byte-exact blocker) L199 @@ -187,11 +192,12 @@ - **§16Z** — SHARPENS *(sharpens §37 "the /s-DEP LATTICE", §136-13, §136-14, §16Xy, §162q, `gcc-2.7.2-map/sched.md` §64)* L14337 - **§3-The** — ADDRESS-CLASS TABLE: which load/store pairs even REACH the `/s` clause (P30 S48 wave 4, `func_80185B44`, ov_SC03_014) L14339 - **§176g** — SIZE A WAVE BY INSTRUCTIONS, NOT BY CARDS (P31 S52 — the adopted doctrine) L16927 -- **§3-F.** — `MEM_IN_STRUCT_P` ASYMMETRY IN `true_dependence` (func_80037144, 124 ins) L17267 -- **§179-E** — A `>2*MAX_MOVE_BYTES` BLOCK COPY IS A **STRUCT ASSIGNMENT**, NOT A HAND LOOP L17469 -- **§176-B** — "REGALLOC-PERM, 1-4 instructions off" ⇒ it is usually NOT register allocation L17626 +- **§3-F.** — `MEM_IN_STRUCT_P` ASYMMETRY IN `true_dependence` (func_80037144, 124 ins) L17278 +- **§179-E** — A `>2*MAX_MOVE_BYTES` BLOCK COPY IS A **STRUCT ASSIGNMENT**, NOT A HAND LOOP L17480 +- **§176-B** — "REGALLOC-PERM, 1-4 instructions off" ⇒ it is usually NOT register allocation L17637 +- **§189** — FIVE COMPILER LAWS MINED FROM THE WAVE R/S JOURNALS (P31 S53), each source-cited and re-derived by a second agent L18194 -### types, signedness & load/store width (36) +### types, signedness & load/store width (38) - **§3-I1** — Unsigned range check: `(x - lo) < (hi-lo)` → `addiu`+`sltiu` L41 - **§3-I2** — Byte mask forces `andi` even after `lbu` L47 @@ -228,9 +234,11 @@ - **§3-C.** — PAC type 1 = the same payload class as type 4, just NOT compressed L10545 - **§155b** — check the TYPE your oracle returns before comparing against it (S45 p5) L10584 - **§3-B.** — Typedef handling — the only strategy that survives contact L16991 -- **§3-D.** — A NARROW TYPE BLOCKS COPY ELISION (func_8001D3FC — new idiom) L17251 +- **§3-D.** — A NARROW TYPE BLOCKS COPY ELISION (func_8001D3FC — new idiom) L17262 +- **§185** — EDIT THE SIDE THAT IS CHEAP TO VERIFY, AND CHECK A TU RETYPE AT ITS USE SITES L18028 +- **§186b** — A NO-SAVE 16-BYTE FRAME IN A LEAF FUNCTION MEANS `s16` LOCALS, NOT A HIDDEN CALL L18081 -### declarations, prototypes & K&R (58) +### declarations, prototypes & K&R (59) - **§3-T4** — Branch polarity: invert the source condition to flip gcc's chosen branch L90 - **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) L437 @@ -290,8 +298,9 @@ - **§159** — THE DECLARATION AXIS: conform to byte-truth, and make every guard state its COVERAGE (P30 S47; ~10,930 sites across 8 axes, fleet byte-identical) L10825 - **§168** — THE COUSIN TIER (P30 S49, 2026-08-12): h_seq's exact-hash brittleness, measured — and the similarity map above it L16208 - **§176f** — THE DECLARATION FORM IS A MATCHING LEVER, SO RECONCILE TOWARD THE FORM THE MATCH NEEDS (P31 S52) L16896 +- **§183** — THE DECLARATION-RECONCILIATION PLAYBOOK (P31 S53, measured on 20 byte-verified drafts) L17929 -### jump tables & switches (28) +### jump tables & switches (29) - **§8** — rodata island (compiler jump tables) — the `.data→.rodata→.data` sandwich (Phase 7) L320 - **§8a** — rodata island in a flat OVERLAY — the tail sandwich, per matched jr-function (Phase 26 — PoC PROVEN) L342 @@ -320,7 +329,8 @@ - **§132b** — When the span's already-matched owner is ITSELF multi-switch: `--span-rel` (P30 S1, `func_8014032C`) L8660 - **§139** — A GATE THAT GREPS FOR VERDICTS MUST ASSERT 1:1 ACCOUNTING; and a `--src` filter must not survive a carve (P30 S38, wave 6: 10 of 16 drafts vanished) L9591 - **§161** — THE RETRY-WAVE HARVEST: a jump table indexed from zero, and two allocator traps (P30 S47) L10966 -- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17495 +- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17506 +- **§188** — 🔴 THE `jr $ra` + `addiu $sp` TAIL IS AN **ASSEMBLER** ARTIFACT, NOT A FRAME SHAPE L18144 ### optimisation level (-O0/-O2) (12) @@ -425,9 +435,9 @@ - **§168** — THE COUSIN TIER (P30 S49, 2026-08-12): h_seq's exact-hash brittleness, measured — and the similarity map above it L16208 - **§171a** — THE MECHANICAL A-PROP DRAFT (P30 S50): 256 members banked with no agent in the loop L16417 - **§171b** — THREE CARRIES THE MECHANICAL DRAFT NEEDS (P30 S50, banking the top-reach families) L16467 -- **§3-G.** — TWO MODELLING TRAPS THAT COST THESE AGENTS SWEEPS OF HUNDREDS OF COMPILES L17273 +- **§3-G.** — TWO MODELLING TRAPS THAT COST THESE AGENTS SWEEPS OF HUNDREDS OF COMPILES L17284 -### integration / TU plumbing (41) +### integration / TU plumbing (43) - **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) L437 - **§8d** — Templating a body INTO a TU must not CHANGE its declaration environment — demote the carried data externs (Phase 26 session 8, byte-proven on `func_8015AE2C` ×133) L483 @@ -470,8 +480,10 @@ - **§166** — THE DESTINATION-TU ORACLE (P30 S48): the seven-attempt bug that was never codegen L14899 - **§173** — THE STORED-PLUMBING RECOVERY RECIPE (P31 T6): symfix-first, per-group isolation, and where the verdicts have no drafts L16603 - **§176d** — THE CONFLICT TABLE MUST BE SEEDED FROM THE TU, AND KEYED PER FILE (P31 S52, 2026-08-15) L16800 +- **§180b** — WHAT THE PRE-GATE LADDER ACTUALLY FINDS IN A COLD PILE (the shape of integration debt) L17810 +- **§185** — EDIT THE SIDE THAT IS CHEAP TO VERIFY, AND CHECK A TU RETYPE AT ITS USE SITES L18028 -### build graph, splat & the harness (117) +### build graph, splat & the harness (125) - **§4** — Flag/toolchain gotchas L190 - **Build** — mechanism — per-file opt override (splat resegmentation) L288 @@ -587,11 +599,19 @@ - **§3-C2.** — RECONCILE BEFORE THE FIRST GATE — a parked draft gets HARDER to bank, not easier L17020 - **§3-D.** — The measured cost shape, and what to build next L17043 - **§176i** — WHAT A STATIC PRE-GATE CHECK CAN AND CANNOT PROVE (P31 S52, wave Q) L17061 -- **§179** — IDIOMS MINED FROM THE WAVE P/Q JOURNALS (P31 S52, harvest pass) L17289 -- **§179-G** — 🟡 A PIN CAN **CREATE** A COMBINE `LOG_LINK` AND DELETE AN `andi` (sixth RC-5 channel, n=1) L17518 -- **§176c** — MAIN (SLUS_007.26) CANNOT BE GATED INCREMENTALLY L17585 +- **§179** — IDIOMS MINED FROM THE WAVE P/Q JOURNALS (P31 S52, harvest pass) L17300 +- **§179-G** — 🟡 A PIN CAN **CREATE** A COMBINE `LOG_LINK` AND DELETE AN `andi` (sixth RC-5 channel, n=1) L17529 +- **§176c** — MAIN (SLUS_007.26) CANNOT BE GATED INCREMENTALLY L17596 +- **§180** — THE LEFTOVER-DRAFT HARVEST: RE-VERIFY WHAT YOU ALREADY HAVE BEFORE DRAFTING ANYTHING NEW L17781 +- **§180b** — WHAT THE PRE-GATE LADDER ACTUALLY FINDS IN A COLD PILE (the shape of integration debt) L17810 +- **§181** — WHAT A WAVE'S GATE ACTUALLY REJECTS (P31 S53, measured on wave R's 45-draft main pile) L17847 +- **§184** — COMMENT-BLINDNESS IS A DEFECT CLASS, NOT A BUG (P31 S53: three tools, one root cause, one session) L18000 +- **§187** — 🔴 "SAME SOURCE" IS NOT "SAME OBJECT": THE SDK BUILD AND THE GAME BUILD DISAGREE ON GTE NOPS L18106 +- **§190** — THREE PRESCRIPTIONS FROM THE SAME HARVEST (weaker evidence than §189, honestly labelled) L18255 +- **§191** — WHAT THIS HARVEST DID **NOT** BANK (4 rejected, 4 narrowed) — recorded so it is not re-derived L18292 +- **§192** — THE PRE-GATE LADDER WAS MAIN-ONLY, AND NOBODY COULD SEE IT (P31 S54) L18314 -### process, measurement & doctrine (79) +### process, measurement & doctrine (85) - **§8e** — The jtbl ALIGNMENT LAW + the pad-spec filter — multi-table .rodata spans (Phase 29, byte-proven; `.run/probe_jtbl/verdict.md`) L530 - **§3-The** — mechanism: game-code dedup is SOURCE-LEVEL, not an object swap (R-D1, the key lesson) L926 @@ -670,10 +690,16 @@ - **§176h** — THE BATCH-SUBSTITUTION HAZARD MAP (P31 S52): seven holes, three wrong fixes, one law L16965 - **§3-D.** — The measured cost shape, and what to build next L17043 - **§176j-2** — THE REPAIR PASS, MEASURED (do this instead of resuming) L17119 -- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17666 -- **§176-E** — Two cheap source spellings, both cc1-probed L17718 +- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17677 +- **§176-E** — Two cheap source spellings, both cc1-probed L17729 +- **§180** — THE LEFTOVER-DRAFT HARVEST: RE-VERIFY WHAT YOU ALREADY HAVE BEFORE DRAFTING ANYTHING NEW L17781 +- **§181** — WHAT A WAVE'S GATE ACTUALLY REJECTS (P31 S53, measured on wave R's 45-draft main pile) L17847 +- **§182** — §177's HONEST NEGATIVE: the epilogue lever cracked 4 of 16, and the `800c3` cluster held L17903 +- **§183** — THE DECLARATION-RECONCILIATION PLAYBOOK (P31 S53, measured on 20 byte-verified drafts) L17929 +- **§185** — EDIT THE SIDE THAT IS CHEAP TO VERIFY, AND CHECK A TU RETYPE AT ITS USE SITES L18028 +- **§190** — THREE PRESCRIPTIONS FROM THE SAME HARVEST (weaker evidence than §189, honestly labelled) L18255 -### (unbucketed — title matched no symptom vocabulary) (188) +### (unbucketed — title matched no symptom vocabulary) (192) - **§3-How** — to use this L30 - **§1** — Idiom catalog (asm pattern → C that produces it) L39 @@ -852,17 +878,21 @@ - **§176b** — BATCH-GATING MECHANICS (P31): what changes when N drafts land in ONE .c L16774 - **§3-C.** — The limit that remains (recorded, not solved) L17013 - **§176k** — TWO SELECTOR BUGS THAT SILENTLY SHRINK A WAVE L17139 -- **§178** — SIX LEVERS MINED FROM THE WAVE-P JOURNALS (P31 S52), each byte-proven and source-cited L17202 -- **§3-A.** — THE `$0`-ADD OPAQUE COPY defeats `make_regs_eqv` (func_80033398, 93 ins) L17214 -- **§3-C.** — SINGLE-SET TEMPS GET THE BIRTHING BOOST (func_8001D3FC, 196 ins) L17243 -- **§3-E.** — THE ZERO-OFFSET ALIAS HOLE (func_80037028, 71 ins) L17258 -- **§179-B** — 🔴 THE HAND-WRITTEN-ASM TRANSCRIPTION CHECKLIST (maspsx + masked_diff; 10 byte-proofs) L17348 -- **§179-C** — 🔴 A FUNCTION WITH NO EPILOGUE (falls into a sibling's shared tail) MUST BE FILE-SCOPE `__asm__` L17398 -- **§179-D** — `gte_stflg` MUST CLOBBER `"$12"` OR THE WHOLE TEMP FILE ROTATES BY ONE L17449 -- **§179-H** — A MID-BODY `.global LABEL` PAIR SLICES A BYTE-COMPARABLE FRAGMENT OUT OF A LARGER ROUTINE L17545 -- **Considered** — and NOT banked L17568 -- **§176-F** — Misdiagnosis triage: four residual verdicts that were lying L17740 -- **What** — is NOT banked here L17757 +- **§178** — SIX LEVERS MINED FROM THE WAVE-P JOURNALS (P31 S52), each byte-proven and source-cited L17213 +- **§3-A.** — THE `$0`-ADD OPAQUE COPY defeats `make_regs_eqv` (func_80033398, 93 ins) L17225 +- **§3-C.** — SINGLE-SET TEMPS GET THE BIRTHING BOOST (func_8001D3FC, 196 ins) L17254 +- **§3-E.** — THE ZERO-OFFSET ALIAS HOLE (func_80037028, 71 ins) L17269 +- **§179-B** — 🔴 THE HAND-WRITTEN-ASM TRANSCRIPTION CHECKLIST (maspsx + masked_diff; 10 byte-proofs) L17359 +- **§179-C** — 🔴 A FUNCTION WITH NO EPILOGUE (falls into a sibling's shared tail) MUST BE FILE-SCOPE `__asm__` L17409 +- **§179-D** — `gte_stflg` MUST CLOBBER `"$12"` OR THE WHOLE TEMP FILE ROTATES BY ONE L17460 +- **§179-H** — A MID-BODY `.global LABEL` PAIR SLICES A BYTE-COMPARABLE FRAGMENT OUT OF A LARGER ROUTINE L17556 +- **Considered** — and NOT banked L17579 +- **§176-F** — Misdiagnosis triage: four residual verdicts that were lying L17751 +- **What** — is NOT banked here L17768 +- **§180c** — WHEN A BINARY'S MASS BAND IS SPENT, THE FLEET-WIDE DRAW IS STRICTLY BETTER L17829 +- **Only** — ONE of 27 blocked drafts was wrong. The other 26 were correct and unbankable. L17848 +- **§180d** — THE `pgrep` BRACKET TRICK PROTECTS THE PATTERN, NOT THE COMMAND LINE L17917 +- **Three** — defects in one call path; the overlay slates that carry most of the wave work were being waved through L18315 ## All sections, in order @@ -1404,30 +1434,51 @@ - **§176j-2** — THE REPAIR PASS, MEASURED (do this instead of resuming) L17119 - **§176k** — TWO SELECTOR BUGS THAT SILENTLY SHRINK A WAVE L17139 - **§177** — 🔴 THE EPILOGUE RETURN-DELAY SLOT IS DECIDED BY YOUR SAVED-REGISTER SET, NOT BY SCHEDULING L17156 -- **§178** — SIX LEVERS MINED FROM THE WAVE-P JOURNALS (P31 S52), each byte-proven and source-cited L17202 -- **§3-A.** — THE `$0`-ADD OPAQUE COPY defeats `make_regs_eqv` (func_80033398, 93 ins) L17214 -- **§3-B.** — A `return ` IS A PRIORITY-1 HARD-REG SET THE SCHEDULER PLACES FIRST (func_8001BE30, 92 ins) L17230 -- **§3-C.** — SINGLE-SET TEMPS GET THE BIRTHING BOOST (func_8001D3FC, 196 ins) L17243 -- **§3-D.** — A NARROW TYPE BLOCKS COPY ELISION (func_8001D3FC — new idiom) L17251 -- **§3-E.** — THE ZERO-OFFSET ALIAS HOLE (func_80037028, 71 ins) L17258 -- **§3-F.** — `MEM_IN_STRUCT_P` ASYMMETRY IN `true_dependence` (func_80037144, 124 ins) L17267 -- **§3-G.** — TWO MODELLING TRAPS THAT COST THESE AGENTS SWEEPS OF HUNDREDS OF COMPILES L17273 -- **§179** — IDIOMS MINED FROM THE WAVE P/Q JOURNALS (P31 S52, harvest pass) L17289 -- **§179-A** — 🔴 A LOOP-WALKED POINTER **PARAMETER** HANDS ITS ARGUMENT REGISTER TO THE GIV (9 byte-proofs) L17299 -- **§179-B** — 🔴 THE HAND-WRITTEN-ASM TRANSCRIPTION CHECKLIST (maspsx + masked_diff; 10 byte-proofs) L17348 -- **§179-C** — 🔴 A FUNCTION WITH NO EPILOGUE (falls into a sibling's shared tail) MUST BE FILE-SCOPE `__asm__` L17398 -- **§179-D** — `gte_stflg` MUST CLOBBER `"$12"` OR THE WHOLE TEMP FILE ROTATES BY ONE L17449 -- **§179-E** — A `>2*MAX_MOVE_BYTES` BLOCK COPY IS A **STRUCT ASSIGNMENT**, NOT A HAND LOOP L17469 -- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17495 -- **§179-G** — 🟡 A PIN CAN **CREATE** A COMBINE `LOG_LINK` AND DELETE AN `andi` (sixth RC-5 channel, n=1) L17518 -- **§179-H** — A MID-BODY `.global LABEL` PAIR SLICES A BYTE-COMPARABLE FRAGMENT OUT OF A LARGER ROUTINE L17545 -- **Considered** — and NOT banked L17568 -- **§176c** — MAIN (SLUS_007.26) CANNOT BE GATED INCREMENTALLY L17585 -- **§176** — SEVEN LEVERS FROM THE P31 OVERNIGHT WAVES (2026-08-15): statement order, false regalloc, and the pin that fights back L17596 -- **§176-A** — "SCHEDULE / DELAY-SLOT / LENGTH-DRIFT ±1" ⇒ check STATEMENT ORDER around the call first L17602 -- **§176-B** — "REGALLOC-PERM, 1-4 instructions off" ⇒ it is usually NOT register allocation L17626 -- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17666 -- **§176-D** — CSE-class levers used in reverse (two sharpenings of §153 and cse_expr §2) L17692 -- **§176-E** — Two cheap source spellings, both cc1-probed L17718 -- **§176-F** — Misdiagnosis triage: four residual verdicts that were lying L17740 -- **What** — is NOT banked here L17757 +- **§178** — SIX LEVERS MINED FROM THE WAVE-P JOURNALS (P31 S52), each byte-proven and source-cited L17213 +- **§3-A.** — THE `$0`-ADD OPAQUE COPY defeats `make_regs_eqv` (func_80033398, 93 ins) L17225 +- **§3-B.** — A `return ` IS A PRIORITY-1 HARD-REG SET THE SCHEDULER PLACES FIRST (func_8001BE30, 92 ins) L17241 +- **§3-C.** — SINGLE-SET TEMPS GET THE BIRTHING BOOST (func_8001D3FC, 196 ins) L17254 +- **§3-D.** — A NARROW TYPE BLOCKS COPY ELISION (func_8001D3FC — new idiom) L17262 +- **§3-E.** — THE ZERO-OFFSET ALIAS HOLE (func_80037028, 71 ins) L17269 +- **§3-F.** — `MEM_IN_STRUCT_P` ASYMMETRY IN `true_dependence` (func_80037144, 124 ins) L17278 +- **§3-G.** — TWO MODELLING TRAPS THAT COST THESE AGENTS SWEEPS OF HUNDREDS OF COMPILES L17284 +- **§179** — IDIOMS MINED FROM THE WAVE P/Q JOURNALS (P31 S52, harvest pass) L17300 +- **§179-A** — 🔴 A LOOP-WALKED POINTER **PARAMETER** HANDS ITS ARGUMENT REGISTER TO THE GIV (9 byte-proofs) L17310 +- **§179-B** — 🔴 THE HAND-WRITTEN-ASM TRANSCRIPTION CHECKLIST (maspsx + masked_diff; 10 byte-proofs) L17359 +- **§179-C** — 🔴 A FUNCTION WITH NO EPILOGUE (falls into a sibling's shared tail) MUST BE FILE-SCOPE `__asm__` L17409 +- **§179-D** — `gte_stflg` MUST CLOBBER `"$12"` OR THE WHOLE TEMP FILE ROTATES BY ONE L17460 +- **§179-E** — A `>2*MAX_MOVE_BYTES` BLOCK COPY IS A **STRUCT ASSIGNMENT**, NOT A HAND LOOP L17480 +- **§179-F** — PINNING A LOOP-WALKED POINTER IS A TOTAL OFF-SWITCH FOR STRENGTH REDUCTION L17506 +- **§179-G** — 🟡 A PIN CAN **CREATE** A COMBINE `LOG_LINK` AND DELETE AN `andi` (sixth RC-5 channel, n=1) L17529 +- **§179-H** — A MID-BODY `.global LABEL` PAIR SLICES A BYTE-COMPARABLE FRAGMENT OUT OF A LARGER ROUTINE L17556 +- **Considered** — and NOT banked L17579 +- **§176c** — MAIN (SLUS_007.26) CANNOT BE GATED INCREMENTALLY L17596 +- **§176** — SEVEN LEVERS FROM THE P31 OVERNIGHT WAVES (2026-08-15): statement order, false regalloc, and the pin that fights back L17607 +- **§176-A** — "SCHEDULE / DELAY-SLOT / LENGTH-DRIFT ±1" ⇒ check STATEMENT ORDER around the call first L17613 +- **§176-B** — "REGALLOC-PERM, 1-4 instructions off" ⇒ it is usually NOT register allocation L17637 +- **§176-C** — 🔴 WALL REFUTATION: a hard-register pin CANNOT schedule around a call, because of a genuine gcc-2.7.2 bug L17677 +- **§176-D** — CSE-class levers used in reverse (two sharpenings of §153 and cse_expr §2) L17703 +- **§176-E** — Two cheap source spellings, both cc1-probed L17729 +- **§176-F** — Misdiagnosis triage: four residual verdicts that were lying L17751 +- **What** — is NOT banked here L17768 +- **§180** — THE LEFTOVER-DRAFT HARVEST: RE-VERIFY WHAT YOU ALREADY HAVE BEFORE DRAFTING ANYTHING NEW L17781 +- **§180b** — WHAT THE PRE-GATE LADDER ACTUALLY FINDS IN A COLD PILE (the shape of integration debt) L17810 +- **§180c** — WHEN A BINARY'S MASS BAND IS SPENT, THE FLEET-WIDE DRAW IS STRICTLY BETTER L17829 +- **§181** — WHAT A WAVE'S GATE ACTUALLY REJECTS (P31 S53, measured on wave R's 45-draft main pile) L17847 +- **Only** — ONE of 27 blocked drafts was wrong. The other 26 were correct and unbankable. L17848 +- **§182** — §177's HONEST NEGATIVE: the epilogue lever cracked 4 of 16, and the `800c3` cluster held L17903 +- **§180d** — THE `pgrep` BRACKET TRICK PROTECTS THE PATTERN, NOT THE COMMAND LINE L17917 +- **§183** — THE DECLARATION-RECONCILIATION PLAYBOOK (P31 S53, measured on 20 byte-verified drafts) L17929 +- **§3-18** — of 20 reconciled while keeping the match. The two that did not are mechanism, not effort. L17930 +- **§184** — COMMENT-BLINDNESS IS A DEFECT CLASS, NOT A BUG (P31 S53: three tools, one root cause, one session) L18000 +- **§185** — EDIT THE SIDE THAT IS CHEAP TO VERIFY, AND CHECK A TU RETYPE AT ITS USE SITES L18028 +- **§186** — CROSS-JUMPING RUNS **AFTER** SCHEDULING, SO NO C-LEVEL BARRIER CAN STEER IT L18060 +- **§186b** — A NO-SAVE 16-BYTE FRAME IN A LEAF FUNCTION MEANS `s16` LOCALS, NOT A HIDDEN CALL L18081 +- **§186c** — WHERE A VALUE IS LOADED DECIDES WHICH ALLOCATOR OWNS IT, AND THEREFORE ITS REGISTER L18089 +- **§187** — 🔴 "SAME SOURCE" IS NOT "SAME OBJECT": THE SDK BUILD AND THE GAME BUILD DISAGREE ON GTE NOPS L18106 +- **§188** — 🔴 THE `jr $ra` + `addiu $sp` TAIL IS AN **ASSEMBLER** ARTIFACT, NOT A FRAME SHAPE L18144 +- **§189** — FIVE COMPILER LAWS MINED FROM THE WAVE R/S JOURNALS (P31 S53), each source-cited and re-derived by a second agent L18194 +- **§190** — THREE PRESCRIPTIONS FROM THE SAME HARVEST (weaker evidence than §189, honestly labelled) L18255 +- **§191** — WHAT THIS HARVEST DID **NOT** BANK (4 rejected, 4 narrowed) — recorded so it is not re-derived L18292 +- **§192** — THE PRE-GATE LADDER WAS MAIN-ONLY, AND NOBODY COULD SEE IT (P31 S54) L18314 +- **Three** — defects in one call path; the overlay slates that carry most of the wave work were being waved through L18315 diff --git a/docs/family-cousins.md b/docs/family-cousins.md index 66afccdc4..aaac21125 100644 --- a/docs/family-cousins.md +++ b/docs/family-cousins.md @@ -1,6 +1,6 @@ # P30 S49 — cousin-unit survey (similarity tier over the open frontier) -> Generated by `tools/family_cousins.py` at HEAD `commit:2416` from `.run/family_hseq.json` + the binary sigs (R32-checked against an independent stub recount — a stale map fails loud). Compare digests only at the same HEAD. +> Generated by `tools/family_cousins.py` at HEAD `commit:2454` from `.run/family_hseq.json` + the binary sigs (R32-checked against an independent stub recount — a stale map fails loud). Compare digests only at the same HEAD. > > **A unit is a CANDIDATE grouping** — same-source cousins split by skeleton drift (insertion/deletion; the li-expansion class). Cousins need a per-member seeded CRACK, never a `family_sweep` remap. The whole-binary byte-gate stays the sole arbiter. @@ -8,11 +8,11 @@ | category | units | open fns | open ins | lever | |---|--:|--:|--:|---| -| A-prop | 395 | 2002 | 87143 | family_sweep (propagation) | -| seeded | 339 | 849 | 30050 | seeded crack — edit a >=0.85-similar MATCHED body | -| cousin-multi | 1349 | 4531 | 208991 | 1 crack seeds the unit's other members | -| cold | 3168 | 3168 | 211433 | full-price crack (the honest unique tail) | -| **total** | **5251** | **10550** | **537617** | | +| A-prop | 398 | 2013 | 87978 | family_sweep (propagation) | +| seeded | 339 | 845 | 30326 | seeded crack — edit a >=0.85-similar MATCHED body | +| cousin-multi | 1340 | 4503 | 206207 | 1 crack seeds the unit's other members | +| cold | 3069 | 3069 | 201371 | full-price crack (the honest unique tail) | +| **total** | **5146** | **10430** | **525882** | | ## Top units by open instructions (whole-cluster weight) @@ -46,15 +46,15 @@ | 26 | cousin-multi | 810 | 10 | 2 | · | · | `ov_SC01_005:0x8017e108` (84 ins ×5) | | 27 | A-prop | 798 | 3 | 1 | 1.0 | `ov_SC01_000:0x80130d48` | `ov_SC01_077:0x80130d48` (266 ins ×3) | | 28 | cousin-multi | 784 | 4 | 1 | · | · | `ov_SC03_108:0x8017ffd0` (196 ins ×4 jr) | -| 29 | cousin-multi | 767 | 15 | 7 | 0.776 | · | `ov_SC03_028:0x8018966c` (63 ins ×4) | +| 29 | cousin-multi | 767 | 15 | 7 | 0.839 | · | `ov_SC03_028:0x8018966c` (63 ins ×4) | | 30 | seeded | 752 | 14 | 10 | 0.921 | `ov_SC02_027:0x8017f6d4` | `ov_SC02_027:0x8017f858` (48 ins ×3) | -| 31 | cousin-multi | 696 | 10 | 3 | · | · | `ov_SC02_041:0x80181164` (72 ins ×4) | -| 32 | cousin-multi | 680 | 8 | 2 | 0.7 | · | `md_SC03_077:0x801efb58` (87 ins ×4 jr) | -| 33 | seeded | 673 | 6 | 4 | 0.921 | `ov_SC02_011:0x80185c7c` | `ov_SC03_118:0x80181e58` (113 ins ×2) | -| 34 | cousin-multi | 668 | 3 | 3 | · | · | `ov_SC02_017:0x80186770` (230 ins ×1) | -| 35 | cousin-multi | 658 | 14 | 5 | · | · | `ov_SC04_018:0x80181124` (52 ins ×4) | -| 36 | cold | 657 | 1 | 1 | · | · | `md_MAIN_003:0x800d12d0` (657 ins ×1) | -| 37 | A-prop | 656 | 29 | 10 | 1.0 | `ov_SC03_001:0x8017f0ec` | `ov_SC03_001:0x8017f0ec` (20 ins ×13) | -| 38 | A-prop | 655 | 16 | 6 | 1.0 | `ov_SC02_005:0x8017df78` | `ov_SC02_005:0x8017df78` (39 ins ×10) | -| 39 | cousin-multi | 648 | 6 | 1 | · | · | `ov_SC06_011:0x8017eeec` (108 ins ×6 jr) | -| 40 | A-prop | 647 | 16 | 6 | 1.0 | `ov_SC01_074:0x8017dba0` | `ov_SC01_077:0x8017d840` (39 ins ×9) | +| 31 | cousin-multi | 680 | 8 | 2 | 0.7 | · | `md_SC03_077:0x801efb58` (87 ins ×4 jr) | +| 32 | seeded | 673 | 6 | 4 | 0.921 | `ov_SC02_011:0x80185c7c` | `ov_SC03_118:0x80181e58` (113 ins ×2) | +| 33 | cousin-multi | 668 | 3 | 3 | · | · | `ov_SC02_017:0x80186770` (230 ins ×1) | +| 34 | cousin-multi | 658 | 14 | 5 | · | · | `ov_SC04_018:0x80181124` (52 ins ×4) | +| 35 | cold | 657 | 1 | 1 | · | · | `md_MAIN_003:0x800d12d0` (657 ins ×1) | +| 36 | A-prop | 656 | 29 | 10 | 1.0 | `ov_SC03_001:0x8017f0ec` | `ov_SC03_001:0x8017f0ec` (20 ins ×13) | +| 37 | A-prop | 655 | 16 | 6 | 1.0 | `ov_SC02_005:0x8017df78` | `ov_SC02_005:0x8017df78` (39 ins ×10) | +| 38 | cousin-multi | 648 | 6 | 1 | · | · | `ov_SC06_011:0x8017eeec` (108 ins ×6 jr) | +| 39 | A-prop | 647 | 16 | 6 | 1.0 | `ov_SC01_074:0x8017dba0` | `ov_SC01_077:0x8017d840` (39 ins ×9) | +| 40 | A-prop | 642 | 16 | 5 | 1.0 | `ov_SC01_000:0x8017d024` | `ov_SC01_000:0x8017d024` (38 ins ×8) | diff --git a/docs/family-hseq.md b/docs/family-hseq.md index db568dc78..d42b80e68 100644 --- a/docs/family-hseq.md +++ b/docs/family-hseq.md @@ -5,13 +5,13 @@ > Generated by `tools/family_hseq.py` from the 212 binary sigs + per-binary src stubs. Ranked by TEMPLATABLE byte-weight (PURE+IMM members × nins × 4). The byte-gate is the arbiter. > -> **Scope (212 binaries): 141 location overlays + 70 md_* modules + the resident · MAIN IS EXCLUDED** · generated at HEAD `commit:2416` · stub set derived from `corpus.stubs` — the same oracle `progress.py` counts, so same-tree overlay totals agree by construction; compare digests only at the same HEAD. +> **Scope (212 binaries): 141 location overlays + 70 md_* modules + the resident · MAIN IS EXCLUDED** · generated at HEAD `commit:2454` · stub set derived from `corpus.stubs` — the same oracle `progress.py` counts, so same-tree overlay totals agree by construction; compare digests only at the same HEAD. -**Fleet (overlays):** 97.1% fn / 96.0% instr / 91.6% distinct-code matched. Unmatched: 10,550 instances / 537,617 ins (9,165 distinct classes). +**Fleet (overlays):** 97.1% fn / 96.1% instr / 91.8% distinct-code matched. Unmatched: 10,430 instances / 525,882 ins (9,045 distinct classes). -**Tail cross-check (Phase-25 close):** 7,036 tail fns / 359,343 ins → 403 h_seq families ≥2, **69 substantial (nins≥80) / 30,270 ins**. +**Tail cross-check (Phase-25 close):** 6,922 tail fns / 348,199 ins → 403 h_seq families ≥2, **69 substantial (nins≥80) / 30,270 ins**. -**Full frontier (all unmatched by h_seq):** 1917 target families (≥2 members or a matched sibling) + 4378 singletons (Step-D residue). Substantial: **252 families / 88,123 templatable ins**, 68 with a matched sibling (zero-crack). Substantial member classes: 614 PURE · 21 IMM · 6 STRUCT-excluded. +**Full frontier (all unmatched by h_seq):** 1917 target families (≥2 members or a matched sibling) + 4262 singletons (Step-D residue). Substantial: **252 families / 87,970 templatable ins**, 69 with a matched sibling (zero-crack). Substantial member classes: 613 PURE · 21 IMM · 6 STRUCT-excluded. ## Top substantial families (by templatable byte-weight) diff --git a/docs/frontier-atlas.md b/docs/frontier-atlas.md index eae7ed9df..ce17023bb 100644 --- a/docs/frontier-atlas.md +++ b/docs/frontier-atlas.md @@ -1,44 +1,44 @@ # The Frontier Atlas (P31 T5) -> Generated at HEAD `commit:2416` by `tools/atlas.py` — REGENERATE, never edit. Groups are ADVISORY (the byte-gate is the arbiter). Calibration: warm merge at norm-ratio ≥ 0.7, kNN floor 0.55. +> Generated at HEAD `commit:2454` by `tools/atlas.py` — REGENERATE, never edit. Groups are ADVISORY (the byte-gate is the arbiter). Calibration: warm merge at norm-ratio ≥ 0.7, kNN floor 0.55. -**Scope:** 11352 open instances / 6911 skeletons across 213 binaries (main: 802 — atlas-layer join (family maps stay non-main)). -**Tiers:** T1.5 h_seqn merges 2 · warm merges 880 (from 19345 ratio calls) · seed sweep: 4714 skeletons carry a ≥0.55 seed from the 3078-skeleton matched pool. +**Scope:** 11183 open instances / 6747 skeletons across 213 binaries (main: 753 — atlas-layer join (family maps stay non-main)). +**Tiers:** T1.5 h_seqn merges 2 · warm merges 876 (from 18404 ratio calls) · seed sweep: 4686 skeletons carry a ≥0.55 seed from the 3197-skeleton matched pool. ## Groups by category | cat | groups | instances | ins | |---|--:|--:|--:| -| A-prop | 394 | 2034 | 87416 | -| cold | 2244 | 2244 | 181913 | -| cousin-multi | 1551 | 5700 | 247842 | -| main-only | 358 | 358 | 31486 | -| seeded | 339 | 851 | 30074 | -| tiny | 99 | 165 | 840 | +| A-prop | 397 | 2045 | 88251 | +| cold | 2149 | 2149 | 172422 | +| cousin-multi | 1539 | 5664 | 244413 | +| main-only | 315 | 315 | 27653 | +| seeded | 339 | 847 | 30350 | +| tiny | 97 | 163 | 827 | ## Groups by lever (confidence) | lever | groups | instances | ins | |---|--:|--:|--:| -| head-crack[default] | 1085 | 3492 | 147552 | -| UNKNOWN[none] | 1882 | 1882 | 130091 | +| head-crack[default] | 1074 | 3458 | 144362 | +| UNKNOWN[none] | 1738 | 1738 | 116262 | | extend-tell[tell] | 579 | 1035 | 77588 | | jtbl-carve[tell] | 190 | 281 | 45673 | -| redraft[measured] | 268 | 1046 | 44562 | -| family-sweep[default] | 201 | 830 | 28324 | -| integration[measured] | 47 | 492 | 21675 | -| seeded-crack[default] | 257 | 572 | 17235 | +| redraft[measured] | 260 | 1037 | 43554 | +| family-sweep[default] | 204 | 841 | 29159 | +| integration[measured] | 47 | 494 | 21707 | +| seeded-crack[default] | 257 | 568 | 17511 | | len-vein[measured] | 148 | 666 | 14310 | -| plumbing[ledger] | 43 | 196 | 9956 | +| plumbing[ledger] | 55 | 211 | 11228 | | swaprepeat-tell[tell] | 70 | 119 | 9316 | | o0-lane[tell] | 31 | 69 | 6564 | | cc1[measured] | 12 | 104 | 6382 | -| needs-autopsy[measured] | 52 | 265 | 5110 | -| cc1[ledger] | 13 | 63 | 4506 | +| needs-autopsy[measured] | 52 | 263 | 5090 | +| cc1[ledger] | 17 | 67 | 4845 | | frame-172[measured] | 8 | 32 | 4078 | | needs-autopsy[ledger] | 9 | 33 | 3914 | -| near-crack[measured] | 11 | 62 | 1765 | -| tiny-direct[default] | 74 | 106 | 536 | +| near-crack[measured] | 8 | 56 | 1416 | +| tiny-direct[default] | 72 | 104 | 523 | | s16-div-tell[tell] | 4 | 6 | 419 | | signload[measured] | 1 | 1 | 15 | @@ -46,7 +46,7 @@ | gid | cat | lever | inst | ins | exemplar | seed | |---|---|---|--:|--:|---|---| -| 57f9bec16c2a | cousin-multi | integration[measured] | 254 | 5864 | ov_SC01_084:func_80180CBC (31) | 0.72 | +| 9607a77b73fe | cousin-multi | integration[measured] | 256 | 5896 | ov_SC01_084:func_80180CBC (31) | 0.72 | | e8bf2260391b | cousin-multi | frame-172[measured] | 16 | 3936 | ov_SC01_077:func_8017C294 (246) | | | 14bc91cb76b0 | cousin-multi | redraft[measured] | 112 | 2879 | main:func_8001FB8C (31) | 0.72 | | 4a254010e122 | A-prop | needs-autopsy[ledger] | 4 | 2444 | ov_SC03_001:func_801898E4 (611) | 1.00 | @@ -82,7 +82,7 @@ | 78627a0eec5d | cousin-multi | redraft[measured] | 10 | 810 | ov_SC01_004:func_8017D964 (84) | 0.62 | | b1509c2e091c | A-prop | integration[measured] | 3 | 798 | ov_MAIN_012:func_80130D48 (266) | 1.00 | | 97e813dd2393 | cousin-multi | integration[measured] | 4 | 784 | ov_SC03_108:func_8017FFD0 (196) | 0.65 | -| dcc70a8601c4 | cousin-multi | head-crack[default] | 15 | 767 | ov_SC03_028:func_8018966C (63) | 0.67 | +| dcc70a8601c4 | cousin-multi | head-crack[default] | 15 | 767 | ov_SC03_028:func_8018966C (63) | 0.74 | | 334bc03beb1f | cousin-multi | integration[measured] | 68 | 760 | md_SC07_003:func_801A38A8 (15) | 0.90 | | ab1569f4386c | seeded | redraft[measured] | 14 | 752 | ov_SC03_096:func_8017D130 (66) | 0.83 | | 101b914c4520 | cousin-multi | extend-tell[tell] | 22 | 721 | ov_SC03_092:func_8017E6A0 (45) | 0.75 | diff --git a/docs/matching-cookbook.md b/docs/matching-cookbook.md index 4269c7d3b..697acc819 100644 --- a/docs/matching-cookbook.md +++ b/docs/matching-cookbook.md @@ -18308,3 +18308,68 @@ banked). artifacts. Every confirmed entry above changed shape under verification — bounds added, a mechanism re-attributed, a sub-claim refuted — and the ones that did not survive were rejected for exactly the reasons §179 predicted: not banked, or banked C contradicting the narrative written about it. + +--- + +## §192 — THE PRE-GATE LADDER WAS MAIN-ONLY, AND NOBODY COULD SEE IT (P31 S54) +### Three defects in one call path; the overlay slates that carry most of the wave work were being waved through + +**The symptom.** Running `pregate_check.py` on an *overlay* slate printed: + +``` +slate 7 -> 7 after resolve_conflicts (0 dropped); checking 0 substituted file(s) + clean — no textual defect found; the batch is worth a rebuild +``` + +Seven byte-perfect drafts, a green light, and **zero files examined**. Every wave since O has been +overlay work; the check that §181 credits with catching 4 of the 5 rejection classes has never once +run on any of it. + +**DEFECT 1 — `gate_main.resolve_conflicts` and `gate_main.substitute` both hardcoded +`corpus.stubs('main')`.** A non-main entry resolved to no stub, so `resolve_conflicts` compared it +against a `''` pseudo-file (never clashing with anything) and `substitute` dropped it on the +floor without a word. This is the R36 citizenship class again — a consumer structurally blind to a +real binary — and the slate shape already carried the answer (`gate_lane` records have a `binary` +field). Fixed with a memoized `_stubs_for(binary)`; absent, the key defaults to `main`, so every +historical main slate behaves byte-identically. + +**DEFECT 2 — `sym_of` returned a KEYWORD for any pointer-to-function declaration.** The regex took +the first identifier followed by `[` or `(`, *by position*, so + +```c +extern void (*D_801923D0[])(void *); +``` + +reported its symbol as `void` — and then every such declaration in the TU "conflicted" with every +other one under that name: **192 phantom CONFLICTING-EXTERN failures on one overlay TU**. Project +symbols are now matched by NAME first, C keywords are excluded from the generic branch, and the +parenthesised declarator `T (*NAME[])(...)` is read explicitly. Negative control across the whole +tree: **5,526,100 declarations, 189,301 changed verdicts, 0 regressions** — every single change is a +keyword becoming the real symbol (`void` → `D_800A4F24`, `s32` → `D_801274D0`), which also means +`gate_main`'s own conflict table has been blind to function-pointer globals in **main** all along. + +**DEFECT 3 — `void f()` and `void f(void)` were normalized to the same thing.** C89 6.5.4.3 makes an +*unspecified* parameter list compatible with any prototype (this project leans on it — §37/§124), but +an *explicit* `(void)` against `f(s32)` is a hard error. Collapsing both to `()` cost 40 more phantom +failures against a TU that compiles today. Now: unspecified → a `UNSPEC` wildcard, explicit `(void)` → +`()`, and the comparison is `gate_main.sig_conflict`, not `!=`. Seven synthetic controls, including +both directions of the pair that matters. + +**The residue is the point.** After the three fixes the same overlay slate reports **2 failures**, and +both are real: a `D_80195AF6_t` typedef defined twice, and `memcpy` declared `(void*, const void*, +u32)` in the TU against a draft's `(void*, void*, s32)`. Each of those is one lost clean rebuild — +which is the whole reason the ladder exists. + +**§192b — A CHECKER THAT CHECKED NOTHING MUST NEVER READ AS A PASS.** Two guards, both earned above: +`pregate_check` now REFUSES (exit 2) when the slate is non-empty and zero files were substituted, and +it prints one `[DROP]` line per draft `resolve_conflicts` rejected — with the clashing symbol, the +side it clashed with, and both signatures. Those drops were already being computed and thrown away, so +a slate that lost **every** draft to declaration conflicts still ended with "clean — the batch is worth +a rebuild". The drop list is not a footnote: it is the §183 playbook's worklist, and it is now the +tool's headline output. + +**The generalizable law.** *A tool written against one binary is a tool with an untested hypothesis +about every other one.* The main-vs-overlay split here is exactly the shape of R36's SC07 bug and +R32's `build_engine_types` bug: the code did not fail, it silently narrowed its own domain and then +reported success over the part it kept. When a checker's verdict is "clean", ask what it counted — and +make the tool answer that question in its own output. diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index 1fab0c36c..5d430a630 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -57,6 +57,47 @@ R22 clean-fleet **213/213** after every banked batch · tools-health green · 0 - 2026-08-14 — **T9 COMPLETE.** `tools/warmstart.py` (the permuter/grinder FEEDER): `--from-banked` walks a banked exemplar's h_seq family's still-open members, builds remapped proven-body drafts (`symbol_map` + `aprop_autodraft.build_draft`, refusing on reloc-count mismatch), **stream-classifies member-vs-seed with ZERO compiles** (masked_diff-shaped dicts from ground-truth bytes → `residual_class.classify_streams`), enqueues ONLY permuter-shaped (bucket==permuter or LENGTH-DRIFT |Δ|≤2) as backlog near-records; `--lenmiss` ingests T8's 49-route. **Grinder patch NOT needed** (its candidates() deliberately keeps unclassified records — "unknown is not a reason to skip" — so pre-filtered enqueues flow as-is; documented in the feeder's docstring). Armed live: **49 + 10 enqueued, 120 refused** by the stream filter (the anti-92%-wasted-CPU discipline working). `family_cousins --weak-cards`: **954 units** (the 0.70–0.85 annotate-only band, never before consumed) as seeded-crack cards, ins-ranked, §168 laws embedded, model-routed **haiku 804 / v3 43 / sonnet 86 / opus 21** (cheap tiers dominate — the token-efficiency shape), 0 unresolved `.s`. +## SESSION S54 (2026-08-17, ultracode) — wave T + the pre-gate ladder learns to see overlays + +- 2026-08-17 — **Session opened with the §180 leftover sweep, before drafting anything new.** + `scan_leftovers.py` over 644 stored drafts in 5 binaries: **29 byte-perfect, still-stubbed drafts = + 2,478 instructions** (main 3/217 · ov_SC04_011 15/1,392 · ov_SC02_005 5/261 · ov_SC06_029 5/521 · + ov_SC03_028 1/87), plus 39 honest NEAR. Zero agent tokens. `reloc_identity --batch`: **29/29 AGREE**. + `fragment_check`: 1 FAIL — `gfx2D_BG0_OBJ_4D8` owns `.L80050D5C`, which `gfx2D_BG0_OBJ_1B4` branches + into (the §181 mirror class; it is also a `GsSortBg` fragment-merge target, so it routes there). + +- 2026-08-17 — **Wave T built and launched (71 cards / 6,557 ins / 5 gate groups / 14.2 drafts per + rebuild), and the selector learned two new principles.** `make atlas` first (it predated 161 banks): + 4,836 groups / 11,183 open instances / 563,916 ins. Then two additions to `build_wave_atlas.py`: + * **`--one-per-gid`** — a fleet-wide draw over SIBLING overlays fills half a wave with the SAME + skeleton at two addresses (the naive draw put 22 of 44 groups in twice), paying an agent for work + `family_sweep --hseq` does free behind the banked exemplar. Collapse to one card per atlas group, + defer the siblings to `.siblings.json`, assert reps + siblings == candidates (R32). + * **`--rank total`** — rank gate groups by the mass a card DELIVERS (its own instructions plus its + deferred siblings'), not by face mass. Measured on this draw: face 6,509 ins with 9,985 sibling ins + behind 57 gids under `--rank mass`, versus **6,557 ins with 12,709 sibling ins behind 69 of 71 + gids** under `--rank total`. **Wave T's potential is 19,266 instructions for 71 agents.** + R39 negative control: on a pool with unique gids, `--one-per-gid` output is byte-identical. + The wave prompt carries the S53 harvest as laws 8-20 (§183.1/§183.3, §186, §186b, §186c, §189-A..E, + §190-A..C), STEP 0 (the cross-overlay magic-literal grep), a per-card sibling note, and a new + **Reconcile phase**: one agent per gate group runs the deterministic ladder over its own slate and + resolves the refusals with the §183 playbook — reconciliation moved INSIDE the wave (§176h.C2). + +- 2026-08-17 — **§192: the pre-gate ladder was main-only and said "clean" about it.** Running + `pregate_check` on an overlay slate printed `checking 0 substituted file(s) ... clean`. Three + defects, all fixed with controls: (1) `resolve_conflicts`/`substitute` hardcoded + `corpus.stubs('main')` → per-binary `_stubs_for()` (main path byte-identical); (2) `sym_of` returned + `void` for every `extern void (*D_x[])(...)` → 192 phantom CONFLICTING-EXTERNs; project symbols now + matched by name, keywords excluded, declarator read explicitly — **NC over 5,526,100 declarations: + 189,301 changed verdicts, 0 regressions**; (3) `void f()` vs `void f(void)` were collapsed together → + 40 more phantoms; C89's unspecified-parameter rule is now `gate_main.sig_conflict` (7 synthetic + controls). Same slate now reports **2 failures, both real**. Plus **§192b**: the tool REFUSES when it + substituted 0 files, and prints one `[DROP]` line per rejected draft — the drop reasons were being + computed and discarded while the tool printed "worth a rebuild". + +- 2026-08-17 — **Leftover-recovery lane launched** (5 agents, one per binary): each gets its slate, its + TU, and the deterministic `[DROP]`/refusal evidence, and applies the §183 playbook draft-side only. + ## SESSION S53 (2026-08-16, ultracode) — wave R + the leftover-draft harvest - 2026-08-16 — **S53-1 PREFLIGHT + ATLAS REGEN.** Tree clean at `commit:2416`, no gate in flight. `make atlas` diff --git a/tools/build_wave_atlas.py b/tools/build_wave_atlas.py index 38a73c686..d14143273 100644 --- a/tools/build_wave_atlas.py +++ b/tools/build_wave_atlas.py @@ -41,7 +41,7 @@ ap.add_argument('--exclude-bins', default='', 'INCREMENTALLY, because its extract runs psyq_integrate/ld_interleave and ' 'rewrites the .ld. Draft main like any binary; gate it with ' 'tools/gate_main.py, never gate_lane/gate_stage.)') -ap.add_argument('--rank', choices=('groups','mass'), default='groups', +ap.add_argument('--rank', choices=('groups','mass','total'), default='groups', help="'groups' (default) ranks gate groups by MEMBER COUNT -- right for overlays, " "where every (binary,TU) group costs its own rebuild. 'mass' ranks purely by " "instruction size across all groups -- right for MAIN, whose gate cost is per " @@ -58,6 +58,17 @@ ap.add_argument('--only-bins', default='', help='comma-separated allow-list; if set, ONLY these binaries are eligible. ' 'Use --only-bins main for a main wave: gate_main.py rebuilds the whole EXE ' 'once per SLATE, so main has no per-TU gate cost and --max-bins can be large.') +ap.add_argument('--one-per-gid', action='store_true', + help="draft ONE card per atlas group and defer its same-gid siblings to " + ".siblings.json for the post-bank mechanical remap " + "(make sig-overlays -> family_hseq.py -> family_sweep --hseq --only). " + "Rationale (P31 S54): a fleet-wide draw over SIBLING overlays (ov_SC04_002 vs " + "ov_SC04_005) fills half the wave with the SAME skeleton at two addresses -- " + "paying an agent twice for work the deterministic remap does for free. The " + "representative is the sibling in the heaviest gate group (then largest nins, " + "then lexical fn) so concentration is unharmed. Every deferred sibling is " + "written to the companion file and accounted (R32): representatives + " + "siblings == candidates, asserted.") a = ap.parse_args() EXCLUDE = {b for b in a.exclude_bins.split(',') if b} ONLY = {b for b in a.only_bins.split(',') if b} @@ -132,13 +143,51 @@ for g in atlas['groups']: 'seed_sim': seed.get('sim'), }) +# principle 4 (P31 S54): ONE CARD PER ATLAS GROUP. Same-gid members are the SAME skeleton in +# different overlays; the deterministic remap (family_sweep --hseq) banks the siblings behind a +# banked exemplar for zero tokens, so drafting both is paying twice. Collapse here, BEFORE the +# gate-group ranking, so the ranking sees distinct work; defer the rest to .siblings.json. +siblings = collections.defaultdict(list) +if a.one_per_gid: + _mass = collections.Counter() + for c in cands: + _mass[(c['binary'], c['tu'])] += c['nins'] + keep = {} + for c in cands: + cur = keep.get(c['gid']) + rank = (_mass[(c['binary'], c['tu'])], c['nins'], c['fn']) + if cur is None or rank > cur[0]: + if cur is not None: + siblings[c['gid']].append(cur[1]) + keep[c['gid']] = (rank, c) + else: + siblings[c['gid']].append(c) + reps = [v[1] for v in keep.values()] + n_sib = sum(len(v) for v in siblings.values()) + assert len(reps) + n_sib == len(cands), \ + f"coverage (R32): {len(reps)} reps + {n_sib} siblings != {len(cands)} candidates" + print(f"--one-per-gid: {len(cands)} candidates -> {len(reps)} groups " + f"({n_sib} same-gid siblings deferred to the mechanical remap)") + cands = reps + # principle 1: CONCENTRATE ON GATE GROUPS. gate_lane groups by (binary, home .c) and each group # is one whole-binary rebuild, so drafts-per-GROUP is the throughput number that matters -- not # drafts per binary. Wave D was 42 drafts over 23 groups (1.8/group, ~40 min of gate). by_tu = collections.defaultdict(list) for c in cands: by_tu[(c['binary'], c['tu'])].append(c) -if a.rank == 'mass': +if a.rank == 'total': + # P31 S54: rank by the mass a card actually DELIVERS -- its own instructions plus the same-gid + # siblings the post-bank remap banks for free. Measured on the wave-T draw: the 70 selected + # cards carried 9,985 sibling instructions, 1.5x the wave's own 6,509, and that leverage is + # very unevenly spread across gate groups (some carry 3 siblings per card, some carry none). + # Ranking by face mass is therefore ranking by the smaller half of the number. + if not a.one_per_gid: + sys.exit("--rank total requires --one-per-gid (there are no deferred siblings otherwise)") + _sibins = {g: sum(c['nins'] for c in v) for g, v in siblings.items()} + ranked = sorted(by_tu, key=lambda k: -sum(c['nins'] + _sibins.get(c['gid'], 0) + for c in by_tu[k]))[:a.max_bins] +elif a.rank == 'mass': ranked = sorted(by_tu, key=lambda k: -sum(c['nins'] for c in by_tu[k]))[:a.max_bins] else: ranked = sorted(by_tu, key=lambda k: -len(by_tu[k]))[:a.max_bins] @@ -153,8 +202,10 @@ def _full(): if a.target_ins: return tot_ins >= a.target_ins or len(wave) >= a.n return len(wave) >= a.n +_deliver = (lambda c: c['nins'] + sum(s['nins'] for s in siblings.get(c['gid'], ()))) \ + if a.rank == 'total' else (lambda c: c['nins']) for k in ranked: # principle 2: within a group, mass first - for c in sorted(by_tu[k], key=lambda c: -c['nins']): + for c in sorted(by_tu[k], key=lambda c: -_deliver(c)): if _full(): break wave.append(c); tot_ins += c['nins'] if _full(): break @@ -163,6 +214,17 @@ if a.target_ins and tot_ins < a.target_ins: f"widen --min-ins/--max-ins/--levers or raise n ({len(wave)} of max {a.n} cards used)") json.dump(wave, open(a.out, 'w'), indent=1) +if a.one_per_gid: + # Only the siblings of gids that ACTUALLY made the wave are actionable this session; the rest + # stay in the atlas for a later draw. Both counts are printed so nothing is silently dropped. + in_wave = {c['gid'] for c in wave} + sib_out = {g: v for g, v in siblings.items() if g in in_wave} + sib_path = a.out.replace('.json', '') + '.siblings.json' + json.dump(sib_out, open(sib_path, 'w'), indent=1) + n_act = sum(len(v) for v in sib_out.values()) + print(f"-> {n_act} siblings ({sum(c['nins'] for v in sib_out.values() for c in v)} ins) behind " + f"{len(sib_out)} of this wave's gids -> {sib_path} (remap after the bank); " + f"{sum(len(v) for v in siblings.values()) - n_act} more sit behind un-drawn gids") tot = sum(c['nins'] for c in wave) print(f"candidates {len(cands)} in {len(by_tu)} gate groups (skipped {dict(skipped)})") ngroups = len({(c['binary'], c['tu']) for c in wave}) diff --git a/tools/gate_main.py b/tools/gate_main.py index 1ef9518ec..8beb0883d 100644 --- a/tools/gate_main.py +++ b/tools/gate_main.py @@ -53,9 +53,28 @@ TYPES = {'void','char','short','int','long','unsigned','signed','float','double' DECL = re.compile(r'^\s*extern\s+([^;]+?)\s*;[ \t]*(?://[^\n]*|/\*(?:[^*]|\*(?!/))*\*/[ \t]*)?$', re.M) +# A DECLARATOR KEYWORD IS NEVER THE SYMBOL (P31 S54). The old single regex took the FIRST +# identifier followed by '[' or '(', wherever it sat -- so a pointer-to-function declaration, +# extern void (*D_801923D0[])(void *); +# reported its symbol as `void`, because `void (` matches first by POSITION. Every such +# declaration in a TU then "collided" with every other one under the name `void`: 192 phantom +# CONFLICTING-EXTERN failures on one overlay TU the first time pregate_check could see overlays. +# Project symbols are matched by NAME first, keywords are excluded from the generic branch, and +# the parenthesised declarator is read explicitly. +_NOT_A_SYMBOL = { + 'void', 'char', 'short', 'int', 'long', 'float', 'double', 'signed', 'unsigned', + 'const', 'volatile', 'struct', 'union', 'enum', 'static', 'extern', 'register', 'typedef', + 's8', 'u8', 's16', 'u16', 's32', 'u32', 's64', 'u64', 'f32', 'f64', +} + + def sym_of(d): - m = re.search(r'\b(D_[0-9A-Fa-f]{8}|func_[0-9A-Fa-f]{8}|[A-Za-z_]\w*)\s*(?:\[|\()', d) + m = re.search(r'\b(D_[0-9A-Fa-f]{8}|func_[0-9A-Fa-f]{8})\s*(?:\[|\()', d) if m: return m.group(1) + m = re.search(r'\b([A-Za-z_]\w*)\s*(?:\[|\()', d) + if m and m.group(1) not in _NOT_A_SYMBOL: return m.group(1) + m = re.search(r'\(\s*\*+\s*([A-Za-z_]\w*)', d) # T (*NAME)(...) / T (*NAME[])(...) + if m and m.group(1) not in _NOT_A_SYMBOL: return m.group(1) m = re.search(r'\b(D_[0-9A-Fa-f]{8}|func_[0-9A-Fa-f]{8})\b', d) return m.group(1) if m else None @@ -110,6 +129,43 @@ def typesig(d): for p in m.group(1).split(',')) return (ret, params) +UNSPEC = '?' # an UNSPECIFIED parameter list -- `void f();`, not `void f(void);` + + +def norm_sig(sig): + """Normalize a `typesig` for COMPATIBILITY comparison, keeping the one distinction C89 makes. + + `extern void f();` declares an UNSPECIFIED parameter list, and C89 6.5.4.3 forms a composite + type with any prototype whose parameters survive the default promotions -- gcc-2.7.2 accepts + the pair, and this project's TUs are full of it (§37/§124, the no-prototype escape). But + `extern void f(void);` declares EXACTLY ZERO parameters, and that against `f(s32)` is a hard + error. Collapsing both to () -- as pregate_check did until P31 S54 -- makes the tool report 40 + phantom failures on one overlay TU that compiles today, while still missing nothing. + + So: unspecified -> UNSPEC (a wildcard for `sig_conflict`), explicit (void) -> ().""" + ret, params = sig + if isinstance(params, tuple): + if params in ((), ('',)): + params = UNSPEC + elif params == ('void',): + params = () + return (ret, params) + + +def sig_conflict(a, b): + """True when two normalized signatures cannot both describe one symbol. + + Return types must agree (S53: even G3P* vs G4P* was rejected by gcc). Parameter lists conflict + only when BOTH are specified and differ.""" + a, b = norm_sig(a), norm_sig(b) + if a[0] != b[0]: + return True + pa, pb = a[1], b[1] + if pa == UNSPEC or pb == UNSPEC: + return False + return pa != pb + + def run(cmd, **kw): return subprocess.run(cmd, shell=True, capture_output=True, text=True, **kw) @@ -117,6 +173,28 @@ def sha(): r = run("sha1sum build/us/SLUS_007.26") return r.stdout.split()[0] if r.returncode == 0 and r.stdout else None +_STUBS_BY_BIN = {} + + +def _stubs_for(binary): + """symbol -> Stub for ONE binary, memoized. + + WHY THIS EXISTS (P31 S54, R36/R33). `resolve_conflicts` and `substitute` both hardcoded + `corpus.stubs('main')`, so every non-main entry resolved to no stub: `resolve_conflicts` compared + it against the '' pseudo-file and `substitute` skipped it silently. The visible symptom + was `pregate_check` printing "checking 0 substituted file(s) ... clean" for an OVERLAY slate -- + a green light from a checker that had examined nothing, which is the exact R32 defect class this + project keeps re-finding. Overlay slates are the majority of wave work (every wave since O), and + 4 of the 5 rejection classes measured in §181 are precisely what these two functions detect. + + The slate shape already carries the answer: gate_lane's records have a 'binary' field. Absent + (every historical main slate), it defaults to 'main', so the main path is byte-identical.""" + b = binary or 'main' + if b not in _STUBS_BY_BIN: + _STUBS_BY_BIN[b] = {st.symbol: st for st in corpus.stubs(b).values()} + return _STUBS_BY_BIN[b] + + def resolve_conflicts(slate): """Drop drafts whose externs contradict (a) the destination TU's OWN existing declarations, or (b) an earlier draft landing in the SAME file. @@ -137,7 +215,6 @@ def resolve_conflicts(slate): and cast at the use site -- including through a function pointer when the TU's prototype takes no argument and your call passes one: ((void (*)(s32))func_8001C9D0)(a0) (byte-identical; verified on all 3 of the above).""" - stubs = {st.symbol: st for st in corpus.stubs('main').values()} kept, dropped = [], [] seen_by_file, from_tu = {}, {} @@ -155,7 +232,7 @@ def resolve_conflicts(slate): return seen_by_file[path] for e in slate: - st = stubs.get(e['fn']) + st = _stubs_for(e.get('binary')).get(e['fn']) path = st.path if st else '' seen = table(path) body = open(e['draft']).read() @@ -169,7 +246,7 @@ def resolve_conflicts(slate): body, re.M) if dm: ds.append((e['fn'], typesig('%s %s(%s)' % (dm.group(1).strip(), e['fn'], dm.group(2))))) - clash = [(s, seen[s], t) for s, t in ds if s in seen and seen[s] != t] + clash = [(s, seen[s], t) for s, t in ds if s in seen and sig_conflict(seen[s], t)] if clash: sym = clash[0][0] dropped.append({'fn': e['fn'], 'symbol': sym, 'file': path, @@ -328,12 +405,21 @@ def substitute(entries, write=True): write=False produces the substituted text WITHOUT touching the tree, which is what tools/pregate_check.py needs: every batch failure this project has hit is a textual property of the file that will be compiled, so it can be checked in ~2s instead of a 5-minute rebuild. - Returns (count, {path: text}).""" - stubs = {st.symbol: st for st in corpus.stubs('main').values()} + Returns (count, {path: text}). + + Per-binary since P31 S54 (see `_stubs_for`): an entry's own 'binary' selects its stub map, and + an entry whose symbol is in NO stub map is reported loudly instead of being dropped on the floor + (R32 -- a silent skip here is what made pregate_check green on overlay slates).""" byfile = collections.defaultdict(list) + unresolved = [] for e in entries: - st = stubs.get(e['fn']) + st = _stubs_for(e.get('binary')).get(e['fn']) if st: byfile[st.path].append((st.addr, e['fn'], st.asm_dir, e['draft'])) + else: unresolved.append((e.get('binary') or 'main', e['fn'])) + if unresolved: + print(" substitute: %d entr%s resolved to NO stub (not open in that binary?): %s" + % (len(unresolved), 'y' if len(unresolved) == 1 else 'ies', + ', '.join('%s:%s' % u for u in unresolved[:8]))) n, texts = 0, {} for path, items in byfile.items(): t = open(path).read() diff --git a/tools/pregate_check.py b/tools/pregate_check.py index a1dba1e34..7e511e103 100644 --- a/tools/pregate_check.py +++ b/tools/pregate_check.py @@ -68,13 +68,15 @@ STRUCT_EXTERN = re.compile( def _norm_sig(sig): - """`void f()` and `void f(void)` are not a conflict worth blocking a rebuild over: C89 calls - the first an unspecified parameter list, and gcc-2.7.2 accepts the pair. Normalize both to - (). Signedness, pointer depth and array-vs-scalar are left alone -- those are real.""" - ret, params = sig - if isinstance(params, tuple) and params in ((), ('',), ('void',)): - params = () - return (ret, params) + """Delegates to the single normalization oracle in gate_main (R33). + + It used to collapse `void f()` AND `void f(void)` to the same (), which is right about the + first (C89's unspecified parameter list, compatible with any prototype) and wrong about the + second (exactly zero parameters, incompatible with `f(s32)`). Since this tool could only ever + see main's TUs, the difference never showed; the first overlay slate it ran on produced 40 + phantom CONFLICTING-EXTERN failures against a file that compiles today. Use `gm.sig_conflict` + for comparisons -- `!=` on these tuples is not the compatibility relation.""" + return gm.norm_sig(sig) def _typedefs(text): @@ -180,7 +182,7 @@ def check_text(path, text): if not s: continue sig = _norm_sig(gm.typesig(d)) - if s in decls and decls[s][1] != sig: + if s in decls and gm.sig_conflict(decls[s][1], sig): findings.append(('FAIL', 'CONFLICTING-EXTERN', f'{path}: `{s}` declared {decls[s][1]} at offset {decls[s][0]} and ' f'{sig} at offset {m.start()}')) @@ -190,7 +192,7 @@ def check_text(path, text): # 5. definition vs a visible prototype for name, (off, sig) in _func_defs(masked).items(): # masked, not raw — see the note above sig = _norm_sig(sig) - if name in decls and decls[name][1] != sig: + if name in decls and gm.sig_conflict(decls[name][1], sig): # SEVERITY CALIBRATED AGAINST THE COMPILER, not against C89 pedantry. Measured on the # wave-P slate that built BYTE-IDENTICAL: gcc-2.7.2 accepted `void f(void*,s32)` vs a # `void f(s8*,s32)` definition, and even `G3P *f(...)` vs `G4P *f(...)`. What it @@ -222,6 +224,16 @@ def main(): kept, dropped = gm.resolve_conflicts(slate) _n, texts = gm.substitute(kept, write=False) + # R32 COVERAGE ASSERTION (P31 S54). Until `gate_main` learned per-binary stub maps, an OVERLAY + # slate resolved to zero stubs and this tool printed "checking 0 substituted file(s) ... clean" + # -- a green light from a checker that had examined nothing, on exactly the slates (overlay + # waves) that carry most of the work. A checker that checked nothing must never read as a pass. + if kept and not texts: + print(f'REFUSING: {len(kept)} kept draft(s) but 0 substituted files — every entry resolved ' + f'to no stub. Does each slate record carry its "binary"? Is the binary extracted ' + f'(make extract BINARY=...)? This is not a clean result.') + sys.exit(2) + findings = [] for path, text in sorted(texts.items()): findings += check_text(path, text) @@ -230,15 +242,25 @@ def main(): if not a.quiet: print(f'slate {len(slate)} -> {len(kept)} after resolve_conflicts ' f'({len(dropped)} dropped); checking {len(texts)} substituted file(s)') + # A DROP IS THE HEADLINE, NOT A FOOTNOTE (P31 S54). The drop reasons were computed and + # thrown away, so a slate that lost EVERY draft to declaration conflicts still printed + # "clean — the batch is worth a rebuild". They are the §183 playbook's actual worklist. + for d in dropped: + print(f' [DROP] {d["fn"]}: `{d["symbol"]}` clashes with {d["against"]} in ' + f'{d["file"]} — kept {d["kept"]} vs this {d["this"]}') for sev, code, msg in findings: print(f' [{sev}] {code}: {msg}') - if not findings: + if not findings and not dropped: print(' clean — no textual defect found; the batch is worth a rebuild') + elif not findings: + print(f' no textual defect in what SURVIVED, but {len(dropped)} draft(s) were dropped ' + f'above — reconcile those before gating (they are unbanked work, not noise)') elif not fails: print(f' {len(findings)} warning(s), no hard failure — worth a rebuild') else: print(f'\n{len(fails)} FAILURE(S) — fix these BEFORE spending a clean rebuild.') - sys.exit(1 if fails else 0) + # exit 1 also when the slate was emptied: 0 kept is not a pass, it is total refusal. + sys.exit(1 if (fails or (slate and not kept)) else 0) if __name__ == '__main__':