mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 13:33:34 -04:00
feat(integration): teach the rest of the toolkit about §378 (self-caller cast)
The lever existed but nothing downstream applied it. Proof it mattered: a wave agent this session diagnosed its own blocker as "§378 THE SELF-CALLER CAST, a TU-level fix (cast_self_callers.py) that requires editing src/, which I'm not permitted to touch" — the knowledge propagated, the automation did not. * recover_integration.py: NEW "self-cast" stage (tier=binary), so the driver can run the whole chain as --stages arity,self-cast. The docstring states WHY the order is not arbitrary: self-cast answers the error that "arity" CREATES. * residual_rules_b.py: both decl-conflict tiers now prescribe the full chain instead of "route to integration / budget for banking", and NOCOMPILE-UNDECLARED-FIXED now says outright NOT to gate the autodecl arm (it is a second conflicting declaration in the real TU). * wave-playbook §4b: replaced the stale two-step recipe with the three-step chain, the one-driver form, the callee variant, and the MANDATORY --undo-journal. * SETUP.md: full inventory row (R21) — it had zero mentions. Not wired, deliberately: gate_stage's ladder rewrites DRAFTS via _xform, while this edits the TU; a src-side edit inside the automatic gate needs revert-on-failure, which recover_integration already owns. Still open: a draft_prechecks rule to catch the self-decl conflict statically, before a build is spent. The new stage's plumbing is verified (CLI + candidate selection); its functional end-to-end run is NOT — gate12 held the tree.
This commit is contained in:
Drew T
@@ -779,6 +779,7 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
|
||||
| | **S68 tooling, round 2 — retrieval, triage, walls, liveness** (P31 S68) | **`tools/neighbor_ref.py` (NEW)** — for an OPEN stub, the already-MATCHED functions worth READING as worked examples, ranked SAME-TU first, then same binary, then shape (li-normalised skeleton / call-sequence hash / reloc-kind sequence / CFG counts / opcode cosine, all precomputed in `.run/feat.*.jsonl`), with a hard penalty for opt-level mismatch (§116). Surfaces the neighbour's HEADER COMMENT — the payload agents actually consume. Answers the question `seed_ref` cannot: seed_ref needs a hash-identical twin, this finds a near-twin. **Run it for every card.** Measured motivation: S68's cheapest large matches all came from a neighbour (555 ins/72k, 657 ins/122k first compile, 753 ins/177k) while neighbour-less main fns ran 200-350k for ~80 ins. **`tools/wall_sweep.py` (NEW)** — enumerates the §332 delay-slot macro walls (10 fns / 1,027 ins); `--emit-exclude` feeds `draw_waves --exclude`. **Run before every draw AND before every escalation** — an escalation cannot beat the toolchain. **`tools/residual_rules.py` + `residual_rules_b.py` (NEW, experimental)** — residual→cookbook-lever classifiers; `_b` is the stronger (113/113 classified, 63% certain/high). NOT yet wired into the pipeline: see `docs/next-session-triage-ladder.md`. **`tools/lane_inflight.py` (NEW)** — the RECORDED liveness ledger (`add` on launch, `done` on verdict); `list` exits non-zero while any agent is live and IS the guard. **`tools/r22_verify.sh` (NEW)** — the clean-fleet verify, EXCLUSIVE by construction: it refuses while `lane_inflight` shows live agents, because `make clean` deletes `asm/` and drafting agents READ it. Clears `.run/R22_DEBT` only on a genuinely green run. |
|
||||
| | **S68 tooling — the gater lane + the -O0 route** (P31 S68) | **`tools/gater_lane.py` (NEW)** — the continuous gater: drains a wave's finished drafts into `parallel_gate`, grouped by binary, `--r22` by default. Ledger AND verdicts keyed **`binary:fn:arm`** (R48 — and an escalation ALWAYS has a prior verdict, so arm-keying is what stops an in-flight fable draft riding the opus one). `--extra BINARY:PATH` for non-wave drafts, `--skip-binary` for lanes that may be writing, and **main is routed IN-TREE via `harvest_verify`** because `parallel_gate`'s worktree cannot stage main's psyq_integrate link inputs. **`tools/workflows/escalate_fable.js` (NEW)** — warm-started escalation: passes the prior draft + its measured closeness + its ruled-out levers, and demands a reusable `new_idiom`. **`tools/o0_boundary.py` (NEW)** — the stranded-boundary -O0 sweep (141 binaries / 288 boundaries / 0 candidates: the class is EXHAUSTED, and that null is negative-controlled). |
|
||||
| | **S69 tooling — the TRIAGE LADDER** (P31 S69) | **`tools/triage_ladder.py` (NEW)** — the zero-token pass that answers *"does this target need an agent at all?"* before one is spent. **The PRE/POST split is the point:** `--pre <wave>` runs the TARGET-SIDE tiers only (BANKED · WALL-332 · PARKED) — no draft, no build, milliseconds — and writes `<wave>/triage.json` + `triage_exclude.txt`; `--post` runs the full `residual_rules_b` residual routing, which needs a draft and runs `match_one`. `--escalate B:FN` exits 2 on a walled/banked target (the check S68 was missing when it escalated a §332 wall at closeness 8). `--acceptance` is the R39/R32 harness: false-skip over EVERY open stub, recall over sampled matched fns, and a wall negative control over already-banked code — all pure filesystem work, so it runs in seconds over the whole corpus. **It REFUSES on a non-quiescent tree** (`pgrep -af` rows for gate/build lanes + `lane_inflight`): a live gate makes the stub oracle transiently wrong in both directions (§377). Wired into `wave_args.py` (drops walled/parked targets at draw time, reusing `pre_classify` — one implementation, R33) and into `tools/workflows/escalate_fable.js`, which now REFUSES any target that does not carry `triage:'DRAFT'`. **Routing correction (§376):** the `INTEG-STANDALONE-MATCH` / `NOCOMPILE-UNDECLARED-*` tiers are GATE-FIRST candidates, never free banks — S69 gated that class **0/28** raw; the route is `fix_arity_callers --any-proto` then the gate. |
|
||||
| | **S69 tooling — the §378 SELF-CALLER CAST** (P31 S69) | **`tools/cast_self_callers.py` (NEW)** — the mirror of `cast_call_sites.py`. That one fixes the DRAFT calling a conflicting CALLEE; this one fixes the TU's OWN already-banked code calling the function the draft is about to DEFINE, which nothing handled and which is the terminal blocker of the §376 pile. **Never run it alone** — it answers the error that `fix_arity_callers --any-proto` CREATES: no-protoing the conflicting decl makes the draft's definition the prototype in scope, so the TU's own call fails anew with `too few arguments`. Order is `arity -> self-cast -> [--sync-decls] -> gate`. Byte-neutral because gcc-2.7.2 folds a cast of a known function symbol back to a direct `jal`. `--sync-decls` covers the narrow-param case a no-proto decl CANNOT legally reach (C89 requires promotion-stable parameter types when one declaration has no prototype, so `void f(s16)` is illegal against `extern void f();` — which is exactly why `fix_arity_callers` skips it as 'narrow-param'); safe only once the call sites are cast, because a declaration then emits no code. It REFUSES a function whose return type it cannot read off the draft (R43). **Journals every edit, and `--undo-journal --keep <banked>` after the gate is MANDATORY** — a leftover cast made `ov_SC07_000` stop compiling and every later gate verdict on it measured a broken baseline. Wired into `recover_integration.py --stages arity,self-cast` (tier `binary`), prescribed by `residual_rules_b`'s decl-conflict tiers, and in the playbook §4b. Measured: **8 banked of 28** including `main/func_80036D58` at zero agent tokens; generalises to the callee the diagnostic NAMES (banked `main/func_80021D38` that way). |
|
||||
| | **S68 fixes — six instances of the overlay-layout assumption** (cookbook **§363**) | `dedup_propagate` could not even IMPORT (`os.` at module level in the one module that imports `os as _os`). `seed_ref` offered main's LINKED-subseg DEAD TEXT as bankable twins (43 of 82 hits — a draft there gates GREEN while wrong); now refuses and COUNTS the refusal. `parallel_gate.stage_generated` hard-coded `build/<b>/<b>.ld`; now asks the Makefile for `<b>_LD_SCRIPT`/`<b>_UNDEF_SYMS`/`<b>_UNDEF_FUNCS` and REFUSES when absent. `rtu_match` gained **`--tu`** (+ `blocker_probe` passes `stub.path` and `stub.asm_dir`) — it reconstructed `src/<source>/<split>.c`, which is the overlay layout; main's sources are LOOSE FILES in `src/`. `gate_stage` no longer synthesises `--out`/`--good-sha` — for main those resolved to a nonexistent path and then **ov_SC01_077's SHA** via `DEF_SHA`. **`psyq_integrate`**: the `*_externals.ld` map is now MONOTONIC — it was re-derived against the CURRENT `.ld`, so `firstfile = 0x80061FA8;` was DROPPED on every incremental relink and main was 2 bytes red before any draft was spliced (**the true identity of the 2026-08-15 'main link defect'**). |
|
||||
| | **S68 — the module-binary -O0 carve route** (cookbook **§371**) | `jr_isolate_all` + `overlay_src_split` + the **Makefile -O0 glob widened to `src/md_*/md_*_o0?.c`** open carving for the single-object `md_*` binaries. Three stacked causes behind one `unaddressable content` message (interior-YAML-comment symbol-list truncation; a trailing verbatim-asm chunk with no region; bare tag forward decls), then the **spimdisasm rodata-migration trap**: migrated rodata follows its function ONLY within the same subseg, so a carve silently drops it and `INCLUDE_RODATA` cannot bring it back — rename the `.rodata` subseg to the object its emitters moved to. **The Makefile hunk MUST be committed with the carve** or a fresh clone loses -O0 on the region and every draft banked there mystery-fails. |
|
||||
| | `tools/recover_rejects.py` | **(P31 S59)** Free recovery of PRE-GATE rejects, wired into the maintenance lane. Two paths exist for a draft that does not bank and only one was recorded: a gate failure gets a backlog row (closeness/class/best draft), while a draft the reloc pre-filter drops reached nothing — **569 of 1,261 drafts over eight waves, 45%**. Of the `MISMATCH?` rejects, **13% carry `shape: MATCH`** — right body, wrong symbol names, i.e. the §171 stale-seed class `aprop_symfix` rebases deterministically. Reads `.run/reloc_rejects.jsonl` (written by `ox_campaign.reloc_filter`), keeps shape-MATCH rows that are STILL open stubs, runs `aprop_symfix --fix`, and STAGES the rebased bodies into `.run/sweep_maint/<bin>/` for the lane's existing free gate. It never substitutes, gates or commits — a bad recovery can waste a build, never a bank. Tried-once is remembered in `.run/recover_rejects_seen.json`. Zero model tokens. |
|
||||
|
||||
+25
-7
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1033 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1038 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -101,7 +101,7 @@
|
||||
- **§346** — `c ? X : -X` TAKES expand_expr's COND_EXPR **SINGLETON** PATH (copy, then negate IN PLACE) — AN if/else STATEMENT GIVES THE TWO-ARM FORM (P31 S67; byte-proven ov_SC03_102/func_80180C38, closed the last instruction) <sub>L31322</sub>
|
||||
- **§370** — ★★ — A **HARD BOUND** FROM sched.c, AND THE reorg SLOT-STEAL DIAGNOSTIC (P31 S68; main/func_8001BC6C, 69 ins, NOT closed — 33 → 28 over ~45 measured compiles) <sub>L31838</sub>
|
||||
|
||||
### instruction scheduling (81)
|
||||
### instruction scheduling (83)
|
||||
|
||||
- **§3-T2** — Source statement order drives instruction scheduling <sub>L78</sub>
|
||||
- **§3** — When a diff is pure scheduling → decomp-permuter (harness built, Phase 6) <sub>L107</sub>
|
||||
@@ -184,6 +184,8 @@
|
||||
- **§373** — ★★★ — THE **DEAD-RESET CSE-BREAKER**, THE PIN THAT BREAKS A sched2 ANTI-DEP, AND WHY AN `asm` CAN NEVER RAISE PRIORITY (P31 S68; byte-proven ov_SC06_010/func_8017E764, 438 ins, fable escalation closed 8 → 0) <sub>L31977</sub>
|
||||
- **§3-2.** — A REGISTER PIN THAT DELETES A sched2 ANTI-DEPENDENCE <sub>L31997</sub>
|
||||
- **§3-3.** — HARD FACT FOR THE SCHEDULING MAP — an `asm` ALWAYS has priority 1 <sub>L32010</sub>
|
||||
- **§379** — ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CAST CANNOT (P31 S69; byte-proven main/func_80021284 220 ins and main/func_8002D904 217 ins, found INDEPENDENTLY by two agents) <sub>L32164</sub>
|
||||
- **§381** — THE `insn_count` HOIST THRESHOLD IS A DIAL YOU CAN READ WITH `cc1 -dL` (P31 S69; four independent uses in one wave) <sub>L32216</sub>
|
||||
|
||||
### register allocation & pins (127)
|
||||
|
||||
@@ -407,7 +409,7 @@
|
||||
- **§361** — ★ — A LOOP-TAIL BYTE SIGNATURE THAT NAMES ITS SOURCE SHAPE — AND THE LAW THAT A "SCHEDULING TIE" MAY BE YOUR OWN EARLIER LEVER (P31 S68; byte-proven main/func_800241C0, fable escalation, 19 → 0 in 3 iterations) <sub>L31637</sub>
|
||||
- **§366** — ★★ — `group_case_nodes` MERGES **STACKED CONSECUTIVE** CASE LABELS: GIVE EVERY CASE ITS OWN BODY (P31 S68; ov_SC01_001/func_8017EC28, **first-try MATCH 360/360**, 96/96 relocs audited) <sub>L31757</sub>
|
||||
|
||||
### structs, block moves & memcpy (83)
|
||||
### structs, block moves & memcpy (84)
|
||||
|
||||
- **§3-T2** — Source statement order drives instruction scheduling <sub>L78</sub>
|
||||
- **§5** — Known hard-residual classes (instruction-identical, one byte-exact blocker) <sub>L199</sub>
|
||||
@@ -492,8 +494,9 @@
|
||||
- **§351** — `/s` (MEM_IN_STRUCT_P) IS A DIAL YOU CHOOSE PER ACCESS: A COMPONENT_REF GRANTS IT AND LETS cse KEEP AN INDEX ACROSS THE STORE; A PLAIN CAST DENIES IT (P31 S67; ov_SC01_000/func_8017DD04, 186 -> 5) <sub>L31456</sub>
|
||||
- **§357** — ONE STRUCT POINTER, NOT TWO: A SECOND SOURCE VARIABLE BUILDS A THIRD IV (P31 S68; byte-proven ov_SC06_029/func_80181DF8, 335 ins, 330 → 13) <sub>L31589</sub>
|
||||
- **§364** — ★ — THE libgpu `P_TAG` BITFIELD SPELLING IS **OPT-LEVEL DEPENDENT** (P31 S68; two functions, opposite verdicts, same session) <sub>L31734</sub>
|
||||
- **§379** — ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CAST CANNOT (P31 S69; byte-proven main/func_80021284 220 ins and main/func_8002D904 217 ins, found INDEPENDENTLY by two agents) <sub>L32164</sub>
|
||||
|
||||
### types, signedness & load/store width (91)
|
||||
### types, signedness & load/store width (92)
|
||||
|
||||
- **§3-I1** — Unsigned range check: `(x - lo) < (hi-lo)` → `addiu`+`sltiu` <sub>L41</sub>
|
||||
- **§3-I2** — Byte mask forces `andi` even after `lbu` <sub>L47</sub>
|
||||
@@ -586,6 +589,7 @@
|
||||
- **§345** — A VOLATILE **STORE** EVICTS THE MEM FROM cse AND KEEPS `sh`; A VOLATILE **LOAD** BLOCKS combine AND DEGRADES `lh` INTO `lhu+sll+sra` (P31 S67; byte-proven ov_SC01_084/func_80181A7C) <sub>L31312</sub>
|
||||
- **§351** — `/s` (MEM_IN_STRUCT_P) IS A DIAL YOU CHOOSE PER ACCESS: A COMPONENT_REF GRANTS IT AND LETS cse KEEP AN INDEX ACROSS THE STORE; A PLAIN CAST DENIES IT (P31 S67; ov_SC01_000/func_8017DD04, 186 -> 5) <sub>L31456</sub>
|
||||
- **§378** — ★★★ — THE **SELF-CALLER CAST**: LET A TU KEEP CALLING THE FUNCTION IT IS ABOUT TO DEFINE (P31 S69; byte-proven ov_SC04_010/func_8017D6CC) <sub>L32114</sub>
|
||||
- **§379** — ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CAST CANNOT (P31 S69; byte-proven main/func_80021284 220 ins and main/func_8002D904 217 ins, found INDEPENDENTLY by two agents) <sub>L32164</sub>
|
||||
|
||||
### declarations, prototypes & K&R (110)
|
||||
|
||||
@@ -780,7 +784,7 @@
|
||||
- **Addendum** — Addendum to §164-XX/§165-06 — at -O0 a bare local IS a memory lvalue, so the increment ope (func_8001212C) <sub>L30477</sub>
|
||||
- **§362** — TWO TRAPS WHEN A CARVE MOVES A STUB INTO THE `-O0` TU (P31 S68; byte-proven, 6 fns / 2,547 ins across ov_MAIN_012 / ov_SC02_037 / ov_SC03_107) <sub>L31682</sub>
|
||||
|
||||
### family propagation & sweeps (115)
|
||||
### family propagation & sweeps (116)
|
||||
|
||||
- **§8d** — Templating a body INTO a TU must not CHANGE its declaration environment — demote the carried data externs (Phase 26 session 8, byte-proven on `func_8015AE2C` ×133) <sub>L502</sub>
|
||||
- **§11** — Cross-binary dedup & code-sharing (Phase 11 — "one match unlocks many") <sub>L927</sub>
|
||||
@@ -897,6 +901,7 @@
|
||||
- **§341** — AN HImode STORE TEMP REWEIGHTS A sched2 TIE-BREAK THAT NO STATEMENT ORDER CAN REACH (P31 S67; byte-proven ov_SC03_006/func_801823B8, last 4 ins) <sub>L31243</sub>
|
||||
- **§355** — A REMAPPED SIBLING'S **SOURCE BIAS IS NOT ITS EMITTED BIAS** — DO NOT HAND-SHIFT OFFSETS TO MATCH THE ASM (P31 S68; byte-proven ov_SC07_007/func_8013DD68, 187/187 in 2 iterations) <sub>L31560</sub>
|
||||
- **§368** — ★★★ — THE **RELOAD-REMAT CONSTANT**: REACH A REGISTER NO PIN CAN REACH (P31 S68; ov_SC03_105/func_80187A30, 339 ins, fable escalation closed 8 → 0 in ONE edit) <sub>L31792</sub>
|
||||
- **§379** — ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CAST CANNOT (P31 S69; byte-proven main/func_80021284 220 ins and main/func_8002D904 217 ins, found INDEPENDENTLY by two agents) <sub>L32164</sub>
|
||||
|
||||
### integration / TU plumbing (71)
|
||||
|
||||
@@ -972,7 +977,7 @@
|
||||
- **§376** — ★★★ — A STANDALONE `match_one` CLOSENESS OF 0 IS A CLAIM ABOUT THE **BODY**, NEVER ABOUT THE **TU** (P31 S69; measured 0/28) <sub>L32055</sub>
|
||||
- **§378** — ★★★ — THE **SELF-CALLER CAST**: LET A TU KEEP CALLING THE FUNCTION IT IS ABOUT TO DEFINE (P31 S69; byte-proven ov_SC04_010/func_8017D6CC) <sub>L32114</sub>
|
||||
|
||||
### build graph, splat & the harness (185)
|
||||
### build graph, splat & the harness (186)
|
||||
|
||||
- **§4** — Flag/toolchain gotchas <sub>L190</sub>
|
||||
- **Build** — mechanism — per-file opt override (splat resegmentation) <sub>L307</sub>
|
||||
@@ -1159,6 +1164,7 @@
|
||||
- **§357** — ONE STRUCT POINTER, NOT TWO: A SECOND SOURCE VARIABLE BUILDS A THIRD IV (P31 S68; byte-proven ov_SC06_029/func_80181DF8, 335 ins, 330 → 13) <sub>L31589</sub>
|
||||
- **§358** — (sharpens §333) — AN **UNREFERENCED** FIXED-SIZE AGGREGATE LOCAL IS LOAD-BEARING (P31 S68; same function) <sub>L31598</sub>
|
||||
- **§371** — ★★ — CARVING A **SINGLE-OBJECT MODULE BINARY**, AND THE spimdisasm RODATA-MIGRATION TRAP THAT FOLLOWS (P31 S68; byte-proven md_MAIN_003, func_800D0D6C 345 ins) <sub>L31890</sub>
|
||||
- **§383** — TWO TOOLCHAIN FACTS THE PACKS DID NOT CARRY (P31 S69) <sub>L32244</sub>
|
||||
|
||||
### process, measurement & doctrine (126)
|
||||
|
||||
@@ -1289,7 +1295,7 @@
|
||||
- **§370** — ★★ — A **HARD BOUND** FROM sched.c, AND THE reorg SLOT-STEAL DIAGNOSTIC (P31 S68; main/func_8001BC6C, 69 ins, NOT closed — 33 → 28 over ~45 measured compiles) <sub>L31838</sub>
|
||||
- **§376** — ★★★ — A STANDALONE `match_one` CLOSENESS OF 0 IS A CLAIM ABOUT THE **BODY**, NEVER ABOUT THE **TU** (P31 S69; measured 0/28) <sub>L32055</sub>
|
||||
|
||||
### (unbucketed — title matched no symptom vocabulary) (305)
|
||||
### (unbucketed — title matched no symptom vocabulary) (307)
|
||||
|
||||
- **§3-How** — to use this <sub>L30</sub>
|
||||
- **§1** — Idiom catalog (asm pattern → C that produces it) <sub>L39</sub>
|
||||
@@ -1596,6 +1602,8 @@
|
||||
- **§372** — ★★★ — THE **COPY-CAPTURE PAIR**, AND THE ONE ZERO-BYTE EDIT THAT DEFEATS BOTH (P31 S68; byte-proven main/func_8003491C, 78 ins, fable escalation closed 5 → 0) <sub>L31939</sub>
|
||||
- **CROSS-REFERENCE** — TO §370 — checked and found INAPPLICABLE here, which is the point <sub>L32018</sub>
|
||||
- **§377** — THREE HARNESS DEFECTS FOUND IN ONE GATING SESSION, ALL "A CONFIDENT NUMBER ABOUT A SMALLER WORLD" (P31 S69) <sub>L32097</sub>
|
||||
- **§380** — ★★★ — **A SECOND SET OF A PSEUDO DISQUALIFIES IT FROM `move_movables`** (P31 S69; main/func_800215F4, 465 ins, closeness 106 → 59 → 39) <sub>L32195</sub>
|
||||
- **§382** — TWO FOLD REASSOCIATIONS THAT NEED THEIR OWN STATEMENT (P31 S69) <sub>L32232</sub>
|
||||
|
||||
|
||||
## All sections, in order
|
||||
@@ -2633,6 +2641,11 @@
|
||||
- **§376** — ★★★ — A STANDALONE `match_one` CLOSENESS OF 0 IS A CLAIM ABOUT THE **BODY**, NEVER ABOUT THE **TU** (P31 S69; measured 0/28) <sub>L32055</sub>
|
||||
- **§377** — THREE HARNESS DEFECTS FOUND IN ONE GATING SESSION, ALL "A CONFIDENT NUMBER ABOUT A SMALLER WORLD" (P31 S69) <sub>L32097</sub>
|
||||
- **§378** — ★★★ — THE **SELF-CALLER CAST**: LET A TU KEEP CALLING THE FUNCTION IT IS ABOUT TO DEFINE (P31 S69; byte-proven ov_SC04_010/func_8017D6CC) <sub>L32114</sub>
|
||||
- **§379** — ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CAST CANNOT (P31 S69; byte-proven main/func_80021284 220 ins and main/func_8002D904 217 ins, found INDEPENDENTLY by two agents) <sub>L32164</sub>
|
||||
- **§380** — ★★★ — **A SECOND SET OF A PSEUDO DISQUALIFIES IT FROM `move_movables`** (P31 S69; main/func_800215F4, 465 ins, closeness 106 → 59 → 39) <sub>L32195</sub>
|
||||
- **§381** — THE `insn_count` HOIST THRESHOLD IS A DIAL YOU CAN READ WITH `cc1 -dL` (P31 S69; four independent uses in one wave) <sub>L32216</sub>
|
||||
- **§382** — TWO FOLD REASSOCIATIONS THAT NEED THEIR OWN STATEMENT (P31 S69) <sub>L32232</sub>
|
||||
- **§383** — TWO TOOLCHAIN FACTS THE PACKS DID NOT CARRY (P31 S69) <sub>L32244</sub>
|
||||
|
||||
|
||||
---
|
||||
@@ -3678,3 +3691,8 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
|
||||
| L32055 | §376 | ★★★ — A STANDALONE `match_one` CLOSENESS OF 0 IS A CLAIM ABOUT THE **BODY**, NEVER ABOUT T |
|
||||
| L32097 | §377 | THREE HARNESS DEFECTS FOUND IN ONE GATING SESSION, ALL "A CONFIDENT NUMBER ABOUT A SMALLER |
|
||||
| L32114 | §378 | ★★★ — THE **SELF-CALLER CAST**: LET A TU KEEP CALLING THE FUNCTION IT IS ABOUT TO DEFINE ( |
|
||||
| L32164 | §379 | ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CA |
|
||||
| L32195 | §380 | ★★★ — **A SECOND SET OF A PSEUDO DISQUALIFIES IT FROM `move_movables`** (P31 S69; main/fun |
|
||||
| L32216 | §381 | THE `insn_count` HOIST THRESHOLD IS A DIAL YOU CAN READ WITH `cc1 -dL` (P31 S69; four inde |
|
||||
| L32232 | §382 | TWO FOLD REASSOCIATIONS THAT NEED THEIR OWN STATEMENT (P31 S69) |
|
||||
| L32244 | §383 | TWO TOOLCHAIN FACTS THE PACKS DID NOT CARRY (P31 S69) |
|
||||
|
||||
@@ -32160,3 +32160,98 @@ changes the call's value category, so it REFUSES instead, R43), skips declaratio
|
||||
call site in that TU is checked against.* A draft is not just a body — it is a new prototype imposed
|
||||
on code that already compiled. Expect to fix the callers, and expect the fix to be a cast, because a
|
||||
cast is the only edit that changes typing without changing codegen.
|
||||
|
||||
## §379 ★★★ — **MEM_IN_STRUCT_P**: THE SAME LOAD, WRITTEN AS A STRUCT MEMBER, SCHEDULES WHERE A CAST CANNOT (P31 S69; byte-proven main/func_80021284 220 ins and main/func_8002D904 217 ins, found INDEPENDENTLY by two agents)
|
||||
|
||||
**The symptom.** A load sits below a store to a scalar global in your output and above it in the
|
||||
target (or vice versa), and no statement permutation moves it. `func_80021284` sat at closeness 7
|
||||
across a **120-order statement-permutation plateau**; the fix took it straight to 0.
|
||||
|
||||
**The mechanism.** gcc-2.7.2's alias machinery asks `true_dependence`/`anti_dependence` whether a
|
||||
load may cross a store. Its answer turns on `MEM_IN_STRUCT_P` — the `/s` flag `expand_expr` sets on
|
||||
a MEM. A raw `*(u32 *)(arg0 + N)` load is a plain MEM: it may alias the scalar global, so the
|
||||
scheduler will not move it across that store. The **same address read through a struct pointer**
|
||||
(`p->field`) is `/s`, declared non-aliasing against a fixed scalar store, and floats freely.
|
||||
|
||||
*(u32 *)(arg0 + 0x18) /* plain MEM — pinned below the store to D_800xxxxx */
|
||||
p->attr /* MEM_IN_STRUCT_P — free to hoist to the block top */
|
||||
|
||||
**The complement, same pass, opposite direction** (`func_8002D904`, worth **26 of its 41 residuals**):
|
||||
assigning an indexed table address to a **pointer variable first** makes the resulting `INDIRECT_REF`
|
||||
*non*-`MEM_IN_STRUCT_P` — gcc sets `/s` whenever "the address was computed by addition", so every
|
||||
array/cast spelling comes out `/s`+varying, and the load floats to the block top. Take the address
|
||||
into a local pointer when you need the load to STAY.
|
||||
|
||||
**So the dial has two ends and both are zero-byte:**
|
||||
|
||||
| you want | write it as |
|
||||
|---|---|
|
||||
| the load to FLOAT (cross a scalar store) | a struct-member access through a pointer |
|
||||
| the load to STAY | take the address into a local pointer variable first |
|
||||
|
||||
Related store-side note at `src/800.c:21942` (W16). This is the alias-analysis sibling of §41b's
|
||||
"global load hoisted above the RTL prologue" wall — that one is unreachable; this one is a spelling.
|
||||
|
||||
## §380 ★★★ — **A SECOND SET OF A PSEUDO DISQUALIFIES IT FROM `move_movables`** (P31 S69; main/func_800215F4, 465 ins, closeness 106 → 59 → 39)
|
||||
|
||||
`move_movables` hoists a loop invariant only while the pseudo has **exactly one set**. Give it a
|
||||
second, and the hoist does not happen — a lever no register pin can reach, because a pin changes
|
||||
*which* register, never *whether* the value is hoisted.
|
||||
|
||||
pz = &otz; /* first set */
|
||||
…
|
||||
pz = &dzp[i >> 1]; /* second set — pz is now ineligible for hoisting */
|
||||
|
||||
k = 0xC000;
|
||||
…
|
||||
k = 0xFFF; /* same trick on a constant */
|
||||
|
||||
Two loop invariants that a `$v0` pin could not un-hoist came down this way. Reach for it whenever
|
||||
the target computes something INSIDE the loop that your draft computes in the preheader.
|
||||
|
||||
**The related pin form** (`func_8001EFE0`, 468 ins): a **call-used hard-reg pin** makes `invariant_p`
|
||||
return 0 when the loop contains a call, which is what stops `move_movables` hoisting an addPrim
|
||||
address. Pin to `$a3` — a call-clobbered register — precisely *because* it is call-clobbered.
|
||||
|
||||
## §381 — THE `insn_count` HOIST THRESHOLD IS A DIAL YOU CAN READ WITH `cc1 -dL` (P31 S69; four independent uses in one wave)
|
||||
|
||||
`move_movables`' willingness to hoist scales with the loop's `insn_count`, and the edge is sharp
|
||||
enough to sit on. Read the count from the `-dL` (loop) dump, then move ONE statement across it:
|
||||
|
||||
* `func_80040DE8` — insn_count **232** hoisted `(s16)param_1` at the exact `58×2×2 = 232` edge; a
|
||||
`key = param_1;` copy inside the loop lifted it to 233 and **un-hoisted** it.
|
||||
* `func_80023BF0` — loop 2's insn_count is **39** and needs **≥41**: below the threshold gcc admits
|
||||
one constant hoist too many (`li 0xFFFF` at threshold 40), which costs a fourth callee-saved
|
||||
register and shifts the whole register file. Still open at 269 for exactly this reason.
|
||||
* `func_8001DA34` — `29×1×9 ≥ 203` hoisted the `0xFF000000` OT mask and spilled `ot` (frame
|
||||
0xB8→0xC8); fixed with `$a1`/`$a2` pins. **BANKED, 408 ins.**
|
||||
|
||||
**Read the dump before you theorise.** Three of these four were found by reading `-dL`, not by
|
||||
guessing, and the fourth is documented as an open threshold miss rather than a mystery.
|
||||
|
||||
## §382 — TWO FOLD REASSOCIATIONS THAT NEED THEIR OWN STATEMENT (P31 S69)
|
||||
|
||||
gcc-2.7.2's `fold` rewrites these before you ever see RTL, so the fix is always "split the
|
||||
expression", never "add a cast":
|
||||
|
||||
* `A * (X / 0x7F * 0x3FFF)` folds to `(A * 0x3FFF) * X` — the 0x3FFF scaling needs its own statement
|
||||
(`func_80040DE8`).
|
||||
* `A + (B + C)` reassociates to `(A + C) + B` — an address split into two statements blocks it
|
||||
(`func_80021284`).
|
||||
* `(x << 9) | CONST | var` reassociates so that only the shift gets hoisted — the four `getTPage`
|
||||
bases must be explicit pre-loop variables (`func_8001EFE0`).
|
||||
|
||||
## §383 — TWO TOOLCHAIN FACTS THE PACKS DID NOT CARRY (P31 S69)
|
||||
|
||||
1. **maspsx requires DECIMAL memory-operand offsets in verbatim asm.** `0x40($sp)` dies with
|
||||
`invalid literal for int() with base 10`; write `64($sp)`. Hex is fine for plain immediates.
|
||||
Found while banking `main/func_80026514` (239 ins) the §265 verbatim-asm way — that function is
|
||||
genuinely HANDWRITTEN: its `D_800B99DA` fetch is a gas macro expansion of `lbu $v1,0xA3AA($t2)`
|
||||
into `lui $at,1 / addu $at,$t2,$at / lbu $v1,-0x5C56($at)`, which gcc cannot emit.
|
||||
2. **A 4-byte struct local is BLKmode**, so `assign_stack_local` gives it 8-byte
|
||||
`BIGGEST_ALIGNMENT`. Declaring `xy0`/`xy1` as `s32` instead of a 4-byte struct is the difference
|
||||
between frame 0x130 and 0x138 (`func_8001EFE0`).
|
||||
3. **`convert_modes` returns x unchanged when `oldmode == GET_MODE(x)`**, so **no cast can emit the
|
||||
`(s16)` `sll`/`sra` pair**. Only an `s16` local assigned in one block and consumed in a distant
|
||||
one does — and the copy must sit ABOVE the guard, or it coalesces away against the target's
|
||||
`addu $a1,$v0,$zero` (`func_8002D904`).
|
||||
|
||||
+22
-1
@@ -179,7 +179,28 @@ a merging gate makes the stub oracle wrong in both directions (§377). If it ref
|
||||
|
||||
**And do not treat its `INTEG-STANDALONE-MATCH` / `NOCOMPILE-UNDECLARED-*` output as banks.** That is
|
||||
the §376 correction: standalone closeness 0 proves the BODY, not that the TU accepts the SIGNATURE.
|
||||
Measured S69: **0 of 28** of that class banked raw. Route them `fix_arity_callers --any-proto` → gate.
|
||||
Measured S69: **0 of 28** of that class banked raw — then **8 banked** once the missing lever
|
||||
existed. Run the full §378 chain, IN ORDER; each step only reveals the next, and stopping at step 1
|
||||
is how the class read as dead for half a session:
|
||||
|
||||
```
|
||||
tools/fix_arity_callers.py --binary B --funcs FN --any-proto --apply --journal J1 # `conflicting types'
|
||||
tools/cast_self_callers.py --binary B --funcs FN --drafts D --apply --journal J2 # `too few arguments'
|
||||
tools/cast_self_callers.py … --sync-decls # narrow-param: C89 forbids no-proto vs `void f(s16)'
|
||||
<gate> # the byte-gate is the sole arbiter
|
||||
tools/cast_self_callers.py --undo-journal J2 --keep <banked> # MANDATORY — see below
|
||||
```
|
||||
|
||||
Or in one driver: `recover_integration.py --binary B --stages arity,self-cast --max-tier fleet --r22`.
|
||||
|
||||
> **THE UNDO IS NOT OPTIONAL.** The casts go in BEFORE the gate, in preparation. A cast left behind
|
||||
> for a draft that did not bank made `ov_SC07_000` fail to COMPILE at HEAD, so every later gate
|
||||
> verdict on that binary measured a broken baseline rather than a draft (found only because two
|
||||
> drafting agents reported BASELINE-RED and I checked their claim). Run `--undo-journal --keep
|
||||
> <banked>` after EVERY gate.
|
||||
|
||||
If the diagnostic names a DIFFERENT symbol (`conflicting types for func_8012AD44`), run the same
|
||||
chain on THAT symbol — it is a callee, not your target. That banked `main:func_80021D38`.
|
||||
|
||||
## 5. Draft
|
||||
|
||||
|
||||
@@ -60,8 +60,16 @@ STAGE_TIER = {"arity": "fleet", # fix_arity_callers writes src/shared/eng
|
||||
"tu-scope": "binary", # §103 STU: moves a contested TU decl into its consumers
|
||||
# (P31 T6 — the sweep-only lever the recovery path lacked;
|
||||
# writes src/<binary>/*.c, covered by the TU snapshot)
|
||||
"macro-externs": "draft"} # §121: rewrite draft decls of DEFINE_-defined callees to
|
||||
"macro-externs": "draft", # §121: rewrite draft decls of DEFINE_-defined callees to
|
||||
# the macro's own signature (draft text only)
|
||||
"self-cast": "binary"} # §378 (P31 S69): cast the TU's OWN call sites of the
|
||||
# function being defined. Writes src/<binary>/*.c only.
|
||||
# THE STAGE ORDER MATTERS AND IS NOT ARBITRARY: `arity`
|
||||
# no-protos the conflicting forward decl, which makes the
|
||||
# DRAFT'S DEFINITION the prototype in scope, so the TU's own
|
||||
# call fails anew with `too few arguments'. self-cast is the
|
||||
# answer to the error `arity` CREATES, so it must run AFTER
|
||||
# it, never instead of it.
|
||||
|
||||
|
||||
def tier_ok(stage, max_tier):
|
||||
@@ -193,7 +201,7 @@ def main():
|
||||
ap.add_argument("--no-propagate", action="store_true")
|
||||
ap.add_argument("--run-id", default="r1", help="run-local scratch under .run/recover/<id>/ (§55b trap 4)")
|
||||
ap.add_argument("--stages", default="demacroize",
|
||||
help="comma-separated: demacroize,arity (default: demacroize — the measured blocker)")
|
||||
help="comma-separated: demacroize,arity,self-cast,tu-scope,macro-externs (default: demacroize). §378 chain = arity,self-cast — in that order.")
|
||||
ap.add_argument("--max-tier", default="binary", choices=("draft", "binary", "fleet"),
|
||||
help="refuse any stage whose blast radius exceeds this (default binary = no shared-state edits)")
|
||||
ap.add_argument("--r22", action="store_true", help="clean-fleet verify after banking (REQUIRED for fleet tier)")
|
||||
@@ -336,6 +344,21 @@ def main():
|
||||
raise SystemExit(f"[recover] fix_arity_callers failed: {(r.stderr or r.stdout)[-300:]}")
|
||||
print(" " + (r.stdout.strip().splitlines()[-1] if r.stdout.strip() else "(fix_arity_callers: no output)"))
|
||||
|
||||
if "self-cast" in stages: # P31 S69 — §378: the TU's own call sites of the fn being
|
||||
# DEFINED. `arity` fixes `conflicting types'; this fixes the `too few arguments' that
|
||||
# fixing it produces, because the draft's definition is now the prototype in scope.
|
||||
# Byte-neutral: gcc-2.7.2 folds a cast of a known function symbol back to a direct jal.
|
||||
# --sync-decls additionally handles the narrow-param case (a promotion-affected parameter
|
||||
# like s16 makes a no-proto decl ILLEGAL in C89, which is exactly why fix_arity_callers
|
||||
# skips it), and is safe ONLY because the call sites are cast first.
|
||||
jr = f"{run_dir}/selfcast.json"
|
||||
r = sh([PY, "tools/cast_self_callers.py", "--apply", "--sync-decls", "--binary", a.binary,
|
||||
"--funcs", ",".join(targets), "--drafts", dd,
|
||||
"--journal", jr])
|
||||
if r.returncode:
|
||||
raise SystemExit(f"[recover] cast_self_callers failed: {(r.stderr or r.stdout)[-300:]}")
|
||||
print(" " + (r.stdout.strip().splitlines()[-1] if r.stdout.strip() else "(self-cast: no output)"))
|
||||
|
||||
if "macro-externs" in stages and draft_rewrite: # P31 T6 — §121: a draft's decl of a DEFINE_-defined callee
|
||||
# must match the macro's OWN definition head (a guessed `extern int f();` collides with
|
||||
# the macro's real `void f(s32)` — measured: ONE such draft poisoned an entire probe
|
||||
|
||||
@@ -836,8 +836,12 @@ def classify(binary, fn, draft, tag=None):
|
||||
"the ONLY defect was %d missing extern(s) (%s); with decls COMPUTED "
|
||||
"from the target .s access widths the body byte-matches standalone"
|
||||
% (len(undecl), ", ".join(note["decls"].values())),
|
||||
"add the computed extern(s) and route to integration — the matching "
|
||||
"work is already done.")
|
||||
"GATE-FIRST, NOT A BANK (§376, measured 0/28 raw in P31 S69). Do NOT "
|
||||
"gate the autodecl arm: the extern that satisfies the STANDALONE probe is "
|
||||
"a SECOND conflicting declaration in the real TU. Gate the RAW draft after "
|
||||
"the §378 chain, in this order: fix_arity_callers --any-proto --binary "
|
||||
"<B> --funcs <FN> -> cast_self_callers --binary <B> --funcs <FN> "
|
||||
"--drafts <D> [--sync-decls] -> gate. Each step only reveals the next.")
|
||||
return _finish(out)
|
||||
if res2.get("status") in ("near", "fail") and st2 is not None:
|
||||
out["status"] = res2["status"]
|
||||
@@ -874,8 +878,12 @@ def classify(binary, fn, draft, tag=None):
|
||||
if out["status"] == "match":
|
||||
_fire(rules, "INTEG-STANDALONE-MATCH", "certain", "§58/§59/§367",
|
||||
"closeness 0: the draft is byte-correct standalone",
|
||||
"the blocker is TU integration (decl reconcile ladder, §367 banked-spelling rule); "
|
||||
"matching is done — budget for banking.")
|
||||
"matching is done; BANKING IS NOT. A standalone closeness of 0 proves the BODY and "
|
||||
"says nothing about the TU accepting the SIGNATURE (§376 — this class gated 0/28 raw "
|
||||
"in P31 S69, then 8 banked once the chain existed). Route: fix_arity_callers "
|
||||
"--any-proto -> cast_self_callers [--sync-decls] (§378/§378a) -> gate; or "
|
||||
"recover_integration --stages arity,self-cast. If the diagnostic names a DIFFERENT "
|
||||
"symbol, run the same chain on THAT symbol.")
|
||||
return _finish(out)
|
||||
|
||||
if st is None:
|
||||
|
||||
Reference in New Issue
Block a user