mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 13:33:34 -04:00
fix(tools): reconcile_tu's cc1 premise, the &-cast arms, and the worktree sig gap
The S74 checkpoint's "one unfixed defect that is actively costing banks"
(reconcile_tu manufacturing declaration conflicts), run to ground — plus the
harness gap that produced a false carve-corruption verdict.
reconcile_tu.py — three defects, measured against the real gcc-2.7.2 front end
(cdecl._cc1_accepts, the oracle cdecl.compatible was validated with; R33):
* The premise "a decl BELOW still conflicts" is TRUE at file scope and FALSE
at block scope. cc1 ACCEPTS a block-scope extern against a TU decl below it
(pedwarn "type mismatch with previous external decl"); conforming it is
destructive, because the TU's decl names the TU's TYPE and a type declared
below the splice point is not in scope AT it -- the emitted result gets
"syntax error before 'D_x'". Byte-witnessed on resident:func_800D06E8 (344
ins), whose block-scoped `extern Blk80078E78` became `extern
Struct80078E78`, typedef 388 lines lower. That construct is what this
ladder's OWN scope_demote_drafts (§8d) rung emits on purpose, and three
already-banked functions in that TU use it: one rung undoing another.
* The cast pass rewrote COMMENT PROSE -- 8 rewrites inside one header comment,
including inside a quoted cc1 diagnostic. Now matches on cdecl._mask
(length-preserving, so a mask offset is a source offset) and splices into
the original.
* `&sym` emitted `&` applied to a cast: legal for the scalar arm, `invalid
lvalue in unary '&'` (measured) for the array/fnptr/fnptr_array arms. `&`
now selects a pointer form and consumes itself -- but ONLY with no trailing
subscript, because `&sym[i]` is the address of ELEMENT i and the old code
had that case right. That last clause exists because the R39 negative
control caught the fold as a regression in the first cut of this fix.
gate_stage.py — `--skip-stages` / `GATE_SKIP_STAGES` (loud when used). Stage 0
gates raw drafts first, so a broken rung can only cost a RECOVERY, which is
exactly what makes it invisible: the function it destroys was already failing,
so its DIFF reads as a fact about the function.
verify_worktree.py / jr_isolate_all.py / parallel_gate.py — provision() now
symlinks every .run/sig.*.jsonl (main clone 259, provisioned worktree 0), the
third member of the class holding extracted/ and .run/obj40. parallel_gate was
fixed for this identical bug in S69: two provisioners, no shared list, found
twice; they now cross-reference each other. jr_isolate_all no longer swallows
the resulting FileNotFoundError into `except: continue` -- that turned a missing
index into a confident carve-CORRUPTION verdict over 2,603 of 2,603 functions
(R54). Adds _assert_scan_covered: attempted == raised means the scan measured
nothing, so its zero is an artifact, not a finding (R32).
Verification:
* 4 cc1 probes (the table above), each run on the pinned front end.
* R39 negative control over the stored-draft corpus: 661 adjudicated, 652
IDENTICAL, 9 CHANGED and every one an intended class. 4,173 of 4,864 drafts
unadjudicable (filenames that are not func_<ADDR>) -- stated, not hidden.
* jr_isolate_all ov_SC03_105 --dry-run: unchanged in the main tree.
* make clean/extract/build BINARY=resident -> 8e17e02f... BYTE-IDENTICAL.
Docs ship with the change (R21): cookbook §442/§443, index regenerated (1,112
sections), 3 docs/SETUP.md rows, CURRENT_PHASE S75 log.
This commit is contained in:
Drew T
@@ -767,6 +767,9 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
|
||||
| | `tools/t5_targets.py` · `tools/t5_cards.py` · `tools/wave_card_fuel.py` · `tools/t5_bank.sh` | **(Phase 31 S63 T5)** The wave *draw* and the wave *bank* halves around the T4 harness: `t5_targets.py --wave .run/t5x --n 48 [--residue .run/t5w] [--main-only]` draws a routed slate (open per `corpus.stubs`, ledger `.run/t5/drawn.json` keyed (binary,fn), ≤120 ins → sonnet / >120 → opus, residue → opus once, stratified by band, ≤4 per binary, one fn NAME per wave — the harness keys packs/drafts by bare name, so `claude_wave_packs`/`wave_judge` now REFUSE duplicates, R43/R48); then **`t5_cards.py --wave .run/t5x`** BUILDS the target's OWN card fuel (`tu_ref` = in-TU banked neighbours §194-E, `decl_prior` = fleet signatures §196) from `wave_card_fuel.py` — those helpers extracted VERBATIM out of `build_wave_atlas.py` (which parses argv at import and so was unusable as a library) so both consumers share ONE oracle (R33); measured 88% tu_ref / 98% decl_prior on a fresh K-class slate, against 0% correct fuel before the fix. Then `claude_wave_packs.py --cards <wave>/cards.json` → Workflow `tools/workflows/claude_wave_draft.js` `args={wave, targets}` (one agent per target, `model=target.arm`) → `tools/t5_bank.sh .run/t5x sonnet opus` (refuses if a lane/gate runs or src/ config/ is dirty; `wave_judge --union`; then a RECOVERY pass — every un-banked draft through `fix_tu_ret_decls`, which retypes a destination TU's `extern void` decl on a value-returning function and re-gates (wave t5a: 2/2 banked, both byte-exact bodies refused for that decl alone); then the R22 clean fleet sweep parsed `N passed, 0 failed of N`; commit `— N banked`). |
|
||||
| | `tools/jtbl_rodata_pads.py` — S74 corrections | **(P31 S74, cookbook §436-C/§441)** Three measurement bugs in the island walk, all of which ACCUSE THE CARVE when they fire. (a) `_s_rodata_span` ignored a **trailing `.align`**: a `.s` ending `.asciz "r"` + `.align 2` occupies 4 bytes, not 2. Latent by construction — `--derive`'s `zero_gap` self-corrects a 1-3 byte undershoot whenever the next item is an anchor, and **a C jump table has no anchor**, so it fires only when someone banks a switch function into such an object. (b) `_items` matched a rodata anchor only as `D_xxxxxxxx:`, missing the `labels.inc` macro form **`dlabel D_xxxxxxxx`** that an inline `__asm__` in C produces (as expands it; cc1/maspsx pass it verbatim) — an 8-byte hole in the walk. (c) the `ctable` branch read `word(pos)` without first stepping the sub-word zero gap, so a preceding `.asciz` ending unaligned made it refuse a correct layout. Two agents found (a) independently from opposite directions. Byte-neutral controls: `md_SC07_003` + `md_MAIN_011` clean rebuilds, `gate_main --assert-baseline`, and a full `check-all` 213/213. |
|
||||
| | `tools/jtbl_carve.py` — the `covered` verdict | **(P31 S74, cookbook §440)** A table already inside an existing `.rodata` carve **bound to the function's own subseg** needs NO carve work: in stub state spimdisasm migrates it into the fn's `.s` and the object fills the piece exactly, so banking swaps that block for cc1's identical one. `island_probe` used to classify such a function `tail` on the table's ADDRESS, `apply()` routed it to `build_carve`, which resolves spans out of the RAW data asm where a carved table no longer is → `not found in the raw data asm` → `harvest_verify` booked **CARVE-REFUSED**, a verdict about the route we chose rather than about the function (R43). Now: a `covered` verdict, a `covered-tpad` wall (the retail copy carries a trailing §8a pad word the matched body will not emit — bankable, needs a `0t<n>` spec entry), and a fully-covered batch is a no-op before either route. |
|
||||
| | `tools/reconcile_tu.py` — S75 corrections | **(P31 S75, cookbook §442)** Three defects in the `gate_stage` `-rc` rung, found by running S74's "9/9 via `harvest_verify`, 7/9 via `gate_stage`" measurement to ground. (a) **The premise was false at BLOCK scope.** The tool conformed a draft decl to the TU's in either direction because "a decl BELOW still conflicts" — four `cdecl._cc1_accepts` probes say a block-scope `extern` vs a TU decl BELOW the splice point is **ACCEPTED** (pedwarn `type mismatch with previous external decl`), while file-scope-vs-below and block-scope-vs-above both REJECT. Conforming it is destructive, because the TU's decl names the TU's TYPE and a type declared below the splice point is not in scope at it: the emitted result gets `syntax error before 'D_x'`. Byte-witnessed on `resident:func_800D06E8` (344 ins, `match_one` closeness 0) whose block-scoped `extern Blk80078E78` became `extern Struct80078E78` — typedef 388 lines lower. It is also the construct this ladder's OWN `scope_demote_drafts` (§8d) rung emits on purpose, used by three already-banked functions in that TU: one rung undoing another. Now skipped with a note (R43). (b) **The cast pass rewrote COMMENT PROSE** — it looped over lines skipping only `extern`/`typedef`, so a 40-line header comment had the symbol rewritten 8× including inside a quoted cc1 diagnostic; now matches on `cdecl._mask` (length-preserving, mask offset = source offset) and splices into the original. (c) **`&sym` emitted `&` applied to a cast** — legal for the scalar arm, `invalid lvalue in unary '&'` for the array/fnptr/fnptr_array arms (measured); `&` now SELECTS a pointer form built from the address and consumes itself. |
|
||||
| | `tools/gate_stage.py --skip-stages` | **(P31 S75, cookbook §442)** Switch ladder rungs OFF by tool name (`--skip-stages reconcile_tu,sig_unify`, or `GATE_SKIP_STAGES=`); prints what it skipped, because a silently-disabled recovery rung is the R32 defect class. Exists because stage 0 gates the RAW drafts first, so a broken rung can only cost a RECOVERY — which is exactly what makes it invisible: the function it destroys was already failing, so its DIFF reads as a fact about the function. The next bad rung costs a flag instead of a session. |
|
||||
| | `tools/verify_worktree.py` step 3c + `tools/jr_isolate_all.py` scan guards | **(P31 S75, cookbook §443)** `provision()` now symlinks every `.run/sig.*.jsonl` into the worktree (main clone 259, provisioned worktree **0** before this) — the third member of the same class as `extracted/` (3) and `.run/obj40` (3b), and the SAME defect `parallel_gate.stage_generated` had fixed per-binary in S69: two worktree provisioners, no shared list, so it had to be fixed twice; they now cross-reference each other. At the other end, `jr_isolate_all`'s carve-ownership scan wrapped `reloc_targets` in `try/except Exception: continue` — correct for one unwalkable function, catastrophic when the cause is environmental: **2,603 of 2,603 functions raised, the scan found 0 owners, and the downstream R32 1:1-ownership assert reported carve CORRUPTION that did not exist** (R54 — a guard downstream of the failure is not a guard). `FileNotFoundError` now aborts at the cause naming `make sig-all`, and `_assert_scan_covered` fails loud when attempted == raised (R32: a scan where everything raised measured nothing, so its zero is an artifact). |
|
||||
| | `tools/jr_isolate_all.py` — boundary derivation | **(P31 S74, cookbook §441)** `_region_emit_start()`: the yaml offset for a region is derived from the region's **CONTENT** — min of item addresses and of every `.globl`/`.ent` its text names that resolves inside the object — and taken as `min(cut, emit)`, so a boundary can only move DOWN. Reason: **a §265 verbatim `__asm__` body is not one of `parse_overlay_c`'s four addressed-anchor forms, so it attaches to the NEXT anchor as PREAMBLE — and preamble is assumed byte-neutral when it emits bytes.** A cut at `func_800D0268` would have moved 0x168 bytes of three other functions into the new object while the yaml claimed the region started higher. Where no verbatim asm is in play it equals the cut, so every existing isolate is unchanged. Also: an item-less CLOSING region used to emit a duplicate `- [off, c, …]` line (the empty-region skip covered only region 0, and `_partition`'s empty `footer` made the closing region look non-empty). |
|
||||
| | `tools/jtbl_rodata_pads.py --derive` | **(Phase 31 S62 T3a, cookbook §303)** Module path of the §8e pads filter: the Makefile runs `--derive $(BINARY) --tu <tu>` for every `md_*` object **and for `main` (P31 S72)** — jump-table pads derived at build time from the retail island + the emission stream (trailing pads `0t1`, table-aware, const data passes through). No stored spec; an anchor miss fails the build with the offset. **S72:** `--derive` now serves main too — `_file0_vram` returns the code segment's `vram - start` (the PS-X EXE's 0x800 header), which makes both `raw[a - vram]` and `vram + <yaml offset>` correct for the EXE and leaves flat overlays byte-identical; `_splat_yaml` resolves main to `splat.us.exe.yaml`. |
|
||||
| | `config/wave_exclude.txt` | **(P31 S72)** THE canonical wave exclude list — **tracked**, and **regenerated, never hand-edited**: `tools/exclude_audit.py config/wave_exclude.txt --write <new>`. Consumed as `draw_waves --exclude-file config/wave_exclude.txt`, which AUDITS it as a PREREQUISITE and refuses to draw on a stale one (`--exclude-stale-ok` overrides, loudly). Two classes: **CARVE-BLOCKED** (derived from `split_indicator`; disappears when the subseg is split — **EMPTY since P31 S74**, all four overlays split) and **WALL** (curated, cannot be re-derived — the `# WALL:` annotation is a PIN that survives regeneration, and its note is the refutation list to beat before reopening the entry). It replaces the nine `.run/S*_exclude.txt` snapshots and seven walls ledgers, none of which was authoritative; measured on its predecessor, 88 of 107 entries were stale one day after it was written, 46 of them open drawable work totalling 12,750 instructions. |
|
||||
|
||||
+11
-4
@@ -2,7 +2,7 @@
|
||||
|
||||
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
|
||||
>
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1110 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
> `docs/matching-cookbook.md` is ~716 KB / 1112 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
|
||||
|
||||
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
|
||||
|
||||
@@ -429,7 +429,7 @@
|
||||
- **§418** — ★★★ — TWO LOOP-STRUCTURE LEVERS: MAKE THE SECOND INDEX A GIV, AND KEEP A TABLE ADDRESS UNFOLDED (P31 S71; byte-proven `ov_SC04_016/func_8017DF8C`, 184 ins, 32 → 0 in seven compiles) <sub>L33632</sub>
|
||||
- **§430** — ★★★ — A GOTO INTO A LOOP IS FINE; HAND-HOIST THE CONSTANTS IT COSTS YOU (P31 S73 — **this section previously said the OPPOSITE and was wrong; the refutation is kept below**) <sub>L34073</sub>
|
||||
|
||||
### structs, block moves & memcpy (88)
|
||||
### structs, block moves & memcpy (89)
|
||||
|
||||
- **§3-T2** — Source statement order drives instruction scheduling <sub>L78</sub>
|
||||
- **§5** — Known hard-residual classes (instruction-identical, one byte-exact blocker) <sub>L199</sub>
|
||||
@@ -519,6 +519,7 @@
|
||||
- **§395** — ★★★ — FIVE NARROWING/PLACEMENT LEVERS FROM ONE 91-INSTRUCTION CRACK (P31 S69; byte-proven ov_SC06_018/func_80189E60, warm start 32 off → MATCH 91/91) <sub>L32611</sub>
|
||||
- **§418** — ★★★ — TWO LOOP-STRUCTURE LEVERS: MAKE THE SECOND INDEX A GIV, AND KEEP A TABLE ADDRESS UNFOLDED (P31 S71; byte-proven `ov_SC04_016/func_8017DF8C`, 184 ins, 32 → 0 in seven compiles) <sub>L33632</sub>
|
||||
- **§425** — ★★★ — `sb` ALIASES SCALAR GLOBALS WHILE `sh`/`sw` STRUCT STORES DO NOT, AND TWO MORE ALIAS/BOOST RULES (P31 S71; `md_MAIN_003/func_800CF3E8`, 467 of 469 ins, all four byte-verified from `-dS`/`-dR`/`-dl`/`-dr`) <sub>L33829</sub>
|
||||
- **§442** — ★★★ — A RECOVERY RUNG THAT REWRITES A LEGAL CONSTRUCT INTO AN ILLEGAL ONE READS EXACTLY LIKE A CODEGEN WALL (P31 S75; `reconcile_tu`, 344 ins unblocked) <sub>L34647</sub>
|
||||
|
||||
### types, signedness & load/store width (95)
|
||||
|
||||
@@ -954,7 +955,7 @@
|
||||
- **§420** — ★★★ — A MULTI-CLUSTER SYMBOL REBASE, AND THE BARE-NAME DEDUP THAT HID THREE QUARTERS OF IT (P31 S71; 4 banked in 57 s) <sub>L33692</sub>
|
||||
- **§421** — ★★★ — A `la $tN` + `addiu` PAIR CAN BE A **RELOAD** ARTIFACT THAT NO C SPELLING REACHES (P31 S71; byte-proven `md_SC07_003/func_801A293C`, 313 ins, 6 → 0) <sub>L33729</sub>
|
||||
|
||||
### integration / TU plumbing (81)
|
||||
### integration / TU plumbing (82)
|
||||
|
||||
- **§8c** — Splitting a TU means rebuilding its DECLARATION ENVIRONMENT, not moving text (Phase 26 session 6) <sub>L456</sub>
|
||||
- **§8d** — Templating a body INTO a TU must not CHANGE its declaration environment — demote the carried data externs (Phase 26 session 8, byte-proven on `func_8015AE2C` ×133) <sub>L502</sub>
|
||||
@@ -1037,6 +1038,7 @@
|
||||
- **§435** — ★★★ — AN OVERLAY TU SPLIT IS NEAR-FREE, AND THE GAP TEST IS "IS THIS WORD A CODE ADDRESS", NOT "IS IT ZERO" (P31 S74; four overlays split, all byte-identical, CARVE-BLOCKED class emptied fleet-wide) <sub>L34307</sub>
|
||||
- **§437** — ★★★ — A WRAPPED TRAILING COMMENT BROKE THE TU PARSER, AND THE §8b DECL LAYER WROTE `extern #define` (P31 S74; five independently-MATCHed jr bodies unblocked, `ov_SC06_029` byte-identical) <sub>L34421</sub>
|
||||
- **§440** — ★★★ — A `.rodata` CARVE PIECE BINDS TO A SUBSEG, NOT TO A FUNCTION: EXTEND THE CARVE INSTEAD OF ISOLATING (P31 S74; 6 banked, resident included) <sub>L34576</sub>
|
||||
- **§442** — ★★★ — A RECOVERY RUNG THAT REWRITES A LEGAL CONSTRUCT INTO AN ILLEGAL ONE READS EXACTLY LIKE A CODEGEN WALL (P31 S75; `reconcile_tu`, 344 ins unblocked) <sub>L34647</sub>
|
||||
|
||||
### build graph, splat & the harness (196)
|
||||
|
||||
@@ -1375,7 +1377,7 @@
|
||||
- **§430** — ★★★ — A GOTO INTO A LOOP IS FINE; HAND-HOIST THE CONSTANTS IT COSTS YOU (P31 S73 — **this section previously said the OPPOSITE and was wrong; the refutation is kept below**) <sub>L34073</sub>
|
||||
- **§439** — ★★ — LEVER SET FROM THE S74 WAVE (each entry is one measured crack, not a hypothesis) <sub>L34555</sub>
|
||||
|
||||
### (unbucketed — title matched no symptom vocabulary) (329)
|
||||
### (unbucketed — title matched no symptom vocabulary) (330)
|
||||
|
||||
- **§3-How** — to use this <sub>L30</sub>
|
||||
- **§1** — Idiom catalog (asm pattern → C that produces it) <sub>L39</sub>
|
||||
@@ -1706,6 +1708,7 @@
|
||||
- **§434** — ★★★ — TWO SYMBOLS, ONE FRAME: RUN THE FRAME CHECK BEFORE DRAFTING ANYTHING LARGE (P31 S73; `main/SaveLoadRoutine` + `func_8002B0B4`, byte-verified) <sub>L34259</sub>
|
||||
- **§438** — ★★★ — THE SAME-ADDRESS LEAD IS NOW SIZE-FILTERED: A HOMONYM IS WORSE THAN NO TWIN (P31 S74; ~12 of ~60 cards carried one) <sub>L34528</sub>
|
||||
- **§441** — ★★ — THREE MORE INSTRUMENT DEFECTS FROM THE SAME SESSION, ALL OF WHICH BLAME THE SUBJECT (P31 S74) <sub>L34617</sub>
|
||||
- **§443** — ★★ — A DERIVED DEPENDENCY MUST BE PROVISIONED BY *EVERY* PROVISIONER, AND WE HAVE TWO (P31 S69 → S74, the same defect twice) <sub>L34702</sub>
|
||||
|
||||
|
||||
## All sections, in order
|
||||
@@ -2820,6 +2823,8 @@
|
||||
- **§439** — ★★ — LEVER SET FROM THE S74 WAVE (each entry is one measured crack, not a hypothesis) <sub>L34555</sub>
|
||||
- **§440** — ★★★ — A `.rodata` CARVE PIECE BINDS TO A SUBSEG, NOT TO A FUNCTION: EXTEND THE CARVE INSTEAD OF ISOLATING (P31 S74; 6 banked, resident included) <sub>L34576</sub>
|
||||
- **§441** — ★★ — THREE MORE INSTRUMENT DEFECTS FROM THE SAME SESSION, ALL OF WHICH BLAME THE SUBJECT (P31 S74) <sub>L34617</sub>
|
||||
- **§442** — ★★★ — A RECOVERY RUNG THAT REWRITES A LEGAL CONSTRUCT INTO AN ILLEGAL ONE READS EXACTLY LIKE A CODEGEN WALL (P31 S75; `reconcile_tu`, 344 ins unblocked) <sub>L34647</sub>
|
||||
- **§443** — ★★ — A DERIVED DEPENDENCY MUST BE PROVISIONED BY *EVERY* PROVISIONER, AND WE HAVE TWO (P31 S69 → S74, the same defect twice) <sub>L34702</sub>
|
||||
|
||||
|
||||
---
|
||||
@@ -3942,3 +3947,5 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
|
||||
| L34555 | §439 | ★★ — LEVER SET FROM THE S74 WAVE (each entry is one measured crack, not a hypothesis) |
|
||||
| L34576 | §440 | ★★★ — A `.rodata` CARVE PIECE BINDS TO A SUBSEG, NOT TO A FUNCTION: EXTEND THE CARVE INSTE |
|
||||
| L34617 | §441 | ★★ — THREE MORE INSTRUMENT DEFECTS FROM THE SAME SESSION, ALL OF WHICH BLAME THE SUBJECT ( |
|
||||
| L34647 | §442 | ★★★ — A RECOVERY RUNG THAT REWRITES A LEGAL CONSTRUCT INTO AN ILLEGAL ONE READS EXACTLY LI |
|
||||
| L34702 | §443 | ★★ — A DERIVED DEPENDENCY MUST BE PROVISIONED BY *EVERY* PROVISIONER, AND WE HAVE TWO (P31 |
|
||||
|
||||
@@ -34643,3 +34643,85 @@ where no verbatim asm is in play it equals the cut and nothing changes.
|
||||
**The habit these three share with §436 and §437:** the tool reported something TRUE about a world
|
||||
that was not the one it was asked about, and the true-sounding message pointed at the draft. When a
|
||||
gate rejects a body you have byte-verified standalone, the first suspect is the gate.
|
||||
|
||||
## §442 ★★★ — A RECOVERY RUNG THAT REWRITES A LEGAL CONSTRUCT INTO AN ILLEGAL ONE READS EXACTLY LIKE A CODEGEN WALL (P31 S75; `reconcile_tu`, 344 ins unblocked)
|
||||
|
||||
S74 measured the same nine drafts gating **9/9 through `harvest_verify` and 7/9 through
|
||||
`gate_stage`** and correctly named `reconcile_tu.py` (the `-rc` rung) as the difference — but the
|
||||
diagnosis stopped at "it manufactures declaration conflicts". Running it to ground gave three
|
||||
separate defects, and the load-bearing one is a **false premise about cc1 written into the tool's
|
||||
own docstring.**
|
||||
|
||||
**THE PREMISE, AND THE MEASUREMENT THAT REFUTES IT.** `reconcile_tu` conforms a draft's data decl to
|
||||
the TU's whenever the TU declares the same name, in EITHER direction, on the stated ground that "a
|
||||
decl BELOW still conflicts". Four probes against the real `cc1` (`cdecl._cc1_accepts`, the same
|
||||
oracle `cdecl.compatible` was validated with — R33, do not build a second one):
|
||||
|
||||
| draft's decl | TU's decl | cc1 |
|
||||
|---|---|---|
|
||||
| **block** scope `extern Blk D_x;` | file scope, **BELOW** the splice point | **ACCEPT** — *warning:* `type mismatch with previous external decl` |
|
||||
| **block** scope | file scope, **ABOVE** | REJECT `conflicting types` |
|
||||
| **file** scope | file scope, **BELOW** | REJECT `conflicting types` ← the case the rung genuinely fixes |
|
||||
| the rung's own OUTPUT | — | REJECT **`syntax error before 'D_x'`** |
|
||||
|
||||
So the premise is true at file scope and **false at block scope** — and that is not a missed
|
||||
optimisation, it is destructive: the TU's declaration names the TU's TYPE, and a type declared below
|
||||
the splice point **is not in scope at it**, so "conforming" swaps a construct cc1 compiles for one it
|
||||
cannot parse. Byte-witnessed on `resident:func_800D06E8` (344 ins, `match_one` closeness 0): a
|
||||
deliberate block-scoped `extern Blk80078E78 D_80078E78;` rewritten to `extern Struct80078E78
|
||||
D_80078E78;` whose typedef is declared **388 lines lower**.
|
||||
|
||||
The block-scope extern is not sloppiness to be tidied away either — **it is what this same ladder's
|
||||
`scope_demote_drafts` (§8d) rung produces on purpose**, and three already-banked functions in that TU
|
||||
use it. One rung was undoing another's work. Fix: `if inner and d.name not in above: leave it alone`,
|
||||
with a note saying so (R43 — refuse the class and name it).
|
||||
|
||||
**TWO MORE IN THE SAME PASS.**
|
||||
|
||||
**A. The cast pass rewrote COMMENT PROSE.** It looped over lines, skipped only ones starting with
|
||||
`extern`/`typedef`, and `re.sub`'d the rest — so a 40-line header comment explaining the block-scope
|
||||
idiom had `D_80078E78` rewritten to `(*(Blk80078E78 *)&D_80078E78)` eight times, *inside a quoted cc1
|
||||
diagnostic*. Harmless to the bytes, corrosive to the only artifact that explains why the function is
|
||||
written that way, and it makes a mechanical rewrite look authored. Fix: match on `cdecl._mask`
|
||||
(length-preserving comment/string blanking, so a mask offset is a source offset) and splice into the
|
||||
original — the masker already exists for brace counting.
|
||||
|
||||
**B. `&sym` emitted `&` applied to a CAST.** The regex captured a leading `&` and re-emitted it
|
||||
verbatim in front of the value form. For the scalar arm `&(*(E *)&sym)` is legal and merely ugly;
|
||||
for the other three it is a hard error — measured `invalid lvalue in unary '&'` on `&((E *)&sym)`.
|
||||
Latent for as long as the array/fnptr arms have existed. Fix: `&` SELECTS a pointer form built from
|
||||
the symbol's address (`((E *)&sym)`, `((E (*)[])&sym)`, `((E (**)(P))&sym)`) and consumes itself.
|
||||
|
||||
**THE HABIT.** Stage 0 gates raw drafts first, so a ladder rung can only cost a RECOVERY — which is
|
||||
exactly why a broken one is invisible: the function it destroys was already failing, so its DIFF is
|
||||
read as a fact about the function. `gate_stage` now takes `--skip-stages`/`GATE_SKIP_STAGES` (loud
|
||||
when used), so the next bad rung costs a flag instead of a session. **A verdict from a ladder is a
|
||||
verdict from the ladder** (R40): re-run a stubborn reject through bare `harvest_verify` before you
|
||||
believe anything about the C.
|
||||
|
||||
## §443 ★★ — A DERIVED DEPENDENCY MUST BE PROVISIONED BY *EVERY* PROVISIONER, AND WE HAVE TWO (P31 S69 → S74, the same defect twice)
|
||||
|
||||
`.run/sig.<bin>.jsonl` is gitignored, derived and read-only — so a git worktree never has it, and
|
||||
`family_remap.nins_of` raises `FileNotFoundError` for every function in it. S69 fixed that in
|
||||
`parallel_gate.stage_generated`, per-binary, and wrote it up (§322b). **S74 hit the identical defect
|
||||
in `verify_worktree.provision`** — the *other* worktree provisioner, which nothing linked to the
|
||||
first: main clone 259 sig files, provisioned worktree 0.
|
||||
|
||||
The consequence is the §436 shape at full strength. `jr_isolate_all`'s carve-ownership scan wrapped
|
||||
`reloc_targets` in `try/except Exception: continue` — right for one unwalkable function, catastrophic
|
||||
when the cause is environmental, because then **every** call raises, the scan finds zero owners, and
|
||||
the R32 "every carve resolves to exactly one owner" assertion downstream fires and reports carve
|
||||
CORRUPTION. Measured: **2,603 of 2,603 functions raised, 0 owners, a confident verdict about damage
|
||||
that did not exist.** R54 exactly — a guard downstream of the failure is not a guard.
|
||||
|
||||
Three lessons, in order of reusability:
|
||||
|
||||
1. **Separate the environmental failure from the per-item one.** `FileNotFoundError` on a derived
|
||||
index is a fact about the CHECKOUT, not the subject: abort at the cause, naming the regeneration
|
||||
command, instead of being re-read 100 lines later as evidence.
|
||||
2. **Count attempted vs raised and assert it (R32).** A scan where *everything* raised measured
|
||||
nothing, whatever the exception was, and its zero is an artifact — not a finding.
|
||||
3. **Enumerate provisioners, not fixes.** The S69 write-up fixed a tool; the defect belonged to a
|
||||
CLASS ("untracked derived deps a worktree needs"), and the class had two members. Both now
|
||||
cross-reference each other; add a dependency to one and you must add it to the other. The two
|
||||
are deliberately NOT merged — `parallel_gate` provisions one binary, `verify_worktree` all 213.
|
||||
|
||||
@@ -6635,3 +6635,61 @@ change IS a tool change, and owes the same docs).
|
||||
TU would have destroyed it. Re-gate against HEAD with the fixed tools instead.
|
||||
* **Say the denominator.** "20 waves of 3" was impossible — the pool held 33 undrawn, 4 of them free
|
||||
remaps. Saying so beat drafting 27 functions that already had drafts.
|
||||
|
||||
## S75 progress (2026-09-02) — instrument fixes, and what they uncovered
|
||||
|
||||
- **T10/S75-1 — `reconcile_tu.py` (the S74 "one unfixed defect") ROOT-CAUSED AND FIXED.** S74 named
|
||||
the rung correctly ("manufactures declaration conflicts") but stopped there. Running it to ground
|
||||
gave THREE defects, and the load-bearing one is a **false premise about cc1 written into the tool's
|
||||
own docstring**. Four probes on the real front end (`cdecl._cc1_accepts`, R33 — the same oracle
|
||||
`cdecl.compatible` was validated with):
|
||||
`block-scope extern vs a TU decl BELOW` → **ACCEPT** (pedwarn `type mismatch with previous external
|
||||
decl`) · `block-scope vs ABOVE` → REJECT · `file-scope vs BELOW` → REJECT (the case the rung really
|
||||
fixes) · **the rung's own output** → REJECT `syntax error before 'D_x'`. So "a decl BELOW still
|
||||
conflicts" is true at file scope and FALSE at block scope — and conforming it is destructive,
|
||||
because the TU's decl names the TU's TYPE and a type declared below the splice point is not in
|
||||
scope at it. Byte-witnessed on `resident:func_800D06E8`: a deliberate block-scoped
|
||||
`extern Blk80078E78` rewritten to `extern Struct80078E78`, typedef 388 lines lower. It is also
|
||||
the construct this ladder's OWN `scope_demote_drafts` (§8d) rung emits on purpose, used by three
|
||||
already-banked functions in that TU — **one rung undoing another's work.** Also fixed: the cast
|
||||
pass rewrote COMMENT PROSE (8 rewrites inside one header comment, including inside a quoted cc1
|
||||
diagnostic), and `&sym` emitted `&` applied to a cast (`invalid lvalue in unary '&'`, measured, for
|
||||
the array/fnptr arms). `gate_stage` gained **`--skip-stages`/`GATE_SKIP_STAGES`** (loud) so the next
|
||||
bad rung costs a flag, not a session. Cookbook **§442**, SETUP rows, index regenerated.
|
||||
- **R39 negative control caught a regression in my own fix.** Over the stored-draft corpus: `&sym[i]`
|
||||
is not `&` applied to the symbol — `[]` binds tighter, so `&((E *)&sym)[i]` is legal and correct,
|
||||
and the fold silently changed what it MEANT (`ov_SC02_005:func_8018DFC4`). The pointer form now
|
||||
applies only when no subscript follows. Final control: **661 adjudicated, 652 IDENTICAL, 9 CHANGED,
|
||||
every one an intended class** (4,173 of 4,864 drafts unadjudicable — filenames that are not
|
||||
`func_<ADDR>`, so the harness cannot resolve their binary; stated rather than hidden, R41).
|
||||
- **T10/S75-2 — the S74 worktree harness gap FIXED, and it was the SAME defect twice.**
|
||||
`verify_worktree.provision` now symlinks every `.run/sig.*.jsonl` (main clone 259, provisioned
|
||||
worktree **0**) — the third member of the class that already contains `extracted/` and
|
||||
`.run/obj40`. `parallel_gate.stage_generated` had been fixed for this identical missing-sig bug in
|
||||
**S69**; two worktree provisioners, no shared list, so it had to be found twice. They now
|
||||
cross-reference each other. At the other end `jr_isolate_all`'s ownership scan swallowed the
|
||||
resulting `FileNotFoundError` in a bare `except: continue` — **2,603 of 2,603 functions raised, 0
|
||||
owners found, and the downstream R32 assert reported carve CORRUPTION that did not exist** (R54).
|
||||
Now aborts at the cause naming `make sig-all`, plus `_assert_scan_covered` (attempted == raised ⇒
|
||||
the scan measured nothing, so its zero is an artifact). NC: `jr_isolate_all ov_SC03_105 --dry-run`
|
||||
unchanged in the main tree. Cookbook **§443**.
|
||||
- **THREE FINDINGS FROM RUNNING ONE REJECT TO GROUND** (`resident:func_800D06E8`, 344 ins) —
|
||||
none of them the thing I set out to fix:
|
||||
1. **The gate's jtbl prep MUTATES `config/overlays.mk` AND DOES NOT REVERT IT.** After the
|
||||
rejected gate the file had a 4th `JTBL_PADS` entry and had **lost `--pre hdr.rodata.o`** — the
|
||||
resident's §440 leading-rodata sandwich. The binary then would not build at all
|
||||
(`consumed 3 rodata jump table(s) but 4 pad spec(s) given`) with `src/` perfectly clean, so
|
||||
`git status` on `src/` says nothing is wrong. This damages SHARED COMMITTED CARVE STATE and is
|
||||
more serious than the defect I was chasing. Recovered with `git checkout config/overlays.mk` +
|
||||
a clean `make clean/extract/build BINARY=resident` → `8e17e02f… BYTE-IDENTICAL`.
|
||||
2. **The classified ledger records the LADDER'S FINAL verdict, not the stage-0 one.** The recorded
|
||||
`CC1-FAIL: syntax error before 'D_80078E78'` came from a LATE stage (`sig_unify` hoisting the
|
||||
file-scope type into the block — exactly what the draft's own header warns about). The RAW
|
||||
draft compiles fine and fails on BYTES. A verdict class attributed to the draft was a fact
|
||||
about a transform (R40/R47).
|
||||
3. **The function's real blocker is a JUMP TABLE.** With the raw draft spliced, the built binary is
|
||||
20 bytes longer and `0x800CEDFC` holds `800D01F4, 800D0204, 800D0214 …` — a 4th rodata table;
|
||||
everything downstream shifts (69,571 differing words, 332 inside the function). `match_one`
|
||||
MATCH (344 ins) and `rtu_match` MATCH (344 ins, only the predicted pedwarn) both pass, because
|
||||
both share a `%lo` mask and neither links. **Standalone match ≠ bankable**, again: this needs
|
||||
the §8b/§440 carve, not a better body.
|
||||
|
||||
@@ -58,6 +58,28 @@ def _check_sha(binary):
|
||||
return open(p).read().split()[0] if os.path.exists(p) else None
|
||||
|
||||
|
||||
def _skipped_stages():
|
||||
"""Ladder rungs the operator has switched OFF, by tool name (with or without `.py`).
|
||||
|
||||
Set `GATE_SKIP_STAGES=reconcile_tu,sig_unify` (or pass `--skip-stages`). WHY this exists (P31
|
||||
S75): a recovery rung that REGRESSES a correct draft is indistinguishable, from the outside,
|
||||
from a codegen wall — the gate says DIFF either way, and the verdict gets recorded against the
|
||||
FUNCTION. S74 measured the same nine drafts gating 9/9 through `harvest_verify` and 7/9 through
|
||||
this ladder, and the two casualties were a rung's doing. Fixing that rung is the right answer
|
||||
and was done; this hatch is so the NEXT one costs a flag instead of a session (R43 — a stage you
|
||||
cannot turn off is a stage you have to be right about forever).
|
||||
|
||||
A skip is LOUD: it prints, because a silently-disabled recovery rung is the R32 defect class."""
|
||||
raw = os.environ.get("GATE_SKIP_STAGES", "")
|
||||
names = {s.strip().removesuffix(".py") for s in raw.split(",") if s.strip()}
|
||||
if names:
|
||||
print(f"[gate] ladder rungs SKIPPED by request: {', '.join(sorted(names))}", file=sys.stderr)
|
||||
return names
|
||||
|
||||
|
||||
SKIP = set()
|
||||
|
||||
|
||||
def _xform(tool, ov, indir, suffix, extra=None):
|
||||
"""Run a draft-dir transform; return its out dir, or the in dir if the tool no-ops/fails.
|
||||
|
||||
@@ -70,6 +92,8 @@ def _xform(tool, ov, indir, suffix, extra=None):
|
||||
run banked a function it was never asked to try and would have committed it under a message
|
||||
naming a different one. A stage that silently widens its own input set is the same defect class
|
||||
as a scanner that silently narrows it (R32): in both, the report and the work diverge."""
|
||||
if tool.removesuffix(".py") in SKIP:
|
||||
return indir # switched off by --skip-stages / GATE_SKIP_STAGES
|
||||
out = indir + suffix
|
||||
shutil.rmtree(os.path.join(REPO, out), ignore_errors=True)
|
||||
cmd = [PY, f"tools/{tool}", "--overlay", ov, "--in", indir, "--out", out] + (extra or [])
|
||||
@@ -197,6 +221,8 @@ def run_gate(drafts, binary=OV, src=None, asm=None, out=None, good_sha=None,
|
||||
# and sig_unify canonicalize against THAT file's decls (via --src-file; sig_unify.py:151 reads the
|
||||
# split file's stubs so split-file drafts are NOT dropped). dedup_propagate is already split-aware.
|
||||
# Default None = main .c.
|
||||
global SKIP
|
||||
SKIP = _skipped_stages() # read per RUN, not at import (env may be set late)
|
||||
os.makedirs(os.path.join(REPO, ".run/auto"), exist_ok=True)
|
||||
_lock = open(os.path.join(REPO, lock_path or ".run/auto/gate.lock"), "w")
|
||||
fcntl.flock(_lock, fcntl.LOCK_EX)
|
||||
@@ -648,7 +674,14 @@ def main():
|
||||
help="where harvest_verify writes the verified list (default: per-binary, "
|
||||
".run/harvest_verified.<binary>.txt — do not share across concurrent gates)")
|
||||
ap.add_argument("--failed-out", default=None)
|
||||
ap.add_argument("--skip-stages", default=None,
|
||||
help="comma-separated ladder rungs to switch OFF by tool name (e.g. "
|
||||
"reconcile_tu,sig_unify). Also settable as GATE_SKIP_STAGES. Stage 0 gates "
|
||||
"the RAW drafts first either way, so skipping a rung can only cost a "
|
||||
"RECOVERY, never a correct draft.")
|
||||
a = ap.parse_args()
|
||||
if a.skip_stages:
|
||||
os.environ["GATE_SKIP_STAGES"] = a.skip_stages
|
||||
b = a.binary
|
||||
# `src` RESTRICTS the byte-gate to ONE TU. Defaulting it to the main .c silently pinned the gate
|
||||
# to 13 of 263 stubs (4.9%): A3 taught harvest_verify to derive each draft's home TU *when --src
|
||||
|
||||
+49
-6
@@ -32,12 +32,55 @@ import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import overlay_src_split as oss
|
||||
|
||||
|
||||
import mk_write as MKW # atomic, collapse-refusing overlays.mk writer (P31 S60)
|
||||
|
||||
REPO = oss.REPO
|
||||
O0_SUFFIX = ("_o0", "_o0b")
|
||||
|
||||
|
||||
_SCAN = {"attempted": 0, "raised": 0}
|
||||
|
||||
|
||||
def _reloc_targets_or_die(family_remap, ov, addr, data=None):
|
||||
"""`family_remap.reloc_targets`, with the ENVIRONMENTAL failure separated from the per-function one.
|
||||
|
||||
Both callers used to wrap this in `try/except Exception: continue`. That is right for a single
|
||||
function whose relocations cannot be walked, and catastrophically wrong for a missing
|
||||
`.run/sig.<bin>.jsonl` — because then EVERY call raises, the scan finds zero owners, and the R32
|
||||
"every committed carve resolves to exactly one owner" assertion downstream fires and reports
|
||||
carve CORRUPTION. Measured P31 S74 inside a provisioned worktree: 2,603 of 2,603 functions
|
||||
raised, 0 owners found, run aborted with a confident verdict about damage that did not exist.
|
||||
|
||||
A missing sig index is not a fact about the carves; it is a fact about the checkout (R40 —
|
||||
exonerate the instrument). So it aborts here, at the cause, naming the fix — rather than being
|
||||
re-interpreted 100 lines later as evidence about the subject (R54).
|
||||
|
||||
`_SCAN` counts attempted vs raised: a scan in which EVERYTHING raised measured nothing, whatever
|
||||
the exception was, and may not be reported as a result (R32 — assert your coverage)."""
|
||||
_SCAN["attempted"] += 1
|
||||
try:
|
||||
return family_remap.reloc_targets(ov, addr, data=data) if data is not None \
|
||||
else family_remap.reloc_targets(ov, addr)
|
||||
except FileNotFoundError as e:
|
||||
sys.exit(f"jr_isolate_all({ov}): cannot walk relocations — {e}. This is a MISSING INDEX, "
|
||||
f"not a carve defect: run `make sig-all` (or, in a worktree, provision "
|
||||
f".run/sig.*.jsonl — tools/verify_worktree.py does this). Refusing to report "
|
||||
f"ownership from a scan that could not read the image index (R43).")
|
||||
except Exception:
|
||||
_SCAN["raised"] += 1
|
||||
return None
|
||||
|
||||
|
||||
def _assert_scan_covered(ov, what):
|
||||
"""Fail loud if the ownership scan raised on EVERYTHING it attempted (R32)."""
|
||||
a, r = _SCAN["attempted"], _SCAN["raised"]
|
||||
if a and r == a:
|
||||
sys.exit(f"jr_isolate_all({ov}): {what} attempted {a} function(s) and EVERY ONE raised — "
|
||||
f"the scan measured nothing, so its 0 owners are an artifact, not a finding (R32).")
|
||||
|
||||
|
||||
def sh(cmd):
|
||||
return subprocess.run(cmd, shell=True, cwd=REPO, capture_output=True, text=True)
|
||||
|
||||
@@ -137,9 +180,8 @@ def jr_inventory(ov):
|
||||
for addr, name, kind, _ in items:
|
||||
if kind not in ("def", "define") or not name or addr is None:
|
||||
continue
|
||||
try:
|
||||
targets = family_remap.reloc_targets(ov, addr, data=img)
|
||||
except Exception:
|
||||
targets = _reloc_targets_or_die(family_remap, ov, addr, data=img)
|
||||
if targets is None:
|
||||
continue
|
||||
hits = {t - base for k, t in targets if k == "data" and (t - base) in carve_offs}
|
||||
if hits:
|
||||
@@ -147,6 +189,7 @@ def jr_inventory(ov):
|
||||
for off in hits:
|
||||
owners.setdefault(off, []).append(name)
|
||||
|
||||
_assert_scan_covered(ov, "jr_inventory ownership scan")
|
||||
# R32: every committed carve resolves to EXACTLY ONE owner, or abort loud.
|
||||
#
|
||||
# OWNERSHIP HAS TWO MORE SOURCES THAN THE RELOC SCAN ABOVE (P31 S70). The scan finds an owner
|
||||
@@ -262,13 +305,13 @@ def carve_owners(ov, banked, base, carve_offs):
|
||||
import family_remap
|
||||
owners = {}
|
||||
for addr, fn in banked.items():
|
||||
try:
|
||||
targets = family_remap.reloc_targets(ov, addr)
|
||||
except Exception:
|
||||
targets = _reloc_targets_or_die(family_remap, ov, addr)
|
||||
if targets is None:
|
||||
continue
|
||||
for kind, t in targets:
|
||||
if kind == "data" and (t - base) in carve_offs:
|
||||
owners[t - base] = fn
|
||||
_assert_scan_covered(ov, "carve_owners")
|
||||
return owners
|
||||
|
||||
|
||||
|
||||
@@ -170,7 +170,13 @@ def stage_generated(wt, binary):
|
||||
sp = os.path.join(a_src, f)
|
||||
if os.path.isfile(sp):
|
||||
shutil.copy2(sp, os.path.join(a_dst, f))
|
||||
# THE SIGNATURE REGISTRY (P31 S69, Fable-3). `jr_isolate_all.jr_inventory` resolves every
|
||||
# THE SIGNATURE REGISTRY (P31 S69, Fable-3). SIBLING: `verify_worktree.provision` step 3c does
|
||||
# the same job for the OTHER worktree provisioner, fleet-wide rather than per-binary — it hit
|
||||
# this identical defect independently in S74, five sessions after this fix (2,603/2,603
|
||||
# functions raised, 0 carve owners, a FALSE corruption verdict). Two provisioners, no shared
|
||||
# list, so the fix had to be made twice: if you add a derived dependency here, add it there too
|
||||
# (cookbook §442).
|
||||
# `jr_isolate_all.jr_inventory` resolves every
|
||||
# committed `.rodata` carve's owner through `family_remap.reloc_targets`, whose `nins_of`
|
||||
# reads the gitignored `.run/sig.<binary>.jsonl`. A fresh worktree has no `.run/sig.*`, so
|
||||
# inside a worker EVERY carve reads UNOWNED, jr_inventory R32-aborts, harvest_verify prints
|
||||
|
||||
+104
-16
@@ -29,6 +29,11 @@ WHAT THIS DOES INSTEAD
|
||||
load-bearing (%lo-folding, access width, alignment all key off the declared type), and
|
||||
scope_data_externs (§8d) will demote it to block scope so it establishes no global that the TU's
|
||||
own later block-scope externs would have to agree with.
|
||||
* A visible declaration that is BELOW the splice point while the draft's is at BLOCK SCOPE ->
|
||||
cc1 ACCEPTS that (pedwarn "type mismatch with previous external decl"), so there is nothing to
|
||||
fix and everything to lose: the TU's declaration names the TU's TYPE, and a type declared below
|
||||
the splice point is not in scope AT it, so "conforming" replaces a construct cc1 compiles with
|
||||
one it cannot parse. LEAVE IT ALONE (R43). Measured on resident:func_800D06E8, P31 S75.
|
||||
* A visible, COMPATIBLE declaration -> nothing to do.
|
||||
* A visible, CONFLICTING declaration -> **the TU wins** (it is the environment; we are the guest).
|
||||
Rewrite the draft's decl to the TU's, and CAST AT EVERY USE so the access the draft intended is
|
||||
@@ -97,7 +102,27 @@ def _cast_sub(d, tu):
|
||||
NOTE the fn-ptr arms. reconcile_decls' `data_access_subs` has no fn-ptr kind, so the moment its
|
||||
parser is taught to SEE `extern void (*D_x[])(void);` it would happily rewrite a call-through
|
||||
`D_x[i]()` into `((u8 *)D_x)[i]()` — a dormant transform that fixing the parser would ARM. This
|
||||
tool handles the kind natively, which is why it supersedes that one rather than patching it."""
|
||||
tool handles the kind natively, which is why it supersedes that one rather than patching it.
|
||||
|
||||
TWO FORMS, and the `&` prefix SELECTS between them (P31 S75). The leading `&` used to be
|
||||
captured and re-emitted VERBATIM in front of the value form, which yields `&` applied to a cast
|
||||
— and for three of the four arms that is not merely ugly, it is a hard error. Measured on the
|
||||
real cc1:
|
||||
|
||||
&(*(E *)&sym) scalar/struct/ptr ACCEPT (legal; the fold is cosmetic)
|
||||
&((E *)&sym) array REJECT `invalid lvalue in unary '&'`
|
||||
&((E (*)(P))sym) fnptr REJECT same
|
||||
&((E (**)(P))&sym) fnptr_array REJECT same
|
||||
|
||||
So `&sym` now emits a POINTER form built from the symbol's address directly, which is valid for
|
||||
every arm, is the same link-time constant, and keeps the drafted view's pointee type.
|
||||
|
||||
⚠️ ONLY WHEN NO SUBSCRIPT FOLLOWS. `&sym[i]` is not `&` applied to the symbol at all — `[]`
|
||||
binds tighter, so it is the address of ELEMENT i, and `&((E *)&sym)[i]` is both legal and
|
||||
correct. Folding it to a pointer form silently changes what the expression MEANS. Caught by the
|
||||
R39 negative control over the stored-draft corpus (`ov_SC02_005:func_8018DFC4`, whose
|
||||
`(s32)&((Pair8 *)D_801E4998)[idx]` would have become `(s32)((Pair8 (*)[])D_801E4998)[idx]`) —
|
||||
the trailing-`[` case is exactly the one the old code got RIGHT."""
|
||||
sym = d.name
|
||||
# `(?<![.\w])(?<!->)` — a MEMBER ACCESS is not this symbol. Without it, a struct field that
|
||||
# happens to share a global's name is rewritten at every use: `p->code` became
|
||||
@@ -106,16 +131,29 @@ def _cast_sub(d, tu):
|
||||
rx = re.compile(rf'(&?)(?<![.\w])(?<!->){re.escape(sym)}\b(\s*\[)?')
|
||||
c_arr = tu.kind in ('array', 'fnptr_array')
|
||||
e = _elem(d)
|
||||
p = ", ".join(d.params or [])
|
||||
at = sym if c_arr else f'&{sym}' # the symbol's ADDRESS under TU storage
|
||||
|
||||
if d.kind == 'fnptr': # draft: void (*D_x)(P)
|
||||
acc = f'(({e} (*)({", ".join(d.params or [])})){sym})'
|
||||
val = f'(({e} (*)({p})){sym})'
|
||||
ptr = f'(({e} (**)({p})){at})'
|
||||
elif d.kind == 'fnptr_array': # draft: void (*D_x[])(P)
|
||||
acc = f'(({e} (**)({", ".join(d.params or [])})){"" if c_arr else "&"}{sym})'
|
||||
val = f'(({e} (**)({p})){at})'
|
||||
ptr = f'(({e} (***)({p})){at})'
|
||||
elif d.kind == 'array': # draft: E D_x[]
|
||||
acc = f'(({e} *){"" if c_arr else "&"}{sym})'
|
||||
val = f'(({e} *){at})'
|
||||
ptr = f'(({e} (*)[]){at})'
|
||||
else: # scalar / struct / pointer
|
||||
acc = f'(*({e} *)&{sym})' if not c_arr else f'(*({e} *){sym})'
|
||||
return rx, lambda m: f'{m.group(1)}{acc}{m.group(2) or ""}'
|
||||
val = f'(*({e} *){at})'
|
||||
ptr = f'(({e} *){at})'
|
||||
# group(1) = a leading `&`, group(2) = a trailing `[`. The pointer form is right ONLY for a bare
|
||||
# `&sym`; with a subscript the `&` belongs to the element, so re-emit it in front of the value
|
||||
# form exactly as before (see the docstring's ⚠️).
|
||||
def rep(m):
|
||||
if m.group(1) and not m.group(2):
|
||||
return ptr # `&sym` -> pointer form, `&` consumed
|
||||
return f'{m.group(1)}{val}{m.group(2) or ""}' # `sym`, `sym[i]`, `&sym[i]` -> unchanged shape
|
||||
return rx, rep
|
||||
|
||||
|
||||
def _draft_statements(body):
|
||||
@@ -160,7 +198,10 @@ def _draft_statements(body):
|
||||
|
||||
def fix(body, tu_path, fn):
|
||||
"""Conform the draft's DATA decls to the TU. Returns (new_body, notes)."""
|
||||
full = cdecl.tu_scope(tu_path) # CONFLICT domain: a decl BELOW still conflicts
|
||||
# CONFLICT domain. A decl BELOW the splice point still conflicts — AT FILE SCOPE. At BLOCK
|
||||
# scope cc1 accepts it with a pedwarn, which is why `above` is consulted again in the loop
|
||||
# rather than only for argument order (P31 S75; the four-probe measurement is quoted there).
|
||||
full = cdecl.tu_scope(tu_path)
|
||||
above = set(cdecl.tu_scope(tu_path, above=fn)) # ORDER, for cc1's no-prototype rule
|
||||
|
||||
plan, notes = {}, []
|
||||
@@ -188,6 +229,33 @@ def fix(body, tu_path, fn):
|
||||
tu = full.get(d.name)
|
||||
if tu is None:
|
||||
continue # not declared here -> no conflict is possible
|
||||
# BLOCK SCOPE + TU DECLARATION BELOW -> cc1 ACCEPTS. Do not touch it. (P31 S75)
|
||||
#
|
||||
# This module's own premise — "a decl BELOW still conflicts" (see `full` above) — is
|
||||
# TRUE at file scope and FALSE at block scope, and the difference is not cosmetic. The
|
||||
# four-probe measurement against the real cc1:
|
||||
#
|
||||
# block-scope extern, TU decl BELOW -> ACCEPT (warning: "type mismatch with
|
||||
# previous external decl" — a pedwarn, not an error)
|
||||
# block-scope extern, TU decl ABOVE -> REJECT `conflicting types`
|
||||
# file-scope extern, TU decl BELOW -> REJECT `conflicting types` <- the real job
|
||||
# the conformed output this rung emitted -> REJECT `syntax error before 'D_x'`
|
||||
#
|
||||
# That last line is why this is a refusal and not an optimisation. The TU's declaration
|
||||
# names the TU's TYPE, and a type declared BELOW the splice point is not in scope AT it:
|
||||
# conforming to it substitutes a construct cc1 accepts for one it cannot even parse.
|
||||
# Byte-witnessed on resident:func_800D06E8 (344 ins, match_one closeness 0) — a
|
||||
# deliberate block-scoped `extern Blk80078E78 D_80078E78;` rewritten to
|
||||
# `extern Struct80078E78 D_80078E78;` whose typedef is declared 388 lines LOWER.
|
||||
#
|
||||
# The block-scope extern is not an accident to be cleaned up, either: it is what THIS
|
||||
# LADDER'S OWN `scope_demote_drafts` (§8d) rung produces on purpose, and three already-
|
||||
# banked functions in that TU use it. R43 — refuse the class, name it, leave the draft.
|
||||
if inner and d.name not in above:
|
||||
notes.append(f'-- {d.name}: block-scope extern vs a TU decl BELOW the splice point '
|
||||
f'-> cc1 accepts (pedwarn only); LEFT ALONE (conforming it would name '
|
||||
f'a type that is not yet in scope)')
|
||||
continue
|
||||
a, b = (tu, d) if d.name in above else (d, tu)
|
||||
if cdecl.compatible(a, b):
|
||||
continue # cc1 accepts both -> leave the draft's types
|
||||
@@ -255,16 +323,36 @@ def fix(body, tu_path, fn):
|
||||
notes.append(f'!! {nm}: planned conform to {tu.type!r} did NOT land in the output '
|
||||
f'(its uses would be cast against a declaration that was never rewritten)')
|
||||
|
||||
out = []
|
||||
for line in conformed.split('\n'):
|
||||
if re.match(r'\s*(extern|typedef)\b', line):
|
||||
out.append(line) # never cast inside a declaration line
|
||||
# Cast every USE to the draft's intended view — but ONLY in CODE.
|
||||
#
|
||||
# This pass used to run `rx.sub` over every non-declaration LINE, which is every line of the
|
||||
# draft's doc comment too. Byte-witnessed on resident:func_800D06E8 (P31 S75): a 40-line header
|
||||
# comment explaining the block-scope idiom had `D_80078E78` rewritten to
|
||||
# `(*(Blk80078E78 *)&D_80078E78)` in its PROSE, eight times, including inside a quoted cc1
|
||||
# diagnostic. Harmless to the bytes and corrosive to the only artifact that explains WHY the
|
||||
# function is written the way it is — and it made a mechanical rewrite look like an authored one.
|
||||
#
|
||||
# `cdecl._mask` blanks comments and string/char literals LENGTH-PRESERVINGLY, so an offset into
|
||||
# the mask is an offset into the original: match on the mask, splice into the source (R33 — the
|
||||
# masker already exists and is the same one `split_statements` trusts for brace counting).
|
||||
masked = cdecl._mask(conformed)
|
||||
edits = []
|
||||
for _s, (tu, d, _st) in plan.items():
|
||||
rx, rep = _cast_sub(d, tu)
|
||||
for m in rx.finditer(masked):
|
||||
ls = conformed.rfind('\n', 0, m.start()) + 1
|
||||
le = conformed.find('\n', m.start())
|
||||
if re.match(r'\s*(extern|typedef)\b', conformed[ls:le if le >= 0 else len(conformed)]):
|
||||
continue # never cast inside a declaration line
|
||||
edits.append((m.start(), m.end(), rep(m)))
|
||||
pieces, pos = [], 0
|
||||
for a, b, text in sorted(edits):
|
||||
if a < pos:
|
||||
notes.append(f'!! overlapping cast rewrites at {a}; the later one was skipped')
|
||||
continue
|
||||
for _s, (tu, d, _st) in plan.items(): # cast every USE to the draft's intended view
|
||||
rx, rep = _cast_sub(d, tu)
|
||||
line = rx.sub(rep, line)
|
||||
out.append(line)
|
||||
return '\n'.join(out), notes
|
||||
pieces.append(conformed[pos:a]); pieces.append(text); pos = b
|
||||
pieces.append(conformed[pos:])
|
||||
return ''.join(pieces), notes
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
@@ -39,6 +39,7 @@ USAGE
|
||||
tools/verify_worktree.py --keep # leave the worktree in place for inspection
|
||||
"""
|
||||
import argparse
|
||||
import glob
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
@@ -153,6 +154,32 @@ def provision(wt, prov):
|
||||
os.symlink(src_o40, o40)
|
||||
prov["obj40"] = "symlinked -> main clone (untracked, SDK-derived PsyQ ELF objects)"
|
||||
|
||||
# 3c. .run/sig.<bin>.jsonl — the per-binary function signature index.
|
||||
#
|
||||
# THIRD instance of the same class (extracted/ 3, obj40 3b, this): gitignored because it is
|
||||
# derived, deterministic, regenerable (`make sig-all`), and READ-ONLY during a run. It is what
|
||||
# `family_remap.nins_of/reloc_targets` reads to know a function's length, so ANY tool that walks
|
||||
# relocations inside a worktree needs it.
|
||||
#
|
||||
# Measured P31 S74, and it produced a confident FALSE verdict: main clone 259 sig files,
|
||||
# provisioned worktree 0. `jr_isolate_all`'s carve-ownership scan calls `reloc_targets` per
|
||||
# function inside a `try/except: continue`, so all 2,603 of them raised FileNotFoundError, the
|
||||
# scan found 0 owners, and the R32 "every carve resolves to exactly one owner" assertion fired —
|
||||
# reporting carve CORRUPTION in a tree that had none. The swallow is fixed at that end too
|
||||
# (R54: a guard downstream of the failure is not a guard); this end removes the cause.
|
||||
sigs = sorted(glob.glob(os.path.join(REPO, ".run", "sig.*.jsonl")))
|
||||
if not sigs:
|
||||
return (".run/sig.*.jsonl absent in the main clone — run `make sig-all` first "
|
||||
"(a worktree run without them reports FALSE carve corruption)")
|
||||
os.makedirs(os.path.join(wt, ".run"), exist_ok=True)
|
||||
linked = 0
|
||||
for s in sigs:
|
||||
dst = os.path.join(wt, ".run", os.path.basename(s))
|
||||
if not os.path.exists(dst):
|
||||
os.symlink(s, dst)
|
||||
linked += 1
|
||||
prov["sigs"] = f"symlinked {linked} of {len(sigs)} sig.*.jsonl -> main clone (untracked, derived)"
|
||||
|
||||
# 4. maspsx — a pinned submodule. Record the gitlink the COMMIT expects and what we provide.
|
||||
want_sm = None
|
||||
r = run(["git", "ls-tree", "HEAD", "tools/maspsx"], cwd=wt, quiet=True)
|
||||
|
||||
Reference in New Issue
Block a user