fix(phase-26a): A3e — gate_stage pinned the byte-gate back to 4.9%, OF A3'S OWN FIX

THE WORST DEFECT IN THE AUDIT IS NOT IN A SCANNER. It is one default argument in the CALLER of a
scanner we had already fixed.

    # tools/gate_stage.py:315
    summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...)   # ALWAYS the main .c

`src` RESTRICTS the byte-gate to ONE translation unit, and _gate1 does `if src: cmd += ["--src", src]`
-- always truthy. A3 had just taught harvest_verify to DERIVE each draft's home TU *when --src is
omitted*, lifting the byte-gate's reach from 4.9% to 100%. gate_stage NEVER OMITS IT. The fix was
neutralised by its own caller's default, and the PRIMARY BANKING PATH -- every wave, the grinder, the
orchestrator, bulk_harvest -- remained structurally unable to bank 250 of ov_SC01_077's 263 stubs.

WHY IT SURVIVED 26 PHASES: harvest_verify cannot splice a draft whose stub is not in the TU it was
pointed at, so the draft never verifies -- and is then logged as near/failed, i.e. AS A MATCHING
PROBLEM. The wave reports a poor close-rate; the function goes to the backlog as a compiler residual.

    A tool that CANNOT bank a function is indistinguishable, in every log this project keeps,
    from a function that CANNOT BE banked.

PROOF, same draft / same gate / same second: gate_stage rejected func_80129C40; harvest_verify run
directly (no --src) VERIFIED it byte-identical and banked it.

AND A COUNTING BUG THAT HID THE HIDING (gate_stage:261): when match_one says MATCH but the whole-binary
gate rejects, the record is logged status="near" and THE COUNTER IS NEVER INCREMENTED. A 63-draft run
printed `banked 0, near 0, failed 0` -- three zeros that do not sum to 63 -- for phases. Nobody ever
added them up. (The number was not wrong. It was ABSENT.)

ALSO FIXED, sig_unify (the same disease, one level down): it SILENTLY DROPPED 190 of 196 drafts (97%).
`cur_stubs` was read from the main .c (13 of 263 stubs), so any draft whose stub lives in a _jr_ carve
hit `if fn not in cur_stubs: continue` -- dropped BEFORE THE WRITE: never copied to --out, never gated,
never logged, while the summary printed "drafts unified: 6" and read like success. THIS IS GATE_STAGE'S
STAGE-2 RECOVERY -- the pass whose whole job is to rescue the stage-1 failures -- and it has been a
no-op for nearly every draft it was meant to save. Now: TU derived per draft (corpus.stubs), canon
derived from cdecl.tu_scope (cpp -- macro-injected decls finally visible), and _keep() so an
already-acceptable decl is left alone (the §19 "sig_unify regresses canonical drafts" failure mode).
Reach: 6 -> 196 drafts; callee-externs rewritten 2 -> 90; own def-sig 2 -> 86.

MEASURED, all three consumers migrated (196 never-banked drafts):
    near   5 -> 116        failed  190 -> 17
=> 173 of 190 "failures" were PLUMBING, not codegen: now compiling and SCORED instead of invisible.

THE PRIZE (measured, not claimed): the backlog holds 1,588 entries at closeness==0 -- body byte-exact
per match_one, whole-binary gate rejected. 1,215 have been banked since by other paths. 373 ARE STILL
OPEN STUBS WHOSE BODIES ARE ALREADY BYTE-EXACT, sitting in a ledger that calls them unrecoverable.

⚠ THE HARVEST ITSELF IS NOT IN THIS COMMIT, AND IS NOT CLAIMED (P9). Gating the 63 ov_SC01_077 ones
dragged `dedup_propagate --auto-from --recover` behind it; it ran >1h and hit its timeout -- its
first-ever run over the FULL corpus (A6/A7 unblocked the 407 files it could never see). It MUTATES THE
TREE BEFORE IT GATES, so the kill left 859 files + engine_core.h (+544 lines) written and UN-GATED with
the registry never updated. R22 on that tree: 44 passed / 92 FAILED -> `git checkout -- src/ config/`,
fleet restored to 136/136. Nothing lost (H4: the tree was clean, so the revert was one command).
Two real lessons, recorded: dedup_propagate is NOT crash-safe and must never run under a timeout it can
hit; and a 63-draft experiment must not drag an unbounded fleet-wide propagation behind it.

  R22 clean-fleet after revert: 136 passed, 0 failed of 136.  src/ and config/ clean.
  cookbook §51g LAW 11: A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT. After fixing a
  scanner, grep every call site and ask whether a caller's default re-disables it. An audit that stops
  at the callee is half an audit.
This commit is contained in:
Drew T
2026-07-14 15:39:07 -06:00
parent 4aae5e7589
commit b89fcc2edc
5 changed files with 174 additions and 25 deletions
+31
View File
@@ -3724,3 +3724,34 @@ from **8 → 58 of 196** drafts (7× reach).
> the difference between an **invisible failure that reads as a compiler wall** and a **scored near-miss
> the permuter and the §47/§48 dials can act on.** Which is the audit's thesis exactly. Do not sell it as
> more than it is; three times in one session a confirmed mechanism produced a null consequence.
**LAW 11 — A FIX IS NOT LANDED UNTIL ITS CALLER STOPS OVERRIDING IT.** The single worst defect in the
audit was not in a scanner. It was one default argument in the caller of a scanner we had *already fixed*:
```python
# tools/gate_stage.py:315
summary = run_gate(a.drafts, binary=b, src=a.src or f"src/{b}/{b}.c", ...) # <- ALWAYS the main .c
```
`src` **restricts the byte-gate to ONE translation unit.** A3 had just taught `harvest_verify` to derive
each draft's home TU *when `--src` is omitted*, lifting the byte-gate's reach from **4.9% to 100%** — and
**`gate_stage` never omits it.** So the primary banking path (every wave, the grinder, the orchestrator,
`bulk_harvest`) stayed structurally incapable of banking **250 of 263 stubs**, *after the fix*, *because of
its own caller's default*.
> **AND HERE IS WHY IT SURVIVED 26 PHASES.** `harvest_verify` cannot splice a draft whose stub is not in
> the TU it was pointed at, so the draft simply never verifies — and is then logged as `near`/`failed`,
> i.e. **as a matching problem**. The wave reports a low close-rate and the function goes to the backlog
> as a residual.
> **A tool that CANNOT bank a function is indistinguishable, in every log this project keeps, from a
> function that CANNOT BE banked.**
> Proof, same draft, same gate, same second: `gate_stage` rejected `func_80129C40`; `harvest_verify` run
> directly (no `--src`) **verified it byte-identical and banked it.**
**And a counting bug that hid the hiding:** when `match_one` says MATCH but the whole-binary gate rejects,
`gate_stage` logs `status="near"` and **never increments the counter**. A run of 63 such drafts printed
`banked 0, near 0, failed 0` — *three zeros that do not sum to 63* — for phases. **Nobody added them up.**
(LAW 2 again, and note the shape: the number was not wrong, it was ABSENT.)
**Checklist item, promoted to the top:** after fixing a scanner, `grep` every call site and ask *does any
caller pass a default that re-disables this?* An audit that stops at the callee is half an audit.
+19
View File
@@ -726,6 +726,25 @@ scanners** — the "best outcome is a deleted scanner" rule (R33), applied at sc
3. **A recovery tool once wrote non-C into drafts:** `extern if ((func_80029178(0x119) & 0xFF) != 0);` appears in 33 drafts, *only* in `-canon`/`-recanon`/`-uni`/`-sigfix` output dirs (the pre-recovery draft has none), and gcc rejects it outright. **R14 blast-radius check: the source bug was already fixed in Phase 19** — today's oracle emits **0 garbage over 300 signatures** — so this is dead historical residue, not a live defect. *Mechanism confirmed, consequence nil.* But note what it cost at the time: a draft that cannot compile fails the byte-gate and reads, downstream, as **an intrinsic compiler wall**.
4. **`tu_ambient`'s function regex drops any callee with a fn-ptr parameter.** Its param class is `[^()]*`, so `extern void func_8012A568(void (*a0)(void));` — a real declaration in ov_SC01_077 — lands in **no bucket at all**: not funcs, not data, not typedefs.
### 🔴 A3e — THE ONE THAT MATTERS: `gate_stage` pinned the byte-gate back to 4.9% — *of its own fix*
**`tools/gate_stage.py:315`**
```python
summary = run_gate(a.drafts, binary=b,
src=a.src or f"src/{b}/{b}.c", # <- ALWAYS the main .c
```
`src` **restricts the byte-gate to ONE translation unit.** `_gate1` then does `if src: cmd += ["--src", src]` — and `src` is *always* truthy.
**A3 fixed `harvest_verify` to derive each draft's home TU *when `--src` is omitted*, taking the byte-gate's reach from 4.9% to 100%. `gate_stage` never omits it.** The fix was silently neutralised **by its own caller**, and `gate_stage` — the *primary banking path*, the thing every wave, the grinder, the orchestrator and `bulk_harvest` call — remained structurally incapable of banking **250 of ov_SC01_077's 263 stubs (95%)**.
**How it hid.** `harvest_verify` cannot splice a draft whose stub is not in the TU it was pointed at, so those drafts simply never verify. They are then logged as `near`/`failed` — i.e. *as matching problems* — and the wave reports a low close-rate. **A tool that cannot bank a function is indistinguishable, in every log this project keeps, from a function that cannot be banked.** Proof: `func_80129C40` — `gate_stage` rejected it; `harvest_verify` run directly (no `--src`) **VERIFIED it byte-identical and banked it.** Same draft, same gate, same second.
**And a counting bug that hid the hiding** (`gate_stage:261`): when `match_one` says MATCH but the whole-binary gate rejects, the record is logged `status="near"` and **the counter is never incremented**. A run of 63 such drafts printed `banked 0, near 0, failed 0` — three zeros that do not sum to 63, printed for phases, **and nobody ever added them up.**
**The backlog says how big this is.** 1,588 entries at `closeness == 0` (body byte-exact per `match_one`, whole-binary gate rejected). 1,215 have been banked since by other paths — leaving **373 still-open stubs whose bodies are already byte-exact**, sitting in a backlog that describes them as unrecoverable.
### A3c — the first consumer migration (`cast_call_sites`), and what it measured
| | |
+41
View File
@@ -659,6 +659,47 @@ On approval → `/model opus` + `/effort xHigh` (Tasks 0–4; ALL Fable5 via `Ag
## Log
- **2026-07-14 (session 10, A3e — 🔴 THE BIG ONE: `gate_stage` pinned the byte-gate back to 4.9% — *of A3's own
fix*; Max):** Migrating `sig_unify` exposed a chain, and the last link is the worst defect in the whole audit.
**(1) `sig_unify` silently DROPPED 190 of 196 drafts (97%).** `cur_stubs` was read from the main `.c` (13
stubs of 263), so any draft whose stub lives in a `_jr_*` carve hit `if fn not in cur_stubs: continue` —
dropped **before the write**: never copied to `--out`, never gated, never logged. The summary printed *"drafts
unified: 6"* and read like success. **This is gate_stage's STAGE-2 RECOVERY** — the pass whose entire job is
to rescue stage-1 failures — **and it has been a no-op for almost every draft it was meant to save.** Fixed by
deriving the TU per draft (`corpus.stubs`) + the canon from `cdecl.tu_scope` (cpp, so macro-injected decls are
visible) + `_keep()` (only rewrite when cc1 would actually reject — the §19 "sig_unify regresses canonical
drafts" failure mode). Reach: **6 → 196 drafts; callee-externs rewritten 2 → 90; own def-sig 2 → 86.**
With all three consumers migrated the 196-draft failure profile went **near 5 → 116, failed 190 → 17**:
**173 of 190 "failures" were PLUMBING, not codegen** — now compiling and *scored* instead of invisible.
**(2) 🔴 `gate_stage.py:315`: `src=a.src or f"src/{b}/{b}.c"`.** `src` **RESTRICTS the byte-gate to ONE TU**,
and `_gate1` does `if src: cmd += ["--src", src]` — always truthy. **A3 had just taught `harvest_verify` to
derive each draft's home TU when `--src` is OMITTED (reach 4.9% → 100%), and `gate_stage` never omits it.**
The fix was neutralised **by its own caller's default**, leaving the *primary banking path* (every wave, the
grinder, the orchestrator, `bulk_harvest`) structurally unable to bank **250 of 263 stubs**.
**WHY IT SURVIVED 26 PHASES:** a draft whose stub isn't in the pointed-at TU simply never verifies → it is
logged `near`/`failed`, i.e. **as a MATCHING problem** → the wave reports a poor close-rate → the function
goes to the backlog as a compiler residual. **A tool that CANNOT bank a function is indistinguishable, in
every log this project keeps, from a function that CANNOT BE banked.** Proof, same draft/gate/second:
`gate_stage` rejected `func_80129C40`; **`harvest_verify` run directly (no `--src`) VERIFIED it
byte-identical and banked it.** **(3)** And a counting bug hid the hiding (`gate_stage:261`): `match_one`
MATCH + gate-reject logs `status="near"` and **never increments the counter** — a 63-draft run printed
`banked 0, near 0, failed 0`, *three zeros that do not sum to 63*, for phases, and **nobody added them up**.
**THE PRIZE, MEASURED:** the backlog holds **1,588** `closeness==0` entries (body byte-exact per `match_one`,
gate rejected). 1,215 were banked since by other paths; **373 are STILL OPEN STUBS whose bodies are already
byte-exact**, sitting in a ledger that calls them unrecoverable.
**⚠ THE HARVEST ATTEMPT FAILED AND WAS REVERTED (P9).** Gating the 63 ov_SC01_077 ones dragged
`dedup_propagate --auto-from --recover` behind it, which ran **>1 h and hit its timeout** — its first-ever run
over the FULL corpus (A6/A7 unblocked the 407 files it could never see). It **mutates the tree BEFORE it
gates**, so the kill left **859 files + engine_core.h (+544 lines) written and UN-GATED, and the registry
never updated**. R22 clean-fleet on it: **44 passed / 92 FAILED** → **`git checkout -- src/ config/`**, fleet
restored. **Nothing was lost (H4: the tree was clean, so the revert was one command) — but two real lessons:
`dedup_propagate` is NOT crash-safe and must never run under a timeout it can hit; and a 63-draft experiment
must not drag an unbounded fleet-wide propagation behind it (gate with `propagate=False`, then propagate as
its own bounded job).**
**Committed: the TOOL FIXES only. The harvest is NOT banked and is not claimed.** Knowledge captured live
(R30/R31): cookbook **§51g LAW 11** (*a fix is not landed until its caller stops overriding it* — after
fixing a scanner, grep every call site and ask whether a caller's default re-disables it), tooling-audit
**A3e**. **NEXT: re-run the 373-stub harvest with `propagate=False`, then propagate separately.**
- **2026-07-14 (session 10, A3d — RETIRE the fleet-majority oracle; both banking paths; Max):** R33 applied to
the audit's worst finding: **not fixed, RETIRED.** `reconcile_decls` asks *"what does the FLEET call this
symbol?"*; C asks *"what does THIS TU declare?"*. The engine is loosely typed, so one fleet-wide answer is
+8 -1
View File
@@ -259,6 +259,9 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
note = sm.group(1).strip() if sm else None
if kind == "match": # match_one says MATCH but the whole-binary gate rejected -> plumbing/TU conflict
status, where = "near", note or "match_one MATCH but gate rejected (declaration/TU plumbing)"
near += 1 # …and COUNT it. It was logged as `near` and counted as NOTHING, so a
# run of 63 such drafts printed "banked 0, near 0, failed 0" — three
# zeros that do not sum to 63, and nobody ever added them up (R32).
elif kind == "near":
status, where = "near", note or f"{meta.get('lever') or meta.get('class') or 'residual'}: {close} mismatch"
near += 1
@@ -311,8 +314,12 @@ def main():
ap.add_argument("--commit", action="store_true")
a = ap.parse_args()
b = a.binary
# `src` RESTRICTS the byte-gate to ONE TU. Defaulting it to the main .c silently pinned the gate
# to 13 of 263 stubs (4.9%): A3 taught harvest_verify to derive each draft's home TU *when --src
# is omitted* — and this caller never omitted it, so the fix was neutralised by its own caller.
# Now: pass --src ONLY when a human explicitly restricts the gate. Omitted => derive per draft.
summary = run_gate(a.drafts, binary=b,
src=a.src or f"src/{b}/{b}.c",
src=a.src,
asm=a.asm_subdir or f"asm/{b}/nonmatchings/{b}",
out=a.out or f"build/{b}/{b}",
good_sha=a.good_sha or DEF_SHA,
+75 -24
View File
@@ -145,6 +145,53 @@ def rewrite_def(txt, fn, canon):
return txt[:m.start()] + new_hdr + txt[m.end():], True
_cdecl = _load('cdecl', 'tools/cdecl.py') if '_load' in dir() else None
def _tu_for(overlay, fn):
"""The TU that holds this function's stub — DERIVED (corpus.stubs); None if it is not an open
stub anywhere (already banked, or not ours)."""
try:
import importlib.util
spec = importlib.util.spec_from_file_location('corpus', os.path.join(REPO, 'tools/corpus.py'))
c = importlib.util.module_from_spec(spec)
spec.loader.exec_module(c)
st = c.stubs(overlay).get(int(fn[5:], 16))
return os.path.join(REPO, st.path) if st else None
except Exception:
return os.path.join(REPO, f'src/{overlay}/{overlay}.c')
def _canon_for(tu):
"""{symbol: 'extern <canonical decl>;'} — DERIVED from what cc1 sees in THIS TU (cpp), so the
macro-injected `DEFINE_func_*` externs are finally visible. The old union of three raw-text
scanners (collect_define_sigs + collect_inline_sigs + collect_extern_sigs) could not see them,
and its DATA_DECL_RE was blind to fn-ptr / sized-array / multi-declarator decls (44 of 646
symbols in this very corpus) — so the recovery pass silently did nothing for exactly the symbols
that were failing."""
import cdecl as cd
out = {}
for name, d in cd.tu_scope(tu).items():
if d.storage == 'typedef':
continue
out[name] = d.declaration(storage='extern')
return out
def _keep(draft_line, canon_decl):
"""Leave the draft's decl alone when cc1 would accept it beside the TU's. sig_unify's known
failure mode is REGRESSING already-correct drafts (§19/§25: 'canon-first, sig_unify FALLBACK'),
and a rewrite that changes nothing semantic can still perturb codegen. Rewrite only when the
front end would actually reject the pair."""
try:
import cdecl as cd
a = cd.parse(canon_decl)[0]
b = cd.parse(draft_line.strip())[0]
return cd.compatible(a, b)
except Exception:
return False
def main():
ap = argparse.ArgumentParser()
ap.add_argument('--overlay', default='ov_SC01_077')
@@ -157,38 +204,41 @@ def main():
ap.add_argument('--out', dest='outdir', required=True)
args = ap.parse_args()
c_path = os.path.join(REPO, args.src_file) if args.src_file else \
os.path.join(REPO, f'src/{args.overlay}/{args.overlay}.c')
ec = os.path.join(REPO, 'src/shared/engine_core.h')
fdef = _ght.collect_define_sigs(ec)
fdef.update(_ght.collect_inline_sigs(c_path))
fdecl = _ght.collect_extern_sigs([ec, c_path])
func_canon = {}
for a, s in fdecl.items():
func_canon[f'func_{a:08X}'] = f'extern {s};'
for a, s in fdef.items():
func_canon[f'func_{a:08X}'] = f'extern {s};'
data_canon = collect_data_decls([c_path, ec])
canon = {**func_canon, **data_canon}
override = os.path.join(REPO, args.src_file) if args.src_file else None
os.makedirs(os.path.join(REPO, args.outdir), exist_ok=True)
stub_re = re.compile(r'INCLUDE_ASM\([^,]*,\s*(func_[0-9A-Fa-f]+)\)')
cur_stubs = set(stub_re.findall(open(c_path).read()))
n = ext_rw = def_rw = 0
n = ext_rw = def_rw = passthru = 0
_canon_cache = {}
for p in sorted(glob.glob(os.path.join(REPO, args.indir, '*.c'))):
fn = os.path.basename(p)[:-2]
if fn not in cur_stubs:
continue
txt = open(p).read()
# WHICH TU? DERIVED (Phase 26-A, §51g LAW 10). `--src-file` was an optional hand-passed flag
# defaulting to src/<ov>/<ov>.c, and `cur_stubs` was read from THAT file — so a draft whose
# stub lives in a Phase-26 `_jr_<ADDR>` carve was not in the set and hit
# if fn not in cur_stubs: continue
# which dropped it BEFORE THE WRITE. It never reached --out, was never gated, was never
# logged; the summary just printed a smaller "drafts unified" and read like success.
# MEASURED: 190 of 196 drafts (97%) silently vanished — and this is gate_stage's STAGE-2
# RECOVERY, the pass whose whole job is to rescue the stage-1 failures. It has been a no-op
# for almost every draft it was meant to save.
# corpus.stubs() knows which TU holds each stub (the INCLUDE_ASM line is self-describing),
# so the derivation IS the check and the silent drop is now structurally impossible.
tu = override or _tu_for(args.overlay, fn)
if tu is None: # not an open stub anywhere -> nothing to unify
open(os.path.join(REPO, args.outdir, os.path.basename(p)), 'w').write(txt)
passthru += 1 # PASSED THROUGH and COUNTED, never dropped
continue
if tu not in _canon_cache:
_canon_cache[tu] = _canon_for(tu)
canon = _canon_cache[tu]
ext_changed = False
def repl(m):
nonlocal ext_changed
line = m.group(0)
s = sym_of(line)
if s and s in canon:
if s and s in canon and not _keep(line, canon[s]):
new = canon[s]
if re.sub(r'\s+', ' ', new).strip() != re.sub(r'\s+', ' ', line).strip():
ext_changed = True
@@ -201,7 +251,7 @@ def main():
nonlocal ext_changed
line = m.group(0)
s = sym_of(line)
if s and s != fn and s in canon:
if s and s != fn and s in canon and not _keep(line, canon[s]):
new = canon[s]
if re.sub(r'\s+', ' ', new).strip() != re.sub(r'\s+', ' ', line).strip():
ext_changed = True
@@ -215,8 +265,9 @@ def main():
ext_rw += ext_changed
def_rw += def_changed
print(f'canonical decls: {len(func_canon)} func + {len(data_canon)} data')
print(f'drafts unified: {n} (callee-externs rewritten in {ext_rw}, OWN def-sig rewritten in {def_rw})')
print(f'drafts in: {n + passthru}; unified: {n}; passed through (not an open stub): {passthru}; '
f'TUs: {len(_canon_cache)}')
print(f' callee-externs rewritten in {ext_rw}, OWN def-sig rewritten in {def_rw}')
if __name__ == '__main__':