diff --git a/docs/matching-cookbook.md b/docs/matching-cookbook.md index 8200398448..7ae0fc9102 100644 --- a/docs/matching-cookbook.md +++ b/docs/matching-cookbook.md @@ -8073,3 +8073,35 @@ Note this is the complement of `header_sig_map()`, which reads the `extern` decl its own callees*. Two different macro-derived signature sources; a symbol in neither is a real gap. **Blast radius: 0** beyond this family — like §118 and §120, and unlike §117, a **targeted** lever. + +## §122 — GATE RAW BEFORE TRANSFORMING; the undo belongs to the WRITER, as a per-edit journal (P30 T0a, 2026-07-30) + +**The defect pair this closes** (carried from SESSION-22, reproducible): (1) the `gate_stage` ladder +FAILED drafts that bare `harvest_verify` VERIFIED — `func_8013B6A0`/`func_8013B598` (`_o0`) and +`func_80138C60` (jr split), `rtu_match` confirming real-TU MATCHes. Every casualty lives in a +**split TU**; the plain-TU draft banked through the same run. Root-cause hypothesis (still open, now +harmless): the transforms take ONE batch-wide `--src-file` while the gate derives each draft's home +TU per-draft (Phase 26-A) — a mixed-TU batch gets decls reconciled against the wrong TU. +(2) A bare-gate workflow left `fix_arity_callers --any-proto` residue in **17 unrelated TUs** — the +ladder's snapshot/undo existed only inside the ladder. + +**Law 1 — stage 0: gate the RAW drafts before ANY transform.** A recovery ladder's transforms are +for drafts that FAIL as written; running them on everything lets a "recovery" regress a byte-correct +draft, and the gate then reports the regression as the draft's failure. With stage 0, the +destroyed-good-draft mode is impossible *by construction* — no root-cause required first (the +right sequencing under R35: neutralize, then diagnose). `GATE_NO_STAGE0` restores the old order. + +**Law 2 — undo is the WRITER'S job, recorded per edit, not the orchestrator's file snapshot.** +The snapshot needed two measured special cases (restore-shared-only on a partial bank because a full +restore reverts fresh splices, Task-14; restore-everything on a zero-bank run, §61) because a +file-level restore cannot tell the pre-pass's edits from the gate's splices. A per-edit journal +(`fix_arity_callers --journal` / `--undo-journal --keep `) round-trips each substitution's +literal text: exact by construction, immune to interleaved splices, uniform for shared+local files, +and available to EVERY workflow — ladder or bare. A decl someone else edited since is reported +MISSING loudly (R32), never silently skipped. Negative-control-proven: apply→undo → byte-identical +tree; `--keep` retains exactly the banked set. Bonus closed: the undo now runs AFTER stage 2, so a +stage-2 bank no longer loses its arity edit before its own gate attempt (the old latent parity gap). + +**Generalizes to:** any pipeline where deterministic "fixers" precede a truth gate — the gate goes +first on untouched input, fixers touch only failures, and every shared-state fixer journals its own +writes. (Same family as §19/§25 canon-first and the §61 undo law; this entry is their composition.) diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md new file mode 100644 index 0000000000..8cfdb4e26d --- /dev/null +++ b/phase-ends/CURRENT_PHASE.md @@ -0,0 +1,101 @@ +# CURRENT PHASE — Phase 30: The Recovery & Concentration Campaign (overlays to their ceiling) + +**Opened:** 2026-07-30 · **Effort:** Max (Fable 5; saved defaults) · **Generation:** Gen2 (22nd phase of the arc) · +**Baseline:** PhaseEnd_Phase29 (v1.28.0) · **Roadmap:** `docs/roadmap-to-100.md` **v2** §3 P30 +**Plan approved (P3 gate 1):** 2026-07-30, in-session (Drew's explicit waiver of the fresh-session step at 28% context). +Full plan mirror: `~/.claude/plans/continue-dazzling-newt.md` (out-of-repo; this file is the in-repo authority + crash-recovery log). + +**Fleet at open:** 87.5% instr-weighted / 78.0% distinct-code / 92.00% fn-count · 140/140 +byte-identical · 0 NON_MATCHING · dedup 1886/0 · tools-health RC=0. + +**Drew's Phase-Start decisions (2026-07-30):** +- In-session start (fresh-session step waived at 28% context). +- The Ghidra-C prefetch batch pulled forward from P31 → **T0.5** (tail + main's 1,034 stubs; headless, background). +- T2's primary path = the **two-file atomic o0b substitution driver** (the log's costed no-splat route); Arm-A splat research is the fallback only. +- ROI floor for T3: two consecutive sessions each < +0.3pp instr with all lanes exercised (Drew-adjustable). + +**⚠️ PRECONDITION (R6):** Drew makes the milestone-close commit + push (`PhaseEnd_Phase29.md` + +`phase-ends/logs/Phase29.md`, both in the tree) **before P30's first commit lands**. P30 work +proceeds; the first P30 commit waits on it. + +--- + +## The numbers this phase must pin FIRST (T0 — do not consume until re-derived, R35/R14) +- family_hseq **29,961** vs progress.py **28,296** remaining instances — an unexplained R32 gap. +- The `-O0` population: "10 families / ~1,287 raw distinct" (checkpoints) vs "123 families / + 121,264 ins / 3,100 distinct" (the log's R14 correction) vs "129 distinct on the table" (the + costed o0b route) — three numbers, one T0 job. +- The zero-crack family list (map said **61** on 07-29) and the concentration table (top-20/top-100 shares). + +## Standing invariants (every task) +- Whole-binary byte-gate is the **sole arbiter** (G3/P9); match_one/closeness/masked = candidates (§52b). +- **R22 clean-fleet per banked batch:** `make clean && make extract-all && make check-all` → 140/140. +- `make tools-health` green + fail-closed before matching (corpus · cdecl · audit-binaries(R36) · report/lint/dedup). +- Blast-radius typing on every integration write (T0 draft-only / T1 binary-local / T2 fleet-shared ⇒ R22 mandatory). +- **Probe before costing** (R37-candidate discipline): probe one member before pricing a job; derive + attribution from `corpus.stubs` before/after; diff the artifact to prove an edit ran (§120). +- Flywheel: idioms → cookbook + tooling in-session (R16/R30); pivots → decision-log (R31). +- Effort/model transitions **prompted, never assumed** (R26/R27) — STOP + WAIT for the toggle. +- One commit per task/sub-repair after this file's update; Drew pushes (R6/R20). R23 no-op on db.*.gbf churn. + +--- + +## Task checklist (current-task pointer = ▶) + +- [ ] ▶ **T0 — Frontier regen + instrument repair [Max]** — one commit per lettered repair: + (a) `gate_stage.py` snapshot/restore (the ladder destroys good drafts) · + (b) `rtu_match.py` surface the real cc1 error in the verdict · + (c) reconcile the family_hseq↔progress instance gap (R32) · + (d) purge since-banked rows from the backlog ledger + still-a-stub filter from `corpus.stubs` (R33) · + (e) refresh `.run/autopsy/residuals.jsonl` + fix the 21-file absolute-include defect · + (f) regenerate the frontier (report / family_hseq / worklist / fuel manifest) + **pin the three + contested populations** → the one-page frontier report (**report point #1**). + Verify: tools-health green; digests committed; velocity derives from digest git history (R33). +- [ ] **T0.5 — Fleet Ghidra-C prefetch batch [background]** — `ImportOverlay.java` (~130 missing + programs) + `DecompileFunctions.java` over every remaining distinct stub (overlay tail + main's + 1,034; skip LINKED) → `.run/ghidra_c/`. **R23 lock discipline** (stop the hook-launched MCP + server first or route through it). Verify via `build_fuel_manifest.py` counters + 5 spot-reads. +- [ ] **T1 — Integration-recovery sweep [Ultracode — prompt at launch]** — all close=0 stranded + drafts through `recover_integration.py` tiers + §65b de-macroize + the snapshot-safe ladder; + the 10 named SESSION-16 blocked drafts. Prior: 39% recovery. **Report point #2.** R22 per batch. +- [ ] **T2 — The `-O0` cluster [Max, deep]** — PRIMARY: build the two-file atomic substitution + driver (stage remapped body into `_o0b.c` AND drop the stub's INCLUDE_ASM from + `_after.c` in one edit, gate) and sweep the T0-pinned families. FALLBACK: R17 research on + the Arm-A splat `%lo +0x20`. Resistant residue → wall ledger with evidence. **Report point #3.** +- [ ] **T3 — The standing crack-wave loop [Ultracode waves; Max between]** — lanes interleaved, + propagate behind every crack same-session: + A zero-crack (~61, propagation-only) · B top-mass fresh families (no size cap; jr via + `jtbl_family_bank`; `gate_stage` call-site-casts, never bulk header edits) · C tail mass on + prefetched seeds (local v3 ≤15 → GLM/Haiku ≤~50 → cheap-Opus mid) · D PINS (19) + W4 bounded + diagnoses + permuter backlog via the T4-fixed grinder. + Per-session checkpoint + velocity + 3 metrics. **ROI floor: 2 consecutive sessions < +0.3pp + instr each with all lanes exercised.** +- [ ] **T4 — Carried-tool resolution [xHigh]** — grinder warm-start + the 2 Phase-22 grinder bugs; + verify `--fix-def-sig` is nowhere a default (§119). +- [ ] **T5 — Phase close [Max]** — burn-down from digest history; P7 milestone walk; Roadmap delta; + gate 2; PhaseEnd_Phase30. + +**Milestone:** overlays at their measured ceiling — **≥95% instr fleet, or every remaining overlay +stub on a named wall/behemoth/queue ledger** — 140/140 byte-identical throughout. + +## Blockers +- (none) — precondition noted above (Drew's milestone-close commit before the first P30 commit). + +## Per-task log + +### T0(a) ✅ — gate_stage stage-0 + fix_arity_callers journal undo (2026-07-30) +The carried defect pair closed structurally, not by root-causing: +- **`gate_stage.py`**: **stage 0 gates the RAW drafts before any transform** (`GATE_NO_STAGE0` to + disable) — the destroyed-good-draft mode (SESSION-22 reproduction: `_o0` pair + `func_80138C60`, + ladder-FAILED / bare-VERIFIED) is now impossible by construction; the canon/cast/rc ladder + arity + pre-pass run only on stage-0 failures. Root-cause hypothesis recorded in-code (transforms are + batch-`--src-file` TU-blind where the gate is per-draft TU-aware) — open, now harmless. +- **`fix_arity_callers.py`**: `--journal` (per-edit literal before/after) + `--undo-journal + [--keep ]` — the exact undo now lives in the WRITER, shared by ladder AND bare workflows + (the 17-TU residue class). Replaces gate_stage's two-special-case file snapshot; undo moved to + after stage 2 (closes the latent stage-2 parity gap: a stage-2 bank used to lose its arity edit + before its own gate attempt). Stale-journal guard (`_arity_rc is not None`). +- **Verification:** negative control apply→undo → byte-identical tree; `--keep` retains exactly the + kept edit; in-process flow test `.run/t0a_flowtest/driver.py` **7/7 PASS** (stage-0-first, s1in = + failures-only, accumulation, undo-guard, src/ untouched). Real-tree at-scale proof lands with + T1's first sweep (R22-bracketed there). Cookbook **§122**. diff --git a/tools/fix_arity_callers.py b/tools/fix_arity_callers.py index 5f0f5d3a81..554c3a849b 100644 --- a/tools/fix_arity_callers.py +++ b/tools/fix_arity_callers.py @@ -16,13 +16,24 @@ no-prototype decl is COMPATIBLE with a definition whose params are default-promo tools/fix_arity_callers.py --apply --funcs func_X,func_Y [--drafts DIR] # rewrite (void)->() tools/fix_arity_callers.py --revert --funcs func_X,func_Y # ()->(void) tools/fix_arity_callers.py --apply --from-file .run/list.txt --drafts DIR + tools/fix_arity_callers.py --apply --funcs ... --journal .run/arity_journal.json + tools/fix_arity_callers.py --undo-journal .run/arity_journal.json --keep func_X # exact undo With --drafts, a target whose draft def has a narrow (non-promotion-safe) param is SKIPPED (reported), since no-proto cannot satisfy it. Without --drafts, every listed target is rewritten -(let the gate filter). Run the byte-gate afterwards; --revert the gate-failures to keep the -shared header carrying no-proto only where it bought a match. +(let the gate filter). Run the byte-gate afterwards. + +UNDO (P30 T0a — the "bare gate has no snapshot/restore" carried defect): prefer +--journal on apply + --undo-journal [--keep ] over --revert. --revert rewrites +`()` -> `(void)`, which is the exact inverse of a PLAIN apply but NOT of --any-proto (it relaxes +ANY prototype) — round-tripping an --any-proto edit through --revert INVENTED a `(void)` signature +in the fleet-shared header and 138/140 binaries failed check-all (measured 2026-07-21). The journal +records each substitution's literal before/after text, so --undo-journal restores per-DECL, exact +by construction: it cannot invent a signature, and it is immune to interleaved splices (a banked +draft landing in the same file between apply and undo — the Task-14 hazard the gate_stage snapshot +had to special-case). --keep names the fns whose edits stay (the banked set). """ -import argparse, os, re, glob, sys +import argparse, json, os, re, glob, sys REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) EC = os.path.join(REPO, 'src/shared/engine_core.h') @@ -55,11 +66,41 @@ def draft_is_promotion_safe(fn, drafts): return True +def undo_journal(path, keep): + """Exact per-decl undo: for each journaled edit whose fn is NOT kept, replace the recorded + `after` text back to the recorded `before` text (one occurrence). Reports restored/kept/missing + loudly (R32) — a missing `after` means someone else edited that decl since; it is NOT silently + skipped.""" + entries = json.load(open(os.path.join(REPO, path))) + restored = kept = missing = 0 + texts = {} + for e in entries: + if e['fn'] in keep: + kept += 1 + continue + f = e['file'] + if f not in texts: + texts[f] = open(f).read() + if e['after'] in texts[f]: + texts[f] = texts[f].replace(e['after'], e['before'], 1) + restored += 1 + else: + missing += 1 + print(f" [MISSING] {e['fn']} in {os.path.relpath(f, REPO)} — the edited decl text is " + f"no longer present (later edit?); NOT restored", file=sys.stderr) + for f, t in texts.items(): + open(f, 'w').write(t) + print(f'undo-journal: restored {restored}, kept {kept}, missing {missing}') + return 1 if missing else 0 + + def main(): ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) g = ap.add_mutually_exclusive_group(required=True) g.add_argument('--apply', action='store_true') g.add_argument('--revert', action='store_true') + g.add_argument('--undo-journal', metavar='PATH', + help='exact per-decl undo of a --journal file (see header); honors --keep') ap.add_argument('--funcs', help='comma-separated func_XXXX list') ap.add_argument('--from-file', help='file with one func_XXXX per line') ap.add_argument('--drafts', help='drafts dir: skip targets whose def has a narrow param (apply only)') @@ -69,8 +110,16 @@ def main(): ap.add_argument('--binary', help='ALSO scan+rewrite this overlay\'s own inline caller decls in ' 'src//*.c (a conflicting extern is often in the overlay src, not just ' 'engine_core.h — the T6 integration-recovery gap). Default: engine_core.h only.') + ap.add_argument('--journal', metavar='PATH', + help='(apply) record each substitution\'s literal before/after to this JSON for ' + 'exact --undo-journal restore; written even when no edits were made') + ap.add_argument('--keep', help='(undo-journal) comma-separated fns whose edits are KEPT (the banked set)') a = ap.parse_args() + if a.undo_journal: + keep = set(x.strip() for x in (a.keep or '').split(',') if x.strip()) + sys.exit(undo_journal(a.undo_journal, keep)) + fns = [] if a.funcs: fns += [x.strip() for x in a.funcs.split(',') if x.strip()] @@ -85,6 +134,7 @@ def main(): files += sorted(glob.glob(os.path.join(REPO, f'src/{a.binary}/{a.binary}*.c'))) texts = {f: open(f).read() for f in files} applied = skipped = reverted = notfound = 0 + journal = [] for fn in fns: ad = addr_of(fn) if not ad: @@ -100,7 +150,13 @@ def main(): n = 0 for f in files: if a.apply: - texts[f], k = (anyproto_re if a.any_proto else void_re).subn(r'\1\2', texts[f]) + # replacement FUNCTION (not a template) so each substitution's literal before/after + # is journaled — the substrate of the exact --undo-journal restore (P30 T0a) + def _sub(m, _f=f, _fn=fn): + after = m.group(1) + m.group(2) + journal.append({'fn': _fn, 'file': _f, 'before': m.group(0), 'after': after}) + return after + texts[f], k = (anyproto_re if a.any_proto else void_re).subn(_sub, texts[f]) else: texts[f], k = noproto_re.subn(r'\1void\2', texts[f]) n += k @@ -113,6 +169,12 @@ def main(): print(f' [no caller (void) decl found] {fn}') for f, t in texts.items(): open(f, 'w').write(t) + if a.apply and a.journal: + jp = os.path.join(REPO, a.journal) + os.makedirs(os.path.dirname(jp), exist_ok=True) + with open(jp, 'w') as jf: + json.dump(journal, jf, indent=1) + print(f'journal: {len(journal)} edit(s) -> {a.journal}') if a.apply: print(f'\napplied no-proto to {applied} caller decl(s); skipped {skipped} narrow-param; ' f'{notfound} had no (void) caller decl. Re-run the byte-gate now.') diff --git a/tools/gate_stage.py b/tools/gate_stage.py index 7454d68f9f..00ffdc7221 100644 --- a/tools/gate_stage.py +++ b/tools/gate_stage.py @@ -219,6 +219,33 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_ print(f"[gate] WARNING: 0/{len(draft_fns)} drafts are INCLUDE_ASM stubs in {binary} — " f"binary/src mismatch (drafts for a different binary?); banking will be 0", file=sys.stderr) + # STAGE 0 — gate the RAW drafts before ANY transform touches them (P30 T0a). The SESSION-22 + # reproduction of the carried "ladder destroys good drafts" defect: on one draft set the ladder + # FAILED func_8013B6A0 + func_8013B598 (_o0) and func_80138C60 (jr split) while bare + # harvest_verify VERIFIED all three, rtu_match confirming real-TU MATCHes — every casualty lives + # in a SPLIT TU while the plain-TU draft banked fine. Root-cause hypothesis (open): the + # transforms take ONE batch-wide --src-file while the gate derives each draft's home TU + # per-draft (Phase 26-A) — a mixed-TU batch gets its decls reconciled against the wrong TU. + # Stage 0 makes the failure mode impossible by construction regardless: a byte-correct draft + # banks before any transform can regress it, and the ladder becomes what it was always meant to + # be — RECOVERY for drafts that don't bank as written. Escape hatch: GATE_NO_STAGE0. + verified = [] + remaining_fns = list(draft_fns) + d_stage1_in = drafts + if not os.environ.get("GATE_NO_STAGE0"): + verified = _gate1(binary, src, asm, out, good_sha, drafts, verified_out, failed_out) + remaining_fns = [f for f in draft_fns if f not in verified] + if remaining_fns and len(remaining_fns) != len(draft_fns): + # hand the ladder ONLY the stage-0 failures (an input set that silently widens or + # narrows is the R32 defect class — see _xform's own history) + d_stage1_in = drafts + "-s1in" + abs_s1 = os.path.join(REPO, d_stage1_in) + shutil.rmtree(abs_s1, ignore_errors=True); os.makedirs(abs_s1) + for f in remaining_fns: + p = os.path.join(REPO, drafts, f + ".c") + if os.path.exists(p): + shutil.copy(p, os.path.join(abs_s1, f + ".c")) + # Recovery is CANON-FIRST, sig_unify FALLBACK (§19/§25): sig_unify can REGRESS an already-byte- # correct draft (e.g. a hand-pinned crack — it rewrites the def-sig to a banked caller's wrong # canonical). So gate canon+cast FIRST (stage 1: already-correct drafts bank), then sig_unify @@ -226,8 +253,8 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_ # winners. --src-file makes cast/sig_unify read the SPLIT .c (_a/_o0) so those drafts aren't # dropped. Each transform is a no-op-safe draft rewrite; the byte-gate is the sole arbiter (G3/P9). cast_extra = (["--src-file", src_file] if src_file else None) - d1 = _xform("canon_resident_calls.py", binary, drafts, "-cn") - d1 = _xform("cast_call_sites.py", binary, d1, "-cast", extra=cast_extra) + d1 = _xform("canon_resident_calls.py", binary, d_stage1_in, "-cn") if remaining_fns else d_stage1_in + d1 = _xform("cast_call_sites.py", binary, d1, "-cast", extra=cast_extra) if remaining_fns else d1 # data-symbol analog of cast_call_sites: rewrite each loose D_XXXX extern -> canonical + a # byte-neutral access cast (§33, T7b). No-op/idempotent without a data-decl conflict; the # byte-gate is still the sole arbiter. @@ -244,7 +271,7 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_ # which is why it SUPERSEDES reconcile_decls rather than patching it: teaching the old parser to # see `extern void (*D_x[])(void);` would ARM its fn-ptr-blind data_access_subs to rewrite a # call-through `D_x[i]()` into `((u8 *)D_x)[i]()`. - d1 = _xform("reconcile_tu.py", binary, d1, "-rc", extra=cast_extra) + d1 = _xform("reconcile_tu.py", binary, d1, "-rc", extra=cast_extra) if remaining_fns else d1 # ARITY PRE-PASS (Phase-29 Task-14). The three transforms above all rewrite the DRAFT. The # dominant residual blocker does not live in the draft at all: an already-banked SHARED caller @@ -285,17 +312,19 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_ # mechanism, snapshots the full source set, and undoes per function. _arity_rc = None - _arity_snapshot = {} - if draft_fns and not os.environ.get("GATE_NO_ARITY"): - # snapshot every file the pre-pass may touch, so the undo is a restore, not a re-derivation - for _f in ([os.path.join(REPO, "src/shared/engine_core.h")] - + glob.glob(os.path.join(REPO, f"src/{binary}/{binary}*.c"))): - try: - _arity_snapshot[_f] = open(_f).read() - except OSError: - pass - _arity = [PY, "tools/fix_arity_callers.py", "--apply", "--funcs", ",".join(draft_fns), - "--drafts", d1, "--binary", binary, "--any-proto"] + _arity_journal = f".run/arity_journal.{os.getpid()}.json" + if remaining_fns and not os.environ.get("GATE_NO_ARITY"): + # P30 T0a: the whole-file snapshot is replaced by fix_arity_callers' own per-decl JOURNAL + # (--journal / --undo-journal). The snapshot needed two measured special cases — restore + # ONLY src/shared/ on a partial bank (a full restore reverted 4 fresh banks, Phase-29 + # non-jtbl wave) and restore EVERYTHING on a zero-bank run (§61: "neutral" is not "wanted", + # ~40 TUs of dead diff otherwise) — because a file-level restore cannot distinguish the + # pre-pass's edits from the gate's own splices. A per-decl undo can: it round-trips each + # journaled substitution's literal text, keeps the banked set, and reports (never skips) a + # decl someone else edited since (R32). One mechanism, shared with every BARE-gate workflow + # (the other half of the carried defect: arity residue in 17 unrelated TUs, SESSION-22). + _arity = [PY, "tools/fix_arity_callers.py", "--apply", "--funcs", ",".join(remaining_fns), + "--drafts", d1, "--binary", binary, "--any-proto", "--journal", _arity_journal] try: _r = sh(_arity, timeout=600) _arity_rc = _r.returncode @@ -305,57 +334,12 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_ except Exception as e: # never let the pre-pass sink the gate print(f"[gate] arity pre-pass skipped: {e}", file=sys.stderr) - verified = _gate1(binary, src, asm, out, good_sha, d1, verified_out, failed_out) - - # UNDO for the drafts that did NOT bank — by SNAPSHOT RESTORE, never by `--revert`. - # - # `--revert` rewrites `()` -> `(void)`. That is the exact inverse of a PLAIN apply (which only - # relaxes `(void)`), but NOT of `--any-proto`, which relaxes ANY prototype. Round-tripping an - # unbanked function whose real decl was `extern void func_801708B0(void *a0);` therefore wrote - # back `(void)` — a DIFFERENT signature, in a header all 138 overlays include. - # - # Byte-cost, measured 2026-07-21: ov_SC01_077 gated byte-identical and 138 of 140 binaries then - # FAILED check-all. The single-binary gate cannot see this, because the edit is fleet-wide and - # the gate verifies one binary — so a lossy undo here is invisible until the full R22 sweep. - # Restoring the pre-pass snapshot and re-applying ONLY for the functions that banked is exact by - # construction and cannot invent a signature. - _unbanked = [f for f in draft_fns if f not in verified] - - if _unbanked and _arity_snapshot: - try: - # RESTORE ONLY src/shared/ (the fleet-shared header). The snapshot also captured - # src//*.c, but _gate1 SPLICES each banked draft into those files AFTER the - # snapshot was taken — so restoring them REVERTS the banks that just landed (and the - # re-apply below only re-does arity, not the splice). Bug measured 2026-07-22 (Phase-29 - # non-jtbl wave): a 9-draft run banked 4, and the 5 unbanked triggered this restore, - # silently reverting all 4 back to INCLUDE_ASM. The fleet hazard the snapshot exists for - # (a fix_arity edit lingering in engine_core.h for an unbanked fn, all 138 overlays) is - # entirely in src/shared/; the binary's own TU arity edits are LOCAL + byte-neutral, so - # leaving an unbanked fn's `(void)->()` there is harmless. (R33 — narrow the undo to its - # real write scope; §61's law that undo scope must not EXCEED write scope, from below.) - # - # ZERO-BANK CASE (Phase 29 SESSION-21): when NOTHING banked, the splice hazard above - # does not exist — there are no banks in src//*.c to preserve — so restore the - # binary's own TUs as well. Byte-neutrality is why the old code left them, and that was - # the wrong test: a 0-bank run was leaving ~40 TUs of dead diff in the tree, which a - # `git add -A` commits as pure noise (measured SESSION-20 alongside the identical - # `--normalize-self-decls` defect, 123 files). §61's undo law applied to the SUCCESS - # path: "neutral" is not "wanted" — an edit that bought nothing gets reverted. - _zero_bank = not verified - for _f, _txt in _arity_snapshot.items(): - if not _zero_bank and os.sep + "shared" + os.sep not in _f: - continue - with open(_f, "w") as _fh: - _fh.write(_txt) - if verified: - sh([PY, "tools/fix_arity_callers.py", "--apply", "--funcs", ",".join(verified), - "--drafts", d1, "--binary", binary, "--any-proto"], timeout=600) - except Exception as e: - print(f"[gate] arity snapshot-restore failed: {e}", file=sys.stderr) + if remaining_fns: + verified += _gate1(binary, src, asm, out, good_sha, d1, verified_out, failed_out) d = d1 fails1 = [f for f in draft_fns if f not in verified] - if fails1: # stage 2: sig_unify the stage-1 failures, re-gate + if fails1 and remaining_fns: # stage 2: sig_unify the stage-1 failures, re-gate s2in = drafts + "-s2in" abs_s2in = os.path.join(REPO, s2in) shutil.rmtree(abs_s2in, ignore_errors=True); os.makedirs(abs_s2in) @@ -366,6 +350,25 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_ d = _xform("sig_unify.py", binary, s2in, "-uni", extra=cast_extra) verified += _gate1(binary, src, asm, out, good_sha, d, verified_out, failed_out) + # UNDO the arity edits for everything that did NOT bank — per-decl journal restore, keeping the + # banked set. Never `--revert`: it inverts a PLAIN apply but not `--any-proto`, and once + # round-tripped an unbanked fn's real signature into an invented `(void)` in the fleet-shared + # header — 138 of 140 binaries failed check-all (measured 2026-07-21; invisible to the + # single-binary gate, caught only by the full R22 sweep). Running AFTER stage 2 (not between the + # stages, where the old snapshot-restore ran) also closes a latent parity gap: a stage-2 bank + # that needed its arity edit used to have it reverted before its own gate attempt. + # (_arity_rc is not None) ⇔ the pre-pass ran THIS invocation — a stale same-pid journal from a + # crashed earlier run must not be replayed against today's tree. + if _arity_rc is not None and os.path.exists(os.path.join(REPO, _arity_journal)): + try: + _u = sh([PY, "tools/fix_arity_callers.py", "--undo-journal", _arity_journal] + + (["--keep", ",".join(verified)] if verified else []), timeout=300) + if _u.returncode != 0: + print(f"[gate] arity undo-journal reported missing decls rc={_u.returncode}: " + f"{(_u.stderr or _u.stdout).strip()[:300]}", file=sys.stderr) + except Exception as e: + print(f"[gate] arity undo-journal failed: {e}", file=sys.stderr) + # 5 propagate the banked matches fleet-wide propagated = 0 prop_error = None