mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-10-02 07:59:52 -04:00
feat(phase-30): T0a — gate_stage stage-0 raw gate + fix_arity_callers per-edit journal undo (§122)
- STAGE 0: gate the RAW drafts before any transform (GATE_NO_STAGE0 escape) — the carried 'ladder destroys good drafts' defect (SESSION-22 reproduction: _o0 pair + func_80138C60, ladder-FAILED/bare-VERIFIED) is impossible by construction; ladder+arity now touch only stage-0 failures. TU-blind-transform root-cause hypothesis recorded in-code, open. - fix_arity_callers --journal/--undo-journal --keep: exact per-edit undo in the WRITER, shared by ladder AND bare workflows (the 17-TU residue class); replaces the two-special-case file snapshot; undo moved after stage 2 (closes the stage-2 arity parity gap); stale-journal guard. Negative-control: apply->undo byte-identical; --keep exact. - Flow test .run/t0a_flowtest/driver.py 7/7 PASS. Cookbook §122. CURRENT_PHASE T0(a) logged.
This commit is contained in:
Drew T
@@ -8073,3 +8073,35 @@ Note this is the complement of `header_sig_map()`, which reads the `extern` decl
|
||||
its own callees*. Two different macro-derived signature sources; a symbol in neither is a real gap.
|
||||
|
||||
**Blast radius: 0** beyond this family — like §118 and §120, and unlike §117, a **targeted** lever.
|
||||
|
||||
## §122 — GATE RAW BEFORE TRANSFORMING; the undo belongs to the WRITER, as a per-edit journal (P30 T0a, 2026-07-30)
|
||||
|
||||
**The defect pair this closes** (carried from SESSION-22, reproducible): (1) the `gate_stage` ladder
|
||||
FAILED drafts that bare `harvest_verify` VERIFIED — `func_8013B6A0`/`func_8013B598` (`_o0`) and
|
||||
`func_80138C60` (jr split), `rtu_match` confirming real-TU MATCHes. Every casualty lives in a
|
||||
**split TU**; the plain-TU draft banked through the same run. Root-cause hypothesis (still open, now
|
||||
harmless): the transforms take ONE batch-wide `--src-file` while the gate derives each draft's home
|
||||
TU per-draft (Phase 26-A) — a mixed-TU batch gets decls reconciled against the wrong TU.
|
||||
(2) A bare-gate workflow left `fix_arity_callers --any-proto` residue in **17 unrelated TUs** — the
|
||||
ladder's snapshot/undo existed only inside the ladder.
|
||||
|
||||
**Law 1 — stage 0: gate the RAW drafts before ANY transform.** A recovery ladder's transforms are
|
||||
for drafts that FAIL as written; running them on everything lets a "recovery" regress a byte-correct
|
||||
draft, and the gate then reports the regression as the draft's failure. With stage 0, the
|
||||
destroyed-good-draft mode is impossible *by construction* — no root-cause required first (the
|
||||
right sequencing under R35: neutralize, then diagnose). `GATE_NO_STAGE0` restores the old order.
|
||||
|
||||
**Law 2 — undo is the WRITER'S job, recorded per edit, not the orchestrator's file snapshot.**
|
||||
The snapshot needed two measured special cases (restore-shared-only on a partial bank because a full
|
||||
restore reverts fresh splices, Task-14; restore-everything on a zero-bank run, §61) because a
|
||||
file-level restore cannot tell the pre-pass's edits from the gate's splices. A per-edit journal
|
||||
(`fix_arity_callers --journal` / `--undo-journal --keep <banked>`) round-trips each substitution's
|
||||
literal text: exact by construction, immune to interleaved splices, uniform for shared+local files,
|
||||
and available to EVERY workflow — ladder or bare. A decl someone else edited since is reported
|
||||
MISSING loudly (R32), never silently skipped. Negative-control-proven: apply→undo → byte-identical
|
||||
tree; `--keep` retains exactly the banked set. Bonus closed: the undo now runs AFTER stage 2, so a
|
||||
stage-2 bank no longer loses its arity edit before its own gate attempt (the old latent parity gap).
|
||||
|
||||
**Generalizes to:** any pipeline where deterministic "fixers" precede a truth gate — the gate goes
|
||||
first on untouched input, fixers touch only failures, and every shared-state fixer journals its own
|
||||
writes. (Same family as §19/§25 canon-first and the §61 undo law; this entry is their composition.)
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# CURRENT PHASE — Phase 30: The Recovery & Concentration Campaign (overlays to their ceiling)
|
||||
|
||||
**Opened:** 2026-07-30 · **Effort:** Max (Fable 5; saved defaults) · **Generation:** Gen2 (22nd phase of the arc) ·
|
||||
**Baseline:** PhaseEnd_Phase29 (v1.28.0) · **Roadmap:** `docs/roadmap-to-100.md` **v2** §3 P30
|
||||
**Plan approved (P3 gate 1):** 2026-07-30, in-session (Drew's explicit waiver of the fresh-session step at 28% context).
|
||||
Full plan mirror: `~/.claude/plans/continue-dazzling-newt.md` (out-of-repo; this file is the in-repo authority + crash-recovery log).
|
||||
|
||||
**Fleet at open:** 87.5% instr-weighted / 78.0% distinct-code / 92.00% fn-count · 140/140
|
||||
byte-identical · 0 NON_MATCHING · dedup 1886/0 · tools-health RC=0.
|
||||
|
||||
**Drew's Phase-Start decisions (2026-07-30):**
|
||||
- In-session start (fresh-session step waived at 28% context).
|
||||
- The Ghidra-C prefetch batch pulled forward from P31 → **T0.5** (tail + main's 1,034 stubs; headless, background).
|
||||
- T2's primary path = the **two-file atomic o0b substitution driver** (the log's costed no-splat route); Arm-A splat research is the fallback only.
|
||||
- ROI floor for T3: two consecutive sessions each < +0.3pp instr with all lanes exercised (Drew-adjustable).
|
||||
|
||||
**⚠️ PRECONDITION (R6):** Drew makes the milestone-close commit + push (`PhaseEnd_Phase29.md` +
|
||||
`phase-ends/logs/Phase29.md`, both in the tree) **before P30's first commit lands**. P30 work
|
||||
proceeds; the first P30 commit waits on it.
|
||||
|
||||
---
|
||||
|
||||
## The numbers this phase must pin FIRST (T0 — do not consume until re-derived, R35/R14)
|
||||
- family_hseq **29,961** vs progress.py **28,296** remaining instances — an unexplained R32 gap.
|
||||
- The `-O0` population: "10 families / ~1,287 raw distinct" (checkpoints) vs "123 families /
|
||||
121,264 ins / 3,100 distinct" (the log's R14 correction) vs "129 distinct on the table" (the
|
||||
costed o0b route) — three numbers, one T0 job.
|
||||
- The zero-crack family list (map said **61** on 07-29) and the concentration table (top-20/top-100 shares).
|
||||
|
||||
## Standing invariants (every task)
|
||||
- Whole-binary byte-gate is the **sole arbiter** (G3/P9); match_one/closeness/masked = candidates (§52b).
|
||||
- **R22 clean-fleet per banked batch:** `make clean && make extract-all && make check-all` → 140/140.
|
||||
- `make tools-health` green + fail-closed before matching (corpus · cdecl · audit-binaries(R36) · report/lint/dedup).
|
||||
- Blast-radius typing on every integration write (T0 draft-only / T1 binary-local / T2 fleet-shared ⇒ R22 mandatory).
|
||||
- **Probe before costing** (R37-candidate discipline): probe one member before pricing a job; derive
|
||||
attribution from `corpus.stubs` before/after; diff the artifact to prove an edit ran (§120).
|
||||
- Flywheel: idioms → cookbook + tooling in-session (R16/R30); pivots → decision-log (R31).
|
||||
- Effort/model transitions **prompted, never assumed** (R26/R27) — STOP + WAIT for the toggle.
|
||||
- One commit per task/sub-repair after this file's update; Drew pushes (R6/R20). R23 no-op on db.*.gbf churn.
|
||||
|
||||
---
|
||||
|
||||
## Task checklist (current-task pointer = ▶)
|
||||
|
||||
- [ ] ▶ **T0 — Frontier regen + instrument repair [Max]** — one commit per lettered repair:
|
||||
(a) `gate_stage.py` snapshot/restore (the ladder destroys good drafts) ·
|
||||
(b) `rtu_match.py` surface the real cc1 error in the verdict ·
|
||||
(c) reconcile the family_hseq↔progress instance gap (R32) ·
|
||||
(d) purge since-banked rows from the backlog ledger + still-a-stub filter from `corpus.stubs` (R33) ·
|
||||
(e) refresh `.run/autopsy/residuals.jsonl` + fix the 21-file absolute-include defect ·
|
||||
(f) regenerate the frontier (report / family_hseq / worklist / fuel manifest) + **pin the three
|
||||
contested populations** → the one-page frontier report (**report point #1**).
|
||||
Verify: tools-health green; digests committed; velocity derives from digest git history (R33).
|
||||
- [ ] **T0.5 — Fleet Ghidra-C prefetch batch [background]** — `ImportOverlay.java` (~130 missing
|
||||
programs) + `DecompileFunctions.java` over every remaining distinct stub (overlay tail + main's
|
||||
1,034; skip LINKED) → `.run/ghidra_c/`. **R23 lock discipline** (stop the hook-launched MCP
|
||||
server first or route through it). Verify via `build_fuel_manifest.py` counters + 5 spot-reads.
|
||||
- [ ] **T1 — Integration-recovery sweep [Ultracode — prompt at launch]** — all close=0 stranded
|
||||
drafts through `recover_integration.py` tiers + §65b de-macroize + the snapshot-safe ladder;
|
||||
the 10 named SESSION-16 blocked drafts. Prior: 39% recovery. **Report point #2.** R22 per batch.
|
||||
- [ ] **T2 — The `-O0` cluster [Max, deep]** — PRIMARY: build the two-file atomic substitution
|
||||
driver (stage remapped body into `<ov>_o0b.c` AND drop the stub's INCLUDE_ASM from
|
||||
`<ov>_after.c` in one edit, gate) and sweep the T0-pinned families. FALLBACK: R17 research on
|
||||
the Arm-A splat `%lo +0x20`. Resistant residue → wall ledger with evidence. **Report point #3.**
|
||||
- [ ] **T3 — The standing crack-wave loop [Ultracode waves; Max between]** — lanes interleaved,
|
||||
propagate behind every crack same-session:
|
||||
A zero-crack (~61, propagation-only) · B top-mass fresh families (no size cap; jr via
|
||||
`jtbl_family_bank`; `gate_stage` call-site-casts, never bulk header edits) · C tail mass on
|
||||
prefetched seeds (local v3 ≤15 → GLM/Haiku ≤~50 → cheap-Opus mid) · D PINS (19) + W4 bounded
|
||||
diagnoses + permuter backlog via the T4-fixed grinder.
|
||||
Per-session checkpoint + velocity + 3 metrics. **ROI floor: 2 consecutive sessions < +0.3pp
|
||||
instr each with all lanes exercised.**
|
||||
- [ ] **T4 — Carried-tool resolution [xHigh]** — grinder warm-start + the 2 Phase-22 grinder bugs;
|
||||
verify `--fix-def-sig` is nowhere a default (§119).
|
||||
- [ ] **T5 — Phase close [Max]** — burn-down from digest history; P7 milestone walk; Roadmap delta;
|
||||
gate 2; PhaseEnd_Phase30.
|
||||
|
||||
**Milestone:** overlays at their measured ceiling — **≥95% instr fleet, or every remaining overlay
|
||||
stub on a named wall/behemoth/queue ledger** — 140/140 byte-identical throughout.
|
||||
|
||||
## Blockers
|
||||
- (none) — precondition noted above (Drew's milestone-close commit before the first P30 commit).
|
||||
|
||||
## Per-task log
|
||||
|
||||
### T0(a) ✅ — gate_stage stage-0 + fix_arity_callers journal undo (2026-07-30)
|
||||
The carried defect pair closed structurally, not by root-causing:
|
||||
- **`gate_stage.py`**: **stage 0 gates the RAW drafts before any transform** (`GATE_NO_STAGE0` to
|
||||
disable) — the destroyed-good-draft mode (SESSION-22 reproduction: `_o0` pair + `func_80138C60`,
|
||||
ladder-FAILED / bare-VERIFIED) is now impossible by construction; the canon/cast/rc ladder + arity
|
||||
pre-pass run only on stage-0 failures. Root-cause hypothesis recorded in-code (transforms are
|
||||
batch-`--src-file` TU-blind where the gate is per-draft TU-aware) — open, now harmless.
|
||||
- **`fix_arity_callers.py`**: `--journal` (per-edit literal before/after) + `--undo-journal
|
||||
[--keep <banked>]` — the exact undo now lives in the WRITER, shared by ladder AND bare workflows
|
||||
(the 17-TU residue class). Replaces gate_stage's two-special-case file snapshot; undo moved to
|
||||
after stage 2 (closes the latent stage-2 parity gap: a stage-2 bank used to lose its arity edit
|
||||
before its own gate attempt). Stale-journal guard (`_arity_rc is not None`).
|
||||
- **Verification:** negative control apply→undo → byte-identical tree; `--keep` retains exactly the
|
||||
kept edit; in-process flow test `.run/t0a_flowtest/driver.py` **7/7 PASS** (stage-0-first, s1in =
|
||||
failures-only, accumulation, undo-guard, src/ untouched). Real-tree at-scale proof lands with
|
||||
T1's first sweep (R22-bracketed there). Cookbook **§122**.
|
||||
@@ -16,13 +16,24 @@ no-prototype decl is COMPATIBLE with a definition whose params are default-promo
|
||||
tools/fix_arity_callers.py --apply --funcs func_X,func_Y [--drafts DIR] # rewrite (void)->()
|
||||
tools/fix_arity_callers.py --revert --funcs func_X,func_Y # ()->(void)
|
||||
tools/fix_arity_callers.py --apply --from-file .run/list.txt --drafts DIR
|
||||
tools/fix_arity_callers.py --apply --funcs ... --journal .run/arity_journal.json
|
||||
tools/fix_arity_callers.py --undo-journal .run/arity_journal.json --keep func_X # exact undo
|
||||
|
||||
With --drafts, a target whose draft def has a narrow (non-promotion-safe) param is SKIPPED
|
||||
(reported), since no-proto cannot satisfy it. Without --drafts, every listed target is rewritten
|
||||
(let the gate filter). Run the byte-gate afterwards; --revert the gate-failures to keep the
|
||||
shared header carrying no-proto only where it bought a match.
|
||||
(let the gate filter). Run the byte-gate afterwards.
|
||||
|
||||
UNDO (P30 T0a — the "bare gate has no snapshot/restore" carried defect): prefer
|
||||
--journal on apply + --undo-journal [--keep <banked,fns>] over --revert. --revert rewrites
|
||||
`()` -> `(void)`, which is the exact inverse of a PLAIN apply but NOT of --any-proto (it relaxes
|
||||
ANY prototype) — round-tripping an --any-proto edit through --revert INVENTED a `(void)` signature
|
||||
in the fleet-shared header and 138/140 binaries failed check-all (measured 2026-07-21). The journal
|
||||
records each substitution's literal before/after text, so --undo-journal restores per-DECL, exact
|
||||
by construction: it cannot invent a signature, and it is immune to interleaved splices (a banked
|
||||
draft landing in the same file between apply and undo — the Task-14 hazard the gate_stage snapshot
|
||||
had to special-case). --keep names the fns whose edits stay (the banked set).
|
||||
"""
|
||||
import argparse, os, re, glob, sys
|
||||
import argparse, json, os, re, glob, sys
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
EC = os.path.join(REPO, 'src/shared/engine_core.h')
|
||||
@@ -55,11 +66,41 @@ def draft_is_promotion_safe(fn, drafts):
|
||||
return True
|
||||
|
||||
|
||||
def undo_journal(path, keep):
|
||||
"""Exact per-decl undo: for each journaled edit whose fn is NOT kept, replace the recorded
|
||||
`after` text back to the recorded `before` text (one occurrence). Reports restored/kept/missing
|
||||
loudly (R32) — a missing `after` means someone else edited that decl since; it is NOT silently
|
||||
skipped."""
|
||||
entries = json.load(open(os.path.join(REPO, path)))
|
||||
restored = kept = missing = 0
|
||||
texts = {}
|
||||
for e in entries:
|
||||
if e['fn'] in keep:
|
||||
kept += 1
|
||||
continue
|
||||
f = e['file']
|
||||
if f not in texts:
|
||||
texts[f] = open(f).read()
|
||||
if e['after'] in texts[f]:
|
||||
texts[f] = texts[f].replace(e['after'], e['before'], 1)
|
||||
restored += 1
|
||||
else:
|
||||
missing += 1
|
||||
print(f" [MISSING] {e['fn']} in {os.path.relpath(f, REPO)} — the edited decl text is "
|
||||
f"no longer present (later edit?); NOT restored", file=sys.stderr)
|
||||
for f, t in texts.items():
|
||||
open(f, 'w').write(t)
|
||||
print(f'undo-journal: restored {restored}, kept {kept}, missing {missing}')
|
||||
return 1 if missing else 0
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
g = ap.add_mutually_exclusive_group(required=True)
|
||||
g.add_argument('--apply', action='store_true')
|
||||
g.add_argument('--revert', action='store_true')
|
||||
g.add_argument('--undo-journal', metavar='PATH',
|
||||
help='exact per-decl undo of a --journal file (see header); honors --keep')
|
||||
ap.add_argument('--funcs', help='comma-separated func_XXXX list')
|
||||
ap.add_argument('--from-file', help='file with one func_XXXX per line')
|
||||
ap.add_argument('--drafts', help='drafts dir: skip targets whose def has a narrow param (apply only)')
|
||||
@@ -69,8 +110,16 @@ def main():
|
||||
ap.add_argument('--binary', help='ALSO scan+rewrite this overlay\'s own inline caller decls in '
|
||||
'src/<binary>/<binary>*.c (a conflicting extern is often in the overlay src, not just '
|
||||
'engine_core.h — the T6 integration-recovery gap). Default: engine_core.h only.')
|
||||
ap.add_argument('--journal', metavar='PATH',
|
||||
help='(apply) record each substitution\'s literal before/after to this JSON for '
|
||||
'exact --undo-journal restore; written even when no edits were made')
|
||||
ap.add_argument('--keep', help='(undo-journal) comma-separated fns whose edits are KEPT (the banked set)')
|
||||
a = ap.parse_args()
|
||||
|
||||
if a.undo_journal:
|
||||
keep = set(x.strip() for x in (a.keep or '').split(',') if x.strip())
|
||||
sys.exit(undo_journal(a.undo_journal, keep))
|
||||
|
||||
fns = []
|
||||
if a.funcs:
|
||||
fns += [x.strip() for x in a.funcs.split(',') if x.strip()]
|
||||
@@ -85,6 +134,7 @@ def main():
|
||||
files += sorted(glob.glob(os.path.join(REPO, f'src/{a.binary}/{a.binary}*.c')))
|
||||
texts = {f: open(f).read() for f in files}
|
||||
applied = skipped = reverted = notfound = 0
|
||||
journal = []
|
||||
for fn in fns:
|
||||
ad = addr_of(fn)
|
||||
if not ad:
|
||||
@@ -100,7 +150,13 @@ def main():
|
||||
n = 0
|
||||
for f in files:
|
||||
if a.apply:
|
||||
texts[f], k = (anyproto_re if a.any_proto else void_re).subn(r'\1\2', texts[f])
|
||||
# replacement FUNCTION (not a template) so each substitution's literal before/after
|
||||
# is journaled — the substrate of the exact --undo-journal restore (P30 T0a)
|
||||
def _sub(m, _f=f, _fn=fn):
|
||||
after = m.group(1) + m.group(2)
|
||||
journal.append({'fn': _fn, 'file': _f, 'before': m.group(0), 'after': after})
|
||||
return after
|
||||
texts[f], k = (anyproto_re if a.any_proto else void_re).subn(_sub, texts[f])
|
||||
else:
|
||||
texts[f], k = noproto_re.subn(r'\1void\2', texts[f])
|
||||
n += k
|
||||
@@ -113,6 +169,12 @@ def main():
|
||||
print(f' [no caller (void) decl found] {fn}')
|
||||
for f, t in texts.items():
|
||||
open(f, 'w').write(t)
|
||||
if a.apply and a.journal:
|
||||
jp = os.path.join(REPO, a.journal)
|
||||
os.makedirs(os.path.dirname(jp), exist_ok=True)
|
||||
with open(jp, 'w') as jf:
|
||||
json.dump(journal, jf, indent=1)
|
||||
print(f'journal: {len(journal)} edit(s) -> {a.journal}')
|
||||
if a.apply:
|
||||
print(f'\napplied no-proto to {applied} caller decl(s); skipped {skipped} narrow-param; '
|
||||
f'{notfound} had no (void) caller decl. Re-run the byte-gate now.')
|
||||
|
||||
+65
-62
@@ -219,6 +219,33 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
|
||||
print(f"[gate] WARNING: 0/{len(draft_fns)} drafts are INCLUDE_ASM stubs in {binary} — "
|
||||
f"binary/src mismatch (drafts for a different binary?); banking will be 0", file=sys.stderr)
|
||||
|
||||
# STAGE 0 — gate the RAW drafts before ANY transform touches them (P30 T0a). The SESSION-22
|
||||
# reproduction of the carried "ladder destroys good drafts" defect: on one draft set the ladder
|
||||
# FAILED func_8013B6A0 + func_8013B598 (_o0) and func_80138C60 (jr split) while bare
|
||||
# harvest_verify VERIFIED all three, rtu_match confirming real-TU MATCHes — every casualty lives
|
||||
# in a SPLIT TU while the plain-TU draft banked fine. Root-cause hypothesis (open): the
|
||||
# transforms take ONE batch-wide --src-file while the gate derives each draft's home TU
|
||||
# per-draft (Phase 26-A) — a mixed-TU batch gets its decls reconciled against the wrong TU.
|
||||
# Stage 0 makes the failure mode impossible by construction regardless: a byte-correct draft
|
||||
# banks before any transform can regress it, and the ladder becomes what it was always meant to
|
||||
# be — RECOVERY for drafts that don't bank as written. Escape hatch: GATE_NO_STAGE0.
|
||||
verified = []
|
||||
remaining_fns = list(draft_fns)
|
||||
d_stage1_in = drafts
|
||||
if not os.environ.get("GATE_NO_STAGE0"):
|
||||
verified = _gate1(binary, src, asm, out, good_sha, drafts, verified_out, failed_out)
|
||||
remaining_fns = [f for f in draft_fns if f not in verified]
|
||||
if remaining_fns and len(remaining_fns) != len(draft_fns):
|
||||
# hand the ladder ONLY the stage-0 failures (an input set that silently widens or
|
||||
# narrows is the R32 defect class — see _xform's own history)
|
||||
d_stage1_in = drafts + "-s1in"
|
||||
abs_s1 = os.path.join(REPO, d_stage1_in)
|
||||
shutil.rmtree(abs_s1, ignore_errors=True); os.makedirs(abs_s1)
|
||||
for f in remaining_fns:
|
||||
p = os.path.join(REPO, drafts, f + ".c")
|
||||
if os.path.exists(p):
|
||||
shutil.copy(p, os.path.join(abs_s1, f + ".c"))
|
||||
|
||||
# Recovery is CANON-FIRST, sig_unify FALLBACK (§19/§25): sig_unify can REGRESS an already-byte-
|
||||
# correct draft (e.g. a hand-pinned crack — it rewrites the def-sig to a banked caller's wrong
|
||||
# canonical). So gate canon+cast FIRST (stage 1: already-correct drafts bank), then sig_unify
|
||||
@@ -226,8 +253,8 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
|
||||
# winners. --src-file makes cast/sig_unify read the SPLIT .c (_a/_o0) so those drafts aren't
|
||||
# dropped. Each transform is a no-op-safe draft rewrite; the byte-gate is the sole arbiter (G3/P9).
|
||||
cast_extra = (["--src-file", src_file] if src_file else None)
|
||||
d1 = _xform("canon_resident_calls.py", binary, drafts, "-cn")
|
||||
d1 = _xform("cast_call_sites.py", binary, d1, "-cast", extra=cast_extra)
|
||||
d1 = _xform("canon_resident_calls.py", binary, d_stage1_in, "-cn") if remaining_fns else d_stage1_in
|
||||
d1 = _xform("cast_call_sites.py", binary, d1, "-cast", extra=cast_extra) if remaining_fns else d1
|
||||
# data-symbol analog of cast_call_sites: rewrite each loose D_XXXX extern -> canonical + a
|
||||
# byte-neutral access cast (§33, T7b). No-op/idempotent without a data-decl conflict; the
|
||||
# byte-gate is still the sole arbiter.
|
||||
@@ -244,7 +271,7 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
|
||||
# which is why it SUPERSEDES reconcile_decls rather than patching it: teaching the old parser to
|
||||
# see `extern void (*D_x[])(void);` would ARM its fn-ptr-blind data_access_subs to rewrite a
|
||||
# call-through `D_x[i]()` into `((u8 *)D_x)[i]()`.
|
||||
d1 = _xform("reconcile_tu.py", binary, d1, "-rc", extra=cast_extra)
|
||||
d1 = _xform("reconcile_tu.py", binary, d1, "-rc", extra=cast_extra) if remaining_fns else d1
|
||||
|
||||
# ARITY PRE-PASS (Phase-29 Task-14). The three transforms above all rewrite the DRAFT. The
|
||||
# dominant residual blocker does not live in the draft at all: an already-banked SHARED caller
|
||||
@@ -285,17 +312,19 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
|
||||
# mechanism, snapshots the full source set, and undoes per function.
|
||||
|
||||
_arity_rc = None
|
||||
_arity_snapshot = {}
|
||||
if draft_fns and not os.environ.get("GATE_NO_ARITY"):
|
||||
# snapshot every file the pre-pass may touch, so the undo is a restore, not a re-derivation
|
||||
for _f in ([os.path.join(REPO, "src/shared/engine_core.h")]
|
||||
+ glob.glob(os.path.join(REPO, f"src/{binary}/{binary}*.c"))):
|
||||
try:
|
||||
_arity_snapshot[_f] = open(_f).read()
|
||||
except OSError:
|
||||
pass
|
||||
_arity = [PY, "tools/fix_arity_callers.py", "--apply", "--funcs", ",".join(draft_fns),
|
||||
"--drafts", d1, "--binary", binary, "--any-proto"]
|
||||
_arity_journal = f".run/arity_journal.{os.getpid()}.json"
|
||||
if remaining_fns and not os.environ.get("GATE_NO_ARITY"):
|
||||
# P30 T0a: the whole-file snapshot is replaced by fix_arity_callers' own per-decl JOURNAL
|
||||
# (--journal / --undo-journal). The snapshot needed two measured special cases — restore
|
||||
# ONLY src/shared/ on a partial bank (a full restore reverted 4 fresh banks, Phase-29
|
||||
# non-jtbl wave) and restore EVERYTHING on a zero-bank run (§61: "neutral" is not "wanted",
|
||||
# ~40 TUs of dead diff otherwise) — because a file-level restore cannot distinguish the
|
||||
# pre-pass's edits from the gate's own splices. A per-decl undo can: it round-trips each
|
||||
# journaled substitution's literal text, keeps the banked set, and reports (never skips) a
|
||||
# decl someone else edited since (R32). One mechanism, shared with every BARE-gate workflow
|
||||
# (the other half of the carried defect: arity residue in 17 unrelated TUs, SESSION-22).
|
||||
_arity = [PY, "tools/fix_arity_callers.py", "--apply", "--funcs", ",".join(remaining_fns),
|
||||
"--drafts", d1, "--binary", binary, "--any-proto", "--journal", _arity_journal]
|
||||
try:
|
||||
_r = sh(_arity, timeout=600)
|
||||
_arity_rc = _r.returncode
|
||||
@@ -305,57 +334,12 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
|
||||
except Exception as e: # never let the pre-pass sink the gate
|
||||
print(f"[gate] arity pre-pass skipped: {e}", file=sys.stderr)
|
||||
|
||||
verified = _gate1(binary, src, asm, out, good_sha, d1, verified_out, failed_out)
|
||||
|
||||
# UNDO for the drafts that did NOT bank — by SNAPSHOT RESTORE, never by `--revert`.
|
||||
#
|
||||
# `--revert` rewrites `()` -> `(void)`. That is the exact inverse of a PLAIN apply (which only
|
||||
# relaxes `(void)`), but NOT of `--any-proto`, which relaxes ANY prototype. Round-tripping an
|
||||
# unbanked function whose real decl was `extern void func_801708B0(void *a0);` therefore wrote
|
||||
# back `(void)` — a DIFFERENT signature, in a header all 138 overlays include.
|
||||
#
|
||||
# Byte-cost, measured 2026-07-21: ov_SC01_077 gated byte-identical and 138 of 140 binaries then
|
||||
# FAILED check-all. The single-binary gate cannot see this, because the edit is fleet-wide and
|
||||
# the gate verifies one binary — so a lossy undo here is invisible until the full R22 sweep.
|
||||
# Restoring the pre-pass snapshot and re-applying ONLY for the functions that banked is exact by
|
||||
# construction and cannot invent a signature.
|
||||
_unbanked = [f for f in draft_fns if f not in verified]
|
||||
|
||||
if _unbanked and _arity_snapshot:
|
||||
try:
|
||||
# RESTORE ONLY src/shared/ (the fleet-shared header). The snapshot also captured
|
||||
# src/<binary>/*.c, but _gate1 SPLICES each banked draft into those files AFTER the
|
||||
# snapshot was taken — so restoring them REVERTS the banks that just landed (and the
|
||||
# re-apply below only re-does arity, not the splice). Bug measured 2026-07-22 (Phase-29
|
||||
# non-jtbl wave): a 9-draft run banked 4, and the 5 unbanked triggered this restore,
|
||||
# silently reverting all 4 back to INCLUDE_ASM. The fleet hazard the snapshot exists for
|
||||
# (a fix_arity edit lingering in engine_core.h for an unbanked fn, all 138 overlays) is
|
||||
# entirely in src/shared/; the binary's own TU arity edits are LOCAL + byte-neutral, so
|
||||
# leaving an unbanked fn's `(void)->()` there is harmless. (R33 — narrow the undo to its
|
||||
# real write scope; §61's law that undo scope must not EXCEED write scope, from below.)
|
||||
#
|
||||
# ZERO-BANK CASE (Phase 29 SESSION-21): when NOTHING banked, the splice hazard above
|
||||
# does not exist — there are no banks in src/<binary>/*.c to preserve — so restore the
|
||||
# binary's own TUs as well. Byte-neutrality is why the old code left them, and that was
|
||||
# the wrong test: a 0-bank run was leaving ~40 TUs of dead diff in the tree, which a
|
||||
# `git add -A` commits as pure noise (measured SESSION-20 alongside the identical
|
||||
# `--normalize-self-decls` defect, 123 files). §61's undo law applied to the SUCCESS
|
||||
# path: "neutral" is not "wanted" — an edit that bought nothing gets reverted.
|
||||
_zero_bank = not verified
|
||||
for _f, _txt in _arity_snapshot.items():
|
||||
if not _zero_bank and os.sep + "shared" + os.sep not in _f:
|
||||
continue
|
||||
with open(_f, "w") as _fh:
|
||||
_fh.write(_txt)
|
||||
if verified:
|
||||
sh([PY, "tools/fix_arity_callers.py", "--apply", "--funcs", ",".join(verified),
|
||||
"--drafts", d1, "--binary", binary, "--any-proto"], timeout=600)
|
||||
except Exception as e:
|
||||
print(f"[gate] arity snapshot-restore failed: {e}", file=sys.stderr)
|
||||
if remaining_fns:
|
||||
verified += _gate1(binary, src, asm, out, good_sha, d1, verified_out, failed_out)
|
||||
|
||||
d = d1
|
||||
fails1 = [f for f in draft_fns if f not in verified]
|
||||
if fails1: # stage 2: sig_unify the stage-1 failures, re-gate
|
||||
if fails1 and remaining_fns: # stage 2: sig_unify the stage-1 failures, re-gate
|
||||
s2in = drafts + "-s2in"
|
||||
abs_s2in = os.path.join(REPO, s2in)
|
||||
shutil.rmtree(abs_s2in, ignore_errors=True); os.makedirs(abs_s2in)
|
||||
@@ -366,6 +350,25 @@ def _run_gate_locked(drafts, binary, src, asm, out, good_sha, propagate, source_
|
||||
d = _xform("sig_unify.py", binary, s2in, "-uni", extra=cast_extra)
|
||||
verified += _gate1(binary, src, asm, out, good_sha, d, verified_out, failed_out)
|
||||
|
||||
# UNDO the arity edits for everything that did NOT bank — per-decl journal restore, keeping the
|
||||
# banked set. Never `--revert`: it inverts a PLAIN apply but not `--any-proto`, and once
|
||||
# round-tripped an unbanked fn's real signature into an invented `(void)` in the fleet-shared
|
||||
# header — 138 of 140 binaries failed check-all (measured 2026-07-21; invisible to the
|
||||
# single-binary gate, caught only by the full R22 sweep). Running AFTER stage 2 (not between the
|
||||
# stages, where the old snapshot-restore ran) also closes a latent parity gap: a stage-2 bank
|
||||
# that needed its arity edit used to have it reverted before its own gate attempt.
|
||||
# (_arity_rc is not None) ⇔ the pre-pass ran THIS invocation — a stale same-pid journal from a
|
||||
# crashed earlier run must not be replayed against today's tree.
|
||||
if _arity_rc is not None and os.path.exists(os.path.join(REPO, _arity_journal)):
|
||||
try:
|
||||
_u = sh([PY, "tools/fix_arity_callers.py", "--undo-journal", _arity_journal]
|
||||
+ (["--keep", ",".join(verified)] if verified else []), timeout=300)
|
||||
if _u.returncode != 0:
|
||||
print(f"[gate] arity undo-journal reported missing decls rc={_u.returncode}: "
|
||||
f"{(_u.stderr or _u.stdout).strip()[:300]}", file=sys.stderr)
|
||||
except Exception as e:
|
||||
print(f"[gate] arity undo-journal failed: {e}", file=sys.stderr)
|
||||
|
||||
# 5 propagate the banked matches fleet-wide
|
||||
propagated = 0
|
||||
prop_error = None
|
||||
|
||||
Reference in New Issue
Block a user