mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-27 22:45:39 -04:00
Drew T
73321fe2c2
THE DEFECT CHAIN (byte-witnessed, both ends fixed):
1. harvest_verify._reload_corpus re-applied the `--src` filter AFTER a jtbl carve. Following a
carved stub to its NEW TU is that function's entire documented purpose, and it was deleting the
very stub it had just followed. Then:
_stubs loses fn -> render() raises KeyError -> UNCAUGHT -> _jtbl_restore(snap) never runs
-> the carve is STRANDED in config/ + src/ -> every LATER group in the same gate run then
built against a tree the earlier crashes had mutated.
line 136 already calls --src "an optional filter, not a location oracle"; this was the one place
treating it as one. Fix: the filter never drops a draft under verification, wherever it now
lives, + an R32 loud report if a working stub vanishes across a reload (which also repairs
_touched/baseline — a carved fn missing from _stubs left its new TU unbaselined, so the revert
path could not have restored it either).
2. .run/s6f_gate.py never checked the child's returncode — it grepped stdout for VERIFIED:/FAILED:
and booked "neither" as NOTHING, printing a clean-looking tally over 10 missing verdicts. This
is the §136a defect I logged against my own capture tool last session, in the gate itself.
Fix: 1:1 accounting assertion (banked+failed+no-verdict == drafts), the child's rc + output tail
on anything unaccounted, and exit 1 — a crashed child may have stranded a carve, so it must
never look like success.
PROOF THE FIX IS NOT COSMETIC: func_8017EA84 (579 ins) now carves and banks BYTE-IDENTICAL. The old
tool reported it as nothing at all.
BANKED 7 (R22 clean-fleet 140/140 from `make clean` + extract-all + check-all):
func_8017EA84 ov_SC02_000 (579) · func_8017FEE0 ov_SC02_026 (299) · func_80181CE4 ov_SC03_111 (491)
func_80183AE0 ov_SC03_112 (240) · func_80184C74 ov_SC06_018 (288) · func_80180F98 ov_SC03_097 (263)
func_801841C8 ov_SC02_035 (44)
MY OWN ERROR, RECORDED (R37/R14): after reverting the stranded carves I re-extracted ONE overlay,
not all 16 — the Phase-20 R22 corollary (a reverted CONFIG needs a `make extract`, not just a
revert) which I know and skipped. Run 2 therefore read WORSE than run 1: three genuinely-banked
functions failed against stale asm. Re-extracting the 16 touched binaries produced the honest run.
A gate result measured against stale asm is not a measurement (R35).
.run/w6_diag.py: run the REAL gate path for one (ov,fn) with the child's full output. s36_capture.py
splices without the carve, which is the wrong path for a table-bearing fn (§61b: the carve must
follow the splice) and produces a failure that is an artifact of the diagnosis.
93 lines
4.6 KiB
Python
93 lines
4.6 KiB
Python
#!/usr/bin/env python3
|
|
"""Gate the S6f wave drafts — grouping DERIVED from corpus.stubs, never asserted.
|
|
|
|
The wave manifest I hand-transcribed carried wrong `_jr_*` split names for three overlays (the agents
|
|
caught it). So this driver never trusts a recorded path: for every draft on disk it asks corpus.stubs
|
|
where that function's INCLUDE_ASM actually lives, groups by (binary, split), and runs the whole-binary
|
|
byte-gate once per group. The gate is the sole arbiter (G3/P9).
|
|
"""
|
|
import sys, os, re, glob, subprocess, collections, shutil
|
|
sys.path.insert(0, 'tools')
|
|
import corpus
|
|
|
|
STAGE = '.run/s6f_stage'
|
|
shutil.rmtree(STAGE, ignore_errors=True)
|
|
groups = collections.defaultdict(list)
|
|
skipped = []
|
|
|
|
for d in sorted(glob.glob(sys.argv[1] if len(sys.argv)>1 else '.run/s6f/*/*.c')):
|
|
ov = os.path.basename(os.path.dirname(d))
|
|
fn = os.path.basename(d)[:-2]
|
|
# S35: an agent left scratch files (test_licm*.c) in the drafts dir and this line died on
|
|
# int('full', 16), taking the whole gate with it. A drafts dir is agent-writable, so treat a
|
|
# non-conforming name as a NAMED, COUNTED skip — never a crash (R32).
|
|
m_ = re.fullmatch(r'func_([0-9A-Fa-f]{8})', fn)
|
|
if not m_:
|
|
skipped.append((ov, fn, 'not a func_<ADDR>.c deliverable — agent scratch?'))
|
|
continue
|
|
addr = int(m_.group(1), 16)
|
|
st = corpus.stubs(ov)
|
|
if addr not in st:
|
|
skipped.append((ov, fn, 'not a live stub (already banked?)'))
|
|
continue
|
|
rec = st[addr]
|
|
tu = rec.path if hasattr(rec, 'path') else rec[0]
|
|
subdir = getattr(rec, 'subdir', None) or os.path.basename(tu)[:-2]
|
|
groups[(ov, tu, subdir)].append((fn, d))
|
|
|
|
print(f"[gate] {sum(len(v) for v in groups.values())} drafts / {len(groups)} groups; skipped {len(skipped)}")
|
|
for s in skipped:
|
|
print(" skip:", s)
|
|
|
|
banked, failed, unaccounted = [], [], []
|
|
for (ov, tu, subdir), items in sorted(groups.items()):
|
|
gdir = os.path.join(STAGE, ov)
|
|
os.makedirs(gdir, exist_ok=True)
|
|
for fn, d in items:
|
|
shutil.copy(d, os.path.join(gdir, fn + '.c'))
|
|
good = open(f'config/check.{ov}.sha').read().split()[0]
|
|
r = subprocess.run([
|
|
'.venv/bin/python', 'tools/harvest_verify.py', '--binary', ov, '--src', tu,
|
|
'--asm-subdir', f'{ov}/nonmatchings/{subdir}', '--out', f'build/{ov}/{ov}',
|
|
'--good-sha', good, '--drafts', gdir, '--chunk', '1'], capture_output=True, text=True)
|
|
out = r.stdout + r.stderr
|
|
ver = [l for l in out.splitlines() if l.startswith('VERIFIED:')]
|
|
fal = [l for l in out.splitlines() if l.startswith('FAILED :')]
|
|
v = [x for x in (ver[0][9:].split() if ver else []) if x != '(none)']
|
|
f = [x for x in (fal[0][9:].split() if fal else []) if x != '(none)']
|
|
banked += [(ov, x) for x in v]
|
|
failed += [(ov, x) for x in f]
|
|
|
|
# R32 COVERAGE ASSERTION — the defect this gate shipped with (P30 S38).
|
|
# It used to grep stdout for two line-prefixes and report whatever it found. When
|
|
# harvest_verify CRASHED (a post-carve KeyError, uncaught, which also stranded the carve in
|
|
# config/ + src/), there was no VERIFIED and no FAILED line — so the draft was booked as
|
|
# NOTHING, and the run printed a clean-looking tally. Wave 6 lost 10 of 16 drafts that way,
|
|
# 9 of them claiming MATCH, while every later group ran against the mutated tree.
|
|
# A verdict is now MANDATORY per draft: anything unaccounted for is reported LOUDLY, with the
|
|
# child's returncode and the tail of its output, because "no verdict" is a DEFECT, not a no.
|
|
seen = set(v) | set(f)
|
|
miss = [fn for fn, _ in items if fn not in seen]
|
|
if miss or r.returncode:
|
|
unaccounted += [(ov, fn) for fn in miss]
|
|
print(f" {ov:<14} [{os.path.basename(tu)}] !! NO VERDICT for {len(miss)} draft(s) "
|
|
f"(harvest_verify rc={r.returncode}): {' '.join(miss)}")
|
|
for line in out.strip().splitlines()[-12:]:
|
|
print(f" | {line}")
|
|
else:
|
|
print(f" {ov:<14} [{os.path.basename(tu)}] {len(v)}/{len(items)} banked"
|
|
+ (f" FAILED: {' '.join(f)}" if f else ''))
|
|
shutil.rmtree(gdir, ignore_errors=True)
|
|
|
|
total = sum(len(x) for x in groups.values())
|
|
print(f"\n[gate] BANKED {len(banked)} / FAILED {len(failed)} / NO-VERDICT {len(unaccounted)}"
|
|
f" (of {total} drafts, {len(skipped)} skipped pre-flight)")
|
|
for ov, fn in failed:
|
|
print(" fail:", ov, fn)
|
|
for ov, fn in unaccounted:
|
|
print(" NO VERDICT:", ov, fn)
|
|
assert len(banked) + len(failed) + len(unaccounted) == total, (
|
|
f"gate accounting is not 1:1 — {len(banked)}+{len(failed)}+{len(unaccounted)} != {total}")
|
|
if unaccounted:
|
|
sys.exit(1) # a crashed child may have STRANDED A CARVE; never let this look like success
|