From 41d65af73f2c10cc2de3a758b35c8b103c73fb74 Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Tue, 21 Jul 2026 23:51:16 -0600 Subject: [PATCH] fix(phase-29): per-function jtbl prep + stranded-carve undo; the clean-rebuild blocker named MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit THE BLOCKING FINDING (§61c): the jtbl carve+isolation path yields a state that is INCREMENTALLY valid and CLEAN-INVALID. func_80135A4C banks every time through the automated path (BYTE-IDENTICAL at harvest_verify's gate) and fails `make clean && extract-all && check-all` twice, identically (139/140, [FAIL] ov_SC06_018). The bank is therefore NOT reproducible from committed config + source, and the gate that authorises it cannot see the defect because the gate IS the incremental build (§42b, most expensive form). => NO jtbl core can be banked until that divergence is diagnosed. Next step is to diff the incremental vs clean build/ov_SC06_018/** object set + generated .ld/asm for the carved subseg — NOT to bank more. All 12 wave cracks stay preserved at .run/giants/t5wave_*. TWO REAL DESIGN FAULTS FOUND AND FIXED IN harvest_verify: 1. A stranded carve poisons the overlay. _jtbl_prep carved a draft the gate then REJECTED; the carve remained with NO owner (the fn is still INCLUDE_ASM), and jr_inventory's 1:1 ownership assertion then refused every later isolation in that overlay ([('UNOWNED','0x801d288c')] = func_801299C8's table). THE ASSERTION WAS RIGHT AND CAUGHT IT — R32/R33 working as designed; the defect was mine for leaving the carve behind. Fixed: per-function carve + exact snapshot-restore (config text + only this attempt's region files) on gate rejection. 2. Per-function undo is UNSOUND IN A BATCH: isolation REPARTITIONS shared source, so restoring one draft's snapshot deletes region files now hosting OTHER pending drafts, and their stubs vanish (KeyError in render). jtbl drafts must run one per harvest_verify invocation, or the undo must be region-aware. MEASURED so it is not re-derived: of 11 preserved cracks exactly ONE (func_80135A4C) reaches byte-identical through the carve path; the other 4 table-bearing ones fail one-at-a-time too, on PLUMBING (§57 self-decl et al), not the carve. Tree reverted; clean-fleet 140/140; nothing banked. cookbook §61c. --- docs/matching-cookbook.md | 33 +++++++++++++++++++ tools/harvest_verify.py | 68 +++++++++++++++++++++++++++++++-------- 2 files changed, 88 insertions(+), 13 deletions(-) diff --git a/docs/matching-cookbook.md b/docs/matching-cookbook.md index 223cbe1f56..bede2f7209 100644 --- a/docs/matching-cookbook.md +++ b/docs/matching-cookbook.md @@ -4682,3 +4682,36 @@ the 9 preserved cracks. region files THIS run created (§61's constraint, applied where I had first ignored my own rule). * **Being in a `_jr_*` TU ≠ having a table.** Only 4 of 8 wave drafts in jtbl-carved TUs actually reference a `jtbl_`; the stage correctly prepares only those. The other 4 fail for other classes. + +### §61c — The jtbl bank is INCREMENTALLY valid and CLEAN-INVALID (Phase 29, 2026-07-21) — the blocking finding + +`func_80135A4C` banks through the automated jtbl path every time: `[jtbl] carved` → `+ chunk(1)` → +`verified 1 / failed 0 BYTE-IDENTICAL`. **And it fails a clean rebuild, twice, identically:** + + incremental (harvest_verify's own gate) : BYTE-IDENTICAL + make clean && extract-all && check-all : 139 passed, 1 failed ([FAIL] ov_SC06_018) + +So the carve+isolation path yields a state that is **not reproducible from committed config + source** +— the incremental tree carries something the clean pipeline does not reconstruct (extraction order, or +asm that only exists mid-flow). This is the §42b stale-incremental false pass in its most expensive +form: the gate that authorises the bank cannot see the defect, because the gate IS the incremental +build. + +**Until that reproducibility gap is closed, NO jtbl core can be banked** — not by hand, not by the +ladder. The correct next step is to diagnose the divergence itself (diff the incremental vs clean +`build/ov_SC06_018/**` object set and the generated `.ld`/asm for the carved subseg), NOT to bank more. + +**Two design faults found on the way, both real and both fixed in `harvest_verify`:** +1. **A stranded carve poisons the overlay.** `_jtbl_prep` carved a draft the gate then REJECTED; the + carve stayed with no owner (the fn is still `INCLUDE_ASM`), and `jr_inventory`'s 1:1 ownership + assertion then refused EVERY later isolation in that overlay (`[('UNOWNED','0x801d288c')]` = + func_801299C8's table). **The assertion was RIGHT and caught it** — R32/R33 working exactly as + designed. Fix: per-function carve with snapshot-restore on gate rejection. +2. **Per-function undo is unsound in a BATCH.** Isolation REPARTITIONS shared source, so restoring one + draft's snapshot deletes region files that now host OTHER pending drafts — their stubs vanish + (`KeyError` in render). jtbl drafts must therefore be processed **one per `harvest_verify` + invocation**, or the undo must be region-aware. + +**Measured, so the next session does not re-derive it:** of 11 preserved wave cracks, exactly ONE +(`func_80135A4C`) reaches byte-identical through the carve path; the other four table-bearing ones +fail one-at-a-time too, on the PLUMBING classes (§57 self-decl et al), not on the carve. diff --git a/tools/harvest_verify.py b/tools/harvest_verify.py index 3309778be9..bfc777d4f5 100644 --- a/tools/harvest_verify.py +++ b/tools/harvest_verify.py @@ -208,13 +208,47 @@ def _reload_corpus(): baseline = {q: open(q).read() for q in _touched} -def _jtbl_prep(): - """Make every table-bearing draft's carve valid. Returns the list prepared.""" - todo = [fn for fn in items if _fn_has_jtbl(fn)] - if not todo: - return [] +def _jtbl_snapshot(): + """Text of the config files a carve/isolation may rewrite (for an exact undo).""" + out = {} + for q in (os.path.join(REPO, 'config/splat.%s.yaml' % a.binary), + os.path.join(REPO, 'config/overlays.mk')): + try: + out[q] = open(q).read() + except OSError: + pass + return out + + +def _jtbl_restore(snap, regions_before): + """Undo a carve/isolation EXACTLY: restore the config text and delete only the region files + this attempt created. Then re-extract and re-derive the stub map. + + WHY THIS EXISTS (byte-proven 2026-07-21). A carve left behind by a draft the gate REJECTED has + no owner — the function is still INCLUDE_ASM — so `jr_inventory`'s 1:1 ownership assertion + (R32/R33) then refuses EVERY later isolation in that overlay: + committed .rodata carve ownership is not 1:1 … [('UNOWNED','0x801d288c')] + 0x801d288c is func_801299C8's table: prepped, gate-rejected, carve stranded. The assertion is + CORRECT and caught it; the defect was leaving the carve behind. One failed draft poisoned the + whole overlay for the rest of the batch (4 isolate-FAILs downstream).""" + for q, txt in snap.items(): + open(q, 'w').write(txt) + for rf in set(glob.glob('src/%s/%s_jr_*.c' % (a.binary, a.binary))) - regions_before: + try: + os.remove(rf) + except OSError: + pass + _sh(['make', '--no-print-directory', 'extract', 'BINARY=%s' % a.binary]) + _reload_corpus() + + +def _jtbl_prep_one(fn): + """Prep ONE table-bearing draft's carve. Returns (ok, snapshot, regions_before).""" + if not _fn_has_jtbl(fn): + return True, None, None + snap, regions_before = _jtbl_snapshot(), set(glob.glob('src/%s/%s_jr_*.c' % (a.binary, a.binary))) done = [] - for fn in todo: + for fn in [fn]: st = _stubs.get(fn) if st is None: continue @@ -245,11 +279,13 @@ def _jtbl_prep(): last = ((r.stdout or '') + (r.stderr or '')).strip().splitlines()[-1:] or [''] print(' [jtbl] carve FAILED %s: %s' % (fn, last[0][:120])); continue done.append(fn) - if done: - _sh(['make', '--no-print-directory', 'extract', 'BINARY=%s' % a.binary]) - _reload_corpus() - print(' [jtbl] carved %d/%d table-bearing draft(s): %s' % (len(done), len(todo), ' '.join(done))) - return done + if not done: + _jtbl_restore(snap, regions_before) + return False, None, None + _sh(['make', '--no-print-directory', 'extract', 'BINARY=%s' % a.binary]) + _reload_corpus() + print(' [jtbl] carved %s' % done[0]) + return True, snap, regions_before def _stub_line(fn): @@ -289,16 +325,22 @@ def commit(fns): verified.extend(fns) -_jtbl_prepared = _jtbl_prep() - i = 0 while i < len(items): chunk = items[i:i + a.chunk] i += a.chunk + # per-chunk jtbl prep (chunk==1 is the prescribed default, so this is per-function): carve the + # table, and if the gate then REJECTS the draft, undo the carve — a stranded carve has no owner + # and poisons every later isolation in the overlay (see _jtbl_restore). + _jsnap = _jregions = None + if len(chunk) == 1: + _ok, _jsnap, _jregions = _jtbl_prep_one(chunk[0]) if attempt(chunk): commit(chunk) print(' + chunk(%d): %s' % (len(chunk), ' '.join(chunk))) elif len(chunk) == 1: + if _jsnap is not None: + _jtbl_restore(_jsnap, _jregions) # stranded-carve undo (R32 ownership stays 1:1) # ATOMIC CHUNK — do NOT bisect (Phase-28 T6). The old code fell into the loop below and # re-ran attempt([fn]) on the SAME single element against the SAME baseline: a byte-identical # DUPLICATE build. classify_fail reads _last_sha/_last_err, which the failed attempt(chunk)