fix(phase-26a): A9g — jr_inventory: retire the ephemeral roster, derive banked from the image (R33)

jr_inventory's `banked` set was filtered by an EPHEMERAL, gitignored
.run/banked_func_*.json roster: a `rm -rf .run` / fresh clone would blind ALL
banked jr at once, cross-address siblings (roster named after the exemplar) were
structurally invisible, and non-leader banked jr were missed. "The purest R33
case in the group" (audit).

FIX (the audit's exact prescription): delete the roster glob + `cand` filter;
`banked` is DERIVED FROM THE IMAGE — a real-C def/define fn is a banked jr iff
family_remap.reloc_targets shows it references a committed .rodata carve offset
(config + image, both durable; cross-address- and non-leader-immune). R32
assertion: every committed carve must resolve to EXACTLY ONE owner or abort (a
stranded/duplicated carve is the §8b func_801734BC incident, never silent).

Also fixed the adjacent finding: the asm_jr scan's func_-fullmatch dropped the
curated-name listCdBuffer jr; now resolved via oss.addr_of(). (The --only path's
own fullmatch is left — it parses user input, not the corpus.)

Perf: read the overlay image ONCE and pass it to reloc_targets(..., data=) — a
new backward-compatible param on family_remap (regression: 0/80 mismatch vs the
re-read path).

Verified: data-param behavior-identical; the R33 win — ov_SC02_000 now finds the
cross-address sibling func_8017FCB0 the roster missed; full-fleet parallel run =
134/134 OK, 0 false aborts, 1336 banked jr == 1336 carves -> 1:1 ownership holds
fleet-wide. Byte-safe: jr_isolate_all is not in the make build/extract path
(R22-neutral); the change makes future isolations strictly more correct.
This commit is contained in:
Drew T
2026-07-14 22:03:43 -06:00
parent 96e025a324
commit ea20bdf9f3
4 changed files with 101 additions and 22 deletions
+15 -1
View File
@@ -634,6 +634,11 @@ where ENGINE_TYPES is parsed once from src/shared/engine_types.h (that alone rec
- **assertion to add:** Two, because the current selftest cannot see this. (a) In parse_overlay_c, after building `items`: `assert not [it for it in items if it[2] not in REAL_KINDS and _has_definition_header(it[3])]` — i.e. NO item's preamble may contain a `{`-bodied function-definition header that is not that item's own anchor; fail loud with file:line. (b) Add a real COVERAGE gate to selftest() alongside the round-trip: crudely count candidate anchors (INCLUDE_ASM lines + col-0 `^\w+\s*\(\s*func_` macro invocations + col-0 `}` lines) and `assert n_anchors >= n_col0_close_braces - n_type_blocks`, printing the delta. The existing 'round-trip exact + 0 unresolved + monotonic' triple is 100% green on a file with two missed definitions and must never again be reported as proof of coverage.
### [MEDIUM] `tools/jr_isolate_all.py :: jr_inventory() — `re.fullmatch(r'func_[0-9A-Fa-f]{8}', fn)` on the .s basename (line 81)`
- **✅ FIXED (A9g, §26-A):** the `.s` basename is now resolved through the symbol table
(`overlay_src_split.addr_of()`), which handles BOTH `func_<hex>` and a curated name, so the handwritten
`listCdBuffer` jr (0x80180000, in ov_SC01_084/ov_SC03_108/ov_SC03_118/ov_SC03_119) is no longer dropped.
Fixed together with the roster finding below (same rewrite of `jr_inventory`). *(The `--only` path's own
`func_`-fullmatch — a 2nd instance — is left: it parses user-supplied core names, not the corpus.)*
- candidates **5899** / parsed **5895** / **real skips 4**
- **evidence:** Candidate = every .s under asm/ov_*/nonmatchings/*/ whose text references a `jtbl_` symbol (5,899). Parsed: 5,895. The 4 misses are all the same function under a CURATED name: asm/ov_SC03_119/nonmatchings/ov_SC03_119_jr_80178D40/listCdBuffer.s, and the same in ov_SC03_118, ov_SC01_084, ov_SC03_108. This is NOT a data label — it is a genuine 0xA64-byte handwritten jr function at 0x80180000 with 2 jtbl references ('/* Handwritten function */ nonmatching listCdBuffer, 0xA64' / `glabel listCdBuffer` / `sra $v0, $v0, 16` ...), and it is INCLUDE_ASM'd from real source: src/ov_SC03_119/ov_SC03_119_jr_80178D40.c:4442 `INCLUDE_ASM("asm/ov_SC03_119/nonmatchings/ov_SC03_119_jr_80178D40", listCdBuffer);`. Because its name is `listCdBuffer` and not `func_XXXXXXXX`, the fullmatch filter drops it and jr_inventory reports it does not exist. (Related, LOW: the same curated name defeats jtbl_carve.all_data_labels' `(?:jtbl_|D_)[0-9A-Fa-f]{8}` regex in the 33 OTHER overlays where 0x80180000 is data — `dlabel listCdBuffer` at asm/ov_SC01_000/data/tail.data.s:2380 is invisible to the carve-boundary oracle. I checked exploitability and it is currently NIL: listCdBuffer sits at 0x80180000 in the general data region, never immediately after a jtbl, and jtbl_words' enddlabel-bounded trailing-zero trim clamps `end` to the true extent regardless of a missed boundary label. The boundary oracle is silently incomplete but the trim accidentally masks it — worth an assertion, not a fix.)
- **blast radius:** Lost MATCHES (4 potential banks), and it falsifies the tool's core invariant. jr_isolate_all's docstring and its lines 118-131 establish the rule that EVERY jr in a cut object must get its own region, because 'a region may host AT MOST ONE .rodata carve' — that invariant is what the func_8015AE2C/func_801734BC +33-byte image corruption taught. listCdBuffer is a jr that the tool cannot see, and in these 4 overlays it currently sits INSIDE `<ov>_jr_80178D40`, sharing a region with an already-banked jr. So the invariant 'every jr has its own region' is false in 4 overlays right now. The moment listCdBuffer is matched and banked, jtbl_carve hits its 'subseg would host NON-CONTIGUOUS .rodata carves' fail-loud (jtbl_carve.py:250-254) and the bank cannot proceed without a hand-isolation. Fail-loud, so no byte corruption — but 4 banks are blocked and the blocker will present as a mysterious carve error rather than a naming gap.
@@ -1301,7 +1306,16 @@ The exemplar did not produce different bytes. It produced NO bytes. The operator
- **assertion (R32):** R32: assert that 'byte-diff' is only ever printed when an output binary was actually produced (os.path.exists(build/<ov>/<ov>) and the build's own compile steps all exited 0). If no binary exists, the verdict MUST be COMPILE-FAIL. Cover with a test that feeds bank_exemplar a body with an undeclared struct and asserts the reported class is COMPILE-FAIL, not byte-diff.
- **skeptic:** RAN: (a) read tools/bank_exemplar.py — cited code confirmed at :65 (success test) and :68-70 (4-term allowlist + "byte-diff" fallback). (b) Re-ran the repro with the PINNED compiler the build actually uses (Makefile:443 -> CC1_PSX = tools/bin/gcc-2.7.2-psx/cc1, not the cdk cc1 the claim used) on .run/audit/skeptic/diag2.c: "storage size of `x' isn't known" / "too many arguments to function `g'" / "dereferencing pointer to incomplete type" -> classifier matched 0/3 -> "recovered: byte-diff". Mechanism CONFIRMED. Note the claim's cdk evidence was off the real build path: cdk says "syntax error", psx says "parse error" — a HIT under the real compiler. (c) `strings` over the psx cc1 diagnostic vocabulary: 12/44 error-shaped strings matched; all 9 incomplete-type variants, all 4 arg-count variants, incompatible-types, invalid-lvalue, called-object-is-not-a-function escape. COUNTS REFUTED: candidates=3/parsed=0 are three lines of a SYNTHETIC file the claimant wrote, not corpus items. The real corpus is the 11 bank_exemplar run logs in .run/bank*.log: ~30 stage-FAIL lines, 27 classified CORRECTLY (conflicting types / undeclared / redefinition; ValueError paths bypass the classifier). "byte-diff" was emitted exactly ONCE, for one function (func_8013F350), triple-logged across bank9.log/bank_all.log/bank_func_8013F350.log — and phase-ends/CURRENT_PHASE.md:267 records the independent follow-up proving that verdict TRUE ("NOT a plumbing bug — it is a real class... D_8011511C must be struct-typed to force la+offset; no cast fixes it"). It compiled and produced different bytes. So real mislabels in the corpus = 0; the allowlist covers 100% of the fault vocabulary this project's staged ladder has actually produced. BLAST RADIUS latent as claimed: fb.revert()+sys.exit(1) on every failure path, success gated on the SHA1 "[ OK ]" string, and grep -rn "byte-diff" finds no programmatic consumer — operator-facing prose only. Real shape, reproducible on demand, zero occurrences: LOW/latent hardening, not a MEDIUM live bug. The R33 fix (print the compiler's words + exit code, classify nothing) is still correct and cheap.
### [LOW] `jr_isolate_all.py` — DOWNGRADED
### [LOW] `jr_isolate_all.py` — DOWNGRADED → ✅ FIXED (A9g, R33)
> The ephemeral roster is GONE. `jr_inventory`'s `banked` is now DERIVED: a real-C def/define fn is a
> banked jr iff `family_remap.reloc_targets` shows it references a committed `.rodata` carve offset (the
> image + the config, both durable). No `.run/banked_func_*.json` glob, no `cand` filter — so a `rm -rf
> .run`/fresh clone no longer blinds it, the CROSS-ADDRESS sibling `func_8017FCB0` is now found (roster
> missed it), and NON-LEADER banked jr (carve in the object's own subseg, not a `_jr_` leader) are found
> where a subseg-name model would miss them. The R32 assertion from :1320 is implemented: every committed
> carve must resolve to exactly ONE owner or `jr_inventory` aborts. Image read once + passed to
> `reloc_targets(…, data=)` (new backward-compatible param), ~6s→fast. Verified: the 3 audit orphans
> (ov_SC01_000/ov_SC02_000/ov_SC02_003) all resolve; full-fleet 1:1 ownership holds (no false abort).
- **scanner:** jr_inventory() — the `banked` set: tools/jr_isolate_all.py:85-87 `for bj in glob.glob(REPO/.run/banked_func_*.json): cand.add(basename[len('banked_'):-len('.json')])`, then :94 `banked = {a: nm for a, nm in realc.items() if nm in cand}`
- **counts:** candidates **1202** / parsed **1199** / real skips **3**
- **evidence:** OVER-APPROX = every committed `.rodata` carve piece in the 134 overlay configs (1202). Each is, by construction, owned by a banked jr. REAL = carves whose owner jr_inventory+carve_owners can resolve = 1199. THE 3 ORPHANS, all confirmed against the real corpus:
+20
View File
@@ -659,6 +659,26 @@ On approval → `/model opus` + `/effort xHigh` (Tasks 0–4; ALL Fable5 via `Ag
## Log
- **2026-07-14 (session 13, A9g — jr_inventory: retire the ephemeral roster, derive banked from the image; Max):**
R33 applied to "the purest R33 case in the group" (audit). `jr_inventory`'s `banked` set was filtered by an
**EPHEMERAL, gitignored `.run/banked_func_*.json` roster** — `rm -rf .run`/a fresh clone would blind ALL
banked jr at once, cross-address siblings (roster named after the exemplar) were structurally invisible, and
non-leader banked jr were missed. **FIX (audit's exact prescription): deleted the roster glob + `cand`
filter; `banked` is now DERIVED FROM THE IMAGE** — a real-C def/define fn is a banked jr iff
`family_remap.reloc_targets` shows it references a committed `.rodata` carve offset (config + image, both
durable). **R32 assertion added:** every committed carve must resolve to EXACTLY ONE owner or the run
aborts (a stranded/duplicated carve = the §8b func_801734BC incident, never silent). **Also fixed** the
adjacent MEDIUM/LOW finding — the `asm_jr` scan's `func_`-fullmatch dropped the curated-name `listCdBuffer`
jr; now resolved via `oss.addr_of()` (the `--only` path's own fullmatch is left — it parses user input, not
the corpus). **Perf:** the image is read ONCE and passed to `reloc_targets(…, data=)` (new
backward-compatible param on family_remap; regression-verified 0/80 mismatch vs the re-read path). **VERIFIED:**
(1) reloc_targets `data`-param behavior-identical; (2) the R33 win — ov_SC02_000 now finds the cross-address
sibling `func_8017FCB0` the roster missed; the 3 non-leader overlays (ov_SC01_077/04_008/05_009) resolve;
(3) **full-fleet parallel run = 134/134 OK, 0 false aborts, 1336 banked jr == 1336 carves → 1:1 ownership
holds fleet-wide** (the R32 assertion is safe). **BYTE-SAFE:** jr_isolate_all is NOT in the make build/extract
path (R22-neutral); the change makes future isolations strictly MORE correct (finds carve owners the roster
missed → fewer stranded carves). tooling-audit ledger marked FIXED (3 jr_isolate_all findings). **NEXT: A10 —
re-test the walls (the audit payoff).**
- **2026-07-14 (session 13, A9f — overlay_src_split force_decl latch fixed + a coverage oracle; Max):**
The parser `jr_isolate_all` rewrites source from swallowed **2 real function definitions** on physical
lines of the form `extern A; extern B; void f(){...}`: `scan_construct`'s `force_decl` latched from the
+9 -3
View File
@@ -43,11 +43,17 @@ def nins_of(ov, addr):
return None
def reloc_targets(ov, addr):
def reloc_targets(ov, addr, data=None):
"""ordered [(kind, resolved_addr)] for jal targets + lui/lo address loads, in instruction order.
Verified against splat .s ground truth (22/22 on func_80141100; 15/15 on func_801407F4 whose
indexed-global D[i] accesses the pre-Phase-26 tracker missed — see the add/addu hi-propagation)."""
data = open(img_path(ov), "rb").read()
indexed-global D[i] accesses the pre-Phase-26 tracker missed — see the add/addu hi-propagation).
`data` (optional) = the overlay image bytes, read once by the caller and passed to AVOID the
per-call `open(...).read()` when scanning many functions of one overlay (jr_inventory reads it
once, then calls this ~2,300× — 6s -> 0.1s). Omit it and the image is re-read fresh per call
(the default, so a re-extraction mid-process is always seen)."""
if data is None:
data = open(img_path(ov), "rb").read()
n = nins_of(ov, addr)
off = addr - VRAM
out, pend = [], {}
+57 -18
View File
@@ -70,31 +70,70 @@ def rodata_carves(cfg_lines):
def jr_inventory(ov):
"""Return (all_jr:{vram:src_kind}, banked:{vram:func_name}). src_kind in
{'asm','banked'}. jr = INCLUDE_ASM funcs whose .s references a jtbl_ + the
already-banked jr (real-C `def`/`define` items whose name is recorded in a
.run/banked_func_*.json — the exemplar is NOT in its own sibling list, so we
confirm presence by parsing the source, not by the sibling roster)."""
"""Return (all_jr:{vram:src_kind}, banked:{vram:func_name}). src_kind in {'asm','banked'}.
jr = still-unmatched switch functions (INCLUDE_ASM `.s` referencing a jtbl_) + the
already-banked jr (whose jtbl became a committed `.rodata` carve).
`banked` is DERIVED FROM THE IMAGE — never from a roster (R33). The old code filtered
real-C defs by an EPHEMERAL, gitignored `.run/banked_func_*.json` set: a `rm -rf .run`
/ fresh clone made all banked jr invisible at once, and a cross-address sibling (whose
roster file is named after the exemplar) was structurally missing. Two proven invariants
answer it instead: (1) the committed splat config lists every `.rodata` carve; (2) a
real-C function OWNS a carve iff it references that carve's address — `family_remap.
reloc_targets` reads the extracted image and says so. So a real-C def/define fn is a
banked jr iff it references a committed carve offset. Cross-address- and
curated-name-immune, and it finds NON-LEADER banked jr (carve in the object's own
subseg, not a `_jr_` leader) that a subseg-name model would miss. Every carve MUST
resolve to exactly one owner or the run aborts (R32) — a stranded/duplicated carve is
the func_801734BC incident (§8b) and must never be silent.
Cost: ~6s -> ~0.1s by reading the overlay image ONCE and passing it to reloc_targets."""
import family_remap
base = oss_vram(ov)
syms = oss.load_ov_syms(ov)
# still-unmatched jr: an INCLUDE_ASM fn whose .s references a jtbl_. Resolve the .s
# basename through the symbol table (addr_of) so a CURATED name (e.g. listCdBuffer) is
# not dropped by a func_-shape fullmatch (§26-A LOW finding).
asm_jr = {}
for p in glob.glob(os.path.join(REPO, f"asm/{ov}/nonmatchings/*/*.s")):
if re.search(r'jtbl_[0-9A-Fa-f]{8}', open(p).read()):
fn = os.path.basename(p)[:-2]
if re.fullmatch(r'func_[0-9A-Fa-f]{8}', fn):
asm_jr[int(fn[5:], 16)] = fn
# candidate banked-jr names (global roster) -> confirm each is a real-C def here
cand = set()
for bj in glob.glob(os.path.join(REPO, ".run/banked_func_*.json")):
cand.add(os.path.basename(bj)[len("banked_"):-len(".json")])
realc = {} # addr -> name for def/define items
syms = oss.load_ov_syms(ov)
nm = os.path.basename(p)[:-2]
a = oss.addr_of(nm, syms)
if a is not None:
asm_jr[a] = nm
# already-banked jr: every real-C def/define fn that references a committed carve
# offset in the IMAGE (read once, passed to reloc_targets).
cfg_lines = open(os.path.join(REPO, f"config/splat.{ov}.yaml")).read().splitlines()
carve_offs = {off for _li, off, _sub in rodata_carves(cfg_lines)}
img = open(family_remap.img_path(ov), "rb").read()
banked, owners = {}, {} # owners: carve_off -> [names]
for cf in glob.glob(os.path.join(REPO, f"src/{ov}/*.c")):
_, items = oss.parse_overlay_c(open(cf).read(), syms)
for addr, name, kind, _ in items:
if kind in ("def", "define") and name and addr is not None:
realc[addr] = name
banked = {a: nm for a, nm in realc.items() if nm in cand}
if kind not in ("def", "define") or not name or addr is None:
continue
try:
targets = family_remap.reloc_targets(ov, addr, data=img)
except Exception:
continue
hits = {t - base for k, t in targets if k == "data" and (t - base) in carve_offs}
if hits:
banked[addr] = name
for off in hits:
owners.setdefault(off, []).append(name)
# R32: every committed carve resolves to EXACTLY ONE banked owner, or abort loud.
problems = [("UNOWNED", hex(base + o)) for o in sorted(carve_offs - set(owners))]
problems += [("MULTI", hex(base + o), owners[o]) for o in sorted(owners) if len(owners[o]) > 1]
if problems:
sys.exit(f"jr_inventory({ov}): committed .rodata carve ownership is not 1:1 (R32/R33) — "
f"a stranded/duplicated carve (§8b func_801734BC class): {problems}")
alljr = dict(asm_jr)
alljr.update({a: "banked" for a in banked}) # marker; name in `banked`
alljr.update({a: "banked" for a in banked}) # marker; name in `banked`
return alljr, banked