tools/jtbl_rodata_pads.py --derive walks a TU's rodata emission against the
retail island and validates only what it can SEE. A §265 verbatim-__asm__ body
emits its tables as `.section<TAB>.rodata` + `jtbl_xxxxxxxx:`, and the walk
missed BOTH spellings:
* the rodata directive was matched as the literal one-space string
".section .rodata" / ".rdata", so a tab-spelled directive never entered
rodata at all;
* inside rodata the anchor regex accepted only `D_xxxxxxxx` (the S74 dlabel
fix was one prefix short), so a `jtbl_xxxxxxxx:` label was invisible.
Consequence, traced: the walk skipped the block as if it were .text, every
later C table walked 104 bytes behind its retail address, EVERY WORD in that
range happens to be a valid code address so the entry guard never fired, and
the walk stopped short of the island's single zero word -- so the one trailing
pad was never emitted and the image linked 4 BYTES SHORT. That produced 3,989
differing bytes on a body the verdict layer had already called byte-identical.
Fixed: tokenised directive match (.rdata / .section .rodata, tabs and commas);
anchors keyed on the ADDRESS IN THE NAME for `D_` or `jtbl_`, with an
address-suffixed label of any other prefix now REFUSING loudly (R43) instead of
becoming a silent hole; and `.align N` modelled SECTION-RELATIVE from the walk
origin, as `as` does -- needed for `.align 3` when a section starts = 4 mod 8,
which span B (0x80072E44) does.
Negative control: old vs new derive over ALL 162 md_*/main derive-path TUs ->
160 byte-identical post-derive streams with identical exit codes, 0 DIFF; 2 SKIP
(800c2/800c3 are REORDER TUs with no derive stage).
main SHA1 143dbb89... BYTE-IDENTICAL, 413,696 bytes, cmp identical to retail.
WHAT THIS IS NOT. SaveLoadRoutine is banked as a §265 verbatim __asm__ block --
BYTES, NOT A DECOMPILE. Its 1,165 instructions are byte-correct and unexplained.
tools/progress.py correctly REFUSES to count it, reporting `UNPLACED (parse
hole)` rather than inflating REAL (which moved 880 -> 881 on func_8005DCA0
alone). Two such blocks already exist in this TU, documented as necessary
because those functions have no epilogue and fall into shared tails. A real C
decompile is now being attempted separately; this commit is the revertible
byte-green base for it.
Clean rebuild BYTE-IDENTICAL 87ac0de3; corpus.stubs('md_SC07_004') 10 -> 0, counted from the SOURCE.
THE §376/§378 CHAIN WAS NEVER NEEDED. Zero declaration edits: no fix_arity_callers, no
cast_self_callers, no --any-proto, no --sync-decls, no undo journal. `git diff -U0` on the TU removes
exactly the 10 INCLUDE_ASM stubs plus one hoisted typedef. Every recorded "declaration conflict" was
an INSTRUMENT defect. Four of them, all named, three patched here:
1. `CC1-FAIL(no-diagnostic)` was neither cc1 nor no-diagnostic. The failing stage was
`jtbl_rodata_pads --derive`, which prints to stderr AFTER cc1 exits 0 quietly. `_items` matched a
rodata anchor only as `D_xxxxxxxx:`, but a block written as inline `__asm__` in C arrives in the
labels.inc macro form `dlabel D_xxxxxxxx` — so an 8-byte hole opened in the walk and every C jump
table after it died with "island layout drift". The harness label was wrong twice: it said CC1
when the failure was a post-maspsx filter, and no-diagnostic when there was a precise one.
2. Same file, UNALIGNED ANCHOR: the ctable branch read `word(pos)` without first stepping the
sub-word zero gap, so a preceding `.asciz` ending at an odd address made it refuse a correct
layout. Now reuses the same zero_gap the anchor branches already use — a no-op wherever pos is
already aligned, i.e. everywhere that builds green today.
3. `harvest_verify` computed the typedef strip-set UNSCOPED: `cdecl.typedef_names(path)` without
`above=fn`, which that function supports for exactly this. A typedef declared BELOW the splice
point got stripped out of the draft that needed it -> `parse error`, logged as PLUMBING and
indistinguishable from a real conflict. One line.
4. NOT PATCHED, AND THE MOST IMPORTANT ONE: `reconcile_tu.py` (gate_stage's `-rc` stage) MANUFACTURED
both remaining "conflicts". The same drafts gate 9/9 byte-identical through harvest_verify and
7/9 through gate_stage. Isolated stage by stage, `-rc` (a) rewrites deliberately BLOCK-SCOPED
externs to a file-scope spelling whose typedef is declared ~2,800 lines lower — overwriting the
TU's own byte-proven house style, which three already-banked functions in that file use; and
(b) substitutes identifiers TEXTUALLY, including inside comments and inside `&`-expressions,
emitting `*(T *)&((s32 *)&D_800AE620)`. A correct draft using the block-scope-extern idiom
currently CANNOT survive gate_stage. Left for a deliberate fix: `--stages` should be able to skip
reconcile_tu, or a draft should be able to opt out.
The two surviving non-stub source edits are byte-neutral (proven by the SHA above): a §304
migrated-rodata re-emission (`D_801A01EC`, the exact form this TU already uses three times, needed
because banking the body deletes the .s that carried the island word), and one typedef moved up so a
function above it can see it (typedefs emit no bytes).
Also banked the 10th stub (func_801ADA10) that defect 3 had been silently blocking.
Regression-checked by the agent: main, md_MAIN_003, md_SC07_003, md_SC03_073, md_MAIN_011 all
rebuild BYTE-IDENTICAL. A full R22 follows before this session closes.
md_MAIN_011:func_800CF28C · md_MAIN_003:func_800D0268/func_800D0740/func_800D0C50, all byte-verified
from a clean rebuild and counted from the SOURCE: md_MAIN_011 is now FULLY MATCHED (0 open stubs),
md_MAIN_003 is down to 1 (func_800CF3E8).
THE PREMISE I HANDED THE AGENT WAS WRONG, AND IT SAID SO. md_MAIN_011 is already a whole-object -O0
module — no carve was needed. Its real blocker was tools/jtbl_rodata_pads._s_rodata_span ignoring a
trailing `.align`, the SAME defect this session fixed for md_SC07_003 from the other direction: two
agents converged on it independently. Adopted this agent's stricter form (only a TRULY trailing
align rounds `hi`; an interior one is followed by data that sets `hi` higher anyway).
Note WHY it stayed latent: `derive`'s zero_gap self-corrects a 1-3 byte undershoot whenever the next
stream item is an anchor. A C jump table has NO anchor — so the bug can only fire the moment someone
banks a switch function into such an object, and when it fires it accuses the CARVE ("island layout
drift"), not itself.
md_MAIN_003 needed one new -O0 object, and the boundary I proposed (0x1f74 -> 0x1e58) was both too
narrow and off by 0x2B8. The carve made is `md_MAIN_003_o0e` at 0x1308 (vram 0x800D0100) running to
the existing o0c boundary: everything in that span is a §265 verbatim __asm__ body or an INCLUDE_ASM
stub — zero optimizable C — so the whole tail flips with one cut. Proved byte-identical with NOTHING
banked first (§431 discipline), then the three drafts gated one at a time.
TWO MORE GENERAL DEFECTS FIXED IN jr_isolate_all, both of which silently mis-place a boundary:
* an item-less CLOSING region emitted a duplicate `- [off, c, …]` line and the validator refused;
the empty-region skip covered only region 0, and `_partition`'s empty `footer` made the closing
region look non-empty.
* A §265 VERBATIM __asm__ BODY IS PREAMBLE, AND PREAMBLE IS ASSUMED BYTE-NEUTRAL. It is not — it
emits bytes. `parse_overlay_c` has four addressed-anchor forms and a verbatim body is none of
them, so it attaches to the NEXT anchor: cutting at func_800D0268 would have moved 0x168 bytes of
other functions into the new object while the yaml claimed the region starts higher. New
`_region_emit_start()` derives the yaml offset from the region's CONTENT (item addresses + every
.globl/.ent the text names that resolves inside the object) and takes min(cut, emit), so a
boundary can only move DOWN. Where no verbatim asm is in play it equals the cut — every existing
isolate is unchanged.
BLAST RADIUS PROVEN, not argued: jtbl_rodata_pads is in the build path (`--derive` for md_*/main),
so the agent rebuilt main + all 70 md_* from scratch (71/71) and then ran the full fleet:
**make check-all 213/213 passed, 0 failed**, main 143dbb89 BYTE-IDENTICAL.
CENSUS, denominator asserted (1057 live stubs, 0 without a .s): exactly ONE -O0-prologue stub
remains stranded in an -O2 TU fleet-wide — main:func_8002C410 in src/800_b.c, 299 ins. Nothing more
should be built for this class; the general tool already existed and what was missing was
correctness, not coverage.
`_s_rodata_span` summed only the DATA an included `.s` emits, so a file ending `.asciz "r"` +
`.align 2` measured 0x801A00D8..DA instead of ..DC. The island walk then landed 2 bytes short and
`--derive` aborted with `C table entry 0 at 0x801A00DA ... island layout drift` — a true statement
about a span that was never the real one. Worse, the Makefile pipes md_*/main through `--derive`
without `set -o pipefail`, so the failure could yield a short object rather than stopping the build.
Three lines: round `hi` up to the trailing align, which is what the assembler actually emits.
CONTROLS (both on UNMODIFIED sources, so this is proven byte-neutral, not argued):
md_SC07_003 clean rm -rf + extract + build -j + check -> BYTE-IDENTICAL 46af79a1
gate_main --assert-baseline -> BASELINE GREEN 143dbb89
Found by a drafting agent (md_SC07_003/func_801A09C8) that ran its own gate reject to ground
instead of respelling its body, and proved the patch in scratch first: with the fix its draft's
.rodata is byte-identical to the green control and .text differs in exactly 1 of 6266 words — a
%lo(jtbl) carrying a section-symbol reloc that three already-green C-jtbl functions in the same
object already ship. Cookbook §436-C, with the habit that found it.
* draw_waves Usage advertised [--no-main], which argparse never defined (the flags are
--main / --only-main, and main is excluded by default), and omitted --exclude-file,
which is now a PREREQUISITE that refuses a stale list.
* jr_isolate's STATUS block still declared the tool BLOCKED on split_src_region with the
blocker unbuilt. Five defects were fixed this session and it runs the full chain to
completion; what remains is a duplicate-definition class at assembly. Says so, and
points at §431 as the cheaper route than finishing the item model.
* ld_interleave's layout diagram — the first thing anyone reads — showed the pre-S72
three-piece island with 6324C.data.o. main's island is SEVEN pieces driven by --order;
--front/--tail is the overlay form now.
* jtbl_rodata_pads described a stored-spec-only filter and advertised guards that no
longer all exist; --derive serves main since S72.
main's gate could only ever say "got X want Y". S71 read 7 such verdicts as body
rejects and recorded 11 functions as "PROVEN gate-rejects, §376 in its purest form".
They are not: all 11 are switch functions, and the blocker is that main has had
exactly ONE rodata carve since Phase 7 (LZSS's jtbl_80072A38). Every other main jump
table stayed raw in the tail data, so a drafted switch DOUBLE-EMITTED its table, the
image grew (+28/+52/+76/+84 measured), and all 238 symbols above 0x80072A4C shifted.
* tools/main_diff_locate.py (NEW) — turns a red image into a named list of divergent
symbols via the linker map; per-byte attribution, self-test flips a byte at a known
address and asserts the containing symbol (plus the identical-pair direction).
* gate_main.py — PRESERVES the red image + map before the R40 baseline control
rebuilds over it, and auto-localizes: BODY REJECT vs PLUMBING REJECT vs MIXED. Also
-j on the build (was single-threaded) and the §376 drop list written to
.run/gate_main_dropped.json with the reconciliation chain.
* splat.us.exe.yaml — the .rodata carve extends from the LZSS table alone to the whole
contiguous game-jtbl span 0x80072A38-0x80072C70 (12 tables, one 800.o run).
Byte-neutral with no drafts substituted (probed first).
* jtbl_rodata_pads.py — --derive now works for main: one file-0-vram expression makes
both address->bytes and yaml-piece->address correct for the EXE's 0x800 header and
leaves flat overlays unchanged. Makefile arms it for BINARY=main.
Banked byte-identical: func_8001A114, func_8001AAD0, func_8001AF34 — three of the
eleven. 25 of main's 59 frontier functions (6,215 of 12,912 instructions) are in this
class; the remaining spans need src/800.c split at the TU boundaries the spans reveal.
- ROOT CAUSE PINNED (the session-2 half-pin was INVERTED; both probes were vacuous, R35):
cc1 emits .align 3 before EVERY jump table; maspsx passes it VERBATIM (the :435 'drop' is
an inventory-only pass); as bakes the pad SECTION-RELATIVE; link placement was never guilty
(SUBALIGN(2) + ALIGN(.,4) place 4-mod-8 carve starts tight). Merging originally-separate
TUs fires an intra-TU align where the original packed tight -> +4 at rodata 0xCC ->
image-wide %lo shift. Honest probes persisted: .run/probe_jtbl/ (verdict.md + objdumps).
- NEW tools/jtbl_rodata_pads.py: post-maspsx filter replaces each rodata .align 3 with the
ORIGINAL's exact pad bytes per a JTBL_PADS spec; fail-loud on table-count drift /
non-align-3 / non-jtbl rodata content. Byte-proven: verbatim 0xE4 pad-at-0xCC ->
filtered 0xE0 tight (= the merged carve span).
- jtbl_carve.py: spec-aware same-subseg merge (gap 0 or 4-with-zero-payload-word; else
NON-CONTIGUOUS -> isolate), interval-arithmetic pad specs (committed values CARRIED,
never re-derived), JTBL_PADS target-var emission into overlays.mk + revert() restore +
stale-.o invalidation; the false 'maspsx drops .align' docstring corrected (H5).
- Makefile: $(if $(JTBL_PADS),| jtbl_rodata_pads.py ...) stage in build/src/%.o + file-scope
empty default (env-shield). jtbl_family_bank.stub_file: duplicate-stub fail-loud (the
earlier 'ladder failure' was a wrong-TU splice into a stale _a.c stub, byte-witnessed).
- R22 clean-fleet WITH the fix wired: 140/140 byte-identical, tools-health green
(dedup 1846/0, C1 234205/234205), ZERO new banks -- fleet-neutral by construction.
- cookbook §8e (the jtbl alignment law) + §8a/§8a-pad corrections; decision-log R31 entry;
SETUP.md tool row; .gitignore allowlist for the probe verdict artifacts.
Drew T