mirror of
https://github.com/Druthulu/BFM-decomp
synced 2026-09-26 13:33:34 -04:00
feat(phase-29 §8e): jtbl 8-align pad-spec filter — the 4-giant unblock tooling (fleet-neutral)
- ROOT CAUSE PINNED (the session-2 half-pin was INVERTED; both probes were vacuous, R35): cc1 emits .align 3 before EVERY jump table; maspsx passes it VERBATIM (the :435 'drop' is an inventory-only pass); as bakes the pad SECTION-RELATIVE; link placement was never guilty (SUBALIGN(2) + ALIGN(.,4) place 4-mod-8 carve starts tight). Merging originally-separate TUs fires an intra-TU align where the original packed tight -> +4 at rodata 0xCC -> image-wide %lo shift. Honest probes persisted: .run/probe_jtbl/ (verdict.md + objdumps). - NEW tools/jtbl_rodata_pads.py: post-maspsx filter replaces each rodata .align 3 with the ORIGINAL's exact pad bytes per a JTBL_PADS spec; fail-loud on table-count drift / non-align-3 / non-jtbl rodata content. Byte-proven: verbatim 0xE4 pad-at-0xCC -> filtered 0xE0 tight (= the merged carve span). - jtbl_carve.py: spec-aware same-subseg merge (gap 0 or 4-with-zero-payload-word; else NON-CONTIGUOUS -> isolate), interval-arithmetic pad specs (committed values CARRIED, never re-derived), JTBL_PADS target-var emission into overlays.mk + revert() restore + stale-.o invalidation; the false 'maspsx drops .align' docstring corrected (H5). - Makefile: $(if $(JTBL_PADS),| jtbl_rodata_pads.py ...) stage in build/src/%.o + file-scope empty default (env-shield). jtbl_family_bank.stub_file: duplicate-stub fail-loud (the earlier 'ladder failure' was a wrong-TU splice into a stale _a.c stub, byte-witnessed). - R22 clean-fleet WITH the fix wired: 140/140 byte-identical, tools-health green (dedup 1846/0, C1 234205/234205), ZERO new banks -- fleet-neutral by construction. - cookbook §8e (the jtbl alignment law) + §8a/§8a-pad corrections; decision-log R31 entry; SETUP.md tool row; .gitignore allowlist for the probe verdict artifacts.
This commit is contained in:
Drew T
@@ -36,6 +36,11 @@
|
||||
!/.run/giants/*.c
|
||||
!/.run/giants/*.md
|
||||
!/.run/giants/*.sh
|
||||
!/.run/probe_jtbl/
|
||||
/.run/probe_jtbl/*
|
||||
!/.run/probe_jtbl/verdict.md
|
||||
!/.run/probe_jtbl/rodata_verbatim.objdump
|
||||
!/.run/probe_jtbl/rodata_filtered.objdump
|
||||
!/.run/giants/fable_cd4/
|
||||
/.run/giants/fable_cd4/*
|
||||
!/.run/giants/fable_cd4/*.c
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
|
||||
.run/probe_jtbl/tu_filtered.o: file format elf32-tradlittlemips
|
||||
|
||||
Contents of section .rodata:
|
||||
0000 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0010 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0020 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0030 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0040 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0050 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0060 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0070 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0080 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0090 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
00a0 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
00b0 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
00c0 48000000 48000000 48000000 60670000 H...H...H...`g..
|
||||
00d0 60670000 54690000 6c6a0000 146c0000 `g..Ti..lj...l..
|
||||
[ 9] .rodata PROGBITS 00000000 00f6f0 0000e0 00 A 0 0 4
|
||||
[10] .rel.rodata REL 00000000 017abc 0001c0 08 I 12 9 4
|
||||
@@ -0,0 +1,22 @@
|
||||
|
||||
.run/probe_jtbl/tu_verbatim.o: file format elf32-tradlittlemips
|
||||
|
||||
Contents of section .rodata:
|
||||
0000 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0010 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0020 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0030 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0040 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0050 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0060 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0070 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0080 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
0090 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
00a0 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
00b0 50000000 50000000 50000000 50000000 P...P...P...P...
|
||||
00c0 48000000 48000000 48000000 00000000 H...H...H.......
|
||||
00d0 60670000 60670000 54690000 6c6a0000 `g..`g..Ti..lj..
|
||||
00e0 146c0000 .l..
|
||||
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
|
||||
[ 9] .rodata PROGBITS 00000000 00f6f0 0000e4 00 A 0 0 8
|
||||
[10] .rel.rodata REL 00000000 017ac0 0001c0 08 I 12 9 4
|
||||
@@ -0,0 +1,33 @@
|
||||
# jtbl 8-align probe — the HONEST stage-walk (2026-07-18, replaces the vacuous probe_jtbl_{cc1,maspsx}.s)
|
||||
|
||||
**Target:** func_80131340 (424 ins) draft spliced over its INCLUDE_ASM stub in the committed
|
||||
src/ov_SC01_077/ov_SC01_077_jr_8012ACE0.c (probe copy: tu_spliced.c). Real pipeline, real flags.
|
||||
|
||||
## Verdict — the session-2 checkpoint's half-pin ("cc1 AND maspsx emit .align 2") is INVERTED
|
||||
|
||||
| stage | observable | result |
|
||||
|---|---|---|
|
||||
| cc1 (Sony gcc 2.7.2) | directives before EACH jump table | `.rdata` + **`.align 3`** + label (cc1.s L46-48 → $L15/51 words = jtbl_801D8078; L10578-80 → $L810/5 words = jtbl_801D8144). Exactly one `.align 3` per table, exactly 2 tables in the TU. |
|
||||
| maspsx (pinned 874855c) | `.align` handling | **passed through VERBATIM** (maspsx.s L40-42, L9312-14; `.rdata`→`.section .rodata` only). The `continue` at maspsx/__init__.py:435-437 is inventory-only (`preprocess_lines`); the output path catch-all (L872-873) re-emits `.align`. |
|
||||
| GNU as 2.42 | .rodata content | **size 0xE4, sh_addralign=8, ZERO PAD WORD AT OFFSET 0xCC** (rodata_verbatim.objdump: `00c0 48000000 48000000 48000000 00000000` then table 2 at 0xD0). The +4 interior pad, baked into the object — the linker can never remove it. |
|
||||
| as control | `.section .rodata`+`.word` alone → Al=**4** (ctrl.o); +`.align 3` → Al=**8** (ctrl2.o) | the Al=8 is cc1's directive, NOT an as default. |
|
||||
|
||||
## Why this breaks the image
|
||||
Original layout (asm/ov_SC01_077/data/tail2.data.s): jtbl_801D8078's 51 entries end 0x801D8144 exactly
|
||||
where jtbl_801D8144 begins — originally-SEPARATE TUs, packed TIGHT by the original linker. Merging both
|
||||
into one decomp TU makes cc1's intra-TU `.align 3` fire at 0xCC (%8==4) where the original had a tight TU
|
||||
boundary → every downstream data symbol +4 → %lo relocs break image-wide (.run/build_g2.log SHA1 FAIL).
|
||||
Intra-TU pads ARE real elsewhere (jtbl_801D8158/8170/8188/81A0: 5 entries + one 0x00000000 each, every
|
||||
non-first table landing %8==0) — so the align must be REPLACED by the original's exact pad bytes
|
||||
(pad ∈ {0,4} from carve-interval arithmetic), not blanket-dropped. Isolation is not general: as computes
|
||||
`.align` SECTION-RELATIVE, so a 4-mod-8-start multi-table object would mis-pad internally.
|
||||
|
||||
**Fix:** tools/jtbl_rodata_pads.py post-maspsx filter, per-object JTBL_PADS spec from jtbl_carve.
|
||||
For this TU the spec is `0,0` → expected filtered .rodata = 0xE0, table 2 tight at 0xCC (= the merged
|
||||
carve span 0xaff20..0xB0000).
|
||||
|
||||
## CONFIRMED (same session): the filter's byte proof
|
||||
`maspsx.s | jtbl_rodata_pads.py --pads 0,0 | as` → `.rodata` size **0xE0**, Al=4, table 2 TIGHT at
|
||||
0xCC (rodata_filtered.objdump) = the merged carve span 0xaff20..0xB0000 exactly. Guards verified
|
||||
fail-loud (1-spec / 3-spec / pads[0]=4 all rc=1 with attributable messages). The pad=4 direction
|
||||
verified synthetically (`.word 0` emitted in place of the align).
|
||||
@@ -436,6 +436,12 @@ ASPSX_VERSION := 2.56
|
||||
# maspsx leaves a bare divu and div/rem functions never match. Only affects div/rem,
|
||||
# so the all-INCLUDE_ASM build and div-free functions are unchanged.
|
||||
MASPSX_FLAGS := --expand-div
|
||||
# Phase-29 §8e: per-object jump-table pad spec (tools/jtbl_rodata_pads.py). Set ONLY as a
|
||||
# target-specific var by tools/jtbl_carve.py in config/overlays.mk for multi-table .rodata
|
||||
# carve spans; the file-scope empty default shields the recipe from an inherited environment
|
||||
# variable accidentally arming the filter fleet-wide (a plain `JTBL_PADS=... make` would
|
||||
# otherwise become a global make var). Unset => the compile pipeline is byte-unchanged.
|
||||
JTBL_PADS :=
|
||||
|
||||
# Object set must match the splat linker script's references. After the Phase-6 asm->c
|
||||
# flip the text subseg is src/800.c -> build/src/800.o; the per-function
|
||||
@@ -539,10 +545,15 @@ build/assets/%.o: assets/%.bin
|
||||
# fingerprint ladder PINS the triple (then this block + ASPSX_VERSION are updated, G8).
|
||||
CPPFLAGS := -lang-c -Iinclude -undef -Wall -fno-builtin -Dmips -D__GNUC__=2 -D__OPTIMIZE__ -Dpsx -D_PSYQ -D_MIPSEL -D_LANGUAGE_C
|
||||
CC1FLAGS := -quiet -O2 -G0 -mips1 -mcpu=3000 -mgas -msoft-float -fgnu-linker
|
||||
# The optional jtbl_rodata_pads stage (Phase-29 §8e) is inserted only when the object has a
|
||||
# JTBL_PADS target-specific var (written by tools/jtbl_carve.py for multi-table .rodata carve
|
||||
# spans): it replaces cc1's per-table `.align 3` with the ORIGINAL's exact pad bytes, so a merged
|
||||
# span reproduces the original packing regardless of section-start parity. Unset => stage absent,
|
||||
# pipeline byte-identical to pre-§8e.
|
||||
build/src/%.o: src/%.c
|
||||
@mkdir -p $(dir $@)
|
||||
@echo " CC $@"
|
||||
@set -o pipefail; $(CPP) $(CPPFLAGS) -MMD -MP -MT $@ -MF $(@:.o=.d) $< | $(CC1_PSX) $(CC1FLAGS) | $(VENV_PY) $(MASPSX) --aspsx-version=$(ASPSX_VERSION) $(MASPSX_FLAGS) | $(AS) $(ASFLAGS) -o $@
|
||||
@set -o pipefail; $(CPP) $(CPPFLAGS) -MMD -MP -MT $@ -MF $(@:.o=.d) $< | $(CC1_PSX) $(CC1FLAGS) | $(VENV_PY) $(MASPSX) --aspsx-version=$(ASPSX_VERSION) $(MASPSX_FLAGS) $(if $(JTBL_PADS),| $(VENV_PY) tools/jtbl_rodata_pads.py --pads $(JTBL_PADS)) | $(AS) $(ASFLAGS) -o $@
|
||||
|
||||
# Per-module optimization override (SETUP §5.5 — per-module compiler mixing). The boot/
|
||||
# main/game-mode-dispatch module (src/boot.c, vram 0x80010000-0x800123F0) was compiled at
|
||||
|
||||
@@ -676,6 +676,7 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
|
||||
| | `tools/make_snd_used.py` | **(Phase 8)** Build the combined libspu+libsnd curated dir (alias dedup by byte-match, scattered-`.bss` exclusions; §9.6). |
|
||||
| | `tools/make_apicard_used.py` | **(Phase 8)** Build the combined libapi+libcard curated dir (§9.6). |
|
||||
| | `tools/ld_interleave.py` | Interleave linker inputs to match original section ordering. |
|
||||
| | `tools/jtbl_rodata_pads.py` | **(Phase 29, cookbook §8e)** Post-maspsx filter for multi-table `.rodata` carve spans: REPLACES each cc1 `.align 3` (which is section-relative and would mis-pad a merged/4-mod-8 span) with the ORIGINAL's exact pad bytes per the object's `JTBL_PADS` spec (written by `jtbl_carve` into `config/overlays.mk`; armed via `$(if $(JTBL_PADS),…)` in the Makefile `build/src/%.o` recipe). Fail-loud on table-count drift / non-`.align 3` / non-jtbl rodata content. Unset var ⇒ pipeline byte-identical. |
|
||||
| | `tools/split_src_region.py` | Split a `src/` region file at object boundaries. |
|
||||
| | `tools/rollout_whale_o0.py` | **(Phase 24 W9)** Roll out the -O0 whale `func_80144B9C` ×134: per single-file overlay, line-split `<ov>.c` at the whale, carve the yaml code subseg into before/`_o0b`(-O0)/`_after`, write a thin `<ov>_o0b.c` that `#include`s the shared `src/shared/func_80144B9C.h`. Idempotent; the `WHALE_O0B_OBJS` Makefile wildcard -O0-compiles all `_o0b.o` (cookbook §38). |
|
||||
| **Reports** | `tools/progress.py` | Per-binary decomp progress (`make report`); counts dedup-shared fns as REAL via the registry (Phase 11). **`--fleet`** (Phase 15) aggregates all 136 binaries → `docs/progress.fleet.md` (deterministic, source-derived). **`--weighted`** (Phase 25) prints the two BYTE/instruction-weighted metrics from `.run/sig.*.jsonl` (executable code only): **instr-weighted** (fleet per-overlay, the decomp.dev-display number) + **distinct-code** (dedup, each unique fn once, the distinct-RE number); both also fold into `--fleet` alongside the ×134-inflated function-count %. Needs `make sig-overlays` first; degrades gracefully without sigs. |
|
||||
|
||||
@@ -1353,3 +1353,42 @@ are self-contained") is false whenever `engine_core.h` already declares the memb
|
||||
signature. `--fix-def-sig` should likely be default-on for the h_seq path. And the meta-lesson, hammered
|
||||
three times in one task: **a masked/intermediate MATCH is a candidate, never a diagnosis — reproduce the
|
||||
real build and read the real error before naming the cause (R35).**
|
||||
|
||||
### 2026-07-18 — P29 jtbl 8-align wall: the half-pin was INVERTED (vacuous probes), the fix is a pad-spec filter (§8e)
|
||||
|
||||
**Context.** The 4 jtbl giants (`func_80131340`/`80159C84`/`8013C414`/`8013F350`, all match_one MATCH,
|
||||
~2.6M agent-tokens of preserved drafts) were blocked on ONE tooling gap: banking `func_80131340` into the
|
||||
shared `_jr_8012ACE0` TU produced a +4 pad at rodata 0xCC → image-wide %lo shift → SHA1 fail. The session-2
|
||||
checkpoint recorded a half-pin — "cc1 AND maspsx both emit the jtbl `.align 2`; the +4 is a downstream
|
||||
`as`/`ld_interleave` artifact" — and told the next session to start from there.
|
||||
|
||||
**What the evidence actually said (R35, again).** Both preserved probes were VACUOUS: an empty `j $31`
|
||||
function with NO jump table — the `.align 2` they "showed" was the function-entry `.text` align. The honest
|
||||
stage-walk (real draft spliced into the real TU, `.run/probe_jtbl/`) inverted every clause: cc1 emits
|
||||
**`.align 3` before every table**; **maspsx passes it through verbatim** (the famous `maspsx.py:435` "drops
|
||||
`.align`" is an inventory-only pass — the §8a-pad cookbook claim was false too); `as` bakes the pad
|
||||
section-relative; and the LINK side was never guilty (`SUBALIGN(2)` + `ALIGN(.,4)` place even 4-mod-8 carve
|
||||
starts tight — the banked `0xb07dc` carve proves it). Two Explore subagents produced OPPOSITE readings of
|
||||
maspsx (one read the inventory pass as the output path); the tie was broken by reading the code path myself
|
||||
plus one byte observable — the clean object's `.rodata` sh_addralign=8, which only a surviving `.align 3`
|
||||
explains (R34: make oracles argue; R14: settle on bytes).
|
||||
|
||||
**The design fork and why the filter won.** The obvious fixes all fail a generality test: blanket align-demote
|
||||
breaks the main EXE's island (its intra-TU pads are load-bearing); pure isolation fails multi-table functions
|
||||
whose first table sits at vram ≡4 mod 8 (`.align` is section-relative, so the section-start parity flips every
|
||||
internal pad); sed/as/ld have no per-occurrence mechanism. The winning shape: **replace each rodata `.align 3`
|
||||
with the ORIGINAL's exact pad bytes** — derived per span by interval arithmetic from the carve config
|
||||
(`pad[K] = start[K] − end[K−1]` ∈ {0,4}), emitted as a per-object `JTBL_PADS` make var, applied by a ~50-line
|
||||
post-maspsx filter with fail-loud drift guards. Parity-independent, per-sibling self-adapting (each overlay's
|
||||
own addresses), and structurally fleet-neutral (every pre-existing carve is single-table → no var → pipeline
|
||||
byte-identical). A red-team subagent pre-verified the transform empirically (verbatim 0xE4/pad-at-0xCC vs
|
||||
filtered 0xE0/tight) and surfaced 7 hardening items, including the LATENT bug that produced the original
|
||||
failure (tight abutment silently merged into a bytes-impossible span) and a byte-witnessed wrong-TU splice
|
||||
(`stub_file` first-match returned a stale duplicate stub — the "conflicting types" cascade was never the
|
||||
draft's fault).
|
||||
|
||||
**Hindsight better-path.** The checkpoint's half-pin cost nothing this time because R35 forced re-derivation —
|
||||
but only because the vacuous probes were LOOKED AT. The transferable rule: **a probe whose output contains no
|
||||
instance of the probed thing pins nothing** — check that first, before trusting any recorded verdict. And when
|
||||
a wall involves a multi-stage pipeline, walk it stage-by-stage with one byte observable per stage before
|
||||
designing anything; the whole design fell out of five observables in under an hour.
|
||||
|
||||
@@ -353,7 +353,8 @@ a 25-ins single-jtbl jr-function in ov_SC01_077):**
|
||||
orthogonal — reconcile first, then the carved jtbl lands byte-exact.
|
||||
- **Alignment:** gcc emits the jtbl `.rdata .align 3` (8-byte). If the original jtbl address is 8-aligned
|
||||
(`jtbl_801D8078`, 0x…078) there is no pad and it lands exact. A **4-aligned** original address (`jtbl_801D8AFC`)
|
||||
would force a 4-byte align pad → handle then (not hit by the PoC target).
|
||||
would force a 4-byte align pad → **HANDLED (Phase 29): the §8e `JTBL_PADS` pad-spec filter** (hit for real by
|
||||
`jtbl_801D8144` in the func_80131340 bank).
|
||||
- **rtu_match is NOT a whole-binary gate for jr-functions** — it masks relocs AND excludes the §8 jtbl rodata, so
|
||||
it MATCHes a body whose switch is subtly wrong (e.g. func_80159C84's 2nd jtbl was 5 words vs the real 6 — a
|
||||
false-MATCH). Always confirm jr-function cracks with the whole-binary gate (which now works, via this carve).
|
||||
@@ -364,6 +365,11 @@ a 25-ins single-jtbl jr-function in ov_SC01_077):**
|
||||
|
||||
### §8a-pad — a trailing `.word 0x00000000` under a jtbl dlabel is `.align` PAD, not an entry (Phase 26 session 6, byte-proven)
|
||||
|
||||
> **⚠ CORRECTED by §8e (Phase 29):** the "maspsx drops all `.align`" rationale below is **FALSE** (that
|
||||
> `continue` is in an inventory-only pass; the output path passes `.align` verbatim). The TRIM itself remains
|
||||
> correct — the pad belongs to the NEXT table's `.align 3`, absent when that owner isn't compiled in the same
|
||||
> object. Multi-table spans now reproduce interior pads via the §8e `JTBL_PADS` spec filter.
|
||||
|
||||
**This retroactively explains the §8a `func_80159C84` "5 words vs the real 6" false-MATCH.**
|
||||
|
||||
The raw `dlabel jtbl_XXXXXXXX` in `asm/<ov>/data/*.data.s` can span one word MORE than the switch has
|
||||
@@ -507,6 +513,51 @@ the base for the later recovery stages. **First sibling byte-identical; 562-ins
|
||||
finds only make's `Error 33`. Grep for the diagnostic text (`conflicting types`, `undeclared`, `parse error`,
|
||||
`redefinition`) instead, and remember `warning: conflicting types for built-in function 'memcpy'` is benign.
|
||||
|
||||
## §8e The jtbl ALIGNMENT LAW + the pad-spec filter — multi-table .rodata spans (Phase 29, byte-proven; `.run/probe_jtbl/verdict.md`)
|
||||
|
||||
**Two prior cookbook claims are CORRECTED here** (both were instrument errors, R35):
|
||||
- §8a-pad's "**maspsx drops all `.align`**" is **FALSE**. The `continue` at `maspsx/__init__.py:435` is in
|
||||
`preprocess_lines` — an *inventory-only* pass (sbss/bss/sdata dicts) whose skips produce no output; the real
|
||||
output path (`process_line`, L872-873 catch-all) re-emits `.align` **verbatim**. The §8a-pad *trim* is still
|
||||
right, for a different reason: the trailing pad word belongs to the NEXT table's `.align 3`, which is only
|
||||
emitted when that next owner is compiled in the same object.
|
||||
- The Phase-29 session-2 "half-pin" ("cc1 AND maspsx emit the jtbl `.align 2`") was **inverted**, measured off
|
||||
VACUOUS probes (an empty `j $31` fn — no jtbl in them at all; the only `.align 2` was the function-entry
|
||||
`.text` align). Lesson: **a probe whose output contains no instance of the thing being probed pins nothing.**
|
||||
|
||||
**The law (each link byte-verified in `.run/probe_jtbl/`):**
|
||||
1. cc1 (Sony gcc 2.7.2) emits `.rdata` + **`.align 3`** + label before **every** switch jump table
|
||||
(probe: 2 tables in one TU → 2× `.align 3`; the clean single-table object's `.rodata` sh_addralign=8).
|
||||
2. maspsx passes `.align` through; `as` bakes the pad into the section **SECTION-RELATIVE** — the linker can
|
||||
never remove an intra-object pad. (`as` control: bare `.word` section → Al=4; with `.align 3` → Al=8.)
|
||||
3. Link placement is always TIGHT: `SUBALIGN(2)` (splat `subalign: 2`, fleet-wide) + `ld_interleave`'s
|
||||
`. = ALIGN(., 4)` override input-section alignment — a 4-mod-8 carve start places exactly (the banked
|
||||
`0xb07dc` carve is the byte proof). So **only intra-object pads can diverge from the original.**
|
||||
4. ORIGINAL layout semantics: originally-separate TUs pack **TIGHT** (PSX linker, 4-aligned placement:
|
||||
jtbl_801D8078's 51 entries end 0x801D8144 exactly where the next TU's table begins, %8==4); **intra-TU**
|
||||
consecutive tables carry a REAL zero-word pad wherever the previous table ends ≡4 mod 8
|
||||
(tail2.data.s: jtbl_801D8158/8170/8188/81A0 = 5 entries + one `.word 0` each).
|
||||
5. ⇒ Merging originally-separate TUs into one decomp TU makes cc1's intra-TU `.align 3` fire where the
|
||||
original had a tight TU boundary: a non-first table at original vram ≡4 mod 8 gains a **+4 interior pad**
|
||||
→ every downstream data symbol shifts → `%lo` relocs break image-wide (func_80131340: pad at rodata 0xCC,
|
||||
SHA1 fail from a clean build). Conversely a genuine intra-TU pad must be REPRODUCED.
|
||||
6. **Isolation does NOT fix this in general**: `.align` is section-relative, so an isolated object whose
|
||||
first table starts at vram ≡4 mod 8 flips the parity of every INTERNAL align — a multi-table function
|
||||
with a 4-mod-8 first table would mis-pad inside its own object. Only explicit pad control is general.
|
||||
|
||||
**The mechanism (`tools/jtbl_rodata_pads.py` + `jtbl_carve` + Makefile `JTBL_PADS`):** for a multi-table
|
||||
span, `jtbl_carve` derives each boundary's pad by **interval arithmetic** (`pad[K] = start[K] − end[K−1]`
|
||||
∈ {0,4}; a 4-gap must be a verifiably-zero payload word, else NON-CONTIGUOUS→isolate) and writes a
|
||||
per-object `build/src/<ov>/<sub>.o: JTBL_PADS := 0,4,…` target var into `config/overlays.mk`; the Makefile
|
||||
pipes that object through the filter, which **REPLACES each rodata `.align` with the spec'd pad bytes**
|
||||
(`.word 0` or nothing). Per-sibling self-adapting (each overlay's own addresses), fail-loud on drift
|
||||
(spec count ≠ align count; non-`.align 3`; non-jtbl rodata content). Committed spec values are CARRIED,
|
||||
never re-derived (a committed span's interior boundaries are unrecoverable from its interval).
|
||||
Single-table carves get no var — their pipeline is byte-identical to pre-§8e (their lone `.align 3` at
|
||||
section offset 0 pads nothing and SUBALIGN neutralizes the sh_addralign).
|
||||
- **Object-layer proof before image-layer claims:** verbatim vs filtered objdump — 0xE4/pad-at-0xCC vs
|
||||
0xE0/tight — settled the mechanism before any carve landed. Cheap, decisive, reusable probe shape.
|
||||
|
||||
## §9 Link real PsyQ library objects byte-exact (Phase 7 — GO proven)
|
||||
~350 of BFM's functions are unmodified PsyQ 4.0 SDK code. They are **byte-identical to the real PsyQ library
|
||||
objects**, so link them directly instead of hand-decompiling — and each library `.o` brings its own correct
|
||||
|
||||
@@ -144,6 +144,40 @@ conditional) · main-EXE/B9 + GLM/B6 + resident's 14 walls (P30) · behemoths B7
|
||||
## Per-task log
|
||||
*(appended after each task; the crash-recovery trail — becomes `phase-ends/logs/Phase29.md` at close, R19)*
|
||||
|
||||
- **2026-07-18 — jtbl 8-align fix OPENED (Fable5 Max, plan approved).** Plan mirror:
|
||||
`~/.claude/plans/fable5-max-set-jtbl-ancient-marble.md`. **⚠️ CORRECTION of the session-2 checkpoint's
|
||||
half-pin (R35/R14 — do NOT re-trust it): the claim "cc1 AND maspsx both emit the jtbl `.align 2`" is
|
||||
INVERTED, and both preserved probes (`.run/probe_jtbl_{cc1,maspsx}.s`) are VACUOUS (empty `j $31` fn, no
|
||||
jtbl).** True chain (byte-grounded this session): **(1)** Sony cc1 emits `.align 3` before EACH jump table
|
||||
in `.rdata` (clean `jr_8012ACE0.o` has `.rodata` sh_addralign=8 w/ a single 51-entry table); **(2)** maspsx
|
||||
passes `.align` through VERBATIM (`maspsx/__init__.py:872-873` output catch-all; the L435-437 `continue` is
|
||||
an inventory-only pass — `jtbl_carve.py:132`'s "maspsx drops .align" docstring is FALSE); **(3)** `as` bakes
|
||||
the pad SECTION-RELATIVE (intra-object, linker can't remove); **(4)** placement is always tight
|
||||
(`SUBALIGN(2)` fleet-wide + ld_interleave `ALIGN(.,4)` — the 0xb07dc 4-mod-8 carve proves it); **(5)**
|
||||
original semantics: separate TUs pack TIGHT (078→144 abutment), intra-TU tables 8-align w/ real zero-word
|
||||
pads (tail2.data.s 8158/8170/8188/81A0). **THE BUG:** merging originally-separate TUs into one decomp TU
|
||||
fires an intra-TU `.align 3` where the original had a tight TU boundary → +4 at rodata 0xCC → image-wide
|
||||
%lo shift (build_g2.log SHA1 FAIL). **THE FIX (red-team-hardened, empirically pre-verified):** pad-spec
|
||||
filter `tools/jtbl_rodata_pads.py` (post-maspsx, per-object `JTBL_PADS` make var, fail-loud guards) +
|
||||
`jtbl_carve` interval-arithmetic specs + merge relaxation (gap 4 must be verifiably zero) + revert/stale-.o
|
||||
coverage + `jtbl_family_bank` stub_file uniqueness assert (yesterday's ladder failure spliced into a stale
|
||||
`_a.c` duplicate stub — byte-witnessed in bank_func_80131340.log). Isolation is NOT general (as aligns
|
||||
section-relative; a 4-mod-8-start multi-table object mis-pads internally) — kept as NON-CONTIGUOUS fallback.
|
||||
- **✅ 2026-07-18 — jtbl §8e fix BUILT + fleet-neutral (Steps 1-3 of the approved plan).** Honest probes
|
||||
persisted (`.run/probe_jtbl/verdict.md` + objdumps; the vacuous pair deleted): cc1 `.align 3` per table
|
||||
(2 tables → 2 aligns, byte-listed), maspsx verbatim passthrough in situ, **the +4 pad materialized at object
|
||||
rodata 0xCC** (0xE4 verbatim), as-controls (bare rodata → Al=4; +`.align 3` → Al=8). Built: NEW
|
||||
`tools/jtbl_rodata_pads.py` (spec-driven align→pad-bytes replacement; guards byte-tested rc=1 ×3; pad=4
|
||||
direction synthetically proven), `jtbl_carve.py` (spec-aware merge: gap∈{0,4-zero-checked-in-payload},
|
||||
interval-arithmetic pads CARRIED never re-derived, `JTBL_PADS` target-var emission into overlays.mk +
|
||||
revert() restore + stale-.o invalidation, honest docstring), Makefile `$(if $(JTBL_PADS),…)` stage +
|
||||
env-shield default, `jtbl_family_bank.stub_file` duplicate-stub fail-loud. **Object-layer byte proof:**
|
||||
filtered `.rodata` = 0xE0, table 2 TIGHT at 0xCC = the merged carve span exactly. Dry-run build_carve →
|
||||
span `0xaff20..0xb0000` spec `[0,0]` as designed. **R22 clean-fleet WITH the fix wired: 140/140
|
||||
byte-identical, tools-health green (dedup 1846/0, C1 234205/234205), zero new banks — fleet-neutral.**
|
||||
Flywheel captured in-session (R30): cookbook **§8e** + §8a/§8a-pad corrections, decision-log entry,
|
||||
SETUP.md row. NEXT: Step 4 — bank func_80131340 ×1.
|
||||
|
||||
- **2026-07-16 — Phase opened.** Session Start Protocol complete (all 28 PhaseEnds + roadmap +
|
||||
calibration + decision-log read). 3 Explore surveys grounded the plan (tooling state, frontier
|
||||
data, idiom bank). Plan approved at Max/plan-mode. Beginning Task 1.
|
||||
|
||||
+157
-27
@@ -21,6 +21,18 @@ code object contributes at most ONE contiguous .rodata run, so two matched jr-fu
|
||||
code subseg (non-adjacent jtbls in the island) are UNSATISFIABLE -> this tool fails loud, and the
|
||||
caller must first isolate one into its own code subseg (the whale `_o0b` precedent).
|
||||
|
||||
MULTI-TABLE spans & the 8-align pad spec (Phase-29 §8e; .run/probe_jtbl/verdict.md): cc1 emits
|
||||
`.align 3` before EVERY jump table and maspsx passes it through, so a merged same-subseg span whose
|
||||
non-first table sits at an original vram ≡4 mod 8 would gain a +4 interior pad the original does
|
||||
not have (originally-separate TUs pack TIGHT) — and conversely a real intra-TU pad word must be
|
||||
reproduced where the original HAS one. When a span holds >1 table, this tool derives the per-table
|
||||
pad spec by interval arithmetic (pad[K] = start[K] - end[K-1] ∈ {0,4}; a 4-gap must be a zero word
|
||||
in the payload) and writes a per-object `JTBL_PADS` target var into config/overlays.mk; the
|
||||
Makefile then pipes that object through tools/jtbl_rodata_pads.py, which REPLACES each rodata
|
||||
`.align` with the spec'd pad bytes. Committed spec values are carried, never re-derived (a
|
||||
committed span's interior boundaries are not recoverable from its interval). Single-table carves
|
||||
get NO var and keep today's byte-identical pipeline.
|
||||
|
||||
Usage: jtbl_carve.py <ov> --func func_XXXX [--func ...] # add these matched jr-fns to the carve set
|
||||
jtbl_carve.py <ov> --revert # restore the config from git (drop carves)
|
||||
Idempotent: re-running with the same (accumulated) funcs reproduces the same config.
|
||||
@@ -51,6 +63,46 @@ def overlay_vram_base(ov):
|
||||
return int(m.group(1), 16)
|
||||
|
||||
|
||||
def payload_path(ov):
|
||||
"""The overlay's decompressed payload, derived from the config's target_path (R33)."""
|
||||
m = re.search(r"target_path:\s*(\S+)", open(cfg_path(ov)).read())
|
||||
if not m:
|
||||
sys.exit(f"jtbl_carve: no target_path in {cfg_path(ov)}")
|
||||
p = os.path.join(REPO, m.group(1))
|
||||
if not os.path.exists(p):
|
||||
sys.exit(f"jtbl_carve: payload {p} missing — run `make extract` first (R32: refusing to "
|
||||
f"skip the gap-word check)")
|
||||
return p
|
||||
|
||||
|
||||
def payload_word(ov, off):
|
||||
"""The little-endian u32 at file offset `off` in the overlay's payload."""
|
||||
with open(payload_path(ov), "rb") as f:
|
||||
f.seek(off)
|
||||
b = f.read(4)
|
||||
if len(b) != 4:
|
||||
sys.exit(f"jtbl_carve: short read at payload offset 0x{off:x}")
|
||||
return int.from_bytes(b, "little")
|
||||
|
||||
|
||||
# Per-object pad-spec lines in config/overlays.mk (Phase-29 §8e — consumed by the Makefile's
|
||||
# jtbl_rodata_pads.py stage). One line per multi-table span:
|
||||
# build/src/<ov>/<sub>.o: JTBL_PADS := 0,4 # Phase-29 §8e jtbl pad spec (jtbl_carve.py)
|
||||
PADS_COMMENT = " # Phase-29 §8e jtbl pad spec (jtbl_carve.py)"
|
||||
|
||||
|
||||
def pads_line_re(ov):
|
||||
return rf"^build/src/{re.escape(ov)}/(\w+)\.o: JTBL_PADS := ([\d,]+).*$"
|
||||
|
||||
|
||||
def current_pads_specs(ov, txt=None):
|
||||
"""{subseg: [pad,...]} from the CURRENT overlays.mk (carries committed + in-flight values)."""
|
||||
if txt is None:
|
||||
txt = open(os.path.join(REPO, "config/overlays.mk")).read()
|
||||
return {m.group(1): [int(x) for x in m.group(2).split(",")]
|
||||
for m in re.finditer(pads_line_re(ov), txt, re.M)}
|
||||
|
||||
|
||||
def code_pieces(ov):
|
||||
"""[(vram, subseg)] for every `- [off, c, name]` code piece in the config, ascending."""
|
||||
base = overlay_vram_base(ov)
|
||||
@@ -129,12 +181,16 @@ def jtbl_range(ov, jtbl_hex, labels, region_end_vram):
|
||||
not a jump target, and the function's `sltiu <n>` range check names the true entry count
|
||||
(byte-confirmed: `func_8015AE2C` → `sltiu 0x7` = 7 entries, yet the raw dlabel spans 8 words).
|
||||
|
||||
This matters because **maspsx drops `.align`**, so a C-emitted jump table can never reproduce the
|
||||
pad. Carving to the next dlabel would reserve 8 words while the compiled object supplies only 7 —
|
||||
under-filling the `.rodata` piece by 4 bytes and shifting every later symbol (the same +4 image
|
||||
corruption class as §41d). Trimming leaves the pad where it belongs: in the raw post-carve data
|
||||
piece. This also retroactively explains the §8a `func_80159C84` "5 words vs the real 6"
|
||||
false-MATCH."""
|
||||
Why trim (CORRECTED Phase-29 §8e — the old rationale "maspsx drops `.align`" was FALSE; maspsx
|
||||
passes it through, .run/probe_jtbl/verdict.md): the pad belongs to the NEXT table's `.align 3`,
|
||||
which is emitted only if that next table's owner is compiled in the SAME object. When the next
|
||||
owner is unmatched (or another TU), the carved object ends at the last real entry, so carving
|
||||
to the next dlabel would reserve the pad word the object does not supply — under-filling the
|
||||
`.rodata` piece by 4 bytes and shifting every later symbol (the same +4 image corruption class
|
||||
as §41d). Trimming leaves the pad where it belongs: in the raw post-carve data piece — and when
|
||||
the next owner IS matched into the same span later, the merge re-attributes the pad to that
|
||||
table's JTBL_PADS spec (pad[K]=4) and jtbl_rodata_pads.py emits it. This also retroactively
|
||||
explains the §8a `func_80159C84` "5 words vs the real 6" false-MATCH."""
|
||||
start = int(jtbl_hex, 16)
|
||||
if start not in labels:
|
||||
sys.exit(f"jtbl_carve: jtbl_{jtbl_hex} not found in the raw data asm "
|
||||
@@ -208,14 +264,19 @@ def parse_config(ov):
|
||||
|
||||
|
||||
def build_carve(ov, funcs):
|
||||
"""Return (region_lines, order_arg): the regenerated data-region `- [...]` piece lines and the
|
||||
`ld_interleave --order` object list, for the accumulated carve set (existing + the new funcs)."""
|
||||
"""Return (region_lines, order_arg, pads_map): the regenerated data-region `- [...]` piece
|
||||
lines, the `ld_interleave --order` object list, and {subseg: [pad,...]} for every carve span,
|
||||
for the accumulated carve set (existing + the new funcs)."""
|
||||
base = overlay_vram_base(ov)
|
||||
(_, indent, _, _, tail_start, region_end, trailing_present, existing) = parse_config(ov)
|
||||
region_end_vram = base + region_end
|
||||
|
||||
# carves: (start_off, end_off, subseg). Existing ones come from the config (already migrated).
|
||||
carves = list(existing)
|
||||
# carves: (start_off, end_off, subseg, spec). Existing ones come from the config (already
|
||||
# migrated); their pad spec is CARRIED from overlays.mk (a committed span's interior boundaries
|
||||
# are not recoverable from its interval — never re-derive), default [0] = single-table (the
|
||||
# fleet-wide state before Phase-29 §8e; a multi-table span always writes its spec line).
|
||||
specs = current_pads_specs(ov)
|
||||
carves = [(s, e, sub, specs.get(sub, [0])) for (s, e, sub) in existing]
|
||||
have = {c[0] for c in carves}
|
||||
# A new jtbl's end is bounded by the next RAW data dlabel OR the next EXISTING carve start
|
||||
# (an already-carved adjacent jtbl is gone from the data asm, so the raw dlabels alone would
|
||||
@@ -230,28 +291,42 @@ def build_carve(ov, funcs):
|
||||
s_off, e_off = s_vram - base, e_vram - base
|
||||
if s_off in have:
|
||||
continue # idempotent: already carved
|
||||
carves.append((s_off, e_off, sub))
|
||||
carves.append((s_off, e_off, sub, [0]))
|
||||
have.add(s_off)
|
||||
carves.sort()
|
||||
carves.sort(key=lambda c: (c[0], c[1]))
|
||||
|
||||
# A code object emits its jtbls CONTIGUOUS in .rodata (gcc source order). So two carves in the
|
||||
# SAME subseg are byte-correct only if ADJACENT in the island (no unmatched jtbl between) -> merge
|
||||
# them into one spanning .rodata piece. NON-adjacent same-subseg is unsatisfiable (a single object
|
||||
# can't leave a gap for the raw jtbl between) -> isolate one fn into its own subseg (jr_isolate.py).
|
||||
# SAME subseg are byte-correct only if ADJACENT in the island — where "adjacent" is abutting
|
||||
# (gap 0) OR separated by exactly one original `.align 3` pad word (gap 4, verifiably zero in
|
||||
# the payload; Phase-29 §8e) -> merge them into one spanning .rodata piece, recording the
|
||||
# boundary pad in the span's spec (jtbl_rodata_pads.py reproduces it at compile time).
|
||||
# Any other same-subseg gap is unsatisfiable (a single object can't leave a hole for the raw
|
||||
# jtbl between) -> isolate one fn into its own subseg (jr_isolate_all.py).
|
||||
merged = []
|
||||
for s_off, e_off, sub in carves:
|
||||
if merged and merged[-1][2] == sub and merged[-1][1] == s_off:
|
||||
merged[-1] = (merged[-1][0], e_off, sub) # extend the contiguous same-subseg run
|
||||
else:
|
||||
merged.append((s_off, e_off, sub))
|
||||
for s_off, e_off, sub, spec in carves:
|
||||
if merged and merged[-1][2] == sub:
|
||||
gap = s_off - merged[-1][1]
|
||||
if gap in (0, 4):
|
||||
if gap == 4:
|
||||
w = payload_word(ov, merged[-1][1])
|
||||
if w != 0:
|
||||
sys.exit(
|
||||
f"jtbl_carve: the 4-byte gap at 0x{merged[-1][1]:x} between same-subseg "
|
||||
f"carves is 0x{w:08x}, not a zero .align pad word — treating as "
|
||||
f"NON-CONTIGUOUS. Isolate one matched jr-function into its own code "
|
||||
f"subseg first (tools/jr_isolate_all.py), then re-carve.")
|
||||
ps, _, _, pspec = merged[-1]
|
||||
merged[-1] = (ps, e_off, sub, pspec + [gap] + spec[1:])
|
||||
continue
|
||||
merged.append((s_off, e_off, sub, spec))
|
||||
seen_subsegs = {}
|
||||
for s_off, _, sub in merged:
|
||||
for s_off, _, sub, _ in merged:
|
||||
if sub in seen_subsegs:
|
||||
sys.exit(
|
||||
f"jtbl_carve: subseg '{sub}' would host NON-CONTIGUOUS .rodata carves "
|
||||
f"(0x{seen_subsegs[sub]:x} and 0x{s_off:x}) — a single object can't leave a gap for the "
|
||||
f"unmatched jtbl between them. Isolate one matched jr-function into its own code subseg "
|
||||
f"first (tools/jr_isolate.py, the whale `_o0b` precedent), then re-carve.")
|
||||
f"first (tools/jr_isolate_all.py, the whale `_o0b` precedent), then re-carve.")
|
||||
seen_subsegs[sub] = s_off
|
||||
carves = merged
|
||||
|
||||
@@ -264,7 +339,7 @@ def build_carve(ov, funcs):
|
||||
nonlocal n_data
|
||||
n_data += 1
|
||||
return "tail" if n_data == 1 else f"tail{n_data}"
|
||||
for s_off, e_off, sub in carves:
|
||||
for s_off, e_off, sub, _spec in carves:
|
||||
if cursor < s_off:
|
||||
nm = data_name()
|
||||
pieces.append((cursor, "data", nm))
|
||||
@@ -283,16 +358,20 @@ def build_carve(ov, funcs):
|
||||
for off, kind, name in pieces:
|
||||
comment = " # Phase-26 §8 jtbl-rodata carve (jtbl_carve.py)" if kind == ".rodata" else ""
|
||||
region_lines.append(f"{indent}- [{hex(off)}, {kind}, {name}]{comment}")
|
||||
return region_lines, "--order " + ",".join(order)
|
||||
pads_map = {sub: spec for (_s, _e, sub, spec) in carves}
|
||||
return region_lines, "--order " + ",".join(order), pads_map
|
||||
|
||||
|
||||
def apply(ov, funcs):
|
||||
region_lines, order_arg = build_carve(ov, funcs)
|
||||
region_lines, order_arg, pads_map = build_carve(ov, funcs)
|
||||
lines, indent, lo, hi, *_ = parse_config(ov)
|
||||
new_lines = lines[:lo] + region_lines + lines[hi:]
|
||||
open(cfg_path(ov), "w").write("\n".join(new_lines) + "\n")
|
||||
set_overlays_var(ov, order_arg)
|
||||
print(f"jtbl_carve {ov}: carve set = {len(region_lines)} pieces; JTBL_INTERLEAVE = {order_arg}")
|
||||
set_pads_vars(ov, pads_map)
|
||||
multi = {s: p for s, p in pads_map.items() if len(p) > 1}
|
||||
print(f"jtbl_carve {ov}: carve set = {len(region_lines)} pieces; JTBL_INTERLEAVE = {order_arg}"
|
||||
+ (f"; JTBL_PADS = {multi}" if multi else ""))
|
||||
|
||||
|
||||
def set_overlays_var(ov, args):
|
||||
@@ -310,6 +389,38 @@ def set_overlays_var(ov, args):
|
||||
open(mk, "w").write(txt)
|
||||
|
||||
|
||||
def set_pads_vars(ov, pads_map):
|
||||
"""Write this overlay's per-object JTBL_PADS lines (Phase-29 §8e), preserving carried values.
|
||||
|
||||
Only multi-table spans (len(spec) > 1) get a line; single-table spans get none (their pipeline
|
||||
stays byte-identical to pre-§8e). All of this overlay's current pads lines are replaced by the
|
||||
regenerated block as one unit (values were CARRIED into pads_map by current_pads_specs, so this
|
||||
is a rewrite of the same state plus the new boundary — not a re-derivation). Any object whose
|
||||
spec appears, changes, or disappears gets its stale build/src/<ov>/<sub>.o deleted: the spec is
|
||||
no make-prerequisite, and a padless stale object would fail the SHA gate mystifyingly."""
|
||||
mk = os.path.join(REPO, "config/overlays.mk")
|
||||
txt = open(mk).read()
|
||||
before = current_pads_specs(ov, txt)
|
||||
after = {sub: spec for sub, spec in pads_map.items() if len(spec) > 1}
|
||||
# drop all current lines for this overlay, then insert the regenerated block
|
||||
txt = re.sub(pads_line_re(ov) + r"\n", "", txt, flags=re.M)
|
||||
if after:
|
||||
block = "\n".join(
|
||||
f"build/src/{ov}/{sub}.o: JTBL_PADS := {','.join(map(str, spec))}{PADS_COMMENT}"
|
||||
for sub, spec in sorted(after.items()))
|
||||
m = re.search(rf"^{re.escape(ov)}_JTBL_INTERLEAVE.*$", txt, re.M)
|
||||
if not m:
|
||||
sys.exit(f"jtbl_carve: no {ov}_JTBL_INTERLEAVE line to anchor JTBL_PADS on")
|
||||
txt = txt[:m.end()] + "\n" + block + txt[m.end():]
|
||||
open(mk, "w").write(txt)
|
||||
for sub in set(before) | set(after):
|
||||
if before.get(sub) != after.get(sub):
|
||||
obj = os.path.join(REPO, f"build/src/{ov}/{sub}.o")
|
||||
if os.path.exists(obj):
|
||||
os.remove(obj)
|
||||
print(f"jtbl_carve: JTBL_PADS changed for {sub} — removed stale {obj}")
|
||||
|
||||
|
||||
def revert(ov):
|
||||
"""Restore this overlay's carve state to the COMMITTED one.
|
||||
|
||||
@@ -335,9 +446,28 @@ def revert(ov):
|
||||
txt = txt.replace(anchor, anchor + "\n" + m.group(0), 1)
|
||||
else: # no committed carve -> drop ours
|
||||
txt = re.sub(rf"^{re.escape(ov)}_JTBL_INTERLEAVE.*\n", "", txt, flags=re.M)
|
||||
# JTBL_PADS lines (Phase-29 §8e): restore this overlay's per-object pad specs to the committed
|
||||
# set with the same surgical splice (a failed sibling bank must not leave its spec behind, and
|
||||
# a blunt checkout would wipe OTHER siblings' in-flight lines — overlays.mk is shared).
|
||||
now_pads = current_pads_specs(ov, txt)
|
||||
committed_pads = current_pads_specs(ov, committed)
|
||||
txt = re.sub(pads_line_re(ov) + r"\n", "", txt, flags=re.M)
|
||||
committed_lines = [l for l in committed.splitlines()
|
||||
if re.match(pads_line_re(ov), l)]
|
||||
if committed_lines:
|
||||
m2 = re.search(rf"^{re.escape(ov)}_JTBL_INTERLEAVE.*$", txt, re.M)
|
||||
if not m2:
|
||||
sys.exit(f"jtbl_carve: no {ov}_JTBL_INTERLEAVE line to anchor committed JTBL_PADS on")
|
||||
txt = txt[:m2.end()] + "\n" + "\n".join(committed_lines) + txt[m2.end():]
|
||||
open(mk, "w").write(txt)
|
||||
for sub in set(now_pads) | set(committed_pads):
|
||||
if now_pads.get(sub) != committed_pads.get(sub):
|
||||
obj = os.path.join(REPO, f"build/src/{ov}/{sub}.o")
|
||||
if os.path.exists(obj):
|
||||
os.remove(obj)
|
||||
print(f"jtbl_carve {ov}: reverted config + JTBL_INTERLEAVE restored to committed"
|
||||
f"{'' if m else ' (none)'}")
|
||||
f"{'' if m else ' (none)'}"
|
||||
+ (f" + {len(committed_lines)} JTBL_PADS line(s) restored" if committed_lines else ""))
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
@@ -38,10 +38,17 @@ def sh(cmd):
|
||||
|
||||
|
||||
def stub_file(ov, func):
|
||||
for cf in sorted(glob.glob(f"src/{ov}/{ov}*.c")):
|
||||
if re.search(rf'INCLUDE_ASM\("[^"]*",\s*{func}\);', open(cf).read()):
|
||||
return cf
|
||||
return None
|
||||
"""The ONE .c holding this func's INCLUDE_ASM stub. Fail-loud on duplicates (Phase-29 §8e):
|
||||
the first-sorted-glob behavior once returned a STALE duplicate stub in a different TU
|
||||
(func_80131340 spliced into ov_SC01_077_a.c instead of ..._jr_8012ACE0.c), producing a
|
||||
misattributed `conflicting types` cascade — a wrong-TU splice, not a draft defect."""
|
||||
hits = [cf for cf in sorted(glob.glob(f"src/{ov}/{ov}*.c"))
|
||||
if re.search(rf'INCLUDE_ASM\("[^"]*",\s*{func}\);', open(cf).read())]
|
||||
if len(hits) > 1:
|
||||
sys.exit(f"jtbl_family_bank: {func} has {len(hits)} INCLUDE_ASM stubs in {ov}: {hits} — "
|
||||
f"duplicate/stale stub; fix the source before banking (a first-match splice "
|
||||
f"would hit the wrong TU's decl environment)")
|
||||
return hits[0] if hits else None
|
||||
|
||||
|
||||
def region_files(ov):
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Phase-29 §8e: reproduce the ORIGINAL's jump-table padding in a multi-table .rodata carve.
|
||||
|
||||
The mechanism (byte-proven, .run/probe_jtbl/verdict.md):
|
||||
- cc1 (Sony gcc 2.7.2) emits `.rdata` + `.align 3` + label before EVERY switch jump table.
|
||||
- maspsx passes `.align` through VERBATIM (the maspsx/__init__.py:435 `continue` is in an
|
||||
inventory-only pass; the output path re-emits `.align` unchanged).
|
||||
- GNU as bakes the align pad into the section SECTION-RELATIVE — the linker can never remove it
|
||||
(SUBALIGN(2) caps only the section's *external* placement alignment).
|
||||
- The ORIGINAL image packs originally-separate TUs' tables TIGHT (the PSX linker 4-aligned
|
||||
section placement), while intra-TU consecutive tables carry REAL zero-word pads where the
|
||||
previous table ends ≡4 mod 8.
|
||||
|
||||
So when a decomp TU holds MULTIPLE matched jr-functions (originally separate TUs merged), cc1's
|
||||
`.align 3` fires at boundaries where the original packed tight -> a +4 interior pad -> every
|
||||
downstream data symbol shifts -> %lo relocs break image-wide. And conversely a table's alignment
|
||||
pad must appear exactly where the original HAS one. `.align` cannot express this (it is relative
|
||||
to the section start, whose vram parity varies per overlay sibling) — only explicit pad bytes can.
|
||||
|
||||
This filter sits between maspsx and as (Makefile `build/src/%.o` recipe, armed per-object by a
|
||||
`JTBL_PADS` target-specific make var written by tools/jtbl_carve.py). It REPLACES each rodata
|
||||
`.align` with the exact pad the original has at that table boundary:
|
||||
|
||||
--pads 0,4,0 -> table 1: no pad (specs[0] is always 0 — the table starts the section),
|
||||
table 2: one `.word 0` pad, table 3: no pad.
|
||||
|
||||
Pads are derived by jtbl_carve from the carve intervals (pad[K] = start[K] - end[K-1], values in
|
||||
{0,4}), so each overlay sibling gets its own spec from its own addresses. Unfiltered objects
|
||||
(no JTBL_PADS var — every single-table carve) keep today's byte-identical pipeline.
|
||||
|
||||
Fail-loud guards (R32) — each converts a silent downstream SHA1 mismatch into an immediate,
|
||||
attributable build error:
|
||||
- spec count != rodata `.align` count (table-count drift: a sibling's switch compiled
|
||||
to a different number of tables)
|
||||
- a rodata `.align` other than `.align 3` (unknown emission shape)
|
||||
- rodata content other than $L labels / `.word $L...` / blank (strings, floats — the carve
|
||||
model does not cover them)
|
||||
- pads[0] != 0, or any pad not in {0, 4}
|
||||
"""
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
|
||||
LABEL_RE = re.compile(r"^\$L\d+:$")
|
||||
WORD_RE = re.compile(r"^\.word\s+\$L\d+$")
|
||||
|
||||
|
||||
def run(pads, lines, out):
|
||||
in_rodata = False
|
||||
consumed = 0
|
||||
for line in lines:
|
||||
s = line.strip()
|
||||
if s == ".section .rodata":
|
||||
in_rodata = True
|
||||
out.write(line)
|
||||
continue
|
||||
if in_rodata and (s == ".text" or s.startswith(".section")):
|
||||
in_rodata = False
|
||||
out.write(line)
|
||||
continue
|
||||
if in_rodata:
|
||||
if s.startswith(".align"):
|
||||
parts = s.split()
|
||||
if len(parts) != 2 or parts[1] != "3":
|
||||
sys.exit(f"jtbl_rodata_pads: unexpected rodata align {s!r} "
|
||||
f"(cc1 emits `.align 3` per jump table — see verdict.md)")
|
||||
if consumed >= len(pads):
|
||||
sys.exit(f"jtbl_rodata_pads: more rodata .align directives than pad specs "
|
||||
f"({len(pads)}) — table-count drift vs the carve")
|
||||
if pads[consumed] == 4:
|
||||
out.write(".word 0 # jtbl_rodata_pads: original inter-table .align 3 pad\n")
|
||||
consumed += 1
|
||||
continue
|
||||
if s == "" or LABEL_RE.match(s) or WORD_RE.match(s):
|
||||
out.write(line)
|
||||
continue
|
||||
sys.exit(f"jtbl_rodata_pads: unexpected rodata content {s!r} — "
|
||||
f"the carve model covers only jump tables ($L labels + .word entries)")
|
||||
out.write(line)
|
||||
if consumed != len(pads):
|
||||
sys.exit(f"jtbl_rodata_pads: consumed {consumed} rodata .align(s) but {len(pads)} pad "
|
||||
f"spec(s) given — table-count drift vs the carve")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--pads", required=True,
|
||||
help="comma list of pad BYTES before each rodata jump table, in emission "
|
||||
"order; pads[0] must be 0")
|
||||
a = ap.parse_args()
|
||||
try:
|
||||
pads = [int(x) for x in a.pads.split(",")]
|
||||
except ValueError:
|
||||
sys.exit(f"jtbl_rodata_pads: malformed --pads {a.pads!r}")
|
||||
if not pads or pads[0] != 0:
|
||||
sys.exit(f"jtbl_rodata_pads: pads[0] must be 0 (the first table starts the section): {pads}")
|
||||
if any(p not in (0, 4) for p in pads):
|
||||
sys.exit(f"jtbl_rodata_pads: every pad must be 0 or 4: {pads}")
|
||||
run(pads, sys.stdin, sys.stdout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user