- reports (Task 3): tools/{progress,difficulty,dup_report}.py + make report/sig-refresh
(deterministic digests); harvest to 43 REAL matches incl. LzssDecodeSector
- per-module -O0 split (Task 5.5): src/boot.c + Makefile per-file CC1FLAGS (cookbook §6)
- PsyQ SDK linking (sessions C-G): split/build/identify/link/integrate pipeline
(tools/psyq_*.py, make_libgs.sh, ld_interleave.py, split_src_region.py) — libcd (18 obj)
+ FULL libgs (31 obj / 6 blocks) linked BYTE-IDENTICAL from real PsyQ 4.0 libs; byte-
identical with or without the SDK objects (stub fallback); cookbook §7/§8/§9.1-9.5
- LZSS MATCHED byte-for-byte (session F): §5a cross-jump barrier + §10 regalloc/schedule
idioms, ground-truthed against pinned gcc-2.7.2 source (the Gen1 LZSS gate)
- file-loader cluster: 6 matched + 5 NON_MATCHING-drafted (CdReadStateMachine,
CdReadSectorReadyCB, StreamLoadStateMachine new this session, valid under -DNON_MATCHING);
SaveLoadRoutine DEFERRED to Q#5 (save/memcard, documented stub)
- README Gen1-complete; worklog preserved -> phase-ends/logs/Phase7.md
- rules R17 (web-research compiler quirks), R18 (plain-English session summaries),
R19 (preserve phase worklog out of load order)
- MILESTONE: Gen1 EXIT confirmed — byte-identical build, 43 matches incl LZSS, 7 zero-
regression sessions, loader cluster matched/drafted, overlay map emulator-proven
- bumps project version 1.6.0 -> 1.7.0
28 KiB
CURRENT PHASE — Phase 7: Matching loop at scale + Gen1 close-out
Status: Plan APPROVED 2026-06-14. REORDERED 2026-06-14 (Drew): bank non-switch wins first; the
rodata-island foundation + LZSS match are DEFERRED to a focused sub-project after the easier tasks.
Full plan: ~/.claude/plans/confirmed-plan-mode-enabled-jolly-lemon.md. Project version: 1.6.0 → 1.7.0 at PhaseEnd.
In-phase crash/compaction recovery log (P3 step 4). Update after every task.
Locked decisions (developer-confirmed)
- ≥25 = real substantive matches (the 42 splat-auto empties do NOT count). 14 real now → need ≥11 more.
- Loader cluster = match-tractable / draft-hard (NON_MATCHING-draft the hard state machines).
- REORDER (2026-06-14): rodata foundation hit a structural wall (see below); do reports + harvest + non-switch loader FIRST, then a focused LZSS/rodata sub-project. LZSS is still required for Gen1 exit.
- ⚠️ PHASEEND AFTER LIBGS (Drew 2026-06-14): do NOT write PhaseEnd_Phase7 / delete this CURRENT_PHASE.md until libgs is done. This file holds the libgs placement + working notes (see the session-E "libgs placement" block) and must stay intact so libgs work can resume. Sequence: LZSS → libgs → Task 6 close-out → (only then) Task 7 PhaseEnd. Task graph enforces it (#8 blocked by #9).
Verified baseline (grounded; R14 corrections)
- Build byte-identical (
143dbb89f34491258bbc27810d0a12ec8b43a8dd), reproducible. Only change from committed Phase-6 = one R15 symbol line (func_80047CAC = 0x80047CAC; // data) — fixes a LATENT NON-REPRODUCIBILITY: spimdisasm 1.41.0 auto-detection of that 8-byte inter-fn blob is unstable across clean extracts; declaring it makesmake clean && make extract && make builddeterministic. (Note for PhaseEnd.) - Switch cohort ~46 funcs / 53 tables (51 in-image, single-ref/migratable). LzssDecodeSector 0x80018730 = gcc switch via
jtbl_80072A38(write-free rodata island 0x80072A38–0x80074750 / file 0x63238–0x64F50). - All 13 loader-cluster funcs in text region (< 0x800629DC). Authoritative counts: 14 real / 42 empty / 2 NON_MATCHING / 1998 stubs / 2050 matchable.
Task checklist (REORDERED — execution order top to bottom)
- Task 3 — Report machinery ✓ DONE.
tools/{progress,difficulty,dup_report}.py+make report(deterministic) +make sig-refresh. Authoritative baseline 14 real / 42 empty / 2 NM / 1992 stubs / 2050 matchable (REAL/matchable=0.68%); empties audit 42/42 clean; dup leverage 8 h_exact + 44 h_norm redundant; harvest queue 1119 non-jtbl leaves + 313 trivial. Digests →docs/{progress,difficulty,duplicates}.md. - Task 4 — Harvest to ≥25 real ✓ DONE. 36 real matches (14 + 22 trivial accessor leaves: getters/setters of globals, one mask, one two-store), build BYTE-IDENTICAL → all 22 byte-perfect. ≥25 bar PASS (margin 36/25).
.run/harvest.py= the applier. Dedup-collapse skipped (the sig dup groups are mostly PsyQ library fragments/epilogues, not real funcs). LZSS still required (Task 2′). - Task 5.5 — Per-file -O0 split mechanism ✓ DONE (session B, Drew-approved). MAJOR FINDING: per-module opt mixing is real (§5.5). The boot/main/game-mode-dispatch module @ vram 0x80010000–0x800123F0 (~50 funcs incl. start/main/GameModeDispatch/DebugMenuHandler) is -O0, not the pinned -O2; everything ≥0x800123F0 is -O2. Mechanism: split text c-subseg →
[0x800,c,boot]→src/boot.c(-O0 via Makefile target-specific CC1FLAGS) +[0x2BF0,c,800]→src/800.c(-O2, matched fns untouched). Regression gate PASS (byte-identical at 100% INCLUDE_ASM). Cookbook §6 added (detect via21F0A003frame-ptr sig;register-pointer far-member idiom). Reusable for every future flag-divergent module. - Task 5.6 — PsyQ CD struct header ✓ DONE (session B).
include/psyq/libcd.h(CdlLOC 4B + CdlFILE 24B, offsets from Ghidra/LIBCD.H, G1) + 3 libcd protos; added 4 PsyQ symbols toconfig/symbols.us.txt(CdSearchFile/CdPosToInt/CdIntToPos/VSync — Ghidra-confirmed, R15) and renamed theirfunc_*stubs in src/800.c. Build BYTE-IDENTICAL (label-only); header compiles, sizeof asserts hold. Unlocks the file-loader cluster (LoaderInitFileTable, ResourceLoadStateMachine, the jtbl loaders). Cookbook §7 added (PsyQ types/symbols pattern). NB symbols.us.txt now has 983 lines (4 appended, un-sorted — re-merge note for R15). - [~] Task 5 — Loader cluster PARTIAL (42 real total; non-jtbl 5/7 matched). Matched (score 0, byte-identical): ResourceGetCdLoc (15), LoaderResetReadState (32), GameModeDispatch (29, -O0
register-ptr far member), DebugMenuHandler (30, -O0, reserved-slot local), CdQueueBusy (35, -O2 — if/else order + branch polarity), CdReadRequest (53, -O2 — early-return fall-through). Triage of the rest:- jtbl → Task 2′ (rodata-gated): CdReadStateMachine (385), CdReadSectorReadyCB (424, sole LZSS caller), SaveLoadRoutine (1139), StreamLoadStateMachine (459).
- non-jtbl COMPLETE (6 matched + 2 drafted): LoaderInitFileTable (133) + ResourceLoadStateMachine (211) → NON_MATCHING-drafted (logically faithful, libcd.h-typed; per-function residual notes in src/800.c — LoaderInit: regalloc/loop-invariant hoisting + name=base-0x14; ResourceLoad: block placement (state 0→1→2→3 layout vs nested-if) + reserved-local frame 0x38 vs 0x28 + func_8002D4C8 arg & 0xFFFF). NON_MATCHING count now 4. Byte-match both in a later structural/permuter pass.
- Report tooling fixed for the multi-file split:
tools/{progress,difficulty}.pynow glob allsrc/*.c+ search allasm/nonmatchings/*subdirs, andprogress.pyskipsextern …(…);forward-declarations (they were swallowing the next fn + double-counting). Deterministic; 42 real / 1964 stubs consistent across both. (Note for PhaseEnd.) - Task 2′ — Focused LZSS + rodata-island sub-project ✓ DONE (session F, 2026-06-15). LzssDecodeSector MATCHED byte-for-byte (asm-differ score 0;
make clean && make extract && make build && make check→143dbb89… BYTE-IDENTICAL). The Gen1-exit LZSS gate is satisfied. 43 real matches (42 + LZSS). The NON_MATCHING guard is DROPPED — the C is the default build. Close required 5 load-bearing constructs (all documented in src/800.c + cookbook §10): (A) decouplednhhigh-byte var +(code&0xFF)|(nh<<8)operand order [local-alloc combine_regs]; (B)resultset in save predecessors [reorg fill_simple_delay_slots]; (B/state-2) explicitregister $2local + early read-only input-asm pin [sched list-order]; (B3) NO switchdefault/trailing return → state≥5 reuses thesltiuresult; (§5a) the zero-byte cross-jump barrier. The surgical rodata carve + ld_interleave place gcc's generated jtbl at 0x80072A38 (links byte-identical). Cookbook §10 added (the reusable regalloc/schedule-tail idioms + the floor-free.textmetric); §5a cross-linked. - Task 6 — Gen1-exit close-out (README, checklist, ≥3-session zero-regression evidence). xHigh.
- Task 7 — PhaseEnd_Phase7 (Gen1 synthesis, milestone gate). Max · Tier 1.
Current task
LZSS DONE (session F, 2026-06-15) — the Gen1-exit LZSS gate is SATISFIED. LzssDecodeSector matches
byte-for-byte (asm-differ score 0; full clean build 143dbb89… BYTE-IDENTICAL); the #ifdef NON_MATCHING
guard is dropped — the C is the default build. 43 real matches. Full close detail in the Task 2′ checklist
entry above + the 5 LOAD-BEARING comments in src/800.c + cookbook §10.
- ✅ libcd wired into the byte-identical build (session D) — 58 SDK funcs.
- ✅ LZSS matched (session F) — surgical rodata carve + ld_interleave place gcc's generated switch jtbl at
0x80072A38 (links byte-identical); the §5a cross-jump barrier + 4 regalloc/schedule idioms (§10) closed it.
Method: §3a web-research ground-truthed all levers against the pinned gcc-2.7.2 source; a floor-free
.textmetric (objcopy--only-section=.text+cmp, bypasses the rodata/jtbl floor) measured the close; the permuter could NOT (score floor + random divergence). - ✅ libgs BLOCK 6 integrated byte-identical (session F, Drew: "block 6 alone first, then the integration").
16 contiguous libgs objects → 26 SDK functions banked (vram 0x80053AF8–0x80057928),
make clean/extract/ build/check→143dbb89… BYTE-IDENTICAL, reproducible; fresh-clone stub fallback verified; zero spurious warnings. Mechanism (the libcd §9.3 pattern): resegment 800b → [800b pre][libgs6][800b2 post]; regenerate the 3 split src files;psyq_integrate libgs6. Three new fixes (all in this commit):- splat instability the resegment exposed: shrinking 800b shifted spimdisasm's auto-detected boundaries.
Declared
GsMulCoord2/3(real MATRIX.o funcs, verified via the PsyQ object — R15, Ghidra-mirror pending); and carved the §8 data descriptor table[0x53198,data,53198](boot.c'sD_80062998was being shadowed by a mis-detectedfunc_80062998). ld_interleave FRONT_DATA →53198.data.o. - psyq_integrate multi-library support: NOLOAD section names namespaced per-library (
.nl_<tag>_*) so the 2nd integration's NOLOAD placement isn't skipped (was discarding OBJT3 .rdata/.bss); trial-link now includes already-emitted sibling*_externals.ld(kills the "31 UNRESOLVED" false alarm — they're libcd's externals); NOLOAD lines globally re-sorted by vram across libraries (kills "dot moved backwards"). - reproducibility:
tools/make_libgs_block6.shregenerates the curated 16-object dir (disambiguated: GS_131/137 not RVWUNIT/RVWLUNIT, PRESET3/OBJT3 not PRESET2/OBJT2).
- splat instability the resegment exposed: shrinking 800b shifted spimdisasm's auto-detected boundaries.
Declared
- ✅ DONE (session G) — full libgs integration. All 31 libgs objects in 6 contiguous blocks now linked
from real PsyQ SDK objects, byte-identical. Blocks 1-5 (15 objects / 29 named funcs) added to block 6's 16
(20 named funcs) → ≈49 named SDK functions across 31 objects (count OBJECTS — splat over-segments library
code into 217 INCLUDE_ASM stubs; session-F's per-block "26" was a looser estimate, superseded). GS_001 stays
excluded (gsgap3 stub, §9.1). Unified into ONE
psyq_integratecall (supersedes the block-6-only path). Changes:tools/make_libgs.sh(31-obj regen dir, replaces make_libgs_block6.sh);psyq_integrate.pygained an optional placement-window arg (GS_106 = 8 ins, only anchors uniquely in 0x80051804-0x80057928);config/splat.us.exe.yaml800b resegmented → 800b(pre) + libgs1-5 + gsgap1-5 (+ unchanged libgs6/800b2);MakefileLIBGS6_* → LIBGS_* (window + 6 stubs);src/800b.ctrimmed to <0x80051804; 10 new stub src files (libgs1-5.c, gsgap1-5.c). Split DETERMINISTIC across 2 clean extracts; byte-identical WITH or WITHOUT the SDK objects (fresh-clone stub fallback). Cookbook §9.5. Remaining: Task 6 close-out, Task 7 PhaseEnd (PhaseEnd gated on libgs per Drew — now UNBLOCKED). NOTE: Gen1 exit needs ≥3 SESSIONS of greenmake check— satisfied (A, B, C, D, E, F, +G); Tasks 6/7 pending.
Island ownership map (session E — the decisive finding)
Rodata island 0x80072A38–0x80074750, 102 jtbls total (~52 in-island). By owner segment:
- 0x80072A38–0x800734F4 (~35 jtbls): 800-segment GAME code (targets 0x80018xxx–0x80039xxx; incl.
jtbl_80072A38=LZSS first entry, and S_SCA/SR_SV which resolve toasm/nonmatchings/800/). MIGRATE these. - 0x800737CC–0x800746B0 (~17 jtbls): 800b + LIBRARY — BIOS(libcd, already integrated), GS_123/PRESET/ PRESET2/OBJT/OBJT2(libgs), PRNT(libc2), LIBMCRD. STAY RAW (untouched flat data → byte-identical).
- +24 root cause = 6 library jtbls, ALL in 800b (PRESET_OBJ_744/8FC, PRESET2_OBJ_4D8/A88, OBJT2_OBJ_614, PRNT_OBJ_24C). LZSS+game jtbls are in 800; the only libgs object in 800 (GS_013, 0x8003D40C) has no jtbl → 800-only migration has no library TU → no +24.
libgs placement (session E — preserved for the BONUS task #9)
36/201 located; byte-test disambiguation: PRESET3 (not PRESET2) @0x80055D40, OBJT3 (not OBJT2) @0x80057094
(losers have a spurious .data + .text mismatch); GS_131≡RVWUNIT, GS_137≡RVWLUNIT are .text-identical aliases (keep GS_*).
GS_106 @0x80053308 fills a gap (narrow-window). GS_013 @0x8003D40C = far outlier in 800 (6 ins). 4 ambiguous
(GS_101/102/124/125) don't fit gaps → not linked. Main block 0x80051804–0x80057928 (800b), ~4 sub-blocks; gaps 96/48/304 B.
Patched tools/psyq_identify.py to skip data-only objects (no .text, e.g. GLOBAL.o). PRESET3/OBJT3 .rdata land in-island (0x80073c98/0x80073ee8).
- libgs BYTE-VERIFIED (session E): curated
.run/obj40/libgs_used/(31 objects).psyq_link_region … --emit .run/libgs_region→ 31/32 byte-identical, 0 conflicts, 69 externals;.run/libgs_region.{ld,syms}emitted. Confirms BFM links real PsyQ libgs 4.0 objects byte-for-byte. GS_001 EXCLUDED (known-issue: psyq-obj-parser packs scattered PSD* commons into.bssreferenced via.bss+offset; 35 words differ at the global-zeroing run — needs per-symbol .bss resolution, cookbook §9.1 hard case). Block structure: 6 contiguous blocks + 5 gaps (80/48/1536[GS_001]/48/304 B, all non-libgs → stay stubs). REMAINING (mechanical): resegment splat 800b into the 6 blocks + 5 gap stubs (~13 subsegs) + split_src + wirepsyq_integrate(6 block stubs) → byte-identical → banks ~63 SDK funcs. Simpler first win = block 6 alone (16 objs incl. PRESET3/OBJT3) as [pre][libgs6][post] → ~35 funcs. Deferred at session-E end (long session); clean continuation.
Per-session make check green log (≥3 sessions needed for the milestone)
- 2026-06-14 (session A):
make check→143dbb89… BYTE-IDENTICAL✓ — baseline restored + reproducibility fix, reports built, 38 real matches (22 accessor leaves + ResourceGetCdLoc + LoaderResetReadState), build byte-identical throughout. [need ≥2 more sessions] - 2026-06-14 (session B):
make check→143dbb89… BYTE-IDENTICAL✓ — per-file -O0 split mechanism (src/boot.c + Makefile per-file flags); 4 real matches (GameModeDispatch, DebugMenuHandler, CdQueueBusy, CdReadRequest) → 42 real; PsyQ libcd.h infra (CdlLOC/CdlFILE + 4 named symbols, unlocks the loader cluster); LoaderInitFileTable + ResourceLoadStateMachine NON_MATCHING-drafted (→ 4 NM) — Task 5 non-jtbl loaders COMPLETE (6 matched + 2 drafted); report tooling fixed (multi-file); cookbook §6/§7/T4. Build byte-identical throughout. [need ≥1 more session] - 2026-06-15 (session C):
make check→143dbb89… BYTE-IDENTICAL✓ (fullclean && extract && build, restored after the Task-2′ experiments). ≥3-session bar MET. This session: fully diagnosed + built the rodata-island mechanism (works); root-caused the +24; proved the PsyQ-library-linking GO (see below). No new matches (architectural session). Build green at start and after restore. - 2026-06-14 (session D):
make clean && make extract && make build && make check→143dbb89… BYTE-IDENTICAL✓ — libcd LINKED INTO THE BUILD (Drew-approved push-through). The first real PsyQ library is now sourced from real SDK objects in the byte-identical build: 58 libcd SDK functions linked (not stubs), replacing the libcd-region asm stubs. Idempotent; Makefile-automated; conditional (fresh clone w/otools/psyq/builds via stubs). New committed tooling:tools/psyq_link.py(per-object byte-link engine — recovers externals from resolved relocs, weakens psyq-obj-parser's mislabelled.bsscommons),psyq_link_lib.py(whole-lib verify, 18/18 libcd),psyq_link_region.py(region link via NOLOAD = no data carving),psyq_integrate.py(build wiring: splat resegment + .ld swap + external resolution),split_src_region.py(H5-safe src split). Cookbook §9.1/9.2/9.3 + R16. No data carving (NOLOAD data placement; flat data subseg unchanged). Build green throughout. - 2026-06-15 (session E):
make clean && make extract && make build && make check→143dbb89… BYTE-IDENTICAL✓ — start-of-session baseline + after the LZSS surgical rodata carve (jtbl_80072A38 migrated + sandwiched) + LZSS C NON_MATCHING-guarded (default build = stub). ≥3-session bar already MET (A/B/C/D); E is margin. Findings: lean LZSS path proven (no libgs needed for it); LZSS C structurally matches (111/122) but blocked on a gcc cross-jump-merge hard-tail (see LZSS block above); libgs placement+disambiguation done (bonus, task #9, deferred per Drew until before PhaseEnd). - 2026-06-15 (session F):
make clean && make extract && make build && make check→143dbb89… BYTE-IDENTICAL✓ — TWO milestones. (1) LZSS MATCHED byte-for-byte (asm-differ score 0; C compiled in, NON_MATCHING guard dropped). Closed the session-E ~4 regalloc/scheduling residuals via §3a web-research (ground-truthed against pinned gcc-2.7.2reorg.c/jump.c/local-alloc.c) + a floor-free.textobject metric: decoupled high-byte var + OR operand order (residual A);result-in-predecessor + explicit-$2-local-with-early-pin (residual B); no-defaultsltiu-reuse (residual B3); §5a barrier retained. 43 real matches; Gen1-exit LZSS gate SATISFIED. Cookbook §10. (2) libgs BLOCK 6 integrated (16 objects / 26 SDK funcs byte-identical; the libcd §9.3 pattern). Fixed a resegment-triggered spimdisasm boundary instability (declared GsMulCoord2/3; carved the §8 data table[0x53198,data]) + made psyq_integrate multi-library-safe (per-lib NOLOAD namespacing + global sort; sibling externals in the trial);tools/make_libgs_block6.shfor reproducibility; cookbook §9.4. Build byte-identical with OR without the SDK objects; fresh-clone stub fallback verified. Changed:src/800.c,config/{splat.us.exe.yaml,symbols.us.txt},Makefile,tools/{psyq_integrate.py,ld_interleave.py,make_libgs_block6.sh},docs/matching-cookbook.md(zero generated/ROM bulk staged). Tasks: full libgs → 6 → 7 remain (PhaseEnd gated on libgs per Drew). - 2026-06-15 (session G):
make clean && make extract && make build && make check→143dbb89… BYTE-IDENTICAL✓ (start-of-session baseline + multiple full clean cycles after the change). FULL libgs integration DONE — the remaining 5 blocks unified with block 6 into ONEpsyq_integratecall: all 31 libgs objects / 6 contiguous blocks linked from real SDK objects byte-identical (≈49 named SDK functions).psyq_integrate.pygained an optional placement-window arg (GS_106 = 8 ins anchors uniquely only in the libgs window);tools/make_libgs.shreplacesmake_libgs_block6.sh(31-obj dir); config 800b resegmented into 800b(pre)+libgs1-5+gsgap1-5 (+ unchanged libgs6/800b2);src/800b.ctrimmed (<0x80051804, 0 matched-C lost); 10 new stub src files. Split DETERMINISTIC across 2 clean extracts (no §9.4 boundary instability); byte-identical WITH or WITHOUT the SDK objects (fresh-clone stub fallback verified). Cookbook §9.5. Changed:config/splat.us.exe.yaml,Makefile,src/800b.c+ 10 newsrc/{libgs1-5,gsgap1-5}.c,tools/{psyq_integrate.py,make_libgs.sh}(−make_libgs_block6.sh),docs/matching-cookbook.md,docs/{progress,difficulty}.md(digest refresh). Tasks: 6 (close-out) → 7 (PhaseEnd) remain; libgs gate CLEARED. (session G cont.) Then the 4 jtbl loaders: CdReadStateMachine / CdReadSectorReadyCB / StreamLoadStateMachine NON_MATCHING-drafted (faithful translations of the Ghidra decompile; all valid C under-DNON_MATCHING, verified via a cpp→cc1 syntax check; shared-loader-global types reconciled to the file's s32/void* convention). SaveLoadRoutine DEFERRED to Q#5 (Drew) — it is a 1139-ins multi-entry save/memcard blob (singlejr $ra, 3 saveHeaderTemplate entry points, Ghidra returns only an unaff_-register fragment); left a full structured deferral comment in src/800.c. Task 6 close-out: README → Gen1-complete status; Gen1-exit checklist verified (43 REAL incl LZSS, 7 zero-regression sessions, reports auto-generate, loader cluster matched/drafted, overlay map proven). Gen1 EXIT confirmed by Drew. 3 rules formalized at PhaseEnd: R17 (web-research compiler quirks), R18 (plain-English session recaps), R19 (preserve worklog → phase-ends/logs/, out of load order). This worklog is itself preserved under R19. PhaseEnd_Phase7 written; Gen1 COMPLETE.
Rodata-island foundation — RESOLVED end-to-end (session C, 2026-06-15)
The mechanism now WORKS; only the +24 (a known file-split/alignment artifact) blocks full byte-identity, and
the PsyQ-lib pivot (below) is the chosen fix. Experimental configs saved: .run/{splat.island.yaml, symbols.island.txt,Makefile.island,800.c.fresh-throwaway}; new committed tool tools/ld_interleave.py.
Proven mechanism (3 parts, all validated this session):
- Migration — dotted
.rodatasibling named800([0x63238, .rodata, 800]) → spimdisasm migrates all 50 jtbls + single-ref consts into their owningasm/nonmatchings/800/<fn>.s(99 .s got.section .rodata,.L-refs resolve intra-object). Multi-ref consts → 70INCLUDE_RODATAlines (needs a FRESH800.c; for the PERMANENT file, surgically INSERT those 70 lines into the curated 800.c — do NOT regen-fresh, it drops comments/H5). All 301 jtbl targets ∈ the800text seg (none inboot), so the single sibling is correct. - Placement —
tools/ld_interleave.py(wired intomake extract) rewrites splat's section-major.maininto the real.data→.rodata→.datasandwich order (text, front-data@0x800629DC, rodata@0x80072A38, tail-data@0x80074750). Front/tail split by object basename. rodata + both data sizes came out byte-exact. - Data-in-text carve — the 68-B descriptor table at 0x80062998–0x800629DC (ptrs to start/D_80062998/
D_80074778) must be its own
[0x53198, data, 53198]subseg or the jumptable analyzer mis-extends the last code function across it. The ONLY residual = +24 (ROOT-CAUSED): 6×.align 3jumptable padding nops injected into.text(at PRESET_OBJ_744, PRESET_OBJ_8FC, PRESET2_OBJ_4D8, PRESET2_OBJ_A88, OBJT2_OBJ_614, PRNT_OBJ_24C). GCC 8-aligns each switch jtbl, but the original built these as SEPARATE translation units; our single 800.o concatenation adds padding the original lacked. spimdisasm itself printed 8 file-split suggestions (rodata 0x6324C, 0x63388, 0x633FC, 0x63920, 0x63C94, 0x64420, 0x64AB4, 0x64CA0). Canonical fix = per-file split — OR the PsyQ-lib pivot below (most of these are library code).
PsyQ-library-linking SPIKE — GO PROVEN (session C, the chosen +24 fix + free SDK code)
Finding: BFM's PsyQ library functions are byte-identical to the real PsyQ SDK objects → link them
directly (byte-exact) instead of hand-decompiling, which ALSO gives each library .o correct per-object
alignment (dissolving the library-half of the +24). Validated: CdPosToInt (32 instrs) + CdIntToPos (65)
EXACT vs PsyQ 4.7 libcd.a; CdPosToInt also in 4.0 LIBCD.LIB; PRESET_OBJ_108 (a +24 culprit) is
in 4.0 LIBGS.LIB → library code, not game. (Raw-byte lib search has false-negatives on relocated funcs,
e.g. _spu_FsetPCR/OBJT2/PRNT "missed" — needs the real ELF-link test to classify those.)
Assets staged (gitignored tools/psyq/): psyq-obj-parser (decompme prebuilt, works on .OBJ; rejects
.LIB archives — needs splitting); psyq4.0/ (4.0 tools: CC1PSX/ASPSX/PSYLIB/…); conv47/ (4.7 pre-converted
ELF .a — quick reference); lib40/*.LIB = the 20 PsyQ 4.0 USA libraries (DTL-S2002 R2.0, BFM's exact
version) extracted from the redump via our tools/bfm_extract/iso9660.py walker. Footprint in BFM: ~350 funcs
(libsnd 131, libapi/gs 69+, libmcrd 63, libsn 37, libcd 28, libspu 21, …) of 2050 matchable.
Integration pipeline — BUILT + PROVEN end-to-end (session C):
tools/psyq_lib_split.py(committed) — splits aLIB\x01archive into its member.OBJ(locates each member header by the invariantu32@(header+12) == LNK_offset − header). LIBCD → 25 objects ✓.tools/psyq_build_libs.sh(committed) —.LIB → .OBJ → psyq-obj-parser → ELF .o → arper lib. Built all 14 BFM libs →tools/psyq/lib40_elf/*.a(gitignored): LIBCD 25, LIBGS 201, LIBSPU 129, LIBSND 163, LIBMCRD 2, LIBSN 51, LIBAPI 90, LIBETC 7, LIBGTE 381, LIBGPU 12, LIBMATH 48, LIBCARD 18, LIBC 56, LIBC2 46.- Byte-match PROVEN at object level: in libcd
SYS.o, leavesCdPosToInt/CdIntToPosare EXACT; relocated funcs (CdComstr…) differ ONLY at their relocation sites → link byte-exact once relocs resolve to BFM symbol addrs. So every step (split, convert, leaf-match, reloc-resolve) is validated.
FULL OBJECT LINK — PROVEN BYTE-IDENTICAL (session C): libcd SYS.o (483 instrs, the full TU: leaves +
relocated funcs + 21 externals + internal .rdata/.data) links byte-for-byte identical to BFM. The pipeline
- the 3 last pieces:
- Identify placement (
tools/psyq_identify.py, committed) — relocation-masked search locates each object's.textin BFM. libcd: 18/25 objects found, CONTIGUOUS at 0x80043088–0x80046D1C in object order (the 7 unused — CDPLAY, C_012–015… — BFM doesn't link). So per-library placement = link the used objects in order at the region base; addresses are read off, not guessed. - Recover externals from BFM — symbols the object references but doesn't define (e.g. libcd's
CD_pos,CD_com,DMACallback) are NOT in symbols.us.txt, but their addresses are encoded in the EXE's already- RESOLVED relocations: parse the object's reloc records, read BFM at each site, reconstruct (R_MIPS_26 → target; HI16/LO16 pair → addr). Recovered all 21 for SYS.o. (Feeds symbols.us.txt over time.) - Alignment fix — psyq-obj-parser sets
.text/.rdata/.dataalign=2**3 (8); the original placed them 4-aligned, so an 8-align bumps them +4.objcopy --set-section-alignment .rdata=4 .data=4before linking → exact. (The +4 mismatch is the tell.) - Link recipe:
ld -T <script placing .text@<objaddr> .rdata@<island> .data@<addr>> --defsym <recovered…> obj.o→ objcopy .text → byte-compare. Proven on SYS.o.
REMAINING (replication + wiring, NEXT session, Task #5): generalize the SYS.o recipe to all used objects
per library (identify region → recover externals → set-align → place sections in order → link), then wire into
the build: drop the linked functions' INCLUDE_ASM + carve their raw data, add the lib objects to the link via a
generated .ld fragment. ~350 SDK funcs become byte-exact + the library-region jtbl alignment resolves. GAME
switches + LZSS (jtbl_80072A38 = island's first entry, before any misalignment) take the proven
migration+ld_interleave path above.
Blockers / open items
.LIB→.OBJsplitter (LIB\x01 format) — the gate for the lib-linking integration.- Which 4.x version matches each BFM lib object best (4.0 USA primary; BFM mixes 4.0+4.2 stamps, so some objects may need 4.2/4.3 libs — determine per-lib during integration via the ELF-link byte test).
- LZSS via the proven migration+ld_interleave path (independent of the lib pivot).
Notes
- Rule candidate (PhaseEnd, Drew-flagged session E): web-research the compiler internals (real compiler
source e.g.
pmret/gcc-papermario) + decomp community for compiler-quirk residuals (cross-jump, scheduling, regalloc) — a proven escalation tier above the permuter, below decomp.me. Found the LZSS cross-jump barrier. Captured in cookbook §3a/§5a + memoryweb-research-compiler-quirks. - Rule candidate (PhaseEnd, Drew-flagged session G): end every session/phase with a few-sentence
plain-English recap of what we did and WHY (no jargon) — keeps Drew oriented on real progress without
tracking the byte-level detail. Captured in memory
session-summary-plain-english. Apply immediately. - Session-E checkpoint commit: Drew directed a checkpoint commit (deviates from R8's strict one-commit-at-phase-end; consistent with the session A–D checkpoint commits in the git log). Commit local in WSL (no push, no Co-Authored-By per R5); Drew pushes via GitHub Desktop (R6).
.run/merge_matches.py= regenerate-800.c + re-apply-matches helper. H5 caveat: regen-fresh drops file-level/stub comments; for the permanent 800.c, surgically insert the 70 INCLUDE_RODATA lines instead.tools/ld_interleave.py(committed) = the.data→.rodata→.datalinker-script interleaver.