BFM-decomp/docs/psyq-worklist.md

PsyQ library integration worklist (Phase 8)

Byte-confirmed footprint map of the remaining PsyQ 4.0 libraries linked by SLUS_007.26. Generated by psyq_identify.py over the original EXE (extracted/retail/SLUS_007.26, text 0x80010000..0x800629DC); raw per-object placements in .run/survey/<lib>.txt (regenerable: for L in …; do python3 tools/psyq_identify.py .run/obj40/$L --vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26; done). Replaces the Phase-7 ~350-fn estimate (R14). Placement = candidate footprint; per-object byte-identity is confirmed at integration time by psyq_link_region.py --verify before wiring (the real gate, G3/P9).

DONE (Gen1/Phase 7): libcd (18 objs) + libgs (31 objs) linked byte-identical.

Footprint summary (remaining libraries)

Order	Library	Placed / total	Region(s)	~banked B	Difficulty notes
1	libgpu	3 / 12	800b2 (0x80058890)	~14.9k	✅ DONE (T4): EXT+PRIM linked (curated libgpu_used). SYS.o EXCLUDED — scattered-.bss (§9.1, GS_001 class; stays a stub in 800c).
2	libetc	5 / 7	800 tail (0x8004239C)	~3.3k	VSYNC/INTR/INTR_VB/INTR_DMA/VMODE contiguous, ends exactly at libcd1 (0x80043088). Clean.
3	libmcrd	2 / 2	800b2 (0x8005FC68,0x80062888)	~9.0k	LIBMCRD.o (2186 ins, huge, holds the 55 LIBMCRD_OBJ_*) + USERFUNC.o — 2 blocks. _card_* h_norm dups collapse inside LIBMCRD.
4	libc2	17 / 46	800b2 (0x8005C2C8–0x5CD98 + STRCAT 0x80061E90)	~3.1k	C stdlib (BZERO/MEMCPY/STRCMP/PRINTF/PRNT…); contiguous run + 1 outlier. PRNT.o (418 ins) has an internal jtbl — verify NOLOAD .rodata placement (the PRNT_OBJ_24C rodata note).
5+6	libapi+libcard	22 (800c2)	800c2 (0x80061F38–0x80062888)	~0.7k	✅ DONE (combined apicard region): 22 objs / 4 blocks / +24 fns (tools/make_apicard_used.py, C112 dedup, 0 exclusions). libapi's ~22 objects in the 800c3 region (0x5CE18..) DEFERRED — lowest value, separate resegmentation.
7+8	libspu+libsnd	60 / (38+32)	800 sound (0x8003A444–0x8004239C)	~24k	✅ DONE (combined region): 60 objs / 9 blocks / +225 fns. The two libs interleave, so linked as ONE region (tools/make_snd_used.py dedups + excludes 4 addresses; gen_lib_subsegs.py + window). Excluded: S_R/S_W 0x3C438, S_GRMDT* 0x3D424, S_IH/UT_RON 0x3D94C (false-positive, inside SSSTART), VM_F 0x3FA64 (237 ins) — all scattered-.bss/false-pos, stay stubs. SSGM.o 0x1BD80 deferred (isolated in matched-C, 8 ins).
9	libgte	58 / 381	800b (0x8004787C–0x5082C) + libgs gaps	~48k	✅ DONE (T11): 53 objs / 22 blocks linked in 800b (subsegs via gen_lib_subsegs.py; integrate window 0x4787C..0x51804). 5 libgs-gap objects DEFERRED (MTX_05/07/11/REG03/REG11 → gsgap1/2/4/5 stay stubs; gsgap2≠MTX_07 exactly so needs sub-split).

SKIP — zero footprint (recorded, not linked by the EXE): libmath 0/48, libc 0/56, libsn 0/51 (2 tiny ambiguous). BFM links libc2, not libc; no libmath/libsn. (Also unbuilt/no-footprint: libcomb, libds, libgun, libsio, libtap, libpress — never converted, no symbols.)

Region map (where each library lives, relative to the current splat subsegs)

800   (0x800123F0–0x80043088)  game code + libsnd(SSGM @1BD80; 3D454–42374) + libspu(3A444–422E8) + libetc(4239C–43088, tail)
libcd (0x80043088–0x8004787C)  DONE
800b  (0x8004787C–0x80051804)  libgte(4787C–5082C, multi-block) + game-code tail
libgs (0x80051804–0x80057928)  DONE; gaps gsgap1/2/4/5 are actually libgte (MTX_05/07/11, REG03/11); gsgap3=GS_001 excluded
800b2 (0x80057928–0x800629DC)  game code + libgpu(58890–5Bxxx) + libc2(5C2C8–5CD98,+STRCAT 61E90) + libapi(5CE18–626B8 scattered) + libmcrd(5FC68 + 62888) + libcard(61F38–62808)

Integration order rationale (ascending difficulty)

• 1–2 (libgpu, libetc): clean single contiguous blocks, minimal interleaving → re-validate the §9 mechanism end-to-end on a fresh library before the hard ones.
• 3–4 (libmcrd, libc2): few blocks, contained in 800b2; libc2 introduces the PRNT internal-jtbl wrinkle.
• 5–6 (libcard, libapi): short syscall stubs + cross-library aliases + scatter in 800b2 → need windowed placement + curated _used dirs.
• 7–8 (libspu, libsnd): the interleaved sound pair in the 800 subseg (which still holds the 43 REAL game-code matches lower down) → split_src_region.py trims, windowed placement, alias disambiguation.
• 9 (libgte): largest, multi-region (800b + 4 libgs gaps), game-code-interleaved blocks → last.

Per-library integration recipe (each = one byte-gated task, §9.3/§9.5)

Phase 9: all these tools now REQUIRE --vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26 (the EXE's values; psyq_integrate also --symbols config/symbols.us.txt). For a Gen2 binary, pass its own --vram-base/--exe/--symbols (cookbook §9.7, SETUP §6.7).

1. psyq_link_region.py .run/obj40/<lib> <lo> <hi> --vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26 [--emit <p>] → confirm per-object byte-identity ("per-object .text byte-identical: True") + the block/gap structure (G1: derive blocks empirically, don't trust this table's addresses).
2. Curate .run/obj40/<lib>_used (drop cross-library aliases / keep the variant whose .data/.rdata also matches) when aliases exist (make_libgs.sh pattern).
3. Resegment config/splat.us.exe.yaml (one c subseg per contiguous block; non-library gaps stay stubs); split_src_region.py trim the affected game-code .c; make extract.
4. Makefile: add LIB<X>_ELF/OBJDIR/SYMS + a gated psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) … <stubs> [lo hi] call + -T $(LIB<X>_SYMS) on the ld line.
5. Add recovered SDK names to config/symbols.us.txt (mirror to Ghidra, R15/G6); update progress.py's LINKED config.
6. Dual byte-gate: make clean && extract && build && check → 143dbb89… WITH .run/obj40/<lib> present AND with it absent (stub fallback). make report LINKED up. Commit (R20).

Cross-cutting: windowed placement is mandatory for the short-object / interleaved libraries (the libgs GS_106 precedent); cross-library address collisions (C112 libapi/libcard; UT_RON/S_IH; S_R/S_W) mean total distinct objects < the 199 summed placements — byte-verify decides ownership. Any object that won't link byte-identical after §9.1/§9.4 stays a documented honest stub (does not block exit, P9).

⚠️ Boundary gotcha (found at libc2/T6): the game-code subseg boundary AFTER a library block must be the last object's .text SECTION size (8-aligned), NOT its instruction count × 4. psyq_identify reports the instruction count, which omits the object's trailing 8-byte alignment padding (e.g. libc2 SETJMP.o: 30 ins = 0x78, but .text = 0x80 → block ends 8 bytes later). A too-low boundary overlaps the object's padded tail and the relink inserts +N padding, shifting the whole downstream image (pervasive 1-byte reloc diffs + grown file). Verify the boundary = last_obj.vram + readelf .text size for any library whose last object isn't followed object-to-object. Also: when a library block sits at a subseg's START (no < lo prefix to keep), rm the old .c so splat regenerates it under the new boundaries (split_src_region.trim would leave it header-only).

Excluded / deferred objects (§9.1 scattered-.bss) — honest stubs, byte-identical via asm

The standard mechanism places each object's .bss as ONE NOLOAD section at a single base. An object whose .bss commons the original linker scattered to non-contiguous addresses (referenced as .bss+offset by a single section symbol) cannot be reproduced by one base → excluded, kept as an INCLUDE_ASM stub (still byte-identical). Diagnose: psyq_link_region.py <dir> shows N words differ in the object, and its .bss references resolve to >1 base in the EXE. Curate the library's _used dir to drop these.

Object	Library	Evidence	Status
GS_001.o	libgs	scattered .bss (Phase 7)	excluded (gsgap3 stub)
SYS.o (3109 ins)	libgpu	.bss+0x150 → 0x800c551c but base recovered 0x80078830; commons scattered 0x80078xxx/0x800c5xxx	excluded (T4); stub in 800c
MTX_05/07/11,REG03,REG11	libgte	sit in libgs gaps gsgap1/2/4/5; gsgap2(48B)≠MTX_07(36B) so the gap stub needs a sub-split	deferred (T11); small GTE fns; link byte-identical, just need the gsgap region resegmented (low priority)
0x3C438,0x3D424,0x3D94C,0x3FA64 (S_R/S_GRMDT/S_IH/VM_F)	libspu/snd	scattered-.bss cross-object (S_R/S_GRMDT/VM_F) + a false placement inside SSSTART (S_IH)	excluded (sound region); stay stubs; VM_F (237 ins) the only sizable one
SSGM.o	libsnd	isolated @0x1BD80, inside the matched-C region (near func_8001Bxxx)	deferred; 8 ins; would need a 1-object carve amid matched C
libapi 800c3 cluster (~22 objs)	libapi	C57..L10/L02/L03 @0x5CE18.. in the 800c3 region (separate from the 800c2 apicard region)	deferred; ~22 4-ins BIOS syscall stubs; lowest value; another region resegmentation

If scattered-.bss proves prevalent across libgte/libspu/libsnd, escalate to a Max general fix (split each object's .bss into per-common NOLOAD sections at their EXE-resolved addresses); otherwise excluding the few affected objects is the GS_001-precedent decision.

---

The RESIDENT has no linkable PsyQ footprint — it is custom engine code (Phase 12 T1, byte-verified)

Pre-Phase-12 assumption (now corrected): because the resident detects as PsyQ 4.7.0 (DetectPsyQ + a DsMix/libsnd signature hit), it was expected to link embedded 4.7 libsnd/libgte/libspu (the Phase-11 close-out + R24 "link the resident's 4.7 libs" opener). The bytes refuted this.

Phase 12 T1 survey ran psyq_identify over the 4.7 objects (tools/psyq/conv47/…/lib/, already ELF — ar x into .run/obj47/<lib>/, no psyq-obj-parser needed) and the 4.0 objects, against the resident (MAIN.CD/FILE_010/1.1, --vram-base 0x800CEDF8), over the code window and the whole file. Result — NIL footprint:

Lib (4.7)	placed		Lib (4.0)	placed
libsnd	1/226 (ut_rev_2.o, 4 ins)		libsnd	2/163 (VM_DON.o 5 ins, VM_DOFF.o 4 ins)
libspu	0/134		libspu	0/129
libgte	0/509		libgte	0/381
libgpu	0/61
libcd/libmath/libds	1 each (≤8 ins, two alias the same DsMix addr)

All hits are the §9.5 short-object coincidental class (≤8 ins, masked patterns match by luck). Tool sanity (the negative is real): 4.0 libsnd vs the EXE snd region = 35/163 placed — the tool works and is version-sensitive; the resident genuinely embeds no stock PsyQ object of either version.

Ghidra corroboration (G1, sampled): DsMix decompiles to { FUN_800d1bf8(); return 1; } — a custom 2-line wrapper that ignores its vol arg (NOT the stock libsnd DsMix; the R13 tag in symbols.resident.txt is refuted). Other sampled functions are game logic (global accessors, engine init calling EXE REAL matches, entity-heading math calling the EXE's libgte RATAN). The resident's code makes 61 distinct EXE-range jal calls (vs 37 internal) — it calls the EXE's resident SDK rather than embedding it.

Conclusion / architecture: the PsyQ SDK lives in the EXE (959 LINKED); the resident is ~143 functions of custom engine code that calls the EXE's SDK + engine via fixed addresses (no RAM-wasting SDK duplication in an always-loaded blob). The DetectPsyQ "4.7.0" was a single coincidental DsMix-region signature, not a linked footprint. Phase 12 matches the resident engine by hand (REAL), not by linking (LINKED stays 0). R24's per-binary-provenance principle holds, but for the resident the practical consequence is "nothing to link." (Regenerate: for L in libsnd libspu libgte libgpu; do d=.run/obj47/$L; mkdir -p $d; (cd $d && ar x ../../tools/psyq/conv47/psyq-4_7-converted/lib/$L.a); python3 tools/psyq_identify.py $d 0x800CEDFC 0x800D3408 --vram-base 0x800CEDF8 --exe extracted/retail/MAIN.CD.dir/FILE_010.dir/1.1; done)