BFM-decomp/docs/psyq-worklist.md

# PsyQ library integration worklist (Phase 8)

> Byte-confirmed footprint map of the remaining PsyQ 4.0 libraries linked by `SLUS_007.26`.
> Generated by `psyq_identify.py` over the **original** EXE (`extracted/retail/SLUS_007.26`,
> text `0x80010000..0x800629DC`); raw per-object placements in `.run/survey/<lib>.txt` (regenerable:
> `for L in …; do python3 tools/psyq_identify.py .run/obj40/$L --vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26; done`). Replaces the Phase-7 ~350-fn
> *estimate* (R14). **Placement = candidate footprint; per-object byte-identity is confirmed at
> integration time by `psyq_link_region.py --verify` before wiring (the real gate, G3/P9).**
>
> **DONE (Gen1/Phase 7):** libcd (18 objs) + libgs (31 objs) linked byte-identical.

## Footprint summary (remaining libraries)

| Order | Library | Placed / total | Region(s) | ~banked B | Difficulty notes |
|---|---|---|---|---|---|
| 1 | **libgpu** | 3 / 12 | 800b2 (`0x80058890`) | ~14.9k | ✅ **DONE (T4): EXT+PRIM linked** (curated `libgpu_used`). **SYS.o EXCLUDED** — scattered-`.bss` (§9.1, GS_001 class; stays a stub in 800c). |
| 2 | **libetc** | 5 / 7 | 800 tail (`0x8004239C`) | ~3.3k | VSYNC/INTR/INTR_VB/INTR_DMA/VMODE **contiguous**, ends exactly at libcd1 (`0x80043088`). Clean. |
| 3 | **libmcrd** | 2 / 2 | 800b2 (`0x8005FC68`,`0x80062888`) | ~9.0k | LIBMCRD.o (2186 ins, huge, holds the 55 `LIBMCRD_OBJ_*`) + USERFUNC.o — **2 blocks**. `_card_*` h_norm dups collapse inside LIBMCRD. |
| 4 | **libc2** | 17 / 46 | 800b2 (`0x8005C2C8`–`0x5CD98` + STRCAT `0x80061E90`) | ~3.1k | C stdlib (BZERO/MEMCPY/STRCMP/PRINTF/PRNT…); contiguous run + 1 outlier. **PRNT.o (418 ins) has an internal jtbl** — verify NOLOAD `.rodata` placement (the `PRNT_OBJ_24C` rodata note). |
| 5+6 | **libapi+libcard** | 22 (800c2) | 800c2 (`0x80061F38`–`0x80062888`) | ~0.7k | ✅ **DONE (combined apicard region): 22 objs / 4 blocks / +24 fns** (`tools/make_apicard_used.py`, C112 dedup, 0 exclusions). **libapi's ~22 objects in the 800c3 region (`0x5CE18`..) DEFERRED** — lowest value, separate resegmentation. |
| 7+8 | **libspu+libsnd** | 60 / (38+32) | 800 sound (`0x8003A444`–`0x8004239C`) | ~24k | ✅ **DONE (combined region): 60 objs / 9 blocks / +225 fns.** The two libs interleave, so linked as ONE region (`tools/make_snd_used.py` dedups + excludes 4 addresses; `gen_lib_subsegs.py` + window). **Excluded:** S_R/S_W `0x3C438`, S_GRMDT* `0x3D424`, S_IH/UT_RON `0x3D94C` (false-positive, inside SSSTART), VM_F `0x3FA64` (237 ins) — all scattered-`.bss`/false-pos, stay stubs. SSGM.o `0x1BD80` deferred (isolated in matched-C, 8 ins). |
| 9 | **libgte** | 58 / 381 | 800b (`0x8004787C`–`0x5082C`) + **libgs gaps** | ~48k | ✅ **DONE (T11): 53 objs / 22 blocks linked** in 800b (subsegs via `gen_lib_subsegs.py`; integrate window 0x4787C..0x51804). **5 libgs-gap objects DEFERRED** (MTX_05/07/11/REG03/REG11 → gsgap1/2/4/5 stay stubs; gsgap2≠MTX_07 exactly so needs sub-split). |

**SKIP — zero footprint (recorded, not linked by the EXE):** `libmath` 0/48, `libc` 0/56, `libsn` 0/51 (2 tiny ambiguous). BFM links **libc2**, not libc; no libmath/libsn. (Also unbuilt/no-footprint: libcomb, libds, libgun, libsio, libtap, libpress — never converted, no symbols.)

## Region map (where each library lives, relative to the current splat subsegs)

```
800   (0x800123F0–0x80043088)  game code + libsnd(SSGM @1BD80; 3D454–42374) + libspu(3A444–422E8) + libetc(4239C–43088, tail)
libcd (0x80043088–0x8004787C)  DONE
800b  (0x8004787C–0x80051804)  libgte(4787C–5082C, multi-block) + game-code tail
libgs (0x80051804–0x80057928)  DONE; gaps gsgap1/2/4/5 are actually libgte (MTX_05/07/11, REG03/11); gsgap3=GS_001 excluded
800b2 (0x80057928–0x800629DC)  game code + libgpu(58890–5Bxxx) + libc2(5C2C8–5CD98,+STRCAT 61E90) + libapi(5CE18–626B8 scattered) + libmcrd(5FC68 + 62888) + libcard(61F38–62808)
```

## Integration order rationale (ascending difficulty)
- **1–2 (libgpu, libetc):** clean single contiguous blocks, minimal interleaving → re-validate the §9 mechanism end-to-end on a fresh library before the hard ones.
- **3–4 (libmcrd, libc2):** few blocks, contained in 800b2; libc2 introduces the PRNT internal-jtbl wrinkle.
- **5–6 (libcard, libapi):** short syscall stubs + cross-library aliases + scatter in 800b2 → need windowed placement + curated `_used` dirs.
- **7–8 (libspu, libsnd):** the interleaved sound pair in the 800 subseg (which still holds the 43 REAL game-code matches lower down) → `split_src_region.py` trims, windowed placement, alias disambiguation.
- **9 (libgte):** largest, multi-region (800b + 4 libgs gaps), game-code-interleaved blocks → last.

## Per-library integration recipe (each = one byte-gated task, §9.3/§9.5)
> **Phase 9:** all these tools now REQUIRE `--vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26`
> (the EXE's values; `psyq_integrate` also `--symbols config/symbols.us.txt`). For a Gen2 binary, pass
> its own `--vram-base`/`--exe`/`--symbols` (cookbook §9.7, SETUP §6.7).
1. `psyq_link_region.py .run/obj40/<lib> <lo> <hi> --vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26 [--emit <p>]` → confirm per-object byte-identity ("per-object .text byte-identical: True") + the block/gap structure (G1: derive blocks empirically, don't trust this table's addresses).
2. Curate `.run/obj40/<lib>_used` (drop cross-library aliases / keep the variant whose `.data`/`.rdata` also matches) when aliases exist (make_libgs.sh pattern).
3. Resegment `config/splat.us.exe.yaml` (one `c` subseg per contiguous block; non-library gaps stay stubs); `split_src_region.py trim` the affected game-code `.c`; `make extract`.
4. Makefile: add `LIB<X>_ELF/OBJDIR/SYMS` + a gated `psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) … <stubs> [lo hi]` call + `-T $(LIB<X>_SYMS)` on the `ld` line.
5. Add recovered SDK names to `config/symbols.us.txt` (mirror to Ghidra, R15/G6); update progress.py's LINKED config.
6. **Dual byte-gate:** `make clean && extract && build && check` → `143dbb89…` WITH `.run/obj40/<lib>` present AND with it absent (stub fallback). `make report` LINKED up. Commit (R20).

**Cross-cutting:** windowed placement is mandatory for the short-object / interleaved libraries (the libgs GS_106 precedent); cross-library address collisions (C112 libapi/libcard; UT_RON/S_IH; S_R/S_W) mean total distinct objects < the 199 summed placements — byte-verify decides ownership. Any object that won't link byte-identical after §9.1/§9.4 stays a documented honest stub (does not block exit, P9).

**⚠️ Boundary gotcha (found at libc2/T6):** the game-code subseg boundary AFTER a library block must be the last object's **`.text` SECTION size** (8-aligned), NOT its instruction count × 4. `psyq_identify` reports the instruction count, which omits the object's trailing 8-byte alignment padding (e.g. libc2 SETJMP.o: 30 ins = 0x78, but `.text` = 0x80 → block ends 8 bytes later). A too-low boundary overlaps the object's padded tail and the relink inserts +N padding, shifting the whole downstream image (pervasive 1-byte reloc diffs + grown file). Verify the boundary = `last_obj.vram + readelf .text size` for any library whose last object isn't followed object-to-object. Also: when a library block sits at a subseg's START (no `< lo` prefix to keep), `rm` the old `.c` so splat regenerates it under the new boundaries (split_src_region.trim would leave it header-only).

## Excluded / deferred objects (§9.1 scattered-`.bss`) — honest stubs, byte-identical via asm
The standard mechanism places each object's `.bss` as ONE NOLOAD section at a single base. An object whose
`.bss` commons the **original linker scattered** to non-contiguous addresses (referenced as `.bss`+offset by a
single section symbol) cannot be reproduced by one base → excluded, kept as an INCLUDE_ASM stub (still
byte-identical). Diagnose: `psyq_link_region.py <dir>` shows N words differ in the object, and its `.bss`
references resolve to >1 base in the EXE. Curate the library's `_used` dir to drop these.

| Object | Library | Evidence | Status |
|---|---|---|---|
| `GS_001.o` | libgs | scattered `.bss` (Phase 7) | excluded (gsgap3 stub) |
| `SYS.o` (3109 ins) | libgpu | `.bss`+0x150 → `0x800c551c` but base recovered `0x80078830`; commons scattered 0x80078xxx/0x800c5xxx | **excluded (T4); stub in 800c** |
| `MTX_05/07/11`,`REG03`,`REG11` | libgte | sit in libgs gaps gsgap1/2/4/5; gsgap2(48B)≠MTX_07(36B) so the gap stub needs a sub-split | **deferred (T11)**; small GTE fns; link byte-identical, just need the gsgap region resegmented (low priority) |
| `0x3C438`,`0x3D424`,`0x3D94C`,`0x3FA64` (S_R/S_GRMDT/S_IH/VM_F) | libspu/snd | scattered-`.bss` cross-object (S_R/S_GRMDT/VM_F) + a false placement inside SSSTART (S_IH) | **excluded (sound region)**; stay stubs; VM_F (237 ins) the only sizable one |
| `SSGM.o` | libsnd | isolated @0x1BD80, inside the matched-C region (near func_8001Bxxx) | **deferred**; 8 ins; would need a 1-object carve amid matched C |
| libapi 800c3 cluster (~22 objs) | libapi | C57..L10/L02/L03 @0x5CE18.. in the 800c3 region (separate from the 800c2 apicard region) | **deferred**; ~22 4-ins BIOS syscall stubs; lowest value; another region resegmentation |

*If scattered-`.bss` proves prevalent across libgte/libspu/libsnd, escalate to a Max general fix (split each object's `.bss` into per-common NOLOAD sections at their EXE-resolved addresses); otherwise excluding the few affected objects is the GS_001-precedent decision.*

---

## The RESIDENT has **no linkable PsyQ footprint** — it is custom engine code (Phase 12 T1, byte-verified)

> **Pre-Phase-12 assumption (now corrected):** because the resident detects as **PsyQ 4.7.0** (DetectPsyQ + a `DsMix`/libsnd signature hit), it was expected to link embedded 4.7 libsnd/libgte/libspu (the Phase-11 close-out + R24 "link the resident's 4.7 libs" opener). **The bytes refuted this.**

**Phase 12 T1 survey** ran `psyq_identify` over the 4.7 objects (`tools/psyq/conv47/…/lib/`, already ELF — `ar x` into `.run/obj47/<lib>/`, no psyq-obj-parser needed) **and** the 4.0 objects, against the resident (`MAIN.CD/FILE_010/1.1`, `--vram-base 0x800CEDF8`), over the code window **and** the whole file. Result — **NIL footprint:**

| Lib (4.7) | placed | | Lib (4.0) | placed |
|---|---|---|---|---|
| libsnd | **1/226** (`ut_rev_2.o`, 4 ins) | | libsnd | **2/163** (`VM_DON.o` 5 ins, `VM_DOFF.o` 4 ins) |
| libspu | 0/134 | | libspu | 0/129 |
| libgte | 0/509 | | libgte | 0/381 |
| libgpu | 0/61 | | | |
| libcd/libmath/libds | 1 each (≤8 ins, two alias the same DsMix addr) | | | |

All hits are the **§9.5 short-object coincidental class** (≤8 ins, masked patterns match by luck). **Tool sanity (the negative is real):** 4.0 libsnd vs the **EXE** snd region = **35/163** placed — the tool works and is version-sensitive; the resident genuinely embeds **no stock PsyQ object of either version**.

**Ghidra corroboration (G1, sampled):** `DsMix` decompiles to `{ FUN_800d1bf8(); return 1; }` — a custom 2-line wrapper that **ignores its `vol` arg** (NOT the stock libsnd `DsMix`; the R13 tag in `symbols.resident.txt` is refuted). Other sampled functions are game logic (global accessors, engine init calling EXE REAL matches, entity-heading math calling the EXE's libgte `RATAN`). The resident's code makes **61 distinct EXE-range `jal` calls** (vs 37 internal) — it **calls** the EXE's resident SDK rather than embedding it.

**Conclusion / architecture:** the PsyQ SDK lives in the **EXE** (959 LINKED); the **resident is ~143 functions of custom engine code** that calls the EXE's SDK + engine via fixed addresses (no RAM-wasting SDK duplication in an always-loaded blob). The DetectPsyQ "4.7.0" was a single coincidental DsMix-region signature, not a linked footprint. **Phase 12 matches the resident engine by hand (REAL), not by linking (LINKED stays 0).** R24's per-binary-provenance principle holds, but for the resident the practical consequence is "nothing to link." *(Regenerate: `for L in libsnd libspu libgte libgpu; do d=.run/obj47/$L; mkdir -p $d; (cd $d && ar x ../../tools/psyq/conv47/psyq-4_7-converted/lib/$L.a); python3 tools/psyq_identify.py $d 0x800CEDFC 0x800D3408 --vram-base 0x800CEDF8 --exe extracted/retail/MAIN.CD.dir/FILE_010.dir/1.1; done`)*