mirror/BFM-decomp
1
0
Fork 0
mirror of https://github.com/Druthulu/BFM-decomp synced 2026-09-26 13:33:34 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(phase-31): S79 #4 — scattered-.bss split at link-prepare (psyq_bss_split): SYS.o→libgpu2, VM_F.o→snd12, GS_001.o→libgs8 LINKED; libgpu_used retired

Browse Source 
The §9.1 "scattered .bss commons" exclusion class (Phase 8 → P31) is closed 3/3. New
tools/psyq_bss_split.py (own ELF32 REL reader/writer) cuts an object's packed .bss into
per-base NOBITS pieces: bases derived from the game bytes per HI16/LO16 pair, references
walked in offset order into single-base runs, cuts snapped to symbol starts (the linker
scattered SYMBOLS), symbols moved, a LOCAL section symbol per piece inserted, relocs
retargeted with the addend rewritten in the immediates, self-diffed. It runs inside the one
prepare step shared by psyq_link.link_object / psyq_link_region.build_region /
psyq_integrate.integrate (prepare_object before classify), re-derived every build.

GS_001.o was certified "5 interleaved bases, NOT splittable" by the S77 probe, which grouped
by BASE; by RUN it is six symbol-aligned pieces. All seven cuts across the three objects are
confirmed by the other objects' by-name recoveries (_que 0x800C5510, _svm_sreg_buf
0x800B9B58, PSDBASEX/CLIP2/PSDBASEY/POSITION/GsDRAWENV). R39 negative control: 235 placed
objects across 9 curated dirs, 0 refusals, exactly 3 splits (a libcd .bss+size end pointer
refused the first build → reference problems are fatal only when a split is needed).

Wiring: yaml 800c→libgpu2, sgap_6→sgap_6+snd12, gsgap3→libgs8 (comments rewritten);
LIBGPU_ELF := .run/obj40/libgpu (curated libgpu_used retired); libgs 34 objs/8 blocks
(make_libgs.sh +GS_001); snd 63/12 (make_snd_used.py exclusions 4→3). src/800c.c and
src/gsgap3.c removed (Sony code hand-matched as REAL/verbatim), sgap_6.c keeps only
func_8003FA54; splat-emitted libgpu2.c/libgs8.c/snd12.c stubs for the no-SDK fallback.

Verified: main 143dbb89f34491258bbc27810d0a12ec8b43a8dd WITH the SDK objects and WITHOUT
them from a fresh extract; make tools-health OK; R22 fleet clean extract-all 212/212 +
check-all 213/213. Metrics: main REAL 886→839, LINKED 1,040→1,150, VERBATIM 85→29, stubs 29
(unchanged); game-code weighted 91.1% (40,895/44,870) — both terms lost the 3,667 SDK ins;
the remainder is still exactly the 3,975-ins open-stub sum. Verbatim manifest --update
200→33 rows (subtractive). Docs: cookbook §489 (+index), psyq-worklist rows + "S78 task #4",
SETUP S79 R21 table, decision-log S79 addendum, accelerators S79, CURRENT_PHASE S79 FINAL 🛑.
This commit is contained in:
Drew T
2026-09-04 17:19:29 -06:00
parent 4c32475299
commit 757bd82a0f
26 changed files with 1354 additions and 6665 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Makefile
+18 -14
View File
@@ -549,11 +549,12 @@ LIBCD_ELF := .run/obj40/libcd
LIBCD_OBJDIR := build/psyq/libcd
LIBCD_SYMS := build/psyq/libcd_externals.ld
# libgs (Phase 7 Task #9, FULL integration): 31 libgs objects in 6 contiguous blocks linked in place
# of the libgs1..libgs6 block stubs (the 5 non-libgs gaps stay gsgapN asm stubs). Same conditional/
# libgs (Phase 7 Task #9, FULL integration; S78 #3/#4): 34 libgs objects in 8 blocks linked in place
# of the libgs1..libgs8 block stubs (libgs7 = 2D_BG0/2D_BG1, S78 #3; libgs8 = GS_001 via the link-prepare
# .bss split, S78 #4; the 4 remaining gaps are libgte objects, libgte27-30). Same conditional/
# idempotent model as libcd. The curated object dir is SDK-derived (gitignored), regenerated by
# tools/make_libgs.sh (needs the LIBGS ELF from psyq_build_libs.sh LIBGS). GS_106 (block 4) anchors
# uniquely only within the libgs window, so the integrate call passes 0x80051804 0x80057928.
# uniquely only within the libgs window, so the integrate call passes 0x8005080C 0x80057928.
LIBGS_ELF := .run/obj40/libgs_used
LIBGS_OBJDIR := build/psyq/libgs
LIBGS_SYMS := build/psyq/libgs_externals.ld
@@ -565,10 +566,12 @@ LIBETC_ELF := .run/obj40/libetc
LIBETC_OBJDIR := build/psyq/libetc
LIBETC_SYMS := build/psyq/libetc_externals.ld
# libgpu (Phase 8): EXT+PRIM only — SYS.o EXCLUDED (scattered-.bss, cookbook §9.1, GS_001 class; stays a
# stub in 800c). Curated dir libgpu_used = {EXT,PRIM}; regenerate: tools/psyq_build_libs.sh LIBGPU then
# `mkdir -p .run/obj40/libgpu_used && cp .run/obj40/libgpu/{EXT,PRIM}.o .run/obj40/libgpu_used/`.
LIBGPU_ELF := .run/obj40/libgpu_used
# libgpu (Phase 8; S78 #4): EXT+PRIM (block `libgpu`) + SYS.o (block `libgpu2`, 3109 ins). SYS.o was
# EXCLUDED from Phase 8 to P31 S78 as scattered-.bss (cookbook §9.1, the GS_001 class) and lived in 800c
# as hand-matched SDK C + verbatim frags; psyq_integrate now splits such a .bss into per-base NOLOAD
# pieces at link-prepare (tools/psyq_bss_split.py, cookbook §489), so the raw psyq_build_libs.sh LIBGPU
# output links directly — no curated dir (psyq_identify drops the 8 objects the EXE does not link).
LIBGPU_ELF := .run/obj40/libgpu
LIBGPU_OBJDIR := build/psyq/libgpu
LIBGPU_SYMS := build/psyq/libgpu_externals.ld
@@ -595,13 +598,14 @@ LIBGTE_OBJDIR := build/psyq/libgte
LIBGTE_SYMS := build/psyq/libgte_externals.ld
LIBGTE_STUBS := libgte1,libgte2,libgte3,libgte4,libgte5,libgte6,libgte7,libgte8,libgte9,libgte10,libgte11,libgte12,libgte13,libgte14,libgte15,libgte16,libgte17,libgte18,libgte19,libgte20,libgte21,libgte22,libgte23,libgte24,libgte25,libgte26,libgte27,libgte28,libgte29,libgte30
# Combined libspu+libsnd sound region (Phase 8): the two SDK sound libs interleave in 0x3A444..0x4239C
# so they link as one 60-object region (snd1..snd9). Curated dir .run/obj40/snd_used built by
# tools/make_snd_used.py (4 addresses excluded as scattered-.bss/false-positive stubs). Window arg below.
# Combined libspu+libsnd sound region (Phase 8; S78 #3/#4): the two SDK sound libs interleave in
# 0x3A444..0x4239C so they link as one 63-object region (snd1..snd12). Curated dir .run/obj40/snd_used
# built by tools/make_snd_used.py (3 addresses excluded as cross-object-common/false-positive stubs;
# VM_F rejoined in S78 #4 via the link-prepare .bss split, cookbook §489). Window arg below.
SND_ELF := .run/obj40/snd_used
SND_OBJDIR := build/psyq/snd
SND_SYMS := build/psyq/snd_externals.ld
SND_STUBS := snd1,snd2,snd3,snd4,snd5,snd6,snd7,snd8,snd9,snd10,snd11
SND_STUBS := snd1,snd2,snd3,snd4,snd5,snd6,snd7,snd8,snd9,snd10,snd11,snd12
# Combined libapi+libcard 800c2 region (Phase 8): 22 objects in 4 blocks (apicard1..4). Curated dir
# .run/obj40/apicard_used (tools/make_apicard_used.py). Window 0x61F38..0x62888. (libapi's ~22 objects
@@ -839,7 +843,7 @@ ifeq ($(BINARY),main)
echo " (no $(LIBCD_ELF) — libcd region stays asm stubs; run tools/psyq_build_libs.sh LIBCD)"
fi
if [ -d "$(LIBGS_ELF)" ]; then
$(PYTHON) tools/psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) --yaml $(main_SPLAT_YAML) $(LIBGS_ELF) $(LD_SCRIPT) $(LIBGS_OBJDIR) $(LIBGS_SYMS) libgs1,libgs2,libgs3,libgs4,libgs5,libgs6,libgs7 0x8005080C 0x80057928
$(PYTHON) tools/psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) --yaml $(main_SPLAT_YAML) $(LIBGS_ELF) $(LD_SCRIPT) $(LIBGS_OBJDIR) $(LIBGS_SYMS) libgs1,libgs2,libgs3,libgs4,libgs5,libgs6,libgs7,libgs8 0x8005080C 0x80057928
else
echo " (no $(LIBGS_ELF) — libgs region stays asm stubs; run tools/make_libgs.sh)"
fi
@@ -849,9 +853,9 @@ ifeq ($(BINARY),main)
echo " (no $(LIBETC_ELF) — libetc region stays asm stubs; run tools/psyq_build_libs.sh LIBETC)"
fi
if [ -d "$(LIBGPU_ELF)" ]; then
$(PYTHON) tools/psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) --yaml $(main_SPLAT_YAML) $(LIBGPU_ELF) $(LD_SCRIPT) $(LIBGPU_OBJDIR) $(LIBGPU_SYMS) libgpu
$(PYTHON) tools/psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) --yaml $(main_SPLAT_YAML) $(LIBGPU_ELF) $(LD_SCRIPT) $(LIBGPU_OBJDIR) $(LIBGPU_SYMS) libgpu,libgpu2
else
echo " (no $(LIBGPU_ELF) — libgpu region stays asm stubs; run tools/psyq_build_libs.sh LIBGPU + curate libgpu_used)"
echo " (no $(LIBGPU_ELF) — libgpu region stays asm stubs; run tools/psyq_build_libs.sh LIBGPU)"
fi
if [ -d "$(LIBMCRD_ELF)" ]; then
$(PYTHON) tools/psyq_integrate.py --vram-base $(main_VRAM_BASE) --exe $(main_EXE) --symbols $(main_SYMBOLS) --yaml $(main_SPLAT_YAML) $(LIBMCRD_ELF) $(LD_SCRIPT) $(LIBMCRD_OBJDIR) $(LIBMCRD_SYMS) libmcrd1,libmcrd2
config/splat.us.exe.yaml
+35 -26
View File
@@ -92,10 +92,12 @@ segments:
- [0x1D0BC, c, 800_b_2] # -O2 -> src/800_b_2.c (vram 0x8002C8BC-0x80035270); owns .rodata span B (tail)
- [0x25A70, c, 800_c] # -O2 game code -> src/800_c.c (vram 0x80035270-0x8003A444); owns .rodata span C
# PsyQ libspu+libsnd COMBINED sound region (Phase 8): the two SDK sound libs are interleaved here,
# so they link as one 60-object region (curated by tools/make_snd_used.py; 4 addresses excluded as
# scattered-.bss/false-positive stubs: S_R/S_W 0x3C438, S_GRMDT* 0x3D424, S_IH/UT_RON 0x3D94C,
# VM_F 0x3FA64). Subsegs via gen_lib_subsegs.py; integrate window 0x3A444..0x4239C. snd1..snd9 link;
# sgap* = game code + the 4 excluded stubs. (libsnd SSGM.o @0x1BD80 stays a stub in 800.)
# so they link as one 63-object region (curated by tools/make_snd_used.py; 3 addresses excluded as
# cross-object-common/false-positive stubs: S_R/S_W 0x3C438, S_GRMDT* 0x3D424, S_IH/UT_RON 0x3D94C).
# VM_F 0x3FA64 was the 4th exclusion (scattered-.bss) until P31 S78 #4: psyq_integrate now splits
# such a .bss into per-base NOLOAD pieces at link-prepare (psyq_bss_split, cookbook §489) and it
# links as snd12. Subsegs via gen_lib_subsegs.py; integrate window 0x3A444..0x4239C. snd1..snd12
# link; sgap* = game code + the 3 excluded stubs. (libsnd SSGM.o @0x1BD80 stays a stub in 800.)
- [0x2AC44, c, snd1] # snd block 1: 15 obj (S_INI.o..S_SK.o) vram 0x8003A444-0x8003C438
- [0x2CC38, c, sgap] # game code (vram 0x8003C438-0x8003C498)
- [0x2CC98, c, snd2] # snd block 2: 8 obj (S_STSA.o..S_SRMD.o) vram 0x8003C498-0x8003D424
@@ -107,7 +109,8 @@ segments:
- [0x2EA48, c, snd5] # snd block 5: 1 obj (PAUSE.o) vram 0x8003E248-0x8003E2E4
- [0x2EAE4, c, sgap_5] # game code (vram 0x8003E2E4-0x8003E310)
- [0x2EB10, c, snd6] # snd block 6: 12 obj (MIDIREAD.o..VM_ALOC2.o) vram 0x8003E310-0x8003FA54
- [0x30254, c, sgap_6] # game code + excluded VM_F stub (vram 0x8003FA54-0x8003FE18)
- [0x30254, c, sgap_6] # game code: func_8003FA54, 4 ins (vram 0x8003FA54-0x8003FA64)
- [0x30264, c, snd12] # snd block 12 (S78 #4): VM_F.o (237 ins, exact tile; .bss split ×2 at link-prepare, §489) vram 0x8003FA64-0x8003FE18 — carved off sgap_6
- [0x30618, c, snd7] # snd block 7: 5 obj (S_SNV.o..VM_N2P.o) vram 0x8003FE18-0x800403A4
- [0x30BA4, c, snd10] # snd block 10 (S78): VM_NO1.o (305 ins, exact tile) vram 0x800403A4-0x80040868 — was 'sgap_7' game code (§485)
- [0x31068, c, snd8] # snd block 8: 1 obj (VM_NOWOF.o) vram 0x80040868-0x80040938
@@ -124,13 +127,16 @@ segments:
- [0x33888, c, libcd1] # libcd block 1 -> src/libcd1.c (vram 0x80043088-0x80046980, 11 objs)
- [0x37180, c, gap] # non-libcd gap -> src/gap.c (vram 0x80046980-0x800469CC, stub)
- [0x371CC, c, libcd2] # libcd block 2 -> src/libcd2.c (vram 0x800469CC-0x8004787C, 7 objs)
# PsyQ libgs (Phase 7 Task #9, FULL integration): 31 libgs objects in 6 contiguous blocks
# across vram 0x80051804-0x80057928, separated by 5 non-libgs gaps. psyq_integrate swaps each
# block stub's build/src/libgsN.o(.text) for the real objects + NOLOAD data (same mechanism as
# libcd). The 5 gaps stay asm stubs (gsgapN): 80 / 48 / 1536[GS_001] / 48 / 304 B. GS_001 is
# EXCLUDED (scattered-.bss hard case, cookbook §9.1) and stays a stub (gsgap3). GS_106 (block 4)
# is an 8-ins object that only anchors uniquely within the libgs window, so make extract passes
# 0x80051804 0x80057928 to psyq_integrate. Object lists + disambiguation: tools/make_libgs.sh.
# PsyQ libgs (Phase 7 Task #9, FULL integration; S78 #3/#4): 34 libgs objects in 8 blocks
# across vram 0x8005080C-0x80057928. psyq_integrate swaps each block stub's
# build/src/libgsN.o(.text) for the real objects + NOLOAD data (same mechanism as libcd).
# The 4 remaining gaps (gsgap1/2/4/5: 80 / 48 / 48 / 304 B) are libgte objects, wired as
# libgte27-30 (S78 #3). The 1536 B gap gsgap3 was GS_001, EXCLUDED from Phase 7 to P31 S78 as the
# scattered-.bss "hard case" (cookbook §9.1; six bases): psyq_integrate now splits such a .bss
# into per-base NOLOAD pieces at link-prepare (psyq_bss_split, cookbook §489), so it links as
# libgs8 (S78 #4). GS_106 (block 4) is an 8-ins object that only anchors uniquely within the
# libgs window, so make extract passes 0x8005080C 0x80057928 to psyq_integrate. Object lists +
# disambiguation: tools/make_libgs.sh.
# PsyQ libgte (Phase 8): GTE matrix/vector math, 53 objects in 22 blocks across the old 800b region,
# interleaved with game code (800b/800b_2..800b_7). Subseg lines generated by gen_lib_subsegs.py
# (section-size-correct boundaries). integrate window 0x4787C..0x51804. The 5 libgs-gap libgte
@@ -167,26 +173,29 @@ segments:
- [0x425C8, c, libgs2] # libgs block 2: 2D_LIN0/COM1/SP0 -> src/libgs2.c (vram 0x80051DC8-0x80052430, 3 objs)
- [0x42C30, c, libgte28] # libgte block 28 (S78): MTX_07.o (12 ins, exact tile) vram 0x80052430-0x80052460 — was gsgap2
- [0x42C60, c, libgs3] # libgs block 3: 2D_SP1 -> src/libgs3.c (vram 0x80052460-0x800525DC, 1 obj)
- [0x42DDC, c, gsgap3] # gap 1536 B: GS_001 excluded -> src/gsgap3.c (vram 0x800525DC-0x80052BDC)
- [0x42DDC, c, libgs8] # libgs block 8 (S78 #4): GS_001.o (384 ins, exact tile; .bss split ×6 at link-prepare, §489) -> src/libgs8.c (vram 0x800525DC-0x80052BDC) — was the 'gsgap3' stub (hand-matched as game C)
- [0x433DC, c, libgs4] # libgs block 4: GS_002/003/MATRIX/103/104/105/106/107 -> src/libgs4.c (vram 0x80052BDC-0x800538BC, 8 objs)
- [0x440BC, c, libgte29] # libgte block 29 (S78): MTX_11.o (12 ins, exact tile) vram 0x800538BC-0x800538EC — was gsgap4
- [0x440EC, c, libgs5] # libgs block 5: GS_108/109 -> src/libgs5.c (vram 0x800538EC-0x800539C8, 2 objs)
- [0x441C8, c, libgte30] # libgte block 30 (S78): REG03.o + REG11.o (76 ins, exact tile) vram 0x800539C8-0x80053AF8 — was gsgap5
- [0x442F8, c, libgs6] # libgs block 6 (16 objs) -> src/libgs6.c (vram 0x80053AF8-0x80057928)
- [0x48128, c, 800b2] # -O2 game code (post-libgs, pre-libgpu) -> src/800b2.c (vram 0x80057928-0x80058890)
# PsyQ libgpu (Phase 8): EXT/PRIM linked; SYS.o EXCLUDED — scattered-.bss commons (cookbook §9.1,
# the GS_001 class: SYS references .bss by section+offset but the original linker scattered the
# commons across 0x80078xxx/0x800c5xxx, so no single NOLOAD base reproduces it). SYS stays a stub
# in 800c. Curated dir .run/obj40/libgpu_used = {EXT,PRIM}.
# ⚠ P31 S77 — "no single base" is TRUE; "therefore unlinkable" is NOT (cookbook §484).
# tools/psyq_bss_probe.py derives SYS.o's bases from the bytes: exactly TWO, and their .bss
# offset ranges are DISJOINT (0x0000-0x0044 @ 0x80078830, 0x0148-0x0150 @ 0x800c53cc), so a
# split at 0x148 places each half NOLOAD. Only .text references .bss (.data: zero), so the
# split is complete. GS_001.o is the genuine wall (5 interleaved bases); 2D_BG0.o and VM_NO1.o
# have NO .bss at all and are excluded for some other, unrecorded reason.
# src/800c.c is 100% SYS.o — its span is exactly the object's .text size, not "game code".
- [0x49090, c, libgpu] # libgpu block -> src/libgpu.c (vram 0x80058890-0x80059234, EXT+PRIM)
- [0x49A34, c, 800c] # -O2 game code (incl. excluded libgpu SYS stub) -> src/800c.c (vram 0x80059234-0x8005C2C8)
# PsyQ libgpu (Phase 8; S78 #4): EXT/PRIM linked as `libgpu`; SYS.o (3109 ins) was EXCLUDED from
# Phase 8 to P31 S78 as scattered-.bss commons (cookbook §9.1, the GS_001 class: SYS references
# .bss by section+offset but the original linker scattered the commons across 0x80078xxx/
# 0x800c5xxx, so no single NOLOAD base reproduces it) and sat in `800c` as 56 hand-matched SDK
# functions + 62 verbatim frags — 100% SYS.o, the span IS the object's .text size.
# P31 S77 (cookbook §484): "no single base" is TRUE; "therefore unlinkable" is NOT —
# tools/psyq_bss_probe.py derived exactly TWO bases from the bytes with DISJOINT offset ranges
# (0x0000-0x0044 @ 0x80078830, 0x0148-0x0150 @ 0x800c53cc).
# P31 S78 #4 (cookbook §489): psyq_integrate now SPLITS such a section into per-base NOLOAD
# pieces at link-prepare (tools/psyq_bss_split.py: .bss [0,0x144) @0x80078830 + .bss2 = `_que`
# @0x800C5510), so SYS.o links byte-identical as `libgpu2`. No curated dir any more: LIBGPU_ELF is
# the raw .run/obj40/libgpu (psyq_identify drops the 8 objects the EXE does not link). The same
# split takes GS_001.o (libgs8, six pieces) and VM_F.o (snd12). 2D_BG0.o / VM_NO1.o have NO .bss
# and were wired in S78 #3.
- [0x49090, c, libgpu] # libgpu block 1 -> src/libgpu.c (vram 0x80058890-0x80059234, EXT+PRIM)
- [0x49A34, c, libgpu2] # libgpu block 2 (S78 #4): SYS.o (3109 ins, exact tile; .bss split ×2 at link-prepare, §489) -> src/libgpu2.c (vram 0x80059234-0x8005C2C8) — was '800c'
# PsyQ libc2 (Phase 8): C stdlib. Main block = 16 objs (BZERO/MEMCPY/STRCMP/PRINTF/PRNT[jtbl ok]/
# …/SETJMP); STRCAT.o is a 2nd block at 0x80061E90 (start of the old 800c2). integrate libc2_1,libc2_2.
- [0x4CAC8, c, libc2_1] # libc2 main block (16 objs) -> src/libc2_1.c (vram 0x8005C2C8-0x8005CE18);
config/verbatim_manifest.json
+47 -2051
View File
File diff suppressed because it is too large Load Diff
docs/SETUP.md
+14 -3
View File
@@ -729,14 +729,15 @@ Every script under `tools/` (plus the two report make-targets), grouped by purpo
| **PsyQ library linking** (cookbook §8/§9) | `tools/psyq_lib_split.py` | Split a PsyQ `.LIB` into per-object members. |
| | `tools/psyq_build_libs.sh` | Build the PsyQ libs from split members. |
| | `tools/psyq_identify.py` | Identify which SDK objects a region's functions belong to. |
| | `tools/psyq_link.py` | Link identified PsyQ objects into the build. |
| | `tools/psyq_link.py` | Link identified PsyQ objects into the build. **P31 S78 #4:** `link_object` first runs `psyq_bss_split.prepare_object` (the same prepare step the region verify and the build use), so its PASS/FAIL is the build's verdict for scattered-`.bss` objects; links with `--no-check-sections` like the build. |
| | `tools/psyq_bss_split.py` | **(P31 S78 #4, cookbook §489)** Splits a PsyQ object's scattered `.bss` into per-base NOBITS pieces (`.bss`, `.bss2`, …) — a pure-Python ELF32 REL rewrite: bases derived from the game bytes per HI16/LO16 pair, runs in offset order, cuts snapped to symbol starts, relocs retargeted with the addend rewritten in place, self-checked. Refuses (loud, R43) a sized symbol straddling a cut or a HI16 shared across pieces. Runs automatically inside `psyq_link` / `psyq_link_region` / `psyq_integrate` (`prepare_object`); the CLI reports a plan (`--vram --exe --vram-base`, `-o` to write). Takes SYS.o (2 pieces), VM_F.o (2), GS_001.o (6). |
| | `tools/psyq_link_lib.py` | Per-library link driver. |
| | `tools/psyq_link_region.py` | Link a specific address region from PsyQ libs. |
| | `tools/progress.py` (weighted main) | **(P31 S78)** `MAIN game-code weighted` now excludes LINKED subsegs LIVE (`_main_linked_ranges`: Makefile `psyq_integrate` stub lists → yaml ranges, R33); the 2026-08-05 sig had carried all linked-SDK instructions, under-reporting main by ~32 points (59.8% → 91.8%). |
| | `tools/psyq_integrate.py` | Integrate linked PsyQ results back into the source tree. **P31 S78:** `--yaml <splat yaml>` (every main call passes `$(main_SPLAT_YAML)`) maps stub↔objects by SUBSEG RANGE with an exact-tiling check and PRINTS the located-but-unwired objects (`~~ N located object(s) / M ins OUTSIDE the stub subsegs`) — the completion contract's SDK-residue line; and a library object's DEFINED symbol whose recovered address the curated symbol file names differently is `--redefine-sym`'d to the curated name (R15; e.g. libapi 4.0 `A66.o` `firstfile`→`firstfile2`). Without `--yaml` the old contiguity mapping runs (overlay-free libraries only). |
| | `tools/make_libgs.sh` | Build/link the `libgs` block (cookbook §9). |
| | `tools/gen_lib_subsegs.py` | **(Phase 8)** Generate splat subseg lines + integrate stub list for a multi-block library (section-size-correct boundaries; cookbook §9.6). |
| | `tools/make_snd_used.py` | **(Phase 8)** Build the combined libspu+libsnd curated dir (alias dedup by byte-match, scattered-`.bss` exclusions; §9.6). |
| | `tools/make_snd_used.py` | **(Phase 8)** Build the combined libspu+libsnd curated dir (alias dedup by byte-match; 3 address exclusions since S78 #4 — VM_F rejoined via the `.bss` split; §9.6). |
| | `tools/make_apicard_used.py` | **(Phase 8)** Build the combined libapi+libcard curated dir (§9.6). |
| | `tools/ld_interleave.py` | Interleave linker inputs to match the original section ordering. Three forms: `--front/--tail` (the original data->rodata->data sandwich), **`--order`** (P31 S72 — an address-ordered leaf list, needed once a binary owns SEVERAL `.rodata` carves: a `*.data.o` leaf contributes its `(.data)`, a code-object leaf its `(.rodata)`; main's island is a 7-piece sandwich `--front/--tail` cannot express), and **`--pre`** (P31 S74, cookbook §440 — a piece that must land BEFORE the text). `--pre` exists for the **resident**, which opens with `- [0x0, rodata, hdr]`, a 1-word `.rodata` header ahead of its code: every `--order` piece is emitted after TEXT_START, so `hdr.rodata.o(.rodata)` would fall into the unchecked `empties` bucket, be parked after the text, and move every byte in the binary. |
| | `tools/jtbl_rodata_pads.py` | **(Phase 29, cookbook §8e)** Post-maspsx filter for multi-table `.rodata` carve spans: REPLACES each cc1 `.align 3` (which is section-relative and would mis-pad a merged/4-mod-8 span) with the ORIGINAL's exact pad bytes per the object's `JTBL_PADS` spec (written by `jtbl_carve` into `config/overlays.mk`; armed via `$(if $(JTBL_PADS),…)` in the Makefile `build/src/%.o` recipe). Fail-loud on table-count drift / non-`.align 3` / non-jtbl rodata content. Unset var ⇒ pipeline byte-identical. |
@@ -983,11 +984,21 @@ fills fast). Nothing is leaking — but the host does not get the memory back on
|---|---|---|
| `tools/gate_main_parallel.py` **(NEW)** | runs the REAL `gate_main` inside N git worktrees to discover which drafts pass, then hands the union to ONE authoritative `gate_main` in the real tree. Workers discover; only the final serial pass banks | a main slate large enough that serial bisection hurts. **Measured: one gate cycle is 16 s**, so MAX_STEPS=24 is ~6.4 min serial and ~90 s across four workers. Run `--negative-control` once per environment first |
| `tools/permuter_sweep.py` **(NEW)** | selects a wave's NEARs the permuter can actually search (small residual + SCHEDULE/DELAY-SLOT/REGALLOC class) and runs `permuter_ils` on them; journals are scoped to the wave so an older run's row cannot win on a bare fn name (R48) | after every wave gate — agents are briefed to STOP at permuter-class residuals, so these arrive unattempted and cost no tokens. Measured yield ~3 in 8; there is NO validated predictor of which 3 |
| `tools/psyq_bss_probe.py` **(NEW)** | derives a PsyQ object's `.bss` bases FROM THE BYTES (base = resolved − addend over each HI16/LO16 pair) and reports whether the offset ranges are DISJOINT, i.e. whether the section can be split and placed | before accepting any "scattered .bss, unlinkable" exclusion. Measured: SYS.o splits at 0x148 (2 bases), GS_001.o does not (5 interleaved), 2D_BG0.o/VM_NO1.o have no `.bss` at all. Locates the object itself by masked search, so a wrong `--vram` cannot fake a clean answer (cookbook §484) |
| `tools/psyq_bss_probe.py` **(NEW)** | derives a PsyQ object's `.bss` bases FROM THE BYTES (base = resolved − addend over each HI16/LO16 pair) and reports whether the section can be cut into single-base pieces | before accepting any "scattered .bss, unlinkable" exclusion. Measured S76: SYS.o splits at 0x148 (2 bases), 2D_BG0.o/VM_NO1.o have no `.bss` at all. **S78 #4 correction:** its "GS_001.o: 5 interleaved bases, NOT splittable" grouped by BASE; grouped by RUN in offset order GS_001 is six pieces, every cut on a symbol start — it now reports over `psyq_bss_split.analyze` (one implementation, R33). Locates the object itself by masked search, so a wrong `--vram` cannot fake a clean answer (cookbook §484/§489) |
| `make sig-main-oracle` **(NEW)** | main's INDEPENDENT second boundary oracle (roadmap contract §1.3): signs the ORIGINAL EXE bytes over 28 game-code ranges derived from the splat yaml's SEGMENT TYPES, never its function boundaries | run by `tools-health`; needed before any 100% claim on main. Distinct from `make sig-main`, which is splat-SEEDED on purpose — `corpus.ORACLE_SIG` keeps the audit off that one. Result: 986 fns, 0 phantom / 0 truncated / 1 pad-tail (docs/second-oracle.md) |
| `tools/recover_route.py` **(NEW)** | reads a gate's DROP ledger and names the tool that actually applies — SELF / CALLEE / DATA, each with its ordered ladder; refuses a verbatim draft up front. Wired into `gate_main`'s dropped report | after any gate that drops a draft. Replaces the single hardcoded chain gate_main used to print for every drop, which was the SELF chain and measured RED on a callee (S69) and inert on data. Negative-controlled against all 7 S77 drops whose winning tool was already known |
| `tools/sync_tu_decls.py` **(NEW)** | banks a draft the gate refuses by copying the TU's OWN `extern` line for whatever symbol the gate names, re-gating, and repeating | a draft that is byte-correct but rejected on a declaration conflict. Refuses `self_decl_tu` (use `cast_self_callers --sync-decls`) and refuses a NEAR up front, since syncing declarations makes a body COMPILE, never MATCH |
### Tools added / changed 2026-09-04 (S79, task #4) — R21 record
| tool | what it does | when you need it |
|---|---|---|
| `tools/psyq_bss_split.py` **(NEW)** | rewrites a PsyQ ELF object whose `.bss` is referenced through the SECTION symbol at several bases into per-base NOBITS pieces (`.bss`/`.bss2`/…): bases from the game bytes, runs in offset order, cuts snapped to symbol starts, symbols moved, a LOCAL section symbol per piece inserted (REL symbol indices bumped), relocs retargeted with the addend rewritten in the instruction immediates, then self-diffed. No pyelftools — its own 60-line ELF32 reader/writer | never by hand: `psyq_link.link_object`, `psyq_link_region.build_region` and `psyq_integrate.integrate` all call `prepare_object()` before `classify()`, so every library build splits what needs splitting and passes the rest through untouched (R39: 235 placed objects, 0 refusals, exactly 3 splits). CLI = the plan report (`<obj> --vram --exe --vram-base [-o out.o]`) |
| `tools/psyq_bss_probe.py` | now a thin reporter over `psyq_bss_split.analyze` (its own pairing code deleted, R33); "SPLITTABLE" means "tiles into single-base runs at symbol boundaries", which is what the build does | as before — before believing any scattered-`.bss` exclusion |
| `tools/psyq_link.py` / `psyq_link_region.py` / `psyq_integrate.py` | the NOBITS predicate is `^\.s?bss\d*$` (was the literal pair `.bss`/`.sbss`), so split pieces are NOLOAD-placed from their own section symbols; `link_object` links with `--no-check-sections` like the build (piece extents tile the PACKED section, so unreferenced commons overlap other pieces harmlessly) | automatic |
| `LIBGPU_ELF := .run/obj40/libgpu` | the `libgpu_used` curated dir is retired (it existed only to exclude SYS.o); `psyq_identify` drops the 8 objects the EXE never links | fresh clone: `tools/psyq_build_libs.sh LIBGPU` is now the whole libgpu step |
| `tools/make_libgs.sh` (+GS_001) / `tools/make_snd_used.py` (VM_F rejoined) | curated dirs now carry the two objects the split unlocked: libgs 34 objects / 8 blocks (`libgs8`), sound 63 / 12 (`snd12`) | re-run after `psyq_build_libs.sh` on a fresh clone |
**Oracle corrections — re-read any verdict recorded before these:**
* `match_one` and `rtu_match` now route the Makefile's `REORDER_TUS` (`800c2 800c2_2 800c2_3 800c3`)
through `reorder_passthrough.py + as -O2`, the real build path. They previously modelled
docs/accelerators.md
+14
View File
@@ -663,3 +663,17 @@ frontier's own stub sum (~4,000 ins), and nobody subtracted the two. One subtrac
the denominator was carrying the 31,000 linked-SDK instructions. Accelerator: **every headline % ships
with its remainder, and the remainder is reconciled against an independently-derived list of what is
actually open.**
## S79 — "no single base" means "partition it", and partition along the producer's structure
Three SDK objects sat excluded for twenty-three phases as "scattered `.bss`, no single NOLOAD base".
The general fix (cut the section into per-base NOBITS pieces at link-prepare, ~400 lines of ELF
surgery, `tools/psyq_bss_split.py`) took one afternoon and closed the class 3/3 — including the object
the previous session's probe had certified as a genuine wall, because that probe grouped by base while
the original linker had scattered SYMBOLS. Accelerators: **(1)** a "no single X" verdict is a
partition problem, not a wall — build the partition the day the exclusion is written; **(2)** partition
along the structure the original producer used (the symbol table), and confirm every cut against an
independent oracle (here the other objects' by-name recoveries agreed on all seven cuts); **(3)** put
the transformation in the shared prepare path, not in a curated artifact, so it is re-derived from the
bytes on every build and negative-controlled over the whole placed population for free.
docs/cookbook-index.md
+5 -2
View File
@@ -2,7 +2,7 @@
> **Generated by `tools/cookbook_index.py` — do not hand-edit** (R33). Regenerate after adding a cookbook section.
>
> `docs/matching-cookbook.md` is ~716 KB / 1156 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
> `docs/matching-cookbook.md` is ~716 KB / 1157 sections. Grepping it blind is how three P30 wave-1 agents each "discovered" an idiom that was already written down. **Start here, then read the section.** A section appears under every symptom it addresses.
**How to use:** name what you SEE in the diff (a stolen delay slot, an extra `la`, a swapped register pair, a `conflicting types` error), find that symptom below, read those sections first. If nothing fits, THEN grind — and add a section when you win.
@@ -1080,7 +1080,7 @@
- **§442** — ★★★ — A RECOVERY RUNG THAT REWRITES A LEGAL CONSTRUCT INTO AN ILLEGAL ONE READS EXACTLY LIKE A CODEGEN WALL (P31 S75; `reconcile_tu`, 344 ins unblocked) <sub>L34647</sub>
- **§477** — ★★★ — THE `self_decl_tu` CLASS IS A SOLVED, MECHANICAL LANE: 16 DRAFTS, 16 BANKS (P31 S77) <sub>L35713</sub>
### build graph, splat & the harness (208)
### build graph, splat & the harness (209)
- **§4** — Flag/toolchain gotchas <sub>L190</sub>
- **Build** — mechanism — per-file opt override (splat resegmentation) <sub>L307</sub>
@@ -1290,6 +1290,7 @@
- **§485** — ★★★ — THE PLACEMENT MAP WAS PARSING A PRETTY-PRINTER: 25 PsyQ OBJECTS WERE INVISIBLE, NOT ABSENT (P31 S77) <sub>L36069</sub>
- **§487** — ★★★ — THE PSX LOADER'S PER-VERSION SIGNATURE SETS ARE A FREE PROVENANCE ORACLE: main's "WALL" BAND IS LIBPAD 4.2.1 (P31 S78) <sub>L36150</sub>
- **§488** — ★★ — THE "GAME CODE" GAPS BETWEEN LIBRARY BLOCKS WERE LIBRARY OBJECTS: 13 SUBSEGS → LINKED, EXACT-TILED, ZERO TOKENS (P31 S78) <sub>L36202</sub>
- **§489** — ★★★ — SCATTERED `.bss` IS A PARTITION PROBLEM: SPLIT THE SECTION INTO PER-BASE PIECES AT LINK-PREPARE (P31 S78 #4; closes the §9.1 exclusion class 3/3) <sub>L36242</sub>
### process, measurement & doctrine (141)
@@ -2934,6 +2935,7 @@
- **§486** — ★★★ — CARVING AN `-O0` ISLAND IN **main**: FIVE COUPLED PIECES, AND THE TWO THAT ANNOUNCE THEMSELVES (P31 S77) <sub>L36105</sub>
- **§487** — ★★★ — THE PSX LOADER'S PER-VERSION SIGNATURE SETS ARE A FREE PROVENANCE ORACLE: main's "WALL" BAND IS LIBPAD 4.2.1 (P31 S78) <sub>L36150</sub>
- **§488** — ★★ — THE "GAME CODE" GAPS BETWEEN LIBRARY BLOCKS WERE LIBRARY OBJECTS: 13 SUBSEGS → LINKED, EXACT-TILED, ZERO TOKENS (P31 S78) <sub>L36202</sub>
- **§489** — ★★★ — SCATTERED `.bss` IS A PARTITION PROBLEM: SPLIT THE SECTION INTO PER-BASE PIECES AT LINK-PREPARE (P31 S78 #4; closes the §9.1 exclusion class 3/3) <sub>L36242</sub>
---
@@ -4102,3 +4104,4 @@ Notes routinely quote that as a section id. This table resolves it. Grep bait: `
| L36105 | §486 | ★★★ — CARVING AN `-O0` ISLAND IN **main**: FIVE COUPLED PIECES, AND THE TWO THAT ANNOUNCE |
| L36150 | §487 | ★★★ — THE PSX LOADER'S PER-VERSION SIGNATURE SETS ARE A FREE PROVENANCE ORACLE: main's "WA |
| L36202 | §488 | ★★ — THE "GAME CODE" GAPS BETWEEN LIBRARY BLOCKS WERE LIBRARY OBJECTS: 13 SUBSEGS → LINKED |
| L36242 | §489 | ★★★ — SCATTERED `.bss` IS A PARTITION PROBLEM: SPLIT THE SECTION INTO PER-BASE PIECES AT L |
docs/decision-log.md
+30
View File
@@ -3111,3 +3111,33 @@ exclusion derived live (Makefile stub lists → yaml ranges), main reads **91.8%
the remainder equals the open-stub instruction sum to the instruction. Hindsight: a metric whose
denominator is a snapshot will drift the first time the thing it snapshots changes; derive it (R33),
and check it against a case whose answer you already know (the 28 stubs' size).
### S79 addendum (2026-09-04, task #4) — the "scattered-`.bss`" wall class is closed, and the probe that measured it was too strict
**Belief.** From Phase 8 to P31 S77 three SDK objects were excluded from the LINKED build as "scattered
`.bss` commons — no single NOLOAD base reproduces them": SYS.o (3,109 ins, kept as 56 hand-matched
Sony functions + 62 verbatim frags in `src/800c.c`), VM_F.o (237, hand-matched as game code in
`sgap_6`), GS_001.o (384, hand-matched as game code in `gsgap3`). S77's probe reframed two of them as
"disjoint ranges → splittable" and confirmed GS_001 as the genuine wall ("5 interleaved bases").
**What happened.** Task #4 built the split as a link-time ELF rewrite (`psyq_bss_split.py`, §489)
instead of a curated-dir artifact, and modelled the section as RUNS of one base in offset order with
cuts snapped to symbol starts — because the original linker scattered *symbols*, not offset ranges.
Under that model GS_001 is six symbol-aligned pieces, and the five cut symbols recover by name from
the other libgs objects at exactly the piece bases. All three link byte-identical; main is `143dbb89`
with and without the SDK objects; 235 placed objects across nine curated dirs pass through with zero
refusals.
**Why the probe was wrong.** It grouped references BY BASE and asked whether the per-base offset ranges
were disjoint. A common that the linker placed between two others (PSDBASEY at +0x38 sits between
PSDBASEX at +0x28 and CLIP2 at +0x30 in the packed section, but in the game X and Y are adjacent)
makes two ranges interleave while every run is still single-base. The right unit was the run; the
right tie-breaker was the symbol table.
**Hindsight.** The worklist's own italic note from Phase 8 said "escalate to a Max general fix (split
each object's `.bss` into per-common NOLOAD sections)". That fix is ~400 lines and one afternoon; it
waited twenty-three phases because three exclusions never looked worth a general mechanism, and the
probe's stricter test then ratified one of them as a wall. General form: when a tool says "no single
X reproduces it", the next question is "can X be partitioned", and the partition should follow the
structure the ORIGINAL producer used (here: symbols), not the structure the measurement happened to
group by.
docs/difficulty.md
+10 -10
View File
@@ -1,8 +1,8 @@
# Unmatched difficulty inventory (generated by tools/difficulty.py — harvest queue)
unmatched functions : 1069
trivial (<=5 ins) : 228
non-jtbl leaves : 734 (best harvest targets)
unmatched functions : 1179
trivial (<=5 ins) : 238
non-jtbl leaves : 797 (best harvest targets)
jump-table funcs : 36 (deferred — need the rodata-island workflow, Task 2')
## Easiest 120 unmatched (score asc) — the work queue
@@ -30,6 +30,7 @@ jump-table funcs : 36 (deferred — need the rodata-island workflow, Task
| 1 | PRNT_OBJ_2A0 | 1 | 0 | 0 | - | Y |
| 1 | PRNT_OBJ_348 | 1 | 0 | 0 | - | Y |
| 1 | PRNT_OBJ_3D0 | 1 | 0 | 0 | - | Y |
| 1 | SYS_OBJ_1C28 | 1 | 0 | 0 | - | Y |
| 1 | S_SR_OBJ_B0 | 1 | 0 | 0 | - | Y |
| 2 | INTR_DMA_OBJ_274 | 2 | 0 | 0 | - | Y |
| 2 | LIBMCRD_OBJ_1A60 | 2 | 0 | 0 | - | Y |
@@ -39,6 +40,7 @@ jump-table funcs : 36 (deferred — need the rodata-island workflow, Task
| 2 | PRIM_OBJ_228 | 2 | 0 | 0 | - | Y |
| 2 | SR_SV_OBJ_224 | 2 | 0 | 0 | - | Y |
| 2 | SR_SV_OBJ_2F4 | 2 | 0 | 0 | - | Y |
| 2 | SYS_OBJ_191C | 2 | 0 | 0 | - | Y |
| 2 | S_ITC_OBJ_88 | 2 | 0 | 0 | - | Y |
| 2 | S_SCA_OBJ_15C | 2 | 0 | 0 | - | Y |
| 2 | S_SCA_OBJ_94 | 2 | 0 | 0 | - | Y |
@@ -49,6 +51,7 @@ jump-table funcs : 36 (deferred — need the rodata-island workflow, Task
| 3 | MIDIREAD_OBJ_6C0 | 3 | 0 | 0 | - | Y |
| 3 | PRESET2_OBJ_A88 | 3 | 0 | 0 | - | Y |
| 3 | PRNT_OBJ_380 | 3 | 0 | 0 | - | Y |
| 3 | SYS_OBJ_1B78 | 3 | 0 | 0 | - | Y |
| 3 | SetDQA | 3 | 0 | 0 | - | Y |
| 3 | SetDQB | 3 | 0 | 0 | - | Y |
| 3 | SetData32 | 3 | 0 | 0 | - | Y |
@@ -59,6 +62,8 @@ jump-table funcs : 36 (deferred — need the rodata-island workflow, Task
| 4 | CdLastCom | 4 | 0 | 0 | - | Y |
| 4 | GEO_00_OBJ_2C | 4 | 0 | 0 | - | Y |
| 4 | GS_108_OBJ_7C | 4 | 0 | 0 | - | Y |
| 4 | GetGraphDebug | 4 | 0 | 0 | - | Y |
| 4 | GetGraphType | 4 | 0 | 0 | - | Y |
| 4 | GsGetActiveBuff | 4 | 0 | 0 | - | Y |
| 4 | LIBMCRD_OBJ_15C | 4 | 0 | 0 | - | Y |
| 4 | LIBMCRD_OBJ_1944 | 4 | 0 | 0 | - | Y |
@@ -71,6 +76,8 @@ jump-table funcs : 36 (deferred — need the rodata-island workflow, Task
| 4 | SSSTART_OBJ_240 | 2 | 0 | 1 | - | - |
| 4 | SSSTART_OBJ_248 | 2 | 0 | 1 | - | - |
| 4 | SSSTART_OBJ_328 | 4 | 0 | 0 | - | Y |
| 4 | SYS_OBJ_1AF0 | 4 | 0 | 0 | - | Y |
| 4 | SYS_OBJ_2F7C | 4 | 0 | 0 | - | Y |
| 4 | S_RRWA_OBJ_40 | 4 | 0 | 0 | - | Y |
| 4 | S_SAV_OBJ_1E8 | 4 | 0 | 0 | - | Y |
| 4 | S_SAV_OBJ_1F8 | 4 | 0 | 0 | - | Y |
@@ -121,10 +128,3 @@ jump-table funcs : 36 (deferred — need the rodata-island workflow, Task
| 5 | PRNT_OBJ_F4 | 2 | 1 | 0 | - | Y |
| 5 | RATAN_OBJ_148 | 5 | 0 | 0 | - | Y |
| 5 | SPU_OBJ_848 | 5 | 0 | 0 | - | Y |
| 5 | SR_SV_OBJ_1EC | 2 | 1 | 0 | - | Y |
| 5 | SR_SV_OBJ_1F4 | 2 | 1 | 0 | - | Y |
| 5 | SR_SV_OBJ_1FC | 2 | 1 | 0 | - | Y |
| 5 | SR_SV_OBJ_204 | 2 | 1 | 0 | - | Y |
| 5 | SR_SV_OBJ_20C | 2 | 1 | 0 | - | Y |
| 5 | SR_SV_OBJ_214 | 2 | 1 | 0 | - | Y |
| 5 | SR_SV_OBJ_21C | 2 | 1 | 0 | - | Y |
docs/matching-cookbook.md
+59
View File
@@ -36238,3 +36238,62 @@ the fresh-clone fallback: move `.run/obj40` aside, **re-extract**, build, restor
deleted, REAL −4 (inline-asm wrappers), 0 agent tokens. Remaining LINKED residue: `SYS.o` (3,109),
`VM_F` (237), the libpad/libapi band pieces (task #5), `SSGM.o` (8 ins amid matched C), and the
GS_001 / S_R / S_GRMDT scattered-`.bss` genuine walls.
#### §489 ★★★ — SCATTERED `.bss` IS A PARTITION PROBLEM: SPLIT THE SECTION INTO PER-BASE PIECES AT LINK-PREPARE (P31 S78 #4; closes the §9.1 exclusion class 3/3)
**The wall (§9.1, Phase 8).** psyq-obj-parser packs an object's common-style globals into ONE `.bss` with
sequential offsets; the original linker allocated those commons individually, so the game has them at
unrelated addresses. A common referenced BY NAME is weakened and `--defsym`'d (§9.2). But the compiler
references the object's own statics through the `.bss` SECTION SYMBOL + offset — no name to defsym — and
one section can be NOLOAD-placed at only one base. SYS.o (2 bases), VM_F.o (2) and GS_001.o (6) were
excluded on that reason for twenty-three phases; §484 (S77) asked whether the bases' offset ranges were
disjoint and said yes for two of them, "no, interleaved" for GS_001.
**The model that is actually right: RUNS, cut at SYMBOL starts.** Walk the section-symbol references in
offset order; each maximal run with one base is a piece. The cut between two runs snaps to the largest
symbol start between them, because the linker scattered *symbols*: SYS.o's second run begins inside `_que`
(+0x148) and the piece begins at `_que` (+0x144) — and `_que`, recovered BY NAME from SYS.o's own four
named references, is 0x800C5510 = base2 + 0x144; VM_F.o's second run begins at `_svm_sreg_buf` (+0x508),
which 62 other sound objects recover to 0x800B9B58 = base2 + 0x508; GS_001.o's five cuts land on
PSDBASEX / CLIP2 / PSDBASEY / POSITION / GsDRAWENV, all five recovered by the other libgs objects at exactly
the piece bases. Two unrelated oracles agree on all seven cuts (R34). §484's "interleaved" verdict came
from grouping by BASE: PSDBASEY (+0x38) sits between PSDBASEX (+0x28) and CLIP2 (+0x30) in the packed
section while X and Y are adjacent in the game — two base-ranges interleave, every run is single-base.
Refuse (R43) only what the run model cannot tile: a SIZED symbol straddling a cut (one common at two
bases), a HI16 whose LO16s need different pieces or high halves, an orphan LO16, a far-out addend.
**The rewrite** (`tools/psyq_bss_split.py`, its own 60-line ELF32 REL reader/writer — no pyelftools):
new NOBITS sections `.bss2…` sized [s_k, s_k+1); the original shrunk to [0, s_2); symbols at/after a cut
moved (value −= piece start); one LOCAL section symbol per piece inserted with the existing section
symbols and every later symbol index in every REL entry bumped; each reference retargeted to its piece's
symbol with the addend rewritten in place — HI16/LO16 immediates in `.text` (`hi' = (A'+0x8000)>>16`,
`lo' = A' & 0xFFFF`, `A' = A − s_k`; a shared `lui` is one cluster and all its LO16s must agree), or the
R_MIPS_32 word in data. `classify()` then recovers one base per piece from the piece's own section symbol
(base = resolved − addend, tautologically the game's address) and NOLOAD-places each; the NOBITS predicate
in `psyq_link` / `psyq_link_region` / `psyq_integrate` is `^\.s?bss\d*$`. Self-check: the code/data
sections differ from the original at exactly the retargeted sites whose value changed (R37, the tool
diffs its own artifact).
**Where it runs, and why there.** Not in the curated dirs — inside the ONE prepare step that
`psyq_link.link_object` (per-object verify), `psyq_link_region.build_region` (region verify) and
`psyq_integrate.integrate` (the build) share (`prepare_object()`, before `classify()`). The split is
re-derived from the bytes on every build — no recorded offset to go stale (R51) — and the same call is the
negative control: 235 placed objects across the 9 curated dirs, 0 refusals, exactly 3 splits (R39). The
first build DID refuse: a libcd object references `.bss + size` (an end-of-buffer pointer) and the strict
`addend < size` bound fired on an object one base already served. Law that fell out: **problems found
while classifying references are fatal only when a split is actually needed** — an object that passed
before this tool existed must pass through it untouched.
**Two things the extents are NOT.** A piece's extent tiles the PACKED section, so an unreferenced common
inside piece k may in truth live inside piece j's game range (GS_001's PSDBASEX/PSDBASEY: adjacent in the
game, 16 bytes apart in the object) — NOLOAD pieces therefore overlap, harmlessly (zero bytes), and the
verifiers link with `--no-check-sections` exactly like the build. And a named common whose recovered
address disagrees with its piece is still handled by §9.2's weaken+defsym — the split only serves the
section-symbol references; the two mechanisms compose.
**Yield.** `800c` (56 hand-matched Sony functions + 62 verbatim frags, 100% SYS.o) → `libgpu2`;
`gsgap3` (hand-matched as game C) → `libgs8`; `_SsVmFlush` out of `sgap_6` → `snd12`. `libgpu_used`
retired (`LIBGPU_ELF` = the raw dir; identify drops the 8 objects the EXE never links). Main `143dbb89`
with and without the SDK objects. The class that remains in the sound region — `S_R`/`S_W`, `S_GRMDT*` —
has NO `.bss` of its own (cross-object commons at a minority address): a different wall, not this one.
docs/progress.fleet.md
+8 -8
View File
@@ -4,21 +4,21 @@
# cross-binary collapsible-byte leverage: docs/duplicates.cross.md.
# THREE progress metrics (all matter — see the labels):
FLEET fn-count byte-ident: 363098 / 363149 = 99.99% (REAL+LINKED+empties; FUNCTION-count, ×134-inflated — one crack counts per overlay)
FLEET instr-weighted : 13485298 / 13492892 = 99.9% (shipped .text across main + resident + 211 overlays; the decomp.dev-DISPLAY number)
FLEET distinct-code(uniq): 5813544 / 5820999 = 99.9% (90909/90929 unique fns; the DISTINCT-RE number)
MAIN game-code weighted : 44562 / 48537 = 91.8% (INCLUDED in the fleet numbers above since 2026-07-22 — roadmap §1 metrics contract; LINKED-excluding Ghidra sig dated 2026-08-05; boundaries INDEPENDENTLY VERIFIED since P31 S77 — `make sig-main-oracle` + `make audit-corpus`: 0 phantom, 0 truncated, 1 explained pad-tail)
FLEET fn-count byte-ident: 363105 / 363156 = 99.99% (REAL+LINKED+empties; FUNCTION-count, ×134-inflated — one crack counts per overlay)
FLEET instr-weighted : 13481631 / 13489225 = 99.9% (shipped .text across main + resident + 211 overlays; the decomp.dev-DISPLAY number)
FLEET distinct-code(uniq): 5809877 / 5817332 = 99.9% (90909/90929 unique fns; the DISTINCT-RE number)
MAIN game-code weighted : 40895 / 44870 = 91.1% (INCLUDED in the fleet numbers above since 2026-07-22 — roadmap §1 metrics contract; LINKED-excluding Ghidra sig dated 2026-08-05; boundaries INDEPENDENTLY VERIFIED since P31 S77 — `make sig-main-oracle` + `make audit-corpus`: 0 phantom, 0 truncated, 1 explained pad-tail)
(fleet EXCLUDING main, for continuity with pre-2026-07-22 readings: 13440736 / 13444355 = 100.0%)
FLEET REAL substantive : 360755 (of which dedup-shared 255632 via 2220 groups / 255708 instances)
FLEET LINKED PsyQ objs : 1040
FLEET REAL substantive : 360708 (of which dedup-shared 255632 via 2220 groups / 255708 instances)
FLEET LINKED PsyQ objs : 1150
FLEET NON_MATCHING : 0 (0 in any default build — G4)
FLEET INCLUDE_ASM stubs : 51
FLEET matchable : 363149
FLEET matchable : 363156
| binary | REAL | shared | LINKED | byte-ident | matchable | byte-ident % |
|---|---:|---:|---:|---:|---:|---:|
| main | 886 | 2 | 1040 | 2054 | 2083 | 98.6% |
| main | 839 | 2 | 1150 | 2061 | 2090 | 98.6% |
| resident | 141 | 0 | 0 | 143 | 145 | 98.6% |
| md_MAIN_001 | 11 | 0 | 0 | 11 | 11 | 100.0% |
| md_MAIN_003 | 53 | 0 | 0 | 59 | 64 | 92.2% |
docs/progress.md
+8 -8
View File
File diff suppressed because one or more lines are too long
docs/psyq-worklist.md
+33 -7
View File
@@ -13,12 +13,12 @@
| Order | Library | Placed / total | Region(s) | ~banked B | Difficulty notes |
|---|---|---|---|---|---|
| 1 | **libgpu** | 3 / 12 | 800b2 (`0x80058890`) | ~14.9k | ✅ **DONE (T4): EXT+PRIM linked** (curated `libgpu_used`). **SYS.o EXCLUDED** — scattered-`.bss` (§9.1, GS_001 class; stays a stub in 800c). |
| 1 | **libgpu** | 3 / 12 | 800b2 (`0x80058890`) | ~14.9k | ✅ **DONE (T4 + S78 #4): EXT+PRIM+SYS linked** (raw `.run/obj40/libgpu`, no curated dir). SYS.o was EXCLUDED Phase 8→P31 S78 as scattered-`.bss` (§9.1); linked since S78 #4 via the link-prepare `.bss` split (cookbook §489) as `libgpu2`. |
| 2 | **libetc** | 5 / 7 | 800 tail (`0x8004239C`) | ~3.3k | VSYNC/INTR/INTR_VB/INTR_DMA/VMODE **contiguous**, ends exactly at libcd1 (`0x80043088`). Clean. |
| 3 | **libmcrd** | 2 / 2 | 800b2 (`0x8005FC68`,`0x80062888`) | ~9.0k | LIBMCRD.o (2186 ins, huge, holds the 55 `LIBMCRD_OBJ_*`) + USERFUNC.o — **2 blocks**. `_card_*` h_norm dups collapse inside LIBMCRD. |
| 4 | **libc2** | 17 / 46 | 800b2 (`0x8005C2C8`–`0x5CD98` + STRCAT `0x80061E90`) | ~3.1k | C stdlib (BZERO/MEMCPY/STRCMP/PRINTF/PRNT…); contiguous run + 1 outlier. **PRNT.o (418 ins) has an internal jtbl** — verify NOLOAD `.rodata` placement (the `PRNT_OBJ_24C` rodata note). |
| 5+6 | **libapi+libcard** | 22 (800c2) | 800c2 (`0x80061F38`–`0x80062888`) | ~0.7k | ✅ **DONE (combined apicard region): 22 objs / 4 blocks / +24 fns** (`tools/make_apicard_used.py`, C112 dedup, 0 exclusions). **libapi's ~22 objects in the 800c3 region (`0x5CE18`..) DEFERRED** — lowest value, separate resegmentation. |
| 7+8 | **libspu+libsnd** | 60 / (38+32) | 800 sound (`0x8003A444`–`0x8004239C`) | ~24k | ✅ **DONE (combined region): 60 objs / 9 blocks / +225 fns.** The two libs interleave, so linked as ONE region (`tools/make_snd_used.py` dedups + excludes 4 addresses; `gen_lib_subsegs.py` + window). **Excluded:** S_R/S_W `0x3C438`, S_GRMDT* `0x3D424`, S_IH/UT_RON `0x3D94C` (false-positive, inside SSSTART), VM_F `0x3FA64` (237 ins) — all scattered-`.bss`/false-pos, stay stubs. SSGM.o `0x1BD80` deferred (isolated in matched-C, 8 ins). |
| 7+8 | **libspu+libsnd** | 63 / (38+32) | 800 sound (`0x8003A444`–`0x8004239C`) | ~24k | ✅ **DONE (combined region; S78 #3/#4): 63 objs / 12 blocks.** The two libs interleave, so linked as ONE region (`tools/make_snd_used.py` dedups + excludes 3 addresses; `gen_lib_subsegs.py` + window). **Excluded:** S_R/S_W `0x3C438`, S_GRMDT* `0x3D424` (cross-object commons — these objects have NO `.bss`, so the S78 split cannot apply), S_IH/UT_RON `0x3D94C` (false-positive, inside SSSTART). VM_F `0x3FA64` (237 ins) rejoined in S78 #4 (`snd12`, `.bss` split at `_svm_sreg_buf`). SSGM.o `0x1BD80` deferred (isolated in matched-C, 8 ins). |
| 9 | **libgte** | 58 / 381 | 800b (`0x8004787C`–`0x5082C`) + **libgs gaps** | ~48k | ✅ **DONE (T11): 53 objs / 22 blocks linked** in 800b (subsegs via `gen_lib_subsegs.py`; integrate window 0x4787C..0x51804). **5 libgs-gap objects DEFERRED** (MTX_05/07/11/REG03/REG11 → gsgap1/2/4/5 stay stubs; gsgap2≠MTX_07 exactly so needs sub-split). |
**SKIP — zero footprint (recorded, not linked by the EXE):** `libmath` 0/48, `libc` 0/56, `libsn` 0/51 (2 tiny ambiguous). BFM links **libc2**, not libc; no libmath/libsn. (Also unbuilt/no-footprint: libcomb, libds, libgun, libsio, libtap, libpress — never converted, no symbols.)
@@ -29,7 +29,7 @@
800 (0x800123F0–0x80043088) game code + libsnd(SSGM @1BD80; 3D454–42374) + libspu(3A444–422E8) + libetc(4239C–43088, tail)
libcd (0x80043088–0x8004787C) DONE
800b (0x8004787C–0x80051804) libgte(4787C–5082C, multi-block) + game-code tail
libgs (0x80051804–0x80057928) DONE; gaps gsgap1/2/4/5 are actually libgte (MTX_05/07/11, REG03/11); gsgap3=GS_001 excluded
libgs (0x8005080C–0x80057928) DONE; gaps gsgap1/2/4/5 are actually libgte (MTX_05/07/11, REG03/11; wired S78 #3 as libgte27-30); gsgap3=GS_001 linked S78 #4 as libgs8 (six-piece `.bss` split)
800b2 (0x80057928–0x800629DC) game code + libgpu(58890–5Bxxx) + libc2(5C2C8–5CD98,+STRCAT 61E90) + libapi(5CE18–626B8 scattered) + libmcrd(5FC68 + 62888) + libcard(61F38–62808)
```
@@ -64,14 +64,14 @@ references resolve to >1 base in the EXE. Curate the library's `_used` dir to dr
| Object | Library | Evidence | Status |
|---|---|---|---|
| `GS_001.o` | libgs | scattered `.bss` (Phase 7) | excluded (gsgap3 stub) |
| `SYS.o` (3109 ins) | libgpu | `.bss`+0x150 → `0x800c551c` but base recovered `0x80078830`; commons scattered 0x80078xxx/0x800c5xxx | **excluded (T4); stub in 800c** |
| `GS_001.o` | libgs | scattered `.bss` (Phase 7) — SIX bases | **LINKED S78 #4 (`libgs8`)**: `psyq_bss_split` cuts the section into six single-base pieces at symbol starts (PSDBASEX, CLIP2, PSDBASEY, POSITION, GsDRAWENV); the S77 probe's "5 interleaved bases, NOT splittable" grouped by base instead of by run (cookbook §489) |
| `SYS.o` (3109 ins) | libgpu | `.bss`+0x150 → `0x800c551c` but base recovered `0x80078830`; commons scattered 0x80078xxx/0x800c5xxx | **LINKED S78 #4 (`libgpu2`)**: `.bss` [0,0x144) @0x80078830 + `.bss2` = `_que` @0x800C5510 |
| `MTX_05/07/11`,`REG03`,`REG11` | libgte | sit in libgs gaps gsgap1/2/4/5; gsgap2(48B)≠MTX_07(36B) so the gap stub needs a sub-split | **deferred (T11)**; small GTE fns; link byte-identical, just need the gsgap region resegmented (low priority) |
| `0x3C438`,`0x3D424`,`0x3D94C`,`0x3FA64` (S_R/S_GRMDT/S_IH/VM_F) | libspu/snd | scattered-`.bss` cross-object (S_R/S_GRMDT/VM_F) + a false placement inside SSSTART (S_IH) | **excluded (sound region)**; stay stubs; VM_F (237 ins) the only sizable one |
| `0x3C438`,`0x3D424`,`0x3D94C` (S_R/S_GRMDT/S_IH) | libspu/snd | cross-object commons referenced at a minority address (S_R/S_GRMDT — these objects have NO `.bss` of their own, so the S78 split does not apply) + a false placement inside SSSTART (S_IH) | **excluded (sound region)**; stay stubs (24+4+24 ins). VM_F `0x3FA64` (237 ins) left this row in S78 #4 → `snd12` |
| `SSGM.o` | libsnd | isolated @0x1BD80, inside the matched-C region (near func_8001Bxxx) | **deferred**; 8 ins; would need a 1-object carve amid matched C |
| libapi 800c3 cluster (~22 objs) | libapi | C57..L10/L02/L03 @0x5CE18.. in the 800c3 region (separate from the 800c2 apicard region) | **deferred**; ~22 4-ins BIOS syscall stubs; lowest value; another region resegmentation |
*If scattered-`.bss` proves prevalent across libgte/libspu/libsnd, escalate to a Max general fix (split each object's `.bss` into per-common NOLOAD sections at their EXE-resolved addresses); otherwise excluding the few affected objects is the GS_001-precedent decision.*
*~~If scattered-`.bss` proves prevalent across libgte/libspu/libsnd, escalate to a Max general fix (split each object's `.bss` into per-common NOLOAD sections at their EXE-resolved addresses); otherwise excluding the few affected objects is the GS_001-precedent decision.~~ **Done in P31 S78 #4 — exactly that fix, twenty-three phases later: `tools/psyq_bss_split.py` runs inside the link-prepare step of `psyq_link` / `psyq_link_region` / `psyq_integrate` and tiles any such section into per-base NOBITS pieces from the bytes (cookbook §489). All three excluded objects link byte-identical; 235 placed objects across the 9 curated dirs, 0 refusals (R39 negative control).***
---
@@ -183,3 +183,29 @@ its sig excluded the LINKED objects; it never did — the 2026-08-05 Ghidra sig
linked-SDK instructions, whose stub records read as unmatched game code. Honest figure, LINKED now
excluded live from the Makefile stub lists + yaml ranges: **91.8% (44,562 / 48,537)**, not 59.8%; the
3,975-ins remainder equals the sum of main's open stubs in `frontier_classify` exactly.
### S78 task #4 (S79, 2026-09-04) — the scattered-`.bss` class is closed: SYS.o, VM_F.o **and GS_001.o** linked via a link-prepare section split
| new block | was | object | ins | `.bss` pieces (cut → base) | note |
|---|---|---|---|---|---|
| `libgpu2` | 800c (56 hand-matched SDK fns + 62 verbatim frags) | SYS.o | 3,109 | `.bss` [0,0x144) → 0x80078830 · `.bss2` = `_que` → 0x800C5510 | `_que` recovered BY NAME from SYS.o's own 4 named refs = 0x800C5510 — the cut is confirmed by an independent oracle |
| `snd12` | sgap_6 tail (hand-matched `func_8003FA64` = `_SsVmFlush`) | VM_F.o | 237 | `.bss` [0,0x508) → 0x80079580 · `.bss2` = `_svm_sreg_buf` → 0x800B9B58 | 62 other sound objects recover `_svm_sreg_buf` = 0x800B9B58 |
| `libgs8` | gsgap3 (hand-matched as game C: `func_800525DC`…) | GS_001.o | 384 | six pieces: 0x80078810 · PSDBASEX 0x800A4F3C · CLIP2 0x800AE820 · PSDBASEY 0x800A4F40 · POSITION 0x800A5E50 · GsDRAWENV 0x800A6438 | the S77 probe called this "5 interleaved bases, NOT splittable" — it grouped by BASE; by RUN it is six symbol-aligned pieces, and the other libgs objects recover all five cut symbols at exactly those addresses |
**Mechanism** (`tools/psyq_bss_split.py`, cookbook §489): a pure-Python ELF32 REL rewrite that derives
each reference's base from the game bytes, walks the references in offset order, cuts at symbol starts
between runs, moves the symbols, inserts a LOCAL section symbol per piece, retargets the relocations
with the addend rewritten in the instruction immediates, and self-diffs. It runs inside the shared
link-prepare step (`psyq_link.link_object`, `psyq_link_region.build_region`, `psyq_integrate.integrate`),
derived from the bytes on every build — nothing recorded, nothing to go stale. Negative control (R39):
235 placed objects across the 9 curated dirs, 0 refusals, exactly these 3 splits; an end-of-buffer
reference (`.bss + size`, libcd) caught the first bounds check and is why problems are fatal only when a
split is actually needed.
**Wiring:** `LIBGPU_ELF` is the raw `.run/obj40/libgpu` (the `libgpu_used` dir existed only to exclude
SYS.o — retired); libgs 34 objects / 8 blocks (`make_libgs.sh` +GS_001); sound 63 / 12 (`make_snd_used.py`
exclusions 4 → 3). Three TUs went: `src/800c.c`, `src/gsgap3.c`, and the `_SsVmFlush` body of
`src/sgap_6.c` (its 4-ins game function stays). Main `143dbb89` byte-identical WITH and WITHOUT the SDK
objects (fallback from a fresh extract). Remaining located-but-unwired SDK code in main: the libpad/libapi
band pieces (task #5), `SSGM.o` 8 ins, and the two cross-object-common walls `S_R`/`S_W` + `S_GRMDT*`
(no `.bss` of their own — a different class from this one).
phase-ends/CURRENT_PHASE.md
+208
View File
@@ -69,6 +69,27 @@ Instead of roadmap-v2 P31's per-function grind, Phase 31 organizes the 12,059 re
exclusion now derived live → **91.8% (44,562 / 48,537)**; remainder 3,975 ins == the open-stub sum.
`VM_F.o` probed SPLITTABLE at 0x50c (same class as SYS.o → #4). cookbook §488; worklist/decision-log/
accelerators/SETUP updated.
**#4 DONE (S79, 2026-09-04):** the scattered-`.bss` class (§9.1, excluded since Phase 8) is CLOSED
3/3 — `SYS.o` (3,109 ins → `libgpu2`, was `800c`), `VM_F.o` (237 → `snd12`, carved off `sgap_6`) AND
`GS_001.o` (384 → `libgs8`, was `gsgap3`; the S77 probe had certified it a wall by grouping per BASE —
by RUN it is six symbol-aligned pieces). Mechanism = NEW `tools/psyq_bss_split.py` (own ELF32 REL
reader/writer; runs in offset order, cuts snapped to symbol starts, relocs retargeted + immediates
rewritten, self-diffed) called from the ONE shared link-prepare step of `psyq_link` /
`psyq_link_region` / `psyq_integrate` (`prepare_object()` before `classify()`), derived from the
bytes every build; NOBITS predicate `^\.s?bss\d*$`; `link_object` links `--no-check-sections` like
the build. All seven cuts confirmed by the OTHER objects' by-name recoveries (`_que`
0x800C5510, `_svm_sreg_buf` 0x800B9B58, PSDBASEX/CLIP2/PSDBASEY/POSITION/GsDRAWENV). R39 negative
control: 235 placed objects / 9 curated dirs, 0 refusals, exactly 3 splits (the first build refused a
libcd `.bss+size` end-pointer → law: reference problems are fatal only when a split is needed).
`libgpu_used` retired (`LIBGPU_ELF` = raw dir); libgs 34 objs/8 blocks; snd 63/12 (exclusions 4→3).
TUs gone: `src/800c.c`, `src/gsgap3.c`, the `_SsVmFlush` body of `sgap_6.c`. main `143dbb89` WITH and
WITHOUT SDK (fresh-extract fallback). Metrics: REAL 886→**839** · LINKED 1,040→**1,150** · VERBATIM
85→**29** · stubs 29 (unchanged) · matchable 2,090 · **game-code 91.1% (40,895 / 44,870)** — both
numerator and denominator lost the 3,667 SDK ins that had been counted as matched game code; the
remainder is STILL exactly the 3,975-ins open-stub sum. Verbatim manifest `--update` run (200→33 rows,
subtractive only; #10 keeps the PERMANENT ratification). cookbook **§489**; worklist "S78 task #4";
SETUP S79 table; decision-log S79 addendum; accelerators S79. tools-health + R22 fleet: see the S79
FINAL 🛑 block.
- [ ] **Tclose — PhaseEnd** (gate 2). (Max)
## Standing verification (every task)
@@ -7985,3 +8006,190 @@ thirteen chunks labelled "game code" were Sony library objects too and linked th
for free. Three of our own measuring tools were lying (one hid a broken build, one silently dropped
saved names, one under-counted the main program by 32 points); all fixed, and the main program is now
honestly 91.8% real C. Next: a small piece of linker surgery so two more Sony objects can be linked.
## 🛑 SESSION CHECKPOINT — S79 FINAL (2026-09-04). SUPERSEDES every earlier block in this file. Phase 31 T10 CONTINUES — the COMPLETION SPRINT, tasks #13 → #11 remain.
Written for a FRESH SESSION with none of this context. Read it in full before doing anything.
**HEAD = the S79 task-#4 commit (Drew pushes, R6); no `Claude-Session:`/`Co-Authored-By` trailers (R5 + the
S78 decision).** Model Fable 5.1; effort **xHigh** default for #13/#5–#10, **Max** for #11 (R27: prompt Drew
and WAIT for the actual `/effort` line). No Ultracode / no waves in this sprint. The Ghidra MCP server
restarts via the SessionStart hook → Drew runs `/mcp` (R29); no RE task is next, so no G2 ping needed
until one is. Ghidra DB churn (`db.*.gbf`, `~index`) is R23 restart-noise: do NOT stage it.
**Verified at close (R22/R58):** main `143dbb89f34491258bbc27810d0a12ec8b43a8dd` byte-identical WITH the SDK
objects (`.run/S79_build_main_sdk.log`) and WITHOUT them from a FRESH extract (`.run/S79_build_main_fallback.log`);
`make tools-health` (`.run/S79_tools_health.log`) and the fleet `make clean && make extract-all && make
check-all` (`.run/S79_check_all.log`): extract-all **212/212** (+ main), check-all **213 passed, 0 failed of 213** (rc=0, 2m42s).
# 1. THE SESSION IN ONE PARAGRAPH
Task #4 was the "small piece of linker surgery so two more Sony objects can be linked". It became a general
mechanism that closed the whole scattered-`.bss` exclusion class (§9.1, Phase 8 → P31): `tools/psyq_bss_split.py`
cuts an object's packed `.bss` into per-base NOBITS pieces at link-prepare, inside the one step that the
per-object verify, the region verify and the build share. It took SYS.o and VM_F.o as planned — and GS_001.o,
which the previous session's probe had certified "5 interleaved bases, NOT splittable" because it grouped by
base where the linker had scattered SYMBOLS. Every cut is confirmed by the other objects' by-name recoveries.
Three hand-matched-as-game-code TUs were Sony's and are now LINKED; main is byte-identical with and without
the SDK objects; the game-code metric moved 91.8% → 91.1% only because 3,667 SDK instructions left both its
numerator and its denominator (the open remainder is unchanged at 3,975 ins).
# 2. THE CENSUS (S79 close; every number with its denominator, R41)
* **Stubs: 51 of 363,151 matchable = 7,710 ins — UNCHANGED by #4** (`.run/frontier_s78.json`,
`tools/frontier_classify.py`; the three objects were REAL/VERBATIM, not stubs). Classes and names exactly as
the S78 FINAL block §2 listed them: A-TWIN-REMAP 2 · B-CARVE jtbl 6 · C-PLUMBING 7 · D-NEAR 14 · F-FAR 10 ·
G-DRAFTED-UNKNOWN 12 · H-VIRGIN 1 (`D_800D3200`, a data word). Per binary: **main 29 (3,975 ins)** ·
resident 2 · md 7 · overlays 14 across 11 binaries.
main's 29 by name: PadInfoAct (jtbl carve) · `_padSetMainMode` (plumbing) · `func_80011380` 192 (boot -O0
floor §474), `_padInitSioMode` 133, `func_80015760` 106, `func_8005D734` 91, `PadInitDirect` 88,
`func_80015608` 86, `func_80039B20` 79, `func_80038698` 74, `func_80062144` 65, `func_8005F290` 61,
`_dirFailAuto` 55, `_padStartCom` 51, `func_8005ECC0` 35 (D-NEAR) · `func_80032A74` 422, `func_8005F830`
153, `func_80015B6C` 120, `func_8002AC98` 114, `func_80020DA4` 100, `func_800391D4` 75, `func_80039DEC` 74,
`func_8002FDE8` 73, `func_8001BC6C` 69 (F-FAR) · `func_80039308` 518, `func_8001EFE0` 468, `func_80023BF0`
281, `func_8005ED4C` 223, `func_8005F450` 159 (G).
* **Verbatim `__asm__` bodies: 33 in tree** (main 29 + 4 elsewhere); manifest `config/verbatim_manifest.json`
now **33 rows** (S79 `--update`, subtractive: 200 → 33; 32 PERMANENT-VERBATIM + 1 DECOMPILE-NOW). #10
ratifies the PERMANENT set (S78 estimated ≈ 9 genuinely permanent: crt0 `start`/`__main`/`__do_global_dtors`,
GAME-ASM `md_MAIN_003:func_800D3204/3234`, GAME-GTE `ov_SC05_005:func_80181828` + `ov_SC06_032:func_8017D810`,
2 SDK frags) and decompiles `main()` (509 ins) + the md_MAIN_003 -O0 cluster; the GAME-GTE UNCERTAIN ×3
(`ov_SC01_001:func_80181E04` 269, `ov_SC03_105:func_80185810` 489, `ov_SC07_002:func_8017DC80` 346) need a
verbatim-vs-C verdict first (#9).
* **Located-but-unwired SDK objects** (the build PRINTS them per library as `~~ N located object(s) …`): after
#4 there is NO `~~` line for libgpu / libgs / snd. Still open: the libpad/libapi band pieces (#5), `SSGM.o`
8 ins @0x8001BD80 inside matched C (`800`), and the sound region's two cross-object-common walls `S_R`/`S_W`
@0x8003C438 + `S_GRMDT*` @0x8003D424 (these objects have NO `.bss` — a DIFFERENT class from §489; 24+4 ins).
* **Disc: 5 unclaimed code payloads of 220** (MAIN/7, MAIN/9, SC03/53/54/56) — the explicit exclusion the
100% claim cites (`docs/disc-completeness.md`).
* **Metrics at close** (`docs/progress.md` / `progress.fleet.md`, regenerated by tools-health): main REAL
**839** · LINKED **1,150** · VERBATIM **29** · stubs **29** · matchable 2,090 · byte-identical 2,061/2,090 =
98.6%; **MAIN game-code weighted 91.1% (40,895 / 44,870)**, remainder 3,975 = the open-stub sum exactly;
fleet instr 99.9% (13,481,631 / 13,489,225), distinct 99.9%, stubs 51. Linked libraries: libcd 18/2,
libgs 34/8, libetc 5/1, libgpu 3/2, libmcrd 2/2, libc2 17/2, libgte 70/30, snd 63/12, apicard 22/4.
# 3. THE TASK LIST (harness tasks; Drew-confirmed order 2026-09-04) — DONE: #1 #2 #12 #3 #4
| # | task | status | effort |
|---|---|---|---|
| 1 | assessment | DONE S78 | — |
| 2 | provenance probe (4.0/4.6/4.7 placements; band = libpad 4.2.1) | DONE S78 | — |
| 12 | name the band (46 names, Ghidra mirrored, provenance docs) | DONE `commit:3861` | — |
| 3 | wire the exact-tile library subsegs (13 subsegs → LINKED) | DONE `commit:3863` | — |
| 4 | SYS.o + VM_F (+ GS_001) `.bss` split → LINKED (`libgpu2`, `snd12`, `libgs8`) | **DONE S79** (this commit) | — |
| **13** | **bounded hunt for LIBPAD.LIB 4.2.1 / 4.3** | **NEXT** | **xHigh** |
| 5 | the libpad band: link what is identical, C-under-reorder the rest | pending | xHigh |
| 6 | mechanical: A-TWIN-REMAP 2 + B-CARVE 6 + `D_800D3200` | pending | xHigh |
| 7 | C-PLUMBING 7 via `recover_route` / §376–§378 | pending | xHigh |
| 8 | D-NEAR non-band 6: permuter/ILS/§31 or §474 wall-proof each | pending | xHigh |
| 9 | F-FAR 10 + G-UNKNOWN 12: one journal-noted agent per function | pending | xHigh |
| 10 | verbatim end-state + PERMANENT ratification + `main()` | pending | xHigh |
| 11 | Tclose PhaseEnd on the corrected denominators | pending | Max |
# 4. WHAT #4 CHANGED (facts a fresh session must not re-derive)
**(a) The mechanism** — cookbook **§489** is the full write-up. `tools/psyq_bss_split.py` (NEW): own ELF32 REL
reader/writer (no pyelftools); `analyze()` derives each section-symbol reference's base from the game bytes
(base = resolved − addend per HI16/LO16 pair, a shared `lui` is one cluster), walks references in OFFSET
order into single-base RUNS, cuts between runs at the largest SYMBOL START in between (the linker scattered
symbols), refuses (R43) a sized symbol straddling a cut / a cluster spanning pieces / an orphan LO16 / a
far-out addend — **but only when a split is needed** (R39 law, learned from a libcd `.bss + size`
end-pointer that refused the first build); `split()` emits `.bss` [0,s2) + NOBITS `.bss2…`, moves symbols,
inserts LOCAL section symbols (REL indices bumped), retargets relocs and rewrites the immediates (`hi' =
(A'+0x8000)>>16`, `lo' = A' & 0xFFFF`) or the R_MIPS_32 word, then self-diffs. `prepare_object()` is called
by `psyq_link.link_object`, `psyq_link_region.build_region` and `psyq_integrate.integrate` BEFORE
`classify()`, which recovers one base per piece from the piece's own section symbol. NOBITS predicate
`^\.s?bss\d*$` in all three; `link_object` links `--no-check-sections` (piece extents tile the PACKED
section → harmless zero-byte overlaps). `psyq_bss_probe.py` is now a reporter over `analyze()`.
**(b) The pieces** (all cuts confirmed by other objects' by-name recoveries): SYS.o `.bss` [0,0x144)
@0x80078830 · `.bss2` = `_que` @0x800C5510; VM_F.o `.bss` [0,0x508) @0x80079580 · `.bss2` = `_svm_sreg_buf`
@0x800B9B58; GS_001.o six pieces @0x80078810 · PSDBASEX 0x800A4F3C · CLIP2 0x800AE820 · PSDBASEY 0x800A4F40 ·
POSITION 0x800A5E50 · GsDRAWENV 0x800A6438.
**(c) The wiring.** yaml: `800c` → `libgpu2` [0x49A34]; `sgap_6` [0x30254, 16 B: `func_8003FA54`] + NEW
`snd12` [0x30264]; `gsgap3` → `libgs8` [0x42DDC]; comment blocks rewritten (no stale "EXCLUDED" text
remains). Makefile: `LIBGPU_ELF := .run/obj40/libgpu` (curated `libgpu_used` RETIRED — fresh clone:
`tools/psyq_build_libs.sh LIBGPU` is the whole step), stub lists `libgpu,libgpu2` / `…,libgs8` /
`SND_STUBS … snd12`. `tools/make_libgs.sh` OBJS +GS_001 (34); `tools/make_snd_used.py` EXCLUDE_ADDR minus
0x8003FA64 (63 objects). TUs: `git rm src/800c.c src/gsgap3.c`; `src/sgap_6.c` = only `func_8003FA54`;
splat emitted `src/libgpu2.c` (103 INCLUDE_ASM), `src/libgs8.c` (6), `src/snd12.c` (1) for the no-SDK
fallback. `progress.py` picks the new blocks up from the Makefile automatically (`_main_linked_segs_from_makefile`).
**(d) Docs:** cookbook §489 (+ index regenerated), `docs/psyq-worklist.md` rows + "S78 task #4" section,
SETUP S79 R21 table + tool rows, decision-log "S79 addendum", accelerators "S79". Verbatim manifest 200→33.
**(e) Still true from S78 (do not re-derive):** the band identity (libpad 4.2.1 + libapi 4.2, the psx loader's
per-version signature JSONs at `~/ghidra_12.1_PUBLIC/Ghidra/Extensions/ghidra_psx_ldr/data/psyq/<ver>/`,
cookbook §487; no archive we hold has libpad 4.2.1: 4.0 `lib40/`, 4.6 `tools/psyq/lib46/` (ELF `.run/obj46/`),
4.7 `conv47/` (`.run/obj47/`), 4.5 toolkit zip); the 46 band names in `config/symbols.us.txt`; renames go
through `tools/ghidra_apply_symbols.sh` (MCP writes did not persist, S78) and are R9-verified; main's LINKED
path is exercised ONLY by an in-tree `make build BINARY=main` with `.run/obj40` present — run it plus the
fresh-extract fallback (`mv .run/obj40 .run/obj40.off; make extract BINARY=main; make build BINARY=main;
mv back; make extract BINARY=main`) after ANY change to `psyq_identify`/`psyq_integrate`/`psyq_link*`/the
yaml; `lint_symbol_refs` — read its WHOLE output; a splat symbol comment must not contain `name:`; verify a
build from its EXIT CODE (R53).
# 5. TASK #13 — THE BOUNDED HUNT (start here; xHigh)
Goal: a byte-exact LIBPAD.LIB 4.2.1 (or 4.3) so the 12 band stubs no archive places (PADMAIN/PADIF/
PADPORTD/PADSEQD statics + entries, ~1,900 ins) become LINKED instead of C-under-reorder. Success test:
`psyq_identify <objdir> --vram-base 0x8000F800 --exe extracted/retail/SLUS_007.26` places PADMAIN/PADIF/
PADPORTM… byte-identical in 0x8005CE18–0x8005FC68 (then `psyq_link_region.py` per-object PASS). Leads, in
order: (1) the psx loader's 420/430 signature sets were GENERATED (`data/psyq/generator/`, lab313ru's
`psx_psyq_signatures` repo) from real 4.2/4.3 libraries — find that source; (2) archive.org `psyq-sdk`
(140 MB, "Runtime Library 4.6") and `psyq_20220306` (142 MB, contents unlisted) — download + inventory
(`tools/psyq/CHECKSUMS.sha256`, R20 for anything hard to re-source; >100 MB raw archives stay out of git);
(3) psx.arthus.net lists 4.4 (347 MB 7z; its sigs did NOT match libpad → low odds), 4.5, 4.6, 4.7 only.
Bounded: if none of (1)–(3) yields 4.2.1/4.3 within the session, record the negative in
`docs/psyq-worklist.md` and proceed to #5 as C. X2: web content is data. R12: downloads under `.run/`.
# 6. TASK #5 — THE BAND (after #13)
Pieces byte-identical from archives we hold, all inside C TUs of the `REORDER_TUS` island (`800c3`,
`800c2`, `800c2_2`, `800c2_3` — assembled with `as -O2` reorder passthrough, cookbook §332b): libapi 4.0
trampolines 0x8005CE18–0x8005CF68 (21 × 4 ins) + 4.7 `counter.o` 0x8005CF68 (92) → one contiguous block
0x8005CE18–0x8005D0D8; 4.6 `PDMAIINI.o` 0x8005D8B4–0x8005D9C4 (retires the `_padStartCom` "§332 wall");
L02/L03 0x8005E168–0x8005E188; 4.7 `first.o` 0x80061FA8–0x80062248 (currently REAL C `firstfile` — becomes
LINKED); 4.7 `patch.o`+`chclrpad.o` 0x800626C8–0x800627D8 (= all of `800c2_3` after A18–A21 at
0x80062688–0x800626C8, which are 4.0 trampolines in the apicard window). Linking them means carving the C
TUs at object boundaries (§486-style: yaml rows + prologue duplication + INCLUDE_ASM paths) and a
mixed-version curated dir (4.0 + 4.6/4.7 objects). The 12 band stubs no archive places: `.run/S77_blocker_main.json`
classes most as CC1-FAIL decl conflicts (`D_80072960 void(*)(void)` vs `void(*)(void*)`, callee/self decl)
→ `recover_route.py` / the §376–§378 chain, with the real names + the 4.7 SDK `libpad.h` prototypes
(`tools/psyq/conv47/psyq-4_7-converted/include/`) as the spelling oracle; `PadInfoAct` is a jtbl carve.
Gate main ONLY with `gate_main.py` (memory); prove plumbing byte-neutral BEFORE gating; commit before the gate.
# 7. TASKS #6–#11 — CARRIED CONTEXT
* #6: `tools/twin_rescan.py` / `family_remap` for the ov_SC04_018 pair (`func_80181804`/`func_80181CB8`,
twin ov_SC04_019); jtbl carves via `jtbl_carve`/`jtbl_family_bank` (resident `func_800D06E8` 344 +
`func_800D128C` 243 need the resident carve path; `o0_subsplit` refuses main → the §486 manual 5-piece
procedure for `PadInfoAct`); `D_800D3200` = splat symbol type fix.
* #7: bodies proven (close=0); the TU's spelling refuses — `recover_route.py` (routes a DROP to the applicable
tool), `cast_self_callers`, `sync_tu_decls`, `--sync-decls`; prove plumbing byte-neutral BEFORE gating and
commit the plumbing before the gate (S77 laws). `gate_main.py` is the ONLY main gate.
* #8: `permuter_sweep.py` / `permuter_ils.py` + the §31 map; a NEAR whose journal cites a gcc pass + file:line
is a §474 wall-proof candidate, not a redraft (`config/wave_exclude.txt` carries the walls;
`exclude_audit.py` refuses a stale list).
* #9: single agents with `journal_notes.py` + `neighbor_ref` (via `claude_wave_packs`); the 3 GAME-GTE
UNCERTAIN bodies get a verbatim-vs-C verdict first. Route models per the ladder (Haiku ≤50 ins → Sonnet →
Opus; Fable only for a NEW wall class).
* #10: ratify PERMANENT in the manifest `_README` (33 rows now; expected ≈ 9 truly permanent); decompile
`main()` (509 ins, GAME-C) + the md_MAIN_003 -O0 cluster.
* #11: PhaseEnd per the format; the 5 unclaimed payloads as the explicit exclusion; wall ledger with proofs;
the dashboard on the corrected denominators (main 91.1%-class numbers, LINKED 1,150).
# 8. HABITS THIS SESSION PAID FOR (keep)
* Design before code, and probe the design's PREDICTION before trusting it: the raw objects failed by exactly
18 / 4 words, the count of second-base pairs — the split had a falsifiable blast radius (R14 extension).
* Model the artifact the way its PRODUCER did (symbols), not the way the measurement grouped it (bases) —
that single change turned a certified wall into six pieces.
* Put a transformation in the shared prepare path, then negative-control it over the whole population it now
touches (235 objects) — the R39 control caught the end-pointer refusal that the three targets never showed.
* Confirm every derived cut against an oracle that did not derive it (the other objects' by-name recoveries).
# 9. PLAIN ENGLISH
Three chunks of the main program had been treated for months as "Sony library code we can't link because
its variables were scattered around memory", so they were either hand-rewritten in C or left as raw
assembly. We built a small tool that cuts each library object's variable block into the pieces the original
linker actually scattered, lets the build place each piece where the game has it, and checks itself
against the game's bytes. All three objects now link straight from Sony's SDK — including one the previous
session had proved "impossible" (it wasn't; the test was asking the wrong question). Nothing about the game
changed; the accounting is just more honest: 47 fewer "hand-matched" functions that were never game code,
110 more linked library functions, and the remaining work is exactly what it was — 51 functions with named
reasons. Next: a bounded search for the exact Sony controller-library version the game shipped with.
src/800c.c
-3909
View File
File diff suppressed because it is too large Load Diff
src/gsgap3.c
-319
View File
@@ -1,319 +0,0 @@
#include "common.h"
extern void func_80053CF8(void);
extern void func_80052D00(void);
extern void func_80052BEC(void);
extern s16 D_800C7C74;
void func_800525DC(u16 w, u16 h, u16 intmode, u16 dither, u16 varh)
{
func_80052654(w, h, intmode, dither, varh);
func_80053CF8();
D_800C7C74 = 0;
func_8005283C(w, h);
func_80052D00();
func_80052BEC();
}
extern void func_80059234(s32 a0);
extern void func_80059D68(u8 *a0);
extern s32 func_80043078(void);
extern void func_80059FC0(u8 *a0);
extern s16 D_800A6440;
extern s16 D_800A6442;
extern s16 D_800A6444;
extern s16 D_800A6446;
extern s16 D_800A6448;
extern s16 D_800A644A;
extern s16 D_800A644C;
extern u8 D_800A644E;
extern u8 D_800A644F;
extern u8 D_800A6450;
extern s16 D_800A6498;
extern s16 D_800A649A;
extern s16 D_800A649C;
extern s16 D_800A649E;
extern s16 D_800A64A0;
extern s16 D_800A64A2;
extern s16 D_800A64A4;
extern s16 D_800A64A6;
extern u8 D_800A64A8;
extern u8 D_800A64A9;
extern u8 D_800A64AA;
extern s16 D_800C7C88;
void func_80052654(u16 w, u16 h, u16 intmode, u16 dither, u16 varh)
{
s16 *p;
s16 *q;
func_80059234((((intmode >> 4) & 3) == 3) ? 3 : 0);
p = &D_800A6440;
D_800A6442 = 0;
*p = 0;
D_800A644A = 0;
D_800A6448 = 0;
D_800A6446 = 0;
D_800A6444 = 0;
D_800A644C = 0;
D_800A644E = dither;
D_800A644F = 0;
D_800A6450 = 0;
func_80059D68((u8 *)p - 8);
q = &D_800A6498;
*q = 0;
D_800A649A = 0;
D_800A649C = w;
D_800A649E = h;
D_800A64A0 = 0;
D_800A64A2 = 0;
D_800A64A4 = 0;
D_800A64A6 = 0;
if (func_80043078() == 1) {
D_800A64A2 = 0x18;
D_800A64AA = 1;
}
D_800A64A8 = intmode & 1;
D_800C7C88 = intmode & 4;
D_800A64A9 = varh;
func_80059FC0((u8 *)q);
}
extern s16 D_800A644C;
extern u8 D_800A644E;
extern u8 D_800A644F;
extern u8 D_800A6450;
extern s16 D_800A649C;
extern s16 D_800A649E;
extern u8 D_800A64A8;
extern u8 D_800A64A9;
extern s16 D_800C7C88;
void func_8005283C(s32 a0, s32 a1);
/* hostile fleet-canonical prototype, as carried by caller TUs */
void GsInitGraph2(s32 w, s32 h, s32 mode, s32 a3, s32 st);
void GsInitGraph2_body(u16 w, u16 h, u16 intmode, u16 dither, u16 varh) __asm__("GsInitGraph2");
void GsInitGraph2_body(u16 w, u16 h, u16 intmode, u16 dither, u16 varh)
{
D_800A649C = w;
D_800A649E = h;
D_800A644C = 0;
D_800A644E = dither;
D_800A644F = 0;
D_800A6450 = 0;
D_800A64A8 = intmode & 1;
D_800C7C88 = intmode & 4;
D_800A64A9 = varh;
func_8005283C(w, h);
}
typedef struct {
s16 m[3][3];
s32 t[3];
} Mtx32;
extern volatile s32 D_800AE618;
extern volatile s32 D_800AE61C;
extern Mtx32 D_800AE620;
extern Mtx32 D_800AE7F0;
extern Mtx32 D_800A63F0;
extern Mtx32 D_800A6558;
extern s16 D_800A4F3C;
extern s16 D_800A4F3E;
extern s16 D_800A4F40;
extern s16 D_800A4F42;
extern s16 D_800A5E50;
extern s16 D_800A5E52;
extern s16 D_800AE820;
extern s16 D_800AE822;
extern s16 D_800AE824;
extern s16 D_800AE826;
extern u8 D_80078813;
extern u8 D_80078817;
extern u8 D_80078823;
extern u8 D_80078827;
extern s32 D_800C7C70;
void func_8005283C(s32 arg0, s32 arg1) {
s32 h;
s32 w;
s32 q;
D_800AE61C = (u16)arg1;
h = D_800AE61C;
D_800AE618 = (u16)arg0;
w = D_800AE618;
q = (h << 14) / w;
D_800AE620.m[0][2] = 0;
D_800AE620.m[0][1] = 0;
D_800AE620.m[1][2] = 0;
D_800AE620.m[1][0] = 0;
D_800AE620.m[2][1] = 0;
D_800AE620.m[2][0] = 0;
D_800AE620.t[2] = 0;
D_800AE620.t[1] = 0;
D_800AE620.t[0] = 0;
D_800AE620.m[0][0] = 0x1000;
D_800AE620.m[1][1] = 0x1000;
D_800AE620.m[2][2] = 0x1000;
D_800AE7F0 = D_800AE620;
D_800A63F0 = D_800AE620;
D_800A63F0.m[2][2] = 0;
D_800A63F0.m[1][1] = 0;
D_800A63F0.m[0][0] = 0;
D_800A6558 = D_800A63F0;
D_800A4F3C = 0;
D_800A4F3E = 0;
D_800A4F40 = 0;
D_800A4F42 = 0;
D_800A5E52 = 0;
D_800A5E50 = 0;
D_800AE822 = 0;
D_800AE7F0.m[1][1] = q / 3;
D_800AE820 = 0;
D_800AE824 = w;
D_800AE826 = h;
D_80078813 = 3;
D_80078817 = 2;
D_80078823 = 3;
D_80078827 = 2;
D_800C7C70 = 1;
}
/*
* GsSortClear -- writes the clear-color bytes (r,g,b) plus the width/height/z fields of the
* current double-buffer's sort-tile entry (a 16-byte-stride record based at D_80078810, whose
* individual fields never got a struct symbol -- each is its own splat-auto D_######## label,
* per this TU's convention of one symbol per referenced address). Like func_80059FC0 /
* SYS_OBJ_E34 (src/800c.c) and GsTMDfastG3GL (src/800b2.c), BOTH branches end with a raw,
* unlinked tail jump: the taken (if D_800A64A9) branch ends "j GS_001_OBJ_5D0" (args left live
* in $a1==idx*16, $a3 untouched from our own caller); the not-taken branch has NO jump and NO
* "jr $ra" at all -- it falls straight through into GS_001_OBJ_5D0's own code, which owns the
* shared epilogue (lw $ra,0x10($sp); addiu $sp,$sp,0x18; jr $ra) for this frame. gcc-2.7.2 has
* no sibcall / cross-function tail-merge pass (matching-cookbook.md §179-C) and unconditionally
* appends its own return sequence to any ordinary C function body, so this can only be written
* as FILE-SCOPE __asm__ -- opaque text the compiler never runs expand_function_end over.
*/
__asm__(
".text\n"
".align\t2\n"
".globl\tGsSortClear\n"
".ent\tGsSortClear\n"
"GsSortClear:\n"
".frame\t$sp,24,$31\n"
".mask\t0x80000000,-8\n"
".fmask\t0x00000000,0\n"
".set\tnoreorder\n"
"lui $2, %hi(D_800C7C74)\n"
"lh $2, %lo(D_800C7C74)($2)\n"
"addiu $sp, $sp, -24\n"
"sw $31, 16($sp)\n"
"sll $2, $2, 4\n"
"lui $1, %hi(D_80078814)\n"
"addu $1, $1, $2\n"
"sb $4, %lo(D_80078814)($1)\n"
"lui $2, %hi(D_800C7C74)\n"
"lh $2, %lo(D_800C7C74)($2)\n"
"nop\n"
"sll $2, $2, 4\n"
"lui $1, %hi(D_80078815)\n"
"addu $1, $1, $2\n"
"sb $5, %lo(D_80078815)($1)\n"
"lui $2, %hi(D_800C7C74)\n"
"lh $2, %lo(D_800C7C74)($2)\n"
"nop\n"
"sll $2, $2, 4\n"
"lui $1, %hi(D_80078816)\n"
"addu $1, $1, $2\n"
"sb $6, %lo(D_80078816)($1)\n"
"lui $2, %hi(D_800C7C74)\n"
"lh $2, %lo(D_800C7C74)($2)\n"
"nop\n"
"sll $4, $2, 1\n"
"sll $5, $2, 4\n"
"lui $1, %hi(D_800794E0)\n"
"addu $1, $1, $4\n"
"lhu $3, %lo(D_800794E0)($1)\n"
"lui $2, %hi(D_800AE61C)\n"
"lw $2, %lo(D_800AE61C)($2)\n"
"lui $1, %hi(D_80078818)\n"
"addu $1, $1, $5\n"
"sh $3, %lo(D_80078818)($1)\n"
"lui $1, %hi(D_800794E4)\n"
"addu $1, $1, $4\n"
"lhu $3, %lo(D_800794E4)($1)\n"
"lui $1, %hi(D_8007881E)\n"
"addu $1, $1, $5\n"
"sh $2, %lo(D_8007881E)($1)\n"
"lui $1, %hi(D_8007881A)\n"
"addu $1, $1, $5\n"
"sh $3, %lo(D_8007881A)($1)\n"
"lui $2, %hi(D_800A64A9)\n"
"lbu $2, %lo(D_800A64A9)($2)\n"
"nop\n"
"beqz $2, 1f\n"
" nop\n"
"lui $2, %hi(D_800AE618)\n"
"lw $2, %lo(D_800AE618)($2)\n"
"nop\n"
"sll $3, $2, 1\n"
"addu $3, $3, $2\n"
"srl $2, $3, 31\n"
"addu $3, $3, $2\n"
"sra $3, $3, 1\n"
"lui $1, %hi(D_8007881C)\n"
"addu $1, $1, $5\n"
"sh $3, %lo(D_8007881C)($1)\n"
"j GS_001_OBJ_5D0\n"
" nop\n"
"1:\n"
"lui $2, %hi(D_800AE618)\n"
"lw $2, %lo(D_800AE618)($2)\n"
"lui $1, %hi(D_8007881C)\n"
"addu $1, $1, $5\n"
"sh $2, %lo(D_8007881C)($1)\n"
".set\treorder\n"
".end\tGsSortClear\n"
);
__asm__(
".text\n"
".align\t2\n"
".globl\tGS_001_OBJ_5D0\n"
".ent\tGS_001_OBJ_5D0\n"
"GS_001_OBJ_5D0:\n"
".set\tnoreorder\n"
"lui $2, %hi(D_80078810)\n"
"addiu $2, $2, %lo(D_80078810)\n"
"lui $5, %hi(D_800C7C74)\n"
"lh $5, %lo(D_800C7C74)($5)\n"
"lw $4, 16($7)\n"
"sll $5, $5, 4\n"
"jal AddPrim\n"
"addu $5, $5, $2\n"
"lw $31, 16($sp)\n"
"addiu $sp, $sp, 24\n"
"jr $31\n"
"nop\n"
".set\treorder\n"
".end\tGS_001_OBJ_5D0\n"
);
src/libgpu2.c
+213
View File
@@ -0,0 +1,213 @@
#include "common.h"
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059234);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_11C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_16C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetGraphReverse);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_210);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_800594CC);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetGraphQueue);
INCLUDE_ASM("asm/nonmatchings/libgpu2", GetGraphType);
INCLUDE_ASM("asm/nonmatchings/libgpu2", GetGraphDebug);
INCLUDE_ASM("asm/nonmatchings/libgpu2", DrawSyncCallback);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059658);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_800596F4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059760);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_604);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_640);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059888);
INCLUDE_ASM("asm/nonmatchings/libgpu2", ClearImage2);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_800599B8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", StoreImage);
INCLUDE_ASM("asm/nonmatchings/libgpu2", MoveImage);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_8F4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", ClearOTag);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059BFC);
INCLUDE_ASM("asm/nonmatchings/libgpu2", DrawPrim);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059CF4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059D68);
INCLUDE_ASM("asm/nonmatchings/libgpu2", DrawOTagEnv);
INCLUDE_ASM("asm/nonmatchings/libgpu2", GetDrawEnv);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_80059FC0);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_E34);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_F00);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_F24);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_F58);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_FD8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1034);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_11C0);
INCLUDE_ASM("asm/nonmatchings/libgpu2", GetDispEnv);
INCLUDE_ASM("asm/nonmatchings/libgpu2", GetODE);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetTexWindow);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetDrawArea);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetDrawOffset);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetPriority);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005A600);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SetDrawEnv);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1578);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005A870);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1790);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_18AC);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005AB00);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_191C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005AB58);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1964);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_19A4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_19D8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005AC24);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1A30);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1A70);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1AA4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005ACF0);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1AF0);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005AD34);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1B78);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005ADB8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1BC4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1C28);
void SYS_OBJ_1C2C(void) {
}
INCLUDE_ASM("asm/nonmatchings/libgpu2", _status);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005AE80);
INCLUDE_ASM("asm/nonmatchings/libgpu2", _clr);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1D84);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1DC0);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1F64);
INCLUDE_ASM("asm/nonmatchings/libgpu2", _dws);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_1FF4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_202C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_21A4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", _drs);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_222C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_2264);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_242C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005B684);
INCLUDE_ASM("asm/nonmatchings/libgpu2", _getctl);
INCLUDE_ASM("asm/nonmatchings/libgpu2", _cwb);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005B710);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005B75C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", _addque);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005B7B0);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005B7E4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_25C8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_26EC);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_27A8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_283C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005BA90);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005BD7C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_2C6C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005BED8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_2CC4);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_2CDC);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_2DD8);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005C020);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005C054);
INCLUDE_ASM("asm/nonmatchings/libgpu2", SYS_OBJ_2F7C);
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005C1C0);
void SYS_OBJ_3060(void) {
}
INCLUDE_ASM("asm/nonmatchings/libgpu2", func_8005C29C);
src/libgs8.c
+13
View File
@@ -0,0 +1,13 @@
#include "common.h"
INCLUDE_ASM("asm/nonmatchings/libgs8", func_800525DC);
INCLUDE_ASM("asm/nonmatchings/libgs8", func_80052654);
INCLUDE_ASM("asm/nonmatchings/libgs8", GsInitGraph2);
INCLUDE_ASM("asm/nonmatchings/libgs8", func_8005283C);
INCLUDE_ASM("asm/nonmatchings/libgs8", GsSortClear);
INCLUDE_ASM("asm/nonmatchings/libgs8", GS_001_OBJ_5D0);
src/sgap_6.c
-153
View File
@@ -4,156 +4,3 @@ void func_8003FA54(void) {
extern u16 D_800A5F60;
D_800A5F60 = 0;
}
extern s32 D_800BA1B0;
extern u8 D_800AE614;
extern u32 D_800C7EC0[16];
extern u16 D_8007958E;
extern u8 D_800BA100;
extern u8 D_800795A3;
extern u16 D_800C7F04;
extern u16 D_800C7F06;
extern s16 D_800795A4;
extern s16 D_800795B0;
extern u8 D_800A4620;
extern u16 D_80078D8C;
extern u16 D_80078D8E;
extern u16 D_80078D90;
extern u16 D_80078D92;
extern u16 D_800B9B58;
extern u16 D_800B9B5C;
extern u16 D_800B9B5E;
extern void (*D_800BA0D0)(s32);
extern void (*D_800AE828)(s32);
extern u16* D_8006B8FC;
extern void func_8003FE18(s32, u32);
void func_8003FA64(void) {
s32 pad[2]; /* frame pad: target vars=16 (cookbook §226 / §162i1 dead-local lever) */
s32 i = 0;
s32 n;
u32 mask;
u32 off;
u8* fp;
u16* pa2;
u16* pa3;
u16* pt0;
u16* pt1;
u16* pend;
u16* pb;
u16 r1, r2, r3, r4, r5, r6;
u16* dp;
D_800BA1B0 = (D_800BA1B0 + 1) & 0xF;
D_800C7EC0[D_800BA1B0] = 0;
n = D_800AE614;
if (n > 0) {
u32* mp = &D_800C7EC0[D_800BA1B0];
u32 one = 1;
s32 count = D_800AE614;
u16* vp = D_8006B8FC;
u32 off0 = 0;
do {
u16 val = *(u16*)((char*)vp + 0xC);
*(u16*)((char*)&D_8007958E + off0) = val;
if (val == 0) {
*mp |= one << i;
}
vp += 8;
off0 += 0x34;
i++;
} while (i < count);
}
i = 0;
if (D_800BA100 == 0) {
mask = -1;
do {
mask &= D_800C7EC0[i];
i++;
} while (i < 0xF);
for (i = 0; i < (s32)D_800AE614; i++) {
u32 one = 1;
u32 two = 2;
off = i * 0x34;
if ((mask & (one << i)) != 0) {
if (*(u8*)((char*)&D_800795A3 + off) == two) {
func_8003FE18(0, 0xFFFFFF);
}
*(u8*)((char*)&D_800795A3 + off) = 0;
}
}
i = 0;
}
D_80078D8C &= ~D_800C7F04;
D_80078D8E &= ~D_800C7F06;
off = 0;
do {
if (*(s16*)((char*)&D_800795A4 + off) != 0) {
D_800BA0D0(i);
}
if (*(s16*)((char*)&D_800795B0 + off) != 0) {
D_800AE828(i);
}
i++;
off += 0x34;
} while (i < 0x18);
fp = &D_800A4620;
pb = &D_800B9B58;
pt1 = pb + 5;
pt0 = pb + 4;
{
u32 off4 = 0;
pa3 = pb + 1;
pa2 = pb;
pend = &D_800A4620 + 24;
do {
if ((*fp & 1) != 0) {
u16* d1 = (u16*)(off4 + (u32)D_8006B8FC);
*d1 = *pa2;
*(u16*)((char*)d1 + 2) = *pa3;
}
if ((*fp & 4) != 0) {
u16* d2 = (u16*)(off4 + (u32)D_8006B8FC);
*(u16*)((char*)d2 + 4) = *(u16*)((char*)&D_800B9B5C + off4);
}
if ((*fp & 8) != 0) {
u16* d3 = (u16*)(off4 + (u32)D_8006B8FC);
*(u16*)((char*)d3 + 6) = *(u16*)((char*)&D_800B9B5E + off4);
}
if ((*fp & 0x10) != 0) {
u16* d4 = (u16*)(off4 + (u32)D_8006B8FC);
*(u16*)((char*)d4 + 8) = *pt0;
*(u16*)((char*)d4 + 0xA) = *pt1;
}
*fp = 0;
fp++;
pt1 += 8;
pt0 += 8;
off4 += 0x10;
pa3 += 8;
pa2 += 8;
} while ((s32)fp < (s32)pend);
}
dp = D_8006B8FC;
r1 = D_800C7F04;
r2 = D_800C7F06;
r3 = D_80078D8C;
r4 = D_80078D8E;
r5 = D_80078D90;
r6 = D_80078D92;
D_800C7F04 = 0;
D_800C7F06 = 0;
D_80078D8C = 0;
D_80078D8E = 0;
*(u16*)((char*)dp + 0x18C) = r1;
*(u16*)((char*)dp + 0x18E) = r2;
*(u16*)((char*)dp + 0x188) = r3;
*(u16*)((char*)dp + 0x18A) = r4;
*(u16*)((char*)dp + 0x198) = r5;
*(u16*)((char*)dp + 0x19A) = r6;
}
src/snd12.c
+3
View File
@@ -0,0 +1,3 @@
#include "common.h"
INCLUDE_ASM("asm/nonmatchings/snd12", func_8003FA64);
tools/make_libgs.sh
+10 -6
View File
@@ -5,18 +5,22 @@
# developer who wants the real SDK objects linked. Needs the LIBGS ELF objects from
# tools/psyq_build_libs.sh LIBGS (which needs tools/psyq/lib40/LIBGS.LIB from the DTL-S2002 redump).
#
# These are the 31 libgs objects BFM links, forming 6 contiguous blocks across vram
# 0x80051804-0x80057928, separated by 5 non-libgs gaps (80 / 48 / 1536[GS_001] / 48 / 304 B):
# These are the 34 libgs objects BFM links, forming 8 contiguous blocks across vram
# 0x8005080C-0x80057928 (the 4 remaining non-libgs gaps are libgte objects, wired as libgte27-30):
# block 7: 2D_BG0 2D_BG1 (0x8005080C, S78 #3)
# block 1: 2D_COM0 (0x80051804)
# block 2: 2D_LIN0 2D_COM1 2D_SP0 (0x80051DC8)
# block 3: 2D_SP1 (0x80052460)
# block 8: GS_001 (0x800525DC, S78 #4)
# block 4: GS_002 GS_003 MATRIX GS_103 GS_104 GS_105 GS_106 GS_107 (0x80052BDC)
# block 5: GS_108 GS_109 (0x800538EC)
# block 6: GS_110 GS_119 GS_120 GS_121 GS_122 GS_123 GS_127 GS_131
# GS_133 GS_134 GS_135 GS_137 PRESET PRESET3 OBJT OBJT3 (0x80053AF8)
# GS_001 (the 1536 B gap before block 4) is EXCLUDED — psyq-obj-parser packs its scattered PSD*
# commons into .bss referenced via .bss+offset and 35 words differ at the global-zeroing run
# (cookbook §9.1 hard case); it stays an asm stub (gsgap3) and reproduces its bytes.
# GS_001 (the 1536 B gap before block 4) was EXCLUDED from Phase 7 to P31 S78 — psyq-obj-parser packs
# its scattered PSD* commons into one .bss referenced via .bss+offset at SIX bases (35 words differed
# at the global-zeroing run; cookbook §9.1 "hard case"). Since S78 #4 psyq_integrate splits such a
# section into per-base NOBITS pieces at link-prepare (tools/psyq_bss_split.py, cookbook §489), so
# GS_001 links byte-identical like any other object; it is block libgs8 (was the gsgap3 stub).
# The list is DISAMBIGUATED (session E): psyq_identify reports byte-identical-.text aliases — keep
# GS_131 (not RVWUNIT), GS_137 (not RVWLUNIT), PRESET3 (not PRESET2), OBJT3 (not OBJT2); the losers
# have a spurious .data/.rdata mismatch and do NOT link byte-exact. GS_106 is an 8-instruction object
@@ -28,7 +32,7 @@ cd "$(dirname "$0")/.."
LIBGS=.run/obj40/libgs
DST=.run/obj40/libgs_used
OBJS=(2D_COM0 2D_LIN0 2D_COM1 2D_SP0 2D_SP1 2D_BG0 2D_BG1 \
GS_002 GS_003 MATRIX GS_103 GS_104 GS_105 GS_106 GS_107 GS_108 GS_109 \
GS_001 GS_002 GS_003 MATRIX GS_103 GS_104 GS_105 GS_106 GS_107 GS_108 GS_109 \
GS_110 GS_119 GS_120 GS_121 GS_122 GS_123 GS_127 GS_131 GS_133 GS_134 GS_135 GS_137 \
PRESET PRESET3 OBJT OBJT3)
tools/make_snd_used.py
+8 -6
View File
@@ -6,12 +6,14 @@ they are linked as ONE combined region rather than two passes. This:
- places both libraries' objects (psyq_identify) and merges them by vram,
- for an aliased address (>1 object, same masked .text) picks the object whose linked .text
byte-matches the EXE (psyq_link.link_object) — the real one,
- EXCLUDES 4 ADDRESSES that don't reconcile in the combined region (ALL candidates there stay
- EXCLUDES 3 ADDRESSES that don't reconcile in the combined region (ALL candidates there stay
byte-identical stubs — excluding by address, not name, since the alias twin fails identically):
0x3C438 (S_R/S_W), 0x3D424 (S_GRMDT/FB/T) — scattered-.bss commons referenced at a minority
address the region's single defsym can't satisfy (cookbook §9.1, cross-object form),
0x3D94C (S_IH/UT_RON) — false placement: 0x3D94C is INSIDE libsnd SSSTART.o,
0x3FA64 (VM_F.o, 237 ins) — scattered-.bss commons (the one real value loss).
0x3C438 (S_R/S_W), 0x3D424 (S_GRMDT/FB/T) — commons referenced at a minority address the
region's single defsym can't satisfy (cookbook §9.1, cross-object form; these objects have NO
.bss of their own — psyq_bss_probe — so the S78 split does not apply to them),
0x3D94C (S_IH/UT_RON) — false placement: 0x3D94C is INSIDE libsnd SSSTART.o.
0x3FA64 (VM_F.o, 237 ins) was the 4th exclusion (scattered-.bss commons) until P31 S78 #4: its
`.bss` is now SPLIT at link-prepare (psyq_bss_split, cookbook §489) and it links as snd12.
- copies the survivors into .run/obj40/snd_used.
The build is byte-identical with OR without snd_used (stub fallback), so a fresh clone need not run
@@ -28,7 +30,7 @@ from psyq_link import link_object # noqa: E402
EXE = "extracted/retail/SLUS_007.26"
VRAM_BASE = 0x8000F800
RLO, RHI = 0x8003A444, 0x8004239C
EXCLUDE_ADDR = {0x8003C438, 0x8003D424, 0x8003D94C, 0x8003FA64} # see module docstring
EXCLUDE_ADDR = {0x8003C438, 0x8003D424, 0x8003D94C} # see module docstring (VM_F 0x8003FA64 rejoined S78 #4)
def place(lib, exe, vram_base):
tools/psyq_bss_probe.py
+60 -99
View File
@@ -1,88 +1,73 @@
#!/usr/bin/env python3
"""psyq_bss_probe.py — can this PsyQ object's scattered `.bss` be SPLIT and placed? (P31 S77)
"""psyq_bss_probe.py — can this PsyQ object's scattered `.bss` be SPLIT and placed? (P31 S77; S78 #4: yes, and it is)
THE WALL THIS INTERROGATES. `config/splat.us.exe.yaml` has excluded four library objects from the
THE WALL THIS INTERROGATES. `config/splat.us.exe.yaml` had excluded library objects from the
LINKED build since Phase 8 with one recorded reason:
SYS.o EXCLUDED — scattered-.bss commons (the GS_001 class: SYS references .bss by section+offset
but the original linker scattered the commons across 0x80078xxx/0x800c5xxx, so no single NOLOAD
base reproduces it)
That reason is CORRECT and this tool confirms it — §9.2's "weaken the `.bss` symbol and `--defsym`
it" cannot reach these sites, because a `R_MIPS_HI16/LO16` against the bare `.bss` SECTION has no
name to defsym. But "no single base" is not the same claim as "unlinkable", and nobody had asked
the next question: **are the offset ranges behind each base DISJOINT?** If they are, `.bss` splits
into pieces and each piece gets its own NOLOAD base.
That reason is CORRECT — §9.2's "weaken the `.bss` symbol and `--defsym` it" cannot reach these
sites, because a `R_MIPS_HI16/LO16` against the bare `.bss` SECTION has no name to defsym. But "no
single base" is not the same claim as "unlinkable", and nobody had asked the next question: **can the
section be cut into pieces that each have ONE base?** This probe asked it (S77) and
`tools/psyq_bss_split.py` (S78 #4) answers it at link-prepare time for every object in the build.
MEASURED ON SYS.o (the biggest, 3,109 ins — `src/800c.c` is 100% this object):
WHAT "SPLITTABLE" MEANS NOW (the S77 test was too strict — corrected S78). The S77 probe grouped the
references BY BASE and required the per-base offset ranges to be disjoint. That verdict was right
for SYS.o and VM_F.o (two bases, two ranges) and WRONG for GS_001.o, which it called "5 interleaved
bases": walked in OFFSET order, GS_001's references form six single-base RUNS, and every cut between
two runs lands on a symbol start (PSDBASEX, CLIP2, PSDBASEY, POSITION, GsDRAWENV) — exactly what
"the linker scattered the commons" predicts. The linker scattered SYMBOLS, so the right question is
whether the runs can be tiled at symbol boundaries without a sized symbol straddling a cut. All
three objects can; all three link byte-identical through the split (psyq_link.py PASS).
base 0x80078830 : 23 refs, .bss offsets 0x0000..0x0044
base 0x800c53cc : 9 refs, .bss offsets 0x0148..0x0150 -> DISJOINT, split at 0x148
and only `.text` references `.bss` at all (`.data` has zero such relocs), so the split covers every
reference in the object. The wall is real and breakable — two different things.
HOW IT DERIVES THE BASE — from the bytes, not from a config. For each HI16/LO16 pair against
`.bss`, the OBJECT's immediates encode the addend and the GAME's immediates encode the resolved
address; `base = resolved − addend`. Grouping by base gives the scatter, and the offsets behind
each group give the split points. The object's placement in the game is found by masking every
relocated field and searching for the unique match, so a wrong `--vram` cannot fake a clean answer
(and the search doubles as proof the object is really there).
HOW IT DERIVES THE BASE — from the bytes, not from a config. For each HI16/LO16 pair against the
section symbol, the OBJECT's immediates encode the addend and the GAME's immediates encode the
resolved address; `base = resolved − addend`. Grouping by base shows the scatter; the runs give the
cut points. The object's placement in the game is found by masking every relocated field and
searching for the unique match, so a wrong `--vram` cannot fake a clean answer (and the search
doubles as proof the object is really there). The analysis itself lives in `psyq_bss_split.analyze`
(R33: one implementation; this file only extracts, locates and reports).
tools/psyq_bss_probe.py --archive tools/psyq/lib40_elf/LIBGPU.a --object SYS.o
tools/psyq_bss_probe.py --archive ... --object GS_001.o --vram 0x800525DC
"""
import argparse
import collections
import os
import re
import struct
import subprocess
import sys
import tempfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from psyq_bss_split import Elf32, SHT_NOBITS, R_MIPS_HI16, R_MIPS_LO16, SplitRefused, analyze, report # noqa: E402
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
OBJDUMP = "mipsel-linux-gnu-objdump"
EXE = os.path.join(REPO, "extracted/retail/SLUS_007.26")
EXE_VRAM0 = 0x8000F800 # file offset 0 -> vram (the 0x800 PS-X EXE header sits below .text)
R_MIPS_26, R_MIPS_32 = 4, 2
def _od(*args):
return subprocess.run([OBJDUMP, *args], capture_output=True, text=True).stdout
def sections(obj):
out = {}
for m in re.finditer(r"^\s*\d+\s+(\.\S+)\s+([0-9a-f]+)\s+\S+\s+\S+\s+([0-9a-f]+)", _od("-h", obj), re.M):
out[m.group(1)] = (int(m.group(2), 16), int(m.group(3), 16)) # (size, file offset)
return out
def relocs(obj, section=".text"):
body = re.search(r"RELOCATION RECORDS FOR \[%s\]:(.*?)(?=\nRELOCATION RECORDS|\Z)" % re.escape(section),
_od("-r", obj), re.S)
if not body:
return []
return [(int(m.group(1), 16), m.group(2), m.group(3))
for m in re.finditer(r"^([0-9a-f]+)\s+(R_MIPS_\w+)\s+(\S+)", body.group(1), re.M)]
def find_vram(text, rels, game):
def find_vram(e, game):
"""The unique game offset where this object's .text sits, with every relocated field masked.
Masking is what makes the search honest: relocated immediates DIFFER between the archive object
and the linked game, so an exact byte search would find nothing and a loose one would find noise.
A unique hit is also proof the object is present at all (R14)."""
ti = e.section_index(".text")
text = bytes(e.data[ti])
mask = bytearray(b"\xff" * len(text))
for off, kind, _t in rels:
if off + 4 <= len(mask):
if kind in ("R_MIPS_HI16", "R_MIPS_LO16"):
mask[off:off + 2] = b"\x00\x00" # low half-word (little-endian immediate)
elif kind == "R_MIPS_26":
mask[off:off + 4] = b"\x00\x00\x00\x00"
elif kind == "R_MIPS_32":
mask[off:off + 4] = b"\x00\x00\x00\x00"
tb, mb = bytes(text), bytes(mask)
for ri, rl in e.rels.items():
if e.sh[ri]["info"] != ti:
continue
for off, _sym, typ in rl:
if off + 4 <= len(mask):
if typ in (R_MIPS_HI16, R_MIPS_LO16):
mask[off:off + 2] = b"\x00\x00" # low half-word (little-endian immediate)
elif typ in (R_MIPS_26, R_MIPS_32):
mask[off:off + 4] = b"\x00\x00\x00\x00"
tb, mb = text, bytes(mask)
hits = []
for base in range(0, len(game) - len(tb), 4):
ok = True
@@ -108,68 +93,44 @@ def main():
ap.add_argument("--vram", default=None, help="skip the search; the object's .text vram")
a = ap.parse_args()
with tempfile.TemporaryDirectory() as td:
with tempfile.TemporaryDirectory(dir=os.path.join(REPO, ".run")) as td:
subprocess.run(["ar", "x", os.path.abspath(a.archive), a.object], cwd=td, check=True)
obj = os.path.join(td, a.object)
secs = sections(obj)
if ".text" not in secs:
e = Elf32(open(obj, "rb").read())
names = [e.secname(i) for i in range(len(e.sh))]
if ".text" not in names:
sys.exit("%s: no .text section" % a.object)
if ".bss" not in secs:
nobits = [i for i, s in enumerate(e.sh) if s["type"] == SHT_NOBITS]
tsize = e.sh[e.section_index(".text")]["size"]
if not nobits:
# NOT AN ERROR, AND SAYING SO MATTERS. The first version exited "no .text/.bss" here,
# which reads as a malformed object; it actually means the scattered-.bss wall CANNOT
# apply. Measured: 2D_BG0.o and VM_NO1.o both land here, and both are excluded in
# apply. Measured: 2D_BG0.o and VM_NO1.o both land here, and both were excluded in
# splat.us.exe.yaml under a reason that is about .bss. (R61a: a tool must not report a
# clean answer as a failure.)
print("%s: .text %d ins, NO .bss SECTION — the scattered-.bss wall does not apply to "
"this object; whatever excludes it is a different reason."
% (a.object, secs[".text"][0] // 4))
"this object; whatever excludes it is a different reason." % (a.object, tsize // 4))
return 0
tsize, toff = secs[".text"]
text = open(obj, "rb").read()[toff:toff + tsize]
rels = relocs(obj, ".text")
game = open(EXE, "rb").read()
if a.vram:
goff = int(a.vram, 0) - EXE_VRAM0
else:
hits = find_vram(text, rels, game)
hits = find_vram(e, game)
if len(hits) != 1:
sys.exit("%s: masked .text search found %d placements — pass --vram" % (a.object, len(hits)))
goff = hits[0]
print("%s: .text %d ins @ vram %#010x (.bss %d bytes)"
% (a.object, tsize // 4, goff + EXE_VRAM0, secs[".bss"][0]))
# every section's .bss references, so the split is provably complete (R32)
other = {s: sum(1 for _o, _k, t in relocs(obj, s) if t == ".bss")
for s in secs if s not in (".text", ".bss", ".note")}
by = collections.defaultdict(list)
i, rl = 0, [r for r in rels if r[2] == ".bss"]
while i < len(rl) - 1:
(o1, k1, _), (o2, k2, _) = rl[i], rl[i + 1]
if k1 == "R_MIPS_HI16" and k2 == "R_MIPS_LO16":
ohi = struct.unpack_from("<I", text, o1)[0] & 0xFFFF
olo = struct.unpack_from("<I", text, o2)[0] & 0xFFFF
addend = (ohi << 16) + ((olo ^ 0x8000) - 0x8000)
ghi = struct.unpack_from("<I", game, goff + o1)[0] & 0xFFFF
glo = struct.unpack_from("<I", game, goff + o2)[0] & 0xFFFF
resolved = (ghi << 16) + ((glo ^ 0x8000) - 0x8000)
by[resolved - addend].append(addend)
i += 2
else:
i += 1
if not by:
print(" no .bss HI16/LO16 pairs — nothing blocking a NOLOAD placement here")
print("%s: .text %d ins @ vram %#010x (%s)"
% (a.object, tsize // 4, goff + EXE_VRAM0,
", ".join("%s %d bytes" % (e.secname(i), e.sh[i]["size"]) for i in nobits)))
try:
plans = analyze(obj, goff + EXE_VRAM0, game, EXE_VRAM0, elf=e)
except SplitRefused as ex:
print(" SPLITTABLE: NO — %s" % ex)
return 1
if not plans:
print(" no references through a NOBITS section symbol — nothing blocking a NOLOAD placement here")
return 0
groups = sorted((b, min(v), max(v), len(v)) for b, v in by.items())
for b, lo, hi, n in groups:
print(" base %#010x : %2d ref(s), .bss offsets %#06x..%#06x" % (b, n, lo, hi))
disjoint = all(groups[i][2] < groups[i + 1][1] for i in range(len(groups) - 1))
extra = {s: n for s, n in other.items() if n}
print(" SPLITTABLE: %s%s" % (
"YES — ranges are disjoint, split at %s" % ", ".join(hex(g[1]) for g in groups[1:])
if disjoint else "NO — the offset ranges interleave; a split cannot separate them",
"" if not extra else " ⚠ but %s also reference .bss (%s) — those must be covered too"
% (",".join(extra), extra)))
print("\n".join(report(plans, a.object).splitlines()[1:])) # report()'s first line repeats the header
return 0
tools/psyq_bss_split.py
+469
View File
@@ -0,0 +1,469 @@
#!/usr/bin/env python3
"""psyq_bss_split.py — split a PsyQ object's scattered `.bss` into per-base pieces so it links byte-exact (P31 S78 #4).
THE WALL, RESTATED. psyq-obj-parser packs an object's common-style globals into ONE `.bss` section
with sequential offsets. The original PsyQ linker allocated those commons individually, so the game
has them at several unrelated addresses ("scattered commons", cookbook §9.1). A common referenced
BY NAME is recovered by name and `--defsym`'d (§9.2). But the compiler references the object's own
statics through the `.bss` SECTION SYMBOL plus an offset — a relocation with no name to defsym — and
the build can NOLOAD-place a section at only ONE base. `psyq_bss_probe.py` (§484) asked the question
nobody had: are the offsets behind each base DISJOINT? For SYS.o and VM_F.o they are, so the section
can be cut into pieces that each have a single base. This tool makes the cut.
WHAT IT DOES TO THE OBJECT (an ELF32 REL rewrite; no pyelftools — the project's readers are regexes
over readelf, and a WRITER needs the real structs anyway):
1. derives the bases FROM THE BYTES: for every HI16/LO16 pair against the section symbol,
addend = the object's immediates, resolved = the game's immediates at the same `.text` offset,
base = resolved − addend (the probe's arithmetic, R33: one implementation, the probe imports it);
2. walks the references in offset order; each maximal run with one base is a piece. A cut between
two runs SNAPS TO A SYMBOL START (the linker scattered SYMBOLS, so a piece that begins mid-symbol
is a fiction): SYS.o's second run begins inside `_que` (+0x148) and the piece begins at `_que`
(+0x144) — and `_que`, recovered independently BY NAME from SYS.o's four named references, is
0x800C5510 = base2 + 0x144. VM_F.o's second run begins at `_svm_sreg_buf` (+0x508), which 62
other sound objects recover to 0x800B9B58 = base2 + 0x508. Two unrelated oracles agree on both
cuts (R34). With no symbol start between the runs the cut is at the run's first reference;
3. REFUSES, loudly (R43), when the references cannot be tiled: a sized symbol straddling a cut (one
common referenced at two bases), a HI16 whose LO16s fall in different pieces, an orphan LO16, a
reference far outside the section, or a relocation type other than HI16/LO16/32 against the
section symbol. Coverage is asserted: every reference against the section symbol is classified
or the split fails (R32). Problems are FATAL ONLY WHEN A SPLIT IS NEEDED — an object one base
already serves passes through untouched, exactly as before this tool existed (R39: the first
build refused a libcd object for an end-of-buffer `.bss + size` reference it never needed split
for; 235 placed objects across the 9 curated dirs now pass with 0 refusals and exactly 3 splits).
GS_001.o, which the S77 probe called "5 interleaved bases, NOT splittable" by grouping per base,
is SIX symbol-aligned runs under this model and links byte-identical;
4. emits `.bss` = [0, s2) and new NOBITS sections `.bss2` = [s2, s3) …; moves each symbol at/after a
cut into its piece (value −= piece start); adds one LOCAL section symbol per piece (inserted with
the existing section symbols — every later symbol index in every REL entry is bumped); retargets
each reference to its piece's symbol and REWRITES ITS ADDEND IN PLACE: the HI16/LO16 immediates
in `.text` (hi' = (A'+0x8000)>>16, lo' = A' & 0xFFFF, A' = A − piece start) or the R_MIPS_32 word;
5. self-checks: the rewritten object's code/data differ from the original at exactly the retargeted
sites whose value changed, and nowhere else (the tool diffs its own artifact, R37).
WHERE IT RUNS. Not in the curated dirs — at LINK-PREPARE time, in the one path `psyq_link.link_object`
(per-object verify), `psyq_link_region.build_region` (region verify) and `psyq_integrate.integrate`
(the build) all share: `prepare_object()`. `classify()` then recovers one base per piece from the
piece's own section symbol and NOLOAD-places each — no config, no recorded split offset that could go
stale (R51). An object with a single base passes through untouched.
tools/psyq_bss_split.py <obj.o> --vram <text_vram> --exe <binary> --vram-base <hex> [-o out.o]
(no -o = report the plan only)
"""
import argparse
import os
import re
import struct
import sys
NOBITS_RE = re.compile(r"^\.s?bss\d*$") # .bss .sbss and the pieces this tool makes (.bss2 …)
SHT_SYMTAB, SHT_STRTAB, SHT_NOBITS, SHT_REL = 2, 3, 8, 9
STT_SECTION, STB_LOCAL = 3, 0
R_MIPS_32, R_MIPS_HI16, R_MIPS_LO16 = 2, 5, 6
RNAME = {2: "R_MIPS_32", 4: "R_MIPS_26", 5: "R_MIPS_HI16", 6: "R_MIPS_LO16"}
EHDR = struct.Struct("<16sHHIIIIIHHHHHH")
SHDR = struct.Struct("<IIIIIIIIII")
SYM = struct.Struct("<IIIBBH")
REL = struct.Struct("<II")
class SplitRefused(Exception):
"""The section's references cannot be tiled into single-base pieces. Say so; never place them wrongly."""
def s16(x):
x &= 0xFFFF
return x - 0x10000 if x & 0x8000 else x
def u16(buf, off):
return struct.unpack_from("<H", buf, off)[0]
def u32(buf, off):
return struct.unpack_from("<I", buf, off)[0]
def cstr(buf, off):
return bytes(buf[off:buf.index(b"\0", off)]).decode()
class Elf32:
"""Minimal ELF32 little-endian relocatable reader/writer: headers + contents, symbols, REL entries.
Sections keep their index order on write; file offsets are recomputed."""
def __init__(self, data):
f = EHDR.unpack_from(data, 0)
if f[0][:4] != b"\x7fELF" or f[0][4] != 1 or f[0][5] != 1:
raise ValueError("not an ELF32 little-endian file")
(self.ident, self.type, self.machine, self.version, self.entry, self.phoff, shoff, self.flags,
self.ehsize, self.phentsize, self.phnum, self.shentsize, shnum, self.shstrndx) = f
self.sh, self.data = [], []
for i in range(shnum):
name, typ, flg, addr, off, size, link, info, align, ent = SHDR.unpack_from(data, shoff + i * SHDR.size)
self.sh.append(dict(name=name, type=typ, flags=flg, addr=addr, size=size, link=link,
info=info, align=align, entsize=ent))
self.data.append(bytearray() if typ == SHT_NOBITS else bytearray(data[off:off + size]))
self.symidx = next(i for i, s in enumerate(self.sh) if s["type"] == SHT_SYMTAB)
st = self.sh[self.symidx]
self.strtab = st["link"]
self.syms = [list(SYM.unpack_from(self.data[self.symidx], k * SYM.size)) # [name value size info other shndx]
for k in range(st["size"] // SYM.size)]
self.rels = {} # rel-section idx -> [[r_offset, r_sym, r_type]]
for i, s in enumerate(self.sh):
if s["type"] == SHT_REL and s["link"] == self.symidx:
self.rels[i] = [[o, info >> 8, info & 0xFF]
for o, info in (REL.unpack_from(self.data[i], k * REL.size)
for k in range(s["size"] // REL.size))]
def secname(self, i):
return cstr(self.data[self.shstrndx], self.sh[i]["name"])
def section_index(self, name):
return next((i for i in range(len(self.sh)) if self.secname(i) == name), None)
def symname(self, k):
return cstr(self.data[self.strtab], self.syms[k][0])
def add_string(self, secidx, s):
off = len(self.data[secidx])
self.data[secidx] += s.encode() + b"\0"
return off
def serialize(self):
self.data[self.symidx] = bytearray(b"".join(SYM.pack(*y) for y in self.syms))
for i, rl in self.rels.items():
self.data[i] = bytearray(b"".join(REL.pack(o, (sym << 8) | typ) for o, sym, typ in rl))
out, offs = bytearray(EHDR.size), []
for i, s in enumerate(self.sh):
if i == 0:
offs.append(0)
continue
a = min(max(s["align"], 1), 16) if s["type"] != SHT_NOBITS else 4
while len(out) % a:
out.append(0)
offs.append(len(out))
if s["type"] != SHT_NOBITS:
out += self.data[i]
while len(out) % 4:
out.append(0)
shoff = len(out)
for i, s in enumerate(self.sh):
size = s["size"] if s["type"] == SHT_NOBITS else len(self.data[i])
out += SHDR.pack(s["name"], s["type"], s["flags"], s["addr"], offs[i], size,
s["link"], s["info"], s["align"], s["entsize"])
EHDR.pack_into(out, 0, self.ident, self.type, self.machine, self.version, self.entry, self.phoff,
shoff, self.flags, self.ehsize, self.phentsize, self.phnum, self.shentsize,
len(self.sh), self.shstrndx)
return bytes(out)
class Piece:
__slots__ = ("index", "lo", "hi", "base", "name", "cut_sym", "shndx", "symidx", "refs")
def __init__(self, index, lo, hi, base, name, cut_sym):
self.index, self.lo, self.hi, self.base, self.name, self.cut_sym = index, lo, hi, base, name, cut_sym
self.shndx = self.symidx = None
self.refs = 0
@property
def addr(self): # where the piece lands in the target binary
return None if self.base is None else (self.base + self.lo) & 0xFFFFFFFF
def _plan_section(e, bi, ssym, text_vram, exe, vram_base):
"""Analyse one NOBITS section referenced through its section symbol `ssym`.
Returns dict(shndx, size, bases, pieces, actions, nrefs, by_section) — `pieces` has ONE entry when
a single base covers every reference (no split needed)."""
sec = e.secname(bi)
size = e.sh[bi]["size"]
text_i = e.section_index(".text")
points, clusters, nrefs, by_section, problems = [], [], 0, {}, []
# Problems found while walking the references are FATAL only if a split turns out to be needed:
# an object one base already serves must pass through exactly as it did before this tool existed
# (R39 — a refusal check ships only after the previously-passing population still passes).
for ri, rl in e.rels.items():
tgt = e.sh[ri]["info"]
body = e.data[tgt]
tname = e.secname(tgt)
cluster = None # [his, los, closed] — LO16s after a HI16 group share it (one lui, several uses)
for k, (off, sym, typ) in enumerate(rl):
if sym != ssym:
continue
nrefs += 1
by_section[tname] = by_section.get(tname, 0) + 1
if typ == R_MIPS_HI16:
if tgt != text_i:
problems.append(f"{sec}: R_MIPS_HI16 in non-text section {tname}")
continue
if cluster is None or cluster[2]:
cluster = [[], [], False, ri] # his, los, closed, rel-section index
clusters.append(cluster)
cluster[0].append(k)
elif typ == R_MIPS_LO16:
if tgt != text_i or cluster is None or not cluster[0]:
problems.append(f"{sec}: orphan R_MIPS_LO16 at {tname}+{off:#x} (no preceding HI16)")
continue
cluster[2] = True
hi_off = rl[cluster[0][-1]][0]
A = (u16(body, hi_off) << 16) + s16(u16(body, off))
site = text_vram + hi_off - vram_base
resolved = (u16(exe, site) << 16) + s16(u16(exe, text_vram + off - vram_base))
base = (resolved - A) & 0xFFFFFFFF
cluster[1].append((k, A, base))
points.append((A, base, ri, k))
elif typ == R_MIPS_32:
A = u32(body, off)
points.append((A, None, ri, k)) # no base derivable without the section's own placement; tiled by offset
else:
problems.append(f"{sec}: unsupported relocation {RNAME.get(typ, typ)} against the section symbol at {tname}+{off:#x}")
for A, base, ri, k in points:
# An addend == size is an END pointer (`&buf[N]`, a loop bound) and belongs to the last piece;
# small negative addends are gcc's folded `&arr[i-1]`. Only an addend far outside the section
# can be a mis-paired HI16/LO16, and THAT is what is refused (libc2 SETJMP-style end pointers
# at .bss+size hit the stricter `< size` form on the first build, S78 #4).
if not (-0x8000 <= A <= size + 0x8000):
problems.append(f"{sec}: reference at {e.secname(e.sh[ri]['info'])}+{e.rels[ri][k][0]:#x} has addend {A:#x} far outside the section (size {size:#x}) — mis-paired relocation?")
based = sorted((A, b) for A, b, _, _ in points if b is not None)
# maximal runs of one base, in offset order
runs = [] # [lo_addend, hi_addend, base, count]
for A, b in based:
if runs and runs[-1][2] == b:
runs[-1][1] = A
runs[-1][3] += 1
else:
runs.append([A, A, b, 1])
bases = {}
for A, b in based:
lo, hi, n = bases.get(b, (A, A, 0))
bases[b] = (min(lo, A), max(hi, A), n + 1)
syms = sorted((y[1], y[2], e.symname(k)) for k, y in enumerate(e.syms)
if y[5] == bi and (y[3] & 0xF) != STT_SECTION)
pieces = []
if len(runs) > 1 and problems:
raise SplitRefused(f"{sec}: needs splitting ({len(runs)} runs) but the references could not all be "
f"classified — {problems[0]}" + (f" (+{len(problems) - 1} more)" if len(problems) > 1 else ""))
if runs:
cuts = [(0, None)]
for prev, cur in zip(runs, runs[1:]):
cands = [(v, nm) for v, _sz, nm in syms if prev[1] < v <= cur[0]]
v, nm = max(cands) if cands else (cur[0], None)
for sv, ssz, snm in syms:
if ssz and sv < v < sv + ssz:
raise SplitRefused(f"{sec}: symbol `{snm}` [{sv:#x},{sv+ssz:#x}) straddles the cut at {v:#x} — "
f"one common is referenced at two bases ({prev[2]:#010x} and {cur[2]:#010x}); not splittable")
cuts.append((v, nm))
for j, (v, nm) in enumerate(cuts):
hi = cuts[j + 1][0] if j + 1 < len(cuts) else size
pieces.append(Piece(j, v, hi, runs[j][2], sec if j == 0 else f"{sec}{j + 1}", nm))
def piece_of(A):
return max((p for p in pieces if p.lo <= A), key=lambda p: p.lo, default=pieces[0])
actions = [] # (rel_sec, entry, piece, new_field_value)
for cl in clusters:
his, los, ri = cl[0], cl[1], cl[3]
target, hi_new = None, None
for k, A, base in los:
p = piece_of(A)
A2 = A - p.lo
lo2 = A2 & 0xFFFF
hi2 = ((A2 + 0x8000) >> 16) & 0xFFFF
if target is None:
target, hi_new = p, hi2
elif p is not target or hi2 != hi_new:
raise SplitRefused(f"{sec}: one HI16 is shared by LO16s that need different pieces/high halves "
f"(addends {[a for _, a, _ in los]}) — not splittable")
actions.append((ri, k, p, lo2))
for k in his:
actions.append((ri, k, target, hi_new))
return dict(shndx=bi, size=size, bases=bases, runs=runs, pieces=pieces, actions=actions,
nrefs=nrefs, by_section=by_section, points=points, syms=syms, problems=problems)
def analyze(obj_path, text_vram, exe, vram_base, elf=None):
"""{section name: plan} for every NOBITS section referenced through its section symbol."""
e = elf or Elf32(open(obj_path, "rb").read())
plans = {}
for bi, s in enumerate(e.sh):
if s["type"] != SHT_NOBITS:
continue
ssym = next((k for k, y in enumerate(e.syms) if (y[3] & 0xF) == STT_SECTION and y[5] == bi), None)
if ssym is None:
continue
plan = _plan_section(e, bi, ssym, text_vram, exe, vram_base)
if plan["nrefs"]:
plans[e.secname(bi)] = plan
return plans
def needs_split(plans):
return {sec: p for sec, p in plans.items() if len(p["pieces"]) > 1}
def describe(plans):
"""One line per split section, for build logs."""
out = []
for sec, p in needs_split(plans).items():
parts = []
for pc in p["pieces"]:
cut = f" (cut at `{pc.cut_sym}`)" if pc.cut_sym else ""
parts.append(f"{pc.name} [{pc.lo:#06x},{pc.hi:#06x}) @{pc.addr:#010x}{cut}")
out.append(f"{sec} split ×{len(p['pieces'])}: " + " · ".join(parts))
return "; ".join(out)
def split(obj_path, text_vram, exe, vram_base, out_path, plans=None):
"""Write the split object to out_path. Returns the plans (empty dict when nothing needed splitting
and nothing was written)."""
raw = open(obj_path, "rb").read()
e = Elf32(raw)
plans = plans if plans is not None else analyze(obj_path, text_vram, exe, vram_base, elf=e)
todo = needs_split(plans)
if not todo:
return {}
# 1. new NOBITS sections for pieces 2..n; shrink the original to piece 1
for sec, p in todo.items():
bi = p["shndx"]
for pc in p["pieces"][1:]:
addr = pc.addr
align = min(addr & -addr, 8) if addr else e.sh[bi]["align"]
pc.shndx = len(e.sh)
e.sh.append(dict(name=e.add_string(e.shstrndx, pc.name), type=SHT_NOBITS, flags=e.sh[bi]["flags"],
addr=0, size=pc.hi - pc.lo, link=0, info=0, align=max(align, 1), entsize=0))
e.data.append(bytearray())
e.sh[bi]["size"] = p["pieces"][0].hi
# 2. one LOCAL section symbol per new piece, inserted right after the existing section symbols
# (locals precede globals); every later symbol index in every REL entry moves up.
insert_at = 1 + max(k for k, y in enumerate(e.syms) if (y[3] & 0xF) == STT_SECTION)
n_new = sum(len(p["pieces"]) - 1 for p in todo.values())
st = e.sh[e.symidx]
if insert_at > st["info"]:
raise SplitRefused("symbol table: section symbols are not in the LOCAL range (sh_info=%d)" % st["info"])
for rl in e.rels.values():
for r in rl:
if r[1] >= insert_at:
r[1] += n_new
st["info"] += n_new
k = insert_at
for sec, p in todo.items():
for pc in p["pieces"][1:]:
e.syms.insert(k, [e.add_string(e.strtab, pc.name), 0, 0, (STB_LOCAL << 4) | STT_SECTION, 0, pc.shndx])
pc.symidx = k
k += 1
# 3. move the symbols that live in a later piece
for sec, p in todo.items():
bi = p["shndx"]
for y in e.syms:
if y[5] == bi and (y[3] & 0xF) != STT_SECTION:
pc = max((q for q in p["pieces"] if q.lo <= y[1]), key=lambda q: q.lo, default=p["pieces"][0])
if pc.index:
y[5], y[1] = pc.shndx, y[1] - pc.lo
# 4. retarget + rewrite the addend field of every reference in a later piece
changed = {} # (target sec idx, offset) -> new word
for sec, p in todo.items():
text_i = e.section_index(".text")
for ri, k, pc, field in p["actions"]:
off, sym, typ = e.rels[ri][k]
tgt = e.sh[ri]["info"]
pc.refs += 1
if pc.index == 0:
continue
e.rels[ri][k][1] = pc.symidx
w = u32(e.data[tgt], off)
nw = (w & 0xFFFF0000) | field
struct.pack_into("<I", e.data[tgt], off, nw)
if nw != w:
changed[(tgt, off)] = nw
for A, base, ri, k in p["points"]: # R_MIPS_32 words (data pointers into the section)
off, sym, typ = e.rels[ri][k]
if typ != R_MIPS_32:
continue
pc = max((q for q in p["pieces"] if q.lo <= A), key=lambda q: q.lo, default=p["pieces"][0])
pc.refs += 1
if pc.index == 0:
continue
tgt = e.sh[ri]["info"]
e.rels[ri][k][1] = pc.symidx
struct.pack_into("<I", e.data[tgt], off, A - pc.lo)
changed[(tgt, off)] = A - pc.lo
blob = e.serialize()
open(out_path, "wb").write(blob)
# 5. self-check: only the retargeted fields changed, in the code/data sections; re-read what we wrote
o, n = Elf32(raw), Elf32(blob)
for i in range(len(o.sh)):
if o.sh[i]["type"] in (SHT_SYMTAB, SHT_STRTAB, SHT_REL, SHT_NOBITS) or i == o.shstrndx:
continue
a, b = o.data[i], n.data[i]
if len(a) != len(b):
raise AssertionError(f"self-check: section {o.secname(i)} changed size")
diff = {j for j in range(0, len(a), 4) if a[j:j + 4] != b[j:j + 4]}
want = {off for (t, off) in changed if t == i}
if diff != want:
raise AssertionError(f"self-check: {o.secname(i)} differs at {sorted(diff ^ want)[:8]} beyond the retargeted sites")
for sec, p in todo.items():
ni = n.section_index(sec)
if n.sh[ni]["size"] != p["pieces"][0].hi or any(n.section_index(pc.name) is None for pc in p["pieces"][1:]):
raise AssertionError(f"self-check: {sec} pieces not present as written")
return plans
def prepare_object(obj_path, text_vram, exe, vram_base, outdir):
"""The object to LINK: the original path, or a split copy under outdir when a NOBITS section is
referenced at more than one base. SplitRefused propagates — a build must not place bytes wrongly."""
plans = analyze(obj_path, text_vram, exe, vram_base)
if not needs_split(plans):
return obj_path, {}
os.makedirs(outdir, exist_ok=True)
out = os.path.join(outdir, os.path.basename(obj_path))
split(obj_path, text_vram, exe, vram_base, out, plans)
return out, plans
def report(plans, name):
lines = []
for sec, p in plans.items():
lines.append(f"{name}: {sec} {p['size']:#x} B, {p['nrefs']} section-symbol ref(s) in "
+ ", ".join(f"{s} ×{n}" for s, n in p["by_section"].items()))
for b, (lo, hi, n) in sorted(p["bases"].items()):
lines.append(f" base {b:#010x} : {n:2d} ref(s), {sec} offsets {lo:#06x}..{hi:#06x}")
if len(p["pieces"]) > 1:
lines.append(f" SPLITTABLE: YES — {len(p['pieces'])} pieces:")
for pc in p["pieces"]:
cut = f" cut snapped to `{pc.cut_sym}`" if pc.cut_sym else ("" if pc.index == 0 else " (no symbol start between the runs)")
lines.append(f" {pc.name:7s} [{pc.lo:#06x},{pc.hi:#06x}) -> {pc.addr:#010x}{cut}")
elif p["runs"]:
lines.append(" single base — no split needed")
else:
lines.append(" no HI16/LO16 pairs against the section symbol — nothing to place")
return "\n".join(lines)
def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("obj")
ap.add_argument("--vram", required=True, help="the object's .text vram in the target binary")
ap.add_argument("--exe", required=True, help="target binary path")
ap.add_argument("--vram-base", required=True, help="fileoff->vram delta of the target binary")
ap.add_argument("-o", "--out", help="write the split object here (omit to report only)")
a = ap.parse_args()
exe = open(a.exe, "rb").read()
name = os.path.basename(a.obj)
try:
plans = analyze(a.obj, int(a.vram, 0), exe, int(a.vram_base, 0))
except SplitRefused as ex:
print(f"{name}: REFUSED — {ex}")
return 2
if not plans:
print(f"{name}: no NOBITS section is referenced through its section symbol — the scattered-.bss wall does not apply")
return 0
print(report(plans, name))
if a.out and needs_split(plans):
split(a.obj, int(a.vram, 0), exe, int(a.vram_base, 0), a.out, plans)
n = sum(pc.refs for p in needs_split(plans).values() for pc in p["pieces"][1:])
print(f" wrote {a.out}: {n} reference(s) retargeted; {describe(plans)}")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/psyq_integrate.py
+20 -7
View File
@@ -3,8 +3,10 @@
Run after `make extract`. Given a library's ELF objects and the stub subsegment(s) splat emitted
for the library's text region(s), this:
1. prepares each object (objcopy: .text/.data/.rdata/.bss align=4; weaken every .bss/.sbss
named symbol so a strong --defsym beats its scattered weak def) -> <objdir>/*.o
1. prepares each object (P31 S78 #4: an object whose .bss is referenced at several bases through
the section symbol is first split into per-base NOBITS pieces by psyq_bss_split, §489; then
objcopy: .text/.data/.rdata/.bss align=4; weaken every .bss/.sbss named symbol so a strong
--defsym beats its scattered weak def) -> <objdir>/*.o
2. rewrites the splat linker script: each `build/src/<stub>.o(.text);` line is replaced by the
real objects' `<objdir>/<obj>.o(.text);` lines (the library's objects form one contiguous
block per stub, so concatenation places them at their exact vrams); each object's
@@ -30,6 +32,7 @@ import glob, os, re, subprocess, sys, tempfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from psyq_link import recover_sym_addrs, AS, sh, DATA_SECTIONS
from psyq_link_region import classify, placement
from psyq_bss_split import prepare_object, describe # P31 S78 #4: scattered-.bss split at link-prepare
def stub_ranges(yaml_path, stubs, vram_base):
@@ -120,9 +123,20 @@ def integrate(elf_dir, ld_path, objdir, syms_path, stubs, lo=None, hi=None,
f"stub subsegs of {os.path.basename(elf_dir)} — byte-placed but NOT wired (LINKED residue):")
for nm, v, n in residue:
print(f" 0x{v:08X} {nm:14s} {n:5d} ins")
recovered, weaken_by, bases_by = {}, {}, {}
recovered, weaken_by, bases_by, srcs = {}, {}, {}, {}
os.makedirs(objdir, exist_ok=True)
for name, (vram, _) in order:
bases, weaken, sym_addr = classify(os.path.join(elf_dir, name), vram, exe, vram_base)
# P31 S78 #4 (cookbook §489): an object whose `.bss` is referenced through the SECTION symbol
# at more than one base (SYS.o, VM_F.o, GS_001.o — the §9.1 "scattered commons" exclusions)
# is rewritten into per-base NOBITS pieces (.bss/.bss2/…) under <objdir>/.split BEFORE
# classify(), whose per-piece section symbols then recover one base each. Derived from the
# bytes on every run — nothing recorded that could go stale (R51). A refusal is fatal (R43).
src, plans = prepare_object(os.path.join(elf_dir, name), vram, exe, vram_base,
os.path.join(objdir, ".split"))
if plans:
print(f" .. {name}: {describe(plans)} — scattered commons tiled at link-prepare")
srcs[name] = src
bases, weaken, sym_addr = classify(src, vram, exe, vram_base)
bases_by[name], weaken_by[name] = bases, weaken
for s, a in sym_addr.items():
if not s.startswith("."):
@@ -143,14 +157,13 @@ def integrate(elf_dir, ld_path, objdir, syms_path, stubs, lo=None, hi=None,
m = re.match(r"(\w+)\s*=\s*0x([0-9A-Fa-f]+)", ln)
if m:
curated_by_addr.setdefault(int(m.group(2), 16), m.group(1))
os.makedirs(objdir, exist_ok=True)
for name, (vram, _) in order:
args = []
for S in (".text",) + DATA_SECTIONS:
args += ["--set-section-alignment", f"{S}=4"]
for w in weaken_by[name]:
args += ["--weaken-symbol", w]
defined = subprocess.run([f"{AS}nm", "--defined-only", os.path.join(elf_dir, name)],
defined = subprocess.run([f"{AS}nm", "--defined-only", srcs[name]],
capture_output=True, text=True).stdout
for dl in defined.splitlines():
parts = dl.split()
@@ -162,7 +175,7 @@ def integrate(elf_dir, ld_path, objdir, syms_path, stubs, lo=None, hi=None,
if cn and cn != sym:
args += ["--redefine-sym", f"{sym}={cn}"]
print(f" == {name}: exported `{sym}` @0x{addr:08X} is curated `{cn}` -> redefined (R15)")
sh(f"{AS}objcopy", *args, os.path.join(elf_dir, name), os.path.join(objdir, name))
sh(f"{AS}objcopy", *args, srcs[name], os.path.join(objdir, name))
if yaml_path:
# Stub<->objects by SUBSEG RANGE (P31 S78), not by run-contiguity: two adjacent stub subsegs are
tools/psyq_link.py
+55 -29
View File
@@ -29,6 +29,8 @@ Usage:
Importable: link_object(obj, text_vram, *, vram_base, exe_path|exe_bytes) -> dict(result).
"""
import struct, subprocess, sys, os, re, tempfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from psyq_bss_split import NOBITS_RE, SplitRefused, prepare_object, describe # P31 S78 #4: scattered-.bss split at link-prepare
# Phase 9: vram_base (the fileoff->vram delta) and the target binary are REQUIRED parameters —
# no EXE default an overlay could silently inherit. AS is the cross-toolchain prefix (universal).
@@ -196,9 +198,26 @@ def link_object(obj, text_vram, name=None, exe_bytes=None, *, vram_base, exe_pat
"""
name = name or os.path.basename(obj)
exe = exe_bytes if exe_bytes is not None else open(exe_path, "rb").read()
tsize = section_table(obj).get(".text", (0, 0))[0]
res = {"name": name, "text_vram": text_vram, "tsize": tsize, "split": ""}
with tempfile.TemporaryDirectory(dir=".run") as td:
# P31 S78 #4: an object whose `.bss` is referenced through the SECTION symbol at more than one
# base (scattered commons, §9.1) is split into per-base NOBITS pieces first — the same prepare
# step the region verify and the build use, so this verdict is the build's verdict. A refusal
# is reported as a FAIL with its reason, never as a crash (this is a verification tool).
try:
obj, plans = prepare_object(obj, text_vram, exe, vram_base, os.path.join(td, "split"))
except SplitRefused as ex:
res.update(ok=False, error=f"bss split refused: {ex}", externals={}, unrecovered=[],
rdata_vram=None, data_vram=None, bss_vram=None)
return res
res["split"] = describe(plans)
return _link_prepared(obj, text_vram, exe, vram_base, td, res)
def _link_prepared(obj, text_vram, exe, vram_base, td, res):
secs = section_table(obj)
tsize = secs.get(".text", (0, 0))[0]
res = {"name": name, "text_vram": text_vram, "tsize": tsize}
tsize = res["tsize"]
symtab = symbol_table(obj)
sym_addr = recover_sym_addrs(obj, text_vram, exe, vram_base)
@@ -210,10 +229,10 @@ def link_object(obj, text_vram, name=None, exe_bytes=None, *, vram_base, exe_pat
# section bases for placement: byte-search (initialised) or the section-symbol reloc.
bases = {}
for S in DATA_SECTIONS:
if S not in secs or secs[S][0] == 0:
for S in secs: # every data-like section, incl. the split pieces (.bss2 …)
if not (S in DATA_SECTIONS or NOBITS_RE.match(S)) or secs[S][0] == 0:
continue
b = unique_byte_vram(obj, S, exe, vram_base) if S not in (".bss", ".sbss") else None
b = unique_byte_vram(obj, S, exe, vram_base) if not NOBITS_RE.match(S) else None
if b is None:
b = sym_addr.get(S) # set iff the object referenced the section symbol
bases[S] = b
@@ -243,30 +262,35 @@ def link_object(obj, text_vram, name=None, exe_bytes=None, *, vram_base, exe_pat
res["externals"] = externals
res["unrecovered"] = sorted(undefined_syms(obj) - set(externals))
with tempfile.TemporaryDirectory(dir=".run") as td:
aligned = os.path.join(td, "a.o")
align_args = ["--set-section-alignment", ".text=4"]
for S in placed:
align_args += ["--set-section-alignment", f"{S}=4"]
for s in weaken:
align_args += ["--weaken-symbol", s]
sh(f"{AS}objcopy", *align_args, obj, aligned)
ld = os.path.join(td, "link.ld")
lines = ["SECTIONS {", f" . = 0x{text_vram:08X};", " .text : { *(.text) }"]
for S, b in placed.items():
lines += [f" . = 0x{b:08X};", f" {S} : {{ *({S}) }}"]
lines += [" /DISCARD/ : { *(*) }", "}"]
open(ld, "w").write("\n".join(lines) + "\n")
cmd = [f"{AS}ld", "-T", ld, "-o", os.path.join(td, "out.elf"), aligned]
for s, a in sorted(defs.items()):
cmd += ["--defsym", f"{s}=0x{a:08X}"]
p = subprocess.run(cmd, capture_output=True)
if p.returncode != 0:
res["ok"] = False
res["error"] = "ld: " + p.stderr.decode().strip().split("\n")[-1]
return res
got = sh(f"{AS}objcopy", "-O", "binary", "--only-section", ".text",
os.path.join(td, "out.elf"), "/dev/stdout").stdout
aligned = os.path.join(td, "a.o")
align_args = ["--set-section-alignment", ".text=4"]
for S in placed:
if re.fullmatch(r"\.s?bss\d+", S):
continue # a split piece carries the alignment of its own base
align_args += ["--set-section-alignment", f"{S}=4"]
for s in weaken:
align_args += ["--weaken-symbol", s]
sh(f"{AS}objcopy", *align_args, obj, aligned)
ld = os.path.join(td, "link.ld")
lines = ["SECTIONS {", f" . = 0x{text_vram:08X};", " .text : { *(.text) }"]
for S, b in placed.items():
lines += [f" . = 0x{b:08X};", f" {S} : {{ *({S}) }}"]
lines += [" /DISCARD/ : { *(*) }", "}"]
open(ld, "w").write("\n".join(lines) + "\n")
# --no-check-sections, exactly as the build and the region verify link: NOLOAD placements are
# addresses only, and a split piece's extent tiles the PACKED section, so an unreferenced common
# inside one piece may in truth live inside another piece's range (GS_001: PSDBASEX/PSDBASEY are
# adjacent in the game, 16 bytes apart in the packed .bss) — zero-byte overlaps, harmless.
cmd = [f"{AS}ld", "--no-check-sections", "-T", ld, "-o", os.path.join(td, "out.elf"), aligned]
for s, a in sorted(defs.items()):
cmd += ["--defsym", f"{s}=0x{a:08X}"]
p = subprocess.run(cmd, capture_output=True)
if p.returncode != 0:
res["ok"] = False
res["error"] = "ld: " + p.stderr.decode().strip().split("\n")[-1]
return res
got = sh(f"{AS}objcopy", "-O", "binary", "--only-section", ".text",
os.path.join(td, "out.elf"), "/dev/stdout").stdout
want = exe[text_vram - vram_base: text_vram - vram_base + tsize]
res["ok"] = (got == want)
@@ -295,6 +319,8 @@ def main():
print(f"[{tag}] {r['name']:14s} .text@0x{text_vram:08X} ({r['tsize']} B) "
f".rdata@{r['rdata_vram'] if not isinstance(r['rdata_vram'],int) else hex(r['rdata_vram'])} "
f".data@{r['data_vram'] if not isinstance(r['data_vram'],int) else hex(r['data_vram'])}")
if r.get("split"):
print(" ", r["split"])
if r.get("error"):
print(" ", r["error"])
if not r["ok"] and "ndiff" in r:
tools/psyq_link_region.py
+14 -8
View File
@@ -20,6 +20,7 @@ import json, os, re, subprocess, sys, tempfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from psyq_link import (section_table, symbol_table, recover_sym_addrs, unique_byte_vram,
DATA_SECTIONS, AS, sh)
from psyq_bss_split import NOBITS_RE, prepare_object, describe # P31 S78 #4
def placement(elf_dir, lo, hi, vram_base, exe):
@@ -47,14 +48,14 @@ def classify(obj, text_vram, exe, vram_base):
sym_addr = recover_sym_addrs(obj, text_vram, exe, vram_base)
bases = {}
for S in DATA_SECTIONS:
if S in secs and secs[S][0] > 0:
b = unique_byte_vram(obj, S, exe, vram_base) if S not in (".bss", ".sbss") else None
for S in secs: # every data-like section, incl. split pieces (.bss2 …, §489)
if (S in DATA_SECTIONS or NOBITS_RE.match(S)) and secs[S][0] > 0:
b = unique_byte_vram(obj, S, exe, vram_base) if not NOBITS_RE.match(S) else None
if b is None:
b = sym_addr.get(S) # the object referenced the section symbol
if b is not None:
bases[S] = b
weaken = [s for s, (sec, _) in symtab.items() if sec in (".bss", ".sbss")]
weaken = [s for s, (sec, _) in symtab.items() if NOBITS_RE.match(sec)]
return bases, weaken, sym_addr
@@ -82,10 +83,16 @@ def build_region(elf_dir, lo=None, hi=None, emit=None, *, vram_base, exe_path):
region_lo = order[0][1][0]
region_hi = order[-1][1][0] + order[-1][1][1] * 4
recovered, weaken_by, bases_by = {}, {}, {}
td = tempfile.mkdtemp(dir=".run")
recovered, weaken_by, bases_by, srcs = {}, {}, {}, {}
conflicts = []
for name, (vram, _) in order:
obj = os.path.join(elf_dir, name)
# P31 S78 #4: split a scattered-.bss object into per-base pieces BEFORE classifying it — the
# pieces' own section symbols then yield one base each (SplitRefused propagates: loud, R43).
obj, plans = prepare_object(os.path.join(elf_dir, name), vram, exe, vram_base, os.path.join(td, "split"))
if plans:
print(f" .. {name}: {describe(plans)}")
srcs[name] = obj
bases, weaken, sym_addr = classify(obj, vram, exe, vram_base)
bases_by[name] = bases
weaken_by[name] = weaken
@@ -96,7 +103,6 @@ def build_region(elf_dir, lo=None, hi=None, emit=None, *, vram_base, exe_path):
conflicts.append((s, recovered[s], a))
recovered[s] = a
td = tempfile.mkdtemp(dir=".run")
prepared = []
for name, (vram, _) in order:
dst = os.path.join(td, name)
@@ -105,7 +111,7 @@ def build_region(elf_dir, lo=None, hi=None, emit=None, *, vram_base, exe_path):
args += ["--set-section-alignment", f"{S}=4"]
for w in weaken_by[name]:
args += ["--weaken-symbol", w]
sh(f"{AS}objcopy", *args, os.path.join(elf_dir, name), dst)
sh(f"{AS}objcopy", *args, srcs[name], dst)
prepared.append((name, vram, dst))
# Each object's .text is placed at its EXACT vram (the region is two contiguous libcd