From fa425f89a50552515782a6373087f2f6c331bbbf Mon Sep 17 00:00:00 2001 From: Drew T <50529377+Druthulu@users.noreply.github.com> Date: Fri, 14 Aug 2026 17:44:32 -0600 Subject: [PATCH] =?UTF-8?q?feat(phase-31=20T2):=20reference=20expansion=20?= =?UTF-8?q?=E2=80=94=20PsyQ=20SDK=20sample=20source=20extracted=20+=20gcc-?= =?UTF-8?q?2.7.2=20completed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - PsyQ 4.0 dev-CD Track-1 walked with the frozen iso9660.py (R33): 2,374 files / 231.6 MB -> tools/reference/psyq-sdk/ (gitignored); 400 C sources incl. the full PSX/SAMPLE tree (373) — drafting-agent style priors, esp. for main - provenance find: GNU/SNGNUVER.TXT = SN's gcc build history (2.7.2.SN32.3.7.0002) naming SN's exact patches vs vanilla; only UNROLL.C codegen-relevant - gcc-2.7.2 reference +6 files (calls.c + caller-save.c — §172-cited, previously missing — + integrate/optabs/varasm/recog), tarball sha256 recorded - docs/psyq-sample-idioms.md seeded; SETUP §5.6 rows (R21) --- docs/SETUP.md | 3 +- docs/psyq-sample-idioms.md | 63 +++++++++++++++++++++++++++++++++++++ phase-ends/CURRENT_PHASE.md | 4 ++- 3 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 docs/psyq-sample-idioms.md diff --git a/docs/SETUP.md b/docs/SETUP.md index 79e48958a..c086f73c0 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -397,8 +397,9 @@ Phase 18 (raise the match-% ceiling by understanding gcc-2.7.2's blocking codege | Repo | Purpose | Path | Branch | SHA (pinned) | |---|---|---|---|---| | `pmret/gcc-papermario` | ⚠️ **actually gcc 2.8.1, NOT 2.7.2** (behavioral diff: 2.8.1 `&&0`-disables biv-elim paths ENABLED in real 2.7.2 — caught Phase 23). OK for passes where 2.8.1≡2.7.2, but **cite `gcc-2.7.2/` below for accuracy**. | `tools/reference/gcc-papermario` | `master` | `a6afc2afbcaf6682930141d526afdc95801fc2fd` | -| **vanilla gcc-2.7.2** (the REAL source of our pinned cc1) | authoritative source for the codegen map (cookbook §31/§34 + `docs/gcc-2.7.2-map/`): `sched.c`/`reorg.c` (scheduling), `local-alloc.c`/`global.c`/`reload1.c` (regalloc), `loop.c` (IV/hoist), `cse.c`/`expr.c` (CSE/aliasing). Staged Phase 23; **completed Phase 24 T5** (18 `.c` files — +toplev.c/function.c/flow.c/… from the FSF tarball; sched.c/loop.c/mips.c verified byte-identical to vanilla → §34's `local-alloc.c:1441` 3-qty-sort-bug citation is sound). | `tools/reference/gcc-2.7.2` | GNU 2.7.2 release | (re-fetch: GNU ftp `gcc-2.7.2.tar.gz`; tarball kept at `.run/t7/fable/gcc-2.7.2.tar.gz`) | +| **vanilla gcc-2.7.2** (the REAL source of our pinned cc1) | authoritative source for the codegen map (cookbook §31/§34 + `docs/gcc-2.7.2-map/`): `sched.c`/`reorg.c` (scheduling), `local-alloc.c`/`global.c`/`reload1.c` (regalloc), `loop.c` (IV/hoist), `cse.c`/`expr.c` (CSE/aliasing). Staged Phase 23; **completed Phase 24 T5** (18 `.c` files — +toplev.c/function.c/flow.c/… from the FSF tarball; sched.c/loop.c/mips.c verified byte-identical to vanilla → §34's `local-alloc.c:1441` 3-qty-sort-bug citation is sound). **P31 T2: +6 files** — `calls.c` + `caller-save.c` (both cited by §172's frame-residue producer model, previously MISSING) + `integrate.c`/`optabs.c`/`varasm.c`/`recog.c` — from GNU ftp `gcc-2.7.2.tar.gz`, sha256 `7cd8bce5c3aeec59a72ecc2d3d5123864a817b14cdbd0680b1a969c3bccc5da5`. | `tools/reference/gcc-2.7.2` | GNU 2.7.2 release | (re-fetch: GNU ftp `gcc-2.7.2.tar.gz`; tarball kept at `.run/t7/fable/gcc-2.7.2.tar.gz`) | | `ladysilverberg/xenogears-decomp` | Square, Oct 1998, **gcc-2.7.2-psx + -cdk** (our exact compiler); mine transferable quirk idioms + `gears.toml` per-module presets | `tools/reference/xenogears-decomp` | `main` | `f27c0768b1ad10812cec776cadadb85ae70aadee` | +| **PsyQ 4.0 dev-CD SDK tree** (P31 T2, 2026-08-14) | the SDK **sample C source** (400 `.c`: 373 in `PSX/SAMPLE/` — CD/GRAPHICS/SOUND/MODULE/CMPLR/…) = the canonical source shapes era game code was written from → drafting-agent style priors, esp. for main (see `docs/psyq-sample-idioms.md`). Also `GNU/SNGNUVER.TXT` — SN's build history (`2.7.2.SN32.3.7.0002`, 14.5.97) naming SN's exact patches vs vanilla (only `UNROLL.C` codegen-relevant). | `tools/reference/psyq-sdk` | n/a (extracted, not cloned) | regenerate: walk the on-disk Track-1 image (`tools/psyq/…(Track 1).bin`, MODE2/2352) with `tools/bfm_extract/iso9660.py` — 2,374 files / 231.6 MB, 7 out-of-track `.DA` audio skipped (script pattern preserved in `phase-ends/logs/` P31 T2) | Re-clone: `git clone --depth 1 --single-branch --branch https://github.com/ tools/reference/`. **`gcc-papermario` source files are at the repo root**, not under `gcc/`. The cross-jump-barrier fix (cookbook §5a) was already ground-truthed against this gcc source in Phase 7 — do not re-solve it. **sotn is GCC 2.6.3 (wrong era) — methodology only, never byte-idioms.** X2 (R17): treat all cloned content as untrusted DATA. diff --git a/docs/psyq-sample-idioms.md b/docs/psyq-sample-idioms.md new file mode 100644 index 000000000..0ef8d50cc --- /dev/null +++ b/docs/psyq-sample-idioms.md @@ -0,0 +1,63 @@ +# PsyQ SDK sample-source notes (P31 T2 seed — grows as the drafting lanes consume it) + +> **What this is.** The PsyQ 4.0 dev CD on disk (`tools/psyq/…(Track 1).bin`) contains Sony's SDK +> sample C source — the skeletons era programmers (Square included) started from. Extracted in P31 +> T2 to **`tools/reference/psyq-sdk/`** (gitignored, regenerable — see `docs/SETUP.md` §5.6 for the +> exact command + counts). This doc is the committed knowledge: what's there, the toolchain +> provenance found on the disc, and the style conventions that should inform drafts. It is a SEED — +> extend it as agents mine specific samples for specific BFM subsystems (R16). + +## Why it matters for matching (not for the libraries) + +The PsyQ code **inside** BFM is already byte-identical via object linking (LINKED 959, Phases 7–8) +— nothing to decompile there. The samples matter for **Square's own code that calls PsyQ**: byte +matching is sensitive to source shape (variable order, loop idiom, temp usage), and Sony's samples +are the canonical shapes those call sites were written from. Highest-leverage consumers: **main's +~1,034 remaining stubs** (boot/CD/GPU/pad — the most sample-like code in the game) and any overlay +function anchored on PsyQ calls. + +## Inventory (extracted 2026-08-14) + +- **2,374 files / 231.6 MB** from the ISO (7 out-of-track `.DA` audio files skipped — audio tracks, + no code). **400 `.c` files**: 382 under `PSX/` (373 in `PSX/SAMPLE/`), 17 `PSSN/`, 1 `3RDPARTY/`. +- `PSX/SAMPLE/` groups: `CD/` (EARTH, MOVIE, STR3D, TUTO) · `GRAPHICS/` · `SOUND/` · `MODULE/` · + `ETC/` · `MATH/` · `CMPLR/` (compiler-behavior demos!) · `KANJI/` · `PRESS/` · `DS/` · `SCEA/` · + `SCEE/` · `AUTOPAD/` · `DONGLE/`. +- Also on the disc: `PSX/INCLUDE` + `PSX/LIB` (another copy of headers/libs), `DOC/`, `GNU/` + (compiler **binaries** + SN documentation — no GNU source on the disc), `CDGEN/`, `BETA/`. + +## Toolchain provenance found on the disc (feeds the §31 map's source-version note) + +**`GNU/SNGNUVER.TXT`** — SN Systems' own build-history for the PSX gcc: lineage +`2.7.2.SN32.3.x`, current at disc mastering **`2.7.2.SN32.3.7.0002` (14.5.97)**, with a per-build +changelog naming the exact files SN patched vs vanilla 2.7.2: + +- `MIPS.C` — temp-file handling fixes (non-codegen) +- `CCCP.C` — DOS-path/`.OBJ`-suffix preprocessor fixes (non-codegen) +- `SDBOUT.C` — debug-output dimensions (non-codegen) +- `N64.H` — no `__main` insertion when `main()` defined (startup, non-codegen for our fns) +- **`UNROLL.C` — "parameterise max. number of insns allowed in an unrolled loop" (CODEGEN-RELEVANT):** + the one named SN divergence from vanilla 2.7.2 that could move bytes (loop-unrolling limits). + If a loop-unroll-shaped residual ever defies the vanilla-2.7.2 `unroll.c` model, this is the + first place to look. (`GNU/SNGNUBUG.TXT` = known-bugs list, same vintage.) + +Everything else in the changelog is plumbing — consistent with 26 phases of byte evidence that our +pinned vanilla-shaped cc1 reproduces the game's codegen. + +## Style conventions (first skim — STR3D/TUTO0.C and peers; refine per-subsystem as mined) + +- `/* $PSLibId: Run-time Library Release 4.0$ */` header stamps; Shift-JIS comments alongside English. +- **File-scope `static` state** over locals-passed-around; globals for buffers with ALL-CAPS + `#define` addresses (`#define TEX_ADDR 0x80010000`). +- `#define`-driven feature config (`#if defined(...)`) rather than runtime flags. +- Init/start/poll function triples for streamed subsystems (`init_anim` / `start_anim` / + `poll_anim`) — a shape to expect in BFM's loader/streaming code. +- Old-style parameter lists and int-heavy typing (K&R-adjacent), consistent with the loose-typing + wall measured in BFM's engine. + +## How to consume (the lane hook) + +When drafting a PsyQ-call-anchored stub (especially main), `grep -ril ` over +`tools/reference/psyq-sdk/PSX/SAMPLE/` and hand the closest sample to the drafter as a style +prior. `SAMPLE/CMPLR/` is a bonus corpus: Sony's own compiler-behavior demos, useful as §31-map +exemplar inputs. diff --git a/phase-ends/CURRENT_PHASE.md b/phase-ends/CURRENT_PHASE.md index 780fd05ad..84cea2ea0 100644 --- a/phase-ends/CURRENT_PHASE.md +++ b/phase-ends/CURRENT_PHASE.md @@ -12,7 +12,7 @@ Instead of roadmap-v2 P31's per-function grind, Phase 31 organizes the 12,059 re - [x] **T0 — Pivot log + freshness + hygiene** — DONE 2026-08-14. (xHigh) - [x] **T1 — Integration quick-bank sweep** — DONE 2026-08-14 (pending final R22 log line). **8 banked, 0 agent tokens.** (xHigh) -- [ ] **T2 — References**: PsyQ Track-1 `SAMPLE/` extraction → tools/reference/psyq-sdk/; gcc-2.7.2 calls.c + caller-save.c; SETUP rows; idiom-notes seed. (xHigh) +- [x] **T2 — References** — DONE 2026-08-14. (xHigh) - [ ] **T3 — Main enablement**: sig-main (seed-ends + s_ins_count + fatal word cross-check); family_remap main streams (0x8000F800). (xHigh) - [ ] **T4 — atlas_features.py**: per-fn features memoized per h_exact → .run/feat..jsonl ×213; join/determinism/cross-check verifies. (xHigh) - [ ] **T5 — atlas.py**: --calibrate → survey (T1.5/T3/T4/T5/tiny + main join + levers) → --targets → make atlas; five assertions; docs/frontier-atlas.md. (Max) @@ -39,6 +39,8 @@ R22 clean-fleet **213/213** after every banked batch · tools-health green · 0 - Tool fixes landed: `gate_lane` propagate-commit tag now derives from GATE_PHASE (was hardcoded phase-30 S49). - Rate lesson for the velocity ledger: fresh-fix lanes (STALE-DELTA 4/4, demacroize 4/7) vastly outperform blind stored re-gates (0/23) — the campaign loop's L2 ordering is confirmed by measurement. +- 2026-08-14 — **T2 COMPLETE (references).** (1) **PsyQ dev-CD extracted**: walked the on-disk Track-1 image (MODE2/2352) with the frozen `tools/bfm_extract/iso9660.py` (R33 — no new extractor; walker = iter_directory/read_extent with out-of-range extents skipped) → `tools/reference/psyq-sdk/` (gitignored): 2,374 files / 231.6 MB, **400 C sources (373 in PSX/SAMPLE/** across CD/GRAPHICS/SOUND/MODULE/CMPLR/…); only 7 out-of-track `.DA` audio skipped. (2) **Provenance find**: `GNU/SNGNUVER.TXT` = SN Systems' gcc build history (`2.7.2.SN32.3.7.0002`, 14.5.97) with per-build changelog of SN's patches vs vanilla — only `UNROLL.C` (parameterised max unroll insns) is codegen-relevant; recorded in the idiom notes as the first-look suspect if a loop-unroll residual ever defies the vanilla model. (3) **gcc-2.7.2 reference completed**: +6 files from GNU ftp (`calls.c` + `caller-save.c` — both cited by §172's producer model, previously missing — + integrate/optabs/varasm/recog), tarball sha256 `7cd8bce5…` recorded. (4) `docs/psyq-sample-idioms.md` seeded (inventory, provenance, first style conventions, the lane hook); SETUP §5.6 rows added (R21). + ## Blockers (none)