diff --git a/.github/raid-discord.png b/.github/raid-discord.png new file mode 100644 index 00000000..f10e0193 Binary files /dev/null and b/.github/raid-discord.png differ diff --git a/.gitignore b/.gitignore index 2b778d27..a1ea0a71 100644 --- a/.gitignore +++ b/.gitignore @@ -79,3 +79,22 @@ __pycache__/ # DuckStation oracle (local-only; private fork at git@github.com:mstan/duckstation.git) /duckstation/ _wip/ + +# Game discs/EXEs and recompiler output (copyrighted, not distributable). +# Only the per-game config (game.toml) is tracked. +games/*/* +!games/*/game.toml +!games/*/seeds.txt + +# Per-game memcards created at runtime +*.mcr +!dummy.*.mcr + +# macOS .app bundle (contains recompiled BIOS-derived binary — not distributable) +/dist/ + +# Session-scratch debug scripts (hardcoded local paths; kept on disk, not tracked) +tools/_options_session/ + +# Compiled test binaries (build artifacts; the runtime/tests/*.c sources are tracked) +runtime/tests/*.exe diff --git a/CLAUDE.md b/CLAUDE.md index 29272378..5b4a19af 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,11 +43,11 @@ loss), never as the starting point — see §0's amendment below and recomp-template `PRINCIPLES.md` → "LLE Is the Baseline; HLE Is a Subsystem Replacement, Not a Starting Point". -**The authoritative game plan is `FAITHFUL_TIMING_PLAN.md` — READ IT EACH +**The authoritative game plan is `docs/internal/FAITHFUL_TIMING_PLAN.md` — READ IT EACH SESSION** (north star, phased plan P1–P6, current status/log). Update its Status/Log section every session. The full-coverage accuracy burndown across ALL axes (semantics, cycle, IRQ, MMIO, peripherals, static-vs-dynamic, determinism) -lives in `ACCURACY_BURNDOWN.md` — every item must be cross-referenced against an +lives in `docs/internal/ACCURACY_BURNDOWN.md` — every item must be cross-referenced against an external comparative (psx-spx / in-tree Beetle source / DuckStation / HW test ROMs), not asserted. Axis 5 (peripherals — esp. SIO/controller, the hybrid-pad bug) is the suspected-weakest second front after the cycle axis. @@ -114,11 +114,19 @@ first-class High-Level Emulation tier alongside LLE, modeled on `F:/Projects/gbarecomp/gbarecomp` (`src/runtime/bios_hle.{h,cpp}`, commits 23a57ce + 168e313): -1. **Two selectable backends, LLE default.** A null-by-default hook intercepts - BIOS service dispatch before the recompiled BIOS runs; selection is - per-game config (`[runtime] bios_hle = true`) + env override (PSX_BIOS_HLE), with a startup - banner naming the active backend. With HLE off the build is byte-identical - to a build without the tier. +1. **Two selectable backends. AMENDMENT 2026-07-06 — HLE is now the DEFAULT, but + this changes NOTHING institutional.** We still BUILD LLE: the recompiled BIOS + is the foundation we architect against, the reference implementation, and the + oracle — fully linked, load-bearing, selectable, and the thing every accuracy + check runs against. HLE is a QoL layer we lay ON TOP (instant boot-skip for + players); the faithful LLE core is proven, so defaulting the convenience on is + an enhancement-phase load-time win, not an architecture change. Mechanically + this only flips the framework runtime default `bios_hle` false→true + (`config_loader.h`); opt OUT per-game with `[runtime] bios_hle = false` or env + `PSX_BIOS_HLE=0`. A null-by-default hook intercepts BIOS service dispatch + before the recompiled BIOS runs; a startup banner names the active backend. + With HLE off the build is byte-identical to a build without the tier. Keep new + bring-up and all verification on LLE; HLE-default is the shipping convenience. 2. **LLE remains the reference implementation and the oracle.** It stays fully linked, load-bearing, and selectable; every BIOS call the HLE layer does not implement transparently falls through to the recompiled BIOS, so HLE is @@ -141,14 +149,14 @@ first-class High-Level Emulation tier alongside LLE, modeled on If you find yourself wanting to violate any of the above three paragraphs **beyond the two amendments just above**, -**stop and re-read PLAN.md**. Every prior attempt failed by +**stop and re-read docs/internal/PLAN.md**. Every prior attempt failed by violating exactly these rules under pressure. --- ## 1. The BIOS is recompilation target #1, the game is target #2 -Phase 1-3 of PLAN.md exist to get the BIOS recompiled and booting on +Phase 1-3 of docs/internal/PLAN.md exist to get the BIOS recompiled and booting on its own. The BIOS must reach the Sony logo and the BIOS shell, running entirely as native C, before any game work begins. There is no path that loads a game EXE before the BIOS is fully working in v4. **Do not @@ -228,7 +236,7 @@ a fake event. The fake delivery was not progress, it was theater. At the start of every session, before any code change: 1. Read this file (CLAUDE.md). -2. Read PLAN.md to confirm what phase we are in and what the next +2. Read docs/internal/PLAN.md to confirm what phase we are in and what the next concrete milestone is. 3. Verify `docs/psx_bios_disasm.txt` exists (primary reference). 4. Verify Ghidra MCP is reachable. If not, stop and ask. @@ -322,7 +330,7 @@ Code without proof is invalid. Before any Phase 2 work: -- FIRST_MILESTONE.md must be complete +- docs/internal/FIRST_MILESTONE.md must be complete - boot_slice must compile - all instructions must be supported @@ -454,7 +462,7 @@ the visibility you need. ## 17. Phase 5 gate — fix hardware stubs before loading a game -`STUBS_TO_FIX.md` lists every known stub in the runtime. Before any +`docs/internal/STUBS_TO_FIX.md` lists every known stub in the runtime. Before any Phase 5 work (loading Tomba or any game EXE), every stub marked "Phase 5+" in that file **must be implemented and oracle-verified**: diff --git a/CODEX_ANALYSIS.md b/CODEX_ANALYSIS.md deleted file mode 100644 index eae84e95..00000000 --- a/CODEX_ANALYSIS.md +++ /dev/null @@ -1,388 +0,0 @@ -# Codex Analysis - PSXRecomp v4 Memory Card Copy - -Date: 2026-05-07 -Branch: `codex/discovery` - -## Executive Summary - -We now have a real, non-no-op reproduction for memory-card COPY. - -The current Card 2 setup before this session was a poor proof target: Card 2 already contained the same first three saves as Card 1, so a Card1 -> Card2 copy could visually "succeed" while the destination file hash stayed unchanged. I replaced Card 2 with a formatted blank card for both native and Beetle, then repeated the same COPY flow. - -Result: - -- Beetle oracle copies successfully. -- Native PSXRecomp does not copy. -- The old `func_1FC0A080.$a1 = 0x83` blocker is not the current blocker in this repro. -- Native enters `func_1FC0A080` with a valid pointer, but the downstream SIO write protocol aborts before the write payload is sent. - -Current localized failure: - -Native starts slot-1 memory-card write commands (`0x57`) but each aborts after only five bytes: - -```text -81 57 00 00 00 -``` - -That is: card select, write command, ID/address phase, then interruption before the 128-byte data payload. Because the state machine never reaches `MC_WRITE_CHK`, `memcard_write_sector()` is never called, so `card2.mcd` remains blank. - -The next target is native vs Beetle comparison around the first slot-1 `0x57` write transaction, with focus on SIO IRQ scheduling / pad polling / transaction isolation. - -## Current Disk State - -Working tree has intentional card-image changes: - -```text -M card2.mcd -M dummy.1.mcr -``` - -Current card hashes: - -```text -card1.mcd 15DBADBB080B8561414DB62847594E4C -card2.mcd 8A250FE6206A1537011166866B509EBD # blank, native did not copy -dummy.0.mcr 97C780723BDA08D8880CFB33421F9FCC -dummy.1.mcr 2CB83FE744EFCD085B9EA8C17D3EBF31 # Beetle copied one save -``` - -Backup of the pre-test Card 2 images: - -```text -F:\tmp\psxrecomp_card_backup_20260507_182441 -``` - -Current hidden processes at time of writing: - -```text -psx-runtime.exe pid 39100, debug port 4370 -psx-beetle.exe pid 10560, debug port 4380 -``` - -Only one instance of each backend was running. - -## Method Of Navigation - -I followed the local repo rules and the `F:\Projects\recomp-template` debugging approach: - -- Treat Beetle as oracle and native as subject. -- Avoid generated-file edits. -- Avoid HLE pokes, stubs, log files, and direct state forcing. -- Use TCP debug commands, ring buffers, screenshots, and card hashes. -- Establish a proof target before inferring behavior. -- Prefer first-divergence evidence over static guessing. - -Concrete navigation steps: - -1. Preserved the current dirty state by committing it before investigation. -2. Created and stayed on `codex/discovery`. -3. Verified both hidden debug servers: - - native: `127.0.0.1:4370` - - Beetle: `127.0.0.1:4380` -4. Verified Ghidra MCP connectivity separately: - - `ghidra_psx` works through Codex at `http://localhost:7777/mcp`. - - Active binary is `SCPH1001.BIN`. - - Ghidra confirmed the static A080 chain from the earlier handoff. -5. Validated that native `fntrace` was not the right tool for static generated function entries. - - Generated function entries are captured by `fn_filter` / `fn_clear` / `fn_entry_dump`. - - `fntrace_arm` is more useful for indirect dispatch / Beetle JAL/JALR comparison. -6. Used screenshots after every navigation step where input ambiguity mattered. -7. Switched from duplicate Card 2 to blank Card 2 to make copy success observable by hash and directory contents. - -## Reproduction Setup - -Build command used earlier in the session: - -```powershell -$env:PATH = 'C:\msys64\mingw64\bin;' + $env:PATH -cmake --build runtime\build --target psx-runtime psx-beetle -- -j8 -``` - -Start exactly one native and one Beetle: - -```powershell -Start-Process -FilePath "F:\Projects\psxrecomp-v4\runtime\build\psx-runtime.exe" -WorkingDirectory "F:\Projects\psxrecomp-v4" -WindowStyle Hidden -Start-Process -FilePath "F:\Projects\psxrecomp-v4\runtime\build\psx-beetle.exe" -WorkingDirectory "F:\Projects\psxrecomp-v4" -WindowStyle Hidden -``` - -Blank Card 2 setup: - -```powershell -Copy-Item runtime\build\card1.mcd card2.mcd -Force -Copy-Item runtime\build\card1.mcd dummy.1.mcr -Force -``` - -`runtime\build\card1.mcd` and `runtime\build\card2.mcd` are formatted blank cards with no used directory entries. - -Native uses: - -```text -card1.mcd -card2.mcd -``` - -Beetle uses: - -```text -dummy.0.mcr -dummy.1.mcr -``` - -## UI Repro Flow - -Starting from BIOS main menu with cursor on `MEMORY CARD`: - -1. CROSS: enter memory-card manager. -2. DOWN: select `COPY`. -3. CROSS: enter source-save selection. -4. CROSS: choose the first Card 1 save. -5. Confirm modal appears: `COPY FILE / Are you sure?` -6. CROSS on `YES`. - -Important: screenshots are needed after steps 2-5. I initially misread the cursor and accidentally pressed CROSS on `EXIT`, which returned to main menu. The reliable proof requires seeing the cursor on `COPY`, then seeing the file/source icon selected, then seeing the modal. - -Useful commands: - -```powershell -python tools\raw_tcp.py 4370 screenshot_file path=F:\tmp\native_step.bmp -python tools\raw_tcp.py 4380 screenshot_file path=F:\tmp\beetle_step.png -``` - -Native screenshot output currently ignores the requested path and writes `psx_screenshot.bmp` in the repo root. Beetle honors the requested path. - -## Key Native Evidence - -Before confirming YES, arm native function-entry capture for A080: - -```powershell -python tools\raw_tcp.py 4370 fn_filter lo=0x1FC0A080 hi=0x1FC0A081 -python tools\raw_tcp.py 4370 fn_clear -python tools\raw_tcp.py 4370 card_data_writes_reset -``` - -After confirming YES: - -```powershell -python tools\raw_tcp.py 4370 fn_stats -python tools\raw_tcp.py 4370 fn_entry_dump addr_lo=0x1FC0A080 addr_hi=0x1FC0A081 count=20 -python tools\raw_tcp.py 4370 card_txn_dump count=400 tail=1 -python tools\raw_tcp.py 4370 card_data_writes count=16 -``` - -Native A080 entries showed: - -```text -func = 0x1FC0A080 -ra = 0x00002D14 -a0 = 0x000086CC -a1 = 0x8007A1D0 -a2 = 0x00000080 -a3 = 0x00000003 -t1 = 0xBFC0A080 -``` - -So the previous `a1 = 0x83` A080 path is not reproduced here. The current path passes a valid RAM pointer. - -Native transaction summary around the failed write: - -```text -cmd counts in recent window: -0x52: 390 -0x57: 10 - -0x57 transactions: -slot=1 -cmd=0x57 -sector=0xFFFF -bytes=5 -end_reason=abort_other -terminal_state=5 -tx = 81 57 00 00 00 -rx = FF 00 5A 5D 00 -``` - -The state naming from `runtime/src/sio.c`: - -```text -MC_IDLE = 0 -MC_CMD = 1 -MC_ID1 = 2 -MC_ID2 = 3 -MC_ADDR_MSB = 4 -MC_ADDR_LSB = 5 -MC_WRITE_DATA starts after address phase -``` - -Because native stops at terminal state 5, it never enters `MC_WRITE_DATA` for the payload and never reaches `MC_WRITE_CHK`. - -Native SIO trace around the first failed write: - -```text -seq 128916 tx=81 rx=FF mc_pre=0 mc_post=1 ctrl=0x3013 func=0x1FC42370 in_exc=0 -seq 128917 tx=57 rx=00 mc_pre=1 mc_post=2 ctrl=0x3013 func=0x1FC42370 in_exc=0 -seq 128918 tx=00 rx=5A mc_pre=2 mc_post=3 ctrl=0x3013 func=0x1FC42370 in_exc=0 -seq 128919 tx=00 rx=5D mc_pre=3 mc_post=4 ctrl=0x3013 func=0x1FC42370 in_exc=0 -seq 128920 tx=00 rx=00 mc_pre=4 mc_post=5 ctrl=0x3013 func=0x1FC42370 in_exc=0 - -then pad polling interrupts: - -seq 128921 tx=01 rx=FF mc_pre=0 mc_post=0 ctrl=0x1013 func=0x0000445C in_exc=1 -seq 128922 tx=42 rx=41 ... -seq 128923 tx=00 rx=5A ... -seq 128924 tx=00 rx=FF ... -seq 128925 tx=00 rx=BF ... -``` - -The transaction ring reports the `0x57` transaction closed as `abort_other` at terminal state 5. This is likely because the card flow is interrupted/reset before the next byte after `MC_ADDR_LSB`. - -Native card status after the failed copy: - -```text -slot1 dirty=false -card2.mcd hash remains 8A250FE6206A1537011166866B509EBD -card2.mcd has no used directory entries -``` - -## Key Beetle Evidence - -Beetle with blank `dummy.1.mcr` successfully copies the first save. - -Before: - -```text -dummy.1.mcr MD5 = 8A250FE6206A1537011166866B509EBD -``` - -After: - -```text -dummy.1.mcr MD5 = 2CB83FE744EFCD085B9EA8C17D3EBF31 -directory slot 1 = BISLPS-01234SAVE00 -``` - -Beetle A080 trace: - -```text -target = 0xBFC0A080 -kind = JALR -caller = 0x00002D0C -ra = 0x00002CDC -a0 = 0x000086CC -a1 = 0x8007A458 -``` - -Beetle hit A080 repeatedly during the copy. The pointer differs from native, but both are valid RAM pointers. The important divergence is not "native never gets to A080"; it does. The divergence is downstream in native SIO write completion. - -## Current Hypothesis - -Native is allowing pad/VBlank polling or interrupt handling to interleave with an in-progress card write at a point where Beetle keeps the write transaction progressing. - -The write command starts correctly: - -```text -81 57 00 00 00 -``` - -Then native switches to pad polling before sending the 128-byte write payload. This causes the card transaction to close at address phase and the BIOS retries the short write several times. No commit occurs. - -Candidate areas: - -- SIO IRQ delay/countdown behavior after card ACKs. -- Whether `sio_card_protocol_active()` considers `MC_ADDR_LSB` / write address phase active strongly enough to defer pad polling. -- VBlank/pad poll re-entry while card write is mid-transaction. -- CTRL/select handling around slot changes (`0x3013` card slot 1 vs `0x1013` pad slot 0). -- Transaction close classification may be diagnostic only, but it exposes the real symptom. - -Do not patch by forcing writes or faking card state. The correct fix should preserve hardware behavior and let the BIOS send the full write transaction. - -## Recommended Next Steps - -1. Capture Beetle SIO/write transaction shape for the successful copy, especially around its first slot-1 `0x57`. -2. Add or use equivalent Beetle SIO byte trace if needed; current Beetle fntrace proves A080 but not the full SIO byte stream. -3. In native, reset rings immediately before confirming YES and capture: - - `sio_trace` - - `sio_pc_trace` filtered to `0x1F801040` and `0x1F80104A` - - `sio_irq_dump src=card` if useful - - `imask_trace` - - `card_txn_dump` -4. Find the first point where native leaves the card write path after `MC_ADDR_LSB`. -5. Fix the runtime SIO/IRQ scheduling behavior, not BIOS state. - -Likely code locations: - -```text -runtime/src/sio.c -runtime/include/sio.h -runtime/src/debug_server.c # instrumentation only -runtime/src/beetle_debug_server.c / beetle_libretro.cpp if oracle trace parity is needed -``` - -## Guardrails - -Keep following the project rules: - -- No generated file edits. -- No HLE-poking card/directory state. -- No stubs. -- No `fprintf` logging or external log files. -- No game ISO/EXE. -- Do not run multiple native or multiple Beetle instances simultaneously. -- Do not seed `0xBFC0989C`. -- Treat `0xBFC0A080.$a1 = 0x83` as an older/stale path unless it is reproduced again with this blank-card proof target. - -## Update 2026-05-08 - COPY Write Path Fixed - -The SIO/IRQ hypothesis above was superseded by the next trace pass. - -Root cause: - -- The write callback did enter `func_000051F4`, but its state jump table at RAM `0x6C70` was not emitted as a C `switch`. -- The table contains low kernel-RAM aliases such as `0x0000542C`, which maps to ROM label `0xBFC14F2C`. -- `full_function_emitter.cpp` and `function_discovery.cpp` only mapped shell RAM aliases (`0x30000..0x5AFFF`) back to ROM when resolving jump tables. They did not map kernel-part-2 RAM aliases (`0x500..0x84FF`) back to `BFC10000+`. -- Because the emitter missed that table, generated `func_000051F4` did `cpu->pc = $t7; return` for state `0x542C`. `psx_dispatch` then fell into `dirty_ram_dispatch`. -- The dirty RAM interpreter executes one basic block. It handled the `jal 0x6380` in the write-byte block, but it could not resume at the generated post-call continuation `BFC14F80`, where `$v0` is cleared to `0`. -- `func_4D6C` saw stale `$v0 = 0x7564`, treated that as an unexpected callback result, reset SIO, and aborted every write at five bytes. - -Fix: - -- Added low kernel RAM alias resolution in both: - - `recompiler/src/function_discovery.cpp` - - `recompiler/src/full_function_emitter.cpp` -- Regenerated `generated/SCPH1001_full.c` and `generated/SCPH1001_dispatch.c`. -- The regenerated `func_000051F4` now emits: - -```text -jump table at 0x00006C70, 10 entries -case 0x0000542C: goto label_BFC14F2C; -``` - -Proof after rebuild: - -```text -Before YES: -card2.mcd MD5 = 8A250FE6206A1537011166866B509EBD - -After YES: -card2.mcd MD5 = 30B8B147D550A409E1C5EF8456B8F18A -Card 2 shows the copied save icon in the BIOS memory-card UI. -``` - -Transaction proof: - -```text -new slot-1 write transactions: 64 -reasons: {'success': 64} -lengths: {138: 64} -sectors: 0x0040..0x007F -``` - -Probe proof: - -```text -BFC14F78 pre-6380 marker: 20 -BFC14F79 post-6380 marker: 20 -BFC14F80 clear-v0 block: 20 -BFC14934 reset path: 0 -``` - -This is a codegen/discovery fix, not a runtime SIO workaround. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..06fbe1c1 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,241 @@ +# Contributing to PSXRecomp + +Thanks for your interest! PSXRecomp is a static recompiler, not an emulator, and +it holds itself to a high correctness bar. This guide covers how the project is +organized, the rules that keep it faithful, and how to get a change merged. +Contributions are welcome — AI-assisted or not — as long as they are reviewed, +tested, and keep the core game-agnostic. + +New to the codebase? Read [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) and +[`docs/EXECUTION_MODEL.md`](docs/EXECUTION_MODEL.md) first, then +[`docs/BUILDING.md`](docs/BUILDING.md) to get a build going. + +## Repository layout + +- **`mstan/psxrecomp`** (this repo) — the framework: the recompiler tool + (`recompiler/`), the runtime engine (`runtime/`), shared tools, and docs. + Framework changes go here. +- **Per-game repos** (`mstan/TombaRecomp`, `mstan/MegaManX6Recomp`, + `mstan/ApeEscapeRecomp`, …) — one per title. Each contains only that game's + config, seeds, and build glue and links this framework in as a submodule at + `psxrecomp/`. Game-specific work goes in the game repo; nothing game-specific + belongs in the framework. + +Each game repo pins an **exact framework commit** — the git submodule pointer at +`psxrecomp/` (some older game repos also carry a human-readable `psxrecomp-v4.pin` +record; the submodule pointer is the source of truth). The framework evolves on +its own cadence; a game only moves to a newer framework when someone deliberately +bumps that submodule pointer. See +[Linking the framework](docs/BUILDING.md#linking-the-framework) for the submodule ++ local-junction dev setup, and +[Framework changes and the pin](#framework-changes-and-the-pin) below for how a +framework change actually reaches a game. + +## The core rules + +These are non-negotiable because violating them is how past efforts turned into +un-debuggable emulators. The full engineering constitution is +[`CLAUDE.md`](CLAUDE.md); the debugging philosophy is +[`PRINCIPLES.md`](PRINCIPLES.md). The short version: + +1. **Build the faithful core; no per-game hacks in the foundation.** The correct + fix is the general, hardware-accurate one, not a surgical workaround for one + title. No title checks in the runtime, no magic PC addresses, no spoofed + return values for one game. (Per-game *enhancements* — widescreen, faster + loads — are legitimate once the faithful core is proven; see + [`ENHANCEMENTS.md`](ENHANCEMENTS.md).) +2. **No stubs.** A function is fully implemented or it fails loudly. No + `return 0;` placeholders, no `// TODO` behavior, no hand-delivered fake + events. If execution reaches code we can't handle, we stop and fix + discovery/codegen — we don't fake the result. +3. **Never hand-edit generated code.** Files under `generated/` and the + recompiler's output are build artifacts. If the emitted C is wrong, fix the + recompiler (`recompiler/src/`), then regenerate. (The one exception: a PR + whose explicit purpose is to *inspect or prove* a generator bug.) +4. **LLE is the baseline; HLE is a validated subsystem replacement, never a + starting point.** The recompiled BIOS is the reference and oracle. An HLE tier + is allowed only where the LLE path has a genuine landmine, must be general + (every game), must operate on the real guest structures, and must be + continuously checked against the Beetle oracle — never a "produce the answer + the BIOS would have" shim. +5. **No `printf`/log-file debugging.** Runtime inspection goes through the TCP + debug server ([`TCP_COMMANDS.md`](TCP_COMMANDS.md)) and always-on ring + buffers, not `fprintf`. +6. **Fix broken tooling immediately** — don't route around it with indirect + evidence, and don't infer correctness from two implementations sharing a bug. + +## AI-assisted and human contributions + +Both are welcome. Using AI does not reduce your responsibility for the result: +review, understand, and test any code before submitting it, and don't submit a +large AI-generated rewrite you can't explain or validate. A PR — however it was +written — is likely to be rejected if it: + +- Introduces broad, unfocused changes, or combines unrelated refactors with + behavioral fixes. +- Hardcodes game-specific behavior into the framework, or adds a title check to + the core runtime. +- Modifies generated output instead of fixing the generator/source. +- Introduces regressions in known-working games. +- Contains code the contributor cannot reasonably explain, or whose verification + cannot be stated. + +## Game-specific work belongs in game repos + +The framework stays game-agnostic. Per-game patches, launcher UI, assets, memory +maps, metadata, config, and workarounds live in the appropriate game repo, not +here. If a behavior differs by game, disc, executable, region, BIOS, or runtime +configuration, it should be **parameterized** (config option, per-game manifest +value, a parameter passed in from the game repo) or **modeled** as the real +hardware/runtime behavior — never special-cased in the core. + +Known game repos: + +- [TombaRecomp](https://github.com/mstan/TombaRecomp) — *Tomba!* +- [MegaManX6Recomp](https://github.com/mstan/MegaManX6Recomp) — *Mega Man X6* +- [ApeEscapeRecomp](https://github.com/mstan/ApeEscapeRecomp) — *Ape Escape* + +### Framework changes and the pin + +Because each game pins an exact framework commit (its `psxrecomp/` submodule +pointer), a framework fix does **not** reach a game until that game bumps that +pointer. When your framework change is meant to fix or improve a specific title: + +- **Link the game repo** in your PR (and a matching game-repo branch/commit/PR if + you have one). This lets maintainers build and test the fix in its real + integration environment. +- **Build the game against your framework branch** to prove it. The game's + `CMakeLists.txt` points `PSXRECOMP_ROOT` at the submodule; you can build + against a working checkout via the junction/symlink dev setup, or override + `PSXRECOMP_ROOT` at configure time to point at your branch — no submodule + surgery required. See [`docs/BUILDING.md`](docs/BUILDING.md#linking-the-framework). +- **Say whether a pin bump is required to consume the fix**, and whether it + requires a **regen** (codegen/emitter changes do; runtime-only changes usually + don't — note which). Bumping the submodule pointer is a deliberate, reviewable + change made in the game repo, not something a framework PR does on its own. + +## Verifying a change + +Correctness is demonstrated, not asserted: + +- **Oracle-check against Beetle.** Run `psx-beetle` alongside `psx-runtime` and + compare via the shared TCP protocol; for timing/codegen, use the co-sim build. + See [`docs/internal/COSIM_ORACLE.md`](docs/internal/COSIM_ORACLE.md). +- **Verify visually, end to end.** A milestone is "the pixels appear on screen" + — take a screenshot, don't trust a counter or a ring-buffer flag alone. +- **A decoder / codegen change requires a playthrough check**, not just a clean + build: cosim/lockstep first, then run the affected area. + +### Regression checklist + +A framework change should not break a game that worked before it. When practical, +build against the known-working game repos above and confirm, for each: + +- The game **boots**. +- The **attract demo** plays without issue. +- A save file can be **created**. +- A save file can be **loaded**. +- **Gameplay is reachable** after saving/loading. + +Perfect compatibility across every game isn't required for every PR, but the PR +must clearly document **what was tested and on which games**. Useful notes: host +OS, compiler/toolchain, game repo + version/region, whether it boots / reaches +gameplay / save-load works, and whether memory cards, audio, video, input, CD +loading, or overlays were touched. Call out any known regression or uncertain +area explicitly — a documented uncertainty is fine; a silent one is not. + +## No copyrighted assets + +Do not submit copyrighted binary content to this repository. This includes, but +is not limited to: + +- PlayStation BIOS files +- Game ROMs, ISOs, BIN/CUE files, or disc images +- Extracted game assets — music, sound, textures, models, video, fonts, artwork +- Proprietary SDK files +- Decompiled copyrighted game source + +Small factual references — addresses, symbols, hashes, structure descriptions, +compatibility notes — are acceptable when used for tooling, debugging, or +documentation. The local artifacts above are gitignored in every repo for exactly +this reason (copyright + size); keep them out of commits. + +## Adding a new game + +At a high level: create a game repo from the template, add this framework as the +`psxrecomp/` submodule, seed function starts (typically from a Ghidra export in +`seeds/`), write `game.toml`, generate the game C with +`psxrecomp-game --config game.toml`, then build and bring the title up boot → +menu → gameplay, oracle-checking as you go. The existing game repos +(TombaRecomp, MegaManX6Recomp) are the working references. + +## Pull requests + +Keep PRs focused — one thing: one bug, one feature, one diagnostic, one isolated +refactor, or docs for one topic. Avoid combining unrelated refactors, formatting +churn, and behavioral fixes in the same PR. + +- Branch off `master`; keep framework changes and game changes in their + respective repos. +- **Do not commit** local artifacts: BIOS images, disc images, generated C + (`generated/`), memory cards, Ghidra databases, build outputs, or overlay + capture files. +- Explain *what hardware behavior* the change matches and *how you verified it* + (oracle diff, screenshot, playthrough). Changes that can't state their + verification are hard to accept. +- Keep the fix minimal and at the right layer (recompiler vs runtime vs game + config). If you find yourself editing generated code or adding a stub, step + back — the real fix is upstream of that. + +When opening a PR, please include: what changed and why; whether it's +game-agnostic or needs per-game configuration; what games/tests were run; any +risks or known limitations; links to affected game repos/branches/commits when +the change targets a specific game; and screenshots, logs, or traces when useful. + +## Bug reports + +Good bug reports are extremely helpful. Please include: + +- The game repo being used, and version/region if known. +- Your OS and build type/configuration. +- Steps to reproduce, expected behavior, and actual behavior. +- Logs, traces, screenshots, or video if available. +- Whether it happens in **LLE (recompiled BIOS)**, **HLE**, and/or the + **dirty-RAM interpreter** path — and in the software vs OpenGL vs Vulkan + renderer, if relevant. +- Whether it's a **regression** from a previous commit, and the last good one if + you know it. + +For gameplay bugs, include the nearest save file or a reproduction path when +possible. For build failures, include `gcc -v` / your toolchain / the generator. + +## Dependencies and portability + +Avoid unnecessary dependencies. When adding one, explain why it's needed and how +it affects portability, build complexity, licensing, or distribution. The project +should stay practical to build and run for contributors. + +## Style and maintainability + +Prefer clear, boring, maintainable code that reads like the code around it: match +the surrounding style and naming, keep changes localized, and comment non-obvious +hardware behavior or timing assumptions. Prefer diagnostics that can be compiled +out when not needed. Avoid cleverness unless it's clearly justified. + +## Licensing + +By contributing, you agree your contribution is licensed under the same license as +the project. Do not submit code you don't have the right to contribute, and do not +copy from incompatible licenses or proprietary sources. + +## Be respectful + +Contributors use different tools, workflows, and levels of AI assistance. Review +the contribution, not the person. Technical criticism is welcome; personal attacks +are not. + +## Questions / community + +Development happens with the **R.A.I.D. (Retro AI Development)** community — +Discord invite in the [README](README.md). Open a GitHub issue for bugs and +build problems (include `gcc -v` / OS / generator for build failures). diff --git a/COSIM_HANDOFF.md b/COSIM_HANDOFF.md deleted file mode 100644 index 748937e6..00000000 --- a/COSIM_HANDOFF.md +++ /dev/null @@ -1,295 +0,0 @@ -# Handoff — MMX6 first-divergence co-sim oracle (2026-06-30) - -Continuation of the MMX6 cutscene→gameplay wedge hunt. Read `COSIM_ORACLE.md` first -(the design + the four validation gates). This file = current state + exact next steps. - -## The mandate (do not drift from this) - -After ~2 weeks of confident-but-wrong "this is it" root causes, the user forbade chasing -any more single hypotheses — **including** the currently-leading-but-UNPROVEN -GPU-draw-time-backpressure idea (Beetle-anchored, bisect-confirmed as a *lever*, never -rendered a frame). We are building a **decision procedure**: a full-architectural-state -lockstep co-sim of the two backends that halts at the FIRST divergence. Whatever it halts -on *is* the first divergence — no hypothesis can be "wrong." It will confirm or kill GPU -backpressure (and everything else) on its own. **Do not implement GPU backpressure or any -other fix until the oracle names the first divergence.** - -## Established facts (proven; do NOT re-chase — each cost days) - -- Compiled backend SKIPS the intro FMV (main-state 0→1, no substate 6) → wedges black at - `02 02`. Interp (`PSX_FORCE_INTERP=1`) plays it. SAME binary. -- Given identical inputs, compiled == interp byte-for-byte (block+function lockstep, 12.9M - + 560K units, zero divergence). So it's NOT codegen — it's WHEN an event is taken - relative to the instruction stream, which flips an early control-flow branch. -- Ruled out with data: async-RFE, CD-DMA-0x79D44 corruption (interp does the same write), - IRQ-storm/timeline, block-edge-vs-per-insn IRQ delivery, load-delay-slot exception - corruption (built + refuted, reverted). -- `PSX_LOAD_DELAY=0` makes compiled play the FMV (pixels) — but that's a secondary phase - modulator, not the faithful root, and it does NOT fix the later `02 02` gameplay wedge. -- TWO separate bugs: (1) FMV-skip, (2) `02 02` gameplay wedge (persists even with FMV fixed). - -## What is BUILT (this session, all uncommitted on branch `codex/mmx6-precise-slice`) - -Worktree: `F:\Projects\psxrecomp\_wt-tomba2\psxrecomp`. All cosim code is `#ifdef PSX_COSIM` -(zero effect on normal builds). - -New files: -- `COSIM_ORACLE.md` — design + gates (READ FIRST). -- `runtime/include/cosim_state.h`, `runtime/src/cosim_state.c` — full guest-arch-state - canonical FNV hash. Reuses `boot_state.c` per-device `_snapshot_write` serializers; ADDS - the CPU micro-state boot_state omits (muldiv_ts_done, gte_ts_done, read_absorb[33], - read_absorb_which, read_fudge, ld_which_t, ld_absorb). Incremental RAM page-hashes via - write hooks. VRAM excluded v1 (downstream, not causal). Gate-4 fault injection built in. -- `runtime/src/cosim.c` — engine: CYCLE-KEYED checkpoints (ring + cumulative chain hash + - park/step lockstep) + standalone minimal TCP server. Commands: status, chain, stride N, - runto , hash, sub, window N, inject ram|reg, reset. -- `tools/cosim.py` — coordinator: launches 2 instances, cycle-locksteps via runto/chain, - reports first divergence + sub-hash diff + window. Its --help documents the gate runs. - -Modified (guarded): -- `recompiler/src/code_generator.cpp` + `full_function_emitter.cpp` — emit guarded - `cosim_block(pc)` at each block leader (+ extern decls). REQUIRES REGEN to take effect - (see below); the CORE compare does NOT need it (see "why regen is optional for v1"). -- `runtime/src/psx_cycles.c` — `cosim_tick()` in `psx_advance_cycles` (THE alignment clock: - the only counter both backends drive identically). -- `runtime/src/dirty_ram_interp.c` — `cosim_block(pc)` in the interp loop. -- `runtime/src/memory.c` — `cosim_note_ram_write` in word/half/byte RAM writes (covers - CPU + DMA; DMA verified to route through psx_write_*_raw via the d44_ring evidence). -- `runtime/src/interrupts.c` — `interrupts_cosim_hash()` (folds cycles_since_vblank + - in_exception ONLY; EXCLUDES total_checks/dispatch_count/cooldown = backend-call-freq - artifacts that would false-positive). ALSO: reverted the refuted load-delay save/restore. -- `runtime/src/main.cpp` — `cosim_init()` at startup. -- `runtime/runtime.cmake` — cosim sources added to shared list; `COSIM` target option → - `PSX_COSIM=1 PSX_NO_DEBUG_TOOLS=1`. -- `MegaManX6Recomp/CMakeLists.txt` — `psx-cosim` target gated on `-DPSX_BUILD_COSIM=ON`. - -Also still uncommitted from earlier (Codex): the precise-slice / `--headless` mode / etc. -(both emitters have `psx_precise_interrupt_at`). Leave as-is; irrelevant to the oracle. - -## Build state (IN PROGRESS at handoff) - -- `build-wt` CANNOT be reconfigured (pre-existing worktree cache-path error, NOT our - changes). So the cosim build uses a FRESH dir: - `F:\Projects\psxrecomp\MegaManX6Recomp\build-cosim` (configured OK with: - `-G Ninja -DPSXRECOMP_V4_ROOT=F:/Projects/psxrecomp/_wt-tomba2/psxrecomp - -DPSX_BUILD_COSIM=ON -DPSX_LAUNCHER=OFF` + the mingw64 gcc/g++/ninja from build-wt cache, - RelWithDebInfo). -- BUILD PROGRESS: ALL translation units compiled CLEAN under PSX_NO_DEBUG_TOOLS+PSX_COSIM - (no unguarded-debug-call errors — the guards are correct). The ONLY failure was one link - error: `cosim_init()` called from main.cpp (C++) needed `extern "C"` — FIXED - (main.cpp:2486). Relinking at handoff. Expect the exe to link now; if any NEW link error - appears it'll be another C/C++ linkage or a missing cosim symbol — all cosim funcs are in - cosim.c/cosim_state.c (C files, added to the shared source list) so they should resolve. -- Build/run commands: - Configure (once): - cmake -S F:/Projects/psxrecomp/MegaManX6Recomp -B F:/Projects/psxrecomp/MegaManX6Recomp/build-cosim \ - -G Ninja -DCMAKE_C_COMPILER=C:/msys64/mingw64/bin/gcc.exe \ - -DCMAKE_CXX_COMPILER=C:/msys64/mingw64/bin/g++.exe \ - -DCMAKE_MAKE_PROGRAM=C:/msys64/mingw64/bin/ninja.exe -DCMAKE_BUILD_TYPE=RelWithDebInfo \ - -DPSXRECOMP_V4_ROOT=F:/Projects/psxrecomp/_wt-tomba2/psxrecomp \ - -DPSX_BUILD_COSIM=ON -DPSX_LAUNCHER=OFF - Build: cmake --build F:/Projects/psxrecomp/MegaManX6Recomp/build-cosim --target psx-cosim -j 4 - (exe at build-cosim/psx-cosim.exe; run coordinator: python tools/cosim.py --a compiled --b compiled) - -## MILESTONE (2026-06-30 late): ORACLE BUILDS + RUNS + EARLY BOOT DETERMINISTIC - -`psx-cosim.exe` links (build-cosim, 89MB). Coordinator smoke test -(`--a compiled --b compiled --stride 262144 --max 8000000`, ~14 frames early boot): -BOTH instances launch, connect (4600/4601), cycle-lockstep via `runto`, and report -**ZERO divergence** — chains match at every checkpoint. So the full mechanism works: -two-process launch, TCP, cycle-stepping, full-state hashing, chain compare. Early boot -is deterministic. Deeper compiled-vs-compiled (to 250M cyc / ~440 frames) running to test -overlay-load-timing determinism (result in build task b5lvwwi0i / next session). - -## HARNESS BUG FOUND + FIXED (the gates working as intended, 2026-06-30) - -Two bugs the validation caught before any real result was trusted: -1. Coordinator `kv()` parser misaligned the `runto` reply (leading bare word `parked`), - so `chain` parsed as None for BOTH sides → every compare was None==None == "equal" → - the tool was SILENTLY BLIND (would report "no divergence" for a real one). Fixed - (parse all adjacent token pairs) + added a FATAL guard if chain is ever unparseable. -2. With the parser fixed, gate-4 immediately exposed a REAL harness bug: the two - processes parked at DIFFERENT cycles for the same `runto` target (A=3.07M, B=4.03M) — - a racy design where the guest free-runs from launch and only notices the async stop - flag at a wall-time-dependent moment. `runto ` did not stop AT that cycle. - FIX (redesign): the guest now parks at EVERY checkpoint boundary (a deterministic - cycle = multiple of stride) and only advances when the coordinator grants budget via - `step N`. Stride is fixed at LAUNCH via env PSX_COSIM_STRIDE (no set-stride race — - the guest hits cp=1 before the coordinator can send a command). Protocol is now - `step N` (not `runto`); coordinator steps both 1 checkpoint at a time and compares - the cumulative chain, with a cycle-skew WARN if the two ever park at different cycles. - REBUILDING psx-cosim with this fix at handoff (task bxuiye5u3). - -GATE 4 PASSES (post-fix, verified 2026-06-30): injected `ram:1000:255` into B before -cp 20 → tool HALTED at cp 21 (exact checkpoint after the fault), BOTH processes at the -IDENTICAL cycle 5242880 (deterministic park — race fixed, no skew warning), and the ONLY -differing subsystem hash was `ram` (cpu/irqctl/timers/clock/all devices identical). So the -tool provably: detects divergence, halts at the right checkpoint, parks deterministically, -localizes the subsystem. cp 1–19 identical before injection = partial Gate 1 passing. -GATE 1 PASSES (compiled-vs-compiled, no inject): 457 checkpoints to ~210 frames, ZERO -divergence, chains matched throughout — INCLUDING past overlay-load territory (frame -118+). So determinism + hashing + host-state exclusion are all correct, and overlay -loading is deterministic in compiled-vs-compiled (the warming concern did not bite). - -==> TOOL VALIDATED (Gate 1 + Gate 4 both pass). THE compiled-vs-interp RUN is now running -(task bhi8hf5p1): `python tools/cosim.py --a compiled --b interp --stride 262144 ---max 1000000000` (~frame 1766, past the FMV-skip ~1650). Interp is ~5 frames/sec so this -takes ~6-10 min. When it halts: the printed cp/cycle = the first divergence; the FIRST -differing subsystem hash (cpu / irqctl / a device / ram) = WHERE it split; the window = -the last 16 checkpoints of both. THAT names the bug — read it, then design the faithful -fix for whatever it actually is (could confirm GPU-backpressure, or something else). -If it shows a cycle-skew WARN, the two backends charge cycles differently somewhere (would -be its own finding). If interp is too slow to reach 1650, use a coarser --stride for a -first bracket then drill finer. - -## OVERLAY CACHE DECISION (user, correct): leave `overlay_cache=true` — the gcc native -overlay shards ARE the compiled path we must measure; forcing them off would make the -"compiled" instance interpret overlays. The gcc compile worker only fires on a cache -MISS, so **do a WARMING RUN first** (free-run one psx-cosim instance through the FMV -region so autocompile populates build-cosim/cache), then every cosim run loads the shards -synchronously + deterministically (worker never fires). NOTE: at 14 frames no cache dir -existed yet (overlays load later in boot). OPEN: verify the autocompile cmd resolves from -build-cosim (game.toml cmd uses `../psxrecomp/...` relative → resolves to the MAIN checkout -F:/Projects/psxrecomp/psxrecomp, NOT the worktree; may need the worktree recompiler path or -a pre-populated cache). If autocompile can't run, overlays fall to dirty-interp in BOTH -compiled instances (still deterministic, but "compiled" won't exercise native overlays). -The Gate-1 deep run will REVEAL whether cold-cache overlay loading is nondeterministic. - -## ★ FIRST RESULT (2026-06-30): oracle HALTED at cp=32, cycle 8,126,465 (frame ~14) - -compiled-vs-interp first divergence is at frame ~14 (EARLY BIOS BOOT, long before the FMV -skip ~frame 1650). Both at the IDENTICAL cycle (no skew). 31 checkpoints matched exactly, -then cp32 splits. The ONLY differing subsystem is `cpu` — ram, scratch, gpu, spu, cdrom, -dma, sio, timers, clock, dirty ALL identical. `istat=00000001` (VBLANK pending) at the -split; interp's last leader = 0x80054240 (BIOS shell). A CPU-only divergence with memory -+ devices identical and an IRQ pending is the SIGNATURE of an interrupt taken at a -different instruction between backends — the exact "when is the event taken" mechanism, -caught at its FIRST occurrence. BUT NOT YET CONFIRMED: need to know WHICH cpu field split. -Added a `cpu` TCP command (full gpr/pc/cop0/micro-state dump) + coordinator -`--cpudiff-at-cp N`. REBUILDING (task b5y3113cg). Next: run -`python tools/cosim.py --a compiled --b interp --stride 262144 --cpudiff-at-cp 32` and read -which field differs: - - pc / cop0[EPC=14] / cop0[SR=12] / cop0[Cause=13] differ => REAL: an IRQ was taken in - one backend and not the other at this instruction. THIS is the root mechanism. Chase - why (the accurate-timing IRQ take-point), fix faithfully. - - ONLY a lone micro-state field (read_fudge/read_absorb_which/ld_which_t/ld_absorb/ - muldiv_ts_done/gte_ts_done) differs, with pc+cop0+gpr identical => the divergence is a - benign load-delay/stall MICRO-STATE difference that hasn't affected architectural state. - Decide: is it a real cycle-model mismatch (compiled vs interp charge differently here) - — investigate psx_cyc_* ordering — or should that field be excluded from the hash as - non-architectural (refine per COSIM_ORACLE.md). Then continue toward frame 1650. - - a gpr differs with pc identical => a real data computation divergence at this PC - (contradicts the lockstep proof — investigate that instruction). -CAVEAT: `pc` in the ring is g_last_leader_pc, only set by cosim_block; WITHOUT REGEN the -compiled side never calls cosim_block for game/BIOS text (only the interp loop does), so -the compiled ring `pc` is STALE/unreliable for labeling. The `cpu` command reads the REAL -debug_cpu_ptr->pc, which IS reliable. (Regen game+BIOS to make ring pc labels meaningful.) - -## ★ cp32 RESOLVED: it was `pc` currency, a FALSE POSITIVE (2026-06-30) - -CPU field-diff at cp32: the ONLY differing field is `pc` (compiled=0x00000000, -interp=0x80054240). EVERY gpr/cop0/hi/lo/micro-state IDENTICAL. Root: the compiled -backend does not keep cpu->pc current mid-execution (writes it only at block transfers; -transiently 0 between dispatches), while interp keeps it exact per-instruction — so at a -mid-instruction cycle checkpoint they hold different pc while in the SAME architectural -state. FIX: excluded cpu->pc from the cross-backend hash (cosim_state.c hash_cpu; still -reported via the `cpu` command). NOT a blind spot — a real control-flow split shows as a -differing gpr/memory value within one checkpoint (stride 262144 = ~50k instructions). -Gate 1 (compiled-vs-compiled) could not catch this because both sides share pc-currency. -REBUILDING (task b2n8aw3v0). This is the tool honing itself exactly as the gates intend: -it surfaced a representational difference at a clean bracketed point with the exact field -named, instead of letting it corrupt a result. - -## ★★ SECOND RESULT (pc excluded): first REAL divergence at cp=42, frame ~18 - -Re-run compiled-vs-interp with pc excluded: CPU now MATCHES (pc fix worked), tool stepped -past cp32 and halted at cp=42, cycle 10,747,916 (frame ~18), same cycle both sides. The -differing subsystems are `irqctl` AND `gpu` — `cpu`, `ram`, `scratch`, `clock`, `timers`, -cdrom, dma, sio, spu ALL identical. This is DEVICE/INTERRUPT-TIMING state, not CPU/data — -cannot be a pc-currency artifact (both drive devices via the same psx_advance_cycles). -`irqctl` = cycles_since_vblank + in_exception + i_mask (+ i_stat, which the window shows =1 -on both). HYPOTHESIS (unconfirmed, must field-check): VBLANK is raised at a different guest -cycle because compiled checks interrupts at BLOCK edges while interp checks per-instruction -→ cycles_since_vblank diverges AND gpu_vblank_tick (GPUSTAT LCF) diverges → both irqctl+gpu -split. That IS the "when is the event taken" mechanism, at its first real occurrence. -CONFIRM: added `dev` TCP cmd (csv=cycles_since_vblank, inexc, imask, istat, gpustat) + -coordinator prints DEV field-diff in --cpudiff-at-cp mode. REBUILDING (task b8nhp6ub1). -Run: `python tools/cosim.py --a compiled --b interp --stride 262144 --cpudiff-at-cp 42`. - - dev.csv differs => CONFIRMED VBLANK-raise-timing divergence (interrupt-check frequency - differs between backends). The faithful fix: raise VBLANK (and all scheduled device - events) at their exact due guest-cycle in BOTH backends, independent of how often - psx_check_interrupts is called — i.e. a due-cycle event scheduler, not a check-at- - block-leader model. (This is the class fix CLAUDE.md/FAITHFUL_TIMING_PLAN point at.) - - dev.gpustat differs but csv same => a GPU-internal divergence (could be the draw-time / - GPUSTAT-ready model = the GPU-backpressure hypothesis, NOW testable at this exact point). - - Add per-field device dumps as needed to pin the exact GPU field. - -## ★★★ CONFIRMED (2026-06-30): first divergence = VBLANK raised at a different guest cycle - -DEV field-diff at cp42 (cycle 10,747,916, frame ~18): - dev.csv (cycles_since_vblank): compiled=0x590c (22796) interp=0x8f60c (587788) - dev.gpustat: compiled=0x94800000 interp=0x14800000 (bit31 LCF differs) -VBLANK_CYCLES=564480. Compiled's last VBLANK was ~22796 cyc ago; interp is at 587788 -(PAST the threshold, next VBLANK not yet fired). Their last-VBLANK cycles differ by -~564992 ≈ ONE VBLANK period → compiled has raised ONE MORE VBLANK than interp at the same -guest cycle; GPUSTAT LCF (bit31, toggled by gpu_vblank_tick) confirms. So the FIRST real -compiled-vs-interp divergence is EVENT-PHASE: VBLANK is raised at a different guest cycle -between backends. This is the measured root of the "when is the event taken" class; the -FMV skip 1600 frames later cascades from it. cpu.pc also shows (A=0 transient, excluded). - -WHY (next investigation — cycles_since_vblank increments identically via psx_advance_cycles -in both; it only RESETS inside psx_check_interrupts when cycles_since_vblank>=VBLANK_CYCLES): -whichever backend calls psx_check_interrupts first after crossing the threshold fires+resets -first. Candidates: (a) psx_check_interrupts call FREQUENCY differs (compiled at block leaders -vs interp per-instruction) so one crosses/fires at a different cycle; (b) the VBLANK DEFER -logic (interrupts.c ~line 388: defer while sio_card_protocol_active() / progress_stale) -evaluates differently between backends. SIO sub-hash matched at cp42, so lean toward (a). -FAITHFUL FIX DIRECTION: raise VBLANK (and all scheduled device events) at their EXACT due -guest-cycle independent of psx_check_interrupts call frequency — a due-cycle event scheduler -(FAITHFUL_TIMING_PLAN.md). CONFER ChatGPT before implementing. Then rebuild psx-cosim, -re-run compiled-vs-interp: the first divergence should move much later; iterate. - -## NEXT (immediate): re-run compiled-vs-interp with pc excluded -`python tools/cosim.py --a compiled --b interp --stride 262144 --max 1000000000` -Expected: proceeds well past cp32 toward the FMV-skip (~frame 1650 / ~933M cyc). Outcomes: - - Halts near frame 1650 with a CPU (cop0/gpr) or device subsystem diff => THE bug's first - divergence. Use `--cpudiff-at-cp N` (or add device field dumps) to name the exact field. - - Halts EARLIER on another currency artifact => field-diff it; if it's another - mid-instruction-currency field (unlikely — only pc had this), exclude/normalize similarly. - - Reaches frame 1650 with NO divergence => the FMV bug is NOT in the guest-architectural - state the hash covers at this stride; drill finer, or the divergence is in a device - internal not yet in the snapshot (audit boot_state device _snapshot completeness). - -## NEXT STEPS (in order) - -1. Finish the `psx-cosim` build; guard any `PSX_NO_DEBUG_TOOLS` link/compile errors. -2. **Why regen is optional for v1:** the compare is the cycle-keyed full-state hash in - `psx_advance_cycles` (works without regen). `cosim_block` only supplies a human-readable - last-leader-PC for the report. So you can run the gates + first compiled-vs-interp pass - on the CURRENT generated C, then regen (game+BIOS) later for nicer PC labels. -3. **RUN THE GATES (do not skip — this is the whole point):** - - Gate 1: `python tools/cosim.py --a compiled --b compiled` → MUST report zero - divergence across boot. If it diverges, the clean build still has NONDETERMINISM - (threads/host-time) OR the hash includes host-only state — fix that FIRST. Likely - suspects: SPU worker thread, overlay compile worker, any wall-clock in the guest path, - autocompile. The clean build should be single-threaded; verify no worker threads run. - - Gate 1b: `--a interp --b interp` → zero divergence. - - Gate 4: `--a compiled --b compiled --inject-at --inject ram:100000:1` → MUST - halt at ~that cycle. If it says "no divergence," the tool is BLIND — fix before trusting. -4. **THE RUN:** `python tools/cosim.py --a compiled --b interp` → first divergence = - the bug. Read the `sub` diff (which subsystem's hash split first) + `window` (last N - checkpoints of both). That names it: cpu / irqctl / a device / RAM address. -5. Only THEN design the faithful fix for whatever it actually is. - -## Traps / notes - -- Determinism is the load-bearing assumption (see COSIM_ORACLE.md "two instances"). Attract - mode needs no input, which helps. Gate 1 PROVES determinism; don't believe any - compiled-vs-interp result until Gate 1 + Gate 4 pass. -- Reading parked state must be side-effect-free (the device _snapshot_write serializers are - reads; verify none mutate). -- Ports: A=4600, B=4601 (env PSX_COSIM_PORT). Coordinator sets BELOW_NORMAL priority. -- `runto` timeout in cosim.c is ~600k ms of 1ms spins; a very large `--stride`×`--max` at - interp speed can be slow — interp is ~10× slower, budget accordingly. -- Pin/reference: `build-master/psx-runtime.PIN.exe` = the working (old-pin) MMX6. `build-wt` - = the regressed accuracy build. Memory: `mmx6_regression_compiled_codegen.md`. diff --git a/CURRENT_STATE.md b/CURRENT_STATE.md deleted file mode 100644 index b197d8eb..00000000 --- a/CURRENT_STATE.md +++ /dev/null @@ -1,201 +0,0 @@ -# CURRENT_STATE — read this before doing anything - -Last updated: 2026-05-17 (Tomba OPTIONS / NEW GAME / save-load milestone) - ---- - -## TL;DR - -- **The active test binary is `F:\Projects\TombaRecomp\build\psx-runtime.exe - --game game.toml`** on debug port 4470. -- Tomba now boots through BIOS, Whoopee Camp FMV, intro FMV, and title. -- User-confirmed working on 2026-05-17: OPTIONS, NEW GAME, SAVE, and LOAD. -- NEW GAME reaches the first in-game area. -- Remaining near-term work is visual correctness: fuzzy title menu glyphs, - missing BIOS PS-logo glyph, and upcoming in-game rendering fixes. -- The historical Windows "Not Responding" hang is mitigated but should stay - under observation during longer gameplay soak tests. - ---- - -## What the project layout actually is - -There are two repos and several binaries. Get this straight before doing -anything. - -``` -F:\Projects\psxrecomp-v4\ ← THIS repo - recompiler\ - src\strict_translator.cpp ← BIOS-only path (psxrecomp-bios) - src\full_function_emitter.cpp ← shared between bios + game paths - src\code_generator.cpp ← PS1-EXE / game-only path (psxrecomp-game) - src\function_analysis.cpp ← game-only path - src\main_bios.cpp ← entry point for psxrecomp-bios.exe - src\main_psx.cpp ← entry point for psxrecomp-game.exe / PSXRecomp.exe - runtime\ ← the runtime library + psx-runtime.exe target - generated\ ← BIOS recomp output (SCPH1001_*.c) - bios\SCPH1001.BIN ← the ROM (and SCPH1001.toml audit config) - -F:\Projects\TombaRecomp\ ← sibling repo - generated\SCUS_942.36_*.c ← Tomba EXE recomp output (separate from BIOS) - build\psx-runtime.exe ← THE BINARY THE USER ACTUALLY TESTS - game.toml ← Tomba config -``` - -`psx-runtime.exe --game game.toml` in TombaRecomp is built from this repo's `runtime/` plus -`TombaRecomp/generated/`. It also links this repo's -`generated/SCPH1001_*.c` for the BIOS half. The cmake macro that does this -is `psxrecomp_v4_add_runtime_target` in this repo's `runtime/CMakeLists.txt`. - -`psx-runtime.exe` is a BIOS-only build target in this repo's `runtime/`. It's -the test runner for the discless BIOS shell (memcard / CD player screen). -Nobody has been actively testing it recently; expect it to be subtly broken. - ---- - -## Two recompilers - -| Binary | Source set | Input | Output | -|---|---|---|---| -| `psxrecomp-bios.exe` | `strict_translator.cpp` + `full_function_emitter.cpp` + `function_discovery.cpp` + `bios_slice_walker.cpp` + `main_bios.cpp` | flat BIOS ROM at 0xBFC00000 | `generated/SCPH1001_*.c` | -| `psxrecomp-game.exe` (also aliased as `PSXRecomp.exe`) | `code_generator.cpp` + `function_analysis.cpp` + `control_flow.cpp` + `ps1_exe_parser.cpp` + `main_psx.cpp` | PS-X EXE header + body | `_full.c` + `_dispatch.c` | - -They share only `mips_decoder.cpp`. **No code path is shared between BIOS -and game translation.** Fixes in one don't apply to the other. - -The seeds file (`recompiler/seeds/phase2_ghidra_seeds.json`) is the BIOS -seeds, passed positionally to `psxrecomp-bios.exe --emit-full`. Tomba seeds -live at `TombaRecomp/seeds/`. `bios/SCPH1001.toml` and -`TombaRecomp/game.toml` exist but are currently only consumed by audit tools, -not by the recompilers themselves. (This is one of the things unification -fixes.) - ---- - -## `.happypath/` — what it contains, when to use it - -- `.happypath/tomba-runtime.exe` — copy of the May 12 11:55 working binary. -- `.happypath/SCPH1001_full.c.obj` — May 11 23:49 BIOS recomp object file - (compiled with `-g` debug; .text size 0x371440 = 3.6 MB; **this is the - reference for "the BIOS output that works"**). -- `.happypath/SCPH1001_dispatch.c.obj` — same era. - -**Restore drill**: if a future session breaks tomba-runtime, copy -`.happypath/tomba-runtime.exe` back to `F:\Projects\TombaRecomp\build\` and -stop touching anything until you understand why. - -**Sanity check that we haven't drifted**: -``` -PATH=/c/msys64/mingw64/bin:$PATH objdump -h .happypath/SCPH1001_full.c.obj - → .text size MUST be 0x00371440 (debug build with -g, no -O) -``` -If a fresh BIOS regen + debug build doesn't produce a matching size, the -generated/ output has changed. - ---- - -## What is currently in working-tree (M / ??) - -These were already modified at session start (carried in from prior session): - -- `M ISSUES.md` — pre-existing Issue #1-#3; **this session also added Issue - #4 and Issue #5**. -- `M recompiler/src/code_generator.cpp` — pre-existing mid-function split - pre-pass cap raised 3 → 16 (Tomba fix; doesn't affect BIOS regen). -- `M runtime/src/debug_server.c` — pre-existing wtrace slot 24/25 ranges - for Tomba pad-poll + gflag observation. -- `D tools/_codegen_audit.py` — pre-existing rename to `codegen_audit.py`. -- `M tools/gte_audit.py`, `M tools/scan_branch_delay_hazards.py` — - pre-existing parameterization for audit-config. - -This session also added: - -- New audit infrastructure: `tools/audit_config.py`, - `tools/codegen_audit.py`, `tools/codegen_audit_game.py`, - `tools/verify_audit_schema.py`, `bios/SCPH1001.toml`. -- `docs/audit_inventory.md`. -- An encoding fix to `tools/gte_audit.py` (Unicode `✓` → ASCII `OK`). -- This file (`CURRENT_STATE.md`). -- `UNIFICATION_PLAN.md` (next-step plan). -- `.happypath/` (backup of working binaries). - -This session's recompiler-source changes were **reverted** after they -caused confusion; the recompiler/src/ tree is back to clean HEAD. - ---- - -## Why prior sessions got lost (and how to not repeat it) - -1. **The handoff said "main menu reachable, Load Game enter/exit works." - That meant Tomba's main menu, not the BIOS shell.** "Load Game" is the - in-game save-load menu inside Tomba, not the BIOS shell's load-from-disc - action. Future handoffs should explicitly write `tomba-runtime: ` - vs `psx-runtime: ` so this can't be mistaken. - -2. **Rebuilding `psxrecomp-bios.exe` is destructive in two ways**: - (a) overwrites the binary itself, (b) any subsequent `--emit-full` run - overwrites `generated/SCPH1001_*.c`. There was no backup of the prior - working `generated/`. **Now there is** — `.happypath/` has the compiled - `.obj` files for it. - -3. **Debug vs Release matters**. The working `tomba-runtime` is a Debug - build (`-g`, no `-O`). `psx-runtime`'s CMakeCache had `BUILD_TYPE=Release` - (`-O3 -DNDEBUG`). The same generated C may behave differently at the - two optimization levels. If you build the wrong one to validate, you - will mis-diagnose. **For now, always test the Debug build.** (The - underlying generated C presumably has undefined-behavior the Release - optimizer exploits — that's a real bug, but not one we're chasing yet.) - -4. **Two runtimes look symmetric but aren't**. `psx-runtime` and - `tomba-runtime` both link the same `generated/SCPH1001_*.c`, but only - `tomba-runtime` is being actively exercised. Always check `tomba-runtime` - first if a "boot" question comes up. - -5. **`generated/` is not in git** (commit 802e0fb stopped tracking it). - The recompiler is the source of truth. **If you regenerate, you cannot - easily undo.** - ---- - -## The actual broken thing right now - -`psx-runtime.exe` does not boot to the BIOS shell. It hangs in the kernel -exception handler / RestoreState loop, with VBlank pending in I_STAT but -never ACK'd. Symptoms (captured in starvation_dump.jsonl earlier): - -- Tight loop between `func_00000C80` (kernel exception handler), - `func_00000EA0` (mfc0 helper), `func_00000F40` (RestoreState), - `func_1FC40628` (the I_STAT ACK helper for CDROM bit 3 — note: NOT bit 0 - for VBlank), `func_1FC40700`, `func_1FC42BA0` (B0 trampoline at RAM 0xB0 - for `B0:0x17 ReturnFromException`), `func_000005E0` (B0 dispatcher). -- `i_stat = 0x09` (bits 0 = VBlank, 3 = CDROM pending) with `i_mask = 0x4D` - (bits 0, 2, 3, 6 enabled). CDROM gets ACK'd via `0xFFFFFFF7`; VBlank - never gets ACK'd. -- The chain handler that should ACK VBlank apparently isn't being invoked, - or returns 0 (not-handled) so the kernel doesn't ACK. - -This is *not* the immediate problem to fix — the unification plan happens -first. But note it here so it doesn't get forgotten. See `ISSUES.md` for -the full open-issue list. - ---- - -## How to verify the happy path is still happy - -```bash -# Kill any running runtimes first. -taskkill //F //IM psx-runtime.exe 2>/dev/null -taskkill //F //IM tomba-runtime.exe 2>/dev/null - -# Launch the backed-up working binary. -"./.happypath/tomba-runtime.exe" # debug port 4470 - -# In a second shell, ping it. If you get a frame number back, it booted. -python tools/_ping_tomba.py -# Expected: {"id":1,"ok":true,"frame":} -``` - -If the SDL window shows the Tomba main menu, you are at the happy path. Do -**not** rebuild `tomba-runtime` (which would re-link against whatever -`generated/SCPH1001_*.c` currently exists) without first verifying the -generated output matches `.happypath/SCPH1001_full.c.obj`'s `.text` size. diff --git a/ENHANCEMENTS.md b/ENHANCEMENTS.md index 4f0051dd..72274a5b 100644 --- a/ENHANCEMENTS.md +++ b/ENHANCEMENTS.md @@ -224,3 +224,65 @@ cost); VK correctness (Tomba2 persistent text boxes + 3-slot HUD icon; Tomba1 lo bar artifact); Tomba2 widescreen sprite-tag hooks; MMX6 Rainy Turtloid perf even on GL 16:9; and the pending MMX6 shipping-default flip software->opengl to close MegaManX6Recomp ISSUES.md #7 (GL validated this session). + +--- + +## W1 — Tomba2 16:9: USER-FLAGGED visible issues queue (2026-07-06, NOT STARTED) + +User-prioritized alongside the P0 crash work (ISSUES.md #8). None of these have +been investigated yet — this section is the work queue + every known lead. +All Tomba2, GL 16:9, worktree `_wt-tomba2-ipr` / `Tomba2Recomp` build-t2. + +### W1.1 — Black background columns in 16:9 (P3; incl. beach area) + +- **Symptom:** huge black regions where the 2D far backdrop should fill the wide + margins (user Image 1: large black sky scene); also a ~24px black column in + beach-village at game-x 85–109. +- **Class:** 2D far-backdrop doesn't cover the wide FBO edges. +- **Prior art / leads:** + - `[widescreen.bg2d]` config (recompiler/src/config_loader.cpp:699). + - Memory `ws_backdrop_preload.md` (Tomba1): far-backdrop void = early + sprite-tagged 0x65 tile grid; fix = centre-stretch gated preload. + - Memory `ws_draw_census_8c.md` (Tomba1 8C scene): void was GTE-3D driver + FUN_8004db3c; fixed via depth-gated un-squash. The **8C draw-census ring** + is the attribution instrument (always-on; query, don't arm). +- **Next step:** per-scene attribution — which producer draws the sky strips, + and why the wide margins get no tiles. Attract may cover some scenes; the + beach needs navigation (user drive or save). + +### W1.2 — 4:3 object culling visible at wide edges (P2 cull widening) + +- **Symptom:** objects pop in/out at the 4:3 boundary in 16:9 (world objects + culled by game code against 4:3 screen extents). +- **Ghidra cull sites already identified** (tomba2_ram.bin; overlay addresses — + validate per scene variant before wiring): + - `FUN_8003e030`: sltiu 0x140 @ 0x8003E228 + - `FUN_80069b6c`: addiu +0xE6 @ 0x80069B84 + sltiu 0x1CD @ 0x80069B8C + - `0x80110A08`: addiu +0x80 / sltiu 0x101 +- **Fix shape:** `[widescreen.cull]` bias_sites/range_sites per-game config + (enhancement-tier per-game shims are legitimate here). Prior art: Ape + bring-up used per-game cull imms (0x181) + signed idioms (slti/bltz) + (memory `ape_widescreen_bringup.md`). + +### W1.3 — Dialogue-box text tearing in 16:9 (P4) + +- **Symptom:** full-width dialogue text splits with gaps (user Image 2: "Water + cam…e out from th…e faucet"). +- **Cause known:** gpu.c `ws_nw_hud` thirds — left/right-third sprites are + anchored apart for HUD proportion; full-width dialogue glyphs tear at the + third boundaries. Needs a smarter anchor (e.g. detect wide text rows / + dialogue-box association) rather than blanket thirds. + +### W1.4 — Beach-area framerate (P1 perf; also user-flagged) + +- 2D isometric scenes run 0.42–0.70× (beach/village worst). The convergence + blockers are FIXED (autocapture futility backoff + entry-based coverage in + tools/compile_overlays.py — see handoff 2026-07-06); the campaign was + interrupted by ISSUES.md #8 and should resume after it: expect 2D scenes to + climb past 0.85 as coverage converges. Residual axes if short of ~1.0: + per-block pump overhead (psx_check_interrupts ~2.6M calls/s), attract + cycling. Measure ONLY with two freeze_check snapshots over a known wall + interval (executed throughput = d(psx_cycle_count) − d(cycles_skipped)); + phase_profile is a ring read and returns instantly. +- CAUTION: any pace numbers taken in diff mode before ISSUES.md #9's redesign + lands are garbage (the wedged shadow silently disabled all native dispatch). diff --git a/GTE_LM_FIX.md b/GTE_LM_FIX.md new file mode 100644 index 00000000..80820eae --- /dev/null +++ b/GTE_LM_FIX.md @@ -0,0 +1,96 @@ +# GTE `lm`/`sf` fix in the depth-cue & lighting family + +**Branch:** `feat/crashbash-fw-fixes` — commit `c4474b9` +**Fixes:** Crash Bash (SCUS-94570) menu-character 30 Hz strobe + in-game +character flattening. Root-caused and verified 2026-07-09. + +## What was wrong + +The runtime GTE hardcoded `lm = 1` (clamp IR results to `[0, 0x7FFF]`) on +every IR write-back in the shared lighting/depth-cue helpers +(`light_transform`, `light_color`, `color_output`, `depth_cue_from_ir`). +Real hardware takes `lm` from **bit 10 of the instruction word** — `lm = 0` +clamps to `[-0x8000, 0x7FFF]` and preserves negative results. + +`depth_cue_from_ir` also ignored `sf` (bit 19) and skipped the hardware's +intermediate step clamp. + +Affected ops: `INTPL`, `DPCS`, `DPCT`, `DPCL`, `NCDS`, `NCDT`, `NCS`, `NCT`, +`NCCS`, `NCCT`, `CDP`, `CC`. (`RTPS`/`RTPT`/`MVMVA`/`OP`/`SQR`/`GPF`/`GPL` +already honored the instruction word and are untouched.) + +## Why it broke Crash Bash + +Crash Bash tweens its 10 fps keyframe vertex animation to the 30 fps render +rate through **INTPL with `lm = 0`** — an off-label, signed-vector use of the +color-interpolation op: + +- pose-A vertex → IR1–3 +- pose-B vertex → far-color regs (FC) via `ctc2` +- blend factor → IR0 +- `INTPL` (`0x0980011`) → lerped vertex read back from IR1–3 (`mfc2`) + +With `lm` forced on, every **negative component** of a tweened vertex clamped +to 0, collapsing the character mesh onto its anchor point on tween frames. +Keyframe frames (no INTPL) stayed healthy, so with alternating draw buffers +exactly one display buffer ever held a healthy pose → the character strobed +at 30 Hz on menus and rendered flattened/half-visible in-game. Beetle and +real hardware honor per-instruction `lm`, so they never showed it. + +## What changed + +- `light_transform` / `light_color` / `color_output`: `lm` read from the + instruction word and passed to every `saturate_ir` write-back. +- `depth_cue_from_ir`: rewritten to the faithful hardware formula — + `base = IR << 12`; `step = lim±0x8000(((FC << 12) − base) >> sf·12)` + (intermediate clamp with `lm` forced OFF, per hardware); `MAC = (base + + IR0·step) >> sf·12`; final `IR = lim(MAC, lm)` with `lm` from the + instruction. MAC overflow flags checked. +- Always-on GTE observability rings (build-dbg debug-server commands): + `gte_ring_dump` (per-RTPS/RTPT inputs+outputs), `gte_intpl_dump` + (per-INTPL inputs+outputs, frame filter + offset paging), + `gte_frame_stats` (per-frame `nproj`/`nsat`/`nflat`/`nintpl`/ + `nintpl_tiny`), `gte_latch_dump` (retained saturated projections). + +## Regression risk + +Low, and skewed toward correctness: + +- Standard libgte **lighting** calls encode `lm = 1`, so games using the + ops as designed see identical clamping to before. +- The two real behavior changes are (a) `lm = 0` callers now keep negative + results — the bug being fixed — and (b) the depth-cue step is now clamped + to ±0x8000 before the IR0 multiply, which only matters when + `|FC − IR| > 0x7FFF` (far color very distant from the lit color). Both + changes move us **toward** hardware/Beetle behavior, so the only regression + class is a game that accidentally depended on our old inaccuracy. +- `sf` handling: all common libgte encodings of these ops use `sf = 1`, + which reproduces the old shift exactly. + +## Cross-game validation — REQUIRED, all titles, before merge + +This branch does not merge to master until every supported title has been +validated against this change. No subset, no sampling. + +**Validated:** + +**Crash Bash (SCUS-94570)** — the fix target. Menu strobe gone; +`nintpl_tiny` 335–352/473 → 0–3; character present in both VRAM +double-buffers and in all frames of a jittered screenshot burst; +menus/animation healthy. + +**Pending (every one required):** + +**Mega Man X6**, **Tomba 2**, **Ape Escape**. Ape Escape is the highest-risk +of the three — a heavy GTE user and the most likely to exercise the +depth-cue step clamp with distant far colors; Tomba 2 and MMX6 exercise the +standard `lm = 1` lighting path that must be bit-identical to before. + +**Per-title procedure (same for all):** + +1. Boot to gameplay through the title's existing verify flow on this branch. +2. Capture `gte_frame_stats` over a representative scene and diff + `nsat`/`nflat` against a pre-fix baseline built from master + this + branch's parent commit — any shift is a stop-and-investigate, not a note. +3. Eyeball lit and fogged/depth-cued scenes side-by-side with the Beetle + oracle; the oracle is ground truth, not the pre-fix recomp. diff --git a/HANDOFF.md b/HANDOFF.md deleted file mode 100644 index 0fac3998..00000000 --- a/HANDOFF.md +++ /dev/null @@ -1,108 +0,0 @@ -# PSXRecomp v4 — Session Handoff (Phase 4, mid-session) - -Read first, in order: - -1. `CLAUDE.md` -2. `PLAN.md` -3. `PRINCIPLES.md` -4. `DEBUG.md` - -Then state exactly: - -`Architecture A is locked. No interpreter. No HLE. No stubs. BIOS first. Game never until Phase 5.` - ---- - -## Current state (DO NOT REDO — all of this is built and working) - -### Infrastructure complete - -| Component | File | Status | -|-----------|------|--------| -| SIO controller state machine | `runtime/src/sio.c` | Ported from v3, audited, wired into memory.c | -| Memory card (SIO protocol) | `runtime/src/memcard.c` | Ported from v3, audited, 128KB .mcd files auto-created | -| SDL keyboard → SIO pad | `runtime/src/main.cpp` | Arrows/XZSA/Enter/RShift mapped to PS1 digital pad | -| TCP debug server | `runtime/src/debug_server.c` | Port 4370, 36K-frame ring buffer, 28 commands | -| GPUSTAT poll-triggered vblank | `runtime/src/gpu.c` | Prevents infinite spin in BIOS VSYNC wait loops | - -### What boots and runs - -- BIOS executes from reset vector (0xBFC00000) -- Sony logo renders (red diamond visible on screen — confirmed via screenshot) -- BIOS reaches shell main loop (confirmed via RA=0xBFC41D18 in diagnostics) -- No dispatch misses, no crashes -- TCP debug server responds to all commands (ping, get_registers, irq_state, read_ram, screenshot, etc.) -- Memory cards initialized with correct PS1 blank format (verified byte-for-byte) -- Controller connected on port 1, keyboard input feeds into SIO - -### Known issue: IEc=0 (interrupts disabled) - -COP0 SR is always 0x00000404 (IEp=1, IEc=0, IM2=1). The BIOS never enables IEc. - -**This blocks:** -- VBlank interrupt delivery → BIOS animation timers don't advance -- The Sony logo → shell transition (which likely needs VBlank counting) -- Any interrupt-driven BIOS behavior (event chain processing) - -**What we know:** -- Exception handler IS installed at 0x80000080: `lui $k0,0 / addiu $k0,0x0C80 / jr $k0 / nop` → handler at RAM 0xC80 (ROM 0xBFC10780), IS in dispatch table -- B0 trampoline IS installed at 0xB0: target 0x5E0 (ROM 0xBFC100E0), IS in dispatch table -- i_mask=0x0000000D (VBLANK+CDROM+DMA enabled) — BIOS DID set up the mask -- i_stat=0x00000009 (VBLANK+DMA pending) — interrupts are pending but never delivered -- ExitCriticalSection (func_1FC0D8C0, syscall $a0=2) exists and is recompiled correctly -- The BIOS never calls ExitCriticalSection during the observed execution - -**Root cause candidates (investigate in order):** -1. The BIOS boot code path that enables IEc may go through a dispatch miss that was silently nop'd (check if `psx_unknown_dispatch` ever fires during early boot by adding a one-time counter) -2. The BIOS may enable IEc via MTC0 inside a function that has an early `return` due to a recompiler bug (e.g., alignment check bailing out) -3. The RFE at 0xBFC10B14 in the exception handler has a `jr $k0` that may not be emitting `cpu->pc = cpu->gpr[26]` in the generated code — check SCPH1001_full.c line 36416 for this - -**Investigation approach (per DEBUG.md):** -1. Use TCP debug server: `{"cmd":"irq_state"}` to confirm IEc -2. Use Ghidra to find the BIOS function that calls ExitCriticalSection during shell init -3. Trace why that function is never reached -4. Fix the root cause (recompiler, dispatch, or traps.c) - ---- - -## Regeneration (idempotent) - -```bash -cd F:/Projects/psxrecomp-v4 -PATH=/c/msys64/mingw64/bin:$PATH - -# Only if recompiler changed: -cd recompiler/build && cmake --build . && cd ../.. - -# Only if seeds changed: -python3 tools/gen_phase2_seeds.py -recompiler/build/psxrecomp-bios.exe bios/SCPH1001.BIN generated/ \ - --emit-full recompiler/seeds/phase2_ghidra_seeds.json - -# Runtime build: -cd runtime/build && cmake --build . && cd ../.. - -# Run (ONE INSTANCE ONLY — always taskkill first): -taskkill //F //IM psx-runtime.exe 2>/dev/null -cd runtime/build && ./psx-runtime.exe ../../bios/SCPH1001.BIN -``` - ---- - -## Hard constraints - -- No HLE behavior anywhere -- No stubs -- No direct BIOS state mutation -- No printf debugging (use TCP debug server) -- No generated C edits -- ONE runtime instance at a time (taskkill before launching) -- Do not touch unrelated systems unless needed for the IEc fix - ---- - -## Next milestone - -Fix IEc=0 so VBlank interrupts fire → BIOS logo animation completes → BIOS shell renders with text → controller input navigates shell → memory card screen works. - -Phase 4 is complete ONLY when all of that works with no fake events. diff --git a/HANDOFF_OVERNIGHT_2026-04-06.md b/HANDOFF_OVERNIGHT_2026-04-06.md deleted file mode 100644 index 60b5c882..00000000 --- a/HANDOFF_OVERNIGHT_2026-04-06.md +++ /dev/null @@ -1,368 +0,0 @@ -# PSXRecomp v4 — Overnight Session Hand-off (2026-04-06) - -> **Audience:** a fresh Claude Code session that has been context-cleared -> and given this file as the canonical entry point. Matthew is asleep -> and will NOT be available to answer questions. You are running -> autonomously overnight. - ---- - -## 1. READ FIRST (in this exact order, no skipping) - -1. `CLAUDE.md` — the project constitution. Re-read it in full. -2. `PLAN.md` — Phase 1 sub-phase structure, hard gates, proof-driven rules. -3. `FIRST_MILESTONE.md` — Phase 1a frozen contract. -4. `README.md` — current milestone pointer. -5. `C:\Users\Matthew\.claude\projects\F--Projects-psxrecomp-v4\memory\MEMORY.md` — auto-memory index. Then read every linked memory file. The most important entries: - - `strict_translator_stays_strict.md` — never add fallthrough cases - - `phase_1a_complete.md` — boot slice baseline frozen 2026-04-06 - - `phase_1a_baseline_changed_for_alignment.md` — the baseline was DELIBERATELY updated during B(7); current baseline is post-B(7), not the original 2026-04-06 form - - `srlv_isa_completeness_exception.md` — SRLV is the ONE allowed no-inventory-hit opcode. Don't generalize. - - `rfe_substitutes_for_eret.md` — RFE handles the FIRST_MILESTONE.md "ERET" terminator. - - `env_path_ordering.md` — `PATH=/c/msys64/mingw64/bin:$PATH` is REQUIRED for every cmake / gcc / psxrecomp-bios.exe invocation. - - `matthew_communication_style.md` — terse, decisive, absolute no-stubs. - -After reading, state out loud (in your first response): -- "Architecture A is locked. No interpreter. No HLE. No stubs. BIOS first. Game never until Phase 5." - -If any of the above files cannot be read, **stop and write a status file `OVERNIGHT_STATUS.md` explaining the blocker.** Do not proceed. - ---- - -## 2. CURRENT STATE (as of session end, 2026-04-06 ~22:30 PT) - -### Phase 1a — COMPLETE -Boot slice `0xBFC00000..0xBFC00074`, 30 instructions, terminates at `j 0xBFC00150`. Frozen. Do NOT redo. - -### Phase 1b — INSTRUCTION COVERAGE COMPLETE -The strict translator (`recompiler/src/strict_translator.cpp`) covers **54 inventory opcodes implemented + 2 implemented-but-unused (RFE, SRLV) + 2 deferred FPU (COP1, LWC1) = 58 total opcode classifications** out of the 56 inventory buckets. Every BIOS opcode in any walker-reachable region has a real audited implementation with the following exceptions: -- COP1 (17 hits) and LWC1 (2 hits): FPU instructions in the BIOS `atof` routine. PS1 R3000A has no FPU. Reachability unknown. Deferred until Phase 1c proves whether they're live. - -#### Phase 1b sub-phase history (in order) -- **B(1)** — ALU R-type: ADD, ADDU, SUBU, AND, OR, XOR, NOR, SLT, SLTU. ADD overflow trap via `psx_arith_overflow` extern. -- **B(2)** — ALU I-type: ADDI, ANDI, XORI, SLTI, SLTIU. ADDI overflow trap via the same extern. SLTIU sign-extends the immediate first then unsigned-compares. -- **B(3)** — Shifts: SRL, SRA, SLLV, SRAV (and SLL was already in Phase 1a). **Plus SRLV via the ISA-completeness exception** — the ONLY opcode in the strict translator implemented without an inventory hit. Matthew explicitly approved this carve-out and it must NOT be generalized. -- **B(4)** — HI/LO + multiply/divide: MFHI, MFLO, MULTU, DIV, DIVU. DIV/DIVU divide-by-zero and INT32_MIN/-1 edge cases handled per documented R3000A behavior. **Remainder computed as `r = n - q*d`** (NOT `n % d`) to be host-`%`-independent — this was Matthew's correction during B(4). -- **B(5)** — Conditional branches: BEQ, BNE, BLEZ, BGTZ, BLTZ, BGEZ. **Pre-delay snapshot pattern via `TranslateResult.pre_delay_code`** — added because 441 of 4617 branches in the BIOS use the operand-mutating delay slot idiom. Snapshot variables are `psx_brA_` / `psx_brB_` declared at function scope before the delay slot, read in the branch terminator c_code. Verified by `tools/scan_branch_delay_hazards.py` which reports `incorrect_in_current_translator: 0`. -- **B(6)** — Loads: LB, LBU, LH, LHU, LW. Sign-extension via `(int32_t)(int8_t)` / `(int32_t)(int16_t)`. `$zero` writes use `(void)cpu->read_*(addr)` to preserve MMIO side effects. Alignment checks added in B(7) — see below. -- **B(7)** — Stores: SB, SH, SWL, SWR. **Also retroactively added alignment-fault checks to LH, LHU, LW, SH, SW** via `psx_unaligned_access` extern. **Phase 1a baseline byte-output for `boot_slice.c` was DELIBERATELY updated** to add the SW alignment guard. The new baseline is the canonical baseline going forward. -- **B(8)** — COP0 access: MFC0, MTC0. Direct `cpu->cop0[rd] <-> cpu->gpr[rt]` transfer with no SR-write hooks, no IRQ delivery, no exception modeling. `cop0[0]` is the Index register, NOT a hardwired zero — do not apply `$zero` rules to `cop0[]`. -- **B(9)** — Unaligned word loads: LWL, LWR. Little-endian R3000A semantics. Merge with old `rt` value via `psx_old_rt = cpu->gpr[rt]` snapshot. No alignment fault by design. -- **B(10)** — Traps: SYSCALL, BREAK. Both are **non-terminators** with `extern_call(...); return;` c_code. Decision forced by the walker: marking them as terminators would force the walker to fetch the next instruction as a "delay slot", and at e.g. `0xbfc0d8c4` the post-syscall instruction is `jr $ra` which is itself a terminator → walker would fail loud. SYSCALL passes `cpu->gpr[2]` (the PS1 ABI syscall number register), BREAK passes the 20-bit immediate from `(raw >> 6) & 0xFFFFF` plus the PC. - -#### What still exists in `missing` / `deferred` / `impl-but-unused` -- `missing`: **0** -- `deferred`: COP1 (17 hits, FPU), LWC1 (2 hits, FPU). Both in `FUN_bfc02324` (an `atof` routine). Reachability TBD in Phase 1c. -- `impl-but-unused`: RFE (lives in copy-to-RAM exception handler not in walker scope), SRLV (ISA-completeness exception). - -### Generated artifacts (all up-to-date) -All files in `generated/`: -- `boot_slice.c` — 30 instructions, post-B(7) form (with SW alignment guard) -- `boot_slice.o` — compiles cleanly with `gcc -c -Wall -Wextra` -- `boot_slice_manifest.json` — unchanged from Phase 1a -- `unsupported_ops.json` — empty `[]` -- `cpu_state.h` — 4 forward-declared externs: `psx_syscall`, `psx_arith_overflow`, `psx_unaligned_access`, `psx_break` -- `instruction_inventory.json` — 56 buckets, 44,629 instructions walked -- `instruction_coverage.json` — 54 implemented / 0 missing / 2 deferred / 2 impl-but-unused -- `branch_delay_hazards.json` — 4617 branches scanned, 441 have operand-mutating delay slots (informational — handled correctly via snapshot), `incorrect_in_current_translator: 0` -- `ghidra_function_starts.json` — all 666 Ghidra function entry points + 9 verified function bodies - -### Tools (all in `tools/`) -- `build_instruction_inventory.py` — BFS reachability walker over the BIOS, deduped, classified, regenerates `instruction_inventory.json`. Verified against 9 Ghidra-sampled function-end checks. **NEVER MODIFY THE WALKER LOGIC** without explicit instruction. -- `build_instruction_coverage.py` — joins inventory against the strict translator's hand-maintained "implemented" table, regenerates `instruction_coverage.json`. -- `scan_branch_delay_hazards.py` — proves the snapshot fix is in place. Reports `incorrect_in_current_translator: 0`. Reads `recompiler/src/strict_translator.cpp` source to verify each branch handler uses the snapshot pattern. **If this scanner ever reports nonzero, STOP and write OVERNIGHT_STATUS.md.** -- `_dump_inventory.py`, `_dump_suspect.py`, `_dump_unknown.py`, `_demo_branch_emit.py` — small dev helpers, leave alone. - -### Recompiler binary -`recompiler/build/psxrecomp-bios.exe` — built. Re-run with: -```bash -PATH=/c/msys64/mingw64/bin:$PATH ./recompiler/build/psxrecomp-bios.exe bios/SCPH1001.BIN generated -``` - -### Documentation -- `PLAN.md` — Phase 1 fully expanded with sub-phases. Phase 1c is the next major milestone. -- `README.md` — current milestone pointer is up-to-date. -- `FIRST_MILESTONE.md` — frozen Phase 1a contract. - ---- - -## 3. WHAT YOU MAY DO OVERNIGHT - -These are tasks that are **safe to perform autonomously** because they're either: -- Pure analysis (no source changes) -- Self-validating (regression baselines exist) -- Strictly within Phase 1b's scope, with the exit gate already met - -### Priority 0 — verify the handoff (do this FIRST, every time you boot) -Run, in order: - -```bash -# 1. Verify Ghidra MCP is reachable (per CLAUDE.md session-start checks) -# Use mcp__ghidra__get_program_info — should return SCPH1001.BIN @ bfc00000..bfc7ffff, 666 functions - -# 2. Rebuild psxrecomp-bios from a forced-clean .obj -cd F:/Projects/psxrecomp-v4/recompiler -rm -f build/CMakeFiles/psxrecomp-bios.dir/src/strict_translator.cpp.obj \ - build/CMakeFiles/psxrecomp-bios.dir/src/bios_slice_walker.cpp.obj -PATH=/c/msys64/mingw64/bin:$PATH /c/msys64/mingw64/bin/cmake.exe --build build --target psxrecomp-bios - -# 3. Snapshot baseline, regenerate, diff -cd F:/Projects/psxrecomp-v4 -cp generated/boot_slice.c /tmp/bs_pre.c -cp generated/boot_slice_manifest.json /tmp/bsm_pre.json -cp generated/unsupported_ops.json /tmp/uo_pre.json -PATH=/c/msys64/mingw64/bin:$PATH ./recompiler/build/psxrecomp-bios.exe bios/SCPH1001.BIN generated -diff -q /tmp/bs_pre.c generated/boot_slice.c -diff -q /tmp/bsm_pre.json generated/boot_slice_manifest.json -diff -q /tmp/uo_pre.json generated/unsupported_ops.json -# All three diffs MUST be silent (no output). If any prints a difference, STOP. - -# 4. Compile-check -PATH=/c/msys64/mingw64/bin:$PATH /c/msys64/mingw64/bin/gcc.exe -c -Wall -Wextra -I generated generated/boot_slice.c -o generated/boot_slice.o -# Exit must be 0, no warnings. - -# 5. Regenerate inventory + coverage + hazard scan -PATH=/c/msys64/mingw64/bin:$PATH python tools/build_instruction_inventory.py -PATH=/c/msys64/mingw64/bin:$PATH python tools/build_instruction_coverage.py -PATH=/c/msys64/mingw64/bin:$PATH python tools/scan_branch_delay_hazards.py -# Expected: -# inventory: 56 buckets, 44629 instructions, 0 unknown -# coverage: 54 implemented, 0 missing, 2 deferred, 2 impl-but-unused -# hazard: incorrect_in_current_translator: 0 -# Any deviation = STOP, write OVERNIGHT_STATUS.md. -``` - -### Priority 1 — strict-translator audit pass (ANALYSIS ONLY, no edits) -Read `recompiler/src/strict_translator.cpp` end-to-end. For each opcode case, verify: -- `r.supported = true;` is set on the success path -- The c_code matches the documented MIPS-I R3000A semantics -- `$zero` handling is correct (either via `emit_gpr_write` or via an explicit `if (rt == 0)` branch that still does any side effects) -- Any signed-int operation that could overflow has a guard or uses unsigned arithmetic -- Variable shift counts use `& 0x1Fu` masks -- Branch targets use `simm * 4` not `simm << 2` (to avoid signed-shift UB) -- Branches use the snapshot pattern (`psx_brA_` / `psx_brB_` only, no `cpu->gpr[` in the c_code) -- Loads/stores compute `addr = (uint32_t)((int32_t)cpu->gpr[base] + (simm))` -- Aligned ops (LH/LHU/LW/SH/SW) check `addr & MASK` before accessing memory and call `psx_unaligned_access(cpu, psx_addr, 0xPCu); return;` on misalignment - -Write findings to a NEW file `generated/strict_translator_audit_2026-04-06.md` (do NOT overwrite anything). If you find a real bug, STOP and write `OVERNIGHT_STATUS.md` explaining it. Do NOT fix the bug — fixes need Matthew's approval. - -### Priority 2 — proof artifact: spot-check a sampled BREAK and SYSCALL -Use `mcp__ghidra__get_code` (format: `disassembly`) on the function containing the first BREAK example (`0xbfc012cc`) and the first SYSCALL example (`0xbfc0d8c4`). Confirm the surrounding code matches the patterns the strict translator expects: -- BREAK should follow a divide instruction or a conditional check (compiler-inserted trap). -- SYSCALL should be in a thin wrapper function with `addiu $a0/$a1/$a2/...; syscall; jr $ra; nop` pattern. - -Write findings to `generated/break_syscall_spot_check_2026-04-06.md`. No source edits. - -### Priority 3 — additional sampled function-end verification -The current `ghidra_function_starts.json` has 9 verified function bodies. Pick ~10 more random functions across the BIOS (not adjacent to the existing 9) and call `mcp__ghidra__get_function_info` on each to capture their bodies. Add them to the `verified_function_bodies` array in `ghidra_function_starts.json`. Re-run `tools/build_instruction_inventory.py` — if any new sample fails verification, STOP. If all pass, you've strengthened the regression baseline. - -**Constraint:** when adding to the verified list, preserve the existing 9 entries exactly. Append new ones. - -### Priority 4 — write `PHASE_1B_COMPLETE.md` -Create a NEW file `PHASE_1B_COMPLETE.md` (do NOT overwrite anything else) summarizing: -- What Phase 1b delivered (54 ops, with the table from this hand-off's section 2) -- What Phase 1a invariants are still preserved (boot slice byte-identical, etc.) -- What is deferred to Phase 1c (FPU reachability, RFE coverage via copy-to-RAM walk) -- What is deferred to Phase 2 (the four trap externs: psx_syscall, psx_arith_overflow, psx_unaligned_access, psx_break) -- The pre_delay_code contract for branches (with the example from `_demo_branch_emit.py`) -- The SRLV ISA-completeness exception -- The Phase 1a baseline change for alignment guards - -Match the tone of `FIRST_MILESTONE.md` — terse, declarative, no fluff. Around 200-300 lines. - -### Priority 5 — Phase 1c PREP (planning only, no code) -Phase 1c is "function discovery pipeline" per `PLAN.md`. The CURRENT state has: -- 666 Ghidra function entry points dumped -- A BFS walker that visits each function's reachable instructions -- A coverage report - -What Phase 1c needs that DOESN'T YET EXIST: -- A C++-side function-discovery walker integrated into the recompiler (not just the Python analysis tool) -- An explicit seed file `recompiler/seeds/phase1c_seeds.json` -- Per-function manifest output `function_manifest.json` with edges, termination reasons, lineage -- Indirect-jump recording for Phase 1d -- Deduplication via normalized addresses (Phase 1e relocation work) - -Write a Phase 1c TODO list to `generated/phase1c_prep_2026-04-06.md`. **Do NOT begin implementing Phase 1c.** This is planning only. - ---- - -## 4. WHAT YOU MUST NOT DO OVERNIGHT - -These are HARD prohibitions. Violating any of these = stop immediately and write `OVERNIGHT_STATUS.md`. - -### Source code -- ❌ Do NOT modify `recompiler/src/strict_translator.cpp`. Phase 1b is closed. -- ❌ Do NOT modify `recompiler/src/strict_translator.h`. The TranslateResult contract is frozen. -- ❌ Do NOT modify `recompiler/src/bios_slice_walker.cpp` or `.h`. The walker is frozen. -- ❌ Do NOT modify `recompiler/src/code_generator.cpp` (the salvaged v3 code). It is NOT trusted. -- ❌ Do NOT modify `generated/boot_slice.c` or `generated/cpu_state.h`. Build artifacts and hand-maintained header. -- ❌ Do NOT add new opcodes to the strict translator, even ones that look "obviously needed". -- ❌ Do NOT remove SRLV (the ISA-completeness exception) or RFE (the unused-but-implemented terminator). -- ❌ Do NOT add MULT, MTHI, MTLO, BLTZAL, or BGEZAL — Matthew explicitly rejected these as speculative. -- ❌ Do NOT touch the Python BFS walker logic. Adding NEW analysis scripts is fine; modifying the existing inventory walker is not. -- ❌ Do NOT modify any test/build infrastructure (CMakeLists.txt, etc.). - -### Architectural moves -- ❌ Do NOT begin Phase 1c implementation. Planning is OK; coding is not. -- ❌ Do NOT begin Phase 1d (indirect jump recording). -- ❌ Do NOT begin Phase 1e (relocation planning). -- ❌ Do NOT create a runtime/ directory. -- ❌ Do NOT add MMIO, GPU, interrupts, DMA, timers, SIO, memcard, CD-ROM, SPU, or any peripheral simulation. -- ❌ Do NOT load a game ISO, PS-X EXE, or anything game-related. -- ❌ Do NOT walk the full BIOS (the BFS walker over Ghidra functions is OK because it's analysis; a recompiler-side full walk is not). -- ❌ Do NOT write any C runtime that defines `psx_syscall`, `psx_break`, `psx_arith_overflow`, or `psx_unaligned_access`. Their forward-declared / link-fail-by-design contract is intentional. -- ❌ Do NOT write a MIPS interpreter for ANY reason. -- ❌ Do NOT add any HLE BIOS shim. -- ❌ Do NOT touch DuckStation. -- ❌ Do NOT push to a remote, do not commit (Matthew controls git). - -### Process -- ❌ Do NOT use `fprintf`, log files, or any printf debugging. The TCP debug server doesn't exist yet; structured JSON artifacts are the only output channel. -- ❌ Do NOT speculate. If something is unknown, write it as `unknown` in an artifact and move on. The phrases "probably", "should work", "looks correct", "seems like" are forbidden in any artifact you write. -- ❌ Do NOT create files with `// TODO`, `// FIXME`, `// XXX`, or `// for now`. The post-emission scanner will reject them and Matthew will be unhappy. -- ❌ Do NOT bulk-search-and-replace anything. -- ❌ Do NOT delete files. -- ❌ Do NOT modify memory files in `C:\Users\Matthew\.claude\projects\F--Projects-psxrecomp-v4\memory\` unless you're appending a new note about something Matthew explicitly told you to remember (he didn't, so don't). - ---- - -## 5. WHEN TO STOP AND WRITE `OVERNIGHT_STATUS.md` - -Stop and write a status file at the project root if ANY of these happen: - -- Any Priority-0 sanity check fails (rebuild fails, diff shows changes to baseline, compile errors, hazard scanner reports nonzero, inventory or coverage counts deviate from the expected values). -- You discover a real bug in the strict translator during the Priority-1 audit. Document the bug; do NOT fix it. -- Ghidra MCP is unreachable. -- A required tool fails (cmake, gcc, python). -- You're tempted to do something that's on the prohibited list. -- You're unsure whether something is allowed. **Default to "stopped" and write the status file.** -- You've completed all 5 priorities and have nothing else safe to do. (Write a "completed all overnight tasks" status.) - -The status file format: -```markdown -# OVERNIGHT_STATUS.md -Generated: -Session uptime: - -## Summary - - -## What I completed -- - -## What I did NOT complete -- - -## Findings (if any) - - -## Sanity-check results -- rebuild: pass/fail -- baseline diff: clean/dirty -- compile: pass/fail -- inventory: -- coverage: -- hazard scan: - -## Next steps for Matthew - -``` - -Place it at `F:/Projects/psxrecomp-v4/OVERNIGHT_STATUS.md`. - ---- - -## 6. ENVIRONMENT CHEAT SHEET - -### Path requirement -**Every** invocation of cmake, gcc, or psxrecomp-bios.exe MUST be prefixed with: -``` -PATH=/c/msys64/mingw64/bin:$PATH -``` -The user's default PATH is broken for the mingw64 toolchain. Ignoring this will cause "command not found" errors that look like the build is broken when it isn't. - -### Working directory -``` -F:/Projects/psxrecomp-v4 -``` -Use forward slashes in paths even on Windows. Bash is the shell. - -### Bash/Windows quirks -- Use `/dev/null`, NOT `NUL` -- Use `cp` and `diff -q`, NOT Windows equivalents -- The `Bash` tool runs in MSYS2 bash on Windows; it understands forward slashes -- Files at `C:\Users\Matthew\...` are accessible via Windows-style paths in `Read`/`Write` tool calls - -### Ghidra MCP -The Ghidra MCP server is auto-started. Tools you may need: -- `mcp__ghidra__get_program_info` — sanity check -- `mcp__ghidra__get_function_info` — verify function bodies -- `mcp__ghidra__get_code` — disassembly per function -- `mcp__ghidra__list_functions` — paginated list (use offset/limit) -- `mcp__ghidra__get_hexdump` — raw bytes at an address - -Active program: `SCPH1001.BIN`, base `0xBFC00000..0xBFC7FFFF`, 666 functions. - -### Tools you should NOT use overnight -- `Edit`, `Write` — only for new artifact files in `generated/` or new audit/planning markdown files. NEVER for source files in `recompiler/src/`, `recompiler/include/`, `bios/`, or existing files in `generated/` other than the analysis JSONs. -- `Bash` with destructive flags — no `rm -rf`, no `git push`, no `git reset --hard`. -- `Agent` (subagent dispatch) — keep work in your own context. Subagents have their own rate-limit pool and will exhaust it quickly on MCP-heavy work. Spawn one ONLY if you need to do >100 sequential MCP calls and would otherwise blow your own context. - ---- - -## 7. KNOWN PHASE 2 / PHASE 1c HAZARDS (read but do nothing about) - -These are issues that WILL need to be addressed in later phases. Knowing about them helps you understand why the current state is what it is. - -### Reset-vector blind spot -Ghidra's first analyzed function is at `0xbfc00420`. The reset vector and pre-function-1 region `0xBFC00000..0xBFC0041F` is covered by the synthetic seed walker but Ghidra does not classify it as a function. Phase 1c needs to seed `0xBFC00000` explicitly. The current Python BFS walker already does this; the C++-side recompiler walker (which doesn't exist yet) will need to as well. - -### `0xBFC00100` is NOT a valid seed -The PLAN.md correction documents this: PS1's R3000A has no TLB, so the R3000 UTLB-miss vector is unused. The bytes there are part of the "Sony Computer Entertainment Inc." copyright string. If you see `0xBFC00100` anywhere as a "seed" or "exception vector", it's wrong. - -### `0xBFC00180` IS a valid seed -The BEV=1 General Exception vector. 168 instructions reachable from there. - -### RFE lives in copy-to-RAM bytes -The exception handler is at `0x80000080` in RAM, not in the ROM-side function set. The strict translator implements RFE but the inventory walker never reaches it (hence "impl-but-unused"). Phase 1e relocation work will add the copy-to-RAM walk. - -### COP1 / LWC1 (FPU) reachability unknown -17 + 2 instances in `FUN_bfc02324` (an `atof`-style routine). PS1 has no FPU; these would trap in real hardware. Whether the BIOS actually executes them depends on whether anything in the reachable code calls `FUN_bfc02324`. Phase 1c reachability analysis will answer this. - -### 441 branches with operand-mutating delay slots -Handled correctly via the pre_delay_code snapshot pattern. The scanner verifies. Don't worry about it unless the scanner ever reports `incorrect_in_current_translator > 0`. - -### Four forward-declared trap externs -- `psx_syscall(cpu, code)` — Phase 1a -- `psx_arith_overflow(cpu)` — Phase 1b B(1) ADD/ADDI overflow -- `psx_unaligned_access(cpu, addr, pc)` — Phase 1b B(7) misalignment -- `psx_break(cpu, code, pc)` — Phase 1b B(10) BREAK trap - -ALL four are declared in `generated/cpu_state.h` and never defined anywhere. Linking the recompiled BIOS into a runnable binary will fail until the Phase 2 runtime provides definitions. This is intentional and required. - ---- - -## 8. FINAL PRE-FLIGHT CHECK - -Before you do ANYTHING, write your first response containing exactly these sections in this order: - -1. **Files read.** List every file you read from section 1 above. If any failed, stop. -2. **Architecture statement.** Verbatim: "Architecture A is locked. No interpreter. No HLE. No stubs. BIOS first. Game never until Phase 5." -3. **Sanity check results.** Run all 5 Priority-0 commands and report results. -4. **Plan.** Which of priorities 1-5 you intend to work on, in order. -5. **Stop conditions.** Restate verbatim that you will write `OVERNIGHT_STATUS.md` and stop if any of the conditions in section 5 are met. - -Do NOT skip the pre-flight check. Do NOT condense it. Matthew will read it in the morning. - ---- - -## 9. ONE LAST THING - -Matthew has been reviewing and approving every batch tonight in real time. The pace and the precision of his corrections (DIV remainder formula, branch snapshot pattern, alignment checks, SRLV exception, MULT comment) show he cares about the small details and trusts you to get them right. Don't squander that trust by inventing changes. - -If you have nothing useful to do that fits within the rules in section 4, **write a status file saying so and stop.** Sleeping is better than drift. - -Good luck. See you in the morning. - -— Claude (B(10) session, 2026-04-06 ~22:30 PT) diff --git a/HANDOFF_PHASE2_2026-04-11.md b/HANDOFF_PHASE2_2026-04-11.md deleted file mode 100644 index 42ff3fb4..00000000 --- a/HANDOFF_PHASE2_2026-04-11.md +++ /dev/null @@ -1,241 +0,0 @@ -# PSXRecomp v4 — Phase 2 Hand-off (2026-04-11) - -> **Audience:** a fresh Claude Code session. Read this FIRST, then -> follow the reading order below. - ---- - -## 1. READ FIRST (in this exact order) - -1. `CLAUDE.md` — the project constitution. Re-read in full. -2. `PLAN.md` — Phase 2 spec starts at line ~1017. -3. `PRINCIPLES.md` — debugging discipline (adapted from recomp-template). -4. `DEBUG.md` — debug loop contract. -5. `C:\Users\Matthew\.claude\projects\F--Projects-psxrecomp-v4\memory\MEMORY.md` — auto-memory index. - -Then state: "Architecture A is locked. No interpreter. No HLE. No stubs. BIOS first. Game never until Phase 5." - ---- - -## 2. GIT STATE - -Repo initialized this session. Single commit: - -``` -7c40d33 Phase 1 complete: recompiler proven correct on SCPH1001.BIN -``` - -All Phase 1 work is committed and clean. - ---- - -## 3. PHASE 1 — COMPLETE (do NOT redo) - -### Phase 1a — Boot Slice -- `generated/boot_slice.c` — 30 instructions, 0xBFC00000..0xBFC00074 -- Compiles clean with `gcc -c -Wall -Wextra` -- Frozen baseline. Do not modify. - -### Phase 1b — Instruction Coverage -- 54 opcodes in `recompiler/src/strict_translator.cpp` -- 44,629 walker-reachable instructions, 0 missing -- 2 deferred (COP1/LWC1 — FPU, PS1 has none) -- 2 impl-but-unused (RFE, SRLV) - -### Phase 1c — Function Discovery -- 76 functions from 2 seeds (0xBFC00000, 0xBFC00180) -- 2,988 instructions validated through StrictTranslator -- 199 direct call edges, 27 indirect jumps -- Seeds file: `recompiler/seeds/phase1c_seeds.json` -- Artifacts: `generated/function_manifest.json`, `function_edges.json`, `discovery_run.log.json` - -### Phase 1d — Indirect Control Flow -- 27 sites: 21 A0/B0/C0 dispatch, 4 JALR, 2 computed tail calls -- 16-instruction context windows captured for each -- Ghidra cross-referenced, 0 disagreements -- Artifacts: `generated/indirect_jumps.json`, `indirect_jump_classes.json`, `indirect_jump_ghidra_xref.json` - -### Phase 1e — Relocation -- ONE ROM-to-RAM copy: `FUN_bfc00420` - - Source: ROM 0xBFC10000..0xBFC18BEF - - Dest: RAM 0xA0000500..0x800090EF - - Length: 0x8BF0 (35,824 bytes) -- Verified by both Ghidra (static) and DuckStation (runtime, TCP debug server port 4371) -- 68/69 function entries match byte-for-byte in RAM -- Normalization: `phys = addr & 0x1FFFFFFF; if phys in [0x1FC10000, 0x1FC18BEF]: phys = phys - 0x1FC10000 + 0x00000500` -- Artifacts: `generated/address_aliases.json`, `normalization_rule.md`, `relocation_proofs/primary_copy/proof.json` - ---- - -## 4. PHASE 2 — WHAT TO BUILD - -From PLAN.md: - -**Goal:** a runtime executable that links the Phase 1 output, calls -into `func_0xBFC00000` (the recompiled reset vector), and runs the -BIOS until it hits its first MMIO access. The MMIO access fatally -errors with a clear message ("BIOS read I_STAT @ 0x1F801070, no -hardware sim available"). - -**Concrete work:** - -1. **Full BIOS recompilation** — the Phase 1c discovery only found 76 - functions from 2 seeds. The full BIOS has 666 Ghidra functions. - Before building the runtime, the recompiler needs to emit ALL - functions into `generated/SCPH1001_full.c`. This requires: - - Walking the full ROM now that Phase 1e relocation is solved - - Handling the ROM-to-RAM aliases (one C function per normalized address) - - Emitting a dispatch table (`SCPH1001_dispatch.c`) - - The strict translator handles every opcode (Phase 1b proved this) - -2. **`runtime/src/main.cpp`** — minimal SDL2 window (or even no window - initially), CPU state allocation, single call into the generated C - reset vector function. - -3. **`runtime/src/memory.c`** — RAM (2 MB), scratchpad (1 KB), BIOS ROM - region (512 KB from disk). Read/write entry points. MMIO fatal errors. - -4. **`runtime/src/cpu_state.h`** — the real CPUState struct (32 GPRs, HI, - LO, PC, COP0 registers). Replaces the compile-only `generated/cpu_state.h`. - -5. **Implement the 4 trap externs** — `psx_syscall`, `psx_break`, - `psx_arith_overflow`, `psx_unaligned_access`. These are currently - forward-declared in `generated/cpu_state.h` and intentionally undefined. - The runtime must provide real definitions. - -6. **Link and run.** Capture which MMIO address the BIOS hits first. - -**Success criterion:** the runtime starts, executes generated C, and -dies with "no hardware sim for MMIO @ 0x..." at a real BIOS MMIO read. - ---- - -## 5. KEY DESIGN DECISIONS FOR PHASE 2 - -### The generated C calling convention -Each recompiled function takes a `CPUState*` and returns void: -```c -void func_BFC00000(CPUState* cpu); -``` - -The strict translator emits code that reads/writes `cpu->gpr[N]`, -`cpu->hi`, `cpu->lo`, `cpu->cop0[N]`, and calls memory access functions -like `cpu->read_word(addr)`, `cpu->write_word(addr, val)`. - -Look at `generated/boot_slice.c` and `generated/cpu_state.h` to see -the exact interface the generated code expects. - -### Memory access pattern -The generated code calls extern functions for memory access: -- `psx_read_byte(cpu, addr)` / `psx_read_half(cpu, addr)` / `psx_read_word(cpu, addr)` -- `psx_write_byte(cpu, addr, val)` / `psx_write_half(cpu, addr, val)` / `psx_write_word(cpu, addr, val)` - -These are declared in `generated/cpu_state.h`. The runtime must implement them, -routing to RAM/ROM/scratchpad/MMIO based on the address. - -### MMIO routing -PS1 MMIO lives at physical 0x1F800000-0x1F802000: -- 0x1F800000-0x1F8003FF: scratchpad (1 KB) -- 0x1F801000-0x1F801FFF: hardware registers (I/O ports) -- 0x1F802000-0x1F802FFF: expansion 2 - -For Phase 2, all MMIO reads/writes should `abort()` with a message -identifying the address. Phase 3 will add real hardware sims. - -### Function dispatch for indirect calls -The 27 indirect jump sites need a dispatch mechanism. For Phase 2, -a simple PC-to-function-pointer lookup table suffices. The recompiler -should emit it. - ---- - -## 6. BUILD ENVIRONMENT - -```bash -# REQUIRED for every cmake/gcc/psxrecomp-bios invocation: -PATH=/c/msys64/mingw64/bin:$PATH - -# Rebuild recompiler: -cd F:/Projects/psxrecomp-v4/recompiler -PATH=/c/msys64/mingw64/bin:$PATH cmake --build build --target psxrecomp-bios - -# Run boot slice (Phase 1a regression check): -cd F:/Projects/psxrecomp-v4 -PATH=/c/msys64/mingw64/bin:$PATH ./recompiler/build/psxrecomp-bios.exe bios/SCPH1001.BIN generated - -# Run discovery (Phase 1c): -PATH=/c/msys64/mingw64/bin:$PATH ./recompiler/build/psxrecomp-bios.exe bios/SCPH1001.BIN generated --discover recompiler/seeds/phase1c_seeds.json - -# Compile boot_slice.c (self-validation): -PATH=/c/msys64/mingw64/bin:$PATH gcc -c -Wall -Wextra -I generated generated/boot_slice.c -o generated/boot_slice.o -``` - -GCC 15.2.0, CMake via msys2/mingw64, Windows 10, bash shell. - ---- - -## 7. DUCKSTATION - -`duckstation/build/bin/duckstation-qt.exe` — modified build with TCP -debug server on port 4371. BIOS at `duckstation/build/bin/bios/SCPH1001.BIN`. - -Launch BIOS-only: `./duckstation-qt.exe -bios` - -Connect via Python: -```python -import socket, json -s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) -s.connect(("127.0.0.1", 4371)) -s.sendall(b'{"cmd":"ping","id":1}\n') -# Response: {"id":1,"ok":true,"frame":N} -``` - -Commands: `ping`, `read_ram`, `get_registers`, `gpu_state`, `screenshot`, -`get_frame`, `frame_range`, `frame_timeseries`, `watch/unwatch`, `pause/continue`, -`step`, `dump_ram`, `dma_state`, `irq_state`, `sio_state`, `quit`. - -36,000-frame ring buffer. `ncat` available at `/c/Program Files (x86)/Nmap/ncat`. - ---- - -## 8. GHIDRA MCP - -Active program: `SCPH1001.BIN`, base `0xBFC00000..0xBFC7FFFF`, 666 functions. -Server auto-starts. Key tools: -- `mcp__ghidra__get_binary_info` -- `mcp__ghidra__get_code` (format: disassembly or decompiler) -- `mcp__ghidra__disassemble_at` -- `mcp__ghidra__analyze_function` -- `mcp__ghidra__xrefs` -- `mcp__ghidra__get_functions` (paginated) - ---- - -## 9. FILES TO READ FOR PHASE 2 - -Before writing any code, read these to understand the interface: - -1. `generated/cpu_state.h` — the compile-only CPUState and extern declarations -2. `generated/boot_slice.c` — example of what generated code looks like -3. `recompiler/src/strict_translator.cpp` — how each opcode emits C -4. `recompiler/src/function_discovery.h` — the discovery pipeline API -5. `recompiler/src/main_bios.cpp` — the recompiler entry point (extend for full BIOS emission) - ---- - -## 10. WHAT NOT TO DO - -- Do NOT modify Phase 1 artifacts or the strict translator -- Do NOT write an interpreter -- Do NOT add HLE shims -- Do NOT create stubs (`return 0;` etc.) -- Do NOT use printf/fprintf for debugging — TCP debug server only -- Do NOT load a game ISO or EXE (Phase 5) -- Do NOT bulk-copy v3 runner files (audit one at a time per CLAUDE.md Rule 7) - ---- - -## 11. COMMIT DISCIPLINE - -Git repo is initialized. Commit after each meaningful milestone. -Do NOT amend the Phase 1 commit. Create new commits. diff --git a/HANDOFF_PHASE3.md b/HANDOFF_PHASE3.md deleted file mode 100644 index 623cbb20..00000000 --- a/HANDOFF_PHASE3.md +++ /dev/null @@ -1,286 +0,0 @@ -# PSXRecomp v4 — Phase 3 HANDOFF (FRESH SESSION) - -You have ZERO prior context. Follow this exactly. - ---- - -## 0. FIRST ACTION - -Read these files in order: - -1. CLAUDE.md -2. PLAN.md (Phase 3 starts ~line 1047) -3. PRINCIPLES.md -4. DEBUG.md -5. MEMORY.md - -Then state EXACTLY: - -"Architecture A is locked. No interpreter. No HLE. No stubs. BIOS first. Game never until Phase 5." - -If you cannot confidently state this, STOP. - ---- - -## 1. CURRENT STATE (DO NOT REDO) - -Phase 1 COMPLETE — recompiler proven correct on SCPH1001.BIN -Phase 2 COMPLETE — runtime executes recompiled BIOS -Phase 3 IN PROGRESS — BIOS boots into shell loop, no display yet - -### What works RIGHT NOW - -The recompiled BIOS executes from reset through kernel init, copies -kernel Part 2 to RAM, sets up A0/B0/C0 function tables, processes a -SYSCALL, copies the shell from ROM to RAM 0x80030000, enters the shell, -and runs in a stable loop for 3+ seconds without crashing. - -**97,000+ trampoline resolutions per 3-second run.** -**~800 seeds, ~793 emitted functions, 46,499 instructions.** - -### Trusted baseline files - -Generated (DO NOT MODIFY — fix the recompiler instead): -- `generated/SCPH1001_full.c` — 793 functions, 46,499 instructions -- `generated/SCPH1001_dispatch.c` — binary-search dispatch with 795 entries - -Runtime: -- `runtime/src/main.cpp` — loads BIOS, initializes CPU, calls dispatch -- `runtime/src/memory.c` — memory + MMIO routing (SEE STUB AUDIT BELOW) -- `runtime/src/traps.c` — SYSCALL handler + trampoline resolver -- `runtime/include/cpu_state.h` — CPU state definition - -Recompiler (changes made this session): -- `recompiler/src/full_function_emitter.cpp` — fallthrough codegen, J/JAL relocation, shell normalization -- `recompiler/src/function_discovery.cpp` — normalization range fix -- `recompiler/src/strict_translator.cpp` — MTHI/MTLO added -- `recompiler/seeds/phase2_ghidra_seeds.json` — 800 seeds - -### Architecture invariants (STILL TRUE) - -- Functions: `void func_XXXXXXXX(CPUState*)` -- All memory via function pointers (read/write) -- Address normalization: `addr & 0x1FFFFFFF` + relocation rules -- Two relocation zones: - - Kernel Part 2: ROM `0x1FC10000-0x1FC17FFF` ↔ RAM `0x500-0x8500` - - Shell: ROM `0x1FC18000-0x1FC427FF` ↔ RAM `0x30000-0x5A7FF` -- Dispatch: `psx_dispatch(cpu, addr)` with binary search -- JAL → direct call; JR $ra → return; JALR/indirect JR → `psx_dispatch` -- Runtime trampoline resolver handles A0/B0/C0 vectors in RAM -- SYSCALL simulates R3000A exception (EPC, Cause, SR push, vector dispatch) - ---- - -## 2. STUB AUDIT — WHAT IS REAL VS FAKE - -This is the critical section. Read `runtime/src/memory.c` carefully. - -### REAL hardware simulation (correct behavior): -- Memory control registers (0x1F801000-0x1F801020): store/recall -- RAM size register (0x1F801060): store/recall -- Cache control (0xFFFE0130): store/recall -- Interrupt I_STAT (0x1F801070): AND-acknowledge semantics ✓ -- Interrupt I_MASK (0x1F801074): store with 0x7FF mask ✓ -- SPU registers (0x1F801C00-0x1F801FFF): store/recall (no audio needed) -- Expansion 1 (0x1F000000-0x1F7FFFFF): returns 0xFFFFFFFF (open bus) ✓ -- Expansion 2 / POST (0x1F802000-0x1F802FFF): ignored ✓ - -### STUBS that MUST be replaced with real simulation: - -**GPU (0x1F801810, 0x1F801814) — CRITICAL for Sony logo:** -- GP0 write: silently dropped (line 150) -- GP1 write: silently dropped (line 151) -- GPUREAD: returns 0 (line 102) -- GPUSTAT: returns fixed 0x1C000000 (line 103) -- REALITY: needs full GP0 command parser, FIFO, VRAM writes, display params - -**DMA (0x1F801080-0x1F8010FF) — CRITICAL for GPU data transfer:** -- All reads return 0 (line 90-92) -- All writes silently dropped (line 138-140) -- REALITY: needs per-channel MADR/BCR/CHCR, DPCR/DICR, transfer execution -- GPU DMA (channel 2) transfers polygon/texture data from RAM to GPU - -**Timers (0x1F801100-0x1F80112F) — NEEDED for vblank:** -- All reads return 0 (line 93-96) -- All writes silently dropped (line 141-143) -- REALITY: needs counter values, mode registers, target values, overflow flags -- Timer 1 (hblank) drives the vblank interrupt - -### STUBS that are ACCEPTABLE for Phase 3: -- SIO (0x1F801040-0x1F80105F): returns TX ready — OK until Phase 4 -- CDROM (0x1F801800-0x1F801803): returns 0 — OK until Phase 5 -- MDEC (0x1F801820-0x1F801824): returns 0 — not needed for Sony logo - -### unmapped_fatal is a concern: -- Currently returns silently instead of aborting (line 62-68) -- This was necessary for BIOS RAM-size probing during init -- BUT it could mask real bugs during shell execution -- Consider: re-enable fatal abort for writes, keep silent for reads - ---- - -## 3. PHASE 3 REMAINING GOAL - -Render the Sony Computer Entertainment logo. - -REQUIREMENT: VRAM output must match DuckStation byte-for-byte. - ---- - -## 4. WHAT MUST BE BUILT NEXT (in order) - -### Step 1: GPU command processor -The shell is writing GP0/GP1 commands that are being silently dropped. -Implement: -- GP1 command parser (display mode, display area, drawing area, reset) -- GP0 command parser (fill rect, copy rect, draw polygon, draw sprite) -- 1MB VRAM buffer (1024×512 × 16-bit) -- GPUSTAT register with real status bits - -### Step 2: DMA channel 2 (GPU) -The shell uses DMA to transfer large command lists to the GPU. -Implement: -- Per-channel registers (MADR, BCR, CHCR) for all 7 channels -- DPCR (channel enable/priority) -- DICR (DMA interrupt control) -- Channel 2 (GPU) linked-list and block transfer modes -- Channel 6 (OTC — ordering table clear) - -### Step 3: Timer + vblank interrupt -The shell waits for vblank to sync frame display. -Implement: -- Timer 0/1/2 counter registers -- Timer mode registers (clock source, target, IRQ) -- Vblank interrupt delivery (I_STAT bit 0) -- Simple frame timing (call vblank at ~60Hz equivalent) - -### Step 4: SDL2 display window -- Add SDL2 dependency to CMakeLists.txt -- Create window (640×480 or 320×240) -- Blit VRAM framebuffer region to SDL surface each frame -- Main loop: execute BIOS → check vblank → blit → SDL_PollEvent - ---- - -## 5. SALVAGE SOURCE (v3 reference, NOT copy-paste) - -v3 hardware sim files at: -`F:/Projects/psxrecomp-projects-v3/psxrecomp/runner/src/` - -- `gpu.c` (1200+ lines) — GP0/GP1 command parser, FIFO, GPUSTAT -- `gpu_sw_renderer.c` — software rasterizer (polygon, sprite, fill) -- `dma.c` (378 lines) — DMA channel transfer engine -- `interrupts.c` (325 lines) — I_STAT/I_MASK + exception dispatch -- `timers.c` (150+ lines) — counter advance, mode flags, clock sources - -AUDIT RULES (from CLAUDE.md §7): -- REMOVE: all bios_*, interpreter_*, events/threading, HLE logic, kernel mirrors -- KEEP ONLY: pure hardware register simulation, state machines -- Integrate ONE file at a time → wire into memory.c → test immediately - ---- - -## 6. BIOS ROM MEMORY MAP (verified) - -Code regions (scan for functions HERE ONLY): -- `BFC00000-BFC0DC60` — Kernel Part 1 (~56KB) -- `BFC10000-BFC16760` — Kernel Part 2 (~26KB, relocated to RAM 0x500) -- `BFC18000-BFC42800` — Shell (~172KB, copied to RAM 0x80030000) - -Data regions (NEVER scan, NEVER treat as code): -- `BFC0DC60-BFC0FFFF` — zero padding -- `BFC16760-BFC17FFF` — data tables + zero padding -- `BFC42800-BFC63FFF` — graphics (TIM textures, strings, CLUTs) -- `BFC64000-BFC7FFFF` — font bitmaps (112KB) - ---- - -## 7. BUILD - -``` -PATH=/c/msys64/mingw64/bin:$PATH -export TMP=/c/msys64/tmp TEMP=/c/msys64/tmp - -# Recompiler (only if recompiler source changes): -cd recompiler/build && cmake --build . - -# Re-emit (only if seeds or recompiler changes): -cd ../.. && recompiler/build/psxrecomp-bios.exe bios/SCPH1001.BIN generated \ - --emit-full recompiler/seeds/phase2_ghidra_seeds.json - -# Runtime: -cd runtime/build && cmake --build . - -# Run: -cd ../.. && runtime/build/psx-runtime.exe bios/SCPH1001.BIN -``` - -Current behavior: runs silently in a loop (shell executing with stubbed hardware). -Expected after GPU impl: should render Sony logo to VRAM, display via SDL2. - ---- - -## 8. DEBUGGING METHOD - -DuckStation TCP debug server: -- Location: `F:/Projects/psxrecomp-v4/duckstation/build/bin/` -- Launch: `duckstation-qt.exe -bios` -- Port: 4371 -- Commands: ping, read_ram, get_registers, gpu_state, pause, step - -For EVERY GPU command or DMA transfer you implement, verify against -DuckStation's output for the same boot sequence. - ---- - -## 9. HARD CONSTRAINTS (still in force) - -NEVER: -- Return dummy MMIO values without understanding what the BIOS expects -- Skip hardware behavior -- Use printf debugging (no fprintf to stderr/stdout in runtime code) -- Modify generated code -- Add HLE behavior -- Load a game ISO or EXE - -If an MMIO read value is unknown: -1. Check DuckStation (TCP debug server) -2. Check Ghidra for the calling BIOS code's usage pattern -3. Implement the correct producing logic - ---- - -## 10. KNOWN ISSUES - -1. `unmapped_fatal` is silent — could mask bugs. Consider re-enabling for writes. - -2. GPUSTAT returns fixed `0x1C000000` — the shell reads this to check GPU - readiness. When GPU is implemented, this must reflect real FIFO/busy state. - -3. SIO_STAT returns fixed `0x00000005` — adequate for Phase 3 but must be - real for Phase 4 (controller/memcard). - -4. The trampoline resolver in `traps.c` handles runtime-written code at - A0/B0/C0 vectors. It simulates delay slots for `li+jr` patterns. - This is NOT an interpreter — it's a dispatch chain resolver. But verify - it handles all patterns the shell uses. - -5. One BIOS function skipped (COP1/FPU at BFC02324) — not reached during boot. - ---- - -## 11. SUCCESS CRITERIA - -Phase 3 is DONE when: -- SDL2 window opens -- Sony Computer Entertainment logo renders on screen -- VRAM contents match DuckStation byte-for-byte - -Phase 3 is NOT done if: -- "The GPU command stream looks right" (PIXELS ON SCREEN) -- "VRAM has some data" (MATCHES DUCKSTATION) -- "It mostly works" (BYTE FOR BYTE) - ---- - -BEGIN. diff --git a/ISSUES.md b/ISSUES.md index 3179caa3..8cff8fc9 100644 --- a/ISSUES.md +++ b/ISSUES.md @@ -557,8 +557,8 @@ fringe on them. Tooling: `tools/crop_launcher_assets.ps1` (crop + knockout), `tools/gen_launcher_assets.ps1` (procedural check/verdict icons), `tools/shot_launcher.ps1` (screenshot the GL launcher window). Mockup -source: `C:\Users\Matthew\Desktop\ef772e04-a7db-4ecd-98bb-eb75a01de0a6.png` -(1448×1086). Pure-cosmetic; does not block Phase 4/5 wiring. +source: a local mockup PNG (1448×1086). Pure-cosmetic; does not block +Phase 4/5 wiring. --- @@ -626,3 +626,123 @@ save-load, FMV, menus, and combat with **0 divergences, 0 wedges**, shards validated-then-promoted, device functions correctly pinned to the interpreter. "Diff is clean" or "shard validated" is NOT success until the full save-load loop closes live. + +--- + +## Issue #8 — Tomba2 P0: wild-jump FAIL-FAST fatal after coverage-wave native promotion + +**Status:** OPEN — root-cause hunt in flight, currently blocked on Issue #9 +**Date opened:** 2026-07-06 (carried from session handoff) +**Area:** overlay native execution (codegen OR dispatch/call-contract layer) +**Rule:** root-cause via first-divergence; do NOT ship a block/disable. +`overlay_native_block` on 0x80073328 acceptable for ATTRIBUTION only. + +### Symptom + +Two FAIL-FAST unknown-dispatch fatals in attract soaks (~15–20 min apart per +run) after the 2026-07-06 entry-based-coverage wave promoted new PCs to native +(the convergence fixes work; they exposed a latent defect): + +- **Crash 1:** addr=ra=0x0000F514 (zeroed kernel heap, page not dirty), + 4 seconds after 0x80073328 went native for the first time (its only-new-entry + DLL `00038000_0A16928B` built 09:04:14). 0x80073328 fired ra=0x80139400 every + 2 frames all scene. +- **Crash 2:** addr=0x009D6830, ra=0x00000001 (beyond 4× RAM mirror), pre-fatal + register soup: target==ra; interp dispatches into 0x13xxxx drivers with + garbage args (a0=0xDF73E104, a1=0xE2000318 = raw GP0 word as pointer). +- Both: same kernel event-delivery dispatch chain (…0x20D4, 0x650, 0x2104) + immediately before; a0=0x800F42E0 (persistent struct) at both fatals. + Corruption PRECEDES the fatal; fail-fast catches the landing. + +Evidence: memory `tomba2_f514_wildjump_first_native.md` + prior session's +scratchpad `crash_f514/` (psx_crash.txt, psx_last_run_report.json, +psx_freeze_heartbeat.json). + +### Suspect set + +The post-08:44 coverage wave DLLs in +`build-t2/cache/SCUS-94454/gcc/win-x64/cg4_d44a063d/`: new region DLLs + +island fragments (incl. F 80073328 in `00038000_0A16928B`; F 80106D7C / +F 80106E58 / F 80024548; jump-table-ladder aliases F 8011040C/43C/45C/478/4CC +in `00106000_1675417D`). + +### Plan (two independent paths) + +1. **Instrumented path (preferred):** finish the shadow-diff segment redesign + (Issue #9), soak with `PSX_OVERLAY_DIFF=1` from boot. Expect either a named + divergence record (→ shard codegen defect, fix recompiler) or clean segments + everywhere + crash persists (→ exit/call-contract class, bugA/bugD family — + fix the dispatch contract layer). +2. **Bisection path (no instrument needed):** quarantine halves of the + post-08:44 DLL wave out of the cache dir, ~20-min attract soak per round + (crash repro interval). First single-candidate probe: `overlay_native_block` + on 0x80073328 (attribution only, never a fix). + +--- + +## Issue #9 — Overlay shadow-diff instrument is structurally unsound (never engages) + +**Status:** ROOT-CAUSED; partial fixes in tree (uncommitted, `_wt-tomba2-ipr`); +segment-granular redesign DESIGNED but NOT implemented +**Date opened:** 2026-07-06 +**Area:** `runtime/src/overlay_loader.c` (run_shadow_diff + dispatch gates), +`runtime/src/dirty_ram_interp.c`, `runtime/src/interrupts.c`, `runtime/src/traps.c` +**Blocks:** Issue #8 path 1. Full detail + design in memory +`tomba2_shadow_diff_unsound_segment_redesign.md`. + +### Symptom + +With `overlay_diff_on` / `PSX_OVERLAY_DIFF=1`, shadow_calls=1 over 51K+ frames +(two independent runs) — the diff instrument validates essentially nothing while +dispatch_native runs millions of calls. The game LOOKS fine, so the failure is +silent. + +### Root cause chain (three stacked defects, found in order) + +1. **CPS interior re-entry path had no diff gate** — continuation re-entries + (the dominant dispatch class under CPS) ran native blind in diff mode. + FIXED in tree: gate mirrors the entry chain (`interior_gated` counter in + `overlay_shadow_dump`). +2. **Longjmp-escape hazard** — a shadow started inside an exception dispatch is + unwound by the guest RFE's longjmp (setjmp frame below run_shadow_diff); + epilogue skipped; s_in_shadow/s_native_exec/s_suppress_irq stuck forever. + HARDENED in tree: `g_exc_setjmp_epoch` (interrupts.c) + + `overlay_loader_shadow_escape_fixup()` at all 3 longjmp sites; dump fields + `escapes`/`escapes_native` must stay 0. +3. **THE WEDGE (design flaw, predates this session): run_shadow_diff never + terminates for non-returning functions.** Interp pass = one + dirty_ram_dispatch call (returns at first exit-to-elsewhere, NOT at stop_ra); + native pass chases via psx_dispatch_call until pc==stop_ra — NEVER for a + main-loop-shaped function. The live game then runs FOREVER inside the + abandoned shadow native pass: **speculative native state becomes the live + timeline**, IRQ-suppress stuck on, all downstream native disabled + (native_exec=0). Verified live both runs: in_shadow=1, native_exec=0, + escapes=0, wedge at frame ~1000. ⇒ Any pace/behavior data from a diff-mode + run is garbage; the instrument was never capable of validating this game. + +### The fix (designed, ready to implement): CPS segment-granular diff + +Diff exactly ONE `c->fn` invocation extent (entry/interior PC → first +jal/jalr/jr/region-exit). Verified against generated DLL C: CPS codegen +surfaces at EVERY call/return/indirect jump (no local jump-table switches), so +a lean interp segment runner (in dirty_ram_interp.c; must intercept +jal/jalr/jr/j BEFORE exec_one — exec_one nests calls inline) can run an +identical extent. Both passes bounded by construction; no chasing, no fiber/RFE +exposure; validates interior re-entries directly; compares pc (boundary target) +as a first-class divergence signal. Abandon (no diff, interp state stands) on: +syscall/break/rfe, unsupported insn, insn cap, MMIO (device_touch as today). +Asymmetry guards: psx_slice_block no-fire in shadow (currently moot — slicing +default OFF); cycle-counter snapshot/restore around the native pass; +post-dispatch IRQ pump shadow guard (already in tree). + +### Uncommitted worktree state (this session, runtime-only, cg tag intact) + +- `overlay_loader.c`: CPS interior diff gate + `interior_gated`; in-exception + shadow-start detour (superseded by segment design — remove when implementing); + `s_shadow_cand` own-interior native route for shadow native pass; escape-fixup + mirrors + `overlay_loader_shadow_escape_fixup()`; shadow dump fields + `in_shadow`/`native_exec`/`escapes`/`escapes_native`; IRQ pump shadow guard. +- `interrupts.c`: `g_exc_setjmp_epoch` + getter; fixup calls in + deferred_exception_longjmp + psx_rfe_escape_check. +- `traps.c`: fixup call at the deferred-honor longjmp site. +- All inert when diff mode is off; normal play unaffected. Builds clean. diff --git a/OVERNIGHT_2026_04_28.md b/OVERNIGHT_2026_04_28.md deleted file mode 100644 index bcc2882c..00000000 --- a/OVERNIGHT_2026_04_28.md +++ /dev/null @@ -1,112 +0,0 @@ -# Overnight session — 2026-04-28 - -Net result: substantial new architectural understanding of the kernel's -memory-card subsystem; **mc_read_done still 0**. No code changes -shipped — investigation only. Running the agent's planned next-steps -list from the handoff revealed the previous memos were partially -incorrect, so the priority of fixes has shifted. - -## What was actually proven - -1. **Outer coord 0x5000 IS the chain trigger.** Single direct caller is - FUN_bfc144bc (RAM 0x49BC), the VBLANK chain handler. Disabling it - via `[0x74BC]=0` poke stops mc_reads dead. -2. **The chain dispatcher 0x4D6C has THREE exit paths:** - - `v0==0` (more bytes): I_MASK |= 0x80, exit. **Does NOT clear - [0x755A].** Counter stays incremented. - - `v0==1` (chain step done): clears [0x755A], resets counter, sets - [0x7568+slot]=0x01. - - `v0==-1` (failure): clears [0x755A], sets [0x7568+slot]=0x21. -3. **R1 ALSO sets [0x755A]=1** at 0xBFC15258 (not just D1). Earlier - memo claimed only D1 sets it — that was wrong. -4. **R-chain DID run 26 times** during the test session; chain trace - shows counter advancing 1..11 each cycle, all v0=0. No run reached - counter=12. **R12 and R13 (the data-checksum and 'G'-byte handlers) - never fire.** -5. **Three chain modes exist:** R (read, entry 0x5688, 13 handlers, - table 0x6C98), W (write, entry 0x51F4), D (detection, entry 0x5B64, - 4 handlers, table 0x6CCC). Three setup functions at 0xBFC159A0, - 0xBFC15A04, 0xBFC15B00 select which by writing [0x7528+slot*4]. -6. **Install handler mode byte:** R-setup writes 2 (RX), W-setup writes - 4 (TX), D-setup writes 8 (no-op). Install handler at 0x641C - processes byte and increments [0x72F0]; clears [0x75C0]=0 after 128 - bytes. -7. **Both slots' status [0x7568+slot] settle at 0x01 in steady state** - — outer coord skip-reset path checks bit 0 and exits. Direct byte - search (`sb zero, 0x7568(?)`) found NO clear sites; whatever clears - [0x7568+slot] back to 0 (to allow a fresh read) uses non-direct - addressing. Detection chains keep running because the dispatcher's - v0=1 path keeps re-arming [0x7568+slot]=0x01 on every cycle. - -## Why R-chain stops at R11 - -Chain dispatcher's v0=0 path doesn't clear [0x755A]. R1 sets it; it -stays set throughout R1..R13 + 128 install handler bytes. On real PSX, -the whole chain finishes in ~4.5ms (well within 16.67ms VBLANK), so -[0x755A] is cleared by R13's v0=1 success path before the next outer -coord call. - -In our runtime: the data phase moves too slowly. Install handler -processes ~16 bytes per cycle, then VBLANK fires (or VBLANK_DEFER_STALE -times out at 500000 dispatch ticks ≈ 125ms with no SIO progress) → -outer coord runs → sees [0x755A]==1 → abort cleanup → chain reset. - -## Why _card_read isn't retriggered - -After the first failed attempt, [0x7568+slot]=0x01 settles; outer -coord skip-reset exits without re-triggering chain. txn_dump for the -recent test window shows ALL 51 transactions are 4-byte detection -(`[0x81, 0x52, 0x00, 0x00]` / `[0xFF, 0x00, 0x5A, 0x5D]`) ending in -`abort_reselect`. R-chain transactions are older (pre-mc_seq 43689). - -The BIOS shell isn't re-issuing _card_read on subsequent X presses. -Either the shell is stuck waiting for a callback that never fired, or -X press doesn't normally trigger another read once you're past the -initial directory load. - -## Files added/changed this session - -- **NO source code changes.** Pure investigation. -- `probes/overnight/` — 11 JSON dumps + 5 Python analyzers + stdout/ - stderr from the test run. -- New memory: - - `phase4_chain_dispatcher_complete_map.md` — full architectural map - - `phase4_real_blocker_2026_04_28.md` — corrected blocker analysis -- Memory NOT yet superseded but should be: `phase4_chain_resets_after_16_bytes.md` - (the underlying observation is right; the "D1 unconditionally" - framing is incorrect — R1 also sets [0x755A]). - -## Concrete next steps (priority) - -1. **Find the `[0x7568+slot]=0` clearer.** It's the gate for fresh - reads. Likely uses `sw zero, X(base)` where base+X = 0x7568+slot. - Search for `addiu reg, reg, 0x7568` then trace stores via reg. - Once found, you know who triggers fresh _card_read calls. -2. **Confirm R-chain's data-phase timing.** Add a wtrace slot for - [0x72F0] (data byte counter) and watch how high it gets per cycle - under different timing settings. Currently appears to top out at - ~16 of 128 needed. -3. **Try lowering SIO_IRQ_DELAY_CARD from 8 to 4** in `runtime/src/sio.c`. - Risk: breaks the BIOS pad/card detection write-clear-check sequence. - But install handler currently does only ~4 SIO accesses post-TX, so - countdown=8 leaves IRQ pending after handler returns. Lower delay = - IRQ fires inside handler = next byte processed immediately = - faster data phase. -4. **Check user-level _card_read in B0 vector** to see what it does - on entry. Specifically: does it clear [0x7568+slot]=0? If yes, why - isn't it being called on X press? Check what calls _card_read. -5. **Examine kernel callback 0xB0006B30** (RAM 0x6B30 = ROM 0xBFC16630). - Called by chain dispatcher v0=1 success path with [0x7520]==0. - Might be the "user notify" hook that BIOS shell's _card_read - blocks on. - -## Hard rules respected - -- No printf debugging. -- No generated/SCPH1001*.c modifications. -- No HLE shims. -- No stubs. -- One runtime instance at a time (always taskkilled before launch). -- Build via mingw64. -- No fprintf added to runtime. -- No game ISO/EXE loaded. diff --git a/OVERNIGHT_PROGRESS_20260427.md b/OVERNIGHT_PROGRESS_20260427.md deleted file mode 100644 index a96b46e3..00000000 --- a/OVERNIGHT_PROGRESS_20260427.md +++ /dev/null @@ -1,215 +0,0 @@ -# Overnight progress — 2026-04-27 - -## What landed (committed) - -### Commit 1 — `fix(memcard): clear FLAG byte after read — the four-day stall` -The FLAG fix in `runtime/src/sio.c` + new `sio_first_divergence` TCP cmd -+ state-12/13 seeds + chain_trace/mc_status from prior session. - -**Real bug, single line:** our card sim returned FLAG=0x08 forever instead -of clearing it after first read. BIOS state-3 chain handler sees bit 3 -set → "fresh card" path → returns v0=-1 → dispatcher resets counter → -infinite loop. Beetle returns 0x00 → chain advances cleanly. - -After fix: -- Both runtimes navigate to MEMORY CARD screen -- Inner read-chain advances states 1..11 cleanly with v0=0 -- byte-1 SIO RX divergence eliminated - -### Commit 2 — `debug: always-on wtrace coverage for card-chain state addrs` -Per global rule "use ring buffers, never time/attach", added static -always-on wtrace at boot for the addresses that determine card-chain -progress. Lets us answer "when was X written?" deterministically by -querying the ring buffer, no racing live samples. - -Always-watched now (`runtime/src/debug_server.c`): -- 0x7514..18 — chain counter -- 0x7528..30 — per-slot chain handler ptr (=0x5688 read / =0x5B64 detect) -- 0x755A — chain abort flag (state-1/D1 set, outer coord aborts on !=0) -- 0x7520 — read-chain success flag -- 0x74A4 — chain status flag - -WTRACE_MAX_RANGES bumped 8→16. - -## What's still broken (next downstream divergence) - -**Memory cards still don't show save data** — same as the user noted -("we're now at the MEMORY CARD screen but card slots show no save data"). - -### Deterministic root cause (per ring buffer evidence) - -Counter at 0x7514 advances via two distinct chain handlers: -- `func_00005B64` (detection chain, table 0x6CCC, 4 entries D1..D4) -- `func_00005688` (READ chain, table 0x6c98, 13 entries states 1..13) - -Read chain DOES get installed at 0x7528[slot] via `func_00005E98` at -seq=12754 (after press X). It runs for ~60 wtrace events then gets -overwritten back to detection by `func_00005FF0` at seq=12814. - -During the read-chain window, counter advances **0→1→2→...→11 then RESETS to 0**. -NEVER reaches 12 (state-12 = data-byte handler) or 13 (state-13 = END byte). - -The reset writer is `0xBFC14B90: sw $0, 0x7514` inside `func_00005000` -(outer coordinator), reached via this path: -1. Some kernel callback runs `func_000049BC` (which JALs `func_00005000`) -2. `func_00005000` JALs DeliverEvent (`func_00001C5C`) -3. DeliverEvent fires events; chain dispatcher runs 11 times advancing counter 0→11 -4. DeliverEvent returns; `func_00005000` continues -5. `func_00005000` reads `[0x755A]` → if !=0, runs abort path (clears 0x755A, - resets 0x7514 counter) -6. State-1 had set `0x755A=1` at start of chain. State-13's success path - would normally clear it — but we never reach state-13. - -**The deadlock:** chain needs to reach state-13 to clear 0x755A; outer coord -aborts at state-11 because 0x755A still set. Chicken-and-egg. - -Counter histogram across ALL 1458 wtrace 0x7514 writes: -``` -counter=0: 596 (resets) -counter=1: 198 detection chain runs -counter=2: 198 -counter=3: 198 -counter=4: 198 -counter=5: 10 read chain runs (only 10 attempts!) -counter=6: 10 -counter=7: 10 -counter=8: 10 -counter=9: 10 -counter=10: 10 -counter=11: 10 -counter=12: 0 ← never reached -counter=13: 0 ← never reached -``` - -### What Beetle does differently (untraced) - -Beetle's 0x7528=0x5B64 too (steady state), so it ALSO uses detection -chain primarily. Beetle's `find_first_divergence` at 0x7560 = 0xC3 -(checksum from state-12+ data XOR; ours = 0). So Beetle DID reach -state-12+ at some point and complete a directory read. - -Hypothesis (NOT yet measured deterministically): -- In Beetle, 13 chain advances complete in ONE DeliverEvent invocation - (so state-13 clears 0x755A before func_00005000 checks it) -- In ours, only 11 chain advances complete per DeliverEvent invocation, - abort fires before state-13 - -WHY only 11? Unknown without further measurement. Possibilities: -- Our SIO IRQ pacing is too slow (each chain advance needs an SIO IRQ; - if IRQs queue up slower than DeliverEvent processes them, only N fit) -- DeliverEvent's exit condition is different in our recompiled C -- Some kernel function (called during state-1..11 path) terminates - DeliverEvent early in our build but not on real hw - -Confirmed dead ends: -- 0x74F8 FLAG-store divergence (red herring — it's only written by 0x57 - WRITE-cmd path, we never send 0x57) -- 0x7264 slot toggle (benign timing skew) -- All cascade-reset SW sites in dispatcher (ruled out by ra=0x4DDC analysis) - -## Next action when you wake up - -The principle-aligned move per "build tools, not guesswork": - -**Build a B0 call ring.** Trace every BIOS B0:XX function call with -the function index (`$t1`), `$ra`, and seq. Beetle vs ours can be -compared: Beetle's B0:6B40 (DeliverEvent) call frequency, B0:6380 -call sequences inside state-11, B0:1B44 calls, etc. The B0 jump table -is at 0x80000060 → 0x000000B0; intercepting that one site gives full -visibility. - -The deadlock IS understood. The fix point requires understanding why -DeliverEvent only fires the chain handler 11 times instead of 13. A -B0-call ring buffer is the cleanest way to see that without racing. - -Alternative quicker probe: arm wtrace on 0x7514 + 0x755A + the SIO -register area, ONE button press, see EVERY event interleaved by seq — -see if there's an SIO IRQ between counter=11 advance and the reset -that should have advanced to 12 but didn't. - -Files modified but not yet committed (from this session): -- `CLAUDE.md` (prior-session edits) -- `recompiler/src/full_function_emitter.cpp` + `.h` (prior session) -- `runtime/src/beetle_psx_bridge.cpp` (prior session) -- `runtime/src/main.cpp` (prior session) -- `runtime/src/memory.c` (prior session) -- `tools/analyze_sio_trace.py` (prior session) - -These are all from earlier work and unrelated to today's FLAG bug fix. -Leave them be unless a new session decides they're ready. - -## Update — function ENTRY/EXIT trace rings landed (commit 84180e2 + 188bdc8) - -Built two new always-on rings hooked off `psx_dispatch`: - -- **`fn_entry`** — every dispatch in CALL ORDER with target, $ra, - $a0..$a3, $t1 (B0/A0/C0 vector index), depth, frame -- **`fn_exit`** — every function in FINISH ORDER with $v0, $v1, plus - link back to entry seq - -Shadow call stack (4096-frame) classifies each dispatch as: -- RETURN — target matches an open frame's saved RA → pop + record exit -- TAIL CALL — $ra unchanged from previous dispatch → replace top frame -- NEW CALL — $ra changed (fresh JAL) → push new frame - -TCP commands: -- `fn_filter` `{lo, hi}` — set [lo, hi) physical addr filter -- `fn_clear` — reset both rings + shadow stack -- `fn_stats` — totals, depth, overflows, tail-call count -- `fn_entry_dump` `{count, seq_lo, seq_hi, addr_lo, addr_hi}` -- `fn_exit_dump` `{count, seq_lo, seq_hi, addr_lo, addr_hi}` - -### Validation revealed new info - -With filter `0x4000..0x6000` (BIOS card area): - -- During press X, captured 4 read-chain runs each going through: - state-1 → state-2 → ... → state-10 → 0x59D0 → state-11 → 0x5A08 → 0x5A48 -- All with `ra=0x4DDC` (chain dispatcher's jalr return) -- The intermediate addresses 0x59D0, 0x5A08, 0x5A48 are functions - reached via **tail-jump from state addresses** that we didn't see - before. They live BETWEEN state-10/11 and AFTER state-11 in the - address space. Worth investigating — they may be where the chain - decides to abort. - -### Known caveat - -`fn_exit` records v0 at "control leaves this function" — for fall- -through dispatches this is the moment the next function dispatches, -not the true return-to-caller. So v0 may reflect intermediate state -rather than intended return value. e.g. state-11 exits with v0=0xB -because that's the COUNTER value (chain dispatcher loaded it into v0 -before jalr; state-11 didn't overwrite before falling through to -0x5A08). - -Real return values still surface for functions that DO set v0 before -returning (e.g., D-states, kernel callbacks) — those are accurate. - -### Workflow for the user - -```bash -# 1. Set tight filter to scope of interest -{"cmd":"fn_filter","lo":"0x00004000","hi":"0x00006000"} - -# 2. Clear before triggering action -{"cmd":"fn_clear"} - -# 3. Trigger (press, etc) -{"cmd":"press","buttons":49151,"frames":120} - -# 4. Inspect -{"cmd":"fn_stats"} -{"cmd":"fn_entry_dump","count":300,"addr_lo":"0x000056E0","addr_hi":"0x00005AB4"} -{"cmd":"fn_exit_dump","count":300,"addr_lo":"0x000056E0","addr_hi":"0x00005AB4"} - -# Use seq_lo/seq_hi to step through specific window -``` - -## Hard rules to keep - -- Never sample-and-hope. Use the always-on rings now extended to cover - card-chain state. Query, don't time. -- 0x7528 wtrace shows EVERY chain-handler-ptr write since boot. If you - need to know "when did read chain get armed?", filter that ring. -- The FLAG fix is correct; don't revert. Subsequent reads correctly - see 0x00 matching real PSX + Beetle. diff --git a/README.md b/README.md index bf206dfe..5bf6963f 100644 --- a/README.md +++ b/README.md @@ -21,12 +21,19 @@ BIOS layer, no stubs, and no general-purpose interpreter fallback for static code. PSXRecomp is a framework. Game-specific projects live in their own -repositories and pull this one in to build a game binary. The active end-to-end -targets are: +repositories and link this one in as a **git submodule** to build a game binary. +The active end-to-end targets are: - [TombaRecomp](https://github.com/mstan/TombaRecomp) — *Tomba!* - [MegaManX6Recomp](https://github.com/mstan/MegaManX6Recomp) — *Mega Man X6* +**New here?** The fastest way in: +[`docs/EXECUTION_MODEL.md`](docs/EXECUTION_MODEL.md) (how a game actually +runs — static / native-overlay / interpreter), then +[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md), +[`docs/BUILDING.md`](docs/BUILDING.md), and +[`CONTRIBUTING.md`](CONTRIBUTING.md). + ## Philosophy — toward 100% static recompilation The goal is simple and absolute: **a PS1 game should run as native code, not be @@ -113,7 +120,7 @@ Known follow-up work: For the current game milestone, build and run the sibling TombaRecomp project: ```sh -cd F:/Projects/TombaRecomp +cd ../TombaRecomp cmake --build build -j16 ./build/psx-runtime.exe --game game.toml ``` @@ -144,29 +151,30 @@ Windows, `ucontext` on POSIX (`runtime/src/psx_fiber.c`) — so the recompiled BIOS's cooperative thread switching (the CD-boot handoff in particular) behaves the same on every platform. -Requirements: +Requirements at a glance (full details, dependency table, and per-platform +prerequisites in [`docs/BUILDING.md`](docs/BUILDING.md)): -- A C/C++ toolchain: MSVC or MinGW (Windows), Apple Clang (macOS), Clang/GCC (Linux). -- CMake 3.20+. On macOS/Linux also `ninja` and `pkg-config`. -- SDL2: the bundled dev pack on Windows; `brew install sdl2 pkg-config ninja` - on macOS; `libsdl2-dev` (or distro equivalent) on Linux. +- A C/C++ toolchain: MSVC or MinGW/MSYS2 (Windows), Apple Clang (macOS), + Clang/GCC (Linux). CMake 3.20+; on macOS/Linux also `ninja` and `pkg-config`. +- SDL2 (system / bundled). RmlUi and FreeType come in as **git submodules** — + clone with `--recurse-submodules`. - A legally obtained `SCPH1001.BIN` BIOS dump. Not included. - For game projects, a legally obtained game disc/EXE dump. Not included. -Build the framework runtime: +Build the framework (recompiler tool + BIOS-only runtime): ```sh -# Windows (MSYS2/MinGW) -cmake -S recompiler -B recompiler/build -G "Unix Makefiles" && cmake --build recompiler/build -cmake -S runtime -B runtime/build -G "Unix Makefiles" && cmake --build runtime/build --target psx-runtime +git clone --recurse-submodules https://github.com/mstan/psxrecomp.git && cd psxrecomp -# macOS / Linux (Ninja) -cmake -S recompiler -B recompiler/build -G Ninja -DCMAKE_BUILD_TYPE=Release && ninja -C recompiler/build -cmake -S runtime -B runtime/build -G Ninja -DCMAKE_BUILD_TYPE=Release && ninja -C runtime/build psx-runtime +cmake -S recompiler -B recompiler/build -G Ninja -DCMAKE_BUILD_TYPE=Release && cmake --build recompiler/build +cmake -S runtime -B runtime/build -G Ninja -DCMAKE_BUILD_TYPE=Release && cmake --build runtime/build --target psx-runtime ``` -Game projects generate their own `generated/_*.c` files and link this -runtime source tree through CMake. +On Windows swap `-G Ninja` for your generator if you prefer (e.g. +`-G "Unix Makefiles"`); always keep an explicit `-DCMAKE_BUILD_TYPE` so the +generated C is optimized. Game projects generate their own +`generated/_*.c` files and link this runtime through CMake — see +[`docs/BUILDING.md`](docs/BUILDING.md#build-and-run-a-game). ## Keyboard Map @@ -209,10 +217,19 @@ The recompiler emits C functions and dispatch tables for BIOS and game code. The runtime loads the BIOS/game assets into emulated PS1 memory, links the generated C as native code, and simulates hardware through MMIO handlers for GPU, DMA, timers, CD-ROM, MDEC, SIO0, memory cards, SPU, GTE, and interrupt -delivery. BIOS A0/B0/C0 vectors go through the recompiled BIOS, not HLE shims. +delivery. The recompiled `SCPH1001.BIN` is the low-level (LLE) kernel and the +correctness oracle; an optional HLE tier lays instant boot-skip and a few BIOS +services on top, always falling through to the recompiled BIOS. -See `CLAUDE.md`, `PLAN.md`, and `CURRENT_STATE.md` for the development rules -and current project context. +Code that can't be seen ahead of time (disc-streamed **overlays**) is captured +and compiled to native code the first time it appears (`static → gcc → tcc` +backend), with a small interpreter as the correctness fallback until it is. Full +story in [`docs/EXECUTION_MODEL.md`](docs/EXECUTION_MODEL.md); component-level +detail in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). + +See [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`CLAUDE.md`](CLAUDE.md) for the +development rules, and [`docs/internal/`](docs/internal/) for the phased plans +and deep design notes (`PLAN.md`, `FAITHFUL_TIMING_PLAN.md`, …). ## Disc Speed @@ -256,6 +273,21 @@ copyrighted material — keep it on your own machine, alongside your disc image. A metadata-only contribution format (addresses and checksums, no game code) is planned so discoveries can be shared safely in the future. +## Contributing + +Contributions are welcome — AI-assisted or not — as long as they're reviewed, +tested, and keep the core game-agnostic. A few things hold this project together: +the faithful recompiled BIOS is the baseline and oracle, generated code is never +hand-edited (fix the recompiler and regenerate), and a change proves itself +against the Beetle oracle / on screen rather than by assertion. Game-specific work +lives in the game repos, which pin an exact framework commit as a submodule. + +Read [`CONTRIBUTING.md`](CONTRIBUTING.md) before opening a PR — it covers the core +rules, how to verify a change, the regression checklist across the known games, +and how a framework fix reaches a game through its pin. Bugs and build problems go +to GitHub issues (include `gcc -v` / OS / generator for build failures); design +discussion happens in the **R.A.I.D.** Discord (invite below). + ## License PolyForm Noncommercial 1.0.0. See `LICENSE`. @@ -267,3 +299,13 @@ collection. Release executables (and per-game overlay caches) contain statically recompiled (machine-translated) builds of the original code, the same distribution model used by other static recompilation projects such as N64: Recompiled. + +--- + +

+ R.A.I.D. — Retro AI Development · a Discord for AI-assisted retro reverse-engineering, decomp & recomp +

+ +

+ Join the Retro AI Development (R.A.I.D.) Discord +

diff --git a/SUMMARY_20260412_2.md b/SUMMARY_20260412_2.md deleted file mode 100644 index c8dc4815..00000000 --- a/SUMMARY_20260412_2.md +++ /dev/null @@ -1,170 +0,0 @@ -# PSXRecomp v4 — Session Summary 2026-04-12 (Session 2) - -## Phase: 4 (Boot to BIOS shell + memory card screen) - -## What was done - -### Interrupt delivery — three critical bugs fixed - -1. **RFE after handler exit (IEc stuck at 0)** - - The BIOS exception handler's normal exit path (0xBFC10944) restores - context from a kernel jmpbuf and does `jr $ra` — but does NOT - execute RFE. On real hardware this is a longjmp to WaitEvent's - caller, which later calls ExitCriticalSection. In our model, - `return;` goes back to `psx_check_interrupts`, not to the jmpbuf - target. IEc stayed 0 after the first interrupt, permanently - disabling all further interrupts. - - **Fix:** After the handler returns, if IEc is still 0, apply RFE - to pop the SR exception stack. - -2. **GPR corruption from handler jmpbuf exit** - - The handler's exit path at 0xBFC10944 restores $ra, $sp, $s0-$s7 - from the kernel jmpbuf. On real hardware this is the non-local - return to WaitEvent's caller. In our model, the interrupted code - resumes with these wrong register values, causing unaligned access - crashes. - - **Fix:** Save all 32 GPRs + HI/LO before dispatching the handler, - restore them after. The interrupted code gets its exact - pre-exception state back, matching real hardware (where the TCB - save/restore mechanism preserves the full register context). - -3. **VBlank stacking inside the handler** - - `psx_check_interrupts` is called at every block leader, including - inside the exception handler. `dispatch_count` was incrementing - during handler execution, firing VBlanks that accumulated in - `i_stat`. After the handler returned with IEc=1, a new interrupt - fired immediately, creating an infinite interrupt loop that starved - the main code. - - **Fix:** Gate `dispatch_count++` and VBlank/timer ticks on - `!in_exception`. - -### DMA IRQ interrupt storm - - `update_master_irq()` in dma.c was **level-triggered**: it set - `i_stat |= (1<<3)` whenever DICR master_flag was 1. Since the BIOS - handler has no registered DMA callback, i_stat bit 3 was never - acknowledged. Combined with the fixes above (IEc now re-enabled), - this caused an interrupt on every single `psx_check_interrupts` - call — complete interrupt storm, VSync wait never exits. - - **Fix:** Made `update_master_irq` edge-triggered (only set i_stat - on 0→1 transition of master_flag). Additionally suppressed DMA - completion IRQ posting entirely since transfers are synchronous in - our model. - -### Debug diagnostics added - - - `gpu_state` now reports: draw_area, draw_offset, GP0 command - counters (nop/fill/draw/env/copy) - - New `vram_peek` debug server command for inspecting arbitrary - VRAM coordinates - -## Current state - -### What works - -- **IEc properly toggles** — interrupts enable/disable correctly - across exception entry/exit (SR=0x00000401 in main code, SR pushed - on exception, RFE pops on exit) -- **VBlank interrupts fire** on schedule, handler processes EvCB - chains, returns cleanly -- **Sony logo displays** and logo animation plays (VBlank-driven timer - advances) -- **Shell main loop reached** — RA=0xBFC41D18 consistently, execution - is in the VSync wait / draw loop -- **GP0 draw commands executing** — 217+ draw commands, 641+ - environment commands observed during shell operation -- **Zero dispatch misses** on clean run -- **Zero crashes** — runtime runs indefinitely - -### What doesn't work yet - -- **VRAM still shows logo fill (0x5AD6) in the display area.** The - shell is sending draw commands (confirmed via GP0 counters) but - the visible VRAM region does not change. The draws may be landing - at coordinates outside the display area, or being clipped/masked. - Draw area = [0,0,639,479], draw offset = [0,0]. - -- **Shell does not visually progress past the Sony logo.** The logo - animation plays ("small little intro") and then the screen stays - static. Internally the shell IS running (register sampling confirms - the main loop), but the GPU rendering isn't producing visible - output. - -## Root cause analysis for remaining issue - -The shell sends GP0 draw commands but VRAM doesn't change in the -display area (0,0)-(639,479). Possible causes (in order of -likelihood): - -1. **Draws land in wrong framebuffer.** The shell uses double - buffering. If GP1(0x05) display start isn't being called (or the - display points at the logo's old framebuffer), draws to the - back buffer are invisible. display_y=2 is the logo's setting; - the shell should switch to display_y=0 or display_y=240. - -2. **Draw commands are environment-only.** The GP0 stats show 217 - draws vs 641 env commands. The "draws" might be small primitives - (the shell's floating particles) that are too sparse to notice in - VRAM spot-checks. The main fill (GP0 0x02) is 0 — the shell - might not use fill-rect for the background gradient. - -3. **GP1 display swap function not executing correctly.** The shell - calls 0x80059DA4 (normalizes to 0x1FC41DA4, IS in dispatch table) - for VSync/display swap. This function has a timeout loop and calls - another function. The swap might silently fail. - -4. **VRAM writes silently dropped.** The mask bit or draw area - settings might prevent writes. check_mask_bit or set_mask_bit - in the GPU could be interfering. VRAM pixel 0x5AD6 has bit 15=0 - (mask clear), so this is unlikely. - -## Build & run - -```bash -cd F:/Projects/psxrecomp-v4 -PATH=/c/msys64/mingw64/bin:$PATH - -# Build runtime (no regen needed): -cd runtime && cmake --build build -j$(nproc) && cd .. - -# Run: -taskkill //F //IM psx-runtime.exe 2>/dev/null -cd runtime/build && ./psx-runtime.exe ../../bios/SCPH1001.BIN - -# Debug server queries: -echo '{"cmd":"gpu_state"}' | ncat -w 2 localhost 4370 -echo '{"cmd":"irq_state"}' | ncat -w 2 localhost 4370 -echo '{"cmd":"vram_peek","x":0,"y":0,"w":16,"h":1}' | ncat -w 2 localhost 4370 -``` - -## Next steps - -1. **Diagnose why draws don't appear in VRAM.** Check whether GP1(0x05) - display start is being sent. Add GP1 command logging. Check if - draw_area/draw_offset change between frames. Verify the "draw" - category GP0 commands are actual polygon/rect draws vs. something - else. - -2. **Compare against DuckStation.** Run the same BIOS in DuckStation, - capture the GP0/GP1 command stream during the logo-to-shell - transition, compare against our command stream. - -3. **VSync wait timing.** The VSync wait exits when GPUSTAT bit 31 - (LCF) toggles. Verify the shell is actually exiting the VSync wait - and reaching the draw phase each frame. - -## Hard rules (always repeat) - -- Architecture A is locked. No interpreter. No HLE. No stubs. BIOS - first. Game never until Phase 5. -- No printf debugging. No log files. -- Never modify generated code. Fix the recompiler. -- One runtime instance at a time (taskkill before launch). -- PATH=/c/msys64/mingw64/bin:$PATH required for all builds. diff --git a/UNIFICATION_PLAN.md b/UNIFICATION_PLAN.md deleted file mode 100644 index 924bee3b..00000000 --- a/UNIFICATION_PLAN.md +++ /dev/null @@ -1,202 +0,0 @@ -# UNIFICATION_PLAN — one recompiler, one runtime, TOML-driven - -Start date: 2026-05-12 (post-validation-confusion session) - -## Read first - -Before any work in this plan, read `CURRENT_STATE.md` end-to-end. The -happy-path binary lives at `.happypath/tomba-runtime.exe`. Every step in -this plan ends with a checkpoint that confirms it still works. - -## Target state - -``` -# ONE recompiler binary -psxrecomp bios/SCPH1001.toml # produces generated/SCPH1001_*.c -psxrecomp games/tomba/game.toml # produces tomba/generated/*.c - # (implicitly also ensures BIOS regen is current) - -# ONE runtime binary, parameterized at link time -psx-runtime # discless BIOS only — boots to memcard / CD player -psx-runtime tomba/game.toml # with game — boots BIOS → Tomba → main menu -psx-runtime games/ff7/game.toml --disc 2 # multi-disc — swap to disc N -``` - -- **Recompiler**: one binary, reads a TOML, dispatches to BIOS-path or - game-path based on the config's `[program] type = "bios" | "game"`. -- **Runtime**: one binary. Always includes BIOS recomp output. Conditionally - includes game recomp output. If `[game]` is in the config, runtime calls - out to game entry point after BIOS shell finishes its boot sequence. -- **Multi-disc**: `[discs]` array in game.toml; runtime exposes a disc-swap - command on the debug port. - -## Why we want this - -Listed roughly in priority: - -1. **Forces translator unification.** Today `strict_translator.cpp` (BIOS) - and `code_generator.cpp` (game) are parallel implementations that have - diverged. Bugs fixed in one don't apply to the other. Memory - `strict_translator_stays_strict.md` directs that strict wins; - code_generator should be absorbed into strict, not the reverse. -2. **Config-driven everything.** `bios/SCPH1001.toml` and - `TombaRecomp/game.toml` already exist but are only used by audit tools. - The recompiler should read them, not take positional `seeds.json` - arguments. -3. **Always boots BIOS.** Today `psx-runtime` is the BIOS-only test runner - and `tomba-runtime` is the BIOS+game runner — but `tomba-runtime` IS the - discless BIOS path too (it boots BIOS first; you can in principle ignore - the game). One runtime, two modes. -4. **Multi-disc** falls naturally out of a `[discs]` config array. Today's - `[game] disc = "tomba.cue"` is a special case of a one-element array. -5. **Stop the two-process confusion.** Memory - `architecture_two_processes_2026_05_05.md` notes we already have - `psx-runtime` + `psx-beetle` (oracle) as parallel processes. Adding - `tomba-runtime` as a third was a mistake we're cleaning up. - -## What stays separate - -- The Beetle oracle (`psx-beetle.exe`) stays its own process. It's an - independent reference implementation; unifying it would defeat its - purpose. - -## Incremental phases - -Each phase has a **checkpoint** that must pass before starting the next. -The checkpoint is always: `.happypath/tomba-runtime.exe` still boots and -reaches Tomba's main menu (or equivalent verification that the working -state has not been broken). - -### Phase A — Config consolidation (low risk) - -Goal: make both existing recompilers read their inputs from TOML, while -keeping `--emit-full` and friends working as backwards-compatible aliases. -No translator code touched. - -A1. Define the unified TOML schema. Today there are two files: -- `bios/SCPH1001.toml` — has `[program]`, `[audit]`. Add `[recompiler]` - with seeds path, output dir. -- `TombaRecomp/game.toml` — has `[game]`, `[recompiler]`, `[runtime]`, - `[audit]`. Already richer. - - Unify into one schema with a top-level `type = "bios" | "game"` discriminator. - -A2. Teach `psxrecomp-bios.exe` to accept `--config .toml` as an -alternative to positional ROM+seeds args. Falling-through to existing -positional behaviour if no `--config`. Verify regen output is byte-for-byte -identical with both invocation styles. - -A3. Teach `psxrecomp-game.exe` the same. - -A4. Update CLAUDE.md, PLAN.md, and the helper cmake macro to use the -config-style invocation. - -**Checkpoint A**: `.happypath/tomba-runtime.exe` still boots (untouched); -fresh rebuild of `tomba-runtime.exe` via the new TOML invocation produces -identical output to a fresh rebuild via the old positional invocation. - -### Phase B — Runtime consolidation (medium risk) - -Goal: collapse `psx-runtime` + `tomba-runtime` into one parameterized -runtime. After this phase, `psx-runtime` is gone as a separate target; -`tomba-runtime` is renamed to `psx-runtime` and takes an optional game -config. - -B1. Identify the cmake macro `psxrecomp_v4_add_runtime_target` and audit -how `psx-runtime` (in this repo) and `tomba-runtime` (in TombaRecomp) -diverge. They probably mostly agree, but with paths and link inputs -parameterised differently. - -B2. Define the merged target shape. Inputs: -- BIOS C source: always `generated/SCPH1001_*.c` (this repo). -- Game C source (optional): path from `[game] generated_dir`. -- Game config TOML: embedded as a runtime-readable resource (or path - passed at runtime). - -B3. Migrate `tomba-runtime` to be built via the merged macro, with -the game inputs supplied. **Do not touch the old `tomba-runtime.exe` -on disk yet** — build the new one alongside, verify it boots, then -swap. - -B4. Migrate `psx-runtime` to be built via the merged macro, with no -game inputs. Confirm discless BIOS boot (or document the existing -psx-runtime-can't-boot-shell bug separately). - -**Checkpoint B**: rebuilt-via-new-macro `tomba-runtime.exe` boots to -Tomba's main menu identically to `.happypath/tomba-runtime.exe`. - -### Phase C — Recompiler translator unification (high risk) - -This is the big one. Merge `strict_translator.cpp` + `code_generator.cpp` -into a single translator. Memory `strict_translator_stays_strict.md` -directs that the strict approach wins; `code_generator` content is -absorbed in. - -C1. Inventory both translators' supported opcodes. For each, mark: -- supported on both sides (likely most) -- supported only in strict (BIOS-specific, e.g. CACHE, RFE handling) -- supported only in game (whatever Tomba uses that BIOS doesn't) -- handled differently between the two - -C2. For divergences, decide which version wins: -- Hard rule: no fall-through cases sneak in (memory rule). -- Hard rule: no stubs (CLAUDE.md rule 0). -- For each divergence, document the resolution in - `docs/translator_unification.md` so this audit is reviewable. - -C3. Migrate the merged translator behind a feature flag. Build both -binaries (psxrecomp-bios and psxrecomp-game) against the new translator -in parallel with the old, run both regens, diff the output. - -C4. When outputs match, flip the flag, delete the old translator. - -C5. Rename `psxrecomp-bios.exe` and `psxrecomp-game.exe` → `psxrecomp.exe`. -Single binary, dispatches by config `type`. - -**Checkpoint C**: same regen output, byte-for-byte, before and after the -flip. `.happypath/tomba-runtime.exe` checkpoint still passes when -relinked against the new regen output. - -### Phase D — Multi-disc support - -Goal: support `[discs] paths = [...]` in game.toml and runtime -disc-swapping. Not blocking on game work; can land any time after Phase B. - -D1. Schema: `[game]` learns `discs = ["disc1.cue", "disc2.cue", ...]`. -For backwards compatibility, accept either `disc = "x.cue"` (single) -or `discs = [...]` (multi). - -D2. Runtime exposes a `swap_disc ` debug command. CD-ROM subsystem -gates on which disc is currently mounted. - -D3. Boot-time: mount disc 0 by default. - -D4. Test on Tomba (single disc) to confirm no regression, then on a -real multi-disc game (FF7 / RE2 — TBD). - -## Rollback / abort criteria - -At any phase, if the checkpoint fails: - -1. Restore `.happypath/tomba-runtime.exe` to its build location: - ``` - cp .happypath/tomba-runtime.exe \ - F:/Projects/TombaRecomp/build/tomba-runtime.exe - ``` -2. Stop work, update `CURRENT_STATE.md` with what was learned, and - surface the blocker to the user before retrying. - -## Decisions made 2026-05-12 - -1. **BIOS config location**: extend `bios/SCPH1001.toml` in place. Add - `[recompiler]` and `[runtime]` sections to the existing file. The - `[program]` and `[audit]` blocks already there stay. -2. **Game configs**: stay in sibling game repos (e.g. - `TombaRecomp/game.toml`). Matches existing sibling-junction + - `.pin` pattern shared with nesrecomp / segagenesisrecomp. -3. **Runtime name**: keep `psx-runtime`. Becomes parameterized; the - sibling-repo `tomba-runtime` target retires and the merged binary - is invoked as `psx-runtime ` (optional). -4. **Multi-disc test target**: deferred. Phase D designs the - `[discs] = [...]` schema; we pick a concrete game (FF7 / RE2 / MGS) - when we're actually ready to validate disc-swapping. diff --git a/WIDESCREEN.md b/WIDESCREEN.md index f87b9b91..113959b3 100644 --- a/WIDESCREEN.md +++ b/WIDESCREEN.md @@ -65,6 +65,11 @@ sprite_anchor_addr = "0x1F800070" # scratchpad holding the prim's # GTE-projected anchor SXY at tag time. hud_sprt_squash = true # center/edge-squash untagged SPRTs # (screen-space HUD/menus). +clear_reveal = true # clear synthetic native-wide side margins + # at opted-in scene/map boundaries (default false). +nw_left_hud_packet_lo = "0x000E3400" # optional targeted left-HUD packet range +nw_left_hud_packet_hi = "0x000E4100" # (half-open); avoids shifting 2D scenery. +offer_ultrawide = true # separate experimental 21:9 launcher row. ``` Tomba's values are Ghidra-evidenced: `0x8005E08C` is the shared per-prim helper @@ -219,12 +224,12 @@ Runtime-only changes (everything except `sprite_tag_funcs` / cull-widen) just need a rebuild: ``` PATH=/c/msys64/mingw64/bin:$PATH -cmake --build F:/Projects/psxrecomp/TombaRecomp/build-stable --target psx-runtime -j16 +cmake --build ../TombaRecomp/build-stable --target psx-runtime -j16 ``` Changes that emit into generated C (`sprite_tag_funcs`, future cull-widen) need a regen first (see `memory/regen_build_recipe.md`): ``` -F:/Projects/psxrecomp/psxrecomp/recompiler/build/psxrecomp-game.exe --config game.toml +recompiler/build/psxrecomp-game.exe --config game.toml ``` `build-stable` is RelWithDebInfo + debug tools (TCP port 4470) + launcher ON. Always `taskkill /F /IM psx-runtime.exe` before relaunching. diff --git a/accuracy/axis5_gpu.md b/accuracy/axis5_gpu.md index d1428d04..94f2f402 100644 --- a/accuracy/axis5_gpu.md +++ b/accuracy/axis5_gpu.md @@ -216,6 +216,25 @@ vs `beetle-psx/mednafen/psx/:line` (oracle). and can scribble across VRAM where hardware would have dropped it). - **Effect**: rare; mainly a robustness/timing divergence. Low priority. +### D12 — Textured-polygon texpage attribute did not update GPU draw-mode state — FIXED 2026-07-05 +- **Oracle**: every textured polygon's texpage word is copied into the GPU + draw-mode state before drawing — `SetTPage(CB[4 + ((cc>>4)&1)] >> 16)` runs + for all GP0 0x20-0x3F with the texture bit (gpu.cpp:1070-1072). The poly's + own word (not the last E1) selects its semi-transparency mode, and later + rect/sprite prims (no texpage word) consume the state the poly left behind. + Poly words cannot alter dither/draw-to-display (E1-only bits 9-10) nor + texture-disable (bit 11 latches only under GP1(09h) TexDisableAllowChange). +- **Ours (pre-fix)**: textured polys fetched texels from their own tpage but + blended with the stale E1-set global `semi_transparency`, and never updated + `texpage_x/y/colors` for subsequent rects. +- **Effect**: additive glows (mode 1 in the prim word) blended with whatever + mode the last E1 set — Ape Escape's headlamp halos / siren flares / firework + bursts rendered as dark blobs or black starbursts, intermittently (depends + on OT bucket order, hence camera/distance). Proven on Ape attract frame + 60052: nine op-0x2E quads at the artifact coords carried prim mode 1 vs E1 + mode 0. Fixed in `gpu.c set_tpage_from_poly()`, called by all four textured + poly handlers; all backends inherit via `gr_set_semi_transparency`. + ### Items that are FAITHFUL (no action) - Semi-transparency 4-mode selection + STP-bit gating (SW and GL). - Mask set/check including in copies (SW and GL). diff --git a/accuracy/axis5_mdec.md b/accuracy/axis5_mdec.md index de637551..e7ad4b21 100644 --- a/accuracy/axis5_mdec.md +++ b/accuracy/axis5_mdec.md @@ -1,7 +1,7 @@ # Axis 5 — MDEC (Motion Decoder / FMV) accuracy cross-reference Scope: `runtime/src/mdec.c` (our impl) vs the in-tree Beetle/Mednafen oracle -`F:\Projects\psxrecomp\psxrecomp\beetle-psx\mednafen\psx\mdec.cpp`, plus nocash +`beetle-psx/mednafen/psx/mdec.cpp`, plus nocash psx-spx "Macroblock Decoder (MDEC)". READ-ONLY analysis; no source/build touched. References below cite `mdec.c:NN` for ours and `mdec.cpp:NN` for Beetle. diff --git a/accuracy/axis5_sio_controller.md b/accuracy/axis5_sio_controller.md index 5372cca4..2263f5f6 100644 --- a/accuracy/axis5_sio_controller.md +++ b/accuracy/axis5_sio_controller.md @@ -2,7 +2,7 @@ Cross-reference of `runtime/src/sio.c` (+ `main.cpp` pad driver, `memcard.c`) against the in-tree Beetle/Mednafen oracle -(`F:\Projects\psxrecomp\psxrecomp\beetle-psx\mednafen\psx\`: +(`beetle-psx/mednafen/psx/`: `frontio.cpp`, `input/dualshock.cpp`, `input/gamepad.cpp`, `input/memcard.cpp`) and nocash psx-spx "Controllers and Memory Cards". diff --git a/accuracy/axis5_spu.md b/accuracy/axis5_spu.md index 8cf61db2..9810e140 100644 --- a/accuracy/axis5_spu.md +++ b/accuracy/axis5_spu.md @@ -5,7 +5,7 @@ oracle and nocash psx-spx. - **Our impl:** `runtime/src/spu.c` (799 LOC) + `runtime/src/spu_shadow.c` (float resample shadow, opt-in, verify-only — irrelevant to hardware accuracy). -- **Oracle:** `F:/Projects/psxrecomp/psxrecomp/beetle-psx/mednafen/psx/spu.cpp` +- **Oracle:** `beetle-psx/mednafen/psx/spu.cpp` (1664 LOC, `PS_SPU`). - **Spec:** nocash psx-spx "Sound Processing Unit (SPU)". diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 00000000..3293157c --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,159 @@ +# Architecture + +How PSXRecomp is put together, end to end. For the higher-level "why three ways +of running code" story, read [`EXECUTION_MODEL.md`](EXECUTION_MODEL.md) first; +this doc is the component-level view. + +## Two programs: the recompiler and the runtime + +PSXRecomp is split into two CMake projects that are built and run separately: + +``` + recompiler/ runtime/ + ─────────── ──────── + MIPS ──▶ C C + hardware simulation ──▶ native game binary + (build-time tool) (linked with the generated C) +``` + +- **`recompiler/`** (C++20) is an offline tool. It reads MIPS R3000A machine code + and emits C source files under `generated/`. +- **`runtime/`** (C99 + C++17) is the engine. It loads the game's assets into an + emulated PS1 address space, links the generated C in as native functions, and + simulates the console's hardware around them. + +A **game repository** (e.g. TombaRecomp) contains no framework source — it links +this framework in (as a submodule) and provides the game's config, seeds, and +build glue. See [`BUILDING.md`](BUILDING.md#linking-the-framework). + +## The recompiler (`recompiler/`) + +Two entry points share the same translation core: + +- `src/main_bios.cpp` — ingests the flat BIOS ROM (`SCPH1001.BIN`, loaded at + `0xBFC00000`) and emits `generated/SCPH1001_*.c`. +- `src/main_psx.cpp` — ingests a `PS-X EXE` extracted from a game disc and emits + `generated/_*.c`. + +The translation pipeline: + +| Stage | File | Job | +|---|---|---| +| Decode | `mips_decoder.cpp` (+ vendored `rabbitizer`) | Decode MIPS instructions | +| Control flow | `control_flow.cpp`, `basic_block.cpp` | Build basic blocks / CFG | +| Discovery | `function_discovery.cpp`, `function_analysis.cpp` | Find function entry points (seeded from Ghidra exports) | +| Codegen | `code_generator.cpp`, `full_function_emitter.cpp`, `strict_translator.cpp` | Emit C: one function per guest function, plus a dispatch table | + +Output is **two files** per program: a `_full.c` (the function bodies) and a +`_dispatch.c` (the address→function dispatch table). Both are build artifacts and +are **never hand-edited** — if the C is wrong, the fix is in the recompiler. + +GTE (the PS1 geometry coprocessor) instructions are emitted inline; +COP0 kernel-mode instructions the BIOS needs are handled in codegen. + +## The runtime (`runtime/`) + +The runtime is assembled by one CMake helper, `psxrecomp_add_runtime_target()` +(defined in `runtime/runtime.cmake`), which a game's `CMakeLists.txt` calls with +the paths to its generated C. The runtime provides: + +- **Memory & address space** (`memory.c`) — the 2 MB main RAM, scratchpad, BIOS + ROM, and MMIO regions, with the dispatch that routes a guest PC to its native + function (static, overlay, or interpreter). +- **Hardware simulation via MMIO handlers** — GPU (`gpu.c`, `gpu_sw_renderer.c`, + `gpu_gl_renderer.c`), DMA (`dma.c`), timers (`timers.c`), CD-ROM + (`cdrom.c`, `iso_reader.cpp`), MDEC, SIO0 controllers/memory cards + (`sio.c`, `memcard.c`), SPU (`spu.c`), GTE (`gte.cpp`), and interrupt delivery + (`interrupts.c`). +- **Host services** — window/input/audio via SDL2, a cooperative-thread + scheduler on host fibers (`psx_fiber.c`: Win32 Fibers / POSIX `ucontext`), and + the optional debug TCP server (`debug_server.c`). + +### BIOS: LLE baseline + a swappable HLE tier + +The recompiled `SCPH1001.BIN` is the **low-level (LLE) baseline**: it *is* the +kernel, and it is the reference implementation and the correctness oracle. There +are no per-vector HLE shims replacing it. + +On top of that, PSXRecomp carries an optional **HLE tier** (`bios_hle`, on by +default for player convenience) that can synthesize the post-boot kernel handoff +state to skip the BIOS boot sequence, and intercept a small set of BIOS services +— always falling through to the recompiled BIOS for anything it doesn't +implement. LLE stays fully linked and is what every accuracy check runs against. +Turn it off with `[runtime] bios_hle = false` or `PSX_BIOS_HLE=0`; with it off the +build behaves as pure LLE. (Design notes: +[`docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md`](internal/HLE_SCHEDULER_CARVEOUT_PLAN.md).) + +### Static / overlay / interpreter dispatch + +This is the heart of the system and has its own doc, +[`EXECUTION_MODEL.md`](EXECUTION_MODEL.md). In brief: a guest PC resolves to +statically-recompiled native code (BIOS + main EXE), a runtime-compiled **native +overlay** (`overlay_capture.c` → `tools/compile_overlays.py` → +`overlay_loader.c`), or the small **dirty-RAM interpreter** +(`dirty_ram_interp.c`) — in that priority order. + +### Overlay compile backend + +When an overlay needs compiling, the runtime spawns a C compiler on the +recompiler-emitted C and loads the resulting DLL — it does not JIT in-process. +The backend tier is resolved in `overlay_backend.c` (see `main.cpp` around the +`code_provider` setup): + +``` + static → gcc → tcc +``` + +- **static** — the overlay was baked into the binary at build time (best case). +- **gcc** — used when a system `gcc` is on `PATH` (the development default). +- **tcc** — TinyCC, the toolchain-free fallback **bundled beside shipped + executables** in `overlay_toolchain/` (an embedded Python + `tcc.exe`) so + players never need a compiler installed. + +Compiled overlays are stored in a content-addressed cache namespaced by +compiler and target ABI (`/gcc//…` vs `/tcc/…`); a `gcc` +shard wins over a `tcc` shard for the same region. See +[`docs/FEATURES.md`](FEATURES.md), [`docs/OVERLAY_CACHE_V2.md`](OVERLAY_CACHE_V2.md), +and [`docs/ASYNC_OVERLAY_COMPILE.md`](ASYNC_OVERLAY_COMPILE.md). + +## Renderers + +Three GPU backends behind one interface: + +- **Software rasterizer** — CPU, most portable, the reference look. +- **OpenGL** — GPU-authoritative VRAM/FBO renderer, the default; moves + rasterization and supersampling onto the GPU. Falls back to software if GL + init fails. (See [`docs/internal/GL_RENDERER_HANDOFF.md`](internal/GL_RENDERER_HANDOFF.md).) +- **Vulkan** — experimental, off by default (`PSX_ENABLE_VULKAN=OFF`). + +Widescreen (a genuine wider GTE FOV, not a stretch) is opt-in and gen-time; see +[`WIDESCREEN.md`](../WIDESCREEN.md) and +[`docs/internal/NATIVE_WIDE_PLAN.md`](internal/NATIVE_WIDE_PLAN.md). + +## The oracle model (how correctness is checked) + +PSXRecomp validates itself against **Beetle PSX** (the mednafen-psx libretro +core), run as a **separate process** with an identical TCP JSON debug protocol: + +- `psx-runtime` — the recompiled runtime (debug server on port 4370). +- `psx-beetle` — Beetle PSX (debug server on port 4380). + +A tool written against one works against the other by switching ports; cross- +checking is done by querying both, never by sharing state in one process. There +is also a first-divergence **co-sim** build that cycle-locksteps the compiled +backend against the interpreter. See +[`docs/internal/COSIM_ORACLE.md`](internal/COSIM_ORACLE.md), +[`docs/config_schema.md`](config_schema.md), and [`TCP_COMMANDS.md`](../TCP_COMMANDS.md). + +## Configuration + +Every process has a **BIOS config**; a game adds a **game config** (`game.toml`), +and the two are merged (game wins on scalar keys; program blocks and generated +dispatch tables are additive). Full schema: +[`docs/config_schema.md`](config_schema.md). + +## Where to go next + +- [`EXECUTION_MODEL.md`](EXECUTION_MODEL.md) — static/native/interp in depth. +- [`BUILDING.md`](BUILDING.md) — dependencies + build steps. +- [`../CONTRIBUTING.md`](../CONTRIBUTING.md) — dev workflow and rules. +- [`../CLAUDE.md`](../CLAUDE.md) — the exhaustive engineering constitution. diff --git a/docs/BUILDING.md b/docs/BUILDING.md new file mode 100644 index 00000000..ce2423b7 --- /dev/null +++ b/docs/BUILDING.md @@ -0,0 +1,181 @@ +# Building PSXRecomp + +PSXRecomp builds natively on **Windows** (MSVC or MinGW/MSYS2), **macOS** (Apple +Silicon & Intel), and **Linux**. There are two things you can build: + +- the **framework** itself (this repo) — a BIOS-only runtime plus the recompiler + tool; useful for BIOS work and as the thing game repos link against; +- a **game** — done from that game's own repository, which links this framework + in as a submodule (see [Linking the framework](#linking-the-framework)). + +You always supply your own **`SCPH1001.BIN` BIOS** and, for a game, your own +legally-obtained **disc image**. This project ships none of those. + +## Toolchain requirements + +- A C/C++ toolchain: + - **Windows:** MSYS2 MinGW-w64 (`mingw-w64-x86_64` toolchain) *or* MSVC. + - **macOS:** Apple Clang (Xcode command-line tools). + - **Linux:** GCC or Clang. +- **CMake ≥ 3.20**. On macOS/Linux also **Ninja** and **pkg-config**. +- Language standards used: recompiler is **C++20**; runtime is **C99 + C++17**. +- `ccache` is auto-detected and used if present (optional, speeds rebuilds). + +## Dependencies + +| Dependency | How it's provided | Used for | +|---|---|---| +| **SDL2** | System. Windows/MSVC: prebuilt pack auto-found at `../sdl2-msvc/SDL2-*`. Windows/MinGW + macOS + Linux: found via pkg-config. | Window, input, GL/Vulkan context, audio, threads | +| **RmlUi** | **git submodule** `lib/RmlUi` (static) | The launcher UI | +| **FreeType** | **git submodule** `lib/freetype` (static) | RmlUi's font engine | +| **fmt** 9.1.0 | vendored `recompiler/lib/fmt` | String formatting (runtime uses it header-only) | +| **toml11** | vendored `recompiler/lib/toml11` | Parsing `game.toml` / configs | +| **ELFIO** | vendored `recompiler/lib/ELFIO` | ELF parsing (recompiler only) | +| **rabbitizer** | vendored `recompiler/lib/rabbitizer` | MIPS instruction decoding (recompiler only) | +| **TinyCC (TCC) 0.9.27** | Not in this repo — downloaded at release-packaging time and bundled beside the game exe in `overlay_toolchain/`. | Toolchain-free overlay compilation for players (run as a subprocess) | +| **Python 3** | System (development) or an embedded copy bundled in releases | Runs `tools/compile_overlays.py` in the overlay pipeline | +| **OpenGL** | System (`opengl32` on Windows; `find_package(OpenGL)` elsewhere) | The GL renderer | +| **Vulkan** | Headers only, optional, **off by default** (`PSX_ENABLE_VULKAN=OFF`); loaded dynamically via SDL. Shader compilation needs `glslc` from the Vulkan SDK. | The experimental Vulkan renderer | + +Developers building overlays locally just need `gcc` on `PATH` (the `gcc` tier); +the bundled `tcc` matters only for end-user release packages. + +### Get the source (with submodules) + +RmlUi and FreeType are submodules, so clone recursively (or init after the fact): + +```sh +git clone --recurse-submodules https://github.com/mstan/psxrecomp.git +# or, in an existing clone: +git submodule update --init --recursive +``` + +## Per-platform prerequisites + +**Windows (MSYS2/MinGW — recommended for release parity):** +```sh +# In an MSYS2 MinGW64 shell: +pacman -S --needed mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake \ + mingw-w64-x86_64-ninja mingw-w64-x86_64-SDL2 mingw-w64-x86_64-ccache +``` + +**macOS:** +```sh +brew install sdl2 pkg-config ninja cmake +``` + +**Linux (Debian/Ubuntu):** +```sh +sudo apt install build-essential cmake ninja-build pkg-config libsdl2-dev +``` + +## Build the framework + +Two CMake trees: the recompiler (a tool) and the runtime (the engine). + +```sh +# 1. Recompiler tool → produces psxrecomp-bios and psxrecomp-game +cmake -S recompiler -B recompiler/build -G Ninja -DCMAKE_BUILD_TYPE=Release +cmake --build recompiler/build + +# 2. (Optional) regenerate the BIOS C from your SCPH1001.BIN +# Requires bios/SCPH1001.BIN to be present. +bash tools/regen_bios.sh + +# 3. Runtime → produces psx-runtime (BIOS-only for this repo) +cmake -S runtime -B runtime/build -G Ninja -DCMAKE_BUILD_TYPE=Release +cmake --build runtime/build --target psx-runtime +``` + +On Windows with MSVC or plain MinGW makefiles, swap `-G Ninja` for your generator +(e.g. `-G "Unix Makefiles"`); everything else is identical. + +> **Build type matters.** With no `-DCMAKE_BUILD_TYPE`, the framework defaults to +> **Release** (optimized). The huge generated C compiles unusably slowly at `-O0`, +> so never build it debug-by-accident. Use `-DCMAKE_BUILD_TYPE=RelWithDebInfo` +> when you need symbols. Release turns the debug TCP server **off**; RelWithDebInfo +> and Debug turn it **on** (`PSX_DEBUG_TOOLS`). + +### Useful CMake options + +| Option | Default | Effect | +|---|---|---| +| `PSX_DEBUG_TOOLS` | ON for Debug/RelWithDebInfo, OFF for Release | TCP debug server + heartbeat + per-block recording | +| `PSX_STATIC_RUNTIME` | ON for MinGW Release | Self-contained exe (statically links SDL2 + libgcc/libstdc++) | +| `PSX_LAUNCHER` | ON | Build the RmlUi launcher UI | +| `PSX_ENABLE_VULKAN` | OFF | Build the experimental Vulkan renderer | +| `PSX_BUILD_COSIM` | OFF | Build the first-divergence co-sim oracle target | + +## Build and run a game + +From the **game's** repository (not this one). Each game repo links this +framework in and provides its own `game.toml`, seeds, and generated C. Example +for TombaRecomp: + +```sh +# Extract the game's PS-X EXE from your disc (helper included in the game repo): +python3 tools/extract_psx_exe.py tomba/tomba.bin SCUS_942.36 tomba/SCUS_942.36 + +# Regenerate the game's C from the disc/EXE (game repos invoke the recompiler +# binary directly with their config): +../psxrecomp/recompiler/build/psxrecomp-game --config game.toml + +# Configure + build the game runtime: +cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release +cmake --build build --target psx-runtime +./build/psx-runtime --game game.toml --disc tomba/tomba.cue +``` + +## Linking the framework + +Game repositories do **not** vendor the framework. They reference it as a git +**submodule** at path `psxrecomp/`, which the game's `CMakeLists.txt` points +`PSXRECOMP_ROOT` at: + +```cmake +set(PSXRECOMP_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/psxrecomp" CACHE PATH "Path to psxrecomp") +include("${PSXRECOMP_ROOT}/runtime/runtime.cmake") +``` + +A fresh clone of a game repo gets the framework automatically: + +```sh +git clone --recurse-submodules https://github.com/mstan/TombaRecomp.git +``` + +**Local dev tip — share one framework checkout across games.** If you work on +several game repos plus the framework at once, you don't want N multi-GB copies. +Replace each game's `psxrecomp/` submodule working directory with a **junction / +symlink** to a single shared framework checkout: + +```sh +# Windows (junction): +cmd /c mklink /J psxrecomp F:\path\to\shared\psxrecomp +# macOS / Linux (symlink): +ln -s /path/to/shared/psxrecomp psxrecomp +``` + +Git treats the link's target as the submodule working tree, so +`git submodule status` stays clean **as long as the shared checkout is at the +commit the game pins**. Bump a game to a newer framework by checking out the new +commit in the shared repo and committing the updated submodule pointer in the +game repo. This is the setup the game repos are configured for. + +## Troubleshooting + +**MinGW: `Error: too many sections` / a `*.c.o` (or `.obj`) fails to assemble.** +Windows COFF object files have a 32,768-section limit; very large translation +units (the generated game C, and `debug_server.c` in debug-info builds) can +exceed it on older binutils. The fix is to assemble those units as *big objects*: +add `-Wa,-mbig-obj` to their compile options (recent binutils, ≥ ~2.40, handle +these files without it). If you hit this on a framework source file, please open +an issue with your `gcc -v` / `as --version` — the build should apply the flag +for you. + +**`SDL2 MSVC dev package not found`.** The MSVC build expects the prebuilt SDL2 +pack beside the repo at `../sdl2-msvc/SDL2-*`. Use the MSYS2/MinGW toolchain +(which finds SDL2 via pkg-config) or place the pack there. + +**Overlays never compile / stay slow.** In development you need `gcc` on `PATH` +for the `gcc` tier; otherwise areas stay in the interpreter. See +[`EXECUTION_MODEL.md`](EXECUTION_MODEL.md). diff --git a/docs/EXECUTION_MODEL.md b/docs/EXECUTION_MODEL.md new file mode 100644 index 00000000..abc68745 --- /dev/null +++ b/docs/EXECUTION_MODEL.md @@ -0,0 +1,160 @@ +# The Execution Model — static, native-overlay, and interpreter + +*A semi-technical tour of how PSXRecomp actually runs a PlayStation game. If you +only read one design doc, read this one.* For the low-level plumbing, see +[`ARCHITECTURE.md`](ARCHITECTURE.md). + +## The one-sentence version + +PSXRecomp turns a PS1 game into a **native program** by translating its MIPS code +to C ahead of time; the small amount of code it *can't* see ahead of time is +either compiled to native code the first time it appears, or run by a tiny +built-in interpreter until it is — so the game is always **correct**, and gets +**faster** the more it's played. + +## Why a PS1 game can't just be "decompiled once" + +A PlayStation game is not one fixed blob of code sitting in memory. The main +program (the `PS-X EXE` on the disc) is, but games are far bigger than the PS1's +2 MB of RAM, so they **stream code off the disc as you play**. A level's logic, +a boss's AI, a menu — these are loaded from the CD into a fixed region of RAM, +run, and then *overwritten* by the next chunk when you move on. These streamed +chunks are called **overlays**. + +That creates a hard problem for any ahead-of-time recompiler: + +- The main EXE exists on disc at build time → we can translate all of it. +- An overlay's code **does not exist anywhere at build time** in a form we can + find. It only appears in RAM at a specific moment during play, and the same + RAM addresses get reused by dozens of different overlays over a playthrough. + +So "recompile the game" can never be a single up-front pass. PSXRecomp handles +this with **three tiers of execution**, in strict priority order. + +``` + ┌─────────────────────────────────────────────────────────┐ + │ Guest wants to run code at some address │ + └─────────────────────────────────────────────────────────┘ + │ + ┌─────────────────────┼─────────────────────┐ + ▼ ▼ ▼ + 1. STATIC (AOT) 2. NATIVE OVERLAY 3. INTERPRETER + Translated at Compiled to native Runs the guest's + build time from code the first time own instructions + the disc's EXE the overlay is seen, directly, as a + and the BIOS. then cached forever. correctness safety + The bulk of Fills the "streamed net for anything + execution. from disc" gap. not yet native. + ── fastest ── ── fast ── ── slowest, always + correct ── +``` + +## Tier 1 — Static (ahead-of-time) + +At build time the **recompiler** reads two things and translates their MIPS +machine code into C, function by function: + +- the real PS1 **BIOS** (`SCPH1001.BIN`) — this becomes the kernel, and +- the game's **main executable** extracted from your disc. + +That C is compiled and linked into the runtime as ordinary native functions, so +when the game calls one of them the CPU just runs native code — no emulation of +individual instructions. **This is the overwhelming majority of what executes**: +boot, the engine, the main loop, most gameplay systems. It is as fast as the C +compiler can make it. + +Everything a pure AOT recompiler can see, it recompiles. The only things left +over are the two categories below. + +## Tier 2 — Native overlays (compiled the moment they're discovered) + +Overlays can't be seen at build time — but they *can* be seen the instant the +game loads one. PSXRecomp watches for that moment and turns each overlay into +native code so that, from then on, it runs at Tier-1 speed: + +1. **Capture.** Every time the game DMAs code from the disc into the overlay RAM + window, the runtime records the exact bytes (before the game can modify them) + and the addresses that were actually executed. These go into a per-game file, + `overlay_captures.json`. +2. **Compile.** Those captured bytes are fed back through the *same recompiler* + used for the main EXE, producing C for that overlay. The C is compiled to a + small native library (a DLL) — using `gcc` in development, or a bundled + `tcc` (TinyCC) toolchain in shipped releases so players need no compiler + installed. Each library is keyed by the overlay's content hash. +3. **Cache & dispatch.** The compiled overlay lands in a content-addressed + cache. On this run and every future run, when that overlay loads, the runtime + finds the matching native library and dispatches into it **before** falling + back to the interpreter. + +The practical effect: **the first time anyone visits an area, it may run in the +slow tier for a moment; after it's been captured and compiled, that area is +native for them — and, once the capture is shared into a release, for everyone.** +Coverage grows toward "100% native" the more the game is played. + +## Tier 3 — The interpreter (the correctness safety net) + +There is always a window between "an overlay just appeared in RAM" and "we have +native code for it." There's also a genuinely hard corner: code that is +**rewritten to different bytes on every load** (self-modifying / per-load +relocated), which by definition has no single ahead-of-time translation. + +For both, PSXRecomp includes a **small MIPS interpreter** that runs the guest's +*own* instructions directly on the CPU register state. It is deliberately narrow: + +- It runs **only** code that was installed into RAM at runtime (dirty pages) and + is not yet covered by a native overlay. It is **never** on the BIOS or + main-EXE path — those are always static. +- It is a **safety net and a coverage feeder**, never a substitute for + recompiling. If the interpreter is running something, that's a signal to + capture-and-compile it (Tier 2), not to leave it interpreted. + +Crucially, this is **not** high-level emulation (HLE). The interpreter executes +the program's real instructions, exactly as the game's authors wrote them — the +*only* difference from static recompilation is where the instructions came from +(RAM-at-runtime vs disc-at-build-time). We never synthesize "the answer the code +would have produced." + +## The guiding rule: precision over recall + +The tiers are ordered so the system can only ever fail *toward being slow*, never +*toward being wrong*: + +- Code we **haven't** compiled yet safely drops to the interpreter and gets + captured for next time — under-coverage **self-heals**. +- Code we might compile **wrong** would corrupt the machine — so native code is + dispatched *only* when its source RAM is provably unchanged, and a compiled + overlay is instantly revoked the moment the RAM it came from is overwritten. + +> **The worst case is always performance, never correctness.** Anything not yet +> native simply runs interpreted, correctly. + +## How the gaps actually get plugged + +Two complementary loops close the gap toward full native coverage: + +- **While you play (automatic).** The capture step runs continuously in the + background. New areas you reach are recorded and compiled to native code — + often within a minute, without interrupting play — and reused on every later + launch. Your discoveries live in `overlay_captures.json` next to the game. +- **In releases (shared head start).** A game release ships a `cache/` folder of + overlays already compiled from everywhere players have collectively visited, + so common areas are native from the very first moment you arrive. + +The frontier of the project is pushing Tier-2 coverage up (so the interpreter +goes idle) and, eventually, handling the self-modifying corner by baking in the +relocation pattern at build time. Until then, that corner stays correctly +interpreted — a narrow, accepted cost, not a wall. + +## Where this lives in the code + +| Concern | Where | +|---|---| +| MIPS→C translation (static + overlay) | `recompiler/src/` (`code_generator.cpp`, `mips_decoder.cpp`, …) | +| Overlay capture | `runtime/src/overlay_capture.c` | +| Overlay compile pipeline | `tools/compile_overlays.py`, `runtime/src/overlay_backend.c`, `overlay_compile_worker.c` | +| Native overlay load/dispatch | `runtime/src/overlay_loader.c` | +| Runtime interpreter (dirty RAM) | `runtime/src/dirty_ram_interp.c` | + +See [`ARCHITECTURE.md`](ARCHITECTURE.md) for how these connect to the hardware +simulation, and [`docs/FEATURES.md`](FEATURES.md) for the overlay-system feature +reference. diff --git a/docs/MEMCARD_AUDIT_2026_04_28.md b/docs/MEMCARD_AUDIT_2026_04_28.md deleted file mode 100644 index 27717e97..00000000 --- a/docs/MEMCARD_AUDIT_2026_04_28.md +++ /dev/null @@ -1,294 +0,0 @@ -# Memory-Card End-to-End Audit — 2026-04-28 - -Branch: `feature/cycle-paced-sio-irq` @ commit `5b3c7dd` -Date: 2026-04-28 -Operator: full audit run on a freshly-rebuilt native runtime (`psx-runtime.exe`). - -End-to-end success criterion: real `.mcr` save data (e.g. the -`BISLPS-01234SAVE00` save title in `card1.mcd`) appears as a list entry -on the BIOS MEMORY CARD screen, having travelled through the real SIO -protocol — no HLE shims, no fake directory entries, no edits to -generated code. - -This audit walks all seven layers, fills the table, and identifies the -first failed layer with a fix proposal scoped to it. No source change -beyond two infrastructure additions made during the audit -(`tools/debug_client.py` JSON-read fix; new `card_buffer_dump` TCP -command). - ---- - -## Audit table - -| # | Layer | Component | Expected | Observed | Evidence | Verdict | -|---|---|---|---|---|---|---| -| 1.1 | L1 | `bios/dummy.0.mcr` exists | exists per session prompt | does NOT exist | `ls -la bios/` shows only `SCPH1001.BIN`, `dummy.bin`, `dummy.cue`, `scph5501.bin` | **FAIL (prompt-only — informational)** | -| 1.2 | L1 | All `.mcr`/`.mcd` files are 131072 B | 131072 | all four are 131072 B | `ls -la card1.mcd card2.mcd dummy.0.mcr dummy.1.mcr` | PASS | -| 1.3 | L1 | "MC" magic at offset 0 | `4D 43` | `4D 43` on all four | `xxd -s 0 -l 0x80` on each | PASS | -| 1.4 | L1 | Frame 1 dir entry semantics | "used" or "free" with valid checksum | card1.mcd: 2 used (BISLPS-01234SAVE00, BASLUS-56789DATA01); card2.mcd: empty; dummy.0.mcr: 2 used (same first two as card1 but blocks 3-15 differ); dummy.1.mcr: 1 used (BASLUS-0125100000-00) | hex dump | PASS | -| 1.5 | L1 | Native source = Beetle source (sha256) | byte-equal | sha256 differs across all four pairs (even the first 16-frame directory region differs) | `sha256sum card1.mcd dummy.0.mcr` → `f79bfc41…` vs `e146de7c…` | **FAIL (test-setup, blocks oracle byte-diff)** | -| 2.1 | L2 | slot 0 in-memory buffer = `card1.mcd` on disk | byte-equal sha256 | MATCH | `python tools/verify_card_buffer.py` slot 0 sha = disk sha = `f79bfc41…` | PASS | -| 2.2 | L2 | slot 1 in-memory buffer = `card2.mcd` on disk | byte-equal sha256 | MATCH | same script slot 1 sha = `7706c7d4…` | PASS | -| 3.1 | L3 | Card transactions occur after entering MC screen | ≥1 closed READ txn | 542 attempted | `card_txn_dump count=10000` yields 542 entries | PASS | -| 3.2 | L3 | Each READ TX byte sequence matches protocol | 0x81, 0x52, 0x00, 0x00, … | first 4 TX correct, txn never gets past byte 4 | `card_txn_dump`: every entry has `tx:[0x81,0x52,0x00,0x00]` | PARTIAL | -| 3.3 | L3 | Each READ RX matches no$psx protocol | 0xFF, 0x00 (flag), 0x5A, 0x5D, 0x00, 0x00, 0x5C, 0x5D, echo_msb, echo_lsb, 128 data, chk, 0x47 | `[0xFF, 0x00, 0x5A, 0x5D]` only — protocol-correct for the first 4 bytes (mc_flag=0 after first read; ID bytes 0x5A 0x5D follow) | `card_txn_dump` rx field | PASS for the bytes seen | -| 3.4 | L3 | Transaction reaches END (terminal_state=12) | success | **all aborted** — 537 / 542 = `abort_reselect` at terminal_state=4 (MC_ADDR_MSB); 6 = `abort_reset` at terminal_state=0; 1 open; 0 success | `card_txn_dump` aggregate (see transcript A) | **FAIL** | -| 3.5 | L3 | RX matches Beetle byte-for-byte | byte equal | NOT MEASURED — gated on Layer 1 file equality, currently FAIL | (would be `card_txn_diff`) | BLOCKED-BY-L1 | -| 4.1 | L4 | Install handler at RAM 0xCF0 fires per byte | one fire per RX byte | install handler fired 26,030 times (3 insns/fire); chain entry 0x641C fired 26,030 times | `dirty_ram_stats` → per_pc 0xCF0 hits=26030 | PASS (handler runs) | -| 4.2 | L4 | `[0x72F0]` data-byte counter advances 0..127 | monotonic increase to 127 per sector | currently `[0x72F0] = 0`; never observed > 1 in any window | `read_ram addr=0x800072F0 len=4` | **FAIL** | -| 4.3 | L4 | `[0x755A]` abort flag = 0 across a transaction | 0 | currently 0; setter pattern matches prior memo `phase4_real_blocker_2026_04_28.md` (R1/D1 set unconditionally) | `read_ram addr=0x8000755A` (and `wtrace_dump` for slot 7) | needs deeper trace | -| 5.1 | L5 | I_MASK bit 7 (SIO) gets set during chain enter | ≥1 set | bit7_sets=205, bit7_clears=205 — toggled 205× during exception entries | `imask_trace count=64` | PASS | -| 5.2 | L5 | One IRQ #7 fires per card-protocol byte | 4-138 card IRQs per txn | **0 card-source IRQs in 200,000 most-recent IRQ entries** (full ring) — every recent IRQ is `src=pad` | `sio_irq_dump count=200000 src=1` → `entries:[]`, `emitted:0` | **FAIL — root signal** | -| 5.3 | L5 | `sio_irq_pending` arms after card TX byte | nonzero for ~delay cycles | `sio_irq_countdown` is set to `SIO_IRQ_DELAY_CARD=8` on every TX, but ticks happen only on SIO register access (`sio.c:830,961`) and on interpreter-step (`psx_interpreter.c:517`); between two card TX bursts the BIOS does <8 SIO accesses, so the countdown is rearmed before reaching 0 → IRQ never fires for cards | code read of `sio.c:854-894,974-1008` | **FAIL — proximate cause of 5.2** | -| 6.1 | L6 | `[0x80007568]` slot gates non-zero after card detect | ≥1 byte set | `01 01 00 00 01 …` — both slots flagged | `read_ram addr=0x80007568 len=16` | PASS | -| 6.2 | L6 | `[0x800074BC]` read flag set | 1 during read | `01 00 00 00` | `read_ram` | PASS | -| 6.3 | L6 | `[0x800075C0]` mode word | nonzero during read | `00 00 00 00` (currently idle) | `read_ram` | inconclusive (state machine returned idle by audit time) | -| 6.4 | L6 | `[0x80007550]` buffer ptr set before data phase | nonzero RAM pointer | `00 00 00 00` | `read_ram` | **FAIL** (buffer never set up — read never reached data phase) | -| 7.1 | L7 | MEMORY CARD screen reaches per-card directory listing | 15-row directory grid with save titles | screen shows MEMORY CARD operations menu (EXIT/COPY/COPY ALL/DELETE) only — no per-slot directory listing rendered; CARD 1 and CARD 2 column headers are visible but directory contents are blank | `screenshot_file` → `psx_screenshot.bmp` (see image: `MEMORY CARD operations` menu, no titles) | **FAIL** | - ---- - -## Evidence transcripts - -### Transcript A — card transaction outcome distribution - -``` -$ python tools/raw_tcp.py 4370 card_txn_dump count=10000 | grep -oE '"end_reason":"[^"]+"' | sort | uniq -c - 537 "end_reason":"abort_reselect" - 6 "end_reason":"abort_reset" - 1 "end_reason":"open" - -$ python tools/raw_tcp.py 4370 card_txn_dump count=10000 | grep -oE '"terminal_state":[0-9]+' | sort | uniq -c - 7 "terminal_state":0 # MC_IDLE — never even started CMD - 539 "terminal_state":4 # MC_ADDR_MSB — got 4 bytes, then aborted - -$ python tools/debug_client.py card_txn_dump count=2 | head -40 -{ - "id": 0, "ok": true, "total_closed": 94, "open": true, … - "entries": [ - { - "txn_seq": 78, "slot": 0, "cmd": "0x52", "sector": "0xFFFF", - "bytes": 4, "acks": 1, - "end_reason": "abort_reselect", "terminal_state": 4, - "tx": ["0x81", "0x52", "0x00", "0x00"], - "rx": ["0xFF", "0x00", "0x5A", "0x5D"] - }, … -``` - -Note: the `acks` field counts only the SELECT-byte ACK (`sio_mc_ack_count` -incremented at `sio.c:442`; subsequent state transitions set -`SIO_STAT_ACK` but don't increment that counter). It is misleading -without context. - -### Transcript B — IRQ ring delivery profile - -``` -$ python tools/raw_tcp.py 4370 sio_irq_dump count=200000 src=1 -{"id":1,"ok":true,"total":200000,"shown":200000,"src_filter":1,"entries":[ -],"emitted":0} - -$ python tools/raw_tcp.py 4370 sio_irq_dump count=20000 | grep -c '"src": "card"' -0 - -$ python tools/raw_tcp.py 4370 sio_irq_dump count=20000 | grep -c '"src": "pad"' -~20000 # 100% pad -``` - -### Transcript C — I_MASK toggle history (bit 7 SIO) - -``` -$ python tools/debug_client.py imask_trace count=4 -… "bit7_sets": 205, "bit7_clears": 205, … -{"old":"0x04D","new":"0x0CD","func":"0x00005D98","pc":"0xBFC14930","b7s":1,"exc":1} -{"old":"0x0CD","new":"0x04D","func":"0x00001444","pc":"0xBFC14A28","b7c":1,"exc":1} -… (repeated 205 times) -``` - -I_MASK bit 7 IS being toggled in exception entry — masking is not the -problem. The problem is upstream of mask: the IRQ never fires. - -### Transcript D — install-handler aggregate - -``` -$ python tools/raw_tcp.py 4370 dirty_ram_stats -{"blocks_run":26036, "insns_run":135391627, "aborts":38264, … - per_pc:[ - {"pc":"0x00000CF0", "hits":26030, "insns":78090}, ← install handler stub - {"pc":"0x0000641C", "hits":26030, "insns":104120}, ← chain dispatcher entry - {"pc":"0x00006594", "hits":26024, "insns":26024}, - {"pc":"0x000000B0", "hits":33642758, "insns":100928274}, ← B0 trampoline - … - ]} -``` - -### Transcript E — state-word snapshot - -``` -$ python tools/raw_tcp.py 4370 read_ram addr=0x80007568 len=16 -{"hex":"01010000010000000000000000000000"} # slot 0 + slot 1 detected -$ python tools/raw_tcp.py 4370 read_ram addr=0x800074BC len=4 -{"hex":"01000000"} # read-flag = 1 -$ python tools/raw_tcp.py 4370 read_ram addr=0x800072F0 len=4 -{"hex":"00000000"} # data-byte counter = 0 -$ python tools/raw_tcp.py 4370 read_ram addr=0x80007550 len=4 -{"hex":"00000000"} # buffer ptr never set -$ python tools/raw_tcp.py 4370 read_ram addr=0x8000755A len=4 -{"hex":"00000000"} # abort flag currently clear -``` - -### Transcript F — visual ground truth - -`psx_screenshot.bmp` (after pressing CROSS to enter MEMORY CARD): the -"MEMORY CARD" operations menu is visible with EXIT / COPY / COPY ALL / -DELETE entries and CARD 1 / CARD 2 column headers, but **no per-slot -directory listing is rendered**. The save title "BISLPS-01234SAVE00" -that exists in `card1.mcd` frame 1 does not appear anywhere on screen. - ---- - -## First failed layer - -**Layer 3 — transaction completion** is the highest-numbered layer -where the user-visible failure crystallises (transactions truncate at -byte 4 of 138, `mc_read_done = 0`), but the **proximate cause is -Layer 5 — IRQ delivery**. Every other failed row (4.2, 4.3, 6.4, 7.1) -is downstream of "no card IRQ ever fires". - -Specifically: in the 200,000-entry IRQ ring (full SIO_IRQ_RING_CAP), -**zero entries have source = `card`**. 542 transactions were attempted, -each rearmed `sio_irq_countdown = SIO_IRQ_DELAY_CARD = 8` on the TX -write, and zero of those countdowns ever reached 0. - -### Why the existing access-paced model can't deliver card IRQs - -`sio_tick()` decrements the countdown on each call. It is called from -(`grep -rn 'sio_tick' runtime/`): - -- every `sio_read()` (sio.c:830) — SIO_RX_DATA / SIO_STAT / etc. -- every I_STAT read (memory.c:220, 309) -- every read of an `0x1F801070..0x1F801074` register (memory.c:365) -- every interpreter step (psx_interpreter.c:517) - -For pads (delay = 4), the BIOS pad-poll routine touches SIO heavily — -it tight-polls SIO_STAT after each TX byte until RX_RDY appears. -4 ticks accumulate easily, IRQ fires. - -For cards (delay = 8), the BIOS card-protocol path is **chain-driven, -not poll-driven**: the chain handler writes one TX byte and returns, -expecting the next IRQ to drive the next byte. Between TX writes the -BIOS performs only a handful of SIO accesses. The countdown does not -reach 0 before the next TX rearms it. The IRQ never fires, so the -chain never re-enters, so the next byte never goes out, so the BIOS -times out and writes SIO_CTRL.RESET — which `txn_close` records as -`abort_reselect`. - -This is not an unmasked-IRQ issue (Transcript C confirms I_MASK bit 7 -is set on every chain enter). It is not a state-machine-byte issue -(Transcript A confirms the bytes our state machine returns are correct -for the first 4 positions). It is a pacing-model issue. - ---- - -## Proposed fix (scoped to Layer 5) - -The fix must move from access-paced to **time/cycle-paced** SIO IRQ -delivery, which is what real hardware does (~170 cycles after each -ACK, independent of register-access count). The branch name -`feature/cycle-paced-sio-irq` already states this intention; the -implementation has not landed. - -### Concrete proposal - -1. **Replace `sio_irq_countdown` with a frame-clock deadline.** Add a - monotonic counter ticked once per dispatched recompiled function - (or once per VBlank — the runtime does not have a true cycle - counter). On TX write, set - `sio_irq_deadline = sio_clock + N` where N is calibrated to "fires - within the same VBlank for a 138-byte card sector at 60 Hz" → at - most ~430 µs per byte → calibrate to ~1 dispatch loop iteration for - pads, similar for cards. -2. **Drive `sio_tick()` from the dispatch loop**, not from register - accesses. Call `sio_tick()` from `psx_dispatch` once per dispatched - function (or once per N functions). This gives consistent pacing - regardless of what the BIOS happens to be doing between bytes. -3. **Keep the access-paced ticks as a faster fast-path** so existing - pad protocol stays correct (or — preferably — re-validate pad - timing after the change and consolidate to the single model). - -### Risk-bounded interim step - -Before doing the full rework, run a **calibration probe**: change -`SIO_IRQ_DELAY_CARD` from 8 → 4 (matching pad). If 4 is enough to let -even one card IRQ through, we will see card-source entries appear in -`sio_irq_dump`, mc_max_state will exceed 4, and `card_txn_dump` will -show a transaction that gets past `terminal_state=4`. If yes — the -diagnosis is correct and we plan the cycle-paced rewrite. If no — the -problem is elsewhere (chain abort flag at [0x755A], install-handler -miswire) and we revisit. - -This calibration is exactly the kind of "is the hypothesis true" -measurement the audit was designed to gate. **No code change should -land until the calibration probe demonstrates a measurable shift in -the metrics this audit captured.** - -### Out of scope for the first fix - -- Layer 1 file-name divergence (card?.mcd vs dummy.?.mcr) — should be - unified before the next cross-oracle byte-diff session, but does not - block the native-side card path from working. -- The 23k-line/min `[dirty_ram_interp] WARN` fprintf flood - (CLAUDE.md §3 violation in `runtime/src/dirty_ram_interp.c`) — - separate cleanup task; doesn't block this fix. -- Visual sprite corruption on highlighted UI items (cursor sprite - rendering) — not on the card data path. -- The MEMORY CARD operations menu rendering after no-data — that's - expected BIOS behavior when the directory read fails. Once Layer 5 - is fixed, we re-test Layer 7. - ---- - -## Verification (after fix lands) - -End-to-end success of the fix: - -1. `python tools/raw_tcp.py 4370 sio_irq_dump count=200 src=1` — must - show non-empty entries (≥1 card-source IRQ). -2. `python tools/raw_tcp.py 4370 card_txn_dump count=20` — must show - ≥1 entry with `terminal_state` reaching `MC_READ_END=12` and - `end_reason: success`. -3. `python tools/raw_tcp.py 4370 read_ram addr=0x800072F0 len=4` — - must show `[0x72F0]` reaching values > 1 (confirming the - `phase4_72f0_one_byte_per_vblank` hypothesis is resolved). -4. `python tools/raw_tcp.py 4370 sio_state` — `mc_read_done` must be - ≥ 1. -5. `screenshot_file` after navigating into CARD 1 — the BMP must show - the save title `BISLPS-01234SAVE00` (from `card1.mcd` frame 1) as - a list entry. - -The screenshot (5) is ground truth. Until the save title is on screen -the session is not complete. - ---- - -## Tooling additions / fixes made during this audit - -These are minor infrastructure touch-ups, not BIOS-path changes. They -are committed regardless of whether the fix proposal is accepted, so -the audit harness stays repeatable. - -- `tools/debug_client.py` — `send_cmd()` now reads multi-line JSON via - brace/bracket balance instead of stopping at the first `\n`; the - `pretty_regs` formatter now also reads the server's `gpr` array - (positional) when no `regs` dict is present. Both were broken - before, hiding all GPRs and corrupting `card_txn_dump` parsing. -- `tools/raw_tcp.py` — same brace-balance read; `0x…` arg values are - preserved as JSON strings (server's `read_ram` etc. expect strings). -- `runtime/include/memcard.h` + `runtime/src/memcard.c` — new - `memcard_debug_read_buffer()` accessor. -- `runtime/src/debug_server.c` — new `card_buffer_dump` TCP command - (slot, offset, len → hex). Used by `tools/verify_card_buffer.py` to - prove Layer 2. -- `tools/verify_card_buffer.py` — pulls slot 0/1 buffers and - sha256-compares to disk. - -No edits to generated code (Rule 4 / Rule 14). No fprintf added. diff --git a/docs/RECURSION_BUG.md b/docs/RECURSION_BUG.md index 455657fa..a084d4fa 100644 --- a/docs/RECURSION_BUG.md +++ b/docs/RECURSION_BUG.md @@ -196,7 +196,7 @@ No stubs, no HLE, never edit generated C; the fix is in the recompiler/runtime. ## 6. Build & run (this project) ``` -cd /f/Projects/psxrecomp/TombaRecomp +cd ../TombaRecomp export PATH=/c/msys64/mingw64/bin:$PATH TMP=/c/msys64/tmp TEMP=/c/msys64/tmp # regen tools + BIOS + game (generated/ is gitignored; off master tip): cmake --build ../psxrecomp/recompiler/build --target psxrecomp-game psxrecomp-bios -j8 @@ -580,7 +580,7 @@ idles indefinitely, **USER-CONFIRMED** — and tips the per-frame loop SYMPTOM; the time-based divergence is the ROOT. Every soak so far was **us-vs-us**; we have never diffed against real hardware over the idle window. -**NEXT = the oracle.** psxref (`F:\Projects\psxref`, Beetle PSX, TCP **4380**, shares +**NEXT = the oracle.** psxref (``, Beetle PSX, TCP **4380**, shares `card1.mcd`; see [[psxref_oracle]] / `memory/psxref_oracle.md`). Run BOTH our recomp and psxref idle (**turbo** if possible, to reach ~frame 50k in well under 14 min wall clock) and find the **FIRST** state / control-flow divergence around frame ~50k — that diff --git a/docs/SHADOW_ENHANCEMENTS.md b/docs/SHADOW_ENHANCEMENTS.md index 2d2c3ae5..7f37272f 100644 --- a/docs/SHADOW_ENHANCEMENTS.md +++ b/docs/SHADOW_ENHANCEMENTS.md @@ -2,15 +2,15 @@ Backport of the gbarecomp "verified-enhancement" QoL layer to psxrecomp. All work lives on the `feat/shadow-enhancements` branch / `_shadow_psxrecomp` -worktree (sibling of the core repo `F:\Projects\psxrecomp\psxrecomp`, branched +worktree (sibling of the core repo `psxrecomp`, branched off `master`). It does not touch the in-flight `feat/gl-renderer`, `feat/mmx6-input`, `dev/ape-escape`, or any other branch/worktree. ## Which PSX core / language / build -- **Core chosen:** `F:\Projects\psxrecomp\psxrecomp` — the real git repo +- **Core chosen:** `psxrecomp` — the real git repo (`origin git@github.com:mstan/psxrecomp.git`, default branch `master`, current - to 2026-06-10). `F:\Projects\psxrecomp-v4` is a stale stub (symlink to + to 2026-06-10). `` is a stale stub (symlink to `psxrecomp/duckstation` + an empty `recompiler/build`), NOT a maintained core. - **Language:** runtime is **C99** (`spu.c`, `gpu.c`, …) with a little C++17 (`main.cpp`, `gte.cpp`). All shadow code is written in **C** to match the diff --git a/docs/STRING_TRANSLATION.md b/docs/STRING_TRANSLATION.md new file mode 100644 index 00000000..b9823de3 --- /dev/null +++ b/docs/STRING_TRANSLATION.md @@ -0,0 +1,909 @@ +# On-the-Fly String Translation / Localization for psxrecomp + +**Status:** SPEC (design only — no implementation yet). Reusable framework +feature; **Tsumu Light (SLPS-02253)** is the first consumer. + +**Goal.** Let a Japanese-only PSX title be played in English (or any target +language) by **capturing the game's source strings from memory and substituting +translated bytes on the fly**, so a non-Japanese speaker can navigate menus, +read tutorials/dialogue, and understand the game. **Ultimate success = every +string we can find renders in the target language.** Multilingual is a +first-class requirement (English is just the first target); the mechanism is a +**shared psxrecomp module** any title can adopt. + +This design is a direct PSX adaptation of the author's proven N64 prior art in +the `n64recomp` fork for **Pocket Monsters Stadium (Japan)** +(`F:\Projects\n64recomp\PocketMonstersStadiumRecomp\`). Section 1 documents that +mechanism; Section 2 documents Tsumu's concrete text reality; Sections 3+ define +the PSX framework. + +--- + +## 0. TL;DR architecture + +Three always-on subsystems hung off the **`psx_dispatch` call chokepoint** (the +one place every guest function call funnels through — see §3.1): + +1. **Capture ring (always-on).** On every dispatch, scan the argument registers + (and a few stack slots) for pointers to Shift-JIS text records. Hash each + distinct record (FNV-1a64 over its raw bytes), record `{hash, bytes, addr, + target-PC, call-site}`, persist immediately. This *enumerates every string to + translate* and *discovers which PCs draw text* — no arm-then-capture (per the + project's ring-buffer rule). +2. **Translation table (per-title, human-editable, multilingual).** + `TsumuRecomp/translations/tsumu.toml` — keyed by source-byte hash (+ optional + `src_addr`), one target string per language, with encoding/length metadata. + Hot-reloaded on mtime change. +3. **Apply hook (always-on).** A dispatch hook (sibling to the existing + `g_psx_bios_hle_hook`) that, when the target is a registered text-draw PC and + an arg points at a translated source record, writes the target-language bytes + into transient guest scratch (below `$sp`), repoints the arg register, and + lets the **original** draw routine render it. No regen, no engine change, + unbounded replacement length, dynamic format strings preserved. + +The only title-specific pieces are (a) the translation table and (b) the font +situation (§4). Everything else — capture ring, apply hook, config, coverage +tooling — is shared framework code. + +--- + +## 1. Prior art: the n64recomp PMS-J mechanism + +Files: `PocketMonstersStadiumRecomp/src/main/diagnostics.cpp` (the engine), +`include/trace.h` (the hook wiring), `tools/pms_build_translations.py` (authoring), +`translations.json` (the table). Verified by reading the source. + +### 1.1 Capture — always-on string inventory + text-PC census + +The recompiler emits a bare `TRACE_ENTRY()` at the top of **every** recompiled +function when `game.toml` has `trace_mode = true`. `trace.h` expands it to call, +with `rdram`/`ctx` in scope: + +```c +pkmnstadium_textdraw_probe(__func__, ra, a0, a1, a2, a3); // census + inventory +pkmnstadium_text_xlate(__func__, rdram, ctx); // apply +``` + +`pkmnstadium_textdraw_probe` (diagnostics.cpp:1235) is the **always-on capture** +(explicitly *no* env gate — "we never 'arm' capture and hope to catch the +window, we record every unique on-screen string as it is first drawn and persist +it immediately"): + +- **`read_text_arg()`** (1152) validates that an arg register points at a + NUL-terminated EUC-JP/ASCII run ≥2 bytes: every byte must be newline, + printable ASCII, or a EUC-JP lead/kana byte (`0x8E/0x8F/0xA1–0xFE`). The + RDRAM-range check is the real discriminator — coords/ints/non-pointers are + rejected before the byte scan. +- **Class-1 convention gate:** most text draws use `x=a0, y=a1, str=a2`; the + `a0<0x1000 && a1<0x1000` "looks like screen coords" test rejects binary + headers (`Yay0`/`PRESJPEG`) that decode as valid EUC-JP by chance. +- **`_Printf` chokepoint** (`kPrintfPC`): the universal vararg formatter is + special-cased — its format string (a2) is inventoried regardless of the coord + gate, so battle/overlay text built via `_Printf` is captured too. +- **`strinv_upsert()`** records every *distinct* string (content-hash keyed) into + `stringdump.log`, flushed on every newly-seen string so a crash never loses + captures. +- **`textpc_add()`** maintains a lock-free open-addressing set of PCs proven to + draw text (`~70` distinct), so the apply hook only pays the arg-scan cost on + real text PCs. + +There is also a capture-only **discovery probe** (`xlate_discovery`, gated by +`PMS_TEXTPROBE`) that finds *which register/stack slot* a known-untranslated +watch-string arrives in — used to learn non-standard conventions (battle labels, +Pokédex descriptions in `r7`/`r4`, the `%sポケモン` category classifier in `r5`). + +### 1.2 Translation table format + +`translations.json` next to the exe, hot-reloaded on mtime (1550): + +```json +{ "src_hex": "a5d0a5c8a5eb", "src_jp": "バトル", "target": "Battle", "orig_w": 3 } +``` + +- **`src_hex`** — the raw source bytes (EUC-JP) in hex. The load path converts + hex→bytes and keys the in-memory map by **`FNV-1a64` of those bytes** (robust + to RDRAM buffer reuse — the key is content, not address). +- **`target`** — the replacement (ASCII; may contain `%s`/`%d` specifiers to + preserve dynamic args, and `\n` for line breaks). +- **`orig_w`** — Japanese footprint in glyph units (documentation + default fit + budget). Optional **`fit_w`** overrides the width budget (use more/less space + than the original when there is room). `src_jp` is documentation only. + +Authoring workflow (`pms_build_translations.py`): `stringdump.log` → decode +EUC-JP → `strings_decoded.tsv` (persistent master across sessions) → author +`key_en.tsv` (`\t`) → `build` joins them into `translations.json`, +**merge-preserving** hand-authored entries. A `todo` mode lists still-untranslated +strings sorted by draw count. + +### 1.3 Apply — on-the-fly substitution + +`pkmnstadium_text_xlate` (2100) runs at every function entry; cheap no-ops +(`g_xlate_active` reentrancy guard, lock-free `g_xlate_armed` reject, reload +every 1024 calls). On a text PC it takes one of three paths: + +- **Pointer-swap (`write_guest_str` + repoint `ctx->r6`).** Content-hash the + source at the fmt arg; on a KV hit, write the replacement (ASCII, NUL-term) + into **transient scratch on the calling thread's guest stack** (`sp-0x800` for + format templates, `sp-0xC00` for literal args — two disjoint slots so a paired + `_Printf` swap can't self-clobber) and repoint the arg register. The + **original function then runs unchanged** — `_Printf` formats it (dynamic + `%d`/`%s` stay intact) and renders glyph-by-glyph. No engine change, no regen, + unbounded replacement length (never writes back into game RAM). +- **Proportional self-render (`self_render_static`, 1887).** For plain strings at + coordinate drawers, it draws the English itself by calling the game's glyph + routine (`FUN_8001a3e4`) per character with **per-glyph proportional advances** + (calibrated by measuring real ink width of each glyph tile in VRAM, + `calibrate_slot`/`measure_glyph`) and **auto-fit** condensing so English never + spills past the Japanese footprint (`fit_w` budget or per-line JP width). Then + it suppresses the caller's own draw with an empty fmt. +- **Register-specific paths** (`xlate_desc`, `xlate_general`) for the + discovered non-`r6` conventions, including a subtle `_Printf` writeproc + byte-count re-sync when a shorter replacement would otherwise embed a NUL and + truncate a composed string. + +### 1.4 Multilingual & font + +- **Multilingual** is table-driven: the KV value is per-language. (PMS shipped + English only, but the schema and hook are language-agnostic — selecting a + language means loading a different column/file.) +- **Font/glyph:** PMS-J's font sheet **already contained Latin glyphs**, so + English renders directly through the existing glyphs (the whole "no glyph + injection needed" premise). `pkmnstadium_fontdump` (2227) dumps the font-state + struct + glyph sheets (`.i8`) to answer exactly this gating question; the + fallback contingency (documented in comments/tools) is to inject an ASCII + glyph set if a game lacks Latin glyphs. The renderer measures glyph tiles + directly from VRAM to get proportional widths. + +### 1.5 What carries to PSX vs. what changes + +| Aspect | N64 (PMS-J) | PSX (Tsumu) | +|---|---|---| +| Hook point | `TRACE_ENTRY()` emitted per function (needs `trace_mode=true` regen) | **`psx_dispatch` chokepoint** — one install site, no regen, works in Release | +| Encoding | EUC-JP (JIS X 0208 via `A1/A3/A4/A5` leads) | **Shift-JIS** (2-byte fullwidth + 1-byte halfwidth kana `A1–DF`) + custom control framing | +| String delimiter | NUL | **NUL (menu labels) AND `0xFFFF` (framed message records)** | +| Memory swizzle | big-endian `^3` byte swap | **little-endian, no swizzle** (simpler) | +| Arg access | `ctx->r4..r7` | `cpu->gpr[4..7]` | +| Glyph routine | `FUN_8001a3e4` (known) | text-draw PC pinned at runtime by the census (Phase 0) | +| Latin glyphs | present in font | fullwidth Latin present (proven); halfwidth ASCII TBD (§4) | + +--- + +## 2. Tsumu's concrete text reality + +Evidence gathered from the headerless boot EXE +(`TsumuRecomp/ghidra/SLPS_022.53_no_header.bin`, 512 KB, loads at `0x80010000`, +entry `0x800691F0`) via direct byte analysis + Ghidra, plus `DISC.md`. + +### 2.1 Encoding — Shift-JIS (confirmed) + +The text is **Shift-JIS**, not ASCII and not EUC-JP (EUC was the N64 game). +Proven by decoding real records: + +| Address | Decoded | Notes | +|---|---|---| +| `0x80010C5C` | `ＴＳＵＭＵ　ｌｉｇｈｔ　プレイデータ` | fullwidth Latin + fullwidth space + katakana ("TSUMU light / Play Data") | +| `0x80010C94` | `ＴＳＵＭＵ　マップデータ` | "TSUMU / Map Data" | +| `0x80010190` | `？？？？？？？？` | fullwidth question marks (placeholder score row) | +| `0x80070AA8` | `それじゃ、がんばってね！`… | tutorial dialogue (hiragana) | + +The character mix is: + +- **2-byte fullwidth** (JIS X 0208): kanji, hiragana, katakana, **fullwidth + Latin `0x8260–0x8279` (Ａ–Ｚ), `0x8281–0x829A` (ａ–ｚ), fullwidth digits + `0x824F–0x8258`**, symbols. Lead byte `0x81–0x9F` / `0xE0–0xFC`, trail + `0x40–0x7E` / `0x80–0xFC`. +- **1-byte halfwidth katakana** `0xA1–0xDF` (e.g. `0xBB`=ｻ) — mixed inline with + 2-byte chars. This means the font has a **halfwidth grid**. +- **Custom control framing** interleaved with text (raw bytes, not SJIS): + - `0xFF 0xFF` — **message terminator** (record end). + - `0x82 0xFE 0xFF` / `0x81 …` sequences — **line break / formatting** markers. + - Menu labels (e.g. `0x80010C5C`) are simpler: plain 2-byte SJIS, **NUL**-terminated. + +**Consequence for the framework:** the PSX text validator must be Shift-JIS +aware (2-byte lead+trail, 1-byte halfwidth kana `A1–DF`, ASCII `20–7E`), must +**pass through control bytes**, and must treat a "string" as a record delimited +by **either** NUL (menu labels) **or** `0xFFFF` (framed messages). The hash key +is the raw record bytes up to (not including) the terminator. + +### 2.2 String storage + +- **In-EXE Shift-JIS text block:** a large body of tutorial/hint/system messages + lives in the EXE at roughly **`0x80070900`–`0x80072000`** (offsets + `0x60900`–`0x62000`), reached through a **pointer/index table**. +- **Text pointer table** at **`0x80071474`** (preceded by header word + `0x0000FC10` at `0x80071470`): a contiguous array of 32-bit little-endian + pointers into the text records, e.g. + `0x80070918, 0x80070958, 0x800709A4, 0x800709F0, … 0x80070AA8, 0x80070AC4, …`. + This is the message-ID → record-address map; the text system indexes it by ID. +- **Menu/UI labels** live earlier in `.rodata` as NUL-terminated SJIS + (`0x80010C5C`, `0x80010C94`, the `？？？？` row at `0x80010190`, memcard save + titles `BISLPS-02253 TSMLGT` / `TSMMP` for "TSUMU light" / "TSUMU MaP"). +- **Disc-streamed text/data:** `DISC.md` lists **`MOJI.BIN`** (文字 = "glyphs/ + characters" → the font sheet) and **`MONDAI.BIN`** (問題 = "problems/puzzles" → + puzzle text/data), plus `GRAPH1-4.BIN`, `SOUND.BIN`, FMV (`DOUGA.BIN`/ + `END.STR`), `TSUMU.XA`. Disc image + files are present locally under + `TsumuRecomp/tsumu/`. So some text is **static (patchable in the EXE)** and + some is **loaded/composed from disc at runtime** (must be caught at draw time). + +Static vs. dynamic: the in-EXE block and menu labels are static and could in +principle be patched at load, **but the framework deliberately intercepts at +draw time** (like PMS) so it uniformly covers disc-loaded and dynamically- +composed strings too, without touching game RAM. + +**Recompiler note (not blocking):** the recompiler's flow walker currently +mis-decodes the text/pointer-table region as code (it emitted +`func_80070300` over `0x80070300…`, decoding the pointer words as load-byte +instructions — classic data-walk over-reach). This does not affect the runtime +translation hook (which works on live memory), but a data/code separation pass +would clean the generated C. Track separately. + +### 2.3 Text render path + +`psx_dispatch` funnels every call, so the exact draw routine is **pinned at +runtime by the capture census** (Phase 0), exactly as PMS discovered its +`~70` text PCs. Static anchors already found to seed/confirm it: + +- The text-lookup routine loads base **`0x80071474`** and indexes it by message + ID to fetch a record pointer; that pointer flows (as `a0`/`a1`/`a2`) into a + glyph-draw routine that walks the SJIS bytes, maps each char to a glyph cell in + the font texture (uploaded from `MOJI.BIN` into VRAM), and issues textured + sprite primitives via the GPU. Menu labels reach the same drawer directly. +- Phase 0 confirms the exact draw PC(s), the arg register carrying the string + pointer (PSX convention is commonly `a0`=str or `a0`=x,`a1`=y,`a2`=str), and + the glyph-draw leaf (the PSX analogue of `FUN_8001a3e4`) if a self-render path + is wanted for fit control. + +(Full static disassembly of the drawer was not completed here because Ghidra +auto-analysis produced 0 functions on the raw dump and the region is +data-poisoned; the runtime census is the reliable, spec-sanctioned way to pin it +and is a Phase-0 deliverable.) + +### 2.4 Font / glyph availability + +- The game **renders fullwidth Latin and fullwidth digits today** (`ＴＳＵＭＵ + ｌｉｇｈｔ`, score digits), so its font sheet **already contains a fullwidth Latin + A–Z / a–z / 0–9 block** (SJIS `0x8260–0x8279`, `0x8281–0x829A`, + `0x824F–0x8258`). **English can therefore render with zero glyph injection** by + emitting fullwidth-Latin SJIS codes — the same "existing glyphs" premise that + made PMS work. +- The game also uses **halfwidth katakana** (`0xA1–0xDF`), so a **halfwidth grid + exists**. Whether a **halfwidth ASCII block (`0x20–0x7E`)** is also present — + which would let English render *compactly* (half width) instead of fullwidth — + is the **open question**, resolved in Phase 0 by dumping `MOJI.BIN` / the VRAM + font page. PSX SJIS fonts frequently include halfwidth ASCII, so this is + likely but must be confirmed. +- **Contingency** (if no compact Latin): inject an ASCII glyph set into the font + page — append a Latin row to the `MOJI.BIN` glyph upload or upload a + supplementary sheet into free VRAM and map ASCII codes to it (mirrors PMS's + documented injection fallback). + +### 2.5 Length / layout constraints + +- **Menu labels** sit in fixed-width boxes; **framed messages** are laid out with + the game's own `0xFFFF`/line-break control codes into a text window. +- English is usually longer than Japanese, and **fullwidth Latin doubles the + width**, so overflow is the primary layout risk. Mitigations (§3.4): prefer + halfwidth ASCII when available; re-insert the game's own line-break control + codes to reflow multi-line messages; optionally a self-render fit path + (PMS-style advance condensing) for coordinate drawers; and per-entry + `max_w`/`fit` metadata in the table to cap width. + +--- + +## 3. Framework design (PSX) + +Shared module, proposed location: `runtime/src/text_xlate.c` + +`runtime/include/text_xlate.h` (peers of `fntrace.*` and `bios_hle.*`). + +### 3.1 The hook point — `psx_dispatch` + +`psx_dispatch` / `psx_dispatch_impl` (declared `cpu_state.h:164`) is the single +chokepoint every guest call routes through. Two precedents already hang off it: + +- **`fntrace_record(cpu, target)`** (`fntrace.c:58`) — an **always-on ring** of + every dispatch (`{frame, target, ra, a0..a3, s3, sp}`), explicitly designed as + "record from boot, query the window after the fact" (the project ring-buffer + rule). This is the capture analogue. +- **`g_psx_bios_hle_hook`** (`bios_hle.h:54`) — a hook slot consulted at the top + of `psx_dispatch_impl` with the pre-normalize physical address; it can + intercept/replace the target (HLE call + boot shell-skip). This is the apply + analogue. + +The translation module adds the same two shapes, so it needs **no per-function +codegen** and works in Release. + +### 3.2 Capture subsystem (always-on ring) + +Add `text_capture_record(cpu, target)` called from `psx_dispatch` right beside +`fntrace_record` (or fold into it): + +1. For each candidate arg (`gpr[4..7]`, plus a few stack slots `sp+0x10..`), + run **`sjis_read_record(addr, out, &len, &term)`**: + - Reject if `addr` outside main RAM (`0x80000000–0x80200000` and mirrors). + - Walk bytes until NUL **or** `0xFFFF` terminator (cap `kSrcMax=512`), classi- + fying each: ASCII `0x20–0x7E`, halfwidth kana `0xA1–0xDF`, SJIS 2-byte + lead+trail, or known control byte. Reject on any non-textish byte (filters + structs/pointers). Require ≥1 real text char. +2. Hash the record bytes: **`h = fnv1a64(bytes, len)`**. +3. Upsert into an **always-on inventory ring** `{h, bytes[≤64 sample], full-len, + src_addr, target_pc, ra, first_frame, count}`; on first sight, append to a + persistent `tsumu_stringdump.log` (flush immediately — crash-safe, per the + ring-buffer rule). +4. `textpc_add(target)` into a lock-free set so the apply hook only scans args on + proven text PCs. + +This runs from boot for every player (no arm gate). "Enumerate every string" = +drive the game (§6) and read the ever-growing inventory; "find untranslated" = +query the ring for records with no table entry. + +**Coord/`_Printf` disambiguation:** reuse PMS's precision filters adapted to +PSX — a coordinate gate (`a0,a1 < 0x1000`) for the class-1 drawer, plus explicit +handling for any universal formatter PC once the census reveals it. + +### 3.3 Translation table (multilingual, human-editable) + +**Location:** `TsumuRecomp/translations/tsumu.toml` (single multilingual file; +TOML fits the project's config conventions — `game_options.toml`, +`SCPH1001.toml`). Loaded by the shared module from a path the runtime injects +(peer of the overlay-cache dir injection). Hot-reloaded on mtime change. + +**Schema:** + +```toml +# tsumu.toml — Tsumu Light string translations (multilingual) +schema = 1 +default_lang = "en" +langs = ["en", "es", "de", "fr"] # first-class multilingual; add columns freely + +[[entry]] +src_hex = "82bb82ea82b6..." # raw Shift-JIS record bytes (hex), the hash key +src_jp = "それじゃ、がんばってね！" # documentation (decoded), not used for matching +src_addr = 0x80070AA8 # optional disambiguator when two records collide on hash +term = "ffff" # terminator that delimited this record: "nul" | "ffff" +orig_w = 12 # JP footprint in glyph cells (fit budget / docs) +max_w = 12 # optional hard width cap (glyph cells) for overflow control +note = "tutorial: end-of-lesson encouragement" +en = "Well then, do your best!" +es = "¡Pues venga, esfuérzate!" +de = "Also dann, gib dein Bestes!" + +[[entry]] +src_hex = "8273827282758..." # ＴＳＵＭＵ　ｌｉｇｈｔ　プレイデータ +term = "nul" +en = "TSUMU light Play Data" +``` + +- **Key = `fnv1a64(src bytes)`** (content hash — robust to buffer reuse), with + `src_addr` as a tiebreak if two distinct records ever collide. +- Per-language target strings; **untranslated / missing-language falls back to + the source** (render the original Japanese) so the game is never broken by a + gap. +- Target strings are authored in **UTF-8**; the apply layer transcodes UTF-8 → + game glyph codes (§3.4). `\n` requests a line break (emitted as the game's own + line-break control code). `%d`/`%s`-style specifiers are preserved for dynamic + args if the census shows a formatter path. +- `orig_w`/`max_w` drive width fitting. + +**Authoring tools** (shared, `psxrecomp/tools/`, adapted from PMS): +`text_xlate_decode.py` (stringdump → decoded TSV master, Shift-JIS via Python +`cp932`), `text_xlate_build.py` (join `\tUTF-8` → `tsumu.toml`, +merge-preserving), `text_xlate_todo.py` (list untranslated by draw count → +coverage report). + +### 3.4 Apply subsystem (dispatch hook) + +Install `g_psx_text_xlate_hook` (sibling to `g_psx_bios_hle_hook`), consulted at +the top of `psx_dispatch_impl` **after** the BIOS-HLE hook, gated by a lock-free +`armed` flag (table non-empty) and the `textpc_contains(target)` set: + +1. Read the source record at the string-arg register (`sjis_read_record`). +2. `h = fnv1a64(...)`; look up `{lang}` target. Miss → **return without change** + (fallback to source). +3. **Transcode** the UTF-8 target → a byte sequence the game's drawer will + render: + - **Halfwidth-ASCII path** (preferred, if Phase 0 confirms halfwidth Latin + glyphs): emit ASCII `0x20–0x7E` directly — compact, closest to JP width. + - **Fullwidth-Latin path** (guaranteed): map each ASCII char to its fullwidth + SJIS code (`A→0x8260…`, `a→0x8281…`, `0→0x824F…`, space→`0x8140`). Always + works because those glyphs are proven present. Double width → rely on width + fitting. + - **Injected-glyph path** (contingency): map to injected ASCII glyph cells. + - Re-emit the record's framing: preserve/insert the game's `0xFFFF` terminator + and line-break control codes; convert `\n` to the game's line-break code. +4. **Write to guest scratch** below the caller's `$sp` (`gpr[29]`) using the + runtime store accessors (little-endian, no swizzle): pick a scratch slot with + stack headroom (e.g. `sp - 0x800`, `& ~7`), bounds-checked against RAM; NUL/ + `0xFFFF`-terminate. +5. **Repoint the string-arg register** at the scratch address. Return "not + handled" so the **original draw routine runs and renders the replacement** + (the whole no-regen premise). A reentrancy `thread_local` guard prevents the + hook re-triggering on any nested draw calls. + +**Length / layout handling:** +- Prefer halfwidth ASCII (compact). Use table `max_w` to cap width. +- For framed multi-line messages, reflow by inserting the game's own line-break + control codes at word boundaries to fit the text window. +- **Optional self-render fit path** (Phase 1+, mirrors PMS `self_render_static`): + once the glyph-draw leaf PC is known, draw the target ourselves per-glyph with + condensed advances and suppress the caller's draw — gives pixel-accurate + fit-to-JP-footprint. Not required for first light; the pointer-swap path is. + +### 3.5 Language selection + +- **`game.toml` / `settings.toml`:** add `[localization] language = "en"` (peer + of the existing renderer/aspect/FMV/turbo keys; `settings.toml` overrides + `game.toml`, CLI overrides both — per the project's settings precedence). + `"jp"`/`"off"` = passthrough (no translation). +- **RmlUi launcher:** a "Language" dropdown populated from the table's `langs` + list; writes the choice to `game_options.toml`. Runtime switch = reselect → + the module reloads the active language column (no restart needed; hot-reload + already covers table edits). +- Default = `default_lang` from the table (English for Tsumu). + +### 3.6 Coverage workflow + +The loop to reach "all strings target-language" (§0 goal): + +1. **Capture pass.** Fresh run; drive every screen/menu/tutorial/dialogue/ending + (self-driven via the debug port + input injection, or a scripted playthrough). + The always-on ring logs every distinct record to `tsumu_stringdump.log`. +2. **Decode + author.** `text_xlate_decode.py` → master TSV (Japanese visible). + Fill translations; `text_xlate_build.py` → `tsumu.toml`. +3. **Apply + verify.** Re-run with the table; the hook substitutes. **Verify + visually** (screenshot both windows — never infer from a counter), per the + project's verification rules. +4. **Re-capture to find gaps.** `text_xlate_todo.py` diffs the live inventory + against the table → coverage report (`N/total translated`, untranslated + sorted by draw count). Repeat until the untranslated set is empty across a + full playthrough. +5. **Coverage report** artifact: total distinct records seen, translated %, + per-screen breakdown, list of dynamic/composed strings needing special + handling. "We've found them all" = a full playthrough adds zero new records to + the inventory. + +### 3.7 Framework generality (what's shared vs. title-specific) + +| Shared (framework) | Title-specific | +|---|---| +| Capture ring + `sjis_read_record` + inventory persistence | The translation table (`.toml`) | +| `fnv1a64` keying, text-PC census set | The font situation (Latin present? inject?) | +| Apply dispatch hook + guest-scratch write + arg repoint | Encoding profile (SJIS here; another game might be custom-tiled) | +| UTF-8 → glyph transcoders (ASCII, fullwidth-Latin, injected) | The string-arg register convention (pinned by census) | +| Config keys + launcher wiring + hot-reload | Control-framing profile (terminators, line-break codes) | +| Authoring tools (decode/build/todo/coverage) | — | + +Encoding is abstracted behind an **`EncodingProfile`** (validator + terminator +rules + transcoders) selected per title in `game.toml` +(`[localization] encoding = "shift_jis"`), so a game using a custom +tile-index encoding plugs in its own profile without touching the capture/apply +core. + +--- + +## 4. Font/glyph plan — concrete Tsumu situation + +1. **Guaranteed today:** fullwidth Latin + fullwidth digits are in the font + (proven by `ＴＳＵＭＵ ｌｉｇｈｔ`). Ship English on the **fullwidth-Latin path** + for first light — zero asset work, guaranteed to render. +2. **Phase-0 check:** dump `MOJI.BIN` / the live VRAM font page and inspect for a + **halfwidth ASCII block** (`0x20–0x7E`). The presence of halfwidth katakana + (`0xA1–0xDF`) means a halfwidth grid exists; if ASCII is in it, switch to the + compact **halfwidth-ASCII path** (better width fit). +3. **If neither compact option:** inject an ASCII glyph set — extend the + `MOJI.BIN` upload or blit a supplementary Latin sheet into free VRAM and map + ASCII to those cells. Add a small `font_probe`/`font_dump` tool (PSX analogue + of `pkmnstadium_fontdump`) that reports the font page layout and which code + ranges have ink. + +--- + +## 5. Risks / open questions + +1. **Halfwidth ASCII glyphs present?** Determines compact vs. fullwidth (width + budget). Phase-0 blocker for layout quality (not for first light). +2. **Exact text-draw PC(s) + string-arg register.** Pinned by the Phase-0 + census; menu labels vs. framed messages may use different drawers/registers + (PMS had `~70` PCs and multiple conventions — expect the same spread). +3. **Control-code framing semantics.** `0xFFFF` (terminator) and the `0x81…`/ + `0x82 0xFE 0xFF` sequences (line break/format) must be decoded so we preserve + them across substitution and can reflow. Reverse the exact code set in + Phase 0. +4. **Dynamically composed strings** (scores, counts, names concatenated at + runtime, e.g. the `？？？？` placeholder row): the source bytes vary, so a + pure content-hash misses them. Handle via formatter-template entries + (`"%d"`-style) once the census reveals a composing routine, or per-fragment + translation — same class of problem PMS solved for `%sポケモン`. +5. **Length overflow / fixed boxes.** English (esp. fullwidth) overruns menu + boxes; needs `max_w` capping, reflow via line-break codes, and possibly the + self-render fit path. +6. **Disc-loaded text (`MONDAI.BIN`/`MOJI.BIN`).** Confirm puzzle/problem text is + caught at draw time (it should be — the capture is draw-time, not load-time); + if a puzzle *is* Japanese-character gameplay, translation may be + gameplay-affecting and should be scoped carefully (menus/UI/tutorials first). +7. **DTE / dictionary compression.** Not observed (text is plain SJIS in the + EXE), but confirm no compressed text table exists on disc before declaring + full coverage. +8. **Hot path cost.** The apply hook runs at every dispatch; keep the fast reject + (lock-free armed flag + text-PC set membership) ahead of any RAM scan, exactly + as PMS does, so non-text dispatch pays ~nothing. + +--- + +## 6. Phased implementation plan + +- **Phase 0 — Discovery (runtime census).** Add the capture ring + + `sjis_read_record` + inventory persistence + text-PC census. Drive Tsumu; dump + `tsumu_stringdump.log`. Pin the text-draw PC(s), the string-arg register, the + control-code set, and the font situation (`MOJI.BIN` dump). Deliver the encoding + profile. *(No visible change yet — pure observability.)* +- **Phase 1 — Apply (pointer-swap).** Install `g_psx_text_xlate_hook`; implement + the fullwidth-Latin transcoder + guest-scratch write + arg repoint. Author a + starter `tsumu.toml` (main menu). Verify menus render English visually. +- **Phase 2 — Encoding/layout polish.** Halfwidth-ASCII path if available; + reflow via line-break codes; `max_w` fitting; framed-message coverage. +- **Phase 3 — Multilingual + launcher.** Language dropdown, `game.toml`/ + `settings.toml` key, runtime switch, second language column to prove the + pipeline. +- **Phase 4 — Coverage to 100%.** Iterate capture→author→verify with the + coverage report until a full playthrough surfaces no untranslated records. + Handle dynamic/composed strings and any injected-glyph work. +- **Phase 5 — Generalize.** Lift the module to a clean framework API + (`EncodingProfile`, config-driven table path), document a "new title" recipe, + and (optionally) fix the data/code separation for the mis-decoded text region. + +--- + +## Appendix Z — Implementation status & Phase-0 empirical findings (2026-07-05) + +The mechanism is IMPLEMENTED and verified working headless on Tsumu; the +translation table is an ongoing coverage effort. Hard-won findings from driving +the real game (these correct/extend §2, which was written pre-implementation): + +- **Hook point (as specced):** `text_xlate_on_dispatch()` is called from + `fntrace_record()` at the `psx_dispatch` chokepoint. No generated-code edits, + no BIOS regen, works in the Release build. Capture + apply confirmed live. +- **Two encodings, two endiannesses.** The EXE menu labels (e.g. `0x80010C5C` + `ＴＳＵＭＵ ｌｉｇｈｔ`) are **big-endian** Shift-JIS, NUL-terminated. The + message-table text (tutorial/hint/dialogue, drawn via a string pointer) is + **little-endian 16-bit** Shift-JIS (bytes `bf 82` = ち), framed with control + codes and **0xFFFF**-terminated. The transcoder must emit the matching + endianness (`PSX_XLATE_LE`); this should become an `EncodingProfile` flag. +- **Control framing (confirmed):** `FE FF` = line break; `FF FF` = end of + message; `30 FC` / `10 FC` = page/prompt breaks; `＄`/`％` (`8x` codes) = + on-screen button icons; fullwidth space `81 40` pads fixed-width fields. +- **Two draw paths — only one is interceptable.** Message/dialogue text is drawn + by passing a **string pointer** to a formatter → captured & translatable. + The **title prompt, HUD ("ステージ"/"バッテリー"), and menu labels are drawn + per-glyph as sprites** (glyph index, no string pointer) → **NOT reachable by + arg-scanning**. Full HUD/menu localization needs a second hook that intercepts + the per-glyph sprite draw and remaps the glyph index — a larger, separate + effort (open item). +- **Struct-embedded strings crash if naively replaced.** Some NUL records are + fixed-width struct fields with binary params packed after the text (e.g. the + `ちゅーとりあるの5` "Tutorial N" label carries trailing coordinate bytes). + Replacing the whole record corrupts the struct and derails the game (observed + `PC=0`). **Apply is therefore gated to standalone 0xFFFF-framed messages by + default** (`PSX_XLATE_ALLOW_NUL=1` to override for vetted labels). This is the + concrete form of the §5.4 "dynamically-composed / struct" risk. +- **Capture quality gate.** The relaxed reader admits vertex/coordinate binary + that passes by chance (~20k records). A gate requiring ≥2 hiragana/katakana/ + fullwidth chars (SJIS lead `0x82`/`0x83`) keeps the always-on inventory a clean + enumeration (~52 real records). Authoring reads it via TCP `xlate dump/todo`. +- **Verification.** Framed tutorial hints apply on real draws (hits climb, game + stays alive, English written little-endian). These particular hints are + composed during a load transition and displayed only in **interactive tutorial + mode**; a full on-screen visual pass needs menu navigation to that mode (and, + for the HUD/menu, the per-glyph hook above). Debug-server `screenshot` is + flaky after extended headless runs — a tooling bug to fix (CLAUDE.md rule 15). + +**Coverage status (138 entries shipped):** the full in-EXE message table was +statically enumerated — pointer table at **0x80071474** plus the block +**0x80070900–0x80073700** — decoded (little-endian Shift-JIS, control framing +honored) and translated to English: mode-select descriptions, the entire +interactive tutorial, ~90 per-stage hints, all memory-card save/load/format +messages, the Yes/No confirmation dialogs, and the contributor credits. Every +entry's `src_hex` is byte-exact to what the game draws — cross-checked by +hashing against the always-on runtime capture ring (confirmed matches), so the +apply hook fires on the real draws (verified: hits climb, no crash, English +written little-endian). + +**Reader limitation (small gap):** the byte-based reader rejects a record whose +first byte isn't "textish" — which for LE storage means a first glyph whose low +byte is 0xA0 (i.e. records starting with **あ**, e.g. the name-entry prompt +"あなたの名前を教えてね"). Those few stay Japanese until the reader is made +LE-word-aware (would rehash the table). + +**Still out of scope / open:** (1) per-glyph sprite-draw text — HUD +(ステージ/バッテリー), title prompt, menu labels, the "CLEAR!!" banner, the +name-entry glyph grid — needs a separate glyph-index-remap hook (deferred +feature, by design); (2) `MONDAI.BIN` disc puzzle data (if it carries +string-pointer text, the always-on capture will surface it during play); +(3) fold endianness + the LE-first-byte fix into `EncodingProfile`; +(4) `screenshot` writes fail to some temp paths — use a project-dir path. + +## Appendix Y — Per-glyph text: investigation & why it resists (2026-07-05) + +The HUD / title / menu labels are drawn **per-glyph as sprites** (glyph index → +font tile → sprite), NOT via the string-pointer formatter, so the message-path +hook cannot reach them. Investigation results (concrete, for the eventual hook): + +**Font glyph-index map** (table at EXE `0x80074ACA`, one LE-SJIS code per index): +- **Latin `Ａ–Ｚ` = indices 73–98**, **digits `０–９` = 105–114** (primary bank); + a second Latin/digit bank at **213–252**. Hiragana fills 0–64 and 121–207. + So for any English char the glyph index is known. **The font already contains + fullwidth Latin/digit glyphs** (confirmed) — no glyph injection needed. +- Note: this table has **no katakana** (it is the name-entry character set); + the katakana glyphs the HUD uses live at other indices in the VRAM font + (`MOJI.BIN`) — dumping `MOJI.BIN` / the VRAM font page is still needed to map + katakana → index if a katakana label is to be recognized. + +**Label storage** — per-glyph labels are static **little-endian Shift-JIS** in +the EXE, in **fixed-width, space-padded slots** (patchable): +- Level names ~`0x80070755–0x800708E4`: `はこにわ`(0x8007075E) `たわー`(0x800707AC) + `くらげ`(0x800707C6) `せんしゃ`(0x800707E0) `せぱれーと`(0x800707FA) + `おてほん`(0x80070814) `ろじうら`(0x800708E4) … each padded with fullwidth + spaces (`81 40`) to a fixed slot. +- Tutorial labels ~`0x80070300+` (`ちゅーとりある` ×6); mode name `のーまるげーむ`; + `はい`/`いいえ` at several sites. + +**Cleanest hook (designed, not shipped):** since fullwidth Latin glyphs exist, +a **load-time source patch** — after game-start, overwrite a label's katakana/ +hiragana SJIS in RAM with fullwidth-Latin English SJIS (`ステージ`→`ＳＴＡＧＥ`), +padded to the slot with `81 40` — makes the **existing** per-glyph routine draw +English, with no new draw routine and no glyph injection. English must fit the +slot (the space padding gives the budget; e.g. `たわー`→`ＴＯＷＥＲ`, +`くらげ`→`ＪＥＬＬＹＦＩＳＨ`). For labels drawn instead via the string-pointer +path, they simply go in `tsumu.toml` like any message entry. + +**Why it is not shipped/verified here:** the bar is a screenshot of English on +these screens, and **the screens are unreachable in the headless build**: the +attract never auto-plays the gameplay demo (150 s observed: title/Sony-logo +only), and reaching the mode/level-select or in-game HUD requires completing the +**name-entry glyph grid** (enter a char, navigate to the "End" cell, confirm) — +which is intractable via blind debug-input injection (input reaches the pad, +verified, but precise grid navigation to "End" did not land). Committing a +RAM-patch feature that overwrites game memory without a rendered-English +screenshot would be unverified coverage, so it is documented here instead. + +**Remaining resisters within per-glyph:** the HUD counter `ステージ：X–Y` (label +storage not found statically — composed at runtime / in an overlay; needs +gameplay capture to pin); the title prompt and `CLEAR!!` (likely baked sprite +graphics, not glyph-drawn → would need image replacement, out of the glyph +model). Next step for whoever has a reachable build: dump `MOJI.BIN`, confirm the +katakana index range, ship the load-time source patch for the padded label +slots, and screenshot the level-select / HUD. + +## Appendix A — Key source references + +**n64recomp prior art** (`F:\Projects\n64recomp\PocketMonstersStadiumRecomp\`): +- `src/main/diagnostics.cpp` — capture (`pkmnstadium_textdraw_probe` 1235, + `read_text_arg` 1152, `strinv_upsert`/`textpc_add`), apply + (`pkmnstadium_text_xlate` 2100, `self_render_static` 1887, `xlate_desc`/ + `xlate_general`), `load_translations` 1550, `pkmnstadium_fontdump` 2227. +- `include/trace.h` — `TRACE_ENTRY()` hook wiring. +- `tools/pms_build_translations.py` — decode/build/todo authoring. +- `translations.json` — table format. + +**psxrecomp hook points** (`F:\Projects\psxrecomp\_wt-tsumu\psxrecomp\`): +- `runtime/include/fntrace.h` + `runtime/src/fntrace.c:58` — always-on dispatch + ring (capture analogue). +- `runtime/include/bios_hle.h:54` (`g_psx_bios_hle_hook`) + `runtime/src/bios_hle.c:297` + — dispatch hook slot precedent (apply analogue). +- `runtime/include/cpu_state.h:164` — `psx_dispatch` / `psx_dispatch_call`. + +**Tsumu evidence** (`F:\Projects\psxrecomp\TsumuRecomp\`): +- `ghidra/SLPS_022.53_no_header.bin` — boot EXE (load `0x80010000`, entry + `0x800691F0`). +- Menu labels `0x80010C5C`, `0x80010C94`, `0x80010190`; tutorial text block + `~0x80070900–0x80072000`; **text pointer table `0x80071474`** (header + `0x0000FC10` @ `0x80071470`). +- `DISC.md` — `MOJI.BIN` (font), `MONDAI.BIN` (puzzle text), disc file list. +- `game.toml` — `load_address`, `entry_pc`, `debug_port = 4510`. + +## Appendix X — Per-glyph coverage pass: shipped, and the composed-HUD resisters (2026-07-05) + +Follow-up to Appendix Y. Concrete outcomes of the per-glyph work, with the +static evidence that decides what is source-patchable vs. what genuinely resists. + +### Shipped (RAM source-patch layer — `[[glyph_label]]`) +- **Stage/level/tutorial names** — the full uniform 26-byte-stride table at + **`0x80070300`–`0x800708FE`** (59 slots, one per puzzle). Confirmed clean: + every slot start is +26 from the last (`text` + fullwidth-space pad), so slot + bounds are certain. Patched in RAM once resident, verified byte-for-byte + against the source before writing exactly 26 bytes. (commit `f00631e` runtime, + `31ab3c2` table.) +- **Name-entry prompt** `あなたの名前を教えてね。` @ **`0x80071360`** — a static + 0xFFFF-framed record whose renderer reads the source address *directly* (a + pointer-repoint `[[entry]]` captures it with `xl=true` and `hits` increments, + but the pixels don't change). Source-patched instead: `[[glyph_label]]` width + 24 (12 cells) → fullwidth "YOUR NAME?", trailing `FF FF` @ `0x80071378` left + intact. Renders English on-screen. (commit `31ba92d`.) +- **Name-entry Latin input is NATIVE.** The kana grid already draws the font's + fullwidth Latin `Ａ–Ｚ` and digits `０–９` (char table `0x80074ACA`, indices + 73–98 / 213–252). A non-JP player can type an English name today — no change + needed; the only JP on that screen was the prompt above. + +### Key discriminator: static source address ⇒ source-patchable +The RAM source-patch works for **any** label the game draws from a fixed EXE +address, *regardless of draw path* (per-glyph sprite loop OR string formatter) — +both read the same bytes. It CANNOT touch text that is **composed at runtime** +(no static source to overwrite). + +### `CLEAR!!` banner — ALREADY ENGLISH (no work needed) +Reaching a gameplay stage (attract demo, HUD on-screen) and screenshotting the +stage-clear moment shows the banner **already renders "CLEAR!!" in stylized +Latin letters** — it is not Japanese. Consistent with the static evidence: `クリア` +(`4e 83 8a 83 41 83`) appears only message-embedded ("…をクリアしていく"…), there +is no standalone `クリア` / fullwidth `ＣＬＥＡＲ` / ASCII `CLEAR` in the EXE, and +the disc's only pre-rendered text graphic is the `TSUMU light` logo. The clear +banner is drawn from Latin glyphs the game already owns. **No translation +required** (the earlier "resister" assumption was wrong — corrected here). + +### In-game HUD — RESOLVED 2026-07-06, see Appendix Z (the analysis below was superseded) +The characterization below assumed the HUD labels were composed per-glyph at +stage load. They are not: the labels exist as **pre-rendered pixel strips +inside the glyph-sheet asset itself** (the 64hw-wide block uploaded to VRAM +832,256 — MOJI.BIN carries the font AND the baked label strips). The stage-load +compose just MoveImages the whole strip (`src=(834,352) 18×16` ステージすう：, +`src=(852,352) 18×16` のこりバッテリー：) into the HUD cache row at y=32. The +shipped fix is the `[[vram_patch]]` upload-time pixel patch (Appendix Z) — no +glyph remap hook was needed. Historical analysis kept for the record: + +### In-game HUD `ステージ：X–Y` / `バッテリー：N` — RESISTER (characterized, NOT shipped) +The HUD label is the one genuinely-unshipped item. Rendering model, pinned via +the always-on GP0 ring (`gpu_frame_dump`, records every primitive with its +submitting PC + DMA `src_addr`): +- The katakana `ステージ` (`58 83 65 83 5b 81 57 83`) / `バッテリー` + (`6f 83 62 83 65 83 8a 83 5b 81`) exist in a full 2 MB live RAM scan **only** at + the static message-text addresses (`0x800709xx`…), each followed by a particle + (を/が/の/は) — never as a standalone HUD label, and there is no `ステージ：` + template. So there is **no static source the RAM source-patch could rewrite.** +- The game composites heavily: it averages **~6 draw primitives/frame** + (`gp0_draw` ÷ frames) and rebuilds only the animated foreground each frame + (character + a few blocks, ~8 textured `0x2C` quads at screen y ≥ 31, issued + via the BIOS DMA library from an OT at RAM `~0x138xxx`). The HUD sits at screen + y ≈ 8 and is **NOT among the per-frame draws** — it is composed **once at + stage-load** into the display/background and thereafter just shown. +- The stage-load compose is therefore the only place to intercept it, and it + happens before the GP0 ring's steady-state window; catching it needs a **fresh + stage load while observing** (force real gameplay past name-entry, or clear a + stage so the next one composes), then decoding whether the label is blitted as + CPU→VRAM glyph bitmaps (`0xA0`) or drawn as font-tile quads, and remapping that + specific compose — a scoped per-glyph/blit intercept, not a source-patch. + +This remains the **per-glyph interception** effort flagged in Appendix Y. It was +NOT shipped: a clean, pixel-verified remap of the composited HUD wasn't achieved +in this pass, and a blanket VRAM font swap (the only "quick" alternative) would +corrupt katakana in menus/messages and can't handle the `ステージ`→`STAGE` +4-vs-5-glyph layout change. Per project Rule 5 (pixels or it's not done) and the +no-fragile-hacks rule, no partial was shipped. Next step for a reachable build: +force a fresh stage-load, dump the GP0 ring across the compose, and remap there. + +### Disc asset map (all files extracted from `Tsumu Light (Japan).bin`) +ISO9660 root: `DOUGA.BIN` (FMV), `END.STR` (ending FMV), `FB_TEX.BIN` (4bpp block +textures @VRAM 768,0), `GRAPH1/4.BIN` (16bpp `TSUMU light` title @320,0), +`GRAPH2/3.BIN` (8bpp character sprite sheets @320,256 / 576,256), `HECTLOGO.TIM` +(4bpp Hect logo), `TSU_KOK.TIM` (16bpp), **`MOJI.BIN`** (22528 B raw font, lba +7119 — the HUD/per-glyph font sheet), `MONDAI.BIN` (puzzle data), `SOUND.BIN`, +`TSUMU.XA`. Rendered PNGs confirm no CLEAR!! banner among the baked graphics. + +### HUD remap — the drawer is now pinned (mechanism cracked, for the next pass) +Live GP0-ring capture on the name-entry screen (which draws per-glyph text at the +screen top, same class as the HUD) pinned the per-glyph text drawer concretely: +- **Drawer PC `0x8005443C`** — issues one textured quad per glyph via **direct + GP0** (not DMA), building the primitives in game RAM at **`~0x80092xxx`**. Each + glyph is a `0x2C` textured quad: screen rect (X,Y, 16–32 px cell) + **U,V into + texpage `0x1E`** selecting the font tile, CLUT `0x7C10`. +- **The font page (texpage 0x1E) holds hiragana AND Latin** — proven because the + name-entry grid draws `Ａ–Ｚ`/`０–９` through this exact drawer. So the Latin + tile U,Vs the HUD remap needs are directly readable off the grid's letter quads + (correlate each grid cell's on-screen glyph ↔ its quad U,V). +- **Remaining blocker for a clean HUD remap:** catch the HUD label's own compose + (it draws its katakana quads via this drawer at stage-load) to get its source + prims + katakana U,Vs, then either rewrite those prims' U,V to the Latin tiles + or hook `0x8005443C` to remap the HUD-position glyph quads. Reaching a fresh + stage-load is gated on confirming a name in the grid (blind debug-input grid + navigation to the "decide" cell did not land) or a memcard LOAD path — a + reachable interactive build (the USER's) clears this immediately. With the + drawer + font-tile scheme now known, the remap itself is a scoped U,V rewrite. + +### HUD remap — reachability blockers (why it's not finishable headless) +Attempted the LOAD-GAME unblock (skip the name grid) to reach a dynamic HUD: +- **Both memcards are empty.** `saves/card1.mcd` + `card2.mcd` are formatted + (magic `MC`) but have **0 used blocks** and no `SLPS`/`TSM` save markers — so + **LOAD GAME has nothing to load** and cannot reach a stage. +- **NEW GAME** still routes through the name-entry glyph grid; blind debug-input + navigation to its "decide" cell did not land (confirmed again this pass). +- **The attract demo's HUD is not per-frame-drawn.** With the GP0 ring recording + from the game's first GP0 write (frame 554), a full 554→3754 capture shows the + HUD (screen y≈8) is **never** drawn as glyph quads or any top-region primitive; + steady gameplay frames contain only the animated 3D scene (~8 textured `0x2C` + quads at screen y≥31, via BIOS DMA `0xBFC38B1C`). The HUD is composed once at + stage-load and thereafter only displayed. In the demo its values are static + (`0-5` / `∞`), so there is **no dynamic HUD draw to intercept** and no + discriminator to capture. + +**Net:** the drawer + font-tile scheme are cracked (see above), but capturing the +HUD label's own glyph quads (kana U,Vs + a call-context/screen-region +discriminator) requires **real gameplay where the HUD is dynamically drawn** +(battery counting down / stage advancing) — reachable only via a real save +(create one first) or driving NEW GAME past the name grid on an interactive +build. No hook was shipped: without the captured discriminator, any remap would +risk rewriting non-HUD quads (blocks/menus) — a fragile hack (Rule 5, forbidden). +Precise remaining step for the USER's reachable build: enter a stage so the HUD +draws, dump the GP0 ring at drawer `0x8005443C`, read the HUD glyph quads' U,Vs + +screen row, then rewrite those to the font's Latin tiles for STAGE / BATTERY. + +## Appendix Z — VRAM-strip patch layer (`[[vram_patch]]`) — HUD labels SHIPPED (2026-07-06) + +### The missing model: pre-rendered strips, not composed glyphs +The HUD labels (and, it turns out, the entire menu system) are **pre-rendered +pixel strips baked into the glyph-sheet asset** — not glyph-code strings, not +per-glyph composes. Evidence chain: +- The 144-entry font order table at `0x80074a9c` contains **no katakana** (only + hiragana + fullwidth Latin/digits + a little punctuation), yet the HUD shows + katakana `ステージすう：`/`のこりバッテリー：` — so that text cannot come from + the glyph-code path at all. +- No SJIS code string for either label exists anywhere in the EXE or in a full + 2 MB RAM scan (searched as both hiragana and katakana, both byte orders). +- Rendering VRAM `(832,256) 64×256` as 4bpp shows the sheet rows 0–4 are the + font, and rows past the font are **baked strips**: the HUD labels at y=352, + `○ボタンで すすむ !!` at y=368, results-screen labels (`ステージすう`/ + `のこりバッテリー`/`ランク`/`ほじゅうバッテリー`/`ノーマル`/`スペシャル`) at + y≈448–511; block `(896,256)` holds the main-menu strips (`ちゅーとりある`, + `のーまる げーむ`, `ふりー ぷれい`, `すとろーく げーむ`, `でーた せーぶ`, + `にゅーげーむ`, `こんてぃにゅー`, `ハイスコア`) and `(896,384)` the high-score + table (`１位/２位/３位`, `なまえ`, `のこりバッテリー`, `グループ`); block + `(768,256)` the pause menu (`このままつづける`, `さいしょからはじめる`, + `あきらめる`, `そうさタイプのへんこう`, `目標残バッテリー：`) and misc + (`〜を選択`, `Tブロック…不正です`, `○ボタンでせーぶへ!!`); `(960,~440)` + `データ１〜４`. (`CRASH`/`CLEAR!!`/`EMPTY` strips are already English.) +- The sheet arrives as four `GP0 0xA0` LoadImage uploads of 64×256 at + `(768,256) (832,256) (896,256) (960,256)` (~boot frame 1500; ring-attributed + to the libgpu LoadImage path, guest buffer transient — a CD-read bounce + buffer, which is why the RAM scan finds nothing). +- At stage load the game MoveImages whole label strips into a HUD cache row: + `src=(834,352) 18×16` + `src=(852,352) 18×16` → `dst=(642..,32)`, digits + blitted separately from the big-digit row y=336 (ring `bld[]` chains: + `0x80041E78`/`0x80041EA4` return sites). Patching the SHEET therefore fixes + every downstream consumer. + +### Mechanism (general, runtime): patch rides the game's own upload +`text_xlate.cpp` gains a third layer, `[[vram_patch]]`: on completion of every +CPU→VRAM transfer (`gpu.c` GP0 0xA0 state machine, both completion paths call +`text_xlate_vram_upload(x,y,w,h)`), any configured rect fully contained in the +upload is verified **halfword-for-halfword** against `src_hex` (expected JP +pixels) and only on an exact match rewritten with `<lang>_hex` through +`gr_vram_read`/`gr_vram_write` — the renderer facade the upload itself uses, so +software / GL / supersampling mirrors all stay coherent. Re-applies on every +matching re-upload (source-path patch, NOT a one-shot VRAM poke — survives +scene reloads; a different asset at the same coords fails verify and is left +alone). Config lives with the other translation surfaces in +`translations/tsumu.toml`; language-gated like `[[glyph_label]]`. Debug: +`xlate stats` → `vram_patches`/`vram_applied`; `xlate vpatch` force-applies +against current VRAM (verify still gates) for config iteration without waiting +for a re-upload. + +### Shipped entries + authoring recipe +Shipped: `hud_stage` `(834,352,18,16)` → `STAGE`, `hud_battery` +`(852,352,18,16)` → `BATTERY` — rects exactly the game's own compose-blit strip +units; the replacement preserves the bullet-dot and colon pixels verbatim and +swaps only the JP text texels for text composed from the sheet's own fullwidth +Latin glyphs (row y=304, 8 texels/glyph, palette nibbles {1=stroke, 2=shadow} — +proven HUD-visible because the y=320 row, same palette, is blitted onto the +HUD). Authoring loop (scratchpad `composestrips2.py`): peek the strip rect → +`src_hex`; overwrite the measured text-texel span with Latin glyph texels → +`en_hex`; preview-render both; append to toml; `xlate reload` + `vpatch` to +test live. Same-day follow-up shipped the rest of the inventory — 37 entries +total: title menu (TUTORIAL/NEW GAME/CONTINUE ×2 variants), pause menu +(CONTINUE/RESTART/GIVE UP/CHANGE CONTROLS + TARGET BATT), results +(STAGES/BATTERY/RANK/REFILL/NORMAL/SPECIAL), mode menu (TUTORIAL/NORMAL +GAME/FREE PLAY/STROKE GAME/DATA SAVE), DATA 1-4, STAGE SELECT, SELECT MOVIE +PIECE, BAD T BLOCK COUNT/LAYOUT, HIGH SCORE + FREE pills, TO CONTINUE / TO +SAVE prompts, high-score headers NAME/BATTERY. Per-strip palette families +matter: pause/mode boxes bg=1 with {c,d,e,f} text (Latin glyphs must be +remapped, e.g. {1→d,2→c}, or the stroke nibble collides with the box bg and +vanishes); gray prompt strips bg=5 keep the Latin {1,2} palette; the +high-score pill is light-on-dark (map {1→f,2→d}). Remaining: グループ box + +１/２/３位 badges, and the level-select レベル１/２/３ labels which are NOT in +this sheet (likely the 8bpp character sheets GRAPH2/3) — separate hunt. diff --git a/docs/TOMBA2_EXC_CONSULT.md b/docs/TOMBA2_EXC_CONSULT.md deleted file mode 100644 index 2dcf03d5..00000000 --- a/docs/TOMBA2_EXC_CONSULT.md +++ /dev/null @@ -1,91 +0,0 @@ -# Recomp consult — Tomba 2 exception-model fix (EPC de-overload) - -Context for the "Recomp" GPT. This is a **static MIPS→C recompiler** (Architecture A): -the PSX BIOS (`SCPH1001.BIN`) is recompiled to native C; the game's disc overlays / -install-at-runtime RAM code are executed by a small **dirty-RAM interpreter**. There is -**no general interpreter and no HLE BIOS**. The exception/interrupt path is shared by -three already-shipped titles (Tomba 1, Mega Man X6, Ape Escape) — do not regress them. - -## The bug (data-confirmed, not theory) - -Tomba 2 boots and runs 60fps to frame ~1997, then the top-level dispatch returns -`cpu->pc=0` ("execution completed" abnormal exit). Measured root cause: - -- Our async-interrupt delivery (`psx_check_interrupts`) sets **`COP0_EPC = 0x80000048`**, - a host longjmp **sentinel** (a NOP we write), NOT the real interrupted PC. It then - saves all guest GPRs to host locals, `setjmp`s, dispatches the recompiled BIOS - exception handler, and on `ReturnFromException` **`longjmp`s back** and **restores the - GPRs from the host locals**. So normal resume is a *host-stack return*, not an - architectural RFE-to-EPC. The sentinel exists so the BIOS handler's COP2-branch-delay - check reads a benign NOP. -- The recompiled BIOS handler saves EPC (= sentinel) into the **current TCB save area**. - Confirmed live: TCB@0x108 → save_area off128 (EPC) = `0x80000048`, while off124 (ra) = - `0x80050CC8` and off116 (sp) = `0x801FFFB8` are the correct interrupted context. -- Tomba 2 installs its **own** interrupt handler via **HookEntryInt (B0:0x19)**. That - handler (`0x80085D8C`, dirty-interpreted) walks a per-source handler table and ends by - calling **ReturnFromException (B0:0x17) itself** (full-takeover pattern). -- At frame ~1997 the game's handler calls `ReturnFromException` while **`in_exception=0`** - (measured: `freeze_check` shows in_exception=0 + post_exception_cooldown active; the - event ring's last event is a COOLDOWN gate, which is only set after in_exception - clears). RFE reads `saved_epc = sentinel`, sets `cpu->pc = 0x80000048`; the - sentinel→longjmp gate in `psx_unknown_dispatch` requires `in_exception`, so it FAILS; - `0x80000048` is a bare NOP → resolves to `pc=0` → top dispatch exits. - -**One line:** EPC is overloaded with a host sentinel; resume only works synchronously -inside our `psx_check_interrupts` window. The game drives `ReturnFromException` -*asynchronously* (outside that window), and the sentinel has no valid resume → exit. - -## The key asymmetry that enables a fix - -When the interrupt is delivered from the **dirty-RAM interpreter pump** -(`dirty_ram_dispatch`, every ~4096 insns), `cpu->pc` already holds the **precise next -guest PC** (the interpreter committed it at a clean poll boundary). Tomba 2's main code -is entirely dirty-interpreted, so a real interrupted guest PC **exists** at exception -entry — we are simply discarding it and writing the sentinel. - -When the interrupt is delivered at a **compiled-code** block boundary, the interrupted -code resumes by *host return*, and there is no single precise guest PC — this is the -case the sentinel/host-restore hack was built for, and it works for T1/MMX6/Ape. - -## Proposed fix (Option A — scoped EPC de-overload) — please pressure-test - -When `psx_check_interrupts` enters an exception **and** the interrupted context is -dirty-interpreted (a valid guest resume PC is live in `cpu->pc`): -1. Set `EPC = cpu->pc` (real guest PC) instead of the sentinel. The TCB then saves the - real PC. -2. `ReturnFromException` (sync or async) sets `cpu->pc = saved_epc = real PC`; the - dispatch loop / dirty-interp re-entry resumes there. No host-stack longjmp needed for - this path. -3. Keep the existing sentinel + host-GPR-restore + longjmp path **only** for - compiled-interrupted contexts. - -Open questions for you: -- For the dirty-interp path, do we still need the **host-GPR save/restore**? On real HW - the BIOS handler saves/restores the full context via the TCB; if our recompiled - handler does that faithfully, the host hack should be unnecessary on the real-EPC - path — but the existing comment says the handler's *normal* (non-RFE) exit corrupts - the interrupted GPRs. How do we cleanly separate "RFE exit" (resume via EPC) from - "normal/longjmp exit" (resume via host) without the GPRs being clobbered? -- Is there a hazard where the **same** exception is entered from a *nested* dirty pump - (we already had a dispatch-depth desync bug, fixed by save/restore of - `g_psx_dispatch_depth`)? Does real-EPC resume interact with that? -- Async RFE with `in_exception=0`: should we drop the `in_exception` gate on the - sentinel→longjmp entirely and instead **always** resume via `cpu->pc=EPC` when EPC is a - real PC, reserving longjmp strictly for the sentinel-and-in_exception case? -- Alternative (Option B): force the game's HookEntryInt handler to run *inside* our - exception window so the proven sync longjmp applies. We have not yet pinned WHY our - model runs it with in_exception=0 (the BIOS DeliverEvent chain-walk that calls it - appears to run post-cooldown). Is chasing the deferral safer than de-overloading EPC? - -## Files -- `runtime/src/interrupts.c` — `psx_check_interrupts` (sentinel @ ~L362, save/restore, - setjmp loop @ ~L443). -- `runtime/src/traps.c` — `psx_syscall` case 3 (ReturnFromException, ~L497), - `psx_unknown_dispatch` sentinel gate (~L628). -- `runtime/src/dirty_ram_interp.c` — pump (`psx_check_interrupts` @ L1377), sentinel - handling (L1388). - -## Validation plan (the gate) -Oracle = Beetle PSX (`psx-beetle.exe`, port 4380) on the same BIOS+disc. After the fix: -Tomba 2 must roll past the splash into the intro FMV; **and** Tomba 1 / MMX6 / Ape must -remain frame-for-frame healthy vs the oracle. diff --git a/docs/TOMBA2_NATIVE_FMV_FREEZE.md b/docs/TOMBA2_NATIVE_FMV_FREEZE.md deleted file mode 100644 index 5b811579..00000000 --- a/docs/TOMBA2_NATIVE_FMV_FREEZE.md +++ /dev/null @@ -1,411 +0,0 @@ -# Tomba 2 — native-overlay FMV freeze (end of Whoopee Camp): root-cause findings - -Status: **FIXED + VERIFIED** (2026-06-23, wt/tomba2 bd60381). Was a deterministic -overlay range-ownership / dispatch-legality bug (not a race). See "RESOLUTION" at -the very bottom for the implemented fix and verification; the sections below are the -investigation trail that led there. - ---- - -## RESOLUTION (implemented + verified) - -Fix shipped: the fail-closed native entry guard (the runtime/dispatch-legality layer -of the ChatGPT-validated plan; "ownership is not entry legality"). The generated CPS -entry-switch for OVERLAY functions no longer falls through to the function top on an -unknown interior PC — it restores the PC and calls psx_native_bad_entry, and the -overlay dispatch routes that PC to the sanctioned dirty-RAM interpreter (the live -bytes run correctly; NOT a stub/HLE). A true-prologue entry still runs from the top. -Overlay-scoped via config_.overlay_mode (static BIOS/boot-EXE keep legacy behavior; -their generated trampoline doesn't consume the signal and their ranges have no -multi-variant ambiguity). Wired through OverlayCallbacks (ABI v5→v6 to reject stale -caches), the DLL glue (compile_overlays.py), and the runtime (overlay_loader.c, -cpu_state.h). Files: recompiler/src/code_generator.cpp, runtime/src/overlay_loader.c, -runtime/include/{cpu_state.h,overlay_api.h}, tools/compile_overlays.py. - -VERIFIED (clean-warm native): boot passes the freeze — frame 1823 → 18001+ at -emu_cpu 16.5ms / 60fps, i_mask=0x0D (VBLANK live), reval_crc_miss=0, invalidations=0, -MDEC decoding; foreign-interior PCs (0x89788) fail closed to the interpreter -(dispatch_interp_fallback; cheap flag-guarded init-array). - -DONE — B/C completeness (the discovery/ownership layer; recompiler main_psx.cpp overlay -mode): in overlay discovery, after walking the seed roots, every DIRECT CALL (jal) -target that lands strictly INSIDE an already-discovered function is registered as an -ALIAS ENTRY of that host (the existing alias machinery — materialize_alias_groups), so -the host's entry-switch routes a dispatch of that PC to the correct absorbed interior -block (runs NATIVE), instead of relying on it having been captured/executed. This is -derived purely from static jal evidence, so it does not depend on the capture. Why this -over an explicit range-clip: the absorbed code IS the right code; it just needed a -routable entry. VERIFIED: regenerated overlay cache (codegen hash cg4_42de29a8) -func_800896E0 now emits `case 0x80089788u`; the cps_probe for 0x89788 drops from 1021 -hits (broken CPS find-by-range -> func_800896E0 default) to 0 (now a registered entry, -main-chain native); freeze still gone, 60fps sustained to frame 6261+. The fail-closed -default remains as the safety net for any genuinely-unrouted interior PC. -NOTE: the static two-phase FunctionDiscovery::discover already enforces hard_cap = next -known entry (in_bounds: addr < hard_cap), so only the overlay path needed this. - -STILL OPEN: NOT yet visually confirmed split-frame-free (user visual-verify). -Framework-wide change — regression-test Tomba 1 / MMX6 / Ape before any master merge. - ---- - -## Symptom - -With the warm overlay cache delivering full native execution (emu_cpu ~16.5ms / -60fps, dirty-interp idle, `dispatch_interp_fallback=0`, `reval_crc_miss=0`, -`invalidations=0`), the boot **hard-freezes at guest frame ~1823** at the end of -the Whoopee Camp FMV. `overlay_native_off` (route overlays through the sanctioned -dirty-RAM interpreter instead of native DLLs) makes it sail past. Near-deterministic -freeze frame (1823 ×3, 1831 ×1). - -This is the same ~frame-1828 hang a prior session believed fixed by the -alias/interior-entry rollup (commit 47f3f15). It was not fixed — it was MASKED: -the messy accumulated captures audit-failed the relevant region's compile, so -those overlay functions fell back to dirty-interp (correct). A clean warm cache -compiles the region successfully → it runs native → the divergence re-surfaces. - -## Freeze state (always-on rings) - -- `irq_state`: `i_stat=0x01` (VBLANK pending), `i_mask=0x0C` (VBLANK bit0 MASKED). - `cop0_sr=0x40000401` (IEc=1, IM2=1). The pending VBLANK can never be taken. -- `dirty_ram_stats`: `insns_run=0` (interp idle). `overlay dispatch_native` climbing, - `dispatch_interp_fallback=0` → the guest is busy-spinning in NATIVE code. -- `dispatch_tail`: last 64 dispatch targets all `0x000000A0` (kernel A0 syscall - vector). `dispatch_stats static_hits=25M, miss=0`. The mainline re-enters the A0 - vector in a tight loop — an event-wait that never completes. -- `imask_trace`: under native, every i_mask write is BIOS ROM `0xBFC12210`; the - game's own critical-section handler `0x80085CB0` (base mask 0x0D, VBLANK on) - NEVER runs. Under dirty-interp it does, and the game progresses. - -## Proof it is CODEGEN, not a race - -The runtime already has a **sandboxed native↔interp differential** (built a prior -session — see `SHADOW_ENHANCEMENTS.md`): - -- `overlay_diff_on` / `overlay_diff_off` — each matched overlay function runs BOTH - ways from identical CPU+RAM state; the interp result is kept (the game stays - correct and PROGRESSES), and native computation divergences are logged. -- `overlay_shadow_dump` / `overlay_shadow_detail` — the divergence records. -- `overlay_native_ring` — always-on ring of native overlay calls + in-progress entry. -- `overlay_irq_ratelimit` / `overlay_irq_suppress_*` — independently test whether the - divergence is interrupt-delivery timing. - -Method: `overlay_diff_on` early (frame 793), ran PAST the freeze (frame 2344) on -interp-kept results, `overlay_shadow_dump`. - -Because the shadow runs both engines from IDENTICAL state with no interrupts and no -timing, a divergence there is purely computational. **Native diverges → deterministic -codegen bug, definitively not a race.** - -## First divergence - -`overlay_shadow_dump` seq 1 (earliest), and `overlay_shadow_detail`: - -``` -function 0x800896E0 (a stack/heap/context init; see disasm below) - t1 (r9): native 0x000000E4 interp 0x00000039 - t0 (r8): native 0xBFC01E24 interp 0x80000000 -``` - -- `0xE4 = 0x39 << 2` → native executed the **A0 dispatcher's `sll t1,t1,2`** - (kernel A0 dispatch stub at 0x5C4: `addiu t0,zero,0x200; sll t1,t1,2; add; lw t0,0(t0); jr t0`). -- `0xBFC01E24 = A0_table[0x39]` (= `*(0x200 + 0x39*4)` = InitHeap's ROM entry). -- interp `t0=0x80000000` comes from `0x80089714: lui t0,0x8000` in 0x800896E0 itself; - interp `t1=0x39` is the untouched entry value. - -Interpretation: **native erroneously routes through the A0:0x39 (InitHeap) dispatch -path that the interpreter does NOT take.** It re-inits every spin-loop iteration and -never satisfies the loop's exit condition → endless A0-vector dispatch → VBLANK stays -masked (the game disabled it via a B0 syscall, `imask 0x0D->0x0C`, and the path that -would re-enable it never runs) → deadlock. - -Spin set from `overlay_native_ring`: `0x80050B08 → 0x80089788 → 0x800896E0` -(in-progress), cycling `0x89770` / `0x89860`. The actual wrong branch is somewhere -in the 0x800896E0 callee tree (it decides to take the A0:0x39 path); shadow currently -gives function granularity, not the exact instruction. - -### 0x800896E0 disasm (head) - -``` -800896e0: lui v0,0x800c ; addiu v0,v0,-7976 ; v0 = 0x800CE0D8 -800896e8: lui v1,0x8010 ; addiu v1,v1,25128 ; v1 = 0x80106228 -800896f0: sw zero,0(v0) ; addiu v0,v0,4 ; sltu at,v0,v1 ; bne at,..0x896f0 ; memset 0x800CE0D8..0x80106228 -80089704: lui v0,0x800a ; lw v0,16264(v0) ; v0 = *(0x800A3F88) -80089710: addi v0,v0,-8 -80089714: lui t0,0x8000 ; or sp,v0,t0 ; sp = (v0-8) | 0x80000000 <-- interp t0 here -8008971c: ... builds a0 from 0x80106228, a1 = size, stores to 0x800Bbef8/bef4 -80089758: sw ra,-7976(at) ; addiu gp,..; addu fp,sp,zero -80089768: jal 0x80089860 ; addi a0,a0,4 -``` - -## Suspected codegen construct (to confirm) - -ChatGPT consult (Overlay Cache Architecture chat, with our debug principles -shared) ranked, for this static-MIPS CPS recompiler: -1. **delay-slot / continuation-PC handling at a cross-unit control transfer around - an interior/alias entry** (the area commit 47f3f15 touched) — top suspect. -2. R3000A load-delay feeding a branch/jump/syscall-index (see separate finding below). -3. entry-switch fallthrough to the wrong interior block. - -## Separate latent CLASS bug: recompiler does not model R3000A load-delay - -- `recompiler/src/strict_translator.cpp:830` — "No load-delay-slot modeling: per - project decision, the recompiler writes the destination register at the load - instruction's position ... relies on the BIOS already respecting the architectural - load-delay rule (assemblers/compilers schedule loads correctly)." -- The dirty-RAM interpreter (`runtime/src/psx_interpreter.c`) DOES model it faithfully - (`set_load_delay`/`apply_load_delay`, 1-instruction pending-load slot). -- That assumption is VIOLATED in this game's overlays: a scan found **50 load-delay - hazards** (load rt immediately followed by an instruction reading rt) in the resident - overlay code — **0 of them control-flow**, and the immediate 0x896E0 freeze path had - none. So load-delay is a real latent divergence class (worth a class fix), but is NOT - proven to be THIS freeze. -- Class fix (future): detect load-delay hazards in the recompiler and emit the - old-value-preserving (deferred-write) semantics for the hazard pair, matching the - interpreter. No load-delay hazard detection currently exists (only a branch-delay - scanner, `tools/scan_branch_delay_hazards.py`). - -## Deeper investigation (2026-06-23, session 2) — codegen DOUBLY confirmed; red herrings ruled out - -Raw RAM dumped + loaded into Ghidra (MIPS:LE:32 @ 0x80000000). Dumps are now -KEYED by resident-overlay identity (tools dump_ram.py writes -`<name>_f<frame>_<variant-fingerprint>.bin` + a .json manifest of game/frame/ -resident-fn CRCs) — a bare RAM image is ambiguous because the same addresses -hold different overlay variants per scene. - -Codegen-vs-race settled by TWO independent methods: -1. Sandboxed `overlay_diff` shadow (identical state, no IRQ/timing) diverges. -2. `overlay_irq_ratelimit n=4096` (coarse, interp-like IRQ cadence) set BEFORE - the freeze → STILL freezes at ~1826. Not IRQ-cadence/timing. -=> CODEGEN, definitively not a race. - -Red herrings ruled OUT as the direct cause: -- BREAK no-op: the spin-set's break is at 0x80089784. The dirty-RAM interpreter - (`dirty_ram_interp.c:1041`, the engine under overlay_native_off) routes break - to `psx_break` which CRASHES (traps.c:594 = trap_crash + exit(1)). Since - overlay_native_off PROGRESSES (no crash), 0x89784 is NEVER reached in the - working path => main (0x80050B08) does not return there normally. So the - recompiler's break-no-op (code_generator.cpp:973) is a real latent bug (should - fail-loud like strict_translator's `psx_break(...)`/the interp), but it is NOT - what breaks the FMV. It only matters once something upstream wrongly reaches it. -- Load-delay: path clean (see above). - -Operative divergence (what actually happens): under native, control flow reaches -the bootstrap re-loop — the native overlay ring shows a repeating cycle -{0x89860 InitHeap-thunk → 0x89770 → 0x80050B08 "main" → 0x89788 init-array} with -0x800896E0 (the crt0-style stack/heap/context init) re-entered each lap, calling -InitHeap (→ A0 vector) every iteration — which is the `dispatch_tail` all-0xA0 -spin. Under interp this does not loop. i.e. native's 0x80050B08 ("main") RETURNS -(native_ring returned:1) or the bootstrap otherwise re-enters, where interp keeps -running. Root is a mis-emitted control transfer in the 0x80050B08 subtree (large -fn, internal loops, many interior/alias entries) — NOT yet pinned to one insn. - -Shadow caveat (rule 15): `overlay_shadow_dump` reports 13 divergences; the seq-1 -(0x896E0 t0/t1) is suspect as a SYSCALL-BOUNDARY ARTIFACT — native continues -through the A0 dispatch (t1=0x39<<2=0xE4, t0=A0_table[0x39]) while the interp -shadow stops at the overlay→kernel(0xA0) boundary (t1=0x39). Later records have -real RAM divergences (e.g. fn 0x85D70: RAM native=0 vs interp=0xBFC050A4, a BIOS -ROM pointer) suggesting a systemic BIOS-interaction/return-path codegen issue, -not one function. Need finer (instruction-level) isolation. - -## Session 3 (2026-06-23): per-function native-disable BUILT + bug localized to func_80050B08 - -Built the per-function native-disable (the tool ChatGPT recommended): a phys-keyed -blocklist in overlay_loader.c that forces a named overlay function through the -sanctioned dirty-RAM interpreter (NOT skip/stub/HLE — it still runs), exposed as the -`overlay_native_block` debug command ({"addr":"0x.."} add / {"clear":1} clear / -{} report). Runtime-only change (overlay_loader.c + debug_server.c), no regen. - -BISECTION RESULT — culprit is **func_80050B08's own native body**: -- `overlay_native_block addr=0x80050B08` set early → boot PASSES the freeze (frame - 2306+), hits=1. Because the blocked function runs via interp while its CALLEES stay - native, this proves the bug is in 0x50B08's OWN compiled code, not a callee. -- 0x80050B08 = the game main: a 33-`jal` subsystem-init sequence then an infinite main - loop (no `jr $ra`; loops via goto). It is entered ONCE (hits=1); native exec of it - causes the bootstrap re-loop / A0-spin; interp exec runs the game (frame advances). -- Verified clean: all 33 CPS jal TARGETS match the RAM-decoded jal instructions; all - jal return-continuations are present in the entry-switch (no missing/`default` - fall-through-to-top). So it is NOT a wrong-target or missing-continuation bug. - -Additional rule-outs (Ghidra raw-RAM analysis): -- 0x80050CE8 interior entry is SPURIOUS — Ghidra finds NO xrefs to it (over-detected - by the recompiler; only fall-through reaches it). Harmless red herring, not the bug. -- No load-delay hazard in 0x50B08 (loads have compiler-scheduled nop delay slots). - -SMOKING-GUN TRANSITION (overlay_native_ring): the repeating cycle is -0x89860 -> 0x89770 -> 0x50B08 -> 0x89788 -> [back to 0x89860]. After native 0x50B08's -FIRST jal (to 0x89788) returns (ra=0x50B30), control should continue to 0x50B08's -NEXT jal (0x85B20); instead it goes back to 0x89860 (InitHeap inside the 0x896E0 -bootstrap) — i.e. the BOOTSTRAP RE-RUNS. So the 0x89788 return / the 0x50B30 CPS -continuation re-entry is mis-handled under native. This is consistent with why -BLOCKING 0x50B08 fixes it: running 0x50B08 in the interp BYPASSES the native CPS -continuation re-entry entirely (the interp runs jal callees as units and continues -sequentially), so the broken re-entry never happens. - -REFINED HYPOTHESIS: the bug is in the CPS continuation RE-ENTRY for 0x50B08's interior -PCs — the trampoline dispatching pc=0x50B30 (an interior continuation, head<0) through -the g_psx_cps_mode path (overlay_loader.c ~line 1075: overlay_find_by_range + -generation/crc validation + entry-switch) across the 23 resident variants of the -0x38000 region. Either overlay_find_by_range picks the wrong variant for the interior -PC, or the re-entry doesn't route to block_80050B30, so control escapes back to the -bootstrap. This is a RUNTIME dispatch issue (overlay_loader.c), possibly compounded by -the multi-variant region, NOT necessarily a code_generator.cpp per-instruction bug. -NEXT: instrument the CPS continuation re-entry (which candidate/variant is chosen for -pc=0x50B30, and where c->fn(cpu) lands) — add it to the always-on native ring, OR -diff the interp's call-as-unit return vs native's trampoline re-entry at this exact -site. The 0x89788->0x89860 transition is the window to query. - -GENERATED ENTRY-SWITCH CLEARED as the cause: checked all 5 resident _patched.c -variants with func_80050B08 — ALL include `case 0x80050B30`. The only inter-variant -diff is the SPURIOUS `0x80050CE8` case (present in 4, absent in 77548F26; harmless — -0x50CE8 has no xrefs). So re-entry at 0x50B30 would land at block_80050B30 correctly -IF the runtime dispatched pc=0x50B30 into func_80050B08. Since 0x85B20/0x50B30 never -execute, the runtime is NOT re-entering func_80050B08 for pc=0x50B30 at all — the -failure is in overlay_loader_dispatch's interior-continuation path (overlay_loader.c -~1075: head<0 -> g_psx_cps_mode -> overlay_find_by_range(0x50B30) -> crc-validate -> -cpu->pc=addr; c->fn). Across 43 resident 0x38000 variant DLLs, the candidate chosen -for the interior PC 0x50B30 (or its crc validation) is wrong, so control escapes to -the interp/bootstrap instead of block_80050B30. This is a RUNTIME dispatch bug -(multi-variant interior-continuation re-entry), to confirm with live instrumentation -of that exact path (log: for pc=0x50B30, the ci/variant overlay_find_by_range returns, -crc match y/n, and the dispatch return value). - -KEY RING FACT: after 0x89788 returns (pc should be 0x50B30), NEITHER 0x50B30 (native -re-entry) NOR 0x85B20 (0x50B30's jal, via interp) ever appears in the native ring — -control goes straight to 0x89860 (bootstrap). So the trampoline neither native-re- -enters func_80050B08 nor interp-runs 0x50B30; pc=0x50B30 is routed somewhere that -lands back in the 0x896E0 bootstrap. - -overlay_find_by_range is FIRST-BY-RANGE: it returns the first s_cand (registration -order) whose [range_lo, range_lo+range_len) contains phys, then crc-validates ONLY -that one (no fallthrough to other range-containing candidates). With 43 resident -0x38000 variant DLLs (+ interior-entry island fragments, c5efd70) all carrying -func_80050B08-range entries, the candidate chosen for interior PC 0x50B30 can be the -WRONG unit (a different variant or an island fragment whose range overlaps 0x50B30 but -whose entry-switch lacks 0x50B30 -> default -> runs from ITS top = wrong code), or one -whose crc fails -> return 0. Prime suspect for "control escapes to bootstrap." - -FIX DIRECTION (runtime, overlay_loader.c — to design after live confirmation): make -interior-continuation re-entry select the candidate that genuinely OWNS the PC as a -continuation (entry-switch/known-continuation membership, or nearest function-start -<= phys whose body+crc covers phys), not merely first-by-range; and/or try all -range-containing candidates until one validates AND owns the continuation. Until then -the per-function native-disable (overlay_native_block) is the workaround. - -STATUS: localized to the runtime interior-continuation dispatch; exact fault to be -confirmed by instrumenting the trampoline/overlay_find_by_range for pc=0x50B30. - -## ROOT CAUSE CONFIRMED (2026-06-23, session 3) — break fall-through pollutes function range - -Built a CPS-dispatch probe (overlay_cps_probe debug cmd: arm a watched interior PC, -records what overlay_find_by_range + validation decide). Live result for the spin: - - probe pc=0x89788: count=1021, chosen_addr=0x000896E0, ci=96, cands_in_range=6, - crc_matched=1, outcome=2 (ran native) - -i.e. when func_80050B08 does `jal 0x89788` (its first subsystem-init call), the -trampoline can't find 0x89788 as a registered entry (idx_head<0), so the CPS path -runs overlay_find_by_range(0x89788). SIX loaded candidates have a range containing -0x89788; it returns **func_800896E0** (the crt0 bootstrap, crc 0x1D663E23, whose range -spans 0x896E0..past 0x89788). crc matches → it runs func_800896E0 with cpu->pc=0x89788. -0x89770 IS a case in func_800896E0's entry-switch (so the 0x89770 dispatch works), but -**0x89788 is NOT a case → default → func_800896E0 runs from its TOP** = the bootstrap -re-init (zero BSS, InitHeap [the 0xA0 spin], call main 0x50B08 again...). Infinite -re-loop → VBLANK stays masked → deadlock. - -WHY func_800896E0's range covers 0x89788: function_discovery.cpp:493-499 treats BREAK -(and SYSCALL) as FALL-THROUGH ("BIOS uses these as in-line traps that return") and -pushes addr+4 onto the discovery work-queue. The bootstrap has `jal 0x50B08; break` -at 0x89780/0x89784 (break guards "main returned"). Discovery falls through the break -at 0x89784 into 0x89788 — a SEPARATE function (the init-array, called directly by -0x50B08) — absorbing it into func_800896E0's body/range. So 0x89788 is owned by -func_800896E0 (no matching entry-switch case) instead of being its own dispatchable -entry. The interp is immune: it executes 0x89788's bytes (init-array) directly, no -range ownership. - -This is the SAME break-no-op flagged earlier; the damage is at COMPILE/DISCOVERY time -(range pollution), not runtime execution of break — which is why it survived the -"break isn't reached in the working path" rule-out. - -### Fix options (framework-wide; needs regression on Tomba1/MMX6/Ape) -- A. BREAK terminates discovery (don't push addr+4). Clean root fix, matches - strict_translator/interp/HW. RISK: regresses div-by-zero break (PsyQ relies on the - break handler returning to EPC+4, i.e. fall-through) and any trap-that-returns. -- B. Clip a function's discovered range at the next discovered function entry, so even - with break fall-through func_800896E0 stops at 0x89788. Preserves in-function - fall-through (div-by-zero) while preventing cross-function swallowing. Likely the - best fix; needs the next-entry set during range finalization. -- C. Fall through past BREAK only if addr+4 is NOT a known function entry (terminate - when it would cross into another function). Targeted variant of A. -- D. Runtime: overlay_find_by_range prefer the candidate whose c->addr is the nearest - start <= phys (most-specific entry) and whose crc matches, over first-by-range. Only - helps if a candidate with addr==0x89788 (func_80089788) is loaded + crc-matches. -RECOMMEND B or C (don't change break execution semantics; stop the range from crossing -a function boundary). Workaround meanwhile: overlay_native_block 0x80050B08. - -### CHATGPT-VALIDATED FIX PLAN (2026-06-23, debug principles shared) -Verdict: not a BREAK runtime-semantics bug — a discovery/range-ownership bug PLUS an -unsafe runtime "contained range implies callable entry" assumption. Fix BOTH layers -(defense-in-depth). Implement as a three-part class correction (all inside Arch A): - -1. DISCOVERY FINALIZATION (B, required): clip a function's ownership range at the next - STRONG known function entry: owner_end = min(raw_end, next_strong_function_entry_after(start)). - STRONG = direct jal target, direct j/tailcall target, executed dispatch root, - game.toml symbol, manifest overlay entry. WEAK (do NOT clip on these) = linear-sweep - artifact, speculative recovered host, unvalidated jump-table target, range-derived - interior-only guess (e.g. the spurious 0x50CE8, which has no xrefs). 0x89788 is a - direct jal target = strong, so clipping func_800896E0 at 0x89788 is valid. - Compute next-entry from THAT candidate's OWN compile/discovery known-entry set — - NOT from the runtime list of simultaneously-loaded variants (that would mix variants; - overlapping scene variants each independently produce equivalent ownership). -2. DISCOVERY TRAVERSAL (C, good): BREAK and SYSCALL are EXCEPTION EDGES, not ordinary - fall-through. Treat them the SAME (the distinction is NOT "syscall falls through / - break terminates"). Enqueue addr+4 as a trap-RESUME edge only if addr+4 is not a - strong known function entry. Neither may swallow a known function entry. (Preserves - div-by-zero break / returning-trap idioms; prevents cross-function absorption.) -3. RUNTIME DISPATCH (D, required — fail-closed): add per-candidate LEGAL-ENTRY metadata - (the entry-switch case set / legal continuation PCs). overlay_find_by_range becomes: - (a) find candidates whose owner_range contains phys, (b) CRC/byte validate, - (c) REQUIRE candidate.legal_entries contains phys, (d) prefer most-specific owner if - several remain, (e) if none legally serves phys -> sanctioned dirty-RAM interpreter. - This does NOT violate Arch A (fallback is the sanctioned interp, not a stub/HLE). - Correct degradation = interp runs the exact live bytes at that PC. -PLUS codegen: the generated entry-switch DEFAULT must NOT fall through to the top block. - Emit `default: return psx_native_bad_entry(cpu, unit_id, cpu->pc);` — dev/strict = - abort with ring context (matches "fully implemented or aborts"); normal runtime = - reject provider and fall to sanctioned dirty interp. It must NEVER run from the top. -NUANCE: keep dispatch OwnerRange (for range-based containment) separate from - CRC CoveredRanges (bytes the native body may execute + validate). Don't conflate - "clipped ownership range" with "the only bytes this function may execute." - -INVARIANT (the takeaway): "containment is not ownership, and ownership is not entry -legality." Expected post-fix: func_800896E0 owns [0x896E0,0x89788); 0x89788 either has -its own native candidate with an exact legal entry or falls to dirty interp; -overlay_find_by_range(0x89788) can neither choose func_800896E0 nor run it from the top. - -## Next steps - -1. Build a PER-FUNCTION / per-entry-PC native-disable (ChatGPT's #1 tool; does not - exist yet — only the global overlay_native_off). It must route the disabled - HostUnit through the sanctioned dirty-RAM interpreter (NOT skip/stub/HLE). Then - bisect: start by disabling native for 0x80050B08, then its callees, to find the - single function whose native execution flips the freeze. This gives causal proof - where the shadow (function-granular, with syscall-boundary artifacts) cannot. -2. With the culprit function isolated, instruction-diff native-vs-interp within it - (extend the shadow to record the first diverging guest PC, or step the dirty - interp and the native side from the same entry and compare per-block) to pin the - exact mis-emitted transfer. -3. Confer with ChatGPT on that disasm to confirm the codegen construct (likely a - jr/jalr/branch continuation-PC or interior/alias-entry transfer in 0x50B08's tree). -4. Class-fix in `recompiler/src` (never generated C, never a Tomba 2 workaround). -5. Verify: regen overlay cache, `overlay_diff` shows 0 divergences, live native run - passes the freeze (frame > 2031) at 60fps with MDEC advancing. -6. Adjudicate the exact divergent MIPS against the Beetle oracle + BIOS disasm + Ghidra; - generated C is evidence only. - -Separately (independent class fixes, lower priority): emit BREAK as fail-loud/exception -in code_generator.cpp (match strict_translator/interp), and model the R3000A -load-delay slot. - -Diagnostic principle reminder: a shadow mismatch is "native differs from interp at -fn X / insn Y", NOT automatically "native is wrong" — the interpreter can also be -wrong; adjudicate against Beetle/disasm before committing a fix. diff --git a/docs/TOMBA2_SPLASH_ATTRACT_PLAN.md b/docs/TOMBA2_SPLASH_ATTRACT_PLAN.md deleted file mode 100644 index fafa5045..00000000 --- a/docs/TOMBA2_SPLASH_ATTRACT_PLAN.md +++ /dev/null @@ -1,317 +0,0 @@ -# Tomba 2 splash→attract — investigation plan (Recomp GPT, 2026-06-22) - -Frame-1997 is FIXED. Next: the recomp softlocks on the Whoopee-Camp splash; the Beetle -oracle reaches the attract. The splash state machine never advances. CD ruled out (attract -data loads correctly). This is a first-divergence in the scene-transition logic. - -## ⚠️ CORRECTION (2026-06-22 sess2, after fixing the CD trace flood) — CD streaming is NOT the immediate blocker -After gating the per-word DMA 'D' trace off (commit 598d1aa) the CD COMMAND flow is finally readable, and it -**refutes the "CD streaming is the root cause" conclusion below.** The game does the directory traversal via -SINGLE-SECTOR DATA reads that WORK: `SetLoc`+`ReadN`+`Pause` per sector — PVD LBA16 → root LBA22 → subdir -LBA373 (all delivered, INT1, parsed perfectly). After LBA 373 is delivered (frame 1986), the game issues -**NO further CD command** and stalls (now `reading=0`, a half-state `pending Pause 0x09` phase=1). So the -recomp never even REACHES the CD-audio/XA streaming — it stalls earlier, at the **post-directory FILE-LOAD -DECISION**. The CD-audio streaming IS still missing (the oracle hammers DMA3 to `0x1Axxxxx` ~1.1M times; the -recomp doesn't), but since the game issues no streaming command, that's downstream, not the gate. - -**Corrected blocker:** after a correct directory read, the game's code decides what/whether to load next, and -the recomp's decision diverges (issues no SetLoc+ReadN for the next file). Same disc data + same game code + -correct directory ⇒ a STATE the game tests after the dir read differs (candidate: an event/flag/return value, -possibly from the interrupt/event model — the frame-1997 neighborhood). NEXT: find the code that runs right -after the LBA-373 directory delivery (the CD-read completion at `pc≈0x8008ADE4` / `0x8008594C` returns into -the file-open/dir-walk routine) and trace its branch + the state it tests on recomp vs oracle. - ---- - -## (SUPERSEDED) CONFIRMED ROOT CAUSE — incomplete CD STREAMING path (first exercised by Tomba 2) -NOTE: superseded by the CORRECTION above — streaming is real & missing but NOT the immediate gate. -The splash softlock is the **CD streaming / XA / CD-DA + DMA-3 audio path being incomplete** — Tomba 2 is -the FIRST title psxrecomp has run that actually drives it (Tomba 1 / MMX6 / Ape use SPU-voice/SEQ music on -single-track data discs; none stream CD audio). Evidence chain (all measured): -- Splash state machine `[0x80050D00]` stuck at state 0; advance writer `0x80107268` (sets state=2/frame) - is in a FILE that never loads (`0x80107xxx` = zeros in recomp, real code on oracle). -- The recomp's directory read (LBA 373) is PERFECT (full file list A00-A0L/DEMO/GAME/OPN/SOP/START.BIN) — - not a CD-fidelity or game-code bug. It just never issues the next file load. -- The recomp's CD froze ~frame 1987; its CD interrupt handler spins at `pc=0x8008B6F4` reading - `[0x800AC2C4]=0x1F8010B8` (**DMA-3/CDROM CHCR**) waiting for CHCR bit `0x01000000` (DMA start/busy) to - clear — the CD-streaming DMA-drain loop. The oracle hammers DMA-3 to `0x1Axxxxx` (~1.1M transfers via - game routine `0x8008DB88`); the recomp gets EMPTY reads → the stream never flows. -- `cdrom.c` Play(0x03) handler comment: *"Red Book sample output is not yet decoded — silent for now — - but the flow no longer stalls."* XA decode fns exist but the continuous-stream delivery is not built. - -So: the splash sequence streams CD audio+data; the recomp's streaming doesn't deliver continuously, so -the IRQ-handler DMA-wait never completes / the audio/stream sequence never progresses, so the game never -loads the `0x107xxx` advance-task file → state stuck 0. **The fix is to BUILD the CD streaming path -(Architecture-A hardware-sim, no stubs):** DMA-3 CD-streaming must deliver the game's requested sectors -continuously (data sectors to the game's buffer; XA-audio sectors decoded to SPU; CD-DA Red Book output + -position/end reporting for multi-track), so the IRQ DMA-wait completes and the stream advances. This is -the STUBS_TO_FIX S3/S4/S5 (MDEC/SPU/DMA) streaming family, now actually exercised. NOTE: must not regress -T1/MMX6/Ape (they don't stream, so the streaming additions should be inert for them) — but this is an -isolated branch; Tomba 2 first, regression after. - ---- - -## Strategy — REVERSE WRITER HUNT (not a PC diff) [historical — how we got here] -Both recomp and oracle reach the same idle PC `0x80050CE8`, so a raw PC-stream diff buries -the signal (code reconverges, state diverged). Instead: make the ORACLE stop itself at the -successful transition and dump the write history *before* it (the recomp never reaches that -trigger). Find the FIRST oracle write/branch that selects "advance" while the recomp selects -"stay in state 0." It may NOT be the state byte itself — could be a nearby flag the state -machine later consumes. - -## Likely gate (ranked by ChatGPT, given our evidence) -1. **GPU / DrawSync / render-completion flag (most likely).** State-0 path is render-heavy - (`jal 0x8008179C / 0x800815D0 / 0x80081560` + double-buffer toggle + wait-2-VBlank + loop). - VBlank pacing works + CD loaded, but a DrawSync / GPU-done / fade-complete predicate never - becomes true → state stays 0. Symptom matches exactly. -2. **A countdown/fade variable, NOT the state byte** (`if(++fade>=limit) state=1` / - `if(palette_fade_done) state++`). → trace the WHOLE scratchpad + nearby globals. -3. **CD load-COMPLETE EVENT** (CdSync/DeliverEvent/TestEvent/WaitEvent/callback flag): bytes - loaded but the completion event/flag differs. (Data movement already ruled out; the EVENT is not.) -4. Less likely: input timeout; thread/TCB scheduling (callback writes the flag while the loop polls). -5. Unlikely: MDEC/FMV (only chase if the oracle pre-trigger trace shows MDEC/DMA0 activity). - -## Concrete capture plan -**Step A — transition trigger (both runtimes), fire on ANY of:** -- PC enters attract code `0x80041800..0x80052000` -- PC leaves splash loop `0x80050C40..0x80051FFF` -- write to `0x1F8001A4` changes it, OR nearby cluster `0x1F8001A0` changes -(Do NOT require the state byte as the only trigger — the real flag may be elsewhere.) - -**Step B — trace ALL scratchpad writes in Beetle** `0x1F800000..0x1F8003FF`, plus clusters -`0x800E8080..0x800E80C0` (VBlank gate counter), `0x1F800180..0x1F800260` (splash state -cluster), `0x80041000..0x80052000` (attract overlay). Per write record: seq, cycle/frame/ -vblank, pc, ra, opcode, width, addr, old, new, state byte, threshold, counter, irq stat/mask, -BIOS event/callback id, cd state, gpu dma/status/drawsync. - -**Step C — outer-loop snapshots** at each `0x80050CE8` / `0x80050C40` hit: iter, pc, ra, state, -threshold, vblank_counter, hash(scratchpad), hash(0x800E8000..9000), hash(attract code), -irq, cd_state, gpu_status, drawsync, pad. Compare Beetle vs recomp BY OUTER-LOOP ITERATION, -not global frame. Expect: equal for iters 0..N, then Beetle's hash changes at N+1 and advances -at N+2; recomp missing the same write. - -**Step D — identify the writer PC** of the first advance-flag change + the predicate loads -immediately before it. Decode that PC's live RAM. Branch-trace around it. Run recomp to the -same iteration and compare the loaded values. Then classify: -- GPU busy/drawsync wrong → GPU event/status bug -- BIOS event flag → DeliverEvent/TestEvent/WaitEvent bug -- CD sync/callback flag → CD event-completion bug (not DMA/data) -- pad state → input bug -- timer/frame counter → root-counter/VBlank-accounting bug -- stale local → dirty-RAM interp / block-return / store-width / ENDIAN bug - -## TWO TRAPS TO CHECK FIRST (cheap, do before the hunt) -1. **Committed-PC in the trace.** This is dirty-interp overlay code; verify the write/branch - trace records the ACTUAL committed PC (after the frame-1997 committed-PC fix) — else the - writer/caller attribution lies. -2. **Byte/halfword scratchpad endian.** A scratchpad BYTE state (`0x1F8001A4`/`0x1F80019C`) is - exactly where a store-width / endian / logging mismatch makes the trace lie. Verify sb/sh - into scratchpad are endian-correct in BOTH tracing and execution. - -## NOTE / correction to verify -Re-derive the real state byte from the live disasm: `0x80050D00: lbu v1,412(s5)` with -s5=0x1F800000 ⇒ state byte = **`0x1F80019C`** (offset 0x19C), not 0x1F8001A4 as first sampled. -Confirm which byte the branch-out-of-state-0 actually consumes before instrumenting. - -## First run order (ChatGPT) -1. Beetle only: arm scratchpad + `0x800E8080..0x800E80C0` write rings; run to attract; dump last ~8k writes/branches. -2. Find first write changing the state byte / nearby flag / a RAM flag read by the state-0 exit branch. -3. Decode the writer's live RAM code (read_ram). -4. Branch-trace the writer/caller block (source PC, condition regs, taken/not, memory loads). -5. Run recomp to the same splash outer-loop iteration; compare those exact loaded values → classify the subsystem. - -Highest-value first artifact: the **Beetle pre-attract write ring ending at the first -transition out of state 0** + the writer PC + the predicate loads right before it. - ---- - -## EXECUTION PROGRESS (2026-06-22 sess2, partial) -- Splash loop is at `0x80050C40+`; state machine reads byte `[0x1F80019C]` (`lbu v1,412(s5)`, - s5=0x1F800000 @ `0x80050D00`): state 0 → render handler `0x80050D44` which just renders and - `j 0x80050C6C` (LOOPS, never advances); state 1 → loop; state 2 → handler `0x80050D98` which - FADES + `sb s2,0x19C(s5)` sets state=1; state 3 → `0x80050DD8`. **The state-0 handler never - advances the state** → the 0→2 advance must come from an EXTERNAL writer (callback/event). -- Scanned ALL loaded code `0x80040000..0x80092000` for stores to offset `0x19C`: only TWO — - `0x80050A30 sb zero` (init→0) and `0x80050DA8 sb s2` (state-2 handler →1). **NOTHING writes - the state byte to 2.** So the 0→2 writer uses different addressing (base≠0x1F800000 / offset - ≠0x19C) OR is in code not matched by the scan — i.e. it is event/callback-driven (ChatGPT - hypothesis #1/#3/#4), reachable only at the transition. -- Confirmed `0x80050CE4/CE8` is a COMMON VSync-wait loop: the Beetle oracle (visually in the - ATTRACT scene) ALSO samples PC=`0x80050CE4`. So PC-sampling cannot distinguish scenes — the - scene state is in RAM/scratchpad (recomp cluster `0x1F800198..1A4` = all ZERO; oracle = real - data). Apples-to-apples requires catching the oracle AT the splash, not at attract. -- **DECISIVE NEXT STEP:** reverse writer hunt on the oracle — restart Beetle with a write - watch/trace on `[0x1F80019C]` (+ cluster `0x1F800180..0x1F8001C0`) armed from boot; catch the - write that sets it 0→non-zero while PC is in the splash range `0x80050xxx`; that writer PC + - its predicate loads are the answer. Then check whether the recomp ever executes that PC and - why the predicate differs. (TCP debug works fine; the browser flakiness is unrelated.) -- Recomp runs STABLY stuck on the splash to frame 78k+ this run (the earlier ~17k death was - not reproduced — likely the window was closed), so it is live-inspectable for the comparison. - -## ✅ ROOT CAUSE FOUND (2026-06-22 sess2) — missing splash-advance code at 0x80107xxx -Ran the oracle reverse-writer-hunt (wtrace_arm on `[0x1F800190,0x1F8001A8]` from a fresh boot; -TCP raw cmd since the client maps `wtrace_add` but the oracle wants `wtrace_arm`). The oracle's -state byte `[0x1F80019C]` ping-pongs EVERY frame: -- `PC 0x80107268` writes **state=2** (the advance trigger), then -- `PC 0x80050DA8` (state-2 handler) writes **state=1**. -So on HW, `0x80107268` runs every frame and drives the splash forward. - -**The recomp NEVER runs it because the code isn't there.** Disassembled `0x80107200..0x8010728C`: -- ORACLE = the real splash-advance function (the `sb v0,412(v1)` state=2 write is at 0x80107268, - preceded by a counter `addiu s1,s1,-1` / `bne s1` loop + GPU/struct setup). -- RECOMP = **ALL ZEROS (NOPs).** The function is simply not loaded. - -Memory map sampling `0x80100000..0x80110000` (recomp vs oracle): recomp has `0x80104368` and -`0x8010A000` loaded (but with DIFFERENT content than the oracle) while `0x80107000`, `0x80110000` -are ZERO in the recomp and real code on the oracle. So the recomp's high-RAM overlay -(`0x80100000+`) is PARTIALLY loaded and DIVERGENT — a **load / relocation divergence during boot**: -the recomp failed to bring the `0x80107xxx` overlay region into RAM (or loaded it elsewhere). - -This is the real softlock cause (NOT the JIT/cache — confirmed pure interp: overlay -`dispatch_native=0`, `dispatch_interp_fallback=125778`, sljit `live=0/compiles=0`, autocompile off). - -### NEXT (loader hunt) -Find how the oracle gets `0x80107xxx` into RAM and why the recomp doesn't. It loads EARLY (oracle -has it by frame 630), so it's a boot/splash-setup load — a CD read OR a RAM→RAM memcpy/decompress. -The cd_read_log ring (256 entries) has already evicted the early loads, so capture from boot: -- Arm an always-on/early wtrace on `0x80107000..0x80108000` (writes that INSTALL the code) on BOTH - the oracle and the recomp; compare. The oracle's installer write (DMA or memcpy store) PC + source - tells us the load mechanism; the recomp either never does it (skipped load / wrong branch) or writes - it to a different address (relocation/base divergence). -- Also dump the full early CD read log from boot (restart with cd logging) to see if a CD read to - 0x107xxx happens on the oracle but not the recomp. - -## LOADER HUNT RESULT (2026-06-22 sess2) — it's a CD DMA load, late, that the recomp never gets -Armed `wtrace 0x80107000..0x80108000` on a fresh oracle from frame 411. Findings: -- The ONLY CPU-store writes caught to the region were the BIOS RAM-zeroing pass (PC `0xBFC0D864`, - ra `0xBFC06BBC`, writing 0x00000000 to `0xA0107C00+` at frame ~848). NOT the code install. -- The real code at `0x80107268` is **still ZERO at oracle frame 2289**, but **present by frame 3960** - — and the wtrace caught **ZERO CPU stores** carrying those code bytes. ⇒ the splash-advance code - is installed by **CD DMA (channel 3, CD→RAM)**, not CPU stores. It loads LATE (≈frame 2289–3960), - i.e. DURING the splash, not at initial boot. -- So **the CD is NOT fully ruled out** after all: the EARLY load (`0x41800+`, 250 sectors) works, but - this LATER DMA load of the `0x80107xxx` overlay is the one the recomp never receives (its region - stays zero even at frame 78k). cd_read_log cross-compare blocked by tooling (beetle has no - `cd_read_log`; recomp dump came back empty once — Rule 15 follow-up). - -### Refined model + next step -Likely chicken-and-egg / CD-completion: the splash state-0 loop (both run) initiates or waits on a CD -load that brings in the `0x80107xxx` advance task; on HW the DMA completes (~frame 3960) and the task -then drives state 2↔1; on the recomp the load never lands so state stays 0. This points back at the -**CD/DMA + completion-event path for a SECOND (mid-splash) load**, distinct from the working first load. -NEXT: (1) fix/confirm cd_read_log on the recomp + add it to beetle (Rule 15) so we can diff CD reads; -(2) restart the RECOMP with full from-boot CD logging and check whether it ever issues a read whose -DMA dest covers `0x107xxx` — if NO, a code-path/branch divergence skips the load (the state-0 loop -never reaches the load trigger); if YES-but-data-absent, a CD/DMA targeting or completion bug drops it. -(3) On the oracle, trace what the state-0 loop does right before the load fires (the CD command + -DeliverEvent/CdSync) and compare the recomp's same loop. - -## RECOMP-SIDE RESULT (2026-06-22 sess2) — the recomp NEVER issues the 0x107xxx CD read -Restarted the recomp, dumped cd_read_log: total=2651 reads, dest range `0x41800..0x104368`, -distinct LBAs {16,18,24,373}. **`covers 0x107xxx = False`.** The recomp does the directory lookup -(LBA 16/18/373 → scratch buffer `0x104368`) but **never issues the actual file load to `0x107xxx`**, -then goes quiet (total stuck at 2651 — no further CD activity while softlocked). So this is a -**code-path / task divergence in the file-load DECISION**, not a DMA-targeting or CD-fidelity bug at -the read level (the earlier `0x41800` load proved raw CD reads land correct, advancing MIPS code). - -So the chain is: -1. splash state-0 loop runs (recomp + oracle). -2. on HW something issues a CD load of the `0x80107xxx` overlay (the splash-advance task) mid-splash - (~frame 2289–3960) and DMAs it in. -3. the recomp never issues that load → `0x80107xxx` stays zero → advance writer `0x80107268` never - runs → state stuck 0 → softlock. - -### Where the load-issuing task likely lives (next investigation) -The state-0 handler (`0x80050D44`) only renders + loops — it does NOT issue a CD load. So the load is -issued by ANOTHER task/callback during the splash that the recomp doesn't run. Strong candidates, -in order: (a) a per-frame scheduler/event/callback driven off the game's VBlank handler — SAME -neighborhood as the frame-1997 HookEntryInt handler, so the recomp's event/scheduler model may not -run the scene-loader task; (b) a CD-complete-event (CdSync/DeliverEvent) callback that chains the -next load; (c) a separate thread/TCB the recomp's scheduler never resumes. - -### Concrete next steps -1. Trace the ORACLE's CD command stream (mmio writes @0x1F801801) around frame ~2289–3960 to find the - SetLoc+ReadN that targets the 0x107xxx file, and the PC/caller that issues it. (recomp has - mmio_dump / a CD command trace; beetle needs the same — Rule 15.) -2. Find that issuing PC/function; check whether the recomp ever executes it. If it's reached from a - VBlank-callback / event chain, compare the recomp's event delivery + DeliverEvent/WaitEvent + - TCB/callback execution rings to the oracle's at the splash. -3. If the issuing task is event/scheduler-driven, this likely shares root with the interrupt-model - work — verify the game's per-frame callback chain fully runs in the recomp. -Tooling debt (Rule 15): raw-socket cd_read_log parse fails on the big response (use debug_client); -beetle lacks cd_read_log + a cdrom command trace — add them for clean cross-backend diffs. - -## DMA-3 trace + refined divergence point (2026-06-22 sess2) -Armed `wtrace 0x1F8010B0` (DMA3/CDROM MADR) + `0x1F801800` (CD cmd) on a fresh oracle. During the -splash the oracle does HEAVY CD streaming: DMA3 MADR is written ~1.1M times (to `0x1A2xxx` / -`0x1A8xxx-0x1AExxx`, all via game DMA routine `pc=0x8008DB88`) — this is the splash's CD-DA / XA -AUDIO (the Whoopee-Camp jingle, TRACK 02). The single `0x107xxx` code-load DMA is buried among them -and evicted from the (small) ring before I can dump it — wtrace-on-MADR can't isolate it by value. - -**Refined divergence point (the precise lead):** the recomp's cd_read_log shows it DID the directory -lookup (LBA 16/18/373 → scratch `0x104368`) **recently, right before it got stuck**, then CD activity -FROZE (total stuck at 2651). The oracle does the SAME lookup and then issues the `0x107xxx` file load -+ keeps streaming. So the divergence is **immediately after the directory lookup: the recomp does not -follow it with the file load.** Two sub-cases to decide: -- (a) the recomp's directory-entry RESULT (the LBA 373 / `0x104368` record: file extent-LBA + size) - is WRONG → it computes "nothing to load" / wrong target. (Raw CD fidelity is OK — the 0x41800 load - was valid code — but verify the specific LBA-373 record content vs the oracle.) -- (b) the record is correct but a later branch (a state/flag/event check after the lookup) diverges so - the load is skipped. -Also notable: the recomp's cd_read_log dests are only `0x41800`/`0x104368` — NOT the `0x1Axxxxx` audio -stream the oracle hammers — so the recomp may also not be streaming the CD-DA/XA splash audio (could be -a related symptom of the same post-lookup divergence, or a separate XA path difference). - -### Concrete next step -Restart the RECOMP with `wtrace` on the CD command reg `0x1F801801` (+ SetLoc params `0x1F801802`) and -on `0x1F8010B0` from boot; catch the SetLoc+ReadN sequence for LBA 16/18/373 and the issuing PC/ra. -That PC is the directory-lookup/file-open routine. Then disasm it (live RAM) and single-step/trace what -it does AFTER the lookup — find the branch where the oracle issues the next ReadN (file→0x107xxx) and -the recomp doesn't, and read the directory-entry record both sides consume. That pins (a) vs (b). - -## SUB-CASE (b) CONFIRMED — directory read is VALID; it's a file-search/load-DECISION divergence -Decoded the recomp's `0x104368` buffer: it is a well-formed ISO9660 directory: rec0 = {len 48, -extent_LBA 373, size 2048, flags 0x02 (directory), name "."}, rec1 = {extent_LBA 22, ".."}, more -records follow. So the recomp READ AND PARSED the directory at LBA 373 correctly — the data is fine, -CD fidelity is fine. ⇒ The divergence is **(b): after a correct directory read, the code that walks -the directory to find the target file and issue its load (→0x107xxx) takes a different path in the -recomp and never issues the ReadN.** So the next investigation is the FILE-SEARCH / LOAD-DECISION -code, not the CD layer. - -Likely shapes of the (b) divergence: a filename compare that mis-matches; a directory-walk loop that -exits early; a "load already done?" / state / event-flag check that's in the wrong state in the recomp; -or a return value from the dir-read/open routine that differs. Find the routine that issued the LBA-373 -read (wtrace CD cmd reg from boot → PC/ra), disasm it live, and compare its post-read branch + the -values it tests on recomp vs oracle. - -### Directory contents (recomp parsed it perfectly) + NEW audio-streaming clue -The recomp's `0x104368` subdir (LBA 373) decodes to the full game file list — `A00.BIN`..`A0L.BIN` -(level/scene overlays), `CRD.BIN`, `DEMO.BIN`, `GAME.BIN`, `OPN.BIN`, `SOP.BIN`, `START.BIN`, all with -valid extents/sizes. So directory parsing is perfect; the recomp simply never LOADS the next file. -(The big LBA-24 / `0x41800` 512KB load holds the splash code at `0x80050xxx`; the `0x107xxx` advance -task is a SEPARATE file from this dir that never loads.) - -**NEW CLUE — recomp's CD-DA/XA audio streaming is failing.** The recomp's CD activity FROZE at -~frame 1987 (now stuck at frame 4379, no CD since). cdrom_trace is 100% flooded with `dma` empty-read -events (`val=0 addr=0`, `func=0x00000E10` = the EXCEPTION HANDLER chain-walk, `pc=0x8008B6F4`). Combined -with: the recomp's CD dests are only `0x41800`/`0x104368` while the ORACLE hammers DMA3 to `0x1Axxxxx` -(~1.1M MADR writes, game routine `0x8008DB88`) — the recomp is **NOT streaming the splash CD-DA/XA audio -(the Whoopee-Camp jingle, multi-track TRACK 02 / XA), it's getting EMPTY reads in the CD interrupt -handler.** Strong hypothesis: the splash→attract progression is gated on the audio stream (jingle end / -stream-position / XA-sector event), and because the recomp's audio streaming returns empty, that -gate never fires → the file-load that brings the `0x107xxx` advance task is never issued → state stuck 0. - -This ties the softlock to the **CD-DA/XA streaming path for the multi-track Tomba 2 disc**, executed -from the CD/VBlank interrupt handler — i.e. likely the SAME interrupt/event neighborhood as frame-1997. - -### Blocking tooling (Rule 15 — fix next) -`cdrom_trace_dump` ring is fully evicted by `dma` (empty-read 'D') events — can't see the CD commands. -Edge-suppress the 'D'/dma empty-read events (the memory's documented fix) so the SetLoc/ReadN/Play -commands + their PCs are visible. Also add `cd_read_log` + a CD-command trace to beetle for cross-diff. - -### Revised next steps -1. Fix the cdrom_trace dma-flood (edge-suppress empty reads) → see the CD command sequence + PCs. -2. Investigate the CD-DA/XA AUDIO streaming for the multi-track disc: does the recomp read TRACK 02 / - XA audio sectors at all, or return empty? Compare with the oracle's DMA3 stream to `0x1Axxxxx`. -3. Determine whether the splash advance is gated on the audio stream; if so, fix the multi-track - CD-DA/XA read path (the empty reads from `pc=0x8008B6F4` in the exception handler are the thread to pull). diff --git a/docs/TOMBA2_WHOOPEE_BAIL.md b/docs/TOMBA2_WHOOPEE_BAIL.md deleted file mode 100644 index 8e9b86e5..00000000 --- a/docs/TOMBA2_WHOOPEE_BAIL.md +++ /dev/null @@ -1,411 +0,0 @@ -# Tomba 2 — Whoopee-Camp splash freeze (wild-call/return-contract bail storm) - -Branch: `wt/tomba2`. Single source of truth for the splash freeze. Update as we learn. - ---- - -## 🎯 STATUS 2026-06-22 sess2 — FRAME-1997 EXIT FIXED; new frontier = splash→attract - -**FIXED (commit d016e3b):** the frame-1997 abnormal exit. Tomba 2 now runs past it — -the Whoopee-Camp splash renders and the frame climbs steadily (16k+), `dirty_ram_aborts=0`. -Root cause was **longjmp-landing-depth** (NOT async, NOT EPC de-overload): a game RFE -longjmp at a BLOCK-RETURN pump left `cpu->pc=0`, which unwinds to the outermost dispatch -and reads as a clean exit. Fix = restore the committed guest PC after the block-return -pumps so the trampoline re-dispatches (details in the CORRECTION section below). - -**NEW FRONTIER — splash→attract divergence (first-divergence, NOT a CD bug).** The recomp -softlocks on the splash; the Beetle oracle on the same disc reaches the **attract/intro -gameplay** (3D Tomba on a grassy field). BOTH sit at the same idle PC `0x80050CE8`, so the -recomp tracks the oracle's CODE but has diverged in game STATE. - -Investigated 2026-06-22 sess2: -- The splash main loop is at `0x80050C40+`; it pumps a 2-frame VBlank gate (counter - `0x800E809C` reset by `sh zero` @`0x80050C8C`, waited up to threshold byte `[0x1F800235]=2` - — works, reaches 2 between samples) and runs a state machine on `[0x1F8001A4]` (observed - stuck at 0). The state never advances to load/show the attract. -- **CD RULED OUT.** The attract data DOES load correctly: consecutive CD-DMA dest addresses - (`0x80041800`, `0x80042000`, …) hold DIFFERENT, valid MIPS code, and `read_msf` advanced to - LBA 374. The `cd_read_log` "lba" field misleadingly logs the SetLoc target (24) for every - DMA read, not the per-sector read position — a TOOL bug to fix (Rule 15): it cost a wrong - "CD stuck at LBA 24" hypothesis. The CD streaming/advance is fine. -- So the divergence is a true first-divergence in the splash state machine / scene-transition - decision, upstream of the common `0x80050CE8` loop. Next: oracle PC-trace from the splash - (psx-beetle @4380) to find the first divergent branch / state write that gates the attract. - -**SECONDARY: ~frame-17k crash.** With the frame-1997 fix the recomp runs ~16-17k frames on -the splash (≈8e9 dirty insns; ~470k insns/frame — the splash spin is expensive) then dies -silently (SEH; watchdog was off). Possible host-stack growth from the fix's re-dispatch on -NESTED block-return-pump RFEs (the restore fires ~1/22 frames) — must check (also a -regression-risk signal for the shared path). Re-run with watchdog + crash dumps armed. - -**PENDING GATES (before master-landing of the fix):** (1) T1/MMX6/Ape regression — the -fix is in the SHARED dirty-pump path; argued low-risk (the restore only fires when a -handler RFE clears a valid committed PC, which playable titles don't hit) but NOT yet -run. (2) Clean up the dead async-EPC scaffolding (inert under the corrected model). - ---- - -## ✅ RESOLVED (2026-06-22) — root cause was the overlay-region FLOOR, not a contract bug - -The splash freeze is **fixed**. Both prior theories below (exception/EPC, then "B0 -non-local-return contract bail") were **symptoms, not the cause** — they were -artifacts of the real bug. Data-confirmed root cause and fixes: - -**Root cause: `OVERLAY_REGION_FLOOR` was hardcoded to Tomba 1's main-EXE text end -(`0x98000` = `0x10000 + 0x88000`).** Tomba 2's boot EXE is only `0x28800`, so its -text ends at **`0x38800`** and its disc overlays load at **`0x80085000`+** — i.e. -in the range `[0x38800, 0x98000)` that the hardcoded floor wrongly classified as -"statically-recompiled main-EXE text." Consequences for every Tomba 2 overlay: -1. `allow_local_dirty_flow` / `is_local_dirty_target` were **false**, so overlay - code ran **block-by-block** (each guest branch surfaces to the trampoline and - re-dispatches). A ~295 KB overlay-init `memset` at `0x800896F0` became ~73 K - trampoline round-trips (the "slow" red-herring symptom). -2. Intra-overlay calls routed through **`dispatch_nonlocal_call` → `psx_dispatch_call` - with a live `stop_addr` contract** instead of chaining in-interpreter. When the - per-frame interrupt pump fired mid-contract, the contract was violated and - **bail/flatten cascaded into the `$ra=$sp=1` storm** (the symptom the prior - handoff chased as a "B0 wild-return contract bug"). - -**Evidence:** the §18 xprobe ring + dispatch_tail + live disasm showed the trigger -code (`0x80085900`, `0x800896F0`, `0x8008AE50`) all at phys `0x85xxx`–`0x8Axxx`, -**below** the `0x98000` floor; the header comment itself (`dirty_ram_interp.h:55`) -documented the floor as "Tomba: load 0x10000 + text 0x88000". - -**Fix 1 (overlay floor, the root fix):** `OVERLAY_REGION_FLOOR` is now the runtime -variable `g_overlay_region_floor`, pinned at game load to -`(gc.load_address + gc.text_size) & 0x1FFFFFFF` (= `0x38800` for Tomba 2). -Default `0x98000` for BIOS-only. For Tomba 1 this evaluates to the same `0x98000` -(no behavior change); it is correct-by-construction for every game. Files: -`dirty_ram_interp.h`, `dirty_ram_interp.c`, `main.cpp`. **Result: bail storm gone -(`bail_first=0`), splash advances at 60 fps past the old frame-1823 wedge.** - -**Fix 2 (latent dispatch-depth desync, exposed next in the cascade):** with overlays -now on the local-flow path, the per-4096-insn pump delivers interrupts from a -**nested** dispatch (depth > 0). `psx_check_interrupts` reset `g_psx_dispatch_depth` -to 0 around the handler but never restored the interrupted code's nesting, and a -ReturnFromException longjmp skips the handler frames' decrements — so the outer -frames unwound to **`dispatch_depth = -1`**, the `outermost` test misfired, and the -top-level dispatch returned to PC=0 ("execution completed" exit). Fix: save the -depth before the handler, run the handler at 0, restore after. File: `interrupts.c`. - -**Current cascade frontier (NEXT bug) — PINNED 2026-06-22:** with both fixes Tomba 2 -runs cleanly to **frame ~1997**, then the **top-level dispatch returns `cpu->pc=0`** -("execution completed" abnormal exit; `dispatch_depth=0`, so NOT the depth bug). - -Pinned via `PSX_FNTRACE_ARM=all` (exit-trace dispatch tail) + `PSX_EXIT_HALT=1` -(halt-and-serve at the PC=0 exit, overlays still loaded) + live disasm: -- `last_store_pc = 0x80086208` decodes to `li $t2,0xB0; jr $t2; li $t1,0x17` = - a call to **B0:0x17 `ReturnFromException`**. The adjacent overlay thunks are - `0x80086210` → **B0:0x18 ResetEntryInt** and `0x80086220` → **B0:0x19 HookEntryInt**. -- So **Tomba 2 installs its OWN interrupt handler via `HookEntryInt` and returns - from it by calling `ReturnFromException` directly.** Its per-frame VBlank callback - (`0x80085D1C → … → 0x80086200`) drives this. -- Our exception model assumes the *default* BIOS path: `psx_check_interrupts` sets - `COP0_EPC = 0x80000048` (a host longjmp sentinel, NOT the real interrupted PC), - dispatches the handler, and resumes via `setjmp/longjmp` + a host GPR save/restore - — it does **not** drive the BIOS's real EPC/TCB exception save+restore that - `ReturnFromException` depends on. When the game's hooked handler calls - `ReturnFromException`, the RFE-to-sentinel eventually resolves to `cpu->pc=0` - (in_exception already cleared at the exit) → top dispatch returns → exit. - -This is the **EPC-overload tension** the (refuted-for-the-bail-storm) design notes -below flagged — refuted as the bail cause (that was the floor), but it **is** the -mechanism here. **Fix direction (delicate, risk of regressing T1/MMX6/Ape):** make -the exception model support a game-installed `HookEntryInt` handler that calls -`ReturnFromException` — i.e. stop overloading guest EPC as a host token; keep host -exception-return state as separate runtime metadata; set `EPC = real interrupted -PC`, let the BIOS/kernel exception path (and the game's hooked handler) run, and -restore the host call-contract frame on RFE. Validate against the oracle -(psxref @4380) frame-by-frame. Diagnostic tooling in place: `PSX_EXIT_HALT`. - ---- - -## ✅✅ CONFIRMED with live data (2026-06-22, session 2) — was inference, now proof - -Reproduced the frame-1997 exit (`PSX_EXIT_HALT=1`, halt-and-serve) and read the real -state over TCP 4500. The prior section was an *inference from `last_store_pc`*; the -following is *measured*: - -**1. The exit sequence (from the `PSX_FNTRACE_ARM=all` dispatch-tail ring):** -``` -0x80085D1C game VBlank entry (sp -> 0x800ABD30, game stack) -0x000000B0 -> B0 vector, ra=0x80085F3C = ReturnFromException (B0:0x17) -0x80000048 RFE restored interrupted ctx (sp=0x801FFFB8 ra=0x80050CC8), pc=SENTINEL - -> psx_unknown_dispatch sentinel-longjmp gate FAILS -> pc=0 -> top exit -``` - -**2. `in_exception == 0` at the failing RFE — measured two independent ways:** -- `freeze_check`: `in_exception=0`, `post_exception_cooldown=7`, `current_func=0x000000B0`, - `last_store_pc=0x80086208`. -- The event ring's last entry is `IRQ_GATE detail=2` = `GATE_COOLDOWN`, which is only - set *after* `in_exception` is cleared. So the game's RFE ran in the post-exception - window, not inside our synthesized exception. - -**3. The TCB save area is the smoking gun.** `TCB ptr @0x108 = 0xA000E1EC` → -save_area `0xA000E1FC`: -``` -saved EPC (off 128) = 0x80000048 <-- OUR SENTINEL (the bug) -saved SR (off 140) = 0x40000404 (exception-pushed SR) -saved ra (off 124) = 0x80050CC8 (real interrupted ctx, correct) -saved sp (off 116) = 0x801FFFB8 (real interrupted ctx, correct) -``` -The interrupted GPRs are saved correctly; only **EPC is the sentinel** instead of the -real interrupted PC. `interrupts.c:362` writes the sentinel; the recompiled BIOS -handler saves it to the TCB; `ReturnFromException` reads it back and RFEs to the -sentinel. - -**4. The game handler is HookEntryInt-installed and self-RFEs (disasm of live RAM):** -`0x80085D8C` walks a per-source handler table (`jalr ra,v0` loop `0x80085E2C–E78`) and -ends with `jal 0x80086200`. `0x80086200 = addiu t2,0xB0; jr t2; addiu t1,0x17` = -B0:0x17 ReturnFromException; `0x80086220` = B0:0x19 HookEntryInt (the install site is -`0x80085D3C jal 0x80086220`). So Tomba 2 installs a full-takeover interrupt handler -that drives `ReturnFromException` itself. - -**5. A precise guest resume PC EXISTS and is being discarded.** The interrupted code is -dirty-interpreted (`event_ring` mode=INTERP, pc≈`0x80050CE8`). The dirty-interp pump -(`dirty_ram_interp.c:1377`) commits `cpu->pc` (the next guest PC) **before** calling -`psx_check_interrupts`. So at exception entry `cpu->pc` holds the real interrupted -guest PC — but `psx_check_interrupts` ignores it and writes the sentinel into EPC. - -### Root cause (one line) -Our exception model overloads COP0 EPC with the host sentinel `0x80000048`. Resume -works only via the host-GPR-save/restore + `longjmp` hack, which is valid **only -synchronously inside `psx_check_interrupts`**. Tomba 2's HookEntryInt handler calls -`ReturnFromException` **asynchronously** (after our exception window has closed, -`in_exception=0`); RFE reads `saved_epc=sentinel`, the longjmp gate fails, and the -sentinel resolves to `pc=0` → "execution completed" abnormal exit. - -### Fix design (recommended — pending ChatGPT consult + oracle grounding) -**Option A (scoped EPC de-overload — recommended):** when `psx_check_interrupts` -delivers an interrupt while the interrupted context is dirty-interpreted (a valid -guest resume PC is live in `cpu->pc`), set `EPC = cpu->pc` (real guest PC) instead of -the sentinel. The TCB then saves the real PC; `ReturnFromException` — synchronous OR -asynchronous — resumes at the real guest PC via the dispatch loop / dirty-interp -re-entry. Keep the sentinel + host-restore path ONLY for compiled-interrupted -contexts (where no precise guest PC exists; T1/MMX6/Ape gameplay handlers RFE -synchronously and are unaffected). The host-GPR-save/restore interplay on the -real-EPC path must be re-derived carefully. -**Option B:** pin *why* our model runs the game handler with `in_exception=0` and make -it run inside the exception (reuse the proven sync longjmp). Needs the deferral -mechanism nailed first. -**Gate:** this touches the shared interrupt path — re-regress Tomba 1 / MMX6 / Ape -frame-by-frame vs the oracle before landing on master. - -Consult package for the Recomp GPT: `docs/TOMBA2_EXC_CONSULT.md`. - ---- - -## ⚠️ CORRECTION (2026-06-22 session 2, impl pass) — it is NOT an async (in_exception=0) RFE - -Implementing the "Option A async-resume" idea and instrumenting it **refuted the -in_exception=0 framing above**. Measured (freeze_check counters added this pass): -`reach_dirty = 19921` (≈ one per exception), **`dirty_in_exc0 = 0`**, `async_rfe_fire = 0`. - -So the sentinel `0x80000048` is dispatched once per exception and **ALWAYS arrives at -the `dirty_ram_dispatch_inner` gate with `in_exception == 1`** — it always takes the -`psx_exception_longjmp()` branch. There is **no** `in_exception==0` dispatch of the -sentinel. (The earlier "in_exception=0 at the RFE" reading was the POST-exit state: -the botched longjmp landing clears `in_exception` and sets the cooldown gate.) - -**Corrected mechanism (longjmp-landing-depth, not async):** -- The game's HookEntryInt handler calls `ReturnFromException`; the recompiled BIOS RFE - restores the interrupted GPRs from the TCB and sets `cpu->pc = saved_epc = sentinel`. -- The trampoline dispatches the sentinel → `dirty_ram_dispatch_inner` sets `cpu->pc = 0` - then `psx_exception_longjmp()` (in_exception==1). -- The longjmp lands in `psx_check_interrupts`' `setjmp` loop (jmp_val==1), which restores - GPRs and **returns** — but `cpu->pc` is still 0. Control returns up through the pump - (`dirty_ram_dispatch` → trampoline). -- For ~19920 exceptions the longjmp lands at a **nested** dispatch (depth>0) and the - enclosing dirty-interp flow re-commits `cpu->pc`, so the game continues. -- For the failing one the longjmp unwinds to the **outermost** trampoline (depth 0), - where `cpu->pc == 0` → `psx_dispatch` returns → main.cpp "execution completed" exit. - -So the bug is: **a `ReturnFromException` longjmp can unwind to the OUTERMOST dispatch and -leave `cpu->pc = 0`, which the top loop reads as a clean exit.** The `g_psx_dispatch_depth` -fix (d10bc12) addressed depth desync but not this: the longjmp itself reaches `psx_dispatch`'s -outermost frame with a null PC. The single global `exception_jmpbuf` + the "resume via host -return" model is the fragile core (matches the consult's "host continuation state bleeding"). - -**Candidate fix (NOT yet implemented/validated):** when the RFE longjmp lands in -`psx_check_interrupts` (jmp_val==1) and `cpu->pc == 0`, set `cpu->pc = -g_async_rfe_resume_pc` (the latched real interrupted guest PC) before returning, so the -trampoline re-dispatches the guest instead of treating the null PC as an exit. MUST be -checked against the ~19920 normal sync RFEs (how do THEY restore the resume PC today? — -trace one normal landing vs the exit landing) and re-regressed on T1/MMX6/Ape. - -**Tooling added this pass (kept, runtime-only):** `freeze_check` now reports -`async_rfe_set/fire`, `reach_dirty/traps`, `reach_async`, `dirty_safe_resume_pc`, -`async_rfe_resume_pc`; `g_dirty_safe_resume_pc` latched around the 3 dirty pumps; -`g_async_rfe_resume_pc` latched at dirty-safe exception entry. The async-resume GATE code -(traps.c / dirty_ram_interp.c sentinel branches) is present but **never fires** under the -corrected model — it is scaffolding for the candidate fix, not a working fix. - ---- - -### Original investigation (history — superseded by the RESOLVED section above) - -## Symptom - -Tomba 2 (SCUS-94454) boots to the Whoopee-Camp publisher splash and **freezes there** -— it never rolls into the intro FMV (the oracle, Beetle PSX on the same disc + BIOS, -sails past it to ~frame 270k). With the interrupt-pump fix (`0a6920a`) the process no -longer hard-deadlocks, but the splash never advances and the process degrades to a -Windows "Not Responding" state within ~2 minutes (the main thread is saturated). - -## What it is NOT (ruled out with always-on heartbeat probes) - -All of the hardware-sim is **healthy** at the spin — verified, not assumed -(probes survive the degradation because they are read from the heartbeat file): - -- **Interrupts** (`75f1468`/`e1dce63`): VBLANK is **raised + delivered + ACKed exactly - 1/frame**; `irq_deliver` = 1/frame. Not an ack failure, not a delivery storm, not a - re-entry storm (`exc_reentry` growth is just the one slow interpreted handler being - polled, not actual nested entries). -- **Timer 1 / RootCounter 1**: free-run HBlank counter, `mode 0x1D00` (no IRQ - configured = correct), `t1_count` advances, `t1_irq_fired = 0` is correct. -- **CD-ROM**: idle / Pause (correct — the splash hasn't issued the FMV load yet). -- **Kernel events (EvCB)**: snapshot at base `0xA000E028`, 16 entries, **all status = - FREE** — no active event, so it is **not** a `WaitEvent`/`TestEvent` wait either. - -## Root cause: wild-call/return-contract corruption at the exception ⟷ B0 boundary - -The freeze is **not** an emulation bug — it is a **recompiler control-flow correctness -bug** in the wild-call/return-contract machinery (`psx_call_contract` in -`cpu_state.h`, and the generated `psx_dispatch_impl` trampoline). Same family as the -Bug A/C/D "wild call contract" freezes. - -The contract bails when a callee C-returns but the guest state doesn't match the call -site: `cpu->gpr[31] != stop_addr` **or** `cpu->gpr[29] != sp_at_call`. On a bail it -sets `cpu->pc = $ra` and "flattens" at the outermost dispatch (re-dispatches the wild -target on a clean host stack). - -Tomba 2's splash drives this **68,000 times per frame** (Tomba 1 at gameplay does -**zero** — so it is Tomba-2-specific, not a generic cost). Captured evidence: - -- **Bail source** (runtime-only capture via `fntrace.c`, `f1aaedd`): dominant bailing - function = **`0x000000B0`** (the **B0 kernel-syscall vector** / last compiled frame - on the boundary). -- **Trigger** (first bail, frame **1823** = spin onset): `0xB0` → wild target - **`0x8008AE50`**. Disassembly of the caller: - ``` - 0x8008AE48: jal 0x80085900 ; call - 0x8008AE4C: li $a0, -1 ; delay slot - 0x8008AE50: lui $v1, 0x8010 ; <- jal+8 return address == the "wild target" - ``` - So `0x8008AE50` is the **correct** return address of `jal 0x80085900` — the bail - target is a *legitimate outer-frame return*, not a corrupt pointer. Both - `0x80085900` and its caller are **high RAM (~0x80085000), OUTSIDE the recompiled - game text** (`0x80010000`–`0x80038800`) → runtime-loaded / install-at-runtime code. -- **Register state at the spin** (clean snapshot): - - `$ra = 0x00000001` — **corrupted** (degenerate dominant key has `$ra=$sp=1`). - - `$sp = 0x801FFF48`, `s2 = 0x8008AE50` (the outer return, saved callee-side), - `s3 = 0x8008ACFC`, `t1 = 0x35`. - - `k0 = 0xBFC0193C` (**BIOS exception handler**), `epc = 0x80000048` (our EPC - sentinel) → the spin runs in **exception-handler context**. - -### Mechanism (grounded model — binary truth + actual code + oracle-aware) - -**Binary truth (PS-EXE header):** `0x80085900` / `0x8008AE50` are **NOT in the boot -EXE** (`t_addr=0x80010000`, `t_size=0x28800`, file = exactly that). They are a **disc -overlay loaded to high RAM (~0x80085000) at runtime**, executed by the **dirty-RAM -interpreter** (the boot EXE is small; the bulk of Tomba 2 is disc overlays). - -So the freeze is at the **dirty-interp/overlay ⟷ compiled-BIOS boundary when an async -interrupt is delivered mid-call** — NOT a broken BIOS exception handler (that platform -infra works for Tomba 1 / MMX6 / Ape; per the recomp-debug "platform infra is almost -always correct" trap, doubting it was the mistake). My interrupt-pump fix (`0a6920a`) -is what now delivers interrupts *during* dirty execution, which exposes this. - -**Root mechanism (confirmed in interrupts.c:355-384):** exception delivery sets -`COP0_EPC = 0x80000048` — a **host longjmp sentinel, NOT the real interrupted PC** — -and relies on saving/restoring GPRs around the handler `return;`. The recompiled -handler RFEs by setting `cpu->pc = EPC = 0x80000048`; `psx_unknown_dispatch` -(traps.c:524) longjmps on that sentinel. **Overloading the guest EPC as a host token -collides with the BIOS's *architectural* use of EPC** (it saves/restores EPC via the -TCB). When the interrupt is delivered mid-dirty-interp/overlay-call, the GPR -save/restore + sentinel-RFE does not preserve the interrupted continuation: the -resumed code's `$ra` is clobbered to `1` (degenerate steady state: `$ra=$sp=1`). The -next return is then "wild" (`$ra != stop_addr`), the contract bails+flattens, -re-dispatches a wrong target, and it **degenerates into an infinite bail loop** = the -68k/frame spin. (Interrupt is healthy at the controller level — raised/delivered/acked -1/frame — yet the guest *continuation* after RFE is corrupted.) - -### CORRECTION (confirm tripwire, commit d965d6e) — it is NOT the exception path - -The `$ra->1` tripwire + first-bail `in_exception` latch **refuted the exception/EPC -model above** (data over theory, per recomp-debug): - -- `bail_first_in_exc = 0` — the `0xB0 -> 0x8008AE50` trigger bail is **NOT** during an - exception. -- `ra_tw`: site = INTERP, `in_exc = 0`, `pc = 0x8008B018`, `prev_ra = 0x8008AF10` -> 1. -- `0x8008B018` disassembles to **`lw $ra, 0x34($sp)`** — a normal function **epilogue** - restoring `$ra` from the stack. The loaded value is `1`, so the **saved `$ra` on the - stack is corrupt** (the stack/`$sp` is wrong), and this happens in **plain - non-exception execution**. - -**Corrected mechanism:** a B0 syscall (reached inside `jal 0x80085900` from caller F at -`0x8008AE48`) returns **non-locally** to F's continuation `0x8008AE50` (a tail-call -shape) — *not* an interrupt. Our bail/flatten handles the control transfer but leaves -the **stack / `$sp` wrong** (the skipped frame isn't unwound / `$sp` not restored). -Later F's epilogue `lw $ra, 0x34($sp)` reads a corrupt saved-`$ra` (`1`) → degenerate -`$ra=$sp=1` → infinite bail loop. So the real bug is a **control-flow / stack-contract -bug in the bail/flatten handling of a B0 non-local (tail-call) return** (recomp-debug -Class 3), NOT exception/RFE/EPC. The EPC-separation fix below is therefore the WRONG -fix and is retained only as a refuted hypothesis. - -### Revised fix direction (data-driven) - -The bail/flatten of a B0 (kernel-vector) non-local return from dirty/overlay code must -restore the **correct guest `$sp`** (and unwind the skipped frame) so the resumed -caller's stack is valid — OR the B0-vector tail-call must be dispatched as a tail -transfer (no spurious `stop_addr`) rather than a call whose contract bails. Next probe: -trace `$sp` and the `stop_addr` across the first B0 bail (latch them at the first bail), -and read what `0x80085900` does at its end (tail `j`/`jr 0xB0` vs `jal`). - -### (REFUTED) Fix design (ChatGPT-converged, Architecture-A-legal) - -1. **Do not overload guest EPC as a host longjmp token.** Keep the host - exception-return mechanism as *separate runtime metadata* - (`host_exception_return_kind` / `host_exception_jmp_target` / - `exception_resume_pc`). At entry set `EPC = real_interrupted_pc`, - `pc = exception_vector`, `in_exception=true` (let the BIOS do its TCB/EPC work - normally). At RFE restore per RFE semantics; **do not invent or use `$ra`/`$sp` as - host sentinels**; `restore_host_call_contract(frame)`. -2. The async interrupt path must either **fully suspend + restore the active host - call-contract frame** across the exception, or **defer delivery to a committed - guest safe point** (a valid resume point for an async HW interrupt — not HLE). - -### Confirm-first (do this BEFORE the fix) - -Single tripwire: latch the **exact moment `$ra` becomes `1`** (`old_ra != 1 && -gpr[31] == 1`), capturing site (interp insn / exception-exit / bail), PC, prev `$ra`, -`in_exception`, frame, full regs — surfaced in the heartbeat. That pins whether the -clobber is an overlay instruction, the exception-restore, or the bail-flatten. - -## Open questions (for the fix) - -1. Exact B0 function index at the trigger (`$t1` captured = `0x35` in the degenerate - state; candidate `B0:0x17 ReturnFromException` fits the non-local-return shape — - confirm by capturing `$t1` at the *first* bail, e.g. latch it in the heartbeat). -2. Precisely where `$ra` is clobbered to `1` — the exception push/RFE path - (`interrupts.c` / `traps.c psx_syscall` ReturnFromException / the EPC-sentinel - `0x80000048` handling) vs the bail-flatten re-dispatch. -3. Whether the first bail is itself a **false positive** (the contract wrongly flags a - legitimate non-local return) or a **genuine** wild return whose *resume* is what - corrupts state. The trigger target being a valid `jal+8` suggests the corruption is - in the resume, not the detection. - -## Fix direction - -The exception/RFE resume across a wild/dirty call boundary must restore the -interrupted frame's `$ra`/`$sp` so the continuation returns to its real call site -instead of being flagged wild. Likely in the recompiler call-contract + the -exception-delivery/RFE path (never in generated C). Design before implementing — the -wild-call-contract family is subtle and has regressed under pressure before. - -## Committed so far (`wt/tomba2`) - -- `0a6920a` interrupt-pump fix (low-RAM kernel loops) — the original hard deadlock. -- `75f1468` Timer1/RCnt1 heartbeat telemetry. -- `e1dce63` IRQ raise/deliver/ack + bail-source ledger telemetry. -- `f1aaedd` `fntrace.c` dispatch-level bail-source capture (runtime-only). diff --git a/docs/beetle-linux.md b/docs/beetle-linux.md new file mode 100644 index 00000000..3134315e --- /dev/null +++ b/docs/beetle-linux.md @@ -0,0 +1,46 @@ +# Building psx-beetle on Linux + +The psx-beetle oracle builds and runs on Linux with the same wire protocol +as on Windows. Verified 2026-07-02 (Kula World boots to demo gameplay; +ping/screenshot served over the TCP debug protocol). + +## beetle-psx checkout + +`runtime/src/beetle_libretro.cpp` targets the C++ beetle-psx tree. Upstream +master has since been converted to plain C (`libretro.cpp` -> `libretro.c`, +`PS_CPU` class dropped), which no longer compiles against our integration. +Pin the checkout to the last compatible commit — the same base +`docs/beetle_wtrace_hook.patch` was generated from: + +```bash +git clone https://github.com/libretro/beetle-psx-libretro.git beetle-psx +cd beetle-psx +git checkout 5759277b # "audit pass" — last C++-tree base we target +patch -p1 < ../docs/beetle_wtrace_hook.patch +patch -p1 < ../docs/beetle_sio_trace_hook.patch +patch -p1 < ../docs/beetle_cdcmd_trace_hook.patch +``` + +`beetle_cdcmd_trace_hook.patch` adds a CD-command trace callback (fires per +command dispatch in cdc.cpp) exposed as the `cdrom_cmd_dump` / `cdrom_cmd_reset` +debug commands — used to oracle-diff the Kula World CD read sequence. + +`beetle_sio_trace_hook.patch` adds `FrontIO::SetSIOTraceCallback` (fires per +completed SIO byte exchange) — an integration hook beetle_libretro.cpp needs +that predated the committed wtrace patch. + +## Build + +```bash +# Static lib. On unix the artifact is named mednafen_psx_libretro.so but is +# an ar archive (STATIC_LINKING=1); stage it under the name cmake expects. +make platform=unix STATIC_LINKING=1 HAVE_LIGHTREC=0 -j"$(nproc)" +cp mednafen_psx_libretro.so libmednafen_psx.a + +cd ../runtime +cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DPSX_LAUNCHER=OFF -DPSX_DEBUG_TOOLS=ON +ninja -C build psx-beetle + +# Headless run (picks the next free port from 4380 if taken; watch stderr) +xvfb-run -a ./runtime/build/psx-beetle ../bios/SCPH1001.BIN --disc <game.cue> +``` diff --git a/docs/beetle_cdcmd_trace_hook.patch b/docs/beetle_cdcmd_trace_hook.patch new file mode 100644 index 00000000..58f1da54 --- /dev/null +++ b/docs/beetle_cdcmd_trace_hook.patch @@ -0,0 +1,87 @@ +diff --git a/libretro.cpp b/libretro.cpp +index d2e82bca..e9c867c8 100644 +--- a/libretro.cpp ++++ b/libretro.cpp +@@ -508,6 +508,11 @@ PS_SPU *PSX_SPU = NULL; + PS_CDC *PSX_CDC = NULL; + FrontIO *PSX_FIO = NULL; + ++/* PSXRecomp oracle hooks: NULL when no listener attached. */ ++psxrecomp_wtrace_cb_t g_psxrecomp_wtrace_cb = NULL; ++psxrecomp_fntrace_cb_t g_psxrecomp_fntrace_cb = NULL; ++psxrecomp_cdcmd_cb_t g_psxrecomp_cdcmd_cb = NULL; ++ + MultiAccessSizeMem<512 * 1024, uint32, false> *BIOSROM = NULL; + MultiAccessSizeMem<65536, uint32, false> *PIOMem = NULL; + MultiAccessSizeMem<2048 * 1024, uint32, false> *MainRAM = NULL; +diff --git a/mednafen/psx/cdc.cpp b/mednafen/psx/cdc.cpp +index 1daaa5b7..a6b31b66 100644 +--- a/mednafen/psx/cdc.cpp ++++ b/mednafen/psx/cdc.cpp +@@ -1318,6 +1318,16 @@ int32_t PS_CDC::Update(const int32_t timestamp) + else + { + const CDC_CTEntry *command = &Commands[PendingCommand]; ++ /* PSXRecomp oracle hook: CD command about to execute. ++ * PC of the guest at dispatch time — the command was ++ * WRITTEN earlier (latency), so this is the poll/idle ++ * PC, but it still localizes kernel vs shell vs game. */ ++ if(MDFN_UNLIKELY(g_psxrecomp_cdcmd_cb)) ++ g_psxrecomp_cdcmd_cb(PendingCommand, ArgsReceiveIn, ++ ArgsReceiveIn > 0 ? ArgsReceiveBuf[0] : 0, ++ ArgsReceiveIn > 1 ? ArgsReceiveBuf[1] : 0, ++ ArgsReceiveIn > 2 ? ArgsReceiveBuf[2] : 0, ++ PSX_CPU ? PSX_CPU->GetRegister(PS_CPU::GSREG_PC, NULL, 0) : 0); + next_time = (this->*(command->func))(ArgsReceiveIn, ArgsReceiveBuf); + PendingCommandPhase = 2; + } +diff --git a/mednafen/psx/psx.h b/mednafen/psx/psx.h +index 704c0a19..fc53362d 100644 +--- a/mednafen/psx/psx.h ++++ b/mednafen/psx/psx.h +@@ -137,4 +137,45 @@ extern unsigned psx_gpu_overclock_shift; + extern uint8_t analog_combo[2]; + extern uint8_t analog_combo_hold; + ++/* PSXRecomp oracle: SIO byte trace callback. ++ * Fired by FrontIO when a complete byte exchange finishes. */ ++class FrontIO; ++extern FrontIO *PSX_FIO; ++typedef void (*psxrecomp_sio_trace_cb_t)(uint8_t tx, uint8_t rx, uint16_t ctrl); ++ ++/* PSXRecomp oracle: RAM write trace callback. ++ * Fired by PS_CPU::SB/SH/SW after the write completes. ++ * addr = virtual address (caller masks for physical filter) ++ * value = stored value (sized by `size`) ++ * pc = PC of the executing store instruction ++ * ra = $ra (GPR[31]) at time of store, for one frame of caller context ++ * size = 1, 2, or 4 (byte / halfword / word) */ ++typedef void (*psxrecomp_wtrace_cb_t)(uint32_t addr, uint32_t value, ++ uint32_t pc, uint32_t ra, uint8_t size); ++extern psxrecomp_wtrace_cb_t g_psxrecomp_wtrace_cb; ++ ++/* PSXRecomp oracle: function-call (JAL/JALR/J/JR) trace callback. ++ * Fired by PS_CPU on every J/JAL/JR/JALR retire, BEFORE the link write. ++ * Caller filters by target_pc to keep volume bounded. ++ * caller_pc = PC of the J/JAL/JR/JALR instruction ++ * target_pc = computed branch target ++ * parent_ra = GPR[31] at time of call (parent's return address) ++ * a0, a1 = argument registers at time of call ++ * kind = 1=J, 2=JAL, 3=JR, 4=JALR */ ++typedef void (*psxrecomp_fntrace_cb_t)(uint32_t caller_pc, uint32_t target_pc, ++ uint32_t parent_ra, uint32_t a0, ++ uint32_t a1, uint8_t kind); ++extern psxrecomp_fntrace_cb_t g_psxrecomp_fntrace_cb; ++ ++/* PSXRecomp oracle: CD command trace callback. ++ * Fired by PS_CDC just before a command handler executes. ++ * cmd = command byte (0x01 GetStat, 0x02 Setloc, 0x06 ReadN, 0x09 Pause, ++ * 0x0A Init, 0x0E Setmode, 0x15 SeekL, 0x1A GetID, 0x1B ReadS, ...) ++ * nargs = number of argument bytes ++ * arg0..arg2 = first three argument bytes (0 if absent) */ ++typedef void (*psxrecomp_cdcmd_cb_t)(uint8_t cmd, uint8_t nargs, ++ uint8_t arg0, uint8_t arg1, uint8_t arg2, ++ uint32_t pc); ++extern psxrecomp_cdcmd_cb_t g_psxrecomp_cdcmd_cb; ++ + #endif diff --git a/docs/beetle_sio_trace_hook.patch b/docs/beetle_sio_trace_hook.patch new file mode 100644 index 00000000..935b5315 --- /dev/null +++ b/docs/beetle_sio_trace_hook.patch @@ -0,0 +1,35 @@ +diff --git a/mednafen/psx/frontio.cpp b/mednafen/psx/frontio.cpp +index a35b022f..02e7ce0b 100644 +--- a/mednafen/psx/frontio.cpp ++++ b/mednafen/psx/frontio.cpp +@@ -783,6 +783,10 @@ int32_t FrontIO::Update(int32_t timestamp) + ReceiveInProgress = false; + ReceiveBufferAvail = true; + ++ /* PSXRecomp oracle hook: byte exchange complete. */ ++ if(MDFN_UNLIKELY(sio_trace_cb)) ++ sio_trace_cb(TransmitBuffer, ReceiveBuffer, (uint16_t)Control); ++ + if(Control & 0x800) + { + istatus = true; +diff --git a/mednafen/psx/frontio.h b/mednafen/psx/frontio.h +index 8a67f5af..d9a260ca 100644 +--- a/mednafen/psx/frontio.h ++++ b/mednafen/psx/frontio.h +@@ -95,8 +95,15 @@ class FrontIO + + int StateAction(StateMem* sm, int load, int data_only); + ++ /* PSXRecomp oracle hook: fires once per completed SIO byte exchange ++ * (tx byte sent, rx byte received, current Control register). */ ++ typedef void (*sio_trace_cb_t)(uint8_t tx, uint8_t rx, uint16_t ctrl); ++ void SetSIOTraceCallback(sio_trace_cb_t cb) { sio_trace_cb = cb; } ++ + private: + ++ sio_trace_cb_t sio_trace_cb = nullptr; ++ + void DoDSRIRQ(void); + void CheckStartStopPending(int32_t timestamp, bool skip_event_set = false); + diff --git a/docs/config_schema.md b/docs/config_schema.md index 0588ee30..c7eab4c1 100644 --- a/docs/config_schema.md +++ b/docs/config_schema.md @@ -1,14 +1,13 @@ # PSXRecomp v4 — config TOML schema Consumed by: -- `psxrecomp-bios.exe` and `psxrecomp-game.exe` (once Phase A2/A3 ships - `--config <toml>` support) +- `psxrecomp-bios.exe` and `psxrecomp-game.exe` (via `--config <toml>`) - `tools/audit_config.py` and the audit tooling that wraps it - The runtime cmake macro (`runtime/runtime.cmake`) Examples: - `bios/SCPH1001.toml` — BIOS-only config (no game; psx-runtime targets this) -- `F:/Projects/TombaRecomp/game.toml` — game config (tomba-runtime targets this) +- `../TombaRecomp/game.toml` — game config (tomba-runtime targets this) ## How configs combine diff --git a/ACCURACY_BURNDOWN.md b/docs/internal/ACCURACY_BURNDOWN.md similarity index 91% rename from ACCURACY_BURNDOWN.md rename to docs/internal/ACCURACY_BURNDOWN.md index f5dc975e..fb8ae9ad 100644 --- a/ACCURACY_BURNDOWN.md +++ b/docs/internal/ACCURACY_BURNDOWN.md @@ -169,6 +169,25 @@ bug**, not timing. Beetle spu.cpp; psx-spx "SPU"; validate by audio-sample diff. - [ ] **CDROM**: command set, response timing, sector read (data/XA/CD-DA), seek, shell/lid — Beetle cdrom.cpp; psx-spx "CDROM"; validate by sector/response diff. + - **KNOWN OK-TO-DIVERGE (we are MORE faithful than the oracle) — from PR #9:** + Suppressed in `tools/devtrace_diff.py` (`KNOWN_DIVERGENCES["cdrom"]`; use + `--strict` to see raw). Beetle stays the independent oracle (unpatched); the + tool LABELS these deltas, it does not silence real ones. + - **CD controller version (Test `19h`,`20h`)**: we return `94 09 19 C0` (real + SCPH-1001 sub-CPU, 1994-09-19, per psx-spx "CDROM Test Commands"). Beetle + hardcodes `97 01 10 C2` (`cdc.cpp:2253`, a later PSone board) regardless of + BIOS. Version byte `< 95h` keeps shell CD-init flag `[0xA000DFFC]` clear; + `>= 95h` sets it and forces a spurious boot ReadTOC. So native (94h) issues + NO ReadTOC where Beetle (97h) does → expect a CDROM event delta at CD-init, + possibly cascading briefly. Verified vs psx-spx, NOT the oracle. MMX6+Tomba + soak PASSED (2026-07-06). Trivially revertable (4 bytes, `cdrom.c` Test 0x20). + - **Status reg `0x1F801800` bit 2 (ADPBUSY)**: idles at 0 (was wrongly pinned + set under an "ADPCM empty" label). psx-spx: bit2 = XA playback → 0 when idle. + - **GetID license region**: derived from disc serial (`SCEA`/`SCEE`/`SCEI`) + instead of hardcoded `SCEI`; NTSC-U (MMX6/Tomba) now correctly report `SCEA`. + - Method reminder (top-of-doc LESSON): these were HYPOTHESES until OUTPUT + validation — the MMX6/Tomba playtest soak WAS that validation. Revert any + that regress a shipped title. - [ ] **DMA**: all 7 channels, block/linked-list/chain modes, timing, DICR/DPCR — Beetle dma.cpp; psx-spx "DMA". - [ ] **MDEC**: macroblock decode, IDCT, color conversion, RLE — Beetle mdec.cpp; diff --git a/BACKDROP_PRELOAD.md b/docs/internal/BACKDROP_PRELOAD.md similarity index 100% rename from BACKDROP_PRELOAD.md rename to docs/internal/BACKDROP_PRELOAD.md diff --git a/COSIM_ORACLE.md b/docs/internal/COSIM_ORACLE.md similarity index 100% rename from COSIM_ORACLE.md rename to docs/internal/COSIM_ORACLE.md diff --git a/CYCLE_MODEL_BEETLE.md b/docs/internal/CYCLE_MODEL_BEETLE.md similarity index 100% rename from CYCLE_MODEL_BEETLE.md rename to docs/internal/CYCLE_MODEL_BEETLE.md diff --git a/FAITHFUL_TIMING_PLAN.md b/docs/internal/FAITHFUL_TIMING_PLAN.md similarity index 100% rename from FAITHFUL_TIMING_PLAN.md rename to docs/internal/FAITHFUL_TIMING_PLAN.md diff --git a/FIRST_MILESTONE.md b/docs/internal/FIRST_MILESTONE.md similarity index 100% rename from FIRST_MILESTONE.md rename to docs/internal/FIRST_MILESTONE.md diff --git a/GL_RENDERER_HANDOFF.md b/docs/internal/GL_RENDERER_HANDOFF.md similarity index 100% rename from GL_RENDERER_HANDOFF.md rename to docs/internal/GL_RENDERER_HANDOFF.md diff --git a/HLE_SCHEDULER_CARVEOUT_PLAN.md b/docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md similarity index 100% rename from HLE_SCHEDULER_CARVEOUT_PLAN.md rename to docs/internal/HLE_SCHEDULER_CARVEOUT_PLAN.md diff --git a/NATIVE_WIDE_PLAN.md b/docs/internal/NATIVE_WIDE_PLAN.md similarity index 100% rename from NATIVE_WIDE_PLAN.md rename to docs/internal/NATIVE_WIDE_PLAN.md diff --git a/PLAN.md b/docs/internal/PLAN.md similarity index 100% rename from PLAN.md rename to docs/internal/PLAN.md diff --git a/PRECISE_IRQ_SLICE.md b/docs/internal/PRECISE_IRQ_SLICE.md similarity index 100% rename from PRECISE_IRQ_SLICE.md rename to docs/internal/PRECISE_IRQ_SLICE.md diff --git a/docs/internal/README.md b/docs/internal/README.md new file mode 100644 index 00000000..5ab8d0de --- /dev/null +++ b/docs/internal/README.md @@ -0,0 +1,42 @@ +# Internal dev-logs & design notes + +These are the working design documents and burn-down logs the project was built +from. They are **historical/working notes**, not contributor onboarding docs — +they capture decisions, phased plans, and deep subsystem investigations in the +order they happened, and some sections are superseded by later ones. + +If you're new here, start with the top-level [`README.md`](../../README.md), +then [`docs/ARCHITECTURE.md`](../ARCHITECTURE.md), +[`docs/EXECUTION_MODEL.md`](../EXECUTION_MODEL.md), +[`docs/BUILDING.md`](../BUILDING.md), and +[`CONTRIBUTING.md`](../../CONTRIBUTING.md). Come here when you need the *why* +behind a specific subsystem. + +## Plans & roadmap +- [`PLAN.md`](PLAN.md) — the original phased build plan (P1–P6). +- [`FAITHFUL_TIMING_PLAN.md`](FAITHFUL_TIMING_PLAN.md) — the authoritative + cycle-accuracy north-star plan and status log. +- [`FIRST_MILESTONE.md`](FIRST_MILESTONE.md) — the first-boot milestone gate. +- [`STUBS_TO_FIX.md`](STUBS_TO_FIX.md) — hardware-stub burn-down (pre-game gate). + +## Cycle accuracy & timing +- [`PRECISE_IRQ_SLICE.md`](PRECISE_IRQ_SLICE.md) — exact-instruction IRQ delivery. +- [`CYCLE_MODEL_BEETLE.md`](CYCLE_MODEL_BEETLE.md) — the Beetle-derived cycle model. +- [`ACCURACY_BURNDOWN.md`](ACCURACY_BURNDOWN.md) — full accuracy burn-down across + all axes (semantics, cycle, IRQ, MMIO, peripherals, determinism). +- [`COSIM_ORACLE.md`](COSIM_ORACLE.md) — the first-divergence co-sim oracle. + +## BIOS / scheduler +- [`HLE_SCHEDULER_CARVEOUT_PLAN.md`](HLE_SCHEDULER_CARVEOUT_PLAN.md) — the + cooperative-thread scheduler HLE carve-out. +- [`WEDGE_load4_shell_rootcause.md`](WEDGE_load4_shell_rootcause.md) — a specific + boot-shell wedge root-cause writeup. + +## Renderer / widescreen +- [`GL_RENDERER_HANDOFF.md`](GL_RENDERER_HANDOFF.md) — OpenGL renderer design. +- [`NATIVE_WIDE_PLAN.md`](NATIVE_WIDE_PLAN.md) — native-wide 16:9 (GTE FOV) plan. +- [`BACKDROP_PRELOAD.md`](BACKDROP_PRELOAD.md) — widescreen backdrop preload. + +## Overlay cache internals +- [`SLJIT_PERSIST_CACHE.md`](SLJIT_PERSIST_CACHE.md) — persisted overlay-shard + cache design. diff --git a/SLJIT_PERSIST_CACHE.md b/docs/internal/SLJIT_PERSIST_CACHE.md similarity index 100% rename from SLJIT_PERSIST_CACHE.md rename to docs/internal/SLJIT_PERSIST_CACHE.md diff --git a/STUBS_TO_FIX.md b/docs/internal/STUBS_TO_FIX.md similarity index 100% rename from STUBS_TO_FIX.md rename to docs/internal/STUBS_TO_FIX.md diff --git a/WEDGE_load4_shell_rootcause.md b/docs/internal/WEDGE_load4_shell_rootcause.md similarity index 100% rename from WEDGE_load4_shell_rootcause.md rename to docs/internal/WEDGE_load4_shell_rootcause.md diff --git a/psxrecomp-toml-readme.md b/psxrecomp-toml-readme.md new file mode 100644 index 00000000..d182b438 --- /dev/null +++ b/psxrecomp-toml-readme.md @@ -0,0 +1,39 @@ +# psxrecomp-toml 🛠️🎮 + +`psxrecomp-toml` is a companion tool designed to streamline the configuration process for `psxrecomp` static recompilation projects. It analyzes a PlayStation 1 executable (`psx-exe`), automatically extracts key deployment metadata, and generates a valid `game.toml` configuration along with initial function jump seeds (`JAL` targets). + +No more digging around with Hex Editors or Ghidra just to find the entry point, load addresses, and basic text sizing. + +## Features + +- **Automated `game.toml` Assembly:** Dumps necessary headers including `load_address`, `entry_pc`, and `text_size` based on the binary's actual PS-X EXE structure. +- **Static Discovery Disambiguation:** Scans for standard jump-and-link (`JAL`) destination blocks. +- **Coverage Tuning:** Includes edge-case address mapping right after subroutine returns (`jr $ra`) to capture maximum layout structure out-of-the-box. + +--- + +Usage: psxrecomp-toml <PS1-EXE> [options] + +Options: + --output <path> Write game.toml to <path> (default: stdout) + --seeds <path> Also write JAL-target seed file to <path> + --name <str> Game name in TOML (default: derived from EXE) + --id <str> Game ID (default: auto-detect or empty) + --stdout Force output to stdout even with --output + --include-after-return Add addresses after jr $ra to seeds (more coverage, + may include some data addresses) + -h, --help Show this help + +### Example Workflow + +psxrecomp-toml ./isos/SCES_028.34 \ + --output ./projects/CrashBash/game.toml \ + --seeds ./projects/CrashBash/recompiler/seeds/seeds.txt \ + --name "Crash-Bash-EUR" + + +#### ⚠️ A Note on Seed Coverage + +While psxrecomp-toml does the heavy lifting by scanning code segments and extracting immediate jump addresses for you, it does not statically resolve 100% of the game's execution paths (such as dynamic function tables or indirect register dispatches). + +If the recompiler throws dynamic discovery gaps or unknown dispatches later during active runtime simulation, you will still need to feed those newly discovered addresses back into your seeds.txt manually to close the gap. diff --git a/recompiler/CMakeLists.txt b/recompiler/CMakeLists.txt index cd882e1f..c146d7e8 100644 --- a/recompiler/CMakeLists.txt +++ b/recompiler/CMakeLists.txt @@ -89,6 +89,20 @@ if(MINGW OR WIN32) target_link_options(psxrecomp-game PRIVATE -Wl,--stack,16777216) endif() +# psxrecomp-toml — PS-X EXE → game.toml generator. +# Standalone tool that analyses a PS-X EXE and generates a complete game.toml +# config with auto-detected fields (load_address, entry_pc, text_size, +# stack_base) and optional JAL-target seed generation. +add_executable(psxrecomp-toml + src/ps1_exe_parser.cpp + src/main_toml.cpp +) +target_include_directories(psxrecomp-toml PRIVATE + include + lib/fmt/include +) +target_link_libraries(psxrecomp-toml PRIVATE fmt) + # ---- baked codegen hash (stale-binary guard) -------------------------------- # Bake the SAME emitter-source hash the runtime folds into its overlay cache # tag (runtime/runtime.cmake -> hash_codegen.cmake, shared canonical list in diff --git a/recompiler/include/code_generator.h b/recompiler/include/code_generator.h index c526eb74..4c2795fa 100644 --- a/recompiler/include/code_generator.h +++ b/recompiler/include/code_generator.h @@ -50,6 +50,10 @@ struct CodeGenConfig { // proportion correction at GP0 submission. Empty = no hooks (default). std::set<uint32_t> ws_sprite_tag_funcs; + // MMX6-class 2D tile-ring stage initializer. Emit a reveal-invalidation + // callback at entry so host-only wide pixels cannot survive a stage swap. + uint32_t ws_bg2d_init_func = 0; + // Widescreen far-backdrop un-squash ([widescreen] backdrop_unsquash_funcs): // functions whose body is bracketed by gte_ws_set_suppress(1)/(0) so the // GTE X-squash is OFF for their (far-backdrop) draws — the backdrop fills @@ -67,6 +71,17 @@ struct CodeGenConfig { std::set<uint32_t> ws_cull_a1_sites; std::set<uint32_t> ws_cull_screen_x_sites; + // Explicit signed right-edge widen sites ([widescreen.cull] slti_sites): + // `slti rt, sx, W` emitted through psx_ws_cull_slti for funnel functions + // the auto-detector cannot qualify (X-only test, no height compare). + std::set<uint32_t> ws_cull_slti_sites; + + // Screen-extent signature immediates ([widescreen.cull] screen_w_imms / + // screen_h_imms) — per-game display-width-derived bounds. Defaults are the + // Tomba signature; Ape Escape uses 0x181 (+ 0xF1 height). + std::vector<uint32_t> ws_cull_w_imms = { 0x140, 0x141 }; + std::vector<uint32_t> ws_cull_h_imms = { 0xE0, 0xF1 }; + // Widescreen automatic horizontal-FOV cull widening ([widescreen.cull] // auto_screen_x). When true, any function carrying the GTE screen-extent // reject signature — at least one `sltiu …,0x140/0x141` (right/width) AND @@ -233,6 +248,11 @@ private: // translate_instruction to widen that function's width compares. See // func_has_screen_extent_cull(). bool ws_auto_cull_func_ = false; + // LEFT-edge funnel bltz addresses in the current function (auto_screen_x, + // signed min/max + center±halfwidth idioms — ws_cull_detect.h). Populated + // per function alongside ws_auto_cull_func_; generate_branch_condition + // emits a listed bltz through psx_ws_cull_bltz. Cleared per function. + std::set<uint32_t> ws_cull_bltz_pcs_; // True iff the function's instruction stream contains both a screen-width // reject (`sltiu …,0x140` or `…,0x141`) and a screen-height reject @@ -250,6 +270,9 @@ private: // Populate ws_backdrop_sites_ from the shared psx_ws_find_backdrop_windows() // detector over this function's instruction words; returns the window count. int detect_backdrop_windows(const ControlFlowGraph& cfg); + // Populate ws_cull_bltz_pcs_ (LEFT-edge funnel bltz addresses) for the + // current function; no-op unless ws_auto_cull_func_ is set. + int detect_cull_bltz_sites(const ControlFlowGraph& cfg); // Instruction translation std::string translate_instruction(uint32_t addr, uint32_t instr); @@ -270,7 +293,7 @@ private: std::map<uint32_t, std::vector<uint32_t>>& out_edges); // Control flow translation - std::string generate_branch_condition(uint32_t instr); + std::string generate_branch_condition(uint32_t instr, uint32_t addr); std::string translate_branch(const ControlFlowInstr& cf, uint32_t fall_through); std::string translate_jump(const ControlFlowInstr& cf); diff --git a/recompiler/src/code_generator.cpp b/recompiler/src/code_generator.cpp index a93b39fe..1c3b3443 100644 --- a/recompiler/src/code_generator.cpp +++ b/recompiler/src/code_generator.cpp @@ -5,6 +5,7 @@ // included via relative path to avoid an include-dir collision (recompiler and // runtime both ship a gte.h). #include "../../runtime/include/ws_backdrop_detect.h" +#include "../../runtime/include/ws_cull_detect.h" #include "../../runtime/include/psx_instr_cost.h" /* single-source CPU cycle cost (shared with interp) */ #include <fmt/format.h> #include <algorithm> @@ -699,7 +700,7 @@ std::string CodeGenerator::translate_mtlo(uint32_t instr) { return fmt::format("cpu->lo = {};", reg_name(rs)); } -std::string CodeGenerator::generate_branch_condition(uint32_t instr) { +std::string CodeGenerator::generate_branch_condition(uint32_t instr, uint32_t addr) { uint32_t opcode = (instr >> 26) & 0x3F; uint32_t rs = get_rs(instr); uint32_t rt = get_rt(instr); @@ -708,6 +709,11 @@ std::string CodeGenerator::generate_branch_condition(uint32_t instr) { if (opcode == 0x01) { uint32_t regimm_op = (instr >> 16) & 0x1F; if (regimm_op == 0x00) { // bltz + // Classified LEFT-edge funnel bltz (auto_screen_x, signed idioms): + // reject only past the revealed margin. Identity at 4:3 (margin 0). + if (ws_cull_bltz_pcs_.count(addr)) + return fmt::format("psx_ws_cull_bltz({}) /* ws auto screen-x cull (left edge) */", + reg_name(rs)); return fmt::format("(int32_t){} < 0", reg_name(rs)); } else if (regimm_op == 0x01) { // bgez return fmt::format("(int32_t){} >= 0", reg_name(rs)); @@ -861,20 +867,45 @@ std::string CodeGenerator::translate_instruction(uint32_t addr, uint32_t instr) // than culled at the 320 boundary. Same shape as an explicit range_site but // applied by signature — no per-address list. (Reached only when the addr is // not already an explicit cull site, which returns above.) - if (ws_auto_cull_func_ && opcode == 0x0B) { // sltiu + if (ws_auto_cull_func_ && (opcode == 0x0B || opcode == 0x0A)) { // sltiu / slti uint16_t uimm = get_imm16_u(instr); - if (uimm == 0x140 || uimm == 0x141) { + if (psx_ws_cull_imm_in(uimm, config_.ws_cull_w_imms.data(), + (int)config_.ws_cull_w_imms.size())) { uint32_t rs = get_rs(instr), rt = get_rt(instr); - // Route through the shared runtime helper psx_ws_cull_sltiu so the - // gcc emit, the sljit JIT, and the interpreter all widen identically. - // It sign-extends SX and shifts by +margin (wide window - // -margin <= SX < imm+margin, both edges); at margin=0 it reduces - // bit-for-bit to the vanilla `SX <u imm` verdict (4:3 byte-identical). - return fmt::format("{} = psx_ws_cull_sltiu({}, {});" - " /* ws auto screen-x cull (both edges) */{}", + if (opcode == 0x0B) { + // Route through the shared runtime helper psx_ws_cull_sltiu so the + // gcc emit, the sljit JIT, and the interpreter all widen identically. + // It sign-extends SX and shifts by +margin (wide window + // -margin <= SX < imm+margin, both edges); at margin=0 it reduces + // bit-for-bit to the vanilla `SX <u imm` verdict (4:3 byte-identical). + return fmt::format("{} = psx_ws_cull_sltiu({}, {});" + " /* ws auto screen-x cull (both edges) */{}", + reg_name(rt), reg_name(rs), (int)uimm, comment); + } + // Signed min/max funnel: `slti v, minSX, W` is the RIGHT edge only + // (the paired left edge is the classified bltz — see + // generate_branch_condition); widen by +margin. + return fmt::format("{} = psx_ws_cull_slti({}, {});" + " /* ws auto screen-x cull (right edge) */{}", reg_name(rt), reg_name(rs), (int)uimm, comment); } } + // Explicit signed right-edge widen site ([widescreen.cull] slti_sites) for + // funnel functions the auto-detector cannot qualify (X-only, no H compare). + if (config_.ws_cull_slti_sites.count(addr)) { + if (opcode == 0x0A) { // slti + uint32_t rs = get_rs(instr), rt = get_rt(instr); + uint16_t uimm = get_imm16_u(instr); + return fmt::format("{} = psx_ws_cull_slti({}, {});" + " /* ws cull slti site (right edge) */{}", + reg_name(rt), reg_name(rs), (int)uimm, comment); + } else if (!config_.overlay_mode) { + fmt::print(stderr, "ERROR: [widescreen.cull] slti site 0x{:08X} is not " + "slti (opcode 0x{:02X})\n", addr, opcode); + std::exit(1); + } + // overlay variant: addr is different code here — fall through to vanilla. + } // Widescreen backdrop screenX squash ([widescreen.backdrop] x_sites). The // site is the `sh rt,off(base)` storing a parallax 2D backdrop layer's // final screen-X; squash the stored value around the screen centre so the @@ -1483,7 +1514,7 @@ std::string CodeGenerator::translate_basic_block( // For branch-likely variants, delay slot is conditional if (block.exit_instr.is_likely) { ss << config_.indent << "/* delay slot (likely) - conditional execution */\n"; - ss << config_.indent << "if (" << generate_branch_condition(block.exit_instr.instruction) << ") {\n"; + ss << config_.indent << "if (" << generate_branch_condition(block.exit_instr.instruction, block.exit_instr.address) << ") {\n"; ss << config_.indent << translate_instruction(delay_slot_addr, delay_instr) << "\n"; emit_cosim_instr(delay_slot_addr, config_.indent + config_.indent); ss << config_.indent << "}\n"; @@ -1493,7 +1524,7 @@ std::string CodeGenerator::translate_basic_block( // instruction (before the delay slot). If the delay slot // modifies a condition register we must save the result first. if (block.exit_instr.type == ControlFlowType::Branch) { - std::string cond = generate_branch_condition(block.exit_instr.instruction); + std::string cond = generate_branch_condition(block.exit_instr.instruction, block.exit_instr.address); delay_saved_cond = fmt::format("_bc_{:08X}", addr); ss << config_.indent << fmt::format("int {} = ({}); /* save branch cond before delay slot */\n", @@ -1528,7 +1559,7 @@ std::string CodeGenerator::translate_basic_block( } // Conditional branch - use pre-captured condition if available std::string condition = delay_saved_cond.empty() - ? generate_branch_condition(block.exit_instr.instruction) + ? generate_branch_condition(block.exit_instr.instruction, block.exit_instr.address) : delay_saved_cond; if (block.successors.size() == 2) { ss << config_.indent << "if (" << condition << ") {\n"; @@ -1945,10 +1976,11 @@ std::string CodeGenerator::translate_basic_block( } bool CodeGenerator::func_has_screen_extent_cull(const ControlFlowGraph& cfg) const { - // The GTE per-vertex trivial-reject pairs a width compare (sltiu …,0x140 or - // inclusive 0x141) with a height compare (sltiu …,0xE0 or 0xF1) in the same - // function. Presence of both is the signature of a screen-extent render - // funnel; a lone width compare elsewhere (rare) is left alone. + // The GTE trivial-reject pairs a width compare (slti/sltiu, immediate from + // the per-game W set) with a height compare (immediate from the H set) in + // the same function. Presence of both is the signature of a screen-extent + // render funnel; a lone width compare elsewhere (rare) is left alone. + // Shared scan (ws_cull_detect.h) so every backend derives one verdict. bool has_x = false, has_y = false; for (uint32_t block_addr : cfg.block_order) { const BasicBlock& block = cfg.blocks.at(block_addr); @@ -1956,16 +1988,44 @@ bool CodeGenerator::func_has_screen_extent_cull(const ControlFlowGraph& cfg) con auto io = exe_.read_word(a); if (!io.has_value()) continue; uint32_t in = *io; - if ((in & 0xFC000000u) != 0x2C000000u) continue; // not sltiu - uint16_t im = (uint16_t)(in & 0xFFFF); - if (im == 0x140 || im == 0x141) has_x = true; - else if (im == 0xE0 || im == 0xF1) has_y = true; + uint32_t op = in >> 26; + if (op != 0x0Au && op != 0x0Bu) continue; // not slti/sltiu + uint32_t im = in & 0xFFFFu; + if (psx_ws_cull_imm_in(im, config_.ws_cull_w_imms.data(), + (int)config_.ws_cull_w_imms.size())) has_x = true; + else if (psx_ws_cull_imm_in(im, config_.ws_cull_h_imms.data(), + (int)config_.ws_cull_h_imms.size())) has_y = true; if (has_x && has_y) return true; } } return false; } +int CodeGenerator::detect_cull_bltz_sites(const ControlFlowGraph& cfg) { + // Populate ws_cull_bltz_pcs_ with the function's LEFT-edge funnel bltz + // addresses (ws_cull_detect.h idioms 2/3). Only meaningful when the + // function already qualified (ws_auto_cull_func_). Build a contiguous word + // image over the function range (same approach as detect_backdrop_windows) + // so the structural classifier sees cross-block context. + ws_cull_bltz_pcs_.clear(); + if (!ws_auto_cull_func_) return 0; + uint32_t lo = cfg.function_start, hi = cfg.function_end; + if (hi <= lo || (hi - lo) > 0x40000u) return 0; // sanity bound + std::vector<uint32_t> words; + words.reserve((hi - lo) / 4u); + for (uint32_t a = lo; a < hi; a += 4) { + auto io = exe_.read_word(a); + words.push_back(io.has_value() ? *io : 0u); + } + int n = (int)words.size(); + for (int i = 0; i < n; i++) + if (psx_ws_cull_bltz_here(words.data(), n, i, + config_.ws_cull_w_imms.data(), + (int)config_.ws_cull_w_imms.size())) + ws_cull_bltz_pcs_.insert(lo + (uint32_t)i * 4u); + return (int)ws_cull_bltz_pcs_.size(); +} + int CodeGenerator::detect_backdrop_windows(const ControlFlowGraph& cfg) { // Always clear so a stale map from the previous function can never leak. ws_backdrop_sites_.clear(); @@ -2003,9 +2063,11 @@ GeneratedFunction CodeGenerator::generate_function( // Widescreen auto cull (gated): detect the screen-extent reject signature so // translate_instruction widens this function's width compares. Cleared for - // every function so it never leaks across functions. + // every function so it never leaks across functions. The LEFT-edge funnel + // bltz sites (signed idioms) are classified alongside. ws_auto_cull_func_ = config_.ws_auto_screen_x_cull && func_has_screen_extent_cull(cfg); + detect_cull_bltz_sites(cfg); // Widescreen backdrop preload (gated): detect each scrolling-backdrop column // window so translate_instruction rewrites its START/END finalize. Cleared + @@ -2119,6 +2181,10 @@ GeneratedFunction CodeGenerator::generate_function( body_ss << config_.indent << "psx_ws_sprite_tag(cpu); /* widescreen: record prim ($a0) + anchor */\n"; } + if (config_.ws_bg2d_init_func == func.start_addr) { + body_ss << config_.indent + << "psx_ws_mmx6_bg_stage_init(); /* widescreen: invalidate stale reveal at stage BG init */\n"; + } if (config_.ws_backdrop_unsquash_funcs.count(func.start_addr)) { body_ss << config_.indent << "gte_ws_set_suppress(1); /* widescreen: un-squash far backdrop (8C) */\n"; @@ -2247,6 +2313,7 @@ std::vector<GeneratedFunction> CodeGenerator::generate_alias_group( // generate_function() call can't leak into this body. (See generate_function.) ws_auto_cull_func_ = config_.ws_auto_screen_x_cull && func_has_screen_extent_cull(cfg); + detect_cull_bltz_sites(cfg); // Backdrop preload sites for this alias group's shared CFG (see // generate_function). Set per-group so a stale map can't leak in. @@ -2483,6 +2550,45 @@ std::vector<GeneratedFunction> CodeGenerator::generate_all_functions( } GeneratedFunction gen_func = generate_function(func, cfg, fallthrough_name); + + // PSX_EMIT_PROD (DEFAULT ON): a "function" whose body is overwhelmingly + // untranslatable (TODO-opcode) words is a data region that discovery's + // primary-opcode-only validity check let slip through as code (valid + // primary opcode, garbage sub-field). Re-emit it as the same compact, + // fail-closed data stub used for is_data_section — if it is ever + // dispatched (it should not be: real reachable code never hits a TODO) + // psx_unknown_dispatch fatals LOUDLY rather than silently no-op'ing. + // This is strictly more faithful than the baseline it replaces (which + // emitted a SILENT no-op for the same words). Smoke-validated on Tomba, + // Tomba2, Ape Escape, MMX6. Set PSX_EMIT_PROD=0 to restore the full-scan + // (oracle) emission. Proper productionization moves this judgment UPSTREAM + // into discovery's validity check (sub-field aware) in function_analysis. + static int s_emit_prod = -1; + if (s_emit_prod < 0) { + const char* e = std::getenv("PSX_EMIT_PROD"); + s_emit_prod = (e && *e == '0') ? 0 : 1; // default ON; =0 restores full-scan baseline + } + if (s_emit_prod) { + size_t todo = 0, pos = 0; + while ((pos = gen_func.body.find("TODO: opcode", pos)) != std::string::npos) { todo++; pos += 12; } + pos = 0; + while ((pos = gen_func.body.find("TODO: SPECIAL", pos)) != std::string::npos) { todo++; pos += 13; } + uint32_t instrs = func.size / 4u; + if (instrs > 0 && todo * 2u >= instrs) { // >= 50% untranslatable => data + GeneratedFunction data_stub; + data_stub.function_name = func.name; + data_stub.signature = fmt::format("void {}(CPUState* cpu)", func.name); + data_stub.body = fmt::format( + "{{\n psx_unknown_dispatch(cpu, 0x{:08X}u, 0x{:08X}u);\n}}\n", + func.start_addr, func.start_addr & 0x1FFFFFFFu); + data_stub.full_code = data_stub.signature + "\n" + data_stub.body; + data_stub.line_count = 4; + total_lines += data_stub.line_count; + results.push_back(std::move(data_stub)); + continue; + } + } + total_lines += gen_func.line_count; results.push_back(gen_func); } @@ -2514,8 +2620,11 @@ std::string CodeGenerator::generate_file( ss << "extern void cosim_instr(uint32_t pc);\n"; ss << "#endif\n"; ss << "extern void psx_ws_sprite_tag(CPUState* cpu); /* widescreen prim tag (gpu.c) */\n"; + ss << "extern void psx_ws_mmx6_bg_stage_init(void); /* ws 2D stage reveal invalidation (gpu.c) */\n"; ss << "extern int psx_ws_x_margin(void); /* widescreen cull-margin term (gpu.c) */\n"; ss << "extern int psx_ws_cull_sltiu(uint32_t sx, uint32_t imm); /* ws auto screen-x cull (gpu.c) */\n"; + ss << "extern int psx_ws_cull_slti(uint32_t sx, uint32_t imm); /* ws cull signed right edge (gpu.c) */\n"; + ss << "extern int psx_ws_cull_bltz(uint32_t v); /* ws cull signed left edge (gpu.c) */\n"; ss << "extern int psx_ws_backdrop_x(int x); /* widescreen backdrop screenX squash (gpu.c) */\n"; ss << "extern int psx_ws_bg2d_cols(int base); /* ws 2D bg tile-loop widen: col count (gpu.c) */\n"; ss << "extern int psx_ws_bg2d_startcol(int col, unsigned mask); /* ws 2D bg tile-loop widen: start tile col (gpu.c) */\n"; diff --git a/recompiler/src/config_loader.cpp b/recompiler/src/config_loader.cpp index e69fcc1a..67f37c74 100644 --- a/recompiler/src/config_loader.cpp +++ b/recompiler/src/config_loader.cpp @@ -8,6 +8,7 @@ #include <string> #include "fmt/format.h" +#include "ps1_exe_parser.h" // toml11 is header-only. #define TOML11_USE_UNRELEASED_TOML_FEATURES @@ -80,8 +81,30 @@ static bool parse_aspect_ratio(const std::string& s, int* num, int* den) { // to `root` (project root). static RuntimeConfig parse_runtime_block(const toml::value& cfg, const fs::path& root) { RuntimeConfig rt; + // [localization] language = "en" (top-level; independent of [runtime]). + if (cfg.contains("localization")) { + const toml::value& loc = toml::find(cfg, "localization"); + if (loc.contains("language")) + rt.language = toml::find<std::string>(loc, "language"); + // `default` is the launcher-facing alias for the pre-selected language. + if (loc.contains("default")) + rt.language = toml::find<std::string>(loc, "default"); + // Optional launcher dropdown options: languages = [ {code, label}, ... ]. + if (loc.contains("languages") && toml::find(loc, "languages").is_array()) { + for (const auto& e : toml::find(loc, "languages").as_array()) { + if (!e.contains("code")) continue; + RuntimeConfig::LanguageOption lo; + lo.code = toml::find<std::string>(e, "code"); + lo.label = e.contains("label") ? toml::find<std::string>(e, "label") + : lo.code; + if (!lo.code.empty()) rt.languages.push_back(std::move(lo)); + } + } + } if (!cfg.contains("runtime")) return rt; const toml::value& runtime = toml::find(cfg, "runtime"); + if (runtime.contains("language")) // [runtime].language convenience alias + rt.language = toml::find<std::string>(runtime, "language"); if (runtime.contains("debug_port")) { const auto port = toml::find<int64_t>(runtime, "debug_port"); @@ -296,6 +319,9 @@ static RuntimeConfig parse_runtime_block(const toml::value& cfg, const fs::path& if (ct.contains("lock_mode")) { rt.controller_lock_mode = toml::find<bool>(ct, "lock_mode"); } + if (ct.contains("lock_device")) { + rt.controller_lock_device = toml::find<bool>(ct, "lock_device"); + } if (ct.contains("deadzone")) { const auto n = toml::find<int64_t>(ct, "deadzone"); if (n < 0 || n > 32767) @@ -541,18 +567,67 @@ GameConfig load_game_config(const fs::path& config_path_in) { } const fs::path exe_path = fs::absolute(root / exe_field); + // Auto-detect EXE header values for any field not explicitly set in TOML. + // Parses the PS-X EXE header once and fills in load_address, entry_pc, + // text_size, and stack_base from the binary. If the EXE cannot be read + // (malformed header, file not found) and a required field is missing + // from TOML, the error surfaces when the EXE parse itself fails. + struct ExeHeaderAuto { + bool ok = false; + uint32_t load_address = 0; + uint32_t entry_pc = 0; + uint32_t text_size = 0; + uint32_t stack_base = 0; + }; + ExeHeaderAuto auto_hdr; + // We only parse the EXE header if at least one field is missing from TOML. + const bool need_auto = + !game.contains("load_address") || !game.contains("entry_pc") || + !game.contains("text_size") || !game.contains("stack_base"); + if (need_auto) { + std::string err; + if (auto exe = PSXRecomp::PS1ExeParser::parse_file(exe_path, err)) { + auto_hdr.ok = true; + auto_hdr.load_address = exe->load_address(); + auto_hdr.entry_pc = exe->entry_point(); + auto_hdr.text_size = exe->code_size(); + auto_hdr.stack_base = exe->header.stack_base; + // Round text_size up to next 4K page if it's not already + // page-aligned (the runtime's overlay region floor uses this + // as a guard, and the text segment in RAM is always fully + // page-reserved even if the EXE doesn't fill the last page). + if (auto_hdr.text_size & 0xFFF) { + auto_hdr.text_size = (auto_hdr.text_size + 0xFFF) & ~0xFFFu; + } + } + } + const uint32_t load_address = - parse_hex(toml::find<std::string>(game, "load_address"), "game.load_address"); + game.contains("load_address") + ? parse_hex(toml::find<std::string>(game, "load_address"), "game.load_address") + : (auto_hdr.ok ? auto_hdr.load_address : []() -> uint32_t { + throw std::runtime_error( + "game.toml: missing 'load_address' in [game] and could not " + "auto-detect from EXE header"); + }()); const uint32_t entry_pc = game.contains("entry_pc") ? parse_hex(toml::find<std::string>(game, "entry_pc"), "game.entry_pc") - : load_address; + : (auto_hdr.ok ? auto_hdr.entry_pc : load_address); const uint32_t text_size = - parse_hex(toml::find<std::string>(game, "text_size"), "game.text_size"); + game.contains("text_size") + ? parse_hex(toml::find<std::string>(game, "text_size"), "game.text_size") + : (auto_hdr.ok ? auto_hdr.text_size : []() -> uint32_t { + throw std::runtime_error( + "game.toml: missing 'text_size' in [game] and could not " + "auto-detect from EXE header"); + }()); const uint32_t stack_base = game.contains("stack_base") ? parse_hex(toml::find<std::string>(game, "stack_base"), "game.stack_base") - : 0x801FFFF0u; + : (auto_hdr.ok && auto_hdr.stack_base != 0 + ? auto_hdr.stack_base + : 0x801FFFF0u); // Disc paths: accept either single `disc` or array `discs`. std::vector<fs::path> discs; @@ -620,6 +695,14 @@ GameConfig load_game_config(const fs::path& config_path_in) { uint32_t ws_sprite_anchor_addr = 0; bool ws_hud_sprt_squash = false; bool ws_full_2d = false; + bool ws_gte_game_mode = false; + bool ws_nw_hud_corners = false; + uint32_t ws_nw_left_hud_packet_lo = 0; + uint32_t ws_nw_left_hud_packet_hi = 0; + bool ws_nw_backdrop = false; + bool ws_clear_reveal = false; + bool ws_offered = true; + bool ws_ultrawide_offered = false; if (cfg.contains("widescreen")) { const toml::value& ws = toml::find(cfg, "widescreen"); if (ws.contains("sprite_tag_funcs")) { @@ -641,11 +724,49 @@ GameConfig load_game_config(const fs::path& config_path_in) { ws_hud_sprt_squash = toml::find<bool>(ws, "hud_sprt_squash"); if (ws.contains("full_2d")) ws_full_2d = toml::find<bool>(ws, "full_2d"); + if (ws.contains("gte_game_mode")) + ws_gte_game_mode = toml::find<bool>(ws, "gte_game_mode"); + if (ws.contains("nw_hud_corners")) + ws_nw_hud_corners = toml::find<bool>(ws, "nw_hud_corners"); + const bool has_nw_left_hud_lo = ws.contains("nw_left_hud_packet_lo"); + const bool has_nw_left_hud_hi = ws.contains("nw_left_hud_packet_hi"); + if (has_nw_left_hud_lo != has_nw_left_hud_hi) + throw std::runtime_error(fmt::format( + "{}: [widescreen] nw_left_hud_packet_lo and " + "nw_left_hud_packet_hi must be set together", + config_path.string())); + if (has_nw_left_hud_lo) { + ws_nw_left_hud_packet_lo = parse_hex( + toml::find<std::string>(ws, "nw_left_hud_packet_lo"), + "widescreen.nw_left_hud_packet_lo"); + ws_nw_left_hud_packet_hi = parse_hex( + toml::find<std::string>(ws, "nw_left_hud_packet_hi"), + "widescreen.nw_left_hud_packet_hi"); + if (ws_nw_left_hud_packet_lo >= ws_nw_left_hud_packet_hi) + throw std::runtime_error(fmt::format( + "{}: [widescreen] nw_left_hud_packet range is empty or reversed", + config_path.string())); + } + if (ws.contains("nw_backdrop")) + ws_nw_backdrop = toml::find<bool>(ws, "nw_backdrop"); + if (ws.contains("clear_reveal")) + ws_clear_reveal = toml::find<bool>(ws, "clear_reveal"); + if (ws.contains("offer")) + ws_offered = toml::find<bool>(ws, "offer"); + if (ws.contains("offer_ultrawide")) + ws_ultrawide_offered = toml::find<bool>(ws, "offer_ultrawide"); } // Optional [widescreen.cull] block — world-space draw-cull widening. std::vector<uint32_t> ws_cull_bias_sites, ws_cull_range_sites, ws_cull_a1_sites; std::vector<uint32_t> ws_cull_screen_x_sites; + std::vector<uint32_t> ws_cull_slti_sites; + int ws_cull_guard_pixels = 0; + // Cull-signature immediates (screen_w_imms / screen_h_imms). Defaults are + // the original Tomba signature (320-display: 0x140/0x141 + 0xE0/0xF1); a + // game with a different display width overrides them (Ape Escape: 0x181). + std::vector<uint32_t> ws_cull_w_imms = { 0x140, 0x141 }; + std::vector<uint32_t> ws_cull_h_imms = { 0xE0, 0xF1 }; bool ws_auto_screen_x_cull = false; bool ws_auto_backdrop_preload = false; if (cfg.contains("widescreen")) { @@ -661,6 +782,22 @@ GameConfig load_game_config(const fs::path& config_path_in) { load_sites("range_sites", ws_cull_range_sites); load_sites("a1_sites", ws_cull_a1_sites); load_sites("screen_x_sites", ws_cull_screen_x_sites); + load_sites("slti_sites", ws_cull_slti_sites); + if (cull.contains("guard_pixels")) { + ws_cull_guard_pixels = toml::find<int>(cull, "guard_pixels"); + if (ws_cull_guard_pixels < 0 || ws_cull_guard_pixels > 256) + throw std::runtime_error(fmt::format( + "{}: [widescreen.cull] guard_pixels must be in [0, 256]", + config_path.string())); + } + if (cull.contains("screen_w_imms")) { + ws_cull_w_imms.clear(); + load_sites("screen_w_imms", ws_cull_w_imms); + } + if (cull.contains("screen_h_imms")) { + ws_cull_h_imms.clear(); + load_sites("screen_h_imms", ws_cull_h_imms); + } if (cull.contains("auto_screen_x")) ws_auto_screen_x_cull = toml::find<bool>(cull, "auto_screen_x"); if (cull.contains("auto_backdrop")) @@ -676,6 +813,7 @@ GameConfig load_game_config(const fs::path& config_path_in) { uint32_t ws_bg2d_map_size_addr = 0x800CD338u, ws_bg2d_layer_stride_addr = 0x8008EC10u; uint32_t ws_bg2d_ring_cols = 64, ws_bg2d_layer_count = 3; uint32_t ws_bg2d_layer_struct_stride = 0x54; + uint32_t ws_bg2d_init_func = 0; if (cfg.contains("widescreen")) { const toml::value& ws = toml::find(cfg, "widescreen"); if (ws.contains("bg2d")) { @@ -716,6 +854,7 @@ GameConfig load_game_config(const fs::path& config_path_in) { if ((ws_bg2d_ring_cols & (ws_bg2d_ring_cols - 1u)) != 0u) throw std::runtime_error( "widescreen.bg2d.ring_cols must be a power of two"); + ws_bg2d_init_func = load1("init_func"); } } @@ -765,11 +904,23 @@ GameConfig load_game_config(const fs::path& config_path_in) { /*ws_cull_range_sites*/ ws_cull_range_sites, /*ws_cull_a1_sites*/ ws_cull_a1_sites, /*ws_cull_screen_x_sites*/ ws_cull_screen_x_sites, + /*ws_cull_slti_sites*/ ws_cull_slti_sites, + /*ws_cull_guard_pixels*/ ws_cull_guard_pixels, + /*ws_cull_w_imms*/ ws_cull_w_imms, + /*ws_cull_h_imms*/ ws_cull_h_imms, /*ws_backdrop_x_sites*/ ws_backdrop_x_sites, /*ws_backdrop_unsquash_funcs*/ ws_backdrop_unsquash_funcs, /*ws_auto_screen_x_cull*/ ws_auto_screen_x_cull, /*ws_auto_backdrop_preload*/ ws_auto_backdrop_preload, /*ws_full_2d*/ ws_full_2d, + /*ws_gte_game_mode*/ ws_gte_game_mode, + /*ws_nw_hud_corners*/ ws_nw_hud_corners, + /*ws_nw_left_hud_packet_lo*/ ws_nw_left_hud_packet_lo, + /*ws_nw_left_hud_packet_hi*/ ws_nw_left_hud_packet_hi, + /*ws_nw_backdrop*/ ws_nw_backdrop, + /*ws_clear_reveal*/ ws_clear_reveal, + /*ws_offered*/ ws_offered, + /*ws_ultrawide_offered*/ ws_ultrawide_offered, /*ws_bg2d_count_site*/ ws_bg2d_count_site, /*ws_bg2d_startcol_site*/ ws_bg2d_startcol_site, /*ws_bg2d_startx_site*/ ws_bg2d_startx_site, @@ -784,6 +935,7 @@ GameConfig load_game_config(const fs::path& config_path_in) { /*ws_bg2d_ring_cols*/ ws_bg2d_ring_cols, /*ws_bg2d_layer_count*/ ws_bg2d_layer_count, /*ws_bg2d_layer_struct_stride*/ ws_bg2d_layer_struct_stride, + /*ws_bg2d_init_func*/ ws_bg2d_init_func, }; } @@ -953,6 +1105,13 @@ UserSettings load_user_settings(const fs::path& path) { s.memcard2_enabled = toml::find<bool>(m, "enable2"); s.has_memcard2_enabled = true; }); } + if (doc.contains("localization")) { + const toml::value& lc = toml::find(doc, "localization"); + if (lc.contains("language")) try_get([&]{ + const auto v = toml::find<std::string>(lc, "language"); + if (!v.empty()) { s.language = v; s.has_language = true; } + }); + } if (doc.contains("controller")) { const toml::value& ct = toml::find(doc, "controller"); if (ct.contains("p1_device")) try_get([&]{ @@ -1084,6 +1243,11 @@ bool save_user_settings(const fs::path& path, const UserSettings& s) { f << "deadzone = " << s.deadzone << "\n"; } + if (s.has_language) { + f << "\n[localization]\n"; + f << "language = \"" << s.language << "\"\n"; + } + return f.good(); } diff --git a/recompiler/src/config_loader.h b/recompiler/src/config_loader.h index 8cca8e68..cb5225c0 100644 --- a/recompiler/src/config_loader.h +++ b/recompiler/src/config_loader.h @@ -48,6 +48,19 @@ struct RuntimeConfig { bool has_debug_port = false; uint16_t debug_port = 0; + // Localization / on-the-fly string translation (docs/STRING_TRANSLATION.md). + // language selects the translations/*.toml column; "jp"/"off"/"" disables + // APPLY (capture still runs). From [localization].language or [runtime]. + // language; env PSX_LANG overrides at runtime. Also the launcher default. + std::string language = "en"; + + // Optional launcher-facing language menu. When a game declares + // [localization].languages, the launcher shows a "Localization" dropdown of + // these {code,label} options (code feeds `language`; "off"/"jp"/"" = the + // untranslated native game). Empty => no dropdown (the general default). + struct LanguageOption { std::string code; std::string label; }; + std::vector<LanguageOption> languages; + bool has_window_title = false; std::string window_title; @@ -79,15 +92,18 @@ struct RuntimeConfig { // at host speed. Kept so existing game.toml/settings.toml keep working. bool fast_boot = false; - // bios_hle: opt-in High-Level Emulation tier for BIOS kernel services - // (CLAUDE.md §0 amendment 2026-07-02, the gbarecomp model). Default off = - // LLE (the recompiled BIOS), which remains the reference implementation - // and the oracle. When on, implemented kernel services are computed - // in-runtime against the real guest kernel structures and every other - // call falls through to LLE. Implies the HLE boot shell-skip unless - // bios_hle_keep_intro. PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env - // override at launch. Runtime: runtime/src/bios_hle.c. - bool bios_hle = false; + // bios_hle: High-Level Emulation tier for BIOS kernel services + // (CLAUDE.md §0 amendment 2026-07-02, the gbarecomp model). DEFAULT ON as of + // 2026-07-06 (user-directed player default: instant boot-skip for every + // game). Opt OUT with [runtime] bios_hle = false or env PSX_BIOS_HLE=0 to run + // pure LLE (the recompiled BIOS), which REMAINS the reference implementation + // and the oracle — this only flips the default, LLE is still fully linked and + // selectable. When on, implemented kernel services are computed in-runtime + // against the real guest kernel structures and every other call falls through + // to LLE. Implies the HLE boot shell-skip unless bios_hle_keep_intro. + // PSX_BIOS_HLE / PSX_BIOS_HLE_KEEP_INTRO env override at launch. + // Runtime: runtime/src/bios_hle.c. + bool bios_hle = true; bool bios_hle_keep_intro = false; // hle_scheduler: the HLE tier's standing SUBSYSTEM REPLACEMENT for guest @@ -257,6 +273,14 @@ struct RuntimeConfig { // allow_hybrid (which only hides the Hybrid segment). Default false. bool controller_lock_mode = false; + // lock_device: when true the launcher HIDES the Player 1/2 controller cards + // entirely (no device picker, no config) — the game's controller type is + // fixed and auto-bound. For a title that presents exactly one hardcoded pad + // type (e.g. Ape Escape = DualShock analog) where exposing a device/mode + // choice is pointless. Distinct from lock_mode (which only hides the pad-mode + // segment but keeps the device dropdown). Default false. + bool controller_lock_device = false; + // deadzone: default analog-stick deadzone in raw SDL axis units (0..32767). // Applied both to the stick->d-pad press threshold and the analog-axis centre // dead-band. Absent => runtime default (12000). Overridden per-install by @@ -396,6 +420,23 @@ struct GameConfig { // splitting separates the paired vertical test from auto_screen_x. std::vector<uint32_t> ws_cull_screen_x_sites; + // [widescreen.cull] slti_sites — explicit signed right-edge widen sites + // (`slti rt, sx, W` → psx_ws_cull_slti) for funnel functions the + // auto-detector cannot qualify (e.g. an X-only test with no height compare + // in the same function — Ape Escape 0x8004AB64). Empty by default; regen. + std::vector<uint32_t> ws_cull_slti_sites; + // Extra per-side actor overdraw beyond the visible widescreen edge. + int ws_cull_guard_pixels = 0; + + // [widescreen.cull] screen_w_imms / screen_h_imms — the width/height + // immediates of the GTE screen-extent reject signature, per game (the + // display width varies: Tomba 320 → 0x140/0x141; Ape Escape 368 → 0x181). + // Consumed by the auto_screen_x detector + emit on every backend (the + // runtime mirrors get them via gpu_ws_set_cull_imms). Defaults preserve + // the original Tomba signature. + std::vector<uint32_t> ws_cull_w_imms; + std::vector<uint32_t> ws_cull_h_imms; + // Backdrop screen-X squash ([widescreen.backdrop] x_sites). The parallax // 2D backdrop layer (ocean/cloud/mountain/grass — overlay actor handlers) // computes screenX = (worldX - camX) >> parallax in pure integer math and @@ -450,6 +491,52 @@ struct GameConfig { // default; the env var PSX_WS_FORCE_2D=1 forces it on for testing. bool ws_full_2d = false; + // [widescreen] gte_game_mode — GTE-activity gameplay detector for a fully + // 3D title with no sprite-tag helper (e.g. Ape Escape). When true, a frame + // that projects enough vertices through RTPS/RTPT is stamped as gameplay + // (native-wide engages); genuine full-2D screens (save/options) still + // pillarbox 4:3. Runtime-only — no regen required. Off by default. + bool ws_gte_game_mode = false; + + // [widescreen] nw_hud_corners — in native-wide, push outer-third screen- + // space HUD sprites out to the true wide-frame corners (they otherwise sit + // inset by the reveal offset). Runtime-only — no regen. Off by default. + bool ws_nw_hud_corners = false; + + // [widescreen] nw_left_hud_packet_lo / nw_left_hud_packet_hi — optional + // half-open physical-RAM source range for a specifically identified HUD + // pool. In native-wide, primitives from this pool anchor by screen third + // (left/right), without moving similarly placed scenery from other pools. + // Both values must be present together. Runtime-only; no regen required. + uint32_t ws_nw_left_hud_packet_lo = 0; + uint32_t ws_nw_left_hud_packet_hi = 0; + + // [widescreen] nw_backdrop — in native-wide, stretch a screen-space quad + // that covers the whole 4:3 framebuffer (sky gradient / backdrop image) to + // fill the wide frame, so it stops pillarboxing at the reveal margins. + // Runtime-only — no regen. Off by default. + bool ws_nw_backdrop = false; + + // [widescreen] clear_reveal — opt a title into synthetic native-wide margin + // cleanup. A game-specific stage/map boundary can clear only proven-void + // sides while preserving the canonical 4:3 surface and guest VRAM. Runtime- + // only gate; any game-specific init hook remains regen-class. Off by default. + bool ws_clear_reveal = false; + + // [widescreen] offer — whether the launcher OFFERS its EXPERIMENTAL + // Widescreen toggle for this title. Default true. Set false while a + // title's widescreen is unported/unvalidated (e.g. MMX4 at bring-up): + // the toggle is hidden in the launcher UI AND the runtime clamps the + // display aspect to 4:3 after all config sources, so a stale persisted + // 16:9 in settings.toml can never engage the hack. Runtime-only (read at + // startup; no codegen impact) — no regen required. + bool ws_offered = true; + + // [widescreen] offer_ultrawide — expose a separate experimental 21:9 + // launcher choice for titles that have explicitly tested it. Default off; + // ordinary widescreen offer remains the independent 16:9 choice. + bool ws_ultrawide_offered = false; + // [widescreen.bg2d] — pure-2D background tile-loop widen (e.g. MMX6's // FUN_800270d0). Three instruction addresses in the per-layer BG renderer // whose column count and loop start are rewritten so the loop draws the @@ -476,7 +563,6 @@ struct GameConfig { // raised to match the bigger buffer. Together they cure the dense-stage overflow. uint32_t ws_bg2d_bufbase_site = 0; uint32_t ws_bg2d_cap_site = 0; - // Tile-ring layout used by the once-per-frame freshness refill. Defaults // describe MMX6; sibling engines such as MMX5 override the shifted RAM // addresses and smaller ring width in game.toml. @@ -487,6 +573,10 @@ struct GameConfig { uint32_t ws_bg2d_ring_cols = 64; uint32_t ws_bg2d_layer_count = 3; uint32_t ws_bg2d_layer_struct_stride = 0x54; + // init_func: full tile-ring initializer called only when an independent + // layer's stage data is dirty. A callback at entry invalidates stale host + // reveal pixels once before the new stage background is submitted. + uint32_t ws_bg2d_init_func = 0; }; // UserSettings — the launcher-written, user-editable override layer. @@ -565,6 +655,10 @@ struct UserSettings { // Analog-stick deadzone, raw SDL axis units (0..32767). The launcher edits // this as 0-100% (raw = pct*32767/100), mirroring snesrecomp's GamepadDeadzone. bool has_deadzone = false; int deadzone = 12000; + // Localization: the launcher's chosen language code (feeds RuntimeConfig + // .language / g_lang). "off"/"jp"/"" = untranslated native game. Persisted to + // settings.toml [localization].language. + bool has_language = false; std::string language = "en"; }; // GameOptions — the game's OWN native OPTION-screen settings, declared in a diff --git a/recompiler/src/full_function_emitter.cpp b/recompiler/src/full_function_emitter.cpp index d2c69a31..e92a5ab5 100644 --- a/recompiler/src/full_function_emitter.cpp +++ b/recompiler/src/full_function_emitter.cpp @@ -1453,9 +1453,23 @@ void FullFunctionEmitter::emit_dispatch( out += " * it to the stale shell image. A genuine pre-start shell address\n"; out += " * (game not started) still falls through to normalize(). */\n"; out += " extern int fntrace_is_game_started(void);\n"; + out += " extern int dirty_ram_text_native_ok(uint32_t phys);\n"; out += " int game_shell_overlap = fntrace_is_game_started() &&\n"; out += " game_phys >= 0x00030000u && game_phys <= 0x0005AFFFu;\n"; - out += " if (!found && (game_shell_overlap ||\n"; + out += " /* Shell-window addresses the loaded game EXE image covers with\n"; + out += " * matching bytes are GAME code, not the BIOS shell \xE2\x80\x94 route them to\n"; + out += " * the game path even when the game-start latch (a dispatch to\n"; + out += " * entry_pc) never tripped. Fixes titles whose static text fills\n"; + out += " * 0x30000-0x5AFFF: Crash Bash reaches its entry 0x2E7B0 via compiled\n"; + out += " * internal flow, so `started` never latched and normalize() shadowed\n"; + out += " * real text at 0x3358C to dead shell ROM (unknown-dispatch abort).\n"; + out += " * The native-ok compare is ground truth: pre-game the RAM holds shell\n"; + out += " * bytes (differ from the game image -> not routed here -> shell\n"; + out += " * dispatches); post-load it holds the game EXE (matches -> game path). */\n"; + out += " int game_text_in_shell_window =\n"; + out += " game_phys >= 0x00030000u && game_phys <= 0x0005AFFFu &&\n"; + out += " psx_game_address_in_text(addr) && dirty_ram_text_native_ok(game_phys);\n"; + out += " if (!found && (game_shell_overlap || game_text_in_shell_window ||\n"; out += " (psx_game_address_in_text(addr) && dirty_ram_is_dirty(game_phys)))) {\n"; out += " found = dirty_ram_dispatch(cpu, addr, stop_addr);\n"; out += " }\n"; diff --git a/recompiler/src/function_analysis.cpp b/recompiler/src/function_analysis.cpp index 3fa05507..6efa827c 100644 --- a/recompiler/src/function_analysis.cpp +++ b/recompiler/src/function_analysis.cpp @@ -1133,22 +1133,240 @@ FunctionAnalysisResult FunctionAnalyzer::analyze() { std::vector<uint32_t> starts_vec(function_starts.begin(), function_starts.end()); std::sort(starts_vec.begin(), starts_vec.end()); + // ── Control-flow-aware function extent + code/data classification ── + // + // The legacy estimate set end_addr to the NEXT discovered function start + // and then ran is_likely_data_section over that whole span. In Psy-Q / + // Capcom titles (e.g. Mega Man X4) a real function is immediately followed + // by an INLINE data / jump table (build-signature strings, dispatch + // tables). That trailing data inflated the function's measured extent, and + // is_likely_data_section's undefined-opcode density then misclassified the + // real function as a data section — emitting a psx_unknown_dispatch stub + // for real, directly-called code (the entire 0x800EE3A4.. text tail, + // ~261 KB / 7,373 entries, became stubs, and the boot jal into it hit the + // stub → fail-fast). + // + // Fix (mirrors the proven analyze_exact_entries overlay walk): walk each + // function's control flow from its entry to its real terminus (the jr $ra / + // tail-jump reachable from the entry). If the reachable body is provably + // clean code — reaches a return/tail exit, decodes with zero invalid + // primary opcodes, and every indirect jump was resolved to in-function + // targets — bound end_addr to that terminus (so trailing inline data is + // excluded and never linearly disassembled as code) and mark it code. Only + // functions that are NOT provably clean code fall back to the legacy + // boundary + density heuristic, which still correctly rejects genuine data + // tables minted as function starts (those never walk as clean code: + // ~44% of random data words carry an invalid primary opcode). + auto in_exe_p3 = [&](uint32_t a) { + return a >= exe_.header.load_address && a < exe_.end_address() && (a & 3u) == 0; + }; + // Same primary-opcode validity table as is_likely_data_section so the two + // agree on what "clean code" means (COP2/GTE, LWC2/SWC2 are valid PS1 ops). + auto valid_primary_p3 = [](uint32_t instr) -> bool { + static const bool valid[64] = { + true, true, true, true, true, true, true, true, // 0x00-0x07 + true, true, true, true, true, true, true, true, // 0x08-0x0F + true, false, true, false, false, false, false, false, // 0x10-0x17 + false, false, false, false, false, false, false, false, // 0x18-0x1F + true, true, true, true, true, true, true, false, // 0x20-0x27 + true, true, true, true, false, false, true, false, // 0x28-0x2F + true, false, true, false, false, false, false, false, // 0x30-0x37 + true, false, true, false, false, false, false, false, // 0x38-0x3F + }; + return valid[(instr >> 26) & 0x3Fu]; + }; + // Resolve a `jr $rN` jump table by the standard Psy-Q pattern + // (sltiu bound; sll idx,2; lui/addiu base; addu; lw target; jr). Returns + // true and fills `targets` (in-function entries) when a table is + // recognized; false when the indirect jump is unresolved. + auto resolve_jt_p3 = [&](uint32_t entry, uint32_t hard_cap, uint32_t jr_pc, + uint32_t jr_rs, std::vector<uint32_t>& targets) -> bool { + uint32_t lw_base = 0xFFu; int32_t lw_offset = 0; + uint32_t addu_cand[2] = {0xFFu, 0xFFu}; + uint32_t lui_val = 0; int16_t addiu_val[2] = {0, 0}; + bool found_addiu[2] = {false, false}; + bool found_lui = false; uint32_t table_count = 0; + for (int back = 1; back <= 40; back++) { + uint32_t scan_addr = jr_pc - static_cast<uint32_t>(back * 4); + if (scan_addr < entry) break; + auto scan_opt = exe_.read_word(scan_addr); + if (!scan_opt.has_value()) break; + uint32_t instr = *scan_opt; + uint32_t op = (instr >> 26) & 0x3Fu; + uint32_t rs = (instr >> 21) & 0x1Fu; + uint32_t rt = (instr >> 16) & 0x1Fu; + uint32_t rd = (instr >> 11) & 0x1Fu; + uint32_t fn = instr & 0x3Fu; + if (op == 0x23u && rt == jr_rs && lw_base == 0xFFu) { + lw_base = rs; + lw_offset = static_cast<int32_t>(static_cast<int16_t>(instr & 0xFFFFu)); + continue; + } + if (op == 0x00u && fn == 0x21u && rd == lw_base && lw_base != 0xFFu && + addu_cand[0] == 0xFFu) { + addu_cand[0] = rs; addu_cand[1] = rt; continue; + } + if (op == 0x09u && addu_cand[0] != 0xFFu) { + for (int c = 0; c < 2; c++) { + if (!found_addiu[c] && addu_cand[c] != 0xFFu && + rs == addu_cand[c] && rt == addu_cand[c]) { + addiu_val[c] = static_cast<int16_t>(instr & 0xFFFFu); + found_addiu[c] = true; break; + } + } + continue; + } + if (op == 0x0Fu && addu_cand[0] != 0xFFu && !found_lui) { + for (int c = 0; c < 2; c++) { + if (addu_cand[c] != 0xFFu && rt == addu_cand[c]) { + lui_val = (instr & 0xFFFFu) << 16; + if (!found_addiu[c]) addiu_val[c] = 0; + addiu_val[0] = addiu_val[c]; + found_addiu[0] = found_addiu[c]; + found_lui = true; break; + } + } + continue; + } + if (op == 0x0Bu && table_count == 0) { table_count = instr & 0xFFFFu; continue; } + if (found_lui && table_count != 0) break; + } + if (!found_lui || table_count == 0 || table_count >= 512) return false; + uint32_t table_base = lui_val + + (found_addiu[0] ? static_cast<uint32_t>(static_cast<int32_t>(addiu_val[0])) : 0u) + + static_cast<uint32_t>(lw_offset); + for (uint32_t k = 0; k < table_count; k++) { + auto entry_opt = exe_.read_word(table_base + k * 4u); + if (!entry_opt.has_value()) continue; + uint32_t target = *entry_opt; + if (target >= entry && target < hard_cap && in_exe_p3(target)) targets.push_back(target); + } + return true; + }; + struct ExtentP3 { uint32_t terminus; bool clean_code; bool hit_invalid; bool resolved_jt; bool unresolved_indirect; }; + auto compute_extent_p3 = [&](uint32_t entry, uint32_t hard_cap) -> ExtentP3 { + std::set<uint32_t> visited; + std::queue<uint32_t> work; + work.push(entry); + bool hit_invalid = false, reached_exit = false, unresolved_indirect = false; + bool resolved_jt = false; + auto in_fn = [&](uint32_t a) { return in_exe_p3(a) && a >= entry && a < hard_cap; }; + while (!work.empty()) { + uint32_t pc = work.front(); work.pop(); + if (visited.count(pc) || !in_fn(pc)) continue; + auto word_opt = exe_.read_word(pc); + if (!word_opt.has_value()) continue; + uint32_t instr = *word_opt; + visited.insert(pc); + if (!valid_primary_p3(instr)) { hit_invalid = true; continue; } + ExactCf cf = exact_classify_cf(pc, instr); + uint32_t delay = pc + 4u; + switch (cf.kind) { + case ExactCfKind::Normal: + if (in_fn(pc + 4u)) work.push(pc + 4u); + break; + case ExactCfKind::Branch: + if (in_fn(delay)) visited.insert(delay); + if (in_fn(pc + 8u)) work.push(pc + 8u); + if (in_fn(cf.target)) work.push(cf.target); + break; + case ExactCfKind::Jump: + if (in_fn(delay)) visited.insert(delay); + if (in_fn(cf.target)) work.push(cf.target); + else reached_exit = true; // tail call / jump out of function + break; + case ExactCfKind::Jal: + case ExactCfKind::Jalr: + if (in_fn(delay)) visited.insert(delay); + if (in_fn(pc + 8u)) work.push(pc + 8u); + break; + case ExactCfKind::JrRa: + if (in_fn(delay)) visited.insert(delay); + reached_exit = true; + break; + case ExactCfKind::JrOther: { + if (in_fn(delay)) visited.insert(delay); + std::vector<uint32_t> jt; + uint32_t jr_rs = (instr >> 21) & 0x1Fu; + if (resolve_jt_p3(entry, hard_cap, pc, jr_rs, jt)) { + resolved_jt = true; + for (uint32_t t : jt) if (in_fn(t)) work.push(t); + } else { + unresolved_indirect = true; // computed jump we can't bound + } + break; + } + } + } + ExtentP3 info; + info.terminus = visited.empty() ? entry : (*visited.rbegin() + 4u); + // Density guard: a real function's basic blocks densely cover + // [entry, terminus]. A walk that follows a data-decoded branch/jump into + // an inline data region can coincidentally reach a far jr $ra (the word + // 0x03E00008 appears in data) and look "clean" while spanning tens of KB + // it barely visited — that over-extended range then hosts many aliases + // and emits the whole data region as code (huge bloat). Require the + // visited instructions to cover a meaningful fraction (~25%) of the span + // so sparse data-following walks are rejected and fall back to the + // density heuristic. Real functions (incl. those with small inline jump + // tables) stay well above this floor. + uint32_t span = info.terminus - entry; + bool dense = (span == 0) || (static_cast<uint64_t>(visited.size()) * 16u >= span); + info.clean_code = reached_exit && !hit_invalid && !unresolved_indirect && + !visited.empty() && dense; + info.hit_invalid = hit_invalid; + info.resolved_jt = resolved_jt; + info.unresolved_indirect = unresolved_indirect; + return info; + }; + for (size_t i = 0; i < starts_vec.size(); i++) { Function func; func.start_addr = starts_vec[i]; + uint32_t next_start = (i + 1 < starts_vec.size()) ? starts_vec[i + 1] : end_addr; - // Estimate end address (next function start or end of executable) - if (i + 1 < starts_vec.size()) { - func.end_addr = starts_vec[i + 1]; + ExtentP3 ext = compute_extent_p3(func.start_addr, next_start); + if (ext.clean_code) { + // Real function: bound to its control-flow terminus so trailing + // inline data / jump tables are excluded from the emitted extent. + func.end_addr = ext.terminus; + if (func.end_addr <= func.start_addr) func.end_addr = func.start_addr + 4u; + if (func.end_addr > next_start) func.end_addr = next_start; + func.is_data_section = false; + } else if (ext.hit_invalid && !ext.resolved_jt) { + // The reachable body (reached via straight/branch flow, no resolved + // jump table) decodes with an invalid primary opcode — real PS1 code + // never executes an invalid opcode, so this "function start" is + // genuinely data (e.g. a filename/string or pointer-table slot minted + // as a start by pointer-table promotion). Classify as data even when + // the dense-minted-start span is below is_likely_data_section's size + // floor. Guarded by !resolved_jt so a mis-resolved jump table can + // never force a real function to a stub. + func.end_addr = next_start; + func.is_data_section = true; } else { - func.end_addr = end_addr; - } - auto ret_it = function_last_return.find(func.start_addr); - if (ret_it != function_last_return.end()) { - uint32_t return_end = ret_it->second + 8u; /* include jr delay slot */ - if (return_end > func.start_addr && return_end < func.end_addr) { - func.end_addr = return_end; + // Legacy boundary estimate + density heuristic, but bound end_addr to + // the control-flow walk terminus (a valid upper bound on code + // reachable from the entry). Without this, a function-start on a + // data-heavy tail with no intervening discovered entry gets + // end_addr = next_start spanning tens of KB of inline data; that + // over-extended range then hosts many pointer-table aliases and emits + // the whole data region as code (massive bloat). The terminus bound + // is skipped only when an unresolved indirect jump means real code may + // exist beyond the walk that we cannot safely bound. + func.end_addr = next_start; + auto ret_it = function_last_return.find(func.start_addr); + if (ret_it != function_last_return.end()) { + uint32_t return_end = ret_it->second + 8u; /* include jr delay slot */ + if (return_end > func.start_addr && return_end < func.end_addr) { + func.end_addr = return_end; + } } + if (!ext.unresolved_indirect && ext.terminus > func.start_addr && + ext.terminus < func.end_addr) { + func.end_addr = ext.terminus; + } + func.is_data_section = is_likely_data_section(func.start_addr, func.end_addr); } func.size = func.end_addr - func.start_addr; @@ -1176,9 +1394,6 @@ FunctionAnalysisResult FunctionAnalyzer::analyze() { } } - // Check if this "function" is actually a data section - func.is_data_section = is_likely_data_section(func.start_addr, func.end_addr); - result.functions.push_back(func); } diff --git a/recompiler/src/main_bios.cpp b/recompiler/src/main_bios.cpp index b8588bc9..9f410a9f 100644 --- a/recompiler/src/main_bios.cpp +++ b/recompiler/src/main_bios.cpp @@ -304,7 +304,15 @@ std::string find_c_compiler(const std::optional<std::string>& cli_override) { static const std::array<const char*, 3> candidates = { "gcc", "clang", "cl" }; for (const char* c : candidates) { // Probe with --version. We don't parse output; non-zero exit means not present. - const std::string probe = std::string(c) + " --version >NUL 2>&1"; + // The null device differs by platform: NUL on Windows, /dev/null on + // POSIX. Using ">NUL" on macOS/Linux both misdirects the redirect and + // leaves a stray file named "NUL" in the cwd. +#ifdef _WIN32 + const std::string null_dev = "NUL"; +#else + const std::string null_dev = "/dev/null"; +#endif + const std::string probe = std::string(c) + " --version >" + null_dev + " 2>&1"; if (std::system(probe.c_str()) == 0) { return c; } diff --git a/recompiler/src/main_psx.cpp b/recompiler/src/main_psx.cpp index 20ceee0a..f07f68cc 100644 --- a/recompiler/src/main_psx.cpp +++ b/recompiler/src/main_psx.cpp @@ -121,6 +121,9 @@ int main(int argc, char** argv) { std::set<uint32_t> ws_tag_funcs; // [widescreen] sprite_tag_funcs std::set<uint32_t> ws_cull_bias, ws_cull_range, ws_cull_a1; // [widescreen.cull] std::set<uint32_t> ws_cull_screen_x; // [widescreen.cull] screen_x_sites + std::set<uint32_t> ws_cull_slti; // [widescreen.cull] slti_sites + std::vector<uint32_t> ws_cull_w_imms = { 0x140, 0x141 }; // [widescreen.cull] screen_w_imms + std::vector<uint32_t> ws_cull_h_imms = { 0xE0, 0xF1 }; // [widescreen.cull] screen_h_imms std::set<uint32_t> ws_backdrop_x; // [widescreen.backdrop] x_sites std::set<uint32_t> ws_backdrop_unsquash; // [widescreen.backdrop] unsquash_funcs bool ws_auto_screen_x_cull = false; // [widescreen.cull] auto_screen_x @@ -131,7 +134,8 @@ int main(int argc, char** argv) { ws_bg2d_stream_left_site = 0, ws_bg2d_stream_right_site = 0, ws_bg2d_bufbase_site = 0, - ws_bg2d_cap_site = 0; // [widescreen.bg2d] + ws_bg2d_cap_site = 0, + ws_bg2d_init_func = 0; // [widescreen.bg2d] std::filesystem::path out_dir = "generated"; if (!config_path.empty()) { @@ -146,6 +150,9 @@ int main(int argc, char** argv) { ws_cull_range.insert(cfg.ws_cull_range_sites.begin(), cfg.ws_cull_range_sites.end()); ws_cull_a1.insert(cfg.ws_cull_a1_sites.begin(), cfg.ws_cull_a1_sites.end()); ws_cull_screen_x.insert(cfg.ws_cull_screen_x_sites.begin(), cfg.ws_cull_screen_x_sites.end()); + ws_cull_slti.insert(cfg.ws_cull_slti_sites.begin(), cfg.ws_cull_slti_sites.end()); + ws_cull_w_imms = cfg.ws_cull_w_imms; + ws_cull_h_imms = cfg.ws_cull_h_imms; ws_backdrop_x.insert(cfg.ws_backdrop_x_sites.begin(), cfg.ws_backdrop_x_sites.end()); ws_backdrop_unsquash.insert(cfg.ws_backdrop_unsquash_funcs.begin(), cfg.ws_backdrop_unsquash_funcs.end()); ws_auto_screen_x_cull = ws_auto_screen_x_cull || cfg.ws_auto_screen_x_cull; @@ -157,6 +164,7 @@ int main(int argc, char** argv) { if (cfg.ws_bg2d_stream_right_site) ws_bg2d_stream_right_site = cfg.ws_bg2d_stream_right_site; if (cfg.ws_bg2d_bufbase_site) ws_bg2d_bufbase_site = cfg.ws_bg2d_bufbase_site; if (cfg.ws_bg2d_cap_site) ws_bg2d_cap_site = cfg.ws_bg2d_cap_site; + if (cfg.ws_bg2d_init_func) ws_bg2d_init_func = cfg.ws_bg2d_init_func; // [persist_options] init-store hook sites live in a dedicated // game_options.toml next to game.toml (the game's own native OPTION // settings, kept separate from game.toml/settings.toml). Best-effort: @@ -221,10 +229,14 @@ int main(int argc, char** argv) { ws_cull_range.insert(wscfg.ws_cull_range_sites.begin(), wscfg.ws_cull_range_sites.end()); ws_cull_a1.insert(wscfg.ws_cull_a1_sites.begin(), wscfg.ws_cull_a1_sites.end()); ws_cull_screen_x.insert(wscfg.ws_cull_screen_x_sites.begin(), wscfg.ws_cull_screen_x_sites.end()); + ws_cull_slti.insert(wscfg.ws_cull_slti_sites.begin(), wscfg.ws_cull_slti_sites.end()); + ws_cull_w_imms = wscfg.ws_cull_w_imms; + ws_cull_h_imms = wscfg.ws_cull_h_imms; ws_backdrop_x.insert(wscfg.ws_backdrop_x_sites.begin(), wscfg.ws_backdrop_x_sites.end()); ws_backdrop_unsquash.insert(wscfg.ws_backdrop_unsquash_funcs.begin(), wscfg.ws_backdrop_unsquash_funcs.end()); ws_auto_screen_x_cull = ws_auto_screen_x_cull || wscfg.ws_auto_screen_x_cull; ws_auto_backdrop_preload = ws_auto_backdrop_preload || wscfg.ws_auto_backdrop_preload; + if (wscfg.ws_bg2d_init_func) ws_bg2d_init_func = wscfg.ws_bg2d_init_func; fmt::print("ws-config: {} (backdrop_x sites={}, unsquash funcs={})\n", ws_config_path.string(), ws_backdrop_x.size(), ws_backdrop_unsquash.size()); } @@ -764,10 +776,14 @@ int main(int argc, char** argv) { codegen_config.split_mid_function_targets = !overlay_mode; codegen_config.overlay_mode = overlay_mode; codegen_config.ws_sprite_tag_funcs = ws_tag_funcs; + codegen_config.ws_bg2d_init_func = ws_bg2d_init_func; codegen_config.ws_cull_bias_sites = ws_cull_bias; codegen_config.ws_cull_range_sites = ws_cull_range; codegen_config.ws_cull_a1_sites = ws_cull_a1; codegen_config.ws_cull_screen_x_sites = ws_cull_screen_x; + codegen_config.ws_cull_slti_sites = ws_cull_slti; + codegen_config.ws_cull_w_imms = ws_cull_w_imms; + codegen_config.ws_cull_h_imms = ws_cull_h_imms; codegen_config.ws_backdrop_x_sites = ws_backdrop_x; codegen_config.ws_backdrop_unsquash_funcs = ws_backdrop_unsquash; codegen_config.ws_auto_screen_x_cull = ws_auto_screen_x_cull; diff --git a/recompiler/src/main_toml.cpp b/recompiler/src/main_toml.cpp new file mode 100644 index 00000000..48c5807f --- /dev/null +++ b/recompiler/src/main_toml.cpp @@ -0,0 +1,311 @@ +// psxrecomp-toml — PS-X EXE → game.toml generator. +// +// Analyzes a PS-X EXE and generates a complete game.toml config +// with auto-detected fields (load_address, entry_pc, text_size, +// stack_base) and an optional seeds file (JAL targets). +// +// Usage: +// psxrecomp-toml <exe> [--output <path>] [--seeds <path>] +// [--name <str>] [--id <str>] [--stdout] + +#include <cstdio> +#include <filesystem> +#include <fstream> +#include <set> +#include <string> +#include <vector> + +#include "fmt/format.h" +#include "ps1_exe_parser.h" + +namespace fs = std::filesystem; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +static void print_usage(const char* argv0) { + fmt::print(stderr, + "Usage: {} <PS1-EXE> [options]\n" + "\n" + "Options:\n" + " --output <path> Write game.toml to <path> (default: stdout)\n" + " --seeds <path> Also write JAL-target seed file to <path>\n" + " --name <str> Game name in TOML (default: derived from EXE)\n" + " --id <str> Game ID (default: auto-detect or empty)\n" + " --stdout Force output to stdout even with --output\n" + " --include-after-return\n" + " Add addresses after jr $ra to seeds (more coverage,\n" + " may include some data addresses)\n" + " -h, --help Show this help\n", + argv0); +} + +// Derive a game name from the EXE filename (fallback when --name not given). +static std::string name_from_exe(const fs::path& exe_path) { + std::string stem = exe_path.stem().string(); + // Replace underscores/hyphens with spaces + for (char& c : stem) { + if (c == '_' || c == '-') c = ' '; + } + // Capitalise first letter of each word + bool cap = true; + for (char& c : stem) { + if (c == ' ') { + cap = true; + } else if (cap) { + c = (char)std::toupper((unsigned char)c); + cap = false; + } + } + return stem; +} + +// Try to extract a Sony game ID from the EXE filename +// (e.g. "SCES_028.34" -> "SCES-02834", "SLUS-12345" -> "SLUS-12345"). +static std::string id_from_exe(const fs::path& exe_path) { + // Use the full filename (not stem) because Sony IDs use dots as + // field separators within the ID, not as extension separators + // (e.g. "SCES_028.34" -> full ID "SCES-02834"). + std::string fname = exe_path.filename().string(); + // Strip all non-alphanumeric characters, keeping alpha prefix + digits + std::string clean; + for (char c : fname) { + if (std::isalnum((unsigned char)c)) { + clean += (char)std::toupper((unsigned char)c); + } + } + if (clean.empty()) return fname; + // Re-format: prefix (letters) + "-" + number (digits) + // e.g. "SCES02834" -> "SCES-02834" + if (clean.find('-') == std::string::npos) { + size_t split = 0; + while (split < clean.size() && std::isalpha((unsigned char)clean[split])) + split++; + if (split > 0 && split < clean.size()) { + clean.insert(split, "-"); + } + } + return clean; +} + +// Scan the EXE code for JAL instructions and return the set of unique targets. +static std::set<uint32_t> find_jal_targets(const PSXRecomp::PS1Executable& exe) { + std::set<uint32_t> targets; + const uint32_t base = exe.load_address(); + const uint32_t end = exe.end_address(); + + for (uint32_t addr = base; addr + 4 <= end; addr += 4) { + auto word = exe.read_word(addr); + if (!word) continue; + const uint32_t raw = *word; + const uint32_t opcode = (raw >> 26) & 0x3F; + if (opcode == 0x03) { // JAL + const uint32_t target = (addr & 0xF0000000u) | ((raw & 0x03FFFFFFu) << 2); + if (target >= base && target < end) { + targets.insert(target); + } + } + } + return targets; +} + +// Scan the EXE code for addresses that immediately follow a `jr $ra` +// instruction (including its delay slot). These are likely function +// boundaries (the next function after a return). Returns them as seeds. +static std::set<uint32_t> find_after_return(const PSXRecomp::PS1Executable& exe) { + std::set<uint32_t> after_ret; + const uint32_t base = exe.load_address(); + const uint32_t end = exe.end_address(); + + for (uint32_t addr = base; addr + 8 <= end; addr += 4) { + auto word = exe.read_word(addr); + if (!word) continue; + if (*word == 0x03E00008u) { // jr $ra + const uint32_t next_func = addr + 8; // skip delay slot + if (next_func < end) { + after_ret.insert(next_func); + } + } + } + return after_ret; +} + +// --------------------------------------------------------------------------- +// Main +// --------------------------------------------------------------------------- + +int main(int argc, char* argv[]) { + // Parse arguments + fs::path exe_path; + fs::path output_path; + fs::path seeds_path; + std::string game_name; + std::string game_id; + bool force_stdout = false; + bool include_after_return = false; + + for (int i = 1; i < argc; i++) { + std::string arg = argv[i]; + if (arg == "--output" && i + 1 < argc) { + output_path = argv[++i]; + } else if (arg == "--seeds" && i + 1 < argc) { + seeds_path = argv[++i]; + } else if (arg == "--name" && i + 1 < argc) { + game_name = argv[++i]; + } else if (arg == "--id" && i + 1 < argc) { + game_id = argv[++i]; + } else if (arg == "--stdout") { + force_stdout = true; + } else if (arg == "--include-after-return") { + include_after_return = true; + } else if (arg == "-h" || arg == "--help") { + print_usage(argv[0]); + return 0; + } else if (arg[0] == '-') { + fmt::print(stderr, "Unknown option: {}\n", arg); + print_usage(argv[0]); + return 1; + } else { + exe_path = arg; + } + } + + if (exe_path.empty()) { + fmt::print(stderr, "Error: no EXE file specified\n"); + print_usage(argv[0]); + return 1; + } + + if (!fs::exists(exe_path)) { + fmt::print(stderr, "Error: EXE file not found: {}\n", exe_path.string()); + return 1; + } + + // Parse EXE header + std::string parse_err; + auto exe_opt = PSXRecomp::PS1ExeParser::parse_file(exe_path, parse_err); + if (!exe_opt) { + fmt::print(stderr, "Error parsing EXE: {}\n", parse_err); + return 1; + } + const auto& exe = *exe_opt; + + // Derive name/id from filename if not provided + if (game_name.empty()) game_name = name_from_exe(exe_path); + if (game_id.empty()) game_id = id_from_exe(exe_path); + + // Compute fields + const uint32_t load_address = exe.load_address(); + const uint32_t entry_pc = exe.entry_point(); + uint32_t text_size = exe.code_size(); + if (text_size & 0xFFF) { + text_size = (text_size + 0xFFF) & ~0xFFFu; + } + const uint32_t stack_base = exe.header.stack_base != 0 + ? exe.header.stack_base + : 0x801FFFF0u; + + // Generate TOML + const auto exe_rel = fs::relative(exe_path, fs::current_path()); + const std::string exe_str = exe_rel.empty() ? exe_path.filename().string() : exe_rel.string(); + + std::string toml; + toml += "# Auto-generated by psxrecomp-toml\n"; + toml += "# https://github.com/nstse/psxrecomp\n"; + toml += "\n"; + toml += "[game]\n"; + toml += fmt::format("name = \"{}\"\n", game_name); + toml += fmt::format("id = \"{}\"\n", game_id); + toml += fmt::format("exe = \"{}\"\n", exe_str); + toml += fmt::format("load_address = \"0x{:08X}\"\n", load_address); + toml += fmt::format("entry_pc = \"0x{:08X}\"\n", entry_pc); + toml += fmt::format("text_size = \"0x{:X}\"\n", text_size); + toml += fmt::format("stack_base = \"0x{:08X}\"\n", stack_base); + toml += "\n"; + toml += "[recompiler]\n"; + if (!seeds_path.empty()) { + const auto seeds_rel = fs::relative(seeds_path, fs::current_path()); + toml += fmt::format("seeds = \"{}\"\n", seeds_rel.string()); + } else { + toml += "# seeds = \"seeds/funcs.txt\"\n"; + } + toml += "strict = true\n"; + toml += "out_dir = \"generated\"\n"; + toml += "\n"; + toml += "[runtime]\n"; + toml += "# debug_port = 4370\n"; + toml += "window_title = \"" + game_name + " Recompiled\"\n"; + toml += "bios_hle = false\n"; + + // Write output + if (!output_path.empty() && !force_stdout) { + std::ofstream ofs(output_path); + if (!ofs) { + fmt::print(stderr, "Error: cannot write to {}\n", output_path.string()); + return 1; + } + ofs << toml; + ofs.close(); + fmt::print("Wrote {}\n", output_path.string()); + } else { + fmt::print("{}", toml); + } + + // Generate seeds file if requested + std::set<uint32_t> jal_targets; + std::set<uint32_t> after_ret; + if (!seeds_path.empty()) { + jal_targets = find_jal_targets(exe); + after_ret = find_after_return(exe); + + std::set<uint32_t> seeds = jal_targets; + if (include_after_return) { + seeds.insert(after_ret.begin(), after_ret.end()); + } + seeds.insert(entry_pc); + + // Write seeds file + const fs::path seeds_dir = seeds_path.parent_path(); + if (!seeds_dir.empty() && !fs::exists(seeds_dir)) { + fs::create_directories(seeds_dir); + } + + std::ofstream sfs(seeds_path); + if (!sfs) { + fmt::print(stderr, "Error: cannot write seeds to {}\n", seeds_path.string()); + return 1; + } + sfs << fmt::format("# Seeds for {} ({})\n", game_name, game_id); + sfs << fmt::format("# JAL targets: {}\n", jal_targets.size()); + if (include_after_return) { + sfs << fmt::format("# After-return: {}\n", after_ret.size()); + sfs << fmt::format("# (overlap with JAL: {})\n", + jal_targets.size() + after_ret.size() - seeds.size()); + } + sfs << fmt::format("# Total seeds: {}\n", seeds.size()); + sfs << "# Auto-generated by psxrecomp-toml\n\n"; + for (uint32_t s : seeds) { + sfs << fmt::format("0x{:08X}\n", s); + } + sfs.close(); + fmt::print("Wrote {} seeds to {}\n", seeds.size(), seeds_path.string()); + } + + // Summary + fmt::print(stderr, "EXE: {} ({})\n", exe_path.string(), game_id); + fmt::print(stderr, "Entry: 0x{:08X}\n", entry_pc); + fmt::print(stderr, "Load: 0x{:08X}\n", load_address); + fmt::print(stderr, "Text: 0x{:X} ({} KB)\n", exe.code_size(), exe.code_size() / 1024); + fmt::print(stderr, "Stack: 0x{:08X}\n", stack_base); + if (!seeds_path.empty()) { + const uint32_t seed_total = jal_targets.size() + + (include_after_return ? after_ret.size() : 0) + 1; // +1 for entry + fmt::print(stderr, "Seeds: {} JAL targets", jal_targets.size()); + if (include_after_return) + fmt::print(stderr, ", {} after-return", after_ret.size()); + fmt::print(stderr, " -> {}\n", seeds_path.string()); + } + + return 0; +} diff --git a/recompiler/src/ps1_exe_parser.cpp b/recompiler/src/ps1_exe_parser.cpp index d4cdad8b..65b23144 100644 --- a/recompiler/src/ps1_exe_parser.cpp +++ b/recompiler/src/ps1_exe_parser.cpp @@ -160,6 +160,22 @@ std::optional<PS1Executable> PS1ExeParser::parse_buffer( // Copy header (first 2048 bytes) std::memcpy(&exe.header, buffer.data(), sizeof(PS1ExeHeader)); + // Some EXEs (e.g. Kula World SCES-01000) store KUSEG addresses + // (0x00011000) instead of KSEG0 (0x80011000). KUSEG 0x0-0x1FFFFFFF + // mirrors the same physical RAM, so normalize to KSEG0 before + // validation. Only remap addresses that fall inside the 2 MB RAM + // mirror; 0 stays 0 (unused fields like initial_gp). + auto to_kseg0 = [](uint32_t& addr) { + if (addr != 0 && addr < 0x00200000) addr |= 0x80000000; + }; + to_kseg0(exe.header.initial_pc); + to_kseg0(exe.header.initial_gp); + to_kseg0(exe.header.load_address); + to_kseg0(exe.header.memfill_start); + to_kseg0(exe.header.initial_sp); + to_kseg0(exe.header.initial_fp); + to_kseg0(exe.header.stack_base); + // Validate header if (!validate_header(exe.header, error_msg)) { return std::nullopt; diff --git a/runtime/CMakeLists.txt b/runtime/CMakeLists.txt index 31a359f3..c3b013bc 100644 --- a/runtime/CMakeLists.txt +++ b/runtime/CMakeLists.txt @@ -80,6 +80,7 @@ if(NOT MSVC AND EXISTS "${BEETLE_LIB}") src/frame_pacing.c src/parity_trace.c src/device_trace.c + src/audio_trace.c ${LIBRETRO_COMMON_SOURCES} ) diff --git a/runtime/include/audio_trace.h b/runtime/include/audio_trace.h new file mode 100644 index 00000000..61540c7a --- /dev/null +++ b/runtime/include/audio_trace.h @@ -0,0 +1,123 @@ +/* + * audio_trace.h - always-on audio observability rings (PCM taps + event ring). + * + * Port of the snesrecomp audio-campaign measurement harness (audio_trace.{c,h}, + * commit 67ead27 lineage) to the PSX pipeline, following the cross-system tap + * model in F:/Projects/_audio_round2/ROUND2_PLAN.md: always-on ring buffers + * recording every sample/event from process start, dumped AFTER the fact via + * TCP debug commands — never arm-then-reproduce. + * + * Taps (PSX pipeline): + * AUDIO_TAP_SPU_OUT "T1" — canonical spu_render() output @44100, recorded + * inside spu_render so every caller (pump, fade tail) is + * covered, PRE any host-side fade/mute gain. + * AUDIO_TAP_CD_IN "T2" — decoded XA/CD-DA PCM pushed onto the SPU CD + * input bus (already resampled to 44100 by the producer). + * AUDIO_TAP_HOST "T3" — the exact bytes handed to the host audio API + * (post fade ramps; what SDL_QueueAudio receives). + * + * The same module compiles into psx-beetle, where AUDIO_TAP_SPU_OUT records + * the libretro audio_batch callback — Beetle's fully-mixed reference SPU + * output — giving symmetric `audio_*` commands on ports 4370 and 4380. + * + * Threading: each tap has a single writer. Readers (debug-server thread) + * take a stable [oldest, head) snapshot; the PCM rings hold ~95 s at 44100, + * so a dump cannot be lapped mid-read in practice. Indices are C11 atomics + * (release on publish, acquire on read). + */ +#ifndef PSX_AUDIO_TRACE_H +#define PSX_AUDIO_TRACE_H + +#include <stdint.h> + +#ifdef __cplusplus +extern "C" { +#endif + +enum { + AUDIO_TAP_SPU_OUT = 0, + AUDIO_TAP_CD_IN = 1, + AUDIO_TAP_HOST = 2, + AUDIO_TAP_VOICES = 3, /* T0: 24-voice sum only (pre CD mix, pre main) */ + AUDIO_TAP_COUNT = 4 +}; + +/* Event kinds. REG_WRITE covers every SPU register store (addr/value), which + * subsumes KEYON/KEYOFF/pitch/volume writes — the semantic KON ring stays in + * spu.c. The rest instrument the host delivery path. */ +enum { + AUDIO_EV_NONE = 0, + AUDIO_EV_REG_WRITE = 1, /* a=SPU reg addr, b=value */ + AUDIO_EV_RENDER = 2, /* a=frames rendered, b=host queued bytes before */ + AUDIO_EV_PUMP_SKIP = 3, /* backpressure skip; a=queued bytes */ + AUDIO_EV_UNDERRUN = 4, /* host queue empty at pump entry; a=queued bytes */ + AUDIO_EV_MUTE = 5, /* turbo mute engaged; a=fade tail frames */ + AUDIO_EV_UNMUTE = 6, /* pumping resumed; a=fade-in frames pending */ + AUDIO_EV_CD_PUSH = 7, /* a=frames pushed, b=CD ring fill after */ + AUDIO_EV_DMA_WRITE = 8, /* SPU RAM DMA; a=words, b=transfer addr after */ +}; + +typedef struct { + uint64_t seq; /* monotone event counter */ + uint64_t sample_idx; /* AUDIO_TAP_SPU_OUT frames produced when recorded */ + uint32_t frame; /* host-noted frame counter (vblank / retro_run) */ + uint16_t kind; + uint16_t reserved; + uint32_t a; + uint32_t b; +} AudioTraceEvent; + +typedef struct { + /* Per-tap PCM production. */ + uint64_t tap_frames[AUDIO_TAP_COUNT]; /* frames ever recorded */ + uint64_t tap_nonzero[AUDIO_TAP_COUNT]; /* frames with |L| or |R| > 0 */ + uint64_t tap_audible[AUDIO_TAP_COUNT]; /* frames with |L| or |R| > 256 (~ -42 dBFS) */ + int32_t tap_peak[AUDIO_TAP_COUNT]; /* running peak |sample| */ + /* Host pump behavior. */ + uint64_t pump_calls; + uint64_t pump_skips; /* backpressure early-outs */ + uint64_t underruns; /* pump entered with 0 queued bytes (post first audio) */ + uint32_t queue_hiwater; /* max queued bytes observed at pump entry */ + uint32_t queue_lowater; /* min queued bytes observed at pump entry (post first audio) */ + uint64_t mute_events; + uint64_t unmute_events; + /* Event ring. */ + uint64_t events_total; +} AudioTraceStats; + +void audio_trace_init(void); + +/* Record `frames` interleaved stereo s16 frames into tap `tap`. */ +void audio_trace_pcm(int tap, const int16_t *stereo, int frames); + +/* Record a pipeline event (timestamps applied internally). */ +void audio_trace_event(uint16_t kind, uint32_t a, uint32_t b); + +/* Host frame tag for event timestamps (call once per vblank / retro_run). */ +void audio_trace_note_frame(uint32_t frame); + +void audio_trace_get_stats(AudioTraceStats *out); +uint64_t audio_trace_tap_total(int tap); +uint64_t audio_trace_events_total(void); + +/* Taps default to 44100 Hz; the host tap may run at the device rate in + * bridge/pull mode. The WAV dumper stamps this rate — a mislabeled header + * silently poisons every offline A/B. */ +void audio_trace_set_tap_rate(int tap, uint32_t rate); +uint32_t audio_trace_tap_rate(int tap); + +/* Copy the most recent `max` events, oldest first. Returns count copied. */ +uint32_t audio_trace_events_get(AudioTraceEvent *out, uint32_t max); + +/* Write a tap slice as a stereo s16 WAV at the tap's rate. start=-1,count=0 + * dumps the whole currently-buffered ring. `start`/`count` are absolute frame + * indices on the tap's production timeline. Returns frames written, or -1 on + * error (unwritable path / empty ring / slice already evicted). */ +int64_t audio_trace_dump_wav(int tap, const char *path, + int64_t start, uint64_t count); + +#ifdef __cplusplus +} +#endif + +#endif /* PSX_AUDIO_TRACE_H */ diff --git a/runtime/include/bios_hle.h b/runtime/include/bios_hle.h index 6d6211b2..dffc75ce 100644 --- a/runtime/include/bios_hle.h +++ b/runtime/include/bios_hle.h @@ -1,7 +1,7 @@ /* bios_hle.h — opt-in High-Level Emulation tier for PSX BIOS kernel services. * - * CLAUDE.md §0 AMENDMENT 2026-07-02 (the gbarecomp model, see - * F:/Projects/gbarecomp/gbarecomp/src/runtime/bios_hle.h): the recompiled real + * CLAUDE.md §0 AMENDMENT 2026-07-02 (the gbarecomp model, see the sibling + * gbarecomp project's src/runtime/bios_hle.h): the recompiled real * BIOS (LLE) is the DEFAULT, the reference implementation, and the * correctness/timing oracle. HLE is an *alternative backend*, selected * per-game via `[runtime] bios_hle = true` in game.toml (PSX_BIOS_HLE env diff --git a/runtime/include/cdrom.h b/runtime/include/cdrom.h index 265a1d6b..7740bba7 100644 --- a/runtime/include/cdrom.h +++ b/runtime/include/cdrom.h @@ -8,6 +8,9 @@ extern "C" { #endif void cdrom_init(const char* cue_path); +/* Disc license region for GetID ("SCEE"/"SCEA"/"SCEI"); derived from the + * mounted disc's boot serial at launch. Defaults to "SCEA" (SCPH1001). */ +void cdrom_set_disc_scex(const char scex[4]); /* Set disc speed multiplier immediately. divisor=0 → instant, 1 → 1x, 2 → 2x. */ void cdrom_set_speed(int divisor); diff --git a/runtime/include/cpu_state.h b/runtime/include/cpu_state.h index 79634368..844e9dc5 100644 --- a/runtime/include/cpu_state.h +++ b/runtime/include/cpu_state.h @@ -31,6 +31,8 @@ enum { SLJIT_HLP_COP2, /* psx_sljit_cop2 — COP2/GTE, LWC2/SWC2 */ SLJIT_HLP_WS_CULL, /* psx_ws_cull_sltiu — widescreen render-funnel cull */ SLJIT_HLP_CALL, /* psx_sljit_call — jal/jalr call contract */ + SLJIT_HLP_WS_CULL_SLTI, /* psx_ws_cull_slti — signed funnel right edge (appended) */ + SLJIT_HLP_WS_CULL_BLTZ, /* psx_ws_cull_bltz — signed funnel left edge (appended) */ SLJIT_HLP_COUNT }; diff --git a/runtime/include/dirty_ram_interp.h b/runtime/include/dirty_ram_interp.h index 94e8f532..ccc7b3ba 100644 --- a/runtime/include/dirty_ram_interp.h +++ b/runtime/include/dirty_ram_interp.h @@ -50,15 +50,20 @@ int dirty_ram_dispatch(CPUState* cpu, uint32_t addr, uint32_t stop_addr); * Overlay region [OVERLAY_REGION_FLOOR, RAM_SIZE): game overlays loaded by * CD DMA (dirty_ram_mark_executable_range). * - * Main-EXE text [0x10000, OVERLAY_REGION_FLOOR) is statically recompiled and - * is NEVER a capture/candidate window. The floor equals the main EXE text end - * (Tomba: load 0x10000 + text 0x88000). + * Main-EXE text [0x10000, OVERLAY_REGION_FLOOR): CLEAN pages are statically + * recompiled and never captured. But games that discard their boot/init code + * load gameplay overlays OVER this region (Tomba 2: hot gameplay code at + * 0x1DA88/0x24548/0x3116C, well below its 0x38800 floor). Post-baseline + * (dirty_ram_clear_image_baseline) a DIRTY page here is by definition live + * overlay code — the compiled image is stale for it — so dirty boot-text + * pages ARE a capture/candidate window (2026-07-06; before this they could + * only interpret, which dominated wall time at ~300ns/insn). * * The interpreter's LOCAL-FLOW gates (is_local_dirty_target / - * allow_local_dirty_flow) intentionally use OVERLAY_REGION_FLOOR alone, NOT - * this window predicate: kernel-RAM code runs in exception context where the - * verified per-block dispatch cadence is preserved. Native coverage for the - * kernel window comes from the overlay loader, not from interp chaining. */ + * phys_is_overlay_flow_region) use DIRTY_RAM_KERNEL_WINDOW_END alone: only + * the kernel window stays per-block (it runs in exception context where the + * verified dispatch cadence is delicate). Native coverage for the kernel + * window comes from the overlay loader, not from interp chaining. */ #define DIRTY_RAM_KERNEL_WINDOW_END 0x00010000u /* The overlay-region floor is the END of THIS game's statically-recompiled @@ -76,17 +81,34 @@ int dirty_ram_dispatch(CPUState* cpu, uint32_t addr, uint32_t stop_addr); #define OVERLAY_REGION_FLOOR_DEFAULT 0x00098000u extern uint32_t g_overlay_region_floor; #define OVERLAY_REGION_FLOOR (g_overlay_region_floor) -static inline int overlay_cache_window_contains(uint32_t phys) { - return phys < DIRTY_RAM_KERNEL_WINDOW_END || phys >= OVERLAY_REGION_FLOOR; -} /* Test whether a given physical kernel-RAM address is in a page that was * written-to since boot. Defined in memory.c. */ int dirty_ram_is_dirty(uint32_t phys); + +/* Capture/candidate window membership. Kernel window and overlay region are + * unconditional; main-EXE text [KERNEL_WINDOW_END, FLOOR) is included only + * when the page is dirty — i.e. a gameplay overlay overwrote the boot text + * after the game-start baseline, so the bytes there are runtime overlay code + * (see the window model note above). Clean boot text never enters the window + * (it runs compiled; capturing it would violate the static-first design). */ +static inline int overlay_cache_window_contains(uint32_t phys) { + return phys < DIRTY_RAM_KERNEL_WINDOW_END + || phys >= OVERLAY_REGION_FLOOR + || dirty_ram_is_dirty(phys); +} uint32_t dirty_ram_get_bitmap(void); uint32_t dirty_ram_get_bitmap_word(uint32_t word_index); uint32_t dirty_ram_get_bitmap_word_count(void); void dirty_ram_mark_executable_range(uint32_t phys, uint32_t len); +void dirty_ram_register_text_image(uint32_t phys_lo, const uint8_t *bytes, + uint32_t len); +int dirty_ram_text_native_ok(uint32_t phys); +/* Bless an intentional runtime data patch (e.g. text_xlate string/glyph tables) + * into the text reference image so it is not mistaken for self-modifying code. */ +void dirty_ram_text_bless(uint32_t phys, const uint8_t *bytes, uint32_t len); +uint64_t dirty_ram_text_native_blocked(void); +uint32_t dirty_ram_text_diverged_pages(void); /* Counters for visibility / TCP debug. Increment in interpreter; expose * via debug_server.c if helpful. */ diff --git a/runtime/include/gpu.h b/runtime/include/gpu.h index a7d9a2ab..709f373d 100644 --- a/runtime/include/gpu.h +++ b/runtime/include/gpu.h @@ -64,10 +64,18 @@ typedef struct { uint32_t seq; uint32_t src_addr; /* RAM/MMIO source address of command header, if known */ uint32_t pc; /* g_debug_last_store_pc when command completes */ + uint32_t func; /* g_debug_current_func_addr at issue (executing guest fn) */ + uint32_t ra; /* guest $ra at issue: direct caller of the leaf GP0 helper */ uint8_t opcode; uint8_t n_words; /* total command length; >MAX means truncated */ uint16_t pad; uint32_t cmd[GPU_GP0_RING_MAX_WORDS]; + /* Builder attribution (populated only for VRAM->VRAM copies, op 0x80): + * bounded guest-stack unwind of validated return addresses, innermost + * first. Lets a caller skip the libgpu funnel band and name the game-level + * routine that assembled the copy. Zero for non-copy commands. */ + uint32_t bld[6]; + uint32_t csp; /* raw guest $sp at copy time (diagnostic) */ } GpuGp0RingEntry; uint64_t gpu_gp0_ring_total(void); @@ -131,16 +139,55 @@ int gpu_ws_present_native_43(void); /* Per-side X cull-margin (screen/world units) emitted into the game's draw- * cull immediates by the recompiler ([widescreen.cull]); 0 unless stretching. */ int psx_ws_x_margin(void); +void gpu_ws_set_cull_guard_pixels(int pixels); +void gpu_ws_set_explicit_cull_sites(const uint32_t *bias, int nbias, + const uint32_t *slti, int nslti); +int psx_ws_is_cull_bias_site(uint32_t pc); +int psx_ws_is_cull_slti_site(uint32_t pc); /* Shared render-funnel screen-X cull widening ([widescreen.cull] auto_screen_x): * the gcc emit, the sljit JIT, and the interpreter all route a flagged * `sltiu rt, sx, 0x140/0x141` through this one helper so every overlay execution * path widens identically. Returns the sltiu verdict (1 = keep). 0 at 4:3. */ int psx_ws_cull_sltiu(uint32_t sx, uint32_t imm); +/* Signed-funnel variants (min/max + center±halfwidth idioms — see + * ws_cull_detect.h): right-edge widen for `slti v, minSX, W` and left-edge + * widen for the paired `bltz maxSX` reject. Identity at 4:3. */ +int psx_ws_cull_slti(uint32_t sx, uint32_t imm); +int psx_ws_cull_bltz(uint32_t v); /* True if a run of instruction words carries the screen-extent reject signature - * (a sltiu 0x140/0x141 AND a sltiu 0xE0/0xF1). Used by sljit/interp to gate the - * widening to real render funnels. */ + * (a width compare AND a height compare from the configured immediate sets). + * Used by sljit/interp to gate the widening to real render funnels. */ int psx_ws_func_has_screen_cull(const uint32_t *words, int n); +/* Classify words[idx] as an X left-edge reject bltz (runtime imm sets). */ +int psx_ws_cull_bltz_at(const uint32_t *words, int n, int idx); +/* Per-game cull signature immediates ([widescreen.cull] screen_w_imms / + * screen_h_imms); defaults 0x140/0x141 + 0xE0/0xF1. */ +void gpu_ws_set_cull_imms(const uint32_t *w, int nw, const uint32_t *h, int nh); +int psx_ws_is_cull_w_imm(uint32_t imm); +/* Per-game opt-in gates for the pattern-scanned interp/sljit widen hooks + * (auto_screen_x cull + auto_backdrop preload). Default OFF: a title that + * never opted in must never have its live code pattern-scanned and rewritten. */ +void gpu_ws_set_auto_hooks(int cull_on, int backdrop_on); +int psx_ws_auto_cull_on(void); +/* GTE-activity gameplay detector ([widescreen] gte_game_mode) for 3D titles + * with no sprite-tag helper: gte.cpp notes every RTPS/RTPT projection; a frame + * that projects enough vertices is stamped as gameplay. */ +void gpu_ws_set_gte_game_mode(int on); +void psx_ws_note_gte_project(int nverts); +/* Native-wide HUD corner re-anchoring ([widescreen] nw_hud_corners): push + * outer-third screen-space HUD sprites out to the true wide-frame corners + * (they otherwise sit inset by the reveal). Runtime-only. Off by default. */ +void gpu_ws_set_nw_hud_corners(int on); +/* Targeted alternative for sprite-heavy 2D games: corner-anchor only primitives + * whose ordering-table packet lives in the configured half-open RAM range. */ +void gpu_ws_set_nw_left_hud_packet_range(uint32_t lo, uint32_t hi); +void gpu_ws_begin_linked_list(void); +/* Native-wide full-frame 2D backdrop stretch ([widescreen] nw_backdrop): + * stretch a screen-space quad that covers the whole 4:3 framebuffer (sky + * gradient / backdrop image) to fill the wide frame, so it no longer + * pillarboxes at the reveal margins. Runtime-only. Off by default. */ +void gpu_ws_set_nw_backdrop(int on); /* Backdrop screen-X correction ([widescreen.backdrop] x_sites). The parallax * 2D backdrop layer computes screen-X without the GTE, so it misses the diff --git a/runtime/include/gpu_render.h b/runtime/include/gpu_render.h index 81afb281..4953270a 100644 --- a/runtime/include/gpu_render.h +++ b/runtime/include/gpu_render.h @@ -98,6 +98,9 @@ void gr_wide_configure(int wide_w, int offset); void gr_wide_set_target(int base_x); void gr_wide_disable_target(void); void gr_wide_clear(int base_x, int y, int h, uint16_t color); +/* sides: bit 0 = left synthetic margin, bit 1 = right. */ +void gr_wide_clear_margins(int base_x, int y, int h, uint16_t color, int sides); +int gr_wide_dump_full(uint32_t *out, int cap_pixels, int *ow, int *oh, int base_x); int gr_render_wide_display(uint32_t *out, int pitch, int base_x, int disp_y, int disp_h); @@ -165,8 +168,14 @@ typedef struct GpuRenderBackend { void (*wide_set_target)(int base_x); void (*wide_disable_target)(void); void (*wide_clear)(int base_x, int y, int h, uint16_t color); + void (*wide_clear_margins)(int base_x, int y, int h, uint16_t color, int sides); int (*render_wide_display)(uint32_t *out, int pitch, int base_x, int disp_y, int disp_h); + /* Dump the ENTIRE wide compositor surface for base_x (all double-buffer + * bands + both margins), g_wide_w x VRAM_H. Debug/inspection only; returns + * pixel count, writes width/height to ow/oh. NULL if unsupported. */ + int (*wide_dump_full)(uint32_t *out, int cap_pixels, int *ow, int *oh, + int base_x); } GpuRenderBackend; #ifdef __cplusplus diff --git a/runtime/include/gpu_sw_renderer.h b/runtime/include/gpu_sw_renderer.h index a398bfab..8fe351bb 100644 --- a/runtime/include/gpu_sw_renderer.h +++ b/runtime/include/gpu_sw_renderer.h @@ -98,6 +98,7 @@ void sw_wide_configure(int wide_w, int offset); void sw_wide_set_target(int base_x); void sw_wide_disable_target(void); void sw_wide_clear(int base_x, int y, int h, uint16_t color); +void sw_wide_clear_margins(int base_x, int y, int h, uint16_t color, int sides); /* MMX6 native-wide reveal-column tile: rasterize a 16x16 textured tile into the * wide surface ONLY (canonical VRAM + hi-res mirror untouched). See definition. */ void sw_wide_emit_tile(int band_y, int x, int y, int u, int v, @@ -105,6 +106,7 @@ void sw_wide_emit_tile(int band_y, int x, int y, int u, int v, uint32_t color24, int semi_trans, int semi_mode, int raw_texture); int sw_render_wide_display(uint32_t* out_pixels, int out_pitch, int base_x, int disp_y, int disp_h); +int sw_wide_dump_full(uint32_t* out, int cap_pixels, int* ow, int* oh, int base_x); int sw_wide_width(void); #ifdef __cplusplus diff --git a/runtime/include/psx_cycles.h b/runtime/include/psx_cycles.h index 78616cd5..1b130b51 100644 --- a/runtime/include/psx_cycles.h +++ b/runtime/include/psx_cycles.h @@ -29,6 +29,19 @@ void psx_devices_mmio_sync(void); /* Read accessor for telemetry. */ uint64_t psx_get_cycle_count(void); +/* Idle-loop cycle skip (see psx_cycles.c "Idle-loop cycle skip"). Called by + * psx_check_interrupts with the resume PC of the current check; detects + * side-effect-free poll loops and fast-forwards guest time to the next + * internal device event in whole loop quanta (bit-exact vs executing them). + * Counters/toggle are the TCP `idle_skip` command's surface. */ +struct CPUState; +void psx_idle_note_check(struct CPUState *cpu, uint32_t check_pc); +extern int g_idle_skip_enabled; +extern uint64_t g_idle_skip_count; +extern uint64_t g_idle_skip_cycles; +extern uint32_t g_idle_skip_last_pc; +extern uint32_t g_idle_skip_last_quantum; + /* Save-state restore: re-anchor the deadline device model after psx_cycle_count * is overwritten from a snapshot (call once, right after boot_state_load). */ void psx_cycles_resync_after_restore(void); diff --git a/runtime/include/psx_keybinds.h b/runtime/include/psx_keybinds.h new file mode 100644 index 00000000..a9d186ba --- /dev/null +++ b/runtime/include/psx_keybinds.h @@ -0,0 +1,102 @@ +#pragma once +#include <stdint.h> +#include <SDL.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * psx_keybinds — configurable KEYBOARD -> PlayStation DualShock keybinds. + * + * INI-driven, generated next to the exe as `keybinds.ini` on first run. Covers + * every DualShock input: the D-pad, the four face buttons (Cross/Circle/ + * Square/Triangle), the shoulders (L1/L2/R1/R2), Start/Select, the stick + * clicks (L3/R3), and — for the keyboard analog path — the left and right + * analog-stick directions. Each input maps to one SDL_Scancode; "None" + * (SDL_SCANCODE_UNKNOWN) leaves it unbound. + * + * This is the KEYBOARD map only. Game-controller (gamepad) button mapping is a + * separate concern handled by input.ini (see main.cpp load_input_config). + * + * Two players. Player 1 defaults reproduce the framework's historical hardcoded + * keyboard mapping (so out-of-the-box behaviour is unchanged); Player 2 is + * unbound by default (fill it in to drive a second keyboard player). + * + * The launcher's Controls page edits these live through the rebind API below + * (get/set/reset/save) and persists to the same keybinds.ini; the runtime + * re-reads the file at startup via psx_keybinds_init. + */ + +/* Button indices — stable order, matches the order keybinds.ini writes and the + * order the launcher's rebind chips are laid out. Keep in sync with kButtons[] + * in psx_keybinds.c. */ +enum PsxKeybindButton { + PSX_KB_UP = 0, PSX_KB_DOWN, PSX_KB_LEFT, PSX_KB_RIGHT, + PSX_KB_CROSS, PSX_KB_CIRCLE, PSX_KB_SQUARE, PSX_KB_TRIANGLE, + PSX_KB_L1, PSX_KB_R1, PSX_KB_L2, PSX_KB_R2, + PSX_KB_L3, PSX_KB_R3, + PSX_KB_START, PSX_KB_SELECT, + PSX_KB_LS_UP, PSX_KB_LS_DOWN, PSX_KB_LS_LEFT, PSX_KB_LS_RIGHT, + PSX_KB_RS_UP, PSX_KB_RS_DOWN, PSX_KB_RS_LEFT, PSX_KB_RS_RIGHT, + PSX_KB_COUNT +}; + +typedef struct { + SDL_Scancode up, down, left, right; + SDL_Scancode cross, circle, square, triangle; + SDL_Scancode l1, r1, l2, r2; + SDL_Scancode l3, r3; + SDL_Scancode start, select; + SDL_Scancode ls_up, ls_down, ls_left, ls_right; /* left analog stick */ + SDL_Scancode rs_up, rs_down, rs_left, rs_right; /* right analog stick */ +} PsxPlayerBinds; + +typedef struct { + PsxPlayerBinds p1; + PsxPlayerBinds p2; +} PsxKeyBinds; + +/* Initialize from <exe_dir>/keybinds.ini. Generates a default file if one does + * not exist. exe_path may be NULL/argv[0]/the exe directory itself — the .ini is + * placed in that file's directory (or the path itself when it is a directory). */ +void psx_keybinds_init(const char *exe_path); + +/* Read-only view of the current bindings. */ +const PsxKeyBinds *psx_keybinds_get(void); + +/* ── Runtime read helpers (keyboard -> PSX pad), player is 1 or 2 ──────────── */ + +/* Build the 16-bit ACTIVE-LOW PSX button word for `player` from the SDL + * keyboard state (bits per the standard PSX pad word; unbound inputs never + * assert). Returns 0xFFFF (all released) if `keys` is NULL or player is out of + * range. AND this into the pad word (an unpressed source leaves bits high). */ +uint16_t psx_keybinds_pad_word(const uint8_t *keys, int player); + +/* Fill analog-stick bytes (0x80 centred): out[0]=LX, out[1]=LY, out[2]=RX, + * out[3]=RY, from the player's stick-direction binds. A bound direction held + * drives that axis to the extreme (0x00 / 0xFF). Untouched axes stay centred. + * Only overwrites axes that have a live bound press, so callers can pre-seed. */ +void psx_keybinds_sticks(const uint8_t *keys, int player, uint8_t out[4]); + +/* True if any of the player's D-pad direction binds is currently held (used by + * the hybrid pad-mode auto-switch: a D-pad press flips the pad to digital). */ +int psx_keybinds_dpad_active(const uint8_t *keys, int player); + +/* ── Rebind API (launcher Controls page) ──────────────────────────────────── */ +/* Buttons are indexed 0..PSX_KB_COUNT-1 (see PsxKeybindButton). Scancodes, not + * keycodes (keybinds.ini stores SDL scancode names). */ +int psx_keybinds_button_count(void); +const char *psx_keybinds_button_name(int button); /* "up".."rs_right" */ +const char *psx_keybinds_button_label(int button); /* pretty, e.g. "Cross" */ +SDL_Scancode psx_keybinds_get_button(int player, int button); +void psx_keybinds_set_button(int player, int button, SDL_Scancode sc); +/* Reset one player's bindings to the built-in defaults (P2 = all unbound). */ +void psx_keybinds_reset_player(int player); +/* Persist the current bindings to keybinds.ini (path resolved by + * psx_keybinds_init; call that first). */ +void psx_keybinds_save(void); + +#ifdef __cplusplus +} +#endif diff --git a/runtime/include/psx_scheduler.h b/runtime/include/psx_scheduler.h index 9b4bc76e..587b972b 100644 --- a/runtime/include/psx_scheduler.h +++ b/runtime/include/psx_scheduler.h @@ -62,6 +62,15 @@ typedef struct { extern jmp_buf g_scheduler_jmpbuf; extern psx_sched_escape_t g_sched_escape; +/* Deferred cooperative thread switch (Ape Escape memcard fix #2). Thin exports + * over the scheduler's private TCB save/restore + PCB[0] accessors, so the + * interrupt path can re-save a deferred thread cleanly and re-point the kernel + * current-thread pointer at the outermost dispatch boundary. See traps.c. */ +struct CPUState; +void psx_sched_save_context(struct CPUState* cpu, uint32_t tcb, uint32_t resume_pc); +void psx_sched_set_current_tcb(struct CPUState* cpu, uint32_t tcb); +uint32_t psx_sched_current_tcb(struct CPUState* cpu); + /* Fail-closed guard: is `pc` a valid, dispatchable guest resume point? A TCB * EPC / resume PC that is NOT dispatchable (0, the exception sentinel, or not a * known function/re-enterable block-leader entry) is an invariant violation — diff --git a/runtime/include/text_xlate.h b/runtime/include/text_xlate.h new file mode 100644 index 00000000..7cc42900 --- /dev/null +++ b/runtime/include/text_xlate.h @@ -0,0 +1,96 @@ +/* text_xlate.h — on-the-fly string translation / localization (framework). + * + * A reusable psxrecomp feature: capture the game's source strings from guest + * memory and substitute translated bytes on the fly so a JP-only title can be + * played in the target language. See docs/STRING_TRANSLATION.md for the design. + * + * Mechanism (all always-on, hung off the psx_dispatch chokepoint): + * - CAPTURE: on every dispatch, scan the arg registers for pointers to source + * text records (encoding-profile validated). Every distinct record is hashed + * (FNV-1a64) and recorded into an in-memory inventory ring queryable over the + * TCP debug server (the "enumerate every string" authoring source). Never + * arm-then-run — the ring records from boot; the debug command queries it. + * - APPLY: when an arg points at a record present in the loaded translation + * table, transcode the target-language string to the game's glyph codes, + * write it into transient guest-stack scratch (below $sp), and repoint the + * arg register — the ORIGINAL draw routine then renders the replacement. + * KV-gated, so a stray pointer that isn't in the table is left untouched. + * - GLYPH-LABEL PATCH: UI text drawn glyph-by-glyph from a fixed-stride, + * space-padded EXE table (level/stage names) has no string pointer and no + * terminator, so the arg-scan hook can never see it. Instead each confirmed + * slot is patched in guest RAM once the EXE region is resident: verify the + * expected JP bytes are present, then overwrite exactly the slot width with + * target-language fullwidth Shift-JIS re-padded to that width. The game's own + * per-glyph routine then draws the translation. Fed by [[glyph_label]] TOML + * entries; gated on language; never spills past the confirmed slot bounds. + * - VRAM-STRIP PATCH: some UI text is PRE-RENDERED PIXELS in a texture asset + * (e.g. Tsumu's HUD labels live in the glyph-sheet image the game uploads to + * VRAM and samples with quads) — there are no glyph codes anywhere, so + * neither hook above can ever see it. Instead the patch rides the game's own + * upload: on completion of every CPU→VRAM image transfer (GP0 0xA0) any + * configured rect contained in the upload is verified against its expected + * JP pixels and, only on an exact match, rewritten with the target-language + * pixels through the same renderer write path (software/GL/supersample + * coherent). Re-applies automatically whenever the game re-uploads the + * asset — this is a source-path patch, not a one-shot VRAM poke. Fed by + * [[vram_patch]] TOML entries; gated on language; verify-before-patch. + * + * Encoding is abstracted behind an EncodingProfile (validator + terminator rules + * + UTF-8 -> glyph transcoder) selected per title; Tsumu uses the shift_jis + * profile. No generated-code edits, no regen: the hook is called from + * fntrace_record (runtime source), so it works in Release. + */ +#ifndef PSXRECOMP_TEXT_XLATE_H +#define PSXRECOMP_TEXT_XLATE_H + +#include <stdint.h> + +#ifdef __cplusplus +extern "C" { +#endif + +struct CPUState; + +/* Called at the top of every psx_dispatch iteration (from fntrace_record), + * BEFORE the target function runs, with cpu->gpr[4..7] holding this call's + * args. Cheap no-op when the module is uninitialised or disarmed. */ +void text_xlate_on_dispatch(struct CPUState* cpu, uint32_t target); + +/* One-time init. Loads every translations/ *.toml under project_root and + * selects the active `language` column. language "jp"/"off"/"" (or no tables) + * disables APPLY; CAPTURE still runs (always-on inventory). The PSX_LANG env + * var overrides `language` when set. Safe to call once at startup; copies what + * it needs. */ +void text_xlate_init(const char* project_root, const char* language); + +/* Switch the active language and reload the translation tables. For the launcher: + * text_xlate_init runs at config-load (before the launcher), so the launcher's + * chosen language is re-applied through this after it returns. "jp"/"off"/"" + * disables APPLY (untranslated native game). PSX_LANG env still wins if set. + * No-op if `language` matches the already-active one. Thread-safe. */ +void text_xlate_set_language(const char* language); + +/* Called by the GPU on completion of every CPU→VRAM image upload (GP0 0xA0) + * with the upload rect in VRAM halfword coords. Applies any [[vram_patch]] + * whose rect is contained in the upload (verify-then-patch). Cheap no-op when + * no patches are loaded or the language is off. */ +void text_xlate_vram_upload(int x, int y, int w, int h); + +/* Debug-server surface (observability; no log files — Rule 3). subcmd: + * "stats" -> counters (calls, hits, distinct captured, table size, lang) + * "dump" -> JSON array of every captured record {hash, addr, pc, len, + * count, translated, sjis_hex} — the authoring inventory + * "todo" -> JSON array of captured records with NO table entry yet + * "glyph" -> JSON array of per-glyph label RAM-patch slots {addr, width, + * patched, en} — status of the glyph-label source-patch layer + * "vpatch" -> force-apply every [[vram_patch]] against current VRAM (verify + * still gates each) + per-patch status {x,y,w,h,applied} + * "reload" -> re-read the translation table from disk + * Returns bytes written into out (<= cap). */ +int text_xlate_debug_json(const char* subcmd, char* out, int cap); + +#ifdef __cplusplus +} +#endif + +#endif /* PSXRECOMP_TEXT_XLATE_H */ diff --git a/runtime/include/ws_cull_detect.h b/runtime/include/ws_cull_detect.h new file mode 100644 index 00000000..e738a8bf --- /dev/null +++ b/runtime/include/ws_cull_detect.h @@ -0,0 +1,120 @@ +#ifndef PSX_WS_CULL_DETECT_H +#define PSX_WS_CULL_DETECT_H +/* ============================================================================ + * Generic widescreen SCREEN-EXTENT CULL detector ([widescreen.cull] + * auto_screen_x) + * + * Self-contained (depends only on <stdint.h>) so ONE implementation is shared + * verbatim by the recompiler (C++) and the runtime interpreter + sljit emitter + * (C) — the widen must be identical on every execution path or the backends + * diverge (same contract as ws_backdrop_detect.h). + * + * WHAT IT FINDS + * ------------- + * A GTE render funnel trivially rejects a primitive whose projected vertices + * all fall outside the screen. Titles express the horizontal reject three ways + * (immediates are per-game: Tomba tests X < 0x140/0x141 on a 320 display, Ape + * Escape tests X < 0x181 on its 368 display — configured via + * [widescreen.cull] screen_w_imms / screen_h_imms): + * + * (1) UNSIGNED PER-VERTEX (Tomba, Ape sprite path): + * sltiu v, SX, W ; SX loaded lhu / masked andi 0xFFFF + * One compare covers both edges (off-left SX wraps to a large unsigned). + * Widened by psx_ws_cull_sltiu(): window [-m, W+m). + * + * (2) SIGNED MIN/MAX (Ape tri/quad funnel): + * bltz maxSX, reject ; all verts left of 0 + * _slti v, minSX, W ; (delay slot) all verts right of W-1 + * The right bound widens via psx_ws_cull_slti() (W+m); the LEFT bound is + * the bare bltz, widened via psx_ws_cull_bltz() (reject only when + * maxSX < -m). The X pair is discriminated from the identically-shaped Y + * pair by the delay-slot immediate (W-set vs H-set). + * + * (3) SIGNED CENTER±HALFWIDTH (Ape billboard funnel): + * addu t, cx, r + * bltz t, reject ; cx + r < 0 (fully left) + * _<any> + * subu t, cx, r + * slti v, t, W ; cx - r >= W (fully right) + * The bltz is recognised by the addu immediately before it whose (rs,rt) + * pair reappears in a subu feeding a W-immediate slti a few words later. + * + * A function/window QUALIFIES only when it contains BOTH a width compare + * (slti/sltiu, imm in the W set) and a height compare (imm in the H set) — the + * screen-extent signature. A lone width-immediate compare elsewhere stays + * vanilla. + * + * All widen helpers are identity at margin 0 (4:3 / boot / menu / FMV), so the + * transformed code is bit-for-bit the vanilla verdict outside native-wide. + * ========================================================================== */ +#include <stdint.h> + +static inline int psx_ws_cull_imm_in(uint32_t imm, const uint32_t *set, int n) { + for (int i = 0; i < n; i++) + if (set[i] == imm) return 1; + return 0; +} + +/* is words[i] a slti (0x0A) or sltiu (0x0B) with an immediate in `set`? */ +static inline int psx_ws_cull_is_slt_imm(uint32_t w, const uint32_t *set, int n) { + uint32_t op = w >> 26; + if (op != 0x0Au && op != 0x0Bu) return 0; + return psx_ws_cull_imm_in(w & 0xFFFFu, set, n); +} + +/* Screen-extent signature over an instruction window: at least one W compare + * AND one H compare (slti or sltiu). Pure — identical verdicts on the + * recompiler's full-function scan and the runtime's ±window scan. */ +static inline int psx_ws_cull_scan(const uint32_t *words, int n, + const uint32_t *w_imms, int nw, + const uint32_t *h_imms, int nh) { + int has_x = 0, has_y = 0; + for (int i = 0; i < n; i++) { + uint32_t w = words[i]; + uint32_t op = w >> 26; + if (op != 0x0Au && op != 0x0Bu) continue; + uint32_t im = w & 0xFFFFu; + if (psx_ws_cull_imm_in(im, w_imms, nw)) has_x = 1; + else if (psx_ws_cull_imm_in(im, h_imms, nh)) has_y = 1; + if (has_x && has_y) return 1; + } + return 0; +} + +/* Classify words[idx] as an X LEFT-EDGE reject bltz (idiom 2 or 3 above). + * Caller must already have qualified the surrounding window via + * psx_ws_cull_scan. Returns 1 iff the bltz belongs to a width reject. */ +static inline int psx_ws_cull_bltz_here(const uint32_t *words, int n, int idx, + const uint32_t *w_imms, int nw) { + if (idx < 0 || idx >= n) return 0; + uint32_t w = words[idx]; + if ((w >> 26) != 0x01u || ((w >> 16) & 0x1Fu) != 0x00u) return 0; /* bltz only */ + uint32_t bltz_rs = (w >> 21) & 0x1Fu; + + /* Idiom 2: the width slti/sltiu sits in the bltz's delay slot. (The Y pair + * has the same shape with an H immediate — excluded by the W-set test.) */ + if (idx + 1 < n && psx_ws_cull_is_slt_imm(words[idx + 1], w_imms, nw)) + return 1; + + /* Idiom 3: addu t,x,y immediately before; a subu with the same (x,y) pair + * within the next few words feeds a W-immediate slti/sltiu right after. */ + if (idx >= 1) { + uint32_t p = words[idx - 1]; + if ((p >> 26) == 0x00u && (p & 0x3Fu) == 0x21u && /* addu */ + ((p >> 11) & 0x1Fu) == bltz_rs) { /* rd feeds bltz */ + uint32_t ax = (p >> 21) & 0x1Fu, ay = (p >> 16) & 0x1Fu; + for (int j = idx + 1; j <= idx + 4 && j < n; j++) { + uint32_t s = words[j]; + if ((s >> 26) == 0x00u && (s & 0x3Fu) == 0x23u && /* subu */ + ((s >> 21) & 0x1Fu) == ax && ((s >> 16) & 0x1Fu) == ay) { + for (int k = j + 1; k <= j + 2 && k < n; k++) + if (psx_ws_cull_is_slt_imm(words[k], w_imms, nw)) + return 1; + } + } + } + } + return 0; +} + +#endif /* PSX_WS_CULL_DETECT_H */ diff --git a/runtime/launcher/assets/launcher.rml b/runtime/launcher/assets/launcher.rml index d050e683..2f1dbbb1 100644 --- a/runtime/launcher/assets/launcher.rml +++ b/runtime/launcher/assets/launcher.rml @@ -8,7 +8,7 @@ body { background-color: #0b0e14; font-size: 15dp; } -#root { width: 100%; height: 100%; display: block; } +#root { width: 100%; height: 100%; display: block; overflow-y: auto; } #frame { display: block; padding: 22dp 26dp; } /* ---- top bar ---- */ @@ -34,6 +34,12 @@ h1 { display: block; font-size: 27dp; font-weight: bold; color: #ffffff; } padding: 18dp 22dp; margin-bottom: 13dp; } +/* Settings view: two side-by-side columns so the panels use the horizontal + space and the view fits without scrolling. SYSTEM stays full-width below + (its BIOS/disc paths need the room). */ +.settings-cols { display: flex; flex-direction: row; align-items: flex-start; } +.settings-col { display: block; flex: 1 1 0; min-width: 0; } +.settings-col:first-child { margin-right: 13dp; } .cards-row { display: flex; flex-direction: row; align-items: stretch; margin-bottom: 13dp; } .card { display: block; @@ -171,6 +177,23 @@ button:hover { background-color: #232d3c; } white-space: nowrap; overflow: hidden; } .path.empty { color: #f85149; } +/* ---- controls / keybind rebind view ---- */ +.kb-head { display: flex; flex-direction: row; align-items: center; margin-bottom: 16dp; } +.kb-player-lbl { color: #adbac7; margin-right: 10dp; } +.kb-hint { color: #7d8590; font-size: 12dp; margin-left: 18dp; flex: 1 1 auto; + white-space: nowrap; overflow: hidden; } +.kb-reset { background-color: #1b2330; border: 1dp #2b3543; border-radius: 7dp; + color: #e6e9ef; padding: 7dp 14dp; margin-left: 12dp; } +.kb-reset:hover { border-color: #3b82f6; } +.rebind-list { display: block; } +.rb-row { display: flex; flex-direction: row; align-items: center; margin-bottom: 8dp; } +.rb-label { width: 130dp; flex: 0 0 130dp; color: #adbac7; font-size: 13dp; } +.rb-chip { width: 150dp; flex: 0 0 150dp; text-align: center; margin-right: 34dp; + background-color: #161d28; border: 1dp #2b3543; border-radius: 7dp; + color: #e6e9ef; padding: 7dp 10dp; } +.rb-chip:hover { border-color: #3b82f6; } +.rb-chip--scan { border-color: #f0b429; color: #f0b429; } + /* ---- footer ---- */ .footer { display: flex; flex-direction: row; align-items: center; margin-top: 6dp; border-top: 1dp #1e2733; padding-top: 16dp; } @@ -210,8 +233,10 @@ button:hover { background-color: #232d3c; } <div class="subtitle">{{game_name}} launcher</div> </div> <div class="topbar-actions"> + <button class="top-btn" data-if="view == 'dashboard'" data-event-click="show_controls">Controls</button> <button class="top-btn" data-if="view == 'dashboard'" data-event-click="show_settings">Settings</button> <button class="top-btn primary" data-if="view == 'settings'" data-event-click="show_dashboard">‹ Back to dashboard</button> + <button class="top-btn primary" data-if="view == 'controls'" data-event-click="show_dashboard">‹ Back to dashboard</button> </div> </div> @@ -241,7 +266,7 @@ button:hover { background-color: #232d3c; } <button class="change-iso" data-event-click="change_iso">Change ISO</button> </div> - <div class="cards-row"> + <div class="cards-row" data-if="device_locked == false"> <div class="card left"> <div class="card-head"> <span class="card-name">PLAYER 1</span> @@ -338,6 +363,8 @@ button:hover { background-color: #232d3c; } <!-- ================= SETTINGS VIEW ================= --> <div data-if="view == 'settings'"> + <div class="settings-cols"> + <div class="settings-col"> <div class="panel"> <div class="eyebrow">VIDEO</div> <div class="set-row"><span class="set-lbl">Window size</span><span class="set-ctl"> @@ -362,6 +389,16 @@ button:hover { background-color: #232d3c; } <div class="set-row" data-if="ws_eligible"><span class="set-lbl">Widescreen <span class="exp-tag">EXPERIMENTAL</span></span><span class="set-ctl"> <button class="toggle" data-event-click="toggle_widescreen"><span class="value" data-if="widescreen">On</span><span class="value" data-if="widescreen == false">Off</span></button> <span class="ws-hint" data-if="widescreen">16:9 native field-of-view</span></span></div> + <div class="setting-row" data-if="uw_eligible"><span class="setting-name">Ultrawide <span class="exp">EXPERIMENTAL</span></span><span class="setting-control"> + <button class="toggle" data-event-click="toggle_ultrawide"><span class="value" data-if="ultrawide">On</span><span class="value" data-if="ultrawide == false">Off</span></button> + <span class="ws-hint" data-if="ultrawide">21:9 native field-of-view</span></span></div> + </div> + </div> + <div class="settings-col"> + <div class="panel" data-if="lang_menu"> + <div class="eyebrow">LOCALIZATION</div> + <div class="set-row"><span class="set-lbl">Language</span><span class="set-ctl"> + <button class="toggle wide" data-event-click="cycle_language"><span class="value">{{lang_label}}</span></button></span></div> </div> <div class="panel"> <div class="eyebrow">AUDIO</div> @@ -373,6 +410,8 @@ button:hover { background-color: #232d3c; } <div class="set-row"><span class="set-lbl">Analog stick deadzone</span><span class="set-ctl"> <button class="toggle" data-event-click="cycle_deadzone"><span class="value">{{deadzone_pct}}%</span></button></span></div> </div> + </div> + </div> <div class="panel"> <div class="eyebrow">SYSTEM</div> <div class="set-row"><span class="set-lbl">BIOS</span><span class="set-ctl"> @@ -386,6 +425,23 @@ button:hover { background-color: #232d3c; } </div> </div> + <!-- ================= CONTROLS (KEYBOARD KEYBINDS) VIEW ================= --> + <div data-if="view == 'controls'"> + <div class="panel"> + <div class="eyebrow">KEYBOARD CONTROLS</div> + <div class="kb-head"> + <span class="kb-player-lbl">Player</span> + <div class="seg"> + <span class="seg-opt" data-class-on="cfg_player == 0" data-event-click="cfg_player_1">1</span> + <span class="seg-opt" data-class-on="cfg_player == 1" data-event-click="cfg_player_2">2</span> + </div> + <span class="kb-hint">Click a key, then press the new key to bind it. Esc cancels. Changes save instantly.</span> + <button class="kb-reset" data-event-click="rebind_reset">Reset to defaults</button> + </div> + <div id="rebind-list" class="rebind-list"></div> + </div> + </div> + <!-- ================= FOOTER ================= --> <div class="footer"> <span class="switch" data-class-on="skip_launcher" data-event-click="toggle_skip_launcher"><span class="knob"></span></span> diff --git a/runtime/launcher/launcher.cpp b/runtime/launcher/launcher.cpp index fc42510b..0ae47eb2 100644 --- a/runtime/launcher/launcher.cpp +++ b/runtime/launcher/launcher.cpp @@ -12,6 +12,7 @@ extern "C" { #include "memcard.h" +#include "psx_keybinds.h" } #include <RmlUi/Core.h> @@ -29,6 +30,8 @@ extern "C" { #include <cstdio> #include <ctime> #include <filesystem> +#include <functional> +#include <memory> #include <string> #include <vector> @@ -107,6 +110,8 @@ struct LauncherModel { int window_width = 1280; // window size (height = width*den/num per aspect) bool widescreen = false; // EXPERIMENTAL 16:9 native-wide (aspect_index==1) bool ws_eligible = true; // toggle shown only when renderer==software (native-wide is SW-only) + bool ultrawide = false; // separate EXPERIMENTAL 21:9 choice + bool uw_eligible = false; // per-game offer_ultrawide gate bool fullscreen = false; // launch the game window in desktop fullscreen // Skip-launcher: boot straight into the game on subsequent launches. Turning // it ON shows a confirmation modal (show_skip_modal) so the user is told how @@ -135,9 +140,13 @@ struct LauncherModel { Rml::String verdict_detail; // sub line Rml::String verdict_state; // "ok" | "warn" | "bad" | "none" — drives colour - // View toggle: "dashboard" (default) | "settings". + // View toggle: "dashboard" (default) | "settings" | "controls". Rml::String view = "dashboard"; + // Controls page: which player's keyboard binds are being edited (0=P1,1=P2). + int cfg_player = 0; + Rml::String cfg_player_label = "1"; + // Player cards — real device routing. Each port picks a device (None / // Keyboard / a plugged-in SDL controller) and a pad type (DualShock=analog). int p1_dev_index = 1; // index into the shared device option list @@ -148,13 +157,20 @@ struct LauncherModel { int p2_mode = 0; bool allow_hybrid = true; // game.allow_hybrid: when false the Hybrid segment is hidden bool mode_selectable = true; // game.lock_mode == false: when false the whole pad-mode selector is hidden + bool device_locked = false; // game.lock_device: when true the Player 1/2 cards are hidden entirely (fixed, auto-bound pad type) int deadzone_pct = 37; // analog-stick deadzone 0-100% (raw = pct*32767/100) Rml::String p1_dev_label = "Keyboard"; Rml::String p2_dev_label = "None"; Rml::String p1_status, p2_status; // resolved status line Rml::String p1_dot, p2_dot; // "" (on) | "off" Rml::String p1_options, p2_options; // data-rml option-list markup - Rml::String dd_open; // "" | "p1" | "p2" (which list is open) + Rml::String dd_open; // "" | "p1" | "p2" (which device list is open) + + // Localization (only shown when the game declares languages). Settings-view + // cycle button, like Renderer / Screen model. + bool lang_menu = false; // game.languages non-empty => show it + int lang_index = 0; // index into the game's language list + Rml::String lang_label; // current option label ("English") // Memory cards — real introspection of the on-disk .mcd images. Each slot // has a resolved file path, an enable toggle, and parsed directory stats. @@ -344,6 +360,22 @@ void refresh_player(LauncherModel& m, int player, const std::vector<DeviceOption else { status = o.label + Rml::String(" \xE2\x80\x94 ") + type; dot = ""; } } +// Recompute the language button's label from lang_index (Settings cycle toggle). +void refresh_language(LauncherModel& m, + const std::vector<psx_launcher::GameInfo::Language>& langs) { + if (langs.empty()) return; + if (m.lang_index < 0 || m.lang_index >= (int)langs.size()) m.lang_index = 0; + m.lang_label = langs[m.lang_index].label; +} + +// Resolve a saved language code to an index in the game's language list. +int lang_index_for(const std::vector<psx_launcher::GameInfo::Language>& langs, + const std::string& code) { + for (size_t i = 0; i < langs.size(); i++) + if (langs[i].code == code) return (int)i; + return 0; // unknown/first — the game's declared default sits at [0] by convention +} + const char* renderer_name(int v) { return v == 1 ? "OpenGL" : "Software"; } const char* texfilter_name(int v) { return v == 1 ? "Bilinear" : "Nearest"; } const char* crt_name(int v) { @@ -400,7 +432,10 @@ void refresh_labels(LauncherModel& m) { m.aspect_label = aspect_name(m.aspect_index); m.winsize_label = winsize_label_for(m.window_width, m.aspect_index); m.widescreen = (m.aspect_index == 1); // 16:9 == experimental native-wide - m.ws_eligible = true; // native-wide works on BOTH backends now (SW + GL compositor) + m.ultrawide = (m.aspect_index == 2); // 21:9 is offered separately + /* ws_eligible is set once at model init from GameInfo.ws_offered ([widescreen] + * offer): renderer no longer matters (native-wide works on SW + GL), so + * refresh must not overwrite the per-game gate. */ } std::string region_long(const std::string& r) { @@ -607,6 +642,11 @@ Result run(SDL_Window* window, void* gl_context, { (void)gl_context; // already created + current; we only need the window. + // Keyboard keybinds live in keybinds.ini next to the exe (assets_dir == the + // exe dir). Load them so the Controls page edits the real, persisted map; + // the runtime re-reads the same file at startup (psx_keybinds_init). + psx_keybinds_init(assets_dir); + const std::string expected_serial = game.expected_serial ? game.expected_serial : ""; const uint32_t expected_crc = game.expected_crc; const bool has_expected_crc = game.has_expected_crc; @@ -663,6 +703,14 @@ Result run(SDL_Window* window, void* gl_context, m.skip_launcher = io.skip_launcher; m.spu_hq = io.spu_hq; m.aspect_index = io.has_aspect_ratio ? aspect_index_for(io.aspect_num, io.aspect_den) : 0; + // Games whose widescreen is unported/unvalidated declare [widescreen] + // offer=false: hide the toggle AND clamp a stale persisted 16:9 back to + // 4:3 so the hack can't engage from an old settings.toml. + m.ws_eligible = game.ws_offered; + m.uw_eligible = game.ws_ultrawide_offered; + if (!game.ws_offered && m.aspect_index == 1) m.aspect_index = 0; + if (!game.ws_ultrawide_offered && m.aspect_index == 2) + m.aspect_index = game.ws_offered ? 1 : 0; m.window_width = kWinWidths[winsize_index(io.has_window_width ? io.window_width : 1280)]; m.bios_path = io.has_bios_path ? io.bios_path.generic_string() : Rml::String(); m.disc_path = io.has_disc_path ? io.disc_path.generic_string() : Rml::String(); @@ -697,6 +745,9 @@ Result run(SDL_Window* window, void* gl_context, m.p1_mode = game.locked_mode; m.p2_mode = game.locked_mode; } + // lock_device: hide the Player controller cards entirely — the pad type is + // fixed and auto-bound (e.g. Ape Escape ships DualShock analog, no choice). + m.device_locked = game.lock_device; m.deadzone_pct = io.has_deadzone ? (io.deadzone * 100 / 32767) : 37; if (io.has_p1_device) { m.p1_dev_index = find_or_add_device_index(dev_opts, io.p1_device); @@ -708,6 +759,13 @@ Result run(SDL_Window* window, void* gl_context, refresh_player(m, 0, dev_opts); refresh_player(m, 1, dev_opts); + // ---- Seed the localization dropdown (only when the game declares one) ---- + m.lang_menu = !game.languages.empty(); + if (m.lang_menu) { + m.lang_index = lang_index_for(game.languages, io.language); + refresh_language(m, game.languages); + } + // ---- Data model: bind fields + action callbacks ---- Rml::DataModelConstructor c = context->CreateDataModel("settings"); if (!c) { @@ -730,6 +788,8 @@ Result run(SDL_Window* window, void* gl_context, c.Bind("aspect_label", &m.aspect_label); c.Bind("widescreen", &m.widescreen); c.Bind("ws_eligible", &m.ws_eligible); + c.Bind("ultrawide", &m.ultrawide); + c.Bind("uw_eligible", &m.uw_eligible); c.Bind("winsize_label", &m.winsize_label); c.Bind("texfilter_label",&m.texfilter_label); c.Bind("bios_path", &m.bios_path); @@ -744,10 +804,13 @@ Result run(SDL_Window* window, void* gl_context, c.Bind("verdict_detail", &m.verdict_detail); c.Bind("verdict_state", &m.verdict_state); c.Bind("view", &m.view); + c.Bind("cfg_player", &m.cfg_player); + c.Bind("cfg_player_label", &m.cfg_player_label); c.Bind("p1_mode", &m.p1_mode); c.Bind("p2_mode", &m.p2_mode); c.Bind("allow_hybrid", &m.allow_hybrid); c.Bind("mode_selectable",&m.mode_selectable); + c.Bind("device_locked", &m.device_locked); c.Bind("deadzone_pct", &m.deadzone_pct); c.Bind("p1_dev_label", &m.p1_dev_label); c.Bind("p2_dev_label", &m.p2_dev_label); @@ -758,6 +821,8 @@ Result run(SDL_Window* window, void* gl_context, c.Bind("p1_options", &m.p1_options); c.Bind("p2_options", &m.p2_options); c.Bind("dd_open", &m.dd_open); + c.Bind("lang_menu", &m.lang_menu); + c.Bind("lang_label", &m.lang_label); c.Bind("mc1_enabled", &m.mc1_enabled); c.Bind("mc2_enabled", &m.mc2_enabled); c.Bind("mc1_name", &m.mc1_name); @@ -773,6 +838,84 @@ Result run(SDL_Window* window, void* gl_context, Rml::DataModelHandle handle = c.GetModelHandle(); + // ---- keyboard keybind rebinding (Controls page) -------------------------- + // The chip list is built programmatically after the document loads (data-for + // can't generate the per-button chips), and re-wired on each rebuild. A scan + // is armed when a chip is clicked; the SDL loop then swallows the next keydown + // and resolves it (Esc cancels). Bindings save to keybinds.ini immediately on + // capture. Mirrors snesrecomp's launcher Configure view. + Rml::ElementDocument* kbdoc = nullptr; // set after LoadDocument + std::function<void()> build_rebind_list; // set after LoadDocument + bool rebuild_pending = false; + int scan_kind = 0; // 0=idle, 1=capturing + int scan_index = 0; // button being rebound + std::string scan_chip_id; + + auto kb_chip_label = [&m](int button) -> std::string { + SDL_Scancode sc = psx_keybinds_get_button(m.cfg_player + 1, button); + const char* n = (sc != SDL_SCANCODE_UNKNOWN) ? SDL_GetScancodeName(sc) : ""; + return (n && n[0]) ? std::string(n) : std::string("None"); + }; + auto end_scan = [&]() { + if (!scan_kind) return; + if (kbdoc) if (Rml::Element* e = kbdoc->GetElementById(scan_chip_id)) { + e->SetInnerRML(kb_chip_label(scan_index)); + e->SetClass("rb-chip--scan", false); + } + scan_kind = 0; scan_chip_id.clear(); + }; + auto begin_scan = [&](int index, const std::string& chip_id) { + end_scan(); + scan_kind = 1; scan_index = index; scan_chip_id = chip_id; + if (kbdoc) if (Rml::Element* e = kbdoc->GetElementById(chip_id)) { + e->SetInnerRML("Press a key..."); + e->SetClass("rb-chip--scan", true); + } + }; + auto handle_scan_key = [&](const SDL_KeyboardEvent& ke) { + if (ke.keysym.sym == SDLK_ESCAPE) { end_scan(); return; } + const SDL_Scancode sc = ke.keysym.scancode; + // Steal: a key already bound elsewhere (either player) moves here instead + // of silently double-firing. + for (int pl = 1; pl <= 2; pl++) + for (int b = 0; b < psx_keybinds_button_count(); b++) + if (psx_keybinds_get_button(pl, b) == sc && + !(pl == m.cfg_player + 1 && b == scan_index)) + psx_keybinds_set_button(pl, b, SDL_SCANCODE_UNKNOWN); + psx_keybinds_set_button(m.cfg_player + 1, scan_index, sc); + psx_keybinds_save(); + end_scan(); + rebuild_pending = true; // stolen chips refresh too + }; + + c.BindEventCallback("show_controls", + [&m, handle, &end_scan, &rebuild_pending](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + end_scan(); + m.view = "controls"; handle.DirtyVariable("view"); + rebuild_pending = true; + }); + c.BindEventCallback("cfg_player_1", + [&m, handle, &end_scan, &rebuild_pending](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + end_scan(); + m.cfg_player = 0; m.cfg_player_label = "1"; + handle.DirtyVariable("cfg_player"); handle.DirtyVariable("cfg_player_label"); + rebuild_pending = true; + }); + c.BindEventCallback("cfg_player_2", + [&m, handle, &end_scan, &rebuild_pending](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + end_scan(); + m.cfg_player = 1; m.cfg_player_label = "2"; + handle.DirtyVariable("cfg_player"); handle.DirtyVariable("cfg_player_label"); + rebuild_pending = true; + }); + c.BindEventCallback("rebind_reset", + [&m, &end_scan, &rebuild_pending](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + end_scan(); + psx_keybinds_reset_player(m.cfg_player + 1); + psx_keybinds_save(); + rebuild_pending = true; + }); + c.BindEventCallback("cycle_renderer", [&m, handle](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { m.renderer ^= 1; @@ -809,10 +952,10 @@ Result run(SDL_Window* window, void* gl_context, // Off => 4:3 (aspect_index 0). Works on BOTH renderers (SW + the GL wide // compositor), so it is no longer gated on the software renderer. // - // 21:9 (kAspects[2]) is STUBBED but intentionally hidden: the engine handles + // 21:9 (kAspects[2]) is separately gated per game. The engine handles // it (offset / cull / compositor are all aspect-derived), but the parallax + // far-backdrop pipeline only generates ~16:9 of coverage, so 21:9 voids the - // far background. When that pipeline is widened, promote this 2-state toggle + // far background. Titles opt in only after that pipeline is validated; // to a 3-way Off / 16:9 / 21:9 — the existing cycle_aspect callback already // cycles aspect_index 0/1/2 and is the scaffold for it. c.BindEventCallback("toggle_widescreen", @@ -820,9 +963,19 @@ Result run(SDL_Window* window, void* gl_context, m.aspect_index = (m.aspect_index == 1) ? 0 : 1; // 16:9 <-> 4:3 refresh_labels(m); handle.DirtyVariable("widescreen"); + handle.DirtyVariable("ultrawide"); handle.DirtyVariable("aspect_label"); handle.DirtyVariable("winsize_label"); /* height follows aspect */ }); + c.BindEventCallback("toggle_ultrawide", + [&m, handle](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + m.aspect_index = (m.aspect_index == 2) ? 0 : 2; // 21:9 <-> 4:3 + refresh_labels(m); + handle.DirtyVariable("widescreen"); + handle.DirtyVariable("ultrawide"); + handle.DirtyVariable("aspect_label"); + handle.DirtyVariable("winsize_label"); + }); c.BindEventCallback("cycle_winsize", [&m, handle](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { int i = (winsize_index(m.window_width) + 1) % kNumWinWidths; @@ -892,11 +1045,13 @@ Result run(SDL_Window* window, void* gl_context, c.BindEventCallback("change_iso", do_browse_disc); c.BindEventCallback("show_settings", - [&m, handle](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + [&m, handle, &end_scan](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + end_scan(); m.view = "settings"; handle.DirtyVariable("view"); }); c.BindEventCallback("show_dashboard", - [&m, handle](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + [&m, handle, &end_scan](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + end_scan(); m.view = "dashboard"; handle.DirtyVariable("view"); }); // ---- controller: device dropdown + pad-mode segmented selector ---- @@ -929,6 +1084,16 @@ Result run(SDL_Window* window, void* gl_context, dirty_player(player); handle.DirtyVariable("dd_open"); }); + // ---- localization: Language cycle button (Settings > LOCALIZATION) ---- + // Matches the Settings-view idiom (Renderer / Screen model cycle toggles): + // each click advances to the next declared language, wrapping around. + c.BindEventCallback("cycle_language", + [&m, handle, langs = game.languages](Rml::DataModelHandle, Rml::Event&, const Rml::VariantList&) mutable { + if (langs.empty()) return; + m.lang_index = (m.lang_index + 1) % (int)langs.size(); + refresh_language(m, langs); + handle.DirtyVariable("lang_label"); + }); // Pad-mode segmented selector: each segment passes its mode (0=hybrid, // 1=analog, 2=digital) so any mode is one click away. c.BindEventCallback("set_mode_p1", @@ -1007,12 +1172,60 @@ Result run(SDL_Window* window, void* gl_context, } doc->Show(); + // ---- build the keybind chip list (Controls page) ---- + // data-if only hides the controls view, so #rebind-list exists from load and + // GetElementById finds it even while the dashboard is showing. Each chip is a + // <button> whose click arms a scan; listeners are re-wired on every rebuild. + kbdoc = doc; + struct KbClickListener : Rml::EventListener { + std::function<void()> on_click; + void ProcessEvent(Rml::Event&) override { if (on_click) on_click(); } + }; + std::vector<std::unique_ptr<KbClickListener>> kb_listeners; + build_rebind_list = [&]() { + Rml::Element* list = doc->GetElementById("rebind-list"); + if (!list) return; + const int n = psx_keybinds_button_count(); + std::string html; + for (int b = 0; b < n; b += 2) { // two (label, chip) pairs per row + html += "<div class=\"rb-row\">"; + for (int k = b; k < b + 2 && k < n; k++) { + html += "<span class=\"rb-label\">"; + html += rml_escape(psx_keybinds_button_label(k)); + html += "</span><button class=\"rb-chip\" id=\"kb-"; + html += psx_keybinds_button_name(k); + html += "\">" + rml_escape(kb_chip_label(k)) + "</button>"; + } + html += "</div>"; + } + list->SetInnerRML(html); // destroys prior chips... + kb_listeners.clear(); // ...so dropping their listeners is safe + for (int k = 0; k < n; k++) { + const std::string id = std::string("kb-") + psx_keybinds_button_name(k); + if (Rml::Element* e = doc->GetElementById(id)) { + auto lis = std::make_unique<KbClickListener>(); + lis->on_click = [&, k, id]() { begin_scan(k, id); }; + e->AddEventListener(Rml::EventId::Click, lis.get()); + kb_listeners.push_back(std::move(lis)); + } + } + }; + build_rebind_list(); + // ---- Main loop ---- Result result = Result::Quit; bool running = true; while (running) { SDL_Event ev; while (SDL_PollEvent(&ev)) { + // While a rebind scan is armed, swallow keyboard input (the next + // keydown resolves it; Esc cancels) so it can't leak into RmlUi + // controls. + if (scan_kind && + (ev.type == SDL_KEYDOWN || ev.type == SDL_KEYUP || ev.type == SDL_TEXTINPUT)) { + if (ev.type == SDL_KEYDOWN) handle_scan_key(ev.key); + continue; + } switch (ev.type) { case SDL_QUIT: m.quit_requested = true; @@ -1034,6 +1247,11 @@ Result run(SDL_Window* window, void* gl_context, if (m.launch_requested) { result = Result::Launch; running = false; } if (m.quit_requested) { result = Result::Quit; running = false; } + // Deferred chip-list rebuild (set from chip handlers / scan capture / + // player switch / reset — never rebuild a list from inside its own + // listener's dispatch). + if (rebuild_pending) { rebuild_pending = false; build_rebind_list(); } + context->Update(); render_interface.Clear(); @@ -1074,6 +1292,14 @@ Result run(SDL_Window* window, void* gl_context, io.p1_mode = m.p1_mode; io.has_p1_mode = true; io.p2_mode = m.p2_mode; io.has_p2_mode = true; io.deadzone = m.deadzone_pct * 32767 / 100; io.has_deadzone = true; + + // Localization: persist the chosen language code (only meaningful when the + // game declared a menu; otherwise leave io.language untouched). + if (m.lang_menu && m.lang_index >= 0 && + m.lang_index < (int)game.languages.size()) { + io.language = game.languages[m.lang_index].code; + io.has_language = true; + } } Rml::Shutdown(); diff --git a/runtime/launcher/launcher.h b/runtime/launcher/launcher.h index babe74b7..34c0eb08 100644 --- a/runtime/launcher/launcher.h +++ b/runtime/launcher/launcher.h @@ -14,6 +14,8 @@ #pragma once #include <cstdint> +#include <string> +#include <vector> struct SDL_Window; @@ -37,6 +39,16 @@ struct GameInfo { bool allow_hybrid = true; // offer the "Hybrid" pad mode (false => Analog | D-Pad only) bool lock_mode = false; // hide the whole pad-mode selector and force locked_mode (single-pad-type games) int locked_mode = 2; // PAD_MODE_DIGITAL; the mode forced when lock_mode is true + bool lock_device = false; // hide the Player 1/2 controller cards entirely (fixed, auto-bound pad type; e.g. Ape Escape DualShock) + bool ws_offered = true; // offer the EXPERIMENTAL Widescreen toggle (false = hidden, game ships 4:3 only) + bool ws_ultrawide_offered = false; // separately offer experimental 21:9 + + // Optional "Localization" dropdown. Populated from game.toml + // [localization].languages. When non-empty the launcher shows a language + // menu; empty => no menu (the general case — only Tsumu declares languages). + // code feeds the runtime's translation layer ("off"/"jp"/"" = untranslated). + struct Language { std::string code; std::string label; }; + std::vector<Language> languages; }; // Run the launcher loop to completion. `gl_context` is an SDL_GLContext (void* diff --git a/runtime/runtime.cmake b/runtime/runtime.cmake index bf93a279..9f82b584 100644 --- a/runtime/runtime.cmake +++ b/runtime/runtime.cmake @@ -8,6 +8,19 @@ if(NOT DEFINED PSXRECOMP_ROOT) get_filename_component(PSXRECOMP_ROOT "${CMAKE_CURRENT_LIST_DIR}/.." ABSOLUTE) endif() +# Default to an optimized build. The recompiled game is a huge (~270 MB) block of +# generated C; with no CMAKE_BUILD_TYPE the compiler emits it at -O0 and the game +# runs at a small fraction of full speed (terrible framerate). A naive +# `cmake -B build` (as in the README) must NOT produce that, so default to +# Release when the user hasn't chosen a type. Single-config generators only; +# multi-config (VS/Xcode) pick per-build. Overridable with -DCMAKE_BUILD_TYPE=... +if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES) + set(CMAKE_BUILD_TYPE Release CACHE STRING + "Build type (Release/RelWithDebInfo/Debug)" FORCE) + message(STATUS "psxrecomp: no CMAKE_BUILD_TYPE set — defaulting to Release " + "(optimized). Use -DCMAKE_BUILD_TYPE=RelWithDebInfo/Debug to override.") +endif() + # Content-addressed compiler cache (ccache). git branch operations (checkout / # merge / new branch) rewrite working-tree file mtimes, which makes ninja treat # the ~279 MB generated-C objects as stale and recompile them (~15 min) even when @@ -151,6 +164,7 @@ set(PSXRECOMP_RUNTIME_SOURCES ${PSXRECOMP_ROOT}/runtime/src/debug_server.c ${PSXRECOMP_ROOT}/runtime/src/dirty_ram_interp.c ${PSXRECOMP_ROOT}/runtime/src/fntrace.c + ${PSXRECOMP_ROOT}/runtime/src/text_xlate.cpp ${PSXRECOMP_ROOT}/runtime/src/parity_trace.c ${PSXRECOMP_ROOT}/runtime/src/device_trace.c ${PSXRECOMP_ROOT}/runtime/src/boot_state.c @@ -168,6 +182,7 @@ set(PSXRECOMP_RUNTIME_SOURCES ${PSXRECOMP_ROOT}/runtime/src/spu.c ${PSXRECOMP_ROOT}/runtime/src/spu_shadow.c ${PSXRECOMP_ROOT}/runtime/src/audio_shadow.c + ${PSXRECOMP_ROOT}/runtime/src/audio_trace.c ${PSXRECOMP_ROOT}/runtime/src/color_lut.c ${PSXRECOMP_ROOT}/runtime/src/iso_reader.cpp ${PSXRECOMP_ROOT}/runtime/src/iso_reader_c.cpp @@ -186,7 +201,9 @@ set(PSXRECOMP_RUNTIME_SOURCES ${PSXRECOMP_ROOT}/runtime/src/code_provider.c ${PSXRECOMP_ROOT}/runtime/src/event_ring.c ${PSXRECOMP_ROOT}/runtime/src/game_options.c + ${PSXRECOMP_ROOT}/runtime/src/psx_keybinds.c ${PSXRECOMP_ROOT}/recompiler/src/config_loader.cpp + ${PSXRECOMP_ROOT}/recompiler/src/ps1_exe_parser.cpp # Tier-2 in-process JIT backend (sljit, BSD-2-Clause). Single TU; sljit # auto-detects the host arch (SLJIT_CONFIG_AUTO). See lib/sljit/LICENSE. ${PSXRECOMP_ROOT}/lib/sljit/sljit_src/sljitLir.c @@ -195,6 +212,7 @@ set(PSXRECOMP_RUNTIME_SOURCES set(PSXRECOMP_RUNTIME_INCLUDE_DIRS ${PSXRECOMP_ROOT}/runtime/include ${PSXRECOMP_ROOT}/recompiler/src + ${PSXRECOMP_ROOT}/recompiler/include ${PSXRECOMP_ROOT}/recompiler/lib/fmt/include ${PSXRECOMP_ROOT}/recompiler/lib/toml11 ${PSXRECOMP_ROOT}/lib/sljit/sljit_src diff --git a/runtime/src/audio_trace.c b/runtime/src/audio_trace.c new file mode 100644 index 00000000..9008c094 --- /dev/null +++ b/runtime/src/audio_trace.c @@ -0,0 +1,263 @@ +/* + * audio_trace.c - always-on audio observability rings. See audio_trace.h. + * + * Memory: 3 PCM taps x 2^22 stereo frames x 4 bytes = 48 MiB static, plus a + * 2^19-entry event ring (16 MiB). Static BSS, zero cost until touched; the + * recording hot path is a memcpy per audio block and is compiled into + * Release/production builds too (always-on ring-buffer discipline — probes + * query history, they never arm recording). + */ +#include "audio_trace.h" + +#include <stdatomic.h> +#include <stdio.h> +#include <string.h> + +/* 2^22 frames @ 44100 ~= 95 s per tap. Power of two so wrap is a mask. */ +#define PCM_RING_FRAMES (1u << 22) +#define PCM_RING_MASK (PCM_RING_FRAMES - 1u) + +#define EV_RING_CAP (1u << 19) +#define EV_RING_MASK (EV_RING_CAP - 1u) + +/* Audibility threshold shared with snesrecomp's dropped_audible metric: + * |sample| > 256 ~= -42 dBFS. Below that, gaps/drops are inaudible and must + * not trip regression counters (snesrecomp a1c659f). */ +#define AUDIBLE_ABS 256 + +typedef struct { + int16_t pcm[PCM_RING_FRAMES * 2]; + _Atomic uint64_t head; /* frames ever written; ring pos = head & MASK */ + uint64_t nonzero; + uint64_t audible; + int32_t peak; +} PcmTap; + +static PcmTap s_taps[AUDIO_TAP_COUNT]; + +/* Per-tap sample rate for WAV headers. SPU-side taps are natively 44100; + * the host tap follows the device rate in bridge/pull mode. */ +static uint32_t s_tap_rate[AUDIO_TAP_COUNT] = { 44100u, 44100u, 44100u, 44100u }; + +void audio_trace_set_tap_rate(int tap, uint32_t rate) +{ + if (tap < 0 || tap >= AUDIO_TAP_COUNT || rate == 0) return; + s_tap_rate[tap] = rate; +} + +uint32_t audio_trace_tap_rate(int tap) +{ + if (tap < 0 || tap >= AUDIO_TAP_COUNT) return 44100u; + return s_tap_rate[tap]; +} + +static AudioTraceEvent s_events[EV_RING_CAP]; +static _Atomic uint64_t s_event_head; + +static _Atomic uint32_t s_noted_frame; + +/* Host pump counters (single writer: the pump/render thread). */ +static uint64_t s_pump_calls; +static uint64_t s_pump_skips; +static uint64_t s_underruns; +static uint32_t s_queue_hiwater; +static uint32_t s_queue_lowater = 0xFFFFFFFFu; +static uint64_t s_mute_events; +static uint64_t s_unmute_events; + +void audio_trace_init(void) +{ + for (int t = 0; t < AUDIO_TAP_COUNT; t++) { + atomic_store(&s_taps[t].head, 0); + s_taps[t].nonzero = 0; + s_taps[t].audible = 0; + s_taps[t].peak = 0; + } + atomic_store(&s_event_head, 0); + atomic_store(&s_noted_frame, 0); + s_pump_calls = 0; + s_pump_skips = 0; + s_underruns = 0; + s_queue_hiwater = 0; + s_queue_lowater = 0xFFFFFFFFu; + s_mute_events = 0; + s_unmute_events = 0; +} + +void audio_trace_note_frame(uint32_t frame) +{ + atomic_store_explicit(&s_noted_frame, frame, memory_order_relaxed); +} + +void audio_trace_pcm(int tap, const int16_t *stereo, int frames) +{ + if (tap < 0 || tap >= AUDIO_TAP_COUNT || !stereo || frames <= 0) return; + PcmTap *t = &s_taps[tap]; + uint64_t head = atomic_load_explicit(&t->head, memory_order_relaxed); + + for (int f = 0; f < frames; f++) { + int16_t l = stereo[f * 2 + 0]; + int16_t r = stereo[f * 2 + 1]; + uint32_t pos = (uint32_t)((head + (uint64_t)f) & PCM_RING_MASK); + t->pcm[pos * 2 + 0] = l; + t->pcm[pos * 2 + 1] = r; + int32_t al = l < 0 ? -(int32_t)l : l; + int32_t ar = r < 0 ? -(int32_t)r : r; + int32_t a = al > ar ? al : ar; + if (a) t->nonzero++; + if (a > AUDIBLE_ABS) t->audible++; + if (a > t->peak) t->peak = a; + } + atomic_store_explicit(&t->head, head + (uint64_t)frames, + memory_order_release); +} + +void audio_trace_event(uint16_t kind, uint32_t a, uint32_t b) +{ + uint64_t seq = atomic_load_explicit(&s_event_head, memory_order_relaxed); + AudioTraceEvent *e = &s_events[(uint32_t)(seq & EV_RING_MASK)]; + e->seq = seq; + e->sample_idx = atomic_load_explicit(&s_taps[AUDIO_TAP_SPU_OUT].head, + memory_order_relaxed); + e->frame = atomic_load_explicit(&s_noted_frame, memory_order_relaxed); + e->kind = kind; + e->reserved = 0; + e->a = a; + e->b = b; + atomic_store_explicit(&s_event_head, seq + 1, memory_order_release); + + switch (kind) { + case AUDIO_EV_RENDER: + s_pump_calls++; + if (b > s_queue_hiwater) s_queue_hiwater = b; + if (b < s_queue_lowater) s_queue_lowater = b; + break; + case AUDIO_EV_PUMP_SKIP: s_pump_skips++; break; + case AUDIO_EV_UNDERRUN: s_underruns++; break; + case AUDIO_EV_MUTE: s_mute_events++; break; + case AUDIO_EV_UNMUTE: s_unmute_events++; break; + default: break; + } +} + +void audio_trace_get_stats(AudioTraceStats *out) +{ + if (!out) return; + memset(out, 0, sizeof(*out)); + for (int t = 0; t < AUDIO_TAP_COUNT; t++) { + out->tap_frames[t] = atomic_load_explicit(&s_taps[t].head, + memory_order_acquire); + out->tap_nonzero[t] = s_taps[t].nonzero; + out->tap_audible[t] = s_taps[t].audible; + out->tap_peak[t] = s_taps[t].peak; + } + out->pump_calls = s_pump_calls; + out->pump_skips = s_pump_skips; + out->underruns = s_underruns; + out->queue_hiwater = s_queue_hiwater; + out->queue_lowater = s_queue_lowater == 0xFFFFFFFFu ? 0 : s_queue_lowater; + out->mute_events = s_mute_events; + out->unmute_events = s_unmute_events; + out->events_total = atomic_load_explicit(&s_event_head, + memory_order_acquire); +} + +uint64_t audio_trace_tap_total(int tap) +{ + if (tap < 0 || tap >= AUDIO_TAP_COUNT) return 0; + return atomic_load_explicit(&s_taps[tap].head, memory_order_acquire); +} + +uint64_t audio_trace_events_total(void) +{ + return atomic_load_explicit(&s_event_head, memory_order_acquire); +} + +uint32_t audio_trace_events_get(AudioTraceEvent *out, uint32_t max) +{ + if (!out || max == 0) return 0; + uint64_t total = atomic_load_explicit(&s_event_head, memory_order_acquire); + uint64_t avail = total < (uint64_t)EV_RING_CAP ? total : (uint64_t)EV_RING_CAP; + if ((uint64_t)max > avail) max = (uint32_t)avail; + uint64_t first = total - (uint64_t)max; + for (uint32_t i = 0; i < max; i++) + out[i] = s_events[(uint32_t)((first + i) & EV_RING_MASK)]; + return max; +} + +/* ---- WAV dump --------------------------------------------------------- */ + +static void wav_write_header(FILE *fp, uint32_t data_bytes, uint32_t rate) +{ + uint8_t h[44]; + uint32_t byte_rate = rate * 4u; + uint32_t riff_size = 36u + data_bytes; + memcpy(h + 0, "RIFF", 4); + h[4] = (uint8_t)riff_size; h[5] = (uint8_t)(riff_size >> 8); + h[6] = (uint8_t)(riff_size >> 16); h[7] = (uint8_t)(riff_size >> 24); + memcpy(h + 8, "WAVEfmt ", 8); + h[16] = 16; h[17] = 0; h[18] = 0; h[19] = 0; /* fmt chunk size */ + h[20] = 1; h[21] = 0; /* PCM */ + h[22] = 2; h[23] = 0; /* stereo */ + h[24] = (uint8_t)rate; h[25] = (uint8_t)(rate >> 8); + h[26] = (uint8_t)(rate >> 16); h[27] = (uint8_t)(rate >> 24); + h[28] = (uint8_t)byte_rate; h[29] = (uint8_t)(byte_rate >> 8); + h[30] = (uint8_t)(byte_rate >> 16); h[31] = (uint8_t)(byte_rate >> 24); + h[32] = 4; h[33] = 0; /* block align */ + h[34] = 16; h[35] = 0; /* bits per sample */ + memcpy(h + 36, "data", 4); + h[40] = (uint8_t)data_bytes; h[41] = (uint8_t)(data_bytes >> 8); + h[42] = (uint8_t)(data_bytes >> 16); h[43] = (uint8_t)(data_bytes >> 24); + fwrite(h, 1, sizeof(h), fp); +} + +int64_t audio_trace_dump_wav(int tap, const char *path, + int64_t start, uint64_t count) +{ + if (tap < 0 || tap >= AUDIO_TAP_COUNT || !path || !path[0]) return -1; + PcmTap *t = &s_taps[tap]; + + /* Snapshot the head; everything in [head - avail, head) is stable + * (append-only, single writer) unless the writer laps us — the ring + * holds ~95 s, a dump takes well under a second. */ + uint64_t head = atomic_load_explicit(&t->head, memory_order_acquire); + uint64_t avail = head < (uint64_t)PCM_RING_FRAMES ? head + : (uint64_t)PCM_RING_FRAMES; + if (avail == 0) return -1; + uint64_t oldest = head - avail; + + uint64_t s; /* absolute first frame of the slice */ + if (start < 0) { + s = oldest; + if (count == 0 || count > avail) count = avail; + } else { + s = (uint64_t)start; + if (s < oldest) s = oldest; /* evicted prefix clamps */ + if (s >= head) return -1; + uint64_t max = head - s; + if (count == 0 || count > max) count = max; + } + /* Cap a single WAV at the ring capacity (paranoia; avail <= cap). */ + if (count > (uint64_t)PCM_RING_FRAMES) count = PCM_RING_FRAMES; + + FILE *fp = fopen(path, "wb"); + if (!fp) return -1; + /* Stamp the tap's true rate (SPU taps: exactly 44100; host tap: device + * rate in bridge mode). A mislabeled header silently poisons every + * offline A/B (snesrecomp da33c06). */ + wav_write_header(fp, (uint32_t)(count * 4u), s_tap_rate[tap]); + + uint64_t done = 0; + while (done < count) { + uint32_t pos = (uint32_t)((s + done) & PCM_RING_MASK); + uint32_t chunk = PCM_RING_FRAMES - pos; + if ((uint64_t)chunk > count - done) chunk = (uint32_t)(count - done); + if (fwrite(&t->pcm[(size_t)pos * 2u], 4u, chunk, fp) != chunk) { + fclose(fp); + return -1; + } + done += chunk; + } + fclose(fp); + return (int64_t)count; +} diff --git a/runtime/src/autocompile.c b/runtime/src/autocompile.c index b432f3ae..849b65d7 100644 --- a/runtime/src/autocompile.c +++ b/runtime/src/autocompile.c @@ -5,6 +5,7 @@ #include <stdint.h> #include <stdio.h> +#include <stdlib.h> /* getenv — declared here; glibc/windows.h leak it, macOS SDK does not */ #include <string.h> #ifdef _WIN32 @@ -220,6 +221,10 @@ static int json_escape_into(char *dst, int cap, const char *src, int n) { else if (c == '\r') { /* drop */ } else if (c == '\t') { dst[o++] = '\\'; dst[o++] = 't'; } else if (c < 0x20) { o += snprintf(dst + o, cap - o, "\\u%04x", c); } + /* Bytes >= 0x80 are cp1252 console output from the child compile + * (e.g. 0x97 em-dash) — raw they make the JSON invalid UTF-8 and + * break strict clients. Escape as \u00XX (Latin-1 view). */ + else if (c >= 0x80) { o += snprintf(dst + o, cap - o, "\\u%04x", c); } else dst[o++] = (char)c; } dst[o] = '\0'; diff --git a/runtime/src/beetle_debug_server.c b/runtime/src/beetle_debug_server.c index 1675539a..b5d98fca 100644 --- a/runtime/src/beetle_debug_server.c +++ b/runtime/src/beetle_debug_server.c @@ -67,6 +67,15 @@ extern uint32_t beetle_get_sio_trace(uint32_t *out_seq, uint8_t *out_tx, extern uint32_t beetle_get_sio_trace_total(void); extern void beetle_reset_sio_trace(void); +/* CD command trace */ +extern uint32_t beetle_get_cdcmd_trace(uint32_t *out_seq, uint8_t *out_cmd, + uint8_t *out_nargs, uint8_t *out_a0, + uint8_t *out_a1, uint8_t *out_a2, + uint32_t *out_pc, + int max_count); +extern uint32_t beetle_get_cdcmd_trace_total(void); +extern void beetle_reset_cdcmd_trace(void); + /* wtrace */ extern int beetle_wtrace_arm(uint32_t lo, uint32_t hi); extern int beetle_wtrace_disarm(int slot); @@ -139,8 +148,10 @@ extern int beetle_spu_get_global_state( /* Per-frame history ring (parity with runtime's frame_history). */ #include "beetle_history.h" +#include "audio_trace.h" #include "parity_trace.h" #include "device_trace.h" +#include "png_write.h" /* png_write_rgb — shared PNG encoder (matches runtime) */ extern void beetle_history_get_bounds(uint64_t *out_count, uint64_t *out_oldest, uint64_t *out_newest); @@ -160,7 +171,8 @@ extern uint64_t beetle_fntrace_total(void); extern uint32_t beetle_fntrace_get(uint64_t *out_seq, uint32_t *out_caller, uint32_t *out_target, uint32_t *out_ra, uint32_t *out_a0, - uint32_t *out_a1, uint32_t *out_frame, + uint32_t *out_a1, uint32_t *out_a2, + uint32_t *out_a3, uint32_t *out_frame, uint8_t *out_kind, uint32_t *out_sp, int max_count); @@ -387,48 +399,32 @@ static void h_vram_peek(int id, const char *json) { static void h_screenshot_file(int id, const char *json) { char path[512] = {0}; if (!json_get_str(json, "path", path, sizeof(path))) { - strncpy(path, "psx_screenshot.bmp", sizeof(path) - 1); + strncpy(path, "psx_screenshot.png", sizeof(path) - 1); } uint32_t *pixels = NULL; unsigned w = 0, h = 0; if (!beetle_get_framebuffer(&pixels, &w, &h) || !pixels || w == 0 || h == 0) { send_err(id, "no frame"); return; } - /* BMP write (XRGB8888 → 24bpp BGR for portability). */ - FILE *f = fopen(path, "wb"); - if (!f) { send_err(id, "fopen"); return; } - int row_bytes = ((int)w * 3 + 3) & ~3; - int img_size = row_bytes * (int)h; - uint8_t hdr[54] = {0}; - hdr[0] = 'B'; hdr[1] = 'M'; - int filesz = 54 + img_size; - hdr[2] = filesz & 0xFF; hdr[3] = (filesz >> 8) & 0xFF; - hdr[4] = (filesz >> 16) & 0xFF; hdr[5] = (filesz >> 24) & 0xFF; - hdr[10] = 54; - hdr[14] = 40; - hdr[18] = w & 0xFF; hdr[19] = (w >> 8) & 0xFF; - hdr[20] = (w >> 16) & 0xFF; hdr[21] = (w >> 24) & 0xFF; - /* height stored positive → bottom-up rows */ - hdr[22] = h & 0xFF; hdr[23] = (h >> 8) & 0xFF; - hdr[24] = (h >> 16) & 0xFF; hdr[25] = (h >> 24) & 0xFF; - hdr[26] = 1; hdr[28] = 24; - hdr[34] = img_size & 0xFF; hdr[35] = (img_size >> 8) & 0xFF; - hdr[36] = (img_size >> 16) & 0xFF; hdr[37] = (img_size >> 24) & 0xFF; - fwrite(hdr, 1, 54, f); - uint8_t *row = (uint8_t*)malloc(row_bytes); - if (!row) { fclose(f); send_err(id, "alloc"); return; } - memset(row, 0, row_bytes); - for (int y = (int)h - 1; y >= 0; y--) { + /* PNG write (XRGB8888 → 24bpp top-down RGB), matching the runtime's + * screenshot format so both backends emit the same viewer-friendly PNG. */ + uint8_t *rgb = (uint8_t*)malloc((size_t)w * h * 3); + if (!rgb) { send_err(id, "alloc"); return; } + for (unsigned y = 0; y < h; y++) { for (unsigned x = 0; x < w; x++) { - uint32_t px = pixels[y * w + x]; - row[x*3 + 0] = (px ) & 0xFF; /* B */ - row[x*3 + 1] = (px >> 8) & 0xFF; /* G */ - row[x*3 + 2] = (px >> 16) & 0xFF; /* R */ + uint32_t px = pixels[(size_t)y * w + x]; + uint8_t *p = rgb + ((size_t)y * w + x) * 3; + p[0] = (px >> 16) & 0xFF; /* R */ + p[1] = (px >> 8) & 0xFF; /* G */ + p[2] = (px ) & 0xFF; /* B */ } - fwrite(row, 1, row_bytes, f); } - free(row); + FILE *f = fopen(path, "wb"); + if (!f) { free(rgb); send_err(id, "fopen"); return; } + int ok = png_write_rgb(f, rgb, w, h); fclose(f); + free(rgb); + if (!ok) { send_err(id, "png encode failed"); return; } send_fmt("{\"id\":%d,\"ok\":true,\"path\":\"%s\",\"width\":%u,\"height\":%u}\n", id, path, w, h); } @@ -470,6 +466,41 @@ static void h_sio_trace(int id, const char *json) { free(seqs); free(txs); free(rxs); free(ctrl); } +static void h_cdrom_cmd_dump(int id, const char *json) { + int count = json_get_int(json, "count", 256); + if (count < 1) count = 1; + if (count > 8192) count = 8192; + + uint32_t *seqs = (uint32_t*)malloc(count * sizeof(uint32_t)); + uint32_t *pcs = (uint32_t*)malloc(count * sizeof(uint32_t)); + uint8_t *cmds = (uint8_t*)malloc(count), *nargs = (uint8_t*)malloc(count); + uint8_t *a0 = (uint8_t*)malloc(count), *a1 = (uint8_t*)malloc(count), *a2 = (uint8_t*)malloc(count); + if (!seqs || !pcs || !cmds || !nargs || !a0 || !a1 || !a2) { + free(seqs); free(pcs); free(cmds); free(nargs); free(a0); free(a1); free(a2); + send_err(id, "alloc"); return; + } + uint32_t got = beetle_get_cdcmd_trace(seqs, cmds, nargs, a0, a1, a2, pcs, count); + uint32_t total = beetle_get_cdcmd_trace_total(); + + send_fmt("{\"id\":%d,\"ok\":true,\"total\":%u,\"count\":%u,\"entries\":[", + id, total, got); + for (uint32_t i = 0; i < got; i++) { + if (i > 0) send_fmt(","); + send_fmt("{\"seq\":%u,\"cmd\":\"0x%02X\",\"nargs\":%u," + "\"a0\":\"0x%02X\",\"a1\":\"0x%02X\",\"a2\":\"0x%02X\"," + "\"pc\":\"0x%08X\"}", + seqs[i], cmds[i], nargs[i], a0[i], a1[i], a2[i], pcs[i]); + } + send_fmt("]}\n"); + free(seqs); free(pcs); free(cmds); free(nargs); free(a0); free(a1); free(a2); +} + +static void h_cdrom_cmd_reset(int id, const char *json) { + (void)json; + beetle_reset_cdcmd_trace(); + send_ok(id); +} + static void h_sio_write_window(int id, const char *json) { int which = json_get_int(json, "which", 0); int before = json_get_int(json, "before", 4); @@ -872,16 +903,19 @@ static void h_fntrace_dump(int id, const char *json) { uint32_t *ras = (uint32_t*)malloc(count * sizeof(uint32_t)); uint32_t *a0s = (uint32_t*)malloc(count * sizeof(uint32_t)); uint32_t *a1s = (uint32_t*)malloc(count * sizeof(uint32_t)); + uint32_t *a2s = (uint32_t*)malloc(count * sizeof(uint32_t)); + uint32_t *a3s = (uint32_t*)malloc(count * sizeof(uint32_t)); uint32_t *frames = (uint32_t*)malloc(count * sizeof(uint32_t)); uint8_t *kinds = (uint8_t*) malloc(count); uint32_t *sps = (uint32_t*)malloc(count * sizeof(uint32_t)); - if (!seqs || !callers || !targets || !ras || !a0s || !a1s || !frames || !kinds || !sps) { + if (!seqs || !callers || !targets || !ras || !a0s || !a1s || !a2s || !a3s || + !frames || !kinds || !sps) { free(seqs); free(callers); free(targets); free(ras); - free(a0s); free(a1s); free(frames); free(kinds); free(sps); + free(a0s); free(a1s); free(a2s); free(a3s); free(frames); free(kinds); free(sps); send_err(id, "alloc"); return; } uint32_t got = beetle_fntrace_get(seqs, callers, targets, ras, - a0s, a1s, frames, kinds, sps, count); + a0s, a1s, a2s, a3s, frames, kinds, sps, count); uint64_t total = beetle_fntrace_total(); send_fmt("{\"id\":%d,\"ok\":true,\"total\":%llu,\"count\":%u,\"entries\":[", @@ -890,14 +924,16 @@ static void h_fntrace_dump(int id, const char *json) { if (i > 0) send_fmt(","); send_fmt("{\"seq\":%llu,\"caller\":\"0x%08X\",\"target\":\"0x%08X\"," "\"ra\":\"0x%08X\",\"a0\":\"0x%08X\",\"a1\":\"0x%08X\"," + "\"a2\":\"0x%08X\",\"a3\":\"0x%08X\"," "\"frame\":%u,\"kind\":\"%s\",\"sp\":\"0x%08X\"}", (unsigned long long)seqs[i], callers[i], targets[i], - ras[i], a0s[i], a1s[i], frames[i], fn_kind_str(kinds[i]), sps[i]); + ras[i], a0s[i], a1s[i], a2s[i], a3s[i], + frames[i], fn_kind_str(kinds[i]), sps[i]); } send_fmt("]}\n"); free(seqs); free(callers); free(targets); free(ras); - free(a0s); free(a1s); free(frames); free(kinds); free(sps); + free(a0s); free(a1s); free(a2s); free(a3s); free(frames); free(kinds); free(sps); } /* ---- spu_voices: Beetle oracle ground truth via PS_SPU::GetRegister ---- @@ -1016,6 +1052,119 @@ static void h_spu_events(int id, const char *json) { free(vlcs); free(vrcs); free(als); free(ahs); free(kinds); free(vs); } +/* ---- audio_stats / audio_wav / audio_events: always-on oracle PCM tap ---- + * + * Wire-protocol mirror of psx-runtime's audio_* commands (port 4370). + * On psx-beetle only tap 0 (spu_out) is fed — Beetle's fully-mixed + * 44.1 kHz reference SPU output captured at the libretro audio_batch + * callback (beetle_libretro.cpp). Pump/queue fields report zero: this + * binary never opens a host audio device. */ +static void h_audio_stats(int id, const char *json) { + (void)json; + AudioTraceStats st; + audio_trace_get_stats(&st); + send_fmt("{\"id\":%d,\"ok\":true," + "\"taps\":[" + "{\"name\":\"spu_out\",\"frames\":%llu,\"nonzero\":%llu," + "\"audible\":%llu,\"peak\":%d}," + "{\"name\":\"cd_in\",\"frames\":%llu,\"nonzero\":%llu," + "\"audible\":%llu,\"peak\":%d}," + "{\"name\":\"host\",\"frames\":%llu,\"nonzero\":%llu," + "\"audible\":%llu,\"peak\":%d}]," + "\"pump_calls\":%llu,\"pump_skips\":%llu,\"underruns\":%llu," + "\"queue_hiwater\":%u,\"queue_lowater\":%u," + "\"mutes\":%llu,\"unmutes\":%llu,\"events_total\":%llu}\n", + id, + (unsigned long long)st.tap_frames[0], + (unsigned long long)st.tap_nonzero[0], + (unsigned long long)st.tap_audible[0], st.tap_peak[0], + (unsigned long long)st.tap_frames[1], + (unsigned long long)st.tap_nonzero[1], + (unsigned long long)st.tap_audible[1], st.tap_peak[1], + (unsigned long long)st.tap_frames[2], + (unsigned long long)st.tap_nonzero[2], + (unsigned long long)st.tap_audible[2], st.tap_peak[2], + (unsigned long long)st.pump_calls, + (unsigned long long)st.pump_skips, + (unsigned long long)st.underruns, + st.queue_hiwater, st.queue_lowater, + (unsigned long long)st.mute_events, + (unsigned long long)st.unmute_events, + (unsigned long long)st.events_total); +} + +static void h_audio_wav(int id, const char *json) { + char path[512]; + if (!json_get_str(json, "path", path, sizeof(path))) { + send_err(id, "missing path"); + return; + } + int tap = json_get_int(json, "tap", AUDIO_TAP_SPU_OUT); + char buf[32]; + int64_t start = -1; + uint64_t count = 0; + if (json_get_str(json, "start", buf, sizeof(buf))) + start = strtoll(buf, NULL, 0); + else + start = (int64_t)json_get_int(json, "start", -1); + if (json_get_str(json, "count", buf, sizeof(buf))) + count = strtoull(buf, NULL, 0); + else + count = (uint64_t)json_get_int(json, "count", 0); + + int64_t wrote = audio_trace_dump_wav(tap, path, start, count); + if (wrote < 0) { + send_err(id, "dump failed (bad tap/path or empty ring)"); + return; + } + send_fmt("{\"id\":%d,\"ok\":true,\"tap\":%d,\"frames\":%lld," + "\"rate\":44100,\"tap_total\":%llu}\n", + id, tap, (long long)wrote, + (unsigned long long)audio_trace_tap_total(tap)); +} + +static void h_audio_events(int id, const char *json) { + int count = json_get_int(json, "count", 256); + if (count < 1) count = 1; + if (count > 8192) count = 8192; + AudioTraceEvent *evs = + (AudioTraceEvent *)malloc((size_t)count * sizeof(AudioTraceEvent)); + if (!evs) { send_err(id, "alloc"); return; } + uint32_t got = audio_trace_events_get(evs, (uint32_t)count); + uint64_t total = audio_trace_events_total(); + static const char *kind_names[] = { + "?", "REG", "RENDER", "SKIP", "UNDERRUN", + "MUTE", "UNMUTE", "CD_PUSH", "DMA" + }; + send_fmt("{\"id\":%d,\"ok\":true,\"total\":%llu,\"count\":%u,\"events\":[", + id, (unsigned long long)total, (unsigned)got); + for (uint32_t i = 0; i < got; i++) { + const AudioTraceEvent *e = &evs[i]; + const char *kn = (e->kind < sizeof(kind_names) / sizeof(kind_names[0])) + ? kind_names[e->kind] : "?"; + if (i > 0) send_fmt(","); + send_fmt("{\"seq\":%llu,\"smp\":%llu,\"frame\":%u," + "\"kind\":\"%s\",\"a\":\"0x%X\",\"b\":\"0x%X\"}", + (unsigned long long)e->seq, + (unsigned long long)e->sample_idx, + e->frame, kn, e->a, e->b); + } + send_fmt("]}\n"); + free(evs); +} + +/* ---- cdc_volume: CD-audio DecodeVolume matrix ground truth ---- */ +extern int beetle_cdc_decode_volume(unsigned out[4]); +static void h_cdc_volume(int id, const char *json) { + (void)json; + unsigned m[4] = {0, 0, 0, 0}; + if (!beetle_cdc_decode_volume(m)) { send_err(id, "cdc_unavailable"); return; } + send_fmt("{\"id\":%d,\"ok\":true," + "\"ll\":\"0x%02X\",\"lr\":\"0x%02X\"," + "\"rl\":\"0x%02X\",\"rr\":\"0x%02X\"}\n", + id, m[0], m[1], m[2], m[3]); +} + /* ---- wtrace_all (always-on, no-filter, lean fields) ---- * Mirrors runtime's wtrace_all surface so probes that haven't pre-armed * a range can still see the last ~1 second of writes the moment they @@ -1484,6 +1633,8 @@ static const CmdEntry CMDS[] = { { "vram_peek", h_vram_peek }, { "sio_trace_reset", h_sio_trace_reset }, { "sio_trace", h_sio_trace }, + { "cdrom_cmd_dump", h_cdrom_cmd_dump }, + { "cdrom_cmd_reset", h_cdrom_cmd_reset }, { "sio_write_window", h_sio_write_window }, /* wtrace — normalized verb set (parity contract with psx-runtime). */ { "wtrace_arm", h_wtrace_arm }, @@ -1513,6 +1664,10 @@ static const CmdEntry CMDS[] = { { "fntrace_dump", h_fntrace_dump }, { "spu_voices", h_spu_voices }, { "spu_events", h_spu_events }, + { "audio_stats", h_audio_stats }, + { "audio_wav", h_audio_wav }, + { "audio_events", h_audio_events }, + { "cdc_volume", h_cdc_volume }, /* Per-frame history ring (parity with psx-runtime). */ { "history", h_history }, { "get_frame", h_get_frame }, diff --git a/runtime/src/beetle_libretro.cpp b/runtime/src/beetle_libretro.cpp index 1d18d16e..c239e702 100644 --- a/runtime/src/beetle_libretro.cpp +++ b/runtime/src/beetle_libretro.cpp @@ -15,15 +15,18 @@ #include <cstdlib> #include <cstring> #include <cstdarg> +#include <csignal> #include "libretro.h" #include "mednafen/psx/psx.h" #include "mednafen/psx/cpu.h" #include "mednafen/psx/gpu.h" #include "mednafen/psx/spu.h" +#include "mednafen/psx/cdc.h" #include "mednafen/psx/frontio.h" #include "mednafen/psx/irq.h" #include "beetle_history.h" +#include "audio_trace.h" /* always-on oracle PCM tap ring (port 4380 audio_wav) */ #include "parity_trace.h" /* general control-flow parity ring (oracle producer) */ #include "device_trace.h" /* general device-event cycle ring (oracle producer) */ static void irq_event_callback(int which); /* fwd: device-event ring producer */ @@ -59,6 +62,36 @@ static void sio_trace_callback(uint8_t tx, uint8_t rx, uint16_t ctrl) { s_sio_trace_idx = (s_sio_trace_idx + 1) % BEETLE_SIO_TRACE_CAP; } +/* ---- CD command trace (oracle-diff for the Kula World CD-init wedge) ---- */ +#define BEETLE_CDCMD_TRACE_CAP 8192 +struct BeetleCdcmdEntry { + uint32_t seq; uint32_t pc; uint8_t cmd, nargs, a0, a1, a2; +}; +static BeetleCdcmdEntry s_cdcmd_trace[BEETLE_CDCMD_TRACE_CAP]; +static int s_cdcmd_idx = 0; +static uint32_t s_cdcmd_seq = 0; +static void cdcmd_trace_callback(uint8_t cmd, uint8_t nargs, + uint8_t a0, uint8_t a1, uint8_t a2, + uint32_t pc) { + /* Symmetric self-stop trap (mirror of the runtime's PSX_CD_TRAP_CMD): + * SIGSTOP on the Nth matching CD command so a debugger can freeze the + * oracle at the exact boot point and diff its state against ours. */ + { + static long trap_cmd = -2, trap_nth = 1, hits = 0; + if (trap_cmd == -2) { + const char *e2 = getenv("PSX_CD_TRAP_CMD"); + trap_cmd = (e2 && *e2) ? strtol(e2, NULL, 0) : -1; + const char *e3 = getenv("PSX_CD_TRAP_NTH"); + if (e3 && *e3) trap_nth = strtol(e3, NULL, 0); + } + if ((long)cmd == trap_cmd && ++hits == trap_nth) raise(SIGSTOP); + } + BeetleCdcmdEntry *e = &s_cdcmd_trace[s_cdcmd_idx]; + e->seq = s_cdcmd_seq++; e->pc = pc; e->cmd = cmd; e->nargs = nargs; + e->a0 = a0; e->a1 = a1; e->a2 = a2; + s_cdcmd_idx = (s_cdcmd_idx + 1) % BEETLE_CDCMD_TRACE_CAP; +} + /* ---- wtrace_all ring (ALWAYS-ON; no filter; captures every write) ---- * Per global rule "ring buffers always-on": every write hits this ring * unconditionally so probes connected late can still query the last ~1s @@ -249,7 +282,7 @@ static void rtrace_callback(uint32_t addr, uint32_t value, #define BEETLE_FNTRACE_MAX_ARMS 64 struct BeetleFnTraceEntry { uint64_t seq; - uint32_t caller_pc, target_pc, parent_ra, a0, a1, frame, sp; + uint32_t caller_pc, target_pc, parent_ra, a0, a1, a2, a3, frame, sp; uint8_t kind, pad[3]; }; static BeetleFnTraceEntry s_fntrace[BEETLE_FNTRACE_CAP]; @@ -296,6 +329,14 @@ static void fntrace_callback(uint32_t caller_pc, uint32_t target_pc, e->parent_ra = parent_ra; e->a0 = a0; e->a1 = a1; + if (PSX_CPU) { + char dummy[8] = {0}; + e->a2 = PSX_CPU->GetRegister(PS_CPU::GSREG_GPR + 6, dummy, sizeof(dummy)); + e->a3 = PSX_CPU->GetRegister(PS_CPU::GSREG_GPR + 7, dummy, sizeof(dummy)); + } else { + e->a2 = 0; + e->a3 = 0; + } e->frame = s_frame_count; e->kind = kind; /* SP at retire — needed for the func_8001A954 SP/RA-lifecycle oracle @@ -336,8 +377,18 @@ void on_video_refresh(const void *data, unsigned width, unsigned height, size_t } } -void on_audio_sample(int16_t l, int16_t r) { (void)l; (void)r; } -size_t on_audio_sample_batch(const int16_t *data, size_t frames) { (void)data; return frames; } +/* Beetle's fully-mixed 44.1 kHz reference SPU output arrives here every + * retro_run. Record it into the always-on audio trace (tap SPU_OUT) so port + * 4380's audio_wav can dump oracle PCM for offline A/B against the recomp's + * tap on port 4370. The host still plays no audio from psx-beetle. */ +void on_audio_sample(int16_t l, int16_t r) { + int16_t frame[2] = { l, r }; + audio_trace_pcm(AUDIO_TAP_SPU_OUT, frame, 1); +} +size_t on_audio_sample_batch(const int16_t *data, size_t frames) { + audio_trace_pcm(AUDIO_TAP_SPU_OUT, data, (int)frames); + return frames; +} void on_input_poll(void) {} int16_t on_input_state(unsigned port, unsigned device, unsigned index, unsigned id) { @@ -519,6 +570,8 @@ extern "C" int beetle_init_with_disc(const char *bios_path, const char *disc_pat g_psxrecomp_rtrace_cb = rtrace_callback; g_psxrecomp_irq_cb = irq_event_callback; std::fprintf(stderr, "[psx-beetle] wtrace + fntrace + rtrace + irq callbacks registered\n"); + g_psxrecomp_cdcmd_cb = cdcmd_trace_callback; + std::fprintf(stderr, "[psx-beetle] wtrace + fntrace + cdcmd callbacks registered\n"); std::fflush(stderr); return 0; @@ -535,6 +588,7 @@ extern "C" void beetle_run_frame(uint16_t pad1_buttons) { if (!s_loaded) return; s_joypad = pad1_buttons; s_frame_count++; + audio_trace_note_frame(s_frame_count); retro_run(); beetle_history_record_frame(); } @@ -685,6 +739,36 @@ extern "C" void beetle_reset_sio_trace(void) { memset(s_sio_trace, 0, sizeof(s_sio_trace)); } +/* ---- CD command trace accessors ---- */ +extern "C" uint32_t beetle_get_cdcmd_trace(uint32_t *out_seq, uint8_t *out_cmd, + uint8_t *out_nargs, uint8_t *out_a0, + uint8_t *out_a1, uint8_t *out_a2, + uint32_t *out_pc, + int max_count) +{ + int avail = (int)(s_cdcmd_seq < (uint32_t)BEETLE_CDCMD_TRACE_CAP + ? s_cdcmd_seq : BEETLE_CDCMD_TRACE_CAP); + int count = max_count < avail ? max_count : avail; + int start = (s_cdcmd_idx - count + BEETLE_CDCMD_TRACE_CAP) % BEETLE_CDCMD_TRACE_CAP; + for (int i = 0; i < count; i++) { + const BeetleCdcmdEntry *e = &s_cdcmd_trace[(start + i) % BEETLE_CDCMD_TRACE_CAP]; + out_seq[i] = e->seq; + out_cmd[i] = e->cmd; + out_nargs[i] = e->nargs; + out_a0[i] = e->a0; + out_a1[i] = e->a1; + out_a2[i] = e->a2; + out_pc[i] = e->pc; + } + return (uint32_t)count; +} +extern "C" uint32_t beetle_get_cdcmd_trace_total(void) { return s_cdcmd_seq; } +extern "C" void beetle_reset_cdcmd_trace(void) { + s_cdcmd_idx = 0; + s_cdcmd_seq = 0; + memset(s_cdcmd_trace, 0, sizeof(s_cdcmd_trace)); +} + /* ---- wtrace accessors ---- */ extern "C" int beetle_wtrace_arm(uint32_t lo, uint32_t hi) { if (s_wtrace_range_count >= BEETLE_WTRACE_MAX_RANGES) return -1; @@ -924,6 +1008,21 @@ extern "C" void psxrecomp_beetle_spu_event(unsigned kind, unsigned voice, extern "C" uint64_t beetle_spu_event_total(void) { return s_spu_event_seq; } +/* CD-audio volume matrix ground truth (PS_CDC::DecodeVolume; 0x80 = unity). + * Returns 0 if the CDC isn't up. */ +extern "C" int beetle_cdc_decode_volume(unsigned out[4]) +{ + extern PS_CDC *PSX_CDC; + if (!PSX_CDC) return 0; + uint8 m[2][2]; + PSX_CDC->PSXRecomp_GetDecodeVolume(m); + out[0] = m[0][0]; /* L -> L */ + out[1] = m[0][1]; /* L -> R */ + out[2] = m[1][0]; /* R -> L */ + out[3] = m[1][1]; /* R -> R */ + return 1; +} + extern "C" uint32_t beetle_spu_event_get(uint64_t *out_seq, uint32_t *out_frame, uint32_t *out_addr, uint16_t *out_env, uint16_t *out_pitch, @@ -1010,7 +1109,8 @@ extern "C" int beetle_spu_get_global_state( extern "C" uint32_t beetle_fntrace_get(uint64_t *out_seq, uint32_t *out_caller, uint32_t *out_target, uint32_t *out_ra, uint32_t *out_a0, - uint32_t *out_a1, uint32_t *out_frame, + uint32_t *out_a1, uint32_t *out_a2, + uint32_t *out_a3, uint32_t *out_frame, uint8_t *out_kind, uint32_t *out_sp, int max_count) { @@ -1027,6 +1127,8 @@ extern "C" uint32_t beetle_fntrace_get(uint64_t *out_seq, out_ra[i] = e->parent_ra; out_a0[i] = e->a0; out_a1[i] = e->a1; + out_a2[i] = e->a2; + out_a3[i] = e->a3; out_frame[i] = e->frame; out_kind[i] = e->kind; out_sp[i] = e->sp; diff --git a/runtime/src/beetle_main.cpp b/runtime/src/beetle_main.cpp index d8c2c0cf..a6132356 100644 --- a/runtime/src/beetle_main.cpp +++ b/runtime/src/beetle_main.cpp @@ -222,10 +222,22 @@ int main(int argc, char** argv) { pixels[y * w + x] = fb[y * w + x] | 0xFF000000u; } } - SDL_UpdateTexture(tex, NULL, pixels, (int)(w * sizeof(uint32_t))); + /* Upload and present ONLY the valid w×h region. The texture is + * allocated at the max PSX size (640×512), but the live frame + * dimensions vary per-frame (256/320/368/512/640 wide, 240/480 + * tall). Passing NULL rects made SDL treat the source as the full + * 640×512 texture while `pixels` is tightly packed at stride w: + * each 640-texel texture row was filled from a w-stride buffer + * (→ horizontal doubling / shear) and rows past the h*w packed + * data drew uninitialized memory (→ garbage streaks below). A + * per-frame src rect fixes both — the upload's pitch (w*4) now + * matches the packed stride, and only the valid pixels are scaled + * to the window. */ + SDL_Rect src = { 0, 0, (int)w, (int)h }; + SDL_UpdateTexture(tex, &src, pixels, (int)(w * sizeof(uint32_t))); SDL_Rect dst = { 0, 0, 640, 480 }; SDL_RenderClear(ren); - SDL_RenderCopy(ren, tex, NULL, &dst); + SDL_RenderCopy(ren, tex, &src, &dst); SDL_RenderPresent(ren); } diff --git a/runtime/src/bios_hle.c b/runtime/src/bios_hle.c index 5f389fd4..75c9a421 100644 --- a/runtime/src/bios_hle.c +++ b/runtime/src/bios_hle.c @@ -237,7 +237,11 @@ static void hle_deliver_event(CPUState* cpu, uint32_t base, uint32_t n) * $ra is restored before the hook returns. */ uint32_t saved_ra = cpu->gpr[31]; cpu->gpr[31] = KADDR_DELIVER_RET; + extern int g_exec_phase; /* wall-time sampler (dirty_ram_interp.c) */ + int prev_phase = g_exec_phase; + g_exec_phase = 3; /* compiled route; dirty/native callees re-tag inside */ psx_dispatch_call(cpu, func, KADDR_DELIVER_RET); + g_exec_phase = prev_phase; cpu->gpr[31] = saved_ra; } } diff --git a/runtime/src/cdrom.c b/runtime/src/cdrom.c index 3ab86e82..ee062a96 100644 --- a/runtime/src/cdrom.c +++ b/runtime/src/cdrom.c @@ -16,6 +16,9 @@ #include "event_ring.h" #include <string.h> #include <stdlib.h> +#ifndef _WIN32 +# include <signal.h> +#endif #ifdef _WIN32 # define WIN32_LEAN_AND_MEAN # include <windows.h> @@ -49,6 +52,21 @@ static uint8_t request_reg; static uint8_t irq_enable; static uint8_t irq_flag; +/* Disc license region string returned in GetID's last four response bytes + * ("SCEE" PAL / "SCEA" NTSC-U / "SCEI" NTSC-J). Real hardware reports the + * region of the INSERTED DISC (mechacon reads it from the license area); + * the BIOS CD driver revalidates it when the kernel CD subsystem + * reinitializes mid-game, and a mismatch throws it into an endless + * GetStat/Init retry loop (Kula World wedged at its first level load this + * way). Set from the mounted disc's SYSTEM.CNF serial at launch + * (main.cpp); the default matches the console region of the one supported + * BIOS (SCPH1001, NTSC-U). */ +static uint8_t disc_scex[4] = { 'S', 'C', 'E', 'A' }; + +void cdrom_set_disc_scex(const char scex[4]) { + memcpy(disc_scex, scex, 4); +} + /* CPS-native CD interrupt single-outstanding latch. * * The CD controller serializes responses: it presents one INT to the CPU @@ -617,6 +635,31 @@ static void xa_reset_decode(void) { xa_stream_active = 0; } +/* CD-audio volume matrix ([data_source][output_port], 0x80 = unity) applied + * to every decoded XA/CD-DA sample, exactly as the CD controller does on + * real hardware (Beetle PS_CDC::ApplyVolume, cdc.cpp:524). Written via the + * index-2/3 register banks and latched by the "apply changes" bit; games + * drive it CONSTANTLY — X5 fades music between scenes with it (measured + * 0x7E steady, ramping to 0x00 at transitions on the Beetle oracle). Not + * modeling it left fades missing and steady levels ~0.6 dB hot. */ +static uint8_t cd_pending_vol[2][2] = { { 0x80, 0x00 }, { 0x00, 0x80 } }; +static uint8_t cd_decode_vol[2][2] = { { 0x80, 0x00 }, { 0x00, 0x80 } }; + +static void cd_apply_decode_volume(int16_t *stereo, int frames) { + /* Fast path: identity matrix (the reset state). */ + if (cd_decode_vol[0][0] == 0x80 && cd_decode_vol[1][1] == 0x80 && + cd_decode_vol[0][1] == 0x00 && cd_decode_vol[1][0] == 0x00) + return; + for (int i = 0; i < frames; i++) { + int32_t l = stereo[i * 2 + 0]; + int32_t r = stereo[i * 2 + 1]; + int32_t lo = ((l * cd_decode_vol[0][0]) >> 7) + ((r * cd_decode_vol[1][0]) >> 7); + int32_t ro = ((l * cd_decode_vol[0][1]) >> 7) + ((r * cd_decode_vol[1][1]) >> 7); + stereo[i * 2 + 0] = clamp16_cd(lo); + stereo[i * 2 + 1] = clamp16_cd(ro); + } +} + static int xa_decode_sector_4bit_stereo(const uint8_t* data, int16_t* out) { static const int k0[5] = { 0, 60, 115, 98, 122 }; static const int k1[5] = { 0, 0, -52, -55, -60 }; @@ -769,6 +812,9 @@ static int maybe_deliver_xa_audio(const uint8_t* raw_data, : xa_decode_sector_4bit_mono(raw_data + XA_DATA_OFFSET, native); int out_frames = xa_resample_to_44100(native, native_frames, sample_rate, pcm_44100, XA_MAX_44100_FRAMES); + /* Volume is applied after resampling, per PS1 hardware tests (Beetle + * cdc.cpp GetCDAudio comment). */ + cd_apply_decode_volume(pcm_44100, out_frames); spu_cd_audio_push(pcm_44100, out_frames); trace_cdrom('A', 0, ((uint32_t)file << 24) | ((uint32_t)channel << 16) | @@ -971,7 +1017,53 @@ static void queue_or_exec_command(uint8_t cmd) { record_command_history('Q', cmd, queued_cmd.params, queued_cmd.param_count); } +/* Pause (0x09) completion latency — Beetle PS_CDC::Command_Pause. + * + * A Pause issued while the drive is reading/playing completes only after the + * head settles: (1124584 + lba*42596/4500) CPU cycles in double-speed mode, + * doubled at single speed — roughly 1.1M+ cycles (~2 video frames). A Pause + * issued while already paused/stopped acks its second response quickly (5000). + * + * This is a CPU-visible ordering contract, NOT sector cadence: games issue + * Pause from mainline code at every streamed-file boundary and finish their + * driver bookkeeping in the frames before the completion INT2 arrives. The + * previous flat apply_speed(10000) delivered INT2 ~100x early — inside the + * issuing frame — and the game's driver intermittently lost the completion + * (Ape Escape's memcard scene loader wedged at a random file boundary, its + * async CD queue never pumping the next file). Hence also NO apply_speed(): + * disc-speed divisors / 'instant' mode must never compress this latency back + * into the race window. Call BEFORE stop_read_stream()/CDSTAT_READ clear. */ +static int pause_complete_delay_cycles(void) { + if (!reading && !(stat_reg & (CDSTAT_READ | CDSTAT_PLAY))) + return 5000; + int lba = reading ? msf_to_lba(read_min, read_sec, read_sect) + : last_sector_lba; + if (lba < 0) lba = 0; + int64_t cycles = 1124584 + (int64_t)lba * 42596 / (75 * 60); + if (!(mode_reg & 0x80)) + cycles *= 2; + return (int)cycles; +} + static void exec_command(uint8_t cmd) { +#if !defined(PSX_NO_DEBUG_TOOLS) && !defined(_WIN32) + /* Self-stop trap: PSX_CD_TRAP_CMD=<byte> makes the process SIGSTOP + * itself the moment that CD command dispatches, so a debugger can + * attach and read the full native+guest call chain at the exact + * instant with ZERO run-speed cost. (A gdb conditional breakpoint on + * this function slows the emu two orders of magnitude via ptrace — + * unusable for wedges that need minutes of healthy boot first.) */ + { + static long trap_cmd = -2, trap_nth = 1, trap_hits = 0; + if (trap_cmd == -2) { + const char *e = getenv("PSX_CD_TRAP_CMD"); + trap_cmd = (e && *e) ? strtol(e, NULL, 0) : -1; + e = getenv("PSX_CD_TRAP_NTH"); /* stop on the Nth match (default 1st) */ + if (e && *e) trap_nth = strtol(e, NULL, 0); + } + if ((long)cmd == trap_cmd && ++trap_hits == trap_nth) raise(SIGSTOP); + } +#endif trace_cdrom('C', 0, cmd, 0); /* ENQUEUE: a CD command was issued (aux = command byte). */ event_ring_record_aux(EV_ENQ, (uint8_t)SRC_CD_CMD, (uint32_t)cmd); @@ -1021,7 +1113,55 @@ static void exec_command(uint8_t cmd) { set_irq(CDIRQ_ACK); break; - case 0x09: /* Pause */ + case 0x07: /* MotorOn — spin the drive motor up. Two-phase like Stop: + * INT3 (ACK) now with the current status, then a pending INT2 + * (COMPLETE) after spin-up reporting status with the motor bit + * SET (psx-spx "07h MotorOn"). If the motor is ALREADY spinning + * the hardware rejects the command with INT5 (ERROR) and error + * code 0x20 ("wrong condition") — replicate that. + * + * Previously 0x07 had no case and fell through to default -> + * CDIRQ_ERROR (INT5) with no error code, so a game that spins the + * motor up (e.g. after a Stop) and waits for the MotorOn + * completion IRQ would never see it. */ + if (stat_reg & CDSTAT_MOTOR) { + response_push(stat_reg | CDSTAT_ERROR); + response_push(0x20); /* error code: motor already on */ + set_irq(CDIRQ_ERROR); + break; + } + response_push(stat_reg); + set_irq(CDIRQ_ACK); + pending.cmd = 0x07; + pending.pending = 1; + pending.delay = apply_speed(30000); /* motor spin-up */ + pending.phase = 1; + break; + + case 0x08: /* Stop — stop the motor. Two-phase like Pause: INT3 (ACK) now + * with the pre-stop status (motor still spinning), then a pending + * INT2 (COMPLETE) after the motor spins down, reporting the new + * status with the motor bit cleared (psx-spx "08h Stop"). + * + * Previously 0x08 had no case and fell through to default -> + * CDIRQ_ERROR (INT5). A game that stops the drive on a scene + * change then waits for the Stop completion IRQ never sees it and + * hangs: Tsumu Light's CD library retries Stop forever (~90-frame + * timeout) and never advances past its first content load. */ + stop_read_stream(); + xa_reset_decode(); + spu_cd_audio_reset(); + stat_reg &= ~(CDSTAT_READ | CDSTAT_PLAY | CDSTAT_SEEK); + response_push(stat_reg); + set_irq(CDIRQ_ACK); + pending.cmd = 0x08; + pending.pending = 1; + pending.delay = apply_speed(30000); /* motor spin-down */ + pending.phase = 1; + break; + + case 0x09: { /* Pause */ + int complete_delay = pause_complete_delay_cycles(); stop_read_stream(); xa_reset_decode(); spu_cd_audio_reset(); @@ -1030,9 +1170,10 @@ static void exec_command(uint8_t cmd) { set_irq(CDIRQ_ACK); pending.cmd = 0x09; pending.pending = 1; - pending.delay = apply_speed(10000); + pending.delay = complete_delay; pending.phase = 1; break; + } case 0x0A: /* Init */ stop_read_stream(); @@ -1043,7 +1184,11 @@ static void exec_command(uint8_t cmd) { set_irq(CDIRQ_ACK); pending.cmd = 0x0A; pending.pending = 1; - pending.delay = apply_speed(50000); + /* Beetle models the drive-reset busy period as 1136000 cycles + * (PS_CDC::Command_Reset, PSRCounter). Same second-response class as + * Pause: an authentic multi-frame latency games' drivers rely on — + * never scaled by the disc-speed divisor. */ + pending.delay = 1136000; pending.phase = 1; break; @@ -1223,7 +1368,9 @@ static void exec_command(uint8_t cmd) { set_irq(CDIRQ_ACK); pending.cmd = 0x1A; pending.pending = 1; - pending.delay = apply_speed(30000); + /* Beetle PS_CDC::Command_ID: second response after 33868 cycles. + * Unscaled — same authentic-latency class as Pause/Init/ReadTOC. */ + pending.delay = 33868; pending.phase = 1; break; @@ -1243,16 +1390,28 @@ static void exec_command(uint8_t cmd) { set_irq(CDIRQ_ACK); pending.cmd = 0x1E; pending.pending = 1; - pending.delay = apply_speed(100000); + /* Beetle PS_CDC::Command_ReadTOC: ~30M cycles (a near-second TOC + * re-scan; Beetle adds a seek term on top — we keep the dominant + * constant). Unscaled — authentic-latency class. */ + pending.delay = 30000000; pending.phase = 1; break; case 0x19: /* Test */ if (param_count >= 1 && param_fifo[0] == 0x20) { - response_push(0x97); - response_push(0x01); - response_push(0x10); - response_push(0xC2); + /* CD controller firmware version (BCD date + region). This BIOS is + * SCPH-1001, whose sub-CPU reports the 1994 controller: 94/09/19 C0. + * The value must be < 0x95 in the high byte — the shell's CD-init + * (func at ROM 0x1DF50) sets kernel flag [0xA000DFFC]=1 when the + * version byte >= 0x95, which later makes the boot CD-open + * (0xBFC0D570) issue a spurious ReadTOC that wedges the game's + * streaming reads. Beetle hardcodes the PSone-era 0x97 regardless of + * BIOS, which is wrong for SCPH-1001; matching the real 1994 + * controller keeps the flag clear (Kula World demo-load wedge). */ + response_push(0x94); + response_push(0x09); + response_push(0x19); + response_push(0xC0); set_irq(CDIRQ_ACK); } else { response_push(stat_reg); @@ -1303,6 +1462,20 @@ static void process_pending(uint32_t cycles) { } break; + case 0x07: /* MotorOn complete — motor now spinning */ + stat_reg |= CDSTAT_MOTOR; + response_push(stat_reg); + set_irq(CDIRQ_COMPLETE); + fire_cdrom_irq(); + break; + + case 0x08: /* Stop complete — motor has spun down */ + stat_reg &= ~(CDSTAT_MOTOR | CDSTAT_READ | CDSTAT_PLAY | CDSTAT_SEEK); + response_push(stat_reg); + set_irq(CDIRQ_COMPLETE); + fire_cdrom_irq(); + break; + case 0x09: /* Pause complete */ stat_reg &= ~CDSTAT_READ; response_push(stat_reg); @@ -1335,10 +1508,10 @@ static void process_pending(uint32_t cycles) { response_push(0x00); response_push(0x20); response_push(0x00); - response_push('S'); - response_push('C'); - response_push('E'); - response_push('I'); + response_push(disc_scex[0]); + response_push(disc_scex[1]); + response_push(disc_scex[2]); + response_push(disc_scex[3]); set_irq(CDIRQ_COMPLETE); } fire_cdrom_irq(); @@ -1463,11 +1636,19 @@ uint32_t cdrom_read(uint32_t addr) { switch (addr) { case 0x1F801800: { uint8_t s = index_reg & 0x03; - s |= (1 << 2); /* ADPCM empty */ + /* Bit 2 is ADPBUSY (XA-ADPCM playback in progress), NOT "ADPCM + * empty" — it must idle at 0. The oracle (beetle cdc.cpp, Read + * A==0) never raises it. Raising it permanently made every BIOS + * status poll see "XA busy" and steered the kernel CD driver + * init down a different branch from real hardware. */ if (param_count == 0) s |= (1 << 3); if (param_count < PARAM_FIFO_SIZE) s |= (1 << 4); if (response_read < response_count) s |= (1 << 5); if (data_fifo_ready()) s |= (1 << 6); + /* Bit 7 BUSYSTS: command written but not yet executed (our queued + * path; the synchronous path leaves no guest-observable window). + * Mirrors beetle's PendingCommandCounter/phase<=1 busy window. */ + if (queued_cmd.pending) s |= (1 << 7); ret = s; break; } @@ -1515,6 +1696,8 @@ void cdrom_write(uint32_t addr, uint32_t value) { case 0x1F801801: if (index_reg == 0) { queue_or_exec_command(val); + } else if (index_reg == 3) { + cd_pending_vol[1][1] = val; /* Right-CD -> Right-SPU */ } break; @@ -1525,11 +1708,24 @@ void cdrom_write(uint32_t addr, uint32_t value) { } } else if (index_reg == 1) { irq_enable = val & 0x1F; + } else if (index_reg == 2) { + cd_pending_vol[0][0] = val; /* Left-CD -> Left-SPU */ + } else if (index_reg == 3) { + cd_pending_vol[1][0] = val; /* Right-CD -> Left-SPU */ } break; case 0x1F801803: - if (index_reg == 0) { + if (index_reg == 2) { + cd_pending_vol[0][1] = val; /* Left-CD -> Right-SPU */ + } else if (index_reg == 3) { + /* Apply-changes latch (bit 5) copies pending -> live, exactly + * like Beetle cdc.cpp case 0x0B. (ADPMute bit 0 is not modeled + * by the Beetle oracle either.) */ + if (val & 0x20) { + memcpy(cd_decode_vol, cd_pending_vol, sizeof(cd_decode_vol)); + } + } else if (index_reg == 0) { request_reg = val; if (!(request_reg & CDROM_REQUEST_BFRD)) { sector_read_pos = 0; diff --git a/runtime/src/code_provider.c b/runtime/src/code_provider.c index ed5a9041..cbae5926 100644 --- a/runtime/src/code_provider.c +++ b/runtime/src/code_provider.c @@ -1,11 +1,16 @@ -/* code_provider.c — the two CodeProvider implementations + active selection. - * See code_provider.h for the abstraction and SLJIT.md §3/§6/§7 for the plan. +/* code_provider.c — the CodeProvider implementations + active selection. + * See code_provider.h for the abstraction. * - * This is the step-2 seam: the gcc provider is a thin pass-through over the - * validated autocompile_* path (no behavior change), and the sljit provider - * wraps the in-process JIT. Which one is active is decided once at init by - * overlay_backend_resolve (env PSX_OVERLAY_BACKEND > [runtime] overlay_backend > - * auto; auto => gcc if a compile cmd is wired, else sljit). gcc is the default. */ + * The "spawn compiler" provider is a thin pass-through over the validated + * autocompile_* path (compile emitted overlay C to a DLL and load it); it backs + * BOTH the gcc and tcc tiers, which differ only in which compiler the pipeline + * invokes. The in-process JIT provider (code_provider_sljit) is a separate, + * legacy path. Which provider is active is decided once at init by + * overlay_backend_resolve() in overlay_backend.c (env PSX_OVERLAY_BACKEND > + * [runtime] overlay_backend > auto). The auto tier order is: + * static -> gcc (if gcc on PATH) -> tcc (bundled toolchain-free fallback). */ + +#include <stddef.h> #include "code_provider.h" #include "autocompile.h" diff --git a/runtime/src/crash_trace.c b/runtime/src/crash_trace.c index b704371f..af96c7da 100644 --- a/runtime/src/crash_trace.c +++ b/runtime/src/crash_trace.c @@ -8,7 +8,7 @@ * - trap_crash * - TCP "post_mortem_dump" command (future) * - * Mirrors F:/Projects/SuperMarioWorldRecomp/src/post_mortem.c. The file + * Mirrors the sibling SuperMarioWorldRecomp project's src/post_mortem.c. The file * is OVERWRITTEN on each dump (last-write-wins, single file per run); * this is not a log per CLAUDE.md §3 — it's a one-shot final state * snapshot for crashes the running TCP server cannot intercept. diff --git a/runtime/src/debug_server.c b/runtime/src/debug_server.c index e5f97424..9880a868 100644 --- a/runtime/src/debug_server.c +++ b/runtime/src/debug_server.c @@ -30,6 +30,7 @@ #include "sio.h" #include "memcard.h" #include "spu.h" +#include "audio_trace.h" #include "mdec.h" #include "interrupts.h" #include "psx_cycles.h" @@ -71,6 +72,7 @@ # include <unistd.h> # include <fcntl.h> # include <errno.h> +# include <pthread.h> /* phase_profile sampler thread */ typedef int sock_t; # define SOCK_INVALID (-1) # define sock_close close @@ -3457,6 +3459,10 @@ static const char *thread_kind_name(uint32_t kind) case 20: return "syscall3_enter"; /* ChangeThread/RFE syscall, in_exc==0 (switch-eligible) */ case 24: return "syscall3_enter_in_exc"; /* ChangeThread/RFE syscall, in_exc==1 (forced manual-RFE) */ case 26: return "fiber_dispatch_exit_in_exc"; /* fiber's psx_dispatch returned, in_exc==1 */ + case 30: return "inexc_switch_escape"; /* guest moved PCB[0] inside handler -> scheduler escape */ + case 31: return "inexc_switch_defer"; + case 32: return "deferred_switch_escape"; + case 33: return "deferred_switch_stale"; default: return "unknown"; } } @@ -4672,6 +4678,30 @@ static void handle_irq_state(int id, const char *json) dma_get_dpcr(), dma_get_dicr()); } +/* vblank_rate: report the ONE cycle-paced VBlank authority's raise/deliver + * counts, the (normally-off) GPUSTAT-poll fallback raise count, and the + * per-frame GP0(E5) draw-offset-Y range/count. Used to confirm the guest is + * receiving exactly 60 VBlanks/s (not the ~96/s the stale poll fallback caused) + * and to probe double-buffer draw-offset alternation. */ +static void handle_vblank_rate(int id, const char *json) +{ + (void)json; + extern uint64_t g_vblank_raise_count, g_vblank_deliver_count; + extern uint64_t g_pollhack_vblank_count; + extern int32_t g_doff_min_last, g_doff_max_last; + extern uint32_t g_doff_cnt_last; + send_fmt("{\"id\":%d,\"ok\":true," + "\"cycle_paced_raise\":%llu," + "\"delivered\":%llu," + "\"pollhack_raise\":%llu," + "\"doff_min\":%d,\"doff_max\":%d,\"doff_cnt\":%u}", + id, + (unsigned long long)g_vblank_raise_count, + (unsigned long long)g_vblank_deliver_count, + (unsigned long long)g_pollhack_vblank_count, + g_doff_min_last, g_doff_max_last, g_doff_cnt_last); +} + static void handle_timers_state(int id, const char *json) { (void)json; @@ -5309,8 +5339,8 @@ static void handle_gpu_frame_dump(int id, const char *json) int n = gpu_gp0_ring_dump_frame((uint32_t)target, entries, max_entries); - /* ~190 bytes per entry in JSON; budget conservatively. */ - size_t buf_sz = 256 + (size_t)n * 280u; + /* ~190 bytes base + up to ~90 for the copy builder chain; budget conservatively. */ + size_t buf_sz = 256 + (size_t)n * 400u; char *buf = (char *)malloc(buf_sz); if (!buf) { free(entries); send_err(id, "alloc failed"); return; } @@ -5322,16 +5352,26 @@ static void handle_gpu_frame_dump(int id, const char *json) const GpuGp0RingEntry *e = &entries[i]; pos += (size_t)snprintf(buf + pos, buf_sz - pos, "%s{\"seq\":%u,\"op\":\"0x%02X\",\"n\":%u," - "\"src\":\"0x%08X\",\"pc\":\"0x%08X\",\"w\":[", + "\"src\":\"0x%08X\",\"pc\":\"0x%08X\"," + "\"func\":\"0x%08X\",\"ra\":\"0x%08X\",\"w\":[", i ? "," : "", e->seq, e->opcode, e->n_words, - e->src_addr, e->pc); + e->src_addr, e->pc, e->func, e->ra); int show = e->n_words < GPU_GP0_RING_MAX_WORDS ? e->n_words : GPU_GP0_RING_MAX_WORDS; for (int k = 0; k < show && pos < buf_sz - 32; k++) { pos += (size_t)snprintf(buf + pos, buf_sz - pos, "%s\"0x%08X\"", k ? "," : "", e->cmd[k]); } - pos += (size_t)snprintf(buf + pos, buf_sz - pos, "]}"); + pos += (size_t)snprintf(buf + pos, buf_sz - pos, "]"); + if (e->opcode == 0x80) { + pos += (size_t)snprintf(buf + pos, buf_sz - pos, + ",\"csp\":\"0x%08X\",\"bld\":[", e->csp); + for (int k = 0; k < 6 && e->bld[k] && pos < buf_sz - 32; k++) + pos += (size_t)snprintf(buf + pos, buf_sz - pos, + "%s\"0x%08X\"", k ? "," : "", e->bld[k]); + pos += (size_t)snprintf(buf + pos, buf_sz - pos, "]"); + } + pos += (size_t)snprintf(buf + pos, buf_sz - pos, "}"); } snprintf(buf + pos, buf_sz - pos, "]}"); debug_server_send_line(buf); @@ -5388,6 +5428,94 @@ static void handle_gte_state(int id, const char *json) send_fmt("%s", buf); } +/* Dump recent GTE RTPS/RTPT projections (inputs + outputs) from the always-on + * GTE ring. {"cmd":"gte_ring_dump","count":N,"newest":1,"frame":F} — frame + * optional (omit or -1 for all). Used to find flattened/degenerate character + * projections and split game-code input bugs from GTE-math bugs. */ +static void handle_gte_ring_dump(int id, const char *json) +{ + extern unsigned long long gte_rtp_ring_total(void); + extern int gte_rtp_ring_dump_json(char *out, int outsz, int max_count, + int newest_first, long frame_filter); + int count = json_get_int(json, "count", 64); + if (count < 1) count = 1; + if (count > 512) count = 512; + int newest = json_get_int(json, "newest", 1) != 0; + long frame = (long)json_get_int(json, "frame", -1); + + size_t BUF_SZ = 256u + (size_t)count * 720u; + char *entries = (char *)malloc(BUF_SZ); + char *reply = (char *)malloc(BUF_SZ + 256u); + if (!entries || !reply) { free(entries); free(reply); send_err(id, "oom"); return; } + int n = gte_rtp_ring_dump_json(entries, (int)BUF_SZ, count, newest, frame); + snprintf(reply, BUF_SZ + 256u, + "{\"id\":%d,\"ok\":true,\"total\":%llu,\"emitted\":%d,\"entries\":[%s]}", + id, gte_rtp_ring_total(), n, entries); + debug_server_send_line(reply); + free(entries); free(reply); +} + +/* INTPL (vertex-lerp) ring: inputs (ir0 blend, in=IR1-3 pose A, fc=pose B) + * and outputs (mac / out=IR1-3 / flag) per op. offset pages through the + * matching entries after the frame filter, so a whole frame is reachable. */ +static void handle_gte_intpl_dump(int id, const char *json) +{ + extern unsigned long long gte_intpl_ring_total(void); + extern int gte_intpl_ring_dump_json(char *out, int outsz, int max_count, + int newest_first, long frame_filter, + int offset); + int count = json_get_int(json, "count", 64); + if (count < 1) count = 1; + if (count > 512) count = 512; + int newest = json_get_int(json, "newest", 1) != 0; + long frame = (long)json_get_int(json, "frame", -1); + int offset = json_get_int(json, "offset", 0); + if (offset < 0) offset = 0; + + size_t BUF_SZ = 256u + (size_t)count * 420u; + char *entries = (char *)malloc(BUF_SZ); + char *reply = (char *)malloc(BUF_SZ + 256u); + if (!entries || !reply) { free(entries); free(reply); send_err(id, "oom"); return; } + int n = gte_intpl_ring_dump_json(entries, (int)BUF_SZ, count, newest, frame, offset); + snprintf(reply, BUF_SZ + 256u, + "{\"id\":%d,\"ok\":true,\"total\":%llu,\"emitted\":%d,\"offset\":%d,\"entries\":[%s]}", + id, gte_intpl_ring_total(), n, offset, entries); + debug_server_send_line(reply); + free(entries); free(reply); +} + +/* Per-frame GTE projection stats (nproj / nsat / nflat) over recent frames — + * shows the alternating flat/normal render pattern. */ +static void handle_gte_frame_stats(int id, const char *json) +{ + extern int gte_fstat_dump_json(char *out, int outsz, int max_frames); + int n = json_get_int(json, "frames", 120); + if (n < 1) n = 1; if (n > 512) n = 512; + size_t BUF = 256u + (size_t)n * 96u; + char *body = (char *)malloc(BUF), *reply = (char *)malloc(BUF + 128u); + if (!body || !reply) { free(body); free(reply); send_err(id, "oom"); return; } + int emitted = gte_fstat_dump_json(body, (int)BUF, n); + snprintf(reply, BUF + 128u, "{\"id\":%d,\"ok\":true,\"emitted\":%d,\"frames\":[%s]}", + id, emitted, body); + debug_server_send_line(reply); free(body); free(reply); +} + +/* Latched degenerate (saturated-output) GTE projections with full inputs. */ +static void handle_gte_latch_dump(int id, const char *json) +{ + extern unsigned long long gte_latch_total(void); + extern int gte_latch_dump_json(char *out, int outsz, int max_count); + int n = json_get_int(json, "count", 64); + if (n < 1) n = 1; if (n > 256) n = 256; + size_t BUF = 256u + (size_t)n * 720u; + char *body = (char *)malloc(BUF), *reply = (char *)malloc(BUF + 128u); + if (!body || !reply) { free(body); free(reply); send_err(id, "oom"); return; } + int emitted = gte_latch_dump_json(body, (int)BUF, n); + snprintf(reply, BUF + 128u, "{\"id\":%d,\"ok\":true,\"latch_total\":%llu,\"emitted\":%d,\"entries\":[%s]}", + id, gte_latch_total(), emitted, body); + debug_server_send_line(reply); free(body); free(reply); +} + static void handle_sio_state(int id, const char *json) { (void)json; @@ -5641,6 +5769,146 @@ static void handle_spu_events_reset(int id, const char *json) send_fmt("{\"id\":%d,\"ok\":true}\n", id); } +/* ---- Always-on audio tap rings (audio_trace.c) ------------------------- + * audio_stats — counters: per-tap produced/nonzero/audible/peak + pump + * behavior (skips, underruns, queue watermarks). + * audio_wav — dump a tap's PCM ring slice to a 44100 Hz s16 WAV on the + * server side: {"tap":0,"path":"...","start":-1,"count":0}. + * audio_events — most recent N pipeline events (REG_WRITE/RENDER/SKIP/ + * UNDERRUN/MUTE/UNMUTE/CD_PUSH/DMA), sample-clock stamped. + * Protocol mirrored on psx-beetle's port 4380 (beetle_debug_server.c). */ +/* Bridge/legacy output health (main.cpp; C linkage). Returns 0 pre-device. */ +extern int psx_audio_out_stats(double *fill_ms, uint64_t *underruns, + uint64_t *overflow_drops, double *correction, + int *legacy, int *host_rate); + +static void handle_audio_stats(int id, const char *json) +{ + (void)json; + AudioTraceStats st; + audio_trace_get_stats(&st); + double fill_ms = 0.0, correction = 0.0; + uint64_t out_underruns = 0, overflow_drops = 0; + int legacy = 1, host_rate = 44100; + int out_ok = psx_audio_out_stats(&fill_ms, &out_underruns, &overflow_drops, + &correction, &legacy, &host_rate); + send_fmt("{\"id\":%d,\"ok\":true," + "\"taps\":[" + "{\"name\":\"spu_out\",\"frames\":%llu,\"nonzero\":%llu," + "\"audible\":%llu,\"peak\":%d,\"rate\":%u}," + "{\"name\":\"cd_in\",\"frames\":%llu,\"nonzero\":%llu," + "\"audible\":%llu,\"peak\":%d,\"rate\":%u}," + "{\"name\":\"host\",\"frames\":%llu,\"nonzero\":%llu," + "\"audible\":%llu,\"peak\":%d,\"rate\":%u}]," + "\"pump_calls\":%llu,\"pump_skips\":%llu,\"underruns\":%llu," + "\"queue_hiwater\":%u,\"queue_lowater\":%u," + "\"mutes\":%llu,\"unmutes\":%llu,\"events_total\":%llu," + "\"out\":{\"active\":%d,\"mode\":\"%s\",\"host_rate\":%d," + "\"fill_ms\":%.1f,\"underruns\":%llu,\"overflow_drops\":%llu," + "\"correction\":%.5f}}", + id, + (unsigned long long)st.tap_frames[0], + (unsigned long long)st.tap_nonzero[0], + (unsigned long long)st.tap_audible[0], st.tap_peak[0], + audio_trace_tap_rate(0), + (unsigned long long)st.tap_frames[1], + (unsigned long long)st.tap_nonzero[1], + (unsigned long long)st.tap_audible[1], st.tap_peak[1], + audio_trace_tap_rate(1), + (unsigned long long)st.tap_frames[2], + (unsigned long long)st.tap_nonzero[2], + (unsigned long long)st.tap_audible[2], st.tap_peak[2], + audio_trace_tap_rate(2), + (unsigned long long)st.pump_calls, + (unsigned long long)st.pump_skips, + (unsigned long long)st.underruns, + st.queue_hiwater, st.queue_lowater, + (unsigned long long)st.mute_events, + (unsigned long long)st.unmute_events, + (unsigned long long)st.events_total, + out_ok, legacy ? "legacy-push" : "bridge-pull", host_rate, + fill_ms, + (unsigned long long)out_underruns, + (unsigned long long)overflow_drops, + correction); +} + +static void handle_audio_wav(int id, const char *json) +{ + char path[512]; + if (!json_get_str(json, "path", path, sizeof(path))) { + send_fmt("{\"id\":%d,\"ok\":false,\"err\":\"missing path\"}", id); + return; + } + int tap = json_get_int(json, "tap", AUDIO_TAP_SPU_OUT); + /* start/count as strings so 64-bit sample indices survive. */ + char buf[32]; + int64_t start = -1; + uint64_t count = 0; + if (json_get_str(json, "start", buf, sizeof(buf))) + start = strtoll(buf, NULL, 0); + else + start = (int64_t)json_get_int(json, "start", -1); + if (json_get_str(json, "count", buf, sizeof(buf))) + count = strtoull(buf, NULL, 0); + else + count = (uint64_t)json_get_int(json, "count", 0); + + int64_t wrote = audio_trace_dump_wav(tap, path, start, count); + if (wrote < 0) { + send_fmt("{\"id\":%d,\"ok\":false,\"err\":\"dump failed (bad tap/path or empty ring)\"}", id); + return; + } + uint64_t total = audio_trace_tap_total(tap); + send_fmt("{\"id\":%d,\"ok\":true,\"tap\":%d,\"frames\":%lld," + "\"rate\":%u,\"tap_total\":%llu}", + id, tap, (long long)wrote, audio_trace_tap_rate(tap), + (unsigned long long)total); +} + +static void handle_audio_events(int id, const char *json) +{ + int count = json_get_int(json, "count", 256); + if (count < 1) count = 1; + if (count > 8192) count = 8192; + AudioTraceEvent *evs = + (AudioTraceEvent *)malloc((size_t)count * sizeof(AudioTraceEvent)); + if (!evs) { send_fmt("{\"id\":%d,\"ok\":false,\"err\":\"alloc\"}", id); return; } + uint32_t got = audio_trace_events_get(evs, (uint32_t)count); + uint64_t total = audio_trace_events_total(); + static const char *kind_names[] = { + "?", "REG", "RENDER", "SKIP", "UNDERRUN", + "MUTE", "UNMUTE", "CD_PUSH", "DMA" + }; + + size_t cap = 256u + (size_t)got * 192u; + char *out = (char *)malloc(cap); + if (!out) { free(evs); send_fmt("{\"id\":%d,\"ok\":false,\"err\":\"alloc\"}", id); return; } + size_t off = 0; + int n = snprintf(out + off, cap - off, + "{\"id\":%d,\"ok\":true,\"total\":%llu,\"count\":%u,\"events\":[", + id, (unsigned long long)total, (unsigned)got); + if (n > 0) off += (size_t)n; + for (uint32_t i = 0; i < got; i++) { + const AudioTraceEvent *e = &evs[i]; + const char *kn = (e->kind < sizeof(kind_names) / sizeof(kind_names[0])) + ? kind_names[e->kind] : "?"; + n = snprintf(out + off, cap - off, + "%s{\"seq\":%llu,\"smp\":%llu,\"frame\":%u," + "\"kind\":\"%s\",\"a\":\"0x%X\",\"b\":\"0x%X\"}", + i == 0 ? "" : ",", + (unsigned long long)e->seq, + (unsigned long long)e->sample_idx, + e->frame, kn, e->a, e->b); + if (n > 0) off += (size_t)n; + } + n = snprintf(out + off, cap - off, "]}"); + if (n > 0) off += (size_t)n; + send_fmt("%s", out); + free(out); + free(evs); +} + /* ---- SIO IRQ-arm audit ----------------------------------------------- * Reports counts of TX writes that reached the IRQ-arm decision in * sio_write SIO_TX_DATA, partitioned by active_device. Tells us at the @@ -6503,6 +6771,19 @@ static void handle_gl_ws_ablate(int id, const char *json) send_fmt("{\"id\":%d,\"ok\":true,\"mode\":%d}", id, gl_renderer_get_ws_ablate()); } +/* gl_wide_fast on=<0|1>: native-wide centre-blit fast path. 1 (default) = skip + * the redundant centre mirror and copy the canonical 4:3 frame into the wide + * surface centre at present (fast). 0 = re-rasterize the whole wide surface + * every prim (the original full-mirror path). For A/B perf + parity checks. */ +extern void gl_renderer_set_wide_fast(int on); +extern int gl_renderer_get_wide_fast(void); +static void handle_gl_wide_fast(int id, const char *json) +{ + int on = json_get_int(json, "on", -1); + if (on >= 0) gl_renderer_set_wide_fast(on); + send_fmt("{\"id\":%d,\"ok\":true,\"on\":%d}", id, gl_renderer_get_wide_fast()); +} + /* Live GTE widescreen-squash toggle (diagnostic for 8C far-backdrop void): * ws_aspect num=<n> den=<d> calls gte_set_display_aspect at runtime so we can * compare squash ON (e.g. 16/9) vs OFF (1/1) in-place without a relaunch. */ @@ -6630,6 +6911,48 @@ static void handle_ws_far_threshold(int id, const char *json) id, gte_ws_get_far_threshold(), mn, mx, n, far_n); } +/* ws_dome on=<0|1> [num=<W> den=<H>]: native-wide sky-DOME expand. Scales far- + * depth (SZ >= ws_far_threshold) GTE X outward from the projection centre by + * (3*num)/(4*den) so a finite sky dome grows to fill the wider FOV. Tune the + * depth split with ws_far_threshold (its sz_far count = verts being expanded). */ +extern void gte_ws_set_dome_expand(int on, int aspect_num, int aspect_den); +static void handle_ws_dome(int id, const char *json) +{ + int on = json_get_int(json, "on", 1); + int num = json_get_int(json, "num", 16); + int den = json_get_int(json, "den", 9); + gte_ws_set_dome_expand(on, num, den); + send_fmt("{\"id\":%d,\"ok\":true,\"on\":%d,\"num\":%d,\"den\":%d}", id, on ? 1 : 0, num, den); +} + +/* ws_dome_probe on=<0|1> [thr=<SZ>] | dump: tally which guest function projects + * far (SZ>=thr) vertices -> identifies the sky-dome draw fn (top far emitter, + * highest max_sz) for the per-function dome-expand bracket. Arm with on=1, let a + * dome frame render, then call with no args to dump the tally. */ +extern void gte_dome_probe(int on, int thr); +extern int gte_dome_probe_dump(uint32_t* funcs, uint32_t* counts, int32_t* maxsz, int cap); +static void handle_ws_dome_probe(int id, const char *json) +{ + int on = json_get_int(json, "on", -1); + int thr = json_get_int(json, "thr", -1); + if (on >= 0) gte_dome_probe(on, thr); + uint32_t funcs[48], counts[48]; int32_t maxsz[48]; + int n = gte_dome_probe_dump(funcs, counts, maxsz, 48); + /* simple insertion-sort by count desc for readability */ + for (int i = 1; i < n; i++) + for (int j = i; j > 0 && counts[j] > counts[j-1]; j--) { + uint32_t tf=funcs[j];funcs[j]=funcs[j-1];funcs[j-1]=tf; + uint32_t tc=counts[j];counts[j]=counts[j-1];counts[j-1]=tc; + int32_t tm=maxsz[j];maxsz[j]=maxsz[j-1];maxsz[j-1]=tm; + } + char buf[4096]; int p = snprintf(buf, sizeof buf, "{\"id\":%d,\"ok\":true,\"n\":%d,\"funcs\":[", id, n); + for (int i = 0; i < n && p < (int)sizeof(buf)-80; i++) + p += snprintf(buf+p, sizeof(buf)-p, "%s{\"ra\":\"0x%08X\",\"n\":%u,\"max_sz\":%d}", + i?",":"", funcs[i], counts[i], maxsz[i]); + snprintf(buf+p, sizeof(buf)-p, "]}"); + debug_server_send_line(buf); +} + static void handle_ws_census(int id, const char *json) { char act[16] = {0}; @@ -7085,98 +7408,7 @@ static void handle_get_snapshots(int id, const char *json) * blocks. Bigger on disk than a compressed PNG, but a real PNG that every * viewer (and the harness Read tool) accepts, and nothing new to link against * — which keeps the self-contained static runtime self-contained. */ -static uint32_t s_png_crc_tbl[256]; -static int s_png_crc_init = 0; -static void png_crc_build(void) { - for (uint32_t n = 0; n < 256; n++) { - uint32_t c = n; - for (int k = 0; k < 8; k++) c = (c & 1) ? (0xEDB88320u ^ (c >> 1)) : (c >> 1); - s_png_crc_tbl[n] = c; - } - s_png_crc_init = 1; -} -static uint32_t png_crc_update(uint32_t crc, const uint8_t *p, size_t n) { - if (!s_png_crc_init) png_crc_build(); - for (size_t i = 0; i < n; i++) crc = s_png_crc_tbl[(crc ^ p[i]) & 0xFF] ^ (crc >> 8); - return crc; -} -static uint32_t png_adler32(const uint8_t *p, size_t n) { - uint32_t a = 1, b = 0; - /* process in blocks so the 16-bit sums never overflow before the mod */ - while (n) { - size_t k = n < 5552 ? n : 5552; - for (size_t i = 0; i < k; i++) { a += p[i]; b += a; } - a %= 65521; b %= 65521; p += k; n -= k; - } - return (b << 16) | a; -} -static void png_put_be32(FILE *f, uint32_t v) { - uint8_t b[4] = { (uint8_t)(v >> 24), (uint8_t)(v >> 16), - (uint8_t)(v >> 8), (uint8_t)v }; - fwrite(b, 1, 4, f); -} -static void png_chunk(FILE *f, const char *type, const uint8_t *data, size_t len) { - png_put_be32(f, (uint32_t)len); - uint32_t crc = 0xFFFFFFFFu; - crc = png_crc_update(crc, (const uint8_t *)type, 4); - crc = png_crc_update(crc, data, len); - fwrite(type, 1, 4, f); - if (len) fwrite(data, 1, len, f); - png_put_be32(f, crc ^ 0xFFFFFFFFu); -} -/* Write an RGB (3 bytes/pixel, top-down) buffer as a PNG. Returns 1 on success. */ -static int png_write_rgb(FILE *f, const uint8_t *rgb, uint32_t w, uint32_t h) { - static const uint8_t sig[8] = { 137,80,78,71,13,10,26,10 }; - fwrite(sig, 1, 8, f); - - uint8_t ihdr[13]; - ihdr[0]=(uint8_t)(w>>24); ihdr[1]=(uint8_t)(w>>16); ihdr[2]=(uint8_t)(w>>8); ihdr[3]=(uint8_t)w; - ihdr[4]=(uint8_t)(h>>24); ihdr[5]=(uint8_t)(h>>16); ihdr[6]=(uint8_t)(h>>8); ihdr[7]=(uint8_t)h; - ihdr[8]=8; /* bit depth */ - ihdr[9]=2; /* color type 2 = truecolor RGB */ - ihdr[10]=0; /* compression */ - ihdr[11]=0; /* filter */ - ihdr[12]=0; /* interlace */ - png_chunk(f, "IHDR", ihdr, sizeof ihdr); - - /* Filtered raw scanlines: each row prefixed with filter byte 0 (None). */ - size_t raw_len = (size_t)h * (1 + (size_t)w * 3); - uint8_t *raw = (uint8_t *)malloc(raw_len); - if (!raw) return 0; - for (uint32_t y = 0; y < h; y++) { - uint8_t *row = raw + (size_t)y * (1 + (size_t)w * 3); - row[0] = 0; - memcpy(row + 1, rgb + (size_t)y * w * 3, (size_t)w * 3); - } - - /* zlib stream: 2-byte header + stored DEFLATE blocks + 4-byte Adler32. */ - size_t nblocks = (raw_len + 65534) / 65535; if (nblocks == 0) nblocks = 1; - size_t z_len = 2 + nblocks * 5 + raw_len + 4; - uint8_t *z = (uint8_t *)malloc(z_len); - if (!z) { free(raw); return 0; } - size_t zi = 0; - z[zi++] = 0x78; /* CMF: 32K window, deflate */ - z[zi++] = 0x01; /* FLG: check bits make 0x7801 a multiple of 31 */ - size_t off = 0; - while (off < raw_len) { - size_t n = raw_len - off; if (n > 65535) n = 65535; - int final = (off + n >= raw_len); - z[zi++] = (uint8_t)(final ? 1 : 0); /* BFINAL | BTYPE=00 */ - z[zi++] = (uint8_t)(n & 0xFF); z[zi++] = (uint8_t)(n >> 8); - uint16_t nl = (uint16_t)~n; - z[zi++] = (uint8_t)(nl & 0xFF); z[zi++] = (uint8_t)(nl >> 8); - memcpy(z + zi, raw + off, n); zi += n; off += n; - } - uint32_t ad = png_adler32(raw, raw_len); - z[zi++] = (uint8_t)(ad >> 24); z[zi++] = (uint8_t)(ad >> 16); - z[zi++] = (uint8_t)(ad >> 8); z[zi++] = (uint8_t)ad; - free(raw); - - png_chunk(f, "IDAT", z, zi); - free(z); - png_chunk(f, "IEND", NULL, 0); - return 1; -} +#include "png_write.h" /* png_write_rgb + zlib/CRC helpers (shared with Beetle) */ /* Unified screenshot: writes an 8-bit RGB PNG of the current display to "path" * (default psx_screenshot.png in the runtime cwd) and answers with a single @@ -7184,6 +7416,158 @@ static int png_write_rgb(FILE *f, const uint8_t *rgb, uint32_t w, uint32_t h) { * the old "screenshot" inline-hex-row variant streamed h+1 response lines * per request, which violated the one-request/one-response protocol and * poisoned every client connection that used it. */ +/* ---- Always-on display ring ---------------------------------------------- + * The last DISP_RING_CAP vblanks' display areas, raw 15-bit VRAM halfwords, + * captured in debug_server_record_frame (ring-buffer rule: continuous + * capture, observers query a window after the fact). Purpose: FRAME-EXACT + * screenshots. `screenshot_file` returns whatever is displayed when the + * command lands — 1-3 frames after the `frame` query that named it — which + * makes cross-renderer same-frame diffing impossible. With this ring, two + * deterministic runs (GL vs software) can each serve the display for the + * SAME frame number, giving a pixel-exact renderer-divergence census. + * GL reads GPU-side truth via the fbo peek; software reads CPU VRAM (its + * authoritative surface). depth24 scanout frames are tagged and refused + * (15-bit only). */ +#define DISP_RING_CAP 64 /* full-VRAM entries are 1MB each; 32 frames of + * lookback (~0.3-0.5s) for exact-frame forensics */ +#define DISP_RING_MAX_W 640 +#define DISP_RING_MAX_H 256 +/* Full-VRAM aux capture alongside the display: the entire 1024x512 raw VRAM + * as the renderer sees it at this frame (GL: FBO truth; SW: CPU truth), so an + * offline client can micro-cosim ANY prim of the frame — rasterize it from + * the GP0 ring against the exact same-frame texels/CLUTs — and attribute a + * bad pixel to sampling vs content. Streamed texture pages and palette + * cycling make later peeks worthless; only a same-frame capture is evidence. */ +typedef struct { + uint32_t frame; + uint16_t w, h; + uint8_t depth24, valid; + uint16_t *px; /* DISP_RING_MAX_W*DISP_RING_MAX_H halfwords */ + uint16_t *vram; /* full 1024x512 */ +} DispRingEntry; +static DispRingEntry s_disp_ring[DISP_RING_CAP]; +static uint16_t *s_disp_ring_px = NULL; /* one block for all entries */ + +static void disp_ring_capture(void) +{ + if (!s_disp_ring_px) { + size_t per = (size_t)DISP_RING_MAX_W * DISP_RING_MAX_H; + size_t vram = (size_t)1024 * 512; + s_disp_ring_px = (uint16_t *)malloc(DISP_RING_CAP * (per + vram) * sizeof(uint16_t)); + if (!s_disp_ring_px) return; + for (int i = 0; i < DISP_RING_CAP; i++) { + uint16_t *base = s_disp_ring_px + (size_t)i * (per + vram); + s_disp_ring[i].px = base; + s_disp_ring[i].vram = base + per; + } + } + DispRingEntry *e = &s_disp_ring[(uint32_t)(s_frame_count % DISP_RING_CAP)]; + e->frame = (uint32_t)s_frame_count; + e->valid = 0; + GpuDisplayInfo di; + gpu_get_display_info(&di); + if (di.disabled || di.width == 0 || di.height == 0) return; + uint32_t w = di.width, h = di.height; + if (w > DISP_RING_MAX_W) w = DISP_RING_MAX_W; + if (h > DISP_RING_MAX_H) h = DISP_RING_MAX_H; + e->w = (uint16_t)w; e->h = (uint16_t)h; + e->depth24 = (uint8_t)(di.depth24 ? 1 : 0); + /* GPU-side truth when the GL raster pipeline is live (scissored pack of + * just the display rect + one small readback); CPU VRAM otherwise (the + * software backend's authoritative surface). Runs on the present thread, + * where the GL context is current. */ + extern int gl_renderer_fbo_peek(int x, int y, int w_, int h_, uint16_t *out); + int got = 0; + if (di.display_x + w <= 1024 && di.display_y + h <= 512) + got = gl_renderer_fbo_peek((int)di.display_x, (int)di.display_y, + (int)w, (int)h, e->px); + if (!got) { + for (uint32_t y = 0; y < h; y++) + for (uint32_t x = 0; x < w; x++) + e->px[y * w + x] = + gpu_vram_peek((int)(di.display_x + x), (int)(di.display_y + y)); + } + /* Full-VRAM aux capture (same GL-truth/CPU-truth split as the display). */ + if (!gl_renderer_fbo_peek(0, 0, 1024, 512, e->vram)) { + const uint16_t *v = gpu_get_vram(); + memcpy(e->vram, v, (size_t)1024 * 512 * sizeof(uint16_t)); + } + e->valid = 1; +} + +/* Dump an entry's full-VRAM capture as a raw little-endian u16 blob + * (1024x512 row-major). */ +static void handle_display_ring_aux(int id, const char *json) +{ + int f = json_get_int(json, "frame", -1); + if (f < 0) { send_err(id, "missing frame"); return; } + char path[512]; + if (!json_get_str(json, "path", path, sizeof(path))) { + send_err(id, "missing path"); return; + } + if (!s_disp_ring_px) { send_err(id, "display ring not started"); return; } + DispRingEntry *e = &s_disp_ring[(uint32_t)((uint64_t)f % DISP_RING_CAP)]; + if (!e->valid || e->frame != (uint32_t)f) { + send_err(id, "frame not in display ring"); return; + } + FILE *fp = fopen(path, "wb"); + if (!fp) { send_err(id, "cannot open file"); return; } + size_t n1 = fwrite(e->vram, sizeof(uint16_t), (size_t)1024 * 512, fp); + fclose(fp); + send_fmt("{\"id\":%d,\"ok\":true,\"frame\":%d,\"path\":\"%s\"," + "\"vram_words\":%u}", id, f, path, (unsigned)n1); +} + +static void handle_display_ring_stats(int id, const char *json) +{ + (void)json; + uint32_t oldest = 0, newest = 0; + int n = 0; + for (int i = 0; i < DISP_RING_CAP; i++) { + if (!s_disp_ring_px || !s_disp_ring[i].valid) continue; + uint32_t f = s_disp_ring[i].frame; + if (n == 0 || f < oldest) oldest = f; + if (n == 0 || f > newest) newest = f; + n++; + } + send_fmt("{\"id\":%d,\"ok\":true,\"capacity\":%d,\"valid\":%d," + "\"oldest_frame\":%u,\"newest_frame\":%u}", + id, DISP_RING_CAP, n, oldest, newest); +} + +static void handle_display_ring_get(int id, const char *json) +{ + int f = json_get_int(json, "frame", -1); + if (f < 0) { send_err(id, "missing frame"); return; } + char path[512]; + if (!json_get_str(json, "path", path, sizeof(path))) { + send_err(id, "missing path"); return; + } + if (!s_disp_ring_px) { send_err(id, "display ring not started"); return; } + DispRingEntry *e = &s_disp_ring[(uint32_t)((uint64_t)f % DISP_RING_CAP)]; + if (!e->valid || e->frame != (uint32_t)f) { + send_err(id, "frame not in display ring"); return; + } + if (e->depth24) { send_err(id, "frame is 24bpp scanout (unsupported)"); return; } + uint32_t w = e->w, h = e->h; + uint8_t *rgb = (uint8_t *)malloc((size_t)w * h * 3); + if (!rgb) { send_err(id, "alloc failed"); return; } + for (uint32_t i = 0; i < w * h; i++) { + uint16_t p = e->px[i]; + rgb[i * 3 + 0] = (uint8_t)((p & 0x1F) << 3); + rgb[i * 3 + 1] = (uint8_t)(((p >> 5) & 0x1F) << 3); + rgb[i * 3 + 2] = (uint8_t)(((p >> 10) & 0x1F) << 3); + } + FILE *fp = fopen(path, "wb"); + if (!fp) { free(rgb); send_err(id, "cannot open file"); return; } + int ok = png_write_rgb(fp, rgb, w, h); + free(rgb); + fclose(fp); + if (!ok) { send_err(id, "png encode failed"); return; } + send_fmt("{\"id\":%d,\"ok\":true,\"frame\":%d,\"path\":\"%s\"," + "\"width\":%u,\"height\":%u}", id, f, path, w, h); +} + static void handle_screenshot_file(int id, const char *json) { /* Under the OpenGL FBO-present path, CPU VRAM can be stale (the FBO holds @@ -7234,6 +7618,45 @@ static void handle_screenshot_file(int id, const char *json) id, path, w, h); } +/* dump_buffer: dump a raw 512x240 VRAM region starting at display Y = `y` to a + * PNG, regardless of what the game currently displays. Used to inspect BOTH + * double-buffer halves (y=0 and y=256) coherently in one call to see whether a + * strobing character is present in one buffer only. */ +static void handle_dump_buffer(int id, const char *json) +{ + extern void gl_renderer_sync_cpu(void); + gl_renderer_sync_cpu(); + extern void vk_renderer_sync_cpu(void); + vk_renderer_sync_cpu(); + + int y0 = json_get_int(json, "y", 0); + char path[512]; + if (!json_get_str(json, "path", path, sizeof(path))) + strncpy(path, "psx_buffer.png", sizeof(path) - 1); + path[sizeof(path) - 1] = '\0'; + + GpuDisplayInfo di; + di.display_x = 0; di.display_y = (uint32_t)y0; di.depth24 = 0; + di.disabled = 0; di.width = 512; di.height = 240; + + uint32_t w = 512, h = 240; + FILE *f = fopen(path, "wb"); + if (!f) { send_err(id, "cannot open file"); return; } + uint8_t *rgb = (uint8_t *)malloc((size_t)w * h * 3); + if (!rgb) { fclose(f); send_err(id, "alloc failed"); return; } + for (uint32_t y = 0; y < h; y++) + for (uint32_t x = 0; x < w; x++) { + uint8_t r, g, b; + gpu_display_pixel_rgb(&di, x, y, &r, &g, &b); + uint8_t *p = rgb + ((size_t)y * w + x) * 3; + p[0] = r; p[1] = g; p[2] = b; + } + int ok = png_write_rgb(f, rgb, w, h); + free(rgb); fclose(f); + if (!ok) { send_err(id, "png encode failed"); return; } + send_fmt("{\"id\":%d,\"ok\":true,\"path\":\"%s\",\"y\":%d}", id, path, y0); +} + /* wide_shot: capture the NATIVE-WIDE present surface (post-compositor, what the * window actually shows in 16:9) to a PNG — unlike screenshot_file, which dumps * the canonical 320 VRAM region (pre-compositor). Pulls the exact buffer the @@ -7302,7 +7725,12 @@ static void handle_wide_shot(int id, const char *json) * to 0 (the common vertical-double-buffer origin). */ static void handle_wide_full(int id, const char *json) { - extern int sw_wide_dump_full(uint32_t *out, int cap_pixels, int *ow, int *oh, int base_x); + /* Backend-dispatched: gr_wide_dump_full routes to the active renderer's full + * wide-surface dump (SW: sw_wide_dump_full; GL: glb_wide_dump_full), so the + * whole compositor surface can be inspected on either backend over TCP. */ + extern int gr_wide_dump_full(uint32_t *out, int cap_pixels, int *ow, int *oh, int base_x); + extern void gl_renderer_sync_cpu(void); + gl_renderer_sync_cpu(); /* no-op on SW; ensures pending GL frame is flushed */ int base_x = json_get_int(json, "base_x", 0); char path[512]; if (!json_get_str(json, "path", path, sizeof(path))) @@ -7313,7 +7741,7 @@ static void handle_wide_full(int id, const char *json) uint32_t *buf = (uint32_t *)malloc((size_t)cap * sizeof(uint32_t)); if (!buf) { send_err(id, "alloc failed"); return; } int W = 0, H = 0; - int n = sw_wide_dump_full(buf, cap, &W, &H, base_x); + int n = gr_wide_dump_full(buf, cap, &W, &H, base_x); if (n <= 0) { free(buf); send_err(id, "no wide surface (SW backend? native-wide engaged?)"); return; } uint8_t *rgb = (uint8_t *)malloc((size_t)W * H * 3); @@ -8058,38 +8486,56 @@ static void handle_d44_ring(int id, const char *json) * CD DMA was mid-transfer (the VSync-in-DMA-window bug). */ static void handle_irqctx_ring(int id, const char *json) { - (void)json; typedef struct { uint64_t seq, cycle; uint32_t frame, istat, imask, sr, d44, cdrom_active, is_vblank; int dma_depth; uint32_t take_pc, real_epc, exit_pc, exit_reason, same_thread, - restored, v1_exit, v1_saved, ra_exit, ra_saved, redirects; } E; + restored, v1_exit, v1_saved, ra_exit, ra_saved, redirects, + entry_sp, pump_site; } E; extern E g_irqctx_ring[]; extern uint64_t g_irqctx_seq; - uint64_t total = g_irqctx_seq; uint32_t cap = 64u; - uint32_t n = total < cap ? (uint32_t)total : cap; - static char buf[49152]; size_t pos = 0; - pos += snprintf(buf + pos, sizeof(buf) - pos, + /* Ring cap must track IRQCTX_RING_CAP in interrupts.c. */ + uint32_t cap = 4096u; + /* Optional frame-window filter + newest-N count so a deep ring can be + * windowed to the delivery window of interest (ring-buffer discipline). */ + int frame_lo = json_get_int(json, "frame_lo", -1); + int frame_hi = json_get_int(json, "frame_hi", -1); + int count = json_get_int(json, "count", 128); + if (count < 1) count = 1; + if (count > (int)cap) count = (int)cap; + uint64_t total = g_irqctx_seq; + uint32_t avail = total < cap ? (uint32_t)total : cap; + uint32_t n = (uint32_t)count < avail ? (uint32_t)count : avail; + size_t BUF_SZ = 512u + (size_t)n * 410u; + char *buf = (char *)malloc(BUF_SZ); if (!buf) { send_err(id, "oom"); return; } + size_t pos = 0; + pos += snprintf(buf + pos, BUF_SZ - pos, "{\"id\":%d,\"ok\":true,\"total\":%llu,\"entries\":[", id, (unsigned long long)total); - for (uint32_t i = 0; i < n && pos < sizeof(buf) - 512; i++) { + int emitted = 0; + for (uint32_t i = 0; i < n && pos < BUF_SZ - 512; i++) { uint64_t idx = total - n + i; E *e = &g_irqctx_ring[idx & (cap - 1u)]; - pos += snprintf(buf + pos, sizeof(buf) - pos, + if (frame_lo >= 0 && (int)e->frame < frame_lo) continue; + if (frame_hi >= 0 && (int)e->frame > frame_hi) continue; + pos += snprintf(buf + pos, BUF_SZ - pos, "%s{\"seq\":%llu,\"cycle\":%llu,\"frame\":%u,\"vblank\":%u," "\"d44\":\"0x%08X\",\"cdrom_active\":%u,\"dma_depth\":%d," "\"sr\":\"0x%08X\",\"istat\":\"0x%08X\",\"imask\":\"0x%08X\"," "\"take_pc\":\"0x%08X\",\"real_epc\":\"0x%08X\",\"exit_pc\":\"0x%08X\"," "\"exit_reason\":%u,\"same_thread\":%u,\"restored\":%u," "\"v1_exit\":\"0x%08X\",\"v1_saved\":\"0x%08X\"," - "\"ra_exit\":\"0x%08X\",\"ra_saved\":\"0x%08X\",\"redirects\":%u}", - i ? "," : "", (unsigned long long)e->seq, (unsigned long long)e->cycle, + "\"ra_exit\":\"0x%08X\",\"ra_saved\":\"0x%08X\",\"redirects\":%u," + "\"entry_sp\":\"0x%08X\",\"pump_site\":%u}", + emitted ? "," : "", (unsigned long long)e->seq, (unsigned long long)e->cycle, e->frame, e->is_vblank, e->d44, e->cdrom_active, e->dma_depth, e->sr, e->istat, e->imask, e->take_pc, e->real_epc, e->exit_pc, e->exit_reason, e->same_thread, e->restored, e->v1_exit, e->v1_saved, e->ra_exit, e->ra_saved, - e->redirects); + e->redirects, e->entry_sp, e->pump_site); + emitted++; } - pos += snprintf(buf + pos, sizeof(buf) - pos, "]}"); + pos += snprintf(buf + pos, BUF_SZ - pos, "],\"emitted\":%d}", emitted); debug_server_send_line(buf); + free(buf); } static void handle_freeze_check(int id, const char *json) @@ -9887,16 +10333,25 @@ static void handle_autocompile_status(int id, const char *json) extern void overlay_autocapture_get_status(int *enabled, uint32_t *triggers, uint64_t *last_delta); + extern uint64_t overlay_autocapture_last_insns_delta(void); + extern void overlay_autocapture_get_futility(uint32_t *backoff, + uint32_t *futile); (void)json; int ac_en = 0; uint32_t trig = 0; uint64_t delta = 0; + uint32_t backoff = 0, futile = 0; overlay_autocapture_get_status(&ac_en, &trig, &delta); + overlay_autocapture_get_futility(&backoff, &futile); char comp[4096]; autocompile_status_json(comp, sizeof(comp)); send_fmt("{\"id\":%d,\"ok\":true,\"autocapture_enabled\":%d," - "\"triggers\":%u,\"last_pressure\":%llu,\"compile\":%s}\n", - id, ac_en, trig, (unsigned long long)delta, comp); + "\"triggers\":%u,\"futile_skips\":%u,\"backoff_mult\":%u," + "\"last_pressure\":%llu," + "\"last_insns_pressure\":%llu,\"compile\":%s}\n", + id, ac_en, trig, futile, backoff, + (unsigned long long)delta, + (unsigned long long)overlay_autocapture_last_insns_delta(), comp); } /* sljit_status: Tier-2 in-process JIT backend state. Runs the codegen smoke @@ -10864,6 +11319,21 @@ static void handle_game_options(int id, const char *json) send_fmt("{\"id\":%d,\"ok\":true,\"go\":%s}", id, buf); } +/* On-the-fly string translation (text_xlate.cpp). Always-on capture inventory + + * apply stats/dump, queried over TCP (no log files — Rule 3). sub: stats (def) | + * dump | todo | reload. */ +static void handle_xlate(int id, const char *json) +{ + extern int text_xlate_debug_json(const char *subcmd, char *out, int cap); + char sub[32] = {0}; + if (!json_get_str(json, "sub", sub, sizeof(sub))) strcpy(sub, "stats"); + static char buf[1 << 20]; /* 1 MB — the inventory dump can be large */ + int n = text_xlate_debug_json(sub, buf, (int)sizeof(buf)); + if (n < 0) n = 0; + if (n < (int)sizeof(buf)) buf[n] = 0; else buf[sizeof(buf) - 1] = 0; + send_fmt("{\"id\":%d,\"ok\":true,\"xlate\":%s}", id, buf); +} + /* Live host-stack-usage profile (RECURSION_BUG.md §17): read the always-on ring * while the game is still responsive to distinguish a gradual cross-frame leak * (used_kb climbs linearly with frame) from a within-one-frame runaway. */ @@ -10980,10 +11450,226 @@ static void handle_lockstep_func(int id, const char *json) { send_fmt("{\"id\":%d,\"ok\":true,\"lockstep_func\":%s}", id, buf); } +/* ---- phase_profile: statistical wall-time attribution (always-on) --------- + * A dedicated ~1 kHz sampler thread reads the emu thread's phase flags and + * accumulates per-second buckets in a ring. It answers "what fraction of wall + * time is the dirty-RAM interpreter / guest exception context" DIRECTLY, + * without inference from dispatch or instruction counts. Two properties are + * load-bearing: + * - It runs OFF the main thread. The TCP server itself pumps on the main + * thread, so a handler-side busy sampler would freeze the very thing it + * measures. + * - It samples g_exec_phase (dirty_ram_interp.c), the INNERMOST backend + * executing at that instant — NOT g_dirty_interp_active, which stays 1 + * across native overlay calls made from the interpreter's call contract + * and therefore reads "inside the dispatch tree", not "interpreting". + * g_exec_phase is save/restored at every backend boundary and across + * exception delivery, so leaked intervals self-heal at the next bracket. + * SDL init gives the process 1 ms timer resolution, so Sleep(1) really is + * ~1 kHz. Query: {"cmd":"phase_profile","window":N} -> shares over the last + * N whole seconds (default 10, in-progress second excluded) + cumulative. */ +extern int psx_exec_phase(void); /* 0=other 1=interp 2=native 3=static 4=gpu-gp0 */ +extern int psx_get_in_exception(void); +extern uint32_t overlay_loader_native_inprogress(void); + +#define PHASE_RING_SECS 64 +static volatile uint32_t s_phase_total [PHASE_RING_SECS]; +static volatile uint32_t s_phase_interp[PHASE_RING_SECS]; +static volatile uint32_t s_phase_native[PHASE_RING_SECS]; +static volatile uint32_t s_phase_static[PHASE_RING_SECS]; +static volatile uint32_t s_phase_gpu [PHASE_RING_SECS]; +static volatile uint32_t s_phase_exc [PHASE_RING_SECS]; +static volatile uint64_t s_phase_sec [PHASE_RING_SECS]; +static volatile uint64_t s_phase_samples_all = 0, s_phase_interp_all = 0; + +/* Hot-function histogram: when a sample lands in a native overlay shard, + * bucket the shard's registered entry address (cumulative since boot; a + * client diffs two snapshots for a window). Open addressing, sampler-thread + * writes only. */ +#define PHOT_SLOTS 4096 +static volatile uint32_t s_phot_addr[PHOT_SLOTS]; +static volatile uint64_t s_phot_cnt [PHOT_SLOTS]; +static volatile uint64_t s_phot_native_total = 0, s_phot_drops = 0; + +static void phot_add(uint32_t addr) +{ + if (!addr) return; + uint32_t h = (addr >> 2) * 2654435761u; + for (uint32_t p = 0; p < 16; p++) { + uint32_t i = (h + p) & (PHOT_SLOTS - 1u); + uint32_t a = s_phot_addr[i]; + if (a == addr) { s_phot_cnt[i]++; return; } + if (a == 0) { s_phot_addr[i] = addr; s_phot_cnt[i] = 1; return; } + } + s_phot_drops++; +} + +#ifdef _WIN32 +static DWORD WINAPI phase_sampler_main(LPVOID arg) +#else +static void *phase_sampler_main(void *arg) +#endif +{ + (void)arg; + for (;;) { +#ifdef _WIN32 + Sleep(1); +#else + struct timespec ts = {0, 1000000}; + nanosleep(&ts, NULL); +#endif + uint64_t sec = monotonic_ms() / 1000u; + int slot = (int)(sec % PHASE_RING_SECS); + if (s_phase_sec[slot] != sec) { /* bucket rolls to a new second */ + s_phase_sec[slot] = sec; + s_phase_total[slot] = 0; + s_phase_interp[slot] = 0; + s_phase_native[slot] = 0; + s_phase_static[slot] = 0; + s_phase_gpu[slot] = 0; + s_phase_exc[slot] = 0; + } + s_phase_total[slot]++; + s_phase_samples_all++; + switch (psx_exec_phase()) { + case 1: s_phase_interp[slot]++; s_phase_interp_all++; break; + case 2: s_phase_native[slot]++; + s_phot_native_total++; + phot_add(overlay_loader_native_inprogress()); + break; + case 3: s_phase_static[slot]++; break; + case 4: s_phase_gpu[slot]++; break; + default: break; /* 0 = host/other */ + } + if (psx_get_in_exception()) { s_phase_exc[slot]++; } + } +#ifndef _WIN32 + return NULL; +#endif +} + +static void phase_sampler_start(void) +{ +#ifdef _WIN32 + HANDLE h = CreateThread(NULL, 0, phase_sampler_main, NULL, 0, NULL); + if (h) CloseHandle(h); +#else + pthread_t t; + if (pthread_create(&t, NULL, phase_sampler_main, NULL) == 0) + pthread_detach(t); +#endif +} + +static void handle_phase_profile(int id, const char *json) +{ + int window = json_get_int(json, "window", 10); + if (window < 1) window = 1; + if (window > PHASE_RING_SECS - 2) window = PHASE_RING_SECS - 2; + uint64_t now_sec = monotonic_ms() / 1000u; + uint32_t tot = 0, itp = 0, nat = 0, sta = 0, gpu = 0, exc = 0; + for (int k = 1; k <= window; k++) { /* whole seconds only */ + uint64_t sec = now_sec - (uint64_t)k; + int slot = (int)(sec % PHASE_RING_SECS); + if (s_phase_sec[slot] == sec) { + tot += s_phase_total[slot]; + itp += s_phase_interp[slot]; + nat += s_phase_native[slot]; + sta += s_phase_static[slot]; + gpu += s_phase_gpu[slot]; + exc += s_phase_exc[slot]; + } + } + uint32_t oth = tot - itp - nat - sta - gpu; + send_fmt("{\"id\":%d,\"ok\":true,\"window_s\":%d,\"samples\":%u," + "\"interp_samples\":%u,\"interp_share\":%.4f," + "\"native_samples\":%u,\"native_share\":%.4f," + "\"static_samples\":%u,\"static_share\":%.4f," + "\"gpu_samples\":%u,\"gpu_share\":%.4f," + "\"other_samples\":%u,\"other_share\":%.4f," + "\"exc_samples\":%u,\"exc_share\":%.4f," + "\"samples_total\":%llu,\"interp_total\":%llu}", + id, window, tot, + itp, tot ? (double)itp / (double)tot : 0.0, + nat, tot ? (double)nat / (double)tot : 0.0, + sta, tot ? (double)sta / (double)tot : 0.0, + gpu, tot ? (double)gpu / (double)tot : 0.0, + oth, tot ? (double)oth / (double)tot : 0.0, + exc, tot ? (double)exc / (double)tot : 0.0, + (unsigned long long)s_phase_samples_all, + (unsigned long long)s_phase_interp_all); +} + +/* phase_hot: top guest functions by native wall-time samples (cumulative + * since boot — diff two snapshots for a window). {"cmd":"phase_hot","top":N} */ +static void handle_phase_hot(int id, const char *json) +{ + int top = json_get_int(json, "top", 20); + if (top < 1) top = 1; + if (top > 64) top = 64; + uint32_t best_addr[64]; + uint64_t best_cnt[64]; + int n = 0; + for (int i = 0; i < PHOT_SLOTS; i++) { + uint32_t a = s_phot_addr[i]; + if (!a) continue; + uint64_t c = s_phot_cnt[i]; + int j = n < top ? n : top - 1; + if (n < top) n++; + else if (c <= best_cnt[j]) continue; + while (j > 0 && best_cnt[j - 1] < c) { + best_addr[j] = best_addr[j - 1]; + best_cnt[j] = best_cnt[j - 1]; + j--; + } + best_addr[j] = a; + best_cnt[j] = c; + } + uint64_t tot = s_phot_native_total; + char buf[4096]; + int len = snprintf(buf, sizeof(buf), + "{\"id\":%d,\"ok\":true,\"native_samples_total\":%llu," + "\"hash_drops\":%llu,\"top\":[", + id, (unsigned long long)tot, + (unsigned long long)s_phot_drops); + for (int i = 0; i < n && len < (int)sizeof(buf) - 96; i++) { + len += snprintf(buf + len, sizeof(buf) - (size_t)len, + "%s{\"addr\":\"0x%08X\",\"samples\":%llu,\"share\":%.4f}", + i ? "," : "", best_addr[i], + (unsigned long long)best_cnt[i], + tot ? (double)best_cnt[i] / (double)tot : 0.0); + } + len += snprintf(buf + len, sizeof(buf) - (size_t)len, "]}"); + (void)len; + send_fmt("%s", buf); +} + +/* idle_skip: idle-loop cycle-skip status + runtime toggle. + * {"cmd":"idle_skip"} -> counters + * {"cmd":"idle_skip","enable":0|1} -> toggle, then counters */ +static void handle_idle_skip(int id, const char *json) +{ + extern int g_idle_skip_enabled; + extern uint64_t g_idle_skip_count, g_idle_skip_cycles; + extern uint32_t g_idle_skip_last_pc, g_idle_skip_last_quantum; + int en = json_get_int(json, "enable", -1); + if (en == 0 || en == 1) g_idle_skip_enabled = en; + send_fmt("{\"id\":%d,\"ok\":true,\"enabled\":%d," + "\"skips\":%llu,\"cycles_skipped\":%llu," + "\"last_pc\":\"0x%08X\",\"last_quantum\":%u}", + id, g_idle_skip_enabled, + (unsigned long long)g_idle_skip_count, + (unsigned long long)g_idle_skip_cycles, + g_idle_skip_last_pc, g_idle_skip_last_quantum); +} + static const CmdEntry s_commands[] = { + { "phase_profile", handle_phase_profile }, + { "phase_hot", handle_phase_hot }, + { "idle_skip", handle_idle_skip }, { "lockstep", handle_lockstep }, { "lockstep_func", handle_lockstep_func }, { "ping", handle_ping }, + { "xlate", handle_xlate }, { "parity_dump", handle_parity_dump }, { "parity_ctl", handle_parity_ctl }, { "devtrace_dump", handle_devtrace_dump }, @@ -11013,6 +11699,8 @@ static const CmdEntry s_commands[] = { { "ws_backdrop_stretch", handle_ws_backdrop_stretch }, { "ws_dbg_stretch", handle_ws_dbg_stretch }, { "ws_far_threshold", handle_ws_far_threshold }, + { "ws_dome", handle_ws_dome }, + { "ws_dome_probe", handle_ws_dome_probe }, { "ws_census", handle_ws_census }, { "mmx6_freshfix", handle_mmx6_freshfix }, { "mmx6_reveal", handle_mmx6_reveal }, @@ -11022,10 +11710,12 @@ static const CmdEntry s_commands[] = { { "gl_present_ring", handle_gl_present_ring }, { "frame_perf", handle_frame_perf }, { "gl_ws_ablate", handle_gl_ws_ablate }, + { "gl_wide_fast", handle_gl_wide_fast }, { "synth_recurse", handle_synth_recurse }, { "gl_fbo_peek", handle_gl_fbo_peek }, { "gl_vram_diff", handle_gl_vram_diff }, { "irq_state", handle_irq_state }, + { "vblank_rate", handle_vblank_rate }, { "cycles_to_next_event", handle_cycles_to_next_event }, { "timers_state", handle_timers_state }, { "cdrom_state", handle_cdrom_state }, @@ -11046,6 +11736,9 @@ static const CmdEntry s_commands[] = { { "spu_voices", handle_spu_voices }, { "spu_events", handle_spu_events }, { "spu_events_reset", handle_spu_events_reset }, + { "audio_stats", handle_audio_stats }, + { "audio_wav", handle_audio_wav }, + { "audio_events", handle_audio_events }, { "card_buffer_dump", handle_card_buffer_dump }, { "sio_arm_audit", handle_sio_arm_audit }, { "sio_burst_stats", handle_sio_burst_stats }, @@ -11174,6 +11867,10 @@ static const CmdEntry s_commands[] = { { "get_snapshots", handle_get_snapshots }, { "screenshot", handle_screenshot_file }, { "screenshot_file", handle_screenshot_file }, /* alias */ + { "display_ring_get", handle_display_ring_get }, + { "display_ring_aux", handle_display_ring_aux }, + { "display_ring_stats", handle_display_ring_stats }, + { "dump_buffer", handle_dump_buffer }, { "wide_full", handle_wide_full }, { "wide_shot", handle_wide_shot }, { "gpu_opcodes", handle_gpu_opcodes }, @@ -11184,6 +11881,10 @@ static const CmdEntry s_commands[] = { { "capture_quads", handle_capture_quads }, { "get_quads", handle_get_quads }, { "gte_state", handle_gte_state }, + { "gte_ring_dump", handle_gte_ring_dump }, + { "gte_intpl_dump", handle_gte_intpl_dump }, + { "gte_frame_stats", handle_gte_frame_stats }, + { "gte_latch_dump", handle_gte_latch_dump }, { "quit", handle_quit }, { "dispatch_stats", handle_dispatch_stats }, { "dispatch_check", handle_dispatch_check }, @@ -11280,6 +11981,11 @@ static void process_command(const char *line) static CPUState *s_init_cpu = NULL; CPUState *debug_cpu_ptr = NULL; /* Global, used by memory.c watchpoints */ +/* Guest $ra accessor for other TUs (e.g. gpu.c GP0 ring caller capture) that + * must not depend on the CPUState layout. Returns 0 before the CPU is bound. */ +uint32_t debug_guest_ra(void) { return debug_cpu_ptr ? debug_cpu_ptr->gpr[31] : 0; } +uint32_t debug_guest_sp(void) { return debug_cpu_ptr ? debug_cpu_ptr->gpr[29] : 0; } + void debug_server_set_cpu(CPUState *cpu) { s_cpu = cpu; @@ -11346,6 +12052,10 @@ void debug_server_init(int port) * 16 leaves room for a few probes to queue while we investigate. This * is observability infrastructure, not a freeze fix. */ listen(s_listen, 16); + + /* Always-on wall-time phase sampler (phase_profile). Own thread: the + * server pumps on the main thread, so sampling must not live there. */ + phase_sampler_start(); /* Blocking accept — the dedicated I/O thread waits on it (no busy-poll). */ /* Start the TCP I/O thread + its handoff primitives. */ @@ -11497,7 +12207,32 @@ void debug_server_init(int port) s_wtrace_ranges[14].hi = 0x00066BD0u; s_wtrace_ranges[15].lo = 0x00097420u; /* movie/frame handoff state */ s_wtrace_ranges[15].hi = 0x00097430u; - s_wtrace_range_count = 16; + /* Ape LOAD-GAME higher-layer (libcard + game card-manager) always-on + * capture. Oracle diff (2026-07-07 session 3) proved the low-level card + * protocol succeeds on both runtimes, but ours stays in top-scene 1 + * ("Checking") while Beetle advances to scene 4 (file-select). These cells + * are the higher-layer state that diverges; capturing every writer from + * boot (never probe-time armed) attributes WHICH libcard op-state stalls + * and what completion it awaits. Masked-physical (KUSEG) addresses. + * 0x800b4e30 = libcard op struct field 0 (count/op-state; 0 ours vs 1 Beetle) + * 0x800b4e50 = save-name ptr + filename buffer ("bu00:BASCUS-94423SYS" on Beetle, zeros ours) + * 0x800b4ed0 = libcard op-handler ptr (0x80020f4c ours vs 0x80020bc8 Beetle) + * 0x800e3880 = SCENE (no-op flip-flop) + 0x800e3884 = TRIG (= top-scene index; 1 ours vs 4 Beetle) + * 0x8013af50 = game card-menu substate block (0x8013af56 = 0 ours vs 1 Beetle) + * 0x00007264 = kernel card-driver per-state byte M8[0x7264] (handoff: stuck at 1) */ + s_wtrace_ranges[16].lo = 0x000B4E2Cu; + s_wtrace_ranges[16].hi = 0x000B4E40u; + s_wtrace_ranges[17].lo = 0x000B4E50u; + s_wtrace_ranges[17].hi = 0x000B4E68u; + s_wtrace_ranges[18].lo = 0x000B4ECCu; + s_wtrace_ranges[18].hi = 0x000B4ED4u; + s_wtrace_ranges[19].lo = 0x000E3880u; + s_wtrace_ranges[19].hi = 0x000E3888u; + s_wtrace_ranges[20].lo = 0x0013AF50u; + s_wtrace_ranges[20].hi = 0x0013AF60u; + s_wtrace_ranges[21].lo = 0x00007260u; + s_wtrace_ranges[21].hi = 0x00007270u; + s_wtrace_range_count = 22; s_wtrace_boot_ranges[0].lo = 0x00097420u; /* movie/frame handoff state */ s_wtrace_boot_ranges[0].hi = 0x00097430u; @@ -11825,6 +12560,10 @@ void debug_server_record_frame(void) /* Last function */ strcpy(r->last_func, "(no tracking)"); + /* Always-on display ring: raw display-area pixels for THIS frame number + * (pre-increment, matching what the `frame` command reports right now). */ + disp_ring_capture(); + s_history_count = s_frame_count + 1; s_frame_count++; /* Layer-1 first-divergence: snapshot the cumulative write fingerprint for diff --git a/runtime/src/dirty_ram_interp.c b/runtime/src/dirty_ram_interp.c index 4af43f81..24264b01 100644 --- a/runtime/src/dirty_ram_interp.c +++ b/runtime/src/dirty_ram_interp.c @@ -210,6 +210,23 @@ extern uint32_t g_debug_last_store_pc; int g_dirty_interp_active = 0; int dirty_ram_interp_is_active(void) { return g_dirty_interp_active; } +/* Innermost execution phase, for the wall-time sampler (phase_profile). + * g_dirty_interp_active is NOT this: it stays 1 across a native overlay call + * made from the interpreter's jal/jalr contract (overlay_loader_call_native), + * so sampling it reads "inside the dispatch tree", not "interpreting". This + * variable is save/set/restored at EVERY backend boundary, so a sample reads + * the backend actually executing at that instant: + * 0 = host/other (SDL, GPU, top-level dispatch glue, idle) + * 1 = dirty-RAM interpreter (exec_one body / precise slice) + * 2 = native overlay shard (gcc DLL or sljit) + * 3 = compiled static text (game EXE / recompiled BIOS, incl. IRQ handler) + * 4 = GPU GP0 command processing (gpu.c — raster/batch/VRAM-transfer work) + * Longjmp contract mirrors g_dirty_interp_active: exception delivery saves it + * at entry and restores after its setjmp loop (interrupts.c), so a skipped + * inner restore self-heals at the next bracket. */ +int g_exec_phase = 0; +int psx_exec_phase(void) { return g_exec_phase; } + /* TCP-armed instruction-window capture (native↔interp divergence drill). * g_insn_gate_*: an extra runtime-settable PC range that the per-insn log * records (on top of the hardwired kernel ranges). Freeze latch: on the Nth @@ -325,6 +342,34 @@ static int ws_cull_site(uint32_t pc) { return flag; } +/* Widescreen LEFT-edge funnel-bltz classification (auto_screen_x, signed + * min/max + center±halfwidth idioms — ws_cull_detect.h). Same ±512-byte window + * qualification + per-PC cache discipline as ws_cull_site above; additionally + * classifies THIS bltz structurally (delay-slot width compare / addu-subu + * pair), so an unrelated bltz in a qualifying window stays vanilla. */ +static int ws_cull_bltz_site(uint32_t pc) { + enum { WIN = 128 }; /* +/- 128 words = +/- 512 bytes */ + static struct { uint32_t pc; uint32_t gen; uint32_t word; int8_t flag; } cache[WS_SITE_CACHE_SLOTS]; + uint32_t slot = (pc >> 2) & (WS_SITE_CACHE_SLOTS - 1u); + uint32_t phys = pc & 0x1FFFFFFFu; + if (cache[slot].pc == pc && cache[slot].gen == g_dirty_ram_code_gen && + cache[slot].word == fetch_word(phys)) + return cache[slot].flag; + uint32_t lo = (phys > (uint32_t)(WIN * 4)) ? phys - (uint32_t)(WIN * 4) : 0u; + uint32_t hi = phys + (uint32_t)(WIN * 4); + if (hi > 0x200000u) hi = 0x200000u; /* 2 MB main RAM */ + static uint32_t words[2 * WIN + 1]; + int n = 0; + for (uint32_t a = lo; a + 4u <= hi && n < (int)(2 * WIN + 1); a += 4u) + words[n++] = fetch_word(a); + int idx = (int)((phys - lo) / 4u); + int flag = psx_ws_func_has_screen_cull(words, n) && + psx_ws_cull_bltz_at(words, n, idx); + cache[slot].pc = pc; cache[slot].gen = g_dirty_ram_code_gen; + cache[slot].word = fetch_word(phys); cache[slot].flag = (int8_t)flag; + return flag; +} + /* Widescreen far-backdrop column-PRELOAD site classification for the interpreter * ([widescreen.cull] auto_backdrop). The scrolling-backdrop column-window * generators run interpreted in the dev build, so the recompiler emit can't @@ -639,7 +684,12 @@ static int dispatch_nonlocal_call(CPUState *cpu, uint32_t target, uint32_t return_pc, uint32_t *next_pc_out) { cpu->pc = 0; - psx_dispatch_call(cpu, target, return_pc); + { + int prev_phase = g_exec_phase; + g_exec_phase = 3; /* compiled route; dirty/native callees re-tag inside */ + psx_dispatch_call(cpu, target, return_pc); + g_exec_phase = prev_phase; + } /* psx_dispatch_call validated the (return_pc, sp) contract; a bail * unwind in progress surfaces with cpu->pc = the guest's true target. */ if (g_psx_call_bail) return 1; @@ -1010,12 +1060,10 @@ int dirty_ram_xprobe_json(char *out, int cap) { #ifdef PSX_HAS_GAME_DISPATCH static int interp_enter_compiled(CPUState *cpu, uint32_t target) { if (target == 0x8001A954u) site_note(&g_site_interp); - /* Decline if the target page is dirty: an overlay overwrote it after the - * game-start baseline, so the compiled function is STALE. Returning 0 lets the - * JAL/JALR handler fall through to is_local_dirty_target -> local-flow interp of - * the live overlay, instead of running stale compiled code (the same staleness - * the dirty_ram_dispatch_inner gate guards). Clean targets run compiled. */ - if (dirty_ram_is_dirty(target & 0x1FFFFFFFu)) return 0; + /* Decline when the target page no longer matches the static game image. + * Returning 0 lets the JAL/JALR handler fall through to local-flow interp + * of the live RAM bytes instead of running stale compiled code. */ + if (!dirty_ram_text_native_ok(target & 0x1FFFFFFFu)) return 0; if (psx_mixed_owner_enabled() && interp_host_stack_used() > psx_mixed_stack_watermark()) { cpu->pc = target; @@ -1024,7 +1072,10 @@ static int interp_enter_compiled(CPUState *cpu, uint32_t target) { } g_mixed_depth++; ls_func_enter(target, cpu); + int prev_phase = g_exec_phase; + g_exec_phase = 3; int r = psx_dispatch_game_compiled(cpu, target); + g_exec_phase = prev_phase; ls_func_exit(target, cpu, r); g_mixed_depth--; return r; @@ -1455,7 +1506,15 @@ static int exec_one(CPUState *cpu, uint32_t pc, uint32_t *next_pc_out) { case 0x01: { /* REGIMM: BLTZ/BGEZ/BLTZAL/BGEZAL by rt field */ int taken; switch (rt) { - case 0x00: /* BLTZ */ taken = ((int32_t)cpu->gpr[rs] < 0); break; + case 0x00: /* BLTZ */ + /* Widescreen render-funnel LEFT-edge widen (auto_screen_x): a + * classified funnel bltz rejects only past the revealed margin. + * Identity at 4:3 (margin 0). Gated per-game, cheap cached scan. */ + if (psx_ws_auto_cull_on() && ws_cull_bltz_site(pc)) + taken = psx_ws_cull_bltz(cpu->gpr[rs]); + else + taken = ((int32_t)cpu->gpr[rs] < 0); + break; case 0x01: /* BGEZ */ taken = ((int32_t)cpu->gpr[rs] >= 0); break; case 0x10: /* BLTZAL */ taken = ((int32_t)cpu->gpr[rs] < 0); cpu->gpr[31] = pc + 8; break; @@ -1469,23 +1528,32 @@ static int exec_one(CPUState *cpu, uint32_t pc, uint32_t *next_pc_out) { return 1; } case 0x08: /* ADDI rt, rs, simm — same as ADDIU, sans overflow trap (we don't model traps here) */ - cpu->gpr[rt] = cpu->gpr[rs] + (uint32_t)simm; + cpu->gpr[rt] = cpu->gpr[rs] + (uint32_t)simm + + (psx_ws_is_cull_bias_site(pc) ? (uint32_t)psx_ws_x_margin() : 0u); cpu->gpr[0] = 0; return 0; case 0x09: /* ADDIU rt, rs, simm */ - cpu->gpr[rt] = cpu->gpr[rs] + (uint32_t)simm; + cpu->gpr[rt] = cpu->gpr[rs] + (uint32_t)simm + + (psx_ws_is_cull_bias_site(pc) ? (uint32_t)psx_ws_x_margin() : 0u); cpu->gpr[0] = 0; return 0; case 0x0A: /* SLTI */ - cpu->gpr[rt] = ((int32_t)cpu->gpr[rs] < simm) ? 1u : 0u; + /* Widescreen render-funnel RIGHT-edge widen (auto_screen_x) for the + * signed min/max funnel idiom (`slti v, minSX, W`) — the paired left + * edge is the bltz above. Identity at 4:3 (margin 0). */ + if (psx_ws_is_cull_slti_site(pc) || + (psx_ws_auto_cull_on() && psx_ws_is_cull_w_imm(imm) && ws_cull_site(pc))) + cpu->gpr[rt] = (uint32_t)psx_ws_cull_slti(cpu->gpr[rs], imm); + else + cpu->gpr[rt] = ((int32_t)cpu->gpr[rs] < simm) ? 1u : 0u; cpu->gpr[0] = 0; return 0; case 0x0B: /* SLTIU */ - /* Widescreen render-funnel cull widening (auto_screen_x): always apply - * the shared helper for a flagged render-cull site — it is byte-identical + /* Widescreen render-funnel cull widening (auto_screen_x): apply the + * shared helper for a flagged render-cull site — it is byte-identical * to the vanilla compare at 4:3 (margin 0) and widens at 16:9, so the one * code path serves both aspects (no widescreen-specific caching). */ - if ((imm == 0x140 || imm == 0x141) && ws_cull_site(pc)) + if (psx_ws_auto_cull_on() && psx_ws_is_cull_w_imm(imm) && ws_cull_site(pc)) cpu->gpr[rt] = (uint32_t)psx_ws_cull_sltiu(cpu->gpr[rs], imm); else cpu->gpr[rt] = (cpu->gpr[rs] < (uint32_t)simm) ? 1u : 0u; @@ -1717,6 +1785,8 @@ int dirty_ram_dispatch(CPUState* cpu, uint32_t addr, uint32_t stop_addr) { else if (addr == 0x80046264u) site_note(&g_site_dd264); /* loop tail re-dispatch */ int prev = g_dirty_interp_active; g_dirty_interp_active = 1; + int prev_phase = g_exec_phase; + g_exec_phase = 1; int r = dirty_ram_dispatch_inner(cpu, addr, stop_addr); /* pc=0 producer tripwire: the dirty path reported "handled" yet published a @@ -1780,6 +1850,7 @@ int dirty_ram_dispatch(CPUState* cpu, uint32_t addr, uint32_t stop_addr) { } g_dirty_interp_active = prev; + g_exec_phase = prev_phase; return r; } @@ -1833,8 +1904,10 @@ static int precise_pc_dispatchable(uint32_t pc) { static void psx_run_precise(CPUState *cpu, uint32_t bcyc, int deadline_entry) { int prev_precise = g_precise_mode; int prev_active = g_dirty_interp_active; + int prev_phase = g_exec_phase; g_precise_mode = 1; g_dirty_interp_active = 1; + g_exec_phase = 1; uint32_t pc = cpu->pc; g_slice_last_block = pc; @@ -1981,6 +2054,7 @@ static void psx_run_precise(CPUState *cpu, uint32_t bcyc, int deadline_entry) { #endif g_precise_mode = prev_precise; g_dirty_interp_active = prev_active; + g_exec_phase = prev_phase; } /* Block-leader slice guard, called by the emitted prologue of every compiled @@ -2062,24 +2136,34 @@ static int dirty_ram_dispatch_inner(CPUState* cpu, uint32_t addr, uint32_t stop_ #ifdef PSX_HAS_GAME_DISPATCH xprobe_event(cpu->gpr[31], XOP_DD, XSITE_DD, addr, 0u, cpu->gpr[29], cpu->gpr[31], 0); - /* Run the statically-compiled game function ONLY for a CLEAN page (RAM matches - * the compiled image). A dirty page in the game-text region means an overlay - * overwrote it after the game-start baseline (dirty_ram_clear_image_baseline), - * so the compiled function is STALE and we must fall through to interpret the - * live overlay. Without this gate the loader thread's psx_dispatch(0x8001DB38) - * ran the stale compiled func_8001DB38 instead of the START.BIN-loader overlay - * (Tomba 2 Whoopee-Camp splash). Clean text still runs compiled (the fast path, - * unchanged for non-overlaying games once their text is baselined). */ - if (!dirty_ram_is_dirty(phys)) { + /* Run the statically-compiled game function only while the target is still + * native-safe. Dirty overlay pages and pages whose text bytes diverged from + * the original EXE image fall through to interpret the live RAM bytes. */ + if (dirty_ram_text_native_ok(phys)) { g_mixed_depth++; { ls_func_enter(addr, cpu); + int prev_phase = g_exec_phase; + g_exec_phase = 3; int _gc = psx_dispatch_game_compiled(cpu, addr); + g_exec_phase = prev_phase; ls_func_exit(addr, cpu, _gc); g_mixed_depth--; if (_gc) return 1; } clean_game_text_miss = psx_game_address_in_text(addr) ? 1 : 0; + } else if (psx_game_address_in_text(addr)) { + /* RAM at a game-text address diverged from the static EXE image + * (runtime-relocated / overlaid / self-modified code the compiled + * static function no longer reflects). The live RAM is the truth: + * fall through to INTERPRET it here rather than bail (line below) to + * the shell shadow (normalize() -> shell ROM). Crash Bash relocates a + * code page onto 0x30000 (inside the BIOS shell window); without this + * its 0x30FF4 call diverged (native_ok=0) yet was not dirty, so the + * interpreter bailed and normalize() shadowed it to dead shell ROM -> + * unknown-dispatch abort. Marking it a clean game-text miss lets the + * dirty interpreter execute the real RAM bytes. */ + clean_game_text_miss = 1; } #endif @@ -2185,6 +2269,28 @@ static int dirty_ram_dispatch_inner(CPUState* cpu, uint32_t addr, uint32_t stop_ #ifndef PSX_NO_DEBUG_TOOLS extern void debug_server_cyc_observe(uint32_t block_leader_phys); #endif + /* Async-interrupt latency fix. A guest wait loop that lives ENTIRELY in + * dirty RAM (e.g. libcd's CdSync spin, as in Kula World) re-enters this + * interpreter once per short (~7-insn) block and exits before the + * per-invocation `(insns_executed & 0xFFF)` gate below can fire — so a + * pending, IEc+IM2-enabled interrupt is never taken and the loop spins for + * seconds while the CD IRQ that would set its wait-flag is never serviced. + * Poll on a GLOBAL invocation counter so short-block loops still yield to + * interrupts (this is the interpreter analogue of a block-leader poll in + * static code). psx_check_interrupts runs the handler and returns with + * registers restored, so continuing the loop afterward is safe. */ + { + static uint32_t s_interp_entry_poll = 0; + static int s_entry_poll_enabled = -1; /* DIAGNOSTIC toggle (590c236 x kind-30 escape regression hunt) */ + if (s_entry_poll_enabled < 0) { + const char* e = getenv("PSX_DIRTY_ENTRY_POLL"); + s_entry_poll_enabled = (e && e[0] == '0') ? 0 : 1; + } + if (s_entry_poll_enabled && (++s_interp_entry_poll & 0x3Fu) == 0) { + cpu->pc = pc; + psx_check_interrupts(cpu); + } + } for (int i = 0; i < MAX_INSNS_PER_DISPATCH; i++) { uint32_t next_pc = 0; #ifndef PSX_NO_DEBUG_TOOLS diff --git a/runtime/src/disc_identity.cpp b/runtime/src/disc_identity.cpp index 7d546738..04ca7625 100644 --- a/runtime/src/disc_identity.cpp +++ b/runtime/src/disc_identity.cpp @@ -130,6 +130,8 @@ std::string scan_boot_serial(const std::vector<uint8_t>& buf) { j++; if (token.size() > 32) break; } + const size_t slash = token.find_last_of("\\/"); + if (slash != std::string::npos) token = token.substr(slash + 1); const std::string norm = normalize_serial(token); if (!norm.empty()) return norm; } diff --git a/runtime/src/dma.c b/runtime/src/dma.c index 5cef7616..5dcdb716 100644 --- a/runtime/src/dma.c +++ b/runtime/src/dma.c @@ -20,6 +20,7 @@ #include "mdec.h" #include "overlay_capture.h" #include "spu.h" +#include "audio_trace.h" #include "event_ring.h" #include <stdint.h> #include <stdio.h> @@ -664,6 +665,7 @@ static uint32_t execute_ch2_gpu(void) { * Each node: bits 24-31 = number of words following header, * bits 0-23 = next node address (0xFFFFFF = end). * The words following the header are sent to GP0. */ + gpu_ws_begin_linked_list(); uint32_t addr = channels[2].madr & 0x1FFFFCu; uint32_t safety = 0; const uint32_t MAX_NODES = 0x40000; /* prevent infinite loops */ @@ -748,6 +750,10 @@ static uint32_t execute_ch4_spu(void) { spu_dma_write(psx_read_word(addr)); addr = (addr + addr_step) & 0x1FFFFCu; } + /* One aggregated event per SPU-bound transfer (per-word would flood + * the ring: a full sound-bank upload is ~128k words). */ + audio_trace_event(AUDIO_EV_DMA_WRITE, total_words, + channels[4].madr & 0x1FFFFCu); } else { for (uint32_t i = 0; i < total_words; i++) { psx_write_word(addr, 0); diff --git a/runtime/src/fntrace.c b/runtime/src/fntrace.c index 12567008..b5e8a437 100644 --- a/runtime/src/fntrace.c +++ b/runtime/src/fntrace.c @@ -1,6 +1,7 @@ /* fntrace.c — runtime side of psx_dispatch call ring. See fntrace.h. */ #include "fntrace.h" +#include "text_xlate.h" /* on-the-fly string translation hook (framework) */ #include "parity_trace.h" /* general control-flow parity ring (native producer) */ #include <string.h> #include <stdlib.h> @@ -43,6 +44,13 @@ void fntrace_set_game_range(uint32_t lo, uint32_t hi) { int fntrace_is_game_started(void) { return s_game_started; } static inline int armed_match(uint32_t target) { + /* PSX_FNTRACE_ALL=1 records every dispatch from power-on — the ring + * then holds the earliest boot execution (before any TCP client can + * arm it) for offline first-divergence diffs. Checked once. */ + static int boot_all = -1; + if (boot_all == -1) { const char *e = getenv("PSX_FNTRACE_ALL"); + boot_all = (e && *e && *e != '0') ? 1 : 0; } + if (boot_all) return 1; if (s_arm_record_all) return 1; /* Hot path: when nothing is armed, record nothing. Recording every * dispatch by default makes the ring fill in seconds and burns ~10% @@ -56,6 +64,15 @@ static inline int armed_match(uint32_t target) { } void fntrace_record(CPUState* cpu, uint32_t target) { + /* On-the-fly string translation (framework feature — text_xlate.cpp). The + * generated psx_dispatch_impl calls us at the top of each dispatch iteration + * with cpu->gpr[4..7] holding THIS call's args, BEFORE the target function + * runs — the exact chokepoint to (a) capture every source string drawn and + * (b) repoint a string-arg at a translated replacement so the game's own + * renderer draws it. Cheap no-op when uninitialised/disarmed. No BIOS regen + * (runtime-side), works in Release. See docs/STRING_TRANSLATION.md. */ + text_xlate_on_dispatch(cpu, target); + /* General parity ring: one DISPATCH row per leader while current_tcb matches * the watched thread. Gated by the cheap armed flag so disarmed runs pay only * a single branch on this hot path. pc==target (the leader being entered). */ @@ -87,7 +104,22 @@ void fntrace_record(CPUState* cpu, uint32_t target) { } if (!s_game_started && s_game_entry_phys != 0) { - if ((target & 0x1FFFFFFFu) == s_game_entry_phys) { + /* Latch on the entry_pc dispatch (the common case) OR the first dispatch + * to any game-text address whose RAM already holds the loaded game EXE + * image (native-ok). Some titles reach their PS-EXE entry via compiled + * internal flow rather than a dispatcher round-trip, so the entry_pc + * dispatch never arrives and `started` would never latch — leaving the + * shell-window shadow (normalize() maps RAM 0x30000-0x5AFFF -> shell ROM) + * active over REAL game text. Crash Bash: entry 0x2E7B0 is reached + * internally, so its first shell-window call (0x30FF4 / 0x3358C) got + * shadowed to dead shell ROM -> unknown-dispatch abort. The native-ok + * test is safe before the game loads: that RAM still holds shell bytes + * that differ from the game image, so it cannot fire prematurely. */ + extern int psx_game_address_in_text(uint32_t addr); + extern int dirty_ram_text_native_ok(uint32_t phys); + uint32_t _tphys = target & 0x1FFFFFFFu; + if (_tphys == s_game_entry_phys || + (psx_game_address_in_text(target) && dirty_ram_text_native_ok(_tphys))) { s_game_started = 1; /* Establish the clean compiled-image baseline now: the boot EXE is fully * loaded into the game-text region (== compiled image) and no gameplay @@ -96,16 +128,25 @@ void fntrace_record(CPUState* cpu, uint32_t target) { * later overlay overwrites, so the dispatch runs clean text compiled and * interprets only true overlays (Tomba 2 boot-text loader overlay). */ extern void dirty_ram_clear_image_baseline(void); + extern void memory_clear_low_boot_scratch(void); dirty_ram_clear_image_baseline(); + memory_clear_low_boot_scratch(); cdrom_notify_game_started(); boot_state_trigger_capture(cpu); } } - if (!armed_match(target)) return; /* Honor the one-shot capture freeze (insn_freeze): once latched, the ring * preserves the pre-divergence window instead of evicting it. */ extern int g_insn_log_frozen; - if (g_insn_log_frozen) return; + int was_frozen = g_insn_log_frozen; + /* Null-page dispatch guard: a dispatch whose phys target sits below the + * exception vectors (0x80/0xA0) is a jump through a null/garbage pointer — + * the wedge itself, never legitimate code. Latch the capture freeze so the + * ring preserves the window that led here (this culprit entry, with its + * ra, still records; the wedge storm after it does not). */ + if ((target & 0x1FFFFFFFu) < 0x40u) g_insn_log_frozen = 1; + if (!armed_match(target)) return; + if (was_frozen) return; uint64_t idx = g_fntrace_seq++ & (FNTRACE_RING_CAP - 1); FntraceEntry* e = &g_fntrace_ring[idx]; e->frame = (uint32_t)s_frame_count; diff --git a/runtime/src/frame_pacing.c b/runtime/src/frame_pacing.c index 95f9c2e8..7af5bab8 100644 --- a/runtime/src/frame_pacing.c +++ b/runtime/src/frame_pacing.c @@ -23,16 +23,37 @@ uint32_t frame_pacing_sleep_ms(uint64_t now, uint64_t deadline, } #ifndef FRAME_PACING_PURE_ONLY + +/* Bounded catch-up window, in periods. A transient stall (heavy frame, CD + * burst) leaves next_deadline in the past; KEEPING that debt and running + * unpaced until it is repaid preserves the long-term rate at exactly one + * period per frame — which the audio pipeline depends on (the SPU produces + * 768 guest cycles per output sample; every re-anchor that forgives debt + * permanently starves the audio ring by the forgiven amount. Measured on + * MMX5: forgiving all >1-period debt averaged 59.80 Hz against a 59.94 + * target = -0.4% chronic audio underrun). Only debt beyond this window — + * sustained sub-realtime emulation, not a hiccup — is forgiven, else the + * pacer would demand unbounded catch-up. */ +#define FRAME_PACER_CATCHUP_MAX_PERIODS 8u + void frame_pacer_wait(FramePacer *p, double period_ms) { uint64_t freq = SDL_GetPerformanceFrequency(); uint64_t period = (uint64_t)((double)freq * (period_ms / 1000.0)); uint64_t now = SDL_GetPerformanceCounter(); - if (p->next_deadline == 0 || now >= p->next_deadline + period) { - /* First frame, or fell more than one period behind: re-anchor. */ + if (p->next_deadline == 0 || + now >= p->next_deadline + period * FRAME_PACER_CATCHUP_MAX_PERIODS) { + /* First frame, or sustained slowness beyond the catch-up window: + * re-anchor (forgive the debt). */ p->next_deadline = now + period; return; } + if (now >= p->next_deadline) { + /* In debt from a recent stall: run this frame unpaced and advance + * the deadline, repaying one period of debt per fast frame. */ + p->next_deadline += period; + return; + } for (;;) { now = SDL_GetPerformanceCounter(); /* ONE read per iteration */ diff --git a/runtime/src/gpu.c b/runtime/src/gpu.c index e3a244be..2ecdbfd4 100644 --- a/runtime/src/gpu.c +++ b/runtime/src/gpu.c @@ -13,11 +13,13 @@ #include "gpu.h" #include "gpu_sw_renderer.h" #include "gpu_render.h" +#include "text_xlate.h" #include "crash_trace.h" #include "debug_server.h" #include "cpu_state.h" #include "event_ring.h" #include "color_lut.h" +#include "ws_cull_detect.h" #include <stdint.h> #include <stdio.h> #include <stdlib.h> @@ -98,6 +100,7 @@ static int ws_engaged(void) { return ws_mode != 0; } /* Forward decls: defined later but used by psx_ws_backdrop_x above them. */ static int32_t ws_scale_about(int32_t x, int32_t ax); static int32_t ws_disp_w(void); +static void ws_clear_all_reveal_margins(void); /* Gameplay vs full-2D screen. Character/billboard prims tag (psx_ws_sprite_tag) * within the last couple of frames => the actor render funnel is running => @@ -111,6 +114,40 @@ static int32_t ws_disp_w(void); * engage (gpu_ws_set_full_2d); PSX_WS_FORCE_2D=1 forces it on for testing. */ static int ws_full_2d = 0; void gpu_ws_set_full_2d(int on) { ws_full_2d = on ? 1 : 0; } +static int ws_clear_reveal = 0; +static int g_mmx6_void_sides = 0; +static uint32_t g_mmx6_void_generation = 1; +void gpu_ws_set_clear_reveal(int on) { ws_clear_reveal = on ? 1 : 0; } + +/* GTE-activity gameplay detector ([widescreen] gte_game_mode) — the generic + * 3D-title analog of the sprite-tag stamp. A fully-3D game (e.g. Ape Escape) + * has no per-prim tag helper to hook, but every gameplay frame projects a + * meaningful number of vertices through RTPS/RTPT, while a full-2D screen + * (save/options/memory-card) projects none (or a token few). gte.cpp notes + * every projection here; when a frame accumulates >= the threshold the frame + * is stamped as gameplay, with the same 2-frame hysteresis the tag path uses. + * Off unless the game opts in, so existing titles are untouched. */ +static int ws_gte_game_mode_cfg = 0; +static uint32_t ws_gte_frame = (uint32_t)-1; +static uint32_t ws_gte_count = 0; +static uint32_t ws_last_gte_stamp = (uint32_t)-1000; +/* Any frame that projects a handful of vertices is "3D" (a low threshold so a + * sparse close-up cutscene frame still counts — the flicker was frames dipping + * below a high 16-vert bar and pillarboxing for a frame or two). */ +#define WS_GTE_GAME_MODE_MIN_VERTS 3u +/* STICKY: stay in native-wide for ~0.75s after the last 3D frame, so brief + * low-poly frames in a real-time 3D cutscene never flip to a 4:3 pillarbox (the + * intro-cutscene flicker). Only a genuine full-2D screen — no GTE projection for + * this many consecutive frames (save/options/memory-card) — reverts to 4:3. */ +#define WS_GTE_GAME_MODE_HYSTERESIS 45u +void gpu_ws_set_gte_game_mode(int on) { ws_gte_game_mode_cfg = on ? 1 : 0; } +void psx_ws_note_gte_project(int nverts) { + if (!ws_gte_game_mode_cfg) return; + uint32_t f = (uint32_t)s_frame_count; + if (f != ws_gte_frame) { ws_gte_frame = f; ws_gte_count = 0; } + ws_gte_count += (uint32_t)nverts; + if (ws_gte_count >= WS_GTE_GAME_MODE_MIN_VERTS) ws_last_gte_stamp = f; +} /* Full-2D tile-engine mode (MMX6): config [widescreen] full_2d, or the PSX_WS_FORCE_2D * test override. Distinct from ws_game_mode (which also fires on the 3D sprite-tag path, @@ -122,6 +159,8 @@ static int ws_full_2d_mode(void) { } static int ws_game_mode(void) { if (ws_full_2d_mode()) return 1; + if (ws_gte_game_mode_cfg && + (uint32_t)s_frame_count - ws_last_gte_stamp <= WS_GTE_GAME_MODE_HYSTERESIS) return 1; return (uint32_t)s_frame_count - ws_last_tag_stamp <= 2; } @@ -187,7 +226,41 @@ int ws_nw_extra(void) { return 2 * ws_nw_offset(); } * cull while still stretching; large = over-draw) at a fixed camera position. * -1 = normal computed margin. */ static int ws_margin_override = -1; +static int ws_cull_guard_pixels = 0; void gpu_ws_set_margin_override(int v) { ws_margin_override = v; } +void gpu_ws_set_cull_guard_pixels(int pixels) { + if (pixels < 0) pixels = 0; + if (pixels > 256) pixels = 256; + ws_cull_guard_pixels = pixels; +} + +#define WS_EXPLICIT_CULL_SITES_MAX 64 +static uint32_t ws_explicit_bias_sites[WS_EXPLICIT_CULL_SITES_MAX]; +static uint32_t ws_explicit_slti_sites[WS_EXPLICIT_CULL_SITES_MAX]; +static int ws_explicit_bias_n = 0; +static int ws_explicit_slti_n = 0; +void gpu_ws_set_explicit_cull_sites(const uint32_t *bias, int nbias, + const uint32_t *slti, int nslti) { + if (nbias < 0) nbias = 0; + if (nslti < 0) nslti = 0; + if (nbias > WS_EXPLICIT_CULL_SITES_MAX) nbias = WS_EXPLICIT_CULL_SITES_MAX; + if (nslti > WS_EXPLICIT_CULL_SITES_MAX) nslti = WS_EXPLICIT_CULL_SITES_MAX; + ws_explicit_bias_n = nbias; + ws_explicit_slti_n = nslti; + for (int i = 0; i < nbias; i++) ws_explicit_bias_sites[i] = bias[i] & 0x1FFFFFFFu; + for (int i = 0; i < nslti; i++) ws_explicit_slti_sites[i] = slti[i] & 0x1FFFFFFFu; +} +static int ws_explicit_site(const uint32_t *sites, int n, uint32_t pc) { + uint32_t p = pc & 0x1FFFFFFFu; + for (int i = 0; i < n; i++) if (sites[i] == p) return 1; + return 0; +} +int psx_ws_is_cull_bias_site(uint32_t pc) { + return ws_explicit_site(ws_explicit_bias_sites, ws_explicit_bias_n, pc); +} +int psx_ws_is_cull_slti_site(uint32_t pc) { + return ws_explicit_site(ws_explicit_slti_sites, ws_explicit_slti_n, pc); +} int psx_ws_x_margin(void) { if (ws_margin_override >= 0) return ws_margin_override; @@ -196,7 +269,7 @@ int psx_ws_x_margin(void) { * that previously fell outside the 4:3 cull window; the wide compositor then * rasterizes it into the revealed margins. Same recompiler emit sites as the * squash path ([widescreen.cull]); 0 at 4:3 so the cull stays byte-identical. */ - if (ws_native_wide_active()) return ws_nw_offset(); + if (ws_native_wide_active()) return ws_nw_offset() + ws_cull_guard_pixels; if (!ws_active()) return 0; return (160 * (ws_xden - ws_xnum) + ws_xnum / 2) / ws_xnum; } @@ -304,6 +377,19 @@ int psx_ws_mmx6_bg_startx(int x) { return psx_ws_bg2d_startx(x); } int psx_ws_mmx6_bg_stream_left(int x) { return psx_ws_bg2d_stream_left(x); } int psx_ws_mmx6_bg_stream_right(int x) { return psx_ws_bg2d_stream_right(x); } +/* Called at entry to MMX6's full tile-ring initializer (FUN_800269F4). The + * independent layers invoke it only when stage/background data is dirty; gate + * by frame because up to three layers initialize together. This is the exact + * point where pixels retained from the old stage cease to be meaningful. */ +void psx_ws_mmx6_bg_stage_init(void) { + static uint32_t last_frame = 0xFFFFFFFFu; + uint32_t frame = (uint32_t)s_frame_count; + if (!ws_clear_reveal || ws_mode != 2 || frame == last_frame) return; + last_frame = frame; + g_mmx6_void_generation++; + ws_clear_all_reveal_margins(); +} + /* ===== MMX6 BG packet-buffer RELOCATION ([widescreen.bg2d] bufbase_site/cap_site) === * The widened BG render (29 cols) overruns the engine's BG packet double-buffer (driver * base 0x800B91C0, stride 0x4000 = 1024 tile slots/buffer) and its per-frame 1000-tile cap @@ -533,13 +619,26 @@ static int bg2d_fill_column(int layer, int worldX, int worldY, int write, int mapLeft = psx_read_byte(lbase + 0x4d); int mapRight = psx_read_byte(lbase + 0x4e); int scrollX = (int16_t)psx_read_half(lbase + 0xa); + /* C integer division toward zero makes worldX -255..-1 resolve to metatile + * column 0 below. The native 4:3 streamer never requests those coordinates, + * but the widened left window does at a stage starting on mapLeft. Treat + * them as finite-map void before the engine's loop/wrap bookkeeping. */ + if (worldX < mapLeft * 0x100) + return 2; if ((uint32_t)metaCol < (uint32_t)mapLeft) { metaCol = metaCol + 1 + (mapRight - mapLeft); - if (scrollX <= (mapLeft - 1) * 0x100) return 0; /* engine early-return: skip */ + if (scrollX <= (mapLeft - 1) * 0x100) { + /* The native renderer never sees this column, so the engine leaves + * its ring slot alone. Report finite-map void to the DMA-start + * cleanup without mutating ring slots canonical draws can alias. */ + return 2; /* finite-map void, not merely a normal streamed column */ + } } if ((uint32_t)mapRight < (uint32_t)metaCol) { metaCol = (metaCol - 1 - mapRight) + mapLeft; - if ((mapRight + 1) * 0x100 <= scrollX) return 0; + if ((mapRight + 1) * 0x100 <= scrollX) { + return 2; + } } int mapW = psx_read_byte(g_bg2d_map_size_addr); int mapH = psx_read_byte(g_bg2d_map_size_addr + 1u); @@ -604,7 +703,9 @@ void psx_ws_mmx6_bg_refill_all(void) { int sx = (int16_t)psx_read_half(lbase + 0xa); int sy = (int16_t)psx_read_half(lbase + 0xe); for (int ci = -left; ci < g_bg2d_native_cols + left; ci++) { - if (bg2d_fill_column((int)layer, sx + ci * 16, sy - 0x10, 1, NULL, NULL)) + int filled = bg2d_fill_column((int)layer, sx + ci * 16, + sy - 0x10, 1, NULL, NULL); + if (filled) g_mmx6_refill_cols++; } } @@ -651,22 +752,77 @@ int psx_ws_cull_sltiu(uint32_t sx, uint32_t imm) { < (uint32_t)((int32_t)imm + 2 * m)) ? 1 : 0; } -/* Detect the GTE per-vertex trivial-reject signature in a run of instruction - * words: at least one `sltiu …,0x140/0x141` (width) AND one `sltiu …,0xE0/0xF1` - * (height). Lets the sljit JIT + interpreter gate the cull-widening to real - * render funnels (a lone sltiu 0x140 elsewhere must stay vanilla). Mirrors the - * recompiler's func_has_screen_extent_cull. */ -int psx_ws_func_has_screen_cull(const uint32_t *words, int n) { - int has_x = 0, has_y = 0; - for (int i = 0; i < n; i++) { - uint32_t w = words[i]; - if ((w & 0xFC000000u) != 0x2C000000u) continue; /* sltiu */ - uint32_t imm = w & 0xFFFFu; - if (imm == 0x140 || imm == 0x141) has_x = 1; - else if (imm == 0xE0 || imm == 0xF1) has_y = 1; - if (has_x && has_y) return 1; +/* Signed right-edge widen for the min/max funnel idiom (`slti v, minSX, W`): + * the paired LEFT edge is a separate bltz (psx_ws_cull_bltz below), so this + * bound moves out by ONE margin only. Operand is an already sign-extended / + * computed 32-bit screen X. Identity at margin 0 (4:3). */ +int psx_ws_cull_slti(uint32_t sx, uint32_t imm) { + return ((int32_t)sx < (int32_t)imm + psx_ws_x_margin()) ? 1 : 0; +} + +/* Signed left-edge widen for the funnel's `bltz maxSX, reject`: reject only + * when the prim ends left of the REVEALED edge (maxSX < -margin). Returns the + * branch predicate. Identity at margin 0 (4:3). */ +int psx_ws_cull_bltz(uint32_t v) { + return ((int32_t)v < -psx_ws_x_margin()) ? 1 : 0; +} + +/* ---- Cull signature configuration ([widescreen.cull] screen_w_imms / + * screen_h_imms). The width/height immediates are per-game (Tomba: 0x140/0x141 + * + 0xE0/0xF1 on a 320 display; Ape Escape: 0x181 + 0xF1 on 368). Defaults + * keep the original Tomba signature so existing configs are unchanged. The + * sets are consulted by the shared detector on every backend (interp + sljit; + * the recompiler reads the same config at gen time). */ +static uint32_t ws_cull_w_imms[8] = { 0x140, 0x141 }; +static int ws_cull_w_n = 2; +static uint32_t ws_cull_h_imms[8] = { 0xE0, 0xF1 }; +static int ws_cull_h_n = 2; +void gpu_ws_set_cull_imms(const uint32_t *w, int nw, const uint32_t *h, int nh) { + if (w && nw > 0) { + if (nw > 8) nw = 8; + for (int i = 0; i < nw; i++) ws_cull_w_imms[i] = w[i]; + ws_cull_w_n = nw; } - return 0; + if (h && nh > 0) { + if (nh > 8) nh = 8; + for (int i = 0; i < nh; i++) ws_cull_h_imms[i] = h[i]; + ws_cull_h_n = nh; + } +} +int psx_ws_is_cull_w_imm(uint32_t imm) { + return psx_ws_cull_imm_in(imm, ws_cull_w_imms, ws_cull_w_n); +} + +/* ---- Runtime gates for the pattern-scanned widescreen hooks. The interp and + * sljit derive widen sites by scanning live code; that derivation must honor + * the SAME per-game [widescreen.cull] opt-ins the recompiler emit does. These + * default OFF: a title that never opted in must never have its code + * pattern-scanned and rewritten (an ungated backdrop false positive rewrites a + * live GPR = wild-jump fatal — the exact class this gate closes). Set from + * game.toml at startup (main.cpp). */ +static int ws_auto_cull_on_cfg = 0; +static int ws_auto_backdrop_on_cfg = 0; +void gpu_ws_set_auto_hooks(int cull_on, int backdrop_on) { + ws_auto_cull_on_cfg = cull_on ? 1 : 0; + ws_auto_backdrop_on_cfg = backdrop_on ? 1 : 0; +} +int psx_ws_auto_cull_on(void) { return ws_auto_cull_on_cfg; } + +/* Detect the GTE screen-extent trivial-reject signature in a run of + * instruction words: at least one width compare AND one height compare + * (slti or sltiu, immediates from the configured sets). Lets the sljit JIT + + * interpreter gate the cull-widening to real render funnels (a lone width + * compare elsewhere must stay vanilla). Same shared scan the recompiler's + * func_has_screen_extent_cull uses (ws_cull_detect.h). */ +int psx_ws_func_has_screen_cull(const uint32_t *words, int n) { + return psx_ws_cull_scan(words, n, ws_cull_w_imms, ws_cull_w_n, + ws_cull_h_imms, ws_cull_h_n); +} + +/* Classify words[idx] as an X left-edge reject bltz (shared structural + * detector, runtime imm sets). Caller qualifies the window first. */ +int psx_ws_cull_bltz_at(const uint32_t *words, int n, int idx) { + return psx_ws_cull_bltz_here(words, n, idx, ws_cull_w_imms, ws_cull_w_n); } /* Widescreen backdrop screen-X correction ([widescreen.backdrop] x_sites). @@ -700,7 +856,9 @@ int psx_ws_backdrop_x(int x) { * generator's own low/high clamps still bound it at the level edges. Gated on * native-wide only (NOT squash: that path uses psx_ws_backdrop_x()). Returns * `orig` unchanged at 4:3 / squash / boot / FMV, so 4:3 stays byte-identical. */ -int psx_ws_backdrop_preload(void) { return ws_native_wide_active(); } +int psx_ws_backdrop_preload(void) { + return ws_auto_backdrop_on_cfg && ws_native_wide_active(); +} /* Live-tunable widen amount (set via the `ws_backdrop_margin` debug command): * <0 => WHOLE-ROW preload: START->0, END->extent-1 (max generous) @@ -1057,6 +1215,92 @@ static int ws_sprt_fixed_transform(int32_t *x0, int w) { return 0; } +/* ---- Native-wide HUD corner re-anchoring ([widescreen] nw_hud_corners) ------ + * In native-wide the whole frame is composited into a wider surface centred by + * ws_nw_offset() per side (the reveal). Screen-space 2D HUD (drawn with fixed + * rect/sprite GP0 commands, never through the GTE — a 3D title's world is all + * polygons) therefore lands inset from the true wide edges by exactly the + * reveal. This pushes an outer-third HUD element the rest of the way to its + * wide corner with an additive thirds shift: left third −offset, right third + * +offset, middle unchanged (matches the squash-path ws_hud_pivot geometry, but + * as a translate since native-wide does not squash). Composite pieces in one + * zone share a shift and stay aligned. Identity unless native-wide is engaged + * AND the game opts in, so 4:3 and non-opted titles are byte-identical. + * Returns the signed x delta to add to the prim's x before draw_offset. */ +static int ws_nw_hud_corners = 0; +static uint32_t ws_nw_left_hud_packet_lo = 0; +static uint32_t ws_nw_left_hud_packet_hi = 0; +void gpu_ws_set_nw_hud_corners(int on) { ws_nw_hud_corners = on ? 1 : 0; } +void gpu_ws_set_nw_left_hud_packet_range(uint32_t lo, uint32_t hi) { + ws_nw_left_hud_packet_lo = lo & 0x1FFFFFFFu; + ws_nw_left_hud_packet_hi = hi & 0x1FFFFFFFu; +} +static int ws_nw_left_hud_packet(void) { + if (!ws_native_wide_active() || gp0_cmd_source_addr == 0xFFFFFFFFu || + ws_nw_left_hud_packet_hi <= ws_nw_left_hud_packet_lo) + return 0; + uint32_t a = gp0_cmd_source_addr & 0x1FFFFFFFu; + int matched = a >= ws_nw_left_hud_packet_lo && a < ws_nw_left_hud_packet_hi; + return matched; +} +static int32_t ws_nw_hud_shift(int32_t x, int32_t w) { + if (!ws_native_wide_active()) return 0; + int32_t off = ws_nw_offset(); + if (off <= 0) return 0; + if (!ws_nw_left_hud_packet() && !ws_nw_hud_corners) return 0; + int32_t W = ws_disp_w(); + int32_t cx = 2 * x + w; /* 2*centre, avoids losing the half */ + if (3 * cx < 2 * W) return -off; /* left third -> pull to left edge */ + if (3 * cx > 4 * W) return off; /* right third -> push to right edge */ + return 0; /* middle third -> stay centred */ +} + +/* ---- Native-wide full-frame 2D backdrop stretch ([widescreen] nw_backdrop) --- + * A screen-space background plane (sky gradient / backdrop image) is drawn as an + * axis-aligned quad covering the whole 4:3 framebuffer [0,W]x[0,H]. It is NOT + * GTE-projected, so native-wide leaves it at its 4:3 span, composited centred = + * black bars in the revealed side margins (the pillarboxed sky). Detect exactly + * that shape — a 4-vertex quad whose corners form a rectangle spanning ~the full + * display width from ~the left edge — and stretch its X vertices about the + * display centre by the wide ratio so it fills the wider frame; the texture/UV + * (or gradient) simply stretches horizontally (invisible on a sky). GTE-drawn + * world quads are perspective-distorted (not axis-aligned) and partial-width, so + * they never match. Identity unless native-wide is engaged AND the game opts in. + * Transforms vx[0..3] IN PLACE (pre-draw_offset); returns 1 if it applied. */ +static int ws_nw_backdrop = 0; +void gpu_ws_set_nw_backdrop(int on) { ws_nw_backdrop = on ? 1 : 0; } +static int ws_nw_backdrop_stretch_quad(int32_t *vx, const int32_t *vy) { + if (!ws_nw_backdrop || !ws_native_wide_active()) return 0; + int32_t extra = ws_nw_extra(); + if (extra <= 0) return 0; + int32_t W = ws_disp_w(); + const int32_t EDGE = 24; /* slack for "touches the frame edge" */ + int32_t minx = vx[0], maxx = vx[0], miny = vy[0], maxy = vy[0]; + for (int i = 1; i < 4; i++) { + if (vx[i] < minx) minx = vx[i]; + if (vx[i] > maxx) maxx = vx[i]; + if (vy[i] < miny) miny = vy[i]; + if (vy[i] > maxy) maxy = vy[i]; + } + /* Must span the full display width and a real vertical extent. */ + if (minx > EDGE || maxx < W - EDGE || (maxy - miny) < 64) return 0; + /* Axis-aligned: every vertex X sits at either the min or the max edge, and + * every Y at the top or bottom edge (a true screen-space rectangle). */ + for (int i = 0; i < 4; i++) { + int xe = (vx[i] - minx <= EDGE) || (maxx - vx[i] <= EDGE); + int ye = (vy[i] - miny <= EDGE) || (maxy - vy[i] <= EDGE); + if (!xe || !ye) return 0; + } + /* Stretch X about the display centre by (W+extra)/W so [0,W] -> [-off, W+off], + * which the wide compositor (+off) maps onto the full [0, W+extra] surface. */ + int32_t cx = W / 2; + for (int i = 0; i < 4; i++) { + int32_t d = vx[i] - cx; + vx[i] = cx + (d * (W + extra) + (d >= 0 ? W / 2 : -W / 2)) / W; + } + return 1; +} + /* Polyline state */ static uint16_t polyline_color; /* mono polyline: current color */ static int32_t polyline_prev_x, polyline_prev_y; /* previous vertex */ @@ -1097,6 +1341,15 @@ static uint32_t draw_area_right, draw_area_bottom; /* Draw offset (set by GP0(E5h)) */ static int32_t draw_offset_x, draw_offset_y; +/* Instrumentation: per-vblank range/count of GP0(E5) draw-offset-Y sets. If a + * single frame sets offsets in BOTH the top (y<128) and bottom (y>=128) buffer + * bands, the game is drawing different parts of the scene into different display + * buffers in the same frame (candidate root of a character-in-one-buffer + * strobe). Snapshotted each vblank into the *_last copies, exposed via debug. */ +int32_t g_doff_min_this = 0x7fffffff, g_doff_max_this = -0x7fffffff; +uint32_t g_doff_cnt_this = 0; +int32_t g_doff_min_last = 0, g_doff_max_last = 0; +uint32_t g_doff_cnt_last = 0; /* Texture window raw value (set by GP0(E2h), readback via GP1(10h)) */ static uint32_t texture_window_value; @@ -1163,6 +1416,18 @@ static void ws_nw_sync_target(void) { else gr_wide_disable_target(); } +static void ws_clear_all_reveal_margins(void) { + for (int i = 0; i < ws_fb_n; i++) + gr_wide_clear_margins((int)ws_fb_base[i], 0, 512, 0, 3); +} + +/* Stage-init already clears both synthetic margins once. Do not keep clearing + * a guessed finite-map side here: MMX6's authored layers enter the reveal at + * different times, so the side guess produced a moving black trim over valid + * stage art. A stale reveal tile is safer than deleting submitted content. */ +void gpu_ws_begin_linked_list(void) { } + + /* Horizontal display range (GP1(06h)) */ static uint32_t h_display_x1; static uint32_t h_display_x2; @@ -1207,6 +1472,7 @@ static int sq_cap_armed; * flips every ~33,868 GPU clocks (one NTSC frame). */ #define GPUSTAT_POLL_VBLANK_THRESHOLD 1000 static uint32_t gpustat_poll_count; +uint64_t g_pollhack_vblank_count = 0; /* instrumentation: poll-fallback VBlank IRQ count */ /* ---- Initialization ---- */ @@ -1288,11 +1554,29 @@ uint32_t gpu_read_gpustat(void) { gpustat_poll_count++; if (gpustat_poll_count >= GPUSTAT_POLL_VBLANK_THRESHOLD) { gpustat_poll_count = 0; - lcf ^= 1; - psx_irq_raise(0, 0); /* IRQ_VBLANK (GPUSTAT-poll fallback path) */ - /* DEQUEUE: VBlank fired via the GPUSTAT-poll fallback path (distinct - * from the cycle-paced VBlank in psx_check_interrupts). */ - event_ring_record_aux(EV_DEQ, (uint8_t)SRC_VBLANK, 0xFFFFFFFFu); + /* The recompiled BIOS + game carry per-block interrupt checks + * (psx_check_interrupts_at) and per-block cycle charging, so tight + * LCF/VSync poll loops DO advance guest cycles and let the ONE + * cycle-paced VBlank authority (interrupts.c) fire on schedule. The old + * "spin forever" premise that justified raising VBlank+LCF from a raw + * GPUSTAT read-count is stale — and it injected ~38 fake VBlanks/s + * (measured on Crash Bash), delivering ~96/s to the game instead of 60, + * over-advancing the game's VSync frame counter and jittering animation + * (character strobe). Firing is now OFF by default; PSX_POLLHACK_VBLANK=1 + * restores the legacy behavior for A/B. */ + static int poll_fire = -1; + if (poll_fire < 0) { + const char* e = getenv("PSX_POLLHACK_VBLANK"); + poll_fire = (e && e[0] == '1') ? 1 : 0; + } + if (poll_fire) { + g_pollhack_vblank_count++; /* count only when actually firing */ + lcf ^= 1; + psx_irq_raise(0, 0); /* IRQ_VBLANK (GPUSTAT-poll fallback path) */ + /* DEQUEUE: VBlank fired via the GPUSTAT-poll fallback path (distinct + * from the cycle-paced VBlank in psx_check_interrupts). */ + event_ring_record_aux(EV_DEQ, (uint8_t)SRC_VBLANK, 0xFFFFFFFFu); + } } uint32_t stat = 0; @@ -1402,6 +1686,13 @@ static uint16_t rgb888_to_rgb555(uint32_t color24) { void gpu_vblank_tick(void) { lcf ^= 1; + /* snapshot per-frame draw-offset-Y range for the strobe instrumentation */ + if (g_doff_cnt_this) { + g_doff_min_last = g_doff_min_this; + g_doff_max_last = g_doff_max_this; + g_doff_cnt_last = g_doff_cnt_this; + } + g_doff_min_this = 0x7fffffff; g_doff_max_this = -0x7fffffff; g_doff_cnt_this = 0; gpustat_poll_count = 0; psx_irq_raise(0, 0); /* IRQ_VBLANK (gpu_vblank_tick) */ if (vblank_callback) vblank_callback(); @@ -1661,8 +1952,10 @@ static void gp0_exec_mono_quad(void) { int semi_trans = (gp0_cmd_buf[0] >> 25) & 1; uint16_t color = rgb888_to_rgb555(gp0_cmd_buf[0] & 0xFFFFFFu); int32_t vx[4], vy[4]; - for (int i = 0; i < 4; i++) { + for (int i = 0; i < 4; i++) parse_vertex(gp0_cmd_buf[1 + i], &vx[i], &vy[i]); + ws_nw_backdrop_stretch_quad(vx, vy); /* full-frame 2D backdrop stretch (no-op else) */ + for (int i = 0; i < 4; i++) { vx[i] += draw_offset_x; vy[i] += draw_offset_y; } @@ -1705,6 +1998,17 @@ static void gp0_exec_shaded_quad(void) { for (int i = 0; i < 4; i++) { c[i] = rgb888_to_rgb555(gp0_cmd_buf[i * 2] & 0xFFFFFFu); parse_vertex(gp0_cmd_buf[1 + i * 2], &vx[i], &vy[i]); + } + int32_t minx = vx[0], maxx = vx[0]; + for (int i = 1; i < 4; i++) { + if (vx[i] < minx) minx = vx[i]; + if (vx[i] > maxx) maxx = vx[i]; + } + int32_t hud_dx = ws_nw_hud_shift(minx, maxx - minx); + if (hud_dx) + for (int i = 0; i < 4; i++) vx[i] += hud_dx; + ws_nw_backdrop_stretch_quad(vx, vy); /* full-frame 2D backdrop stretch (sky gradient; no-op else) */ + for (int i = 0; i < 4; i++) { vx[i] += draw_offset_x; vy[i] += draw_offset_y; } @@ -1732,6 +2036,21 @@ static uint16_t current_texpage(void) { (semi_transparency << 5) | (texpage_colors << 7)); } +/* Hardware: the texpage attribute word carried inside every textured polygon + * (GP0 0x24-0x3F with the texture bit) is copied into the GPU draw-mode state + * (GPUSTAT bits 0-8) exactly like GP0(E1) bits 0-8 — the poly's own word, not + * the last E1, decides its semi-transparency mode, and later rectangle/sprite + * prims (which carry no texpage word) consume the state the poly left behind. + * Beetle: SetTPage(CB[4 + ((cc>>4)&1)] >> 16) on every textured poly. + * Bits 9-10 (dither / draw-to-display) exist only in E1; bit 11 (texture + * disable) only latches when GP1(09h) allowed it, which polys can't grant. */ +static void set_tpage_from_poly(uint16_t tpage_word) { + texpage_x = tpage_word & 0xF; + texpage_y = (tpage_word >> 4) & 1; + semi_transparency = (tpage_word >> 5) & 3; + texpage_colors = (tpage_word >> 7) & 3; +} + /* Helper: set up SW renderer state before a textured draw. * semi_trans: whether the primitive has semi-transparency enabled (from opcode bit) * raw_texture: 1 if this is a raw-texture opcode (bit 0 of GP0 opcode set) */ @@ -1760,9 +2079,10 @@ static void gp0_exec_textured_tri(void) { uint16_t clut = (uint16_t)(gp0_cmd_buf[2] >> 16); uint16_t clut_x = (clut & 0x3F) * 16; uint16_t clut_y = (clut >> 6) & 0x1FF; - /* Texpage from word 4 bits 16-31, or use current state */ + /* Texpage from word 4 bits 16-31 */ uint16_t tpage_word = (uint16_t)(gp0_cmd_buf[4] >> 16); uint16_t tpage = tpage_word & 0x1FF; + set_tpage_from_poly(tpage_word); for (int i = 0; i < 3; i++) { vx[i] += draw_offset_x; @@ -1795,7 +2115,9 @@ static void gp0_exec_textured_quad(void) { uint16_t clut = (uint16_t)(gp0_cmd_buf[2] >> 16); uint16_t clut_x = (clut & 0x3F) * 16; uint16_t clut_y = (clut >> 6) & 0x1FF; - uint16_t tpage = (uint16_t)(gp0_cmd_buf[4] >> 16) & 0x1FF; + uint16_t tpage_word = (uint16_t)(gp0_cmd_buf[4] >> 16); + uint16_t tpage = tpage_word & 0x1FF; + set_tpage_from_poly(tpage_word); /* Widescreen: tagged billboard quads carry CPU-computed pixel offsets the * GTE squash never saw — re-squash every X around the prim's anchor. */ @@ -1804,6 +2126,7 @@ static void gp0_exec_textured_quad(void) { if (ws_tagged_anchor(&ws_ax)) for (int i = 0; i < 4; i++) vx[i] = ws_scale_about(vx[i], ws_ax); } + ws_nw_backdrop_stretch_quad(vx, vy); /* full-frame 2D backdrop image stretch (no-op else) */ for (int i = 0; i < 4; i++) { vx[i] += draw_offset_x; @@ -1867,7 +2190,9 @@ static void gp0_exec_shaded_textured_tri(void) { uint16_t clut = (uint16_t)(gp0_cmd_buf[2] >> 16); uint16_t clut_x = (clut & 0x3F) * 16; uint16_t clut_y = (clut >> 6) & 0x1FF; - uint16_t tpage = (uint16_t)(gp0_cmd_buf[5] >> 16) & 0x1FF; + uint16_t tpage_word = (uint16_t)(gp0_cmd_buf[5] >> 16); + uint16_t tpage = tpage_word & 0x1FF; + set_tpage_from_poly(tpage_word); for (int i = 0; i < 3; i++) { vx[i] += draw_offset_x; @@ -1904,7 +2229,9 @@ static void gp0_exec_shaded_textured_quad(void) { uint16_t clut = (uint16_t)(gp0_cmd_buf[2] >> 16); uint16_t clut_x = (clut & 0x3F) * 16; uint16_t clut_y = (clut >> 6) & 0x1FF; - uint16_t tpage = (uint16_t)(gp0_cmd_buf[5] >> 16) & 0x1FF; + uint16_t tpage_word = (uint16_t)(gp0_cmd_buf[5] >> 16); + uint16_t tpage = tpage_word & 0x1FF; + set_tpage_from_poly(tpage_word); for (int i = 0; i < 4; i++) { vx[i] += draw_offset_x; @@ -1995,6 +2322,7 @@ static void gp0_exec_textured_rect(void) { ws_w = (int)ws_scale_len(w); } } + x0 += ws_nw_hud_shift(x0, w); /* native-wide HUD corner re-anchor (no-op else) */ x0 += draw_offset_x; y0 += draw_offset_y; setup_textured_draw(color24, semi_trans, raw_texture); @@ -2024,6 +2352,7 @@ static void gp0_exec_textured_8x8(void) { int32_t x0, y0; parse_vertex(gp0_cmd_buf[1], &x0, &y0); int ws_w = ws_sprt_fixed_transform(&x0, 8); + x0 += ws_nw_hud_shift(x0, 8); /* native-wide HUD corner re-anchor (no-op else) */ x0 += draw_offset_x; y0 += draw_offset_y; int u0 = gp0_cmd_buf[2] & 0xFF; int v0 = (gp0_cmd_buf[2] >> 8) & 0xFF; @@ -2057,7 +2386,11 @@ static void gp0_exec_textured_16x16(void) { int raw_texture = (gp0_cmd_buf[0] >> 24) & 1; int32_t x0, y0; parse_vertex(gp0_cmd_buf[1], &x0, &y0); + /* Never suppress MMX6 BG packets at a guessed finite-map boundary. The + * classifier cannot distinguish an authored layer entering the reveal from + * a stale ring slot; suppressing here caused the stage-start black flicker. */ int ws_w = ws_sprt_fixed_transform(&x0, 16); + x0 += ws_nw_hud_shift(x0, 16); /* native-wide HUD corner re-anchor (no-op else) */ x0 += draw_offset_x; y0 += draw_offset_y; int u0 = gp0_cmd_buf[2] & 0xFF; int v0 = (gp0_cmd_buf[2] >> 8) & 0xFF; @@ -2175,6 +2508,9 @@ static void gp0_exec_draw_offset(void) { uint32_t param = gp0_cmd_buf[0] & 0x00FFFFFFu; draw_offset_x = sign_extend(param & 0x7FFu, 11); draw_offset_y = sign_extend((param >> 11) & 0x7FFu, 11); + if (draw_offset_y < g_doff_min_this) g_doff_min_this = draw_offset_y; + if (draw_offset_y > g_doff_max_this) g_doff_max_this = draw_offset_y; + g_doff_cnt_this++; /* Native-wide mirrors framebuffer draws into a separate wide surface (canonical * VRAM stays faithful); the renderer needs the live draw offset to translate * into surface-local coords, so keep gr_set_draw_offset current. */ @@ -2463,6 +2799,44 @@ uint32_t gpu_get_opcode_count(uint8_t op) { return gp0_opcode_count[op]; } extern uint64_t s_frame_count; /* defined in debug_server.c */ extern uint32_t g_debug_last_store_pc; /* defined in debug_server.c */ +extern uint32_t g_debug_current_func_addr; /* defined in debug_server.c */ +uint32_t debug_guest_ra(void); /* accessor in debug_server.c (guest $ra) */ +uint32_t debug_guest_sp(void); /* accessor in debug_server.c (guest $sp) */ +extern uint8_t *memory_get_ram_ptr(void); /* raw 2MB main-RAM base (no lockstep) */ + +/* Bounded guest-stack unwind for VRAM-copy builder attribution. Scans the live + * stack for words that are valid return addresses (main-RAM code range, and the + * instruction at ret-8 is a jal/jalr), innermost first. A consumer skips the + * libgpu funnel band to name the game-level routine that issued the copy. Pure + * reads against the raw RAM array — no lockstep/observer pollution. */ +uint32_t g_gp0_last_copy_sp = 0; /* diag: raw guest $sp at last op-0x80 copy */ +static void gp0_capture_builder_chain(uint32_t out[6]) { + for (int i = 0; i < 6; i++) out[i] = 0; + uint8_t *ram = memory_get_ram_ptr(); + if (!ram) return; + const uint32_t RMASK = 0x001FFFFFu; /* 2 MB, mirror-folded */ + uint32_t rawsp = debug_guest_sp(); + g_gp0_last_copy_sp = rawsp; + if ((rawsp & 0x1FFFFFFFu) >= 0x00800000u) return; /* not in RAM/mirror */ + uint32_t sp = rawsp & RMASK & ~3u; /* fold to 2MB, word-align */ + /* Two-pass: prefer words that are genuine return addresses (call at ret-8), + * but if guest code validation yields none, fall back to any code-range + * stack word so the chain is never silently empty. */ + int found = 0; + for (int pass = 0; pass < 2 && found == 0; pass++) { + for (uint32_t off = 0; off + 4 <= 0x400u && found < 6; off += 4) { + uint32_t w; memcpy(&w, ram + ((sp + off) & RMASK), 4); + if ((w & 3u) || w < 0x80010000u || w >= 0x80120000u) continue; + if (pass == 0) { + uint32_t ins; memcpy(&ins, ram + ((w - 8u) & RMASK), 4); + uint32_t op = ins >> 26; + int is_call = (op == 3u) || (op == 0u && (ins & 0x3Fu) == 9u); + if (!is_call) continue; + } + out[found++] = w; + } + } +} #define GP0_RING_CAP (1u << 20) /* 1 048 576 entries */ /* GpuGp0RingEntry + GPU_GP0_RING_MAX_WORDS are public types in gpu.h. */ @@ -2481,12 +2855,16 @@ static void gp0_ring_record(const uint32_t *words, int n) { e->seq = (uint32_t)gp0_ring_seq; e->src_addr = gp0_cmd_source_addr; e->pc = g_debug_last_store_pc; + e->func = g_debug_current_func_addr; + e->ra = debug_guest_ra(); e->opcode = (uint8_t)((words[0] >> 24) & 0xFF); e->n_words = (uint8_t)(n > 255 ? 255 : (n < 0 ? 1 : n)); e->pad = 0; int copy_n = e->n_words > GPU_GP0_RING_MAX_WORDS ? GPU_GP0_RING_MAX_WORDS : e->n_words; for (int i = 0; i < copy_n; i++) e->cmd[i] = words[i]; for (int i = copy_n; i < GPU_GP0_RING_MAX_WORDS; i++) e->cmd[i] = 0; + if (e->opcode == 0x80) { gp0_capture_builder_chain(e->bld); e->csp = g_gp0_last_copy_sp; } + else { for (int i = 0; i < 6; i++) e->bld[i] = 0; e->csp = 0; } gp0_ring_head = (gp0_ring_head + 1) % GP0_RING_CAP; gp0_ring_seq++; } @@ -2859,7 +3237,7 @@ uint16_t gpu_vram_peek(int x, int y) { return gr_vram_read(x, y); } -void gpu_write_gp0(uint32_t val) { +static void gpu_write_gp0_body(uint32_t val) { gp0_write_count++; /* State: consuming pixel data for CPU→VRAM transfer */ @@ -2896,13 +3274,18 @@ void gpu_write_gp0(uint32_t val) { /* Transfer complete */ gp0_state = GP0_IDLE; vram_write_remaining = 0; + text_xlate_vram_upload(vram_write_x, vram_write_y, + vram_write_w, vram_write_h); return; } } } - if (--vram_write_remaining == 0) + if (--vram_write_remaining == 0) { gp0_state = GP0_IDLE; + text_xlate_vram_upload(vram_write_x, vram_write_y, + vram_write_w, vram_write_h); + } return; } @@ -3031,6 +3414,18 @@ void gpu_write_gp0(uint32_t val) { } } +/* Wall-time sampler bracket (phase_profile): tag GP0 command processing — + * rasterization / batching / VRAM transfer work on the emu thread — as its + * own phase so it is separable from the guest code that issued the write. + * Covers both the MMIO store chokepoint and DMA channel-2 feeds. */ +void gpu_write_gp0(uint32_t val) { + extern int g_exec_phase; + int prev_phase = g_exec_phase; + g_exec_phase = 4; + gpu_write_gp0_body(val); + g_exec_phase = prev_phase; +} + /* ---- GP1 write (0x1F801814 write) ---- */ static void gp1_reset(void) { @@ -3055,7 +3450,16 @@ static void gp1_ack_irq1(void) { static void gp1_display_enable(uint32_t val) { /* GP1(03h): Display enable — bit 0: 0=on, 1=off */ - display_disabled = val & 1; + uint32_t next_disabled = val & 1; + /* Native-wide reveal pixels have no guest-VRAM backing, so they cannot be + * reconstructed from the canonical framebuffer after a scene change. Treat + * the guest's ON->OFF transition as invalidation of those synthetic strips. + * This is deliberately transition-scoped: MMX6 retains valid background + * pixels across ordinary frames, so per-frame clearing pillarboxes gameplay. */ + if (ws_clear_reveal && next_disabled && !display_disabled && ws_mode == 2) { + ws_clear_all_reveal_margins(); + } + display_disabled = next_disabled; } static void gp1_dma_direction(uint32_t val) { diff --git a/runtime/src/gpu_gl_renderer.c b/runtime/src/gpu_gl_renderer.c index a0d0a419..6365709e 100644 --- a/runtime/src/gpu_gl_renderer.c +++ b/runtime/src/gpu_gl_renderer.c @@ -1138,6 +1138,39 @@ static int bd_prim_gate(const int *xs, int n) { return 1; } +/* ---- native-wide FAST path (skip redundant center mirror) ----------------- * + * The wide surface's CENTRE columns [g_wide_off, g_wide_off+native_w) are, by + * construction, identical to the canonical 4:3 framebuffer. So instead of + * re-rasterizing every primitive into the wide surface (the "mirror" pass — the + * dominant native-wide GPU cost, ~2x scene fill), we copy the canonical centre + * into the wide surface once at present (blit_wide_center_from_canonical), and + * the per-prim mirror only needs to produce the reveal MARGINS. Any prim/batch + * whose x-range is fully inside the 4:3 frame contributes nothing to the margins, + * so its mirror is skipped entirely. Correctness does not depend on the skip + * being precise: the centre is authoritatively overwritten by the blit, so the + * ONLY requirement is that a margin-reaching prim is NOT skipped — hence the + * conservative strict-inside test. 4:3 never runs any of this (g_wide_cur == 0). + * Toggle via gl_wide_fast for A/B; default ON. */ +static int s_wide_fast = 1; +void gl_renderer_set_wide_fast(int on) { s_wide_fast = on ? 1 : 0; } +int gl_renderer_get_wide_fast(void) { return s_wide_fast; } +static void wide_blit_center(GLuint wide_fbo, int base_x, int disp_y, int disp_h); /* def below */ +/* True if [lo,hi] (canonical draw-x) lies strictly inside the 4:3 frame, so the + * prim adds nothing to either reveal margin and its mirror can be skipped. */ +static int mirror_x_center_only(int lo, int hi) { + if (!s_wide_fast) return 0; + int base = g_wide_cur_base, native_w = g_wide_w - 2 * g_wide_off; + if (native_w <= 0) return 0; + return (lo >= base) && (hi < base + native_w); +} +static int mirror_geo_center_only(const int *xs, int n) { + if (!s_wide_fast) return 0; + int lo = xs[0], hi = xs[0]; + for (int i = 1; i < n; i++) { if (xs[i] < lo) lo = xs[i]; if (xs[i] > hi) hi = xs[i]; } + return mirror_x_center_only(lo, hi); +} +/* mirror_batch_center_only (textured-batch variant) is defined after s_tb below. */ + /* Set / clear the 2D-backdrop x-stretch for a wide-mirror draw, per the current * gate (s_bd_gate, set by the caller from bd_prim_gate / the batch gate). */ static void wide_set_bd_scale(GLint uScale, GLint uCenter) { @@ -1234,6 +1267,18 @@ static double s_cw_wide_ms = 0.0; /* CPU wall inside glb_wide_* entry points static int s_cw_batches = 0, s_cw_wide_sets = 0, s_cw_wide_cfgs = 0, s_cw_wide_clears = 0, s_cw_fbo_creates = 0, s_cw_flush_depth = 0; +/* Textured-batch variant of mirror_x_center_only: scan the queued verts' x + * (attr 0, stride TEXV). Defined here so s_tb / TEXV are in scope. */ +static int mirror_batch_center_only(int nverts) { + if (!s_wide_fast || nverts <= 0) return 0; + int lo = (int)s_tb[0], hi = (int)s_tb[0]; + for (int i = 1; i < nverts; i++) { + int x = (int)s_tb[i * TEXV]; + if (x < lo) lo = x; if (x > hi) hi = x; + } + return mirror_x_center_only(lo, hi); +} + static void flush_tex_batch(void) { if (s_tb_n == 0) return; int nverts = s_tb_n, semi = s_tb_semi; @@ -1255,7 +1300,12 @@ static void flush_tex_batch(void) { tex_batch_draw_passes(nverts, semi); - if (g_wide_cur && s_ws_ablate != 1) { /* native-wide mirror */ + /* Native-wide mirror — skipped for a batch fully inside the 4:3 frame (its + * centre content comes from the present-time canonical blit; nothing to add + * to the margins). A backdrop-stretched batch (s_tb_gate) widens past the + * frame, so it is never treated as centre-only. */ + if (g_wide_cur && s_ws_ablate != 1 && + !(s_tb_gate == 0 && mirror_batch_center_only(nverts))) { /* native-wide mirror */ int dx = wide_dx(); s_bd_gate = s_tb_gate; /* this batch is uniform-gate (flushed on change) */ gl_perf_mirror_begin(); @@ -1302,7 +1352,8 @@ static void gpu_geometry(GLenum mode, const int *xs, const int *ys, * bound). Geometry positions are unchanged on the host side — the x shift * is applied in the vertex shader via u_xoff, and the wider clip via * u_xhalf. Canonical pass above is untouched (u_xoff=0/u_xhalf=512). */ - if (g_wide_cur && !s_wide_suppress && s_ws_ablate != 1) { + if (g_wide_cur && !s_wide_suppress && s_ws_ablate != 1 && + !(!g_ws_bd_stretch_on && mirror_geo_center_only(xs, n))) { int dx = wide_dx(); s_bd_gate = bd_prim_gate(xs, n); /* flat prims are immediate -> gate per prim */ gl_perf_mirror_begin(); @@ -1329,34 +1380,9 @@ static void gpu_line(int x0,int y0,uint16_t c0,int x1,int y1,uint16_t c1,int sem gpu_geometry(GL_LINES, xs, ys, cs, 2, semi); } -/* Per-prim uv sampling bounds (inclusive), Beetle-PSX model: filtered - * neighbours (and S>1 interpolation overshoot) clamp to these so a sample - * never reads outside the prim's own texture rect. For axis-aligned (2D) - * uv mappings — any zero uv derivative — the max-uv vertex is an exclusive - * edge whose texel the PS1 DDA never samples, so back it off by one. If the - * uv range crosses a 256 wrap boundary the prim relies on page wrapping and - * the bounds widen to the full page (clamp disabled). */ -static void tri_uv_limits(const int *xs, const int *ys, - const int *us, const int *vs, int lim[4]) { - int lo_u = us[0], hi_u = us[0], lo_v = vs[0], hi_v = vs[0]; - for (int i = 1; i < 3; i++) { - if (us[i] < lo_u) lo_u = us[i]; if (us[i] > hi_u) hi_u = us[i]; - if (vs[i] < lo_v) lo_v = vs[i]; if (vs[i] > hi_v) hi_v = vs[i]; - } - long dudx = -(long)(ys[1]-ys[0])*us[2] - (long)(ys[2]-ys[1])*us[0] - (long)(ys[0]-ys[2])*us[1]; - long dvdx = -(long)(ys[1]-ys[0])*vs[2] - (long)(ys[2]-ys[1])*vs[0] - (long)(ys[0]-ys[2])*vs[1]; - long dudy = (long)(xs[1]-xs[0])*us[2] + (long)(xs[2]-xs[1])*us[0] + (long)(xs[0]-xs[2])*us[1]; - long dvdy = (long)(xs[1]-xs[0])*vs[2] + (long)(xs[2]-xs[1])*vs[0] + (long)(xs[0]-xs[2])*vs[1]; - if (dudx == 0 || dudy == 0 || dvdx == 0 || dvdy == 0) { - if (hi_u > lo_u) hi_u--; - if (hi_v > lo_v) hi_v--; - } - if ((lo_u >> 8) == (hi_u >> 8)) { lo_u &= 255; hi_u &= 255; } - else { lo_u = 0; hi_u = 255; } - if ((lo_v >> 8) == (hi_v >> 8)) { lo_v &= 255; hi_v &= 255; } - else { lo_v = 0; hi_v = 255; } - lim[0] = lo_u; lim[1] = lo_v; lim[2] = hi_u; lim[3] = hi_v; -} +/* Shared PS1 uv-sampling model (limits + mirrored-2D compensation) — one + * implementation for GL/VK/SW, see gpu_uv.h. */ +#include "gpu_uv.h" /* Textured triangle. Always two passes split by the per-texel STP bit so the * stencil (mask) write value is constant within each pass; the semi pass is @@ -1368,7 +1394,18 @@ static void gpu_textured_triangle(const int *xs, const int *ys, uint16_t clut_x, uint16_t clut_y, int rawtex, int semi, const int *lim) { int lim_buf[4]; - if (!lim) { tri_uv_limits(xs, ys, us, vs, lim_buf); lim = lim_buf; } + int uv_buf[6]; + if (!lim) { + /* Poly path: exact sampled bounds from the ORIGINAL uvs, then the + * center-sampling mirror compensation (rect prims arrive with their + * own precomputed lim and pre-bumped uvs). */ + int *mu = uv_buf, *mv = uv_buf + 3; + for (int i = 0; i < 3; i++) { mu[i] = us[i]; mv[i] = vs[i]; } + psx_uv_tri_limits(xs, ys, mu, mv, lim_buf); + psx_uv_tri_mirror_offset(xs, ys, mu, mv); + us = mu; vs = mv; + lim = lim_buf; + } s_scene_prims_tex++; int base_x = (texpage & 0xF) * 64; int base_y = ((texpage >> 4) & 1) * 256; @@ -1505,18 +1542,13 @@ static void gpu_textured_rect(int x,int y,int w,int h, if (w <= 0 || h <= 0) return; float mr=s_mod_r/255.0f, mg=s_mod_g/255.0f, mb=s_mod_b/255.0f; float col[9]={mr,mg,mb, mr,mg,mb, mr,mg,mb}; - /* uv sampling bounds: forward mappings sample [u0, u1-1] (u1 is the - * exclusive edge); mirrored ones keep the full inclusive range. Crossing - * a 256 boundary means page wrap — widen to the full page. */ + /* gpu.c routes axis-aligned MIRRORED quads (X/Y-flipped 2D sprites, + * e.g. right-facing MMX entities) through THIS path as scaled rects + * with u0>u1 / v0>v1 — they never reach the poly path. Exact bounds + * from the original corners, then the mirror bump (see gpu_uv.h). */ int lim[4]; - lim[0] = u0 < u1 ? u0 : u1; lim[2] = u0 < u1 ? u1 - 1 : u0; - lim[1] = v0 < v1 ? v0 : v1; lim[3] = v0 < v1 ? v1 - 1 : v0; - if (lim[2] < lim[0]) lim[2] = lim[0]; - if (lim[3] < lim[1]) lim[3] = lim[1]; - if ((lim[0] >> 8) == (lim[2] >> 8)) { lim[0] &= 255; lim[2] &= 255; } - else { lim[0] = 0; lim[2] = 255; } - if ((lim[1] >> 8) == (lim[3] >> 8)) { lim[1] &= 255; lim[3] &= 255; } - else { lim[1] = 0; lim[3] = 255; } + psx_uv_rect_limits(u0, v0, u1, v1, lim); + psx_uv_rect_mirror_offset(&u0, &v0, &u1, &v1); int xs1[3]={x, x+w, x}, ys1[3]={y, y, y+h}; int us1[3]={u0,u1,u0}, vs1[3]={v0,v0,v1}; gpu_textured_triangle(xs1,ys1,us1,vs1,col,tp,clut_x,clut_y,s_mod_raw,semi,lim); @@ -2222,7 +2254,7 @@ static void glb_wide_disable_target(void) { flush_tex_batch(); g_wide_cur = 0; } /* Mirror a framebuffer clear: fill the full wide width over [y, y+h) of the * surface for base_x, so the revealed margins are clean. Mirrors sw_wide_clear: * a scissored glClear with the 1555 color converted to RGBA8 (alpha = bit15). */ -static void glb_wide_clear(int base_x, int y, int h, uint16_t color) { +static void glb_wide_clear(int base_x, int y, int h, uint16_t color) { if (!s_raster_ok || s_ws_ablate == 1) return; double t0 = cw_ms(); s_cw_wide_clears++; flush_tex_batch(); @@ -2249,8 +2281,52 @@ static void glb_wide_clear(int base_x, int y, int h, uint16_t color) { glDisable(GL_SCISSOR_TEST); p_glBindFramebuffer(PSXGL_FRAMEBUFFER, 0); gl_perf_mirror_end(); - s_cw_wide_ms += cw_ms() - t0; -} + s_cw_wide_ms += cw_ms() - t0; +} + +/* Clear only the two synthetic reveal strips, preserving the centred canonical + * framebuffer. This is an opt-in transition cleanup driven by gpu.c. */ +static void glb_wide_clear_margins(int base_x, int y, int h, uint16_t color, int sides) { + if (!s_raster_ok || s_ws_ablate == 1 || g_wide_off <= 0) return; + double t0 = cw_ms(); s_cw_wide_clears++; + flush_tex_batch(); + GLuint fbo = wide_fbo_for(base_x); + if (!fbo) { s_cw_wide_ms += cw_ms() - t0; return; } + gl_perf_mirror_begin(); + int H = VRAM_H * s_scale; + int W = g_wide_w * s_scale; + int margin = g_wide_off * s_scale; + int y0 = y * s_scale, y1 = (y + h) * s_scale; + if (y0 < 0) y0 = 0; + if (y1 > H) y1 = H; + if (y1 <= y0 || margin * 2 >= W) { + gl_perf_mirror_end(); + s_cw_wide_ms += cw_ms() - t0; + return; + } + float r = (color & 0x1F) / 31.0f; + float g = ((color >> 5) & 0x1F) / 31.0f; + float b = ((color >> 10) & 0x1F) / 31.0f; + float a = (color >> 15) & 1 ? 1.0f : 0.0f; + p_glBindFramebuffer(PSXGL_FRAMEBUFFER, fbo); + glViewport(0, 0, W, H); + glEnable(GL_SCISSOR_TEST); + glClearColor(r, g, b, a); + glClearStencil((color >> 15) & 1); + glStencilMask(0xFF); + if (sides & 1) { + glScissor(0, y0, margin, y1 - y0); + glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); + } + if (sides & 2) { + glScissor(W - margin, y0, margin, y1 - y0); + glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); + } + glDisable(GL_SCISSOR_TEST); + p_glBindFramebuffer(PSXGL_FRAMEBUFFER, 0); + gl_perf_mirror_end(); + s_cw_wide_ms += cw_ms() - t0; +} /* Present source: read the wide FBO for the displayed buffer (base_x) into the * CPU present buffer as ARGB8888, byte-identical to sw_render_wide_display's @@ -2278,6 +2354,7 @@ static int glb_render_wide_display(uint32_t *out, int pitch, int base_x, * make sure all wide-FBO draws have completed before the readback. */ flush_tex_batch(); flush_cpu_upload(); + wide_blit_center(fbo, base_x, disp_y, disp_h); /* fast-path: authoritative centre before readback */ glFinish(); int W = g_wide_w * s_scale; @@ -2312,6 +2389,42 @@ static int glb_render_wide_display(uint32_t *out, int pitch, int base_x, return count; } +/* Dump the ENTIRE wide compositor surface for base_x (all double-buffer bands + + * both reveal margins), g_wide_w x VRAM_H at scale. Debug/inspection tool (TCP + * wide_full) — the GL analog of sw_wide_dump_full, so native-wide can be + * inspected without touching the game window. Runs the same authoritative + * centre blit first so the dump matches what present shows. Top-down, alpha=FF + * (matches sw_wide_dump_full / render_wide_display orientation). */ +static int glb_wide_dump_full(uint32_t *out, int cap_pixels, int *ow, int *oh, + int base_x) { + if (!s_raster_ok || !s_ctx || g_wide_w <= 0) return 0; + GLuint fbo = 0; + for (int i = 0; i < WIDE_MAX_SURF; i++) + if (s_wide_fbo[i] && s_wide_base[i] == base_x) { fbo = s_wide_fbo[i]; break; } + if (!fbo) return 0; + flush_tex_batch(); + flush_cpu_upload(); + wide_blit_center(fbo, base_x, 0, VRAM_H); /* authoritative centre (full height) */ + glFinish(); + int W = g_wide_w * s_scale; + int H = VRAM_H * s_scale; + if (cap_pixels > 0 && (long)W * H > cap_pixels) { H = cap_pixels / W; if (H <= 0) return 0; } + uint32_t *tmp = (uint32_t *)malloc((size_t)W * H * sizeof(uint32_t)); + if (!tmp) return 0; + p_glBindFramebuffer(PSXGL_READ_FRAMEBUFFER, fbo); + glPixelStorei(GL_PACK_ALIGNMENT, 4); + glReadPixels(0, 0, W, H, GL_BGRA, GL_UNSIGNED_BYTE, tmp); + p_glBindFramebuffer(PSXGL_READ_FRAMEBUFFER, 0); + /* Same orientation reasoning as glb_render_wide_display: glReadPixels row 0 = + * PS1 top scanline, so copy straight (top-down). */ + int count = 0; + for (int i = 0; i < W * H; i++) { out[i] = tmp[i] | 0xFF000000u; count++; } + free(tmp); + if (ow) *ow = W; + if (oh) *oh = H; + return count; +} + /* THE present path for 15-bit frames: blit the display region from the * authoritative hr FBO into a letterboxed rect. Deterministic — runs * every 15-bit frame regardless of what mix of ops produced it. @@ -2572,13 +2685,46 @@ void gl_renderer_present_vram(int disp_x, int disp_y, int w, int h, int linear, coh_record(GL_COH_PRESENT, disp_x, disp_y, disp_x + w - 1, disp_y + h - 1); } +/* Native-wide fast path: authoritatively copy the canonical 4:3 framebuffer + * into the wide surface's CENTRE columns [g_wide_off, g_wide_off+native_w) for + * the displayed Y band, so the per-prim mirror could skip every centre-only + * prim (the dominant native-wide GPU saving). The reveal margins were already + * produced by the mirror; this leaves them untouched. One FBO->FBO blit, + * x-translated by the reveal offset. No-op when s_wide_fast is off (then the + * mirror drew the full surface, as before). Shared by both present paths. */ +static void wide_blit_center(GLuint wide_fbo, int base_x, int disp_y, int disp_h) { + if (!s_wide_fast || g_wide_w <= 0) return; + int native_w = g_wide_w - 2 * g_wide_off; + if (native_w <= 0) return; + int S = s_scale; + (void)disp_y; (void)disp_h; + /* Copy the canonical framebuffer column into the wide surface CENTRE over the + * FULL VRAM height, not just the current display band [disp_y, disp_y+disp_h]. + * The wide surface holds BOTH vertical double-buffer bands (Ape flips + * display_y 0<->256), and the game's draw area / display band can differ per + * scene (the cityscape intro exposed rows outside disp_h). Copying the whole + * column makes the wide CENTRE bit-identical to what the full mirror would + * have drawn there for every band, so nothing the present reads is ever left + * black. The margins (x outside the centre) are untouched. */ + p_glBindFramebuffer(PSXGL_READ_FRAMEBUFFER, s_hr_fbo); + p_glBindFramebuffer(PSXGL_DRAW_FRAMEBUFFER, wide_fbo); + glDisable(GL_SCISSOR_TEST); + p_glBlitFramebuffer(base_x * S, 0, + (base_x + native_w) * S, VRAM_H * S, + g_wide_off * S, 0, + (g_wide_off + native_w) * S, VRAM_H * S, + GL_COLOR_BUFFER_BIT, GL_NEAREST); + p_glBindFramebuffer(PSXGL_READ_FRAMEBUFFER, 0); + p_glBindFramebuffer(PSXGL_DRAW_FRAMEBUFFER, 0); +} + /* GPU-direct native-wide present: blit the displayed buffer's wide FBO straight * to the window (no glReadPixels / glFinish CPU round-trip). The wide surface is * g_wide_w wide × VRAM_H tall (at scale S); present its [0,g_wide_w] × [disp_y, * disp_y+disp_h] region into the letterbox, V-flipped like present_vram (FBO y * bottom-origin → window top). Returns 0 if there's no wide surface for base_x * (caller falls back). disp_x is the displayed buffer base (the wide-surface key). */ -int gl_renderer_present_wide_fbo(int disp_x, int disp_y, int disp_h, int linear) { +int gl_renderer_present_wide_fbo(int disp_x, int disp_y, int disp_h, int linear) { if (!s_ctx || !s_raster_ok || g_wide_w <= 0) return 0; GLuint fbo = 0; for (int i = 0; i < WIDE_MAX_SURF; i++) @@ -2590,13 +2736,15 @@ int gl_renderer_present_wide_fbo(int disp_x, int disp_y, int disp_h, int linear) int ww = 0, wh = 0; SDL_GL_GetDrawableSize(s_win, &ww, &wh); int lx, ly, lw, lh; letterbox_rect(ww, wh, &lx, &ly, &lw, &lh); + int S = s_scale; - p_glBindFramebuffer(PSXGL_DRAW_FRAMEBUFFER, 0); + wide_blit_center(fbo, disp_x, disp_y, disp_h); /* fast-path: authoritative centre */ + + p_glBindFramebuffer(PSXGL_DRAW_FRAMEBUFFER, 0); glDisable(GL_SCISSOR_TEST); glViewport(0, 0, ww, wh); glClearColor(0.f, 0.f, 0.f, 1.f); glClear(GL_COLOR_BUFFER_BIT); p_glBindFramebuffer(PSXGL_READ_FRAMEBUFFER, fbo); - int S = s_scale; /* Source: full wide width [0, g_wide_w], displayed Y band [disp_y, +disp_h]. * V-flip the dst (ly+lh .. ly) so the top scanline lands at the rect top. */ p_glBlitFramebuffer(0, disp_y * S, g_wide_w * S, (disp_y + disp_h) * S, @@ -2633,8 +2781,10 @@ static const GpuRenderBackend GL_BACKEND = { .wide_configure = glb_wide_configure, .wide_set_target = glb_wide_set_target, .wide_disable_target = glb_wide_disable_target, - .wide_clear = glb_wide_clear, + .wide_clear = glb_wide_clear, + .wide_clear_margins = glb_wide_clear_margins, .render_wide_display = glb_render_wide_display, + .wide_dump_full = glb_wide_dump_full, }; const GpuRenderBackend *gl_backend_get(void) { return &GL_BACKEND; } diff --git a/runtime/src/gpu_render.c b/runtime/src/gpu_render.c index 1811bd98..cae37953 100644 --- a/runtime/src/gpu_render.c +++ b/runtime/src/gpu_render.c @@ -49,7 +49,9 @@ static const GpuRenderBackend SW_BACKEND = { .wide_set_target = sw_wide_set_target, .wide_disable_target = sw_wide_disable_target, .wide_clear = sw_wide_clear, + .wide_clear_margins = sw_wide_clear_margins, .render_wide_display = sw_render_wide_display, + .wide_dump_full = sw_wide_dump_full, }; /* Supplied by gpu_gl_renderer.c; returns NULL until the GL backend is ready. */ @@ -163,9 +165,18 @@ void gr_wide_disable_target(void) { void gr_wide_clear(int base_x, int y, int h, uint16_t color) { if (g_b->wide_clear) g_b->wide_clear(base_x, y, h, color); } +void gr_wide_clear_margins(int base_x, int y, int h, uint16_t color, int sides) { + if (g_b->wide_clear_margins) + g_b->wide_clear_margins(base_x, y, h, color, sides); +} int gr_render_wide_display(uint32_t *out, int pitch, int base_x, int disp_y, int disp_h) { if (g_b->render_wide_display) return g_b->render_wide_display(out, pitch, base_x, disp_y, disp_h); return 0; } +int gr_wide_dump_full(uint32_t *out, int cap_pixels, int *ow, int *oh, int base_x) { + if (g_b->wide_dump_full) + return g_b->wide_dump_full(out, cap_pixels, ow, oh, base_x); + return 0; +} diff --git a/runtime/src/gpu_sw_renderer.c b/runtime/src/gpu_sw_renderer.c index 0a2f8c00..21b22477 100644 --- a/runtime/src/gpu_sw_renderer.c +++ b/runtime/src/gpu_sw_renderer.c @@ -419,47 +419,19 @@ static uint16_t texel_fetch_bilinear(float fu, float fv, uint16_t texpage, return (uint16_t)(r | (g << 5) | (b << 10) | (stp ? 0x8000 : 0)); } -/* uv sampling bounds for a textured triangle (see g_uv_lim): min/max of the - * vertex uvs; for axis-aligned (2D) mappings — any zero uv derivative — the - * max-uv vertex is an exclusive edge whose texel the DDA never samples, so - * back it off by one. Crossing a 256 boundary means page wrap: widen to the - * full page (clamp disabled). */ +/* uv sampling bounds (see g_uv_lim): the shared PS1 uv model in gpu_uv.h. + * The software DDA truncates exactly like the PS1, so unlike the GL/VK + * backends NO mirrored-2D compensation is applied here — these bounds only + * feed the bilinear-filter neighbour clamp. */ +#include "gpu_uv.h" + static void sw_tri_uv_limits(const int *xs, const int *ys, const int *us, const int *vs) { - int lo_u = us[0], hi_u = us[0], lo_v = vs[0], hi_v = vs[0]; - for (int i = 1; i < 3; i++) { - if (us[i] < lo_u) lo_u = us[i]; if (us[i] > hi_u) hi_u = us[i]; - if (vs[i] < lo_v) lo_v = vs[i]; if (vs[i] > hi_v) hi_v = vs[i]; - } - long dudx = -(long)(ys[1]-ys[0])*us[2] - (long)(ys[2]-ys[1])*us[0] - (long)(ys[0]-ys[2])*us[1]; - long dvdx = -(long)(ys[1]-ys[0])*vs[2] - (long)(ys[2]-ys[1])*vs[0] - (long)(ys[0]-ys[2])*vs[1]; - long dudy = (long)(xs[1]-xs[0])*us[2] + (long)(xs[2]-xs[1])*us[0] + (long)(xs[0]-xs[2])*us[1]; - long dvdy = (long)(xs[1]-xs[0])*vs[2] + (long)(xs[2]-xs[1])*vs[0] + (long)(xs[0]-xs[2])*vs[1]; - if (dudx == 0 || dudy == 0 || dvdx == 0 || dvdy == 0) { - if (hi_u > lo_u) hi_u--; - if (hi_v > lo_v) hi_v--; - } - if ((lo_u >> 8) == (hi_u >> 8)) { lo_u &= 255; hi_u &= 255; } - else { lo_u = 0; hi_u = 255; } - if ((lo_v >> 8) == (hi_v >> 8)) { lo_v &= 255; hi_v &= 255; } - else { lo_v = 0; hi_v = 255; } - g_uv_lim[0] = lo_u; g_uv_lim[1] = lo_v; g_uv_lim[2] = hi_u; g_uv_lim[3] = hi_v; + psx_uv_tri_limits(xs, ys, us, vs, g_uv_lim); } -/* uv sampling bounds for rect prims: forward mappings sample [u0, u1-1] - * (u1 is the exclusive edge); mirrored ones keep the full inclusive range. */ static void sw_rect_uv_limits(int u0, int v0, int u1, int v1) { - int lim[4]; - lim[0] = u0 < u1 ? u0 : u1; lim[2] = u0 < u1 ? u1 - 1 : u0; - lim[1] = v0 < v1 ? v0 : v1; lim[3] = v0 < v1 ? v1 - 1 : v0; - if (lim[2] < lim[0]) lim[2] = lim[0]; - if (lim[3] < lim[1]) lim[3] = lim[1]; - if ((lim[0] >> 8) == (lim[2] >> 8)) { lim[0] &= 255; lim[2] &= 255; } - else { lim[0] = 0; lim[2] = 255; } - if ((lim[1] >> 8) == (lim[3] >> 8)) { lim[1] &= 255; lim[3] &= 255; } - else { lim[1] = 0; lim[3] = 255; } - g_uv_lim[0] = lim[0]; g_uv_lim[1] = lim[1]; - g_uv_lim[2] = lim[2]; g_uv_lim[3] = lim[3]; + psx_uv_rect_limits(u0, v0, u1, v1, g_uv_lim); } /* ------------------------------------------------------------------ */ @@ -1620,6 +1592,26 @@ void sw_wide_clear(int base_x, int y, int h, uint16_t color) { } } +/* Clear only the synthetic columns outside the centred canonical framebuffer. + * This provides a defined black void without disturbing guest-owned pixels. */ +void sw_wide_clear_margins(int base_x, int y, int h, uint16_t color, int sides) { + uint16_t *surf = wide_surf_for(base_x); + if (!surf) return; + int s = g_scale; + int W = g_wide_w * s; + int H = VRAM_HEIGHT * s; + int margin = g_wide_off * s; + int y0 = y * s, y1 = (y + h) * s; + if (margin <= 0 || margin * 2 >= W) return; + if (y0 < 0) y0 = 0; + if (y1 > H) y1 = H; + for (int row = y0; row < y1; row++) { + uint16_t *dst = surf + (size_t)row * W; + if (sides & 1) for (int col = 0; col < margin; col++) dst[col] = color; + if (sides & 2) for (int col = W - margin; col < W; col++) dst[col] = color; + } +} + /* Present source: convert the wide surface for the displayed buffer (base_x) to * ARGB. Output is (g_wide_w*scale) wide × (disp_h*scale) tall. Returns 0 if no * surface exists for base_x (caller falls back to the canonical present). */ diff --git a/runtime/src/gpu_uv.h b/runtime/src/gpu_uv.h new file mode 100644 index 00000000..0c2d8568 --- /dev/null +++ b/runtime/src/gpu_uv.h @@ -0,0 +1,114 @@ +/* gpu_uv.h — the ONE shared PS1 uv-sampling model for all render backends. + * + * The PS1 DDA latches interpolants at each pixel's TOP-LEFT corner and + * truncates, so along any axis-aligned 2D mapping the prim's exclusive + * raster edge (+x right / +y bottom) never samples its end texel. Which uv + * extreme that is depends on the mapping direction: forward (uv increasing + * toward the exclusive edge) never samples max-uv; mirrored (uv decreasing + * — X/Y-flipped 2D sprites) never samples min-uv. The helpers here encode + * that model ONCE: + * + * psx_uv_tri_limits / psx_uv_rect_limits + * Exact inclusive sampled texel range for the mapping direction, on the + * prim's ORIGINAL uv values: forward [lo, hi-1], mirrored [lo+1, hi]. + * Crossing a 256 boundary means the prim relies on page wrapping — the + * bounds widen to the full page (clamp disabled). Used as sampling + * clamps (bilinear neighbours, S>1 interpolation overshoot). + * + * psx_uv_tri_mirror_offset / psx_uv_rect_mirror_offset + * The Beetle-PSX / parallel-psx Calc_UVOffsets_Adjust_Verts model for + * CENTER-SAMPLED rasterizers (GL/VK). Their sample-grid shift + * (u_shift = 0.5/S - 1/64) makes floor(uv) land on the exact PS1 texel + * for forward mappings, but mirrored ones interpolate 1/64 SHORT of + * each integer and floor one texel LOW — painting the cel's + * never-sampled edge column/row as detached 1px slivers (MMX6 sprite + * slivers; pink line under rolling wheels). Compensate by bumping the + * prim's uv +1 along each decreasing axis. The software rasterizer's + * own DDA truncates exactly like the PS1 and must NOT apply this. + * + * Derivative direction is area2-normalized (winding-independent); diagonal + * (3D-ish) mappings — both derivatives nonzero on an axis — get no + * compensation and no back-off tightening on that axis. Like Beetle, a rare + * 3D poly that happens to be axis-aligned accepts a one-texel shift in + * exchange for correct 2D sprites. + * + * History: GL, VK and SW each carried a private copy of this math; the + * copies drifted (the mirrored back-off bug lived only in the GPU backends) + * and the fix then had to be hand-mirrored into five sites. One model, one + * file, all backends. */ + +#ifndef PSXRECOMP_GPU_UV_H +#define PSXRECOMP_GPU_UV_H + +/* Signed uv step along each screen axis, area2-scaled (only signs matter; + * area2's sign normalizes winding). du/dv are the axis-aligned direction: + * the nonzero-vs-zero derivative pair, or 0 when the mapping is diagonal. */ +static inline void psx_uv_tri_dirs(const int *xs, const int *ys, + const int *us, const int *vs, + long *du, long *dv) { + long dudx = -(long)(ys[1]-ys[0])*us[2] - (long)(ys[2]-ys[1])*us[0] - (long)(ys[0]-ys[2])*us[1]; + long dvdx = -(long)(ys[1]-ys[0])*vs[2] - (long)(ys[2]-ys[1])*vs[0] - (long)(ys[0]-ys[2])*vs[1]; + long dudy = (long)(xs[1]-xs[0])*us[2] + (long)(xs[2]-xs[1])*us[0] + (long)(xs[0]-xs[2])*us[1]; + long dvdy = (long)(xs[1]-xs[0])*vs[2] + (long)(xs[2]-xs[1])*vs[0] + (long)(xs[0]-xs[2])*vs[1]; + long area2 = (long)(xs[1]-xs[0])*(ys[2]-ys[0]) - (long)(xs[2]-xs[0])*(ys[1]-ys[0]); + if (area2 == 0) { *du = 0; *dv = 0; return; } + *du = dudx == 0 ? dudy : (dudy == 0 ? dudx : 0); + *dv = dvdx == 0 ? dvdy : (dvdy == 0 ? dvdx : 0); + if (area2 < 0) { *du = -*du; *dv = -*dv; } +} + +/* Pre-wrap one axis' exact sampled range into lim_lo/lim_hi. dir > 0 = + * forward (hi exclusive), dir < 0 = mirrored (lo exclusive), 0 = diagonal + * (keep full min..max). */ +static inline void psx_uv_axis_limits(int lo, int hi, long dir, + int *lim_lo, int *lim_hi) { + if (dir > 0 && hi > lo) hi--; + if (dir < 0 && hi > lo) lo++; + if ((lo >> 8) == (hi >> 8)) { lo &= 255; hi &= 255; } + else { lo = 0; hi = 255; } + *lim_lo = lo; *lim_hi = hi; +} + +/* Inclusive sampled uv bounds for a textured triangle, from its ORIGINAL + * (uncompensated) vertex uvs. lim = {lo_u, lo_v, hi_u, hi_v}. */ +static inline void psx_uv_tri_limits(const int *xs, const int *ys, + const int *us, const int *vs, int lim[4]) { + int lo_u = us[0], hi_u = us[0], lo_v = vs[0], hi_v = vs[0]; + for (int i = 1; i < 3; i++) { + if (us[i] < lo_u) lo_u = us[i]; if (us[i] > hi_u) hi_u = us[i]; + if (vs[i] < lo_v) lo_v = vs[i]; if (vs[i] > hi_v) hi_v = vs[i]; + } + long du, dv; + psx_uv_tri_dirs(xs, ys, us, vs, &du, &dv); + psx_uv_axis_limits(lo_u, hi_u, du, &lim[0], &lim[2]); + psx_uv_axis_limits(lo_v, hi_v, dv, &lim[1], &lim[3]); +} + +/* Inclusive sampled uv bounds for a rect prim, from its ORIGINAL uv corners + * (u0/v0 at the top-left raster corner; u1/v1 at the exclusive corner — + * u1 < u0 / v1 < v0 means mirrored). */ +static inline void psx_uv_rect_limits(int u0, int v0, int u1, int v1, int lim[4]) { + int lo_u = u0 < u1 ? u0 : u1, hi_u = u0 < u1 ? u1 : u0; + int lo_v = v0 < v1 ? v0 : v1, hi_v = v0 < v1 ? v1 : v0; + psx_uv_axis_limits(lo_u, hi_u, (long)(u1 - u0), &lim[0], &lim[2]); + psx_uv_axis_limits(lo_v, hi_v, (long)(v1 - v0), &lim[1], &lim[3]); +} + +/* Mirrored-2D compensation for center-sampled rasterizers (GL/VK ONLY — + * never the software DDA): bump the uv +1 along each decreasing axis. + * Triangle form mutates the 3-vertex uv arrays in place. */ +static inline void psx_uv_tri_mirror_offset(const int *xs, const int *ys, + int *us, int *vs) { + long du, dv; + psx_uv_tri_dirs(xs, ys, us, vs, &du, &dv); + if (du < 0) { us[0]++; us[1]++; us[2]++; } + if (dv < 0) { vs[0]++; vs[1]++; vs[2]++; } +} + +/* Rect form: bump each mirrored axis' corner pair. */ +static inline void psx_uv_rect_mirror_offset(int *u0, int *v0, int *u1, int *v1) { + if (*u1 < *u0) { (*u0)++; (*u1)++; } + if (*v1 < *v0) { (*v0)++; (*v1)++; } +} + +#endif /* PSXRECOMP_GPU_UV_H */ diff --git a/runtime/src/gpu_vk_renderer.c b/runtime/src/gpu_vk_renderer.c index a556182b..b3287ee0 100644 --- a/runtime/src/gpu_vk_renderer.c +++ b/runtime/src/gpu_vk_renderer.c @@ -2515,27 +2515,9 @@ static void flush_tex_batch(void) { s_gpu_dirty = 1; } -/* Per-prim uv sampling bounds (inclusive), Beetle-PSX model — see the GL - * backend's tri_uv_limits. */ -static void tri_uv_limits(const int *xs, const int *ys, - const int *us, const int *vs, int lim[4]) { - int lo_u = us[0], hi_u = us[0], lo_v = vs[0], hi_v = vs[0]; - for (int i = 1; i < 3; i++) { - if (us[i] < lo_u) lo_u = us[i]; if (us[i] > hi_u) hi_u = us[i]; - if (vs[i] < lo_v) lo_v = vs[i]; if (vs[i] > hi_v) hi_v = vs[i]; - } - long dudx = -(long)(ys[1]-ys[0])*us[2] - (long)(ys[2]-ys[1])*us[0] - (long)(ys[0]-ys[2])*us[1]; - long dvdx = -(long)(ys[1]-ys[0])*vs[2] - (long)(ys[2]-ys[1])*vs[0] - (long)(ys[0]-ys[2])*vs[1]; - long dudy = (long)(xs[1]-xs[0])*us[2] + (long)(xs[2]-xs[1])*us[0] + (long)(xs[0]-xs[2])*us[1]; - long dvdy = (long)(xs[1]-xs[0])*vs[2] + (long)(xs[2]-xs[1])*vs[0] + (long)(xs[0]-xs[2])*vs[1]; - if (dudx == 0 || dudy == 0 || dvdx == 0 || dvdy == 0) { - if (hi_u > lo_u) hi_u--; - if (hi_v > lo_v) hi_v--; - } - if ((lo_u >> 8) == (hi_u >> 8)) { lo_u &= 255; hi_u &= 255; } else { lo_u = 0; hi_u = 255; } - if ((lo_v >> 8) == (hi_v >> 8)) { lo_v &= 255; hi_v &= 255; } else { lo_v = 0; hi_v = 255; } - lim[0] = lo_u; lim[1] = lo_v; lim[2] = hi_u; lim[3] = hi_v; -} +/* Shared PS1 uv-sampling model (limits + mirrored-2D compensation) — one + * implementation for GL/VK/SW, see gpu_uv.h. */ +#include "gpu_uv.h" /* Append a textured triangle to the batch (per-prim state in the vertex; flush * first if the batch keys differ or the raw mirror needs packing). col = 3x RGB @@ -2546,7 +2528,18 @@ static void gpu_textured_triangle(const int *xs, const int *ys, const int *us, c int semi, const int *lim) { if (!s_ready) return; int lim_buf[4]; - if (!lim) { tri_uv_limits(xs, ys, us, vs, lim_buf); lim = lim_buf; } + int uv_buf[6]; + if (!lim) { + /* Poly path: exact sampled bounds from the ORIGINAL uvs, then the + * center-sampling mirror compensation (rect prims arrive with their + * own precomputed lim and pre-bumped uvs). */ + int *mu = uv_buf, *mv = uv_buf + 3; + for (int i = 0; i < 3; i++) { mu[i] = us[i]; mv[i] = vs[i]; } + psx_uv_tri_limits(xs, ys, mu, mv, lim_buf); + psx_uv_tri_mirror_offset(xs, ys, mu, mv); + us = mu; vs = mv; + lim = lim_buf; + } int base_x = (texpage & 0xF) * 64; int base_y = ((texpage >> 4) & 1) * 256; int depth = (texpage >> 7) & 3; if (depth > 2) depth = 2; @@ -2602,13 +2595,12 @@ static void gpu_textured_rect(int x,int y,int w,int h, int u0,int v0,int u1,int if (w <= 0 || h <= 0) return; float mr = s_mod_r/255.0f, mg = s_mod_g/255.0f, mb = s_mod_b/255.0f; float col[9] = { mr,mg,mb, mr,mg,mb, mr,mg,mb }; + /* Mirrored quads arrive here as scaled rects with u0>u1 / v0>v1 (see + * the GL backend): exact bounds from the original corners, then the + * mirror bump (gpu_uv.h). */ int lim[4]; - lim[0] = u0 < u1 ? u0 : u1; lim[2] = u0 < u1 ? u1 - 1 : u0; - lim[1] = v0 < v1 ? v0 : v1; lim[3] = v0 < v1 ? v1 - 1 : v0; - if (lim[2] < lim[0]) lim[2] = lim[0]; - if (lim[3] < lim[1]) lim[3] = lim[1]; - if ((lim[0] >> 8) == (lim[2] >> 8)) { lim[0] &= 255; lim[2] &= 255; } else { lim[0] = 0; lim[2] = 255; } - if ((lim[1] >> 8) == (lim[3] >> 8)) { lim[1] &= 255; lim[3] &= 255; } else { lim[1] = 0; lim[3] = 255; } + psx_uv_rect_limits(u0, v0, u1, v1, lim); + psx_uv_rect_mirror_offset(&u0, &v0, &u1, &v1); int xs1[3]={x, x+w, x}, ys1[3]={y, y, y+h}; int us1[3]={u0,u1,u0}, vs1[3]={v0,v0,v1}; gpu_textured_triangle(xs1,ys1,us1,vs1,col,tp,clut_x,clut_y,s_mod_raw,semi,lim); @@ -2748,6 +2740,48 @@ static void vkb_wide_clear(int base_x, int y, int h, uint16_t color) { end_oneshot(cb); } +/* Clear only the two synthetic reveal strips, leaving the guest-owned centre + * intact. Mirrors the software/OpenGL opt-in transition cleanup. */ +static void vkb_wide_clear_margins(int base_x, int y, int h, uint16_t color, int sides) { + if (!s_ready || s_wide_w <= 0 || s_wide_offset <= 0) return; + flush_tex_batch(); flush_geometry(); + int i = wide_surf_for(base_x); + if (i < 0) return; + s_perf_cur.wide_clears++; + int S = s_scale, H = VRAM_H * S; + int W = s_wide_w * S, margin = s_wide_offset * S; + int y0 = y * S, y1 = (y + h) * S; + if (y0 < 0) y0 = 0; + if (y1 > H) y1 = H; + if (y1 <= y0 || margin * 2 >= W) return; + VkCommandBuffer cb = begin_oneshot(); + img_to(cb, s_wide_img[i], &s_wide_layout[i], VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL); + VkRenderPassBeginInfo rp = { VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO }; + rp.renderPass = s_rpass; rp.framebuffer = s_wide_fb[i]; + rp.renderArea.extent.width = (uint32_t)W; + rp.renderArea.extent.height = (uint32_t)H; + p_vkCmdBeginRenderPass(cb, &rp, VK_SUBPASS_CONTENTS_INLINE); + VkClearAttachment ca[2] = {0}; + ca[0].aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; + ca[0].colorAttachment = 0; + ca[0].clearValue.color.float32[0] = (float)((color) & 0x1F) / 31.0f; + ca[0].clearValue.color.float32[1] = (float)((color >> 5) & 0x1F) / 31.0f; + ca[0].clearValue.color.float32[2] = (float)((color >> 10) & 0x1F) / 31.0f; + ca[0].clearValue.color.float32[3] = (color >> 15) & 1 ? 1.0f : 0.0f; + ca[1].aspectMask = VK_IMAGE_ASPECT_STENCIL_BIT; + ca[1].clearValue.depthStencil.depth = 0.0f; + ca[1].clearValue.depthStencil.stencil = (color >> 15) & 1; + VkClearRect cr[2]; + uint32_t ncr = 0; + if (sides & 1) + cr[ncr++] = (VkClearRect){ { { 0, y0 }, { (uint32_t)margin, (uint32_t)(y1 - y0) } }, 0, 1 }; + if (sides & 2) + cr[ncr++] = (VkClearRect){ { { W - margin, y0 }, { (uint32_t)margin, (uint32_t)(y1 - y0) } }, 0, 1 }; + if (ncr) p_vkCmdClearAttachments(cb, 2, ca, ncr, cr); + p_vkCmdEndRenderPass(cb); + end_oneshot(cb); +} + /* Present source (CPU): read the wide surface band for the displayed buffer * into an ARGB8888 buffer, byte-compatible with sw_render_wide_display — used * by the shared CPU present fallback and the ws debug dump. The GPU-direct @@ -2829,6 +2863,7 @@ static const GpuRenderBackend VK_BACKEND = { .wide_set_target = vkb_wide_set_target, .wide_disable_target = vkb_wide_disable_target, .wide_clear = vkb_wide_clear, + .wide_clear_margins = vkb_wide_clear_margins, .render_wide_display = vkb_render_wide_display, }; diff --git a/runtime/src/gte.cpp b/runtime/src/gte.cpp index 9bc19765..9f9782f7 100644 --- a/runtime/src/gte.cpp +++ b/runtime/src/gte.cpp @@ -2,6 +2,8 @@ #include "cpu_state.h" #include <algorithm> #include <cstdlib> +#include <cstdio> +#include <cstring> namespace PSXRecomp { namespace GTE { @@ -10,23 +12,70 @@ namespace GTE { // Common helpers // --------------------------------------------------------------------------- -// Perspective division: H * 0x20000 / SZ3, result capped at 0x1FFFF +// Perspective division: H / SZ3, scaled, saturated to 17 bits. +// +// The real PS1 GTE does NOT compute an exact H*0x20000/SZ3. It uses an +// Unsigned Newton-Raphson (UNR) reciprocal approximation driven by a 257-entry +// seed table (documented in PSX-SPX "GTE Division Inaccuracy"; identical to the +// mednafen/Beetle oracle in beetle-psx/mednafen/psx/gte.cpp). Exact division +// diverges from hardware by +/-1 (occasionally up to a few units) on ~25% of +// inputs, which is enough to flip games' distance/intensity threshold branches +// (e.g. Ape Escape's additive-glow CLUT semi-transparency bit). This is the +// faithful hardware algorithm, shared by every RTPS/RTPT caller. +static uint8_t s_gte_div_table[0x101]; +static bool s_gte_div_table_init = false; +static void gte_init_div_table() { + for (uint32_t divisor = 0x8000; divisor < 0x10000; divisor += 0x80) { + uint32_t xa = 512; + for (unsigned i = 1; i < 5; i++) + xa = (xa * (1024u * 512u - ((divisor >> 7) * xa))) >> 18; + s_gte_div_table[(divisor >> 7) & 0xFF] = + (uint8_t)(((xa + 1) >> 1) - 0x101); + } + s_gte_div_table[0x100] = s_gte_div_table[0xFF]; + s_gte_div_table_init = true; +} +static int32_t gte_calc_recip(uint16_t divisor) { + int32_t x = 0x101 + s_gte_div_table[(((divisor & 0x7FFF) + 0x40) >> 7)]; + int32_t tmp = (((int32_t)divisor * -x) + 0x80) >> 8; + int32_t tmp2 = ((x * (131072 + tmp)) + 0x80) >> 8; + return tmp2; +} +// count leading zeros of a 16-bit value (0 -> 16) +static inline unsigned gte_clz16(uint16_t v) { + unsigned n = 0; + for (int b = 15; b >= 0; --b) { if (v & (1u << b)) break; ++n; } + return n; +} static int32_t gte_divide(uint16_t H, uint16_t SZ3, uint32_t& FLAG) { - if (SZ3 == 0) { + if (!s_gte_div_table_init) gte_init_div_table(); + // Hardware: overflow flag + saturate when 2*SZ3 <= H (includes SZ3 == 0). + if ((uint32_t)SZ3 * 2 <= (uint32_t)H) { FLAG |= FLAG_DIV_OVF; return 0x1FFFF; } - // (H * 0x20000 / SZ3 + 1) / 2 — correct PS1 formula with rounding - int64_t result = (((int64_t)H * 0x20000) / SZ3 + 1) / 2; - if (result > 0x1FFFF) { - FLAG |= FLAG_DIV_OVF; - return 0x1FFFF; - } - return static_cast<int32_t>(result); + unsigned shift_bias = gte_clz16(SZ3); + uint32_t dividend = (uint32_t)H << shift_bias; + uint32_t divisor = (uint32_t)SZ3 << shift_bias; + uint32_t result = (uint32_t)(((uint64_t)dividend * + (uint32_t)gte_calc_recip((uint16_t)(divisor | 0x8000)) + + 32768) >> 16); + if (result > 0x1FFFF) result = 0x1FFFF; // 17-bit saturate (no flag; matches hw path) + return (int32_t)result; } +// lm (bit 10) selects the lower clamp bound of every IR write in the +// lighting/depth-cue ops: 0 → -0x8000, 1 → 0. sf (bit 19) selects the +// accumulator shift. Hardware takes BOTH from the instruction word; these +// helpers used to hardcode lm=1, which zeroed every negative component — +// Crash Bash lerps signed VERTICES through INTPL (lm=0), so its characters +// collapsed onto their anchor point on every tween frame (menu strobe bug). +static inline bool gte_instr_lm(uint32_t instr) { return (instr & (1u << 10)) != 0; } +static inline int gte_instr_sf(uint32_t instr) { return (instr & (1u << 19)) ? 12 : 0; } + // Transform a vertex by Light matrix → IR1/IR2/IR3 (step 1 of lighting) -static void light_transform(GTEState* gte, int16_t* V) { +static void light_transform(GTEState* gte, int16_t* V, uint32_t instr) { + const bool lm = gte_instr_lm(instr); int64_t mac1 = ((int64_t)gte->L[0][0] * V[0] + (int64_t)gte->L[0][1] * V[1] + (int64_t)gte->L[0][2] * V[2]) >> 12; @@ -39,13 +88,14 @@ static void light_transform(GTEState* gte, int16_t* V) { gte->MAC1 = static_cast<int32_t>(mac1); gte->MAC2 = static_cast<int32_t>(mac2); gte->MAC3 = static_cast<int32_t>(mac3); - gte->IR1 = gte->saturate_ir(gte->MAC1, 1, true); - gte->IR2 = gte->saturate_ir(gte->MAC2, 2, true); - gte->IR3 = gte->saturate_ir(gte->MAC3, 3, true); + gte->IR1 = gte->saturate_ir(gte->MAC1, 1, lm); + gte->IR2 = gte->saturate_ir(gte->MAC2, 2, lm); + gte->IR3 = gte->saturate_ir(gte->MAC3, 3, lm); } // Apply light color matrix + background color to IR → IR (step 2 of lighting) -static void light_color(GTEState* gte) { +static void light_color(GTEState* gte, uint32_t instr) { + const bool lm = gte_instr_lm(instr); int64_t mac1 = ((int64_t)gte->BK[0] << 12) + (int64_t)gte->LC[0][0] * gte->IR1 + (int64_t)gte->LC[0][1] * gte->IR2 + @@ -61,13 +111,14 @@ static void light_color(GTEState* gte) { gte->MAC1 = static_cast<int32_t>(mac1 >> 12); gte->MAC2 = static_cast<int32_t>(mac2 >> 12); gte->MAC3 = static_cast<int32_t>(mac3 >> 12); - gte->IR1 = gte->saturate_ir(gte->MAC1, 1, true); - gte->IR2 = gte->saturate_ir(gte->MAC2, 2, true); - gte->IR3 = gte->saturate_ir(gte->MAC3, 3, true); + gte->IR1 = gte->saturate_ir(gte->MAC1, 1, lm); + gte->IR2 = gte->saturate_ir(gte->MAC2, 2, lm); + gte->IR3 = gte->saturate_ir(gte->MAC3, 3, lm); } // Multiply IR by RGBC color and push to RGB FIFO (step 3 of color output) -static void color_output(GTEState* gte) { +static void color_output(GTEState* gte, uint32_t instr) { + const bool lm = gte_instr_lm(instr); uint8_t r0 = (gte->RGBC >> 0) & 0xFF; uint8_t g0 = (gte->RGBC >> 8) & 0xFF; uint8_t b0 = (gte->RGBC >> 16) & 0xFF; @@ -77,32 +128,45 @@ static void color_output(GTEState* gte) { gte->MAC1 = static_cast<int32_t>(mac1 >> 12); gte->MAC2 = static_cast<int32_t>(mac2 >> 12); gte->MAC3 = static_cast<int32_t>(mac3 >> 12); - gte->IR1 = gte->saturate_ir(gte->MAC1, 1, true); - gte->IR2 = gte->saturate_ir(gte->MAC2, 2, true); - gte->IR3 = gte->saturate_ir(gte->MAC3, 3, true); + gte->IR1 = gte->saturate_ir(gte->MAC1, 1, lm); + gte->IR2 = gte->saturate_ir(gte->MAC2, 2, lm); + gte->IR3 = gte->saturate_ir(gte->MAC3, 3, lm); gte->push_rgb( gte->saturate_color(gte->MAC1 >> 4, 0), gte->saturate_color(gte->MAC2 >> 4, 1), gte->saturate_color(gte->MAC3 >> 4, 2)); } -// Depth cue from current IR values using IR0 interpolation toward far color -// MAC = FC * 0x1000 - (BK + LC*IR) * 0x1000 ; then IR = (BK+LC*IR) + IR0 * MAC / 0x1000 -static void depth_cue_from_ir(GTEState* gte) { - // Interpolate: result = IR + IR0 * (FC - IR) / 0x1000 - int64_t mac1 = ((int64_t)gte->FC[0] << 12) - ((int64_t)gte->IR1 << 12); - int64_t mac2 = ((int64_t)gte->FC[1] << 12) - ((int64_t)gte->IR2 << 12); - int64_t mac3 = ((int64_t)gte->FC[2] << 12) - ((int64_t)gte->IR3 << 12); - // Scale by IR0 and add back - mac1 = (int64_t)gte->IR1 + ((gte->IR0 * (mac1 >> 12)) >> 12); - mac2 = (int64_t)gte->IR2 + ((gte->IR0 * (mac2 >> 12)) >> 12); - mac3 = (int64_t)gte->IR3 + ((gte->IR0 * (mac3 >> 12)) >> 12); +// Depth cue / interpolate current IR toward the far color using IR0 +// (common tail of DPCS/DPCT/DPCL/INTPL/NCDS/NCDT/CDP). Hardware: +// base = IR << 12 +// step = lim(((FC << 12) - base) >> (sf*12)) ; ±0x8000 clamp, lm FORCED off +// MAC = (base + IR0 * step) >> (sf*12) +// IR = lim(MAC, lm) ; lm from the instruction +static void depth_cue_from_ir(GTEState* gte, uint32_t instr) { + const int shift = gte_instr_sf(instr); + const bool lm = gte_instr_lm(instr); + int64_t base1 = (int64_t)gte->IR1 << 12; + int64_t base2 = (int64_t)gte->IR2 << 12; + int64_t base3 = (int64_t)gte->IR3 << 12; + int16_t step1 = gte->saturate_ir( + (int32_t)(( ((int64_t)gte->FC[0] << 12) - base1) >> shift), 1, false); + int16_t step2 = gte->saturate_ir( + (int32_t)(( ((int64_t)gte->FC[1] << 12) - base2) >> shift), 2, false); + int16_t step3 = gte->saturate_ir( + (int32_t)(( ((int64_t)gte->FC[2] << 12) - base3) >> shift), 3, false); + int64_t mac1 = (base1 + (int64_t)gte->IR0 * step1) >> shift; + int64_t mac2 = (base2 + (int64_t)gte->IR0 * step2) >> shift; + int64_t mac3 = (base3 + (int64_t)gte->IR0 * step3) >> shift; + gte->check_mac_overflow(mac1, 1); + gte->check_mac_overflow(mac2, 2); + gte->check_mac_overflow(mac3, 3); gte->MAC1 = static_cast<int32_t>(mac1); gte->MAC2 = static_cast<int32_t>(mac2); gte->MAC3 = static_cast<int32_t>(mac3); - gte->IR1 = gte->saturate_ir(gte->MAC1, 1, true); - gte->IR2 = gte->saturate_ir(gte->MAC2, 2, true); - gte->IR3 = gte->saturate_ir(gte->MAC3, 3, true); + gte->IR1 = gte->saturate_ir(gte->MAC1, 1, lm); + gte->IR2 = gte->saturate_ir(gte->MAC2, 2, lm); + gte->IR3 = gte->saturate_ir(gte->MAC3, 3, lm); } // --------------------------------------------------------------------------- @@ -120,6 +184,7 @@ static void depth_cue_from_ir(GTEState* gte) { // --------------------------------------------------------------------------- static int32_t s_ws_xnum = 1, s_ws_xden = 1; extern "C" int gpu_ws_present_native_43(void); /* gpu.c — suppress on 4:3 frames */ +extern "C" void psx_ws_note_gte_project(int nverts); /* gpu.c — gte_game_mode stamp */ // Per-draw suppression of the X-squash (8C far-backdrop). The far backdrop // (ocean/cloud/distant mountain) is a parallax layer that is conceptually at @@ -151,6 +216,345 @@ extern "C" void gte_ws_get_sz_stats(int* mn, int* mx, unsigned* n, unsigned* far s_ws_sz_min = 0x7FFFFFFF; s_ws_sz_max = 0; s_ws_sz_n = 0; s_ws_sz_far = 0; } +// ---- Native-wide sky-DOME expand ------------------------------------------ +// In native-wide (mode 2) the GTE is fed identity (no squash) so the world +// fills the wider FOV via the cull widening. But a sky DOME is a FINITE mesh +// authored to fill 4:3 — its curved edge falls short of the wider frame corners +// (black corners). Fix: scale the FAR-depth vertices' projected X OUTWARD from +// the projection centre (OFX) by the frame-widening ratio (3*num)/(4*den) — the +// inverse of the squash — so the dome grows to cover the wider FOV. DEPTH-GATED +// (SZ >= s_ws_far_threshold) so only the farthest layer (the sky) expands while +// nearer world geometry stays put. On + ratio set from main.cpp when native- +// wide engages; threshold tuned live via ws_far_threshold. Off => identity. +static int s_ws_dome_on = 0; +static int32_t s_ws_dome_num = 1, s_ws_dome_den = 1; // expand = num/den (>1) +extern "C" void gte_ws_set_dome_expand(int on, int aspect_num, int aspect_den) { + s_ws_dome_on = on ? 1 : 0; + // widen ratio = wide_w / disp_w = (aspect/(4:3)) = (3*num)/(4*den). + int32_t n = 3 * aspect_num, d = 4 * aspect_den; + if (n <= 0 || d <= 0 || n <= d) { s_ws_dome_num = s_ws_dome_den = 1; return; } + s_ws_dome_num = n; s_ws_dome_den = d; +} + +// ---- Dome-locate probe ----------------------------------------------------- +// Tally which guest function (g_debug_current_func_addr, set at dispatch) +// projects FAR (SZ >= threshold) vertices, so the sky-dome draw function can be +// identified as the top far-vertex emitter — the target for the per-function +// dome-expand bracket (the clean alternative to the scene-dependent depth gate). +extern "C" { extern uint32_t g_debug_current_func_addr; } +static uint32_t s_gte_caller_ra = 0; /* guest ra at gte_execute = return into the GAME fn that issued the projection (not the libgte leaf) */ +#define DOME_PROBE_SLOTS 48 +static struct { uint32_t func; uint32_t count; int32_t max_sz; } s_dome_probe[DOME_PROBE_SLOTS]; +static int s_dome_probe_on = 0; +static int32_t s_dome_probe_thr = 4000; +extern "C" void gte_dome_probe(int on, int thr) { + s_dome_probe_on = on ? 1 : 0; + if (thr > 0) s_dome_probe_thr = thr; + if (on) for (int i = 0; i < DOME_PROBE_SLOTS; i++) { s_dome_probe[i].func = 0; s_dome_probe[i].count = 0; s_dome_probe[i].max_sz = 0; } +} +static inline void dome_probe_note(int32_t sz) { + if (!s_dome_probe_on || sz < s_dome_probe_thr) return; + /* Tally the guest RA (the GAME fn that jal'd to libgte for this projection), + * NOT g_debug_current_func_addr (which is the libgte leaf 0x80000F40). */ + uint32_t f = s_gte_caller_ra; + int slot = -1, empty = -1; + for (int i = 0; i < DOME_PROBE_SLOTS; i++) { + if (s_dome_probe[i].count && s_dome_probe[i].func == f) { slot = i; break; } + if (empty < 0 && s_dome_probe[i].count == 0) empty = i; + } + if (slot < 0) slot = empty; + if (slot < 0) return; + s_dome_probe[slot].func = f; + s_dome_probe[slot].count++; + if (sz > s_dome_probe[slot].max_sz) s_dome_probe[slot].max_sz = sz; +} +extern "C" int gte_dome_probe_dump(uint32_t* funcs, uint32_t* counts, int32_t* maxsz, int cap) { + int n = 0; + for (int i = 0; i < DOME_PROBE_SLOTS && n < cap; i++) + if (s_dome_probe[i].count) { funcs[n] = s_dome_probe[i].func; counts[n] = s_dome_probe[i].count; maxsz[n] = s_dome_probe[i].max_sz; n++; } + return n; +} + +// --------------------------------------------------------------------------- +// GTE projection ring (ALWAYS-ON) — records each RTPS/RTPT's inputs (vertex, +// rotation matrix, translation, projection config) and outputs (screen XY / Z / +// FLAG). A post-hoc probe queries this to find flattened/degenerate character +// projections and decide whether the fault is in the INPUTS supplied by the +// recompiled game code (bad matrix/vertex) or the GTE math itself. +// --------------------------------------------------------------------------- +// NOTE: this block is already inside namespace PSXRecomp::GTE (opened line 8), +// so types are referred to unqualified. +extern "C" { extern uint64_t s_frame_count; } +struct GteRtpRec { + uint32_t seq, frame, caller_ra, cmd; + int16_t V0[3], V1[3], V2[3]; + int16_t RT[9]; + int32_t TR[3]; + uint16_t H; int32_t OFX, OFY; + int32_t SXY0, SXY1, SXY2; + uint16_t SZ1, SZ2, SZ3; + uint32_t FLAG; +}; +#define GTE_RTP_RING_CAP (1u << 18) /* 256K raw entries (~a few seconds) */ +static GteRtpRec* s_gte_rtp_ring = nullptr; +static uint64_t s_gte_rtp_seq = 0; + +/* Per-frame aggregate stats (retained ring over recent distinct frames). Lets a + * probe see the alternating flat/normal render pattern without shipping 84k + * projections/sec over JSON. nsat = projections whose newest output vertex + * saturated off-screen; nflat = RTP whose last-3 output verts are collinear + * (area 0) = a flattened triangle. */ +#define GTE_FSTAT_CAP 512 +struct GteFStat { uint32_t frame, nproj, nsat, nflat, nintpl, nintpl_tiny; }; +static GteFStat s_gte_fstat[GTE_FSTAT_CAP]; +static uint32_t s_gte_fstat_head = 0; /* index of most-recent frame slot */ +static int s_gte_fstat_valid = 0; + +/* Degenerate-latch: full records of flat/saturated projections, retained (small + * ring) so intermittent flat frames are caught regardless of raw-ring roll. */ +#define GTE_LATCH_CAP (1u << 13) /* 8192 */ +static GteRtpRec* s_gte_latch = nullptr; +static uint64_t s_gte_latch_seq = 0; + +static inline int gte_sxx(int32_t p){ int v=p&0xFFFF; return v>=0x8000? v-0x10000:v; } +static inline int gte_syy(int32_t p){ int v=(p>>16)&0xFFFF; return v>=0x8000? v-0x10000:v; } + +static void gte_rtp_record(const GTEState* g, uint32_t cmd) { + if (!s_gte_rtp_ring) { + s_gte_rtp_ring = (GteRtpRec*)calloc(GTE_RTP_RING_CAP, sizeof(GteRtpRec)); + if (!s_gte_rtp_ring) return; + } + GteRtpRec* e = &s_gte_rtp_ring[s_gte_rtp_seq % GTE_RTP_RING_CAP]; + e->seq = (uint32_t)s_gte_rtp_seq; + e->frame = (uint32_t)s_frame_count; + e->caller_ra = s_gte_caller_ra; + e->cmd = cmd; + for (int i = 0; i < 3; i++) { e->V0[i]=g->V0[i]; e->V1[i]=g->V1[i]; e->V2[i]=g->V2[i]; } + for (int r = 0; r < 3; r++) for (int c = 0; c < 3; c++) e->RT[r*3+c]=g->RT[r][c]; + for (int i = 0; i < 3; i++) e->TR[i]=g->TR[i]; + e->H=g->H; e->OFX=g->OFX; e->OFY=g->OFY; + e->SXY0=g->SXY[0]; e->SXY1=g->SXY[1]; e->SXY2=g->SXY[2]; + e->SZ1=g->SZ[1]; e->SZ2=g->SZ[2]; e->SZ3=g->SZ[3]; + e->FLAG=g->FLAG; + s_gte_rtp_seq++; + + /* classify */ + int x0=gte_sxx(e->SXY0),y0=gte_syy(e->SXY0); + int x1=gte_sxx(e->SXY1),y1=gte_syy(e->SXY1); + int x2=gte_sxx(e->SXY2),y2=gte_syy(e->SXY2); + int sat = (x2<=-1023||x2>=1023||y2<=-1023||y2>=1023) ? 1 : 0; + long area2 = (long)(x1-x0)*(y2-y0) - (long)(x2-x0)*(y1-y0); + int flat = (area2==0) ? 1 : 0; /* three output verts exactly collinear */ + + /* per-frame stats */ + uint32_t f=(uint32_t)s_frame_count; + if (!s_gte_fstat_valid || s_gte_fstat[s_gte_fstat_head].frame != f) { + s_gte_fstat_head = s_gte_fstat_valid ? (s_gte_fstat_head+1)%GTE_FSTAT_CAP : 0; + s_gte_fstat_valid = 1; + s_gte_fstat[s_gte_fstat_head].frame=f; + s_gte_fstat[s_gte_fstat_head].nproj=0; + s_gte_fstat[s_gte_fstat_head].nsat=0; + s_gte_fstat[s_gte_fstat_head].nflat=0; + s_gte_fstat[s_gte_fstat_head].nintpl=0; + s_gte_fstat[s_gte_fstat_head].nintpl_tiny=0; + } + s_gte_fstat[s_gte_fstat_head].nproj++; + s_gte_fstat[s_gte_fstat_head].nsat += sat; + s_gte_fstat[s_gte_fstat_head].nflat += flat; + + /* latch degenerate (saturated) projections — the "flat/absent" character */ + if (sat) { + if (!s_gte_latch) { + s_gte_latch = (GteRtpRec*)calloc(GTE_LATCH_CAP, sizeof(GteRtpRec)); + if (!s_gte_latch) return; + } + s_gte_latch[s_gte_latch_seq % GTE_LATCH_CAP] = *e; + s_gte_latch_seq++; + } +} + +extern "C" uint64_t gte_rtp_ring_total(void) { return s_gte_rtp_seq; } +extern "C" uint64_t gte_latch_total(void) { return s_gte_latch_seq; } + +// --------------------------------------------------------------------------- +// GTE INTPL ring (ALWAYS-ON) — records each INTPL (0x11) vertex-lerp: inputs +// (IR0 blend, IR1-3 pose-A vertex, FC pose-B vertex) snapshotted BEFORE the op +// and outputs (MAC1-3 / IR1-3 / FLAG) after. Crash Bash tweens character +// vertex animation through INTPL; this ring decides whether collapsed tween +// output comes from already-tiny inputs (upstream CPU/game-state divergence) +// or from the INTPL math itself. +// --------------------------------------------------------------------------- +struct GteIntplRec { + uint32_t seq, frame, caller_ra; + int16_t ir0, in1, in2, in3; + int32_t fc[3]; + int32_t mac[3]; + int16_t out1, out2, out3; + uint32_t flag; +}; +#define GTE_INTPL_RING_CAP (1u << 17) /* 128K entries (~4s at menu rates) */ +static GteIntplRec* s_gte_intpl_ring = nullptr; +static uint64_t s_gte_intpl_seq = 0; + +static void gte_intpl_record(const GTEState* g, + const int16_t pre_ir[4], const int32_t pre_fc[3]) { + if (!s_gte_intpl_ring) { + s_gte_intpl_ring = (GteIntplRec*)calloc(GTE_INTPL_RING_CAP, sizeof(GteIntplRec)); + if (!s_gte_intpl_ring) return; + } + GteIntplRec* e = &s_gte_intpl_ring[s_gte_intpl_seq % GTE_INTPL_RING_CAP]; + e->seq = (uint32_t)s_gte_intpl_seq; + e->frame = (uint32_t)s_frame_count; + e->caller_ra = s_gte_caller_ra; + e->ir0 = pre_ir[0]; e->in1 = pre_ir[1]; e->in2 = pre_ir[2]; e->in3 = pre_ir[3]; + for (int i = 0; i < 3; i++) e->fc[i] = pre_fc[i]; + e->mac[0]=g->MAC1; e->mac[1]=g->MAC2; e->mac[2]=g->MAC3; + e->out1=g->IR1; e->out2=g->IR2; e->out3=g->IR3; + e->flag=g->FLAG; + s_gte_intpl_seq++; + + /* per-frame stats (shares the fstat ring; nintpl_tiny = all outputs small + * while at least one input was not — the collapse signature) */ + uint32_t f=(uint32_t)s_frame_count; + if (!s_gte_fstat_valid || s_gte_fstat[s_gte_fstat_head].frame != f) { + s_gte_fstat_head = s_gte_fstat_valid ? (s_gte_fstat_head+1)%GTE_FSTAT_CAP : 0; + s_gte_fstat_valid = 1; + s_gte_fstat[s_gte_fstat_head].frame=f; + s_gte_fstat[s_gte_fstat_head].nproj=0; + s_gte_fstat[s_gte_fstat_head].nsat=0; + s_gte_fstat[s_gte_fstat_head].nflat=0; + s_gte_fstat[s_gte_fstat_head].nintpl=0; + s_gte_fstat[s_gte_fstat_head].nintpl_tiny=0; + } + s_gte_fstat[s_gte_fstat_head].nintpl++; + { + int out_tiny = (e->out1>-48 && e->out1<48 && e->out2>-48 && e->out2<48 + && e->out3>-48 && e->out3<48); + int in_big = (e->in1<=-48 || e->in1>=48 || e->in2<=-48 || e->in2>=48 + || e->in3<=-48 || e->in3>=48 + || e->fc[0]<=-48 || e->fc[0]>=48 + || e->fc[1]<=-48 || e->fc[1]>=48 + || e->fc[2]<=-48 || e->fc[2]>=48); + if (out_tiny && in_big) s_gte_fstat[s_gte_fstat_head].nintpl_tiny++; + } +} + +extern "C" uint64_t gte_intpl_ring_total(void) { return s_gte_intpl_seq; } + +/* Dump INTPL ring entries as JSON array body. offset skips the first N + * MATCHING entries (after frame filtering), so a probe can page through a + * whole frame regardless of where it sits in the ring. */ +extern "C" int gte_intpl_ring_dump_json(char* out, int outsz, int max_count, + int newest_first, long frame_filter, + int offset) { + if (!s_gte_intpl_ring || outsz < 64) { if (out && outsz) out[0]=0; return 0; } + uint64_t total = s_gte_intpl_seq; + if (total == 0) { out[0]=0; return 0; } + uint64_t oldest = total > GTE_INTPL_RING_CAP ? total - GTE_INTPL_RING_CAP : 0; + int pos = 0, emitted = 0, matched = 0; + for (uint64_t k = 0; k < (total - oldest) && emitted < max_count; k++) { + uint64_t seq = newest_first ? (total - 1 - k) : (oldest + k); + if (seq < oldest || seq >= total) break; + const GteIntplRec* e = &s_gte_intpl_ring[seq % GTE_INTPL_RING_CAP]; + if (frame_filter >= 0 && e->frame != (uint32_t)frame_filter) continue; + if (matched++ < offset) continue; + if (pos > outsz - 400) break; + pos += snprintf(out+pos, outsz-pos, + "%s{\"seq\":%u,\"frame\":%u,\"ra\":\"0x%08X\"," + "\"ir0\":%d,\"in\":[%d,%d,%d],\"fc\":[%d,%d,%d]," + "\"mac\":[%d,%d,%d],\"out\":[%d,%d,%d],\"flag\":\"0x%08X\"}", + emitted?",":"", e->seq, e->frame, e->caller_ra, + e->ir0, e->in1, e->in2, e->in3, e->fc[0], e->fc[1], e->fc[2], + e->mac[0], e->mac[1], e->mac[2], e->out1, e->out2, e->out3, + e->flag); + emitted++; + } + out[pos] = 0; + return emitted; +} + +/* Emit per-frame stats (newest first) as JSON array body. */ +extern "C" int gte_fstat_dump_json(char* out, int outsz, int max_frames) { + if (!s_gte_fstat_valid) { if(out&&outsz) out[0]=0; return 0; } + int pos=0, emitted=0; + for (int k=0; k<GTE_FSTAT_CAP && emitted<max_frames; k++) { + int idx=(int)((s_gte_fstat_head + GTE_FSTAT_CAP - k)%GTE_FSTAT_CAP); + GteFStat* s=&s_gte_fstat[idx]; + if (s->nproj==0 && !(k==0)) continue; + if (pos>outsz-96) break; + pos+=snprintf(out+pos,outsz-pos, + "%s{\"frame\":%u,\"nproj\":%u,\"nsat\":%u,\"nflat\":%u," + "\"nintpl\":%u,\"nintpl_tiny\":%u}", + emitted?",":"", s->frame,s->nproj,s->nsat,s->nflat, + s->nintpl,s->nintpl_tiny); + emitted++; + } + out[pos]=0; return emitted; +} + +/* Dump latched degenerate projections (newest first). */ +extern "C" int gte_latch_dump_json(char* out, int outsz, int max_count) { + if (!s_gte_latch || s_gte_latch_seq==0){ if(out&&outsz) out[0]=0; return 0; } + uint64_t total=s_gte_latch_seq; + uint64_t oldest = total>GTE_LATCH_CAP ? total-GTE_LATCH_CAP : 0; + int pos=0, emitted=0; + for (uint64_t k=0; k<(total-oldest) && emitted<max_count; k++) { + uint64_t seq=total-1-k; + if (seq<oldest) break; + const GteRtpRec* e=&s_gte_latch[seq % GTE_LATCH_CAP]; + if (pos>outsz-700) break; + pos+=snprintf(out+pos,outsz-pos, + "%s{\"frame\":%u,\"ra\":\"0x%08X\",\"cmd\":\"0x%08X\"," + "\"RT\":[%d,%d,%d,%d,%d,%d,%d,%d,%d],\"TR\":[%d,%d,%d],\"H\":%u," + "\"V0\":[%d,%d,%d],\"V1\":[%d,%d,%d],\"V2\":[%d,%d,%d]," + "\"S0\":[%d,%d],\"S1\":[%d,%d],\"S2\":[%d,%d],\"SZ\":[%u,%u,%u],\"FLAG\":\"0x%08X\"}", + emitted?",":"", e->frame, e->caller_ra, e->cmd, + e->RT[0],e->RT[1],e->RT[2],e->RT[3],e->RT[4],e->RT[5],e->RT[6],e->RT[7],e->RT[8], + e->TR[0],e->TR[1],e->TR[2],(unsigned)e->H, + e->V0[0],e->V0[1],e->V0[2], e->V1[0],e->V1[1],e->V1[2], e->V2[0],e->V2[1],e->V2[2], + gte_sxx(e->SXY0),gte_syy(e->SXY0),gte_sxx(e->SXY1),gte_syy(e->SXY1), + gte_sxx(e->SXY2),gte_syy(e->SXY2),(unsigned)e->SZ1,(unsigned)e->SZ2,(unsigned)e->SZ3,e->FLAG); + emitted++; + } + out[pos]=0; return emitted; +} + +/* Format up to max_count recent entries as a JSON array body (no outer braces). + * If frame_filter >= 0, only entries with that frame are emitted. Returns count. */ +extern "C" int gte_rtp_ring_dump_json(char* out, int outsz, int max_count, + int newest_first, long frame_filter) { + if (!s_gte_rtp_ring || outsz < 64) { if (out && outsz) out[0]=0; return 0; } + uint64_t total = s_gte_rtp_seq; + if (total == 0) { out[0]=0; return 0; } + uint64_t oldest = total > GTE_RTP_RING_CAP ? total - GTE_RTP_RING_CAP : 0; + int pos = 0, emitted = 0; + for (uint64_t k = 0; k < (total - oldest) && emitted < max_count; k++) { + uint64_t seq = newest_first ? (total - 1 - k) : (oldest + k); + if (seq < oldest || seq >= total) break; + const GteRtpRec* e = &s_gte_rtp_ring[seq % GTE_RTP_RING_CAP]; + if (frame_filter >= 0 && e->frame != (uint32_t)frame_filter) continue; + auto sxx = [](int32_t p){ int v=p&0xFFFF; return v>=0x8000? v-0x10000:v; }; + auto syy = [](int32_t p){ int v=(p>>16)&0xFFFF; return v>=0x8000? v-0x10000:v; }; + if (pos > outsz - 700) break; + pos += snprintf(out+pos, outsz-pos, + "%s{\"seq\":%u,\"frame\":%u,\"ra\":\"0x%08X\",\"cmd\":\"0x%08X\"," + "\"V0\":[%d,%d,%d],\"V1\":[%d,%d,%d],\"V2\":[%d,%d,%d]," + "\"RT\":[%d,%d,%d,%d,%d,%d,%d,%d,%d],\"TR\":[%d,%d,%d]," + "\"H\":%u,\"OFX\":%d,\"OFY\":%d," + "\"S0\":[%d,%d],\"S1\":[%d,%d],\"S2\":[%d,%d]," + "\"SZ\":[%u,%u,%u],\"FLAG\":\"0x%08X\"}", + emitted?",":"", e->seq, e->frame, e->caller_ra, e->cmd, + e->V0[0],e->V0[1],e->V0[2], e->V1[0],e->V1[1],e->V1[2], e->V2[0],e->V2[1],e->V2[2], + e->RT[0],e->RT[1],e->RT[2],e->RT[3],e->RT[4],e->RT[5],e->RT[6],e->RT[7],e->RT[8], + e->TR[0],e->TR[1],e->TR[2], (unsigned)e->H, e->OFX, e->OFY, + sxx(e->SXY0),syy(e->SXY0), sxx(e->SXY1),syy(e->SXY1), sxx(e->SXY2),syy(e->SXY2), + (unsigned)e->SZ1,(unsigned)e->SZ2,(unsigned)e->SZ3, e->FLAG); + emitted++; + } + out[pos] = 0; + return emitted; +} + extern "C" void gte_set_display_aspect(int num, int den) { if (num <= 0 || den <= 0) { s_ws_xnum = s_ws_xden = 1; return; } // squash = (4/3) / (num/den) = (4*den) / (3*num); identity for 4:3. @@ -212,6 +616,21 @@ void gte_rtps_internal(GTEState* gte, int16_t* V, bool setMac0) { } if (do_squash) xterm = xterm * s_ws_xnum / s_ws_xden; + // Native-wide sky-dome expand: no squash active (identity), dome mode on, + // this frame is stretched, and the vertex is far (sky). Scale X outward from + // the projection centre so the finite dome mesh reaches the wider frame. + else if (s_ws_dome_on && s_ws_dome_num != s_ws_dome_den && + !gpu_ws_present_native_43()) { + int32_t sz = gte->SZ[3]; + if (sz < s_ws_sz_min) s_ws_sz_min = sz; + if (sz > s_ws_sz_max) s_ws_sz_max = sz; + s_ws_sz_n++; + if (sz >= s_ws_far_threshold) { + xterm = xterm * s_ws_dome_num / s_ws_dome_den; + s_ws_sz_far++; + } + } + dome_probe_note(gte->SZ[3]); /* locate the dome draw fn (far-vertex tally) */ int64_t sx = (gte->OFX + xterm) >> 16; int64_t sy = (gte->OFY + (int64_t)gte->IR2 * h_div_sz) >> 16; gte->push_sxy(sx, sy); @@ -287,15 +706,15 @@ void gte_avsz4(GTEState* gte, uint32_t instr) { // --------------------------------------------------------------------------- // NCCS (0x1B) — Normal Color Color Single // --------------------------------------------------------------------------- -void gte_nccs_internal(GTEState* gte, int16_t* V) { - light_transform(gte, V); - light_color(gte); - color_output(gte); +void gte_nccs_internal(GTEState* gte, int16_t* V, uint32_t instr) { + light_transform(gte, V, instr); + light_color(gte, instr); + color_output(gte, instr); } void gte_nccs(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - gte_nccs_internal(gte, gte->V0); + gte_nccs_internal(gte, gte->V0, instr); gte->set_error_flag(); } @@ -304,25 +723,25 @@ void gte_nccs(GTEState* gte, uint32_t instr) { // --------------------------------------------------------------------------- void gte_ncct(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - gte_nccs_internal(gte, gte->V0); - gte_nccs_internal(gte, gte->V1); - gte_nccs_internal(gte, gte->V2); + gte_nccs_internal(gte, gte->V0, instr); + gte_nccs_internal(gte, gte->V1, instr); + gte_nccs_internal(gte, gte->V2, instr); gte->set_error_flag(); } // --------------------------------------------------------------------------- // NCDS (0x13) — Normal Color Depth Cue Single // --------------------------------------------------------------------------- -void gte_ncds_internal(GTEState* gte, int16_t* V) { - light_transform(gte, V); - light_color(gte); - depth_cue_from_ir(gte); - color_output(gte); +void gte_ncds_internal(GTEState* gte, int16_t* V, uint32_t instr) { + light_transform(gte, V, instr); + light_color(gte, instr); + depth_cue_from_ir(gte, instr); + color_output(gte, instr); } void gte_ncds(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - gte_ncds_internal(gte, gte->V0); + gte_ncds_internal(gte, gte->V0, instr); gte->set_error_flag(); } @@ -331,18 +750,18 @@ void gte_ncds(GTEState* gte, uint32_t instr) { // --------------------------------------------------------------------------- void gte_ncdt(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - gte_ncds_internal(gte, gte->V0); - gte_ncds_internal(gte, gte->V1); - gte_ncds_internal(gte, gte->V2); + gte_ncds_internal(gte, gte->V0, instr); + gte_ncds_internal(gte, gte->V1, instr); + gte_ncds_internal(gte, gte->V2, instr); gte->set_error_flag(); } // --------------------------------------------------------------------------- // NCS (0x1E) — Normal Color Single (no vertex color multiply) // --------------------------------------------------------------------------- -void gte_ncs_internal(GTEState* gte, int16_t* V) { - light_transform(gte, V); - light_color(gte); +void gte_ncs_internal(GTEState* gte, int16_t* V, uint32_t instr) { + light_transform(gte, V, instr); + light_color(gte, instr); // Output directly (no RGBC multiply) gte->push_rgb( gte->saturate_color(gte->MAC1 >> 4, 0), @@ -352,7 +771,7 @@ void gte_ncs_internal(GTEState* gte, int16_t* V) { void gte_ncs(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - gte_ncs_internal(gte, gte->V0); + gte_ncs_internal(gte, gte->V0, instr); gte->set_error_flag(); } @@ -361,9 +780,9 @@ void gte_ncs(GTEState* gte, uint32_t instr) { // --------------------------------------------------------------------------- void gte_nct(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - gte_ncs_internal(gte, gte->V0); - gte_ncs_internal(gte, gte->V1); - gte_ncs_internal(gte, gte->V2); + gte_ncs_internal(gte, gte->V0, instr); + gte_ncs_internal(gte, gte->V1, instr); + gte_ncs_internal(gte, gte->V2, instr); gte->set_error_flag(); } @@ -380,7 +799,7 @@ void gte_dpcs(GTEState* gte, uint32_t instr) { gte->IR2 = gte->saturate_ir(g << 4, 2, false); gte->IR3 = gte->saturate_ir(b << 4, 3, false); // Interpolate toward far color using IR0 - depth_cue_from_ir(gte); + depth_cue_from_ir(gte, instr); // Output gte->push_rgb( gte->saturate_color(gte->MAC1 >> 4, 0), @@ -401,7 +820,7 @@ void gte_dpct(GTEState* gte, uint32_t instr) { gte->IR1 = gte->saturate_ir(r << 4, 1, false); gte->IR2 = gte->saturate_ir(g << 4, 2, false); gte->IR3 = gte->saturate_ir(b << 4, 3, false); - depth_cue_from_ir(gte); + depth_cue_from_ir(gte, instr); gte->push_rgb( gte->saturate_color(gte->MAC1 >> 4, 0), gte->saturate_color(gte->MAC2 >> 4, 1), @@ -422,11 +841,11 @@ void gte_dpcl(GTEState* gte, uint32_t instr) { gte->MAC1 = (r * gte->IR1) >> 8; gte->MAC2 = (g * gte->IR2) >> 8; gte->MAC3 = (b * gte->IR3) >> 8; - gte->IR1 = gte->saturate_ir(gte->MAC1, 1, true); - gte->IR2 = gte->saturate_ir(gte->MAC2, 2, true); - gte->IR3 = gte->saturate_ir(gte->MAC3, 3, true); + gte->IR1 = gte->saturate_ir(gte->MAC1, 1, gte_instr_lm(instr)); + gte->IR2 = gte->saturate_ir(gte->MAC2, 2, gte_instr_lm(instr)); + gte->IR3 = gte->saturate_ir(gte->MAC3, 3, gte_instr_lm(instr)); // Depth cue toward far color - depth_cue_from_ir(gte); + depth_cue_from_ir(gte, instr); gte->push_rgb( gte->saturate_color(gte->MAC1 >> 4, 0), gte->saturate_color(gte->MAC2 >> 4, 1), @@ -439,7 +858,7 @@ void gte_dpcl(GTEState* gte, uint32_t instr) { // --------------------------------------------------------------------------- void gte_intpl(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - depth_cue_from_ir(gte); + depth_cue_from_ir(gte, instr); gte->push_rgb( gte->saturate_color(gte->MAC1 >> 4, 0), gte->saturate_color(gte->MAC2 >> 4, 1), @@ -453,9 +872,9 @@ void gte_intpl(GTEState* gte, uint32_t instr) { void gte_cdp(GTEState* gte, uint32_t instr) { gte->FLAG = 0; // IR1/IR2/IR3 already set (from previous NCS or similar) - light_color(gte); - depth_cue_from_ir(gte); - color_output(gte); + light_color(gte, instr); + depth_cue_from_ir(gte, instr); + color_output(gte, instr); gte->set_error_flag(); } @@ -464,8 +883,8 @@ void gte_cdp(GTEState* gte, uint32_t instr) { // --------------------------------------------------------------------------- void gte_cc(GTEState* gte, uint32_t instr) { gte->FLAG = 0; - light_color(gte); - color_output(gte); + light_color(gte, instr); + color_output(gte, instr); gte->set_error_flag(); } @@ -842,6 +1261,7 @@ extern "C" uint64_t gte_get_exec_count(void) { return s_gte_exec_count; } extern "C" void gte_execute(CPUState* cpu, uint32_t cmd) { using namespace PSXRecomp::GTE; s_gte_exec_count++; + s_gte_caller_ra = cpu->gpr[31]; /* dome-locate probe: game fn that issued this projection */ GTEState gte; // Skip reg 15 (SXYP: push-write, would corrupt SXY FIFO) and @@ -853,6 +1273,20 @@ extern "C" void gte_execute(CPUState* cpu, uint32_t cmd) { for (int i = 0; i < 32; i++) gte_ctc2(>e, i, cpu->gte_ctrl[i]); uint8_t func = cmd & 0x3F; + /* GTE-activity gameplay detector ([widescreen] gte_game_mode): note every + * perspective projection so gpu.c can stamp real 3D frames as gameplay + * (no-op unless the game opts in — early-out on the config flag). */ + if (func == 0x01 || func == 0x30) + psx_ws_note_gte_project(func == 0x30 ? 3 : 1); + /* INTPL ring: snapshot inputs before the op mutates IR (outputs recorded + * after the switch). */ + int16_t intpl_pre_ir[4] = {0,0,0,0}; + int32_t intpl_pre_fc[3] = {0,0,0}; + if (func == 0x11) { + intpl_pre_ir[0]=gte.IR0; intpl_pre_ir[1]=gte.IR1; + intpl_pre_ir[2]=gte.IR2; intpl_pre_ir[3]=gte.IR3; + intpl_pre_fc[0]=gte.FC[0]; intpl_pre_fc[1]=gte.FC[1]; intpl_pre_fc[2]=gte.FC[2]; + } switch (func) { case 0x01: gte_rtps(>e, cmd); break; case 0x06: gte_nclip(>e, cmd); break; @@ -881,6 +1315,9 @@ extern "C" void gte_execute(CPUState* cpu, uint32_t cmd) { break; } + if (func == 0x01 || func == 0x30) gte_rtp_record(>e, cmd); + if (func == 0x11) gte_intpl_record(>e, intpl_pre_ir, intpl_pre_fc); + for (int i = 0; i < 32; i++) cpu->gte_data[i] = gte_mfc2(>e, i); for (int i = 0; i < 32; i++) cpu->gte_ctrl[i] = gte_cfc2(>e, i); diff --git a/runtime/src/interrupts.c b/runtime/src/interrupts.c index 9fdd8cc8..6478e49d 100644 --- a/runtime/src/interrupts.c +++ b/runtime/src/interrupts.c @@ -36,6 +36,7 @@ #include "event_ring.h" #include "lockstep.h" #include "psx_cycles.h" +#include "psx_scheduler.h" #include <stdio.h> #include <stdlib.h> #include <setjmp.h> @@ -47,7 +48,7 @@ extern int g_dirty_interp_active; extern uint32_t g_dirty_safe_resume_pc; /* IRQ-delivery context ring (MMX6 VSync-vs-CD-DMA hunt; dumped via `irqctx_ring`). */ -#define IRQCTX_RING_CAP 64u +#define IRQCTX_RING_CAP 4096u typedef struct { uint64_t seq, cycle; uint32_t frame, istat, imask, sr, d44, cdrom_active, is_vblank; @@ -67,6 +68,13 @@ typedef struct { uint32_t ra_exit; /* cpu->gpr[31] at exit before restore decision */ uint32_t ra_saved; /* saved_gpr[31] */ uint32_t redirects; /* jmp_val==2 RestoreState redirects in this delivery */ + /* Ape memcard native<->interp resume-desync probe: the live sp and the + * dirty pump-site at IRQ ENTRY. The interrupted-thread TCB save captures + * exactly this (resume_pc=take_pc, sp=entry_sp); a pair where take_pc is a + * dirty return point but entry_sp belongs to a native callee's frame is the + * corrupt snapshot (0x801384AC paired with 0x801EFE80). */ + uint32_t entry_sp; /* cpu->gpr[29] at exception entry */ + uint32_t pump_site; /* g_cosim_dirty_pump_site at entry (which delivery path) */ } IrqCtxEntry; IrqCtxEntry g_irqctx_ring[IRQCTX_RING_CAP]; uint64_t g_irqctx_seq = 0; @@ -124,6 +132,7 @@ static uint32_t dispatch_count; static uint64_t total_checks; static uint32_t cycles_since_vblank; /* incremented by interrupts_advance_cycles */ extern uint64_t g_vblank_raise_count; +extern int g_cosim_dirty_pump_site; /* Reentrancy guard: prevent interrupt handler from triggering interrupts. */ static int in_exception; @@ -415,6 +424,15 @@ extern uint64_t g_vblank_ack_count; /* defined in memory.c */ * That fiber's wrapped SwitchToFiber call will observe the flag on * return and execute the longjmp on the correct stack. */ jmp_buf exception_jmpbuf; /* non-static so traps.c can deferred-longjmp */ +/* Monotonic epoch of the exception setjmp frame: bumped each time the setjmp + * loop is (re)armed at exception entry. Lets an observer holding a frame on + * the C stack (the overlay shadow-diff, run_shadow_diff) decide whether a + * longjmp targets a frame BELOW it (armed before it — the unwind escapes its + * frame and skips its epilogue) or a frame armed after it (contained). The + * single global exception_jmpbuf always holds the latest-armed frame, so the + * current epoch identifies every longjmp's target. */ +uint64_t g_exc_setjmp_epoch = 0; +uint64_t psx_exception_setjmp_epoch(void) { return g_exc_setjmp_epoch; } /* The fiber that owns the current exception setjmp. A longjmp must run on * that same fiber/stack, so a non-owner defers by switching back to it * first (see deferred_exception_longjmp). Used on all platforms now that @@ -431,6 +449,34 @@ static uint32_t s_compiled_interrupt_resume_pc = 0; static uint32_t s_last_interrupt_check_pc = 0; static uint64_t s_last_interrupt_check_cycle = UINT64_MAX; +/* Deferred cooperative thread switch from nested exception delivery. + * + * A genuine in-exception ChangeThread (kind-30, escape site below) must be + * honored at the OUTERMOST dispatch boundary, never mid-nested-host-dispatch. + * Only at the outermost boundary is the outgoing thread's guest state fully + * materialized in CPUState (its true block PC is live, GPRs current); nested + * inside a call unit / dirty pump the PC lives only in the resume-PC latch, + * which desyncs from the live GPRs when the IRQ was taken deep inside a nested + * call (the async-poll callback), so switching there saves a poisoned context + * (latched PC ahead of the callback's registers) → the resumed thread dispatches + * a jumptable with a smeared index → misaligned DISPATCH FATAL. When a switch is + * detected nested, we record it here, let the outgoing thread resume + * transparently, and honor it at the next outermost block-leader boundary (top + * of psx_check_interrupts) where the thread can be re-saved cleanly. */ +static int s_defer_switch_pending = 0; +static uint32_t s_defer_switch_target = 0; /* TCB PCB[0] should name after the switch */ +static uint32_t s_defer_switch_from = 0; /* the interrupted thread to re-save cleanly */ + +/* A/B toggle: PSX_DEFER_SWITCH=0 forces the legacy immediate-switch behavior + * (longjmp the instant a mid-exception ChangeThread is detected, regardless of + * nesting) so the deferred-switch fix can be compared against the baseline + * without a rebuild. Default ON. Latched at first query. */ +static int defer_switch_enabled(void) { + static int s = -1; + if (s < 0) { const char *e = getenv("PSX_DEFER_SWITCH"); s = (e && e[0] == '0') ? 0 : 1; } + return s; +} + static int same_guest_pc(uint32_t a, uint32_t b) { return (((a ^ b) & 0x1FFFFFFFu) == 0); } @@ -496,6 +542,9 @@ void psx_rfe_escape_check(CPUState* cpu) { if (g_rfe_escape_pending && in_exception && psx_fiber_current() == g_exception_owner_fiber) { g_rfe_escape_pending = 0; + /* Same shadow-frame hardening as deferred_exception_longjmp. */ + extern void overlay_loader_shadow_escape_fixup(uint64_t target_epoch); + overlay_loader_shadow_escape_fixup(g_exc_setjmp_epoch); longjmp(exception_jmpbuf, 1); /* same fiber: safe; lands in psx_check_interrupts */ } } @@ -553,6 +602,13 @@ void interrupts_init(void) { * check in traps.c psx_change_thread_fiber executes the longjmp on the * correct stack. */ static void deferred_exception_longjmp(int code) { + /* If this unwind blows through a live shadow-diff frame (a frame armed + * BEFORE the shadow started), restore the shadow-scoped globals its + * skipped epilogue would have restored — otherwise the diff instrument + * silently dies for the rest of the run (s_in_shadow stuck). No-op when + * no shadow is live or the target frame was armed inside the shadow. */ + extern void overlay_loader_shadow_escape_fixup(uint64_t target_epoch); + overlay_loader_shadow_escape_fixup(g_exc_setjmp_epoch); if (!g_exception_owner_fiber || psx_fiber_current() == g_exception_owner_fiber) { longjmp(exception_jmpbuf, code); } @@ -638,6 +694,15 @@ void psx_check_interrupts(CPUState* cpu) { interrupts_service_scheduled_events(); + /* Idle-loop cycle skip (psx_cycles.c): detect a side-effect-free poll + * loop by its repeated same-PC checks and fast-forward guest time to the + * next internal device event in whole loop quanta. Runs BEFORE the + * deliverability evaluation below so an event raised by the skip is + * delivered in this same check, exactly at the boundary real execution + * would have taken it. No-ops in exception context / precise mode / + * lockstep (gated inside). */ + psx_idle_note_check(cpu, s_last_interrupt_check_pc); + /* User save states: this is a block-leader boundary with a known resume PC; * only act outside the exception handler so a restore's stack-unwind can't * strand a half-finished handler. Near-free when nothing is staged. A load @@ -647,6 +712,63 @@ void psx_check_interrupts(CPUState* cpu) { savestate_poll(cpu, s_last_interrupt_check_pc); } + /* Deferred cooperative thread switch: honor at the next real thread-save + * boundary (see s_defer_switch_* above). A + * genuine in-exception ChangeThread was deferred because it was detected at a + * poisoned point (the IRQ fired deep inside a nested call, e.g. the + * async-poll callback, so the resume-PC latch desynced from the live GPRs). + * + * The dirty-RAM interpreter also pumps interrupts at synthetic transfer and + * call-return sites. Those sites are precise enough to deliver hardware IRQs, + * but not necessarily safe to snapshot a cooperative thread: the interpreter + * may have committed a candidate resume PC while the live CPUState stack/GPRs + * still describe the previous local frame. Keep the deferral pending there and + * honor it only once execution reaches a normal interrupt poll with site 0. + * Near-free when nothing is pending. */ + if (s_defer_switch_pending && !in_exception) { + if (psx_hle_scheduler_enabled()) { + uint32_t from_tcb = s_defer_switch_from; + uint32_t to_tcb = s_defer_switch_target; + uint32_t resume_pc = s_last_interrupt_check_pc; /* materialized block PC */ + if (from_tcb == 0u || to_tcb == 0u || + psx_sched_current_tcb(cpu) != from_tcb) { + /* STALE: the guest already re-scheduled cooperatively (PCB[0] no + * longer names the deferred thread) - abandon the deferral. This is + * the ONLY case we clear pending without honoring; dropping when a + * clean resume PC merely isn't available yet would LOSE the guest's + * ChangeThread and wedge (the bug this replaces). */ + s_defer_switch_pending = 0; + s_defer_switch_target = 0; + s_defer_switch_from = 0; + debug_server_log_thread_event(33, cpu, from_tcb, to_tcb, resume_pc); + } else if (g_cosim_dirty_pump_site != 0) { + /* Not a real suspend boundary: dirty interpreter pump sites expose + * committed PCs for IRQ timing, but the matching CPUState may not + * be materialized yet. */ + } else if (resume_pc != 0u && (resume_pc & 0x3u) == 0u && + psx_is_dispatchable(resume_pc)) { + /* Materialized clean boundary: re-save the deferred thread cleanly + * (this OVERWRITES any poisoned/sentinel EPC the guest handler wrote + * for it while nested), re-point PCB[0], and honor the switch. */ + s_defer_switch_pending = 0; + s_defer_switch_target = 0; + s_defer_switch_from = 0; + psx_sched_save_context(cpu, from_tcb, resume_pc); + psx_sched_set_current_tcb(cpu, to_tcb); + debug_server_log_thread_event(32, cpu, from_tcb, to_tcb, resume_pc); + g_dirty_interp_active = 0; + s_compiled_interrupt_resume_pc = 0; + g_sched_escape.target_tcb = to_tcb; + g_sched_escape.resume_pc = 0; + g_sched_escape.reason = PSX_RUN_YIELD_TO_TCB; + longjmp(g_scheduler_jmpbuf, 1); /* unwind to psx_scheduler_run */ + } + /* else: KEEP pending - this outermost boundary has no materialized + * resume PC (both latches 0, e.g. a scheduled-event check not tied to a + * block leader). Wait for the next block-leader boundary that does. */ + } + } + /* Dispatch-loop maintenance only when NOT inside the exception handler. */ if (!in_exception) { #if SIO_MODEL_CYCLE_PACED @@ -860,6 +982,9 @@ void psx_check_interrupts(CPUState* cpu) { e->exit_pc = 0; e->exit_reason = 0; e->same_thread = 0; e->restored = 0; e->v1_exit = 0; e->v1_saved = 0; e->ra_exit = 0; e->ra_saved = 0; e->redirects = 0; + e->entry_sp = cpu->gpr[29]; + { extern int g_cosim_dirty_pump_site; + e->pump_site = (uint32_t)g_cosim_dirty_pump_site; } } } @@ -942,6 +1067,43 @@ void psx_check_interrupts(CPUState* cpu) { * so the EPC-sentinel longjmp can't leave the flag wrong. */ int prev_interp_active = g_dirty_interp_active; g_dirty_interp_active = 0; + /* Ape memcard fix #3 (2026-07-07): clear the dirty-interp resume-PC latch + * across the handler dispatch, exactly like g_dirty_interp_active above. + * + * The interrupted dirty block set g_dirty_safe_resume_pc = its committed PC + * and left it LIVE across its own psx_check_interrupts call (the dirty pump + * only restores it AFTER that call returns). Without clearing it here, a + * NESTED interrupt taken inside the handler — e.g. a VBLANK delivered while + * the kernel's async-poll memory-card callback runs (in_exception, IEc + * re-enabled) — inherits the OUTER interrupted block's stale resume PC as its + * EPC. same-thread-restore then matches (cpu->pc == that stale EPC) and + * resumes the OUTER block with the CALLBACK's registers: the Ape memcard + * consumer at 0x801382CC gets v1 (jumptable index) = the callback's pointer + * 0x800B4E30, bounds-check already passed, `jr jumptable[garbage]` faults + * (DISPATCH FATAL misaligned 0x3) — or, in the soft variant, drives a wrong + * branch that skips DeliverEvent(0xF0000011,4) and the card scene wedges. + * (irqctx_ring proof: exit_pc=0x801382CC, ra_saved=0x80010094 [callback], + * v1_saved=0x800B4E30, restored=1.) + * + * Cleared here so a nested delivery latches the ACTUALLY-running code's PC: + * the callback's own dirty pump re-sets g_dirty_safe_resume_pc for its + * blocks, and compiled handler code uses s_compiled_interrupt_resume_pc. + * Restored after the dispatch loop (below), which the RFE/escape longjmp + * lands in — so the escape can't leave the latch wrong. A/B off: + * PSX_EXC_CLEAR_RESUME_LATCH=0. */ + static int s_clear_resume_latch = -1; + if (s_clear_resume_latch < 0) { + const char *e = getenv("PSX_EXC_CLEAR_RESUME_LATCH"); + s_clear_resume_latch = (e && *e) ? atoi(e) : 1; + } + uint32_t saved_dirty_resume_pc = g_dirty_safe_resume_pc; + if (s_clear_resume_latch) g_dirty_safe_resume_pc = 0; + /* Wall-time sampler phase: the handler dispatch runs recompiled static + * BIOS text. Same save/restore contract as g_dirty_interp_active above — + * the restore sits after the setjmp loop, covering longjmp landings. */ + extern int g_exec_phase; + int prev_exec_phase = g_exec_phase; + g_exec_phase = 3; /* Dispatch-depth contract across the async handler (Tomba 2 splash, post * overlay-floor fix). The interrupt can be delivered from a NESTED dispatch * (the local-flow pump runs inside dirty_ram_dispatch at depth > 0). The @@ -954,6 +1116,7 @@ void psx_check_interrupts(CPUState* cpu) { * ("execution completed" abnormal exit). */ int saved_dispatch_depth = g_psx_dispatch_depth; g_psx_dispatch_depth = 0; + g_exc_setjmp_epoch++; /* new setjmp frame armed (see decl above) */ for (;;) { int jmp_val = setjmp(exception_jmpbuf); if (jmp_val == 2) { @@ -982,6 +1145,12 @@ void psx_check_interrupts(CPUState* cpu) { g_exception_owner_fiber = prev_owner_fiber; g_pending_exception_longjmp = prev_pending; g_dirty_interp_active = prev_interp_active; + /* Ape memcard fix #3: restore the resume-PC latch cleared at handler entry + * (see above). Sits with the other post-dispatch-loop restores so an + * RFE/RestoreState longjmp — which lands in the setjmp loop above — can't + * strand it at 0. */ + if (s_clear_resume_latch) g_dirty_safe_resume_pc = saved_dirty_resume_pc; + g_exec_phase = prev_exec_phase; /* Restore the interrupted code's dispatch nesting (see save above). The * handler's frames were unwound by longjmp without decrementing, so the * live counter is meaningless here — overwrite, don't decrement. */ @@ -1117,6 +1286,118 @@ void psx_check_interrupts(CPUState* cpu) { : psx_get_cycle_count() + POST_EXC_CLAIMED_COOLDOWN_CYCLES; } if (g_ls_suppress_record > 0) g_ls_suppress_record--; + + /* In-exception thread switch (Ape Escape NEW GAME memcard scene). + * + * A game can run its OWN cooperative scheduler inside the kernel exception + * handler: its VBlank/exception callback moves dword_108->entry (PCB[0], the + * current-thread pointer) to the next thread, and ReturnFromException (0xF40) + * then restores THAT thread. On real hardware the RFE atomically switches + * execution to the new thread. (docs/psx_bios_disasm.txt: both ExceptionHandler + * 0xC80 and ReturnFromException 0xF40 key off dword_108->entry.) + * + * Under the HLE TCB scheduler the interrupted thread is running via a + * structured dispatch on the host stack. Returning normally here does NOT + * transfer to the new thread — the generated block leader just falls through + * to the interrupted thread's next block (it never re-reads cpu->pc). The + * interrupted thread therefore keeps physically executing while PCB[0] already + * names a different thread; a subsequent exception then saves the running + * (old) thread's EPC into the NEW thread's TCB, corrupting it. That is the + * Ape "Checking… MEMORY CARD" wedge: the transition thread's ec8 PC is written + * into main's TCB EPC, main is later RFE'd there with main's regs, and its + * store lands on the transition thread's TCB status word (kill → target_missing + * forever). See ape_thread_smear_rootcause.md. + * + * Honor the switch exactly as a cooperative ChangeThread is honored: unwind to + * psx_scheduler_run and dispatch whatever PCB[0] now points at, from its + * committed TCB context (the guest handler already saved the interrupted + * thread into ITS own TCB before the switch). Gated to the OUTERMOST exception + * (prev_in_exception==0 → no live outer exception_jmpbuf frame to skip) and to + * the HLE scheduler (the legacy fiber bridge has no g_scheduler_jmpbuf and + * switches threads by fiber). */ + if (prev_in_exception == 0 && psx_hle_scheduler_enabled() && + entry_tcb != 0u && exit_tcb != 0u && entry_tcb != exit_tcb) { + extern uint32_t psx_read_word(uint32_t addr); /* memory.c (plain RAM read) */ + uint32_t new_state = psx_read_word(exit_tcb & 0x1FFFFFFFu); + if (new_state == 0x4000u) { /* the new current thread must be runnable */ + /* Fix #2: honor the switch ONLY at the outermost dispatch boundary, + * where the outgoing thread's guest state is fully materialized in + * CPUState. Nested inside a host call unit / dirty pump, the outgoing + * thread's PC lives only in the resume-PC latch, which desyncs from the + * live GPRs when the IRQ was taken deep inside a nested call (the + * async-poll callback) — switching there commits a poisoned TCB context + * and the resumed thread faults on a smeared jumptable index. */ + extern int g_call_unit_depth; + int at_outermost = (g_psx_dispatch_depth == 0 && g_call_unit_depth == 0); + /* Defer only when nested AND the outgoing thread has a valid resume PC + * of its OWN (g_exception_real_epc — the architectural EPC latched for + * THIS delivery). The guest handler's in-exception ReturnFromException + * (traps.c syscall 3, in_exception path) has already restored the TARGET + * thread's GPRs and set cpu->pc to the target's EPC; to resume the + * OUTGOING thread transparently we must put cpu->pc back to ITS resume PC + * as well, not leave it pointing at the target. If we have no clean + * outgoing resume PC (sentinel / clean-trampoline boundary), or the + * toggle is off, or we are already outermost, switch immediately. */ + /* Low BIOS/kernel code is already scheduler code; deferring it can + * starve a target thread by re-entering the same VBlank EPC forever. */ + uint32_t epc_phys = g_exception_real_epc & 0x1FFFFFFFu; + int low_kernel_epc = (epc_phys < 0x00010000u); + int can_defer = defer_switch_enabled() && !low_kernel_epc && !at_outermost && + g_exception_real_epc != 0u && + (g_exception_real_epc & 0x3u) == 0u && + g_exception_real_epc != (uint32_t)PSX_EXC_SENTINEL_PC && + psx_is_dispatchable(g_exception_real_epc); + if (!can_defer) { + /* Outermost / no clean deferral: honor immediately — the guest + * handler's save of the outgoing thread is consistent here (or this + * is the pre-existing baseline path). */ + debug_server_log_thread_event(30, cpu, entry_tcb, exit_tcb, cpu->pc); + /* The longjmp skips psx_check_interrupts_at's restore of the compiled + * resume-PC latch; clear it so the next thread's block-leader delivery + * recomputes real_pc cleanly rather than inheriting a stale value. */ + s_compiled_interrupt_resume_pc = 0; + /* This escape uses g_scheduler_jmpbuf (unwinding to psx_scheduler_run), + * NOT exception_jmpbuf — so it BYPASSES the landing above that restores + * g_dirty_interp_active. From the dirty-RAM entry poll the flag is 1 + * and would LEAK across the unwind; we abandon the dirty-interp + * dispatch here, so it MUST be 0 (dirty_ram_interp.c ~205-210 contract; + * no-op from compiled code where the flag was already 0). */ + g_dirty_interp_active = 0; + g_sched_escape.target_tcb = exit_tcb; + g_sched_escape.resume_pc = 0; + g_sched_escape.reason = PSX_RUN_YIELD_TO_TCB; + longjmp(g_scheduler_jmpbuf, 1); /* unwind to psx_scheduler_run; never returns */ + } + /* Nested in host dispatch: DEFER. The guest handler has already saved a + * POISONED context for the outgoing thread (latched PC ahead of the live + * registers), but we do NOT switch off it. Instead resume the outgoing + * thread transparently — restoring its clean pre-exception GPRs/HI/LO/SR + * (do_restore above ran false for this genuine cross-thread case) — so it + * finishes its nested calls; record the switch as pending and honor it at + * the next outermost boundary (top of this function), re-saving the thread + * cleanly there. Keep PCB[0] naming the STILL-RUNNING thread during the + * interim so any interim exception saves into the right TCB (closes the + * mis-save window the immediate switch used to avoid); the guest's intent + * to run exit_tcb is preserved as the pending switch. Coalesces: the newest + * guest ChangeThread wins the target. */ + for (int i = 0; i < 32; i++) cpu->gpr[i] = saved_gpr[i]; + cpu->hi = saved_hi; + cpu->lo = saved_lo; + cpu->cop0[COP0_SR] = sr; /* pre-exception SR (IEc was set — gate above) */ + cpu->pc = g_exception_real_epc; /* resume the OUTGOING thread at ITS own PC, + * not the target PC the in-exception RFE + * left in cpu->pc — else it runs the target's + * code under the outgoing thread and never + * reaches its own block leaders to be honored. */ + psx_sched_set_current_tcb(cpu, entry_tcb); + s_defer_switch_from = entry_tcb; + s_defer_switch_target = exit_tcb; + s_defer_switch_pending = 1; + s_compiled_interrupt_resume_pc = 0; + debug_server_log_thread_event(31, cpu, entry_tcb, exit_tcb, cpu->pc); + /* fall through: return normally — the outgoing thread resumes. */ + } + } #ifdef PSX_COSIM #undef COSIM_IRQ_NOTE #undef COSIM_IRQ_TAKE_PC diff --git a/runtime/src/main.cpp b/runtime/src/main.cpp index 56f3cb70..12b4239f 100644 --- a/runtime/src/main.cpp +++ b/runtime/src/main.cpp @@ -12,6 +12,7 @@ #include "psx_interpreter.h" #include "cdrom.h" #include "fntrace.h" +#include "text_xlate.h" #include "boot_state.h" #include "bios_hle.h" #include "savestate.h" @@ -30,7 +31,16 @@ #include "latency_ring.h" #include "sio.h" #include "spu.h" +#include "audio_trace.h" #include "spu_shadow.h" + +/* Shared clock-domain bridge: band-limited polyphase resampler + P-only DRC. + * The SPU renders at a fixed 44100/60 per wall-clock frame while the host + * consumes on its own crystal; with no resampling/DRC the queue slowly drifts + * to underrun (silence gaps). The bridge resamples ~1:1 with a <=+/-0.5% ratio + * trim to hold the ring near target -- no gaps. See recomp_audio_drc.h. */ +#define RECOMP_AUDIO_DRC_IMPL +#include "recomp_audio_drc.h" #include "memcard.h" #include "debug_server.h" #include "crash_trace.h" @@ -39,12 +49,14 @@ #include "game_options.h" #include "crc32.h" #include "disc_identity.h" +#include "psx_keybinds.h" /* configurable keyboard->DualShock keybinds (keybinds.ini) */ #if defined(PSX_LAUNCHER) #include "launcher.h" #endif #include <SDL.h> #include <algorithm> #include <cctype> +#include <cmath> #include <cstdio> #include <cstring> #include <cstdlib> @@ -83,6 +95,9 @@ extern "C" uint64_t gte_get_exec_count(void); extern "C" void memory_init(const char* bios_path); extern "C" void memory_set_sr_ptr(const uint32_t *p); extern "C" uint32_t memory_get_bios_checksum(void); +extern "C" void dirty_ram_register_text_image(uint32_t phys_lo, + const uint8_t *bytes, + uint32_t len); /* dma.c */ extern "C" void dma_init(void); @@ -200,6 +215,9 @@ static int g_video_aspect_den = 3; * tagged sprite prims + HUD SPRT center-squash. Inert at 0/false. */ static uint32_t g_ws_anchor_addr = 0; static bool g_ws_hud_sprt = false; +/* Runtime-only transition cleanup; kept out of gpu.h because generated game + * units include that ABI header and do not need this frontend-only setter. */ +extern "C" void gpu_ws_set_clear_reveal(int on); /* Widescreen engages at game entry (fntrace_is_game_started): the BIOS boot * — Sony logo, PS logo, shell — presents authentic 4:3 with no GTE squash. * Starts true when the configured aspect is already 4:3 (nothing to engage). */ @@ -275,6 +293,41 @@ uint64_t g_turbo_loads_frames = 0; /* vblanks run unpaced (observability) */ static SDL_AudioDeviceID sdl_audio_device; static int16_t sdl_audio_buf[2048 * 2]; +/* DRC bridge. Producer (sdl_audio_pump) runs on the main loop thread under + * SDL_LockAudioDevice; consumer (sdl_drc_callback) runs on the SDL audio + * thread. */ +static rab_bridge s_drc; +static bool s_drc_ready = false; + +/* Observability + A/B. PSXRECOMP_AUDIO_LEGACY=1 keeps the historical push model + * (SDL_QueueAudio, no bridge) so the underrun baseline can be measured against + * the bridge from a single build; default (unset) = bridge/pull model. Output + * health is surfaced through the audio_stats TCP command (no stderr probe). */ +static bool audio_legacy_mode(void) { + static int v = -1; + if (v < 0) { const char* e = getenv("PSXRECOMP_AUDIO_LEGACY"); v = (e && *e && *e != '0') ? 1 : 0; } + return v != 0; +} +/* Legacy-mode underrun counter: incremented when the SDL queue is found empty + * at pump time (the device was silence-filling = an audible gap). */ +static uint64_t g_legacy_underruns = 0; +/* Set at unmute so the pump resyncs its bridge-underrun baseline past the + * intentional mute-drain instead of reporting it as gaps. */ +int g_audio_unmute_resync = 0; +/* Actual device rate the host opened at (bridge mode may differ from 44100; + * the T3 tap ring runs at this rate and its WAV dump must say so). */ +int g_audio_host_rate = 44100; + +static void sdl_drc_callback(void* /*user*/, Uint8* stream, int len) { + if (!s_drc_ready) { std::memset(stream, 0, (size_t)len); return; } + int frames = len / (int)(2 * sizeof(int16_t)); /* stereo S16 */ + rab_pull(&s_drc, reinterpret_cast<int16_t*>(stream), frames); + /* T3 tap in bridge mode: the exact device-rate bytes the host consumes. + * This callback is the tap's single writer while the bridge is active. */ + audio_trace_pcm(AUDIO_TAP_HOST, reinterpret_cast<const int16_t*>(stream), + frames); +} + static std::filesystem::path find_upward(std::filesystem::path start, const std::filesystem::path& marker) { std::error_code ec; @@ -635,9 +688,10 @@ static void shutdown_runtime(void) { overlay_capture_write_json(); if (sdl_audio_device) { SDL_ClearQueuedAudio(sdl_audio_device); - SDL_CloseAudioDevice(sdl_audio_device); + SDL_CloseAudioDevice(sdl_audio_device); /* stops the pull callback */ sdl_audio_device = 0; } + if (s_drc_ready) { rab_free(&s_drc); s_drc_ready = false; } close_controller(); debug_server_shutdown(); } @@ -664,16 +718,74 @@ static void sdl_audio_pump(void) { if (!sdl_audio_device) return; const uint32_t bytes_per_frame = sizeof(int16_t) * 2u; - const uint32_t max_queue_bytes = 44100u * bytes_per_frame / 5u; - if (SDL_GetQueuedAudioSize(sdl_audio_device) > max_queue_bytes) return; + const bool legacy = audio_legacy_mode(); + static int had_audio = 0; + uint32_t queued = 0; /* RENDER event b: bytes (legacy) / fill ms (bridge) */ + if (legacy) { + /* Historical push model + baseline measurement: a drained (==0) queue + * means the device was silence-filling since the last pump = a gap. + * Only meaningful after the first audio has been queued. */ + const uint32_t max_queue_bytes = 44100u * bytes_per_frame / 5u; + queued = SDL_GetQueuedAudioSize(sdl_audio_device); + if (queued == 0 && had_audio) { + g_legacy_underruns++; + audio_trace_event(AUDIO_EV_UNDERRUN, 0, 0); + } + if (queued > max_queue_bytes) { + audio_trace_event(AUDIO_EV_PUMP_SKIP, queued, 0); + return; + } + } else { + /* No host-queue backpressure check: the bridge's ring + DRC hold the + * fill near target, so we always render this frame and push it. */ + if (!s_drc_ready) return; + /* Surface bridge underruns (counted on the SDL audio thread) into the + * event ring from this thread — the event ring is single-writer. + * Across a turbo mute the ring intentionally runs dry (the pump stops + * while the callback keeps pulling); those dry pulls are the mute, + * not gaps — resync past them instead of reporting them. */ + rab_stats st; + rab_get_stats(&s_drc, &st); + static uint64_t prev_underruns = 0; + extern int g_audio_unmute_resync; + if (g_audio_unmute_resync) { + prev_underruns = st.underrun_events; + g_audio_unmute_resync = 0; + } else if (st.underrun_events > prev_underruns) { + audio_trace_event(AUDIO_EV_UNDERRUN, + (uint32_t)(st.underrun_events - prev_underruns), 1); + prev_underruns = st.underrun_events; + } + queued = (uint32_t)st.last_fill_ms; + } - static double sample_accum = 0.0; - sample_accum += 44100.0 / 60.0; - int frames = (int)sample_accum; - sample_accum -= (double)frames; + /* Faithful sample budget: the SPU is clocked by the GUEST, not by host + * presents. 33.8688 MHz / 44100 Hz = exactly 768 guest cycles per output + * frame, so production tracks guest time precisely — including the real + * NTSC 59.94 Hz vblank — instead of assuming 60.00 Hz per present, which + * built in a systematic -0.1% production deficit (measured: 43950/s + * produced vs 44100/s consumed = recurring ring underruns no +/-0.5% + * DRC trim could absorb during jitter spikes). */ + extern uint64_t psx_cycle_count; + static uint64_t last_cycles = 0; + static uint64_t cycle_carry = 0; + const uint64_t now_cycles = psx_cycle_count; + if (last_cycles == 0) last_cycles = now_cycles; + uint64_t delta = (now_cycles - last_cycles) + cycle_carry; + last_cycles = now_cycles; + int frames = (int)(delta / 768u); + cycle_carry = delta % 768u; if (frames <= 0) return; - if (frames > 2048) frames = 2048; + if (frames > 2048) { + /* A burst beyond one buffer (e.g. right after an unmute or a long + * stall): render one full buffer and DROP the remainder of the debt — + * same semantic as the mute model (voice positions freeze across the + * gap) rather than time-compressing a backlog into garble. */ + frames = 2048; + cycle_carry = 0; + } + audio_trace_event(AUDIO_EV_RENDER, (uint32_t)frames, queued); spu_render(sdl_audio_buf, frames); if (sdl_audio_fadein_left > 0) { const float g0 = 1.0f - (float)sdl_audio_fadein_left @@ -684,8 +796,21 @@ static void sdl_audio_pump(void) { sdl_audio_gain_ramp(sdl_audio_buf, ramp, g0, g1); sdl_audio_fadein_left -= ramp; } - SDL_QueueAudio(sdl_audio_device, sdl_audio_buf, - (uint32_t)frames * bytes_per_frame); + if (legacy) { + /* T3 tap: the exact post-fade bytes handed to the host audio queue. */ + audio_trace_pcm(AUDIO_TAP_HOST, sdl_audio_buf, frames); + SDL_QueueAudio(sdl_audio_device, sdl_audio_buf, + (uint32_t)frames * bytes_per_frame); + had_audio = 1; + } else { + /* Hand to the bridge (band-limited resample + DRC) instead of + * SDL_QueueAudio. Lock guards the SPSC ring against the pull callback. + * The T3 tap moves to sdl_drc_callback: what the device actually + * receives is the bridge's device-rate output, not this buffer. */ + SDL_LockAudioDevice(sdl_audio_device); + rab_push(&s_drc, sdl_audio_buf, frames); + SDL_UnlockAudioDevice(sdl_audio_device); + } } /* Audio gating across turbo-loads transitions. @@ -701,8 +826,40 @@ static void sdl_audio_pump(void) { * re-trigger within a few frames; without the debounce the mute would * flicker audibly), then resume pumping with a rising ramp applied * across the first ~50 ms of samples (sdl_audio_pump above). */ +/* Bridge/legacy output health, surfaced through the audio_stats TCP command + * (debug_server.c) — no stderr probe; rule 3. */ +extern "C" int psx_audio_out_stats(double *fill_ms, uint64_t *underruns, + uint64_t *overflow_drops, double *correction, + int *legacy, int *host_rate) +{ + extern int g_audio_host_rate; /* set at device open */ + *legacy = audio_legacy_mode() ? 1 : 0; + *host_rate = g_audio_host_rate; + if (*legacy || !s_drc_ready) { + *fill_ms = sdl_audio_device + ? (double)SDL_GetQueuedAudioSize(sdl_audio_device) + / (44100.0 * 4.0) * 1000.0 + : 0.0; + *underruns = g_legacy_underruns; + *overflow_drops = 0; + *correction = 0.0; + return sdl_audio_device != 0; + } + rab_stats st; + rab_get_stats(&s_drc, &st); + *fill_ms = st.last_fill_ms; + *underruns = st.underrun_events; + *overflow_drops = st.overflow_drops; + *correction = st.last_correction; + return 1; +} + static void sdl_audio_update(int turbo_active) { if (!sdl_audio_device) return; + { /* Tag audio events with the vblank frame counter. */ + extern uint64_t s_frame_count; + audio_trace_note_frame((uint32_t)s_frame_count); + } const int HANGOVER_FRAMES = 8; /* ~133 ms at 60 fps */ static int muted = 0; static int hangover = 0; @@ -719,8 +876,16 @@ static void sdl_audio_update(int turbo_active) { sdl_audio_fadein_left = 0; spu_render(sdl_audio_buf, tail); sdl_audio_gain_ramp(sdl_audio_buf, tail, g0, 0.0f); - SDL_QueueAudio(sdl_audio_device, sdl_audio_buf, - (uint32_t)tail * sizeof(int16_t) * 2u); + audio_trace_event(AUDIO_EV_MUTE, (uint32_t)tail, 0); + if (audio_legacy_mode()) { + audio_trace_pcm(AUDIO_TAP_HOST, sdl_audio_buf, tail); + SDL_QueueAudio(sdl_audio_device, sdl_audio_buf, + (uint32_t)tail * sizeof(int16_t) * 2u); + } else if (s_drc_ready) { + SDL_LockAudioDevice(sdl_audio_device); + rab_push(&s_drc, sdl_audio_buf, tail); + SDL_UnlockAudioDevice(sdl_audio_device); + } muted = 1; } hangover = HANGOVER_FRAMES; @@ -730,6 +895,9 @@ static void sdl_audio_update(int turbo_active) { if (hangover > 0) { hangover--; return; } muted = 0; sdl_audio_fadein_left = sdl_audio_fade_samples; + audio_trace_event(AUDIO_EV_UNMUTE, (uint32_t)sdl_audio_fadein_left, 0); + extern int g_audio_unmute_resync; + g_audio_unmute_resync = 1; } sdl_audio_pump(); } @@ -1016,6 +1184,12 @@ static void load_input_config(const char* argv0) { if (!controller_enabled) { for (auto& entry : controller_map) entry.sources.clear(); } + + /* Configurable KEYBOARD keybinds (keybinds.ini, next to the exe) — separate + * from input.ini's gamepad map. Loads the user's map (or writes defaults on + * first run). The launcher may have just edited+saved this file; we re-read + * it here so the runtime always reflects the current bindings. */ + psx_keybinds_init(argv0); } static void close_player(PlayerInput& p) { @@ -1134,26 +1308,13 @@ static bool controller_source_pressed_h(SDL_GameController* h, const ControllerS } } -static uint16_t pad_from_keyboard(void) { +/* Keyboard -> PSX button word for `player` (1 or 2), via the configurable + * keybinds.ini map (psx_keybinds). Defaults reproduce the old hardcoded layout + * (arrows=d-pad, X/S/Z/A=Cross/Circle/Square/Triangle, Q/W/E/R=L1/R1/L2/R2, + * Return=Start, RShift=Select), so out-of-the-box behaviour is unchanged. */ +static uint16_t pad_from_keyboard(int player) { const Uint8* keys = SDL_GetKeyboardState(NULL); - uint16_t buttons = 0xFFFF; /* all released */ - - if (keys[SDL_SCANCODE_UP]) buttons &= ~PAD_UP; - if (keys[SDL_SCANCODE_DOWN]) buttons &= ~PAD_DOWN; - if (keys[SDL_SCANCODE_LEFT]) buttons &= ~PAD_LEFT; - if (keys[SDL_SCANCODE_RIGHT]) buttons &= ~PAD_RIGHT; - if (keys[SDL_SCANCODE_RETURN]) buttons &= ~PAD_START; - if (keys[SDL_SCANCODE_RSHIFT]) buttons &= ~PAD_SELECT; - if (keys[SDL_SCANCODE_X]) buttons &= ~PAD_CROSS; - if (keys[SDL_SCANCODE_Z]) buttons &= ~PAD_SQUARE; - if (keys[SDL_SCANCODE_S]) buttons &= ~PAD_CIRCLE; - if (keys[SDL_SCANCODE_A]) buttons &= ~PAD_TRIANGLE; - if (keys[SDL_SCANCODE_Q]) buttons &= ~PAD_L1; - if (keys[SDL_SCANCODE_W]) buttons &= ~PAD_R1; - if (keys[SDL_SCANCODE_E]) buttons &= ~PAD_L2; - if (keys[SDL_SCANCODE_R]) buttons &= ~PAD_R2; - - return buttons; + return psx_keybinds_pad_word(keys, player); } static uint16_t controller_pad_buttons(SDL_GameController* h) { @@ -1170,28 +1331,39 @@ static uint16_t controller_pad_buttons(SDL_GameController* h) { return buttons; } -/* Map an SDL axis (-32768..32767) to a PSX analog byte (0..255, 0x80 centred), - * applying the configured centre deadzone: travel within controller_deadzone - * reads as centred (0x80) and the remaining travel is rescaled to the full range - * so the stick still reaches the extremes. Gives clean variable analog speed - * with no centre drift. */ -static uint8_t axis_to_pad_byte(int16_t v) { - const int dz = controller_deadzone; - int av = v < 0 ? -(int)v : (int)v; /* |v|, 0..32768 */ - if (av <= dz) return 0x80; /* inside deadzone -> centred */ - int range = 32767 - dz; - if (range < 1) range = 1; - int mag = (av - dz) * 32767 / range; /* 0..32767 past the deadzone */ - if (mag > 32767) mag = 32767; - int sv = v < 0 ? -mag : mag; - int b = (sv + 32768) >> 8; /* 0..255 */ - if (b < 0) b = 0; else if (b > 255) b = 255; - return (uint8_t)b; +/* Radial deadzone: process a stick's X and Y together so the dead region is a + * small CIRCLE, not a per-axis square. Travel within controller_deadzone reads + * centred (0x80/0x80); beyond it the vector MAGNITUDE is rescaled to the full + * range along the stick's true direction, so diagonals are preserved (a + * per-axis deadzone snaps diagonals toward the cardinals and feels notchy — + * bad for directional analog like Ape Escape's net swing). The magnitude is + * capped at 32767 before rescale so a full-diagonal push (raw mag ~46341) maps + * to ~0x9E/0x9E per axis — the circular gate a real DualShock stick reports, + * not 0xFF/0xFF. At dz==0 it reduces to a plain magnitude-preserving map. */ +static void axes_to_pad_pair(int16_t vx, int16_t vy, uint8_t* obx, uint8_t* oby) { + const double dz = controller_deadzone; + double x = vx, y = vy; + double mag = std::sqrt(x * x + y * y); /* 0 .. ~46341 */ + if (mag <= dz || mag <= 0.0) { *obx = 0x80; *oby = 0x80; return; } + double capped = mag > 32767.0 ? 32767.0 : mag; + double range = 32767.0 - dz; + if (range < 1.0) range = 1.0; + double newmag = (capped - dz) * 32767.0 / range; /* 0 .. 32767 */ + double scale = newmag / mag; /* along true direction */ + int sx = (int)std::lround(x * scale); + int sy = (int)std::lround(y * scale); + if (sx > 32767) sx = 32767; else if (sx < -32768) sx = -32768; + if (sy > 32767) sy = 32767; else if (sy < -32768) sy = -32768; + int bx = (sx + 32768) >> 8; + int by = (sy + 32768) >> 8; + *obx = (uint8_t)(bx < 0 ? 0 : (bx > 255 ? 255 : bx)); + *oby = (uint8_t)(by < 0 ? 0 : (by > 255 ? 255 : by)); } -/* Buttons for a player's selected device (0xFFFF = none pressed). */ -static uint16_t pad_buttons_for(const PlayerInput& p) { - if (p.kind == 1) return pad_from_keyboard(); +/* Buttons for a player's selected device (0xFFFF = none pressed). `player` is + * 1 or 2 — selects which keybinds.ini section drives a keyboard port. */ +static uint16_t pad_buttons_for(const PlayerInput& p, int player) { + if (p.kind == 1) return pad_from_keyboard(player); if (p.kind == 2) return controller_pad_buttons(p.handle); return 0xFFFF; } @@ -1209,21 +1381,23 @@ static uint16_t pad_buttons_for(const PlayerInput& p) { * D-pad instead flips the pad to digital, so the game runs its own d-pad path) * and true in pinned-ANALOG mode. The keyboard branch always folds the arrows * — they are that player's only stick source. */ -static void pad_sticks_for(const PlayerInput& p, uint8_t out[4], bool fold_dpad) { +static void pad_sticks_for(const PlayerInput& p, int player, uint8_t out[4], bool fold_dpad) { out[0] = out[1] = out[2] = out[3] = 0x80; if (p.kind == 1) { + /* Keyboard analog: the configurable left/right stick-direction binds + * (default = arrow keys on the LEFT stick; RIGHT stick unbound), so the + * old keyboard analog behaviour is preserved unless the user rebinds. */ const Uint8* keys = SDL_GetKeyboardState(NULL); - if (keys[SDL_SCANCODE_LEFT]) out[0] = 0x00; - if (keys[SDL_SCANCODE_RIGHT]) out[0] = 0xFF; - if (keys[SDL_SCANCODE_UP]) out[1] = 0x00; - if (keys[SDL_SCANCODE_DOWN]) out[1] = 0xFF; + psx_keybinds_sticks(keys, player, out); return; } if (p.kind == 2 && p.handle) { - out[0] = axis_to_pad_byte(SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_LEFTX)); - out[1] = axis_to_pad_byte(SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_LEFTY)); - out[2] = axis_to_pad_byte(SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_RIGHTX)); - out[3] = axis_to_pad_byte(SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_RIGHTY)); + axes_to_pad_pair(SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_LEFTX), + SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_LEFTY), + &out[0], &out[1]); + axes_to_pad_pair(SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_RIGHTX), + SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_RIGHTY), + &out[2], &out[3]); if (fold_dpad) { if (SDL_GameControllerGetButton(p.handle, SDL_CONTROLLER_BUTTON_DPAD_LEFT)) out[0] = 0x00; if (SDL_GameControllerGetButton(p.handle, SDL_CONTROLLER_BUTTON_DPAD_RIGHT)) out[0] = 0xFF; @@ -1245,7 +1419,7 @@ static bool hybrid_stick_active(const PlayerInput& p) { int ly = SDL_GameControllerGetAxis(p.handle, SDL_CONTROLLER_AXIS_LEFTY); return lx > dz || lx < -dz || ly > dz || ly < -dz; } -static bool hybrid_dpad_active(const PlayerInput& p, bool kb_always) { +static bool hybrid_dpad_active(const PlayerInput& p, int player, bool kb_always) { if (p.kind == 2 && p.handle) { if (SDL_GameControllerGetButton(p.handle, SDL_CONTROLLER_BUTTON_DPAD_LEFT) || SDL_GameControllerGetButton(p.handle, SDL_CONTROLLER_BUTTON_DPAD_RIGHT) || @@ -1255,9 +1429,7 @@ static bool hybrid_dpad_active(const PlayerInput& p, bool kb_always) { } if (p.kind == 1 || kb_always) { const Uint8* keys = SDL_GetKeyboardState(NULL); - if (keys[SDL_SCANCODE_LEFT] || keys[SDL_SCANCODE_RIGHT] || - keys[SDL_SCANCODE_UP] || keys[SDL_SCANCODE_DOWN]) - return true; + if (psx_keybinds_dpad_active(keys, player)) return true; } return false; } @@ -1272,17 +1444,64 @@ static bool hybrid_dpad_active(const PlayerInput& p, bool kb_always) { * (stick -> analog, D-pad -> digital) so the game runs its own analog or * digital input path exactly as on hardware (mirrors the inline block this * helper was extracted from for the low-latency re-sample). */ -/* Dev builds (PSX_DEBUG_TOOLS — the dev/debug configuration) ALWAYS feed the - * keyboard into Player 1 alongside whatever device the launcher routed to port 1, - * so a tester can drive the game from EITHER the keyboard OR the selected - * controller (e.g. a DualShock) at the same time, with no reconfiguration. The - * keyboard map and the controller are merged (active-low AND) onto the P1 pad - * word. Release builds keep the single launcher-selected device per port. */ -#if defined(PSX_DEBUG_TOOLS) -static const bool g_dev_kb_p1 = true; -#else -static const bool g_dev_kb_p1 = false; -#endif +/* Dev input mode: while enabled, Player 1 is driven by the keyboard AND EVERY + * connected game controller, all merged together (active-low AND) onto the P1 pad + * word — so a tester can navigate P1 from whatever is plugged in (keyboard, the + * launcher-assigned pad, or any other controller) with no reconfiguration. The P1 + * pad TYPE is left unchanged: a launcher-assigned analog DualShock still presents + * as analog (so the game's analog input path / SIO handshake cadence is preserved + * exactly), and merged sources only contribute button/stick STATE, never a type + * downgrade. Controlled by PSX_DEV_INPUT (default ON for the dev workflow); set + * PSX_DEV_INPUT=0 to restore strict single-device-per-port routing. */ +static bool dev_any_input_enabled() { + static int cached = -1; + if (cached < 0) { + const char* e = std::getenv("PSX_DEV_INPUT"); + cached = (e && (e[0] == '0' || e[0] == 'n' || e[0] == 'N' || e[0] == 'f' || e[0] == 'F')) ? 0 : 1; + } + return cached != 0; +} + +/* Merge (active-low AND) the button words of every connected game controller. + * Handles are resolved from SDL's already-open set when possible and opened once + * on first sight otherwise (kept open for the process lifetime; if a slot later + * closes a shared device this self-heals by reopening next frame). This does NOT + * disturb per-slot routing — SDL returns the same handle for an already-open + * device, so reads are shared and harmless. */ +static uint16_t dev_all_controllers_buttons() { + uint16_t btn = 0xFFFF; + const int n = SDL_NumJoysticks(); + for (int i = 0; i < n; i++) { + if (!SDL_IsGameController(i)) continue; + SDL_JoystickID inst = SDL_JoystickGetDeviceInstanceID(i); + SDL_GameController* h = SDL_GameControllerFromInstanceID(inst); + if (!h) h = SDL_GameControllerOpen(i); /* open once; SDL keeps it */ + if (h) btn &= controller_pad_buttons(h); + } + return btn; +} + +/* Fold the first live (non-neutral) left/right stick from ANY connected + * controller into st[] (lx,ly,rx,ry). Lets an analog-mode P1 steer from + * whatever pad is plugged in, not only the launcher-assigned one. Only axes + * that read past the deadzone override; a neutral pad leaves st[] untouched. */ +static void dev_any_controller_sticks(uint8_t st[4]) { + const int n = SDL_NumJoysticks(); + for (int i = 0; i < n; i++) { + if (!SDL_IsGameController(i)) continue; + SDL_JoystickID inst = SDL_JoystickGetDeviceInstanceID(i); + SDL_GameController* h = SDL_GameControllerFromInstanceID(inst); + if (!h) h = SDL_GameControllerOpen(i); + if (!h) continue; + uint8_t lx, ly, rx, ry; + axes_to_pad_pair(SDL_GameControllerGetAxis(h, SDL_CONTROLLER_AXIS_LEFTX), + SDL_GameControllerGetAxis(h, SDL_CONTROLLER_AXIS_LEFTY), &lx, &ly); + axes_to_pad_pair(SDL_GameControllerGetAxis(h, SDL_CONTROLLER_AXIS_RIGHTX), + SDL_GameControllerGetAxis(h, SDL_CONTROLLER_AXIS_RIGHTY), &rx, &ry); + if (lx != 0x80 || ly != 0x80) { st[0] = lx; st[1] = ly; } + if (rx != 0x80 || ry != 0x80) { st[2] = rx; st[3] = ry; } + } +} static void sample_pad_into_sio(int override) { if (override >= 0) { @@ -1291,39 +1510,53 @@ static void sample_pad_into_sio(int override) { } for (int s = 0; s < 2; s++) { PlayerInput& p = g_players[s]; - const bool kb_here = (g_dev_kb_p1 && s == 0); - if (p.kind == 0 && !kb_here) continue; /* no device in this port */ + const int player = s + 1; /* keybinds.ini section (1|2) */ + /* Dev input: P1 is driven by the keyboard AND every connected controller, + * so a tester can navigate from whatever is plugged in (P2 keeps strict + * per-port routing). */ + const bool dev_here = (dev_any_input_enabled() && s == 0); + if (p.kind == 0 && !dev_here) continue; /* no device in this port */ /* Buttons: merge the assigned device with the keyboard (PSX pad word is - * active-low, so AND combines "pressed on either source"). */ - uint16_t btn = (p.kind != 0) ? pad_buttons_for(p) : (uint16_t)0xFFFF; - if (kb_here) btn &= pad_from_keyboard(); + * active-low, so AND combines "pressed on either source"). In dev mode P1 + * also folds in the keyboard binds and EVERY connected controller. */ + uint16_t btn = (p.kind != 0) ? pad_buttons_for(p, player) : (uint16_t)0xFFFF; + if (dev_here) { + btn &= pad_from_keyboard(1); /* keyboard drives P1 binds */ + btn &= dev_all_controllers_buttons(); /* any plugged-in controller too */ + } sio_set_pad_state_slot(s, btn); - /* Resolve the pad type this frame from the player's mode (a port driven - * by the keyboard alone presents as a digital pad). */ - const int mode = (p.kind != 0) ? p.mode : (int)PSXRecompV4::PAD_MODE_DIGITAL; + /* Resolve the pad type this frame. An assigned device keeps its configured + * mode (a launcher-selected analog DualShock stays analog, so its input + * path / SIO handshake cadence is preserved exactly). A P1 with no assigned + * device but dev-any-input on presents as HYBRID — boots analog like a + * DualShock and auto-drops to digital on the d-pad — so any plugged + * controller and the keyboard both navigate. */ + int mode; + if (p.kind != 0) mode = p.mode; + else if (dev_here) mode = (int)PSXRecompV4::PAD_MODE_HYBRID; + else mode = (int)PSXRecompV4::PAD_MODE_DIGITAL; int eff_analog; uint8_t st[4] = { 0x80, 0x80, 0x80, 0x80 }; if (mode == PSXRecompV4::PAD_MODE_DIGITAL) { eff_analog = 0; } else if (mode == PSXRecompV4::PAD_MODE_ANALOG) { eff_analog = 1; - pad_sticks_for(p, st, /*fold_dpad=*/true); + pad_sticks_for(p, player, st, /*fold_dpad=*/true); } else { /* HYBRID */ - if (hybrid_stick_active(p)) p.hybrid_analog = true; - else if (hybrid_dpad_active(p, kb_here)) p.hybrid_analog = false; + if (hybrid_stick_active(p)) p.hybrid_analog = true; + else if (hybrid_dpad_active(p, player, dev_here)) p.hybrid_analog = false; eff_analog = p.hybrid_analog ? 1 : 0; - if (eff_analog) pad_sticks_for(p, st, /*fold_dpad=*/false); + if (eff_analog) pad_sticks_for(p, player, st, /*fold_dpad=*/false); } - /* In dev builds also fold the keyboard arrows onto the analog stick, so an - * analog-mode P1 still steers from the keyboard. */ - if (kb_here && eff_analog) { + /* Dev mode: fold the keyboard's stick binds AND any connected controller's + * sticks onto the analog stick, so an analog-mode P1 steers from whatever + * is plugged in (P1 binds). */ + if (dev_here && eff_analog) { const Uint8* keys = SDL_GetKeyboardState(NULL); - if (keys[SDL_SCANCODE_LEFT]) st[0] = 0x00; - if (keys[SDL_SCANCODE_RIGHT]) st[0] = 0xFF; - if (keys[SDL_SCANCODE_UP]) st[1] = 0x00; - if (keys[SDL_SCANCODE_DOWN]) st[1] = 0xFF; + psx_keybinds_sticks(keys, 1, st); + dev_any_controller_sticks(st); } /* Push sticks every frame; request the pad type (digital/analog) through * the coherent channel so a hybrid stick<->d-pad flip is applied only at @@ -1361,8 +1594,19 @@ static void sample_headless_pad_into_sio(int override) { * audio sample generation (735 samples/vblank * 60 = 44100/sec) matched * to the SDL audio device drain rate, eliminating queue overflow drops * and underruns. Uncapped, the host runs the simulation at whatever - * speed it can — typically several × realtime — and audio glitches. */ + * speed it can — typically several × realtime — and audio glitches. + * + * The wall-clock pacer target; nudged to the host display refresh at + * window-creation time (sync-to-host-refresh) so the pacer and SDL PRESENTVSYNC + * do not fight — a fixed 59.94 pacer against a 60.00 Hz panel makes rendered + * frames land on an uneven vblank count (a 2/3/1 beat) that reads as + * moving-object judder/flicker. See g_frame_period_ms. */ static constexpr double PSX_FRAME_PERIOD_MS = 1000.0 / 59.94; +/* Live pacer period (ms). Defaults to the PSX rate; set to the host refresh + * period when the panel is within ~2% of 60 Hz so 30fps content pads evenly to + * two host refreshes. Left at the PSX rate on non-~60Hz panels to avoid running + * the sim at the wrong speed. */ +static double g_frame_period_ms = PSX_FRAME_PERIOD_MS; /* ── Host-stack-usage profile (RECURSION_BUG.md §17) ────────────────────────── * The decisive instrument for the long-run freeze. The guest call graph mirrors @@ -1660,7 +1904,7 @@ static void sdl_vblank_present(void) { * hard freeze). */ { static FramePacer pacer = { 0 }; - frame_pacer_wait(&pacer, PSX_FRAME_PERIOD_MS); + frame_pacer_wait(&pacer, g_frame_period_ms); } latency_ring_mark(LAT_PACED); @@ -1833,6 +2077,47 @@ static void sdl_vblank_present(void) { } } } + + /* Frame blending (CRT-persistence masker for 30fps double-buffered + * content). Some games (e.g. Crash Bash menus/characters) leave a + * dynamic object in only one of the two display buffers per 30fps cycle, + * so it strobes on/off as the display alternates buffers — the static + * background, present in both, stays put. Real CRTs hid this via phosphor + * persistence; accurate emulators either time it away (cycle accuracy) or + * offer a "blend frames" option that averages the last two presented + * frames. This is the latter: presentation only, game logic untouched. + * Guarded to the plain software present path (no GL/VK/wide/hires/FMV) so + * the buffer is a simple present_w*h ARGB grid. PSX_FRAME_BLEND=0 off. */ + { + static int blend_cfg = -1; /* -1 unresolved, 0 off, 1 on */ + if (blend_cfg < 0) { + const char* e = getenv("PSX_FRAME_BLEND"); + blend_cfg = (e && e[0] == '1') ? 1 : 0; /* default off (masker only) */ + } + const bool simple_sw = !g_gl_active && !g_vk_active && !wide_present && + !di.depth24 && active_scale == 1; + if (blend_cfg && simple_sw) { + static uint32_t prev_buf[640 * 512]; + static uint32_t prev_px = 0; + const uint32_t npx = present_w * h; + if (npx <= (uint32_t)(640 * 512)) { + if (prev_px == npx) { + for (uint32_t i = 0; i < npx; i++) { + const uint32_t a = sdl_pixel_buf[i]; + const uint32_t b = prev_buf[i]; + prev_buf[i] = a; /* store this frame (pre-blend) */ + sdl_pixel_buf[i] = + 0xFF000000u | + (((a & 0xFEFEFEu) >> 1) + ((b & 0xFEFEFEu) >> 1)); + } + } else { + /* Resolution changed / first frame: seed, no blend. */ + for (uint32_t i = 0; i < npx; i++) prev_buf[i] = sdl_pixel_buf[i]; + prev_px = npx; + } + } + } + } } /* Update only the active display rectangle. The backing texture is sized @@ -1995,10 +2280,11 @@ int main(int argc, char** argv) { bool memcard2_enabled = true; /* [controller] device routing (defaults: P1 keyboard/digital, P2 none). */ /* Dev builds default Player 1 to the first connected controller ("auto"): - * combined with the always-on keyboard merge (g_dev_kb_p1) this means the - * selected controller AND the keyboard both drive P1 with no launcher setup. - * If no controller is present, "auto" opens nothing and the keyboard merge - * still drives P1. Release keeps "keyboard" (the launcher assigns devices). */ + * combined with dev-any-input (dev_any_input_enabled(), default ON) the + * selected controller, EVERY other plugged-in controller, AND the keyboard + * all drive P1 with no launcher setup. If no controller is present, "auto" + * opens nothing and the keyboard/any-controller merge still drives P1. + * Release keeps "keyboard" (the launcher assigns devices). */ #if defined(PSX_DEBUG_TOOLS) std::string p1_device = "auto"; #else @@ -2009,7 +2295,22 @@ int main(int argc, char** argv) { int p2_mode = PSXRecompV4::PAD_MODE_HYBRID; bool ctrl_allow_hybrid = true; /* game.toml [controller] allow_hybrid; false hides Hybrid in the launcher */ bool ctrl_lock_mode = false; /* game.toml [controller] lock_mode; true hides the whole pad-mode selector */ + bool ctrl_lock_device = false; /* game.toml [controller] lock_device; true hides the Player controller cards entirely */ + /* The game-DECLARED port modes, captured at game.toml load and immune to the + * settings.toml overrides below. Under lock_mode these are the only valid + * modes (the game supports exactly one pad type), so they are what the + * runtime clamps to and what the launcher locks its selector to. */ + int ctrl_locked_p1_mode = PSXRecompV4::PAD_MODE_HYBRID; + int ctrl_locked_p2_mode = PSXRecompV4::PAD_MODE_HYBRID; + bool ws_offered = true; /* game.toml [widescreen] offer; false hides the launcher toggle + clamps 4:3 */ + bool ws_ultrawide_offered = false; int resolved_deadzone = -1; /* <0 => keep input.ini/runtime default (12000) */ + /* Localization: the effective language (game.toml default -> settings.toml -> + * launcher choice), applied to the translation layer AFTER the launcher runs. + * lang_menu_options drives the launcher's "Localization" dropdown (empty => + * no dropdown; only games that declare [localization].languages get one). */ + std::string resolved_language = "en"; + std::vector<PSXRecompV4::RuntimeConfig::LanguageOption> lang_menu_options; std::filesystem::path resolved_disc; std::string window_title = PSX_WINDOW_TITLE; uint16_t debug_port = (uint16_t)DEFAULT_DEBUG_PORT; @@ -2035,6 +2336,14 @@ int main(int argc, char** argv) { if (gc.runtime.has_memcard_dir) memcard_dir = gc.runtime.memcard_dir; if (gc.runtime.has_window_title) window_title = gc.runtime.window_title; if (gc.runtime.has_debug_port) debug_port = gc.runtime.debug_port; + /* On-the-fly string translation / localization (framework feature — + * text_xlate.cpp): load translations/ *.toml under the project root + * and select the language. Capture inventory is always-on; APPLY is + * gated by language + table presence. See docs/STRING_TRANSLATION.md. */ + text_xlate_init(gc.project_root.string().c_str(), + gc.runtime.language.c_str()); + resolved_language = gc.runtime.language; /* launcher/settings may override */ + lang_menu_options = gc.runtime.languages; /* launcher localization dropdown */ if (gc.runtime.has_disc_speed) disc_speed = gc.runtime.disc_speed; if (gc.runtime.has_instant_max_per_frame) instant_rate = gc.runtime.instant_max_per_frame; @@ -2071,6 +2380,32 @@ int main(int argc, char** argv) { gc.ws_bg2d_ring_cols, gc.ws_bg2d_layer_count, gc.ws_bg2d_layer_struct_stride); + /* [widescreen] gte_game_mode — 3D-title gameplay detector (Ape). */ + gpu_ws_set_gte_game_mode(gc.ws_gte_game_mode ? 1 : 0); + /* [widescreen] nw_hud_corners — push HUD to the true wide corners. */ + gpu_ws_set_nw_hud_corners(gc.ws_nw_hud_corners ? 1 : 0); + /* Targeted left-HUD packet range — avoids shifting 2D scenery. */ + gpu_ws_set_nw_left_hud_packet_range(gc.ws_nw_left_hud_packet_lo, + gc.ws_nw_left_hud_packet_hi); + /* [widescreen] nw_backdrop — stretch full-frame 2D sky backdrop. */ + gpu_ws_set_nw_backdrop(gc.ws_nw_backdrop ? 1 : 0); + /* [widescreen] clear_reveal — enable opted-in scene/map-boundary + * cleanup of synthetic native-wide margins. */ + gpu_ws_set_clear_reveal(gc.ws_clear_reveal ? 1 : 0); + gpu_ws_set_cull_guard_pixels(gc.ws_cull_guard_pixels); + gpu_ws_set_explicit_cull_sites( + gc.ws_cull_bias_sites.data(), (int)gc.ws_cull_bias_sites.size(), + gc.ws_cull_slti_sites.data(), (int)gc.ws_cull_slti_sites.size()); + /* [widescreen.cull] per-game gates + signature immediates for the + * pattern-scanned interp/sljit widen hooks. A title that never + * opted in must never have its live code scanned and rewritten. */ + gpu_ws_set_auto_hooks(gc.ws_auto_screen_x_cull ? 1 : 0, + gc.ws_auto_backdrop_preload ? 1 : 0); + if (!gc.ws_cull_w_imms.empty() || !gc.ws_cull_h_imms.empty()) + gpu_ws_set_cull_imms(gc.ws_cull_w_imms.data(), (int)gc.ws_cull_w_imms.size(), + gc.ws_cull_h_imms.data(), (int)gc.ws_cull_h_imms.size()); + ws_offered = gc.ws_offered; + ws_ultrawide_offered = gc.ws_ultrawide_offered; /* Register the [widescreen.backdrop] store PCs so the dirty-RAM * interpreter applies the backdrop screenX squash on the interp * path (overlay backdrop handlers run interpreted when no cache @@ -2086,8 +2421,11 @@ int main(int argc, char** argv) { p1_mode = gc.runtime.default_p1_mode; p2_mode = gc.runtime.default_p2_mode; } + ctrl_locked_p1_mode = gc.runtime.default_p1_mode; + ctrl_locked_p2_mode = gc.runtime.default_p2_mode; ctrl_allow_hybrid = gc.runtime.controller_allow_hybrid; ctrl_lock_mode = gc.runtime.controller_lock_mode; + ctrl_lock_device = gc.runtime.controller_lock_device; if (gc.runtime.has_deadzone) resolved_deadzone = gc.runtime.deadzone; /* LEGACY per-game pad-config opt-in (default modern). Only Tomba sets * it, so its launcher Hybrid mode's analog<->digital flip doesn't make @@ -2101,6 +2439,34 @@ int main(int argc, char** argv) { fast_boot = gc.runtime.fast_boot; bios_hle = gc.runtime.bios_hle; bios_hle_keep_intro = gc.runtime.bios_hle_keep_intro; + /* Let the dispatch layer distinguish "dirty because text was + * loaded" from "diverged because runtime wrote different code over + * the original EXE image". Packed/self-modifying games can rewrite + * their own text; those pages must execute from live RAM, not stale + * static native code. Best effort: without the local EXE file, the + * guard remains on the existing dirty-page behavior. */ + if (!gc.exe_path.empty()) { + std::ifstream ef(gc.exe_path, std::ios::binary | std::ios::ate); + if (ef) { + std::streamsize sz = ef.tellg(); + if (sz > 2048) { + uint32_t img_len = (uint32_t)(sz - 2048); + uint8_t *img = (uint8_t *)std::malloc(img_len); + if (img) { + ef.seekg(2048, std::ios::beg); + if (ef.read((char *)img, img_len)) { + dirty_ram_register_text_image( + gc.load_address & 0x1FFFFFFFu, img, img_len); + std::fprintf(stdout, + "psxrecomp: text image guard armed (0x%08X..0x%08X)\n", + gc.load_address, gc.load_address + img_len); + } else { + std::free(img); + } + } + } + } + } /* HLE-tier scheduler subsystem replacement default (env * PSX_HLE_SCHEDULER still wins; latched at first dispatch). */ psx_hle_scheduler_set_default(gc.runtime.hle_scheduler ? 1 : 0); @@ -2274,6 +2640,7 @@ int main(int argc, char** argv) { if (us.has_memcard2_path) memcard2_path = us.memcard2_path; if (us.has_memcard1_enabled) memcard1_enabled = us.memcard1_enabled; if (us.has_memcard2_enabled) memcard2_enabled = us.memcard2_enabled; + if (us.has_language) resolved_language = us.language; if (us.has_p1_device) p1_device = us.p1_device; if (us.has_p2_device) p2_device = us.p2_device; if (us.has_p1_mode) p1_mode = us.p1_mode; @@ -2283,6 +2650,39 @@ int main(int argc, char** argv) { if (us.has_vsync) g_video_vsync = us.vsync; } + /* lock_mode: the game supports exactly ONE pad type (e.g. X4 / Tomba 2 are + * digital-only — X4's pre-DualShock libpad silently discards input from a + * pad answering id 0x73). The launcher hides its selector for such games, + * but that alone left two holes: (a) launcher-less builds still honoured a + * settings.toml p1_mode/p2_mode, and (b) a settings.toml persisted BEFORE + * the game declared lock_mode fed the stale mode back as the launcher's + * locked_mode. Clamp to the game-declared modes here, after every + * config/settings source has been applied, so a locked game can never boot + * a pad type it doesn't support. */ + if (ctrl_lock_mode) { + p1_mode = ctrl_locked_p1_mode; + p2_mode = ctrl_locked_p2_mode; + } + + /* [widescreen] offer=false: this title's widescreen is unported/unvalidated, + * so the launcher hides its toggle — and, same completeness treatment as + * lock_mode above, the runtime clamps the display aspect to native 4:3 here + * so a stale persisted 16:9 in settings.toml can't engage the hack in + * launcher-less builds either. */ + if (!ws_offered && (g_video_aspect_num != 4 || g_video_aspect_den != 3)) { + std::fprintf(stdout, "psxrecomp: widescreen not offered for this title; " + "clamping display aspect %d:%d -> 4:3\n", + g_video_aspect_num, g_video_aspect_den); + g_video_aspect_num = 4; + g_video_aspect_den = 3; + } + if (!ws_ultrawide_offered && g_video_aspect_num * 9 == g_video_aspect_den * 21) { + std::fprintf(stdout, "psxrecomp: 21:9 is not offered for this title; clamping to %s\n", + ws_offered ? "16:9" : "4:3"); + g_video_aspect_num = ws_offered ? 16 : 4; + g_video_aspect_den = ws_offered ? 9 : 3; + } + /* Latency knobs: env overrides win over config (for A/B measurement). * PSX_LOW_LATENCY_INPUT=0/1 ; PSX_VSYNC=1(vsync)/0(immediate)/-1(adaptive). */ if (const char *e = std::getenv("PSX_LOW_LATENCY_INPUT")) g_low_latency_input = atoi(e) ? 1 : 0; @@ -2370,6 +2770,7 @@ int main(int argc, char** argv) { seed.memcard2_enabled = memcard2_enabled; seed.has_memcard2_enabled = true; if (!memcard1_path.empty()) { seed.memcard1_path = memcard1_path; seed.has_memcard1_path = true; } if (!memcard2_path.empty()) { seed.memcard2_path = memcard2_path; seed.has_memcard2_path = true; } + seed.language = resolved_language; seed.has_language = true; seed.p1_device = p1_device; seed.has_p1_device = true; seed.p2_device = p2_device; seed.has_p2_device = true; seed.p1_mode = p1_mode; seed.has_p1_mode = true; @@ -2410,6 +2811,11 @@ int main(int argc, char** argv) { ginfo.allow_hybrid = ctrl_allow_hybrid; ginfo.lock_mode = ctrl_lock_mode; ginfo.locked_mode = p1_mode; /* force the game's declared mode (default_mode) */ + ginfo.lock_device = ctrl_lock_device; + ginfo.ws_offered = ws_offered; + ginfo.ws_ultrawide_offered = ws_ultrawide_offered; + for (const auto& lo : lang_menu_options) + ginfo.languages.push_back({ lo.code, lo.label }); lr = psx_launcher::run(lwin, lctx, seed, ginfo, assets.c_str()); SDL_GL_DeleteContext(lctx); } @@ -2446,6 +2852,7 @@ int main(int argc, char** argv) { memcard2_enabled = seed.memcard2_enabled; if (seed.has_memcard1_path) memcard1_path = seed.memcard1_path; if (seed.has_memcard2_path) memcard2_path = seed.memcard2_path; + if (seed.has_language) resolved_language = seed.language; p1_device = seed.p1_device; p2_device = seed.p2_device; p1_mode = seed.p1_mode; p2_mode = seed.p2_mode; if (seed.has_deadzone) resolved_deadzone = seed.deadzone; @@ -2458,6 +2865,11 @@ int main(int argc, char** argv) { } #endif + /* Re-apply the resolved language to the translation layer. text_xlate_init + * (at config load) only saw the game.toml default; this folds in the + * settings.toml override and the launcher's choice. No-op when unchanged. */ + text_xlate_set_language(resolved_language.c_str()); + /* CLI overrides win over config — applied last, before backend/port init. * Enables a soak fleet: several instances on distinct ports + renderers, * including BIOS instances that have no [game]-block config. */ @@ -2554,10 +2966,10 @@ int main(int argc, char** argv) { set_player_device(g_players[0], p1_device, p1_mode); set_player_device(g_players[1], p2_device, p2_mode); for (int s = 0; s < 2; s++) { - /* Dev builds keep P1 connected even with no controller so the always-on - * keyboard (g_dev_kb_p1) can drive port 1 standalone. */ - const bool kb_p1 = (g_dev_kb_p1 && s == 0); - sio_set_pad_connected(s, (g_players[s].kind != 0 || kb_p1) ? 1 : 0); + /* Dev-any-input keeps P1 connected even with no assigned controller so the + * keyboard / any plugged-in controller can drive port 1 standalone. */ + const bool dev_p1 = (dev_any_input_enabled() && s == 0); + sio_set_pad_connected(s, (g_players[s].kind != 0 || dev_p1) ? 1 : 0); sio_set_pad_analog(s, pad_mode_boot_analog(g_players[s].mode), 0x80, 0x80, 0x80, 0x80); } /* SPU float-shadow gate must be set before spu_init() (which runs @@ -2567,6 +2979,20 @@ int main(int argc, char** argv) { std::fprintf(stdout, "psxrecomp: SPU float-shadow enabled (verified-enhancement)\n"); spu_init(); cdrom_init(disc_path_str.empty() ? NULL : disc_path_str.c_str()); + if (!disc_path_str.empty()) { + /* GetID must report the inserted disc's license region (the BIOS CD + * driver revalidates it mid-game). Derive it from the disc's boot + * serial via the same disc_identity module the launch check uses. */ + const auto ident = PSXRecompV4::identify_disc( + disc_path_str, /*expected_serial*/"", /*expected_crc*/0, + /*has_expected_crc*/false, /*compute_crc*/false); + if (ident.region == "PAL") cdrom_set_disc_scex("SCEE"); + else if (ident.region == "NTSC-J") cdrom_set_disc_scex("SCEI"); + else if (ident.region == "NTSC-U") cdrom_set_disc_scex("SCEA"); + if (!ident.region.empty()) + std::fprintf(stdout, "psxrecomp: disc region %s (serial %s)\n", + ident.region.c_str(), ident.detected_serial.c_str()); + } { int divisor = 1; /* default: authentic 1x timing */ if (disc_speed == "instant") divisor = 0; @@ -2642,6 +3068,7 @@ int main(int argc, char** argv) { controller_deadzone = std::max(0, std::min(32767, resolved_deadzone)); refresh_player_devices(); /* open SDL handles to match the player config */ #ifndef PSX_SDL_NO_AUDIO + audio_trace_init(); if (SDL_InitSubSystem(SDL_INIT_AUDIO) == 0) { SDL_AudioSpec want; SDL_AudioSpec have; @@ -2650,8 +3077,21 @@ int main(int argc, char** argv) { want.format = AUDIO_S16SYS; want.channels = 2; want.samples = 1024; - sdl_audio_device = SDL_OpenAudioDevice(NULL, 0, &want, &have, 0); + const bool legacy = audio_legacy_mode(); + if (!legacy) + want.callback = sdl_drc_callback; /* pull model: bridge resamples + DRC */ + sdl_audio_device = SDL_OpenAudioDevice(NULL, 0, &want, &have, + legacy ? 0 : SDL_AUDIO_ALLOW_FREQUENCY_CHANGE); if (sdl_audio_device) { + if (!legacy) { + rab_config cfg; rab_config_defaults(&cfg); + cfg.channels = 2; + cfg.source_rate = 44100.0; /* SPU render rate */ + cfg.host_rate = (double)have.freq; /* actual device rate */ + if (rab_init(&s_drc, &cfg) == 0) s_drc_ready = true; + } + g_audio_host_rate = have.freq; + audio_trace_set_tap_rate(AUDIO_TAP_HOST, (uint32_t)have.freq); SDL_PauseAudioDevice(sdl_audio_device, 0); } } @@ -2682,6 +3122,29 @@ int main(int argc, char** argv) { return 1; } + /* Sync-to-host-refresh: with SDL PRESENTVSYNC on, a fixed 59.94 Hz wall-clock + * pacer fights a 60.00 Hz panel — rendered frames slip onto an uneven vblank + * count (2/3/1 beat) that reads as moving-object judder. If the panel is + * within ~2% of 60 Hz, nudge the pacer to the exact panel period so the pacer + * and vsync agree and 30fps content pads to a steady 2 refreshes each. + * Non-~60Hz panels keep the PSX rate (vsync then governs; wrong-speed sim is + * worse than a benign slow beat). */ + { + SDL_DisplayMode dm; + int disp_idx = SDL_GetWindowDisplayIndex(sdl_window); + if (disp_idx >= 0 && SDL_GetCurrentDisplayMode(disp_idx, &dm) == 0 && dm.refresh_rate > 0) { + double host_hz = (double)dm.refresh_rate; + if (host_hz >= 58.8 && host_hz <= 61.2) { + g_frame_period_ms = 1000.0 / host_hz; + std::printf("psxrecomp: sync-to-host-refresh: pacing to %d Hz panel " + "(%.4f ms/frame)\n", dm.refresh_rate, g_frame_period_ms); + } else { + std::printf("psxrecomp: host panel %d Hz not ~60 Hz; keeping PSX " + "59.94 Hz pacing\n", dm.refresh_rate); + } + } + } + /* OpenGL backend: create the GL context now. On failure, relabel the * facade back to software (rasterization already runs through software in * this phase) and fall through to the SDL_Renderer present path below. */ diff --git a/runtime/src/memory.c b/runtime/src/memory.c index 8bd0df30..3a2391b2 100644 --- a/runtime/src/memory.c +++ b/runtime/src/memory.c @@ -12,6 +12,7 @@ #include "cdrom.h" #include "crash_trace.h" #include "dma.h" +#include "fntrace.h" #include "gpu.h" #include "mdec.h" #include "sio.h" @@ -32,10 +33,27 @@ static uint8_t ram[RAM_SIZE]; static uint8_t scratchpad[SCRATCHPAD_SIZE]; static uint8_t bios_rom[BIOS_ROM_SIZE]; +/* Physical address translation for guest accesses. The 2 MB main RAM is + * mirrored 4x across the first 8 MB of each segment (mem-ctrl RAM_SIZE + * register, default 0x0B88) and games rely on it — Kula World's crt0 + * parks $sp in the 4th mirror (0x807FFFF8). Fold the mirrors here so the + * RAM bounds checks below see canonical offsets; without this, mirror + * writes fell into the open-bus no-op and mirror reads returned 0 (the + * guest's stack silently vanished and $ra came back as 0). */ +static inline uint32_t psx_phys_addr(uint32_t addr) { + uint32_t phys = addr & 0x1FFFFFFFu; + if (phys < 0x00800000u) phys &= (uint32_t)(RAM_SIZE - 1); + return phys; +} + /* Expose RAM pointer for oracle comparison (find_first_divergence). */ uint8_t *memory_get_ram_ptr(void) { return ram; } uint8_t *memory_get_scratchpad_ptr(void) { return scratchpad; } +void memory_clear_low_boot_scratch(void) { + memset(ram, 0, 0x10u); +} + /* ---- Dirty-page tracking for install-at-runtime code (CLAUDE.md Rule 18) ---- * * The PS1 BIOS writes 4-instruction dispatch stubs into kernel RAM at runtime @@ -87,6 +105,8 @@ static inline void dirty_ram_mark_kernel_write(uint32_t phys) { dirty_ram_mark_page(phys); } +int dirty_ram_is_dirty(uint32_t phys); + /* Establish the clean compiled-image baseline for the game-EXE text region. * Called ONCE when the game entry is first reached (fntrace game-start): by then * the BIOS has fully loaded the boot EXE into [0x10000, FLOOR) — which IS the @@ -109,6 +129,120 @@ void dirty_ram_clear_image_baseline(void) { dirty_ram_bitmap[page >> 5] &= ~(1u << (page & 31u)); } +/* Text-image divergence guard. + * + * dirty_ram_is_dirty() is intentionally coarse: it says a page was touched by + * runtime code loading. For deciding whether the original static game dispatch + * is still safe, a dirty page is not enough information: data in a code page can + * be written without changing the instructions at a target, while packed games + * can also rewrite their own text with completely different instructions. + * + * main.cpp registers the boot EXE bytes as the reference image. Writes inside + * that range mark pages whose bytes differ from the reference. At dispatch time, + * a modified page is still allowed to use the static native entry if the code + * bytes at the target match the original image; otherwise it is sticky-diverged + * and must execute from live RAM through the dirty interpreter. */ +/* Mutable: the runtime owns this heap buffer (main.cpp mallocs the EXE image and + * never frees it). Intentional data patches by the translation layer are blessed + * into it via dirty_ram_text_bless so they are not mistaken for self-modifying + * code — see dirty_ram_text_native_ok / dirty_ram_text_bless. */ +static uint8_t *text_ref_image = NULL; +static uint32_t text_ref_lo = 0, text_ref_hi = 0; +static uint32_t text_modified_bitmap[DIRTY_RAM_BITMAP_WORDS]; +static uint32_t text_diverged_bitmap[DIRTY_RAM_BITMAP_WORDS]; +static uint64_t g_text_native_blocked = 0; +static uint32_t g_text_diverged_pages = 0; + +void dirty_ram_register_text_image(uint32_t phys_lo, const uint8_t *bytes, + uint32_t len) { + if (!bytes || len == 0 || phys_lo >= RAM_SIZE) return; + if (len > RAM_SIZE - phys_lo) len = RAM_SIZE - phys_lo; + text_ref_image = (uint8_t *)bytes; /* runtime-owned mutable heap buffer */ + text_ref_lo = phys_lo; + text_ref_hi = phys_lo + len; + memset(text_modified_bitmap, 0, sizeof(text_modified_bitmap)); + memset(text_diverged_bitmap, 0, sizeof(text_diverged_bitmap)); + g_text_native_blocked = 0; + g_text_diverged_pages = 0; +} + +static inline void text_guard_note_write(uint32_t phys, uint32_t val, int size) { + if (!text_ref_image) return; + if (phys < text_ref_lo || phys + (uint32_t)size > text_ref_hi) return; + const uint8_t *ref = text_ref_image + (phys - text_ref_lo); + uint8_t buf[4] = { (uint8_t)val, (uint8_t)(val >> 8), + (uint8_t)(val >> 16), (uint8_t)(val >> 24) }; + if (memcmp(ref, buf, (size_t)size) != 0) { + uint32_t page = phys >> DIRTY_RAM_PAGE_SHIFT; + text_modified_bitmap[page >> 5] |= (1u << (page & 31u)); + } +} + +int dirty_ram_text_native_ok(uint32_t phys) { + if (!text_ref_image || phys < text_ref_lo || phys >= text_ref_hi) + return !dirty_ram_is_dirty(phys); + + uint32_t page = phys >> DIRTY_RAM_PAGE_SHIFT; + uint32_t bit = 1u << (page & 31u); + if (text_diverged_bitmap[page >> 5] & bit) { + g_text_native_blocked++; + return 0; + } + /* Fast path: a page never touched by a guarded write AND not runtime-dirty + * still holds the pristine compiled image — native is safe with no compare. */ + if (!(text_modified_bitmap[page >> 5] & bit) && !dirty_ram_is_dirty(phys)) + return 1; + + /* Flagged (a guarded write that differed, an overlay-dirty page, or an + * intentional runtime data patch): the compiled native code is valid IFF the + * bytes at the entry still match the reference image. Decide by the ACTUAL + * bytes, not the page-dirty heuristic — so a data-only change in a page that + * also holds code (e.g. a translation string table blessed into the ref via + * dirty_ram_text_bless) does not needlessly block a still-valid function and + * route it to psx_unknown_dispatch. A genuine code overwrite still diverges. */ + uint32_t n = 256; + if (n > text_ref_hi - phys) n = text_ref_hi - phys; + if (memcmp(ram + phys, text_ref_image + (phys - text_ref_lo), n) == 0) + return 1; + + text_diverged_bitmap[page >> 5] |= bit; + g_text_diverged_pages++; + g_text_native_blocked++; + return 0; +} + +/* Bless an INTENTIONAL data patch into the reference image so the text-divergence + * guard does not mistake it for self-modifying code. The runtime's own + * translation layer (text_xlate) patches string/glyph tables that share 4 KB + * pages — and the guard's 256-byte compare window — with real compiled functions. + * Without this, patching a data table diverges the page and blocks native + * dispatch of every function in it (a real function fatally routes to + * psx_unknown_dispatch). By updating text_ref_image to match the patched bytes, + * the compare passes for the intentional change while a genuine game write of + * DIFFERENT bytes still diverges and is still caught. Clears the sticky diverged + * bit for affected pages so the guard re-evaluates against the updated reference. + * Byte-for-byte identical writes are a no-op. */ +void dirty_ram_text_bless(uint32_t phys, const uint8_t *bytes, uint32_t len) { + if (!text_ref_image || !bytes || len == 0) return; + if (phys >= text_ref_hi || phys + len <= text_ref_lo) return; /* out of range */ + uint32_t lo = phys < text_ref_lo ? text_ref_lo : phys; + uint32_t hi = phys + len > text_ref_hi ? text_ref_hi : phys + len; + if (hi <= lo) return; + uint8_t *ref = text_ref_image + (lo - text_ref_lo); + const uint8_t *src = bytes + (lo - phys); + if (memcmp(ref, src, hi - lo) == 0) return; /* already in sync */ + memcpy(ref, src, hi - lo); + /* Re-open the affected pages: clear the sticky diverged bit so the next + * dispatch re-runs the compare against the now-updated reference. */ + uint32_t first_page = lo >> DIRTY_RAM_PAGE_SHIFT; + uint32_t last_page = (hi - 1u) >> DIRTY_RAM_PAGE_SHIFT; + for (uint32_t page = first_page; page <= last_page; page++) + text_diverged_bitmap[page >> 5] &= ~(1u << (page & 31u)); +} + +uint64_t dirty_ram_text_native_blocked(void) { return g_text_native_blocked; } +uint32_t dirty_ram_text_diverged_pages(void) { return g_text_diverged_pages; } + void dirty_ram_mark_executable_range(uint32_t phys, uint32_t len) { if (len == 0 || phys >= RAM_SIZE) return; uint32_t end = phys + len - 1u; @@ -267,7 +401,21 @@ uint32_t i_mask; /* 0x1F801074 — interrupt enable mask */ * watching (adds 0). */ int g_shadow_mmio_watch = 0; uint64_t g_shadow_mmio_hits = 0; -#define SHADOW_NOTE_MMIO() do { g_shadow_mmio_hits += (uint64_t)g_shadow_mmio_watch; } while (0) +/* Always-on MMIO access counter (reads AND writes of the device page). The + * idle-loop cycle skip (psx_cycles.c) requires "no MMIO touched since the + * last interrupt check" before fast-forwarding time: an MMIO read can return + * time-varying or side-effecting values (timer counters, CD response FIFO), + * so any touch disqualifies the window. One increment per access. */ +uint64_t g_mmio_access_count = 0; +#define SHADOW_NOTE_MMIO() do { g_mmio_access_count++; \ + g_shadow_mmio_hits += (uint64_t)g_shadow_mmio_watch; } while (0) + +/* Always-on guest store counter, bumped at the top of every raw store + * (RAM, scratchpad, MMIO, even discarded KSEG2/IsC stores — over-counting + * is conservative). Second gate of the idle-loop cycle skip: a window with + * ANY store is not a pure poll loop. CPU stores and in-flight DMA writes + * both funnel through the psx_write_*_raw chokepoints. */ +uint64_t g_guest_store_count = 0; /* ---- Card protocol trace: tracks I_MASK bit 7 transitions ---- */ #define IMASK_TRACE_CAP 4096 @@ -837,6 +985,16 @@ uint32_t psx_read_word(uint32_t addr) { if (!psx_get_in_exception()) ls_read_hook(addr, 4, v); return v; } +/* Physical address of a CPU/DMA main-RAM access. Fold KUSEG/KSEG0/KSEG1 first + * (0x1FFFFFFF), then fold the 2nd-4th main-RAM mirrors: real hardware mirrors the + * 2 MB DRAM across the WHOLE 0..0x7FFFFF physical window (Beetle libretro.cpp:874 + * `A < 0x00800000` routes to main RAM; psx-spx "2048K RAM ... mirrored 4x"). A game + * may legitimately place its stack at the top of that window — Tsumu Light computes + * sp = (ramtop-8)|0x80000000 with its ramtop constant 0x00800000, giving sp=0x807FFFF8 + * (top of the 4th mirror). Without this fold those accesses miss DRAM (`< RAM_SIZE` + * fails) and hit open bus: stores drop, loads return 0, so the first saved return + * address reads back 0 and `jr ra` derails to PC=0. Identity for phys < RAM_SIZE, so + * normal-range RAM is byte-identical; MMIO/scratchpad/BIOS (>= 0x800000) untouched. */ static uint32_t psx_read_word_raw(uint32_t addr) { /* KSEG2 cache control — before physical translation. */ if (addr == 0xFFFE0130u) return cache_ctrl; @@ -850,7 +1008,7 @@ static uint32_t psx_read_word_raw(uint32_t addr) { * fatal (frame 9705). Ignore writes, read as 0, count for telemetry. */ if (addr >= 0xC0000000u) { g_kseg2_ignored_reads++; return 0; } - uint32_t phys = addr & 0x1FFFFFFFu; + uint32_t phys = psx_phys_addr(addr); if (phys < RAM_SIZE) { uint32_t v = (uint32_t)ram[phys] @@ -931,6 +1089,7 @@ void psx_write_word(uint32_t addr, uint32_t val) { s_ls_op_active = 0; } static void psx_write_word_raw(uint32_t addr, uint32_t val) { + g_guest_store_count++; /* KSEG2 cache control — before physical translation. */ if (addr == 0xFFFE0130u) { cache_ctrl = val; return; } /* KSEG2 guard — see psx_read_word_raw. */ @@ -940,14 +1099,67 @@ static void psx_write_word_raw(uint32_t addr, uint32_t val) { * We have no cache model, so silently discard RAM/scratchpad writes. */ if (sr_ptr && (*sr_ptr & 0x10000u)) return; - uint32_t phys = addr & 0x1FFFFFFFu; + uint32_t phys = psx_phys_addr(addr); + + /* The generated BIOS mirrors its exception trampoline to 0x80000000 during + * boot. On hardware this mirror copy is not visible in RAM; only the real + * exception vector at 0x80000080 is. Tomba 2 later passes buffer=0 to the + * BIOS card write routine, so stale mirror bytes at 0 corrupt the sector 63 + * management write and leave the load menu stuck checking the card. */ + if (fntrace_is_game_started() && + phys < 0x10u && g_debug_last_store_pc == 0xBFC10A00u) return; + + /* BIOS helpers use RAM address zero as a tiny delay-loop scratch between + * device-register polls. Treat these two dummy stores as non-visible; real + * hardware / Beetle preserve Tomba 2's sector-63 card-management payload + * when it passes buffer=0 to _card_write. */ + if (fntrace_is_game_started() && phys == 0u) { + switch (g_debug_last_store_pc) { + case 0xBFC04E90u: + case 0xBFC04EF0u: + case 0xBFC05164u: + case 0xBFC0D634u: + case 0xBFC3EEB4u: + case 0xBFC405E4u: + case 0xBFC40788u: + case 0xBFC41C50u: + case 0x80012434u: + case 0x800125ACu: + return; + default: + break; + } + } if (phys < RAM_SIZE) { + /* Tomba 2 load-game card check: the BIOS card-manager cleanup helper + * at kernel RAM 0x1C5C scans MARK events and re-arms any READY event + * matching F0000011/{4,8000,100,200,2000}. In the recomp timing path it + * can run after the card completion DeliverEvent but before the game's + * TestEvent poll, consuming the public card event and leaving the UI + * stuck at "Checking MEMORY CARD...". Let the real TestEvent consume + * public MARK card events instead; keep this narrowly keyed to the + * helper's status store and the EvCB layout. */ + if (fntrace_is_game_started() && + g_debug_last_store_pc == 0xBFC117E4u && + val == 0x2000u && + phys >= 4u && (phys + 8u) < RAM_SIZE && + read_ram_word(phys) == 0x4000u && + read_ram_word(phys - 4u) == 0xF0000011u && + read_ram_word(phys + 8u) == 0x2000u) { + uint32_t spec = read_ram_word(phys + 4u); + if (spec == 0x00000004u || spec == 0x00008000u || + spec == 0x00000100u || spec == 0x00000200u || + spec == 0x00002000u) { + return; + } + } if (phys == D44_PHYS) d44_note(phys, read_ram_word(phys), val); debug_server_trace_write_check(phys, read_ram_word(phys), val, 4); parity_trace_note_write(phys, 4, effective_store_pc()); card_data_writes_check(phys, val, 4); dirty_ram_mark_kernel_write(phys); + text_guard_note_write(phys, val, 4); overlay_watch_note_write(phys, 4); #ifdef PSX_COSIM { extern void cosim_note_ram_write(uint32_t,uint32_t); cosim_note_ram_write(phys, 4); } @@ -998,7 +1210,7 @@ uint16_t psx_read_half(uint32_t addr) { static uint16_t psx_read_half_raw(uint32_t addr) { /* KSEG2 guard — see psx_read_word_raw. */ if (addr >= 0xC0000000u) { g_kseg2_ignored_reads++; return 0; } - uint32_t phys = addr & 0x1FFFFFFFu; + uint32_t phys = psx_phys_addr(addr); if (phys < RAM_SIZE) { return (uint16_t)ram[phys] | ((uint16_t)ram[phys + 1] << 8); @@ -1029,17 +1241,19 @@ void psx_write_half(uint32_t addr, uint16_t val) { s_ls_op_active = 0; } static void psx_write_half_raw(uint32_t addr, uint16_t val) { + g_guest_store_count++; if (sr_ptr && (*sr_ptr & 0x10000u)) return; /* KSEG2 guard — see psx_read_word_raw. */ if (addr >= 0xC0000000u) { g_kseg2_ignored_writes++; return; } - uint32_t phys = addr & 0x1FFFFFFFu; + uint32_t phys = psx_phys_addr(addr); if (phys < RAM_SIZE) { debug_server_trace_write_check(phys, (uint32_t)read_ram_half(phys), (uint32_t)val, 2); parity_trace_note_write(phys, 2, effective_store_pc()); card_data_writes_check(phys, (uint32_t)val, 2); dirty_ram_mark_kernel_write(phys); + text_guard_note_write(phys, (uint32_t)val, 2); overlay_watch_note_write(phys, 2); #ifdef PSX_COSIM { extern void cosim_note_ram_write(uint32_t,uint32_t); cosim_note_ram_write(phys, 2); } @@ -1081,7 +1295,7 @@ uint8_t psx_read_byte(uint32_t addr) { static uint8_t psx_read_byte_raw(uint32_t addr) { /* KSEG2 guard — see psx_read_word_raw. */ if (addr >= 0xC0000000u) { g_kseg2_ignored_reads++; return 0; } - uint32_t phys = addr & 0x1FFFFFFFu; + uint32_t phys = psx_phys_addr(addr); if (phys < RAM_SIZE) { return ram[phys]; @@ -1256,17 +1470,19 @@ void psx_write_byte(uint32_t addr, uint8_t val) { s_ls_op_active = 0; } static void psx_write_byte_raw(uint32_t addr, uint8_t val) { + g_guest_store_count++; if (sr_ptr && (*sr_ptr & 0x10000u)) return; /* KSEG2 guard — see psx_read_word_raw. */ if (addr >= 0xC0000000u) { g_kseg2_ignored_writes++; return; } - uint32_t phys = addr & 0x1FFFFFFFu; + uint32_t phys = psx_phys_addr(addr); if (phys < RAM_SIZE) { debug_server_trace_write_check(phys, (uint32_t)ram[phys], (uint32_t)val, 1); parity_trace_note_write(phys, 1, effective_store_pc()); card_data_writes_check(phys, (uint32_t)val, 1); dirty_ram_mark_kernel_write(phys); + text_guard_note_write(phys, (uint32_t)val, 1); overlay_watch_note_write(phys, 1); #ifdef PSX_COSIM { extern void cosim_note_ram_write(uint32_t,uint32_t); cosim_note_ram_write(phys, 1); } diff --git a/runtime/src/overlay_capture.c b/runtime/src/overlay_capture.c index 71356aa1..80e2f3cf 100644 --- a/runtime/src/overlay_capture.c +++ b/runtime/src/overlay_capture.c @@ -110,10 +110,13 @@ void overlay_capture_on_dma(uint32_t load_addr, uint32_t size, /* ---- JSON output — assembled dirty_ram regions -------------------------- */ /* Emit every dirty-page run within [win_lo_page, win_hi_page) as a capture - * region. Runs are clamped to the window: the kernel window must never merge - * with main-EXE pages (both can be dirty and adjacent — kernel via CPU - * stores, main EXE via CD DMA), and main-EXE text is statically recompiled, - * never captured. */ + * region. Runs are clamped to the window so region_start keys are stable + * across window boundaries: the kernel window must never merge with + * boot-text pages (both can be dirty and adjacent — kernel via CPU stores, + * boot-text via overlay CD DMA), and a boot-text run must never merge + * across the overlay-region floor (existing [FLOOR,...) DLL keys would + * shift). Dirty boot-text runs ARE captured — post-baseline they are live + * overlay code (see dirty_ram_interp.h window model). */ static void write_json_window(FILE *f, uint32_t win_lo_page, uint32_t win_hi_page, int *first_region) { @@ -259,10 +262,18 @@ void overlay_capture_write_json(void) fprintf(f, "[\n"); first_region = 1; - /* Kernel window first, then the overlay region (see dirty_ram_interp.h - * for the window model). Main-EXE text between them is never captured. */ + /* Kernel window, then dirty boot-text, then the overlay region (see + * dirty_ram_interp.h for the window model). The boot-text window + * [KERNEL_WINDOW_END, FLOOR) emits only genuinely-overlaid runs: the + * game-start baseline cleared the EXE-load false positives, so a dirty + * page there is runtime overlay code by definition. Windows stay + * separate so runs clamp at the boundaries and region_start keys match + * try_load_region's clamped walkback (existing overlay-region DLL keys + * are unchanged). */ write_json_window(f, 0u, DIRTY_RAM_KERNEL_WINDOW_END / page_sz, &first_region); + write_json_window(f, DIRTY_RAM_KERNEL_WINDOW_END / page_sz, + OVERLAY_REGION_FLOOR / page_sz, &first_region); write_json_window(f, OVERLAY_REGION_FLOOR / page_sz, bw * 32u, &first_region); @@ -285,17 +296,47 @@ int overlay_capture_count(void) * skips image CRCs already in the cache — and the loop self-limits: once * the hot code goes native the pressure signal drops to zero and no more * captures fire. */ -#define AUTOCAP_CHECK_FRAMES 600u /* ~10 s between pressure samples */ -#define AUTOCAP_MIN_DISPATCHES 256u /* interp pressure gate per sample */ -#define AUTOCAP_COOLDOWN_FRAMES 3600u /* >= 60 s between auto-fires */ -#define AUTOCAP_MAX_TRIGGERS 16u /* session backstop (TCP-visible) */ +/* Cadence/threshold retune 2026-07-06 (measured, Tomba2 attract): a single + * interp dispatch can chain ~129K insns, so DISPATCH count under-reports + * pressure by orders of magnitude — a scene burning >50% of wall time in the + * interpreter produced ~250 dispatches per old 10s window, hovering at the + * old 256 gate (triggers effectively never fired mid-scene). Pressure is now + * EITHER signal: dispatch count (many short stubs) OR interpreted-insn count + * (few long chains). ~250K insns/window ≈ interp >~2.5% of wall — worth a + * capture. Faster cadence + short cooldown are safe: fires are still gated + * on a coherent moment (not loading, provider idle) and the loop self-limits + * as coverage converges (pressure → 0). */ +/* There is deliberately NO lifetime trigger cap. A 64-fire "session backstop" + * shipped 2026-07-06 and was exhausted ~4h into an attract soak, silently + * freezing coverage growth with interp still ~50% of wall in 2D scenes — + * convergence IS many fires over a long session. The runaway case a cap was + * papering over (pressure that never converts: excluded PCs, failing shards + * — fires every cooldown forever, spawning futile provider runs) is handled + * by FUTILITY BACKOFF instead: a fire is skipped when the capture manifest + * is byte-identical to the last compile request AND that request registered + * no new candidates; skips retry on exponential backoff and any change in + * either signal resets to normal cadence. Concurrency is already bounded + * (cp->busy defers fires while a compile is in flight) and identical images + * dedup downstream (compile_overlays.py skips cached CRCs). */ +#define AUTOCAP_CHECK_FRAMES 120u /* ~2 s between pressure samples */ +#define AUTOCAP_MIN_DISPATCHES 256u /* dispatch-count pressure gate */ +#define AUTOCAP_MIN_INSNS 250000ull /* interp-insn pressure gate */ +#define AUTOCAP_COOLDOWN_FRAMES 300u /* >= ~5 s between auto-fires */ +#define AUTOCAP_BACKOFF_MAX 64u /* futile-retry ceiling: 64*5s ≈ 5min */ static int s_autocap_enabled = 0; static uint64_t s_autocap_last_check = 0; static uint64_t s_autocap_last_fire = 0; static uint64_t s_autocap_last_disp = 0; +static uint64_t s_autocap_last_insns = 0; static uint32_t s_autocap_triggers = 0; static uint64_t s_autocap_last_delta = 0; +static uint64_t s_autocap_last_insns_delta = 0; +static uint64_t s_autocap_next_ok = 0; /* frame gate for next attempt */ +static uint64_t s_autocap_sig_at_req = 0; /* manifest FNV at last request */ +static int s_autocap_reg_at_req = -1; /* loader candidates at last req */ +static uint32_t s_autocap_backoff = 1; /* cooldown multiplier (futile) */ +static uint32_t s_autocap_futile = 0; /* futile skips (telemetry) */ void overlay_autocapture_set_enabled(int on) { s_autocap_enabled = on ? 1 : 0; } @@ -306,6 +347,35 @@ void overlay_autocapture_get_status(int *enabled, uint32_t *triggers, if (last_delta) *last_delta = s_autocap_last_delta; } +uint64_t overlay_autocapture_last_insns_delta(void) { + return s_autocap_last_insns_delta; +} + +void overlay_autocapture_get_futility(uint32_t *backoff, uint32_t *futile) { + if (backoff) *backoff = s_autocap_backoff; + if (futile) *futile = s_autocap_futile; +} + +/* FNV-1a of the just-written capture manifest. The manifest is a pure + * function of the capture inputs (live overlay bytes + observed seed-PC + * sets, both grow-only), so an unchanged hash means a compile request + * would redo exactly the work of the previous one. */ +static uint64_t autocap_manifest_sig(void) +{ + char path[600]; + FILE *f; + uint64_t h = 1469598103934665603ull; + unsigned char buf[65536]; + size_t n, i; + snprintf(path, sizeof(path), "%s/overlay_captures.json", s_out_dir); + f = fopen(path, "rb"); + if (!f) return 0; + while ((n = fread(buf, 1, sizeof(buf), f)) > 0) + for (i = 0; i < n; i++) { h ^= buf[i]; h *= 1099511628211ull; } + fclose(f); + return h; +} + void overlay_autocapture_tick(void) { extern uint64_t s_frame_count; @@ -320,21 +390,44 @@ void overlay_autocapture_tick(void) uint64_t delta = disp - s_autocap_last_disp; s_autocap_last_disp = disp; s_autocap_last_delta = delta; + uint64_t insns = g_dirty_ram_insns_run; + uint64_t insns_delta = insns - s_autocap_last_insns; + s_autocap_last_insns = insns; + s_autocap_last_insns_delta = insns_delta; - if (delta < AUTOCAP_MIN_DISPATCHES) return; + if (delta < AUTOCAP_MIN_DISPATCHES && insns_delta < AUTOCAP_MIN_INSNS) + return; if (cdrom_load_in_progress()) return; /* coherent moment only */ if (cp->busy && cp->busy()) return; - if (s_autocap_triggers >= AUTOCAP_MAX_TRIGGERS) return; - if (s_autocap_last_fire != 0 && - s_frame_count - s_autocap_last_fire < AUTOCAP_COOLDOWN_FRAMES) return; + if (s_frame_count < s_autocap_next_ok) return; /* cooldown (x backoff) */ + + /* Always write the backend-neutral coverage manifest (the player-shareable + * contribution file). It doubles as the futility probe: unchanged content + * since the last compile request, with no new candidates registered by + * that request, means the provider would redo identical work — skip and + * back off instead of spawning it. */ + overlay_capture_write_json(); + { + uint64_t sig = autocap_manifest_sig(); + int reg = overlay_loader_registered_count(); + if (sig == s_autocap_sig_at_req && reg == s_autocap_reg_at_req) { + s_autocap_futile++; + if (s_autocap_backoff < AUTOCAP_BACKOFF_MAX) + s_autocap_backoff <<= 1; + s_autocap_next_ok = s_frame_count + + (uint64_t)AUTOCAP_COOLDOWN_FRAMES * s_autocap_backoff; + return; + } + s_autocap_backoff = 1; + s_autocap_sig_at_req = sig; + s_autocap_reg_at_req = reg; + } s_autocap_last_fire = s_frame_count; s_autocap_triggers++; - /* Always write the backend-neutral coverage manifest (the player-shareable - * contribution file) — then ask the active provider to produce code from it. - * gcc spawns the background compile; sljit (sync producer) declines here and - * JITs on the dispatch miss instead. */ - overlay_capture_write_json(); + s_autocap_next_ok = s_frame_count + AUTOCAP_COOLDOWN_FRAMES; + /* gcc spawns the background compile; sljit (sync producer) declines here + * and JITs on the dispatch miss instead. */ if (cp->request) cp->request(); } diff --git a/runtime/src/overlay_loader.c b/runtime/src/overlay_loader.c index 80f4ddb8..a8ea39b1 100644 --- a/runtime/src/overlay_loader.c +++ b/runtime/src/overlay_loader.c @@ -132,6 +132,9 @@ static NRingEnt s_nring[NRING_CAP]; static uint32_t s_nring_pos = 0; static uint64_t s_nring_seq = 0; static uint32_t s_native_inprogress = 0; /* addr of fn currently in native, 0 = none */ +/* Sampler accessor (phase_profile hot-function histogram): the innermost native + * candidate's registered entry. Read off-thread; a torn read is harmless. */ +uint32_t overlay_loader_native_inprogress(void) { return s_native_inprogress; } static uint64_t s_native_calls_total = 0; extern uint64_t s_frame_count; @@ -248,6 +251,27 @@ uint32_t overlay_loader_get_inprogress(void) { return s_native_inprogress; } /* Same-state differential — defined fully below; declared here for dispatch. */ static int s_diff_mode = 0; static int s_in_shadow = 0; +/* Candidate whose shadow NATIVE pass is currently executing (NULL outside it). + * Set ONLY around run_shadow_diff's native pass — never during the interp pass, + * which must stay pure interp. Lets the CPS continuation re-entry path below + * run THIS candidate's own interiors natively inside its shadow run (so the + * diff exercises the whole function, not just the first CPS segment), while + * every other dispatch inside the shadow stays on the interpreter. */ +static const void *s_shadow_cand = NULL; +/* Longjmp-escape hardening. run_shadow_diff saves the pre-shadow values of + * s_native_exec / s_suppress_irq here (in addition to its locals) so that + * overlay_loader_shadow_escape_fixup() can restore them if an exception + * longjmp ever unwinds through a live shadow frame (should be impossible — + * the dispatch gate refuses to start a shadow while in an exception dispatch — + * but an escape must fail LOUD and self-heal, not silently kill the + * instrument). s_shadow_epoch is the exception-setjmp epoch at shadow start: + * a longjmp targeting a frame with epoch <= s_shadow_epoch escapes the shadow; + * a larger epoch is an exception armed INSIDE the shadow (contained). */ +static int s_shadow_saved_native_exec = 0; +static int s_shadow_saved_supp = 0; +static uint64_t s_shadow_epoch = 0; +static uint32_t s_shadow_escapes = 0; +static uint32_t s_shadow_escapes_native = 0; /* Live sljit mode (PSX_OVERLAY_SLJIT_LIVE / sljit_live debug cmd): JIT overlay * functions on-miss and run them LIVE without the per-shard differential — the * production model (sljit is the sync on-miss producer; safety is the emitter's @@ -272,6 +296,9 @@ static uint32_t s_rehash_miss = 0; /* hashes that didn't match crc_code */ static uint64_t s_gen_fastpath = 0; /* dispatches that skipped the crc32 */ /* via the unchanged page-generation */ /* fast path (overlay-cache v2 P2) */ +static uint64_t s_diffgate_interp = 0; /* CPS interior re-entries sent to the */ + /* interp because their candidate is */ + /* still inside the diff verify budget */ static uint32_t s_last_crc = 0; static uint32_t s_no_manifest = 0; /* exports skipped (no manifest range)*/ static uint32_t s_selfmod = 0; /* entries blacklisted (self-mod) */ @@ -498,7 +525,13 @@ static void persist_sljit_shard(uint32_t entry_phys, uint32_t lo, uint32_t len, * sets s_async_cache_dirty; the DISPATCH thread rescans-and-registers on its * next miss (candidate table stays single-threaded). PSX_SLJIT_SYNC=1 forces * the legacy synchronous JIT-on-miss path. */ +#ifdef _WIN32 static volatile LONG s_async_cache_dirty = 0; +# define async_cache_dirty_exchange(v) InterlockedExchange(&s_async_cache_dirty, (v)) +#else +static volatile int s_async_cache_dirty = 0; +# define async_cache_dirty_exchange(v) __atomic_exchange_n(&s_async_cache_dirty, (v), __ATOMIC_SEQ_CST) +#endif static int g_sljit_async = 1; /* MASTER SAFETY GATE (2026-06-25): the sljit EMITTER mistranslates some overlay @@ -935,7 +968,7 @@ void overlay_loader_async_publish(uint32_t entry_phys, uint32_t lo, uint32_t len if (!wok) { remove(tmp); return; } if (!MoveFileExA(tmp, path, MOVEFILE_REPLACE_EXISTING)) { remove(tmp); return; } s_sljit_persist_writes++; /* telemetry-only; benign cross-thread incr */ - InterlockedExchange(&s_async_cache_dirty, 1); + async_cache_dirty_exchange(1); #else (void)entry_phys; (void)lo; (void)len; (void)crc; (void)blob; (void)blob_size; #endif @@ -1053,6 +1086,41 @@ static int s_irq_defer_cdrom = 0; extern uint32_t i_stat; extern uint32_t i_mask; +/* Nested call-unit depth (Ape memcard native<->interp resume-desync fix). + * + * When the dirty interpreter (or a shard) issues a guest jal/jalr to a callee + * run as a UNIT — an overlay-native shard (overlay_loader_call_native) OR a + * non-local dirty/kernel routine (dispatch_nonlocal_call -> psx_dispatch_call) — + * that callee must be ATOMIC w.r.t. the cooperative thread switch, exactly like + * statically-compiled code (psx_dispatch_impl checks interrupts ONLY at the + * outermost dispatch return; a nested callee never interrupts before its caller + * runs the post-call continuation). Both the overlay CI wrappers AND the dirty + * per-transfer IRQ pumps, however, checked interrupts at EVERY block/transfer + * regardless of nesting; an IRQ + cooperative ChangeThread landing inside such a + * nested unit suspended the interrupted thread with an INCONSISTENT snapshot + * (resume PC at the caller's post-call point, sp still mid-callee) -> a leaked + * stack frame that compounded across cooperative cycles into a smeared jumptable + * index (the Ape "Checking MEMORY CARD" softlock/fatal). While this depth is >0 + * both backends defer the IRQ check; the callee runs to completion, then the + * enclosing top-level dirty pump / outermost dispatch return delivers the IRQ at + * a consistent (pc, sp) boundary. The TOP-LEVEL flow (depth 0) still pumps IRQs, + * so the consumer's poll loop and long overlay game-loops keep their + * responsiveness. Incremented only when the A/B toggle is on (below). */ +int g_call_unit_depth = 0; + +/* A/B toggle for the nested-unit IRQ deferral (PSX_OVERLAY_UNIT_DEFER, default + * ON). Gates the depth INCREMENTS (so =0 leaves g_call_unit_depth at 0 and + * nothing defers — the pre-fix behavior), letting us attribute a behavior change + * to this fix without a rebuild. */ +int overlay_unit_defer_enabled(void) { + static int cached = -1; + if (cached < 0) { + const char *e = getenv("PSX_OVERLAY_UNIT_DEFER"); + cached = (e && (e[0] == '0' || e[0] == 'n' || e[0] == 'N')) ? 0 : 1; + } + return cached; +} + void overlay_loader_set_irq_suppress(int mode, uint32_t ratelimit) { s_suppress_irq = mode ? 1 : 0; s_irq_ratelimit = ratelimit; @@ -1092,6 +1160,9 @@ static int overlay_irq_suppressed_now(void) { } static void overlay_ci_wrapper(CPUState *cpu) { + /* Defer while inside a nested call unit — a callee must not interrupt + * mid-call (static-call atomicity). See g_call_unit_depth. */ + if (g_call_unit_depth > 0) return; if (overlay_irq_suppressed_now()) return; if (s_irq_defer_cdrom && (i_stat & (1u << IRQ_CDROM))) { uint32_t saved_cd = i_stat & (1u << IRQ_CDROM); @@ -1104,6 +1175,10 @@ static void overlay_ci_wrapper(CPUState *cpu) { } static void overlay_ci_at_wrapper(CPUState *cpu, uint32_t resume_pc) { + /* Defer while inside a nested call unit (see g_call_unit_depth): suspending + * here would save resume_pc at the callee's block leader while the enclosing + * dirty caller expects an atomic unit — the resume-desync bug. */ + if (g_call_unit_depth > 0) return; if (overlay_irq_suppressed_now()) return; if (s_irq_defer_cdrom && (i_stat & (1u << IRQ_CDROM))) { uint32_t saved_cd = i_stat & (1u << IRQ_CDROM); @@ -1129,6 +1204,11 @@ static int overlay_irq_budget_blocks_now(void) { static void overlay_post_dispatch_irq_pump(CPUState *cpu) { if (!s_irq_post_dispatch_pump) return; + /* Never deliver an IRQ inside a shadow run: the handler would ack device + * state (I_STAT) that survives the sandbox while the guest state it ran on + * is rolled back — a lost interrupt. Shadow runs are IRQ-suppressed by + * design (s_suppress_irq in run_shadow_diff); this pump must match. */ + if (s_in_shadow) return; if (overlay_irq_budget_blocks_now()) return; if (cpu->pc != 0u) psx_check_interrupts_at(cpu, cpu->pc); else psx_check_interrupts(cpu); @@ -1138,9 +1218,13 @@ static void overlay_post_dispatch_irq_pump(CPUState *cpu) { * xprobe-watched targets (before + after) so overlay-DLL call-outs are visible * in the xprobe `watched` dump the way interp JAL/JALR sites already are. */ extern void dirty_ram_xprobe_call_note(CPUState *cpu, uint32_t target, uint32_t ra, uint8_t phase); +extern int g_exec_phase; /* wall-time sampler phase (dirty_ram_interp.c) */ static void overlay_dispatch_call_probed(CPUState *cpu, uint32_t addr, uint32_t ra) { dirty_ram_xprobe_call_note(cpu, addr, ra, 0); + int prev_phase = g_exec_phase; + g_exec_phase = 3; /* compiled route; dirty/native callees re-tag inside */ psx_dispatch_call(cpu, addr, ra); + g_exec_phase = prev_phase; dirty_ram_xprobe_call_note(cpu, addr, ra, 1); } @@ -1471,7 +1555,9 @@ void overlay_loader_check_cache(uint32_t load_addr, uint32_t size, /* ---- Lazy region cache check (first dispatch miss for a region) -------- */ -#define MAX_CHECKED 64 +/* 64 -> 256 (2026-07-06): the dirty boot-text window adds per-variant regions + * below FLOOR; a full memo silently stops caching new region checks. */ +#define MAX_CHECKED 256 static uint32_t s_checked[MAX_CHECKED]; static int s_nchecked = 0; static int s_last_file_found = 0; @@ -1501,7 +1587,10 @@ void overlay_loader_rescan(void) { * ALL of them; each contributes its functions as separate candidates, and * per-entry validity (the live-RAM hash) decides which candidate is callable * at any moment. Nothing is ever clobbered — discoveries accumulate. */ -#define MAX_LOADED_DLLS 128 +/* 128 -> 512 (2026-07-06): the warmed vault already loads 190 DLLs, past the + * old cap — dll_already_loaded() then misses and a rescan can double-register + * a DLL's candidates. Sized for vault + autocompile + boot-text regions. */ +#define MAX_LOADED_DLLS 512 static char s_loaded_paths[MAX_LOADED_DLLS][768]; static int s_nloaded_paths = 0; @@ -1548,19 +1637,20 @@ static void try_load_region(uint32_t phys) { * DLLs are keyed by this start address in their filename. * * CRITICAL: the capture clamps each region to its WINDOW — kernel - * [0, DIRTY_RAM_KERNEL_WINDOW_END) or overlay [OVERLAY_REGION_FLOOR, end) — - * so a DLL's region_start is the first dirty page AT OR ABOVE the window - * floor. But the dirty run is contiguous ACROSS the static-EXE gap between - * those windows: the boot EXE image (0x10000..floor) was written at load - * time, so its pages are dirty too. Walking back without the same floor - * clamp overshoots through the EXE region and yields a region_start (e.g. - * 0x10000) that NO DLL filename matches — so the overlay's DLLs never load - * and its functions interpret forever (the dominant slowness). Clamp the - * walkback to the window floor so region_start matches the capture. */ - uint32_t floor_pg = (phys < DIRTY_RAM_KERNEL_WINDOW_END) - ? 0u - : (OVERLAY_REGION_FLOOR / page_sz); + * [0, KERNEL_WINDOW_END), dirty boot-text [KERNEL_WINDOW_END, FLOOR), or + * overlay [FLOOR, end) — so a DLL's region_start is the first dirty page + * AT OR ABOVE the window floor. The walkback must apply the SAME clamp, + * chosen by PAGE (the capture windows are page-granular and FLOOR need + * not be page-aligned): an overlay-region walk stopping anywhere below + * FLOOR's page, or a boot-text walk crossing into the kernel window, + * yields a region_start that NO DLL filename matches — the overlay's + * DLLs never load and its functions interpret forever. */ + uint32_t kern_pg = DIRTY_RAM_KERNEL_WINDOW_END / page_sz; + uint32_t ovl_pg = OVERLAY_REGION_FLOOR / page_sz; uint32_t pg = phys / page_sz; + uint32_t floor_pg = (pg < kern_pg) ? 0u + : (pg < ovl_pg) ? kern_pg + : ovl_pg; while (pg > floor_pg) { uint32_t pp = pg - 1; if (!((dirty_ram_get_bitmap_word(pp >> 5) >> (pp & 31u)) & 1u)) break; @@ -1608,7 +1698,7 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) { if (head < 0 && s_active && overlay_cache_window_contains(phys)) { /* Pick up shards the background compile worker just published (idempotent * rescan; runs only when a publish has flipped the dirty flag). */ - if (g_sljit_tier_enabled && g_sljit_async && InterlockedExchange(&s_async_cache_dirty, 0)) { + if (g_sljit_tier_enabled && g_sljit_async && async_cache_dirty_exchange(0)) { scan_sljit_cache_dir(); head = idx_head(phys); } @@ -1691,8 +1781,33 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) { if (matched) { if (c->state != ENTRY_VALID) { c->state = ENTRY_VALID; s_valid_count++; } if (c->device_touch) { if (_probe) s_cps_probe_outcome = 3; s_disp_interp++; return 0; } - if (!s_native_exec || overlay_native_blocked(c->addr) || overlay_native_blocked(addr)) - { if (_probe) s_cps_probe_outcome = 4; s_would_run_native++; s_disp_interp++; return 0; } + /* Diff instrument — same contract as the entry chain's want_diff + * gate below. A continuation re-entry must NOT run native blind + * while its candidate is still inside the verify budget: CPS + * interiors dominate dispatch, so without this gate ~all native + * execution bypassed the diff harness (shadow_calls=1 over 51K + * frames while gen_fastpath took ~600M hits). Interiors are + * validated transitively: the entry-level shadow diff runs the + * WHOLE function natively (in_own_shadow below), so until the + * candidate passes its budget its interiors take the interpreter + * (the authority). Inside the candidate's OWN shadow native pass + * interiors DO run native — that IS the diff exercising the + * continuation blocks; the blocklist is not consulted there, + * matching the entry gate (which shadow-diffs blocked candidates + * too — the blocklist blocks LIVE native, not the sandbox). */ + int in_own_shadow = (s_in_shadow && (const void *)c == s_shadow_cand); + if (!in_own_shadow) { + int want_diff = s_diff_mode || (s_sljit_live && c->dll < 0); + if (want_diff && addr < 0x10000u) want_diff = 0; + if (want_diff && c->diff_passes < OVERLAY_DIFF_BUDGET) { + if (_probe) s_cps_probe_outcome = 5; + s_diffgate_interp++; + s_disp_interp++; + return 0; + } + if (!s_native_exec || overlay_native_blocked(c->addr) || overlay_native_blocked(addr)) + { if (_probe) s_cps_probe_outcome = 4; s_would_run_native++; s_disp_interp++; return 0; } + } if (_probe) s_cps_probe_outcome = 2; uint32_t slot = s_nring_pos++ & (NRING_CAP - 1u); s_nring[slot].addr = addr; @@ -1707,7 +1822,12 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) { s_active_stack[s_active_depth++] = ci; s_disp_native++; cpu->pc = addr; /* route the func's entry-switch to the block */ - c->fn(cpu); + { + int prev_phase = g_exec_phase; + g_exec_phase = 2; + c->fn(cpu); + g_exec_phase = prev_phase; + } overlay_post_dispatch_irq_pump(cpu); if (s_active_depth > 0) s_active_depth--; s_nring[slot].returned = 1; @@ -1779,7 +1899,34 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) { * the trusted tier (validated at dev time) and run native directly; * they are diffed only in explicit dev diff mode. */ int want_diff = s_diff_mode || (s_sljit_live && c->dll < 0); + /* Kernel-window candidates (call gates 0xA0/0xB0/0xC0 and the RAM + * kernel) are NOT shadow-diffable: the gates tail-jump via RAM + * tables into kernel SERVICES whose behavior depends on scheduler/ + * event state and can block (TestEvent/WaitEvent) — the two passes + * legitimately run different instruction paths and every call + * "diverges" (observed: 2016/2017 calls, all 0xB0, flooding the + * ring). The diff harness validates OVERLAY shard codegen against + * the interp oracle; kernel gates stay on their normal route. */ + if (want_diff && addr < 0x10000u) want_diff = 0; if (want_diff && !s_in_shadow && c->diff_passes < OVERLAY_DIFF_BUDGET) { + /* NEVER start a shadow inside an exception dispatch: the guest's + * ReturnFromException longjmps to the setjmp frame BELOW us + * (psx_check_interrupts), unwinding run_shadow_diff WITHOUT its + * epilogue — s_in_shadow/s_native_exec/s_suppress_irq stay stuck + * and the diff instrument is dead for the rest of the run + * (observed: shadow_calls=1 over 51K frames). Exceptions ENTERED + * during a shadow are contained — their setjmp is armed inside + * the shadow frame — so gating the START is structurally + * sufficient (every psx_exception_longjmp site is guarded by + * psx_get_in_exception()). The candidate still must not run + * native unvalidated: route to the interpreter (the authority); + * it gets diffed at its next non-exception dispatch. */ + extern int psx_get_in_exception(void); + if (psx_get_in_exception()) { + s_diffgate_interp++; + s_disp_interp++; + return 0; + } run_shadow_diff(cpu, c, addr); return 1; } @@ -1811,7 +1958,12 @@ int overlay_loader_dispatch(CPUState *cpu, uint32_t addr) { extern void dirty_ram_log_marker(uint32_t addr, uint32_t tag, int kind); uint32_t mtag = (uint32_t)s_nring[slot].seq; /* stable across nesting */ dirty_ram_log_marker(c->addr, mtag, 0); - c->fn(cpu); + { + int prev_phase = g_exec_phase; + g_exec_phase = 2; + c->fn(cpu); + g_exec_phase = prev_phase; + } overlay_post_dispatch_irq_pump(cpu); dirty_ram_log_marker(c->addr, mtag, 1); if (s_active_depth > 0) s_active_depth--; @@ -2028,9 +2180,12 @@ static void run_shadow_diff(CPUState *cpu, Candidate *c, uint32_t addr) { uint8_t *ram = memory_get_ram_ptr(); uint8_t *spad = memory_get_scratchpad_ptr(); + extern uint64_t psx_exception_setjmp_epoch(void); s_in_shadow = 1; + s_shadow_epoch = psx_exception_setjmp_epoch(); int saved_supp = s_suppress_irq; s_suppress_irq = 1; /* isolate computation; longjmp-safe */ + s_shadow_saved_supp = saved_supp; /* escape-fixup mirror (see decl) */ /* Validate ONE function at a time: nested OVERLAY calls run via the * INTERPRETER on BOTH passes (s_native_exec=0). Otherwise the native pass * dispatches each callee as its own native shard while the interp pass runs @@ -2041,6 +2196,7 @@ static void run_shadow_diff(CPUState *cpu, Candidate *c, uint32_t addr) { * tree soundness then follows by induction. */ int sv = s_native_exec; s_native_exec = 0; + s_shadow_saved_native_exec = sv; /* escape-fixup mirror (see decl) */ CPUState cpu0 = *cpu; memcpy(s_ram0, ram, SHADOW_RAM_SIZE); @@ -2083,7 +2239,19 @@ static void run_shadow_diff(CPUState *cpu, Candidate *c, uint32_t addr) { memcpy(spad, s_spad0, SHADOW_SPAD_SIZE); uint32_t stop_ra = cpu->gpr[31]; /* entry $ra = the function's return point */ - c->fn(cpu); + /* Arm the own-interior native route for the NATIVE pass only (see + * s_shadow_cand decl): the candidate's CPS continuation re-entries run + * native so the diff exercises every block of the function, not just the + * first segment (nested CALLS still run interp on both passes — + * s_native_exec=0 above). Never armed during the interp pass, which must + * stay pure interp. */ + s_shadow_cand = c; + { + int prev_phase = g_exec_phase; + g_exec_phase = 2; + c->fn(cpu); + g_exec_phase = prev_phase; + } /* CPS shards exit with cpu->pc set to the next tail target. Chain through * the normal dispatcher to the original caller return, with s_native_exec=0 * above so nested overlay calls still run through the interpreter on both @@ -2095,9 +2263,13 @@ static void run_shadow_diff(CPUState *cpu, Candidate *c, uint32_t addr) { uint32_t tv = cpu->pc; if ((tv & 0x1FFFFFFFu) == (stop_ra & 0x1FFFFFFFu)) break; /* returned */ cpu->pc = 0; + int prev_phase = g_exec_phase; + g_exec_phase = 3; /* compiled route; dirty/native callees re-tag inside */ psx_dispatch_call(cpu, tv, stop_ra); + g_exec_phase = prev_phase; } } + s_shadow_cand = NULL; CPUState cpuN = *cpu; memcpy(s_ramN, ram, SHADOW_RAM_SIZE); @@ -2130,7 +2302,13 @@ static void run_shadow_diff(CPUState *cpu, Candidate *c, uint32_t addr) { s_detail_nat_hi = cpuN.hi; s_detail_nat_lo = cpuN.lo; s_detail_int_hi = cpuI.hi; s_detail_int_lo = cpuI.lo; } - if (s_sdiv_n < SDIV_CAP) { + /* Ring-flood guard: one persistently-diverging site must not evict + * every other site's records (512x 0xB0 drowned the 0xF514 hunt). + * Cap records per address; the divergence COUNTER still increments. */ + int addr_recs = 0; + for (int i = 0; i < s_sdiv_n; i++) + if (s_sdiv[i].addr == c->addr && ++addr_recs >= 16) break; + if (s_sdiv_n < SDIV_CAP && addr_recs < 16) { ShadowDiv *d = &s_sdiv[s_sdiv_n++]; d->seq = s_shadow_calls; d->addr = c->addr; d->reg = reg; @@ -2157,6 +2335,29 @@ static void run_shadow_diff(CPUState *cpu, Candidate *c, uint32_t addr) { s_in_shadow = 0; } +/* Called by deferred_exception_longjmp() (interrupts.c) before it unwinds. + * If the longjmp target frame predates a live shadow run (target_epoch <= + * s_shadow_epoch), the unwind blows through run_shadow_diff and its epilogue + * never runs — restore the shadow-scoped globals here and count the escape. + * The dispatch gate (no shadow start while in an exception dispatch) makes + * this structurally unreachable; if the counters ever move, that invariant is + * broken and must be investigated. An escape during the NATIVE pass + * (s_shadow_cand set) additionally means speculative native state leaked into + * the live timeline — counted separately; it must stay 0. A longjmp to a + * frame armed AFTER shadow start (target_epoch > s_shadow_epoch) is an + * exception contained inside the shadow: it lands inside the shadow frame and + * the shadow continues — the flags must NOT be touched. */ +void overlay_loader_shadow_escape_fixup(uint64_t target_epoch) { + if (!s_in_shadow) return; + if (target_epoch > s_shadow_epoch) return; /* contained: leave armed */ + s_shadow_escapes++; + if (s_shadow_cand != NULL) s_shadow_escapes_native++; + s_in_shadow = 0; + s_shadow_cand = NULL; + s_native_exec = s_shadow_saved_native_exec; + s_suppress_irq = s_shadow_saved_supp; +} + int overlay_loader_dump_shadow_detail(char *out, int cap) { int n = 0; n += snprintf(out + n, cap - n, @@ -2182,10 +2383,15 @@ int overlay_loader_dump_shadow(char *out, int cap) { int n = 0; n += snprintf(out + n, cap - n, "{\"diff_mode\":%d,\"shadow_calls\":%llu,\"divergences\":%llu," - "\"skipped_device\":%llu,\"records\":[", + "\"skipped_device\":%llu,\"interior_gated\":%llu," + "\"in_shadow\":%d,\"native_exec\":%d," + "\"escapes\":%u,\"escapes_native\":%u,\"records\":[", s_diff_mode, (unsigned long long)s_shadow_calls, (unsigned long long)s_shadow_divs, - (unsigned long long)s_shadow_skipped_dev); + (unsigned long long)s_shadow_skipped_dev, + (unsigned long long)s_diffgate_interp, + s_in_shadow, s_native_exec, + s_shadow_escapes, s_shadow_escapes_native); for (int i = 0; i < s_sdiv_n && n < cap - 200; i++) { ShadowDiv *d = &s_sdiv[i]; n += snprintf(out + n, cap - n, @@ -2254,7 +2460,17 @@ int overlay_loader_call_native(CPUState *cpu, uint32_t addr) { if (idx_head(phys) < 0) return 0; /* fast reject: not a candidate */ uint32_t in_regs[34]; overlay_regs_snap(in_regs, cpu); - if (!overlay_loader_dispatch(cpu, addr)) return 0; + /* Run the callee as an atomic UNIT: IRQ delivery is deferred inside it (both + * the overlay CI wrappers and the dirty IRQ pumps check g_call_unit_depth) so + * a cooperative ChangeThread cannot suspend the interrupted thread mid-callee + * with an inconsistent (resume_pc, sp) snapshot. Restore (not just decrement) + * so a bail/longjmp-out unwinds the depth correctly; the scheduler landing + * also resets it to 0 as a backstop. Gated by the A/B toggle. */ + int prev_unit_depth = g_call_unit_depth; + if (overlay_unit_defer_enabled()) g_call_unit_depth = prev_unit_depth + 1; + int ran = overlay_loader_dispatch(cpu, addr); + g_call_unit_depth = prev_unit_depth; + if (!ran) return 0; overlay_fp_log(addr, in_regs, cpu, 1); return 1; } @@ -2282,13 +2498,16 @@ static int psx_sljit_call_inner(CPUState *cpu, uint32_t target, uint32_t return_ int check_contract) { uint32_t site_sp = cpu->gpr[29]; /* sp at the call (after the delay slot) */ #ifdef PSX_HAS_GAME_DISPATCH - /* Skip the compiled game function if its page is dirty (an overlay overwrote it - * after the game-start baseline -> compiled code is stale); fall through to the - * native/interp paths which run the live overlay. Clean pages run compiled. */ - if (!dirty_ram_is_dirty(target & 0x1FFFFFFFu)) { + /* Skip the compiled game function when its target is no longer native-safe; + * fall through to the native/interp paths that run the live RAM bytes. */ + if (dirty_ram_text_native_ok(target & 0x1FFFFFFFu)) { extern int psx_dispatch_game_compiled(CPUState *cpu, uint32_t addr); cpu->pc = 0; - if (psx_dispatch_game_compiled(cpu, target)) { + int prev_phase = g_exec_phase; + g_exec_phase = 3; + int _gc = psx_dispatch_game_compiled(cpu, target); + g_exec_phase = prev_phase; + if (_gc) { if (g_psx_call_bail) return 1; if (cpu->pc != 0) return 1; if (check_contract && psx_call_contract(cpu, return_pc, site_sp)) return 1; @@ -2307,7 +2526,12 @@ static int psx_sljit_call_inner(CPUState *cpu, uint32_t target, uint32_t return_ * interpreter's nonlocal path — handles interp/dirty callees + the return * contract internally). */ cpu->pc = 0; - psx_dispatch_call(cpu, target, return_pc); + { + int prev_phase = g_exec_phase; + g_exec_phase = 3; /* compiled route; dirty/native callees re-tag inside */ + psx_dispatch_call(cpu, target, return_pc); + g_exec_phase = prev_phase; + } if (g_psx_call_bail) return 1; if (cpu->pc != 0) return 1; return 0; diff --git a/runtime/src/overlay_sljit.c b/runtime/src/overlay_sljit.c index 676da248..66e39a69 100644 --- a/runtime/src/overlay_sljit.c +++ b/runtime/src/overlay_sljit.c @@ -18,6 +18,7 @@ #include <stdlib.h> #include "sljitLir.h" +#include "gpu.h" #include "ws_backdrop_detect.h" /* shared backdrop-window detector (auto_backdrop) */ #ifdef _WIN32 @@ -161,8 +162,18 @@ static uint32_t s_gpr_lru; * overlay widens identically to the gcc cache + interp (widescreen FOV). */ static int s_frag_ws_cull; extern int psx_ws_cull_sltiu(uint32_t sx, uint32_t imm); /* gpu.c — shared widen */ +extern int psx_ws_cull_slti(uint32_t sx, uint32_t imm); /* gpu.c — signed right edge */ +extern int psx_ws_cull_bltz(uint32_t v); /* gpu.c — signed left edge */ +extern int psx_ws_is_cull_w_imm(uint32_t imm); /* gpu.c — per-game W-imm set */ +extern int psx_ws_auto_cull_on(void); /* gpu.c — per-game opt-in gate */ +extern int psx_ws_cull_bltz_at(const uint32_t *words, int n, int idx); /* gpu.c */ +extern int psx_ws_func_has_screen_cull(const uint32_t *words, int n); /* gpu.c */ #define SLJIT_MAX_FRAG_INSNS 2048u #define SLJIT_MAX_FRAG_CTRL 512u /* branches/jumps per fragment cap */ +/* Per-fragment (PASS 1) map of classified LEFT-edge funnel bltz words + * (ws_cull_detect.h idioms 2/3): emit_cond_to_S1 routes those through + * psx_ws_cull_bltz so the reject widens with the revealed margin. */ +static uint8_t s_ws_bltz[SLJIT_MAX_FRAG_INSNS]; /* Per-fragment (PASS 1) backdrop-preload site map ([widescreen.cull] * auto_backdrop): s_bd_kind[i] = WS_BD_START_ZERO/END_WIDEN for the window * START/END finalize at fragment word i, else WS_BD_NONE. emit_one routes those @@ -354,26 +365,31 @@ static void emit_helper2(struct sljit_compiler *C, int helper_idx, uint32_t insn * of the persisted sljit cache). Order MUST match the SLJIT_HLP_* enum. Called * once at startup, after CPUState's memory fn-pointers are wired. */ void overlay_sljit_init_helpers(CPUState *cpu) { - cpu->sljit_helpers[SLJIT_HLP_MEMX] = (void *)psx_sljit_memx; - cpu->sljit_helpers[SLJIT_HLP_COP2] = (void *)psx_sljit_cop2; - cpu->sljit_helpers[SLJIT_HLP_WS_CULL] = (void *)psx_ws_cull_sltiu; - cpu->sljit_helpers[SLJIT_HLP_CALL] = (void *)psx_sljit_call; + cpu->sljit_helpers[SLJIT_HLP_MEMX] = (void *)psx_sljit_memx; + cpu->sljit_helpers[SLJIT_HLP_COP2] = (void *)psx_sljit_cop2; + cpu->sljit_helpers[SLJIT_HLP_WS_CULL] = (void *)psx_ws_cull_sltiu; + cpu->sljit_helpers[SLJIT_HLP_CALL] = (void *)psx_sljit_call; + cpu->sljit_helpers[SLJIT_HLP_WS_CULL_SLTI] = (void *)psx_ws_cull_slti; + cpu->sljit_helpers[SLJIT_HLP_WS_CULL_BLTZ] = (void *)psx_ws_cull_bltz; } -/* Emit `rt = psx_ws_cull_sltiu(gpr[rs], imm)` for a flagged auto_screen_x cull - * site, so a sljit-JIT'd overlay widens the render-funnel screen-X reject the - * same way the gcc backend does. The helper reads only its two args (+ the - * runtime widescreen margin); it does NOT touch cpu->gpr[], so the GPR cache - * survives — same icall discipline as emit_load (materialise the source into a - * scratch before the call, write the result into rt after). */ -static void emit_ws_cull(struct sljit_compiler *C, uint32_t rt, uint32_t rs, uint32_t imm) { +/* Emit `rt = <cull helper>(gpr[rs], imm)` for a flagged auto_screen_x cull + * site (helper_idx = SLJIT_HLP_WS_CULL for the unsigned per-vertex idiom, + * SLJIT_HLP_WS_CULL_SLTI for the signed min/max right edge), so a sljit-JIT'd + * overlay widens the render-funnel screen-X reject the same way the gcc + * backend does. The helper reads only its two args (+ the runtime widescreen + * margin); it does NOT touch cpu->gpr[], so the GPR cache survives — same + * icall discipline as emit_load (materialise the source into a scratch before + * the call, write the result into rt after). */ +static void emit_ws_cull(struct sljit_compiler *C, uint32_t rt, uint32_t rs, uint32_t imm, + int helper_idx) { sljit_s32 a; sljit_sw aw; gpr_src(C, rs, &a, &aw); sljit_emit_op1(C, SLJIT_MOV32, SLJIT_R0, 0, a, aw); /* sx -> R0 (arg0) */ sljit_emit_op1(C, SLJIT_MOV32, SLJIT_R1, 0, SLJIT_IMM, (sljit_sw)imm); /* imm -> R1 (arg1) */ sljit_emit_op1(C, SLJIT_MOV, SLJIT_R2, 0, SLJIT_MEM1(R_CPU), /* fn (cpu-relative) */ (sljit_sw)(offsetof(CPUState, sljit_helpers) - + (size_t)SLJIT_HLP_WS_CULL * sizeof(void *))); + + (size_t)helper_idx * sizeof(void *))); sljit_emit_icall(C, SLJIT_CALL, SLJIT_ARGS2(32, 32, 32), SLJIT_R2, 0); /* R0 = verdict */ sljit_emit_op1(C, SLJIT_MOV32, gpr_dst(C, rt), 0, SLJIT_R0, 0); gpr_unpin(); @@ -577,11 +593,14 @@ static int emit_one(struct sljit_compiler *C, uint32_t insn, int bd_kind, int bd return EMIT_OK; } case 0x0A: /* SLTI (signed) */ - emit_slt(C, SLJIT_SUB32 | SLJIT_SET_SIG_LESS, SLJIT_SIG_LESS, rt, rs, 0, 0, simm); + if (s_frag_ws_cull && psx_ws_is_cull_w_imm(imm)) + emit_ws_cull(C, rt, rs, imm, SLJIT_HLP_WS_CULL_SLTI); /* signed funnel right edge */ + else + emit_slt(C, SLJIT_SUB32 | SLJIT_SET_SIG_LESS, SLJIT_SIG_LESS, rt, rs, 0, 0, simm); return EMIT_OK; case 0x0B: /* SLTIU (unsigned, simm sign-extended then compared unsigned) */ - if (s_frag_ws_cull && (imm == 0x140 || imm == 0x141)) - emit_ws_cull(C, rt, rs, imm); /* widescreen render-funnel cull widen (auto_screen_x) */ + if (s_frag_ws_cull && psx_ws_is_cull_w_imm(imm)) + emit_ws_cull(C, rt, rs, imm, SLJIT_HLP_WS_CULL); /* widescreen render-funnel cull widen (auto_screen_x) */ else emit_slt(C, SLJIT_SUB32 | SLJIT_SET_LESS, SLJIT_LESS, rt, rs, 0, 0, simm); return EMIT_OK; @@ -663,9 +682,25 @@ static int classify_control(uint32_t insn, uint32_t off, uint32_t entry_phys, * the source registers at the BRANCH instruction (before its delay slot runs). * Reads go through the GPR cache; the predicate lands in S1, which is never a * cache slot, so it survives the delay slot + the pre-jump flush. */ -static void emit_cond_to_S1(struct sljit_compiler *C, uint32_t insn) { +static void emit_cond_to_S1(struct sljit_compiler *C, uint32_t insn, int ws_bltz) { uint32_t op = f_op(insn), rs = f_rs(insn), rt = f_rt(insn); sljit_s32 a, b; sljit_sw aw, bw; + /* Classified LEFT-edge funnel bltz (auto_screen_x): predicate comes from + * psx_ws_cull_bltz(gpr[rs]) — reject only past the revealed margin + * (identity at 4:3). Helper reads only its arg, but the icall discipline + * still flushes/resets the GPR cache via emit-helper conventions; here we + * call through the cpu-relative table and land the verdict in S1. */ + if (ws_bltz && op == 0x01 && rt == 0x00) { + gpr_src(C, rs, &a, &aw); + sljit_emit_op1(C, SLJIT_MOV32, SLJIT_R0, 0, a, aw); + sljit_emit_op1(C, SLJIT_MOV, SLJIT_R2, 0, SLJIT_MEM1(R_CPU), + (sljit_sw)(offsetof(CPUState, sljit_helpers) + + (size_t)SLJIT_HLP_WS_CULL_BLTZ * sizeof(void *))); + sljit_emit_icall(C, SLJIT_CALL, SLJIT_ARGS1(32, 32), SLJIT_R2, 0); + sljit_emit_op1(C, SLJIT_MOV32, R_CTRL, 0, SLJIT_R0, 0); + gpr_unpin(); + return; + } switch (op) { case 0x04: /* BEQ rs==rt */ case 0x05: /* BNE rs!=rt */ @@ -800,6 +835,17 @@ void overlay_sljit_try_compile(uint32_t entry, if (!found_term || frag_words == 0 || frag_words > SLJIT_MAX_FRAG_INSNS) { s_declines++; return; } + + /* Explicit address-listed cull hooks carry a live aspect term. Generated + * overlay DLLs and the interpreter implement it directly; keep these rare + * fragments out of the generic JIT so a vanilla immediate is never cached. */ + for (uint32_t i = 0; i < frag_words; i++) { + uint32_t pc = entry + i * 4u; + if (psx_ws_is_cull_bias_site(pc) || psx_ws_is_cull_slti_site(pc)) { + s_declines++; + return; + } + } if (off0 + frag_words * 4u > size) { s_declines++; return; } /* delay slot off image */ /* Validate branch targets: in-range, aligned, not landing on a delay slot. */ @@ -816,31 +862,30 @@ void overlay_sljit_try_compile(uint32_t entry, * see gpr_ref_mask). distinct >= the max regs in any single instruction, so * gpr_alloc always finds a slot. */ uint32_t used_mask = 0; - int ws_hx = 0, ws_hy = 0; /* render-funnel screen-cull signature (auto_screen_x) */ - for (uint32_t i = 0; i < frag_words; i++) { - uint32_t w = img_word(bytes, off0 + i * 4u); - used_mask |= gpr_ref_mask(w); - if ((w & 0xFC000000u) == 0x2C000000u) { /* sltiu */ - uint32_t im = w & 0xFFFFu; - if (im == 0x140 || im == 0x141) ws_hx = 1; - else if (im == 0xE0 || im == 0xF1) ws_hy = 1; - } - } - s_frag_ws_cull = ws_hx && ws_hy; + for (uint32_t i = 0; i < frag_words; i++) + used_mask |= gpr_ref_mask(img_word(bytes, off0 + i * 4u)); - /* Backdrop-preload window detection ([widescreen.cull] auto_backdrop): scan - * the fragment for column-window generators and mark the START/END finalize - * words so emit_one routes them through psx_ws_backdrop_value. base_pc = - * entry (virtual) so a site PC maps to its fragment word index. A window - * whose magic load falls outside this fragment is simply not detected here - * (the contiguous case body is the common shape; the interp/gcc paths cover - * the rest), so the worst case is a missed widen, never a mis-compile. */ + /* Widescreen cull + backdrop-preload detection ([widescreen.cull] + * auto_screen_x / auto_backdrop): both run over the same fragment word + * image via the shared detectors (ws_cull_detect.h / ws_backdrop_detect.h) + * so the sljit JIT derives the SAME sites as the gcc emit + interp. Both + * are gated on the per-game opt-in (a title that never opted in must never + * have its code pattern-scanned and rewritten). A window whose context + * falls outside this fragment is simply not detected here (the interp/gcc + * paths cover the rest) — worst case a missed widen, never a mis-compile. */ for (uint32_t i = 0; i < frag_words && i < SLJIT_MAX_FRAG_INSNS; i++) { - s_bd_kind[i] = WS_BD_NONE; s_bd_cols[i] = 0; + s_bd_kind[i] = WS_BD_NONE; s_bd_cols[i] = 0; s_ws_bltz[i] = 0; } + s_frag_ws_cull = 0; { static uint32_t bdwords[SLJIT_MAX_FRAG_INSNS]; for (uint32_t i = 0; i < frag_words; i++) bdwords[i] = img_word(bytes, off0 + i * 4u); + if (psx_ws_auto_cull_on()) { + s_frag_ws_cull = psx_ws_func_has_screen_cull(bdwords, (int)frag_words); + if (s_frag_ws_cull) + for (uint32_t i = 0; i < frag_words && i < SLJIT_MAX_FRAG_INSNS; i++) + s_ws_bltz[i] = (uint8_t)psx_ws_cull_bltz_at(bdwords, (int)frag_words, (int)i); + } WsBackdropSite bds[16]; int nb = psx_ws_find_backdrop_windows(bdwords, (int)frag_words, entry, bds, 16); for (int b = 0; b < nb; b++) { @@ -909,7 +954,8 @@ void overlay_sljit_try_compile(uint32_t entry, sljit_emit_op1(C, SLJIT_MOV32, gpr_dst(C, 31), 0, SLJIT_IMM, (sljit_sw)(entry + i * 4u + 8u)); } - emit_cond_to_S1(C, insn); /* predicate read BEFORE the delay slot */ + emit_cond_to_S1(C, insn, /* predicate read BEFORE the delay slot */ + s_frag_ws_cull && i < SLJIT_MAX_FRAG_INSNS && s_ws_bltz[i]); } pending = PEND_BR; pend_cond = (ctrl == CTRL_BRANCH); diff --git a/runtime/src/png_write.h b/runtime/src/png_write.h new file mode 100644 index 00000000..8347b46a --- /dev/null +++ b/runtime/src/png_write.h @@ -0,0 +1,112 @@ +/* png_write.h — minimal, dependency-free RGB→PNG writer. + * + * A real PNG (8-bit truecolor) whose IDAT is a zlib stream of "stored" + * (uncompressed) DEFLATE blocks: bigger on disk than a compressed PNG, but + * accepted by every viewer (and the harness Read tool), and nothing new to link + * against — which keeps the self-contained static binaries self-contained. + * + * Header-only with `static` linkage so both the runtime debug server and the + * Beetle debug server (separate binaries / translation units) can emit PNG from + * one source, with no shared object to add to either build. */ +#ifndef PSX_PNG_WRITE_H +#define PSX_PNG_WRITE_H + +#include <stdio.h> +#include <stdint.h> +#include <stdlib.h> +#include <string.h> + +static uint32_t s_png_crc_tbl[256]; +static int s_png_crc_init = 0; +static void png_crc_build(void) { + for (uint32_t n = 0; n < 256; n++) { + uint32_t c = n; + for (int k = 0; k < 8; k++) c = (c & 1) ? (0xEDB88320u ^ (c >> 1)) : (c >> 1); + s_png_crc_tbl[n] = c; + } + s_png_crc_init = 1; +} +static uint32_t png_crc_update(uint32_t crc, const uint8_t *p, size_t n) { + if (!s_png_crc_init) png_crc_build(); + for (size_t i = 0; i < n; i++) crc = s_png_crc_tbl[(crc ^ p[i]) & 0xFF] ^ (crc >> 8); + return crc; +} +static uint32_t png_adler32(const uint8_t *p, size_t n) { + uint32_t a = 1, b = 0; + /* process in blocks so the 16-bit sums never overflow before the mod */ + while (n) { + size_t k = n < 5552 ? n : 5552; + for (size_t i = 0; i < k; i++) { a += p[i]; b += a; } + a %= 65521; b %= 65521; p += k; n -= k; + } + return (b << 16) | a; +} +static void png_put_be32(FILE *f, uint32_t v) { + uint8_t b[4] = { (uint8_t)(v >> 24), (uint8_t)(v >> 16), + (uint8_t)(v >> 8), (uint8_t)v }; + fwrite(b, 1, 4, f); +} +static void png_chunk(FILE *f, const char *type, const uint8_t *data, size_t len) { + png_put_be32(f, (uint32_t)len); + uint32_t crc = 0xFFFFFFFFu; + crc = png_crc_update(crc, (const uint8_t *)type, 4); + crc = png_crc_update(crc, data, len); + fwrite(type, 1, 4, f); + if (len) fwrite(data, 1, len, f); + png_put_be32(f, crc ^ 0xFFFFFFFFu); +} +/* Write an RGB (3 bytes/pixel, top-down) buffer as a PNG. Returns 1 on success. */ +static int png_write_rgb(FILE *f, const uint8_t *rgb, uint32_t w, uint32_t h) { + static const uint8_t sig[8] = { 137,80,78,71,13,10,26,10 }; + fwrite(sig, 1, 8, f); + + uint8_t ihdr[13]; + ihdr[0]=(uint8_t)(w>>24); ihdr[1]=(uint8_t)(w>>16); ihdr[2]=(uint8_t)(w>>8); ihdr[3]=(uint8_t)w; + ihdr[4]=(uint8_t)(h>>24); ihdr[5]=(uint8_t)(h>>16); ihdr[6]=(uint8_t)(h>>8); ihdr[7]=(uint8_t)h; + ihdr[8]=8; /* bit depth */ + ihdr[9]=2; /* color type 2 = truecolor RGB */ + ihdr[10]=0; /* compression */ + ihdr[11]=0; /* filter */ + ihdr[12]=0; /* interlace */ + png_chunk(f, "IHDR", ihdr, sizeof ihdr); + + /* Filtered raw scanlines: each row prefixed with filter byte 0 (None). */ + size_t raw_len = (size_t)h * (1 + (size_t)w * 3); + uint8_t *raw = (uint8_t *)malloc(raw_len); + if (!raw) return 0; + for (uint32_t y = 0; y < h; y++) { + uint8_t *row = raw + (size_t)y * (1 + (size_t)w * 3); + row[0] = 0; + memcpy(row + 1, rgb + (size_t)y * w * 3, (size_t)w * 3); + } + + /* zlib stream: 2-byte header + stored DEFLATE blocks + 4-byte Adler32. */ + size_t nblocks = (raw_len + 65534) / 65535; if (nblocks == 0) nblocks = 1; + size_t z_len = 2 + nblocks * 5 + raw_len + 4; + uint8_t *z = (uint8_t *)malloc(z_len); + if (!z) { free(raw); return 0; } + size_t zi = 0; + z[zi++] = 0x78; /* CMF: 32K window, deflate */ + z[zi++] = 0x01; /* FLG: check bits make 0x7801 a multiple of 31 */ + size_t off = 0; + while (off < raw_len) { + size_t n = raw_len - off; if (n > 65535) n = 65535; + int final = (off + n >= raw_len); + z[zi++] = (uint8_t)(final ? 1 : 0); /* BFINAL | BTYPE=00 */ + z[zi++] = (uint8_t)(n & 0xFF); z[zi++] = (uint8_t)(n >> 8); + uint16_t nl = (uint16_t)~n; + z[zi++] = (uint8_t)(nl & 0xFF); z[zi++] = (uint8_t)(nl >> 8); + memcpy(z + zi, raw + off, n); zi += n; off += n; + } + uint32_t ad = png_adler32(raw, raw_len); + z[zi++] = (uint8_t)(ad >> 24); z[zi++] = (uint8_t)(ad >> 16); + z[zi++] = (uint8_t)(ad >> 8); z[zi++] = (uint8_t)ad; + free(raw); + + png_chunk(f, "IDAT", z, zi); + free(z); + png_chunk(f, "IEND", NULL, 0); + return 1; +} + +#endif /* PSX_PNG_WRITE_H */ diff --git a/runtime/src/psx_cycles.c b/runtime/src/psx_cycles.c index c8057da5..01ada9b6 100644 --- a/runtime/src/psx_cycles.c +++ b/runtime/src/psx_cycles.c @@ -2,6 +2,7 @@ #include "psx_cycles.h" #include "cpu_state.h" +#include <stdlib.h> #include "cdrom.h" #include "dma.h" #include "interrupts.h" @@ -205,6 +206,159 @@ uint64_t psx_get_cycle_count(void) { return psx_cycle_count; } +/* ===== Idle-loop cycle skip (wait-loop elision, 2026-07-06) ================== + * + * A pure poll loop (Tomba2 main loop: `do {} while (vbl_count < target)`) + * burns its full guest-cycle budget at real emulation cost even though every + * iteration is a provable no-op: no stores, no MMIO, and register state at + * the loop's interrupt-check boundary identical each time. Nothing such a + * loop can observe changes except via device servicing, which happens at + * deterministic cycle deadlines. So time is fast-forwarded to the next + * INTERNAL device event (mask-blind min over vblank pacing / timers / cdrom / + * dma / sio — the full set of async guest-visible mutators) in whole loop + * quanta, bit-exact vs executing the iterations: + * - real execution takes a deliverable IRQ at the first check boundary at + * or after the event cycle; the skip lands on exactly that boundary + * (k = ceil(dist / quantum) whole iterations). + * - device RAM writes (DMA) replay at their exact cycles inside + * psx_advance_cycles' deadline-servicing path either way. + * - if the event does not release the loop, the detector re-arms and + * skips to the next internal event. + * + * Detection is evidence-based; ALL gates must hold across IDLE_STREAK_MIN + * consecutive interrupt checks at the SAME resume PC with a STABLE quantum: + * 1. zero guest stores (g_guest_store_count unchanged — covers RAM, + * scratchpad, MMIO and DMA writes; memory.c chokepoints) + * 2. zero MMIO accesses (g_mmio_access_count unchanged — an MMIO read can + * be side-effecting or time-varying, strictly disqualifying. I_STAT- + * polling kernel loops therefore never skip; whitelisting side-effect- + * free I_STAT reads is a possible later refinement.) + * 3. identical GPR/hi/lo fingerprint at the check boundary — rules out + * progressing pure-load loops (strlen/memcmp-style scans) whose exit + * depends on register progress, not time. (COP0/GTE-held loop state is + * not fingerprinted; no realistic poll loop keys its exit on those.) + * + * Excluded contexts: exception delivery, bail unwinds, precise slicing, + * lockstep record/replay, and PSX_COSIM builds entirely (the cosim oracle + * compares per-instruction streams; elided iterations would false-diverge). + * + * Off switch: PSX_IDLE_SKIP=0 (env) or {"cmd":"idle_skip","enable":0} (TCP). + * The always-on counters below are the observability surface. */ + +enum { + IDLE_STREAK_MIN = 4, /* stable same-PC windows before a skip */ + IDLE_QUANTUM_MAX = 32768, /* interp pump gap fits; junk deltas don't */ + IDLE_SKIP_MAX_CYCLES = 1200000 /* defensive cap (> one vblank interval) */ +}; + +int g_idle_skip_enabled = -1; /* -1 = read PSX_IDLE_SKIP once; TCP-togglable */ +uint64_t g_idle_skip_count = 0; /* skips performed */ +uint64_t g_idle_skip_cycles = 0; /* guest cycles fast-forwarded */ +uint32_t g_idle_skip_last_pc = 0; /* loop PC of the last skip */ +uint32_t g_idle_skip_last_quantum = 0; + +static uint32_t s_idle_pc = 0; +static uint32_t s_idle_quantum = 0; +static uint32_t s_idle_streak = 0; +static uint32_t s_idle_fp = 0; +static uint64_t s_idle_last_cycle = 0; +static uint64_t s_idle_last_stores = 0; +static uint64_t s_idle_last_mmio = 0; + +static uint32_t idle_reg_fp(const CPUState *cpu) { + uint32_t h = 0x811C9DC5u; /* FNV-1a over r1..r31 + hi/lo */ + for (int i = 1; i < 32; i++) { h ^= cpu->gpr[i]; h *= 16777619u; } + h ^= cpu->hi; h *= 16777619u; + h ^= cpu->lo; h *= 16777619u; + return h; +} + +static int idle_skip_on(void) { + if (g_idle_skip_enabled < 0) { + /* DISABLED BY DEFAULT (2026-07-06). The skip fast-forwards guest time + * to the next INTERNAL device event, but an SIO/memory-card transfer + * in flight is not currently modeled as a bounding deadline, so the + * skip jumps past the write-completion the card driver polls for at + * boot — Tomba2's "checking memory card" livelocks (frame rate + * collapses to a few fps; the game never advances). Opt in with + * PSX_IDLE_SKIP=1. Re-enable by default only once the skip is bounded + * by the SIO transfer/IRQ deadline (devices_cycles_to_next_internal_ + * event must include the in-flight card-byte completion). */ + const char *e = getenv("PSX_IDLE_SKIP"); + g_idle_skip_enabled = (e && e[0] == '1'); + } + return g_idle_skip_enabled; +} + +void psx_idle_note_check(CPUState *cpu, uint32_t check_pc) { +#ifdef PSX_COSIM + (void)cpu; (void)check_pc; + return; +#else + extern int g_psx_call_bail, g_precise_mode, g_ls_mode, g_ls_replay_active; + extern int psx_get_in_exception(void); + extern uint64_t g_guest_store_count, g_mmio_access_count; + + if (!idle_skip_on()) return; + if (check_pc == 0 || psx_get_in_exception() || g_psx_call_bail || + g_precise_mode || g_ls_mode != 0 || g_ls_replay_active) { + s_idle_pc = 0; + s_idle_streak = 0; + return; + } + + uint64_t cyc = psx_cycle_count; + if (check_pc == s_idle_pc && + g_guest_store_count == s_idle_last_stores && + g_mmio_access_count == s_idle_last_mmio) { + uint32_t fp = idle_reg_fp(cpu); + uint32_t delta = (uint32_t)(cyc - s_idle_last_cycle); + if (fp == s_idle_fp && delta > 0 && delta <= IDLE_QUANTUM_MAX) { + if (delta == s_idle_quantum) { + if (s_idle_streak < 1000000u) s_idle_streak++; + } else { + s_idle_quantum = delta; + s_idle_streak = 1; + } + } else { + s_idle_streak = 0; + s_idle_quantum = 0; + } + s_idle_fp = fp; + } else { + s_idle_pc = check_pc; + s_idle_streak = 0; + s_idle_quantum = 0; + s_idle_fp = idle_reg_fp(cpu); + } + s_idle_last_cycle = cyc; + s_idle_last_stores = g_guest_store_count; + s_idle_last_mmio = g_mmio_access_count; + + if (s_idle_streak < IDLE_STREAK_MIN) return; + uint32_t q = s_idle_quantum; + uint32_t dist = devices_cycles_to_next_internal_event(); + if (dist <= q) return; /* one real iteration reaches it */ + uint32_t k = (dist + q - 1u) / q; /* first check boundary >= event */ + uint64_t skip = (uint64_t)k * q; + if (skip > IDLE_SKIP_MAX_CYCLES) return; + + g_idle_skip_count++; + g_idle_skip_cycles += skip; + g_idle_skip_last_pc = check_pc; + g_idle_skip_last_quantum = q; + psx_advance_cycles((uint32_t)skip); /* replays device events exactly */ + + /* Device servicing may have written RAM / raised I_STAT — the caller + * (psx_check_interrupts) evaluates deliverability right after this with + * the post-skip state. Re-detect from scratch either way. */ + s_idle_streak = 0; + s_idle_last_cycle = psx_cycle_count; + s_idle_last_stores = g_guest_store_count; + s_idle_last_mmio = g_mmio_access_count; +#endif +} + /* Save-state restore: psx_cycle_count was just overwritten from the snapshot, so * the deadline-model bookkeeping (synced position + next deadline) is stale and * would try to replay a bogus gap. Re-anchor devices at the restored cycle and diff --git a/runtime/src/psx_keybinds.c b/runtime/src/psx_keybinds.c new file mode 100644 index 00000000..6124fe2a --- /dev/null +++ b/runtime/src/psx_keybinds.c @@ -0,0 +1,348 @@ +/* + * psx_keybinds.c — configurable keyboard -> DualShock keybinds, INI-driven. + * + * INI lives next to the exe as keybinds.ini. Auto-generated with the framework's + * historical default keyboard layout when missing, so behaviour is unchanged out + * of the box. Edit + restart (or rebind live in the launcher's Controls page) to + * apply. See psx_keybinds.h for the API contract and the PSX pad-word bit layout. + */ +#include "psx_keybinds.h" + +#include <stdio.h> +#include <string.h> +#include <ctype.h> +#include <stddef.h> + +/* PSX pad word bits (active-low), standard DualShock layout. Matches the + * PAD_* masks in main.cpp / beetle_main.cpp. */ +#define PSXKB_BIT_SELECT (1u << 0) +#define PSXKB_BIT_L3 (1u << 1) +#define PSXKB_BIT_R3 (1u << 2) +#define PSXKB_BIT_START (1u << 3) +#define PSXKB_BIT_UP (1u << 4) +#define PSXKB_BIT_RIGHT (1u << 5) +#define PSXKB_BIT_DOWN (1u << 6) +#define PSXKB_BIT_LEFT (1u << 7) +#define PSXKB_BIT_L2 (1u << 8) +#define PSXKB_BIT_R2 (1u << 9) +#define PSXKB_BIT_L1 (1u << 10) +#define PSXKB_BIT_R1 (1u << 11) +#define PSXKB_BIT_TRIANGLE (1u << 12) +#define PSXKB_BIT_CIRCLE (1u << 13) +#define PSXKB_BIT_CROSS (1u << 14) +#define PSXKB_BIT_SQUARE (1u << 15) + +/* ── Defaults ─────────────────────────────────────────────────────────────── */ +/* + * Player 1 reproduces the framework's historical hardcoded keyboard mapping + * (pad_from_keyboard / pad_sticks_for in main.cpp), so shipping keybinds.ini + * with defaults changes nothing until the user edits it: + * D-pad: Arrow keys Start: Return Select: Right Shift + * Cross: X Circle: S Square: Z Triangle: A + * L1: Q R1: W L2: E R2: R L3/R3: unbound + * Left analog stick: Arrow keys (matches the old keyboard analog path) + * Right analog stick: unbound (the keyboard never drove it before) + * Player 2 is fully unbound (add binds to enable a 2nd keyboard player). + */ +#define PSXKB_DEFAULTS { \ + .p1 = { \ + .up = SDL_SCANCODE_UP, .down = SDL_SCANCODE_DOWN, \ + .left = SDL_SCANCODE_LEFT, .right = SDL_SCANCODE_RIGHT, \ + .cross = SDL_SCANCODE_X, .circle = SDL_SCANCODE_S, \ + .square = SDL_SCANCODE_Z, .triangle = SDL_SCANCODE_A, \ + .l1 = SDL_SCANCODE_Q, .r1 = SDL_SCANCODE_W, \ + .l2 = SDL_SCANCODE_E, .r2 = SDL_SCANCODE_R, \ + .l3 = SDL_SCANCODE_UNKNOWN, .r3 = SDL_SCANCODE_UNKNOWN, \ + .start = SDL_SCANCODE_RETURN, .select = SDL_SCANCODE_RSHIFT, \ + .ls_up = SDL_SCANCODE_UP, .ls_down = SDL_SCANCODE_DOWN, \ + .ls_left = SDL_SCANCODE_LEFT, .ls_right = SDL_SCANCODE_RIGHT, \ + .rs_up = SDL_SCANCODE_UNKNOWN, .rs_down = SDL_SCANCODE_UNKNOWN, \ + .rs_left = SDL_SCANCODE_UNKNOWN, .rs_right = SDL_SCANCODE_UNKNOWN, \ + }, \ + .p2 = { \ + .up = SDL_SCANCODE_UNKNOWN, .down = SDL_SCANCODE_UNKNOWN, \ + .left = SDL_SCANCODE_UNKNOWN, .right = SDL_SCANCODE_UNKNOWN, \ + .cross = SDL_SCANCODE_UNKNOWN, .circle = SDL_SCANCODE_UNKNOWN, \ + .square = SDL_SCANCODE_UNKNOWN, .triangle = SDL_SCANCODE_UNKNOWN, \ + .l1 = SDL_SCANCODE_UNKNOWN, .r1 = SDL_SCANCODE_UNKNOWN, \ + .l2 = SDL_SCANCODE_UNKNOWN, .r2 = SDL_SCANCODE_UNKNOWN, \ + .l3 = SDL_SCANCODE_UNKNOWN, .r3 = SDL_SCANCODE_UNKNOWN, \ + .start = SDL_SCANCODE_UNKNOWN, .select = SDL_SCANCODE_UNKNOWN, \ + .ls_up = SDL_SCANCODE_UNKNOWN, .ls_down = SDL_SCANCODE_UNKNOWN, \ + .ls_left = SDL_SCANCODE_UNKNOWN, .ls_right = SDL_SCANCODE_UNKNOWN, \ + .rs_up = SDL_SCANCODE_UNKNOWN, .rs_down = SDL_SCANCODE_UNKNOWN, \ + .rs_left = SDL_SCANCODE_UNKNOWN, .rs_right = SDL_SCANCODE_UNKNOWN, \ + }, \ +} + +static PsxKeyBinds s_binds = PSXKB_DEFAULTS; +static const PsxKeyBinds s_default_binds = PSXKB_DEFAULTS; + +typedef struct { + const char *name; /* ini key */ + const char *label; /* pretty label for the launcher */ + size_t offset; /* offset into PsxPlayerBinds */ + uint16_t bit; /* PSX pad-word bit, 0 for non-button (stick) inputs */ +} ButtonDef; + +static const ButtonDef s_buttons[] = { + { "up", "Up", offsetof(PsxPlayerBinds, up), PSXKB_BIT_UP }, + { "down", "Down", offsetof(PsxPlayerBinds, down), PSXKB_BIT_DOWN }, + { "left", "Left", offsetof(PsxPlayerBinds, left), PSXKB_BIT_LEFT }, + { "right", "Right", offsetof(PsxPlayerBinds, right), PSXKB_BIT_RIGHT }, + { "cross", "Cross (X)", offsetof(PsxPlayerBinds, cross), PSXKB_BIT_CROSS }, + { "circle", "Circle (O)", offsetof(PsxPlayerBinds, circle), PSXKB_BIT_CIRCLE }, + { "square", "Square", offsetof(PsxPlayerBinds, square), PSXKB_BIT_SQUARE }, + { "triangle", "Triangle", offsetof(PsxPlayerBinds, triangle), PSXKB_BIT_TRIANGLE }, + { "l1", "L1", offsetof(PsxPlayerBinds, l1), PSXKB_BIT_L1 }, + { "r1", "R1", offsetof(PsxPlayerBinds, r1), PSXKB_BIT_R1 }, + { "l2", "L2", offsetof(PsxPlayerBinds, l2), PSXKB_BIT_L2 }, + { "r2", "R2", offsetof(PsxPlayerBinds, r2), PSXKB_BIT_R2 }, + { "l3", "L3 (stick)", offsetof(PsxPlayerBinds, l3), PSXKB_BIT_L3 }, + { "r3", "R3 (stick)", offsetof(PsxPlayerBinds, r3), PSXKB_BIT_R3 }, + { "start", "Start", offsetof(PsxPlayerBinds, start), PSXKB_BIT_START }, + { "select", "Select", offsetof(PsxPlayerBinds, select), PSXKB_BIT_SELECT }, + { "ls_up", "L-Stick Up", offsetof(PsxPlayerBinds, ls_up), 0 }, + { "ls_down", "L-Stick Down", offsetof(PsxPlayerBinds, ls_down), 0 }, + { "ls_left", "L-Stick Left", offsetof(PsxPlayerBinds, ls_left), 0 }, + { "ls_right", "L-Stick Right", offsetof(PsxPlayerBinds, ls_right), 0 }, + { "rs_up", "R-Stick Up", offsetof(PsxPlayerBinds, rs_up), 0 }, + { "rs_down", "R-Stick Down", offsetof(PsxPlayerBinds, rs_down), 0 }, + { "rs_left", "R-Stick Left", offsetof(PsxPlayerBinds, rs_left), 0 }, + { "rs_right", "R-Stick Right", offsetof(PsxPlayerBinds, rs_right), 0 }, +}; +#define PSXKB_N ((int)(sizeof(s_buttons) / sizeof(s_buttons[0]))) + +/* ── INI parsing helpers ──────────────────────────────────────────────────── */ + +static void trim(char *s) { + size_t n = strlen(s); + while (n > 0 && isspace((unsigned char)s[n-1])) s[--n] = '\0'; + char *start = s; + while (*start && isspace((unsigned char)*start)) start++; + if (start != s) memmove(s, start, strlen(start) + 1); +} + +static SDL_Scancode name_to_scancode(const char *name) { + if (!name || !*name) return SDL_SCANCODE_UNKNOWN; + SDL_Scancode sc = SDL_GetScancodeFromName(name); + if (sc != SDL_SCANCODE_UNKNOWN) return sc; + char buf[32]; + size_t i = 0; + for (; name[i] && i < sizeof(buf) - 1; i++) buf[i] = (char)tolower((unsigned char)name[i]); + buf[i] = '\0'; + if (!strcmp(buf, "enter") || !strcmp(buf, "return")) return SDL_SCANCODE_RETURN; + if (!strcmp(buf, "tab")) return SDL_SCANCODE_TAB; + if (!strcmp(buf, "space")) return SDL_SCANCODE_SPACE; + if (!strcmp(buf, "lshift")) return SDL_SCANCODE_LSHIFT; + if (!strcmp(buf, "rshift")) return SDL_SCANCODE_RSHIFT; + if (!strcmp(buf, "lctrl")) return SDL_SCANCODE_LCTRL; + if (!strcmp(buf, "rctrl")) return SDL_SCANCODE_RCTRL; + if (!strcmp(buf, "lalt")) return SDL_SCANCODE_LALT; + if (!strcmp(buf, "ralt")) return SDL_SCANCODE_RALT; + if (!strcmp(buf, "backslash")) return SDL_SCANCODE_BACKSLASH; + if (!strcmp(buf, "escape") || !strcmp(buf, "esc")) return SDL_SCANCODE_ESCAPE; + if (!strcmp(buf, "backspace")) return SDL_SCANCODE_BACKSPACE; + if (!strcmp(buf, "none") || !strcmp(buf, "")) return SDL_SCANCODE_UNKNOWN; + return SDL_SCANCODE_UNKNOWN; +} + +static const char *scancode_to_name(SDL_Scancode sc) { + if (sc == SDL_SCANCODE_UNKNOWN) return "None"; + const char *name = SDL_GetScancodeName(sc); + return (name && name[0]) ? name : "None"; +} + +/* ── File I/O ─────────────────────────────────────────────────────────────── */ + +static char s_ini_path[1024] = {0}; + +/* Resolve keybinds.ini alongside exe_path. exe_path may be a file (argv[0] / + * the exe) or a directory (the exe dir the launcher passes) — either works. */ +static void derive_ini_path(const char *exe_path) { + if (!exe_path || !*exe_path) { + strcpy(s_ini_path, "keybinds.ini"); + return; + } + /* Find the last path separator; if none, treat the whole thing as a dir. */ + const char *slash = NULL; + for (const char *p = exe_path; *p; p++) + if (*p == '/' || *p == '\\') slash = p; + + /* Heuristic: a path ending in a separator, or with no extension in the last + * component, is treated as a directory; otherwise strip the file name. */ + size_t len = strlen(exe_path); + int ends_sep = (exe_path[len-1] == '/' || exe_path[len-1] == '\\'); + const char *last = slash ? slash + 1 : exe_path; + int has_ext = strchr(last, '.') != NULL; + + char dir[1024]; + if (ends_sep) { + snprintf(dir, sizeof(dir), "%s", exe_path); + } else if (!has_ext) { + /* directory path without trailing sep */ + snprintf(dir, sizeof(dir), "%s/", exe_path); + } else if (slash) { + size_t dl = (size_t)(slash - exe_path) + 1; + if (dl >= sizeof(dir)) dl = sizeof(dir) - 1; + memcpy(dir, exe_path, dl); + dir[dl] = '\0'; + } else { + dir[0] = '\0'; + } + snprintf(s_ini_path, sizeof(s_ini_path), "%skeybinds.ini", dir); +} + +static void write_player_section(FILE *f, const char *section, const PsxPlayerBinds *pb) { + fprintf(f, "[%s]\n", section); + for (int i = 0; i < PSXKB_N; i++) { + SDL_Scancode sc = *(const SDL_Scancode *)((const char *)pb + s_buttons[i].offset); + fprintf(f, "%-9s = %s\n", s_buttons[i].name, scancode_to_name(sc)); + } + fprintf(f, "\n"); +} + +static void write_ini(const char *path) { + FILE *f = fopen(path, "w"); + if (!f) return; + fprintf(f, + "# PSXRecomp Keyboard Keybinds (keyboard -> DualShock).\n" + "# Edit values and restart, or rebind live in the launcher's Controls page.\n" + "# Use SDL key names. Common: A B C ... Z, 0-9, F1-F12, Up Down Left Right,\n" + "# Return, Tab, Space, Left Shift, Right Shift, Left Ctrl, Right Ctrl,\n" + "# Backspace, Escape, Backslash. Use \"None\" to leave an input unbound.\n" + "#\n" + "# Buttons: up/down/left/right, cross/circle/square/triangle, l1/r1/l2/r2,\n" + "# l3/r3 (stick clicks), start/select. ls_* / rs_* are the left/right\n" + "# analog-stick DIRECTIONS driven from the keyboard (analog pad modes).\n" + "#\n" + "# Player 2 is unbound by default — fill in keys to enable a second\n" + "# keyboard player (route a port to \"Keyboard\" in the launcher).\n" + "\n"); + write_player_section(f, "player1", &s_binds.p1); + write_player_section(f, "player2", &s_binds.p2); + fclose(f); + printf("[Keybinds] Wrote %s\n", path); +} + +static void load_ini(const char *path) { + FILE *f = fopen(path, "r"); + if (!f) return; + PsxPlayerBinds *current = NULL; + char line[256]; + while (fgets(line, sizeof(line), f)) { + trim(line); + if (!line[0] || line[0] == '#' || line[0] == ';') continue; + if (line[0] == '[') { + char *end = strchr(line, ']'); + if (end) *end = '\0'; + const char *section = line + 1; + current = NULL; + if (!strcmp(section, "player1")) current = &s_binds.p1; + else if (!strcmp(section, "player2")) current = &s_binds.p2; + continue; + } + char *eq = strchr(line, '='); + if (!eq) continue; + *eq = '\0'; + char *key = line, *val = eq + 1; + trim(key); trim(val); + for (char *c = key; *c; c++) *c = (char)tolower((unsigned char)*c); + if (!current) continue; + for (int i = 0; i < PSXKB_N; i++) { + if (!strcmp(key, s_buttons[i].name)) { + *(SDL_Scancode *)((char *)current + s_buttons[i].offset) = name_to_scancode(val); + break; + } + } + } + fclose(f); + printf("[Keybinds] Loaded %s\n", path); +} + +/* ── Public API ───────────────────────────────────────────────────────────── */ + +void psx_keybinds_init(const char *exe_path) { + derive_ini_path(exe_path); + FILE *test = fopen(s_ini_path, "r"); + if (test) { fclose(test); load_ini(s_ini_path); } + else write_ini(s_ini_path); +} + +const PsxKeyBinds *psx_keybinds_get(void) { return &s_binds; } + +static const PsxPlayerBinds *player_binds_c(int player) { + return (player == 2) ? &s_binds.p2 : &s_binds.p1; +} +static PsxPlayerBinds *player_binds(int player) { + return (player == 2) ? &s_binds.p2 : &s_binds.p1; +} + +/* Is the scancode at button-def index i currently held for this player? */ +static int held(const uint8_t *keys, const PsxPlayerBinds *pb, int i) { + SDL_Scancode sc = *(const SDL_Scancode *)((const char *)pb + s_buttons[i].offset); + return sc != SDL_SCANCODE_UNKNOWN && keys[sc]; +} + +uint16_t psx_keybinds_pad_word(const uint8_t *keys, int player) { + if (!keys) return 0xFFFF; + const PsxPlayerBinds *pb = player_binds_c(player); + uint16_t b = 0xFFFF; /* active-low: all released */ + for (int i = 0; i < PSXKB_N; i++) { + if (s_buttons[i].bit && held(keys, pb, i)) + b &= (uint16_t)~s_buttons[i].bit; + } + return b; +} + +void psx_keybinds_sticks(const uint8_t *keys, int player, uint8_t out[4]) { + if (!keys || !out) return; + const PsxPlayerBinds *pb = player_binds_c(player); + if (held(keys, pb, PSX_KB_LS_LEFT)) out[0] = 0x00; + if (held(keys, pb, PSX_KB_LS_RIGHT)) out[0] = 0xFF; + if (held(keys, pb, PSX_KB_LS_UP)) out[1] = 0x00; + if (held(keys, pb, PSX_KB_LS_DOWN)) out[1] = 0xFF; + if (held(keys, pb, PSX_KB_RS_LEFT)) out[2] = 0x00; + if (held(keys, pb, PSX_KB_RS_RIGHT)) out[2] = 0xFF; + if (held(keys, pb, PSX_KB_RS_UP)) out[3] = 0x00; + if (held(keys, pb, PSX_KB_RS_DOWN)) out[3] = 0xFF; +} + +int psx_keybinds_dpad_active(const uint8_t *keys, int player) { + if (!keys) return 0; + const PsxPlayerBinds *pb = player_binds_c(player); + return held(keys, pb, PSX_KB_UP) || held(keys, pb, PSX_KB_DOWN) || + held(keys, pb, PSX_KB_LEFT) || held(keys, pb, PSX_KB_RIGHT); +} + +/* ── Rebind API ───────────────────────────────────────────────────────────── */ + +int psx_keybinds_button_count(void) { return PSXKB_N; } + +const char *psx_keybinds_button_name(int button) { + if (button < 0 || button >= PSXKB_N) return "?"; + return s_buttons[button].name; +} +const char *psx_keybinds_button_label(int button) { + if (button < 0 || button >= PSXKB_N) return "?"; + return s_buttons[button].label; +} + +SDL_Scancode psx_keybinds_get_button(int player, int button) { + if (button < 0 || button >= PSXKB_N) return SDL_SCANCODE_UNKNOWN; + return *(SDL_Scancode *)((char *)player_binds(player) + s_buttons[button].offset); +} + +void psx_keybinds_set_button(int player, int button, SDL_Scancode sc) { + if (button < 0 || button >= PSXKB_N) return; + *(SDL_Scancode *)((char *)player_binds(player) + s_buttons[button].offset) = sc; +} + +void psx_keybinds_reset_player(int player) { + *player_binds(player) = (player == 2) ? s_default_binds.p2 : s_default_binds.p1; +} + +void psx_keybinds_save(void) { + if (!s_ini_path[0]) strcpy(s_ini_path, "keybinds.ini"); + write_ini(s_ini_path); +} diff --git a/runtime/src/recomp_audio_drc.h b/runtime/src/recomp_audio_drc.h new file mode 100644 index 00000000..ec758ba3 --- /dev/null +++ b/runtime/src/recomp_audio_drc.h @@ -0,0 +1,362 @@ +/* recomp_audio_drc.h - shared audio clock-domain bridge for the *recomp ecosystems. + * + * One persistent band-limited polyphase windowed-sinc fractional resampler driven + * by a P-only normalized-fill controller, fed by a millisecond-measured SPSC ring, + * with smooth fade-based startup / underrun / overflow handling. + * + * It exists because every runner (NES/SNES/GBA/N64/Genesis) is video-master / + * audio-slaved: guest audio is produced ~1 frame per wall-clock frame while the + * host device consumes at its own crystal rate. The unsynchronized clocks drift, + * and the historical "fix" was to discard samples (drop/flush/decimate or a + * nearest-neighbor servo) -- every discard is a click, and continuous micro + * discard/dup is a sustained crackle. This bridge replaces all of that: the + * controller nudges the *resample ratio* by <= +/-0.5% to hold the ring near a + * target fill, and the band-limited resampler keeps the waveform continuous. + * + * Single-file. Define RECOMP_AUDIO_DRC_IMPL in exactly ONE translation unit. + * C99 and C++ compatible. Depends only on the C stdlib + libm. + * + * Threading: rab_push() is called by the producer (emulation thread), rab_pull() + * by the consumer (host audio callback). One producer, one consumer. The shared + * state touched across threads (in_count / out_pos / ring) is accessed in a + * single-producer/single-consumer pattern; integate with whatever memory ordering + * the host runner already uses for its existing queue (a mutex around push/pull is + * fine and is what most of these runners already have). + */ +#ifndef RECOMP_AUDIO_DRC_H +#define RECOMP_AUDIO_DRC_H + +#include <stdint.h> +#include <stddef.h> + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct { + int channels; /* 1 or 2 */ + double source_rate; /* nominal guest source sample rate (Hz) */ + double host_rate; /* host device sample rate (Hz) */ + int taps; /* prototype length; 16 min, 32 preferred */ + int phases; /* fractional phases; 256-1024 (512 default) */ + double target_ms; /* steady-state ring fill target (default 80) */ + double ring_ms; /* ring capacity in ms (default 250) */ + double kp; /* proportional gain (default 0.02) */ + double ki_per_s; /* integral gain per second (default 0.02): holds + a steady clock-skew correction with ZERO + standing fill error (P-only must run a permanent + fill deficit of corr/kp to sustain correction) */ + double max_correction; /* clamp on ratio correction (default 0.005) */ + double err_lp_ms; /* error low-pass time constant ms (default 75) */ + double slew_pp_per_s; /* correction slew, percent-points/sec (default 0.75) */ + double deadband_ms; /* +/- band around target with no correction (1.0) */ + double em_low_ms; /* below this = underrun emergency (default 12) */ + double em_high_ms; /* above this = overflow emergency (default 105) */ +} rab_config; + +typedef struct { + uint64_t pushed_frames; + uint64_t pulled_frames; + uint64_t underrun_events; /* times the ring ran dry mid-pull */ + uint64_t overflow_drops; /* source frames dropped on ring overflow */ + double last_fill_ms; + double last_correction; /* applied ratio correction (signed) */ +} rab_stats; + +typedef struct rab_bridge { + rab_config cfg; + int half; /* taps/2 */ + float *coeffs; /* phases*taps, DC-normalized per phase */ + float *ring; /* cap_frames*channels, interleaved float */ + int64_t cap; /* ring capacity in frames */ + + /* SPSC stream position */ + uint64_t in_count; /* frames ever pushed (monotonic) */ + double out_pos; /* fractional source-frame read cursor */ + + /* controller */ + double cur_step; /* current source frames consumed per output frame */ + double err_lp; /* low-passed normalized error */ + double corr; /* current (slew-limited) correction */ + double corr_i; /* integral term: holds steady clock-skew with + zero standing fill error (see controller) */ + + /* fade / emergency */ + double gain; /* 0..1 smooth gate for startup/underrun */ + int primed; /* set once fill first reaches target */ + float last_out[2]; /* last emitted sample per channel (for holds) */ + + rab_stats stats; +} rab_bridge; + +/* Fill a config with the locked-in ecosystem defaults. You must still set + * channels, source_rate and host_rate afterwards. */ +void rab_config_defaults(rab_config *c); + +/* Allocate internal buffers and build the polyphase filter bank. Returns 0 on + * success, non-zero on allocation/parameter failure. */ +int rab_init(rab_bridge *b, const rab_config *cfg); +void rab_free(rab_bridge *b); + +/* Producer: append `frames` interleaved int16 source samples. Thread: emu side. */ +void rab_push(rab_bridge *b, const int16_t *interleaved, int frames); + +/* Consumer: render exactly `frames` interleaved int16 host samples (never + * blocks; emits faded silence on underrun / before prime). Thread: callback. */ +void rab_pull(rab_bridge *b, int16_t *out, int frames); + +/* Current ring fill expressed in milliseconds of source audio. */ +double rab_fill_ms(const rab_bridge *b); + +void rab_get_stats(const rab_bridge *b, rab_stats *out); + +#ifdef __cplusplus +} +#endif + +/* ---------------------------------------------------------------------------- */ +#ifdef RECOMP_AUDIO_DRC_IMPL + +#include <stdlib.h> +#include <string.h> +#include <math.h> + +#ifndef RAB_PI +#define RAB_PI 3.14159265358979323846 +#endif + +static double rab__sinc(double x) { + if (x > -1e-9 && x < 1e-9) return 1.0; + double p = RAB_PI * x; + return sin(p) / p; +} + +/* Blackman window over [-1,1] mapped from t/half. */ +static double rab__blackman(double tn) { /* tn in [-1,1] */ + double a = RAB_PI * (tn + 1.0); /* 0..2pi */ + return 0.42 - 0.5 * cos(a) + 0.08 * cos(2.0 * a); +} + +void rab_config_defaults(rab_config *c) { + c->channels = 2; + c->source_rate = 48000.0; + c->host_rate = 48000.0; + c->taps = 32; + c->phases = 512; + /* target/ring raised 50/150 -> 80/250: a producer paced by an emulated + * machine stalls for tens of ms at a time (heavy frames, CD bursts) — + * jitter far beyond a host audio driver's. 80 ms keeps the servo's + * operating point clear of the dry floor through those spikes. */ + c->target_ms = 80.0; + c->ring_ms = 250.0; + c->kp = 0.02; + c->ki_per_s = 0.02; + c->max_correction = 0.005; + c->err_lp_ms = 75.0; + c->slew_pp_per_s = 0.75; + c->deadband_ms = 1.0; + c->em_low_ms = 12.0; + c->em_high_ms = 205.0; +} + +int rab_init(rab_bridge *b, const rab_config *cfg) { + if (!b || !cfg) return 1; + if (cfg->channels < 1 || cfg->channels > 2) return 2; + if (cfg->taps < 4 || (cfg->taps & 1)) return 3; + if (cfg->phases < 16) return 4; + if (cfg->source_rate <= 0.0 || cfg->host_rate <= 0.0) return 5; + + memset(b, 0, sizeof(*b)); + b->cfg = *cfg; + b->half = cfg->taps / 2; + + /* Prototype lowpass cutoff (normalized to source Nyquist). When downsampling + * (host < source) we must pull the cutoff in to suppress aliases; when + * upsampling we keep the full source band. A small guard avoids the corner. */ + double fc = (cfg->host_rate < cfg->source_rate) + ? (cfg->host_rate / cfg->source_rate) : 1.0; + fc *= 0.92; + + size_t ncoef = (size_t)cfg->phases * (size_t)cfg->taps; + b->coeffs = (float *)malloc(ncoef * sizeof(float)); + if (!b->coeffs) return 6; + + for (int p = 0; p < cfg->phases; ++p) { + double frac = (double)p / (double)cfg->phases; /* [0,1) sub-sample delay */ + double sum = 0.0; + float *row = b->coeffs + (size_t)p * cfg->taps; + for (int k = 0; k < cfg->taps; ++k) { + /* input tap index relative to the interpolation point */ + double t = (double)(k - (b->half - 1)) - frac; + double tn = t / (double)b->half; /* window arg in [-~1,~1] */ + double w = (tn <= -1.0 || tn >= 1.0) ? 0.0 : rab__blackman(tn); + double h = fc * rab__sinc(fc * t) * w; + row[k] = (float)h; + sum += h; + } + /* normalize each phase to unit DC gain to kill level ripple / zipper */ + if (sum > 1e-12) { + float inv = (float)(1.0 / sum); + for (int k = 0; k < cfg->taps; ++k) row[k] *= inv; + } + } + + double cap_ms = cfg->ring_ms; + if (cap_ms < cfg->target_ms + 40.0) cap_ms = cfg->target_ms + 40.0; + b->cap = (int64_t)(cfg->source_rate * cap_ms / 1000.0) + cfg->taps + 2; + b->ring = (float *)calloc((size_t)b->cap * cfg->channels, sizeof(float)); + if (!b->ring) { free(b->coeffs); b->coeffs = NULL; return 7; } + + b->in_count = 0; + /* Start the read cursor half a window in so the polyphase filter always has + * valid history behind it; otherwise the first half-1 outputs can't be + * computed and the cursor would never advance. */ + b->out_pos = (double)(b->half - 1); + b->cur_step = cfg->source_rate / cfg->host_rate; + b->err_lp = 0.0; + b->corr = 0.0; + b->gain = 0.0; + b->primed = 0; + return 0; +} + +void rab_free(rab_bridge *b) { + if (!b) return; + free(b->coeffs); b->coeffs = NULL; + free(b->ring); b->ring = NULL; +} + +double rab_fill_ms(const rab_bridge *b) { + double fill = (double)b->in_count - b->out_pos; + if (fill < 0.0) fill = 0.0; + return fill * 1000.0 / b->cfg.source_rate; +} + +void rab_push(rab_bridge *b, const int16_t *in, int frames) { + int ch = b->cfg.channels; + for (int f = 0; f < frames; ++f) { + double fill = (double)b->in_count - b->out_pos; + if (fill >= (double)(b->cap - 1)) { + /* Overflow emergency: producer outran consumer past the ring. Drop the + * oldest source frame by nudging the read cursor forward. Rare under + * DRC; the resampler's continuous history limits the audible seam. */ + b->out_pos += 1.0; + b->stats.overflow_drops++; + } + int64_t w = (int64_t)(b->in_count % (uint64_t)b->cap); + float *slot = b->ring + w * ch; + for (int c = 0; c < ch; ++c) + slot[c] = (float)in[f * ch + c] * (1.0f / 32768.0f); + b->in_count++; + } + b->stats.pushed_frames += (uint64_t)frames; +} + +static void rab__update_controller(rab_bridge *b) { + rab_config *c = &b->cfg; + double fill_ms = rab_fill_ms(b); + b->stats.last_fill_ms = fill_ms; + + if (!b->primed && fill_ms >= c->target_ms) b->primed = 1; + + /* one control update per pull; low-pass the normalized error. Use the pull + * block duration as dt (approx via host_rate is unnecessary here -- we fold + * the time constant into a per-update smoothing alpha sized for ~10-20ms). */ + double err = (fill_ms - c->target_ms) / c->target_ms; + if (fill_ms > c->target_ms - c->deadband_ms && + fill_ms < c->target_ms + c->deadband_ms) { + err = 0.0; /* deadband */ + } + /* smoothing alpha: assume ~one update per host audio block (~10ms typical). */ + double dt_ms = 12.0; + double alpha = dt_ms / (c->err_lp_ms + dt_ms); + b->err_lp += alpha * (err - b->err_lp); + + /* PI control. P alone cannot hold a sustained source-clock skew without a + * standing fill error of corr/kp (a 0.3% skew cost 7.5 ms of permanent + * fill deficit at kp=0.02/target=50 — margin that jitter spikes then + * drained to underrun). The integral accumulates the steady skew so fill + * returns to target; anti-windup: never integrate deeper into the clamp. */ + double target_corr = c->kp * b->err_lp + b->corr_i; + int clamped_hi = target_corr > c->max_correction; + int clamped_lo = target_corr < -c->max_correction; + if ((!clamped_hi && !clamped_lo) || + (clamped_hi && b->err_lp < 0.0) || (clamped_lo && b->err_lp > 0.0)) { + b->corr_i += c->ki_per_s * (dt_ms / 1000.0) * b->err_lp; + if (b->corr_i > c->max_correction) b->corr_i = c->max_correction; + if (b->corr_i < -c->max_correction) b->corr_i = -c->max_correction; + } + if (target_corr > c->max_correction) target_corr = c->max_correction; + if (target_corr < -c->max_correction) target_corr = -c->max_correction; + + /* slew limit (percentage points per second -> fraction per update) */ + double slew = (c->slew_pp_per_s / 100.0) * (dt_ms / 1000.0); + double d = target_corr - b->corr; + if (d > slew) d = slew; + if (d < -slew) d = -slew; + b->corr += d; + + /* A high queue (positive error) must be consumed FASTER -> larger step. */ + b->cur_step = (c->source_rate / c->host_rate) * (1.0 + b->corr); + b->stats.last_correction = b->corr; +} + +void rab_pull(rab_bridge *b, int16_t *out, int frames) { + int ch = b->cfg.channels; + rab__update_controller(b); + + double gstep = 1.0 / (b->cfg.host_rate * 0.003); /* ~3ms fade in/out */ + double step = b->cur_step; + + for (int f = 0; f < frames; ++f) { + int64_t i = (int64_t)floor(b->out_pos); + double fr = b->out_pos - (double)i; + int64_t newest_needed = i + b->half; + int have = b->primed && (newest_needed < (int64_t)b->in_count) + && ((i - b->half + 1) >= 0); + + float s[2] = {0.0f, 0.0f}; + if (have) { + int phase = (int)(fr * b->cfg.phases); + if (phase >= b->cfg.phases) phase = b->cfg.phases - 1; + const float *cz = b->coeffs + (size_t)phase * b->cfg.taps; + for (int k = 0; k < b->cfg.taps; ++k) { + int64_t m = i - b->half + 1 + k; + const float *sl = b->ring + (m % b->cap) * ch; + float w = cz[k]; + s[0] += w * sl[0]; + if (ch == 2) s[1] += w * sl[1]; + } + b->last_out[0] = s[0]; + if (ch == 2) b->last_out[1] = s[1]; + b->out_pos += step; /* advance only when we consumed */ + } else { + /* underrun / not yet primed: hold last sample, let gain fade it out */ + s[0] = b->last_out[0]; + if (ch == 2) s[1] = b->last_out[1]; + if (b->primed && newest_needed >= (int64_t)b->in_count) + b->stats.underrun_events++; + } + + /* smooth gate: fade toward 1 when delivering real audio, toward 0 else */ + double target_gain = have ? 1.0 : 0.0; + if (b->gain < target_gain) { + b->gain += gstep; if (b->gain > target_gain) b->gain = target_gain; + } else if (b->gain > target_gain) { + b->gain -= gstep; if (b->gain < target_gain) b->gain = target_gain; + } + + for (int c = 0; c < ch; ++c) { + double v = (double)s[c] * b->gain * 32768.0; + if (v > 32767.0) v = 32767.0; + if (v < -32768.0) v = -32768.0; + out[f * ch + c] = (int16_t)lrint(v); + } + } + b->stats.pulled_frames += (uint64_t)frames; +} + +void rab_get_stats(const rab_bridge *b, rab_stats *out) { *out = b->stats; } + +#endif /* RECOMP_AUDIO_DRC_IMPL */ +#endif /* RECOMP_AUDIO_DRC_H */ diff --git a/runtime/src/sio.c b/runtime/src/sio.c index 670938a6..2df0f109 100644 --- a/runtime/src/sio.c +++ b/runtime/src/sio.c @@ -127,6 +127,7 @@ typedef enum { MC_READ_CHK, MC_READ_END, /* Write states */ + MC_WRITE_LSB_ECHO, /* rx = sector_lsb while accepting first data byte */ MC_WRITE_DATA, MC_WRITE_CHK, MC_WRITE_ACK1, @@ -1066,11 +1067,17 @@ static void mc_process_byte(uint8_t tx_byte) { for (int i = 0; i < 128; i++) mc_checksum ^= mc_data[i]; mc_state = MC_READ_ACK1; + sio_rx_data = 0x00; } else { mc_data_idx = 0; - mc_state = MC_WRITE_DATA; + mc_state = MC_WRITE_LSB_ECHO; + /* Hardware/Beetle echo the high address byte on the address-LSB + * transfer (zero for all 1Mbit card sectors), then echo the low + * address byte while receiving the first payload byte. Some BIOS + * card-write paths validate this handshake before they schedule the + * follow-up directory reads. */ + sio_rx_data = mc_sector_msb; } - sio_rx_data = 0x00; sio_stat |= SIO_STAT_ACK; break; @@ -1135,6 +1142,13 @@ static void mc_process_byte(uint8_t tx_byte) { } /* ---- WRITE states ---- */ + case MC_WRITE_LSB_ECHO: + mc_data[mc_data_idx++] = tx_byte; + mc_state = MC_WRITE_DATA; + sio_rx_data = mc_sector_lsb; + sio_stat |= SIO_STAT_ACK; + break; + case MC_WRITE_DATA: mc_data[mc_data_idx++] = tx_byte; sio_rx_data = 0x00; @@ -1313,7 +1327,8 @@ static void sio_process_byte(uint8_t tx_byte) { /* ---- Card transaction tracking: record + maybe-close ---- */ if (txn_was_card_byte && sio_txn_open) { int got_ack = (sio_mc_ack_count > trace_ack_before) ? 1 : 0; - uint16_t sector_now = (mc_state >= MC_READ_ACK1 || mc_state == MC_WRITE_DATA + uint16_t sector_now = (mc_state >= MC_READ_ACK1 || mc_state == MC_WRITE_LSB_ECHO + || mc_state == MC_WRITE_DATA || mc_state >= MC_WRITE_ACK1) ? mc_sector : 0xFFFF; txn_record_byte(tx_byte, sio_rx_data, mc_cmd, sector_now, diff --git a/runtime/src/spu.c b/runtime/src/spu.c index 3f4a2386..711048b3 100644 --- a/runtime/src/spu.c +++ b/runtime/src/spu.c @@ -9,6 +9,7 @@ #include "spu.h" #include "spu_shadow.h" +#include "audio_trace.h" #include <string.h> @@ -284,6 +285,10 @@ void spu_cd_audio_push(const int16_t* stereo, int frames) { } cd_frame_count += in_frames; cd_push_frames += in_frames; + + /* T2 tap: what the CD/XA decoder feeds the SPU CD input bus. */ + audio_trace_pcm(AUDIO_TAP_CD_IN, stereo, (int)in_frames); + audio_trace_event(AUDIO_EV_CD_PUSH, in_frames, cd_frame_count); } static int cd_audio_pop(int16_t* left, int16_t* right) { @@ -327,7 +332,15 @@ static void decode_block(SpuVoice *v) { } } - if (flags & 0x04u) v->repeat_addr = addr; + if (flags & 0x04u) { + v->repeat_addr = addr; + /* The auto-latch updates the guest-visible register too (Beetle + * spu.cpp:386 + read path 1302): drivers read it back to save + * loop points. */ + int v_idx = (int)(v - voices); + if (v_idx >= 0 && v_idx < SPU_VOICE_COUNT) + spu_regs[(uint32_t)v_idx * 8u + 7u] = (uint16_t)(addr >> 3); + } v->flags = flags; v->sample_idx = 0; v->cur_addr = (addr + 16u) & (SPU_RAM_SIZE - 1u); @@ -558,6 +571,13 @@ void spu_render(int16_t* out_stereo, int frames) { mix_l += ((int32_t)s * vl) >> 14; mix_r += ((int32_t)s * vr) >> 14; } + { /* T0 tap: voice sum only (pre CD mix, pre main volume) so the + * final mix can be decomposed source-by-source offline. */ + int16_t vsum[2]; + vsum[0] = clamp16(mix_l); + vsum[1] = clamp16(mix_r); + audio_trace_pcm(AUDIO_TAP_VOICES, vsum, 1); + } if (ctrl & 0x0001u) { int16_t cd_l = 0; int16_t cd_r = 0; @@ -596,6 +616,11 @@ void spu_render(int16_t* out_stereo, int frames) { * while proven. No-op (byte-identical) when disabled. The canon mix above * stays the authoritative output AND the verify oracle. */ spu_shadow_process(out_stereo, frames); + + /* T1 tap: the SPU's final output block as handed to the host layer + * (post-shadow, pre host fade/mute). Placed here so every spu_render + * caller — the vblank pump and the turbo fade tail — is covered. */ + audio_trace_pcm(AUDIO_TAP_SPU_OUT, out_stereo, frames); } void spu_debug_info(SpuDebugInfo* out) { @@ -660,8 +685,28 @@ void spu_write(uint32_t addr, uint32_t value) { if (addr >= 0x1F801C00u && addr <= 0x1F801DFFu) { uint32_t idx = reg_index(addr); if (idx < SPU_REG_COUNT) { + /* Event-ring every register store with its SPU_OUT sample-clock + * timestamp: this is what lets offline analysis correlate a + * write (key-on, pitch, volume) with the exact output sample + * the render loop first honored it at — the write-to-render + * quantization audit (snesrecomp 8ffc797 class). */ + audio_trace_event(AUDIO_EV_REG_WRITE, addr, value & 0xFFFFu); spu_regs[idx] = (uint16_t)value; + /* Voice repeat/loop address (voice reg 7) is LIVE state on real + * hardware: writing it after KEYON retargets where the next + * END+REPEAT block jumps (Beetle spu.cpp:1150/333). X5's driver + * uses exactly this to end one-shots on looped samples — KEYON, + * then point the loop register at a silent tail block; no KEYOFF + * is ever sent. Caching repeat_addr only at KEYON left voices + * looping the loud sample body forever (measured: 3 stuck voices + * re-looping ~1400x/s, +11 dB over the oracle, clipping). */ + if (idx < (uint32_t)SPU_VOICE_COUNT * 8u && (idx & 7u) == 7u) { + int v = (int)(idx >> 3); + voices[v].repeat_addr = + ((uint32_t)(uint16_t)value << 3) & (SPU_RAM_SIZE - 1u); + } + if (addr == 0x1F801D88u) { kon_latch = (kon_latch & 0xFFFF0000u) | (uint32_t)(uint16_t)value; key_on((uint32_t)(uint16_t)value); diff --git a/runtime/src/text_xlate.cpp b/runtime/src/text_xlate.cpp new file mode 100644 index 00000000..562630ee --- /dev/null +++ b/runtime/src/text_xlate.cpp @@ -0,0 +1,1021 @@ +/* text_xlate.cpp — on-the-fly string translation / localization (framework). + * See text_xlate.h + docs/STRING_TRANSLATION.md. */ + +#include "text_xlate.h" +#include "cpu_state.h" + +#include <cstdint> +#include <cstring> +#include <cstdio> +#include <cstdlib> +#include <string> +#include <vector> +#include <unordered_map> +#include <unordered_set> +#include <mutex> +#include <atomic> +#include <filesystem> + +#include "toml.hpp" + +extern "C" uint8_t* memory_get_ram_ptr(void); +extern "C" { extern uint64_t s_frame_count; } +/* Renderer VRAM facade (gpu_render.h) — the same read/write path the GP0 0xA0 + * upload uses, so vram-strip patches stay coherent across software/GL backends + * and the supersampling mirror. */ +extern "C" uint16_t gr_vram_read(int x, int y); +extern "C" void gr_vram_write(int x, int y, uint16_t pixel); +// Bless an in-place RAM patch into the text-image guard so it isn't mistaken for +// self-modifying code (memory.c). Our patches (string/glyph tables) share pages +// with real compiled functions; without this the guard blocks their native +// dispatch and the game fatally routes to psx_unknown_dispatch. +extern "C" void dirty_ram_text_bless(uint32_t phys, const uint8_t* bytes, uint32_t len); + +namespace { + +namespace fs = std::filesystem; + +// --------------------------------------------------------------------------- +// Guest RAM access (little-endian, no swizzle). Main RAM is 2 MB, mirrored +// across [0,0x800000). Returns 0 / no-op for out-of-range. +// --------------------------------------------------------------------------- +constexpr uint32_t kRamSize = 2u * 1024u * 1024u; + +inline bool ram_fold(uint32_t va, uint32_t* pa_out) { + uint32_t p = va & 0x1FFFFFFFu; + if (p < 0x00800000u) { *pa_out = p & (kRamSize - 1u); return true; } + return false; // I/O / BIOS / scratchpad — not translatable text storage +} +inline uint8_t grb(uint8_t* ram, uint32_t va) { + uint32_t pa; return ram_fold(va, &pa) ? ram[pa] : 0u; +} +inline void gwb(uint8_t* ram, uint32_t va, uint8_t v) { + uint32_t pa; if (ram_fold(va, &pa)) ram[pa] = v; +} +inline bool va_in_ram(uint32_t va) { uint32_t pa; return ram_fold(va, &pa); } +// Bless a just-written patch region into the text-image guard (see the extern +// above). gwb writes RAM directly; this syncs the guard's reference so the patch +// reads as intentional data, not self-modifying code. Reads the now-current bytes +// straight from guest RAM, so it always blesses exactly what was written. +inline void bless_patch(uint8_t* ram, uint32_t va, uint32_t len) { + uint32_t pa; + if (len && ram_fold(va, &pa)) dirty_ram_text_bless(va & 0x1FFFFFFFu, ram + pa, len); +} + +// --------------------------------------------------------------------------- +// FNV-1a 64 over raw source bytes — the translation KV key. +// --------------------------------------------------------------------------- +inline uint64_t fnv1a(const uint8_t* p, uint32_t n) { + uint64_t h = 1469598103934665603ull; + for (uint32_t i = 0; i < n; ++i) { h ^= p[i]; h *= 1099511628211ull; } + return h; +} + +// =========================================================================== +// EncodingProfile — general text-encoding abstraction (validator + terminator +// rules + UTF-8 -> game-glyph transcoder). A title selects one; Tsumu uses +// shift_jis. Keeps the capture/apply core encoding-agnostic. +// =========================================================================== +constexpr uint32_t kSrcMax = 512; // max source record bytes captured/hashed + +enum class Term { None, Nul, FFFF }; + +struct EncodingProfile { + const char* name; + // Cheap first-byte gate: could `c` begin a text record? + bool (*first_byte_textish)(uint8_t c); + // Read a NUL/terminator-delimited record from guest RAM at va. Fills + // out[0..*len) with the raw content bytes (terminator excluded), sets + // *term. Returns true only for a plausible text record (>= min real chars). + bool (*read_record)(uint8_t* ram, uint32_t va, uint8_t* out, uint32_t* len, Term* term); + // Transcode a UTF-8 target string into game glyph bytes appended to `out`. + void (*transcode)(const std::string& utf8, std::vector<uint8_t>& out); + // Authoring quality gate: is this decoded record worth recording in the + // always-on capture inventory (vs. binary that passed the reader by chance)? + bool (*capture_worthy)(const uint8_t* bytes, uint32_t len); +}; + +// ---- Shift-JIS profile (Tsumu) -------------------------------------------- +inline bool sj_lead(uint8_t c) { return (c >= 0x81 && c <= 0x9F) || (c >= 0xE0 && c <= 0xFC); } +inline bool sj_trail(uint8_t c) { return (c >= 0x40 && c <= 0x7E) || (c >= 0x80 && c <= 0xFC); } +inline bool sj_kana(uint8_t c) { return c >= 0xA1 && c <= 0xDF; } // halfwidth katakana +inline bool sj_ascii(uint8_t c) { return c >= 0x20 && c <= 0x7E; } + +bool sj_first_byte_textish(uint8_t c) { + return sj_ascii(c) || sj_kana(c) || sj_lead(c); +} + +// A record ends at NUL (menu/UI labels) or the game's 0xFFFF message terminator +// (framed multi-line messages). Content bytes are kept VERBATIM — including the +// game's control framing: a lone SJIS-lead byte with no valid trail, 0x8x/0xFE, +// and the "FE FF" line-break code — so the hash is byte-exact and the original +// renderer sees identical framing. Binary structs are filtered by requiring a +// minimum count of decodable 2-byte SJIS units. +bool sj_read_record(uint8_t* ram, uint32_t va, uint8_t* out, uint32_t* len, Term* term) { + if (!va_in_ram(va)) return false; + uint8_t b0 = grb(ram, va); + if (!sj_first_byte_textish(b0)) return false; + uint32_t n = 0, real = 0, two = 0; // real=text units; two=decodable 2-byte SJIS + Term t = Term::None; + for (uint32_t i = 0; i < kSrcMax; ++i) { + uint8_t c = grb(ram, va + i); + if (c == 0x00) { t = Term::Nul; break; } + if (c == 0xFF && grb(ram, va + i + 1) == 0xFF) { t = Term::FFFF; break; } + if (sj_lead(c) && sj_trail(grb(ram, va + i + 1))) { + if (n + 2 > kSrcMax) break; + out[n++] = c; out[n++] = grb(ram, va + i + 1); ++i; ++real; ++two; continue; + } + if (sj_kana(c) || sj_ascii(c) || c == 0x0A) { + if (n + 1 > kSrcMax) break; + out[n++] = c; if (c != 0x0A) ++real; continue; + } + // Control / framing byte: lone SJIS lead (0x81..0x9F/0xE0..0xFC with no + // valid trail — e.g. the "81 FF" / "FE FF" framing), 0x01..0x1F, 0x80, + // 0xFD/0xFE, or a lone 0xFF (second half of a FE-FF line break). Keep it. + if (n + 1 > kSrcMax) break; + out[n++] = c; + } + // Reject binary: a genuine string carries at least two decodable 2-byte SJIS + // chars (or, for a short pure label, at least three text units). + if (t == Term::None) return false; + if (two < 2 && real < 3) return false; + *len = n; *term = t; + return true; +} + +// UTF-8 -> fullwidth-Shift-JIS. Tsumu's font already contains fullwidth Latin + +// digits (proven by "ＴＳＵＭＵ ｌｉｇｈｔ"), so English renders with zero glyph +// injection. '\n' -> raw 0x0A (the drawer's line break); unknown codepoints -> +// fullwidth space. (Halfwidth-ASCII path is a future EncodingProfile variant +// once MOJI.BIN confirms a halfwidth Latin block.) +uint16_t fw_sjis_for_ascii(uint32_t cp) { + if (cp >= 'A' && cp <= 'Z') return (uint16_t)(0x8260 + (cp - 'A')); + if (cp >= 'a' && cp <= 'z') return (uint16_t)(0x8281 + (cp - 'a')); + if (cp >= '0' && cp <= '9') return (uint16_t)(0x824F + (cp - '0')); + switch (cp) { + case ' ': return 0x8140; // fullwidth (ideographic) space + case '!': return 0x8149; + case '?': return 0x8148; + case '.': return 0x8144; + case ',': return 0x8143; + case ':': return 0x8146; + case ';': return 0x8147; + case '\'': return 0x8166; + case '"': return 0x8168; + case '(': return 0x8169; + case ')': return 0x816A; + case '[': return 0x816D; + case ']': return 0x816E; + case '-': return 0x817C; // fullwidth minus + case '/': return 0x815E; + case '&': return 0x8195; + case '$': return 0x8190; // fullwidth $ (Tsumu font: ○ button glyph) + case '%': return 0x8193; // fullwidth % (Tsumu font: ✕ button glyph) + case '+': return 0x817B; + case '=': return 0x8181; + case '*': return 0x8196; + case '@': return 0x8197; + case '<': return 0x8183; + case '>': return 0x8184; + case '_': return 0x8151; + default: return 0x8140; // unknown -> fullwidth space + } +} + +void sj_transcode(const std::string& utf8, std::vector<uint8_t>& out) { + size_t i = 0; + while (i < utf8.size()) { + uint8_t c = (uint8_t)utf8[i]; + uint32_t cp; size_t adv; + if (c < 0x80) { cp = c; adv = 1; } + else if ((c & 0xE0) == 0xC0 && i + 1 < utf8.size()) { + cp = ((c & 0x1F) << 6) | (utf8[i+1] & 0x3F); adv = 2; + } else if ((c & 0xF0) == 0xE0 && i + 2 < utf8.size()) { + cp = ((c & 0x0F) << 12) | ((utf8[i+1] & 0x3F) << 6) | (utf8[i+2] & 0x3F); adv = 3; + } else { cp = ' '; adv = 1; } + i += adv; + // Framing tokens the author writes into the English (remapped to the + // game's control codes in apply_to_reg): \n line break, \f page break, + // \r prompt/wait. Preserve them as their raw ASCII control bytes here. + if (cp == '\n') { out.push_back(0x0A); continue; } + if (cp == 0x0C) { out.push_back(0x0C); continue; } // \f -> page break + if (cp == 0x0D) { out.push_back(0x0D); continue; } // \r -> prompt + // A codepoint already in the fullwidth SJIS range would be authored as + // raw JP; here targets are Latin. Map ASCII-range codepoints; fullwidth + // Unicode letters (U+FF01..) map back to their ASCII then to SJIS. + if (cp >= 0xFF01 && cp <= 0xFF5E) cp = cp - 0xFF00 + 0x20; + if (cp == 0x3000) cp = ' '; + uint16_t g = fw_sjis_for_ascii(cp <= 0x7E ? cp : ' '); + // Endianness of the 16-bit glyph code as the game's drawer reads it. + // EXE menu labels are big-endian SJIS; the message-table text is stored + // little-endian. Selectable per run (PSX_XLATE_LE=1) until confirmed; + // once known this becomes an EncodingProfile flag. + static const bool le = [] { const char* e = std::getenv("PSX_XLATE_LE"); + return e && e[0] == '1'; }(); + if (le) { out.push_back((uint8_t)(g & 0xFF)); out.push_back((uint8_t)(g >> 8)); } + else { out.push_back((uint8_t)(g >> 8)); out.push_back((uint8_t)(g & 0xFF)); } + } +} + +// Transcode a UTF-8 label into fixed-width little-endian fullwidth-Shift-JIS for +// an in-place RAM source patch. Each ASCII char maps to its fullwidth glyph +// (2 bytes, stored low/high as the game's per-glyph drawer reads it), truncated +// to whole cells so at most `width` bytes are produced, then padded with the +// fullwidth space (0x8140 -> bytes 40 81) to exactly `width` bytes. NEVER emits +// more than `width` bytes — the hard guarantee against corrupting the next slot. +void label_transcode_le(const std::string& utf8, uint32_t width, std::vector<uint8_t>& out) { + out.clear(); + const uint32_t maxbytes = width & ~1u; // whole 2-byte cells only + size_t i = 0; + while (i < utf8.size() && out.size() + 2 <= maxbytes) { + uint8_t c = (uint8_t)utf8[i]; + uint32_t cp; size_t adv; + if (c < 0x80) { cp = c; adv = 1; } + else if ((c & 0xE0) == 0xC0 && i + 1 < utf8.size()) { + cp = ((c & 0x1F) << 6) | (utf8[i+1] & 0x3F); adv = 2; + } else if ((c & 0xF0) == 0xE0 && i + 2 < utf8.size()) { + cp = ((c & 0x0F) << 12) | ((utf8[i+1] & 0x3F) << 6) | (utf8[i+2] & 0x3F); adv = 3; + } else { cp = ' '; adv = 1; } + i += adv; + if (cp >= 0xFF01 && cp <= 0xFF5E) cp = cp - 0xFF00 + 0x20; // fullwidth Latin -> ASCII + if (cp == 0x3000) cp = ' '; // ideographic space + uint16_t g = fw_sjis_for_ascii(cp <= 0x7E ? cp : ' '); + out.push_back((uint8_t)(g & 0xFF)); // trail (low) — little-endian storage + out.push_back((uint8_t)(g >> 8)); // lead (high) + } + while (out.size() + 2 <= maxbytes) { out.push_back(0x40); out.push_back(0x81); } // fw-space pad +} + +// Capture-inventory quality gate (authoring signal only — APPLY is KV-gated and +// unaffected). Real Tsumu text is hiragana/katakana/fullwidth-alnum/kanji heavy, +// whose SJIS lead bytes concentrate in the 0x81/0x82/0x83 and 0x88-0x9F/0xE0-0xEA +// rows; vertex/coordinate binary that passes the record reader by chance has +// diverse, out-of-row leads. Gate: (a) a minimum real length, (b) >= 2 real +// kana/fullwidth-alnum chars, and (c) the decoded 2-byte units are >= 80% in +// valid Shift-JIS text rows — so the always-on inventory stays a clean +// enumeration of drawn strings and binary noise is dropped. +bool sj_capture_worthy(const uint8_t* b, uint32_t n) { + if (n < 4) return false; + uint32_t units = 0, good = 0, kanafw = 0, unit_bytes = 0; + for (uint32_t i = 0; i + 1 < n; ) { + uint8_t c = b[i], d = b[i + 1]; + if (sj_lead(c) && sj_trail(d)) { + ++units; + // Valid Shift-JIS text rows: 0x81 symbols/punct, 0x82 hiragana + + // fullwidth alnum, 0x83 katakana, 0x88-0x9F + 0xE0-0xEA kanji. + if (c == 0x81 || c == 0x82 || c == 0x83 || + (c >= 0x88 && c <= 0x9F) || (c >= 0xE0 && c <= 0xEA)) { ++good; unit_bytes += 2; } + if (c == 0x83 || (c == 0x82 && d >= 0x4F)) ++kanafw; // kana / fullwidth alnum + i += 2; + } else i += 1; + } + if (kanafw < 2) return false; // needs real JP / fullwidth content + if (units == 0 || good * 5 < units * 4) return false; // >= 80% of units in valid rows + if (unit_bytes * 2 < n) return false; // >= 50% of record is 2-byte SJIS text + // (drops long single-byte fill buffers) + return true; +} + +// Canonical record-START gate (capture only). A capture is ingested only when +// `va` is a genuine record boundary — the unit immediately preceding it is a +// terminator (NUL / 0xFFFF) or a region edge. This collapses the partial-offset +// explosion: a register that lands mid-string (or mid-binary-struct) is preceded +// by content bytes, not a terminator, so the SAME record is no longer re-ingested +// at every byte offset a register happens to point. APPLY is unaffected — it must +// still match the exact bytes the game draws at whatever offset the register +// carries, so this gate is never consulted on the apply path. +inline bool sj_at_record_start(uint8_t* ram, uint32_t va) { + if (!va_in_ram(va - 1)) return true; // region edge + uint8_t p1 = grb(ram, va - 1); + if (p1 == 0x00) return true; // after NUL terminator + if (p1 == 0xFF && grb(ram, va - 2) == 0xFF) return true; // after 0xFFFF terminator + return false; // mid-record offset — skip +} + +// Struct-vs-text safety: a genuine message record contains only text units and +// the known control framing (0x0A newline; 0x10/0x30 page-break leads; 0xFC/0xFE/ +// 0xFF framing; fullwidth-space pad). A fixed-width struct field packs binary +// params after the text (e.g. the "Tutorial N" label's trailing 09 08 01 02 F4), +// which shows up as low-control bytes 0x01..0x09 / 0x0B..0x0F / 0x11..0x1F. +// Replacing such a record corrupts the struct and derails the game — so apply is +// refused for records carrying those wild bytes even when a table entry exists. +bool sj_clean_text(const uint8_t* b, uint32_t n) { + for (uint32_t i = 0; i < n; ++i) { + uint8_t c = b[i]; + if (sj_lead(c) && i + 1 < n && sj_trail(b[i + 1])) { ++i; continue; } + if (c == 0x0A || c == 0x10 || c == 0x30) continue; // newline / page-break leads + if (c >= 0x20) continue; // text / framing / high bytes + return false; // wild low-control => struct + } + return true; +} + +const EncodingProfile kShiftJisProfile = { + "shift_jis", sj_first_byte_textish, sj_read_record, sj_transcode, sj_capture_worthy +}; + +// =========================================================================== +// Module state +// =========================================================================== +struct TableEntry { + std::string target; // active-language UTF-8 target (empty => fall back to source) + Term term = Term::Nul; +}; +struct CapRec { + uint8_t bytes[96]; + uint8_t sample_len; // bytes stored in `bytes` (<=96) + uint16_t full_len; // full record length + uint32_t addr; // last source VA seen + uint32_t pc; // a dispatch target that carried it + uint32_t ra; + uint32_t first_frame; + uint64_t count; + Term term; // terminator that delimited the record (nul|ffff) + bool translated; +}; + +// A per-glyph label source-patch entry. Some UI text (level/stage names) is +// drawn glyph-by-glyph as sprites from a fixed-stride, space-padded table in the +// EXE — there is NO string pointer and NO terminator, so the dispatch/arg-scan +// message hook above can never see it. Instead we patch the label's bytes in +// guest RAM once the EXE region is resident: overwrite the confirmed slot with +// the target-language text (fullwidth-Latin Shift-JIS, little-endian) re-padded +// to the exact slot width, and the game's OWN per-glyph routine then draws +// English. Verify-before-patch (RAM must equal `src` first) guarantees we never +// write unless the exact expected JP label is present, and writing exactly +// `width` bytes (== the table stride) guarantees we never spill into the next +// slot. See docs/STRING_TRANSLATION.md Appendix Y. +struct GlyphLabel { + uint32_t addr = 0; // slot base VA + uint32_t width = 0; // slot stride in bytes (never exceeded) + std::vector<uint8_t> src; // expected JP source bytes (verify key) + std::string target; // active-language UTF-8 (empty => skip) + bool patched = false; +}; + +// A VRAM-strip patch entry. Some UI text is pre-rendered pixels inside a +// texture asset the game uploads to VRAM and samples with quads (Tsumu's HUD +// labels live in the glyph-sheet image) — no glyph codes exist anywhere, so +// neither the message hook nor the glyph-label patch can reach it. The patch +// rides the game's own upload path instead: whenever a GP0 0xA0 CPU→VRAM +// transfer completes and fully contains the patch rect, the rect is verified +// halfword-for-halfword against the expected JP pixels and, only on an exact +// match, rewritten with the target-language pixels through the renderer's own +// VRAM write facade. Because it re-applies on every matching upload it is a +// source-path patch (survives scene reloads / re-uploads), not a one-shot +// VRAM poke. +struct VramPatch { + int x = 0, y = 0; // rect origin, VRAM halfword coords + int w = 0, h = 0; // rect size (halfwords × rows) + std::vector<uint16_t> src; // expected JP pixels (verify key), w*h + std::vector<uint16_t> rep; // replacement pixels, w*h + uint64_t applied = 0; // times applied (per-upload re-apply) +}; + +// Table-driven message record. Some UI text is addressed through a POINTER +// TABLE the renderer indexes internally — e.g. Tsumu's memory-card dialogs: +// show_card_dialog(id) loads table[id] and draws it, so the string pointer is +// never a dispatch argument and the a0..a3 message hook can never see it (nor +// can it be captured). The fix is the glyph-label pattern applied to an SJIS +// message record: verify the exact JP bytes are resident at a known address, +// then overwrite them IN PLACE with the transcoded target text (little-endian +// fullwidth-Shift-JIS body + the game's FE-FF line-break / 30-FC page-break / +// 10-FC prompt framing), terminated, never exceeding the source byte length — +// so the game's own message renderer reads the translation. Verify-before- +// patch + the length cap guarantee no corruption; idempotent (a patched slot +// no longer matches `src`). Authored via a [[entry]] carrying a `ram_addr`. +struct MsgInplace { + uint32_t addr = 0; // message record VA (game data region) + std::vector<uint8_t> src; // expected JP record bytes (verify key) + std::string target; // active-language UTF-8 + Term term = Term::Nul; // record terminator (nul / ffff) + bool patched = false; +}; + +const EncodingProfile* g_prof = &kShiftJisProfile; + +std::unordered_map<uint64_t, TableEntry> g_table; // hash -> translation +std::unordered_map<uint64_t, CapRec> g_inv; // hash -> capture record +std::vector<GlyphLabel> g_glyph_labels;// per-glyph RAM patches +std::atomic<int> g_glyph_pending{0}; // unpatched count (0 => idle) +std::vector<VramPatch> g_vram_patches; // pre-rendered strip patches +std::atomic<int> g_vram_patch_n{0}; // count (0 => upload hook idle) +std::vector<MsgInplace> g_msg_inplace; // table-driven message RAM patches +std::atomic<int> g_msg_inplace_pending{0}; // unpatched count (0 => idle) +// Card-manager messages are packed into NUL-terminated *chunks*; within a chunk, +// distinct messages (each with its own pointer-table entry) are separated ONLY by +// runs of the fullwidth space (0x8140), with no terminator between them. The +// game's renderer reads a message from its pointer until the next NUL, so any +// non-last message in a chunk bleeds through its neighbours into the chunk tail +// (e.g. the "Reading Card" / "Saving" status messages rendered the format prompt +// and its Yes/No wait). A separator drops a NUL at the start of the trailing +// space-run so each message reads to its own terminator. Verify-then-patch: only +// a resident fullwidth space is ever overwritten, so drift can never corrupt text. +struct NulSep { uint32_t addr = 0; bool patched = false; }; +std::vector<NulSep> g_msg_seps; // inter-message NUL separators +std::atomic<int> g_msg_sep_pending{0}; // unpatched count (0 => idle) +std::mutex g_mtx; + +std::atomic<bool> g_apply_armed{false}; // table non-empty AND language enabled +std::atomic<bool> g_capture_on{true}; // always-on inventory (default) +std::atomic<uint64_t> g_calls{0}; +std::atomic<uint64_t> g_hits{0}; +std::string g_lang = "en"; +std::string g_dir; // translations/ directory (for reload) + +// Recently-written scratch VAs — so our own transcoded (fullwidth-SJIS) English +// scratch isn't re-captured as a bogus "Japanese" record. +uint32_t g_scratch_ring[32] = {0}; +std::atomic<uint32_t> g_scratch_pos{0}; +bool scratch_recent(uint32_t va) { + for (uint32_t i = 0; i < 32; ++i) { + uint32_t s = g_scratch_ring[i]; + if (s && (va >= s) && (va < s + kSrcMax + 4)) return true; + } + return false; +} +void scratch_note(uint32_t va) { + g_scratch_ring[g_scratch_pos.fetch_add(1, std::memory_order_relaxed) & 31u] = va; +} + +// --------------------------------------------------------------------------- +// Table loading (all translations/*.toml). Hot-reloadable via debug "reload". +// --------------------------------------------------------------------------- +std::vector<uint8_t> hex_to_bytes(const std::string& hex) { + std::vector<uint8_t> b; + auto hv = [](char c) -> int { + if (c >= '0' && c <= '9') return c - '0'; + if (c >= 'a' && c <= 'f') return c - 'a' + 10; + if (c >= 'A' && c <= 'F') return c - 'A' + 10; + return -1; + }; + for (size_t i = 0; i + 1 < hex.size(); i += 2) { + int hi = hv(hex[i]), lo = hv(hex[i+1]); + if (hi < 0 || lo < 0) { b.clear(); break; } + b.push_back((uint8_t)((hi << 4) | lo)); + } + return b; +} + +// hex string -> little-endian halfwords (VRAM pixel stream order). +std::vector<uint16_t> hex_to_halfwords(const std::string& hex) { + auto b = hex_to_bytes(hex); + std::vector<uint16_t> hw; + if (b.empty() || (b.size() & 1)) return hw; + hw.reserve(b.size() / 2); + for (size_t i = 0; i + 1 < b.size(); i += 2) + hw.push_back((uint16_t)(b[i] | (b[i + 1] << 8))); + return hw; +} + +void load_tables_locked() { + g_table.clear(); + g_glyph_labels.clear(); + g_glyph_pending.store(0, std::memory_order_relaxed); + g_vram_patches.clear(); + g_vram_patch_n.store(0, std::memory_order_relaxed); + g_msg_inplace.clear(); + g_msg_inplace_pending.store(0, std::memory_order_relaxed); + g_msg_seps.clear(); + g_msg_sep_pending.store(0, std::memory_order_relaxed); + for (auto& kv : g_inv) kv.second.translated = false; + if (g_dir.empty() || !fs::exists(g_dir)) { g_apply_armed.store(false); return; } + const bool lang_off = g_lang.empty() || g_lang == "jp" || g_lang == "off"; + size_t files = 0, entries = 0, glyphs = 0, vpatches = 0; + std::error_code ec; + for (auto& de : fs::directory_iterator(g_dir, ec)) { + if (!de.is_regular_file()) continue; + if (de.path().extension() != ".toml") continue; + toml::value data; + try { data = toml::parse(de.path().string()); } + catch (...) { continue; } + ++files; + if (!data.contains("entry")) continue; + const auto& arr = toml::find(data, "entry"); + if (!arr.is_array()) continue; + for (const auto& e : arr.as_array()) { + if (!e.contains("src_hex")) continue; + std::string hex = toml::find_or<std::string>(e, "src_hex", ""); + auto bytes = hex_to_bytes(hex); + if (bytes.empty()) continue; + std::string tgt; + if (e.contains(g_lang)) tgt = toml::find_or<std::string>(e, g_lang, ""); + else if (e.contains("en")) tgt = toml::find_or<std::string>(e, "en", ""); + if (tgt.empty()) continue; // no translation for this lang => source shown + Term term = Term::Nul; + std::string ts = toml::find_or<std::string>(e, "term", "nul"); + if (ts == "ffff" || ts == "FFFF") term = Term::FFFF; + else if (ts == "none") term = Term::None; // mid-blob sub-record: no + // terminator write (content follows) + // Optional in-place RAM patch: a record the game addresses through a + // pointer table (never a dispatch arg) can't be reached by the a0..a3 + // hook, so overwrite the JP bytes in place at `ram_addr`. The entry + // still joins g_table (harmless if the record is also ever passed). + if (e.contains("ram_addr")) { + uint32_t ra = (uint32_t)toml::find_or<int64_t>(e, "ram_addr", 0); + if (ra) g_msg_inplace.push_back(MsgInplace{ ra, bytes, tgt, term, false }); + } + uint64_t key = fnv1a(bytes.data(), (uint32_t)bytes.size()); + g_table[key] = TableEntry{ std::move(tgt), term }; + ++entries; + } + // Per-glyph label source-patch entries (RAM-patch layer). Only load when + // a target for the active language exists; otherwise the slot stays JP. + if (!lang_off && data.contains("glyph_label")) { + const auto& garr = toml::find(data, "glyph_label"); + if (garr.is_array()) for (const auto& g : garr.as_array()) { + if (!g.contains("addr") || !g.contains("src_hex")) continue; + std::string ghex = toml::find_or<std::string>(g, "src_hex", ""); + auto gbytes = hex_to_bytes(ghex); + if (gbytes.empty()) continue; + std::string tgt; + if (g.contains(g_lang)) tgt = toml::find_or<std::string>(g, g_lang, ""); + else if (g.contains("en")) tgt = toml::find_or<std::string>(g, "en", ""); + if (tgt.empty()) continue; // no translation for this lang => slot stays JP + GlyphLabel gl; + gl.addr = (uint32_t)toml::find_or<int64_t>(g, "addr", 0); + gl.width = (uint32_t)toml::find_or<int64_t>(g, "width", (int64_t)gbytes.size()); + if (gl.addr == 0 || gl.width == 0 || gbytes.size() > gl.width) continue; + gl.src = std::move(gbytes); + gl.target = std::move(tgt); + g_glyph_labels.push_back(std::move(gl)); + ++glyphs; + } + } + // VRAM-strip patch entries (pre-rendered-pixel layer). The replacement + // pixels come from a per-language "<lang>_hex" key ("en_hex" fallback); + // no key for the active language => the strip stays JP. + if (!lang_off && data.contains("vram_patch")) { + const auto& varr = toml::find(data, "vram_patch"); + if (varr.is_array()) for (const auto& v : varr.as_array()) { + if (!v.contains("src_hex")) continue; + VramPatch vp; + vp.x = (int)toml::find_or<int64_t>(v, "x", -1); + vp.y = (int)toml::find_or<int64_t>(v, "y", -1); + vp.w = (int)toml::find_or<int64_t>(v, "w", 0); + vp.h = (int)toml::find_or<int64_t>(v, "h", 0); + if (vp.x < 0 || vp.y < 0 || vp.w <= 0 || vp.h <= 0) continue; + if (vp.x + vp.w > 1024 || vp.y + vp.h > 512) continue; + vp.src = hex_to_halfwords(toml::find_or<std::string>(v, "src_hex", "")); + std::string rk = g_lang + "_hex"; + if (v.contains(rk)) vp.rep = hex_to_halfwords(toml::find_or<std::string>(v, rk, "")); + else if (v.contains("en_hex")) vp.rep = hex_to_halfwords(toml::find_or<std::string>(v, "en_hex", "")); + size_t need = (size_t)vp.w * (size_t)vp.h; + if (vp.src.size() != need || vp.rep.size() != need) continue; + g_vram_patches.push_back(std::move(vp)); + ++vpatches; + } + } + // Inter-message NUL separators (structure fix for packed message chunks). + // Each entry names a VA that currently holds a fullwidth space at the start + // of an inter-message gap; a NUL is dropped there so the preceding message + // reads to its own terminator instead of bleeding into its neighbour. + if (!lang_off && data.contains("msg_sep")) { + const auto& sarr = toml::find(data, "msg_sep"); + if (sarr.is_array()) for (const auto& s : sarr.as_array()) { + if (!s.contains("addr")) continue; + uint32_t a = (uint32_t)toml::find_or<int64_t>(s, "addr", 0); + if (a) g_msg_seps.push_back(NulSep{ a, false }); + } + } + } + g_glyph_pending.store((int)g_glyph_labels.size(), std::memory_order_relaxed); + g_vram_patch_n.store((int)g_vram_patches.size(), std::memory_order_relaxed); + g_msg_inplace_pending.store((int)g_msg_inplace.size(), std::memory_order_relaxed); + g_msg_sep_pending.store((int)g_msg_seps.size(), std::memory_order_relaxed); + g_apply_armed.store(!lang_off && (!g_table.empty() || !g_glyph_labels.empty() || + !g_vram_patches.empty() || !g_msg_inplace.empty() || + !g_msg_seps.empty()), + std::memory_order_relaxed); + // Mark inventory records that now have a translation. + for (auto& kv : g_inv) + kv.second.translated = (g_table.find(kv.first) != g_table.end()); + std::fprintf(stderr, + "[xlate] loaded %zu entries + %zu glyph-labels + %zu vram-patches + %zu msg-inplace from %zu file(s) in %s (lang=%s apply=%s)\n", + entries, glyphs, vpatches, g_msg_inplace.size(), files, g_dir.c_str(), g_lang.c_str(), + g_apply_armed.load() ? "on" : "off"); +} + +// --------------------------------------------------------------------------- +// Capture inventory upsert. +// --------------------------------------------------------------------------- +void inv_upsert_locked(uint64_t key, const uint8_t* bytes, uint32_t len, + uint32_t addr, uint32_t pc, uint32_t ra, Term term) { + auto it = g_inv.find(key); + if (it != g_inv.end()) { it->second.count++; it->second.addr = addr; + it->second.pc = pc; it->second.ra = ra; return; } + if (g_inv.size() >= 60000) return; // sane bound + CapRec r{}; + uint32_t n = len < 96 ? len : 96; + std::memcpy(r.bytes, bytes, n); + r.sample_len = (uint8_t)n; r.full_len = (uint16_t)len; r.term = term; + r.addr = addr; r.pc = pc; r.ra = ra; + r.first_frame = (uint32_t)s_frame_count; r.count = 1; + r.translated = (g_table.find(key) != g_table.end()); + g_inv[key] = r; +} + +// --------------------------------------------------------------------------- +// Glyph-label RAM source-patch: scan the label table and patch any slot whose +// confirmed JP source bytes are currently resident in guest RAM. Idempotent — +// a patched slot no longer matches `src`, so it is written exactly once. Runs +// under g_mtx. +// --------------------------------------------------------------------------- +void glyph_labels_patch_locked(uint8_t* ram) { + int pending = 0; + for (auto& gl : g_glyph_labels) { + if (gl.patched || gl.target.empty() || gl.width == 0) continue; + // Verify-before-patch: the exact expected JP label must be resident and + // the whole slot must fit in RAM. If not, the region isn't loaded yet — + // leave it JP and try again on a later scan (never corrupt). + if (!va_in_ram(gl.addr) || !va_in_ram(gl.addr + gl.width - 1)) { ++pending; continue; } + bool match = gl.src.size() <= gl.width; + for (uint32_t i = 0; match && i < gl.src.size(); ++i) + if (grb(ram, gl.addr + i) != gl.src[i]) match = false; + if (!match) { ++pending; continue; } + std::vector<uint8_t> enc; + label_transcode_le(gl.target, gl.width, enc); + if (enc.empty() || enc.size() > gl.width) { ++pending; continue; } // paranoia + for (uint32_t i = 0; i < enc.size(); ++i) gwb(ram, gl.addr + i, enc[i]); + bless_patch(ram, gl.addr, (uint32_t)enc.size()); + gl.patched = true; + } + g_glyph_pending.store(pending, std::memory_order_relaxed); +} + +// Build a message body: little-endian fullwidth-Shift-JIS (as the message-table +// drawer reads it) with the author's \n/\f/\r framing tokens mapped to the +// game's control codes (FE-FF line break, 30-FC page break, 10-FC prompt). +// Mirrors sj_transcode's glyph mapping + apply_to_reg's framing, fused for the +// in-place path (which writes the framed bytes directly rather than repointing). +void msg_inplace_build(const std::string& utf8, std::vector<uint8_t>& out) { + out.clear(); + size_t i = 0; + while (i < utf8.size()) { + uint8_t c = (uint8_t)utf8[i]; + uint32_t cp; size_t adv; + if (c < 0x80) { cp = c; adv = 1; } + else if ((c & 0xE0) == 0xC0 && i + 1 < utf8.size()) { + cp = ((c & 0x1F) << 6) | (utf8[i+1] & 0x3F); adv = 2; + } else if ((c & 0xF0) == 0xE0 && i + 2 < utf8.size()) { + cp = ((c & 0x0F) << 12) | ((utf8[i+1] & 0x3F) << 6) | (utf8[i+2] & 0x3F); adv = 3; + } else { cp = ' '; adv = 1; } + i += adv; + if (cp == '\n') { out.push_back(0xFE); out.push_back(0xFF); continue; } // line break + if (cp == 0x0C) { out.push_back(0x30); out.push_back(0xFC); continue; } // \f page break + if (cp == 0x0D) { out.push_back(0x10); out.push_back(0xFC); continue; } // \r prompt/wait + if (cp >= 0xFF01 && cp <= 0xFF5E) cp = cp - 0xFF00 + 0x20; // fullwidth Latin -> ASCII + if (cp == 0x3000) cp = ' '; + uint16_t g = fw_sjis_for_ascii(cp <= 0x7E ? cp : ' '); + out.push_back((uint8_t)(g & 0xFF)); // little-endian storage + out.push_back((uint8_t)(g >> 8)); + } +} + +// Table-driven message in-place RAM patch. Mirrors glyph_labels_patch_locked +// but for SJIS message records. Runs under g_mtx. +void msg_inplace_patch_locked(uint8_t* ram) { + int pending = 0; + for (auto& m : g_msg_inplace) { + if (m.patched || m.target.empty() || m.src.empty()) continue; + uint32_t len = (uint32_t)m.src.size(); + if (!va_in_ram(m.addr) || !va_in_ram(m.addr + len - 1)) { ++pending; continue; } + // Verify-before-patch: the exact JP record must be resident (else the + // region isn't loaded yet, or it's already patched — never corrupt). + bool match = true; + for (uint32_t i = 0; match && i < len; ++i) + if (grb(ram, m.addr + i) != m.src[i]) match = false; + if (!match) { ++pending; continue; } + std::vector<uint8_t> body; + msg_inplace_build(m.target, body); + // Structure-preserving in-place patch. The author writes the SAME control + // framing as the source (\n \f \r), so an English body sized to the exact + // source length reproduces the record's control layout — critically the + // trailing \r (0x10-FC prompt/wait) an interactive dialog needs to accept + // input and advance. Truncate to whole 2-byte cells if over-long, then pad + // with the fullwidth space to the source length so no stale JP tail + // remains, and reuse the record's own terminator slot (offset len) so the + // NUL/0xFFFF lands exactly where the game expects it. + while (!body.empty() && body.size() > len) body.resize(body.size() - 2); + if (body.empty()) { m.patched = true; continue; } // nothing fit; don't spin + // term=none marks a mid-blob sub-record (e.g. a Yes/No prompt whose \r is + // followed by another message in the same NUL/FFFF span): write the body + // exactly, WITHOUT padding or a terminator, so the following content is + // left intact. Otherwise pad to the record length and stamp the terminator + // in the record's own terminator slot. + if (m.term != Term::None) + while (body.size() + 2 <= len) { body.push_back(0x40); body.push_back(0x81); } + for (uint32_t i = 0; i < body.size(); ++i) gwb(ram, m.addr + i, body[i]); + uint32_t ti = (uint32_t)body.size(); + if (m.term == Term::FFFF) { + if (va_in_ram(m.addr + ti + 1)) { gwb(ram, m.addr + ti, 0xFF); gwb(ram, m.addr + ti + 1, 0xFF); } + } else if (m.term == Term::Nul && va_in_ram(m.addr + ti)) { + gwb(ram, m.addr + ti, 0x00); + } + // Bless the whole written span (body + up to a 2-byte terminator) so the + // guard sees the patched record as intentional data, not modified code. + bless_patch(ram, m.addr, ti + 2); + m.patched = true; + } + g_msg_inplace_pending.store(pending, std::memory_order_relaxed); +} + +// Drop an end-of-message terminator (0xFFFF) at each inter-message separator VA +// once the region is resident. The text renderer (FUN at 0x80058xxx, message +// pointer at gp+0x26c) stops a message at the \r prompt-wait (0xFC10) or the +// 0xFFFF end marker — NOT at 0x0000 (that byte is drawn as a glyph). 0xFFFF ends +// the message with no input-wait (as the game's own term=ffff status messages +// do), so the preceding packed message reads to its own terminator instead of +// bleeding through its neighbours to the next \r. Verify-then-patch: the two +// bytes must currently be a fullwidth space (LE 0x40 0x81) — never overwrite +// text — so an already-patched or drifted slot is simply skipped. +void msg_seps_patch_locked(uint8_t* ram) { + int pending = 0; + for (auto& s : g_msg_seps) { + if (s.patched) continue; + if (!va_in_ram(s.addr) || !va_in_ram(s.addr + 1)) { ++pending; continue; } + if (grb(ram, s.addr) != 0x40 || grb(ram, s.addr + 1) != 0x81) { ++pending; continue; } + gwb(ram, s.addr, 0xFF); + gwb(ram, s.addr + 1, 0xFF); + bless_patch(ram, s.addr, 2); + s.patched = true; + } + g_msg_sep_pending.store(pending, std::memory_order_relaxed); +} + +// --------------------------------------------------------------------------- +// VRAM-strip patch: verify the rect holds the exact expected JP pixels, then +// rewrite it with the target-language pixels through the renderer facade. +// `force` skips the upload-containment gate (debug "vpatch" re-apply). Runs +// under g_mtx. Returns patches applied this call. +// --------------------------------------------------------------------------- +int vram_patches_apply_locked(int ux, int uy, int uw, int uh, bool force) { + int applied = 0; + for (auto& vp : g_vram_patches) { + if (!force && + (vp.x < ux || vp.y < uy || vp.x + vp.w > ux + uw || vp.y + vp.h > uy + uh)) + continue; + // Verify-before-patch: every halfword must match the expected JP + // pixels. A different asset at the same coords (or an already-patched + // strip) fails the compare and is left untouched. + bool match = true; + for (int r = 0; match && r < vp.h; ++r) + for (int c = 0; match && c < vp.w; ++c) + if (gr_vram_read(vp.x + c, vp.y + r) != vp.src[(size_t)r * vp.w + c]) + match = false; + if (!match) continue; + for (int r = 0; r < vp.h; ++r) + for (int c = 0; c < vp.w; ++c) + gr_vram_write(vp.x + c, vp.y + r, vp.rep[(size_t)r * vp.w + c]); + vp.applied++; + ++applied; + } + return applied; +} + +// --------------------------------------------------------------------------- +// Apply: transcode + write guest scratch + repoint the arg register. +// --------------------------------------------------------------------------- +bool apply_to_reg(uint8_t* ram, CPUState* cpu, uint32_t sp, uint32_t* reg, + const TableEntry& te, Term src_term) { + std::vector<uint8_t> enc; + g_prof->transcode(te.target, enc); + if (enc.empty()) return false; + Term t = (te.term != Term::Nul) ? te.term : src_term; // preserve source terminator + // For framed (0xFFFF) messages the game's line break is the "FE FF" code, not + // a raw 0x0A — remap so multi-line English reflows in the text box. + std::vector<uint8_t> body; + body.reserve(enc.size() + 16); + for (uint8_t c : enc) { + if (c == 0x0A && t == Term::FFFF) { body.push_back(0xFE); body.push_back(0xFF); } // line break + else if (c == 0x0A) { /* NUL record: drop stray newline */ } + else if (c == 0x0C) { body.push_back(0x30); body.push_back(0xFC); } // \f page break + else if (c == 0x0D) { body.push_back(0x10); body.push_back(0xFC); } // \r prompt/wait + else body.push_back(c); + } + if (body.empty() || body.size() > kSrcMax) return false; + // Scratch below the caller's $sp, 8-byte aligned, with headroom. + if (sp < 0x80001000u) return false; + uint32_t scratch = (sp - 0x900u) & ~7u; + if (!va_in_ram(scratch) || !va_in_ram(scratch + (uint32_t)body.size() + 2)) return false; + for (size_t i = 0; i < body.size(); ++i) gwb(ram, scratch + (uint32_t)i, body[i]); + if (t == Term::FFFF) { gwb(ram, scratch + (uint32_t)body.size(), 0xFF); + gwb(ram, scratch + (uint32_t)body.size() + 1, 0xFF); } + else { gwb(ram, scratch + (uint32_t)body.size(), 0x00); } + scratch_note(scratch); + *reg = scratch; + return true; +} + +} // namespace + +// =========================================================================== +// Public API +// =========================================================================== +extern "C" void text_xlate_on_dispatch(CPUState* cpu, uint32_t target) { + const bool cap = g_capture_on.load(std::memory_order_relaxed); + const bool app = g_apply_armed.load(std::memory_order_relaxed); + if (!cap && !app) return; + uint8_t* ram = memory_get_ram_ptr(); + if (!ram) return; + uint64_t call_n = g_calls.fetch_add(1, std::memory_order_relaxed); + + // Per-glyph label RAM source-patch (load-time). Runs while any label is still + // unpatched — i.e. until the EXE label region becomes resident and every + // confirmed slot has been overwritten. Once g_glyph_pending hits 0 this is a + // single relaxed load on the hot path. Throttled so the (cheap) verify scan + // costs nothing per-dispatch before the region loads. + if (app && g_glyph_pending.load(std::memory_order_relaxed) > 0 && + (call_n & 0x1FFu) == 0) { + std::lock_guard<std::mutex> lk(g_mtx); + glyph_labels_patch_locked(ram); + } + // Table-driven message records (never a dispatch arg): patch in place while + // any is still unpatched. Same cheap throttle as glyph labels. + if (app && g_msg_inplace_pending.load(std::memory_order_relaxed) > 0 && + (call_n & 0x1FFu) == 0) { + std::lock_guard<std::mutex> lk(g_mtx); + msg_inplace_patch_locked(ram); + } + // Inter-message NUL separators: same cheap throttle; idle once all placed. + if (app && g_msg_sep_pending.load(std::memory_order_relaxed) > 0 && + (call_n & 0x1FFu) == 0) { + std::lock_guard<std::mutex> lk(g_mtx); + msg_seps_patch_locked(ram); + } + + const uint32_t sp = cpu->gpr[29]; + // Scan the argument registers a0..a3 for source-text pointers. KV-gated + // apply means swapping only the reg that pointed at a KNOWN record — a + // coincidental non-text arg won't be in the table, so it is never touched. + uint32_t* argregs[4] = { &cpu->gpr[4], &cpu->gpr[5], &cpu->gpr[6], &cpu->gpr[7] }; + for (int a = 0; a < 4; ++a) { + uint32_t va = *argregs[a]; + if (!va_in_ram(va)) continue; + if (!g_prof->first_byte_textish(grb(ram, va))) continue; + if (scratch_recent(va)) continue; // don't recapture our own English scratch + uint8_t buf[kSrcMax]; uint32_t len = 0; Term term = Term::None; + if (!g_prof->read_record(ram, va, buf, &len, &term)) continue; + uint64_t key = fnv1a(buf, len); + + // CAPTURE (always-on inventory): ingest each real string ONCE at its + // genuine record start, and only if it clears the noise gate. The + // canonical-start test collapses the partial-offset explosion (the same + // record was previously re-ingested at every offset a register landed on). + // APPLY below is deliberately NOT gated by record-start — it matches the + // exact bytes the game draws wherever the pointer lands. + if (cap && sj_at_record_start(ram, va) && + (!g_prof->capture_worthy || g_prof->capture_worthy(buf, len))) { + std::lock_guard<std::mutex> lk(g_mtx); + inv_upsert_locked(key, buf, len, va, target, cpu->gpr[31], term); + } + if (app) { + // Safety: by default only substitute standalone framed (0xFFFF) + // messages. NUL-terminated records are frequently fixed-width struct + // fields with binary params packed after the text (e.g. the "Tutorial + // N" label carries trailing coordinate bytes); replacing those + // corrupts the struct and derails the game. A NUL record is only + // safe when its table entry opts in with term="nul" AND matches + // byte-for-byte (the author vetted it). PSX_XLATE_ALLOW_NUL=1 lifts + // the gate for experimentation. + TableEntry te; + { + std::lock_guard<std::mutex> lk(g_mtx); + auto it = g_table.find(key); + if (it == g_table.end()) continue; + te = it->second; + } + // The table entry is author-vetted, but guard against corrupting a + // struct that shares the key: apply only when the source record is + // framed (0xFFFF, always standalone) OR pure clean text (no wild + // binary params). PSX_XLATE_ALLOW_NUL=1 forces through for debugging. + static const bool allow_nul = [] { + const char* e = std::getenv("PSX_XLATE_ALLOW_NUL"); return e && e[0] == '1'; }(); + if (term != Term::FFFF && !allow_nul && !sj_clean_text(buf, len)) continue; + if (apply_to_reg(ram, cpu, sp, argregs[a], te, term)) + g_hits.fetch_add(1, std::memory_order_relaxed); + } + } +} + +extern "C" void text_xlate_vram_upload(int x, int y, int w, int h) { + // Hot path: one relaxed load when no patches are configured / lang off. + if (g_vram_patch_n.load(std::memory_order_relaxed) <= 0) return; + if (!g_apply_armed.load(std::memory_order_relaxed)) return; + std::lock_guard<std::mutex> lk(g_mtx); + vram_patches_apply_locked(x, y, w, h, false); +} + +extern "C" void text_xlate_init(const char* project_root, const char* language) { + const char* env = std::getenv("PSX_LANG"); + if (env && *env) g_lang = env; + else if (language && *language) g_lang = language; + if (project_root && *project_root) + g_dir = (fs::path(project_root) / "translations").string(); + const char* capenv = std::getenv("PSX_XLATE_CAPTURE"); + if (capenv && capenv[0] == '0') g_capture_on.store(false); + std::lock_guard<std::mutex> lk(g_mtx); + load_tables_locked(); +} + +extern "C" void text_xlate_set_language(const char* language) { + // PSX_LANG env is an authoring/testing override — if set it pins the language + // and the launcher's choice is ignored (matches text_xlate_init precedence). + const char* env = std::getenv("PSX_LANG"); + std::string want = (env && *env) ? env : (language ? language : ""); + std::lock_guard<std::mutex> lk(g_mtx); + if (want == g_lang) return; // no change => skip the reload + g_lang = want; + load_tables_locked(); // re-selects the target column + re-arms APPLY +} + +extern "C" int text_xlate_debug_json(const char* subcmd, char* out, int cap) { + if (!out || cap <= 0) return 0; + std::string sc = subcmd ? subcmd : "stats"; + std::lock_guard<std::mutex> lk(g_mtx); + + if (sc == "reload") { load_tables_locked(); + return std::snprintf(out, cap, "{\"reloaded\":true,\"entries\":%zu}", g_table.size()); } + + if (sc == "stats") { + size_t gp = 0; for (auto& gl : g_glyph_labels) if (gl.patched) ++gp; + uint64_t va = 0; for (auto& vp : g_vram_patches) va += vp.applied; + size_t mp = 0; for (auto& m : g_msg_inplace) if (m.patched) ++mp; + return std::snprintf(out, cap, + "{\"lang\":\"%s\",\"apply\":%s,\"capture\":%s,\"table_entries\":%zu," + "\"distinct_captured\":%zu,\"calls\":%llu,\"hits\":%llu," + "\"glyph_labels\":%zu,\"glyph_patched\":%zu,\"glyph_pending\":%d," + "\"vram_patches\":%zu,\"vram_applied\":%llu," + "\"msg_inplace\":%zu,\"msg_inplace_patched\":%zu,\"msg_inplace_pending\":%d}", + g_lang.c_str(), g_apply_armed.load() ? "true" : "false", + g_capture_on.load() ? "true" : "false", + g_table.size(), g_inv.size(), + (unsigned long long)g_calls.load(), (unsigned long long)g_hits.load(), + g_glyph_labels.size(), gp, g_glyph_pending.load(), + g_vram_patches.size(), (unsigned long long)va, + g_msg_inplace.size(), mp, g_msg_inplace_pending.load()); + } + + // vpatch: force-apply every VRAM-strip patch against current VRAM content + // (verify still gates each one) and report per-patch status. Lets a freshly + // reloaded patch be exercised without waiting for the game to re-upload. + if (sc == "vpatch") { + int n = g_apply_armed.load() ? vram_patches_apply_locked(0, 0, 0, 0, true) : 0; + int w = 0; + w += std::snprintf(out + w, cap - w, "{\"applied_now\":%d,\"patches\":[", n); + bool first = true; + for (auto& vp : g_vram_patches) { + if (w > cap - 160) break; + if (!first) w += std::snprintf(out + w, cap - w, ","); + first = false; + w += std::snprintf(out + w, cap - w, + "{\"x\":%d,\"y\":%d,\"w\":%d,\"h\":%d,\"applied\":%llu}", + vp.x, vp.y, vp.w, vp.h, (unsigned long long)vp.applied); + } + w += std::snprintf(out + w, cap - w, "]}"); + return w; + } + + // glyph: per-label patch status (JP addr -> patched? + target). Observability + // for the RAM source-patch layer. + if (sc == "glyph") { + int w = 0; + w += std::snprintf(out + w, cap - w, "["); + bool first = true; + for (auto& gl : g_glyph_labels) { + if (w > cap - 256) break; + if (!first) w += std::snprintf(out + w, cap - w, ","); + first = false; + w += std::snprintf(out + w, cap - w, + "{\"addr\":\"%08x\",\"width\":%u,\"patched\":%s,\"en\":\"%s\"}", + gl.addr, gl.width, gl.patched ? "true" : "false", gl.target.c_str()); + } + w += std::snprintf(out + w, cap - w, "]"); + return w; + } + + // dump / todo: JSON array of captured records. + const bool todo_only = (sc == "todo"); + int w = 0; + w += std::snprintf(out + w, cap - w, "["); + bool first = true; + for (auto& kv : g_inv) { + const CapRec& r = kv.second; + if (todo_only && r.translated) continue; + if (w > cap - 512) break; // leave room; truncate cleanly + if (!first) w += std::snprintf(out + w, cap - w, ","); + first = false; + w += std::snprintf(out + w, cap - w, + "{\"hash\":\"%016llx\",\"addr\":\"%08x\",\"pc\":\"%08x\",\"ra\":\"%08x\"," + "\"len\":%u,\"count\":%llu,\"term\":\"%s\",\"xl\":%s,\"hex\":\"", + (unsigned long long)kv.first, r.addr, r.pc, r.ra, + (unsigned)r.full_len, (unsigned long long)r.count, + r.term == Term::FFFF ? "ffff" : "nul", + r.translated ? "true" : "false"); + for (uint32_t i = 0; i < r.sample_len && w < cap - 4; ++i) + w += std::snprintf(out + w, cap - w, "%02x", r.bytes[i]); + w += std::snprintf(out + w, cap - w, "\"}"); + } + w += std::snprintf(out + w, cap - w, "]"); + return w; +} diff --git a/runtime/src/traps.c b/runtime/src/traps.c index d6b200f5..0285932f 100644 --- a/runtime/src/traps.c +++ b/runtime/src/traps.c @@ -348,6 +348,34 @@ static uint32_t psx_restore_context_from_tcb(CPUState* cpu, uint32_t tcb) return cpu->gpr[26]; } +/* Deferred cooperative thread switch (Ape Escape memcard fix #2). + * + * A genuine in-exception ChangeThread (interrupts.c kind-30) must be honored at + * the OUTERMOST dispatch boundary, never mid-nested-host-dispatch: only at the + * outermost boundary is the interrupted thread's guest state fully materialized + * in CPUState (cpu->pc == the true block PC, GPRs live) rather than split across + * a stale resume-PC latch + the host C call stack. Switching mid-nest saves a + * poisoned TCB context (latched PC ahead of the live registers) → the resumed + * thread dispatches a jumptable with a smeared index → misaligned DISPATCH FATAL. + * + * These thin exports let the interrupt path (interrupts.c) re-save the deferred + * thread cleanly and re-point PCB[0] at the outermost boundary, using the same + * TCB layout the scheduler save/restore already owns. */ +void psx_sched_save_context(CPUState* cpu, uint32_t tcb, uint32_t resume_pc) +{ + psx_save_context_to_tcb(cpu, tcb, resume_pc); +} + +void psx_sched_set_current_tcb(CPUState* cpu, uint32_t tcb) +{ + psx_set_current_tcb(cpu, tcb); +} + +uint32_t psx_sched_current_tcb(CPUState* cpu) +{ + return psx_current_tcb_ptr(cpu); +} + typedef struct HostThreadFiber { uint32_t tcb; psx_fiber_t fiber; @@ -565,6 +593,13 @@ static int psx_change_thread_fiber(CPUState* cpu, uint32_t target_tcb) if (g_pending_exception_longjmp && psx_fiber_current() == g_exception_owner_fiber) { int code = g_pending_exception_longjmp; g_pending_exception_longjmp = 0; + /* Same shadow-frame hardening as deferred_exception_longjmp: this is + * the deferred-honor site — the actual unwind of the OWNER fiber's + * frames happens here, so a live shadow frame on this stack must have + * its globals restored before the jump. */ + extern uint64_t g_exc_setjmp_epoch; + extern void overlay_loader_shadow_escape_fixup(uint64_t target_epoch); + overlay_loader_shadow_escape_fixup(g_exc_setjmp_epoch); longjmp(exception_jmpbuf, code); } @@ -707,6 +742,11 @@ void psx_scheduler_run(CPUState* cpu) * with in_exception==0), so interrupt state needs no fixup here. */ g_psx_dispatch_depth = 0; g_psx_call_bail = 0; + /* A longjmp-out through a nested call unit (overlay_loader_call_native + * or dispatch_nonlocal_call) would skip its depth restore; clear the + * nested-unit gate so IRQ checks are not wedged-off after the escape + * (backstop for the Ape memcard native<->interp fix). */ + { extern int g_call_unit_depth; g_call_unit_depth = 0; } } switch (g_sched_escape.reason) { diff --git a/runtime/tests/test_chain_handler_check.exe b/runtime/tests/test_chain_handler_check.exe deleted file mode 100644 index 55d7e4b0..00000000 Binary files a/runtime/tests/test_chain_handler_check.exe and /dev/null differ diff --git a/runtime/tests/test_frame_pacing.exe b/runtime/tests/test_frame_pacing.exe deleted file mode 100644 index deec31ca..00000000 Binary files a/runtime/tests/test_frame_pacing.exe and /dev/null differ diff --git a/runtime/tests/test_sio_card_protocol.exe b/runtime/tests/test_sio_card_protocol.exe deleted file mode 100644 index dda16929..00000000 Binary files a/runtime/tests/test_sio_card_protocol.exe and /dev/null differ diff --git a/tools/_options_session/_analyze_baseline.py b/tools/_options_session/_analyze_baseline.py deleted file mode 100644 index 2226441f..00000000 --- a/tools/_options_session/_analyze_baseline.py +++ /dev/null @@ -1,109 +0,0 @@ -"""Analyze the captured options_baseline_*.json snapshot. - -Goal: characterize the divergence between runtime (stuck in OPTIONS-black, -constantly rebuilding display list) and beetle (stable OPTIONS, only -touching state cells). - -For each side, group the wtrace_all entries by (addr, pc, ra) and -report counts. Tells us: - - which PCs are writing the OPTIONS-area cells on each backend - - whether the same PCs exist on both (= same code, wrong data) or - different PCs (= different code paths entirely) -""" -from __future__ import annotations -import sys, os, json, glob -from collections import defaultdict - -if len(sys.argv) > 1: - path = sys.argv[1] -else: - out_dir = os.path.join(os.environ.get("TEMP", "/tmp"), "optsess") - matches = sorted(glob.glob(os.path.join(out_dir, "options_baseline_*.json"))) - if not matches: - print("no snapshot found", file=sys.stderr); sys.exit(1) - path = matches[-1] -print(f"using {path}") - -with open(path) as f: - snap = json.load(f) - - -def hx(v): - if isinstance(v, str): return int(v, 16) if v.lower().startswith("0x") else int(v) - return v - - -def analyze_side(side_name, side): - print(f"\n{'='*70}\n {side_name.upper()}\n{'='*70}") - print(f" frame={side['ping']['frame']}") - if "freeze_check" in side: - fc = side["freeze_check"] - if isinstance(fc, dict) and fc.get("ok"): - print(f" freeze: current_func={fc.get('current_func')} " - f"last_store_pc={fc.get('last_store_pc')} " - f"in_exc={fc.get('in_exception')} " - f"exc_reentry={fc.get('exception_reentry_blocks')}") - - # OPTIONS-area cluster (filtered) - dmp = side["wtrace_all_dump_optstate"] - entries = dmp.get("entries", []) - print(f"\n optstate writes in ring: {len(entries)} (filter 0x80090C00..0x80091000)") - if not entries: return - - # Group by addr (which cells are written most) - by_addr = defaultdict(int) - for e in entries: - by_addr[e["addr"]] += 1 - top_addr = sorted(by_addr.items(), key=lambda x: -x[1])[:12] - print(f"\n top 12 addrs (count of writes):") - for a, c in top_addr: - print(f" {a} {c}") - - # Group by (pc, ra) — who is writing - by_pcra = defaultdict(int) - for e in entries: - by_pcra[(e["pc"], e["ra"])] += 1 - top_pcra = sorted(by_pcra.items(), key=lambda x: -x[1])[:10] - print(f"\n top 10 (pc, ra) (count):") - for (pc, ra), c in top_pcra: - print(f" pc={pc} ra={ra} {c}") - - # Distinct PCs (writers) and distinct RAs (callers of writers) - distinct_pcs = sorted({e["pc"] for e in entries}) - distinct_ras = sorted({e["ra"] for e in entries}) - print(f"\n distinct write PCs: {len(distinct_pcs)}") - print(f" distinct write RAs (callers): {len(distinct_ras)}") - print(f" first 6 distinct PCs: {distinct_pcs[:6]}") - print(f" first 6 distinct RAs: {distinct_ras[:6]}") - - # Frame range covered in this window - frames = sorted({e["frame"] for e in entries}) - print(f"\n frame range in ring: [{min(frames)}, {max(frames)}] ({len(frames)} distinct frames)") - - # Unfiltered tail — where IS the rest of the activity? - tail = side["wtrace_all_dump_tail"] - tail_entries = tail.get("entries", []) - if tail_entries: - # Top 8 addr bins (granularity: 256-byte buckets) - bucket = defaultdict(int) - for e in tail_entries: - a = hx(e["addr"]) - bucket[a & ~0xFF] += 1 - top_bucket = sorted(bucket.items(), key=lambda x: -x[1])[:8] - print(f"\n top 8 256-byte buckets in unfiltered tail of {len(tail_entries)} entries:") - for a, c in top_bucket: - print(f" 0x{a:08X}-0x{a+0xFF:08X} {c}") - - -analyze_side("runtime", snap["runtime"]) -analyze_side("beetle", snap["beetle"]) - -# Cross-side: are the same PCs writing the same cells on both? -r_pc_set = {e["pc"] for e in snap["runtime"]["wtrace_all_dump_optstate"]["entries"]} -b_pc_set = {e["pc"] for e in snap["beetle"]["wtrace_all_dump_optstate"]["entries"]} -print(f"\n{'='*70}\n CROSS-SIDE\n{'='*70}") -print(f" runtime distinct PCs writing optstate: {len(r_pc_set)}") -print(f" beetle distinct PCs writing optstate: {len(b_pc_set)}") -print(f" intersection: {len(r_pc_set & b_pc_set)}") -print(f" runtime-only PCs (first 10): {sorted(r_pc_set - b_pc_set)[:10]}") -print(f" beetle-only PCs (first 10): {sorted(b_pc_set - r_pc_set)[:10]}") diff --git a/tools/_options_session/_analyze_dump.py b/tools/_options_session/_analyze_dump.py deleted file mode 100644 index e869af69..00000000 --- a/tools/_options_session/_analyze_dump.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Quick analysis of a freeze auto-dump file.""" -import json, sys -from collections import Counter - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -print(f'== Dump: {path}') -print(f' wall_clock_epoch={d["wall_clock_epoch"]}') -print(f' wedge={d["wedge_kind_name"]} frame={d["frame_count"]} ' - f'exc_re={d["exception_reentry_blocks"]} ' - f'dispatch_count={d["dispatch_count"]}') -print(f' in_exc={d["in_exception"]} cur_fn={d["current_func"]} ' - f'last_store_pc={d["last_store_pc"]}') -print(f' i_stat={d["i_stat"]} i_mask={d["i_mask"]} ' - f'sio_card_active={d["sio_card_active"]} mc_max={d["mc_max_state"]}') - -print() -print(f'== wtrace_all: {len(d["wtrace_all"])} entries') -addr_c = Counter(e['addr'] for e in d['wtrace_all']) -pc_c = Counter(e['pc'] for e in d['wtrace_all']) -print('Top 15 ADDRS:') -for a, c in addr_c.most_common(15): - print(f' {a}: {c}') -print('Top 15 PCs:') -for p, c in pc_c.most_common(15): - print(f' {p}: {c}') - -print() -print(f'== fn_entry: {len(d["fn_entry"])} entries') -fn_c = Counter(e['func'] for e in d['fn_entry']) -print('Top 10 funcs entered:') -for f_, c in fn_c.most_common(10): - print(f' {f_}: {c}') - -print() -print(f'== restore_trace: {len(d["restore_trace"])} entries') -if d['restore_trace']: - kind_c = Counter(e['name'] for e in d['restore_trace']) - print('Kinds:', dict(kind_c)) - pc_c2 = Counter(e['cpu_pc'] for e in d['restore_trace']) - print('Top 10 cpu_pcs:') - for p, c in pc_c2.most_common(10): - print(f' {p}: {c}') - -print() -print(f'== thread_trace: {len(d["thread_trace"])} entries') -if d['thread_trace']: - kind_c = Counter(e['name'] for e in d['thread_trace']) - print('Kinds:', dict(kind_c)) - -print() -print(f'== sio_pc_trace: {len(d["sio_pc_trace"])} entries') -if d['sio_pc_trace']: - addr_c = Counter(e['addr'] for e in d['sio_pc_trace']) - print('Top 5 SIO addrs:') - for a, c in addr_c.most_common(5): - print(f' {a}: {c}') - -print() -print(f'== frame_history: {len(d["frame_history"])} frames') -if d['frame_history']: - first = d['frame_history'][0] - last = d['frame_history'][-1] - print(f' first frame {first["frame"]}: sr={first["sr"]} i_stat={first["i_stat"]} pad={first["pad"]} disp={first["disp"]}') - print(f' last frame {last["frame"]}: sr={last["sr"]} i_stat={last["i_stat"]} pad={last["pad"]} disp={last["disp"]}') - # see if display went off - off_count = sum(1 for f in d['frame_history'] if f['disp']['off']) - print(f' display_off frames: {off_count}/{len(d["frame_history"])}') diff --git a/tools/_options_session/_arm_init_watch.py b/tools/_options_session/_arm_init_watch.py deleted file mode 100644 index aa58afae..00000000 --- a/tools/_options_session/_arm_init_watch.py +++ /dev/null @@ -1,51 +0,0 @@ -"""Arm watchpoints on the init-target addresses (0x8009B3B4 = +0x14) on both -backends. Then we poll for a few seconds to catch any writes happening in -the OPTIONS state.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -ADDR = 0x8009B3B4 # struct base + 0x14 - -# DuckStation (port 4371) uses mem_break + mem_hit_last -print("=== DuckStation (port 4371) ===") -call(4371, 'mem_hit_clear') -r = call(4371, 'mem_break', addr=f'0x{ADDR:08X}') -print(f" mem_break: {r}") - -# Runtime (port 4470) uses wtrace_arm -print() -print("=== Runtime (port 4470) ===") -r = call(4470, 'wtrace_arm', lo=f'0x{(ADDR & 0x1FFFFFFF):08X}', hi=f'0x{(ADDR & 0x1FFFFFFF) + 4:08X}') -print(f" wtrace_arm: {r}") - -# Poll for 5 seconds -print() -print("--- Polling for 5 seconds (both backends still at OPTIONS) ---") -end = time.time() + 5 -duck_hits = 0 -while time.time() < end: - h = call(4371, 'mem_hit_last') - if h.get('valid'): - duck_hits += 1 - print(f" DUCK HIT: pc={h.get('pc')} ra={h.get('ra')} mem_addr={h.get('mem_addr')}") - call(4371, 'mem_hit_clear') - time.sleep(0.1) - -print(f"\nDuckStation: {duck_hits} hits on 0x{ADDR:08X}") -# Now dump runtime's wtrace -rt = call(4470, 'wtrace_dump', count=50) -es = rt.get('entries', []) -matching = [e for e in es if int(e.get('addr','0'), 16) == ADDR] -print(f"Runtime: {len(matching)} entries matching 0x{ADDR:08X}") -for e in matching[:10]: - print(f" RT HIT: seq={e.get('seq')} pc={e.get('pc')} new={e.get('new')} fr={e.get('frame')}") diff --git a/tools/_options_session/_b9d0_writes.py b/tools/_options_session/_b9d0_writes.py deleted file mode 100644 index 55a378d2..00000000 --- a/tools/_options_session/_b9d0_writes.py +++ /dev/null @@ -1,33 +0,0 @@ -"""Search for any writes to the polled flag region 0xB9D0..0xB9E4.""" -import json, sys - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -LO = 0xB9C0 -HI = 0xB9F0 - -hits = [] -for e in d['wtrace_all']: - a = int(e['addr'], 16) - if LO <= a < HI: - hits.append(e) - -print(f'Writes in [{hex(LO)}..{hex(HI)}): {len(hits)} of {len(d["wtrace_all"])} total') -for e in hits[:30]: - print(f' seq={e["seq"]} addr={e["addr"]} new={e["new"]} pc={e["pc"]} ra={e["ra"]} frame={e["frame"]}') - -# Also check broader kernel-data scan to find what IS being written nearby -print() -print('--- broader scan 0xB000..0xC800 ---') -addr_c = {} -for e in d['wtrace_all']: - a = int(e['addr'], 16) - if 0xB000 <= a < 0xC800: - addr_c[e['addr']] = addr_c.get(e['addr'], 0) + 1 -print(f'Total in 0xB000..0xC800: {sum(addr_c.values())}') -for a, c in sorted(addr_c.items(), key=lambda x: -x[1])[:20]: - print(f' {a}: {c}') diff --git a/tools/_options_session/_beetle_diff.py b/tools/_options_session/_beetle_diff.py deleted file mode 100644 index 49ce8c3c..00000000 --- a/tools/_options_session/_beetle_diff.py +++ /dev/null @@ -1,83 +0,0 @@ -"""Diff critical memory addresses between runtime (4470) and beetle (4380) -to find where Tomba's state machine diverges in OPTIONS-black.""" -import sys, struct -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -def read_word(port, addr): - try: - return _dbg.read_word(port, addr) - except Exception as e: - return f"ERR: {e}" - -def read_bytes(port, addr, n): - try: - r = _dbg.call(port, 'read_ram', addr=f'0x{addr:08X}', len=n, timeout=4) - if not r.get('ok'): return f"ERR: {r}" - return bytes.fromhex(r['hex']) - except Exception as e: - return f"ERR: {e}" - -# Ping both -p_rt = _dbg.call(4470, "ping", timeout=2) -p_be = _dbg.call(4380, "ping", timeout=2) -print(f"runtime fr={p_rt.get('frame','?')} | beetle fr={p_be.get('frame','?')}") -print() - -# Critical addresses to compare -print(f"{'addr':>12} {'runtime':>12} {'beetle':>12} {'match':>6}") -addrs = [ - (0x1F8001D4, "scratchpad+0x1D4 (state-struct ptr)"), - (0x80097520, "fn ptr table+0"), - (0x80097524, "fn ptr table+4 (state-0 helper)"), - (0x80097528, "fn ptr table+8"), - (0x8009752C, "fn ptr table+C"), - (0x8009B3A0, "state-0 helper return ptr (first word)"), - (0x8009B3D4, "*** mem[ptr+0x34] high half check"), - (0x8009EB58, "side-path read"), -] -for addr, label in addrs: - rt = read_word(4470, addr) - be = read_word(4380, addr) - if isinstance(rt, int) and isinstance(be, int): - m = "OK" if rt == be else "*DIFF*" - print(f" 0x{addr:08X} 0x{rt:08X} 0x{be:08X} {m} {label}") - else: - print(f" 0x{addr:08X} {rt} {be} ? {label}") - -# Compare the full state struct (256 bytes from runtime's ptr) -print() -print("=== Full state struct comparison ===") -rt_ptr = read_word(4470, 0x1F8001D4) -be_ptr = read_word(4380, 0x1F8001D4) -print(f"runtime ptr = 0x{rt_ptr:08X}, beetle ptr = 0x{be_ptr:08X}") -if rt_ptr == be_ptr and isinstance(rt_ptr, int): - rt_buf = read_bytes(4470, rt_ptr, 256) - be_buf = read_bytes(4380, be_ptr, 256) - if isinstance(rt_buf, bytes) and isinstance(be_buf, bytes): - for i in range(0, 256, 16): - rt_row = rt_buf[i:i+16] - be_row = be_buf[i:i+16] - if rt_row != be_row: - diff = ' '.join('!!' if rt_row[j] != be_row[j] else '..' for j in range(16)) - print(f" +0x{i:03X} RT: {' '.join(f'{x:02x}' for x in rt_row)}") - print(f" BE: {' '.join(f'{x:02x}' for x in be_row)}") - print(f" {diff}") - print() - -# Compare 0x8009B3A0 helper return struct too -print() -print("=== 0x8009B3A0 helper-return struct (256 bytes) ===") -rt_buf = read_bytes(4470, 0x8009B3A0, 256) -be_buf = read_bytes(4380, 0x8009B3A0, 256) -if isinstance(rt_buf, bytes) and isinstance(be_buf, bytes): - any_diff = False - for i in range(0, 256, 16): - rt_row = rt_buf[i:i+16] - be_row = be_buf[i:i+16] - if rt_row != be_row: - any_diff = True - print(f" +0x{i:03X} RT: {' '.join(f'{x:02x}' for x in rt_row)}") - print(f" BE: {' '.join(f'{x:02x}' for x in be_row)}") - if not any_diff: - print(" (IDENTICAL)") diff --git a/tools/_options_session/_bmp2png.py b/tools/_options_session/_bmp2png.py deleted file mode 100644 index f966ab84..00000000 --- a/tools/_options_session/_bmp2png.py +++ /dev/null @@ -1,7 +0,0 @@ -"""Convert one or more BMPs (cmd-line args) into PNGs of the same name.""" -from PIL import Image -import sys -for f in sys.argv[1:]: - out = f.rsplit('.', 1)[0] + '.png' - Image.open(f).convert('RGB').save(out) - print(f"{f} -> {out}") diff --git a/tools/_options_session/_capture_now.py b/tools/_options_session/_capture_now.py deleted file mode 100644 index b9c44eda..00000000 --- a/tools/_options_session/_capture_now.py +++ /dev/null @@ -1,56 +0,0 @@ -"""Capture live state on whichever backend is reachable, RIGHT NOW. - -Reads: - - dispatch miss totals (runtime only) - - frame counters - - the two probe cells (0x80090CAC, 0x80090DA0) on both - - surrounding range 0x80090C80..0x80090DE0 on both - - hex diff of the surrounding range - -No probes are armed — this is a pure ring/state read. Always-on rings -are preserved; we only read what's already captured. -""" -from __future__ import annotations -import sys, os -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from _dbg import call, read_block, read_word, RUNTIME_PORT, BEETLE_PORT - -PROBE_CELLS = [0x80090CAC, 0x80090DA0] -RANGE_LO = 0x80090C80 -RANGE_HI = 0x80090DE0 - -def safe(port, cmd, **kw): - try: return call(port, cmd, **kw) - except Exception as e: return {"error": str(e)} - -print("=== ping / frame / dispatch ===") -rp = safe(RUNTIME_PORT, "ping") -bp = safe(BEETLE_PORT, "ping") -print(f"runtime: {rp}") -print(f"beetle : {bp}") - -print("\n=== probe cells ===") -print(f"{'addr':<14} {'runtime':<12} {'beetle':<12} {'eq?':<4}") -for a in PROBE_CELLS: - try: rv = read_word(RUNTIME_PORT, a) - except Exception as e: rv = -1; rv_err = str(e) - try: bv = read_word(BEETLE_PORT, a) - except Exception as e: bv = -1; bv_err = str(e) - eq = "Y" if rv == bv else "N" - print(f"0x{a:08X} 0x{rv:08X} 0x{bv:08X} {eq}") - -print(f"\n=== range 0x{RANGE_LO:08X}..0x{RANGE_HI:08X} (diff only) ===") -try: - r = read_block(RUNTIME_PORT, RANGE_LO, RANGE_HI - RANGE_LO) - b = read_block(BEETLE_PORT, RANGE_LO, RANGE_HI - RANGE_LO) - n_diff = 0 - print(f"{'addr':<12} runtime beetle") - for i in range(0, len(r), 4): - rw = int.from_bytes(r[i:i+4], "little") - bw = int.from_bytes(b[i:i+4], "little") - if rw != bw: - n_diff += 1 - print(f"0x{RANGE_LO+i:08X} {rw:08X} {bw:08X}") - print(f"\n{n_diff} divergent words out of {(RANGE_HI-RANGE_LO)//4}") -except Exception as e: - print(f"range read failed: {e}") diff --git a/tools/_options_session/_capture_options_baseline.py b/tools/_options_session/_capture_options_baseline.py deleted file mode 100644 index 0b90bda9..00000000 --- a/tools/_options_session/_capture_options_baseline.py +++ /dev/null @@ -1,86 +0,0 @@ -"""Capture beetle's OPTIONS-screen baseline + runtime's current state. - -After this, when the user presses OPTIONS on runtime (with delay) we -can query the SAME windows on runtime and diff against this baseline. -No racing — always-on rings give us retroactive visibility. -""" -from __future__ import annotations -import sys, os, json, time -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from _dbg import call, RUNTIME_PORT, BEETLE_PORT - - -def safe(port, cmd, **kw): - try: return call(port, cmd, timeout=20, **kw) - except Exception as e: return {"error": str(e)} - - -def capture(port, label): - out = {"label": label, "port": port, "ts": time.time()} - p = safe(port, "ping") - out["ping"] = p - out["history"] = safe(port, "history") - out["wtrace_all_stats"] = safe(port, "wtrace_all_stats") - - # Pull the entire always-on ring (just the OPTIONS area writes via filter) - # Cluster around 0x80090C00..0x80091000 — wider net than first probe. - out["wtrace_all_dump_optstate"] = safe(port, "wtrace_all_dump", - addr_lo="0x80090C00", addr_hi="0x80091000", count=2048) - - # Also full last 4K writes (unfiltered) so we can mine for unexpected cells. - out["wtrace_all_dump_tail"] = safe(port, "wtrace_all_dump", count=4096) - - h = out["history"] - if h.get("ok"): - newest = h["newest"] - start = max(0, newest - 199) - out["frame_range_start"] = start - out["frame_range_end"] = newest - out["frame_timeseries"] = safe(port, "frame_timeseries", start=start, end=newest) - out["get_frame_newest"] = safe(port, "get_frame", frame=newest) - - if label == "runtime": - out["freeze_check"] = safe(port, "freeze_check", window=512) - return out - - -print("Capturing OPTIONS baseline (beetle = good state; runtime = whatever it is now)…") -snap = { - "ts": time.strftime("%Y-%m-%dT%H:%M:%S"), - "purpose": "OPTIONS-screen baseline (beetle), runtime current-state", - "runtime": capture(RUNTIME_PORT, "runtime"), - "beetle": capture(BEETLE_PORT, "beetle"), -} - -out_dir = os.path.join(os.environ.get("TEMP", "/tmp"), "optsess") -os.makedirs(out_dir, exist_ok=True) -path = os.path.join(out_dir, f"options_baseline_{int(time.time())}.json") -with open(path, "w") as f: - json.dump(snap, f, indent=2) -print(f"wrote {path} ({os.path.getsize(path)} bytes)") - -# Summary -for side in ("runtime", "beetle"): - s = snap[side] - pg = s["ping"] - wa = s["wtrace_all_stats"] - ws = s["wtrace_all_dump_optstate"] - print(f"\n=== {side} ===") - print(f" frame={pg.get('frame')}") - print(f" wtrace_all total={wa.get('total')} cap={wa.get('capacity')}") - print(f" optstate-range writes in ring: {ws.get('emitted', 0)} of {ws.get('available', 0)} ring entries") - # Distinct addrs in optstate window - entries = ws.get("entries", []) - distinct = sorted({e["addr"] for e in entries}) - print(f" distinct optstate addrs: {len(distinct)}") - if distinct: - print(f" first 8: {distinct[:8]}") - -# Live read of the probe cells we care about -print("\n=== live read of OPTIONS-probe cells ===") -def lr(port, addr): - r = safe(port, "read_ram", addr=addr, len=4) - return r.get("hex", "??") -for cell in ("0x80090CAC", "0x80090DA0", "0x80090DB8", "0x80090DA4", "0x80090DAC", "0x80090DB4"): - rv = lr(RUNTIME_PORT, cell); bv = lr(BEETLE_PORT, cell) - print(f" {cell} runtime={rv} beetle={bv} {'DIFF' if rv != bv else 'same'}") diff --git a/tools/_options_session/_capture_stuck.py b/tools/_options_session/_capture_stuck.py deleted file mode 100644 index 14d04a50..00000000 --- a/tools/_options_session/_capture_stuck.py +++ /dev/null @@ -1,75 +0,0 @@ -"""Capture the current divergence snapshot between runtime and beetle -when runtime is in a stuck/black-screen state. - -Reads from the rings we just built — no arming, no probes: - - always-on wtrace_all on BOTH (last 200 writes) - - per-frame ring last 200 frames on BOTH (CPU/display/IRQ over time) - - freeze_check on runtime (handler/dispatch/dirty diagnostics) - -Writes everything to /tmp/optsess/stuck_<timestamp>.json so we don't -keep re-querying as state churns. -""" -from __future__ import annotations -import sys, os, json, time -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from _dbg import call, RUNTIME_PORT, BEETLE_PORT - - -def safe(port, cmd, **kw): - try: return call(port, cmd, **kw, timeout=15) - except Exception as e: return {"error": str(e)} - - -def capture(port, label): - out = {"label": label, "port": port} - out["ping"] = safe(port, "ping") - out["history"] = safe(port, "history") - out["wtrace_all_stats"] = safe(port, "wtrace_all_stats") - out["wtrace_all_dump"] = safe(port, "wtrace_all_dump", count=512) - # Per-frame ring: pull last 200 frames - h = out["history"] - if h.get("ok"): - newest = h["newest"] - start = max(0, newest - 199) - out["frame_range_start"] = start - out["frame_range_end"] = newest - out["frame_timeseries"] = safe(port, "frame_timeseries", start=start, end=newest) - out["get_frame_newest"] = safe(port, "get_frame", frame=newest) - if label == "runtime": - out["freeze_check"] = safe(port, "freeze_check", window=512) - out["wtrace_ranges"] = safe(port, "wtrace_ranges") - return out - - -print("Capturing snapshot…") -snap = { - "ts": time.strftime("%Y-%m-%dT%H:%M:%S"), - "runtime": capture(RUNTIME_PORT, "runtime"), - "beetle": capture(BEETLE_PORT, "beetle"), -} - -out_dir = os.path.join(os.environ.get("TEMP", "/tmp"), "optsess") -os.makedirs(out_dir, exist_ok=True) -out_path = os.path.join(out_dir, f"stuck_{int(time.time())}.json") -with open(out_path, "w") as f: - json.dump(snap, f, indent=2) -print(f"wrote {out_path} ({os.path.getsize(out_path)} bytes)") - -# Print a one-screen summary -print(f"\nruntime: frame={snap['runtime']['ping']['frame']} disp_miss={snap['runtime']['ping']['dispatch_miss_total']}") -print(f"beetle : frame={snap['beetle']['ping']['frame']}") -fc = snap["runtime"]["freeze_check"] -if fc.get("ok"): - print(f"\nruntime freeze_check:") - for k in ("current_func", "last_store_pc", "i_stat", "i_mask", - "exception_reentry_blocks", "dirty_ram_blocks", "dispatch_count", - "frame_count", "in_exception", "sio_stat", "mc_max_state"): - if k in fc: print(f" {k:32s} = {fc[k]}") - -# Last 5 writes on each backend's catch-all -print(f"\nruntime wtrace_all newest 3:") -for e in snap["runtime"]["wtrace_all_dump"]["entries"][-3:]: - print(f" seq={e['seq']} addr={e['addr']} new={e['new']} pc={e['pc']} ra={e['ra']} fr={e['frame']}") -print(f"beetle wtrace_all newest 3:") -for e in snap["beetle"]["wtrace_all_dump"]["entries"][-3:]: - print(f" seq={e['seq']} addr={e['addr']} new={e['new']} pc={e['pc']} ra={e['ra']} fr={e['frame']}") diff --git a/tools/_options_session/_card_bytes.py b/tools/_options_session/_card_bytes.py deleted file mode 100644 index a102c83a..00000000 --- a/tools/_options_session/_card_bytes.py +++ /dev/null @@ -1,95 +0,0 @@ -"""Extract the card-data byte stream the BIOS received during OPTIONS-black. - -The kernel SIO byte ISR at 0xBFC16024 emits `sb v0,0x0(k0)` — every byte -the card delivers gets stored at k0 (which increments per byte). Filter -wtrace_all by that PC, sort by seq, and reconstruct the byte stream. -Compare to tomba.0.mcr / tomba.1.mcr. -""" -import json, os, sys -from collections import defaultdict - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' -mcr_root = 'F:/Projects/TombaRecomp/saves' - -with open(path) as f: - d = json.load(f) - -# Filter by the kernel-SIO-handler byte-store PC. The handler lives at -# BIOS 0xBFC16024, but the recompiler runs the RAM mirror at 0x00006524 -# (= 0xBFC16024 - 0xBFC10000 + 0x500). Either could show up depending -# on whether code came from ROM (BIOS path) or RAM-mirror (kernel). -STORE_PCS = ('0xBFC16024', '0x00006524', '0x06524') - -hits = [e for e in d['wtrace_all'] - if e['pc'].upper() in (p.upper() for p in STORE_PCS) - and e.get('w', 1) == 1] -print(f'BIOS card-byte stores: {len(hits)} hits') -if not hits: - sys.exit(1) - -hits.sort(key=lambda e: int(e['seq'])) - -# Group by contiguous bursts. Each card command reads N bytes consecutively -# into a buffer; bursts are separated by gaps in seq (handler not running). -bursts = [] -cur = [] -for i, e in enumerate(hits): - if cur and int(e['seq']) > int(cur[-1]['seq']) + 5000: - bursts.append(cur) - cur = [] - cur.append(e) -if cur: - bursts.append(cur) -print(f'Bursts: {len(bursts)}') -for i, b in enumerate(bursts): - addrs = [int(e['addr'], 16) for e in b] - print(f' Burst {i}: {len(b)} bytes, addr range ' - f'{hex(min(addrs))}..{hex(max(addrs))}, seq ' - f'{b[0]["seq"]}..{b[-1]["seq"]}') - -# Examine the most-recent (likely complete) burst. -b = bursts[-1] if bursts else [] -print(f'\n=== Last burst ({len(b)} bytes) ===') -print('First 32 entries:') -for e in b[:32]: - print(f' seq={e["seq"]} addr={e["addr"]} val=0x{int(e["new"], 16) & 0xFF:02X} frame={e["frame"]}') -print('...') -print('Last 16 entries:') -for e in b[-16:]: - print(f' seq={e["seq"]} addr={e["addr"]} val=0x{int(e["new"], 16) & 0xFF:02X} frame={e["frame"]}') - -# Bytes in the order they were stored. -recv = bytes(int(e['new'], 16) & 0xFF for e in b) -print(f'\nReceived stream hex (first 256 bytes):') -for i in range(0, min(256, len(recv)), 32): - print(f' {i:04x}: ' + ' '.join(f'{b:02x}' for b in recv[i:i+32])) - -# Compute the BIOS-style XOR checksum over the first 128 bytes. -if len(recv) >= 129: - xor = 0 - for byte in recv[:128]: - xor ^= byte - print(f'\nXOR of first 128 bytes: 0x{xor:02X}') - print(f'Byte at offset 128 (claimed checksum): 0x{recv[128]:02X}') - print(f'Match: {xor == recv[128]}') - -# Compare to the actual save files. -for fp in ('F:/Projects/TombaRecomp/saves/card1.mcd', - 'F:/Projects/TombaRecomp/saves/card2.mcd'): - fname = os.path.basename(fp) - if not os.path.exists(fp): - print(f'\n{fname}: not found') - continue - with open(fp, 'rb') as fh: - mcr = fh.read() - print(f'\n{fname}: {len(mcr)} bytes') - print('First 256 bytes:') - for i in range(0, 256, 32): - print(f' {i:04x}: ' + ' '.join(f'{b:02x}' for b in mcr[i:i+32])) - # See if our recv stream matches any window of the mcr. - if len(recv) > 32: - needle = recv[:32] - idx = mcr.find(needle) - print(f' Match of first 32 received bytes anywhere in {fname}: ' - f'{hex(idx) if idx >= 0 else "NO"}') diff --git a/tools/_options_session/_card_bytes2.py b/tools/_options_session/_card_bytes2.py deleted file mode 100644 index 4d23f7f3..00000000 --- a/tools/_options_session/_card_bytes2.py +++ /dev/null @@ -1,96 +0,0 @@ -"""Properly split the SIO byte stream into sector-sized groups. - -Each sector read fills the buffer at 0xBE48..0xBEC7 (128 bytes). k0 resets -when a new sector starts. Group by addr-decrease, then compare each -captured sector against the real card1.mcd content. -""" -import json, os, sys - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -STORE_PCS = {'0xBFC16024', '0x00006524'} -hits = [e for e in d['wtrace_all'] - if e['pc'].upper() in {p.upper() for p in STORE_PCS} - and e.get('w', 1) == 1] -hits.sort(key=lambda e: int(e['seq'])) -print(f'{len(hits)} byte-store events captured') - -# Group by sector: a new sector starts when the destination address -# decreases (or, more reliably, when it equals the lowest seen base). -addrs = [int(e['addr'], 16) for e in hits] -base = min(addrs) -print(f'Buffer base: 0x{base:04X}') - -sectors = [] -cur = [] -prev_addr = -1 -for e in hits: - a = int(e['addr'], 16) - if a < prev_addr or (a == base and cur): - sectors.append(cur) - cur = [] - cur.append(e) - prev_addr = a -if cur: - sectors.append(cur) - -print(f'Sectors identified: {len(sectors)}') -for i, s in enumerate(sectors): - addrs = [int(e['addr'], 16) for e in s] - print(f' Sector {i}: {len(s)} bytes, addr range 0x{min(addrs):04X}..0x{max(addrs):04X}, ' - f'frames {s[0]["frame"]}..{s[-1]["frame"]}') - -def sector_bytes(s): - """Reconstruct sector buffer from the stores (last write wins per addr).""" - buf = {} - for e in s: - buf[int(e['addr'], 16)] = int(e['new'], 16) & 0xFF - base = min(buf) - return bytes(buf[base + i] if (base + i) in buf else 0 - for i in range(max(buf) - base + 1)) - -# Pull real card data for comparison. -with open('F:/Projects/TombaRecomp/saves/card1.mcd', 'rb') as fh: - card1 = fh.read() - -# A PS1 memcard read typically starts at the directory (sector 0), then -# reads sectors 1..15. Each "frame" in the .mcd is 128 bytes. -for i, s in enumerate(sectors): - recv = sector_bytes(s) - print(f'\n=== Sector {i}: {len(recv)} bytes ===') - print('Received hex (full):') - for off in range(0, len(recv), 32): - print(f' {off:03x}: ' + ' '.join(f'{b:02x}' for b in recv[off:off+32])) - - # Try to find a window in card1.mcd that this matches. - if len(recv) >= 8: - for fr in range(0, 16): - real = card1[fr*128:(fr+1)*128] - if recv[:len(recv)] == real[:len(recv)]: - print(f' EXACT match: card1.mcd frame {fr}') - break - # Try matching the prefix - common = 0 - for j in range(min(len(recv), len(real))): - if recv[j] == real[j]: - common += 1 - else: - break - if common >= 8: - print(f' Prefix match with card1.mcd frame {fr}: {common}/{len(recv)} bytes') - # If mismatch starts mid-stream, show the divergence - if common < len(recv): - div = common - print(f' Divergence at offset 0x{div:02X}:') - print(f' received[{div}..{div+16}]: ' + ' '.join(f'{b:02x}' for b in recv[div:div+16])) - print(f' real [{div}..{div+16}]: ' + ' '.join(f'{b:02x}' for b in real[div:div+16])) - # See if received[div:] matches real[div+1:] (shifted by 1) - if div+1 < len(real): - shifted = real[div+1:div+1+min(16, len(recv)-div)] - match_shift = recv[div:div+len(shifted)] == shifted - print(f' SHIFT-BY-1 match: {match_shift}') - break diff --git a/tools/_options_session/_check_caller.py b/tools/_options_session/_check_caller.py deleted file mode 100644 index d3fe2cd7..00000000 --- a/tools/_options_session/_check_caller.py +++ /dev/null @@ -1,38 +0,0 @@ -"""Check if 0x80028B34 (caller of 0x80068DDC) fires on runtime. -If not, we go further upstream.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -# Arm fntrace on 0x80028B34 on runtime -call(4470, 'fntrace_arm', target='0x80028B34') -# Also arm 0x80068DDC (the immediate caller of init function) -call(4470, 'fntrace_arm', target='0x80068DDC') -# And 0x80022458 (caller of 0x80028B34) -call(4470, 'fntrace_arm', target='0x80022458') -# And 0x8006A0C0 itself -call(4470, 'fntrace_arm', target='0x8006A0C0') - -# Wait 3 sec -print("Waiting 3 sec for hits...") -time.sleep(3) - -# Dump -d = call(4470, 'fntrace_dump', count=500) -es = d.get('entries', []) -print(f"Total entries: {len(es)}") - -import collections -funcs = collections.Counter(e.get('target') for e in es) -print("Top hit counts:") -for f, c in funcs.most_common(10): - print(f" {f}: {c}") diff --git a/tools/_options_session/_check_checksum.py b/tools/_options_session/_check_checksum.py deleted file mode 100644 index 7c0a53c8..00000000 --- a/tools/_options_session/_check_checksum.py +++ /dev/null @@ -1,38 +0,0 @@ -"""Compute expected per-frame XOR for card1.mcd frames 0..3 and compare to -the running XOR captured in the dump trace (at 0x7560 after byte 127 store).""" -import json, sys - -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open('F:/Projects/TombaRecomp/saves/card1.mcd', 'rb') as fh: - card = fh.read() - -print(f"Card file: {len(card)} bytes ({len(card)//128} frames)") - -# Compute XOR over each frame for sectors 0..3 -for frame in range(0, 16): - fbytes = card[frame*128:(frame+1)*128] - msb = (frame >> 8) & 0xFF - lsb = frame & 0xFF - xor_bytes = 0 - for b in fbytes: - xor_bytes ^= b - expected = msb ^ lsb ^ xor_bytes - print(f" Frame {frame:2d}: MSB={msb:02X} LSB={lsb:02X} " - f"XOR(bytes 0..127)={xor_bytes:02X} " - f"expected_checksum_byte={expected:02X} " - f"on-disk-byte-127={fbytes[127]:02X} " - f"first_4_bytes={fbytes[0]:02X} {fbytes[1]:02X} {fbytes[2]:02X} {fbytes[3]:02X}") - -# Now find each sector's running XOR end value in the dump -with open(dump) as f: - d = json.load(f) - -print("\n\nRunning XOR end-of-sector values from trace (writes to 0x7560 from pc=0xBFC155AC):") -events = d['wtrace_all'] -ac_writes = [(i, e) for i, e in enumerate(events) - if e['pc'].upper() == '0XBFC155AC' and int(e['addr'], 16) == 0x7560] -for i, e in ac_writes: - val = int(e['new'], 16) & 0xFF - print(f" idx={i} seq={e['seq']} fr={e.get('frame')} 0x7560 -> 0x{val:02X}") diff --git a/tools/_options_session/_check_e6.py b/tools/_options_session/_check_e6.py deleted file mode 100644 index b294fa2e..00000000 --- a/tools/_options_session/_check_e6.py +++ /dev/null @@ -1,40 +0,0 @@ -"""Check mem[0x8009B3A0+0xE6] on both backends.""" -import socket, json -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -# Read 0x8009B486 (= struct base + 0xE6) halfword -# But read as word at 0x8009B484 to be safe -for port, name in [(4470, 'runtime'), (4371, 'duckstation')]: - r = call(port, 'read_ram', addr='0x8009B400', len=0x100) - if r.get('ok'): - b = bytes.fromhex(r['hex']) - # +0xE6 from struct base (0x8009B3A0) = byte at offset 0xE6 - # We're reading from 0x8009B400 = struct base + 0x60 - # So mem[+0xE6] = byte at offset (0xE6 - 0x60) = 0x86 in this buffer - off_in_buf = 0xE6 - 0x60 - v = (b[off_in_buf+1] << 8) | b[off_in_buf] # halfword LE - print(f"{name:>12} mem[0x8009B486] (struct+0xE6, halfword) = 0x{v:04X}") - # also +0x46 - off2 = 0x46 - 0x60 - if off2 < 0: - # need a different read - r2 = call(port, 'read_ram', addr='0x8009B3E6', len=1) - v46 = int(r2['hex'], 16) - print(f"{name:>12} mem[0x8009B3E6] (struct+0x46, byte) = 0x{v46:02X}") - else: - print(f"{name:>12} mem[+0x46] (byte) = 0x{b[off2]:02X}") - # Also dump 0x60..0x100 of buffer - print(f"{name:>12} struct[+0x60..+0xFF]:") - for i in range(0, len(b), 16): - line = ' '.join(f'{x:02x}' for x in b[i:i+16]) - print(f" +0x{0x60+i:03X}: {line}") - print() diff --git a/tools/_options_session/_check_init_fields.py b/tools/_options_session/_check_init_fields.py deleted file mode 100644 index 8e25a27a..00000000 --- a/tools/_options_session/_check_init_fields.py +++ /dev/null @@ -1,41 +0,0 @@ -"""Are the missing-init fields STATICALLY missing on runtime, or do they cycle? -Sample several offsets over 2 sec on both backends.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -def read_word(port, addr): - r = call(port, 'read_ram', addr=f'0x{addr:08X}', len=4) - return int.from_bytes(bytes.fromhex(r['hex']), 'little') if r.get('ok') else None - -# Critical offsets where runtime is suspected wrong -fields = [ - (0x8009B3A0, "+0x00 first pointer"), - (0x8009B3A4, "+0x04 second pointer"), - (0x8009B3A8, "+0x08 third pointer"), - (0x8009B3B4, "+0x14 fn ptr 1"), - (0x8009B3B8, "+0x18 fn ptr 2"), - (0x8009B3C0, "+0x20 pointer"), -] - -print(f"{'addr':>14} {'runtime':>40} {'duckstation':>40}") -print(f"{'':14} {'(5 samples)':>40} {'(5 samples)':>40}") -for addr, label in fields: - rt_vals = [read_word(4470, addr) for _ in range(5)] - du_vals = [read_word(4371, addr) for _ in range(5)] - rt_str = ','.join(f'{v:08X}' for v in rt_vals) - du_str = ','.join(f'{v:08X}' for v in du_vals) - rt_static = len(set(rt_vals)) == 1 - du_static = len(set(du_vals)) == 1 - print(f" 0x{addr:08X} RT[{'static' if rt_static else 'cyclic'}]: {rt_str}") - print(f" {'':10} DU[{'static' if du_static else 'cyclic'}]: {du_str} ({label})") - print() diff --git a/tools/_options_session/_check_init_path.py b/tools/_options_session/_check_init_path.py deleted file mode 100644 index 81f9cc2f..00000000 --- a/tools/_options_session/_check_init_path.py +++ /dev/null @@ -1,48 +0,0 @@ -"""FUN_8006A0C0 is the init that writes struct+0x14/0x18/0x20/0x46. -It bails early if mem[0x8009752C](a0) returns non-zero. -Check that fn ptr + other init-related fields.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(3); buf=b''; out=None; deadline=time.time()+4 - while time.time() < deadline: - try: chunk = s.recv(65536) - except socket.timeout: break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: obj = json.loads(line.decode()) - except: continue - if obj.get('id') == 1: out = obj; deadline = 0; break - s.close() - return out or {'err': 'no resp'} - -def rd32(port, addr): - r = call(port, 'read_ram', addr=f'0x{addr:08X}', len=4) - if 'hex' not in r: return None - return int.from_bytes(bytes.fromhex(r['hex']), 'little') - -for _ in range(5): call(4371, 'mem_hit_clear') - -# The bailout fn ptr -print("== mem[0x8009752C] (gate fn ptr — called by 0x8006A0C0) ==") -print(f" runtime: 0x{rd32(4470, 0x8009752C):08X}") -print(f" duck: 0x{rd32(4371, 0x8009752C):08X}") - -# Surrounding fn-ptr area at 0x80097500..0x80097540 -print("\n== mem[0x80097500..0x80097540] fn-ptr area ==") -for port, name in [(4470, 'runtime'), (4371, 'duck')]: - r = call(port, 'read_ram', addr='0x80097500', len=0x40) - if 'hex' in r: - b = bytes.fromhex(r['hex']) - print(f" {name}:") - for i in range(0, len(b), 16): - words = [int.from_bytes(b[i+k:i+k+4],'little') for k in range(0, 16, 4)] - print(f" +0x{i:02X}: {' '.join(f'{w:08X}' for w in words)}") - -# Check the struct's "is alloc'd" indicator -- looks like +0x10 is a "next ptr" or "header" -# Per the disasm of 0x8006A0C0, the gate fn was called with a0=struct. Maybe it's an -# allocator that returns 0 on success, nonzero on already-alloc'd diff --git a/tools/_options_session/_check_liveness.py b/tools/_options_session/_check_liveness.py deleted file mode 100644 index 85915445..00000000 --- a/tools/_options_session/_check_liveness.py +++ /dev/null @@ -1,17 +0,0 @@ -import socket, json -def call(port, cmd, **kw): - try: - s = socket.create_connection(('127.0.0.1', port), timeout=2) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - except Exception as e: - return {'err': str(e)} -for port, name in [(4470, 'runtime'), (4371, 'duck'), (4380, 'beetle')]: - r = call(port, 'ping') - print(f'{name:>8} (port {port}): {r}') diff --git a/tools/_options_session/_check_more_callers.py b/tools/_options_session/_check_more_callers.py deleted file mode 100644 index ea0e3e23..00000000 --- a/tools/_options_session/_check_more_callers.py +++ /dev/null @@ -1,42 +0,0 @@ -"""Check if 0x800223E0 fires on runtime, plus its callers 0x80019898 and 0x8001A5D4 -(actually, the FUNCTIONS containing those PCs).""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -# Arm fntrace at every level we care about -targets = [ - '0x800223E0', # function containing call to 0x80028B34 - '0x80019898', # caller PC of 0x800223E0 - '0x8001A5D4', # other caller PC of 0x800223E0 - '0x80028D70', # also called from 0x800223E0 — see if these run - '0x8006A0C0', # the init we know doesn't fire -] -for t in targets: - call(4470, 'fntrace_arm', target=t) - -print(f"Armed {len(targets)} targets. Waiting 5 sec...") -time.sleep(5) - -d = call(4470, 'fntrace_dump', count=500) -es = d.get('entries', []) -print(f"Total entries: {len(es)}") - -import collections -funcs = collections.Counter(e.get('target') for e in es) -print("Hit counts:") -for t in targets: - addr_int = int(t, 16) - matches = [k for k in funcs if int(k, 16) == addr_int] - n = sum(funcs[k] for k in matches) - label = '[' + t + ']' - print(f" {label:>14} hits: {n}") diff --git a/tools/_options_session/_compare_frames.py b/tools/_options_session/_compare_frames.py deleted file mode 100644 index b135645e..00000000 --- a/tools/_options_session/_compare_frames.py +++ /dev/null @@ -1,16 +0,0 @@ -"""Are card1.mcd frames 1..15 byte-identical? If yes, the 'frame 1' label -in _card_bytes2.py output may actually be frames 1, 2, 3, 4 etc. -Print full hex for each first-byte-differing pair.""" -with open('F:/Projects/TombaRecomp/saves/card1.mcd', 'rb') as fh: - card = fh.read() - -f1 = card[128:256] -for fr in range(0, 16): - fb = card[fr*128:(fr+1)*128] - # diff vs f1 - diffs = [(i, f1[i], fb[i]) for i in range(128) if f1[i] != fb[i]] - if diffs: - diff_str = ', '.join(f'@{i}(f1={a:02X},f{fr}={b:02X})' for i, a, b in diffs[:6]) - else: - diff_str = '(IDENTICAL to frame 1)' - print(f"Frame {fr:2d}: {diff_str}") diff --git a/tools/_options_session/_compare_gpu_activity.py b/tools/_options_session/_compare_gpu_activity.py deleted file mode 100644 index 717f78fd..00000000 --- a/tools/_options_session/_compare_gpu_activity.py +++ /dev/null @@ -1,45 +0,0 @@ -"""Compare GPU activity at OPTIONS between runtime and DuckStation. -Look at: frame rate, recent GPU commands.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while b'\n' not in buf: - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.partition(b'\n')[0].decode()) - -PORTS = [(4470, 'runtime'), (4371, 'duck')] - -# Frame rate -print("=== Frame rate over 2 sec ===") -for port, name in PORTS: - f0 = call(port, 'ping').get('frame', 0) - time.sleep(2) - f1 = call(port, 'ping').get('frame', 0) - print(f" {name}: {f0} -> {f1} (d={f1-f0} frames in 2s = {(f1-f0)/2:.1f} fps)") - -# probe gpu/display commands -print("\n=== Display state probes ===") -for port, name in PORTS: - print(f"--- {name} ---") - for cmd, kw in [ - ('gpu_stats', {}), - ('gpu_state', {}), - ('gpu_status', {}), - ('gpu_ring', {'count': 10}), - ('gp0_ring', {'count': 10}), - ('display_state', {}), - ('vram_stat', {}), - ('screenshot', {}), - ]: - try: - r = call(port, cmd, **kw) - txt = str(r)[:200] - print(f" {cmd}: {txt}") - except Exception as e: - print(f" {cmd}: EX {e}") diff --git a/tools/_options_session/_ctx_ring_analyze.py b/tools/_options_session/_ctx_ring_analyze.py deleted file mode 100644 index c3893187..00000000 --- a/tools/_options_session/_ctx_ring_analyze.py +++ /dev/null @@ -1,48 +0,0 @@ -"""Pull thread_ctx_ring from runtime at OPTIONS. -For each TCB, show the sequence of save -> restore -> save -> restore. -The 'resume_pc' field should EVOLVE if the thread does real work between yields. -If it's static (same PC every save), the thread is stuck.""" -import socket, json, collections - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4470), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -r = call('thread_ctx_ring', count=256) -es = r.get('entries', []) -print(f"Got {len(es)} entries (total={r.get('total')}, available={r.get('available')})") -print() - -# Group by TCB -per_tcb = collections.defaultdict(list) -for e in es: - per_tcb[e['tcb']].append(e) - -for tcb, entries in per_tcb.items(): - print(f"=== TCB {tcb} ({len(entries)} entries) ===") - saves = [e for e in entries if e['op'] == 'save'] - restores = [e for e in entries if e['op'] == 'restore'] - save_pcs = {e['resume_pc'] for e in saves} - restore_pcs = {e['resume_pc'] for e in restores} - print(f" saves: {len(saves)}; distinct resume_pc: {len(save_pcs)} --> {save_pcs}") - print(f" restores: {len(restores)}; distinct resume_pc: {len(restore_pcs)} --> {restore_pcs}") - - # also distinct sp / ra - save_sps = {e['sp'] for e in saves} - save_ras = {e['ra'] for e in saves} - print(f" save SPs: {save_sps}") - print(f" save RAs: {save_ras}") - print() - - # Show last 10 entries chronologically - print(f" Last 8 entries:") - for e in entries[-8:]: - print(f" seq={e['seq']:5} fr={e['frame']:5} {e['op']:>7} resume_pc={e['resume_pc']} sp={e['sp']} ra={e['ra']} sr={e['sr']} epc={e['epc']}") - print() diff --git a/tools/_options_session/_dbg.py b/tools/_options_session/_dbg.py deleted file mode 100644 index a2ae66f2..00000000 --- a/tools/_options_session/_dbg.py +++ /dev/null @@ -1,86 +0,0 @@ -"""Tiny TCP-debug-server client used by the OPTIONS-black session. - -Single source of truth for talking to: - - psx-runtime (TombaRecomp build) on port 4470 - - psx-beetle on port 4380 - -No game-specific knowledge here. Just request/response and command-name -adapters so the rest of the session scripts read the same way against -either backend, even though the wire verbs differ -(e.g. wtrace_add vs wtrace_arm). -""" -from __future__ import annotations -import json, socket, sys, time - -RUNTIME_PORT = 4470 -BEETLE_PORT = 4380 - - -def call(port: int, cmd: str, timeout: float = 30.0, **kw): - payload = {"id": 1, "cmd": cmd, **kw} - s = socket.create_connection(("127.0.0.1", port), timeout=timeout) - try: - s.sendall((json.dumps(payload) + "\n").encode()) - buf = bytearray() - while not buf.endswith(b"\n"): - chunk = s.recv(1 << 20) - if not chunk: - break - buf.extend(chunk) - finally: - s.close() - return json.loads(buf.decode("utf-8", "replace").strip() or "{}") - - -def hx(v): - if isinstance(v, int): - return v - if isinstance(v, str): - return int(v, 16) if v.lower().startswith("0x") else int(v) - return v - - -def read_word(port: int, addr: int) -> int: - """Little-endian word read at addr via read_ram.""" - r = call(port, "read_ram", addr=f"0x{addr:08X}", len=4) - if not r.get("ok"): - raise RuntimeError(f"read_ram failed at 0x{addr:08X}: {r}") - b = bytes.fromhex(r["hex"]) - return int.from_bytes(b, "little") - - -def read_block(port: int, addr: int, n: int) -> bytes: - out = bytearray() - off = 0 - while off < n: - chunk = min(0x40000, n - off) - r = call(port, "read_ram", addr=f"0x{addr+off:08X}", len=chunk) - if not r.get("ok"): - raise RuntimeError(f"read_ram failed at 0x{addr+off:08X}: {r}") - out.extend(bytes.fromhex(r["hex"])) - off += chunk - return bytes(out) - - -def ping_both(): - out = {} - for name, port in [("runtime", RUNTIME_PORT), ("beetle", BEETLE_PORT)]: - try: - out[name] = call(port, "ping", timeout=3) - except Exception as e: - out[name] = {"error": str(e)} - return out - - -if __name__ == "__main__": - if len(sys.argv) == 1 or sys.argv[1] == "ping": - for k, v in ping_both().items(): - print(f"{k:8} {v}") - sys.exit(0) - cmd = sys.argv[1] - port = int(sys.argv[2]) - args = {} - for a in sys.argv[3:]: - k, _, v = a.partition("=") - args[k] = v - print(json.dumps(call(port, cmd, **args), indent=2)) diff --git a/tools/_options_session/_decode_b0_callers.py b/tools/_options_session/_decode_b0_callers.py deleted file mode 100644 index 474c360c..00000000 --- a/tools/_options_session/_decode_b0_callers.py +++ /dev/null @@ -1,58 +0,0 @@ -"""Read MIPS at the B0 call sites to identify what BIOS function each calls. -The t1 register holds the B0 function index when calling B0 trampoline.""" -import sys, struct -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -call_sites = [0x800171F8, 0x800170BC, 0x8006825C] -# These are RA values (i.e., return-from-call PCs). The JAL instruction was at ra-8. - -def disasm_li(insn): - """Decode `li/addiu rt, rs, imm` or similar t1-loading instructions.""" - op = (insn >> 26) & 0x3F - rs = (insn >> 21) & 0x1F - rt = (insn >> 16) & 0x1F - imm = insn & 0xFFFF - if imm > 0x7FFF: - imm = imm - 0x10000 - funct = insn & 0x3F - if op == 0x9: # addiu - return f"addiu r{rt},r{rs},{imm:#x}" - if op == 0xD: # ori - return f"ori r{rt},r{rs},{imm:#x}" - if op == 0xF: # lui - return f"lui r{rt},{imm:#x}" - if op == 0: # SPECIAL - if funct == 9: - return f"jalr r{rs}" - if funct == 8: - return f"jr r{rs}" - return f"SPECIAL funct={funct:#x}" - if op == 0x3: # jal - target = (insn & 0x3FFFFFF) << 2 - return f"jal target_relative=0x{target:08x}" - if op == 0x2: # j - target = (insn & 0x3FFFFFF) << 2 - return f"j target_relative=0x{target:08x}" - return f"op={op:#x} insn={insn:08x}" - -for site in call_sites: - print(f"=== Call site (return-to PC) 0x{site:08X} ===") - print(f" JAL was at 0x{site-8:08X}") - # Read 8 instructions before site (32 bytes back) - base = site - 0x20 - r = _dbg.call(4470, "read_ram", addr=f"0x{base:08X}", len=0x30, timeout=3) - if not r.get("ok"): - print(f" read_ram failed: {r}") - continue - b = bytes.fromhex(r["hex"]) - for i in range(0, len(b), 4): - addr = base + i - if addr >= site + 0x4: break - insn = struct.unpack_from("<I", b, i)[0] - mark = "" - if addr == site-8: mark = " <-- JAL" - if addr == site-4: mark = " <-- delay slot" - if addr == site: mark = " <-- return point" - print(f" {addr:08X}: {insn:08x} {disasm_li(insn)}{mark}") - print() diff --git a/tools/_options_session/_diff_at_options.py b/tools/_options_session/_diff_at_options.py deleted file mode 100644 index 447df985..00000000 --- a/tools/_options_session/_diff_at_options.py +++ /dev/null @@ -1,85 +0,0 @@ -"""Compare runtime (4470) vs DuckStation (4371) at OPTIONS. -Handles scratchpad correctly on both sides. -Samples values over 3 seconds to spot static-vs-dynamic differences.""" -import socket, json, time, struct, sys - -def call(port, cmd, **kw): - try: - s = socket.create_connection(('127.0.0.1', port), timeout=3) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - except Exception as e: - return {'error': str(e)} - -def read_word(port, addr): - """Read a 32-bit LE word, handling scratchpad correctly on DuckStation.""" - is_scratchpad = (addr & 0xFFFFE000) == 0x1F800000 - if port == 4371 and is_scratchpad: - r = call(port, 'read_scratch', addr=f'0x{(addr & 0x3FF):04X}', len=4) - else: - r = call(port, 'read_ram', addr=f'0x{addr:08X}', len=4) - if 'error' in r or not r.get('ok'): - return None - return int.from_bytes(bytes.fromhex(r['hex']), 'little') - -def ping(port): - return call(port, 'ping') - -# Verify both alive -rt = ping(4470) -duck = ping(4371) -print(f"runtime: {rt}") -print(f"duckstation: {duck}") -print() - -if 'error' in rt or 'error' in duck: - print("Both backends must be running & at OPTIONS. Re-launch any that errored.") - sys.exit(1) - -# Static comparison — values that should NOT change frame to frame -print("=== STATIC values (these should match if both at same state) ===") -print(f"{'addr':>14} {'runtime':>12} {'duckstation':>12} {'match':>6}") -for addr, label in [ - (0x1F8001D4, "scratchpad[0x1D4] state struct ptr"), - (0x80097520, "fn ptr table+0"), - (0x80097524, "fn ptr table+4 (state-0 helper)"), - (0x80097528, "fn ptr table+8"), - (0x8009752C, "fn ptr table+C"), - (0x80000108, "kernel TCB list ptr"), - (0x80000110, "kernel TCBH"), - (0x80000120, "kernel EvCB ptr"), -]: - rt_v = read_word(4470, addr) - du_v = read_word(4371, addr) - if rt_v is None or du_v is None: - print(f" 0x{addr:08X} err err ? {label}") - continue - m = "OK" if rt_v == du_v else "DIFF" - print(f" 0x{addr:08X} 0x{rt_v:08X} 0x{du_v:08X} {m:>6} {label}") -print() - -# Dynamic comparison — sample 10 times, see which values CHANGE -print("=== DYNAMIC values (sample 10 frames; see who's animating) ===") -addrs = [ - (0x8009B3D4, "state-0 helper struct +0x34"), - (0x8009B3A0, "state-0 helper struct base"), -] - -for addr, label in addrs: - print(f"\n Address 0x{addr:08X} ({label}):") - rt_vals = [] - du_vals = [] - for i in range(10): - rt_vals.append(read_word(4470, addr)) - du_vals.append(read_word(4371, addr)) - time.sleep(0.05) - rt_unique = len(set(rt_vals)) - du_unique = len(set(du_vals)) - print(f" runtime values: {[hex(v) if v is not None else '?' for v in rt_vals]} ({rt_unique} unique --> {'FROZEN' if rt_unique == 1 else 'animating'})") - print(f" duck values: {[hex(v) if v is not None else '?' for v in du_vals]} ({du_unique} unique --> {'FROZEN' if du_unique == 1 else 'animating'})") diff --git a/tools/_options_session/_dirty_block_analysis.py b/tools/_options_session/_dirty_block_analysis.py deleted file mode 100644 index c51a07a9..00000000 --- a/tools/_options_session/_dirty_block_analysis.py +++ /dev/null @@ -1,66 +0,0 @@ -"""Analyze the new dirty_block ring from the freeze dump. -What interpreted RAM PCs dominate during the reentry storm? -""" -import json, sys, collections - -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778822828.json' - -with open(dump) as f: - d = json.load(f) - -print(f"=== Core ===") -print(f" backend: {d['backend']}") -print(f" wall_clock_epoch: {d['wall_clock_epoch']}") -print(f" frame_count: {d['frame_count']}") -print(f" psx_cycle_count: {d['psx_cycle_count']:,}") -print(f" in_exception: {d['in_exception']}") -print(f" current_func: {d['current_func']}") -print(f" last_store_pc: {d['last_store_pc']}") -print(f" exc_entries: {d['exception_entries']:,}") -print(f" exc_reentry_blocks: {d['exception_reentry_blocks']:,}") -print(f" dispatch_count: {d['dispatch_count']:,}") -print(f" dirty_ram_blocks: {d['dirty_ram_blocks']:,}") -print(f" dirty_ram_insns: {d['dirty_ram_insns']:,}") -print(f" wedge_kind: {d.get('wedge_kind','?')} ({d.get('wedge_kind_name','?')})") -print(f" i_stat: {d['i_stat']}") -print(f" i_mask: {d['i_mask']}") -print(f" mc_max_state: {d['mc_max_state']}") -print(f" sio_card_active: {d['sio_card_active']}") -print(f" sio_stat / sio_ctrl: {d['sio_stat']} / {d['sio_ctrl']}") -print(f" caps: {d.get('caps','?')}") - -db = d.get('dirty_block', []) -print(f"\n=== dirty_block ring: {len(db)} entries ===") -if not db: - print(" EMPTY — interpreter never ran (or accessor failed)") - sys.exit(0) - -# Spread by frame -frames = collections.Counter(e['frame'] for e in db) -print(f" frame range: {min(frames)}..{max(frames)} ({len(frames)} distinct)") -print(f" top frames: " + ", ".join(f"fr{f}={c}" for f,c in frames.most_common(8))) - -# Most frequent target -target_count = collections.Counter(e['target'] for e in db) -print(f"\n=== top dirty-block targets (the chain handlers being hammered) ===") -for tgt, cnt in target_count.most_common(20): - print(f" target={tgt}: {cnt} entries") - -# Top RA (caller) -ra_count = collections.Counter(e['ra'] for e in db) -print(f"\n=== top RA values (who called these blocks) ===") -for ra, cnt in ra_count.most_common(15): - print(f" ra={ra}: {cnt} entries") - -# What are the LAST N entries — show full -print(f"\n=== Last 30 dirty_block entries (newest) ===") -for e in db[-30:]: - print(f" seq={e['seq']:>10} fr={e['frame']:>3} target={e['target']} ra={e['ra']} " - f"a0={e.get('a0')} sp={e.get('sp')}") - -# Look for "tight loops": pairs (target, ra) that repeat heavily -pair_count = collections.Counter((e['target'], e['ra']) for e in db) -print(f"\n=== Top (target, ra) pairs — exact tight-loop signature ===") -for (t, r), cnt in pair_count.most_common(15): - print(f" target={t} ra={r}: {cnt} entries") diff --git a/tools/_options_session/_dual_wtrace_slot.py b/tools/_options_session/_dual_wtrace_slot.py deleted file mode 100644 index 9c89e2f2..00000000 --- a/tools/_options_session/_dual_wtrace_slot.py +++ /dev/null @@ -1,49 +0,0 @@ -"""Compare write activity at mem[0x801FD800] between runtime and DuckStation. - -Both backends should support wtrace_addr / wtrace_dump. -If DuckStation writes (esp. value=2 or 3 transitions) and runtime never does, -that confirms the upstream code-path divergence.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while b'\n' not in buf: - c = s.recv(65536) - if not c: break - buf += c - s.close() - line, _, _ = buf.partition(b'\n') - return json.loads(line.decode()) - -PORTS = [(4470, 'runtime'), (4371, 'duck')] - -# Clear any prior wtrace state -for port, name in PORTS: - r1 = call(port, 'wtrace_clear') - r2 = call(port, 'wtrace_addr', addr='0x801FD800') - print(f" {name}: wtrace_clear={r1.get('ok')} wtrace_addr={r2.get('ok')} err={r2.get('err')}") - -print() -print("Sampling for 3 sec...") -time.sleep(3) - -for port, name in PORTS: - r = call(port, 'wtrace_dump', count=64) - entries = r.get('entries') or r.get('hits') or [] - print(f"\n=== {name} (port {port}): {len(entries)} write entries ===") - seen = {} - for e in entries[:40]: - pc = e.get('pc') or e.get('store_pc') - val = e.get('val') or e.get('value') - key = (pc, val) - seen[key] = seen.get(key, 0) + 1 - for (pc, val), n in sorted(seen.items(), key=lambda x: -x[1])[:20]: - print(f" pc={pc} val={val} count={n}") - if not entries: - print(" (empty)") - -# Clean up -for port, name in PORTS: - call(port, 'wtrace_clear') diff --git a/tools/_options_session/_duck_catch_init.py b/tools/_options_session/_duck_catch_init.py deleted file mode 100644 index 68f1aeac..00000000 --- a/tools/_options_session/_duck_catch_init.py +++ /dev/null @@ -1,53 +0,0 @@ -"""Reset DuckStation, arm mem_break on 0x8009B3A0, let it boot, capture the -PC that writes there. That's the init function we need to check on runtime.""" -import socket, json, time - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=10) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -print("--- Reset DuckStation ---") -print(call('system_reset')) - -# Wait for it to come back up -time.sleep(2) -print("ping:", call('ping')) - -# Verify it really reset -print("mem[0x8009B3A0] after reset:", call('read_ram', addr='0x8009B3A0', len=4)) - -# Arm mem_break on the struct head -print() -print("--- Arm mem_break ---") -print("mem_hit_clear:", call('mem_hit_clear')) -print("mem_break: ", call('mem_break', addr='0x8009B3A0')) - -# Poll for the write — write happens early in boot -print() -print("--- Polling for write (60 sec max) ---") -end = time.time() + 60 -caught = [] -while time.time() < end: - h = call('mem_hit_last') - if h.get('valid'): - key = (h.get('pc'), h.get('ra'), h.get('mem_addr')) - if key not in caught: - caught.append(key) - p = call('ping') - v = call('read_ram', addr='0x8009B3A0', len=4) - print(f" fr={p.get('frame','?')} pc={h.get('pc')} ra={h.get('ra')} mem_addr={h.get('mem_addr')} val_now={v.get('hex')}") - call('mem_hit_clear') - # Also peek at value periodically - time.sleep(0.05) - -if not caught: - print(" No writes caught in 60 sec. Value now:", call('read_ram', addr='0x8009B3A0', len=4)) -print() -print(f"Total distinct writes: {len(caught)}") diff --git a/tools/_options_session/_duck_inspect.py b/tools/_options_session/_duck_inspect.py deleted file mode 100644 index 1a04d4f9..00000000 --- a/tools/_options_session/_duck_inspect.py +++ /dev/null @@ -1,28 +0,0 @@ -import socket, json, sys -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - p = {'id':1,'cmd':cmd, **kw} - s.sendall((json.dumps(p)+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -print('ping:', call('ping')) -print() -print('read_ram 0x8009B3D0 len=16:') -r = call('read_ram', addr='0x8009B3D0', len=16) -print(' hex:', r.get('hex')) -print() -print('read_ram 0x801FD950 len=128 (state struct):') -r = call('read_ram', addr='0x801FD950', len=128) -print(' hex:', r.get('hex')) -print() -print('gpu_state:') -print(' ', call('gpu_state')) -print() -print('sio_state:') -print(' ', call('sio_state')) diff --git a/tools/_options_session/_duck_membreak.py b/tools/_options_session/_duck_membreak.py deleted file mode 100644 index 8191a488..00000000 --- a/tools/_options_session/_duck_membreak.py +++ /dev/null @@ -1,29 +0,0 @@ -"""Check if DuckStation already captured a write to 0x8009B3D4. -Arm mem_break, then query mem_hit_last.""" -import socket, json, sys - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -# Address to watch -ADDR = 0x8009B3D4 - -print(f"mem_hit_clear: {call('mem_hit_clear')}") -print(f"mem_break addr=0x{ADDR:08X}: {call('mem_break', addr=f'0x{ADDR:08X}')}") -print() -print("Sleeping 2 sec to see if anything writes...") -import time -time.sleep(2) -print(f"mem_hit_last: {call('mem_hit_last')}") -print() -print("Current memory value:") -r = call('read_ram', addr=f'0x{ADDR:08X}', len=4) -print(f" mem[0x{ADDR:08X}] = {r.get('hex')} (oracle SHOULD be 00010000 LE = 0x00000100)") diff --git a/tools/_options_session/_duck_ping.py b/tools/_options_session/_duck_ping.py deleted file mode 100644 index cdbacec0..00000000 --- a/tools/_options_session/_duck_ping.py +++ /dev/null @@ -1,10 +0,0 @@ -import socket, json -s = socket.create_connection(('127.0.0.1', 4371), timeout=5) -s.sendall(b'{"id":1,"cmd":"ping"}\n') -buf = b'' -while not buf.endswith(b'\n'): - chunk = s.recv(65536) - if not chunk: break - buf += chunk -s.close() -print(buf.decode()) diff --git a/tools/_options_session/_duck_poll_writes.py b/tools/_options_session/_duck_poll_writes.py deleted file mode 100644 index b6f45aa2..00000000 --- a/tools/_options_session/_duck_poll_writes.py +++ /dev/null @@ -1,32 +0,0 @@ -"""Poll mem_hit_last on DuckStation to catch the PC writing to 0x8009B3D4.""" -import socket, json, time - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -call('mem_hit_clear') - -hits_seen = set() -print("Polling for writes to 0x8009B3D4 (5 sec)...") -end = time.time() + 5 -while time.time() < end: - h = call('mem_hit_last') - if h.get('valid'): - key = (h['pc'], h['ra']) - if key not in hits_seen: - hits_seen.add(key) - print(f" WRITE pc={h['pc']} ra={h['ra']} mem_addr={h['mem_addr']}") - call('mem_hit_clear') - time.sleep(0.005) - -print(f"\nDistinct (pc, ra) write sites: {len(hits_seen)}") -for pc, ra in sorted(hits_seen): - print(f" pc={pc} ra={ra}") diff --git a/tools/_options_session/_duck_sample.py b/tools/_options_session/_duck_sample.py deleted file mode 100644 index 92d65cab..00000000 --- a/tools/_options_session/_duck_sample.py +++ /dev/null @@ -1,25 +0,0 @@ -"""Take 20 consecutive snapshots of mem[0x9B3D4] to see if it actually changes -or if the prior 'changes' were just one-off oddities.""" -import socket, json, time - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -prev = None -print(f"{'i':>3} {'frame':>6} {'mem[0x8009B3D4]':>18} {'changed':>8}") -for i in range(20): - p = call('ping') - r = call('read_ram', addr='0x8009B3D4', len=4) - h = r.get('hex') - changed = '*' if (prev and prev != h) else '' - print(f"{i:>3} {p['frame']:>6} {h:>18} {changed:>8}") - prev = h - time.sleep(0.2) diff --git a/tools/_options_session/_duck_scratch.py b/tools/_options_session/_duck_scratch.py deleted file mode 100644 index 969f53bb..00000000 --- a/tools/_options_session/_duck_scratch.py +++ /dev/null @@ -1,20 +0,0 @@ -import socket, json -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -print('ping:', call('ping')) -print() -print('read_ram(0x1F8001D4, 16):', call('read_ram', addr='0x1F8001D4', len=16)) -print() -print('read_scratch(0x1D4, 16):', call('read_scratch', addr='0x1D4', len=16)) -print() -print('read_ram(0x1F800000, 32) (scratchpad start):') -print(' ', call('read_ram', addr='0x1F800000', len=32)) diff --git a/tools/_options_session/_duck_slot_writes.py b/tools/_options_session/_duck_slot_writes.py deleted file mode 100644 index 562acd81..00000000 --- a/tools/_options_session/_duck_slot_writes.py +++ /dev/null @@ -1,53 +0,0 @@ -"""On DuckStation: read current slot states, arm mem_break on slot 0's mem[+0], -poll for writes. Each write tells us a PC that transitions slot state.""" -import socket, json, time - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while b'\n' not in buf: - c = s.recv(65536) - if not c: break - buf += c - s.close() - line, _, _ = buf.partition(b'\n') - return json.loads(line.decode()) - -print(f"ping: {call('ping')}") - -# Read current slot states (all 3 slots, byte at offset 0 of each) -for slot, addr in [(0, 0x801FD800), (1, 0x801FD870), (2, 0x801FD8E0)]: - r = call('read_ram', addr=f'0x{addr:08X}', len=4) - if 'hex' not in r: - print(f" slot {slot} mem[0x{addr:08X}]: ERR {r}") - continue - b = bytes.fromhex(r['hex']) - halfword = b[0] | (b[1]<<8) - print(f" slot {slot} mem[0x{addr:08X}+0] (halfword) = 0x{halfword:04X}") - -# Arm mem_break on slot 0's mem[+0] -print() -print(f"mem_hit_clear: {call('mem_hit_clear')}") -print(f"mem_break 0x801FD800: {call('mem_break', addr='0x801FD800')}") - -# Poll for 5 sec, collect distinct (pc, ra) writes -print() -print("Polling for 5 sec...") -hits = [] -seen_pcs = set() -end = time.time() + 5 -while time.time() < end: - h = call('mem_hit_last') - if h.get('valid'): - key = (h.get('pc'), h.get('ra')) - if key not in seen_pcs: - seen_pcs.add(key) - hits.append(h) - print(f" WRITE pc={h.get('pc')} ra={h.get('ra')} mem_addr={h.get('mem_addr')}") - call('mem_hit_clear') - time.sleep(0.02) - -print(f"\nDistinct write PCs: {len(seen_pcs)}") -for h in hits: - print(f" pc={h.get('pc')} ra={h.get('ra')}") diff --git a/tools/_options_session/_duck_snapshot.py b/tools/_options_session/_duck_snapshot.py deleted file mode 100644 index abea67cb..00000000 --- a/tools/_options_session/_duck_snapshot.py +++ /dev/null @@ -1,65 +0,0 @@ -"""Snapshot DuckStation at OPTIONS — golden oracle data.""" -import socket, json, struct, sys - -def call(cmd, **kw): - s = socket.create_connection(('127.0.0.1', 4371), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -def read_word(addr): - r = call('read_ram', addr=f'0x{addr:08X}', len=4) - return int.from_bytes(bytes.fromhex(r['hex']), 'little') if r.get('ok') else None - -def read_bytes(addr, n): - r = call('read_ram', addr=f'0x{addr:08X}', len=n) - return bytes.fromhex(r['hex']) if r.get('ok') else None - -p = call('ping') -print(f"=== DuckStation oracle @ frame {p['frame']} (OPTIONS) ===") -print() - -# Critical state-machine readings -ptr = read_word(0x1F8001D4) -print(f" scratchpad[0x1D4] (state struct ptr) = 0x{ptr:08X}") - -if ptr: - state_word = read_word(ptr + 0x4C) - state = (state_word >> 16) & 0xFFFF - print(f" state @ ptr+0x4E = 0x{state:04X}") - -print() -print(f" mem[0x80097520..0x8009752F] (fn ptr table):") -b = read_bytes(0x80097520, 16) -if b: - for i in range(0, 16, 4): - w = struct.unpack_from('<I', b, i)[0] - print(f" [0x{0x80097520+i:08X}] = 0x{w:08X}") -print() -print(f" mem[0x8009B3A0..0x8009B43F] (state-0 helper return ptr struct):") -b = read_bytes(0x8009B3A0, 0xA0) -if b: - for i in range(0, len(b), 16): - line = ' '.join(f'{x:02x}' for x in b[i:i+16]) - nz = sum(1 for x in b[i:i+16] if x != 0) - print(f" +0x{i:03X}: {line} ({nz} nz)") -print() -print(f" CRITICAL: mem[0x8009B3D4] = 0x{read_word(0x8009B3D4):08X}") -print(f" high half = 0x{(read_word(0x8009B3D4) >> 16) & 0xFFFF:04X} ({'main path (state advances)' if (read_word(0x8009B3D4) & 0xFFFF0000) == 0 else 'side path (handler skips)'})") - -print() -print(" GPU state:") -g = call('gpu_state') -print(f" disp=({g.get('display_area_x')},{g.get('display_area_y')}) draw_offset=({g.get('draw_offset_x')},{g.get('draw_offset_y')})") -print(f" display_enable={g.get('display_enable')} gpustat={g.get('gpustat')}") - -# Now arm mem_break to catch future writes -print() -print("Arming mem_break on 0x8009B3D4...") -print(f" mem_hit_clear: {call('mem_hit_clear')}") -print(f" mem_break: {call('mem_break', addr='0x8009B3D4')}") diff --git a/tools/_options_session/_duck_state_byte.py b/tools/_options_session/_duck_state_byte.py deleted file mode 100644 index e8fdbfd3..00000000 --- a/tools/_options_session/_duck_state_byte.py +++ /dev/null @@ -1,42 +0,0 @@ -"""Read the state-machine state byte on DuckStation vs runtime.""" -import socket, json - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -def read_word(port, addr): - r = call(port, 'read_ram', addr=f'0x{addr:08X}', len=4) - if not r.get('ok'): return None - b = bytes.fromhex(r['hex']) - return int.from_bytes(b, 'little') - -for label, port in [('runtime', 4470), ('duckstation', 4371)]: - print(f"=== {label} (port {port}) ===") - try: - p = call(port, 'ping') - print(f" ping fr={p.get('frame','?')}") - except Exception as e: - print(f" ping failed: {e}") - continue - - # Read scratchpad ptr at 0x1F8001D4 - ptr = read_word(port, 0x1F8001D4) - print(f" scratchpad[0x1D4] (state struct ptr) = 0x{ptr:08X}") - - # State byte at ptr+0x4E (halfword) - state_word = read_word(port, ptr + 0x4C) # word containing 0x4E,0x4F bytes - state = (state_word >> 16) & 0xFFFF - print(f" state @ ptr+0x4E = 0x{state:04X} ({'normal' if state < 6 else '>=6, handler SKIPS'})") - - # 0x8009B3D4 high half - field_word = read_word(port, 0x8009B3D4) - print(f" mem[0x8009B3D4] = 0x{field_word:08X} high_half=0x{(field_word>>16)&0xFFFF:04X} ({'state-0 main path' if (field_word & 0xFFFF0000)==0 else 'state-0 side path (skips!)'})") - print() diff --git a/tools/_options_session/_duck_vram_cmds.py b/tools/_options_session/_duck_vram_cmds.py deleted file mode 100644 index 33247740..00000000 --- a/tools/_options_session/_duck_vram_cmds.py +++ /dev/null @@ -1,34 +0,0 @@ -import socket, json, time -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(3); buf=b''; out=None; deadline=time.time()+4 - while time.time() < deadline: - try: chunk = s.recv(65536) - except socket.timeout: break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: obj = json.loads(line.decode()) - except: continue - if obj.get('id') == 1: out = obj; deadline = 0; break - s.close() - return out or {'err': 'no resp'} - -for _ in range(3): call(4371, 'mem_hit_clear') - -# Probe what VRAM-read commands duck supports -for cmd, kw in [ - ('vram_peek', {'x':384,'y':256,'w':64,'h':8}), - ('vram_read', {'x':384,'y':256,'w':64,'h':8}), - ('peek_vram', {'x':384,'y':256,'w':64,'h':8}), - ('emu_vram_peek', {'x':384,'y':256,'w':64,'h':8}), - ('emu_vram_read', {'x':384,'y':256,'w':64,'h':8}), - ('screenshot_file', {'path': 'duck_opt.png'}), - ('screenshot', {'path': 'duck_opt.png'}), - ('gpu_ring', {}), - ('gp0_ring', {}), -]: - r = call(4371, cmd, **kw) - print(f" {cmd} {kw}: {str(r)[:160]}") diff --git a/tools/_options_session/_dump_bios_bytes.py b/tools/_options_session/_dump_bios_bytes.py deleted file mode 100644 index 5372ec24..00000000 --- a/tools/_options_session/_dump_bios_bytes.py +++ /dev/null @@ -1,11 +0,0 @@ -import struct, sys -with open('F:/Projects/psxrecomp/psxrecomp/bios/SCPH1001.BIN', 'rb') as f: - data = f.read() -# Read inputs as pc, length -start_pc = int(sys.argv[1], 16) if len(sys.argv) > 1 else 0xBFC16060 -length = int(sys.argv[2], 0) if len(sys.argv) > 2 else 0x40 -off0 = start_pc - 0xBFC00000 -for i in range(0, length, 4): - w = struct.unpack_from('<I', data, off0 + i)[0] - pc = start_pc + i - print(f'{pc:08x}: {w:08x}') diff --git a/tools/_options_session/_dump_keys.py b/tools/_options_session/_dump_keys.py deleted file mode 100644 index 277248f0..00000000 --- a/tools/_options_session/_dump_keys.py +++ /dev/null @@ -1,13 +0,0 @@ -import json -with open('F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json') as f: - d = json.load(f) -for k in d: - v = d[k] - if isinstance(v, list): - print(f'{k}: list len {len(v)}') - if v and isinstance(v[0], dict): - print(f' first entry keys: {list(v[0].keys())}') - elif isinstance(v, dict): - print(f'{k}: dict with keys: {list(v.keys())[:20]}') - else: - print(f'{k}: {type(v).__name__} = {v}') diff --git a/tools/_options_session/_dump_ram.py b/tools/_options_session/_dump_ram.py deleted file mode 100644 index c3bb476d..00000000 --- a/tools/_options_session/_dump_ram.py +++ /dev/null @@ -1,13 +0,0 @@ -import sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg -addr = int(sys.argv[1], 16) -size = int(sys.argv[2], 0) if len(sys.argv) > 2 else 0x200 -r = _dbg.call(4470, 'read_ram', addr=f'0x{addr:08X}', len=size, timeout=4) -b = bytes.fromhex(r['hex']) -print(f"Contents of 0x{addr:08X} ({size} bytes):") -for i in range(0, len(b), 32): - line = ' '.join(f'{x:02x}' for x in b[i:i+32]) - non_zero = sum(1 for x in b[i:i+32] if x != 0) - marker = "" if non_zero == 0 else f" ({non_zero}/32 non-zero)" - print(f" +0x{i:03X}: {line}{marker}") diff --git a/tools/_options_session/_event_spin.py b/tools/_options_session/_event_spin.py deleted file mode 100644 index 1ff41f06..00000000 --- a/tools/_options_session/_event_spin.py +++ /dev/null @@ -1,44 +0,0 @@ -"""Find what's calling TestEvent / what Tomba is spinning on.""" -import json, sys -from collections import Counter - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -# fn_entry: who calls the Tomba chain handlers (RA tells us the caller) -print('=== fn_entry RA distribution for Tomba functions ===') -for target in ('0x80068024', '0x80068098', '0x80068554'): - ra_c = Counter(e['ra'] for e in d['fn_entry'] if e['func'] == target) - print(f'\n{target}:') - for ra, c in ra_c.most_common(5): - print(f' RA {ra}: {c}') - -print() -print('=== Top 20 RA addrs across ALL fn_entry ===') -ra_c = Counter(e['ra'] for e in d['fn_entry']) -for ra, c in ra_c.most_common(20): - print(f' RA {ra}: {c}') - -# Check stores to the EvCB region (typical EvCB starts at 0xE000+) -print() -print('=== Stores in EvCB region (0xE000..0xEC00) ===') -ev_writes = [e for e in d['wtrace_all'] if 0xE000 <= int(e['addr'], 16) < 0xEC00] -print(f'Total EvCB writes: {len(ev_writes)}') -addr_c = Counter(e['addr'] for e in ev_writes) -print('By addr:') -for a, c in addr_c.most_common(15): - print(f' {a}: {c}') -# Show a sample of unique values written, to see what modes are being set -val_c = Counter(e['new'] for e in ev_writes) -print('By value:') -for v, c in val_c.most_common(10): - print(f' {v}: {c}') - -# fn_entry order: see the cycle pattern around the chain handlers -print() -print('=== Last 30 fn_entry events (chronological) ===') -for e in d['fn_entry'][-30:]: - print(f' seq={e["seq"]} fn={e["func"]} ra={e["ra"]} a0={e["a0"]} a1={e["a1"]}') diff --git a/tools/_options_session/_event_waits.py b/tools/_options_session/_event_waits.py deleted file mode 100644 index 2a08478a..00000000 --- a/tools/_options_session/_event_waits.py +++ /dev/null @@ -1,36 +0,0 @@ -"""Sample fntrace and identify the event IDs Tomba is polling each frame.""" -import sys, time, collections, json -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -_dbg.call(4470, "fntrace_arm", target="0xFFFFFFFF", timeout=2) -time.sleep(1) - -d = _dbg.call(4470, "fntrace_dump", count=300, timeout=4) -es = d.get('entries', []) -print(f"Got {len(es)} entries") - -# The B0 trampoline is at 0xB0; calls there carry t1=B0 index, a0/a1 = args. -# Filter B0 calls and TestEvent-style entries. -b0 = [e for e in es if e['target'] == '0x000000B0'] -print(f"\nB0 trampoline calls: {len(b0)}") -# Group by (ra, t1) -sigs = collections.Counter((e['ra'], e.get('t1','?')) for e in b0) -print(f"Top (ra, t1=B0_idx) signatures:") -for (ra, t1), c in sigs.most_common(10): - print(f" ra={ra} t1={t1}: {c}") - -# Find every distinct (ra, a0, a1, t1) for B0 calls -sigs2 = collections.Counter((e['ra'], e.get('a0'), e.get('a1'), e.get('t1','?')) for e in b0) -print(f"\nTop (ra, a0, a1, t1) signatures:") -for (ra, a0, a1, t1), c in sigs2.most_common(15): - print(f" ra={ra} t1={t1} a0={a0} a1={a1}: {c}") - -# Now look at non-B0 calls — what's Tomba running? -non_b0 = [e for e in es if e['target'] != '0x000000B0'] -tomba_funcs = collections.Counter(e['target'] for e in non_b0 if e['target'].startswith('0x800')) -print(f"\nTomba (0x800...) functions called:") -for f, c in tomba_funcs.most_common(15): - callers = collections.Counter(e['ra'] for e in non_b0 if e['target'] == f) - top_ra = callers.most_common(3) - print(f" {f}: {c} hits, top callers: {top_ra}") diff --git a/tools/_options_session/_find_real_changeth_site.py b/tools/_options_session/_find_real_changeth_site.py deleted file mode 100644 index 24a9d436..00000000 --- a/tools/_options_session/_find_real_changeth_site.py +++ /dev/null @@ -1,41 +0,0 @@ -"""For each thread saved at sp=X, BIOS ChangeTh prolog wrote Tomba's caller-ra -to mem[sp+0x14]. Read that to find the actual ChangeTh JAL site in Tomba.""" -import socket, json - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -# TCB A: save sp = 0x801FFF98 -> mem[0x801FFFAC] = Tomba's caller ra -# TCB B: save sp = 0x801FE3A8 -> mem[0x801FE3BC] = Tomba's caller ra -for label, sp in [('TCB A (0xA000E1F4)', 0x801FFF98), ('TCB B (0xA000E2B4)', 0x801FE3A8)]: - print(f"=== {label} ===") - # Stack frame around the saved ra - r = call(4470, 'read_ram', addr=f'0x{sp:08X}', len=0x30) - b = bytes.fromhex(r['hex']) - for i in range(0, len(b), 4): - w = int.from_bytes(b[i:i+4], 'little') - annot = ' <-- saved ra (Tomba caller PC after JAL)' if i == 0x14 else '' - print(f" sp+0x{i:02X} ({sp+i:08X}): 0x{w:08X}{annot}") - print() - -# Also dump on DuckStation for comparison -print(f"=== DuckStation comparison ===") -# DuckStation's threads might have different sp values -# Read its TCB to get current sp from saved area -for tcb, label in [(0xA000E1F4, 'A'), (0xA000E2B4, 'B')]: - save_area = tcb + 8 - r = call(4371, 'read_ram', addr=f'0x{save_area + 29*4:08X}', len=4) # gpr[29] = sp - sp = int.from_bytes(bytes.fromhex(r['hex']), 'little') - print(f" DUCK TCB {label} saved sp = 0x{sp:08X}") - if sp: - r2 = call(4371, 'read_ram', addr=f'0x{sp+0x14:08X}', len=4) - ra = int.from_bytes(bytes.fromhex(r2['hex']), 'little') - print(f" DUCK TCB {label} mem[sp+0x14] = 0x{ra:08X} (Tomba caller PC)") diff --git a/tools/_options_session/_fn_tail.py b/tools/_options_session/_fn_tail.py deleted file mode 100644 index 259b54aa..00000000 --- a/tools/_options_session/_fn_tail.py +++ /dev/null @@ -1,22 +0,0 @@ -import json, sys -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778822828.json' -with open(dump) as f: - d = json.load(f) -fe = d['fn_entry'] -print(f'fn_entry count: {len(fe)}') -print(f'Last 40 entries:') -for e in fe[-40:]: - print(f' seq={e["seq"]} fr={e["frame"]} func={e["func"]} ra={e["ra"]} ' - f'a0={e.get("a0","?")} a1={e.get("a1","?")} t1={e.get("t1","?")}') - -print() -print(f'Last 30 wtrace_all entries:') -for e in d['wtrace_all'][-30:]: - print(f' seq={e["seq"]} fr={e["frame"]} pc={e["pc"]} addr={e["addr"]} ->{e.get("new","?")}') - -print() -print(f'Last 20 restore_trace entries:') -for e in d['restore_trace'][-20:]: - print(f' seq={e["seq"]} fr={e["frame"]} kind={e["kind"]} name={e.get("name","?")} ' - f'target={e["target"]} cpu_pc={e["cpu_pc"]} epc={e["epc"]} in_exc={e["in_exc"]}') diff --git a/tools/_options_session/_fn_what_runs.py b/tools/_options_session/_fn_what_runs.py deleted file mode 100644 index 32f640cd..00000000 --- a/tools/_options_session/_fn_what_runs.py +++ /dev/null @@ -1,30 +0,0 @@ -"""Arm fntrace, sample, find what Tomba functions actually run during black screen.""" -import sys, time, collections, json -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -# Make sure fntrace is armed for the entire Tomba RAM range -_dbg.call(4470, "fntrace_arm", target="0x80000000", timeout=2) - -time.sleep(2) - -d = _dbg.call(4470, "fntrace_dump", count=500, timeout=4) -es = d.get('entries', []) -print(f"Got {len(es)} entries (armed: {d.get('armed')})") - -funcs = collections.Counter(e.get('target') for e in es) -print(f"\nTop 20 hottest functions:") -for f, c in funcs.most_common(20): - print(f" {f}: {c} hits") - -# RA distribution per top target -print(f"\nDecomposition of top 5 functions by caller:") -for f, _ in funcs.most_common(5): - callers = collections.Counter(e['ra'] for e in es if e['target'] == f) - print(f" {f}:") - for ra, c in callers.most_common(5): - print(f" ra={ra}: {c}") - -print(f"\nLast 25 entries (newest first):") -for e in es[:25]: - print(f" fr={e.get('frame')} target={e['target']} ra={e['ra']} a0={e.get('a0')} a1={e.get('a1')}") diff --git a/tools/_options_session/_gate_byte.py b/tools/_options_session/_gate_byte.py deleted file mode 100644 index a63622e7..00000000 --- a/tools/_options_session/_gate_byte.py +++ /dev/null @@ -1,32 +0,0 @@ -import socket, json -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -for port, name in [(4470, 'runtime'), (4371, 'duck')]: - r = call(port, 'read_ram', addr='0x80090C90', len=16) - if r.get('ok'): - b = bytes.fromhex(r['hex']) - c94 = int.from_bytes(b[4:8], 'little') - c98 = int.from_bytes(b[8:12], 'little') - c9e = b[14] - print(f"{name:>8}: mem[0x80090C9E] byte = 0x{c9e:02X} ({c9e}) -> gate: {'work+ChangeTh' if c9e >= 2 else 'ChangeTh only'}") - print(f"{name:>8}: mem[0x80090C94] ptr = 0x{c94:08X}") - print(f"{name:>8}: mem[0x80090C98] ptr = 0x{c98:08X}") - print() - -# Also follow the indirect: mem[mem[0x80090C94]+0x3C] -print("Indirect dispatch target mem[mem[0x80090C94]+0x3C]:") -for port, name in [(4470, 'runtime'), (4371, 'duck')]: - r1 = call(port, 'read_ram', addr='0x80090C94', len=4) - c94 = int.from_bytes(bytes.fromhex(r1['hex']), 'little') - r2 = call(port, 'read_ram', addr=f'0x{c94+0x3C:08X}', len=4) - fn = int.from_bytes(bytes.fromhex(r2['hex']), 'little') - print(f" {name:>8}: mem[0x{c94:08X}+0x3C] = 0x{fn:08X} (function called at 0x8005EBA4)") diff --git a/tools/_options_session/_gp0_ring_inspect.py b/tools/_options_session/_gp0_ring_inspect.py deleted file mode 100644 index b68601aa..00000000 --- a/tools/_options_session/_gp0_ring_inspect.py +++ /dev/null @@ -1,80 +0,0 @@ -"""Inspect runtime's GP0 ring at OPTIONS. -Tally GP0 opcodes; dump a recent frame's draw command stream.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=8) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(5); buf=b''; out=None; deadline=time.time()+6 - while time.time() < deadline: - try: chunk = s.recv(65536) - except socket.timeout: break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: obj = json.loads(line.decode()) - except: continue - if obj.get('id') == 1: out = obj; deadline = 0; break - s.close() - return out or {'err': 'no resp'} - -r = call(4470, 'gpu_ring_stats') -print(f"gpu_ring_stats: {r}") - -oldest = r.get('oldest_frame') -newest = r.get('newest_frame') - -r = call(4470, 'gpu_opcodes') -ops = r.get('opcodes', {}) -print(f"\ngpu_opcodes: {len(ops)} distinct opcodes seen") -GP0_NAMES = { - 0x00: 'NOP', 0x01: 'CLEAR_CACHE', 0x02: 'FILL_RECT', - 0x20: 'POLY_FLAT_TRI', 0x22: 'POLY_FLAT_TRI_T', 0x24: 'POLY_FLAT_TRI_TX', 0x26: 'POLY_FLAT_TRI_TX_T', - 0x28: 'POLY_FLAT_QUAD', 0x2A: 'POLY_FLAT_QUAD_T', 0x2C: 'POLY_FLAT_QUAD_TX', 0x2E: 'POLY_FLAT_QUAD_TX_T', - 0x30: 'POLY_GOUR_TRI', 0x32: 'POLY_GOUR_TRI_T', 0x34: 'POLY_GOUR_TRI_TX', 0x36: 'POLY_GOUR_TRI_TX_T', - 0x38: 'POLY_GOUR_QUAD', 0x3A: 'POLY_GOUR_QUAD_T', 0x3C: 'POLY_GOUR_QUAD_TX', 0x3E: 'POLY_GOUR_QUAD_TX_T', - 0x40: 'LINE_FLAT', 0x48: 'LINE_POLY_FLAT', - 0x60: 'RECT_FLAT', 0x64: 'RECT_FLAT_TX', 0x68: 'RECT_FLAT_1', 0x6C: 'RECT_FLAT_1_TX', - 0x70: 'RECT_FLAT_8', 0x74: 'RECT_FLAT_8_TX', 0x78: 'RECT_FLAT_16', 0x7C: 'RECT_FLAT_16_TX', - 0x80: 'CPY_VRAM_VRAM', 0xA0: 'CPY_CPU_VRAM', 0xC0: 'CPY_VRAM_CPU', - 0xE1: 'DRAW_MODE', 0xE2: 'TEXTURE_WIN', 0xE3: 'DRAW_AREA_TL', 0xE4: 'DRAW_AREA_BR', - 0xE5: 'DRAW_OFFSET', 0xE6: 'MASK_BIT', -} -for op_s, n in sorted(ops.items(), key=lambda x: -x[1]): - op = int(op_s, 16) - nm = GP0_NAMES.get(op, '?') - print(f" 0x{op:02X} {nm:20s}: {n}") - -print(f"\nDumping frame {newest}:") -r = call(4470, 'gpu_frame_dump', frame=newest, count=4096) -entries = r.get('entries', []) -print(f" count: {len(entries)}") - -# Look for actual polygon draws — focus on top 1xx opcodes -classify = {'polygon': 0, 'rect': 0, 'line': 0, 'fill': 0, 'env': 0, 'copy': 0, 'misc': 0} -poly_samples = [] -for e in entries: - op = int(e.get('op','0x0'), 16) - if 0x20 <= op <= 0x3F: - classify['polygon'] += 1 - if len(poly_samples) < 10: - poly_samples.append(e) - elif 0x40 <= op <= 0x5F: - classify['line'] += 1 - elif 0x60 <= op <= 0x7F: - classify['rect'] += 1 - elif 0x80 <= op <= 0xDF: - classify['copy'] += 1 - elif op == 0x02: - classify['fill'] += 1 - elif 0xE0 <= op <= 0xEF: - classify['env'] += 1 - else: - classify['misc'] += 1 -print(f" classify: {classify}") -print(f"\n Sample polygons (first 10):") -for e in poly_samples: - op = int(e['op'],16) - nm = GP0_NAMES.get(op,'?') - print(f" seq={e['seq']:6} op={e['op']} ({nm}) n={e['n']} w={e['w']}") diff --git a/tools/_options_session/_gp0_when_drawing_stopped.py b/tools/_options_session/_gp0_when_drawing_stopped.py deleted file mode 100644 index 065c7e6d..00000000 --- a/tools/_options_session/_gp0_when_drawing_stopped.py +++ /dev/null @@ -1,50 +0,0 @@ -"""Find when runtime stopped drawing polygons. -Sample multiple frames at intervals across the ring.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=10) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(5); buf=b''; out=None; deadline=time.time()+6 - while time.time() < deadline: - try: chunk = s.recv(65536) - except socket.timeout: break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: obj = json.loads(line.decode()) - except: continue - if obj.get('id') == 1: out = obj; deadline = 0; break - s.close() - return out or {'err': 'no resp'} - -r = call(4470, 'gpu_ring_stats') -oldest = r['oldest_frame'] -newest = r['newest_frame'] -print(f"Ring spans frames [{oldest}, {newest}] = {newest-oldest} frames") - -# Sample 12 frames spread across the ring -def classify(entries): - cnt = {'poly': 0, 'rect_tx': 0, 'env': 0, 'fill': 0, 'copy': 0, 'misc': 0} - for e in entries: - op = int(e.get('op','0x0'), 16) - if 0x20 <= op <= 0x3F: cnt['poly'] += 1 - elif op == 0x64 or op == 0x65 or op == 0x74 or op == 0x6C or op == 0x7C: cnt['rect_tx'] += 1 - elif 0x60 <= op <= 0x7F: cnt['rect_tx'] += 1 - elif 0xE0 <= op <= 0xEF: cnt['env'] += 1 - elif op == 0x02: cnt['fill'] += 1 - elif 0x80 <= op <= 0xDF: cnt['copy'] += 1 - else: cnt['misc'] += 1 - return cnt - -print() -n_samples = 14 -for i in range(n_samples): - frame = oldest + int((newest - oldest) * i / (n_samples-1)) - r = call(4470, 'gpu_frame_dump', frame=frame, count=2048) - entries = r.get('entries', []) - c = classify(entries) - poly_total = c['poly'] + c['rect_tx'] - flag = ' <-- HAS DRAWS' if poly_total > 0 else '' - print(f" frame {frame:6}: total={len(entries):4} poly={c['poly']:3} rect_tx={c['rect_tx']:3} fill={c['fill']:2} env={c['env']:3} copy={c['copy']:2}{flag}") diff --git a/tools/_options_session/_gpu_compare_clean.py b/tools/_options_session/_gpu_compare_clean.py deleted file mode 100644 index cdcde634..00000000 --- a/tools/_options_session/_gpu_compare_clean.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Clean compare of GPU state at OPTIONS. -Drain pending events from each socket first.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - # drain async events; wait for response whose id matches - s.settimeout(2) - buf = b'' - out = None - deadline = time.time() + 3 - while time.time() < deadline: - try: - chunk = s.recv(65536) - except socket.timeout: - break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: - obj = json.loads(line.decode()) - except Exception: - continue - if obj.get('id') == 1: - out = obj - deadline = 0 - break - s.close() - return out or {'err': 'no id=1 response'} - -# Clear DuckStation mem_break first (use a session that drains events) -print("Clearing duck mem_hit...") -for _ in range(5): - call(4371, 'mem_hit_clear') - -PORTS = [(4470, 'runtime'), (4371, 'duck')] - -print("\n=== gpu_state ===") -for port, name in PORTS: - r = call(port, 'gpu_state') - print(f" {name}: {r}") - -# screenshot to PNG for visual check -print("\n=== screenshot ===") -for port, name in PORTS: - r = call(port, 'screenshot') - if r.get('ok') and 'hex' in r: - w, h = r['width'], r['height'] - b = bytes.fromhex(r['hex']) - # Quick brightness check: count non-zero pixels - nz = sum(1 for x in b if x != 0) - print(f" {name}: {w}x{h} fmt={r.get('format')} non_zero_bytes={nz}/{len(b)} ({100*nz/max(1,len(b)):.1f}%)") - with open(f'opt_screenshot_{name}.bin', 'wb') as fh: - fh.write(b) - print(f" saved opt_screenshot_{name}.bin") - else: - print(f" {name}: {r}") diff --git a/tools/_options_session/_heartbeat_arc.py b/tools/_options_session/_heartbeat_arc.py deleted file mode 100644 index 77517c6b..00000000 --- a/tools/_options_session/_heartbeat_arc.py +++ /dev/null @@ -1,20 +0,0 @@ -"""Print full heartbeat-ring progression to identify the moment frames stopped.""" -import json, sys -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778822828.json' -with open(dump) as f: - d = json.load(f) - -hb = sorted(d['heartbeat_ring'], key=lambda e: e['wall']) -print(f"{'idx':>4} {'wall':>11} {'frame':>8} {'cyc':>14} {'fr-d':>5} {'cyc-d':>10} " - f"{'exc_re-d':>10} {'dirty-d':>10} {'cur_fn':>12} {'in_exc':>2}") -for i, e in enumerate(hb): - if i == 0: - print(f"{i:>4} {e['wall']:>11} {e['frame']:>8} {e['cyc']:>14,} " - f"{'-':>5} {'-':>10} {'-':>10} {'-':>10} {e['cur_fn']:>12} {e['in_exc']:>2}") - else: - a = hb[i-1] - print(f"{i:>4} {e['wall']:>11} {e['frame']:>8} {e['cyc']:>14,} " - f"{e['frame']-a['frame']:>5} {e['cyc']-a['cyc']:>10,} " - f"{e['exc_re']-a['exc_re']:>10,} {e['dirty_insns']-a['dirty_insns']:>10,} " - f"{e['cur_fn']:>12} {e['in_exc']:>2}") diff --git a/tools/_options_session/_inspect_entries.py b/tools/_options_session/_inspect_entries.py deleted file mode 100644 index 15cf00e7..00000000 --- a/tools/_options_session/_inspect_entries.py +++ /dev/null @@ -1,30 +0,0 @@ -import json,os,glob -out_dir = os.path.join(os.environ.get('TEMP','/tmp'),'optsess') -p = sorted(glob.glob(os.path.join(out_dir,'options_baseline_*.json')))[-1] -print('file:', p) -s = json.load(open(p)) - -print() -print('--- runtime: first 6 writes from PC=0x000000BC ---') -ents = s['runtime']['wtrace_all_dump_optstate']['entries'] -for e in [x for x in ents if x['pc']=='0x000000BC'][:6]: - print(e) -print() -print('--- runtime: writes from PC=0xBFC11BF0 (last 3) ---') -for e in [x for x in ents if x['pc']=='0xBFC11BF0'][-3:]: - print(e) -print() -print('--- beetle: first 4 writes (cell DB8 stream) ---') -ents_b = s['beetle']['wtrace_all_dump_optstate']['entries'] -for e in ents_b[:4]: - print(e) -print() -print('--- beetle: last 2 writes ---') -for e in ents_b[-2:]: - print(e) -print() -print('--- runtime: write SIZES distribution ---') -from collections import Counter -print(Counter(e['w'] for e in ents)) -print('--- beetle: write SIZES distribution ---') -print(Counter(e['w'] for e in ents_b)) diff --git a/tools/_options_session/_live_loop.py b/tools/_options_session/_live_loop.py deleted file mode 100644 index a7e8cb2f..00000000 --- a/tools/_options_session/_live_loop.py +++ /dev/null @@ -1,43 +0,0 @@ -"""Query live runtime's recent activity rings to see what's looping.""" -import sys, json, collections -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -# Try fntrace_dump first -try: - fn = _dbg.call(4470, "fntrace_dump", count=512, timeout=4) - if fn.get('ok') and fn.get('entries'): - print(f"=== fntrace_dump ({len(fn['entries'])} entries, newest-first) ===") - recent = fn['entries'][:60] - # Hot targets - targets = collections.Counter(e.get('target') for e in fn['entries']) - print(f"Top 15 targets in last {len(fn['entries'])}:") - for t, c in targets.most_common(15): - print(f" {t}: {c} hits") - print() - print("Last 15 entries (newest first):") - for e in recent[:15]: - print(f" {e}") - else: - print(f"fntrace_dump empty / error: {fn}") -except Exception as e: - print(f"fntrace_dump exception: {e}") - -# Also try dirty_block_log -print() -try: - db = _dbg.call(4470, "dirty_block_log", count=512, timeout=4) - if db.get('ok') and db.get('entries'): - print(f"=== dirty_block_log ({len(db['entries'])} entries, newest-first) ===") - targets = collections.Counter(e.get('target') for e in db['entries']) - print(f"Top 15 targets:") - for t, c in targets.most_common(15): - print(f" {t}: {c} hits") - print() - print("Last 12 entries (newest-first):") - for e in db['entries'][:12]: - print(f" {e}") - else: - print(f"dirty_block_log empty / error: {db}") -except Exception as e: - print(f"dirty_block_log exception: {e}") diff --git a/tools/_options_session/_outer_caller.py b/tools/_options_session/_outer_caller.py deleted file mode 100644 index 66e90a1c..00000000 --- a/tools/_options_session/_outer_caller.py +++ /dev/null @@ -1,47 +0,0 @@ -"""Look for the outer caller of FUN_bfc08b3c (= 0x00008B3C in recompiled form).""" -import json, sys -from collections import Counter - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -# Print all distinct func addresses in fn_entry, sorted by occurrence -fn_c = Counter(e['func'] for e in d['fn_entry']) -print(f'Total distinct funcs in fn_entry: {len(fn_c)}') -print() - -# Are there any direct entries to 0x00008B3C or 0xBFC08B3C? -candidates = [f for f in fn_c if int(f, 16) in (0x00008B3C, 0xBFC08B3C, 0x80008B3C)] -print(f'Direct entries to FUN_bfc08b3c: {candidates}') - -# Any entries to 0x9144 (FUN_bfc09144)? -candidates = [f for f in fn_c if int(f, 16) in (0x00009144, 0xBFC09144)] -print(f'Direct entries to FUN_bfc09144: {[(f, fn_c[f]) for f in candidates]}') - -# What about all funcs in 0x8000-0x9999 range (BIOS shell card area)? -print() -print('All BIOS shell area funcs in fn_entry (0x8000-0x9FFF):') -for f, c in sorted(fn_c.items(), key=lambda x: int(x[0], 16)): - addr = int(f, 16) - if 0x8000 <= addr < 0xA000: - print(f' {f}: {c}') - -# What about Tomba code (0x80000000+)? -print() -print('All Tomba code funcs in fn_entry (0x80000000+):') -for f, c in sorted(fn_c.items(), key=lambda x: int(x[0], 16)): - addr = int(f, 16) - if 0x80000000 <= addr: - print(f' {f}: {c}') - -# What RA values come back from Tomba functions? -print() -print('Distinct RA values for Tomba-function entries:') -for target in ('0x80068024', '0x80068098', '0x80068554'): - print(f' {target}:') - ra_c = Counter(e['ra'] for e in d['fn_entry'] if e['func'] == target) - for ra, c in ra_c.most_common(5): - print(f' RA={ra}: {c}') diff --git a/tools/_options_session/_pre_wedge.py b/tools/_options_session/_pre_wedge.py deleted file mode 100644 index 1e2410c1..00000000 --- a/tools/_options_session/_pre_wedge.py +++ /dev/null @@ -1,40 +0,0 @@ -"""Find what was happening right before the exception-reentry wedge fired. -The dump captured the wedge — exc_re spikes in heartbeat ticks 62-63. -Look at the LAST wtrace_all entries (frame 763) and the final restore_trace -events to identify the trigger.""" -import json, collections - -dump = 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' -with open(dump) as f: - d = json.load(f) - -events = d['wtrace_all'] -print(f"=== Last 60 wtrace_all entries (frame near 763) ===") -for e in events[-60:]: - pc = e['pc'] - addr = int(e['addr'], 16) - new = e.get('new', '?') - seq = e['seq'] - fr = e['frame'] - print(f" seq={seq:,} fr={fr} pc={pc} addr=0x{addr:08X} ->{new}") - -print() -print("=== Last 30 restore_trace events ===") -rt = d['restore_trace'] -for e in rt[-30:]: - print(f" seq={e.get('seq')} fr={e['frame']} kind={e['kind']} name={e.get('name')} " - f"jmp={e.get('jmp')} target={e.get('target')} cpu_pc={e.get('cpu_pc')} " - f"sr={e.get('sr')} epc={e.get('epc')} istat={e.get('istat')} imask={e.get('imask')} " - f"in_exc={e.get('in_exc')}") - -print() -print("=== Last 30 sio_pc_trace events ===") -for e in d['sio_pc_trace'][-30:]: - print(f" seq={e['seq']} pc={e['pc']} func={e.get('func')} addr={e['addr']} " - f"value={e.get('value')} byte_seq={e.get('byte_seq')}") - -print() -print("=== Last 20 fn_entry events ===") -for e in d['fn_entry'][-20:]: - print(f" seq={e['seq']} fr={e['frame']} func={e['func']} ra={e['ra']} " - f"a0={e.get('a0','?')} a1={e.get('a1','?')}") diff --git a/tools/_options_session/_probe_byte127.py b/tools/_options_session/_probe_byte127.py deleted file mode 100644 index 06fcff40..00000000 --- a/tools/_options_session/_probe_byte127.py +++ /dev/null @@ -1,102 +0,0 @@ -"""Diagnose why the 128th SIO-card-data byte never lands in the BIOS buffer. - -The BIOS handler at 0xBFC15F08 has gates: - (1) mem[0x75c0] != 0 (active flag) - (2) I_STAT bit 0x80 set - (3) I_MASK bit 0x80 set - (4) op-byte at *mem[0x75c8] == 2 (READ mode) -On miss, jr ra (no buffer store). - -For sector 0 (sees 127 stores 0xBE48..0xBEC6), find: - - last buffer store seq + frame - - immediately following writes to 0x75c0 / 0x75c4 / 0x72f0 / I_STAT (1F801070) - - any unrelated SIO traffic between byte 127 and byte 128 that could - have flipped a gate. -""" -import json, sys - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -STORE_PCS_UPPER = {'0xBFC16024', '0x00006524', '0X00006524'} -GATE_ADDRS = { - 0x000075c0: "active_flag", - 0x000075c4: "buffer_ptr", - 0x00007570: "buffer_ptr_alt", # in case the var is at 0x7570 region - 0x000075c8: "op_byte_ptr", - 0x000075cc: "checksum_ptr", - 0x000072f0: "counter", - 0x1f801070: "I_STAT", - 0x1f801074: "I_MASK", - 0x1f801040: "SIO_DATA", - 0x1f80104a: "SIO_CTRL", -} - -events = d['wtrace_all'] -print(f"Total wtrace events: {len(events)}") - -# Find all buffer stores (the 0xBE48..0xBEC7 range) -buf_stores = [] -for i, e in enumerate(events): - pc = e['pc'].upper() - if pc in STORE_PCS_UPPER and int(e['addr'], 16) >= 0xBE48 and int(e['addr'], 16) <= 0xBEC7: - buf_stores.append((i, e)) - -print(f"Buffer stores 0xBE48..0xBEC7: {len(buf_stores)}") - -# Group by sector -sectors = [] -cur = [] -prev_addr = -1 -for i, e in buf_stores: - a = int(e['addr'], 16) - if a < prev_addr or (a == 0xBE48 and cur): - sectors.append(cur) - cur = [] - cur.append((i, e)) - prev_addr = a -if cur: - sectors.append(cur) - -print(f"Sectors: {len(sectors)}") - -# For sector 0, look at events between last byte store and the next -# byte store of sector 1 — that's where the gate-flip must have happened. -for s_idx in range(min(2, len(sectors))): - s = sectors[s_idx] - last_i, last_e = s[-1] - print(f"\n=== Sector {s_idx}: {len(s)} stores ===") - print(f" First store: idx {s[0][0]}, addr {s[0][1]['addr']}, frame {s[0][1].get('frame')}") - print(f" Last store: idx {last_i}, addr {last_e['addr']}, frame {last_e.get('frame')}") - print(f" Expected next store addr: 0xBEC7 (the 128th byte)") - print() - - # Show next 60 events after the last byte 127 store, look for any - # writes to gate addresses, OR any SIO traffic - print(f" Events immediately after last store (idx {last_i}):") - next_buf_idx = sectors[s_idx + 1][0][0] if s_idx + 1 < len(sectors) else last_i + 200 - count = 0 - for j in range(last_i + 1, min(last_i + 200, next_buf_idx)): - e = events[j] - addr = int(e['addr'], 16) - pc = e['pc'] - new = e['new'] - old = e.get('old', '') - seq = e.get('seq', '?') - frame = e.get('frame', '?') - - label = GATE_ADDRS.get(addr) - # Show high-pri gate writes, plus near-buffer writes, plus all SIO writes - if (label or (0xBE48 <= addr <= 0xBED0) - or (0x1F801040 <= addr <= 0x1F80104F) - or (0x1F801070 <= addr <= 0x1F801077) - or (0x7500 <= addr <= 0x76C0)): - tag = f"[{label}]" if label else "" - print(f" idx={j} seq={seq} fr={frame} pc={pc} addr=0x{addr:08X} {old}->{new} {tag}") - count += 1 - if count > 50: - print(" ... (truncated)") - break diff --git a/tools/_options_session/_probe_runtime_cmds.py b/tools/_options_session/_probe_runtime_cmds.py deleted file mode 100644 index be42f546..00000000 --- a/tools/_options_session/_probe_runtime_cmds.py +++ /dev/null @@ -1,26 +0,0 @@ -import socket, json -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while b'\n' not in buf: - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.partition(b'\n')[0].decode()) - -for cmd, kw in [ - ('wtrace_arm', {'lo': '0x801FD800', 'hi': '0x801FD802'}), - ('wtrace_arm', {'lo': '0x801FD800'}), - ('wtrace_addr', {'addr': '0x801FD800'}), - ('mem_break', {'addr': '0x801FD800'}), - ('mem_write_break', {'addr': '0x801FD800'}), - ('wtrace_pc', {'pc': '0x800173FC'}), - ('wtrace_dump', {}), - ('fntrace_arm', {'pc': '0x800173B0'}), - ('fntrace_dump', {}), - ('fn_entry_arm', {'pc': '0x800173B0'}), -]: - r = call(4470, cmd, **kw) - print(f" {cmd} {kw}: {str(r)[:160]}") diff --git a/tools/_options_session/_read_state.py b/tools/_options_session/_read_state.py deleted file mode 100644 index 3c28a1f5..00000000 --- a/tools/_options_session/_read_state.py +++ /dev/null @@ -1,30 +0,0 @@ -import sys, struct -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -# Read scratchpad ptr at 0x1F8001D4 -ptr_word = _dbg.read_word(4470, 0x1F8001D4) -print(f"scratchpad[0x1D4] = ptr 0x{ptr_word:08X}") - -# Read mem[ptr + 0x4E] (halfword), and surrounding bytes for context -r = _dbg.call(4470, 'read_ram', addr=f'0x{ptr_word:08X}', len=0x80, timeout=3) -b = bytes.fromhex(r['hex']) -print(f"\nContents of mem[{ptr_word:08X}..]:") -for i in range(0, len(b), 16): - line = ' '.join(f'{x:02x}' for x in b[i:i+16]) - print(f" +{i:02X}: {line}") - -state = struct.unpack_from('<H', b, 0x4E)[0] -print(f"\nstate_index = mem[ptr+0x4E] (halfword) = 0x{state:04X} ({state} decimal)") -if state < 6: - print(f" -> would dispatch via jump table 0x800E738C[{state}]") -else: - print(f" -> state >= 6, chain handler SKIPS (this is the 'do nothing' branch)") - -# Read the jump table itself to see what each state would do -jt = _dbg.call(4470, 'read_ram', addr='0x800E738C', len=6*4, timeout=3) -jtb = bytes.fromhex(jt['hex']) -print(f"\nJump table 0x800E738C (6 entries):") -for i in range(6): - tgt = struct.unpack_from('<I', jtb, i*4)[0] - print(f" state={i}: target=0x{tgt:08X}") diff --git a/tools/_options_session/_read_tomba_fn.py b/tools/_options_session/_read_tomba_fn.py deleted file mode 100644 index 9815d622..00000000 --- a/tools/_options_session/_read_tomba_fn.py +++ /dev/null @@ -1,35 +0,0 @@ -import sys, struct -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -addr = int(sys.argv[1], 16) if len(sys.argv) > 1 else 0x0005B40C -size = int(sys.argv[2], 0) if len(sys.argv) > 2 else 64 -r = _dbg.call(4470, 'read_ram', addr=f'0x{addr:08X}', len=size, timeout=3) -b = bytes.fromhex(r['hex']) -for i in range(0, len(b), 4): - insn = struct.unpack_from('<I', b, i)[0] - op = (insn>>26)&0x3F - rs = (insn>>21)&0x1F - rt = (insn>>16)&0x1F - rd = (insn>>11)&0x1F - imm16 = insn&0xFFFF - if imm16 > 0x7FFF: imm16 -= 0x10000 - funct = insn&0x3F - target26 = insn & 0x3FFFFFF - s = f'op={op:02x} rs={rs} rt={rt} rd={rd} imm={imm16:#x} funct={funct:02x}' - # Try basic decoding - if op == 0: - if funct == 9: s = f'jalr r{rs}' - elif funct == 8: s = f'jr r{rs}' - elif funct == 0: s = 'nop' - else: s = f'SPECIAL funct={funct:#x}' - elif op == 9: s = f'addiu r{rt}, r{rs}, {imm16}' - elif op == 0xD: s = f'ori r{rt}, r{rs}, 0x{insn&0xFFFF:x}' - elif op == 0xF: s = f'lui r{rt}, 0x{insn&0xFFFF:x}' - elif op == 0x2: s = f'j 0x{target26<<2:08x}' - elif op == 0x3: s = f'jal 0x{target26<<2:08x}' - elif op == 0x4: s = f'beq r{rs}, r{rt}, off={imm16:#x}' - elif op == 0x5: s = f'bne r{rs}, r{rt}, off={imm16:#x}' - elif op == 0x23: s = f'lw r{rt}, {imm16:#x}(r{rs})' - elif op == 0x2B: s = f'sw r{rt}, {imm16:#x}(r{rs})' - print(f' 0x{addr+i:08X}: {insn:08x} {s}') diff --git a/tools/_options_session/_read_word.py b/tools/_options_session/_read_word.py deleted file mode 100644 index c93e9bdf..00000000 --- a/tools/_options_session/_read_word.py +++ /dev/null @@ -1,6 +0,0 @@ -import sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg -addr = int(sys.argv[1], 16) -v = _dbg.read_word(4470, addr) -print(f"mem[0x{addr:08X}] = 0x{v:08X}") diff --git a/tools/_options_session/_region_breakdown.py b/tools/_options_session/_region_breakdown.py deleted file mode 100644 index e094810e..00000000 --- a/tools/_options_session/_region_breakdown.py +++ /dev/null @@ -1,46 +0,0 @@ -"""Break down fn_entry and wtrace by execution region.""" -import json, sys -from collections import Counter - -path = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(path) as f: - d = json.load(f) - -def region_of(addr): - if addr >= 0xBFC00000: - return 'BIOS-ROM' - if addr >= 0x10000: - return 'TOMBA' - return 'low-kernel' - -fn_c = Counter(e['func'] for e in d['fn_entry']) -print('Top 30 funcs by fn_entry count:') -for f, c in fn_c.most_common(30): - print(f' {f} {c:5} [{region_of(int(f, 16))}]') - -print() -pc_c = Counter(e['pc'] for e in d['wtrace_all']) -buckets = Counter() -for p, c in pc_c.items(): - buckets[region_of(int(p, 16))] += c -print('wtrace_all distribution by store-PC source:') -for r, c in buckets.most_common(): - print(f' {r:12} {c}') - -print() -# Show top Tomba-region PCs (writes from Tomba code) -tomba_pcs = [(p, c) for p, c in pc_c.items() if region_of(int(p, 16)) == 'TOMBA'] -tomba_pcs.sort(key=lambda x: -x[1]) -print('Top 15 TOMBA-region store PCs:') -for p, c in tomba_pcs[:15]: - print(f' {p} {c}') - -# Same for low-kernel -kern_pcs = [(p, c) for p, c in pc_c.items() if region_of(int(p, 16)) == 'low-kernel'] -kern_pcs.sort(key=lambda x: -x[1]) -print() -print('Top 15 low-kernel-region store PCs:') -for p, c in kern_pcs[:15]: - print(f' {p} {c}') diff --git a/tools/_options_session/_runtime_screenshot.py b/tools/_options_session/_runtime_screenshot.py deleted file mode 100644 index 3c911075..00000000 --- a/tools/_options_session/_runtime_screenshot.py +++ /dev/null @@ -1,73 +0,0 @@ -"""Pull runtime screenshot (multi-line response: meta + 224 row JSON lines).""" -import socket, json, time, sys - -PORT = 4470 if len(sys.argv) < 2 else int(sys.argv[1]) - -s = socket.create_connection(('127.0.0.1', PORT), timeout=15) -s.sendall((json.dumps({'id': 1, 'cmd': 'screenshot'}) + '\n').encode()) -s.settimeout(8) -buf = b'' -meta = None -rows = [] -deadline = time.time() + 8 -while time.time() < deadline: - try: - chunk = s.recv(65536) - except socket.timeout: - break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: - obj = json.loads(line.decode()) - except Exception: - continue - if 'width' in obj and 'height' in obj and 'row' not in obj: - meta = obj - elif 'row' in obj: - rows.append(obj) - if meta and len(rows) >= meta.get('height', 0): - deadline = 0 - break -s.close() - -print(f"meta: {meta}") -print(f"rows received: {len(rows)}") -if not meta: - sys.exit(1) - -W, H = meta['width'], meta['height'] -# Build pixel array -pixels = [] -for r in rows[:H]: - hex_s = r.get('hex', '') - row_pixels = [] - for x in range(W): - if 4*x + 4 <= len(hex_s): - p = int(hex_s[4*x:4*x+4], 16) - row_pixels.append(p) - pixels.append(row_pixels) - -# Statistics -total = 0 -nonzero = 0 -unique = set() -hist = {} -for row in pixels: - for p in row: - total += 1 - if p != 0: - nonzero += 1 - unique.add(p) - hist[p] = hist.get(p, 0) + 1 -print(f"total pixels: {total}") -print(f"non-zero pixels: {nonzero} ({100*nonzero/max(1,total):.2f}%)") -print(f"unique colors: {len(unique)}") -print(f"top 10 colors:") -for p, n in sorted(hist.items(), key=lambda x: -x[1])[:10]: - # RGB555 -> R,G,B - r = (p & 0x1F) << 3 - g = ((p >> 5) & 0x1F) << 3 - b = ((p >> 10) & 0x1F) << 3 - print(f" 0x{p:04X} -> rgb({r:3},{g:3},{b:3}) {n} px ({100*n/total:.1f}%)") diff --git a/tools/_options_session/_runtime_slot_writes.py b/tools/_options_session/_runtime_slot_writes.py deleted file mode 100644 index 57b2b30f..00000000 --- a/tools/_options_session/_runtime_slot_writes.py +++ /dev/null @@ -1,46 +0,0 @@ -"""Watch runtime writes to mem[0x801FD800..0x801FD810] for 4 seconds. -If runtime never writes to it (only the BIOS-ChangeTh-save path), that -confirms divergence: DuckStation has state-transition writes there, -runtime does not.""" -import socket, json, time, collections - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while b'\n' not in buf: - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.partition(b'\n')[0].decode()) - -r = call(4470, 'wtrace_clear') -print(f"wtrace_clear: {r}") -r = call(4470, 'wtrace_arm', lo='0x801FD800', hi='0x801FD810') -print(f"wtrace_arm: {r}") - -print("Sampling 4 sec...") -time.sleep(4) - -r = call(4470, 'wtrace_dump', count=2000) -entries = r.get('entries', []) -total = r.get('total', 0) -print(f"\n=== runtime: total={total} entries_returned={len(entries)} ===") - -# Filter to our address range -hits = [] -for e in entries: - addr_s = e.get('addr', '0') - addr = int(addr_s, 16) if isinstance(addr_s, str) else addr_s - if 0x001FD800 <= addr <= 0x001FD810 or 0x801FD800 <= addr <= 0x801FD810: - hits.append(e) - -print(f"Hits in [0x801FD800..0x801FD810]: {len(hits)}") -seen = collections.Counter() -for e in hits[:40]: - key = (e.get('ra'), e.get('new')) - seen[key] += 1 - print(f" addr={e.get('addr')} old={e.get('old')} new={e.get('new')} ra={e.get('ra')}") - -call(4470, 'wtrace_clear') diff --git a/tools/_options_session/_search_ec_bytes.py b/tools/_options_session/_search_ec_bytes.py deleted file mode 100644 index 097cbd5c..00000000 --- a/tools/_options_session/_search_ec_bytes.py +++ /dev/null @@ -1,40 +0,0 @@ -"""Find where 'EC' bytes (0x45, 0x43) appear in runtime memory but not beetle. -That tells us if 0x43450000 at 0x8009B3D4 is from a stale buffer/string.""" -import sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -# Scan a wide range of low RAM (0x80010000..0x80100000) for any place where -# runtime has bytes "45 43" but beetle doesn't. -# Too wide; let's narrow to 0x8009B000..0x8009B600 (around the diverging struct). - -ranges = [ - (0x80097000, 0x800A0000, "fn ptr table region + 0x8009B... region"), - (0x801FD000, 0x801FE000, "TCB region"), -] - -for lo, hi, label in ranges: - print(f"=== {label}: 0x{lo:08X}..0x{hi:08X} ===") - chunk = 0x800 - for base in range(lo, hi, chunk): - sz = min(chunk, hi - base) - try: - r_rt = _dbg.call(4470, 'read_ram', addr=f'0x{base:08X}', len=sz, timeout=4) - r_be = _dbg.call(4380, 'read_ram', addr=f'0x{base:08X}', len=sz, timeout=4) - if not r_rt.get('ok') or not r_be.get('ok'): continue - bt_rt = bytes.fromhex(r_rt['hex']) - bt_be = bytes.fromhex(r_be['hex']) - for i in range(len(bt_rt) - 1): - if bt_rt[i] == 0x45 and bt_rt[i+1] == 0x43: - if bt_be[i] != 0x45 or bt_be[i+1] != 0x43: - # runtime has "EC", beetle doesn't - addr = base + i - ctx_lo = max(0, i-4) - ctx_hi = min(len(bt_rt), i+8) - rt_ctx = ' '.join(f'{x:02x}' for x in bt_rt[ctx_lo:ctx_hi]) - be_ctx = ' '.join(f'{x:02x}' for x in bt_be[ctx_lo:ctx_hi]) - print(f" 0x{addr:08X} RT: {rt_ctx}") - print(f" BE: {be_ctx}") - except Exception as e: - print(f" err at 0x{base:08X}: {e}") - break diff --git a/tools/_options_session/_show_stack.py b/tools/_options_session/_show_stack.py deleted file mode 100644 index 9450f13e..00000000 --- a/tools/_options_session/_show_stack.py +++ /dev/null @@ -1,23 +0,0 @@ -"""Show main_stack from a freeze dump.""" -import json, sys -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778826348.json' -with open(dump) as f: - d = json.load(f) - -print(f"=== {d['backend']} @ wall={d['wall_clock_epoch']} ===") -print(f" frame_count: {d['frame_count']}, psx_cyc: {d['psx_cycle_count']:,}") -print(f" wedge_kind: {d['wedge_kind']} ({d['wedge_kind_name']})") -print(f" in_exception: {d['in_exception']}, cur_fn: {d['current_func']}, last_store_pc: {d['last_store_pc']}") -print(f" exc_entries: {d['exception_entries']:,}, exc_reentry: {d['exception_reentry_blocks']:,}") -print(f" i_stat: {d['i_stat']}, i_mask: {d['i_mask']}") -print(f" sio_stat: {d['sio_stat']}, sio_ctrl: {d['sio_ctrl']}, sio_card_active: {d['sio_card_active']}, mc_max_state: {d['mc_max_state']}") -print() - -ms = d.get('main_stack', []) -print(f"=== main_stack ({len(ms)} frames) ===") -for f in ms: - s = f.get('symbol', '???') - o = f.get('displacement', '?') - m = f.get('module', '?') - print(f" [{f['depth']:>3}] {f['addr']} {m}!{s}+0x{int(o):X}" if o != '?' else f" [{f['depth']:>3}] {f['addr']} {m}!{s}") diff --git a/tools/_options_session/_smoke_normalized.py b/tools/_options_session/_smoke_normalized.py deleted file mode 100644 index 1414889d..00000000 --- a/tools/_options_session/_smoke_normalized.py +++ /dev/null @@ -1,61 +0,0 @@ -"""Smoke-test the normalized wtrace verb set on BOTH backends. - -Both backends must accept identical JSON for the same verbs. This -script issues the same six probes against runtime (4470) and beetle -(4380) and compares response shapes. Any divergence is a parity bug -in the debug server; fix it before relying on cross-backend tooling. -""" -from __future__ import annotations -import sys, os, json -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from _dbg import call, RUNTIME_PORT, BEETLE_PORT - - -def probe(name, **kw): - out = {} - for label, port in [("runtime", RUNTIME_PORT), ("beetle", BEETLE_PORT)]: - try: - out[label] = call(port, name, **kw) - except Exception as e: - out[label] = {"error": str(e)} - return out - - -def shape(r): - if not isinstance(r, dict): return type(r).__name__ - return sorted(k for k in r.keys() if k != "id") - - -print("== wtrace_disarm_all (idempotent reset of armed ranges) ==") -r = probe("wtrace_disarm_all") -print(" runtime:", r["runtime"]) -print(" beetle :", r["beetle"]) - -print("\n== wtrace_arm lo=0x80090C80 hi=0x80090DE0 ==") -r = probe("wtrace_arm", lo="0x80090C80", hi="0x80090DE0") -print(" runtime:", r["runtime"]) -print(" beetle :", r["beetle"]) - -print("\n== wtrace_ranges ==") -r = probe("wtrace_ranges") -print(" runtime shape:", shape(r["runtime"])) -print(" beetle shape:", shape(r["beetle"])) -print(" runtime count:", r["runtime"].get("count")) -print(" beetle count:", r["beetle"].get("count")) - -print("\n== wtrace_stats ==") -r = probe("wtrace_stats") -print(" runtime:", r["runtime"]) -print(" beetle :", r["beetle"]) - -print("\n== wtrace_dump count=3 ==") -r = probe("wtrace_dump", count=3) -for label in ("runtime", "beetle"): - resp = r[label] - print(f" {label} top-level shape:", shape(resp)) - entries = resp.get("entries", []) - print(f" {label} entries: {len(entries)}") - if entries: - e0 = entries[0] - print(f" {label} entry[0] keys:", sorted(e0.keys())) - print(f" {label} entry[0] sample: addr={e0.get('addr')} new={e0.get('new')} pc={e0.get('pc')} w={e0.get('w')} frame={e0.get('frame')}") diff --git a/tools/_options_session/_snapshot.py b/tools/_options_session/_snapshot.py deleted file mode 100644 index 8cb564a9..00000000 --- a/tools/_options_session/_snapshot.py +++ /dev/null @@ -1,19 +0,0 @@ -"""Compact one-line snapshot of runtime state for diff'ing across nav events.""" -import sys, json -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -label = sys.argv[1] if len(sys.argv) > 1 else "snap" - -p = _dbg.call(4470, "ping", timeout=2) -g = _dbg.call(4470, "gpu_state", timeout=2) -s = _dbg.call(4470, "sio_state", timeout=2) -irq = _dbg.call(4470, "irq_state", timeout=2) -mc = _dbg.call(4470, "mc_status", timeout=2) - -print(f"=== [{label}] fr={p['frame']} ===") -print(f" GPU: draw={g['gp0_draw']} fill={g['gp0_fill']} env={g['gp0_env']} copy={g['gp0_copy']} writes={g['gp0_writes']:,} disabled={g['disabled']}") -print(f" disp=({g['display_x']},{g['display_y']}) draw_area={g['draw_area']} gpustat={g['gpustat']}") -print(f" SIO: mc_probes={s['mc_probes']} mc_reads={s['mc_reads']} mc_max_state={s['mc_max_state']} tx_writes={s['tx_writes']} sio_stat={s['sio_stat']} sio_ctrl={s['sio_ctrl']} card_caller={s['mc_last_caller']}") -print(f" IRQ: i_stat={irq['i_stat']} i_mask={irq['i_mask']} pending={irq['pending']} cop0_sr={irq['cop0_sr']} IEc={irq['IEc']} IM2={irq['IM2']}") -print(f" MC: {mc}") diff --git a/tools/_options_session/_state_diff.py b/tools/_options_session/_state_diff.py deleted file mode 100644 index 18f466ed..00000000 --- a/tools/_options_session/_state_diff.py +++ /dev/null @@ -1,41 +0,0 @@ -import sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -print(f"{'thing':>40} {'runtime':>12} {'beetle':>12}") -for label, addr, width in [ - ("scratchpad+0x1D4 (ptr)", 0x1F8001D4, 'w'), - ("ptr+0x4C (full word w/ state)", 0, 'special_state'), - ("0x8009B3A0+0x34", 0x8009B3D4, 'w'), - ("0x8009B3A0+0x49 (byte at 0x49)", 0x8009B3E9, 'b'), - ("0x8009EB58", 0x8009EB58, 'w'), - ("frame count", -1, 'frame'), -]: - if width == 'frame': - rt = _dbg.call(4470, "ping", timeout=2).get('frame','?') - be = _dbg.call(4380, "ping", timeout=2).get('frame','?') - print(f" {label:>40} {rt:>12} {be:>12}") - continue - if width == 'special_state': - # state is halfword at ptr+0x4E - rt_ptr = _dbg.read_word(4470, 0x1F8001D4) - be_ptr = _dbg.read_word(4380, 0x1F8001D4) - rt_state_word = _dbg.read_word(4470, rt_ptr + 0x4C) - be_state_word = _dbg.read_word(4380, be_ptr + 0x4C) - rt_state = (rt_state_word >> 16) & 0xFFFF - be_state = (be_state_word >> 16) & 0xFFFF - print(f" {'state @ ptr+0x4E':>40} {rt_state:>12} {be_state:>12}") - continue - if width == 'w': - rt = _dbg.read_word(4470, addr) - be = _dbg.read_word(4380, addr) - mark = "" if rt == be else " *DIFF*" - print(f" {label:>40} 0x{rt:08X} 0x{be:08X}{mark}") - elif width == 'b': - rt = _dbg.read_word(4470, addr & ~3) - be = _dbg.read_word(4380, addr & ~3) - byte_off = addr & 3 - rt_byte = (rt >> (byte_off * 8)) & 0xFF - be_byte = (be >> (byte_off * 8)) & 0xFF - mark = "" if rt_byte == be_byte else " *DIFF*" - print(f" {label:>40} 0x{rt_byte:02X}{' '} 0x{be_byte:02X}{' '}{mark}") diff --git a/tools/_options_session/_struct_diff.py b/tools/_options_session/_struct_diff.py deleted file mode 100644 index 6c2d7387..00000000 --- a/tools/_options_session/_struct_diff.py +++ /dev/null @@ -1,35 +0,0 @@ -"""Side-by-side dump of mem[0x8009B3A0..0x8009B440] on runtime vs DuckStation.""" -import socket, json - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -def read(port, addr, n): - r = call(port, 'read_ram', addr=f'0x{addr:08X}', len=n) - return bytes.fromhex(r['hex']) - -BASE = 0x8009B3A0 -N = 0xA0 - -rt = read(4470, BASE, N) -du = read(4371, BASE, N) - -print(f"{'off':>5} {'runtime':>48} {'duckstation':>48}") -for i in range(0, N, 16): - rt_hex = ' '.join(f'{x:02x}' for x in rt[i:i+16]) - du_hex = ' '.join(f'{x:02x}' for x in du[i:i+16]) - rt_full = f"+0x{i:03X}" - diff_marker = ' ' - for j in range(16): - if rt[i+j] != du[i+j]: - diff_marker = '!!' - break - print(f" {rt_full:>5} RT: {rt_hex} {diff_marker} BE: {du_hex}") diff --git a/tools/_options_session/_symbolize.py b/tools/_options_session/_symbolize.py deleted file mode 100644 index ebc36385..00000000 --- a/tools/_options_session/_symbolize.py +++ /dev/null @@ -1,61 +0,0 @@ -"""Map captured stack offsets to nearest preceding symbol from nm output.""" -import subprocess, sys - -# captured stack-frame offsets (relative to runtime image base) -offsets = [ - ("[ 2] SleepEx caller", 0x000B2219), - ("[ 3]", 0x000B4E51), - ("[ 4]", 0x000BE7F7), - ("[ 5]", 0x00460B2D), - ("[ 6]", 0x0063ECC7), - ("[ 7]", 0x006B12B0), - ("[ 8]", 0x000D889C), - ("[ 9]", 0x0045FCF0), - ("[10] dispatcher", 0x0045FDC9), - ("[11]", 0x00011BC0A & 0xFFFFFF), # 0x11BC0A is unusual - ("[12] dispatcher", 0x0045FC97), - ("[14] dispatcher", 0x0045FDC9), - ("[15]", 0x001A37AB), - ("[16] dispatcher", 0x0045FC97), - ("[18]", 0x00142516), - ("[20]", 0x0013D8AC), - ("[23]", 0x00144B00), - ("[26]", 0x000B2B77), - ("[27]", 0x006BB07C), - ("[28]", 0x000B10C9), - ("[29] start", 0x000B1416), -] - -# Load nm output -nm = subprocess.run( - ['nm', 'F:/Projects/TombaRecomp/build/psx-runtime.exe'], - capture_output=True, text=True -).stdout - -# Build sorted list of (addr, name) for text symbols -syms = [] -for line in nm.splitlines(): - parts = line.split() - if len(parts) >= 3 and parts[1] in ('t','T','w','W'): - try: - a = int(parts[0], 16) - except: - continue - # filter to image-base range (0x140000000+ standard mingw) - if 0x140000000 <= a < 0x150000000: - syms.append((a - 0x140000000, ' '.join(parts[2:]))) -syms.sort() -print(f"Loaded {len(syms)} text symbols") - -import bisect -addrs = [a for a,_ in syms] -for label, off in offsets: - i = bisect.bisect_right(addrs, off) - 1 - if i < 0: - print(f" {label}: offset 0x{off:08X} -> (before all symbols)") - continue - sym_off, sym_name = syms[i] - disp = off - sym_off - # only show if displacement is reasonable (< 0x10000 = 64KB) - note = "" if disp < 0x10000 else " (LARGE displacement — probably between symbols)" - print(f" {label}: offset 0x{off:08X} -> {sym_name} + 0x{disp:X}{note}") diff --git a/tools/_options_session/_thread_audit.py b/tools/_options_session/_thread_audit.py deleted file mode 100644 index 3e671ce9..00000000 --- a/tools/_options_session/_thread_audit.py +++ /dev/null @@ -1,44 +0,0 @@ -"""Inspect thread_trace and look for fiber switches near the freeze.""" -import json, sys, collections -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778822828.json' -with open(dump) as f: - d = json.load(f) - -tt = d.get('thread_trace', []) -print(f"thread_trace: {len(tt)} entries") -if tt: - print(f" first: {tt[0]}") - print(f" last : {tt[-1]}") - kinds = collections.Counter(e.get('kind') for e in tt) - print(f" kind distribution: {kinds}") - print() - print(f"Last 30 thread_trace entries:") - for e in tt[-30:]: - print(f" {e}") - -# Count 0x650 dispatches (SwitchThread) -db = d.get('dirty_block', []) -print() -print(f"=== 0x650 (SwitchThread) calls in dirty_block ring ===") -sw = [e for e in db if e['target'] == '0x00000650'] -print(f" count: {len(sw)}") -for e in sw[-15:]: - print(f" seq={e['seq']} fr={e['frame']} target=0x650 ra={e['ra']} " - f"a0={e['a0']} a1={e['a1']} sp={e['sp']}") - -# Look in fn_entry for 0x650 to count and see all params -fe = d.get('fn_entry', []) -print() -sw2 = [e for e in fe if e['func'] == '0x00000650'] -print(f"=== 0x650 calls in fn_entry: {len(sw2)} ===") -for e in sw2[-15:]: - print(f" seq={e['seq']} fr={e['frame']} ra={e['ra']} a0={e['a0']} a1={e['a1']}") - -# Look at any function in fn_entry with a0=3 (likely ChangeThread syscalls) -a0_3 = [e for e in fe if e.get('a0') == '0x00000003'] -print() -print(f"=== fn_entry with a0=0x3 (ChangeThread param) ===") -print(f" count: {len(a0_3)}") -for e in a0_3[-15:]: - print(f" seq={e['seq']} fr={e['frame']} func={e['func']} ra={e['ra']} a1={e['a1']}") diff --git a/tools/_options_session/_thread_live.py b/tools/_options_session/_thread_live.py deleted file mode 100644 index 748ee11f..00000000 --- a/tools/_options_session/_thread_live.py +++ /dev/null @@ -1,29 +0,0 @@ -"""Query live thread_trace and see if ChangeTh ping-pong is active now.""" -import sys, json, collections -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -try: - tt = _dbg.call(4470, "thread_trace", count=512, timeout=4) - if not tt.get('ok'): - print(f"err: {tt}") - sys.exit(1) - es = tt.get('entries', []) - print(f"thread_trace: {len(es)} entries (total={tt.get('total')}, available={tt.get('available')})") - if not es: - print("(empty)") - sys.exit(0) - # Last 20 - print(f"Last 20 (newest first):") - for e in es[:20]: - print(f" seq={e.get('seq')} fr={e.get('frame')} kind={e.get('kind')} name={e.get('name')} " - f"cur={e.get('current_tcb')} tgt={e.get('target_tcb')} pc={e.get('target_pc')}") - - # Pairs of (cur_tcb, target_tcb) - pairs = collections.Counter((e['current_tcb'], e['target_tcb'], e['name']) for e in es) - print() - print("Top transition signatures:") - for (c,t,n), cnt in pairs.most_common(15): - print(f" {n}: {c} -> {t}: {cnt}") -except Exception as e: - print(f"err: {e}") diff --git a/tools/_options_session/_throughput_analysis.py b/tools/_options_session/_throughput_analysis.py deleted file mode 100644 index b7fbcf7b..00000000 --- a/tools/_options_session/_throughput_analysis.py +++ /dev/null @@ -1,108 +0,0 @@ -"""Compute per-byte cycle cost and per-frame breakdown from the freeze dump.""" -import json, sys, collections - -dump = sys.argv[1] if len(sys.argv) > 1 else \ - 'F:/Projects/TombaRecomp/psx_freeze_dump_psx-runtime_1778800465.json' - -with open(dump) as f: - d = json.load(f) - -print("=== Core state at dump time ===") -print(f" frame_count: {d['frame_count']}") -print(f" psx_cycle_count: {d['psx_cycle_count']:,}") -print(f" exc_reentry_blocks: {d['exception_reentry_blocks']:,}") -print(f" exc_entries: {d['exception_entries']:,}") -print(f" dispatch_count: {d['dispatch_count']:,}") -print(f" dirty_ram_blocks: {d['dirty_ram_blocks']:,}") -print(f" dirty_ram_insns: {d['dirty_ram_insns']:,}") -print() - -# Heartbeat ring: psx_cycle_count over time -hb = d['heartbeat_ring'] -print(f"=== Heartbeat ring ({len(hb)} entries) ===") -if len(hb) >= 2: - # entries are time-ordered, oldest first based on 'wall' - hb_sorted = sorted(hb, key=lambda e: e['wall']) - first, last = hb_sorted[0], hb_sorted[-1] - cyc_delta = last['cyc'] - first['cyc'] - fr_delta = last['frame'] - first['frame'] - exc_re_delta = last['exc_re'] - first['exc_re'] - dirty_delta = last['dirty_insns'] - first['dirty_insns'] - wall_delta = last['wall'] - first['wall'] # seconds (integer) - print(f" wall delta: {wall_delta} sec") - print(f" cycle delta: {cyc_delta:,} ({cyc_delta/33868800:.2f}s of PSX time)") - print(f" frame delta: {fr_delta}") - print(f" exc_re delta: {exc_re_delta:,}") - print(f" dirty_insns delta: {dirty_delta:,}") - print() - if wall_delta: - print(f" frames/sec (wall): {fr_delta/wall_delta:.1f}") - print(f" PSX cycles/sec (wall): {cyc_delta/wall_delta:,.0f} (real PSX = 33,868,800)") - print(f" speed pct: {(cyc_delta/wall_delta)/33868800*100:.1f}%") - print(f" dirty_insns/sec: {dirty_delta/wall_delta:,.0f}") - print(f" exc_re/sec: {exc_re_delta/wall_delta:,.0f}") - -# Count buffer stores per frame from wtrace -events = d['wtrace_all'] -# Newest events; estimate window -seqs = [e['seq'] for e in events] -print() -print(f"=== wtrace_all window ===") -print(f" seq range: {min(seqs):,} .. {max(seqs):,}") - -frames_seen = sorted({e['frame'] for e in events}) -print(f" frame range: {min(frames_seen)} .. {max(frames_seen)} ({len(frames_seen)} distinct)") - -# stores per frame -STORE_PCS_UPPER = {'0X00006524', '0XBFC15590'} -stores_per_frame = collections.Counter() -total_buffer_stores = 0 -for e in events: - pc = e['pc'].upper() - if pc in STORE_PCS_UPPER and 0xBE48 <= int(e['addr'], 16) <= 0xBEC7: - stores_per_frame[e['frame']] += 1 - total_buffer_stores += 1 -print(f" card-buffer stores total: {total_buffer_stores}") -print(f" per frame:") -for fr in sorted(stores_per_frame): - print(f" fr {fr}: {stores_per_frame[fr]} stores") - -# How many bytes did SIO process? Look at sio_pc_trace for tx_data writes (1F801040) -sio_pc = d['sio_pc_trace'] -tx_writes = [e for e in sio_pc if int(e['addr'], 16) == 0x1F801040] -print() -print(f"=== SIO TX activity ===") -print(f" total sio_pc_trace entries: {len(sio_pc)}") -print(f" TX writes (to 0x1F801040): {len(tx_writes)}") -tx_seqs = [e['seq'] for e in tx_writes] -if tx_seqs: - seq_window = max(tx_seqs) - min(tx_seqs) - print(f" TX byte_seq range: {min(tx_seqs):,} .. {max(tx_seqs):,} (span {seq_window:,})") - -# What's the largest cycle gap between successive heartbeat entries? -print() -print(f"=== Heartbeat: per-tick analysis ===") -for i in range(1, min(8, len(hb_sorted))): - a, b = hb_sorted[i-1], hb_sorted[i] - cd = b['cyc'] - a['cyc'] - fd = b['frame'] - a['frame'] - rd = b['exc_re'] - a['exc_re'] - dd = b['dirty_insns'] - a['dirty_insns'] - print(f" tick {i}: cyc+{cd:,} fr+{fd} exc_re+{rd} dirty+{dd:,} cur_fn={a['cur_fn']} in_exc={a['in_exc']}") -print(" ...") -for i in range(max(1, len(hb_sorted)-6), len(hb_sorted)): - a, b = hb_sorted[i-1], hb_sorted[i] - cd = b['cyc'] - a['cyc'] - fd = b['frame'] - a['frame'] - rd = b['exc_re'] - a['exc_re'] - dd = b['dirty_insns'] - a['dirty_insns'] - print(f" tick {i}: cyc+{cd:,} fr+{fd} exc_re+{rd} dirty+{dd:,} cur_fn={a['cur_fn']} in_exc={a['in_exc']}") - -# Function entry counts — what's running most -print() -print(f"=== Function entry hotspots ===") -fe = d['fn_entry'] -fc = collections.Counter(e['func'] for e in fe) -print(f" Total fn_entry events: {len(fe)}") -for func, ct in fc.most_common(15): - print(f" {func}: {ct} calls") diff --git a/tools/_options_session/_track_display_origin.py b/tools/_options_session/_track_display_origin.py deleted file mode 100644 index a9df3638..00000000 --- a/tools/_options_session/_track_display_origin.py +++ /dev/null @@ -1,48 +0,0 @@ -"""Track display_x/y over time on runtime + duck. -If double-buffering is working, display origin should toggle between -two values every frame (or every other frame).""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(2) - buf = b'' - out = None - deadline = time.time() + 3 - while time.time() < deadline: - try: - chunk = s.recv(65536) - except socket.timeout: - break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: - obj = json.loads(line.decode()) - except Exception: - continue - if obj.get('id') == 1: - out = obj - deadline = 0 - break - s.close() - return out or {'err': 'no response'} - -for _ in range(3): - call(4371, 'mem_hit_clear') - -print("Sample display origin + draw offset 12 times over 2s:") -for i in range(12): - rt = call(4470, 'gpu_state') - dk = call(4371, 'gpu_state') - rt_disp = f"({rt.get('display_x')},{rt.get('display_y')})" - rt_draw = f"({rt.get('draw_offset',[None,None])[0]},{rt.get('draw_offset',[None,None])[1]})" - dk_disp = f"({dk.get('display_area_x')},{dk.get('display_area_y')})" - dk_draw = f"({dk.get('draw_offset_x')},{dk.get('draw_offset_y')})" - rt_stat = rt.get('gpustat') - dk_stat = dk.get('gpustat') - print(f" [{i:2}] runtime: disp={rt_disp} draw={rt_draw} stat={rt_stat} | " - f"duck: disp={dk_disp} draw={dk_draw} stat={dk_stat}") - time.sleep(0.16) diff --git a/tools/_options_session/_verify_state0_lock.py b/tools/_options_session/_verify_state0_lock.py deleted file mode 100644 index a502c9ed..00000000 --- a/tools/_options_session/_verify_state0_lock.py +++ /dev/null @@ -1,67 +0,0 @@ -"""Verify the previously-identified state-0 lock still holds. -Check mem[0x8009B3D4], state byte, scratchpad ptr, jump table.""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(3); buf=b''; out=None; deadline=time.time()+4 - while time.time() < deadline: - try: chunk = s.recv(65536) - except socket.timeout: break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: obj = json.loads(line.decode()) - except: continue - if obj.get('id') == 1: out = obj; deadline = 0; break - s.close() - return out or {'err': 'no resp'} - -def rd32(port, addr): - r = call(port, 'read_ram', addr=f'0x{addr:08X}', len=4) - if 'hex' not in r: return None - return int.from_bytes(bytes.fromhex(r['hex']), 'little') - -# Drain duck -for _ in range(5): call(4371, 'mem_hit_clear') - -print("== mem[0x1F8001D4] (scratchpad ptr to state struct) ==") -print(f" runtime: 0x{rd32(4470, 0x1F8001D4):08X}") -print(f" duck: 0x{rd32(4371, 0x1F8001D4):08X}") - -print() -print("== mem[0x8009B3D4] (struct + 0x34, holds 'EC' on runtime) ==") -r_val = rd32(4470, 0x8009B3D4) -d_val = rd32(4371, 0x8009B3D4) -print(f" runtime: 0x{r_val:08X}") -print(f" duck: 0x{d_val:08X}") -print(f" diff: {'YES' if r_val != d_val else 'no'}") - -print() -print("== mem[0x8009B3A0..+0x80] struct contents ==") -print(" runtime:") -r = call(4470, 'read_ram', addr='0x8009B3A0', len=0x80) -if 'hex' in r: - b = bytes.fromhex(r['hex']) - for i in range(0, len(b), 16): - chunk = b[i:i+16] - words = [int.from_bytes(chunk[k:k+4],'little') for k in range(0, 16, 4)] - ascii_repr = ''.join(chr(x) if 32 <= x < 127 else '.' for x in chunk) - print(f" +0x{i:02X}: {' '.join(f'{w:08X}' for w in words)} |{ascii_repr}|") - -print(" duck:") -r = call(4371, 'read_ram', addr='0x8009B3A0', len=0x80) -if 'hex' in r: - b = bytes.fromhex(r['hex']) - for i in range(0, len(b), 16): - chunk = b[i:i+16] - words = [int.from_bytes(chunk[k:k+4],'little') for k in range(0, 16, 4)] - ascii_repr = ''.join(chr(x) if 32 <= x < 127 else '.' for x in chunk) - print(f" +0x{i:02X}: {' '.join(f'{w:08X}' for w in words)} |{ascii_repr}|") - -print() -print("== mem[0x80097524] (fn-ptr) ==") -print(f" runtime: 0x{rd32(4470, 0x80097524):08X} (expected 0x8006B028)") -print(f" duck: 0x{rd32(4371, 0x80097524):08X}") diff --git a/tools/_options_session/_vram_peek_both.py b/tools/_options_session/_vram_peek_both.py deleted file mode 100644 index 07a94f45..00000000 --- a/tools/_options_session/_vram_peek_both.py +++ /dev/null @@ -1,61 +0,0 @@ -"""Peek at both double-buffer halves on runtime + duck. -Runtime claims display=(384,256). Both halves should be in y>=256, x=[384..1023].""" -import socket, json, time - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - s.settimeout(3) - buf = b'' - out = None - deadline = time.time() + 4 - while time.time() < deadline: - try: - chunk = s.recv(65536) - except socket.timeout: break - if not chunk: break - buf += chunk - while b'\n' in buf: - line, _, buf = buf.partition(b'\n') - try: - obj = json.loads(line.decode()) - except Exception: continue - if obj.get('id') == 1: - out = obj - deadline = 0 - break - s.close() - return out or {'err': 'no resp'} - -def vram_stats(port, x, y, w=64, h=8): - r = call(port, 'vram_peek', x=x, y=y, w=w, h=h) - if 'hex' not in r: - return None, r - h_s = r['hex'] - pixels = [int(h_s[i*4:i*4+4], 16) for i in range(len(h_s)//4)] - nz = sum(1 for p in pixels if p != 0) - return (nz, len(pixels), len(set(pixels)), pixels[:8]), r - -# Drain duck -for _ in range(3): - call(4371, 'mem_hit_clear') - -REGIONS = [ - ('buffer_A', 384, 256, 64, 8), - ('buffer_B', 704, 256, 64, 8), - ('tex_page_topL', 0, 0, 64, 8), - ('tex_page_320_0', 320, 0, 64, 8), - ('tex_page_640_0', 640, 0, 64, 8), - ('tex_page_0_128', 0, 128, 64, 8), - ('clut_320_240', 320, 240, 64, 8), -] -PORTS = [(4470, 'runtime'), (4371, 'duck')] -for name, x, y, w, h in REGIONS: - print(f"\n=== {name} (vram[{x},{y}] {w}x{h}) ===") - for port, pname in PORTS: - s, r = vram_stats(port, x, y, w, h) - if s: - nz, total, uniq, sample = s - print(f" {pname}: nonzero={nz}/{total} unique={uniq} sample={[hex(p) for p in sample]}") - else: - print(f" {pname}: ERR {r}") diff --git a/tools/_options_session/_walk_up_chain.py b/tools/_options_session/_walk_up_chain.py deleted file mode 100644 index aa828e02..00000000 --- a/tools/_options_session/_walk_up_chain.py +++ /dev/null @@ -1,41 +0,0 @@ -"""Walk up the call chain: arm fntrace at function entries to see who fires.""" -import socket, json, time, collections - -def call(port, cmd, **kw): - s = socket.create_connection(('127.0.0.1', port), timeout=5) - s.sendall((json.dumps({'id':1,'cmd':cmd, **kw})+'\n').encode()) - buf = b'' - while not buf.endswith(b'\n'): - c = s.recv(65536) - if not c: break - buf += c - s.close() - return json.loads(buf.decode()) - -# Function entries to test, top → bottom -targets = [ - ('0x800163B0', 'tomba_main'), - ('0x80019844', 'FUN_80019844 (caller of 0x800223E0 via 0x80019898)'), - ('0x8001A51C', 'FUN_8001A51C (caller of 0x800223E0 via 0x8001A5D4)'), - ('0x8001A328', 'FUN_8001A328 (calls chain handler 0x800E75CC)'), - ('0x800223E0', '0x800223E0 (calls 0x80028B34)'), - ('0x80028B34', '0x80028B34 (calls 0x80068DDC)'), - ('0x80068DDC', '0x80068DDC (calls 0x8006A0C0 = init)'), - ('0x8006A0C0', '0x8006A0C0 (the init function)'), -] - -for t, _ in targets: - call(4470, 'fntrace_arm', target=t) - -print(f"Armed {len(targets)} targets. Waiting 5 sec...") -time.sleep(5) - -d = call(4470, 'fntrace_dump', count=5000) -es = d.get('entries', []) -funcs = collections.Counter(e.get('target') for e in es) - -print(f"{'target':>14} {'hits':>6} label") -for t, label in targets: - t_int = int(t, 16) - n = sum(funcs[k] for k in funcs if int(k, 16) == t_int) - print(f" {t:>12} {n:>6} {label}") diff --git a/tools/_options_session/_watch_display.py b/tools/_options_session/_watch_display.py deleted file mode 100644 index 22b64a79..00000000 --- a/tools/_options_session/_watch_display.py +++ /dev/null @@ -1,27 +0,0 @@ -"""Sample gpu_state every 500ms for 10 sec and see if display origin moves.""" -import time, json, sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -prev = None -for i in range(20): - g = _dbg.call(4470, "gpu_state", timeout=2) - sig = (g['display_x'], g['display_y'], g['draw_area'][0], g['draw_area'][1], - g['gp0_writes'], g['gp0_draw'], g['gp0_fill']) - # Get frame from ping - p = _dbg.call(4470, "ping", timeout=2) - fr = p.get('frame', '?') - sig_disp = f"disp=({g['display_x']},{g['display_y']}) draw_ofs=({g['draw_offset'][0]},{g['draw_offset'][1]}) draw_area=({g['draw_area'][0]},{g['draw_area'][1]}..{g['draw_area'][2]},{g['draw_area'][3]}) gpustat={g['gpustat']}" - sig_cnt = f"gp0_writes={g['gp0_writes']:,} gp0_draw={g['gp0_draw']} gp0_fill={g['gp0_fill']} gp0_copy={g['gp0_copy']} disabled={g['disabled']}" - print(f"[{i:2}] fr={fr:>5} {sig_disp}") - print(f" {sig_cnt}") - if prev and prev != sig: - diff = [] - for k in g: - if k in prev_full and prev_full[k] != g[k]: - diff.append(f"{k}: {prev_full[k]} -> {g[k]}") - if diff: - print(f" CHANGED: {', '.join(diff[:6])}") - prev = sig - prev_full = g - time.sleep(0.5) diff --git a/tools/_options_session/_watch_iec.py b/tools/_options_session/_watch_iec.py deleted file mode 100644 index 7eeeb668..00000000 --- a/tools/_options_session/_watch_iec.py +++ /dev/null @@ -1,17 +0,0 @@ -"""Sample SR / IRQ state to check if IEc ever goes back to 1.""" -import time, sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -print(f"{'i':>3} {'frame':>6} {'sr':>10} {'IEc':>3} {'IEp':>3} {'IEo':>3} " - f"{'i_stat':>10} {'i_mask':>10} {'pending':>10}") -for i in range(20): - p = _dbg.call(4470, "ping", timeout=2) - irq = _dbg.call(4470, "irq_state", timeout=2) - sr_v = int(irq['cop0_sr'], 16) - IEc = sr_v & 1 - IEp = (sr_v >> 2) & 1 - IEo = (sr_v >> 4) & 1 - print(f"{i:>3} {p['frame']:>6} {irq['cop0_sr']:>10} {IEc:>3} {IEp:>3} {IEo:>3} " - f"{irq['i_stat']:>10} {irq['i_mask']:>10} {irq['pending']:>10}") - time.sleep(0.5) diff --git a/tools/_options_session/_watch_state.py b/tools/_options_session/_watch_state.py deleted file mode 100644 index 75d75af2..00000000 --- a/tools/_options_session/_watch_state.py +++ /dev/null @@ -1,19 +0,0 @@ -"""Sample Tomba's state machine state every 500ms for 15 sec.""" -import time, sys -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -prev = None -for i in range(30): - ptr = _dbg.read_word(4470, 0x1F8001D4) - state_word = _dbg.read_word(4470, ptr + 0x4C) # halfword at +0x4E in upper16 - state = (state_word >> 16) & 0xFFFF - # Also read the first 16 bytes of the struct (likely has fields) - head = _dbg.read_word(4470, ptr + 0x00) - flag2 = _dbg.read_word(4470, ptr + 0x44) - p = _dbg.call(4470, "ping", timeout=2) - sig = (ptr, state, head, flag2) - star = " *CHANGED*" if prev and prev != sig else "" - print(f"[{i:2}] fr={p.get('frame','?')} ptr=0x{ptr:08X} state=0x{state:04X} head=0x{head:08X} flag2=0x{flag2:08X}{star}") - prev = sig - time.sleep(0.5) diff --git a/tools/_options_session/_wtrace_query.py b/tools/_options_session/_wtrace_query.py deleted file mode 100644 index 17c89d24..00000000 --- a/tools/_options_session/_wtrace_query.py +++ /dev/null @@ -1,13 +0,0 @@ -import sys, json, time -sys.path.insert(0, 'F:/Projects/psxrecomp/psxrecomp/tools/_options_session') -import _dbg - -# wtrace_dump returns ALL captured events for the armed ranges -d = _dbg.call(4470, "wtrace_dump", count=100, timeout=4) -es = d.get('entries', []) -print(f"wtrace_dump: {len(es)} entries") -for e in es[:20]: - print(f" seq={e.get('seq')} fr={e.get('frame')} pc={e.get('pc')} addr={e.get('addr')} ->{e.get('new')} func={e.get('func','?')} cpu_pc={e.get('cpu_pc','?')}") - -# Also look at wtrace_all filtered by addr (the always-on ring) -d2 = _dbg.call(4470, "wtrace_all_dump", count=100, timeout=4) if False else None diff --git a/tools/_parse_crash.py b/tools/_parse_crash.py index e55047d2..32c39964 100644 --- a/tools/_parse_crash.py +++ b/tools/_parse_crash.py @@ -2,8 +2,8 @@ dispatch_ring entries, last 5 unknown_dispatch entries, last 30 dirty_block_log entries. Highlight the relevant ones for the 0xBFC09790 caller investigation.""" -import json -with open(r'F:\Projects\psxrecomp\psx_last_run_report.json') as f: +import sys, json +with open(sys.argv[1] if len(sys.argv) > 1 else 'psx_last_run_report.json') as f: r = json.load(f) print(f"reason: {r.get('reason')}") diff --git a/tools/_psx_header_dump.py b/tools/_psx_header_dump.py index 9edd822a..552a08f5 100644 --- a/tools/_psx_header_dump.py +++ b/tools/_psx_header_dump.py @@ -1,5 +1,5 @@ import struct, os, sys -p = sys.argv[1] if len(sys.argv) > 1 else 'F:/Projects/TombaRecomp/tomba/SCUS_942.36' +p = sys.argv[1] if len(sys.argv) > 1 else '../TombaRecomp/tomba/SCUS_942.36' with open(p, 'rb') as f: h = f.read(0x800) magic = h[:8] diff --git a/tools/_tomba_bios_thunks.py b/tools/_tomba_bios_thunks.py index 3bbc3d46..af82c627 100644 --- a/tools/_tomba_bios_thunks.py +++ b/tools/_tomba_bios_thunks.py @@ -11,7 +11,7 @@ with a trailing pad/nop, but we accept either stride. import struct, sys, os BIOS_BASE = 0x80010000 -PATH = sys.argv[1] if len(sys.argv) > 1 else 'F:/Projects/TombaRecomp/tomba/SCUS_942.36_no_header' +PATH = sys.argv[1] if len(sys.argv) > 1 else '../TombaRecomp/tomba/SCUS_942.36_no_header' # Standard PSX BIOS function names. Source: docs/psx_bios_disasm.txt + nocash. A0 = { @@ -150,12 +150,12 @@ for a0, kind in seen_sc: print(f" syscall a0=0x{a0:X} {kind:30s} ({cnt} sites)") # Persist to seeds dir for later use by the audit -out_dir = 'F:/Projects/TombaRecomp/seeds' +out_dir = '../TombaRecomp/seeds' os.makedirs(out_dir, exist_ok=True) out = os.path.join(out_dir, 'tomba_bios_thunks.txt') with open(out, 'w') as f: f.write(f"# Tomba PSX BIOS thunk inventory\n") - f.write(f"# Source: F:/Projects/TombaRecomp/tomba/SCUS_942.36_no_header\n") + f.write(f"# Source: ../TombaRecomp/tomba/SCUS_942.36_no_header\n") f.write(f"# Pattern: li t2, 0x{{A0|B0|C0}}; jr t2; li t1, FN (PSY-Q libapi/libetc style)\n") f.write(f"# Total thunks: {len(results)}\n\n") for vec in ('A', 'B', 'C'): diff --git a/tools/audio_ab_diff.py b/tools/audio_ab_diff.py new file mode 100644 index 00000000..9b1a63a5 --- /dev/null +++ b/tools/audio_ab_diff.py @@ -0,0 +1,350 @@ +#!/usr/bin/env python3 +"""audio_ab_diff.py - drift-tolerant A/B comparison of recomp audio vs an +accuracy-grade emulator reference. + +WHY drift-tolerant (not bit-exact): the SNES recomp drives its SPC700/S-DSP on +a *synthetic* clock (cpu_state.c pace_cycles = +256 master cycles per HW-register +touch, converted to APU cycles), not a real cycle count. So even a perfect DSP +will phase-drift against hardware. A sample-exact diff is therefore meaningless +here; what matters is whether the *music* matches: same notes, same onsets, same +pitch, same timbre, with bounded time drift. This tool measures exactly those. + +Inputs are two WAVs at (possibly) different native rates: + - REF : the oracle. Best = bsnes Accuracy via tools/snesref (embeds blargg's + cycle-exact S-DSP). bsnes libretro emits 48000 Hz. + - TEST : the recomp. Dump its always-on native PCM ring (32040 Hz) via the + debug server: audio_wav <path> (runner/src/audio_trace.c). + +Both are resampled to a common rate (default 32040; SNES content is band-limited +well below 16 kHz so nothing is lost). Then: + 1. global alignment - FFT cross-correlation -> best lag + peak correlation + 2. drift - per-window local lag; a growing lag = clock/tempo drift + (reported as ms/s and ppm). This is the headline SNES + metric: it quantifies "off-cue". + 3. onset timing - spectral-flux onsets matched across streams -> timing + error distribution (median / IQR / p90), in ms. + 4. pitch / tuning - dominant-pitch track ratio in cents -> "off-tune". + 5. timbre - log-spectral distance + spectral-centroid delta. + 6. per-stream quality- click rate + noise floor (reused round-2 detectors). + +Only dependency is numpy. Usage: + python audio_ab_diff.py --ref REF.wav --test TEST.wav [--rate 32040] + [--start-s 0] [--dur-s 10] [--json out.json] +""" +import sys, os, wave, argparse, json +import numpy as np + + +# ----------------------------------------------------------------------------- io +def load_wav(path): + """Robust WAV loader: tolerates an UNFINALIZED header (data-size==0 or wrong) + from a force-killed capture by taking all bytes after the data chunk start. + Falls back to raw parse if the stdlib wave module rejects the header.""" + try: + with wave.open(path, "rb") as w: + ch, rate, n = w.getnchannels(), w.getframerate(), w.getnframes() + raw = w.readframes(n) + if len(raw) >= (n * ch * 2) and n > 0: + x = np.frombuffer(raw, dtype="<i2").astype(np.float64) / 32768.0 + return (x.reshape(-1, ch).mean(axis=1) if ch > 1 else x), rate + except Exception: + pass + # manual parse + b = open(path, "rb").read() + assert b[:4] == b"RIFF" and b[8:12] == b"WAVE", "not a WAV" + ch, rate, bits, off = 2, 32040, 16, 12 + data_off, data_len = None, 0 + while off + 8 <= len(b): + cid = b[off:off + 4]; sz = int.from_bytes(b[off + 4:off + 8], "little"); body = off + 8 + if cid == b"fmt ": + ch = int.from_bytes(b[body + 2:body + 4], "little") + rate = int.from_bytes(b[body + 4:body + 8], "little") + bits = int.from_bytes(b[body + 14:body + 16], "little") + elif cid == b"data": + data_off = body + data_len = sz if 0 < sz <= len(b) - body else (len(b) - body) + break + off = body + sz + (sz & 1) + raw = b[data_off:data_off + (data_len - data_len % (ch * (bits // 8)))] + x = np.frombuffer(raw, dtype="<i2").astype(np.float64) / 32768.0 + return (x.reshape(-1, ch).mean(axis=1) if ch > 1 else x), rate + + +def resample_linear(x, src, dst): + if src == dst or x.size == 0: + return x + n_out = int(round(x.size * dst / src)) + t = np.arange(n_out) * (src / dst) + i0 = np.floor(t).astype(int) + i0 = np.clip(i0, 0, x.size - 2) + frac = t - i0 + return x[i0] * (1.0 - frac) + x[i0 + 1] * frac + + +def active_region(x, thr=1e-4): + nz = np.where(np.abs(x) > thr)[0] + if nz.size == 0: + return 0, x.size + return int(nz[0]), int(nz[-1]) + 1 + + +# --------------------------------------------------------------------- alignment +def best_lag(a, b, max_lag): + """Lag (in samples) that best aligns b onto a, via FFT cross-correlation, + restricted to |lag| <= max_lag. Returns (lag, peak_norm_corr).""" + n = 1 << int(np.ceil(np.log2(a.size + b.size))) + fa = np.fft.rfft(a - a.mean(), n) + fb = np.fft.rfft(b - b.mean(), n) + cc = np.fft.irfft(fa * np.conj(fb), n) + cc = np.concatenate([cc[-max_lag:], cc[: max_lag + 1]]) + lags = np.arange(-max_lag, max_lag + 1) + k = int(np.argmax(cc)) + denom = (np.linalg.norm(a) * np.linalg.norm(b)) or 1.0 + return int(lags[k]), float(cc[k] / denom) + + +def drift_track(a, b, rate, win_s=0.75, hop_s=0.375, search_ms=120): + """Local lag per window -> drift slope. a,b already globally aligned.""" + win = int(win_s * rate); hop = int(hop_s * rate); ml = int(search_ms * rate / 1000) + n = min(a.size, b.size) + centers, lags, corrs = [], [], [] + pos = 0 + while pos + win <= n: + aw = a[pos:pos + win]; bw = b[pos:pos + win] + if np.linalg.norm(aw) > 1e-3 and np.linalg.norm(bw) > 1e-3: + lag, c = best_lag(aw, bw, ml) + centers.append((pos + win / 2) / rate); lags.append(lag / rate * 1000.0); corrs.append(c) + pos += hop + if len(centers) < 3: + return None + centers = np.array(centers); lags = np.array(lags); corrs = np.array(corrs) + # weighted least-squares slope (weight by local correlation quality) + wts = np.clip(corrs, 0, None) + if wts.sum() < 1e-6: + wts = np.ones_like(corrs) + A = np.vstack([centers, np.ones_like(centers)]).T + W = np.diag(wts) + coef = np.linalg.lstsq(W @ A, W @ lags, rcond=None)[0] + slope_ms_per_s = float(coef[0]) + return { + "n_windows": len(centers), + "lag_ms_mean": float(lags.mean()), + "lag_ms_std": float(lags.std()), + "drift_ms_per_s": slope_ms_per_s, + "drift_ppm": slope_ms_per_s * 1000.0, # ms/s -> ppm + "local_corr_median": float(np.median(corrs)), + } + + +# ------------------------------------------------------------------------ onsets +def onsets(x, rate, fft=1024, hop=256, k=1.5): + if x.size < fft: + return np.array([]) + win = np.hanning(fft) + frames = 1 + (x.size - fft) // hop + mag = np.empty((frames, fft // 2 + 1)) + for i in range(frames): + seg = x[i * hop:i * hop + fft] * win + mag[i] = np.abs(np.fft.rfft(seg)) + flux = np.maximum(0.0, np.diff(mag, axis=0)).sum(axis=1) + if flux.size == 0: + return np.array([]) + flux /= (flux.max() or 1.0) + # adaptive threshold: local mean + k*local std + w = 16 + thr = np.array([flux[max(0, i - w):i + w].mean() + k * flux[max(0, i - w):i + w].std() + for i in range(flux.size)]) + pk = (flux[1:-1] > flux[:-2]) & (flux[1:-1] >= flux[2:]) & (flux[1:-1] > thr[1:-1]) + idx = np.where(pk)[0] + 1 + return idx * hop / rate # seconds + + +def onset_match(ref_t, test_t, lag_s, tol_ms=50): + """Match test onsets (shifted by -lag) to nearest ref onset; report errors.""" + if ref_t.size == 0 or test_t.size == 0: + return None + tt = test_t - lag_s + errs = [] + matched = 0 + for t in tt: + d = np.abs(ref_t - t) + j = int(np.argmin(d)) + if d[j] * 1000.0 <= tol_ms: + errs.append((t - ref_t[j]) * 1000.0); matched += 1 + if not errs: + return {"matched": 0, "ref_onsets": int(ref_t.size), "test_onsets": int(test_t.size)} + errs = np.array(errs) + return { + "ref_onsets": int(ref_t.size), "test_onsets": int(test_t.size), + "matched": matched, + "match_rate": matched / max(1, test_t.size), + "err_ms_median": float(np.median(errs)), + "err_ms_abs_median": float(np.median(np.abs(errs))), + "err_ms_iqr": float(np.percentile(errs, 75) - np.percentile(errs, 25)), + "err_ms_p90": float(np.percentile(np.abs(errs), 90)), + } + + +# ---------------------------------------------------------------- pitch & timbre +def avg_spectrum(x, rate, fft=4096, hop=2048): + if x.size < fft: + return None, None + win = np.hanning(fft); frames = 1 + (x.size - fft) // hop + acc = np.zeros(fft // 2 + 1) + for i in range(frames): + acc += np.abs(np.fft.rfft(x[i * hop:i * hop + fft] * win)) + acc /= frames + freqs = np.fft.rfftfreq(fft, 1.0 / rate) + return freqs, acc + + +def dominant_pitch_track(x, rate, fft=4096, hop=2048, fmin=80, fmax=2000): + if x.size < fft: + return np.array([]) + win = np.hanning(fft); frames = 1 + (x.size - fft) // hop + freqs = np.fft.rfftfreq(fft, 1.0 / rate) + band = (freqs >= fmin) & (freqs <= fmax) + out = [] + for i in range(frames): + sp = np.abs(np.fft.rfft(x[i * hop:i * hop + fft] * win)) + if sp[band].max() < 1e-3: + continue + out.append(freqs[band][int(np.argmax(sp[band]))]) + return np.array(out) + + +def cents(ratio): + return 1200.0 * np.log2(ratio) if ratio > 0 else float("nan") + + +# ----------------------------------------------------------- per-stream quality +def click_rate(x, rate, k=6.0, rms_ms=10.0): + if x.size < 4: + return 0.0 + d1 = np.abs(np.diff(x, prepend=x[0])); d2 = np.abs(np.diff(x, n=2, prepend=[x[0], x[0]])) + w = max(8, int(rate * rms_ms / 1000.0)) + cs = np.cumsum(np.concatenate([[0.0], x * x])) + lo = np.maximum(0, np.arange(x.size) - w); hi = np.minimum(x.size, np.arange(x.size) + w) + rms = np.sqrt((cs[hi] - cs[lo]) / np.maximum(1, hi - lo)) + score = np.maximum(d1, d2) / np.maximum(rms, 1e-4) + idx = np.where(score > k)[0] + if idx.size: + keep = [idx[0]] + for i in idx[1:]: + if i - keep[-1] >= 3: + keep.append(i) + idx = np.array(keep) + return idx.size / (x.size / rate) + + +def noise_floor_db(x): + if x.size == 0: + return float("nan") + q = np.sqrt(np.mean(np.sort(x * x)[: max(1, x.size // 10)])) # quietest 10% RMS + return 20.0 * np.log10(q + 1e-12) + + +# ------------------------------------------------------------------------- main +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--ref", required=True, help="oracle WAV (bsnes)") + ap.add_argument("--test", required=True, help="recomp WAV (audio_wav dump)") + ap.add_argument("--rate", type=int, default=32040, help="common analysis rate") + ap.add_argument("--start-s", type=float, default=0.0) + ap.add_argument("--dur-s", type=float, default=0.0, help="0 = all") + ap.add_argument("--max-lag-s", type=float, default=2.0) + ap.add_argument("--json", default=None) + a = ap.parse_args() + + rate = a.rate + xr, rr = load_wav(a.ref); xt, rt = load_wav(a.test) + xr = resample_linear(xr, rr, rate); xt = resample_linear(xt, rt, rate) + # trim leading silence on each independently, then optional window + r0, r1 = active_region(xr); t0, t1 = active_region(xt) + xr = xr[r0:r1]; xt = xt[t0:t1] + if a.start_s or a.dur_s: + s = int(a.start_s * rate); e = xr.size if a.dur_s == 0 else s + int(a.dur_s * rate) + xr = xr[s:e]; xt = xt[s:min(e, xt.size)] + n = min(xr.size, xt.size) + xr = xr[:n]; xt = xt[:n] + if n < rate: + print("ERROR: <1s of overlapping active audio; check captures.", file=sys.stderr) + sys.exit(2) + + ml = int(a.max_lag_s * rate) + glag, gcorr = best_lag(xr, xt, ml) + # apply global lag so downstream operate aligned + if glag >= 0: + xa, xb = xr[glag:], xt[: n - glag] + else: + xa, xb = xr[: n + glag], xt[-glag:] + m = min(xa.size, xb.size); xa = xa[:m]; xb = xb[:m] + + drift = drift_track(xa, xb, rate) + o_ref = onsets(xa, rate); o_test = onsets(xb, rate) + onset = onset_match(o_ref, o_test, 0.0) + + fr, sr_ = avg_spectrum(xa, rate); _, st_ = avg_spectrum(xb, rate) + timbre = {} + if sr_ is not None and st_ is not None: + lo = (fr >= 50) & (fr <= 16000) + ls = np.log(sr_[lo] + 1e-9); lt = np.log(st_[lo] + 1e-9) + timbre["log_spectral_dist_db"] = float(np.sqrt(np.mean((ls - lt) ** 2)) * (20 / np.log(10))) + cen = lambda s: float((fr[lo] * s[lo]).sum() / (s[lo].sum() + 1e-9)) + timbre["centroid_ref_hz"] = cen(sr_); timbre["centroid_test_hz"] = cen(st_) + timbre["centroid_delta_hz"] = timbre["centroid_test_hz"] - timbre["centroid_ref_hz"] + + pr = dominant_pitch_track(xa, rate); pt = dominant_pitch_track(xb, rate) + pitch = {} + if pr.size and pt.size: + ratio = np.median(pt) / np.median(pr) + pitch = {"median_ref_hz": float(np.median(pr)), "median_test_hz": float(np.median(pt)), + "tuning_error_cents": float(cents(ratio))} + + quality = { + "ref": {"click_per_s": click_rate(xa, rate), "noise_floor_db": noise_floor_db(xa)}, + "test": {"click_per_s": click_rate(xb, rate), "noise_floor_db": noise_floor_db(xb)}, + } + + result = { + "ref": os.path.basename(a.ref), "test": os.path.basename(a.test), + "rate": rate, "overlap_s": round(m / rate, 2), + "global_align": {"lag_ms": glag / rate * 1000.0, "peak_corr": gcorr}, + "drift": drift, "onset": onset, "pitch": pitch, "timbre": timbre, "quality": quality, + } + + # ----- human verdict ----- + print(f"\n=== A/B audio diff: {result['test']} vs REF {result['ref']} ===") + print(f" overlap {result['overlap_s']}s @ {rate} Hz") + ga = result["global_align"] + print(f" global align : lag {ga['lag_ms']:+.1f} ms, peak corr {ga['peak_corr']:.3f} " + f"({'STRONG' if ga['peak_corr']>0.5 else 'WEAK - streams may differ structurally'})") + if drift: + print(f" drift : {drift['drift_ms_per_s']:+.2f} ms/s ({drift['drift_ppm']:+.0f} ppm), " + f"lag std {drift['lag_ms_std']:.1f} ms, local corr {drift['local_corr_median']:.2f}") + if onset: + if onset.get("matched"): + print(f" onsets : {onset['matched']}/{onset['test_onsets']} matched " + f"({onset['match_rate']*100:.0f}%), |err| median {onset['err_ms_abs_median']:.1f} ms, " + f"p90 {onset['err_ms_p90']:.1f} ms") + else: + print(f" onsets : 0 matched (ref {onset['ref_onsets']}, test {onset['test_onsets']})") + if pitch: + print(f" tuning : test {pitch['median_test_hz']:.1f} Hz vs ref " + f"{pitch['median_ref_hz']:.1f} Hz -> {pitch['tuning_error_cents']:+.0f} cents") + if timbre: + print(f" timbre : log-spectral dist {timbre['log_spectral_dist_db']:.1f} dB, " + f"centroid delta {timbre['centroid_delta_hz']:+.0f} Hz") + q = quality + print(f" quality : clicks ref {q['ref']['click_per_s']:.2f}/s test {q['test']['click_per_s']:.2f}/s; " + f"noise floor ref {q['ref']['noise_floor_db']:.0f} dB test {q['test']['noise_floor_db']:.0f} dB") + print() + + if a.json: + with open(a.json, "w") as f: + json.dump(result, f, indent=2) + print(f" wrote {a.json}") + + +if __name__ == "__main__": + main() diff --git a/tools/audio_analyze.py b/tools/audio_analyze.py new file mode 100644 index 00000000..10258158 --- /dev/null +++ b/tools/audio_analyze.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""audio_analyze.py - turn a recomp_audio_debug dump into a 'first dirty tap' verdict. + +Reads a dump directory written by recomp_audio_debug_dump(): + <tap>.wav (one per tap; names like t0_pulse1, t1_emu, t2_bridge_in, + t3_bridge_out, t4_callback) + events.csv (t_ms,type,metadata) + summary.txt + +Runs the detectors from the round-2 plan and applies the cross-tap decision tree: + anomaly in T1 -> synthesis / upstream + T2 but not T1 -> per-system conversion / decimation + T3 but not T2 -> shared bridge / resampler + only T4 -> SDL callback fill / conversion / starvation + +Usage: + python audio_analyze.py <dump_dir> [--block 735] [--click-k 6.0] + +Only dependency is numpy. +""" +import os, sys, wave, struct, glob, argparse +import numpy as np + + +def load_wav(path): + with wave.open(path, "rb") as w: + ch = w.getnchannels() + rate = w.getframerate() + n = w.getnframes() + raw = w.readframes(n) + x = np.frombuffer(raw, dtype="<i2").astype(np.float64) / 32768.0 + if ch > 1: + x = x.reshape(-1, ch) + mono = x.mean(axis=1) + else: + mono = x + return mono, rate, ch + + +def discontinuity(x, rate, k=6.0, rms_ms=10.0): + """Return (click_indices, click_score array). A click = isolated |d1|/|d2| + spike that towers over the local RMS.""" + if x.size < 4: + return np.array([], dtype=int), np.array([]) + d1 = np.abs(np.diff(x, prepend=x[0])) + d2 = np.abs(np.diff(x, n=2, prepend=[x[0], x[0]])) + win = max(8, int(rate * rms_ms / 1000.0)) + # rolling RMS via cumulative sum of squares + cs = np.cumsum(np.concatenate([[0.0], x * x])) + lo = np.maximum(0, np.arange(x.size) - win) + hi = np.minimum(x.size, np.arange(x.size) + win) + local_rms = np.sqrt((cs[hi] - cs[lo]) / np.maximum(1, hi - lo)) + eps = 1e-4 + score = np.maximum(d1, d2) / np.maximum(local_rms, eps) + idx = np.where(score > k)[0] + # keep only isolated spikes (gap >= 3 samples) to avoid counting one event many times + if idx.size: + keep = [idx[0]] + for i in idx[1:]: + if i - keep[-1] >= 3: + keep.append(i) + idx = np.array(keep, dtype=int) + return idx, score + + +def repeat_runs(x): + """Longest run of identical consecutive samples, and count of runs >= 4.""" + if x.size < 2: + return 0, 0 + same = np.diff(x) == 0.0 + longest = cur = 0 + runs4 = 0 + for s in same: + if s: + cur += 1 + longest = max(longest, cur) + else: + if cur + 1 >= 4: + runs4 += 1 + cur = 0 + if cur + 1 >= 4: + runs4 += 1 + return longest + (1 if longest else 0), runs4 + + +def boundary_hist(click_idx, block): + """Fraction of clicks within +/-2 samples of a block boundary.""" + if click_idx.size == 0 or block <= 0: + return 0.0, None + mod = click_idx % block + near = np.sum((mod <= 2) | (mod >= block - 2)) + return near / click_idx.size, mod + + +def fft_signature(x, rate): + """Return dominant peaks and a crude aliasing/broadband read.""" + if x.size < 2048: + return {} + n = 1 << int(np.floor(np.log2(min(x.size, 1 << 18)))) + seg = x[:n] * np.hanning(n) + mag = np.abs(np.fft.rfft(seg)) + freqs = np.fft.rfftfreq(n, 1.0 / rate) + mag_db = 20 * np.log10(np.maximum(mag, 1e-9)) + nyq = rate / 2.0 + # broadband floor = median of upper half; peak = max + upper = mag_db[freqs > nyq * 0.5] + floor_db = float(np.median(upper)) if upper.size else -120.0 + peak_db = float(np.max(mag_db)) + # top 5 peaks + order = np.argsort(mag_db)[::-1] + peaks = [] + seen = [] + for i in order: + f = freqs[i] + if any(abs(f - s) < rate / n * 8 for s in seen): + continue + seen.append(f) + peaks.append((round(float(f), 1), round(float(mag_db[i]) - peak_db, 1))) + if len(peaks) >= 5: + break + near_nyq = float(np.max(mag_db[freqs > nyq * 0.85]) - peak_db) if np.any(freqs > nyq * 0.85) else -120.0 + return {"floor_dbfs_rel": round(floor_db - peak_db, 1), + "near_nyquist_rel": round(near_nyq, 1), + "peaks": peaks} + + +def canonical(tapname): + n = tapname.lower() + for t in ("t0", "t1", "t2", "t3", "t4"): + if n.startswith(t) or ("_" + t) in n: + return t + return None + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("dump_dir") + ap.add_argument("--block", type=int, default=735, + help="synthesis block size for boundary correlation (NES=735)") + ap.add_argument("--click-k", type=float, default=6.0) + args = ap.parse_args() + + wavs = sorted(glob.glob(os.path.join(args.dump_dir, "*.wav"))) + if not wavs: + print("no .wav stems in", args.dump_dir) + return 2 + + print("=" * 78) + print("recomp audio analysis:", args.dump_dir) + print("=" * 78) + + per_tap = {} + for path in wavs: + name = os.path.splitext(os.path.basename(path))[0] + x, rate, ch = load_wav(path) + dur = x.size / rate if rate else 0 + idx, score = discontinuity(x, rate, k=args.click_k) + cps = idx.size / dur if dur else 0.0 + longest, runs4 = repeat_runs(x) + frac_b, _ = boundary_hist(idx, args.block) + sig = fft_signature(x, rate) + per_tap[name] = dict(rate=rate, ch=ch, dur=dur, clicks=idx.size, cps=cps, + longest_repeat=longest, runs4=runs4, frac_boundary=frac_b, + sig=sig, canon=canonical(name)) + print(f"\n[{name}] rate={rate} ch={ch} dur={dur:.2f}s") + print(f" clicks={idx.size} ({cps:.2f}/s) longest_repeat={longest} " + f"repeat_runs>=4: {runs4}") + if idx.size: + print(f" clicks within +/-2 of {args.block}-sample boundary: {frac_b*100:.0f}%") + if sig: + print(f" fft: floor={sig['floor_dbfs_rel']}dB near-Nyquist={sig['near_nyquist_rel']}dB") + print(f" peaks(rel dB): {sig['peaks']}") + + # ---- cross-tap decision tree ---- + print("\n" + "=" * 78) + print("CROSS-TAP VERDICT") + print("=" * 78) + + def dirty(canon): + # a canonical stage is "dirty" if any tap mapped to it has a meaningful click rate + worst = 0.0 + for n, d in per_tap.items(): + if d["canon"] == canon: + worst = max(worst, d["cps"]) + return worst + + THR = 0.5 # clicks/sec considered meaningful + t1, t2, t3, t4 = dirty("t1"), dirty("t2"), dirty("t3"), dirty("t4") + print(f" stage click rates (clicks/s): T1={t1:.2f} T2={t2:.2f} T3={t3:.2f} T4={t4:.2f}") + + verdict = None + if t1 > THR: + verdict = ("T1", "synthesis / upstream — the emulated console/APU/HLE is " + "producing discontinuous PCM. Fix the generator, not the bridge.") + elif t2 > THR: + verdict = ("T2", "per-system conversion / decimation / staging between native " + "synthesis and the bridge input.") + elif t3 > THR: + verdict = ("T3", "shared bridge / resampler (ratio jumps, phase reset, " + "emergency fade, input starvation).") + elif t4 > THR: + verdict = ("T4", "SDL callback path (underfill, memset/zero fill, callback " + "starvation, format mismatch).") + + if verdict: + print(f"\n >>> FIRST DIRTY TAP: {verdict[0]}\n {verdict[1]}") + else: + print("\n >>> No impulse-class clicks above threshold in any canonical tap.") + print(" If the user still hears static, it is NOT impulse crackle —") + print(" check aliasing (near-Nyquist energy), level/clipping, or tuning.") + + # boundary hint + for n, d in per_tap.items(): + if d["clicks"] and d["frac_boundary"] > 0.6: + print(f"\n NOTE: in [{n}], {d['frac_boundary']*100:.0f}% of clicks sit on a " + f"{args.block}-sample boundary -> block-generation discontinuity " + f"(fix continuous phase/time accumulator).") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/audio_archive_ab.py b/tools/audio_archive_ab.py new file mode 100644 index 00000000..9ae027a8 --- /dev/null +++ b/tools/audio_archive_ab.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python3 +"""audio_archive_ab.py - continuously archive both processes' always-on PCM +rings to disk with a screenshot timeline, so identical attract-demo tracks can +be paired offline. Ring holds ~95 s; dumping every ~75 s with absolute frame +indices yields a gapless per-side archive. + +Usage: python audio_archive_ab.py <out_dir> <minutes> +""" +import json +import os +import socket +import struct +import sys +import time + +SIDES = {"runtime": 4520, "beetle": 4382} + +def call(port, p, timeout=30): + s = socket.create_connection(("127.0.0.1", port), timeout=timeout) + try: + s.sendall((json.dumps(p) + "\n").encode()) + buf = b"" + while True: + c = s.recv(1 << 16) + if not c: + break + buf += c + if buf.endswith(b"\n"): + break + return json.loads(buf.decode()) + finally: + s.close() + +def bmp_mean(path): + try: + with open(path, "rb") as f: + d = f.read() + off = struct.unpack("<I", d[10:14])[0] + px = d[off::37] + return sum(px) / max(1, len(px)) + except Exception: + return -1 + +def main(): + out = sys.argv[1] + minutes = float(sys.argv[2]) if len(sys.argv) > 2 else 8.0 + if not (len(out) > 1 and out[1] == ":"): + raise SystemExit("out dir must be a Windows-style path (F:/...) — " + "the game processes fopen() these paths themselves") + os.makedirs(out, exist_ok=True) + log = open(os.path.join(out, "timeline.jsonl"), "w") + next_dump = {k: 0.0 for k in SIDES} + dump_idx = {k: 0 for k in SIDES} + last_total = {k: 0 for k in SIDES} + t_end = time.time() + minutes * 60 + shot_i = 0 + while time.time() < t_end: + now = time.time() + for side, port in SIDES.items(): + try: + st = call(port, {"id": 1, "cmd": "audio_stats"}) + tap0 = st["taps"][0]["frames"] + sp = os.path.join(out, f"{side}_{shot_i:04d}.bmp").replace("\\", "/") + call(port, {"id": 2, "cmd": "screenshot_file", "path": sp}) + rec = {"t": now, "side": side, "shot": shot_i, "tap0": tap0, + "bright": round(bmp_mean(sp), 1)} + # dump the ring every ~75 s, from where the last dump ended + if now >= next_dump[side]: + start = last_total[side] + avail_oldest = max(0, tap0 - 4194304) + if start < avail_oldest: + start = avail_oldest + count = tap0 - start + if count > 0: + wp = os.path.join( + out, f"{side}_arch{dump_idx[side]:02d}_{start}.wav" + ).replace("\\", "/") + r = call(port, {"id": 3, "cmd": "audio_wav", "tap": 0, + "path": wp, "start": str(start), + "count": str(count)}, timeout=60) + rec["dump"] = {"file": wp, "start": start, + "count": count, "ok": r.get("ok")} + last_total[side] = tap0 + dump_idx[side] += 1 + next_dump[side] = now + 75.0 + log.write(json.dumps(rec) + "\n") + log.flush() + except Exception as e: + log.write(json.dumps({"t": now, "side": side, "err": str(e)}) + "\n") + log.flush() + shot_i += 1 + time.sleep(10) + log.close() + print("archive complete") + +if __name__ == "__main__": + main() diff --git a/tools/audio_capture_ab.py b/tools/audio_capture_ab.py new file mode 100644 index 00000000..c5e6f3c2 --- /dev/null +++ b/tools/audio_capture_ab.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +"""audio_capture_ab.py - dump the always-on audio tap rings from psx-runtime +and psx-beetle (identical `audio_*` wire protocol, ports differ) and run the +drift-tolerant A/B comparison. + +Both processes must already be sitting at the scene whose audio you want to +compare (drive them there first via `press` on their debug ports). The rings +are always-on: this tool dumps HISTORY (the last ~95 s of PCM already +captured), it never arms anything. + +Usage: + python tools/audio_capture_ab.py --runtime-port 4520 --beetle-port 4380 \ + --out F:/Projects/psxrecomp/_wt-audio-mmx5/_audio_captures/title \ + [--secs 30] [--analyze] + +Outputs in --out: + runtime_spu_out.wav T1: recomp SPU render output (pre host fade) + runtime_cd_in.wav T2: recomp XA/CD-DA PCM pushed to the SPU CD bus + runtime_host.wav T3: exact bytes handed to SDL_QueueAudio + beetle_spu_out.wav oracle: Beetle's fully-mixed reference SPU output + stats.json audio_stats from both sides +""" +import argparse +import json +import os +import socket +import subprocess +import sys + +def call(port, payload, timeout=30): + s = socket.create_connection(("127.0.0.1", port), timeout=timeout) + try: + s.sendall((json.dumps(payload) + "\n").encode()) + buf = b"" + while True: + c = s.recv(1 << 16) + if not c: + break + buf += c + if buf.endswith(b"\n"): + break + return json.loads(buf.decode()) + finally: + s.close() + +def dump_tap(port, tap, path, secs): + """Dump the trailing `secs` seconds of a tap ring to `path`.""" + stats = call(port, {"id": 1, "cmd": "audio_stats"}) + total = stats["taps"][tap]["frames"] + count = min(total, int(secs * 44100)) if secs > 0 else 0 + start = total - count if count else -1 + r = call(port, {"id": 2, "cmd": "audio_wav", "tap": tap, + "path": path.replace("\\", "/"), + "start": str(start), "count": str(count)}) + return r, stats + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--runtime-port", type=int, default=4520) + ap.add_argument("--beetle-port", type=int, default=4380) + ap.add_argument("--out", required=True) + ap.add_argument("--secs", type=float, default=30.0, + help="trailing window to dump (0 = whole ring)") + ap.add_argument("--analyze", action="store_true", + help="run audio_ab_diff.py runtime_spu_out vs beetle_spu_out") + args = ap.parse_args() + + os.makedirs(args.out, exist_ok=True) + out = lambda name: os.path.abspath(os.path.join(args.out, name)) + results = {"runtime": {}, "beetle": {}} + + for tap, name in ((0, "runtime_spu_out.wav"), + (1, "runtime_cd_in.wav"), + (2, "runtime_host.wav")): + try: + r, st = dump_tap(args.runtime_port, tap, out(name), args.secs) + results["runtime"][name] = r + results["runtime"]["stats"] = st + print(f"[runtime] tap {tap} -> {name}: {r}") + except Exception as e: + print(f"[runtime] tap {tap} FAILED: {e}") + + try: + r, st = dump_tap(args.beetle_port, 0, out("beetle_spu_out.wav"), args.secs) + results["beetle"]["beetle_spu_out.wav"] = r + results["beetle"]["stats"] = st + print(f"[beetle] tap 0 -> beetle_spu_out.wav: {r}") + except Exception as e: + print(f"[beetle] tap 0 FAILED: {e}") + + with open(out("stats.json"), "w") as f: + json.dump(results, f, indent=2) + print(f"stats -> {out('stats.json')}") + + if args.analyze: + here = os.path.dirname(os.path.abspath(__file__)) + cmd = [sys.executable, os.path.join(here, "audio_ab_diff.py"), + "--ref", out("beetle_spu_out.wav"), + "--test", out("runtime_spu_out.wav"), + "--rate", "44100", + "--json", out("ab_diff.json")] + print("running:", " ".join(cmd)) + subprocess.run(cmd, check=False) + +if __name__ == "__main__": + main() diff --git a/tools/audio_pair_segments.py b/tools/audio_pair_segments.py new file mode 100644 index 00000000..5bfcd107 --- /dev/null +++ b/tools/audio_pair_segments.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python3 +"""audio_pair_segments.py - find the SAME musical content in two archived +ring dumps (runtime vs beetle attract cycles are phase-shifted) and emit the +best-matching aligned window pair for A/B analysis. + +Method: concatenate each side's numbered arch WAVs into one continuous +timeline (they were dumped gaplessly with absolute start indices in the +filename). Compute a coarse spectral fingerprint sequence (log-mel-ish band +energies per 0.5 s hop), normalized-cross-correlate runtime-vs-beetle +fingerprints at every offset, take the best offset, and cut the overlapping +region (minus silent edges) from both sides to <out>/pair_runtime.wav and +<out>/pair_beetle.wav. + +Usage: python audio_pair_segments.py <archive_dir> [--min-secs 20] +Requires numpy. +""" +import argparse +import glob +import os +import re +import sys +import wave + +import numpy as np + +HOP_S = 0.5 +BANDS = 24 + +def load_side(archive_dir, side): + files = [] + for p in glob.glob(os.path.join(archive_dir, f"{side}_arch*_*.wav")): + m = re.search(rf"{side}_arch(\d+)_(\d+)\.wav$", p) + if m: + files.append((int(m.group(1)), int(m.group(2)), p)) + files.sort() + chunks = [] + rate = 44100 + for _idx, _start, p in files: + w = wave.open(p) + rate = w.getframerate() + n = w.getnframes() + data = np.frombuffer(w.readframes(n), dtype=np.int16).reshape(-1, 2) + w.close() + chunks.append(data) + if not chunks: + raise SystemExit(f"no {side}_arch*.wav in {archive_dir}") + return np.concatenate(chunks), rate + +def fingerprint(x, rate): + """Band-energy sequence, one row per HOP_S, log-compressed, per-row norm.""" + mono = x.astype(np.float32).mean(axis=1) / 32768.0 + hop = int(rate * HOP_S) + nfft = 4096 + rows = [] + for i in range(0, len(mono) - nfft, hop): + seg = mono[i:i + nfft] * np.hanning(nfft) + mag = np.abs(np.fft.rfft(seg)) + # log-spaced band edges 60 Hz .. 16 kHz + edges = np.geomspace(60, 16000, BANDS + 1) / (rate / 2) * (nfft // 2) + band = np.array([mag[int(a):max(int(a) + 1, int(b))].mean() + for a, b in zip(edges[:-1], edges[1:])]) + rows.append(np.log1p(band * 1000.0)) + fp = np.array(rows) + # per-row zero-mean so correlation keys on spectral SHAPE sequence + fp -= fp.mean(axis=1, keepdims=True) + n = np.linalg.norm(fp, axis=1, keepdims=True) + fp /= np.maximum(n, 1e-9) + return fp + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("archive_dir") + ap.add_argument("--min-secs", type=float, default=20.0) + args = ap.parse_args() + + rt, rate = load_side(args.archive_dir, "runtime") + bt, _ = load_side(args.archive_dir, "beetle") + print(f"runtime: {len(rt)/rate:.0f}s beetle: {len(bt)/rate:.0f}s") + + frt = fingerprint(rt, rate) + fbt = fingerprint(bt, rate) + + # slide beetle over runtime (both directions), score mean row-dot + best = (-1.0, 0) + min_rows = int(args.min_secs / HOP_S) + for off in range(-(len(fbt) - min_rows), len(frt) - min_rows): + a0, b0 = max(0, off), max(0, -off) + n = min(len(frt) - a0, len(fbt) - b0) + if n < min_rows: + continue + score = float(np.einsum("ij,ij->i", frt[a0:a0 + n], fbt[b0:b0 + n]).mean()) + if score > best[0]: + best = (score, off) + score, off = best + a0, b0 = max(0, off), max(0, -off) + n = min(len(frt) - a0, len(fbt) - b0) + print(f"best offset {off * HOP_S:+.1f}s score {score:.3f} overlap {n * HOP_S:.0f}s") + + # cut the overlapping window, trimming rows where either side is silent + energy_rt = np.abs(rt[:, 0].astype(np.int32)) > 256 + energy_bt = np.abs(bt[:, 0].astype(np.int32)) > 256 + s_rt = int(a0 * HOP_S * rate) + s_bt = int(b0 * HOP_S * rate) + length = int(n * HOP_S * rate) + cut_rt = rt[s_rt:s_rt + length] + cut_bt = bt[s_bt:s_bt + length] + + def write(path, data): + w = wave.open(path, "wb") + w.setnchannels(2) + w.setsampwidth(2) + w.setframerate(rate) + w.writeframes(data.astype(np.int16).tobytes()) + w.close() + print("wrote", path, f"{len(data)/rate:.0f}s") + + write(os.path.join(args.archive_dir, "pair_runtime.wav"), cut_rt) + write(os.path.join(args.archive_dir, "pair_beetle.wav"), cut_bt) + # rough level comparison on the matched content + def rms_db(x): + f = x.astype(np.float64) / 32768.0 + act = f[np.abs(f[:, 0]) > 0.001] + if len(act) == 0: + return -120.0 + return 20 * np.log10(np.sqrt((act ** 2).mean())) + print(f"matched-content RMS: runtime {rms_db(cut_rt):.1f} dBFS, " + f"beetle {rms_db(cut_bt):.1f} dBFS") + +if __name__ == "__main__": + main() diff --git a/tools/audio_spectral_ab.py b/tools/audio_spectral_ab.py new file mode 100644 index 00000000..b4b1cf11 --- /dev/null +++ b/tools/audio_spectral_ab.py @@ -0,0 +1,174 @@ +#!/usr/bin/env python3 +"""audio_spectral_ab.py - deep TIMBRE/tone differential of recomp audio vs an +accuracy-grade reference. Where audio_ab_diff.py answers "same notes, same +timing?", this answers "same TONE?" - the spectral envelope / harmonic content +that onset & correlation metrics are blind to. + +It is drift-tolerant by construction: it compares TIME-AVERAGED spectra over a +matched musical passage, so small timing drift between the streams does not +affect the result. That isolates synthesis/timbre error from timing error. + +Metrics: + - third-octave band energy difference (dB), recomp - reference, 25 Hz..16 kHz + - spectral centroid / 85% rolloff / flatness for each stream + - overall log-spectral distance (dB RMS) + - PNG: overlaid average spectra + both spectrograms + +Uses scipy (resample_poly, welch, spectrogram) + matplotlib. Usage: + python audio_spectral_ab.py --ref REF.wav --test TEST.wav + [--rate 32040] [--win-s 6] [--png out.png] [--json out.json] +""" +import sys, os, wave, argparse, json +import numpy as np +from scipy import signal +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt + + +def load_wav(path): + """Robust to unfinalized (force-killed) headers: fall back to raw parse.""" + try: + with wave.open(path, "rb") as w: + ch, rate, n = w.getnchannels(), w.getframerate(), w.getnframes() + raw = w.readframes(n) + if n > 0 and len(raw) >= n * ch * 2: + x = np.frombuffer(raw, "<i2").astype(np.float64) / 32768.0 + return (x.reshape(-1, ch).mean(1) if ch > 1 else x), rate + except Exception: + pass + b = open(path, "rb").read() + assert b[:4] == b"RIFF" and b[8:12] == b"WAVE", "not a WAV" + ch, rate, bits, off, doff, dlen = 2, 32040, 16, 12, None, 0 + while off + 8 <= len(b): + cid = b[off:off + 4]; sz = int.from_bytes(b[off + 4:off + 8], "little"); body = off + 8 + if cid == b"fmt ": + ch = int.from_bytes(b[body + 2:body + 4], "little") + rate = int.from_bytes(b[body + 4:body + 8], "little") + bits = int.from_bytes(b[body + 14:body + 16], "little") + elif cid == b"data": + doff = body; dlen = sz if 0 < sz <= len(b) - body else len(b) - body; break + off = body + sz + (sz & 1) + raw = b[doff:doff + dlen - dlen % (ch * bits // 8)] + x = np.frombuffer(raw, "<i2").astype(np.float64) / 32768.0 + return (x.reshape(-1, ch).mean(1) if ch > 1 else x), rate + + +def to_rate(x, src, dst): + if src == dst: + return x + from math import gcd + g = gcd(int(src), int(dst)) + return signal.resample_poly(x, dst // g, src // g) + + +def active(x, thr=1e-4): + nz = np.where(np.abs(x) > thr)[0] + return (0, x.size) if nz.size == 0 else (int(nz[0]), int(nz[-1]) + 1) + + +def coarse_lag(a, b, rate, max_s=8): + ml = int(max_s * rate) + n = 1 << int(np.ceil(np.log2(a.size + b.size))) + cc = np.fft.irfft(np.fft.rfft(a - a.mean(), n) * np.conj(np.fft.rfft(b - b.mean(), n)), n) + cc = np.concatenate([cc[-ml:], cc[:ml + 1]]) + return int(np.arange(-ml, ml + 1)[int(np.argmax(cc))]) + + +def welch_psd(x, rate): + f, p = signal.welch(x, rate, nperseg=8192, noverlap=4096, scaling="spectrum") + return f, p + + +def third_octave(f, p, lo=25.0, hi=16000.0): + bands, centers = [], [] + fc = lo + while fc <= hi: + f1 = fc / (2 ** (1 / 6)); f2 = fc * (2 ** (1 / 6)) + m = (f >= f1) & (f < f2) + bands.append(p[m].sum() if m.any() else 0.0); centers.append(fc) + fc *= 2 ** (1 / 3) + return np.array(centers), np.array(bands) + + +def descriptors(f, p): + p = np.maximum(p, 1e-20); tot = p.sum() + centroid = float((f * p).sum() / tot) + cs = np.cumsum(p); rolloff = float(f[np.searchsorted(cs, 0.85 * tot)]) + gm = np.exp(np.mean(np.log(p))); flatness = float(gm / (p.mean())) + return centroid, rolloff, flatness + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--ref", required=True); ap.add_argument("--test", required=True) + ap.add_argument("--rate", type=int, default=32040) + ap.add_argument("--win-s", type=float, default=6.0) + ap.add_argument("--png", default=None); ap.add_argument("--json", default=None) + a = ap.parse_args() + rate = a.rate + + xr, rr = load_wav(a.ref); xt, rt = load_wav(a.test) + xr = to_rate(xr, rr, rate); xt = to_rate(xt, rt, rate) + r0, r1 = active(xr); t0, t1 = active(xt); xr = xr[r0:r1]; xt = xt[t0:t1] + lag = coarse_lag(xr, xt, rate) + if lag >= 0: + xr = xr[lag:] + else: + xt = xt[-lag:] + n = min(xr.size, xt.size); xr = xr[:n]; xt = xt[:n] + # pick the loudest matched win-s window (most musical content) for the average + w = int(a.win_s * rate) + if n > w: + env = np.convolve(np.abs(xr) + np.abs(xt), np.ones(rate) / rate, "same") + c = int(np.argmax(env)); s = max(0, min(c - w // 2, n - w)); xr = xr[s:s + w]; xt = xt[s:s + w] + + fr, pr = welch_psd(xr, rate); ft, pt = welch_psd(xt, rate) + cen_r, ce = third_octave(fr, pr); _, te = third_octave(ft, pt) + # normalize each to equal total energy in the analysis band (compare SHAPE, + # not absolute level - level differences are a separate, trivial fix) + ce_n = ce / (ce.sum() or 1.0); te_n = te / (te.sum() or 1.0) + db = 10 * np.log10(np.maximum(te_n, 1e-12) / np.maximum(ce_n, 1e-12)) + + dr = descriptors(fr, pr); dt = descriptors(ft, pt) + # overall log-spectral distance on normalized PSD shape + lo = (fr >= 50) & (fr <= 16000) + prn = pr / pr[lo].sum(); ptn = pt / pt[lo].sum() + lsd = float(np.sqrt(np.mean((10 * np.log10(ptn[lo] + 1e-12) - 10 * np.log10(prn[lo] + 1e-12)) ** 2))) + + print(f"\n=== SPECTRAL / TONE diff: {os.path.basename(a.test)} vs REF {os.path.basename(a.ref)} ===") + print(f" window {a.win_s:.0f}s @ {rate} Hz (shape-normalized; level ignored)") + print(f" log-spectral distance : {lsd:.1f} dB RMS ({'CLOSE' if lsd<3 else 'AUDIBLE timbre diff' if lsd<6 else 'LARGE timbre diff'})") + print(f" centroid ref {dr[0]:6.0f} Hz test {dt[0]:6.0f} Hz ({dt[0]-dr[0]:+.0f})") + print(f" rolloff85 ref {dr[1]:6.0f} Hz test {dt[1]:6.0f} Hz ({dt[1]-dr[1]:+.0f})") + print(f" flatness ref {dr[2]:.4f} test {dt[2]:.4f}") + print(f" --- third-octave band energy (test - ref, dB; |delta|>3 dB flagged) ---") + for c, d in zip(cen_r, db): + flag = " <<<" if abs(d) > 3 else "" + bar = ("+" if d >= 0 else "-") * min(20, int(abs(d))) + print(f" {c:7.0f} Hz : {d:+5.1f} dB {bar}{flag}") + + result = {"ref": os.path.basename(a.ref), "test": os.path.basename(a.test), + "lsd_db": lsd, + "centroid": {"ref": dr[0], "test": dt[0]}, + "rolloff85": {"ref": dr[1], "test": dt[1]}, + "flatness": {"ref": dr[2], "test": dt[2]}, + "third_octave_db": {float(c): float(d) for c, d in zip(cen_r, db)}} + if a.json: + json.dump(result, open(a.json, "w"), indent=2); print(f"\n wrote {a.json}") + + png = a.png or (os.path.splitext(a.test)[0] + "_spectral.png") + fig, ax = plt.subplots(3, 1, figsize=(10, 11)) + ax[0].semilogx(fr, 10 * np.log10(prn + 1e-12), label="ref (bsnes)", lw=1) + ax[0].semilogx(ft, 10 * np.log10(ptn + 1e-12), label="test (recomp)", lw=1, alpha=0.8) + ax[0].set_xlim(40, 16000); ax[0].set_xlabel("Hz"); ax[0].set_ylabel("dB (norm)") + ax[0].set_title(f"avg spectrum (LSD {lsd:.1f} dB)"); ax[0].legend(); ax[0].grid(True, which="both", alpha=0.3) + for k, (x, t) in enumerate([(xr, "ref (bsnes)"), (xt, "test (recomp)")]): + f2, tt, Sxx = signal.spectrogram(x, rate, nperseg=2048, noverlap=1024) + ax[k + 1].pcolormesh(tt, f2, 10 * np.log10(Sxx + 1e-12), shading="gouraud", cmap="magma") + ax[k + 1].set_ylim(0, 16000); ax[k + 1].set_ylabel("Hz"); ax[k + 1].set_title(t) + fig.tight_layout(); fig.savefig(png, dpi=90); print(f" wrote {png}") + + +if __name__ == "__main__": + main() diff --git a/tools/compile_overlays.py b/tools/compile_overlays.py index 7c18c9b3..9a8ae159 100644 --- a/tools/compile_overlays.py +++ b/tools/compile_overlays.py @@ -997,6 +997,15 @@ int psx_ws_cull_sltiu(uint32_t sx, uint32_t imm) { return ((uint32_t)((int32_t)(int16_t)(uint16_t)sx + m) < (uint32_t)((int32_t)imm + 2 * m)) ? 1 : 0; } +/* Signed-funnel variants (min/max + center±halfwidth idioms). Same contract: + * self-contained, identity at margin 0, MUST stay byte-for-byte identical to + * the gpu.c implementations. */ +int psx_ws_cull_slti(uint32_t sx, uint32_t imm) { + return ((int32_t)sx < (int32_t)imm + psx_ws_x_margin()) ? 1 : 0; +} +int psx_ws_cull_bltz(uint32_t v) { + return ((int32_t)v < -psx_ws_x_margin()) ? 1 : 0; +} void psx_ws_sprite_tag(CPUState *cpu) { if (g_cbs.ws_sprite_tag) g_cbs.ws_sprite_tag(cpu); } @@ -1289,12 +1298,16 @@ def _addr_in_func_ids(addr: int, func_ids: list) -> bool: return False -def load_region_covered_ranges(cache_dir: str, phys_addr: int) -> list: - """List of (lo_phys, length) code ranges already provided by ALL built DLLs - (region + fragment) for this region_start, from their .ranges manifests. Used - to decide whether an executed orphan interior is already covered (so we don't - mint a redundant island fragment for it).""" - out = [] +def load_region_entry_set(cache_dir: str, phys_addr: int) -> set: + """Set of phys-normalized F-line ENTRY addresses provided by ALL built DLLs + (region + fragment) for this region_start. This — not range coverage — is + the dispatchability test: native code is enterable ONLY at F entries, so a + dispatch-proven PC inside a compiled range but absent from every manifest's + F set still runs its whole chain on the interpreter (the 0x80106D7C class, + 2026-07-06: 80% of Tomba2 attract interp residue was two range-covered, + entry-less interior PCs that the range-based orphan test refused to + fragment, while the region compile skipped as 'already covered').""" + out = set() prefix = f'{phys_addr:08X}_' try: names = os.listdir(cache_dir) @@ -1307,9 +1320,9 @@ def load_region_covered_ranges(cache_dir: str, phys_addr: int) -> list: with open(os.path.join(cache_dir, name)) as f: for ln in f: p = ln.split() - if len(p) >= 3 and p[0] == 'R': + if len(p) >= 2 and p[0] == 'F': try: - out.append((int(p[1], 16) & 0x1FFFFFFF, int(p[2], 16))) + out.add(int(p[1], 16) & 0x1FFFFFFF) except ValueError: pass except OSError: @@ -1696,15 +1709,23 @@ def main(): crc32, toml) this_ids = None # region func-ids once recompiled (None if skipped early) - # Record this region's executed interiors for the decoupled fragment pass - # (runs after the loop, regardless of this region's compile outcome). + # Record this region's executed dispatch-proven PCs for the decoupled + # fragment pass (runs after the loop, regardless of this region's + # compile outcome). Interiors AND callable dispatch roots both go in: + # a callable root can be starved by the DLL-already-exists skip when + # an OLDER capture of the same image bytes built this region's DLL + # before the PC became dispatch-proven (same filename, stale entry + # set — the 0x80024548 class), and the fragment pass is the demand- + # driven recovery path for exactly that. if not args.static: _interiors = {a for a, r in seed_audit['included_reasons'].items() if r == 'DISPATCH_INTERIOR'} + _disp_roots = {a for a, r in seed_audit['included_reasons'].items() + if r in ('DISPATCH_ENTRY', 'DISPATCH_ROOT')} _executed = seed_audit.get('executed_pcs', set()) - if _interiors and _executed: + if (_interiors or _disp_roots) and _executed: interior_frag_jobs.append((phys_addr, load_addr, size, data, - _interiors, _executed)) + _interiors | _disp_roots, _executed)) if not args.static: dll_path = os.path.join(cache_dir, f'{phys_addr:08X}_{crc32:08X}.dll') @@ -1885,12 +1906,15 @@ def main(): frag_env['PSX_CPS'] = '1' for job in interior_frag_jobs: phys_addr, load_addr, size, data, interior_pcs, executed = job - covered = load_region_covered_ranges(cache_dir, phys_addr) - def _is_covered(a): - a &= 0x1FFFFFFF - return any(lo <= a < lo + length for lo, length in covered) + # ENTRY-based orphan test, not range-based: native code is + # enterable only at manifest F entries, so "inside a compiled + # range" does NOT make a dispatch target servable — a range- + # covered PC with no F entry anywhere still interps its whole + # chain on every dispatch. Demand an entry at exactly this PC. + covered_entries = load_region_entry_set(cache_dir, phys_addr) orphans = sorted(a for a in interior_pcs - if a in executed and not _is_covered(a)) + if a in executed + and (a & 0x1FFFFFFF) not in covered_entries) if not orphans: continue built = 0 @@ -1900,8 +1924,8 @@ def main(): frag_env) if frag_ids: built += 1 - for ev, _cc, ranges in frag_ids: - covered.extend((lo & 0x1FFFFFFF, L) for lo, L in ranges) + for ev, _cc, _ranges in frag_ids: + covered_entries.add(ev & 0x1FFFFFFF) print(f' interior fragments @0x{phys_addr:08X}: {built}/{len(orphans)} ' f'executed orphan interior(s) -> isolated island shards') diff --git a/tools/cosim.py b/tools/cosim.py index e742188b..882e92e5 100644 --- a/tools/cosim.py +++ b/tools/cosim.py @@ -25,9 +25,9 @@ import socket, subprocess, os, sys, time, argparse # build (e.g. a Tomba regression) without editing this file. Defaults = MMX6. # COSIM_EXE = path to that game's build-cosim/psx-cosim.exe # COSIM_GAME = path to that game's game.toml -EXE = os.environ.get("COSIM_EXE", r"F:\Projects\psxrecomp\MegaManX6Recomp\build-cosim\psx-cosim.exe") +EXE = os.environ.get("COSIM_EXE", r"../MegaManX6Recomp\build-cosim\psx-cosim.exe") CWD = os.environ.get("COSIM_CWD", os.path.dirname(EXE)) -GAME = os.environ.get("COSIM_GAME", r"F:\Projects\psxrecomp\MegaManX6Recomp\game.toml") +GAME = os.environ.get("COSIM_GAME", r"../MegaManX6Recomp\game.toml") LOGDIR = os.path.join(CWD, "cosim-logs") def tail_file(path, max_bytes=8192): diff --git a/tools/crop_launcher_assets.ps1 b/tools/crop_launcher_assets.ps1 index 9f146b06..5ed50c6c 100644 --- a/tools/crop_launcher_assets.ps1 +++ b/tools/crop_launcher_assets.ps1 @@ -6,8 +6,8 @@ # interior dark details (disc PlayStation logo, controller buttons, the centre # hole) are preserved. Logo: simple global luminance key (thin bright glyphs). Add-Type -AssemblyName System.Drawing -$src = "C:\Users\Matthew\Desktop\ef772e04-a7db-4ecd-98bb-eb75a01de0a6.png" -$out = "F:\Projects\psxrecomp\psxrecomp\runtime\launcher\assets\img" +$src = "<desktop>/ef772e04-a7db-4ecd-98bb-eb75a01de0a6.png" +$out = "runtime\launcher\assets\img" $mock = [System.Drawing.Bitmap]::FromFile($src) function Get-Crop($x,$y,$w,$h) { diff --git a/tools/devtrace_diff.py b/tools/devtrace_diff.py index 124e296a..c1908e3c 100644 --- a/tools/devtrace_diff.py +++ b/tools/devtrace_diff.py @@ -24,6 +24,19 @@ import argparse, json, socket, sys SRC = {0:"vblank",1:"gpu",2:"cdrom",3:"dma",4:"timer0",5:"timer1",6:"timer2", 7:"sio0",8:"sio1",9:"spu",10:"pio"} +# KNOWN INTENTIONAL divergences from the Beetle oracle — where OUR runtime is +# MORE faithful to real hardware than Beetle, so a native/beetle delta on this +# source is EXPECTED, not a bug. Labeled (not red-flagged) unless --strict. +# See ACCURACY_BURNDOWN.md (Axis 5 — CDROM). Add entries here (never silence a +# real divergence): source-name -> one-line reason. +KNOWN_DIVERGENCES = { + "cdrom": "SCPH-1001 CD controller version 94/09/19/C0 (psx-spx) vs Beetle's " + "hardcoded 97/01/10/C2 — version <0x95 keeps the shell CD-init flag " + "[0xA000DFFC] clear, so Beetle issues a spurious boot ReadTOC we do " + "not: expect a CDROM event count/timing delta around CD-init. " + "Also GetID region (SCEA/E/I) + status bit2 ADPBUSY idle=0.", +} + def query(port, cmd, **kw): obj = {"cmd": cmd}; obj.update(kw) s = socket.create_connection(("127.0.0.1", port), timeout=15) @@ -61,6 +74,8 @@ def main(): ap.add_argument("--cyc-hi", type=int, default=None) ap.add_argument("--drift", type=int, default=2_000_000) ap.add_argument("--show", type=int, default=0) + ap.add_argument("--strict", action="store_true", + help="do NOT suppress KNOWN_DIVERGENCES (show them as raw drift)") args = ap.parse_args() nat = pull(args.native, args.cyc_lo, args.cyc_hi) @@ -73,6 +88,7 @@ def main(): sources = sorted(set(ns) | set(bs)) print(f"\n{'source':<8} {'native#':>8} {'beetle#':>8} {'first divergent ordinal (cycle delta)':<40}") print("-" * 78) + suppressed = [] for s in sources: n, b = ns.get(s, []), bs.get(s, []) name = SRC.get(s, f"src{s}") @@ -86,10 +102,21 @@ def main(): break if not flag and len(n) != len(b): flag = f"count differs ({len(n)} vs {len(b)}) — no big per-ordinal drift in first {m}" + diverged = bool(flag) if not flag: flag = "tracks (no drift > threshold)" + known = KNOWN_DIVERGENCES.get(name) + if diverged and known and not args.strict: + flag = f"KNOWN✓ intentional (more faithful) — {flag}" + suppressed.append(name) print(f"{name:<8} {len(n):>8} {len(b):>8} {flag}") + if suppressed and not args.strict: + print("\nKNOWN intentional divergences suppressed (re-run with --strict to" + " see raw): " + ", ".join(suppressed)) + for nm in suppressed: + print(f" {nm}: {KNOWN_DIVERGENCES[nm]}") + if args.show: for s in sources: name = SRC.get(s, f"src{s}") diff --git a/tools/duckstation/README.md b/tools/duckstation/README.md index 23f28abb..697dd041 100644 --- a/tools/duckstation/README.md +++ b/tools/duckstation/README.md @@ -13,7 +13,7 @@ PSXRecomp v4 uses a patched build of [stenzek/duckstation](https://github.com/st ## First-time setup ```bash -cd /f/Projects/psxrecomp-v4 +cd <psxrecomp-v4> bash tools/duckstation/setup.sh # ~5 min: clone submodule + fetch/extract deps + apply patch bash tools/duckstation/build.sh # ~15-30 min: compile duckstation-qt Release x64 ``` @@ -41,7 +41,7 @@ The pinned base SHA lives in one place only: `UPSTREAM_BASE` at the top of `tool - Keeps the upstream source untracked in v4's git history — no 2.1GB of upstream code in our blame. - Keeps *our* 1200-line patch reviewable as a single text diff in `tools/duckstation/` — in-tree, versioned, diffable across sessions. -- Matches the nestopia setup in sibling project `F:\Projects\nesrecomp\runner\nestopia_cmake.cmake` + `runner/nestopia_oracle.patch`. Same mental model: submodule upstream, patch on top, auto-apply at setup time. +- Matches the nestopia setup in sibling project `<nesrecomp>/runner\nestopia_cmake.cmake` + `runner/nestopia_oracle.patch`. Same mental model: submodule upstream, patch on top, auto-apply at setup time. - No private/public GitHub fork to maintain or keep in sync. ## Protocol parity with native runtime diff --git a/tools/gen_launcher_assets.ps1 b/tools/gen_launcher_assets.ps1 index 24030c3b..e6b9d83c 100644 --- a/tools/gen_launcher_assets.ps1 +++ b/tools/gen_launcher_assets.ps1 @@ -3,7 +3,7 @@ # logo, verification icons). Swap with real renders later — the <img>/decorator # pipeline (stb_image PNG loader) is the same either way. Add-Type -AssemblyName System.Drawing -$out = "F:\Projects\psxrecomp\psxrecomp\runtime\launcher\assets\img" +$out = "runtime\launcher\assets\img" if (-not (Test-Path $out)) { New-Item -ItemType Directory -Force $out | Out-Null } function New-Canvas($w, $h) { diff --git a/tools/scan_f2_sites.py b/tools/scan_f2_sites.py index 103a07ea..397aba22 100644 --- a/tools/scan_f2_sites.py +++ b/tools/scan_f2_sites.py @@ -1,6 +1,6 @@ #!/usr/bin/env python3 import sys -rom = open('F:/Projects/psxrecomp/bios/SCPH1001.BIN','rb').read() +rom = open('../bios/SCPH1001.BIN','rb').read() matches = [] for rt in range(32): @@ -46,7 +46,7 @@ for addr, rt, op, rs, rt_out, imm, nw, offset in matches: # Read dispatch.c: print("\n--- Functions containing 0xF2000003 sites ---") import re -dispatch = open('F:/Projects/psxrecomp/generated/SCPH1001_dispatch.c').read() +dispatch = open('../generated/SCPH1001_dispatch.c').read() entries = re.findall(r'\{\s*0x([0-9A-F]+)u,\s*func_([0-9A-F]+)\s*\}', dispatch) # Only keep those in code range (phys 0x1FC00000+) fn_list = sorted([(int(a,16), name) for a,name in entries]) diff --git a/tools/shot_launcher.ps1 b/tools/shot_launcher.ps1 index 3622a8f4..2834f9bd 100644 --- a/tools/shot_launcher.ps1 +++ b/tools/shot_launcher.ps1 @@ -1,4 +1,4 @@ -param([string]$Out = "F:\Projects\psxrecomp\TombaRecomp\build-clean\launcher_shot.png") +param([string]$Out = "../TombaRecomp\build-clean\launcher_shot.png") $p = Get-Process psx-runtime -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowHandle -ne 0 } | Select-Object -First 1 if (-not $p) { Write-Output "no window"; exit 1 } Write-Output ("hwnd=" + $p.MainWindowHandle + " title=" + $p.MainWindowTitle) diff --git a/tools/tomba_loadtest.py b/tools/tomba_loadtest.py index cd7755ad..2010d41f 100644 --- a/tools/tomba_loadtest.py +++ b/tools/tomba_loadtest.py @@ -12,11 +12,11 @@ Usage: python tomba_loadtest.py <exe> <game.toml> [port] """ import sys, time, subprocess, socket, json -EXE = sys.argv[1] if len(sys.argv)>1 else r"F:/Projects/psxrecomp/TombaRecomp/build-codex-ape-fw/psx-runtime.exe" +EXE = sys.argv[1] if len(sys.argv)>1 else r"../TombaRecomp/build-codex-ape-fw/psx-runtime.exe" TOML = sys.argv[2] if len(sys.argv)>2 else "game-loadtest.toml" PORT = int(sys.argv[3]) if len(sys.argv)>3 else 4470 -BIOS = r"F:/Projects/psxrecomp/psxrecomp/bios/SCPH1001.BIN" -CWD = r"F:/Projects/psxrecomp/TombaRecomp" +BIOS = r"bios/SCPH1001.BIN" +CWD = r"../TombaRecomp" NONE,RIGHT,CROSS,START = 0xFFFF,0xFFDF,0xBFFF,0xFFF7 def q(cmd, **kw): @@ -70,7 +70,7 @@ def nonblack(): return nz/tot if tot else 0 def shot(name): - b=f"F:/Projects/psxrecomp/_scratch/{name}.bmp"; q("screenshot_file",path=b) + b=f"../_scratch/{name}.bmp"; q("screenshot_file",path=b) try: from PIL import Image; Image.open(b).save(b.replace(".bmp",".png")) except Exception: pass diff --git a/tools/verify_audit_schema.py b/tools/verify_audit_schema.py index b64aa18f..1cf8739e 100644 --- a/tools/verify_audit_schema.py +++ b/tools/verify_audit_schema.py @@ -64,5 +64,5 @@ def validate(path, expect_remaps): return cfg bios = validate(ROOT / "bios" / "SCPH1001.toml", expect_remaps=2) -tomba = validate(Path("F:/Projects/TombaRecomp/game.toml"), expect_remaps=0) +tomba = validate(Path("../TombaRecomp/game.toml"), expect_remaps=0) print("\nOK: both configs parse and validate.")