GetID's last four response bytes were hardcoded 'SCEI' (NTSC-J). Real
hardware reports the region of the INSERTED disc. Kula World uses the
BIOS CD API, and the kernel CD driver revalidates the disc (ReadTOC +
GetID, polled with irq_enable=0x18) when it reinitializes after the
game's first file load. Seeing a 'Japanese' disc, it fell into the
endless GetStat/Init retry loop: CD delivery froze at sector 307 (LBA
318, just before the first level directory) and the game waited forever
on a black screen.
Diagnosed from the always-on cdrom trace ring (grabbed via gdb from the
live process): 307 sectors stream cleanly, Pause completes, then the
BIOS's own ROM CD firmware (0xBFC0Dxxx) masks INTC delivery, issues
ReadTOC + GetID, reads the response... and rejects it. Beetle PSX (the
oracle; boots the same dump + BIOS to demo gameplay) derives the SCEx
string from the disc's SYSTEM.CNF boot serial - so we now do the same:
cdrom_set_disc_scex() is fed at launch from the existing disc_identity
module (PAL->SCEE, NTSC-J->SCEI, NTSC-U->SCEA; default SCEA matches the
one supported BIOS). Verified so far: launch log reports 'disc region
PAL (serial SCES-01000)'; the sector-counter wall-break run is in
flight. Framework-wide note: Tomba/MMX6 (NTSC-U, own CD drivers) never
hit this path but now get 'SCEA' - matching real hardware - instead of
'SCEI'; regression-test before any master merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdjxoRB1f9VQsNzZayCgGS
Brings the fail-closed native entry guard from wt/tomba2 (b5a6940 / bd60381) to
master, where it had never been merged — which is why Tomba (and any master-built
title) still wedged. This is the long-standing "blue screen" / shared-RAM-
divergence-between-interp-and-native class.
Root cause: an overlay function dispatched at a FOREIGN INTERIOR PC (a range-
ownership mismatch routed a PC the function does not own into it) hit the CPS
entry-switch `default: break`, which fell through and ran the function FROM ITS
TOP — re-running its prologue and corrupting shared CPU/RAM state, so the guest's
main loop then waits forever on a condition that never gets set (the 0xCF0 SIO
spin / blue screen). Reproduced on Tomba's Village of All Beginnings: a native
0x000E7000-region shard function entered at a foreign PC wedged; the interpreter
ran the same bytes correctly.
Fix: the generated CPS entry-switch for OVERLAY functions (config_.overlay_mode)
now FAILS CLOSED on an unknown non-zero continuation PC — it restores the PC and
calls psx_native_bad_entry instead of running from the top. overlay_loader_dispatch
consumes g_native_bad_entry and routes the PC to the sanctioned dirty-RAM
interpreter (the live bytes run correctly — NOT a stub/HLE). True-prologue entry
still runs from the top. Static (BIOS/boot-EXE) functions keep the legacy fall-
through. ABI bumped 5->6 so stale ABI-5 caches are rejected and rebuilt with the
guard.
Minimal port: kept only the fail-closed mechanism + its ABI plumbing. The Tomba2
host_aliases interior-routing and the native-block / CPS-probe diagnostics (which
depend on the overlay-cache-v2 multi-range Candidate fields not on master) were
intentionally omitted — the fail-closed default alone is the safety net.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two framework bugs that made overlay variants run interpreted forever (the
"agonizing lag" in overlay-heavy areas like Tomba's dwarf village):
1. Cache read/write divergence. The loader reads <exe>/cache and the runtime
writes captures to <exe>/overlay_captures.json, but the gcc autocompile path
used the game.toml command verbatim, whose hard-coded --out-dir/--captures
pointed elsewhere (build-stable/build-modern). So compiled shards landed where
the loader never looked and captures were read stale -> the variant never went
native. Fix: the runtime now injects PSX_OVERLAY_CACHE_DIR / PSX_OVERLAY_CAPTURES
(the canonical loader paths) into every autocompile spawn; compile_overlays.py
honors them over any CLI flag (--captures now optional). One source of truth for
the cache location, uniform across all games, dev or prod. gcc/tcc stay separated
by the existing <compiler> namespace.
2. Overlay DLL preamble missing psx_ws_cull_sltiu. auto_screen_x emits this helper
at every width-cull site into EVERY build's C (identity at 4:3), and the main EXE
links it from gpu.c, but the overlay DLL preamble never defined it -> any overlay
variant with cull sites failed to link -> no shard -> interpreter. Fix: provide
the self-contained helper (depends only on the already-forwarded psx_ws_x_margin)
in the DLL preamble.
Proven on Tomba dwarf village: dispatch_native 29/s -> 11454/s (was ~97% interp,
now ~97% native).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compiler-neutral tier-selection policy extracted from the mis-named
overlay_sljit.{c,h} into NEW overlay_backend.{c,h} (OverlayBackend enum +
resolve/name/active). overlay_sljit.{c,h} is now just the deprecated emitter.
New tcc tier. gcc and tcc run the IDENTICAL pipeline (recompiler -> C ->
compiler -> DLL -> loader); only the compiler binary differs, so both map to the
autocompile-spawn provider (code_provider.c) and the COMMAND is chosen by the
resolved backend (main.cpp). compile_overlays.py gains --compiler gcc|tcc: tcc
writes a separate tcc/ cache namespace, BOM-strips the C + a copy of the include
headers (tcc 0.9.27 can't skip a UTF-8 BOM), and uses tcc's built-in linker.
Backends (overlay_backend): auto (gcc if a gcc TOOLCHAIN is present, else tcc --
was sljit), gcc, tcc, auto-no-gcc (dev/test: force tcc even with gcc present, to
exercise the toolchain-less user path), sljit (DEPRECATED, gated off, lowest).
Priority: static > gcc shard > tcc shard > sljit(off) > interp. The loader scans
gcc/ then tcc/ and dedups by region+crc (filename) so a gcc shard always wins a
tie.
Self-contained tcc: with no explicit overlay_autocompile_cmd_tcc, the runtime
CONSTRUCTS the tcc command from a toolchain bundled beside the exe
(<exe>/overlay_toolchain/: embedded python + tcc + recompiler + compile_overlays.py
+ runtime headers) -- no system python or gcc needed. config_loader gains
overlay_autocompile_cmd_tcc (a dev override). autocompile cmd buffers enlarged for
the constructed path-heavy command.
Compilation stays OFF the dispatch/render thread (off-process autocompile
subprocess, same model as gcc); the main thread only spawns it and dlopens the
finished DLL.
Validated: MMX6 with overlay_backend=auto-no-gcc fills the tcc/ namespace live
from the bundled toolchain and plays through save-load + gameplay -- the exact
path the sljit tier softlocked on. The gcc path is untouched.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Off-thread sljit compile worker (overlay_compile_worker.c/.h): on a dispatch
gap-fill miss the dispatch thread snapshots the region bytes and ENQUEUES; a
background worker JITs from the snapshot (never live RAM) and publishes the shard
to the on-disk cache via an atomic temp+rename; dispatch picks it up on a later
miss via an idempotent rescan, so the candidate table stays single-threaded. This
removes the synchronous JIT frame hitch. Validated on Tomba + MMX6: compiles
off-thread (one MMX6 compile was 26ms, paid entirely off the dispatch thread with
no frame-pacing spike), shards persist + reload, queue never backs up.
Observability (ring-buffer doctrine): always-on recent-compile ring + per-compile
timing in the worker, the resolved on-disk cache dir, and a tier-enabled flag, all
surfaced via a new sljit_async TCP debug command (documented in TCP_COMMANDS.md).
DISABLE the sljit tier by default (g_sljit_tier_enabled = 0): the sljit EMITTER
mistranslates some MMX6 overlay instruction(s) -- sljit-only save-load softlocks/
loopbacks while PURE INTERP works (discriminator), and it reproduces with the
worker OFF in the synchronous path (PSX_SLJIT_SYNC=1), so it is the emitter, not
the async worker. Until the emitter bug is found + fixed: no shard READ
(scan_sljit_cache_dir skipped), no GENERATE (worker not started + sljit_live
forced 0 + dispatch gap-fill gated), every overlay miss falls to the interpreter.
gcc (DLL cache + autocompile) and overlay capture are untouched, so priority is
gcc > interp and dispatch misses are still recorded for future gcc coverage. gcc
shards are bundled in release zips, so shipped builds are unaffected.
PSX_SLJIT_ENABLE=1 re-enables the tier for emitter-fix testing.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the old incomplete RAM-only fast_boot snapshot with a COMPLETE
hardware save-state captured at the BIOS->game handoff. First launch (and the
first launch after any update) boots the real BIOS normally and captures the
full machine state at the game's PS-EXE entry_pc; every subsequent same-build
launch restores it and presents the game's first frame instantly.
This is not HLE: it persists the real hardware state a real BIOS run produced.
Format (boot_state.h v2): a sectioned file with an integrity key
(bios_checksum + entry_pc + PSX_OVERLAY_CODEGEN_HASH + ABI tag + codegen ver).
Any mismatch -> reject -> normal boot + recapture, so a user/app update can
never silently load a stale snapshot. Missing/short/unknown sections also reject
(no partial restore).
Complete capture (one section per subsystem), the fix for the old snapshot's
real bug (it left GPU/SPU/CD/DMA/SIO at reset while CPU/RAM assumed configured):
- CPU regs, 2 MB RAM, 1 KB scratchpad, IRQ (i_stat/i_mask), root timers, guest
cycle counter, dirty-RAM page bitmap (guest-written code pages).
- GPU: display/draw-area/offset/mask/texpage + GPUSTAT bits + GP0 command
machine + VRAM-transfer state; VRAM (1 MB) via the renderer-agnostic
gr_vram_transfer_out/in hooks (so it is correct on the GPU-authoritative GL
backend, where the FBO -- not gpu.c vram[] -- is truth, and the FBO re-upload
on restore happens for free).
- SPU: register file + 24 voice decode/ADSR state + latches + 512 KB SPU RAM.
- CD-ROM: full controller FSM (regs/IRQ-latch/FIFOs/seek/read/pending/queued +
XA decoder history + disc-speed model).
- DMA: channels[7] + dpcr/dicr + async-transfer state.
- SIO: registers + pad-config FSM + memcard FSM (incl. per-slot mc_slots[]) +
cycle-paced shifter/ACK scheduler.
Host-side/recompiler-derived state (overlay/sljit tables, code-provider) is NOT
serialized -- it re-initializes fresh on the normal startup path.
Per-subsystem accessors use an X-macro field list so bytes/write/read can never
drift. Runtime-only; no recompiler/codegen change, no regen. Reuses the
[runtime] fast_boot key (semantics upgraded; type unchanged).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comprehensive 3-way merge of the dev/ape-escape branch into master for
the isolated wt/ape-escape worktree. Brought in every complementary ape
change automatically (dma.c, memory.c interrupt byte/half lanes,
function_analysis discovery, main_bios vector seeds, iso_reader
iso_sector_count, config_loader, crash-trace fields, headers). Resolved
the 13 conflicts by rule:
- CPS dispatch / recompiler (code_generator, full_function_emitter,
dirty_ram_interp, traps, main_psx) -> KEEP MASTER (evolved system).
- CD model (cdrom.c) -> take ape (the model that booted Ape).
- interrupts.c/.h -> reverted to MASTER (ape's interrupt-delivery is
dispatch-coupled; the merged Frankenstein stormed).
- sio/debug_server/freeze_heartbeat/main.cpp -> keep master (evolved).
- crash_trace.c -> master (ape freeze-dump fns omitted to avoid the
debug-ring cascade).
STATUS: does NOT boot yet. ape-cdrom + master-interrupts derails to
PC=0 at frame ~2092 (insta-close); ape-cdrom + ape-interrupts storms
(297M exception re-entries) at frame 2077. Root finding: ape's CD
anti-storm is coupled to ape's interrupt-delivery + call-depth + EPC
model, which master replaced with CPS continuation-passing, so the CD
subsystem does not forward-port as pure hardware simulation. Preserved
as the reference state for a design consult on bridging the two models.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The launcher Hybrid controller mode (built first, for Tomba, to mimic Tomba:
Special Edition) flips the emulated pad's reported type 0x41<->0x73 on
d-pad<->stick. After the real DualShock config state machine was added for MMX6
(98aa688), that flip regressed every game's libpad re-detect into a 1-frame
"pad unplugged": Tomba unpauses its menu / drops input, MMX6 phantom-dashes.
The modern config SM is correct and required by MMX6, so instead of weakening it
globally, restore the pre-98aa688 "config commands always answer 0xF3" behaviour
as a per-game opt-in: [controller] legacy_pad_config (default OFF). Only Tomba
sets it; with the flag off the modern pad path is byte-for-byte unchanged, so no
other title is affected. A/B verified live: legacy = clean Tomba flip / MMX6
input; modern = Tomba disconnect + MMX6 dash.
Explicitly LEGACY: a correct findpad/re-detect refactor would make the flip
benign for every game with no per-game branch (real hardware tolerates unlimited
analog-button presses); when that lands this whole feature set is to be removed.
Full rationale in sio.c (g_pad_legacy_cfg).
Also includes the deferred coherent pad-type apply (pad_type_req, applied only at
an idle/non-config bus boundary so a flip can't split a poll/handshake) and a
`pad_cfg` debug command + tools/padcfg.py + tools/sio_flip_trace.py used to A/B
the config SM live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 BG widen draws LEFT extra tile columns each side, but the engine's
streamer (FUN_800273e4 -> FUN_800274a0) only refreshes one column per side per
frame at the leading edge. On scene entry / fast scroll the inner reveal columns
held stale tiles from a prior scrollY -> real-but-misplaced tiles spilling
up-and-left (the "staircase"). Census confirmed this is NOT the BG packet-budget
overflow (674-816 prims/frame, well under the 1000-tile cap).
Fix (render window == refresh window): clone FUN_800274a0's inner metatile->ring
fill into mmx6_fill_column() and re-stream every column of the widened window for
all 3 layers each frame (psx_ws_mmx6_bg_refill_all), triggered once/frame from the
existing count_site hook (mmx6_bg_refill_tick, frame-gated). Uses each layer's own
scroll, omits the engine's scrollX wrap-bookkeeping writes (those drive the engine's
own incremental streamer), skips map-loop seams; off-map columns fill tile 0 (the
renderer skips them). Validated byte-exact against the engine's own streaming
(0 mismatches / 1113 cells x 3 frames; debug cmd mmx6_freshfix {validate:1}). Gated
native-wide <=384w so 4:3 and the 512 hi-res title stay byte-identical.
Also removes the 2-col reveal cap (WS_FULL2D_MAX_REVEAL): it never addressed the
staircase and cost screen real-estate; the full 53px/4-col reveal is restored.
Parked, gated-off experiments ride along (inert, default off): host-side reveal
(Attempt A: gpu_ws_mmx6_emit_reveal + sw_wide_emit_tile) and BG packet-buffer
relocation (Attempt B: [widescreen.bg2d] bufbase/cap recompiler hooks). These target
the separate dense-stage budget-overflow concern, not this fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The BG renderer widen (a8f37c1) draws extra tile columns on both sides, but the
engine's 64-column tile ring is only refreshed at its leading edge: the streamer
(FUN_800273e4) re-streams ONE column per side per frame at world-X scrollX-16
(left) and scrollX+16+width (right). So the columns the widened renderer reaches
were never populated — they showed empty (black margin) or stale (old tiles as
the camera scrolled), the "stale margins" the user saw in motion.
Add two more [widescreen.bg2d] sites (stream_left_site / stream_right_site) that
push the streamer's left/right leading-edge world-X out by LEFT*16 via gpu.c
psx_ws_mmx6_bg_stream_{left,right}, so the incremental streamer keeps the ring
valid across the widened window as the camera scrolls. Same number of streams
(the single leading-edge target just moves out LEFT columns) — no extra cost.
Identity at 4:3 / 512 hi-res.
Verified: the previously-black left reveal margin now fills with coherent
adjacent stage; blackL/blackR = 0 across scenes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure-2D sprite games (Mega Man X6) render a true 4:3 field of view with no
overscan to reveal, so 16:9 requires making the engine draw more. Two parts:
- [widescreen] full_2d opts a pure-2D game into the native-wide present path
(it never emits the 3D sprite-tag hook the gameplay detector keys on, so
every in-game frame was misclassified as a 4:3 menu and pillarboxed).
- [widescreen.bg2d] is a gen-time hook on the per-layer background tile
renderer (MMX6 FUN_800270d0) at its column-count / start-tile-column /
start-screen-x instructions, routed through gpu.c psx_ws_mmx6_bg_{cols,
startcol,startx}. It draws LEFT=ceil(nw_offset/16) extra 16px columns on
BOTH sides so the loop fills the 16:9 reveal margins with real adjacent
stage, existing columns staying pixel-aligned with objects. Identity at
4:3 and in the engine's own 512 hi-res mode (so the title is untouched).
Tooling: wide_full debug command + sw_wide_dump_full dump the entire
native-wide compositor surface (both vertical-double-buffer y-bands and both
reveal margins) for diagnosis.
Known follow-ups: object screen-X cull widen (actors in the margins);
top/bottom flicker + stale-margin refresh on the vertical double-buffer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual PSX_OVERLAY_CODEGEN_VER must be hand-bumped on every emitter change;
when it isn't (e.g. between CPS emitter iterations), a stale-but-cgN DLL is reused
-> the v0.3.0 black screen. Now hash_codegen.cmake hashes the recompiler codegen
sources into a (gitignored) overlay_codegen_hash.h at build time; overlay_api.h
exposes PSX_OVERLAY_CODEGEN_HASH and the cache path becomes gcc/<arch-abi>/cg<N>_<hash>/.
The loader and compile_overlays.py read the same value, so any emitter change
auto-invalidates the cache (old dirs coexist, no stale load). Falls back to hash 0
when the header is absent (fresh checkout) — identical to today's single-cgN path.
Verified: build emits hash 0x347751c1; autocompile + loader both use cg4_347751c1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A persisted option value outside its declared [min,max] is now REJECTED at load
(the game's own default is used instead), so a corrupt/stale options file can no
longer inject an out-of-range value into a config global the game may use as a
jump-table index (wild jump = softlock). Ranges declared per option in
game_options.toml (min/max; absent => no validation, e.g. signed screen offsets).
Adds a format_version header (newer-than-known files are rejected wholesale).
Restore stays name-keyed (offset-drift-safe). Runtime-only, no regen.
Validated: corrupt values rejected (defaults applied); valid file restores intact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vulkan backend alongside software + OpenGL: geometry/textured-prim
pipelines, GPU-authoritative VRAM image, deferred-submit + drain-at-present
perf model, GPU-direct present (vk_renderer_present_vram), and a
full-VRAM-pack readback path (ensure_cpu rect_adds the whole VRAM rect
before pack_flush so screenshots/guest VRAM reads reflect live content,
not just the incremental pack-dirty rect — readback-only, no per-frame cost).
Runtime plumbing: --debug-port / --renderer / --window-title CLI overrides
(multi-instance fleet), latency_ring, SPIR-V shaders + embed_spirv.py,
config_loader renderer selection, debug_server screenshot wiring.
All on feat/vulkan-renderer; snapshot in backup/vulkan-wip-20260617.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Oracle disambiguation (RECURSION_BUG.md §23–§24). Traced one func_8001A954
invocation on real HW (psx-beetle) vs the recomp in the IDENTICAL Tomba idle
state (New Game village dialogue; object 0x801FD950, state/scene/jumptable=0).
VERDICT: real HW's per-frame chain is BALANCED — func_8001A954 is entered once
per frame from main_loop (JAL 0x8001A630, sp=0x801FE3E0) and RETURNS once per
frame (jr ra @0x8001A9E8 -> 0x8001A638, SP fully restored); no re-entry, the
overlay 0x80063864 never calls back (refutes §21/§22's "call-cycle"). The recomp
runs the same guest flow and the guest SP unwinds identically, but the HOST stack
leaks +1.17 KB/frame (ce_profile, live-reconfirmed: 20099->22215 KB / 1807 frames)
because the per-frame compiled chain (func_8001A954->AC00->B2B4->B5A8->func_80046264,
emitted as DIRECT func_X(cpu) host calls) re-enters itself via func_80046264's
CD-DMA-dirty hop and never unwinds.
native_stack early-trip (PSX_STACK_GUARD_KB=6144, frame 7592, 6 MB) pins the
leaking host cycle. Cross-checked: xprobe mixed_depth bounded at 6, dirty-insn/
dirty-block show NO literal func_8001A954 transfer, and a954=1/frame is a SURFACED
guest tail-transfer (j/jr) to func_8001A954 that the oracle proves real HW never
makes — i.e. there is both (A) a correctness divergence (spurious per-frame
re-entry) and (B) the architectural host-mirroring limitation that amplifies it.
Fix scope CORRECTED from §23's "quick contained runtime patch": the UNIVERSAL fix
is §11 — route in-range guest calls through the depth-managed psx_dispatch_impl
trampoline instead of mirroring them as direct host calls, so the host stack stays
bounded regardless of guest re-entrancy (real or fabricated). §12 (CD-DMA dirty
precision) and fixing the specific divergence are surgical mitigations, not
universal. Performance loss accepted for now (static recomp: right before fast).
Instruments added (runtime-only, no regen):
- SP capture in the fntrace control-flow ring on BOTH backends (GPR[29] read live):
psx-beetle (beetle_libretro.cpp + beetle_debug_server.c) and the recomp
(fntrace.h/.c + debug_server.c) — enables the func_8001A954 SP/RA-lifecycle oracle.
- psx-beetle default debug port 4380 -> 4382 (+ --port N override): 4380 collided
with psxref and cdirecomp/CdiRuntime.exe; two LISTENers on one port silently
misroute connections (the phantom "oracle keeps dying" early in the session).
Oracle tooling: the control-flow oracle is psx-beetle (Beetle interprets -> records
every J/JAL/JR/JALR), NOT psxref (no PC trace; the assumed emu_step/emu_trace_addr
never existed). Loads Tomba via: psx-beetle.exe --disc tomba.cue <bios> --port 4382.
Next: implement the §11 universal fix (codegen + full regen).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Live-tunable selectable stretch gate to identify which drawn prims form the
native-wide 2D-backdrop void: ws_dbg_stretch modes = OT-range / clut / tpage /
tagged / untagged-tex / all-tex, with per-frame matched + matched-tagged counts
(psx_ws_dbg_gate_frame_snapshot). Routes through the existing GL centre-stretch.
Confirmed live: the void layer is the early-drawn, SPRITE-TAGGED 0x65 background
grid (6 cols x 7, step-64, cluts unique to the backdrop); stretching it about
screen-centre fills the void with no inter-tile gaps (GL scales width too).
Tagged fills it; untagged (3D) does not. The prior content-match plan is
DISPROVEN (template transformed beyond recognition before draw) -- corrected
model recorded in BACKDROP_PRELOAD.md. Fix gate chosen: native-wide + tagged +
background-phase (draw-time), since backdrop X is written by a shared sprite
helper with no clean per-emit source signal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 dwarf/flower-field void is native-wide 2D-backdrop POSITIONING, not a
column-loading problem. The earlier auto_backdrop whole-row preload is proven
IRRELEVANT (ws_backdrop_ring: whole 41-col row loads, void persists; margin off
vs whole-row pixel-identical). The flower-field is a fixed-screen 2D tile field
(static 11-bit vertices, ~4:3 width) that bypasses the GTE, so native-wide (which
widens 3D via the GTE) never widens it. Fix = geometric X-stretch about
screen-centre of the 2D backdrop in the GL wide-mirror — USER-CONFIRMED it fills
the void (with the gate disabled). Full record: BACKDROP_PRELOAD.md (2026-06-16).
Remaining: gate the stretch to ONLY the backdrop (currently also stretches Tomba/
HUD/3D). Path forward = draw-time gp0_cmd_source_addr / sprite-tag classification.
Runtime WIP checkpoint (all behind native-wide + runtime toggles, 4:3 untouched):
- gpu_gl_renderer.c: u_xscale/u_xcenter geometric stretch in the geo+tex
wide-mirror passes; per-frame backdrop-phase reset moved to gl_perf_present_enter
(both present paths); textured-only phase mode; per-prim trace + dbg counters.
- gpu.c: ws_backdrop_ring (always-on rewrite ring); ws_backdrop_margin live
column-widen (proven irrelevant); psx_ws_backdrop_value whole-row path.
- dirty_ram_interp.c: ring capture (extent/camx/count/a1/dl) at the interp hook.
- debug_server.c: ws_backdrop_ring / ws_backdrop_margin / ws_backdrop_stretch.
- main.cpp: keyboard ALWAYS drives player 1 (OR'd with any controller).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist validated sljit shards across launches (SLJIT_PERSIST_CACHE.md). On JIT,
serialize the (position-independent, Stage 1) LIR and write it to
<cache>/<game_id>/sljit/<arch-abi>/cg<N>/<entry8>_<crc8>.sljit with a small header
(magic, format/helper-order versions, entry/lo/len/crc/size). At init, scan that
dir, deserialize + sljit_generate_code for THIS process, and register each as a
dll<0 candidate carrying the blob's crc; dispatch re-hashes live RAM against it so
a reloaded shard re-earns trust exactly like a fresh JIT. Priority gcc > sljit >
interp is unchanged (a gcc DLL still supersedes a reloaded shard via the obsolete
path). Persist is write-on-generate (measured negligible; crash-safe).
Threads the serialized blob through OverlaySljitResult -> CompiledFragment;
register_sljit_candidate gains a crc_override (reloaded shards use the header crc
since the overlay RAM may not be loaded at init). sljit_status exposes
reloaded / persist / reload-diagnostic counters.
Two gotchas found + fixed in testing (vs the original design notes):
- serialize MUST precede generate_code (generate sets compiler->error =
SLJIT_ERR_COMPILED, after which serialize returns NULL);
- serialize with options=0, not SLJIT_SERIALIZE_IGNORE_DEBUG, so an
argument-checks build can deserialize its own blobs.
Validated: two-launch test reloaded 12 shards (deserfail=0), ran them at a fresh
process base with no crash, gameplay renders correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prerequisite for persisting sljit shards to disk (SLJIT_PERSIST_CACHE.md). The
JIT baked absolute host-helper pointers as SLJIT_IMM operands at three sites
(emit_helper2: memx/cop2; emit_ws_cull; PEND_CALL: call), so a shard serialized
in one process would call stale addresses when reloaded at a different ASLR base.
Route all four helpers (SLJIT_HLP_MEMX/COP2/WS_CULL/CALL) through a cpu-relative
table CPUState.sljit_helpers[], loaded via MEM1(R_CPU)[offset] instead of a baked
immediate, so serialized LIR holds zero absolute host addresses. The table is
appended at the END of CPUState (existing gcc overlay-DLL field offsets stay
put) and populated once at startup (overlay_sljit_init_helpers). In-session JIT
is unchanged behaviourally (one extra indirection per helper call).
Smoke-tested: forced sljit with the gcc cache moved aside, 12 shards JIT'd and
ran, title renders, no crash. Broader playtest folds into the Stage 2 reload test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller (item 3): present an analog (DualShock) pad by default for
analog-capable games so the left stick gives variable run speed; fold the
physical D-pad / keyboard arrows onto the left analog axes at full deflection so
they still move the character (avoids the Special Edition full-throttle-D-pad
bug). Add a launcher-editable analog-stick deadzone option (settings.toml +
input.ini + game.toml [controller] default), modeled on snesrecomp.
Settings persistence (item 6): deterministic restore-at-init + save-on-exit for a
game's native OPTION-screen settings, declared per game in a dedicated
game_options.toml. The recompiler rewrites each setting's boot-init store (sb/sh)
to route its value through psx_game_option_store(addr,val), applying a value
persisted from a prior session over the game default (identity when none, so a
fresh install is byte-identical). The runtime saves the live config globals to
<memcard_dir>/<game_id>.options on exit (atexit). Adds a game_options debug
command. Requires a regen for the hook to emit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measured Tomba village = 93% textured prims, so draw batching must target the
textured path (per-prim uv-limits/texpage/clut/depth move to vertex attributes
so consecutive textured prims coalesce into one draw). Flat batching alone
would touch only 7% of prims.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds prims/frame and per_prim_us to the frame_perf aggregate. Proved Tomba's
village is draw-call bound: ~1000 prims/frame at ~14us of GPU time PER
glDrawArrays (a 1-triangle draw should be <1us) — pipeline-flush overhead from
re-binding state between every primitive. 16:9 is 2x slower purely because the
native-wide double-draw issues 2x the draw calls. Points squarely at draw
batching (group consecutive same-state prims) as the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Always-on per-frame ring in the GL renderer: two GL_TIME_ELAPSED queries
bracket each frame's scene draws vs the present clear+blit (true GPU time,
independent of CPU/GPU overlap), plus SDL perf-counter CPU wall time
(present-to-present total, present call). Exposed via the debug server
"frame_perf" command, aggregated over the recent ring and split by 4:3 vs
native-wide so a 16:9-vs-4:3 A/B reads off one query.
Built to answer where a frame goes without guessing. First use proved
Tomba's 16:9 village lag is the native-wide DOUBLE-DRAW (every primitive
re-issued into the wide FBO), NOT supersampling (SS2==SS1) and NOT the
compositor (present_gpu ~1ms): scene_gpu 16:9=28ms vs 4:3=17ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GL native-wide present (Option B) did glFinish + glReadPixels of the whole
wide FBO to the CPU and re-uploaded it EVERY frame — a hard GPU pipeline stall +
round-trip that made OpenGL 16:9 run far slower than the software renderer (whose
wide path is pure CPU, no GPU sync). Confirmed by isolation: GL 4:3 smooth, GL
16:9 slow, independent of SS/AA.
Add gl_renderer_present_wide_fbo: blit the displayed buffer's wide FBO straight to
the window with glBlitFramebuffer (GPU-side, V-flipped, into the letterbox),
exactly like the canonical gl_renderer_present_vram — no readback, no glFinish,
no re-upload. main.cpp routes wide GL frames here (falling back to the CPU readout
only if the wide surface doesn't exist yet). USER-VALIDATED smooth. The CPU
readback path (glb_render_wide_display) stays as the fallback + the wide_shot
probe.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documentation only. Prior sessions kept re-deriving state from breadcrumbs that
were no longer true:
- overlay_sljit.c / code_provider.{h,c}: corrected "declines every fragment" and
"compile_fragment not yet called from dispatch" — the MIPS->sljit emitter is
fully implemented and wired (then validated via the same-state diff).
- ISSUES.md: add Issue #7 (sljit live execution was unvalidated; the pure-live
save-load wedge), root cause and fix recorded.
(SLJIT.md — the §11 forward plan and architecture clarification — lives at the
workspace root F:/Projects/psxrecomp/SLJIT.md, outside this repo, updated
separately.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause class: a toolchain-less machine never self-improved via sljit on the
normal play path. Two blocks: (1) `auto` resolved to gcc on EVERY machine because
the discriminator was autocompile_configured() — "is a command STRING set" — and
the shipped game.toml always carries overlay_autocompile_cmd; (2) live sljit mode
was reachable only via a hand-set env var (dead public setter, no policy).
Fix:
- autocompile.c: autocompile_toolchain_available() probes PATH for a real C
compiler (gcc/cc/clang). main.cpp passes
`autocompile_configured() && autocompile_toolchain_available()` to
code_provider_init, so `auto` picks gcc only when one is actually reachable,
else sljit.
- overlay_loader.c: overlay_loader_apply_live_policy() (called from main.cpp
AFTER backend resolution) defaults live ON when the resolved backend is sljit
— so a toolchain-less box runs VALIDATED shards live (self-improves on the
normal path); PSX_OVERLAY_SLJIT_LIVE still overrides. The env-only block in
overlay_loader_init is removed (the policy fn subsumes it).
Runtime-only (no regen). Refs SLJIT.md §11 Phase C.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render the genuine 16:9 FOV at correct proportions instead of squash-then-stretch,
using a separate per-framebuffer wide surface so canonical PSX VRAM stays faithful
(4:3 / non-opted games byte-identical; the wide canvas only exists when opted in).
- Wide-aspect mode descriptor (off / squash / native-wide) in gpu.c; opt-in via
[video] aspect_ratio. GTE fed the 4:3 ratio (identity, no squash) in native-wide;
canonical draw_offset/draw_area left faithful (no VRAM widening, no wrap).
- Separate wide present-surfaces (gpu_sw_renderer.c) keyed by back-buffer base:
framebuffer prims mirrored at local x = vram_x - base_x + OFFSET; margins cleared
via the frame-clear mirror; present reads the displayed buffer's surface 1:1.
Eliminates the A/B bleed + flicker (and the segfault) of the shared-canvas attempt.
gr_* vtable extended; GL backend reports unsupported -> canonical present until
the GL port (task 8).
- psx_ws_x_margin native-wide-aware (reuses the existing [widescreen.cull] emit).
CONFIRMED a no-op for Tomba's edge objects: they spawn via a non-FUN_80022E44
path (the foreground classifier is already 16:9-safe). See handoff.
- Draw-census extended with per-prim screen-X extent + back-buffer base for the
native-wide void diagnosis; ws_nw TCP toggle; gpu_state ws.mode/nw_extra.
- Controller: "auto"/"gamepad"/"controller" device = first available SDL pad.
WIP checkpoint. SW compositor works: flicker fixed, faithful, terrain platform
fills the wide frame. REMAINING: discrete foreground OBJECTS (rock pillar, tree
branch, bushes) + the parallax backdrop are still culled at the 4:3 edge via a
separate generation/spawn path (wtrace the OT display-list writers next); GL
compositor port. Detail in NATIVE_WIDE_PLAN.md + the session handoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One batch widening to cover most remaining overlay instruction classes;
validated end-to-end via the live differential rather than per-op.
- jr rX (computed / jump tables): CTRL_TAILJUMP — capture the target before
the delay slot, then cpu->pc = target and return; the dispatch loop
re-dispatches (matches the interpreter's jr). jr $ra stays the clean-
return terminator. (A terminator-less tail-dispatch fragment still
declines — safe.)
- GTE/COP2: MFC2/CFC2/MTC2/CTC2/cmd (op 0x12), LWC2 (0x32), SWC2 (0x3A)
via psx_sljit_cop2(), a runtime helper mirroring the interpreter exactly.
- Unaligned mem: LWL/LWR/SWL/SWR via psx_sljit_memx(), likewise.
- BLTZAL/BGEZAL: link $ra = pc+8 before the delay slot, then branch.
Helper-emitted classes get parity by construction (the helper IS the
interpreter's code). Harness 66/66 (regression); runtime builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The big coverage unlock — the emitter now accepts functions that call
others, not just leaves.
- New runtime helper psx_sljit_call(cpu,target,return_pc,check) in
overlay_loader.c, mirroring the dirty-RAM interpreter's call path
EXACTLY (psx_dispatch_game_compiled / overlay_loader_call_native /
psx_dispatch_call + g_psx_call_bail + psx_call_contract). Returns 1 if
the shard must return immediately (transfer/bail unwind), 0 to continue.
- Emitter: jal/jalr -> CTRL_CALL. Link reg + (jalr) dynamic target are
written BEFORE the delay slot; the helper is called AFTER it via S2
(helper address) with the result driving a conditional return. jal uses
an absolute target, jalr a dynamic one captured in S1.
- KSEG integration fix: the loader now passes the VIRTUAL entry to the
emitter (was passing phys with image_base_vram=0), so return_pc and
jal/J targets carry the KSEG bits the guest uses. The standalone harness
passed a virtual entry, which is why it stayed green; the live
differential caught the phys/virtual mismatch (saved $ra off by the
0x80000000 KSEG bit).
Verified: sljit_emit_test.c extended (jal, jalr, transfer/bail) -- 66/66
pass. Full psx-runtime build clean. Live differential re-validation of
the KSEG fix is the next step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MIPS->sljit emitter, paralleling dirty_ram_interp.c's exec_one but
emitting host code. First slice = single-block LEAF functions ending in
`jr $ra`; everything else aborts the whole fragment to the interpreter
(precision over recall).
- overlay_sljit.c: memory-backed GPRs, 32-bit ops (R3000A wraparound),
loads/stores via icall through the cpu read/write callbacks (same
MMIO/page-watch/dirty-tracking path as the interpreter). Opcodes:
SLL/SRL/SRA/SLLV/SRLV/SRAV, ADD(U)/SUB(U), AND/OR/XOR/NOR, SLT(U),
ADDI(U)/SLTI(U)/ANDI/ORI/XORI/LUI, LB/LH/LW/LBU/LHU/SB/SH/SW, SYNC.
- compile_fragment now returns CompiledFragment{fn,code_lo,code_len};
overlay_loader registers an sljit candidate (crc hashed from the live
bytes it JIT'd from, same live-byte validation as gcc candidates).
- Step 5 wiring is VALIDATION-GATED: JIT-on-miss fires only when
backend==sljit AND the same-state differential is armed (never in a
shadow run). Normal play stays byte-identical; live sljit execution
stays off until the differential gate passes.
- Debug: sljit_try <addr> probe; sljit_status reports shards_registered.
Verified: runtime/tests/sljit_emit_test.c (links overlay_sljit.c +
sljitLir.c only) JITs hand-built leaf functions and checks output —
26/26 pass incl. gpr0 invariant, sign/zero-ext, signed/unsigned compare,
and the decline cases. Full psx-runtime build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce one backend-agnostic code-production seam the overlay
capture/dispatch spine talks to, and make the validated gcc spawn->DLL
path its first implementation. Pure refactor; gcc stays the default and
its behavior is byte-identical.
- New code_provider.h/.c: a CodeProvider struct exposing BOTH production
lifecycles so the interface isn't gcc-shaped -- batch/async
(request/busy/poll_main, the gcc model) and sync-on-miss
(compile_fragment, the sljit model). compile_fragment is defined and
implemented per-provider but not yet called from dispatch (step 5).
gcc provider = direct pass-throughs to autocompile_* (compile_fragment
NULL: a compiler spawn is too slow for the dispatch path); sljit
provider wraps overlay_sljit_* (still declines every fragment).
- Wire up overlay_backend_resolve, which was previously dead code:
code_provider_init() resolves the backend at overlay-cache init and
caches the active provider; code_provider_active() is the accessor.
- Add [runtime] overlay_backend config (auto|gcc|sljit; empty=auto) so
resolution has a config source alongside PSX_OVERLAY_BACKEND.
- Route the 3 direct autocompile_* call sites through the active
provider: overlay_capture.c autocapture tick, main.cpp per-frame poll
+ the init call, debug_server.c autocompile_run (now reports backend).
- runtime.cmake += code_provider.c.
Builds + links clean (TombaRecomp/build-dev, MinGW gcc 15.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add sljit as a self-contained, toolchain-less codegen backend for the
overlay self-improvement loop. gcc stays the default (dev: optimized +
inspectable); sljit is the fallback / production producer. Interpreter
remains the correctness floor.
This commit lands the backend-agnostic foundation only; the MIPS->sljit
emitter is not built yet, so every fragment safely declines to the
interpreter (no change to the validated gcc path).
- Vendor sljit (BSD-2-Clause) at lib/sljit/, wired into runtime.cmake
(compile sljitLir.c + include dir; sljit auto-detects the host arch).
- overlay_sljit.{h,c}: backend-selection policy (PSX_OVERLAY_BACKEND=
auto|gcc|sljit), availability, a real codegen smoke test, and
try_compile() honoring the precision-over-recall bail-to-interp contract.
- sljit_status debug-server command (Rule-3 clean: status via getter).
- THIRD_PARTY_ATTRIBUTION.md: sljit entry.
Verified: TombaRecomp/build-dev (MinGW gcc 15.2) compiles + links
psx-runtime.exe; standalone smoke harness JITs f(a)=a+1234 and runs it
correctly (26 bytes x86-64). Design + locked tier decisions in SLJIT.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The far backdrop (ocean/cloud/distant mountain) is GTE-projected 3D drawn
by the main-EXE driver FUN_8004db3c (sole caller of the GTE emitter
FUN_80027600). The global widescreen GTE X-squash compresses it toward
centre, so its geometry falls short of the revealed 16:9 edges -> blue void
that pops as the camera moves.
Fix: bracket the backdrop driver with gte_ws_set_suppress(1)/(0) so its
draws skip the X-squash and fill the stretched frame (skybox treatment),
while the near 3D world keeps the wider FOV.
FUN_8004db3c draws a MIX (far backdrop + nearer props/scenery), so the
suppress is DEPTH-GATED in gte_rtps_internal: only vertices with projected
SZ >= s_ws_far_threshold (default 900, tunable live via the ws_far_threshold
TCP cmd) un-squash; nearer props stay squashed and aligned. Validated:
threshold 900 fills the void, keeps the foreground door + background
scenery (path/tree) aligned. Residual: the mountain grass-top straddles the
depth band (per-vertex limitation) -> flickers when the camera dips; this
is the checkpoint before exploring native-wide rendering to remove the
squash (and this whole artifact class) entirely.
Mechanism:
- recompiler: [widescreen.backdrop] unsquash_funcs emits
gte_ws_set_suppress(1) at entry + (0) before each jr-$ra return
(code_generator + config_loader + main_psx). Main-EXE -> regen-class.
- runtime: gte.cpp depth-gated squash + s_ws_far_threshold; cpu_state.h
decl; debug_server.c ws_aspect (live squash on/off) + ws_far_threshold
(set split + read SZ stats) diagnostics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Menu fix (user-validated): per-prim center-squash mangled composite 2D
UI (dialog boxes built from tiled cap/middle/cap SPRTs no longer
aligned). Root cause was decoupling — content was squashed while the
frame presented 4:3. Now squash applies IFF the frame is stretched:
gpu_ws_present_native_43() is the single predicate (FMV video OR a
full-2D screen = no character billboards this frame), shared by the
GTE squash, the GPU per-prim squash, and the present 4:3 pin. Full-2D
menus get zero squash + 4:3 pillarbox = pixel-native. Removed the old
menu-2D squash paths (lines/flat-rects/mono-sprites/polylines).
Cull-margin widening (recompiler feature): the game's per-object draw
classifier tests (objX-camX+BIAS) <u RANGE sized for the 4:3 screen;
the GTE squash shows ~33% more world so the margin collapses and
objects pop near the wide edges. New [widescreen.cull] config emits
those immediates with a runtime term psx_ws_x_margin() (bias +margin,
range +2*margin, plus a1+=margin for param-margin variants) — 0 at
4:3/boot/menu/FMV so the original cull is byte-identical, ~53px when
stretching 16:9. Site instruction types are verified at emit (fatal on
mismatch). Requires regen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- FMV: 24-bit frames AND streamed 15-bit MDEC video (Tomba plays its
movies 15-bit; depth24 alone missed them — user-reported) present
pinned to native 4:3. Detector = colour MDEC decode within the last
30 vblanks (mdec_recently_active); texture-decompression luma
decodes do not stamp. Both GL present paths take force_4_3; SW path
centers a 4:3 dst rect. User-verified on the intro movies.
- Menu garble: full-2D screens (no tagged prims in flight for 2
frames) now squash lines, polylines, flat rects and mono sprites
too — dialog borders are drawn with these and stuck out past the
squashed SPRT boxes. Full-width TILEs (fades) stay exempt; in-game
non-textured prims stay untouched (may be world-anchored).
- HUD positioning: in-game untagged SPRTs pivot by thirds — outer-
third elements anchor to their screen edge (16:9 corner position,
native proportions), middle third to centre. Full-2D screens keep
the single centre pivot so composites stay coherent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User-validated on Tomba: world, characters and HUD all present at
native proportions on the 16:9 stretch.
Mechanism (all parameterized via game.toml [widescreen], inert by
default and at 4:3):
- recompiler: [widescreen] sprite_tag_funcs emits a
psx_ws_sprite_tag(cpu) callback at the entry of each listed
function (the game engine's shared per-prim helper). The runtime
records prim pointer ($a0) -> GTE-projected anchor (read from
sprite_anchor_addr scratchpad) in a frame-stamped hash.
- gpu.c: at GP0 execution, prims whose guest address (from the DMA
linked-list walk's source tracking) is tagged get every X
re-squashed around their own anchor — undoing the present stretch
for CPU-computed pixel offsets the GTE squash never saw. Covers
POLY_FT4 billboards and SPRT parts; SPRT texels keep full coverage
via gr_draw_textured_rect_scaled.
- hud_sprt_squash: untagged textured rects (SPRT family — pure
screen-space 2D: HUD, menus) center-squash around the display
centre. Untextured TILEs (fades) untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the DuckStation-style widescreen hack as a config option,
implemented in the runtime (no regen needed):
- gte.cpp: gte_set_display_aspect(num, den). RTPS/RTPT scale the
IR1*h/sz screen-X term by (4*den)/(3*num) around the game's own
OFX projection centre (identity at 4:3). Games cull against the
SXY values they read back from us, so post-projection screen-
bounds culls stay aligned with the visible frame.
- present: GL letterbox_rect generalised to the configured aspect
(gl_renderer_set_display_aspect); SDL_Renderer path gets an
aspect-derived logical size + dst rect. Window opens at the
configured aspect (clamp_window_4x3 -> clamp_window_aspect).
- config: [video] aspect_ratio = W:H (4:3 default, 16:9, up
to 32:9) in game.toml, settings.toml (load+save), launcher seed.
World geometry keeps correct proportions on the stretched present;
screen-space 2D (HUD, FMV, sprite quad widths) stretches — the
known limitation of the squash tier. EXPERIMENTAL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The overlay cache keys DLLs by guest-bytes CRC, which is flavor-blind — the
same guest overlay yields the same filename whether compiled by the base
recompiler or a variant (e.g. widescreen, which emits psx_ws_*/GTE-adjust
symbols). Two flavors sharing a cache dir would collide: a base runtime could
pick up a widescreen DLL (or vice versa) with no guard, since both report ABI
v2.
Fold a codegen-flavor id into overlay_abi() (high 16 bits) and check the
combined tag in the loader's existing ABI gate. Base/master flavor is 0, so the
tag stays == PSX_OVERLAY_ABI_VERSION — fully backward compatible with every
existing base DLL (verified: an 8-DLL base cache loads with zero rejections).
A variant build sets PSX_OVERLAY_FLAVOR (e.g. widescreen: -DPSX_OVERLAY_FLAVOR=1
for the runtime, --flavor 1 for compile_overlays.py); its overlays then report a
distinct tag, so the base runtime rejects+regenerates them and vice versa. The
caches stay separate automatically, even in a shared directory — no new paths.
overlay_api.h: PSX_OVERLAY_FLAVOR (default 0) + PSX_OVERLAY_ABI_TAG.
overlay_loader.c: both ABI gates check the tag.
compile_overlays.py: --flavor arg -> -DPSX_OVERLAY_FLAVOR, baked into overlay_abi().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an opt-in "skip full-motion videos" feature ([video] auto_skip_fmv,
default off). A streaming FMV is detected as active MDEC decode
(mdec_get_decode_count climbing per vblank) + cdrom_xa_stream_active();
while detected, the frontend injects START into pad slot 0 so the game's
OWN FMV player aborts the movie and jumps cleanly to the next screen —
instant, invisible, and desync-free, with all guest state correct.
Mechanism notes (why this and not the alternatives):
- Flooding XA sectors ('instant' CD during streaming) desyncs the player
and hangs the movie on a white screen — rejected.
- Uncapped-pacing fast-forward is safe but slow and choppy (plays the
whole stream) — rejected.
- Triggering the game's native skip is instant and correct. Injection is
gated on raw xa_stream_active so START stops the moment the FMV ends and
never bleeds into the next screen (verified: title stays interactive).
Present/pacing are suppressed + audio muted as a safety net for the 1-2
frames before the game's abort takes effect.
Plumbed end-to-end: config_loader [video] auto_skip_fmv + UserSettings
(settings.toml load/save), main.cpp detector/injection + settings layering
+ launcher seed/apply, and a "Skip FMVs" toggle in the launcher Settings
(launcher.cpp + launcher.rml), mirroring the existing SPU/AA toggles.
Runtime-only; no emitter/regen impact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the launcher's Memory Card 1/2 panels to real on-disk data and make
the per-card enable/path choices flow through to the runtime.
- memcard.c/.h: add stateless introspection (memcard_summary_path parses the
directory frames for used-block count + mtime + validity, no live state),
memcard_format_file for "New", and memcard_init_slots for per-slot paths +
enable flags. memcard_init now delegates to it.
- config_loader: settings.toml [memcard] gains card1/card2 (explicit paths)
and enable1/enable2; load + save round-trip them (UserSettings extended).
- main.cpp: resolve the effective memcard dir before the launcher so it can
introspect the real files; seed/apply per-card paths+enable; boot via
memcard_init_slots (a disabled slot is left absent — SIO reports no device).
- launcher: MC cards now show real filename, size, used-block count, last-
modified date, and a live 15-cell block grid; Browse picks an existing
.mcd, New formats a fresh one (overwrite-prompt save dialog). Per-card
enable + paths are committed to settings on launch.
- The block grid is injected via data-rml (the structural data-for view does
not capture inner-xml in this RmlUi build; data-rml is the robust path and
the markup is fully launcher-controlled).
- Add LauncherSystemInterface so RmlUi warnings/errors reach stdout (they
otherwise go to OutputDebugString on Windows); drop an unsupported
text-overflow RCSS declaration it surfaced.
Verified on Tomba: card1 3/15 + card2 1/15 match the on-disk directory
frames; disabling slot 2 in settings.toml leaves it not-present via mc_status.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a shared disc-identity module (runtime/{include,src}/disc_identity)
that opens a .cue/.bin/.iso, checks for the ISO9660 PVD (raw 2352 and
cooked 2048 layouts), reads the PVD volume id, parses the boot serial
from SYSTEM.CNF, derives the region from the serial prefix, and
optionally streams a full-file CRC32 and compares against expected
serial/CRC. This is now the single source of truth for "is this the
right disc?" — main.cpp's launch-time validate_disc_image is rewritten
to delegate to it (dropping its duplicated raw-read logic and the now
dead cue_data_path/is_iso_pvd/psx_exe_id helpers) so the launch check
and the launcher badge can never drift.
game.toml [game] gains optional disc_crc / disc_sha1 (GameConfig +
config_loader). Left unset by default: the serial check is dump-
agnostic, whereas a baked CRC would flag every legitimate dump of a
different revision/region.
Launcher: GameInfo carries the expected serial + CRC; the launcher
recomputes verification on entry and on every Browse, showing a
colour-coded badge (Serial verified / Disc verified / CRC mismatch /
Wrong game? / Not a PlayStation disc) plus a detail line with the
detected serial · region · CRC. CRC is only hashed when an expected
CRC is configured.
Verified all three tiers on Tomba: Serial verified (no CRC),
CRC mismatch (wrong CRC), Disc verified (CRC B00ECB0D match).
tools/shot_launcher.ps1: bring the launcher GL window to front and
screenshot its client area (GL windows screenshot black via PrintWindow).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude