OVERLAY_REGION_FLOOR was hardcoded to 0x98000 = Tomba 1's main-EXE text end
(0x10000 + 0x88000). Tomba 2's boot EXE is only 0x28800 (text ends 0x38800),
so its disc overlays load at 0x80085000+ — inside [0x38800, 0x98000), which the
hardcoded floor wrongly classified as statically-recompiled main-EXE text. That
forced every Tomba 2 overlay onto (a) slow block-by-block dispatch and (b) the
bail-prone dispatch_nonlocal_call contract path; when the per-frame interrupt
pump fired mid-contract it cascaded into the $ra=$sp=1 bail storm = the
Whoopee-Camp splash freeze.
Make the floor a runtime value (g_overlay_region_floor), pinned at game load to
(load_address + text_size) & 0x1FFFFFFF. Defaults to 0x98000 for BIOS-only;
evaluates to the same 0x98000 for Tomba 1 (no behavior change), and is
correct-by-construction for every game.
Resolves: bail storm gone (bail_first=0); splash advances at 60 fps past the
old frame-1823 wedge. NEW failure point exposed next: dispatch_depth desync on
the now-local-flow overlay path (fixed in the following commit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Always-on, heartbeat-resident diagnostics for the Whoopee-Camp splash spin:
- VBLANK raise/deliver/ack counters (interrupts.c + memory.c) + irq_deliver.
- Deduped bail-source ledger keyed by (call-site $ra, wild target) (traps.c,
hooked at psx_call_contract in cpu_state.h) + heartbeat top-key/first-bail.
Result: REFUTES the exception-reentry-storm hypothesis. VBLANK is raised,
delivered AND acked exactly 1/frame (textbook-healthy); irq_deliver is 1/frame,
not a storm. So the freeze is NOT an interrupt/timer/CD problem -- the hardware
sim is healthy. The ~68k bails/frame are the GAME splash loop busy-polling a
kernel event (TestEvent/WaitEvent) that never fires -- the DeliverEvent/WaitEvent
discriminator is the next probe.
(The dominant dispatch-trampoline bails are emitted in generated BIOS code, so
the precise bail call-site needs a recompiler-emit change + BIOS regen; the
direct-call ledger here is partial. Deferred -- not needed to localize to the
event wait.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The .cue parser only extracted the FILE line and the CD model hardcoded a
single data track: GetTN always returned 01..01 and GetTD returned LBA 0 for
any track. Multi-track discs with a CD-DA audio track (Tomba 2: TRACK 02 AUDIO
@ 38:48:74) therefore reported a bogus TOC.
- iso_reader: parse TRACK/INDEX lines into a CDTrack table; synthesize a single
data track for bare .bin/.iso. New TrackCount/TrackStartLBA/TrackIsAudio +
iso_track_* C wrappers.
- cdrom.c GetTN returns first=1/last=track_count (BCD); GetTD returns each
track's real start MSF (lead-out via 0xAA/track 0 unchanged).
Framework fix for every CD-DA title. NOTE: does not by itself unblock Tomba 2's
boot — that still stalls in a GetStat/GetID/Init disc-init loop (display toggles,
reading=0) which does NOT call GetTN/GetTD; next is an oracle CD command/response
diff at that loop (and the hardcoded GetID region "SCEI" vs USA "SCEA" is a
candidate). cdrom_trace_dump floods with DMA 'D' events — fix that (Rule 15)
before the diff.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the fail-closed native entry guard from wt/tomba2 (b5a6940 / bd60381) to
master, where it had never been merged — which is why Tomba (and any master-built
title) still wedged. This is the long-standing "blue screen" / shared-RAM-
divergence-between-interp-and-native class.
Root cause: an overlay function dispatched at a FOREIGN INTERIOR PC (a range-
ownership mismatch routed a PC the function does not own into it) hit the CPS
entry-switch `default: break`, which fell through and ran the function FROM ITS
TOP — re-running its prologue and corrupting shared CPU/RAM state, so the guest's
main loop then waits forever on a condition that never gets set (the 0xCF0 SIO
spin / blue screen). Reproduced on Tomba's Village of All Beginnings: a native
0x000E7000-region shard function entered at a foreign PC wedged; the interpreter
ran the same bytes correctly.
Fix: the generated CPS entry-switch for OVERLAY functions (config_.overlay_mode)
now FAILS CLOSED on an unknown non-zero continuation PC — it restores the PC and
calls psx_native_bad_entry instead of running from the top. overlay_loader_dispatch
consumes g_native_bad_entry and routes the PC to the sanctioned dirty-RAM
interpreter (the live bytes run correctly — NOT a stub/HLE). True-prologue entry
still runs from the top. Static (BIOS/boot-EXE) functions keep the legacy fall-
through. ABI bumped 5->6 so stale ABI-5 caches are rejected and rebuilt with the
guard.
Minimal port: kept only the fail-closed mechanism + its ABI plumbing. The Tomba2
host_aliases interior-routing and the native-block / CPS-probe diagnostics (which
depend on the overlay-cache-v2 multi-range Candidate fields not on master) were
intentionally omitted — the fail-closed default alone is the safety net.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two framework bugs that made overlay variants run interpreted forever (the
"agonizing lag" in overlay-heavy areas like Tomba's dwarf village):
1. Cache read/write divergence. The loader reads <exe>/cache and the runtime
writes captures to <exe>/overlay_captures.json, but the gcc autocompile path
used the game.toml command verbatim, whose hard-coded --out-dir/--captures
pointed elsewhere (build-stable/build-modern). So compiled shards landed where
the loader never looked and captures were read stale -> the variant never went
native. Fix: the runtime now injects PSX_OVERLAY_CACHE_DIR / PSX_OVERLAY_CAPTURES
(the canonical loader paths) into every autocompile spawn; compile_overlays.py
honors them over any CLI flag (--captures now optional). One source of truth for
the cache location, uniform across all games, dev or prod. gcc/tcc stay separated
by the existing <compiler> namespace.
2. Overlay DLL preamble missing psx_ws_cull_sltiu. auto_screen_x emits this helper
at every width-cull site into EVERY build's C (identity at 4:3), and the main EXE
links it from gpu.c, but the overlay DLL preamble never defined it -> any overlay
variant with cull sites failed to link -> no shard -> interpreter. Fix: provide
the self-contained helper (depends only on the already-forwarded psx_ws_x_margin)
in the DLL preamble.
Proven on Tomba dwarf village: dispatch_native 29/s -> 11454/s (was ~97% interp,
now ~97% native).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compiler-neutral tier-selection policy extracted from the mis-named
overlay_sljit.{c,h} into NEW overlay_backend.{c,h} (OverlayBackend enum +
resolve/name/active). overlay_sljit.{c,h} is now just the deprecated emitter.
New tcc tier. gcc and tcc run the IDENTICAL pipeline (recompiler -> C ->
compiler -> DLL -> loader); only the compiler binary differs, so both map to the
autocompile-spawn provider (code_provider.c) and the COMMAND is chosen by the
resolved backend (main.cpp). compile_overlays.py gains --compiler gcc|tcc: tcc
writes a separate tcc/ cache namespace, BOM-strips the C + a copy of the include
headers (tcc 0.9.27 can't skip a UTF-8 BOM), and uses tcc's built-in linker.
Backends (overlay_backend): auto (gcc if a gcc TOOLCHAIN is present, else tcc --
was sljit), gcc, tcc, auto-no-gcc (dev/test: force tcc even with gcc present, to
exercise the toolchain-less user path), sljit (DEPRECATED, gated off, lowest).
Priority: static > gcc shard > tcc shard > sljit(off) > interp. The loader scans
gcc/ then tcc/ and dedups by region+crc (filename) so a gcc shard always wins a
tie.
Self-contained tcc: with no explicit overlay_autocompile_cmd_tcc, the runtime
CONSTRUCTS the tcc command from a toolchain bundled beside the exe
(<exe>/overlay_toolchain/: embedded python + tcc + recompiler + compile_overlays.py
+ runtime headers) -- no system python or gcc needed. config_loader gains
overlay_autocompile_cmd_tcc (a dev override). autocompile cmd buffers enlarged for
the constructed path-heavy command.
Compilation stays OFF the dispatch/render thread (off-process autocompile
subprocess, same model as gcc); the main thread only spawns it and dlopens the
finished DLL.
Validated: MMX6 with overlay_backend=auto-no-gcc fills the tcc/ namespace live
from the bundled toolchain and plays through save-load + gameplay -- the exact
path the sljit tier softlocked on. The gcc path is untouched.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Off-thread sljit compile worker (overlay_compile_worker.c/.h): on a dispatch
gap-fill miss the dispatch thread snapshots the region bytes and ENQUEUES; a
background worker JITs from the snapshot (never live RAM) and publishes the shard
to the on-disk cache via an atomic temp+rename; dispatch picks it up on a later
miss via an idempotent rescan, so the candidate table stays single-threaded. This
removes the synchronous JIT frame hitch. Validated on Tomba + MMX6: compiles
off-thread (one MMX6 compile was 26ms, paid entirely off the dispatch thread with
no frame-pacing spike), shards persist + reload, queue never backs up.
Observability (ring-buffer doctrine): always-on recent-compile ring + per-compile
timing in the worker, the resolved on-disk cache dir, and a tier-enabled flag, all
surfaced via a new sljit_async TCP debug command (documented in TCP_COMMANDS.md).
DISABLE the sljit tier by default (g_sljit_tier_enabled = 0): the sljit EMITTER
mistranslates some MMX6 overlay instruction(s) -- sljit-only save-load softlocks/
loopbacks while PURE INTERP works (discriminator), and it reproduces with the
worker OFF in the synchronous path (PSX_SLJIT_SYNC=1), so it is the emitter, not
the async worker. Until the emitter bug is found + fixed: no shard READ
(scan_sljit_cache_dir skipped), no GENERATE (worker not started + sljit_live
forced 0 + dispatch gap-fill gated), every overlay miss falls to the interpreter.
gcc (DLL cache + autocompile) and overlay capture are untouched, so priority is
gcc > interp and dispatch misses are still recorded for future gcc coverage. gcc
shards are bundled in release zips, so shipped builds are unaffected.
PSX_SLJIT_ENABLE=1 re-enables the tier for emitter-fix testing.
Runtime-only, no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the old incomplete RAM-only fast_boot snapshot with a COMPLETE
hardware save-state captured at the BIOS->game handoff. First launch (and the
first launch after any update) boots the real BIOS normally and captures the
full machine state at the game's PS-EXE entry_pc; every subsequent same-build
launch restores it and presents the game's first frame instantly.
This is not HLE: it persists the real hardware state a real BIOS run produced.
Format (boot_state.h v2): a sectioned file with an integrity key
(bios_checksum + entry_pc + PSX_OVERLAY_CODEGEN_HASH + ABI tag + codegen ver).
Any mismatch -> reject -> normal boot + recapture, so a user/app update can
never silently load a stale snapshot. Missing/short/unknown sections also reject
(no partial restore).
Complete capture (one section per subsystem), the fix for the old snapshot's
real bug (it left GPU/SPU/CD/DMA/SIO at reset while CPU/RAM assumed configured):
- CPU regs, 2 MB RAM, 1 KB scratchpad, IRQ (i_stat/i_mask), root timers, guest
cycle counter, dirty-RAM page bitmap (guest-written code pages).
- GPU: display/draw-area/offset/mask/texpage + GPUSTAT bits + GP0 command
machine + VRAM-transfer state; VRAM (1 MB) via the renderer-agnostic
gr_vram_transfer_out/in hooks (so it is correct on the GPU-authoritative GL
backend, where the FBO -- not gpu.c vram[] -- is truth, and the FBO re-upload
on restore happens for free).
- SPU: register file + 24 voice decode/ADSR state + latches + 512 KB SPU RAM.
- CD-ROM: full controller FSM (regs/IRQ-latch/FIFOs/seek/read/pending/queued +
XA decoder history + disc-speed model).
- DMA: channels[7] + dpcr/dicr + async-transfer state.
- SIO: registers + pad-config FSM + memcard FSM (incl. per-slot mc_slots[]) +
cycle-paced shifter/ACK scheduler.
Host-side/recompiler-derived state (overlay/sljit tables, code-provider) is NOT
serialized -- it re-initializes fresh on the normal startup path.
Per-subsystem accessors use an X-macro field list so bytes/write/read can never
drift. Runtime-only; no recompiler/codegen change, no regen. Reuses the
[runtime] fast_boot key (semantics upgraded; type unchanged).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comprehensive 3-way merge of the dev/ape-escape branch into master for
the isolated wt/ape-escape worktree. Brought in every complementary ape
change automatically (dma.c, memory.c interrupt byte/half lanes,
function_analysis discovery, main_bios vector seeds, iso_reader
iso_sector_count, config_loader, crash-trace fields, headers). Resolved
the 13 conflicts by rule:
- CPS dispatch / recompiler (code_generator, full_function_emitter,
dirty_ram_interp, traps, main_psx) -> KEEP MASTER (evolved system).
- CD model (cdrom.c) -> take ape (the model that booted Ape).
- interrupts.c/.h -> reverted to MASTER (ape's interrupt-delivery is
dispatch-coupled; the merged Frankenstein stormed).
- sio/debug_server/freeze_heartbeat/main.cpp -> keep master (evolved).
- crash_trace.c -> master (ape freeze-dump fns omitted to avoid the
debug-ring cascade).
STATUS: does NOT boot yet. ape-cdrom + master-interrupts derails to
PC=0 at frame ~2092 (insta-close); ape-cdrom + ape-interrupts storms
(297M exception re-entries) at frame 2077. Root finding: ape's CD
anti-storm is coupled to ape's interrupt-delivery + call-depth + EPC
model, which master replaced with CPS continuation-passing, so the CD
subsystem does not forward-port as pure hardware simulation. Preserved
as the reference state for a design consult on bridging the two models.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The launcher Hybrid controller mode (built first, for Tomba, to mimic Tomba:
Special Edition) flips the emulated pad's reported type 0x41<->0x73 on
d-pad<->stick. After the real DualShock config state machine was added for MMX6
(98aa688), that flip regressed every game's libpad re-detect into a 1-frame
"pad unplugged": Tomba unpauses its menu / drops input, MMX6 phantom-dashes.
The modern config SM is correct and required by MMX6, so instead of weakening it
globally, restore the pre-98aa688 "config commands always answer 0xF3" behaviour
as a per-game opt-in: [controller] legacy_pad_config (default OFF). Only Tomba
sets it; with the flag off the modern pad path is byte-for-byte unchanged, so no
other title is affected. A/B verified live: legacy = clean Tomba flip / MMX6
input; modern = Tomba disconnect + MMX6 dash.
Explicitly LEGACY: a correct findpad/re-detect refactor would make the flip
benign for every game with no per-game branch (real hardware tolerates unlimited
analog-button presses); when that lands this whole feature set is to be removed.
Full rationale in sio.c (g_pad_legacy_cfg).
Also includes the deferred coherent pad-type apply (pad_type_req, applied only at
an idle/non-config bus boundary so a flip can't split a poll/handshake) and a
`pad_cfg` debug command + tools/padcfg.py + tools/sio_flip_trace.py used to A/B
the config SM live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 BG widen draws LEFT extra tile columns each side, but the engine's
streamer (FUN_800273e4 -> FUN_800274a0) only refreshes one column per side per
frame at the leading edge. On scene entry / fast scroll the inner reveal columns
held stale tiles from a prior scrollY -> real-but-misplaced tiles spilling
up-and-left (the "staircase"). Census confirmed this is NOT the BG packet-budget
overflow (674-816 prims/frame, well under the 1000-tile cap).
Fix (render window == refresh window): clone FUN_800274a0's inner metatile->ring
fill into mmx6_fill_column() and re-stream every column of the widened window for
all 3 layers each frame (psx_ws_mmx6_bg_refill_all), triggered once/frame from the
existing count_site hook (mmx6_bg_refill_tick, frame-gated). Uses each layer's own
scroll, omits the engine's scrollX wrap-bookkeeping writes (those drive the engine's
own incremental streamer), skips map-loop seams; off-map columns fill tile 0 (the
renderer skips them). Validated byte-exact against the engine's own streaming
(0 mismatches / 1113 cells x 3 frames; debug cmd mmx6_freshfix {validate:1}). Gated
native-wide <=384w so 4:3 and the 512 hi-res title stay byte-identical.
Also removes the 2-col reveal cap (WS_FULL2D_MAX_REVEAL): it never addressed the
staircase and cost screen real-estate; the full 53px/4-col reveal is restored.
Parked, gated-off experiments ride along (inert, default off): host-side reveal
(Attempt A: gpu_ws_mmx6_emit_reveal + sw_wide_emit_tile) and BG packet-buffer
relocation (Attempt B: [widescreen.bg2d] bufbase/cap recompiler hooks). These target
the separate dense-stage budget-overflow concern, not this fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The BG renderer widen (a8f37c1) draws extra tile columns on both sides, but the
engine's 64-column tile ring is only refreshed at its leading edge: the streamer
(FUN_800273e4) re-streams ONE column per side per frame at world-X scrollX-16
(left) and scrollX+16+width (right). So the columns the widened renderer reaches
were never populated — they showed empty (black margin) or stale (old tiles as
the camera scrolled), the "stale margins" the user saw in motion.
Add two more [widescreen.bg2d] sites (stream_left_site / stream_right_site) that
push the streamer's left/right leading-edge world-X out by LEFT*16 via gpu.c
psx_ws_mmx6_bg_stream_{left,right}, so the incremental streamer keeps the ring
valid across the widened window as the camera scrolls. Same number of streams
(the single leading-edge target just moves out LEFT columns) — no extra cost.
Identity at 4:3 / 512 hi-res.
Verified: the previously-black left reveal margin now fills with coherent
adjacent stage; blackL/blackR = 0 across scenes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure-2D sprite games (Mega Man X6) render a true 4:3 field of view with no
overscan to reveal, so 16:9 requires making the engine draw more. Two parts:
- [widescreen] full_2d opts a pure-2D game into the native-wide present path
(it never emits the 3D sprite-tag hook the gameplay detector keys on, so
every in-game frame was misclassified as a 4:3 menu and pillarboxed).
- [widescreen.bg2d] is a gen-time hook on the per-layer background tile
renderer (MMX6 FUN_800270d0) at its column-count / start-tile-column /
start-screen-x instructions, routed through gpu.c psx_ws_mmx6_bg_{cols,
startcol,startx}. It draws LEFT=ceil(nw_offset/16) extra 16px columns on
BOTH sides so the loop fills the 16:9 reveal margins with real adjacent
stage, existing columns staying pixel-aligned with objects. Identity at
4:3 and in the engine's own 512 hi-res mode (so the title is untouched).
Tooling: wide_full debug command + sw_wide_dump_full dump the entire
native-wide compositor surface (both vertical-double-buffer y-bands and both
reveal margins) for diagnosis.
Known follow-ups: object screen-X cull widen (actors in the margins);
top/bottom flicker + stale-margin refresh on the vertical double-buffer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual PSX_OVERLAY_CODEGEN_VER must be hand-bumped on every emitter change;
when it isn't (e.g. between CPS emitter iterations), a stale-but-cgN DLL is reused
-> the v0.3.0 black screen. Now hash_codegen.cmake hashes the recompiler codegen
sources into a (gitignored) overlay_codegen_hash.h at build time; overlay_api.h
exposes PSX_OVERLAY_CODEGEN_HASH and the cache path becomes gcc/<arch-abi>/cg<N>_<hash>/.
The loader and compile_overlays.py read the same value, so any emitter change
auto-invalidates the cache (old dirs coexist, no stale load). Falls back to hash 0
when the header is absent (fresh checkout) — identical to today's single-cgN path.
Verified: build emits hash 0x347751c1; autocompile + loader both use cg4_347751c1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A persisted option value outside its declared [min,max] is now REJECTED at load
(the game's own default is used instead), so a corrupt/stale options file can no
longer inject an out-of-range value into a config global the game may use as a
jump-table index (wild jump = softlock). Ranges declared per option in
game_options.toml (min/max; absent => no validation, e.g. signed screen offsets).
Adds a format_version header (newer-than-known files are rejected wholesale).
Restore stays name-keyed (offset-drift-safe). Runtime-only, no regen.
Validated: corrupt values rejected (defaults applied); valid file restores intact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vulkan backend alongside software + OpenGL: geometry/textured-prim
pipelines, GPU-authoritative VRAM image, deferred-submit + drain-at-present
perf model, GPU-direct present (vk_renderer_present_vram), and a
full-VRAM-pack readback path (ensure_cpu rect_adds the whole VRAM rect
before pack_flush so screenshots/guest VRAM reads reflect live content,
not just the incremental pack-dirty rect — readback-only, no per-frame cost).
Runtime plumbing: --debug-port / --renderer / --window-title CLI overrides
(multi-instance fleet), latency_ring, SPIR-V shaders + embed_spirv.py,
config_loader renderer selection, debug_server screenshot wiring.
All on feat/vulkan-renderer; snapshot in backup/vulkan-wip-20260617.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Oracle disambiguation (RECURSION_BUG.md §23–§24). Traced one func_8001A954
invocation on real HW (psx-beetle) vs the recomp in the IDENTICAL Tomba idle
state (New Game village dialogue; object 0x801FD950, state/scene/jumptable=0).
VERDICT: real HW's per-frame chain is BALANCED — func_8001A954 is entered once
per frame from main_loop (JAL 0x8001A630, sp=0x801FE3E0) and RETURNS once per
frame (jr ra @0x8001A9E8 -> 0x8001A638, SP fully restored); no re-entry, the
overlay 0x80063864 never calls back (refutes §21/§22's "call-cycle"). The recomp
runs the same guest flow and the guest SP unwinds identically, but the HOST stack
leaks +1.17 KB/frame (ce_profile, live-reconfirmed: 20099->22215 KB / 1807 frames)
because the per-frame compiled chain (func_8001A954->AC00->B2B4->B5A8->func_80046264,
emitted as DIRECT func_X(cpu) host calls) re-enters itself via func_80046264's
CD-DMA-dirty hop and never unwinds.
native_stack early-trip (PSX_STACK_GUARD_KB=6144, frame 7592, 6 MB) pins the
leaking host cycle. Cross-checked: xprobe mixed_depth bounded at 6, dirty-insn/
dirty-block show NO literal func_8001A954 transfer, and a954=1/frame is a SURFACED
guest tail-transfer (j/jr) to func_8001A954 that the oracle proves real HW never
makes — i.e. there is both (A) a correctness divergence (spurious per-frame
re-entry) and (B) the architectural host-mirroring limitation that amplifies it.
Fix scope CORRECTED from §23's "quick contained runtime patch": the UNIVERSAL fix
is §11 — route in-range guest calls through the depth-managed psx_dispatch_impl
trampoline instead of mirroring them as direct host calls, so the host stack stays
bounded regardless of guest re-entrancy (real or fabricated). §12 (CD-DMA dirty
precision) and fixing the specific divergence are surgical mitigations, not
universal. Performance loss accepted for now (static recomp: right before fast).
Instruments added (runtime-only, no regen):
- SP capture in the fntrace control-flow ring on BOTH backends (GPR[29] read live):
psx-beetle (beetle_libretro.cpp + beetle_debug_server.c) and the recomp
(fntrace.h/.c + debug_server.c) — enables the func_8001A954 SP/RA-lifecycle oracle.
- psx-beetle default debug port 4380 -> 4382 (+ --port N override): 4380 collided
with psxref and cdirecomp/CdiRuntime.exe; two LISTENers on one port silently
misroute connections (the phantom "oracle keeps dying" early in the session).
Oracle tooling: the control-flow oracle is psx-beetle (Beetle interprets -> records
every J/JAL/JR/JALR), NOT psxref (no PC trace; the assumed emu_step/emu_trace_addr
never existed). Loads Tomba via: psx-beetle.exe --disc tomba.cue <bios> --port 4382.
Next: implement the §11 universal fix (codegen + full regen).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Live-tunable selectable stretch gate to identify which drawn prims form the
native-wide 2D-backdrop void: ws_dbg_stretch modes = OT-range / clut / tpage /
tagged / untagged-tex / all-tex, with per-frame matched + matched-tagged counts
(psx_ws_dbg_gate_frame_snapshot). Routes through the existing GL centre-stretch.
Confirmed live: the void layer is the early-drawn, SPRITE-TAGGED 0x65 background
grid (6 cols x 7, step-64, cluts unique to the backdrop); stretching it about
screen-centre fills the void with no inter-tile gaps (GL scales width too).
Tagged fills it; untagged (3D) does not. The prior content-match plan is
DISPROVEN (template transformed beyond recognition before draw) -- corrected
model recorded in BACKDROP_PRELOAD.md. Fix gate chosen: native-wide + tagged +
background-phase (draw-time), since backdrop X is written by a shared sprite
helper with no clean per-emit source signal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 dwarf/flower-field void is native-wide 2D-backdrop POSITIONING, not a
column-loading problem. The earlier auto_backdrop whole-row preload is proven
IRRELEVANT (ws_backdrop_ring: whole 41-col row loads, void persists; margin off
vs whole-row pixel-identical). The flower-field is a fixed-screen 2D tile field
(static 11-bit vertices, ~4:3 width) that bypasses the GTE, so native-wide (which
widens 3D via the GTE) never widens it. Fix = geometric X-stretch about
screen-centre of the 2D backdrop in the GL wide-mirror — USER-CONFIRMED it fills
the void (with the gate disabled). Full record: BACKDROP_PRELOAD.md (2026-06-16).
Remaining: gate the stretch to ONLY the backdrop (currently also stretches Tomba/
HUD/3D). Path forward = draw-time gp0_cmd_source_addr / sprite-tag classification.
Runtime WIP checkpoint (all behind native-wide + runtime toggles, 4:3 untouched):
- gpu_gl_renderer.c: u_xscale/u_xcenter geometric stretch in the geo+tex
wide-mirror passes; per-frame backdrop-phase reset moved to gl_perf_present_enter
(both present paths); textured-only phase mode; per-prim trace + dbg counters.
- gpu.c: ws_backdrop_ring (always-on rewrite ring); ws_backdrop_margin live
column-widen (proven irrelevant); psx_ws_backdrop_value whole-row path.
- dirty_ram_interp.c: ring capture (extent/camx/count/a1/dl) at the interp hook.
- debug_server.c: ws_backdrop_ring / ws_backdrop_margin / ws_backdrop_stretch.
- main.cpp: keyboard ALWAYS drives player 1 (OR'd with any controller).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist validated sljit shards across launches (SLJIT_PERSIST_CACHE.md). On JIT,
serialize the (position-independent, Stage 1) LIR and write it to
<cache>/<game_id>/sljit/<arch-abi>/cg<N>/<entry8>_<crc8>.sljit with a small header
(magic, format/helper-order versions, entry/lo/len/crc/size). At init, scan that
dir, deserialize + sljit_generate_code for THIS process, and register each as a
dll<0 candidate carrying the blob's crc; dispatch re-hashes live RAM against it so
a reloaded shard re-earns trust exactly like a fresh JIT. Priority gcc > sljit >
interp is unchanged (a gcc DLL still supersedes a reloaded shard via the obsolete
path). Persist is write-on-generate (measured negligible; crash-safe).
Threads the serialized blob through OverlaySljitResult -> CompiledFragment;
register_sljit_candidate gains a crc_override (reloaded shards use the header crc
since the overlay RAM may not be loaded at init). sljit_status exposes
reloaded / persist / reload-diagnostic counters.
Two gotchas found + fixed in testing (vs the original design notes):
- serialize MUST precede generate_code (generate sets compiler->error =
SLJIT_ERR_COMPILED, after which serialize returns NULL);
- serialize with options=0, not SLJIT_SERIALIZE_IGNORE_DEBUG, so an
argument-checks build can deserialize its own blobs.
Validated: two-launch test reloaded 12 shards (deserfail=0), ran them at a fresh
process base with no crash, gameplay renders correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prerequisite for persisting sljit shards to disk (SLJIT_PERSIST_CACHE.md). The
JIT baked absolute host-helper pointers as SLJIT_IMM operands at three sites
(emit_helper2: memx/cop2; emit_ws_cull; PEND_CALL: call), so a shard serialized
in one process would call stale addresses when reloaded at a different ASLR base.
Route all four helpers (SLJIT_HLP_MEMX/COP2/WS_CULL/CALL) through a cpu-relative
table CPUState.sljit_helpers[], loaded via MEM1(R_CPU)[offset] instead of a baked
immediate, so serialized LIR holds zero absolute host addresses. The table is
appended at the END of CPUState (existing gcc overlay-DLL field offsets stay
put) and populated once at startup (overlay_sljit_init_helpers). In-session JIT
is unchanged behaviourally (one extra indirection per helper call).
Smoke-tested: forced sljit with the gcc cache moved aside, 12 shards JIT'd and
ran, title renders, no crash. Broader playtest folds into the Stage 2 reload test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller (item 3): present an analog (DualShock) pad by default for
analog-capable games so the left stick gives variable run speed; fold the
physical D-pad / keyboard arrows onto the left analog axes at full deflection so
they still move the character (avoids the Special Edition full-throttle-D-pad
bug). Add a launcher-editable analog-stick deadzone option (settings.toml +
input.ini + game.toml [controller] default), modeled on snesrecomp.
Settings persistence (item 6): deterministic restore-at-init + save-on-exit for a
game's native OPTION-screen settings, declared per game in a dedicated
game_options.toml. The recompiler rewrites each setting's boot-init store (sb/sh)
to route its value through psx_game_option_store(addr,val), applying a value
persisted from a prior session over the game default (identity when none, so a
fresh install is byte-identical). The runtime saves the live config globals to
<memcard_dir>/<game_id>.options on exit (atexit). Adds a game_options debug
command. Requires a regen for the hook to emit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measured Tomba village = 93% textured prims, so draw batching must target the
textured path (per-prim uv-limits/texpage/clut/depth move to vertex attributes
so consecutive textured prims coalesce into one draw). Flat batching alone
would touch only 7% of prims.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds prims/frame and per_prim_us to the frame_perf aggregate. Proved Tomba's
village is draw-call bound: ~1000 prims/frame at ~14us of GPU time PER
glDrawArrays (a 1-triangle draw should be <1us) — pipeline-flush overhead from
re-binding state between every primitive. 16:9 is 2x slower purely because the
native-wide double-draw issues 2x the draw calls. Points squarely at draw
batching (group consecutive same-state prims) as the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Always-on per-frame ring in the GL renderer: two GL_TIME_ELAPSED queries
bracket each frame's scene draws vs the present clear+blit (true GPU time,
independent of CPU/GPU overlap), plus SDL perf-counter CPU wall time
(present-to-present total, present call). Exposed via the debug server
"frame_perf" command, aggregated over the recent ring and split by 4:3 vs
native-wide so a 16:9-vs-4:3 A/B reads off one query.
Built to answer where a frame goes without guessing. First use proved
Tomba's 16:9 village lag is the native-wide DOUBLE-DRAW (every primitive
re-issued into the wide FBO), NOT supersampling (SS2==SS1) and NOT the
compositor (present_gpu ~1ms): scene_gpu 16:9=28ms vs 4:3=17ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GL native-wide present (Option B) did glFinish + glReadPixels of the whole
wide FBO to the CPU and re-uploaded it EVERY frame — a hard GPU pipeline stall +
round-trip that made OpenGL 16:9 run far slower than the software renderer (whose
wide path is pure CPU, no GPU sync). Confirmed by isolation: GL 4:3 smooth, GL
16:9 slow, independent of SS/AA.
Add gl_renderer_present_wide_fbo: blit the displayed buffer's wide FBO straight to
the window with glBlitFramebuffer (GPU-side, V-flipped, into the letterbox),
exactly like the canonical gl_renderer_present_vram — no readback, no glFinish,
no re-upload. main.cpp routes wide GL frames here (falling back to the CPU readout
only if the wide surface doesn't exist yet). USER-VALIDATED smooth. The CPU
readback path (glb_render_wide_display) stays as the fallback + the wide_shot
probe.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documentation only. Prior sessions kept re-deriving state from breadcrumbs that
were no longer true:
- overlay_sljit.c / code_provider.{h,c}: corrected "declines every fragment" and
"compile_fragment not yet called from dispatch" — the MIPS->sljit emitter is
fully implemented and wired (then validated via the same-state diff).
- ISSUES.md: add Issue #7 (sljit live execution was unvalidated; the pure-live
save-load wedge), root cause and fix recorded.
(SLJIT.md — the §11 forward plan and architecture clarification — lives at the
workspace root F:/Projects/psxrecomp/SLJIT.md, outside this repo, updated
separately.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause class: a toolchain-less machine never self-improved via sljit on the
normal play path. Two blocks: (1) `auto` resolved to gcc on EVERY machine because
the discriminator was autocompile_configured() — "is a command STRING set" — and
the shipped game.toml always carries overlay_autocompile_cmd; (2) live sljit mode
was reachable only via a hand-set env var (dead public setter, no policy).
Fix:
- autocompile.c: autocompile_toolchain_available() probes PATH for a real C
compiler (gcc/cc/clang). main.cpp passes
`autocompile_configured() && autocompile_toolchain_available()` to
code_provider_init, so `auto` picks gcc only when one is actually reachable,
else sljit.
- overlay_loader.c: overlay_loader_apply_live_policy() (called from main.cpp
AFTER backend resolution) defaults live ON when the resolved backend is sljit
— so a toolchain-less box runs VALIDATED shards live (self-improves on the
normal path); PSX_OVERLAY_SLJIT_LIVE still overrides. The env-only block in
overlay_loader_init is removed (the policy fn subsumes it).
Runtime-only (no regen). Refs SLJIT.md §11 Phase C.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render the genuine 16:9 FOV at correct proportions instead of squash-then-stretch,
using a separate per-framebuffer wide surface so canonical PSX VRAM stays faithful
(4:3 / non-opted games byte-identical; the wide canvas only exists when opted in).
- Wide-aspect mode descriptor (off / squash / native-wide) in gpu.c; opt-in via
[video] aspect_ratio. GTE fed the 4:3 ratio (identity, no squash) in native-wide;
canonical draw_offset/draw_area left faithful (no VRAM widening, no wrap).
- Separate wide present-surfaces (gpu_sw_renderer.c) keyed by back-buffer base:
framebuffer prims mirrored at local x = vram_x - base_x + OFFSET; margins cleared
via the frame-clear mirror; present reads the displayed buffer's surface 1:1.
Eliminates the A/B bleed + flicker (and the segfault) of the shared-canvas attempt.
gr_* vtable extended; GL backend reports unsupported -> canonical present until
the GL port (task 8).
- psx_ws_x_margin native-wide-aware (reuses the existing [widescreen.cull] emit).
CONFIRMED a no-op for Tomba's edge objects: they spawn via a non-FUN_80022E44
path (the foreground classifier is already 16:9-safe). See handoff.
- Draw-census extended with per-prim screen-X extent + back-buffer base for the
native-wide void diagnosis; ws_nw TCP toggle; gpu_state ws.mode/nw_extra.
- Controller: "auto"/"gamepad"/"controller" device = first available SDL pad.
WIP checkpoint. SW compositor works: flicker fixed, faithful, terrain platform
fills the wide frame. REMAINING: discrete foreground OBJECTS (rock pillar, tree
branch, bushes) + the parallax backdrop are still culled at the 4:3 edge via a
separate generation/spawn path (wtrace the OT display-list writers next); GL
compositor port. Detail in NATIVE_WIDE_PLAN.md + the session handoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One batch widening to cover most remaining overlay instruction classes;
validated end-to-end via the live differential rather than per-op.
- jr rX (computed / jump tables): CTRL_TAILJUMP — capture the target before
the delay slot, then cpu->pc = target and return; the dispatch loop
re-dispatches (matches the interpreter's jr). jr $ra stays the clean-
return terminator. (A terminator-less tail-dispatch fragment still
declines — safe.)
- GTE/COP2: MFC2/CFC2/MTC2/CTC2/cmd (op 0x12), LWC2 (0x32), SWC2 (0x3A)
via psx_sljit_cop2(), a runtime helper mirroring the interpreter exactly.
- Unaligned mem: LWL/LWR/SWL/SWR via psx_sljit_memx(), likewise.
- BLTZAL/BGEZAL: link $ra = pc+8 before the delay slot, then branch.
Helper-emitted classes get parity by construction (the helper IS the
interpreter's code). Harness 66/66 (regression); runtime builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The big coverage unlock — the emitter now accepts functions that call
others, not just leaves.
- New runtime helper psx_sljit_call(cpu,target,return_pc,check) in
overlay_loader.c, mirroring the dirty-RAM interpreter's call path
EXACTLY (psx_dispatch_game_compiled / overlay_loader_call_native /
psx_dispatch_call + g_psx_call_bail + psx_call_contract). Returns 1 if
the shard must return immediately (transfer/bail unwind), 0 to continue.
- Emitter: jal/jalr -> CTRL_CALL. Link reg + (jalr) dynamic target are
written BEFORE the delay slot; the helper is called AFTER it via S2
(helper address) with the result driving a conditional return. jal uses
an absolute target, jalr a dynamic one captured in S1.
- KSEG integration fix: the loader now passes the VIRTUAL entry to the
emitter (was passing phys with image_base_vram=0), so return_pc and
jal/J targets carry the KSEG bits the guest uses. The standalone harness
passed a virtual entry, which is why it stayed green; the live
differential caught the phys/virtual mismatch (saved $ra off by the
0x80000000 KSEG bit).
Verified: sljit_emit_test.c extended (jal, jalr, transfer/bail) -- 66/66
pass. Full psx-runtime build clean. Live differential re-validation of
the KSEG fix is the next step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MIPS->sljit emitter, paralleling dirty_ram_interp.c's exec_one but
emitting host code. First slice = single-block LEAF functions ending in
`jr $ra`; everything else aborts the whole fragment to the interpreter
(precision over recall).
- overlay_sljit.c: memory-backed GPRs, 32-bit ops (R3000A wraparound),
loads/stores via icall through the cpu read/write callbacks (same
MMIO/page-watch/dirty-tracking path as the interpreter). Opcodes:
SLL/SRL/SRA/SLLV/SRLV/SRAV, ADD(U)/SUB(U), AND/OR/XOR/NOR, SLT(U),
ADDI(U)/SLTI(U)/ANDI/ORI/XORI/LUI, LB/LH/LW/LBU/LHU/SB/SH/SW, SYNC.
- compile_fragment now returns CompiledFragment{fn,code_lo,code_len};
overlay_loader registers an sljit candidate (crc hashed from the live
bytes it JIT'd from, same live-byte validation as gcc candidates).
- Step 5 wiring is VALIDATION-GATED: JIT-on-miss fires only when
backend==sljit AND the same-state differential is armed (never in a
shadow run). Normal play stays byte-identical; live sljit execution
stays off until the differential gate passes.
- Debug: sljit_try <addr> probe; sljit_status reports shards_registered.
Verified: runtime/tests/sljit_emit_test.c (links overlay_sljit.c +
sljitLir.c only) JITs hand-built leaf functions and checks output —
26/26 pass incl. gpr0 invariant, sign/zero-ext, signed/unsigned compare,
and the decline cases. Full psx-runtime build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce one backend-agnostic code-production seam the overlay
capture/dispatch spine talks to, and make the validated gcc spawn->DLL
path its first implementation. Pure refactor; gcc stays the default and
its behavior is byte-identical.
- New code_provider.h/.c: a CodeProvider struct exposing BOTH production
lifecycles so the interface isn't gcc-shaped -- batch/async
(request/busy/poll_main, the gcc model) and sync-on-miss
(compile_fragment, the sljit model). compile_fragment is defined and
implemented per-provider but not yet called from dispatch (step 5).
gcc provider = direct pass-throughs to autocompile_* (compile_fragment
NULL: a compiler spawn is too slow for the dispatch path); sljit
provider wraps overlay_sljit_* (still declines every fragment).
- Wire up overlay_backend_resolve, which was previously dead code:
code_provider_init() resolves the backend at overlay-cache init and
caches the active provider; code_provider_active() is the accessor.
- Add [runtime] overlay_backend config (auto|gcc|sljit; empty=auto) so
resolution has a config source alongside PSX_OVERLAY_BACKEND.
- Route the 3 direct autocompile_* call sites through the active
provider: overlay_capture.c autocapture tick, main.cpp per-frame poll
+ the init call, debug_server.c autocompile_run (now reports backend).
- runtime.cmake += code_provider.c.
Builds + links clean (TombaRecomp/build-dev, MinGW gcc 15.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add sljit as a self-contained, toolchain-less codegen backend for the
overlay self-improvement loop. gcc stays the default (dev: optimized +
inspectable); sljit is the fallback / production producer. Interpreter
remains the correctness floor.
This commit lands the backend-agnostic foundation only; the MIPS->sljit
emitter is not built yet, so every fragment safely declines to the
interpreter (no change to the validated gcc path).
- Vendor sljit (BSD-2-Clause) at lib/sljit/, wired into runtime.cmake
(compile sljitLir.c + include dir; sljit auto-detects the host arch).
- overlay_sljit.{h,c}: backend-selection policy (PSX_OVERLAY_BACKEND=
auto|gcc|sljit), availability, a real codegen smoke test, and
try_compile() honoring the precision-over-recall bail-to-interp contract.
- sljit_status debug-server command (Rule-3 clean: status via getter).
- THIRD_PARTY_ATTRIBUTION.md: sljit entry.
Verified: TombaRecomp/build-dev (MinGW gcc 15.2) compiles + links
psx-runtime.exe; standalone smoke harness JITs f(a)=a+1234 and runs it
correctly (26 bytes x86-64). Design + locked tier decisions in SLJIT.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The far backdrop (ocean/cloud/distant mountain) is GTE-projected 3D drawn
by the main-EXE driver FUN_8004db3c (sole caller of the GTE emitter
FUN_80027600). The global widescreen GTE X-squash compresses it toward
centre, so its geometry falls short of the revealed 16:9 edges -> blue void
that pops as the camera moves.
Fix: bracket the backdrop driver with gte_ws_set_suppress(1)/(0) so its
draws skip the X-squash and fill the stretched frame (skybox treatment),
while the near 3D world keeps the wider FOV.
FUN_8004db3c draws a MIX (far backdrop + nearer props/scenery), so the
suppress is DEPTH-GATED in gte_rtps_internal: only vertices with projected
SZ >= s_ws_far_threshold (default 900, tunable live via the ws_far_threshold
TCP cmd) un-squash; nearer props stay squashed and aligned. Validated:
threshold 900 fills the void, keeps the foreground door + background
scenery (path/tree) aligned. Residual: the mountain grass-top straddles the
depth band (per-vertex limitation) -> flickers when the camera dips; this
is the checkpoint before exploring native-wide rendering to remove the
squash (and this whole artifact class) entirely.
Mechanism:
- recompiler: [widescreen.backdrop] unsquash_funcs emits
gte_ws_set_suppress(1) at entry + (0) before each jr-$ra return
(code_generator + config_loader + main_psx). Main-EXE -> regen-class.
- runtime: gte.cpp depth-gated squash + s_ws_far_threshold; cpu_state.h
decl; debug_server.c ws_aspect (live squash on/off) + ws_far_threshold
(set split + read SZ stats) diagnostics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Menu fix (user-validated): per-prim center-squash mangled composite 2D
UI (dialog boxes built from tiled cap/middle/cap SPRTs no longer
aligned). Root cause was decoupling — content was squashed while the
frame presented 4:3. Now squash applies IFF the frame is stretched:
gpu_ws_present_native_43() is the single predicate (FMV video OR a
full-2D screen = no character billboards this frame), shared by the
GTE squash, the GPU per-prim squash, and the present 4:3 pin. Full-2D
menus get zero squash + 4:3 pillarbox = pixel-native. Removed the old
menu-2D squash paths (lines/flat-rects/mono-sprites/polylines).
Cull-margin widening (recompiler feature): the game's per-object draw
classifier tests (objX-camX+BIAS) <u RANGE sized for the 4:3 screen;
the GTE squash shows ~33% more world so the margin collapses and
objects pop near the wide edges. New [widescreen.cull] config emits
those immediates with a runtime term psx_ws_x_margin() (bias +margin,
range +2*margin, plus a1+=margin for param-margin variants) — 0 at
4:3/boot/menu/FMV so the original cull is byte-identical, ~53px when
stretching 16:9. Site instruction types are verified at emit (fatal on
mismatch). Requires regen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- FMV: 24-bit frames AND streamed 15-bit MDEC video (Tomba plays its
movies 15-bit; depth24 alone missed them — user-reported) present
pinned to native 4:3. Detector = colour MDEC decode within the last
30 vblanks (mdec_recently_active); texture-decompression luma
decodes do not stamp. Both GL present paths take force_4_3; SW path
centers a 4:3 dst rect. User-verified on the intro movies.
- Menu garble: full-2D screens (no tagged prims in flight for 2
frames) now squash lines, polylines, flat rects and mono sprites
too — dialog borders are drawn with these and stuck out past the
squashed SPRT boxes. Full-width TILEs (fades) stay exempt; in-game
non-textured prims stay untouched (may be world-anchored).
- HUD positioning: in-game untagged SPRTs pivot by thirds — outer-
third elements anchor to their screen edge (16:9 corner position,
native proportions), middle third to centre. Full-2D screens keep
the single centre pivot so composites stay coherent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User-validated on Tomba: world, characters and HUD all present at
native proportions on the 16:9 stretch.
Mechanism (all parameterized via game.toml [widescreen], inert by
default and at 4:3):
- recompiler: [widescreen] sprite_tag_funcs emits a
psx_ws_sprite_tag(cpu) callback at the entry of each listed
function (the game engine's shared per-prim helper). The runtime
records prim pointer ($a0) -> GTE-projected anchor (read from
sprite_anchor_addr scratchpad) in a frame-stamped hash.
- gpu.c: at GP0 execution, prims whose guest address (from the DMA
linked-list walk's source tracking) is tagged get every X
re-squashed around their own anchor — undoing the present stretch
for CPU-computed pixel offsets the GTE squash never saw. Covers
POLY_FT4 billboards and SPRT parts; SPRT texels keep full coverage
via gr_draw_textured_rect_scaled.
- hud_sprt_squash: untagged textured rects (SPRT family — pure
screen-space 2D: HUD, menus) center-squash around the display
centre. Untextured TILEs (fades) untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the DuckStation-style widescreen hack as a config option,
implemented in the runtime (no regen needed):
- gte.cpp: gte_set_display_aspect(num, den). RTPS/RTPT scale the
IR1*h/sz screen-X term by (4*den)/(3*num) around the game's own
OFX projection centre (identity at 4:3). Games cull against the
SXY values they read back from us, so post-projection screen-
bounds culls stay aligned with the visible frame.
- present: GL letterbox_rect generalised to the configured aspect
(gl_renderer_set_display_aspect); SDL_Renderer path gets an
aspect-derived logical size + dst rect. Window opens at the
configured aspect (clamp_window_4x3 -> clamp_window_aspect).
- config: [video] aspect_ratio = W:H (4:3 default, 16:9, up
to 32:9) in game.toml, settings.toml (load+save), launcher seed.
World geometry keeps correct proportions on the stretched present;
screen-space 2D (HUD, FMV, sprite quad widths) stretches — the
known limitation of the squash tier. EXPERIMENTAL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The overlay cache keys DLLs by guest-bytes CRC, which is flavor-blind — the
same guest overlay yields the same filename whether compiled by the base
recompiler or a variant (e.g. widescreen, which emits psx_ws_*/GTE-adjust
symbols). Two flavors sharing a cache dir would collide: a base runtime could
pick up a widescreen DLL (or vice versa) with no guard, since both report ABI
v2.
Fold a codegen-flavor id into overlay_abi() (high 16 bits) and check the
combined tag in the loader's existing ABI gate. Base/master flavor is 0, so the
tag stays == PSX_OVERLAY_ABI_VERSION — fully backward compatible with every
existing base DLL (verified: an 8-DLL base cache loads with zero rejections).
A variant build sets PSX_OVERLAY_FLAVOR (e.g. widescreen: -DPSX_OVERLAY_FLAVOR=1
for the runtime, --flavor 1 for compile_overlays.py); its overlays then report a
distinct tag, so the base runtime rejects+regenerates them and vice versa. The
caches stay separate automatically, even in a shared directory — no new paths.
overlay_api.h: PSX_OVERLAY_FLAVOR (default 0) + PSX_OVERLAY_ABI_TAG.
overlay_loader.c: both ABI gates check the tag.
compile_overlays.py: --flavor arg -> -DPSX_OVERLAY_FLAVOR, baked into overlay_abi().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an opt-in "skip full-motion videos" feature ([video] auto_skip_fmv,
default off). A streaming FMV is detected as active MDEC decode
(mdec_get_decode_count climbing per vblank) + cdrom_xa_stream_active();
while detected, the frontend injects START into pad slot 0 so the game's
OWN FMV player aborts the movie and jumps cleanly to the next screen —
instant, invisible, and desync-free, with all guest state correct.
Mechanism notes (why this and not the alternatives):
- Flooding XA sectors ('instant' CD during streaming) desyncs the player
and hangs the movie on a white screen — rejected.
- Uncapped-pacing fast-forward is safe but slow and choppy (plays the
whole stream) — rejected.
- Triggering the game's native skip is instant and correct. Injection is
gated on raw xa_stream_active so START stops the moment the FMV ends and
never bleeds into the next screen (verified: title stays interactive).
Present/pacing are suppressed + audio muted as a safety net for the 1-2
frames before the game's abort takes effect.
Plumbed end-to-end: config_loader [video] auto_skip_fmv + UserSettings
(settings.toml load/save), main.cpp detector/injection + settings layering
+ launcher seed/apply, and a "Skip FMVs" toggle in the launcher Settings
(launcher.cpp + launcher.rml), mirroring the existing SPU/AA toggles.
Runtime-only; no emitter/regen impact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the launcher's Memory Card 1/2 panels to real on-disk data and make
the per-card enable/path choices flow through to the runtime.
- memcard.c/.h: add stateless introspection (memcard_summary_path parses the
directory frames for used-block count + mtime + validity, no live state),
memcard_format_file for "New", and memcard_init_slots for per-slot paths +
enable flags. memcard_init now delegates to it.
- config_loader: settings.toml [memcard] gains card1/card2 (explicit paths)
and enable1/enable2; load + save round-trip them (UserSettings extended).
- main.cpp: resolve the effective memcard dir before the launcher so it can
introspect the real files; seed/apply per-card paths+enable; boot via
memcard_init_slots (a disabled slot is left absent — SIO reports no device).
- launcher: MC cards now show real filename, size, used-block count, last-
modified date, and a live 15-cell block grid; Browse picks an existing
.mcd, New formats a fresh one (overwrite-prompt save dialog). Per-card
enable + paths are committed to settings on launch.
- The block grid is injected via data-rml (the structural data-for view does
not capture inner-xml in this RmlUi build; data-rml is the robust path and
the markup is fully launcher-controlled).
- Add LauncherSystemInterface so RmlUi warnings/errors reach stdout (they
otherwise go to OutputDebugString on Windows); drop an unsupported
text-overflow RCSS declaration it surfaced.
Verified on Tomba: card1 3/15 + card2 1/15 match the on-disk directory
frames; disabling slot 2 in settings.toml leaves it not-present via mc_status.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matthew Stanley