The ~14-min Tomba idle freeze is a gradual per-frame host-stack LEAK (not a
within-one-frame spiral, not the gradual-marking theory). Confirmed via a new
compiled-entry depth profile (§19): per-frame MAX guest-stack-used climbs dead-linear
+1.17 KB/frame from gameplay start to 48 MB on the 64 MB main-thread stack -> native
stack guard -> freeze. Once per frame an interpreted overlay (reached via
func_80046264's dirty hop jal 0x80063864) does jalr func_8001A954, nesting a host call
that never unwinds (the per-frame loop never returns). Full writeup RECURSION_BUG.md
§16-§21.
Instruments (always-on rings, runtime-only, no regen):
- §17 stack_profile: per-vblank host-stack sample (main.cpp; stack_profile TCP cmd).
- §18 xprobe / xprobe_arm: interp<->compiled boundary crossing flight recorder
(dirty_ram_interp.c; per-frame summary + per-crossing detail).
- §19 ce_profile: per-compiled-entry depth profile + raw TEB at the trip
(debug_server.c; ce_profile TCP cmd) — the trustworthy instrument; the regression
check for the fix (max_kb must stay FLAT).
- PSX_STACK_GUARD_KB early-trip + PSX_FIBER_STACK_KB (debug_server.c / traps.c).
Fix progress:
- Block-loop tail-transfer surface (dirty_ram_interp.c ~1514): surface compiled
tail-transfers to the trampoline instead of nesting. Correct for tail-transfers but
RULED OUT as the leak (§19 climb unchanged) — the leak is the CALL path (jalr/jal).
- Real fix (next): surface compiled-game CALLS to the owner trampoline, gated to
compiled-game targets only (a naive all-targets surface reintroduces the
dwarf->overworld lost-continuation bug). Needs a non-nesting membership test
(generated psx_dispatch_game_has) or §12 CD-DMA dirty-marking precision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Live-tunable selectable stretch gate to identify which drawn prims form the
native-wide 2D-backdrop void: ws_dbg_stretch modes = OT-range / clut / tpage /
tagged / untagged-tex / all-tex, with per-frame matched + matched-tagged counts
(psx_ws_dbg_gate_frame_snapshot). Routes through the existing GL centre-stretch.
Confirmed live: the void layer is the early-drawn, SPRITE-TAGGED 0x65 background
grid (6 cols x 7, step-64, cluts unique to the backdrop); stretching it about
screen-centre fills the void with no inter-tile gaps (GL scales width too).
Tagged fills it; untagged (3D) does not. The prior content-match plan is
DISPROVEN (template transformed beyond recognition before draw) -- corrected
model recorded in BACKDROP_PRELOAD.md. Fix gate chosen: native-wide + tagged +
background-phase (draw-time), since backdrop X is written by a shared sprite
helper with no clean per-emit source signal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 dwarf/flower-field void is native-wide 2D-backdrop POSITIONING, not a
column-loading problem. The earlier auto_backdrop whole-row preload is proven
IRRELEVANT (ws_backdrop_ring: whole 41-col row loads, void persists; margin off
vs whole-row pixel-identical). The flower-field is a fixed-screen 2D tile field
(static 11-bit vertices, ~4:3 width) that bypasses the GTE, so native-wide (which
widens 3D via the GTE) never widens it. Fix = geometric X-stretch about
screen-centre of the 2D backdrop in the GL wide-mirror — USER-CONFIRMED it fills
the void (with the gate disabled). Full record: BACKDROP_PRELOAD.md (2026-06-16).
Remaining: gate the stretch to ONLY the backdrop (currently also stretches Tomba/
HUD/3D). Path forward = draw-time gp0_cmd_source_addr / sprite-tag classification.
Runtime WIP checkpoint (all behind native-wide + runtime toggles, 4:3 untouched):
- gpu_gl_renderer.c: u_xscale/u_xcenter geometric stretch in the geo+tex
wide-mirror passes; per-frame backdrop-phase reset moved to gl_perf_present_enter
(both present paths); textured-only phase mode; per-prim trace + dbg counters.
- gpu.c: ws_backdrop_ring (always-on rewrite ring); ws_backdrop_margin live
column-widen (proven irrelevant); psx_ws_backdrop_value whole-row path.
- dirty_ram_interp.c: ring capture (extent/camx/count/a1/dl) at the interp hook.
- debug_server.c: ws_backdrop_ring / ws_backdrop_margin / ws_backdrop_stretch.
- main.cpp: keyboard ALWAYS drives player 1 (OR'd with any controller).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On macOS, defining _POSIX_C_SOURCE restricts the visible namespace and HIDES
BSD symbols the TCP debug server needs — notably INADDR_LOOPBACK (and
struct timeval). clang then fails with 'use of undeclared identifier
INADDR_LOOPBACK'. Use _DARWIN_C_SOURCE on Apple (which exposes the full
BSD+POSIX namespace incl. clock_gettime), keeping _POSIX_C_SOURCE for glibc.
Mirrors the same __APPLE__/_DARWIN_C_SOURCE guard already used in psx_fiber.c.
handle_synth_recurse (a command handler, compiled even in Release) called
debug_server_synth_recurse_arm, which is defined only under #ifndef
PSX_NO_DEBUG_TOOLS — so a Release build (PSX_DEBUG_TOOLS=OFF) failed to link with
an undefined reference. The self-test is debug-only and the TCP server never
starts in Release, so guard the call; the handler is dead code there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist validated sljit shards across launches (SLJIT_PERSIST_CACHE.md). On JIT,
serialize the (position-independent, Stage 1) LIR and write it to
<cache>/<game_id>/sljit/<arch-abi>/cg<N>/<entry8>_<crc8>.sljit with a small header
(magic, format/helper-order versions, entry/lo/len/crc/size). At init, scan that
dir, deserialize + sljit_generate_code for THIS process, and register each as a
dll<0 candidate carrying the blob's crc; dispatch re-hashes live RAM against it so
a reloaded shard re-earns trust exactly like a fresh JIT. Priority gcc > sljit >
interp is unchanged (a gcc DLL still supersedes a reloaded shard via the obsolete
path). Persist is write-on-generate (measured negligible; crash-safe).
Threads the serialized blob through OverlaySljitResult -> CompiledFragment;
register_sljit_candidate gains a crc_override (reloaded shards use the header crc
since the overlay RAM may not be loaded at init). sljit_status exposes
reloaded / persist / reload-diagnostic counters.
Two gotchas found + fixed in testing (vs the original design notes):
- serialize MUST precede generate_code (generate sets compiler->error =
SLJIT_ERR_COMPILED, after which serialize returns NULL);
- serialize with options=0, not SLJIT_SERIALIZE_IGNORE_DEBUG, so an
argument-checks build can deserialize its own blobs.
Validated: two-launch test reloaded 12 shards (deserfail=0), ran them at a fresh
process base with no crash, gameplay renders correctly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller (item 3): present an analog (DualShock) pad by default for
analog-capable games so the left stick gives variable run speed; fold the
physical D-pad / keyboard arrows onto the left analog axes at full deflection so
they still move the character (avoids the Special Edition full-throttle-D-pad
bug). Add a launcher-editable analog-stick deadzone option (settings.toml +
input.ini + game.toml [controller] default), modeled on snesrecomp.
Settings persistence (item 6): deterministic restore-at-init + save-on-exit for a
game's native OPTION-screen settings, declared per game in a dedicated
game_options.toml. The recompiler rewrites each setting's boot-init store (sb/sh)
to route its value through psx_game_option_store(addr,val), applying a value
persisted from a prior session over the game default (identity when none, so a
fresh install is byte-identical). The runtime saves the live config globals to
<memcard_dir>/<game_id>.options on exit (atexit). Adds a game_options debug
command. Requires a regen for the hook to emit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The native stack guard captures a report when a runaway guest recursion is
about to overflow, but the report couldn't reliably NAME the recursing function:
the SEH stack_scan (host offsets, build-specific) is absent on the graceful-halt
path, last_func_addr comes from the dispatcher (blind to direct func_X
recursion), and the existing fn-entry ring is debug-only + fntrace-gated (empty
in release).
Add an always-on (debug AND release) ring of the last 64 recompiled function
entries in GUEST addresses, plus g_psx_recursion_func = the function being
entered when the guard trips. crash_trace dumps recursion_func + recent_fn. For
a runaway the recent_fn tail repeats the recursing func, so the next ORGANIC
crash is root-causable in Ghidra with no symbol map — pair it with cpu.gpr (the
args/corrupt value at the trip).
Validated via the new `synth_recurse` debug self-test (forces a runaway): guard
halts cleanly, report shows recursion_func=0x8000DEAD + recent_fn all 0x8000DEAD.
No measurable overhead (16:9 stays 60fps); no false-trips across village play.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The watchtower force-close (and the matching GitHub issue) is a host C-stack
overflow (SEH 0xC00000FD) from runaway guest recursion — the seesaw/Bug-D
wild-call family, where a corrupt jump-table loops a handler back on itself.
An in-range guest `jal` emits a DIRECT func_X(cpu) call, so this recursion grows
the native stack WITHOUT going through psx_dispatch_impl: g_psx_dispatch_depth
(and the existing depth-counter guard in dirty_ram_dispatch) is blind to it —
the captured report shows STACK_OVERFLOW with dispatch_depth=0.
Add a path-independent guard at debug_server_log_call_entry (emitted at the top
of every recompiled function, so it covers direct-call recursion too, and runs
in release since it precedes the PSX_NO_DEBUG_TOOLS early-return). It reads the
running fiber's real stack bounds from the x64 TEB (StackBase GS:[0x08],
DeallocationStack GS:[0x1478]) and, when headroom drops below 1/4 of the stack,
halts via psx_fatal_halt — captured crash report + halt-and-serve in debug,
clean exit + report in release — BEFORE the stack overflows. Guest fibers are
1 MB (traps.c); the 1/4 headroom only trips on runaway recursion (thousands of
frames), never on legitimate gameplay depth. Verified: no false-trigger across
extended village play (frame 6117); compiles + present in both debug and release.
TENTATIVE containment + diagnostics, NOT a root fix — converts a silent
force-close into a captured, parseable report (dirty_block cycle names the
recursing PCs) so the intermittent crash can finally be root-caused from a
repro. The upstream data corruption is unaddressed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measured Tomba village = 93% textured prims, so draw batching must target the
textured path (per-prim uv-limits/texpage/clut/depth move to vertex attributes
so consecutive textured prims coalesce into one draw). Flat batching alone
would touch only 7% of prims.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds prims/frame and per_prim_us to the frame_perf aggregate. Proved Tomba's
village is draw-call bound: ~1000 prims/frame at ~14us of GPU time PER
glDrawArrays (a 1-triangle draw should be <1us) — pipeline-flush overhead from
re-binding state between every primitive. 16:9 is 2x slower purely because the
native-wide double-draw issues 2x the draw calls. Points squarely at draw
batching (group consecutive same-state prims) as the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Always-on per-frame ring in the GL renderer: two GL_TIME_ELAPSED queries
bracket each frame's scene draws vs the present clear+blit (true GPU time,
independent of CPU/GPU overlap), plus SDL perf-counter CPU wall time
(present-to-present total, present call). Exposed via the debug server
"frame_perf" command, aggregated over the recent ring and split by 4:3 vs
native-wide so a 16:9-vs-4:3 A/B reads off one query.
Built to answer where a frame goes without guessing. First use proved
Tomba's 16:9 village lag is the native-wide DOUBLE-DRAW (every primitive
re-issued into the wide FBO), NOT supersampling (SS2==SS1) and NOT the
compositor (present_gpu ~1ms): scene_gpu 16:9=28ms vs 4:3=17ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugs in the GL native-wide present (both root-caused via the new wide_shot
capture):
1. ZOOM + wide_shot crash: glb_scale() was a stub returning 1 while the GL
renderer actually renders at s_scale (e.g. 2). main.cpp:1801
`g_video_scale = gr_scale()` collapsed g_video_scale to 1 on GL, so the wide
CPU present sized buffers/dims for scale 1 (426x224) while gr_render_wide_
display produced scale-2 data (852x448) -> present uploaded it as 426-wide and
showed a magnified sub-region (the zoom); the wide_shot probe overflowed its
buffer (crash). Fixed glb_scale() to return s_scale (matches the SW backend's
sw_renderer_scale contract; both gr_scale() callers want the real scale; the
GL FBO-direct present is unaffected — it never reads gr_scale()).
2. UPSIDE-DOWN: glb_render_wide_display flipped rows on readback, but the wide
FBO stores PS1-y inverted (vram_y=0 -> NDC -1 -> FBO bottom) and glReadPixels
reads bottom-up, so the two inversions already cancel — the extra flip turned
the frame upside-down. Removed the reversal (copy straight = top-down, matching
sw_render_wide_display + the PRESENT_VS V-flip).
Adds the `wide_shot` debug command (debug_server.c): dumps the composited
native-wide present surface to a PNG (gr_render_wide_display), unlike
screenshot_file which captures only the canonical pre-compositor VRAM. This is
how both bugs were seen + verified.
Verified: wide_shot returns a right-side-up, full-frame 16:9 capture (no zoom,
no flip, no crash). Final visual sign-off pending (human).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause class: load priority is static > gcc shard > sljit shard > interp,
but it was only emergent from chain registration order (LIFO prepend), not
enforced, and a superseded sljit shard was never retired. A shipped/optimized
gcc DLL must outrank — and obsolete — a user machine's locally-JIT'd sljit shard
for the same content.
Fix (overlay_loader.c): when a gcc/DLL candidate registers (cand_register),
retire any same-PC sljit shard JIT'd from IDENTICAL bytes (same crc_code) —
blacklist it so it can never run even if the gcc candidate is later self-mod
blacklisted. Precise by content: a sljit shard for a DIFFERENT overlay variant
at the same address is distinct coverage and is kept. New counter
s_sljit_obsoleted, surfaced via overlay_loader_sljit_obsoleted().
debug_server.c: sljit_status now reports live + obsoleted (observability for
the validation playtest — measure, don't eyeball).
Runtime-only (no regen). Refs SLJIT.md §11 Phase B.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render the genuine 16:9 FOV at correct proportions instead of squash-then-stretch,
using a separate per-framebuffer wide surface so canonical PSX VRAM stays faithful
(4:3 / non-opted games byte-identical; the wide canvas only exists when opted in).
- Wide-aspect mode descriptor (off / squash / native-wide) in gpu.c; opt-in via
[video] aspect_ratio. GTE fed the 4:3 ratio (identity, no squash) in native-wide;
canonical draw_offset/draw_area left faithful (no VRAM widening, no wrap).
- Separate wide present-surfaces (gpu_sw_renderer.c) keyed by back-buffer base:
framebuffer prims mirrored at local x = vram_x - base_x + OFFSET; margins cleared
via the frame-clear mirror; present reads the displayed buffer's surface 1:1.
Eliminates the A/B bleed + flicker (and the segfault) of the shared-canvas attempt.
gr_* vtable extended; GL backend reports unsupported -> canonical present until
the GL port (task 8).
- psx_ws_x_margin native-wide-aware (reuses the existing [widescreen.cull] emit).
CONFIRMED a no-op for Tomba's edge objects: they spawn via a non-FUN_80022E44
path (the foreground classifier is already 16:9-safe). See handoff.
- Draw-census extended with per-prim screen-X extent + back-buffer base for the
native-wide void diagnosis; ws_nw TCP toggle; gpu_state ws.mode/nw_extra.
- Controller: "auto"/"gamepad"/"controller" device = first available SDL pad.
WIP checkpoint. SW compositor works: flicker fixed, faithful, terrain platform
fills the wide frame. REMAINING: discrete foreground OBJECTS (rock pillar, tree
branch, bushes) + the parallax backdrop are still culled at the 4:3 edge via a
separate generation/spawn path (wtrace the OT display-list writers next); GL
compositor port. Detail in NATIVE_WIDE_PLAN.md + the session handoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The server was pumped from the emu thread, so when the emu thread is
buried (e.g. the same-state differential's per-dispatch overhead) the
socket goes unanswered and clients time out.
- A dedicated SDL I/O thread now owns accept/recv/send. Commands still
EXECUTE on the emu thread at the debug_server_poll() safe point, via a
single in-flight request slot (the client is one-command-per-connection)
-- so there are no new state races and the ~200 send sites are unchanged:
send_line now buffers into a growable response buffer (s_in_command),
which the I/O thread sends. Async pushes (watchpoint events) have no
persistent client under this protocol and are dropped, as before.
- `ping` is answered directly on the I/O thread, lock-free, so liveness is
queryable even while the emu thread is buried or frozen.
- The I/O thread bounds its wait for a response (30s) so a wedged emu
doesn't hang the connection; the emu discards a response the I/O thread
already gave up on.
Deadlock-free: the emu thread only ever signals the I/O thread, never
waits on it. Builds clean; exercised live in the next validation playtest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MIPS->sljit emitter, paralleling dirty_ram_interp.c's exec_one but
emitting host code. First slice = single-block LEAF functions ending in
`jr $ra`; everything else aborts the whole fragment to the interpreter
(precision over recall).
- overlay_sljit.c: memory-backed GPRs, 32-bit ops (R3000A wraparound),
loads/stores via icall through the cpu read/write callbacks (same
MMIO/page-watch/dirty-tracking path as the interpreter). Opcodes:
SLL/SRL/SRA/SLLV/SRLV/SRAV, ADD(U)/SUB(U), AND/OR/XOR/NOR, SLT(U),
ADDI(U)/SLTI(U)/ANDI/ORI/XORI/LUI, LB/LH/LW/LBU/LHU/SB/SH/SW, SYNC.
- compile_fragment now returns CompiledFragment{fn,code_lo,code_len};
overlay_loader registers an sljit candidate (crc hashed from the live
bytes it JIT'd from, same live-byte validation as gcc candidates).
- Step 5 wiring is VALIDATION-GATED: JIT-on-miss fires only when
backend==sljit AND the same-state differential is armed (never in a
shadow run). Normal play stays byte-identical; live sljit execution
stays off until the differential gate passes.
- Debug: sljit_try <addr> probe; sljit_status reports shards_registered.
Verified: runtime/tests/sljit_emit_test.c (links overlay_sljit.c +
sljitLir.c only) JITs hand-built leaf functions and checks output —
26/26 pass incl. gpr0 invariant, sign/zero-ext, signed/unsigned compare,
and the decline cases. Full psx-runtime build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce one backend-agnostic code-production seam the overlay
capture/dispatch spine talks to, and make the validated gcc spawn->DLL
path its first implementation. Pure refactor; gcc stays the default and
its behavior is byte-identical.
- New code_provider.h/.c: a CodeProvider struct exposing BOTH production
lifecycles so the interface isn't gcc-shaped -- batch/async
(request/busy/poll_main, the gcc model) and sync-on-miss
(compile_fragment, the sljit model). compile_fragment is defined and
implemented per-provider but not yet called from dispatch (step 5).
gcc provider = direct pass-throughs to autocompile_* (compile_fragment
NULL: a compiler spawn is too slow for the dispatch path); sljit
provider wraps overlay_sljit_* (still declines every fragment).
- Wire up overlay_backend_resolve, which was previously dead code:
code_provider_init() resolves the backend at overlay-cache init and
caches the active provider; code_provider_active() is the accessor.
- Add [runtime] overlay_backend config (auto|gcc|sljit; empty=auto) so
resolution has a config source alongside PSX_OVERLAY_BACKEND.
- Route the 3 direct autocompile_* call sites through the active
provider: overlay_capture.c autocapture tick, main.cpp per-frame poll
+ the init call, debug_server.c autocompile_run (now reports backend).
- runtime.cmake += code_provider.c.
Builds + links clean (TombaRecomp/build-dev, MinGW gcc 15.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add sljit as a self-contained, toolchain-less codegen backend for the
overlay self-improvement loop. gcc stays the default (dev: optimized +
inspectable); sljit is the fallback / production producer. Interpreter
remains the correctness floor.
This commit lands the backend-agnostic foundation only; the MIPS->sljit
emitter is not built yet, so every fragment safely declines to the
interpreter (no change to the validated gcc path).
- Vendor sljit (BSD-2-Clause) at lib/sljit/, wired into runtime.cmake
(compile sljitLir.c + include dir; sljit auto-detects the host arch).
- overlay_sljit.{h,c}: backend-selection policy (PSX_OVERLAY_BACKEND=
auto|gcc|sljit), availability, a real codegen smoke test, and
try_compile() honoring the precision-over-recall bail-to-interp contract.
- sljit_status debug-server command (Rule-3 clean: status via getter).
- THIRD_PARTY_ATTRIBUTION.md: sljit entry.
Verified: TombaRecomp/build-dev (MinGW gcc 15.2) compiles + links
psx-runtime.exe; standalone smoke harness JITs f(a)=a+1234 and runs it
correctly (26 bytes x86-64). Design + locked tier decisions in SLJIT.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The far backdrop (ocean/cloud/distant mountain) is GTE-projected 3D drawn
by the main-EXE driver FUN_8004db3c (sole caller of the GTE emitter
FUN_80027600). The global widescreen GTE X-squash compresses it toward
centre, so its geometry falls short of the revealed 16:9 edges -> blue void
that pops as the camera moves.
Fix: bracket the backdrop driver with gte_ws_set_suppress(1)/(0) so its
draws skip the X-squash and fill the stretched frame (skybox treatment),
while the near 3D world keeps the wider FOV.
FUN_8004db3c draws a MIX (far backdrop + nearer props/scenery), so the
suppress is DEPTH-GATED in gte_rtps_internal: only vertices with projected
SZ >= s_ws_far_threshold (default 900, tunable live via the ws_far_threshold
TCP cmd) un-squash; nearer props stay squashed and aligned. Validated:
threshold 900 fills the void, keeps the foreground door + background
scenery (path/tree) aligned. Residual: the mountain grass-top straddles the
depth band (per-vertex limitation) -> flickers when the camera dips; this
is the checkpoint before exploring native-wide rendering to remove the
squash (and this whole artifact class) entirely.
Mechanism:
- recompiler: [widescreen.backdrop] unsquash_funcs emits
gte_ws_set_suppress(1) at entry + (0) before each jr-$ra return
(code_generator + config_loader + main_psx). Main-EXE -> regen-class.
- runtime: gte.cpp depth-gated squash + s_ws_far_threshold; cpu_state.h
decl; debug_server.c ws_aspect (live squash on/off) + ws_far_threshold
(set split + read SZ stats) diagnostics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New debug server commands for the GL backend: gl_fbo_peek reads a rect
of GPU-side VRAM (via the pack pass) without touching the CPU array,
gl_vram_diff reports full-VRAM FBO-vs-CPU mismatch count/bbox/samples,
and gl_coh_ring dumps the always-on coherency event ring (last n events,
optional frame_min filter).
crash_trace SEH dumps now report the faulting module + base + offset and
scan the stack for executable return addresses, so dumps stay
addr2line-able despite ASLR.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The FBO-present crop shader sampled a sub-region of the 1024x512 VRAM
texture with vertex-interpolated UVs; sub-texel precision over the
fullscreen triangle produced a vertical seam and text smearing on screen
(the GPU render itself was correct — forcing the software present path
showed a clean image, which localized the bug to the present sampling).
Replace the crop shader with glBlitFramebuffer: an exact integer-rect
copy of the display region from the FBO to the window, with the vertical
flip folded into the dst rect. No per-fragment UV math, no sub-texel
error. linear=GL_LINEAR (AA) else GL_NEAREST.
Also:
- screenshot now calls gl_renderer_sync_cpu() first, so captures reflect
the on-screen frame under the FBO-present path (was reading stale CPU
VRAM). Safe: the debug server is pumped on the main GL-context thread.
- PSX_GL_FORCE_CPU_PRESENT=1 forces the software readout present path —
a fallback/diagnostic that also keeps CPU VRAM current every frame.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
handle_screenshot_file emitted a 24-bit BMP regardless of extension, so
"screenshot_file path=foo.png" produced a file that PNG-only consumers
(and the agent Read tool) rejected. Replace with a self-contained PNG
encoder: 8-bit truecolor, IDAT carried as a zlib stream of stored
(uncompressed) DEFLATE blocks with correct CRC32 + Adler32. No new link
dependency, so the static self-contained runtime stays self-contained.
Default name is now psx_screenshot.png.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two pre-fix attract-idle 'degradations' (2026-06-10 dumps ...393 and
...707) were observer interference, not guest bugs:
- both wedge dumps show 8/8 main-thread stack samples inside WS2_32!send
(same call site) — the main-thread-pumped server blocked sending to a
trickle-draining client, throttling the loop to ~6 fps;
- the 'frame counter reset to 0' was a second process instance (no reset
path exists for s_frame_count or psx_cycle_count; dump ...707 has lower
cycles than ...393);
- the 'kernel-only dispatch loop' signature (0xB0/0x650, restore target
0x80068024, EPC 0x80000048, dispatch_count 0) is the normal Tomba
title-attract idle pattern — identical in a healthy 60-min post-fix soak.
Fix the loophole and make recurrence self-evident:
- send_all_blocking gains a 15 s TOTAL budget per response (the 2 s
SO_SNDTIMEO only bounded zero-progress chunks; a slow-but-alive client
could throttle the emulator indefinitely while heartbeating the
starvation watchdog). Over budget -> client dropped, runtime unharmed.
- cumulative tcp_send_stall_ms + tcp_clients_dropped exported in the
freeze heartbeat (top level + per-tick ring 'tcp_ms') and in every
wedge/fatal dump header, so a TCP-throttled window is attributable at
a glance. Documented in TCP_COMMANDS.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- psx_fatal_halt: deliberate fatal sites (unhandled MMIO access,
trap_crash) now write the structured crash report AND a full
freeze-style ring dump (wedge kind "fatal"), then keep pumping the
TCP debug server forever in debug-tools builds so every ring stays
queryable against the exact crash state (ring-buffer-first: crashes
are interrogated, not snapshotted). Release builds exit(1) as before.
Re-entry guarded; heartbeat JSON gains a "fatal" field.
- gp1_dump: dedicated always-on GP1 (0x1F801814 display control) ring,
512K entries (~15 min of gameplay) with frame-window server-side
filtering. The general MMIO ring rolls over in under a minute of SPU/
DMA traffic, which kept evicting boot-window display history before a
probe could ask "who toggled the display during the logo" (bug 1).
- mmio_dump/mmio_clear documented in TCP_COMMANDS.md.
The halt-and-serve path is what made the pig-crash root cause a
one-repro diagnosis.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- debug_server.c: dump_ram = alias of read_ram (one request, one response,
up to full 2 MB); screenshot = screenshot_file (24-bit BMP, optional
"path", default psx_screenshot.bmp); vram_peek cap 64->128; read_ram hex
encode via nibble table (was 2M snprintf calls stalling the main thread);
send_all_blocking bounded (1 MB chunks, SO_SNDTIMEO 2s, heartbeats the
starvation watchdog per chunk) and drops the client on send failure
instead of letting the watchdog kill the runtime; handle_quit tags exit
origin.
- beetle_debug_server.c: read_ram cap 64 KB->2 MB via heap+send_raw;
dump_ram/screenshot aliases; screenshot w/h->width/height parity;
set_input now holds until clear_input (frames param optional) matching
psx-runtime semantics — the old default expired after 1 frame and the
off-by-one in the getter cleared the override before it was ever read,
so scripted beetle navigation silently did nothing; wtrace_dump response
heap-built + send_raw (per-entry send_fmt dropped chunks on multi-MB
responses, producing broken JSON).
- crash_trace: new psx_crash_trace_set_exit_origin(); report gains
exit_origin (tcp_quit / sdl_window_close / unknown).
- debug_client.py: O(n) string-aware brace parser (old per-byte loop was
O(n^2) — minutes on a 2 MB response, which starved the server's send and
got the runtime killed); generic key=value passthrough for any server
command without a bespoke CLI mapping.
- TCP_COMMANDS.md updated for all of the above.
Verified: 2 MB read_ram round-trip 0.75 s; scripted beetle navigation
reaches the memcard file-select; wtrace_dump returns valid JSON at 21 MB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runtime detects an uncovered overlay/kernel variant by sustained dirty-RAM
interp pressure inside the capture windows (g_dirty_window_dispatches,
sampled every 600 vblanks; fires only at a coherent non-loading moment via
cdrom_load_in_progress, 60s cooldown, 16/session cap), writes the capture
JSON, and spawns the game.toml-configured compile command
([runtime] overlay_autocompile_cmd, cmd.exe /C, cwd=project root, child
output kept in an in-memory ring — no log files). On child exit 0 the emu
thread rescans the cache (overlay_loader_rescan: idempotent dir re-scan +
checked-regions reset) so the new DLL loads on the next dispatch and the
variant goes native in-session. Dedup is tool-side (existing-CRC skip);
pressure drops to zero once code is native, so the loop self-limits.
New TCP: autocompile_status / autocompile_run / overlay_rescan.
compile_dll now uses absolute native paths and reports exit codes.
VERIFIED live (user-played route): 2 auto-triggers fired, captures written,
child ran recompiler+audit (new variants 1E7DDE2C, A62D0A7C discovered and
audit-clean), rescan picked the compiled DLL up in-session. OPEN ISSUE: gcc
exits 1 with no output inside the spawned cmd.exe child (same absolute-path
command succeeds from an interactive shell) — compile step currently
completed manually; suspect child-env difference, to be isolated next
session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
While the game is loading, skip wall-clock pacing in sdl_vblank_present
so the guest runs at host speed (presents 1-in-30 for visual progress).
Gated three ways: game.toml [runtime] turbo_loads (OPT-IN — streaming
titles must stay off), post-BIOS-handoff only (fntrace_is_game_started),
and cdrom_load_in_progress() — a data-sector read stream is active or a
data sector landed within the burst-gap window (bridges per-file
Setloc/seek gaps); XA streaming (FMV/CD audio) is never a load. New
turbo_loads TCP command gets/sets the enable and reports load_active +
unpaced-vblank count.
Measured (Tomba boot, vs step-3 baseline; user-confirmed in play):
intro load 1066 sectors 7.59s -> 6.20s; later loads 3.3-5.1 ms/frame
vs 16.7 paced (3-5x); 704 vblanks ran unpaced; no watchdog events.
Compression ceiling is guest speed during loads, which is currently
interpreter-bound (the 0x11xxxx loader-region family, ~12M insns) —
step 2.8 variant-capture automation multiplies turbo directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CDROM_INSTANT_MAX_PER_FRAME becomes a runtime variable (default 32,
clamp [1,4096], period still floored at CDROM_MIN_DELAY). Three writers
share the knob: game.toml [runtime] instant_max_per_frame, the new
cdrom_instant_rate TCP command (in-session A/B without rebuilds), and
the upcoming turbo-through-loads predicate (step 4).
New always-on CD load-burst ring (ring-buffer rule): one record per
gap-separated run of delivered data sectors — start/end frame, host
wall ms, sector count, rate+divisor in effect. cdrom_bursts TCP command
dumps newest-first, making "load duration" a measured quantity.
MEASURED CONCLUSION (Tomba boot, identical deterministic workload):
the 32/frame cap is NOT the load-time bottleneck. Intro load at rate 32
= 1066 sectors / 456 frames / 7.59s; at rate 256 = 1064 / 449 / 7.48s
(identical within noise; actual delivery 2.2-4.8 sectors/frame, never
near either cap). Tomba's loads are paced by the game's own per-sector
processing at real-time speed — wall-clock load time is a job for
step 4 turbo (run the guest above real-time while loading), not for a
bigger IRQ budget. Default stays 32. No watchdog transient at either
rate (the old frame ~184-213 reentry storm did not reproduce — likely
calmed by step 2.7's native exception tail).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two pieces from the variant-coverage investigation:
1. dirty_ram_stats per_pc rows gain an `entries` counter: dispatches
that arrived from NATIVE code (call / dispatch loop), distinguished
from the interpreter's own block-to-block chaining via a consumed
chain-target latch. `hits` conflates the two — below the local-flow
floor every taken branch re-dispatches, so chain blocks dominate and
misattribute hot families. First use REFUTED the main-EXE-interior-
entry hypothesis: the 0x1DFF8/0x1E0D8 family (~1M insns) has ZERO
external entries — it is purely downstream of overlay interp runs,
and collapsed to 57K insns once the overlay variant was compiled.
No main-EXE aliasing needed.
2. compile_overlays.py merges prior-manifest evidence on rebuild: every
F entry in an existing .ranges for the SAME image bytes was proven
compilable, so a fresh (poorer) capture cannot regress coverage.
Callable entries re-enter as captured function entries; non-callable
ones (alias wrappers from a prior build) re-enter as dispatch entries
so the classifier re-derives their interior disposition — feeding
them back as walk roots would truncate their hosts.
Village variant DLL (000E7000_9707AAB5) force-rebuilt with merged
evidence: 195 -> 244 functions incl. interior aliases. Verified on a
user-played village+overworld lap: 0x12A60C (~1M interp insns),
0x138084, 0xED2F4 now native and absent from per_pc; 0 dispatch misses,
0 aborts, 0 invalidations.
Remaining interp load is (a) the variant long tail — the same overlay
region holds scene-dependent variants where one variant's code address
is another variant's DATA (0x800E889C: code in the overworld-family
image, a pointer table in the village image); per-entry hashing
correctly refuses stale native code, and each variant gets covered when
captured — and (b) the kernel-RAM SIO stub at 0xCF0 (~4.6M insns/
session), a separate class.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inline multi-MB dumps wedge the single-threaded debug server (third
occurrence: dirty_block_log count=16384). Block ring joins the insn ring
and event ring in having a file-dump path; client plumbing included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the dwarf-village->overworld native-overlay blue screen
(deterministic, every run, at native call #70):
The dirty-RAM interpreter's jal/jalr handlers preserved the call's
return-address obligation for statically-compiled callees
(psx_dispatch_game_compiled -> *next_pc_out = return_pc) and for non-local
targets (dispatch_nonlocal_call passes return_pc as stop_addr), but a LOCAL
DIRTY target fell through to a bare `cpu->pc = target; return 1`. When that
target was a NATIVE overlay candidate, the dispatch loop ran it as a C
function; its pc==0 return made psx_dispatch_impl unwind to the C caller,
silently dropping the suspended interpreted caller's continuation.
Concrete case: main-EXE field dispatcher F=0x800338A8 (undiscovered entry —
discovery walked it from 0x338B0; reached via data-table jalr, so it runs
under the interpreter) does `jal 0x8011B0D4` per entity. Under native exec
F's epilogue never ran: guest sp leaked 0x18 per entity until the world
corrupted. Interp-only mode is immune (the callee's own jr ra is interpreted).
Evidence chain (full-register fingerprint diff ON vs OFF localized first
divergence to sp at exec #70; frozen dirty-block ring shows F re-entered at
sp-0x18 with no epilogue between) is in docs §9 and the session memory.
Fix: overlay_loader_call_native() — run a validated native candidate as a
unit and resume at return_pc — called from both jal and jalr handlers,
mirroring the psx_dispatch_game_compiled contract exactly. Side effect:
native overlay coverage rose from 86 calls per village->overworld run to
~112,000 (interpreted calls now route to native candidates).
Verified: dwarf->overworld renders and plays under native overlays.
Same-scene A/B: host CPU 7.31 (native) vs 7.83 (interp) CPU-s per 15s wall.
Timing hypotheses refuted by experiment before fixing: IRQ-check cadence
(suppress armed, identical failure) and device-event granularity
(conservative stepping, identical failure).
Also in this commit (the capture tooling that found it):
- fingerprint ring now stores full entry/exit register files + guest cycle
(FP_CAP 2^20 crc-only -> 2^16 full-reg); overlay_fp_dump takes a path.
- insn_freeze <addr> <nth>: one-shot latch freezing insn/block/flow/fntrace
rings immediately BEFORE the Nth dispatch of a watched candidate, so the
pre-divergence window survives until dumped (ring-first, no arm-then-hope).
- dirty_insn_gate <lo> <hi>: runtime-settable extra PC range for the per-insn
interp log; native entry/exit markers (transferred=200/201) in that ring.
- dirty_insn_dump_file: file dump for the insn ring — the 16MB inline TCP
dump wedged the single-threaded debug server.
- freeze gating in the flow-log path gates ONLY the ring write, never flow.
- debug_client plumbing for the new commands + fntrace_arm/fntrace_dump.
- docs/overlay-recompilation-design.md §9: the dispatch call-contract
invariant, the Bug A case study, verification numbers, and known residue
(data-jump-table discovery-miss class; one unreproduced black-screen
one-off; §8.3 tail-position caveat).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reload-on-return does not fire against the current merged-capture DLL. Added
code-span watch/hash (vs whole-region) + diagnostics, and the counters pinpoint
why:
r0_fn_lo=0x000E73C0 r0_fn_hi=0x0013B184 -> 343 KB function span
r0_writes_since_invalid=107222 -> below span/2 threshold (171 KB)
reval_attempts=0 -> never triggered
The DLL was compiled from the merged 1.1 MB roaming capture, so its functions
are scattered across the whole region with mutable data interleaved. There is no
coherent reload unit to threshold or hash: the re-validate trigger never fires,
and a 343 KB hash including data would never match anyway. OV-1 (correctness) is
unaffected — invalidation still fires (no blue screen); only the native speedup
on return is blocked.
Conclusion: reload-on-return requires coherent per-overlay segmentation (split
the merged region into real load units). The reload machinery stays in as inert
scaffolding until segmentation lands. Diagnostics exposed via
overlay_loader_status (r0_*, reval_attempts, reval_crc_miss, last_reval_crc).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two additions:
1. overlay_capture_dump TCP command — flush overlay_captures.json on demand,
so a freeze/kill can't lose a capture session (no clean window-close needed).
2. Reload-on-return: when an invalidated overlay region is loaded back with the
SAME content, re-enable its functions natively instead of staying interpreted.
- At register, hash the live region (crc_live).
- On invalidation, keep watching and count writes into the region (instead of
clearing the watch).
- When the region is substantially rewritten (>= size/2 bytes — a real reload,
not incidental data writes), the next dispatch re-hashes once; on a match to
crc_live, flip the region valid again. O(1) — dyntab entries never left, no
DLL reload. Cheap-gated so steady-state churn doesn't pay per-dispatch hashes.
- New counter: revalidations (in overlay_loader_status).
Ghidra note: searched the Tomba main EXE for the overlay load address (0x800E7000
virt / 0x000E7000 phys) — no flat overlay table; the address is computed and the
overlay is copied/decompressed to the code region by a BIOS routine. So overlay
identity must be a runtime content hash (this commit), not a static table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A registered overlay function is only callable while the RAM it was compiled
from is unchanged. Adds coarse, region-level write-invalidation:
- overlay_loader: each loaded DLL gets a region record {base,size,generation,
valid}; dyntab entries carry their region index. dyntab_lookup returns a
function only if its region is still valid (else counts stale_blocked and
falls through). Invalidation is O(1) (flip valid=0) — no open-addressing
deletion. DLL stays loaded; only callability is revoked
(Compiled != Registered != Callable).
- memory.c: a watched-overlay-pages bitmap, set when a region registers; one
bitmap test added to the RAM-store branch of psx_write_{word,half,byte}
(the single audited store chokepoint — all CPU + DMA writes funnel here).
A write into a watched page calls overlay_loader_invalidate_at(), which
flips the region invalid and stops watching it.
- Counters via overlay_loader_status: loads, invalidations, unregistered_funcs,
dispatch_native, dispatch_interp_fallback, stale_blocked, last_write_{pc,
addr,size}, regions.
Coarse by design: ANY write into a registered region invalidates the WHOLE
region (over-invalidation is safe — it falls back to the interpreter, which
self-heals). Per-function/dependency-range narrowing waits for the Inc2
manifest. After invalidation the region is not auto-reloaded this session
(re-registering a CRC-matched cache is also Inc2); the new content runs
interpreted, which is correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.
Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).
Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
targets and set split_mid_function_targets=false, so overlay basic-block
targets are NOT split into bogus standalone functions (the mid-function-seed
failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
mid-function split pre-pass behind split_mid_function_targets (default true,
so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).
Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).
Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Purely additive diagnostic instrumentation (0 deletions):
- debug_server.c: always-on BIOS-call ring (psx_bioscall_record) hooked at
the central dispatch chokepoint + unique-call summary table.
- interrupts.{c,h}: psx_check_interrupts_at(cpu, resume_pc) compat shim that
forwards to psx_check_interrupts, so the baseline runtime can link
ape-flavored generated code for instrumented good-vs-bad comparison.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Framework work on codex/mmx6-support that makes Tomba build and play
(title screen, 0 dispatch misses) and MMX6 boot through BIOS to its
intro. Touches recompiler codegen (code_generator, function_analysis)
and runtime CD-ROM/DMA/MDEC/GPU/SIO timing (cdrom, dma, mdec, gpu,
psx_cycles, traps, fntrace, debug_server) + headers + runtime.cmake.
Milestone snapshot for collapsing onto master. Build dir excluded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The CD DMA hook (overlay_log.c) could not distinguish code overlays from
FMV streaming data — Tomba buffers FMV through the same physical address
range and DMA channel. Ripped it out entirely.
Replace with overlay_dump TCP command in debug_server.c: scans the
dirty_ram bitmap for contiguous dirty pages above a configurable physical
threshold (default 0x98000), writes <crc32>.bin files to a caller-supplied
directory, and returns a JSON manifest. dirty_ram IS the oracle for
"executed as code" — no false positives from data transfers.
tools/dump_overlays.py queries overlay_dump post-session and appends new
entries to logs/<game_id>/overlay_map.jsonl with (crc32, load_addr, size,
bytes_file). First run on Tomba captured 4 regions (8KB, 264KB, 356KB,
748KB) matching the known dirty_ram miss clusters.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
dispatch_stats TCP command: g_dispatch_static_hits incremented on every
binary-search hit in generated dispatch; exposed alongside miss_total and
miss_unique so static coverage ratio is measurable numerically.
fast_boot = true in [runtime]: first launch runs BIOS normally (logos
visible) and captures full system state at game handoff to
fast_boot_<bios_cksum>_<entry_pc>.bin adjacent to the BIOS. Every
subsequent launch restores the snapshot and dispatches directly to the
game entry PC — no BIOS execution, instant start.
Snapshot captures: CPU (gpr/hi/lo/cop0/gte), RAM (2 MB), scratchpad,
dirty-RAM bitmap, i_stat/i_mask, timer state. GPU/SPU/VRAM not included
(game reinitialises). Display-disabled vblanks now clear SDL to black so
the window never shows OS-background white during game crt0 setup.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two distinct bugs were causing OPTIONS-black hard freezes after 12K-65K
frames:
1. interrupts.c + traps.c: exception_jmpbuf is a single global. setjmp
captures the running fiber, but longjmp may be requested from a
different fiber via psx_exception_longjmp / psx_restore_state_escape
during chain-handler dispatch. Cross-fiber longjmp restores RSP to
the wrong stack while Win32 still tracks the calling fiber, corrupting
fiber state until SwitchToFiber eventually hangs.
Fix: track g_exception_owner_fiber at setjmp time. New
deferred_exception_longjmp checks the current fiber; if it's not the
owner, sets a pending flag and SwitchToFiber's back to the owner.
psx_change_thread_fiber's post-SwitchToFiber path observes the flag
and executes the longjmp on the correct stack.
2. gpu.c: gpu_read_gpustat fired gpu_vblank_tick() after 1000 polls,
which invoked sdl_vblank_present (SDL_Delay -> Sleep) from inside
an MMIO read code path. That meant recompiled MIPS spent wall time
in Sleep deep inside the fiber call tree -- confusing to reason
about and a contributor to fiber/longjmp interactions.
Fix: synthetic VBLANK from a poll-storm only updates emulation state
(LCF + I_STAT). The host-side render+sleep happens only at the proper
cycle-paced VBLANK trigger in psx_check_interrupts.
Also adds: dirty_block ring slice to the freeze auto-dump, Win32 main-
thread stack capture for wedge_kind=1 hard freezes (SuspendThread +
StackWalk64 + SymFromAddr), and dbghelp link.
Validated: TombaRecomp psx-runtime ran 30+ minutes / 105K+ frames at
OPTIONS with no hard freeze. Previous freezes consistently hit at
11900, 20493, and 64596 frames.
Diagnostic tooling under tools/_options_session/ that produced this
diagnosis is included.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two observability additions, both gated by the new
-DPSX_DEBUG_TOOLS=OFF flag for production builds.
1. Freeze heartbeat thread (`runtime/src/freeze_heartbeat.c`):
Background thread spawned at startup that snapshots runtime
state (frame_count, psx_cycle_count, in_exception, exception
counters, sio/mc state, i_stat/i_mask, dirty_ram counters) to
`psx_freeze_heartbeat.json` every 100 ms. Survives main-thread
freezes — writer is independent of the SDL loop and the TCP
debug server. Atomic file replace via .tmp + MoveFileEx so a
reader never catches a half-written JSON.
Already paid off: captured a freeze pattern (frame_count stuck,
psx_cycle_count stuck, in_exception=0, current_func in
install-stub kernel code) that the TCP server couldn't show
because its accept queue was wedged by the same stalled main
thread.
2. PSX_DEBUG_TOOLS CMake flag (`runtime/runtime.cmake`):
- Default ON: builds with the TCP debug server, heartbeat
thread, and per-block recording (debug_server_log_call_entry
fires for every recompiled function entry).
- OFF (`cmake -DPSX_DEBUG_TOOLS=OFF`): production build. The
hot-path log functions early-return; TCP server doesn't
start; heartbeat doesn't spawn. The ring buffer arrays still
exist in the binary (their definitions live in
debug_server.c) but no writes happen so the demand-paged
memory stays committed at a much lower level (~77 MB at
idle vs ~245 MB in debug mode).
The flag propagates to psx-beetle via the option declared at
the top of runtime.cmake.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure observability change — not a fix claim.
Both psx-runtime (was 1) and psx-beetle (was 4) had small accept-
queue backlogs. When the SDL main thread stalls (the exact state
we want to probe during a freeze investigation), the kernel
queues the first pending connect and rejects subsequent attempts
with RST. From the client side this looks like ConnectionRefused,
making it impossible to diagnose the stalled process from another
shell.
Backlog 16 lets a handful of probes queue up. Main thread will
eventually accept them in order if it ever resumes. If the
main thread is permanently dead the probes still fail, but at
least the failure mode changes from "looks like the server is
gone" to "the queue is genuinely deep + still stalled."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NES recomp template PRINCIPLES.md §13a calls a dispatch miss a
"SILENT GAME-BREAKING BUG" and mandates checking
`dispatch_misses.log` after every game run. We don't write that log
(no log files per CLAUDE.md §3) — we have an in-memory ring +
`unknown_dispatch_log` TCP command instead. But nothing routinely
surfaces the count, so silent misses can accumulate across sessions
unnoticed.
Now `ping` includes `dispatch_miss_total` and `dispatch_miss_unique`
fields. Every routine health check reveals if discovery has gaps.
0 = healthy. Non-zero on any session = stop and add the addresses
to game.toml / regen before continuing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matthew Stanley