Commit Graph

141 Commits

Author SHA1 Message Date
Matthew Stanley 45cc5c3887 psx_fiber: SIGSTKSZ compile-time fallback for glibc >= 2.34
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 05:19:17 -07:00
Matthew Stanley ca6f423edb debug: gl_coh_ring/gl_fbo_peek/gl_vram_diff commands; richer SEH dumps
New debug server commands for the GL backend: gl_fbo_peek reads a rect
of GPU-side VRAM (via the pack pass) without touching the CPU array,
gl_vram_diff reports full-VRAM FBO-vs-CPU mismatch count/bbox/samples,
and gl_coh_ring dumps the always-on coherency event ring (last n events,
optional frame_min filter).

crash_trace SEH dumps now report the faulting module + base + offset and
scan the stack for executable return addresses, so dumps stay
addr2line-able despite ASLR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 05:19:17 -07:00
Matthew Stanley b228c6270e runtime: GL present wiring + turbo-loads audio fade
Present path: 15-bit frames always blit the display rect straight from
the GL VRAM FBO (4:3 letterboxed); 24-bit FMV frames sync the FBO down
and present the CPU conversion. One deterministic path per depth — the
old per-frame FBO-vs-CPU alternation (menu jitter) is structurally gone.

Audio: sdl_audio_update() ramps gain over 40 ms at turbo-loads
mute/unmute edges with an 8-frame unmute hangover to debounce burst
gaps. The fade tail is sized to the ramp (1764 frames) and clamped to
the buffer — the first version rendered 2205 frames into the 2048-frame
sdl_audio_buf and corrupted adjacent statics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 05:19:17 -07:00
Matthew Stanley 0631538407 gl-renderer v2: GPU-authoritative VRAM FBO + Beetle-style filtered sampling
Full rewrite of the GL backend around a single authoritative hr VRAM FBO
(RGBA8 + stencil mask-bit mirror, internal scale from [video]
supersampling): GPU fills/copies/draws rasterize into it, CPU uploads
batch through a pending-rect quad blit, a PACK pass re-encodes dirty
regions into the native R16UI raw mirror for texture sampling and
readback, and gpu.c routes GPUREAD/mask-check/peek reads through the
facade so the CPU array syncs before any CPU-side read.

Texture sampling adopts the Beetle-PSX model (structure only, studied
from beetle-psx-libretro):
- Per-primitive uv sampling bounds, clamped in the fragment shader when
  no texture window is active. Filtered neighbours and S>1 interpolation
  overshoot can never read outside the prim's own texture rect. Bounds
  back off the exclusive max edge for axis-aligned (2D) uv mappings and
  widen to the full page when the uv range crosses a 256 wrap boundary.
- Bilinear takes the NEAREST texel as base (cutout + STP authority),
  neighbours toward the sub-texel offset, with per-texel opacity weights
  and renormalised colour. The old "-0.5 then floor" base sampled one
  texel OUTSIDE the prim on top/left edges; where that neighbour was
  empty VRAM (e.g. tiled menu backgrounds with u0=0) the whole edge
  column/row discarded — the vertical line on Tomba's title and the
  cross seams on the memory-card screens.

The software renderer's bilinear gets the same reformulation so GL and
SW render filtered output identically; the SW nearest path (shipping
default) is untouched.

Also: always-on coherency event ring (flushes, fills, copies, draw
bboxes, packs, readbacks, presents, probe perturbations, with rects +
frame numbers) queryable via the debug server, and gl_vram_diff now
forces a full pack so it reads FBO truth even where the raw-mirror
invariant is broken.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 05:19:01 -07:00
Matthew Stanley b592447136 gl-renderer: present via glBlitFramebuffer; screenshot syncs FBO
The FBO-present crop shader sampled a sub-region of the 1024x512 VRAM
texture with vertex-interpolated UVs; sub-texel precision over the
fullscreen triangle produced a vertical seam and text smearing on screen
(the GPU render itself was correct — forcing the software present path
showed a clean image, which localized the bug to the present sampling).

Replace the crop shader with glBlitFramebuffer: an exact integer-rect
copy of the display region from the FBO to the window, with the vertical
flip folded into the dst rect. No per-fragment UV math, no sub-texel
error. linear=GL_LINEAR (AA) else GL_NEAREST.

Also:
- screenshot now calls gl_renderer_sync_cpu() first, so captures reflect
  the on-screen frame under the FBO-present path (was reading stale CPU
  VRAM). Safe: the debug server is pumped on the main GL-context thread.
- PSX_GL_FORCE_CPU_PRESENT=1 forces the software readout present path —
  a fallback/diagnostic that also keeps CPU VRAM current every frame.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:49:31 -07:00
Matthew Stanley af49fb02d6 gl-renderer: GPU rects/fills/lines + present straight from the FBO
Move the remaining 2D primitives (flat/textured rects, fill, lines) onto
the GPU, and present directly from the FBO color texture cropped to the
display region instead of reading the whole 1 MB FBO back to CPU VRAM
every frame.

Why both at once: the native present read the `vram` array directly and
relied on an incidental software op each frame (the HUD/sprites, drawn
last) to trigger the FBO->CPU sync. Moving every primitive to the GPU
removed that trigger, so a fully-GPU frame left `vram` stale -> black
present (the earlier FMV->black regression). The fix is to present from
the FBO when the GPU holds the freshest frame (gl_renderer_have_gpu_frame)
and to sync explicitly (gl_renderer_sync_cpu) on the software/24-bit path.

Result: the per-frame readback is gone for GPU scenes (the forest-lag
source), software/FMV frames keep the CPU present path (verified: the
MDEC intro renders correctly), and loads are faster.

KNOWN-FOLLOWUP cosmetic artifacts to clean up: the GPU textured shader
doesn't yet implement the texture window (wrap/mask), so tiled/wrapped
sprites show seams; bilinear texture_filtering isn't honored on the GPU
path; and the debug screenshot still reads CPU VRAM (stale under the
FBO-present path) so captures can lag the on-screen image.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:39:51 -07:00
Matthew Stanley 846e402b98 gl-renderer: fix double-applied draw offset dropping terrain layers
gpu.c applies draw_offset_x/y to every primitive's coordinates before
calling the gr_* facade (see gp0_exec_* — triangles, rects, sprites,
lines all pre-offset), and the software raster consumes them directly
(g_offset_x is never read during rasterization). The GPU triangle paths
were adding s_off again, double-applying the offset.

At the logos the offset is 0 so it was invisible, but in the scrolling
overworld it displaced every GPU-rendered triangle by 2x the scroll,
pushing the background/foreground terrain quads off-screen while
sprites (drawn as rects -> software, single offset) stayed put. Net
effect: "the entire overworld is gone" — only sprites survived.

Drop the s_off addition in gpu_triangle/gpu_textured_triangle. Verified:
the attract-demo overworld now renders full terrain (grass, mushrooms,
cliffs, signpost, coins) instead of sprites on an empty gradient.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:17:22 -07:00
Matthew Stanley 6c30b65b3d gl-renderer phase 2b step 4: GPU semi-transparency (4 PS1 blend modes)
Route semi-transparent flat/gouraud/textured polys to the GPU instead of
falling back to software (which forced a full FBO readback per prim).
The four PS1 modes (B/2+F/2, B+F, B-F, B+F/4) map to fixed-function
blending: FUNC_ADD/REVERSE_SUBTRACT with ONE/CONSTANT_ALPHA factors, the
fragment as source and the FBO as dest; framebuffer clamp matches the PS1
saturate. glBlendEquation/glBlendColor loaded via the modern-GL loader.

Textured semi-transparency honors the per-texel STP bit via a two-pass
split (new u_semipass uniform): pass 1 lays opaque texels with blend off,
pass 2 blends only STP texels. Untextured semi prims blend every pixel in
one pass. Opaque prims are unchanged (semi = -1, blend disabled).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:06:39 -07:00
Matthew Stanley ee0a454a56 debug server: screenshot writes a real PNG instead of a .png-named BMP
handle_screenshot_file emitted a 24-bit BMP regardless of extension, so
"screenshot_file path=foo.png" produced a file that PNG-only consumers
(and the agent Read tool) rejected. Replace with a self-contained PNG
encoder: 8-bit truecolor, IDAT carried as a zlib stream of stored
(uncompressed) DEFLATE blocks with correct CRC32 + Adler32. No new link
dependency, so the static self-contained runtime stays self-contained.
Default name is now psx_screenshot.png.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:06:39 -07:00
Matthew Stanley c2b2959af4 gl-renderer phase 2b step 3: GPU rasterization of textured polys
Route opaque textured/shaded-textured triangles to the GPU via a raw
16-bit VRAM integer sampler (GL_R16UI) + a CLUT-decoding fragment shader
(4/8/15-bit depths, texel-0 discard, PS1 *2-around-0x80 modulation). The
raw VRAM texture mirrors CPU VRAM in ensure_gpu; render-to-texture within
a single GPU batch is a known later-step limitation. Semi-transparent
textured polys still fall back to software.

Two play-test fixes folded in:
- Present at native res under the GL backend (glb_set_scale/glb_scale
  return 1). GPU draws write the FBO only, not the software SSAA hi-res
  mirror g_hr, so presenting via g_hr dropped every GPU layer ("entire
  layers missing"). GPU-side hi-res scaling is a later step.
- noperspective on GEO/TEX varyings for PS1 affine (non-perspective)
  parity ("logos look horrible" -> clean).

Verified: PlayStation BIOS logo + Tomba face cutscene render correctly
via the GPU path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 14:58:05 -07:00
Matthew Stanley d7b430cf4f gl-renderer phase 2b step 2: GPU rasterization of opaque flat/gouraud polys
First real GPU rasterization. VRAM is mirrored as an RGBA8 GL texture bound to
an FBO (the render target); opaque flat and gouraud triangles are rasterized on
the GPU into that FBO via a geometry shader. Everything else (textured polys,
rects/sprites, lines, fills, semi-transparent polys, VRAM transfers/copies)
stays on the software rasterizer over CPU VRAM.

Coherency: two dirty flags keep CPU VRAM and the FBO in sync — a GPU draw
uploads CPU VRAM if the CPU side is ahead; a software op or the display readout
reads the FBO back (glReadPixels) if the GPU side is ahead. So both paths
interleave correctly in PS1 draw order. The sync is entirely inside the backend
vtable, so main.cpp/present are unchanged.

Verified on Tomba: GPU raster path enabled (FBO complete), renders correctly
and matches software A/B (Whoopee Camp logo intro), no corruption/crash.

Known/expected: the per-op full-VRAM sync is simple, not fast — with most
drawing still software it thrashes and runs slow. That disappears as textured
polys move to the GPU (next steps): textured-poly CLUT sampling shader, then
semi-transparency/dither/mask, then internal-res scaling, then drop the
readback and keep VRAM GPU-side.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 12:08:16 -07:00
Matthew Stanley b509e8ffa7 gl-renderer phase 2b step 1: modern-GL loader + shader present pipeline
Replaces the legacy immediate-mode present quad with a modern-GL pipeline:
SDL_GL_GetProcAddress loader for the GL 2.0+/3.0 entry points, a compiled
vertex+fragment present program, and a fullscreen-triangle draw via an empty
VAO. Verified: Tomba renders correctly via the shader path (log "GL present
path = shader", GL 4.6 NVIDIA). On any loader/compile/link failure it degrades
to the immediate-mode quad, so present never black-screens.

Rasterization still delegates to software (vtable unchanged) — this isolates
and validates the shader/VAO plumbing that the GPU rasterizer builds on next:
VRAM as a GL texture + an FBO at internal resolution, primitive shaders.

Avoids GLchar/GLsizeiptr (absent in MinGW GL 1.1 headers) by spelling loaded
prototypes with char/ptrdiff_t (ABI-identical).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 11:50:22 -07:00
Matthew Stanley 25fc85b2e5 gl-renderer: show active renderer ([OpenGL]/[Software]) in the window title
So it's unambiguous at a glance which backend is live during play/testing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 11:39:18 -07:00
Matthew Stanley 1eaee11299 gl-renderer phase 2a: real GL context + VRAM-texture present
The opengl backend now creates a genuine OpenGL context and presents through
it. Verified: Tomba boots and renders gameplay correctly via GL (context
"4.6.0 NVIDIA"), pixel-identical to software.

- gpu_gl_renderer.c: gl_renderer_init_context (SDL_GL context on a
  SDL_WINDOW_OPENGL window), gl_renderer_present (upload ARGB image -> GL
  texture -> full-screen quad -> SDL_GL_SwapWindow), present_blank, shutdown.
  Legacy GL 1.x (immediate mode) so no GL loader needed yet (-lopengl32).
  Backend vtable still delegates rasterization to sw_* this phase, so the GL
  path is pixel-identical — this isolates/validates the context+present.
- main.cpp: window gains SDL_WINDOW_OPENGL when renderer=opengl; GL context
  created with graceful fallback to the SDL_Renderer path on failure; present
  + disabled-frame + cleanup branch on g_gl_active. SW path untouched.
- runtime.cmake: link opengl32 (Win) / OpenGL::GL (other).

Phase 2b (next): move rasterization onto the GPU — VRAM as a GL texture/FBO,
modern-GL loader + shaders, primitives at internal resolution (the actual
scaling-without-perf-loss win).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 11:34:46 -07:00
Matthew Stanley 72799bda93 gl-renderer phase 1: renderer facade + [video] renderer config option
Foundation for a selectable hardware (OpenGL) renderer alongside the software
rasterizer. No behavior change: software remains the default and routes through
the facade to the exact same sw_* functions.

- config: [video] renderer = "software" (default) | "opengl" (config_loader).
- gpu_render.h/.c: gr_* facade + GpuRenderBackend vtable. gpu.c (50 call sites)
  and main.cpp (present/scale/filter) now call gr_* instead of sw_* directly.
  The software backend table points straight at sw_*, so the SW path is
  byte-identical.
- gpu_gl_renderer.c: Phase-1 stub. gl_backend_get() returns NULL, so
  requesting renderer="opengl" logs a notice and falls back to software.
- main.cpp: selects the backend (gr_set_backend) before gpu_init().

Phase 2 (next) builds the real GL backend: SDL GL context, VRAM-as-texture/FBO,
shader rasterization at internal resolution, present. Phase 3: coherency,
transparency/dither/mask, then default-with-fallback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 11:14:06 -07:00
Matthew Stanley 34561b5418 video: optional bilinear texture filtering
New [video] texture_filtering = "nearest" | "bilinear" (default nearest).
Bilinear smooths textures and 2D backgrounds by sampling the 4 neighbouring
texels and blending them. Done correctly for PSX:
- blends in RGB after the CLUT lookup (never interpolates palette indices)
- transparency-aware: keeps cutouts where the nearest texel is transparent,
  and replaces transparent neighbours with the nearest colour so edges don't
  bleed to black
- neighbours wrap within the 256-texel page, so the blend stays inside one
  texture instead of bleeding across packed VRAM
Applied to all textured primitives (triangles, shaded triangles, sprites,
scaled sprites); independent of supersampling and resolution, so cost is in
texel fetches only, not pixel count. Nearest path is byte-identical when off.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 23:43:08 -07:00
Matthew Stanley 9474765acd video: fix red/blue swap in hi-res display readout
rgb555_to_argb packed the ARGB bytes as A|B|G|R instead of A|R|G|B, so the
supersampling display path (scale>1) showed red and blue swapped. The
native scale==1 path uses gpu_display_pixel_argb and was unaffected. Match
the authoritative packing exactly: A<<24 | R<<16 | G<<8 | B.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 23:28:15 -07:00
Matthew Stanley 2a0536963c video: configurable internal-resolution supersampling (SSAA) + AA
Adds a software-renderer supersampling path so PSX games can render at a
higher internal resolution with anti-aliased edges, configured per game via
a new [video] block:

  [video]
  supersampling = 1..4   # internal-res SSAA factor (1 = native, default)
  antialiasing  = true   # linear present filtering

How it works: when scale > 1, gpu_sw_renderer maintains g_hr, an S*-scaled
mirror of VRAM. Every primitive is rasterized into native VRAM (byte-
identical to the scale==1 path — VRAM stays authoritative for framebuffer
readback, render-to-texture, transfers) AND into g_hr at S* resolution;
textures are always sampled from native VRAM so texels keep native res while
geometry/shading gain resolution. Block ops (fill/copy/CPU->VRAM/pixel write)
replicate into the mirror. Display reads the mirror and the present path
downsamples to the window (true ordered-grid supersampling). 24-bit (FMV)
frames fall back to the native path. scale==1 allocates nothing and behaves
exactly as before.

To keep the mirror coherent, the two GP0 handlers that wrote vram[] directly
(fill-rect 0x02 and the A0 CPU->VRAM image transfer) now funnel through
sw_fill_rect / sw_vram_write — no VRAM write site bypasses the renderer.

main.cpp sizes the SDL texture/logical-size/staging buffer to the internal
resolution and selects linear vs nearest present filtering from the config.

Verified on Tomba at supersampling=2: boots, renders a correct frame, holds
59.96 fps (pacing-limited, render headroom), dispatch_miss_total 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 23:21:27 -07:00
Matthew Stanley 2439d4dd44 dispatch call contract: continuations only run when the guest really returns
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.

The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).

- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
  pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
  psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
  handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
  bail aborts the interp run; tail transfers into compiled code now run
  with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
  deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
  headers; anomaly guard at exception entry; TCP_COMMANDS.md docs

Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 22:30:41 -07:00
Matthew Stanley 4bdb7e23e9 freeze dumps: serialize concurrent dumps, snapshot wedge_kind, never self-suspend
The chest-freeze postmortem (2026-06-10) exposed a three-way race when
psx_fatal_halt (main thread) and the watchdog (heartbeat thread) dump
concurrently: shared static io_buf corrupted both 159MB JSONs, and the
watchdog flipping s_last_wedge_kind mid-write sent the fatal dump (which
runs ON the main thread) into the SuspendThread stack walkers -> main
thread suspended itself two instructions before its own ResumeThread,
killing TCP on the held state. Fixes: CAS gate (first dump wins),
wedge_kind snapshotted at entry, walkers refuse to suspend self.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 16:44:35 -07:00
Matthew Stanley aa536b899c debug server: total send budget + serve-stall telemetry (root cause of 'attract degradation')
The two pre-fix attract-idle 'degradations' (2026-06-10 dumps ...393 and
...707) were observer interference, not guest bugs:
- both wedge dumps show 8/8 main-thread stack samples inside WS2_32!send
  (same call site) — the main-thread-pumped server blocked sending to a
  trickle-draining client, throttling the loop to ~6 fps;
- the 'frame counter reset to 0' was a second process instance (no reset
  path exists for s_frame_count or psx_cycle_count; dump ...707 has lower
  cycles than ...393);
- the 'kernel-only dispatch loop' signature (0xB0/0x650, restore target
  0x80068024, EPC 0x80000048, dispatch_count 0) is the normal Tomba
  title-attract idle pattern — identical in a healthy 60-min post-fix soak.

Fix the loophole and make recurrence self-evident:
- send_all_blocking gains a 15 s TOTAL budget per response (the 2 s
  SO_SNDTIMEO only bounded zero-progress chunks; a slow-but-alive client
  could throttle the emulator indefinitely while heartbeating the
  starvation watchdog). Over budget -> client dropped, runtime unharmed.
- cumulative tcp_send_stall_ms + tcp_clients_dropped exported in the
  freeze heartbeat (top level + per-tick ring 'tcp_ms') and in every
  wedge/fatal dump header, so a TCP-throttled window is attributable at
  a glance. Documented in TCP_COMMANDS.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 14:12:11 -07:00
Matthew Stanley 31377b6d4a observability: full ring dumps on involuntary death + minutes-scale wedge dump windows
- crash_trace.c: SEH and signal handlers now also write the freeze-style
  full ring dump (wedge kind 4), so a host fault no longer takes the
  rings with it. freeze_heartbeat_fatal_dump gains a first-dump-wins
  guard so a post-mortem fault during halt-and-serve can't overwrite the
  original fatal dump.
- freeze_heartbeat.c: raise per-ring auto-dump caps (wtrace_all/dirty
  16K->256K, others 4-8K->64K, frame history 600->3600). The old caps
  spanned under a second of activity — too short to cross a degradation
  transition; the underlying rings hold 4M entries. Dumps grow to
  ~50-80 MB, which prior investigations handled fine.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:55:30 -07:00
Matthew Stanley 9cd98b0c0c harden: refuse corrupt DMA lengths, fatal-halt all DMA/GPU death sites
A pre-fix degraded session died via host SEH (wild write, frame 282184)
with garbage DMA ch2 regs in flight, losing every ring. Convert the
corruption escape paths into held, fully-queryable states:

- dma.c: validate BCR-derived transfer length on every kick (cap 0x80000
  words = 2 MB; nothing legitimate exceeds RAM/VRAM size). Violation ->
  psx_fatal_halt with MADR/BCR/CHCR in the reason. Linked-list mode is
  exempt (BCR unused; node walker already has MAX_NODES).
- dma.c: clamp the CD overlay-capture span to host RAM end — the DMA
  word loop wraps inside the 2 MB mask but the capture memcpy/CRC took a
  flat ram+offset span and could read past the RAM array.
- dma.c, gpu.c: all six fprintf+exit(1) fatal sites (unimplemented DMA
  channel, unknown DMA reg, unimplemented/unknown GP0, unknown GP1) now
  route through psx_fatal_halt — rings stay live for post-mortem TCP.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:55:21 -07:00
Matthew Stanley 724912338c turbo_loads: mute audio while turbo is engaged
During turbo the guest runs at host speed, so SPU output rendered in
that window is time-compressed garble — queueing it is the audible
stutter players hear at every load burst. Hoist the turbo-active test
above the audio pump and skip queueing while it holds: the device
drains the last real-time samples, underruns to silence for the load,
and the SPU (no longer sampled) resumes from the same musical position
when pacing returns.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:27:18 -07:00
Matthew Stanley 2d6a75d9a2 tooling: fatal halt-and-serve + always-on GP1 display ring
- psx_fatal_halt: deliberate fatal sites (unhandled MMIO access,
  trap_crash) now write the structured crash report AND a full
  freeze-style ring dump (wedge kind "fatal"), then keep pumping the
  TCP debug server forever in debug-tools builds so every ring stays
  queryable against the exact crash state (ring-buffer-first: crashes
  are interrogated, not snapshotted). Release builds exit(1) as before.
  Re-entry guarded; heartbeat JSON gains a "fatal" field.
- gp1_dump: dedicated always-on GP1 (0x1F801814 display control) ring,
  512K entries (~15 min of gameplay) with frame-window server-side
  filtering. The general MMIO ring rolls over in under a minute of SPU/
  DMA traffic, which kept evicting boot-window display history before a
  probe could ask "who toggled the display during the logo" (bug 1).
- mmio_dump/mmio_clear documented in TCP_COMMANDS.md.

The halt-and-serve path is what made the pig-crash root cause a
one-repro diagnosis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:17:10 -07:00
Matthew Stanley c496dc1db1 fix: dirty-interp local-flow must honor the dispatch stop_addr return contract
Pig-into-pig crash (TombaRecomp issue #1, v0.1.5): when a cached-DLL
native overlay function called uncached overlay code, the dirty interp's
overlay local-flow chained the callee's jr-ra straight past the nested
dispatch's return point (the target page is always dirty for overlays),
executing the native caller's tail in the interpreter. The suspended
native host frames then resumed and double-executed those tails against
a moved guest stack, restoring garbage callee-saved registers — main
loop's s1 mode-switch constant destroyed -> zombie loop (blue screen,
music on IRQs) or wild pointer -> fatal MMIO READ16.

Thread stop_addr (psx_dispatch_call's return_addr) through
dirty_ram_dispatch; local flow exits when any transfer or straight-line
fall-through reaches it, handing control back to the dispatch loop's
existing pc==stop_addr return. Signature change is deliberate: stale
generated dispatch fails to compile, forcing regen (BIOS + Tomba
regenerated; other titles regen on their own branches).

Sibling of the Bug A call-contract fix, on the return side. Root-caused
live via always-on rings on a wedged instance; user-verified fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 12:16:53 -07:00
Matthew Stanley cd32149a08 tooling: unify dump/screenshot surface, bound TCP sends, fix beetle input + big-dump truncation
- debug_server.c: dump_ram = alias of read_ram (one request, one response,
  up to full 2 MB); screenshot = screenshot_file (24-bit BMP, optional
  "path", default psx_screenshot.bmp); vram_peek cap 64->128; read_ram hex
  encode via nibble table (was 2M snprintf calls stalling the main thread);
  send_all_blocking bounded (1 MB chunks, SO_SNDTIMEO 2s, heartbeats the
  starvation watchdog per chunk) and drops the client on send failure
  instead of letting the watchdog kill the runtime; handle_quit tags exit
  origin.
- beetle_debug_server.c: read_ram cap 64 KB->2 MB via heap+send_raw;
  dump_ram/screenshot aliases; screenshot w/h->width/height parity;
  set_input now holds until clear_input (frames param optional) matching
  psx-runtime semantics — the old default expired after 1 frame and the
  off-by-one in the getter cleared the override before it was ever read,
  so scripted beetle navigation silently did nothing; wtrace_dump response
  heap-built + send_raw (per-entry send_fmt dropped chunks on multi-MB
  responses, producing broken JSON).
- crash_trace: new psx_crash_trace_set_exit_origin(); report gains
  exit_origin (tcp_quit / sdl_window_close / unknown).
- debug_client.py: O(n) string-aware brace parser (old per-byte loop was
  O(n^2) — minutes on a 2 MB response, which starved the server's send and
  got the runtime killed); generic key=value passthrough for any server
  command without a bespoke CLI mapping.
- TCP_COMMANDS.md updated for all of the above.

Verified: 2 MB read_ram round-trip 0.75 s; scripted beetle navigation
reaches the memcard file-select; wtrace_dump returns valid JSON at 21 MB.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 07:49:35 -07:00
Matthew Stanley 035a9fa968 release hygiene: explicit two-step file pickers, prod event-ring gate, decouple capture from compile
- First-run pickers now state exactly which user-supplied file is being
  requested: an info box + step-numbered dialog title for the BIOS
  (Step 1 of 2, SCPH1001.BIN) and the game disc (Step 2 of 2, named
  game + accepted formats), each noting what it is NOT. Game name comes
  from the loaded game config.
- event_ring recording is now compiled out under PSX_NO_DEBUG_TOOLS:
  it is only readable via the TCP debug server (stripped in production)
  and is not part of the freeze dump, so recording was pure hot-path
  cost. All other rings were already no-op gated.
- Overlay autocapture (which writes the player-shareable
  overlay_captures.json) is now enabled whenever overlay_cache is on,
  no longer coupled to the dev-only autocompile command — player builds
  produce the contribution file the README promises; the background
  compile still requires a configured command and no-ops without one.

Production surface after this: no console (-mwindows), no TCP server,
no ring recording; keeps overlay cache + captures, freeze watchdog JSON,
and crash report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 20:28:10 -07:00
Matthew Stanley ffc15219fd fix: Bug B hard freeze — TOCTOU underflow in frame pacing slept ~24.7 days
Root cause (freeze dump 1781046069, addr2line-verified): the wall-clock
pacing loop in sdl_vblank_present read the performance counter twice per
iteration — once in the while-condition, once to compute remaining time.
If the frame deadline was crossed between the two reads (a preemption
window), the unsigned subtraction underflowed, *1000 overflowed, and
SDL_Delay received ~2^64/10^4 ms mod 2^32 ≈ 24.7 days at the standard
10 MHz QPC. The app parks in one SDL_Delay with every counter frozen,
wedged between 'i_stat |= VBLANK' (interrupts.c:286) and exception
delivery — which is why dumps showed i_stat=1 pinned. VBLANK was being
ACK'd every frame right up to the wedge in every dump examined; the
'VBLANK-ACK reentry storm' framing was a symptom misread.

- frame_pacing.c/h: shared race-free pacer (single counter read per
  decision, compare-before-subtract, sleep clamped to one period).
  Both call sites switched: main.cpp sdl_vblank_present and
  beetle_main.cpp. Unit test pins the underflow case.
- Vsync self-heal: the 'recovered storm' dump (1781045865) was 8/8
  main-thread samples inside wglSwapBuffers — NV driver blocking ~1.5s
  per present. After 3 presents >250ms, drop driver vsync (we pace to
  59.94 Hz ourselves). Counters surfaced in the freeze dump.
- Watchdog classifier: 'reentry_storm' threshold (5K/sec) was below the
  healthy baseline (~2K reentry blocks/frame × 30-60 fps), mislabeling
  normal play and host stalls — that label misdirected two prior
  investigations. Now normalized per frame (>20K/frame), plus a new
  slow_frames kind (<5 fps) with multi-sample stack capture.

Verified: test_frame_pacing ALL PASS; live Tomba paced at 59.962 fps
over 20s (single-process measurement).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 20:09:25 -07:00
Matthew Stanley 739994cfcd Step 2.8: variant-capture automation (detect -> capture -> compile -> rescan)
Runtime detects an uncovered overlay/kernel variant by sustained dirty-RAM
interp pressure inside the capture windows (g_dirty_window_dispatches,
sampled every 600 vblanks; fires only at a coherent non-loading moment via
cdrom_load_in_progress, 60s cooldown, 16/session cap), writes the capture
JSON, and spawns the game.toml-configured compile command
([runtime] overlay_autocompile_cmd, cmd.exe /C, cwd=project root, child
output kept in an in-memory ring — no log files). On child exit 0 the emu
thread rescans the cache (overlay_loader_rescan: idempotent dir re-scan +
checked-regions reset) so the new DLL loads on the next dispatch and the
variant goes native in-session. Dedup is tool-side (existing-CRC skip);
pressure drops to zero once code is native, so the loop self-limits.

New TCP: autocompile_status / autocompile_run / overlay_rescan.
compile_dll now uses absolute native paths and reports exit codes.

VERIFIED live (user-played route): 2 auto-triggers fired, captures written,
child ran recompiler+audit (new variants 1E7DDE2C, A62D0A7C discovered and
audit-clean), rescan picked the compiled DLL up in-session. OPEN ISSUE: gcc
exits 1 with no output inside the spawned cmd.exe child (same absolute-path
command succeeds from an interactive shell) — compile step currently
completed manually; suspect child-env difference, to be isolated next
session.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 16:16:36 -07:00
Matthew Stanley 0466f867f9 Step 4: turbo-through-loads (opt-in per game)
While the game is loading, skip wall-clock pacing in sdl_vblank_present
so the guest runs at host speed (presents 1-in-30 for visual progress).
Gated three ways: game.toml [runtime] turbo_loads (OPT-IN — streaming
titles must stay off), post-BIOS-handoff only (fntrace_is_game_started),
and cdrom_load_in_progress() — a data-sector read stream is active or a
data sector landed within the burst-gap window (bridges per-file
Setloc/seek gaps); XA streaming (FMV/CD audio) is never a load. New
turbo_loads TCP command gets/sets the enable and reports load_active +
unpaced-vblank count.

Measured (Tomba boot, vs step-3 baseline; user-confirmed in play):
intro load 1066 sectors 7.59s -> 6.20s; later loads 3.3-5.1 ms/frame
vs 16.7 paced (3-5x); 704 vblanks ran unpaced; no watchdog events.
Compression ceiling is guest speed during loads, which is currently
interpreter-bound (the 0x11xxxx loader-region family, ~12M insns) —
step 2.8 variant-capture automation multiplies turbo directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 15:46:48 -07:00
Matthew Stanley 3e326acef3 Step 3: runtime-tunable instant-CD budget + always-on load-burst ring
CDROM_INSTANT_MAX_PER_FRAME becomes a runtime variable (default 32,
clamp [1,4096], period still floored at CDROM_MIN_DELAY). Three writers
share the knob: game.toml [runtime] instant_max_per_frame, the new
cdrom_instant_rate TCP command (in-session A/B without rebuilds), and
the upcoming turbo-through-loads predicate (step 4).

New always-on CD load-burst ring (ring-buffer rule): one record per
gap-separated run of delivered data sectors — start/end frame, host
wall ms, sector count, rate+divisor in effect. cdrom_bursts TCP command
dumps newest-first, making "load duration" a measured quantity.

MEASURED CONCLUSION (Tomba boot, identical deterministic workload):
the 32/frame cap is NOT the load-time bottleneck. Intro load at rate 32
= 1066 sectors / 456 frames / 7.59s; at rate 256 = 1064 / 449 / 7.48s
(identical within noise; actual delivery 2.2-4.8 sectors/frame, never
near either cap). Tomba's loads are paced by the game's own per-sector
processing at real-time speed — wall-clock load time is a job for
step 4 turbo (run the guest above real-time while loading), not for a
bigger IRQ budget. Default stays 32. No watchdog transient at either
rate (the old frame ~184-213 reentry storm did not reproduce — likely
calmed by step 2.7's native exception tail).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 15:38:47 -07:00
Matthew Stanley 11799d4189 Step 2.7: kernel-window native coverage for install-at-runtime stubs
Extend the overlay JIT cache below the 0x98000 floor to kernel RAM
[0, 0x10000) — the BIOS part-2 image + install-at-runtime stubs
(Rule 18), dominated by the SIO data-byte slot at 0xCF0 whose
install-slot hook tail-dispatches the whole exception-handler tail
into the interpreter on EVERY exception once installed.

Window model (dirty_ram_interp.h): two capture/candidate windows —
kernel [0, 0x10000) and overlay [0x98000, RAM); main-EXE text between
them is statically recompiled, never captured. Interp local-flow gates
intentionally stay overlay-only (kernel runs per-block in exception
context; named constant documents this).

- overlay_capture: per-window dirty-run enumeration, runs clamped at
  window edges (kernel CPU-write pages and main-EXE DMA pages can be
  adjacent; never merge them).
- overlay_loader: dispatch lazy-load gate covers both windows;
  scan_cache_dir validates the <addr8>_<crc8>.dll shape explicitly
  (kernel region_start 0 is legal, zero-parse is no longer a sentinel).
- classifier (compile_overlays.py): kernel-window-only DISPATCH_ROOT
  promotion — an orphan dispatch interior no rooted walk covers has no
  host to truncate and is a real execution root by construction (the
  static install-slot hooks tail-dispatch into it). One orphan per
  iteration so remaining orphans alias into the new host. Overlay
  regions stay ineligible (cross-variant per-PC evidence: 0xE889C class).
- main_psx --overlay: 'dispatch_root' seed lines are trusted walk roots
  (exempt from the boundary re-check); seed range is now the loaded
  image's own bounds (the old hardcoded [0x80010000,0x80200000) window
  silently dropped kernel seeds).
- overlay_api/loader/DLL preamble: route psx_restore_state_escape
  through OverlayCallbacks (appended last, by-value copy keeps old
  DLLs compatible) — exception-context kernel code longjmp-returns.

Tomba verified (user-played: title -> LOAD list renders -> save loads
-> overworld laps): kernel interp per-PC rows 3 -> 0 (was 0xCF0 =
61,775 dispatches / 2.71M insns at the LOAD screen alone, ~4.6M
insns/session); host func [0xCF0,0xEA0) + 0xDF8/0xE08 aliases, audit
clean; dispatch_native 357,763; dispatch_miss_total 0; interp aborts 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 15:24:30 -07:00
Matthew Stanley 864dff187d runtime: per-PC external-entry attribution; tool: prior-manifest evidence merge
Two pieces from the variant-coverage investigation:

1. dirty_ram_stats per_pc rows gain an `entries` counter: dispatches
   that arrived from NATIVE code (call / dispatch loop), distinguished
   from the interpreter's own block-to-block chaining via a consumed
   chain-target latch. `hits` conflates the two — below the local-flow
   floor every taken branch re-dispatches, so chain blocks dominate and
   misattribute hot families. First use REFUTED the main-EXE-interior-
   entry hypothesis: the 0x1DFF8/0x1E0D8 family (~1M insns) has ZERO
   external entries — it is purely downstream of overlay interp runs,
   and collapsed to 57K insns once the overlay variant was compiled.
   No main-EXE aliasing needed.

2. compile_overlays.py merges prior-manifest evidence on rebuild: every
   F entry in an existing .ranges for the SAME image bytes was proven
   compilable, so a fresh (poorer) capture cannot regress coverage.
   Callable entries re-enter as captured function entries; non-callable
   ones (alias wrappers from a prior build) re-enter as dispatch entries
   so the classifier re-derives their interior disposition — feeding
   them back as walk roots would truncate their hosts.

Village variant DLL (000E7000_9707AAB5) force-rebuilt with merged
evidence: 195 -> 244 functions incl. interior aliases. Verified on a
user-played village+overworld lap: 0x12A60C (~1M interp insns),
0x138084, 0xED2F4 now native and absent from per_pc; 0 dispatch misses,
0 aborts, 0 invalidations.

Remaining interp load is (a) the variant long tail — the same overlay
region holds scene-dependent variants where one variant's code address
is another variant's DATA (0x800E889C: code in the overworld-family
image, a pointer table in the village image); per-entry hashing
correctly refuses stale native code, and each variant gets covered when
captured — and (b) the kernel-RAM SIO stub at 0xCF0 (~4.6M insns/
session), a separate class.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 14:42:46 -07:00
Matthew Stanley 6c7bb649e3 runtime: dirty_block_dump_file — file dump for the block ring
Inline multi-MB dumps wedge the single-threaded debug server (third
occurrence: dirty_block_log count=16384). Block ring joins the insn ring
and event ring in having a file-dump path; client plumbing included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 13:04:11 -07:00
Matthew Stanley 26d03e643a runtime: fix Bug A — interp jal/jalr must honor the call contract for native overlay callees
Root cause of the dwarf-village->overworld native-overlay blue screen
(deterministic, every run, at native call #70):

The dirty-RAM interpreter's jal/jalr handlers preserved the call's
return-address obligation for statically-compiled callees
(psx_dispatch_game_compiled -> *next_pc_out = return_pc) and for non-local
targets (dispatch_nonlocal_call passes return_pc as stop_addr), but a LOCAL
DIRTY target fell through to a bare `cpu->pc = target; return 1`. When that
target was a NATIVE overlay candidate, the dispatch loop ran it as a C
function; its pc==0 return made psx_dispatch_impl unwind to the C caller,
silently dropping the suspended interpreted caller's continuation.

Concrete case: main-EXE field dispatcher F=0x800338A8 (undiscovered entry —
discovery walked it from 0x338B0; reached via data-table jalr, so it runs
under the interpreter) does `jal 0x8011B0D4` per entity. Under native exec
F's epilogue never ran: guest sp leaked 0x18 per entity until the world
corrupted. Interp-only mode is immune (the callee's own jr ra is interpreted).
Evidence chain (full-register fingerprint diff ON vs OFF localized first
divergence to sp at exec #70; frozen dirty-block ring shows F re-entered at
sp-0x18 with no epilogue between) is in docs §9 and the session memory.

Fix: overlay_loader_call_native() — run a validated native candidate as a
unit and resume at return_pc — called from both jal and jalr handlers,
mirroring the psx_dispatch_game_compiled contract exactly. Side effect:
native overlay coverage rose from 86 calls per village->overworld run to
~112,000 (interpreted calls now route to native candidates).

Verified: dwarf->overworld renders and plays under native overlays.
Same-scene A/B: host CPU 7.31 (native) vs 7.83 (interp) CPU-s per 15s wall.
Timing hypotheses refuted by experiment before fixing: IRQ-check cadence
(suppress armed, identical failure) and device-event granularity
(conservative stepping, identical failure).

Also in this commit (the capture tooling that found it):
- fingerprint ring now stores full entry/exit register files + guest cycle
  (FP_CAP 2^20 crc-only -> 2^16 full-reg); overlay_fp_dump takes a path.
- insn_freeze <addr> <nth>: one-shot latch freezing insn/block/flow/fntrace
  rings immediately BEFORE the Nth dispatch of a watched candidate, so the
  pre-divergence window survives until dumped (ring-first, no arm-then-hope).
- dirty_insn_gate <lo> <hi>: runtime-settable extra PC range for the per-insn
  interp log; native entry/exit markers (transferred=200/201) in that ring.
- dirty_insn_dump_file: file dump for the insn ring — the 16MB inline TCP
  dump wedged the single-threaded debug server.
- freeze gating in the flow-log path gates ONLY the ring write, never flow.
- debug_client plumbing for the new commands + fntrace_arm/fntrace_dump.
- docs/overlay-recompilation-design.md §9: the dispatch call-contract
  invariant, the Bug A case study, verification numbers, and known residue
  (data-jump-table discovery-miss class; one unreproduced black-screen
  one-off; §8.3 tail-position caveat).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 12:25:03 -07:00
Matthew Stanley 5041712a05 runtime: phase-5 checkpoint — event-timeline ring, instant-disc rate fix, overlay diagnostics
Accumulated working-tree state for the overlay-JIT-cache phase, committed as a
checkpoint before the Bug A fix (next commit):

- event_ring.{c,h}: always-on unified event-timeline ring (IRQ deliver/gate/
  raise, I_STAT changes, DMA kick/sched/done, ENQ/DEQ per source, exec-mode
  tagged). Edge-recorded, eviction-bounded, survives a blue screen; dump via
  event_ring_dump/_tail/_clear.
- cdrom.c: disc_speed="instant" no longer collapses every sector read to the
  500cy floor (which flooded ~1100 DMA3 completions/frame and starved main
  code). CDROM_INSTANT_PERIOD bounds sector IRQs to CDROM_INSTANT_MAX_PER_FRAME
  (32)/frame. Known caveat: a transient ~1-2s reentry storm at boot frame ~210
  still trips the watchdog once, then recovers.
- interrupts.c: claimed-IRQ cooldown is now per-source — SIO stays 0 (card
  reads need back-to-back IRQs); DMA/VBLANK/timer get CLAIMED_PROGRESS_QUANTUM
  (8) blocks of guaranteed main-code progress between deliveries. Root cause of
  the instant-disc freeze: cooldown=0 let the block leader re-take the
  exception before its body ran (pinned at the 0x8005A1D8 field loop).
- gpu/dma/memory/sio/timers/psx_cycles: event-ring instrumentation at raise/
  kick/complete/schedule sites; g_event_step_conservative (single-cycle event
  stepping A/B used to refute the event-granularity hypothesis for Bug A).
- overlay_capture.c, overlay_loader.h, crc32: capture/manifest plumbing for
  per-entry validity (Inc3).
- recompiler: overlay-mode emission updates (code_generator, main_psx).
- tools: compile_overlays.py audit/patch updates; tomba_loadtest.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 12:24:40 -07:00
Matthew Stanley 1804859983 overlay: reload-on-return diagnostics — proves segmentation is the prerequisite
Reload-on-return does not fire against the current merged-capture DLL. Added
code-span watch/hash (vs whole-region) + diagnostics, and the counters pinpoint
why:

  r0_fn_lo=0x000E73C0  r0_fn_hi=0x0013B184  -> 343 KB function span
  r0_writes_since_invalid=107222            -> below span/2 threshold (171 KB)
  reval_attempts=0                          -> never triggered

The DLL was compiled from the merged 1.1 MB roaming capture, so its functions
are scattered across the whole region with mutable data interleaved. There is no
coherent reload unit to threshold or hash: the re-validate trigger never fires,
and a 343 KB hash including data would never match anyway. OV-1 (correctness) is
unaffected — invalidation still fires (no blue screen); only the native speedup
on return is blocked.

Conclusion: reload-on-return requires coherent per-overlay segmentation (split
the merged region into real load units). The reload machinery stays in as inert
scaffolding until segmentation lands. Diagnostics exposed via
overlay_loader_status (r0_*, reval_attempts, reval_crc_miss, last_reval_crc).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 18:01:20 -07:00
Matthew Stanley 3de1832350 overlay: reload-on-return (Inc2 first cut) + on-demand overlay_capture_dump
Two additions:

1. overlay_capture_dump TCP command — flush overlay_captures.json on demand,
   so a freeze/kill can't lose a capture session (no clean window-close needed).

2. Reload-on-return: when an invalidated overlay region is loaded back with the
   SAME content, re-enable its functions natively instead of staying interpreted.
   - At register, hash the live region (crc_live).
   - On invalidation, keep watching and count writes into the region (instead of
     clearing the watch).
   - When the region is substantially rewritten (>= size/2 bytes — a real reload,
     not incidental data writes), the next dispatch re-hashes once; on a match to
     crc_live, flip the region valid again. O(1) — dyntab entries never left, no
     DLL reload. Cheap-gated so steady-state churn doesn't pay per-dispatch hashes.
   - New counter: revalidations (in overlay_loader_status).

Ghidra note: searched the Tomba main EXE for the overlay load address (0x800E7000
virt / 0x000E7000 phys) — no flat overlay table; the address is computed and the
overlay is copied/decompressed to the code region by a BIOS routine. So overlay
identity must be a runtime content hash (this commit), not a static table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 17:48:39 -07:00
Matthew Stanley 478e2352c9 overlay: Inc1-D registration-lifetime write-invalidation (fixes OV-1)
A registered overlay function is only callable while the RAM it was compiled
from is unchanged. Adds coarse, region-level write-invalidation:

- overlay_loader: each loaded DLL gets a region record {base,size,generation,
  valid}; dyntab entries carry their region index. dyntab_lookup returns a
  function only if its region is still valid (else counts stale_blocked and
  falls through). Invalidation is O(1) (flip valid=0) — no open-addressing
  deletion. DLL stays loaded; only callability is revoked
  (Compiled != Registered != Callable).
- memory.c: a watched-overlay-pages bitmap, set when a region registers; one
  bitmap test added to the RAM-store branch of psx_write_{word,half,byte}
  (the single audited store chokepoint — all CPU + DMA writes funnel here).
  A write into a watched page calls overlay_loader_invalidate_at(), which
  flips the region invalid and stops watching it.
- Counters via overlay_loader_status: loads, invalidations, unregistered_funcs,
  dispatch_native, dispatch_interp_fallback, stale_blocked, last_write_{pc,
  addr,size}, regions.

Coarse by design: ANY write into a registered region invalidates the WHOLE
region (over-invalidation is safe — it falls back to the interpreter, which
self-heals). Per-function/dependency-range narrowing waits for the Inc2
manifest. After invalidation the region is not auto-reloaded this session
(re-registering a CRC-matched cache is also Inc2); the new content runs
interpreted, which is correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 15:11:06 -07:00
Matthew Stanley b2929aa504 overlay: restore overlay DLL cache + capture (faithful port of codex/cache-discovery)
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.

Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
  cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
  LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
  enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
  writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
  decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).

Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
  targets and set split_mid_function_targets=false, so overlay basic-block
  targets are NOT split into bogus standalone functions (the mid-function-seed
  failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
  mid-function split pre-pass behind split_mid_function_targets (default true,
  so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).

Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).

Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 12:32:19 -07:00
Matthew Stanley d22c9cec42 runtime: always-on A0/B0/C0 bioscall ring + psx_check_interrupts_at shim
Purely additive diagnostic instrumentation (0 deletions):
- debug_server.c: always-on BIOS-call ring (psx_bioscall_record) hooked at
  the central dispatch chokepoint + unique-call summary table.
- interrupts.{c,h}: psx_check_interrupts_at(cpu, resume_pc) compat shim that
  forwards to psx_check_interrupts, so the baseline runtime can link
  ape-flavored generated code for instrumented good-vs-bad comparison.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 11:15:01 -07:00
Matthew Stanley ec4306fbb3 mmx6 framework: CD/DMA/MDEC + codegen fixes (Tomba builds+plays, MMX6 boots to intro)
Framework work on codex/mmx6-support that makes Tomba build and play
(title screen, 0 dispatch misses) and MMX6 boot through BIOS to its
intro. Touches recompiler codegen (code_generator, function_analysis)
and runtime CD-ROM/DMA/MDEC/GPU/SIO timing (cdrom, dma, mdec, gpu,
psx_cycles, traps, fntrace, debug_server) + headers + runtime.cmake.

Milestone snapshot for collapsing onto master. Build dir excluded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 12:06:02 -07:00
Matthew Stanley 963a87a327 Replace DMA overlay hook with overlay_dump TCP command
The CD DMA hook (overlay_log.c) could not distinguish code overlays from
FMV streaming data — Tomba buffers FMV through the same physical address
range and DMA channel. Ripped it out entirely.

Replace with overlay_dump TCP command in debug_server.c: scans the
dirty_ram bitmap for contiguous dirty pages above a configurable physical
threshold (default 0x98000), writes <crc32>.bin files to a caller-supplied
directory, and returns a JSON manifest. dirty_ram IS the oracle for
"executed as code" — no false positives from data transfers.

tools/dump_overlays.py queries overlay_dump post-session and appends new
entries to logs/<game_id>/overlay_map.jsonl with (crc32, load_addr, size,
bytes_file). First run on Tomba captured 4 regions (8KB, 264KB, 356KB,
748KB) matching the known dirty_ram miss clusters.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-30 12:33:45 -07:00
Matthew Stanley 38ec2e7426 Add overlay logger (B+A Layer B-1)
Hooks CD DMA channel 3 completion to log code overlay loads to
logs/<game_id>/overlay_map.jsonl for build-time pre-compilation.

Hot path cost: two integer comparisons (address + size filters) and
return. CRC32 is NOT computed inline — too expensive for FMV-heavy
scenes that stream many small DMA transfers through channel 3.

Filters:
- phys_addr < 0x80000: skip (main EXE, kernel, data)
- size_bytes < 4096:   skip (streaming data, not code)

Deduplicates on (addr, size) in a 128-entry table; appends new entries
to an open file handle (no fopen/fclose per transfer). overlay_log_init
is called from main.cpp with the game_id and game.toml directory.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-30 11:28:00 -07:00
Matthew Stanley b29065aa56 Add dispatch hit counter and fast boot with BIOS snapshot
dispatch_stats TCP command: g_dispatch_static_hits incremented on every
binary-search hit in generated dispatch; exposed alongside miss_total and
miss_unique so static coverage ratio is measurable numerically.

fast_boot = true in [runtime]: first launch runs BIOS normally (logos
visible) and captures full system state at game handoff to
fast_boot_<bios_cksum>_<entry_pc>.bin adjacent to the BIOS. Every
subsequent launch restores the snapshot and dispatches directly to the
game entry PC — no BIOS execution, instant start.

Snapshot captures: CPU (gpr/hi/lo/cop0/gte), RAM (2 MB), scratchpad,
dirty-RAM bitmap, i_stat/i_mask, timer state. GPU/SPU/VRAM not included
(game reinitialises). Display-disabled vblanks now clear SDL to black so
the window never shows OS-background white during game crt0 setup.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-30 10:40:54 -07:00
Matthew Stanley 8b59da8202 Disc speed: post-BIOS switch via entry_pc, XA guard for FMVs
Boot stays at authentic 1x. Speed switch fires once when the game's
entry_pc is first dispatched — safely after the BIOS disc init and
Sony/PlayStation logo sequences complete. Uses entry_pc exact-match
rather than text range to avoid false-triggering on the BIOS shell
(RAM 0x30000-0x5B000 overlaps the game text range).

XA guard in apply_speed(): reverts to 1x whenever xa_stream_active,
protecting FMV playback from speed-up regardless of disc_speed setting.

CDROM_MIN_DELAY floor of 500 cycles prevents instant mode from firing
IRQs faster than the interrupt handler can process them.

Tomba game.toml: disc_speed = "4x" (instant proved too aggressive
for early game disc reads; 4x safe with XA guard protecting FMVs).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-29 23:59:35 -07:00
Matthew Stanley aef852e11b SIO alias dispatch + instant disc speed + game miss collector
SIO handler alias (0x0CF0 → func_0000641C):
  Runtime capture showed BIOS installs lui+addiu+jalr to RAM 0x641C at 0x0CF0.
  New [[recompiler.bios_aliases]] TOML mechanism emits a one-liner wrapper and
  registers it in the dispatch table. Eliminates 30K dirty_ram blocks/session.

Instant disc speed (disc_speed = "instant" in game.toml [runtime]):
  cdrom_set_speed(0) collapses all seek/read delays to 1 cycle. Correct INT
  sequence still fires; no artificial wait. "2x" and "4x" divisors available.
  Default remains authentic 1x. Tomba set to instant.

Persistent game miss collector (tools/collect_game_misses.py):
  Appends newly-seen dirty_ram dispatch misses in the game text range to a
  file (TombaRecomp/seeds/dirty_ram_misses.txt) for later seeding.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-29 23:42:16 -07:00
Matthew Stanley bd60a359fc Controller: fall back to first available, add Steam Remote Play hints
- open_configured_controller: two-pass scan — first tries the configured
  device index, then falls back to the first available game controller.
  Fixes Steam Remote Play where the virtual Xbox controller appears at an
  unpredictable SDL joystick index, causing silent failure when device=0
  in input.ini doesn't match.
- SDL_HINT_JOYSTICK_HIDAPI=1 / SDL_HINT_JOYSTICK_RAWINPUT=0 set before
  SDL_Init so Steam Input's virtual controller is enumerated via HIDAPI
  rather than the raw Windows HID path, making it visible as a proper
  SDL_GameController.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-29 21:40:36 -07:00
Matthew Stanley 5c87ee2837 Fix psx_fiber_convert_thread on Windows: drop sentinel, use ERROR_ALREADY_FIBER
The 0x1E00000000000000 TEB sentinel for "not yet a fiber" is MSVC-
internal. MinGW's GetCurrentFiber() before ConvertThreadToFiber may
return a different unspecified TEB value, causing the sentinel check
to silently skip ConvertThreadToFiber. The thread stays as a non-fiber
and the first SwitchToFiber crashes (SIGSEGV at frame 714, CD-boot).

Fix: call ConvertThreadToFiber unconditionally and detect the already-
converted case via GetLastError() == ERROR_ALREADY_FIBER, then fall
back to GetCurrentFiber(). This matches the pre-abstraction behavior
in the old traps.c which gated on s_main_fiber == NULL.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-29 21:29:40 -07:00