Measured Tomba village = 93% textured prims, so draw batching must target the
textured path (per-prim uv-limits/texpage/clut/depth move to vertex attributes
so consecutive textured prims coalesce into one draw). Flat batching alone
would touch only 7% of prims.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds prims/frame and per_prim_us to the frame_perf aggregate. Proved Tomba's
village is draw-call bound: ~1000 prims/frame at ~14us of GPU time PER
glDrawArrays (a 1-triangle draw should be <1us) — pipeline-flush overhead from
re-binding state between every primitive. 16:9 is 2x slower purely because the
native-wide double-draw issues 2x the draw calls. Points squarely at draw
batching (group consecutive same-state prims) as the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Always-on per-frame ring in the GL renderer: two GL_TIME_ELAPSED queries
bracket each frame's scene draws vs the present clear+blit (true GPU time,
independent of CPU/GPU overlap), plus SDL perf-counter CPU wall time
(present-to-present total, present call). Exposed via the debug server
"frame_perf" command, aggregated over the recent ring and split by 4:3 vs
native-wide so a 16:9-vs-4:3 A/B reads off one query.
Built to answer where a frame goes without guessing. First use proved
Tomba's 16:9 village lag is the native-wide DOUBLE-DRAW (every primitive
re-issued into the wide FBO), NOT supersampling (SS2==SS1) and NOT the
compositor (present_gpu ~1ms): scene_gpu 16:9=28ms vs 4:3=17ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GL native-wide present (Option B) did glFinish + glReadPixels of the whole
wide FBO to the CPU and re-uploaded it EVERY frame — a hard GPU pipeline stall +
round-trip that made OpenGL 16:9 run far slower than the software renderer (whose
wide path is pure CPU, no GPU sync). Confirmed by isolation: GL 4:3 smooth, GL
16:9 slow, independent of SS/AA.
Add gl_renderer_present_wide_fbo: blit the displayed buffer's wide FBO straight to
the window with glBlitFramebuffer (GPU-side, V-flipped, into the letterbox),
exactly like the canonical gl_renderer_present_vram — no readback, no glFinish,
no re-upload. main.cpp routes wide GL frames here (falling back to the CPU readout
only if the wide surface doesn't exist yet). USER-VALIDATED smooth. The CPU
readback path (glb_render_wide_display) stays as the fallback + the wide_shot
probe.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documentation only. Prior sessions kept re-deriving state from breadcrumbs that
were no longer true:
- overlay_sljit.c / code_provider.{h,c}: corrected "declines every fragment" and
"compile_fragment not yet called from dispatch" — the MIPS->sljit emitter is
fully implemented and wired (then validated via the same-state diff).
- ISSUES.md: add Issue #7 (sljit live execution was unvalidated; the pure-live
save-load wedge), root cause and fix recorded.
(SLJIT.md — the §11 forward plan and architecture clarification — lives at the
workspace root F:/Projects/psxrecomp/SLJIT.md, outside this repo, updated
separately.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause class: a toolchain-less machine never self-improved via sljit on the
normal play path. Two blocks: (1) `auto` resolved to gcc on EVERY machine because
the discriminator was autocompile_configured() — "is a command STRING set" — and
the shipped game.toml always carries overlay_autocompile_cmd; (2) live sljit mode
was reachable only via a hand-set env var (dead public setter, no policy).
Fix:
- autocompile.c: autocompile_toolchain_available() probes PATH for a real C
compiler (gcc/cc/clang). main.cpp passes
`autocompile_configured() && autocompile_toolchain_available()` to
code_provider_init, so `auto` picks gcc only when one is actually reachable,
else sljit.
- overlay_loader.c: overlay_loader_apply_live_policy() (called from main.cpp
AFTER backend resolution) defaults live ON when the resolved backend is sljit
— so a toolchain-less box runs VALIDATED shards live (self-improves on the
normal path); PSX_OVERLAY_SLJIT_LIVE still overrides. The env-only block in
overlay_loader_init is removed (the policy fn subsumes it).
Runtime-only (no regen). Refs SLJIT.md §11 Phase C.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render the genuine 16:9 FOV at correct proportions instead of squash-then-stretch,
using a separate per-framebuffer wide surface so canonical PSX VRAM stays faithful
(4:3 / non-opted games byte-identical; the wide canvas only exists when opted in).
- Wide-aspect mode descriptor (off / squash / native-wide) in gpu.c; opt-in via
[video] aspect_ratio. GTE fed the 4:3 ratio (identity, no squash) in native-wide;
canonical draw_offset/draw_area left faithful (no VRAM widening, no wrap).
- Separate wide present-surfaces (gpu_sw_renderer.c) keyed by back-buffer base:
framebuffer prims mirrored at local x = vram_x - base_x + OFFSET; margins cleared
via the frame-clear mirror; present reads the displayed buffer's surface 1:1.
Eliminates the A/B bleed + flicker (and the segfault) of the shared-canvas attempt.
gr_* vtable extended; GL backend reports unsupported -> canonical present until
the GL port (task 8).
- psx_ws_x_margin native-wide-aware (reuses the existing [widescreen.cull] emit).
CONFIRMED a no-op for Tomba's edge objects: they spawn via a non-FUN_80022E44
path (the foreground classifier is already 16:9-safe). See handoff.
- Draw-census extended with per-prim screen-X extent + back-buffer base for the
native-wide void diagnosis; ws_nw TCP toggle; gpu_state ws.mode/nw_extra.
- Controller: "auto"/"gamepad"/"controller" device = first available SDL pad.
WIP checkpoint. SW compositor works: flicker fixed, faithful, terrain platform
fills the wide frame. REMAINING: discrete foreground OBJECTS (rock pillar, tree
branch, bushes) + the parallax backdrop are still culled at the 4:3 edge via a
separate generation/spawn path (wtrace the OT display-list writers next); GL
compositor port. Detail in NATIVE_WIDE_PLAN.md + the session handoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One batch widening to cover most remaining overlay instruction classes;
validated end-to-end via the live differential rather than per-op.
- jr rX (computed / jump tables): CTRL_TAILJUMP — capture the target before
the delay slot, then cpu->pc = target and return; the dispatch loop
re-dispatches (matches the interpreter's jr). jr $ra stays the clean-
return terminator. (A terminator-less tail-dispatch fragment still
declines — safe.)
- GTE/COP2: MFC2/CFC2/MTC2/CTC2/cmd (op 0x12), LWC2 (0x32), SWC2 (0x3A)
via psx_sljit_cop2(), a runtime helper mirroring the interpreter exactly.
- Unaligned mem: LWL/LWR/SWL/SWR via psx_sljit_memx(), likewise.
- BLTZAL/BGEZAL: link $ra = pc+8 before the delay slot, then branch.
Helper-emitted classes get parity by construction (the helper IS the
interpreter's code). Harness 66/66 (regression); runtime builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The big coverage unlock — the emitter now accepts functions that call
others, not just leaves.
- New runtime helper psx_sljit_call(cpu,target,return_pc,check) in
overlay_loader.c, mirroring the dirty-RAM interpreter's call path
EXACTLY (psx_dispatch_game_compiled / overlay_loader_call_native /
psx_dispatch_call + g_psx_call_bail + psx_call_contract). Returns 1 if
the shard must return immediately (transfer/bail unwind), 0 to continue.
- Emitter: jal/jalr -> CTRL_CALL. Link reg + (jalr) dynamic target are
written BEFORE the delay slot; the helper is called AFTER it via S2
(helper address) with the result driving a conditional return. jal uses
an absolute target, jalr a dynamic one captured in S1.
- KSEG integration fix: the loader now passes the VIRTUAL entry to the
emitter (was passing phys with image_base_vram=0), so return_pc and
jal/J targets carry the KSEG bits the guest uses. The standalone harness
passed a virtual entry, which is why it stayed green; the live
differential caught the phys/virtual mismatch (saved $ra off by the
0x80000000 KSEG bit).
Verified: sljit_emit_test.c extended (jal, jalr, transfer/bail) -- 66/66
pass. Full psx-runtime build clean. Live differential re-validation of
the KSEG fix is the next step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MIPS->sljit emitter, paralleling dirty_ram_interp.c's exec_one but
emitting host code. First slice = single-block LEAF functions ending in
`jr $ra`; everything else aborts the whole fragment to the interpreter
(precision over recall).
- overlay_sljit.c: memory-backed GPRs, 32-bit ops (R3000A wraparound),
loads/stores via icall through the cpu read/write callbacks (same
MMIO/page-watch/dirty-tracking path as the interpreter). Opcodes:
SLL/SRL/SRA/SLLV/SRLV/SRAV, ADD(U)/SUB(U), AND/OR/XOR/NOR, SLT(U),
ADDI(U)/SLTI(U)/ANDI/ORI/XORI/LUI, LB/LH/LW/LBU/LHU/SB/SH/SW, SYNC.
- compile_fragment now returns CompiledFragment{fn,code_lo,code_len};
overlay_loader registers an sljit candidate (crc hashed from the live
bytes it JIT'd from, same live-byte validation as gcc candidates).
- Step 5 wiring is VALIDATION-GATED: JIT-on-miss fires only when
backend==sljit AND the same-state differential is armed (never in a
shadow run). Normal play stays byte-identical; live sljit execution
stays off until the differential gate passes.
- Debug: sljit_try <addr> probe; sljit_status reports shards_registered.
Verified: runtime/tests/sljit_emit_test.c (links overlay_sljit.c +
sljitLir.c only) JITs hand-built leaf functions and checks output —
26/26 pass incl. gpr0 invariant, sign/zero-ext, signed/unsigned compare,
and the decline cases. Full psx-runtime build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce one backend-agnostic code-production seam the overlay
capture/dispatch spine talks to, and make the validated gcc spawn->DLL
path its first implementation. Pure refactor; gcc stays the default and
its behavior is byte-identical.
- New code_provider.h/.c: a CodeProvider struct exposing BOTH production
lifecycles so the interface isn't gcc-shaped -- batch/async
(request/busy/poll_main, the gcc model) and sync-on-miss
(compile_fragment, the sljit model). compile_fragment is defined and
implemented per-provider but not yet called from dispatch (step 5).
gcc provider = direct pass-throughs to autocompile_* (compile_fragment
NULL: a compiler spawn is too slow for the dispatch path); sljit
provider wraps overlay_sljit_* (still declines every fragment).
- Wire up overlay_backend_resolve, which was previously dead code:
code_provider_init() resolves the backend at overlay-cache init and
caches the active provider; code_provider_active() is the accessor.
- Add [runtime] overlay_backend config (auto|gcc|sljit; empty=auto) so
resolution has a config source alongside PSX_OVERLAY_BACKEND.
- Route the 3 direct autocompile_* call sites through the active
provider: overlay_capture.c autocapture tick, main.cpp per-frame poll
+ the init call, debug_server.c autocompile_run (now reports backend).
- runtime.cmake += code_provider.c.
Builds + links clean (TombaRecomp/build-dev, MinGW gcc 15.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add sljit as a self-contained, toolchain-less codegen backend for the
overlay self-improvement loop. gcc stays the default (dev: optimized +
inspectable); sljit is the fallback / production producer. Interpreter
remains the correctness floor.
This commit lands the backend-agnostic foundation only; the MIPS->sljit
emitter is not built yet, so every fragment safely declines to the
interpreter (no change to the validated gcc path).
- Vendor sljit (BSD-2-Clause) at lib/sljit/, wired into runtime.cmake
(compile sljitLir.c + include dir; sljit auto-detects the host arch).
- overlay_sljit.{h,c}: backend-selection policy (PSX_OVERLAY_BACKEND=
auto|gcc|sljit), availability, a real codegen smoke test, and
try_compile() honoring the precision-over-recall bail-to-interp contract.
- sljit_status debug-server command (Rule-3 clean: status via getter).
- THIRD_PARTY_ATTRIBUTION.md: sljit entry.
Verified: TombaRecomp/build-dev (MinGW gcc 15.2) compiles + links
psx-runtime.exe; standalone smoke harness JITs f(a)=a+1234 and runs it
correctly (26 bytes x86-64). Design + locked tier decisions in SLJIT.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The far backdrop (ocean/cloud/distant mountain) is GTE-projected 3D drawn
by the main-EXE driver FUN_8004db3c (sole caller of the GTE emitter
FUN_80027600). The global widescreen GTE X-squash compresses it toward
centre, so its geometry falls short of the revealed 16:9 edges -> blue void
that pops as the camera moves.
Fix: bracket the backdrop driver with gte_ws_set_suppress(1)/(0) so its
draws skip the X-squash and fill the stretched frame (skybox treatment),
while the near 3D world keeps the wider FOV.
FUN_8004db3c draws a MIX (far backdrop + nearer props/scenery), so the
suppress is DEPTH-GATED in gte_rtps_internal: only vertices with projected
SZ >= s_ws_far_threshold (default 900, tunable live via the ws_far_threshold
TCP cmd) un-squash; nearer props stay squashed and aligned. Validated:
threshold 900 fills the void, keeps the foreground door + background
scenery (path/tree) aligned. Residual: the mountain grass-top straddles the
depth band (per-vertex limitation) -> flickers when the camera dips; this
is the checkpoint before exploring native-wide rendering to remove the
squash (and this whole artifact class) entirely.
Mechanism:
- recompiler: [widescreen.backdrop] unsquash_funcs emits
gte_ws_set_suppress(1) at entry + (0) before each jr-$ra return
(code_generator + config_loader + main_psx). Main-EXE -> regen-class.
- runtime: gte.cpp depth-gated squash + s_ws_far_threshold; cpu_state.h
decl; debug_server.c ws_aspect (live squash on/off) + ws_far_threshold
(set split + read SZ stats) diagnostics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Menu fix (user-validated): per-prim center-squash mangled composite 2D
UI (dialog boxes built from tiled cap/middle/cap SPRTs no longer
aligned). Root cause was decoupling — content was squashed while the
frame presented 4:3. Now squash applies IFF the frame is stretched:
gpu_ws_present_native_43() is the single predicate (FMV video OR a
full-2D screen = no character billboards this frame), shared by the
GTE squash, the GPU per-prim squash, and the present 4:3 pin. Full-2D
menus get zero squash + 4:3 pillarbox = pixel-native. Removed the old
menu-2D squash paths (lines/flat-rects/mono-sprites/polylines).
Cull-margin widening (recompiler feature): the game's per-object draw
classifier tests (objX-camX+BIAS) <u RANGE sized for the 4:3 screen;
the GTE squash shows ~33% more world so the margin collapses and
objects pop near the wide edges. New [widescreen.cull] config emits
those immediates with a runtime term psx_ws_x_margin() (bias +margin,
range +2*margin, plus a1+=margin for param-margin variants) — 0 at
4:3/boot/menu/FMV so the original cull is byte-identical, ~53px when
stretching 16:9. Site instruction types are verified at emit (fatal on
mismatch). Requires regen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- FMV: 24-bit frames AND streamed 15-bit MDEC video (Tomba plays its
movies 15-bit; depth24 alone missed them — user-reported) present
pinned to native 4:3. Detector = colour MDEC decode within the last
30 vblanks (mdec_recently_active); texture-decompression luma
decodes do not stamp. Both GL present paths take force_4_3; SW path
centers a 4:3 dst rect. User-verified on the intro movies.
- Menu garble: full-2D screens (no tagged prims in flight for 2
frames) now squash lines, polylines, flat rects and mono sprites
too — dialog borders are drawn with these and stuck out past the
squashed SPRT boxes. Full-width TILEs (fades) stay exempt; in-game
non-textured prims stay untouched (may be world-anchored).
- HUD positioning: in-game untagged SPRTs pivot by thirds — outer-
third elements anchor to their screen edge (16:9 corner position,
native proportions), middle third to centre. Full-2D screens keep
the single centre pivot so composites stay coherent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User-validated on Tomba: world, characters and HUD all present at
native proportions on the 16:9 stretch.
Mechanism (all parameterized via game.toml [widescreen], inert by
default and at 4:3):
- recompiler: [widescreen] sprite_tag_funcs emits a
psx_ws_sprite_tag(cpu) callback at the entry of each listed
function (the game engine's shared per-prim helper). The runtime
records prim pointer ($a0) -> GTE-projected anchor (read from
sprite_anchor_addr scratchpad) in a frame-stamped hash.
- gpu.c: at GP0 execution, prims whose guest address (from the DMA
linked-list walk's source tracking) is tagged get every X
re-squashed around their own anchor — undoing the present stretch
for CPU-computed pixel offsets the GTE squash never saw. Covers
POLY_FT4 billboards and SPRT parts; SPRT texels keep full coverage
via gr_draw_textured_rect_scaled.
- hud_sprt_squash: untagged textured rects (SPRT family — pure
screen-space 2D: HUD, menus) center-squash around the display
centre. Untextured TILEs (fades) untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the DuckStation-style widescreen hack as a config option,
implemented in the runtime (no regen needed):
- gte.cpp: gte_set_display_aspect(num, den). RTPS/RTPT scale the
IR1*h/sz screen-X term by (4*den)/(3*num) around the game's own
OFX projection centre (identity at 4:3). Games cull against the
SXY values they read back from us, so post-projection screen-
bounds culls stay aligned with the visible frame.
- present: GL letterbox_rect generalised to the configured aspect
(gl_renderer_set_display_aspect); SDL_Renderer path gets an
aspect-derived logical size + dst rect. Window opens at the
configured aspect (clamp_window_4x3 -> clamp_window_aspect).
- config: [video] aspect_ratio = W:H (4:3 default, 16:9, up
to 32:9) in game.toml, settings.toml (load+save), launcher seed.
World geometry keeps correct proportions on the stretched present;
screen-space 2D (HUD, FMV, sprite quad widths) stretches — the
known limitation of the squash tier. EXPERIMENTAL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The overlay cache keys DLLs by guest-bytes CRC, which is flavor-blind — the
same guest overlay yields the same filename whether compiled by the base
recompiler or a variant (e.g. widescreen, which emits psx_ws_*/GTE-adjust
symbols). Two flavors sharing a cache dir would collide: a base runtime could
pick up a widescreen DLL (or vice versa) with no guard, since both report ABI
v2.
Fold a codegen-flavor id into overlay_abi() (high 16 bits) and check the
combined tag in the loader's existing ABI gate. Base/master flavor is 0, so the
tag stays == PSX_OVERLAY_ABI_VERSION — fully backward compatible with every
existing base DLL (verified: an 8-DLL base cache loads with zero rejections).
A variant build sets PSX_OVERLAY_FLAVOR (e.g. widescreen: -DPSX_OVERLAY_FLAVOR=1
for the runtime, --flavor 1 for compile_overlays.py); its overlays then report a
distinct tag, so the base runtime rejects+regenerates them and vice versa. The
caches stay separate automatically, even in a shared directory — no new paths.
overlay_api.h: PSX_OVERLAY_FLAVOR (default 0) + PSX_OVERLAY_ABI_TAG.
overlay_loader.c: both ABI gates check the tag.
compile_overlays.py: --flavor arg -> -DPSX_OVERLAY_FLAVOR, baked into overlay_abi().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an opt-in "skip full-motion videos" feature ([video] auto_skip_fmv,
default off). A streaming FMV is detected as active MDEC decode
(mdec_get_decode_count climbing per vblank) + cdrom_xa_stream_active();
while detected, the frontend injects START into pad slot 0 so the game's
OWN FMV player aborts the movie and jumps cleanly to the next screen —
instant, invisible, and desync-free, with all guest state correct.
Mechanism notes (why this and not the alternatives):
- Flooding XA sectors ('instant' CD during streaming) desyncs the player
and hangs the movie on a white screen — rejected.
- Uncapped-pacing fast-forward is safe but slow and choppy (plays the
whole stream) — rejected.
- Triggering the game's native skip is instant and correct. Injection is
gated on raw xa_stream_active so START stops the moment the FMV ends and
never bleeds into the next screen (verified: title stays interactive).
Present/pacing are suppressed + audio muted as a safety net for the 1-2
frames before the game's abort takes effect.
Plumbed end-to-end: config_loader [video] auto_skip_fmv + UserSettings
(settings.toml load/save), main.cpp detector/injection + settings layering
+ launcher seed/apply, and a "Skip FMVs" toggle in the launcher Settings
(launcher.cpp + launcher.rml), mirroring the existing SPU/AA toggles.
Runtime-only; no emitter/regen impact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the launcher's Memory Card 1/2 panels to real on-disk data and make
the per-card enable/path choices flow through to the runtime.
- memcard.c/.h: add stateless introspection (memcard_summary_path parses the
directory frames for used-block count + mtime + validity, no live state),
memcard_format_file for "New", and memcard_init_slots for per-slot paths +
enable flags. memcard_init now delegates to it.
- config_loader: settings.toml [memcard] gains card1/card2 (explicit paths)
and enable1/enable2; load + save round-trip them (UserSettings extended).
- main.cpp: resolve the effective memcard dir before the launcher so it can
introspect the real files; seed/apply per-card paths+enable; boot via
memcard_init_slots (a disabled slot is left absent — SIO reports no device).
- launcher: MC cards now show real filename, size, used-block count, last-
modified date, and a live 15-cell block grid; Browse picks an existing
.mcd, New formats a fresh one (overwrite-prompt save dialog). Per-card
enable + paths are committed to settings on launch.
- The block grid is injected via data-rml (the structural data-for view does
not capture inner-xml in this RmlUi build; data-rml is the robust path and
the markup is fully launcher-controlled).
- Add LauncherSystemInterface so RmlUi warnings/errors reach stdout (they
otherwise go to OutputDebugString on Windows); drop an unsupported
text-overflow RCSS declaration it surfaced.
Verified on Tomba: card1 3/15 + card2 1/15 match the on-disk directory
frames; disabling slot 2 in settings.toml leaves it not-present via mc_status.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a shared disc-identity module (runtime/{include,src}/disc_identity)
that opens a .cue/.bin/.iso, checks for the ISO9660 PVD (raw 2352 and
cooked 2048 layouts), reads the PVD volume id, parses the boot serial
from SYSTEM.CNF, derives the region from the serial prefix, and
optionally streams a full-file CRC32 and compares against expected
serial/CRC. This is now the single source of truth for "is this the
right disc?" — main.cpp's launch-time validate_disc_image is rewritten
to delegate to it (dropping its duplicated raw-read logic and the now
dead cue_data_path/is_iso_pvd/psx_exe_id helpers) so the launch check
and the launcher badge can never drift.
game.toml [game] gains optional disc_crc / disc_sha1 (GameConfig +
config_loader). Left unset by default: the serial check is dump-
agnostic, whereas a baked CRC would flag every legitimate dump of a
different revision/region.
Launcher: GameInfo carries the expected serial + CRC; the launcher
recomputes verification on entry and on every Browse, showing a
colour-coded badge (Serial verified / Disc verified / CRC mismatch /
Wrong game? / Not a PlayStation disc) plus a detail line with the
detected serial · region · CRC. CRC is only hashed when an expected
CRC is configured.
Verified all three tiers on Tomba: Serial verified (no CRC),
CRC mismatch (wrong CRC), Disc verified (CRC B00ECB0D match).
tools/shot_launcher.ps1: bring the launcher GL window to front and
screenshot its client area (GL windows screenshot black via PrintWindow).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full rewrite of the GL backend around a single authoritative hr VRAM FBO
(RGBA8 + stencil mask-bit mirror, internal scale from [video]
supersampling): GPU fills/copies/draws rasterize into it, CPU uploads
batch through a pending-rect quad blit, a PACK pass re-encodes dirty
regions into the native R16UI raw mirror for texture sampling and
readback, and gpu.c routes GPUREAD/mask-check/peek reads through the
facade so the CPU array syncs before any CPU-side read.
Texture sampling adopts the Beetle-PSX model (structure only, studied
from beetle-psx-libretro):
- Per-primitive uv sampling bounds, clamped in the fragment shader when
no texture window is active. Filtered neighbours and S>1 interpolation
overshoot can never read outside the prim's own texture rect. Bounds
back off the exclusive max edge for axis-aligned (2D) uv mappings and
widen to the full page when the uv range crosses a 256 wrap boundary.
- Bilinear takes the NEAREST texel as base (cutout + STP authority),
neighbours toward the sub-texel offset, with per-texel opacity weights
and renormalised colour. The old "-0.5 then floor" base sampled one
texel OUTSIDE the prim on top/left edges; where that neighbour was
empty VRAM (e.g. tiled menu backgrounds with u0=0) the whole edge
column/row discarded — the vertical line on Tomba's title and the
cross seams on the memory-card screens.
The software renderer's bilinear gets the same reformulation so GL and
SW render filtered output identically; the SW nearest path (shipping
default) is untouched.
Also: always-on coherency event ring (flushes, fills, copies, draw
bboxes, packs, readbacks, presents, probe perturbations, with rects +
frame numbers) queryable via the debug server, and gl_vram_diff now
forces a full pack so it reads FBO truth even where the raw-mirror
invariant is broken.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SPU Catmull-Rom float shadow (spu_shadow), 15-bit CRT color LUT (raw bit-exact), verifier. Opt-in (env-gated), default OFF: raw output byte-identical and remains the verify oracle. Reverts loud (DEGRADED) on divergence. ShadowVerifier ported from JRickey/gba-recomp (MIT OR Apache-2.0). Compiles clean; full game-build A/B is the remaining integration gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the remaining 2D primitives (flat/textured rects, fill, lines) onto
the GPU, and present directly from the FBO color texture cropped to the
display region instead of reading the whole 1 MB FBO back to CPU VRAM
every frame.
Why both at once: the native present read the `vram` array directly and
relied on an incidental software op each frame (the HUD/sprites, drawn
last) to trigger the FBO->CPU sync. Moving every primitive to the GPU
removed that trigger, so a fully-GPU frame left `vram` stale -> black
present (the earlier FMV->black regression). The fix is to present from
the FBO when the GPU holds the freshest frame (gl_renderer_have_gpu_frame)
and to sync explicitly (gl_renderer_sync_cpu) on the software/24-bit path.
Result: the per-frame readback is gone for GPU scenes (the forest-lag
source), software/FMV frames keep the CPU present path (verified: the
MDEC intro renders correctly), and loads are faster.
KNOWN-FOLLOWUP cosmetic artifacts to clean up: the GPU textured shader
doesn't yet implement the texture window (wrap/mask), so tiled/wrapped
sprites show seams; bilinear texture_filtering isn't honored on the GPU
path; and the debug screenshot still reads CPU VRAM (stale under the
FBO-present path) so captures can lag the on-screen image.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The opengl backend now creates a genuine OpenGL context and presents through
it. Verified: Tomba boots and renders gameplay correctly via GL (context
"4.6.0 NVIDIA"), pixel-identical to software.
- gpu_gl_renderer.c: gl_renderer_init_context (SDL_GL context on a
SDL_WINDOW_OPENGL window), gl_renderer_present (upload ARGB image -> GL
texture -> full-screen quad -> SDL_GL_SwapWindow), present_blank, shutdown.
Legacy GL 1.x (immediate mode) so no GL loader needed yet (-lopengl32).
Backend vtable still delegates rasterization to sw_* this phase, so the GL
path is pixel-identical — this isolates/validates the context+present.
- main.cpp: window gains SDL_WINDOW_OPENGL when renderer=opengl; GL context
created with graceful fallback to the SDL_Renderer path on failure; present
+ disabled-frame + cleanup branch on g_gl_active. SW path untouched.
- runtime.cmake: link opengl32 (Win) / OpenGL::GL (other).
Phase 2b (next): move rasterization onto the GPU — VRAM as a GL texture/FBO,
modern-GL loader + shaders, primitives at internal resolution (the actual
scaling-without-perf-loss win).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation for a selectable hardware (OpenGL) renderer alongside the software
rasterizer. No behavior change: software remains the default and routes through
the facade to the exact same sw_* functions.
- config: [video] renderer = "software" (default) | "opengl" (config_loader).
- gpu_render.h/.c: gr_* facade + GpuRenderBackend vtable. gpu.c (50 call sites)
and main.cpp (present/scale/filter) now call gr_* instead of sw_* directly.
The software backend table points straight at sw_*, so the SW path is
byte-identical.
- gpu_gl_renderer.c: Phase-1 stub. gl_backend_get() returns NULL, so
requesting renderer="opengl" logs a notice and falls back to software.
- main.cpp: selects the backend (gr_set_backend) before gpu_init().
Phase 2 (next) builds the real GL backend: SDL GL context, VRAM-as-texture/FBO,
shader rasterization at internal resolution, present. Phase 3: coherency,
transparency/dither/mask, then default-with-fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New [video] texture_filtering = "nearest" | "bilinear" (default nearest).
Bilinear smooths textures and 2D backgrounds by sampling the 4 neighbouring
texels and blending them. Done correctly for PSX:
- blends in RGB after the CLUT lookup (never interpolates palette indices)
- transparency-aware: keeps cutouts where the nearest texel is transparent,
and replaces transparent neighbours with the nearest colour so edges don't
bleed to black
- neighbours wrap within the 256-texel page, so the blend stays inside one
texture instead of bleeding across packed VRAM
Applied to all textured primitives (triangles, shaded triangles, sprites,
scaled sprites); independent of supersampling and resolution, so cost is in
texel fetches only, not pixel count. Nearest path is byte-identical when off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a software-renderer supersampling path so PSX games can render at a
higher internal resolution with anti-aliased edges, configured per game via
a new [video] block:
[video]
supersampling = 1..4 # internal-res SSAA factor (1 = native, default)
antialiasing = true # linear present filtering
How it works: when scale > 1, gpu_sw_renderer maintains g_hr, an S*-scaled
mirror of VRAM. Every primitive is rasterized into native VRAM (byte-
identical to the scale==1 path — VRAM stays authoritative for framebuffer
readback, render-to-texture, transfers) AND into g_hr at S* resolution;
textures are always sampled from native VRAM so texels keep native res while
geometry/shading gain resolution. Block ops (fill/copy/CPU->VRAM/pixel write)
replicate into the mirror. Display reads the mirror and the present path
downsamples to the window (true ordered-grid supersampling). 24-bit (FMV)
frames fall back to the native path. scale==1 allocates nothing and behaves
exactly as before.
To keep the mirror coherent, the two GP0 handlers that wrote vram[] directly
(fill-rect 0x02 and the A0 CPU->VRAM image transfer) now funnel through
sw_fill_rect / sw_vram_write — no VRAM write site bypasses the renderer.
main.cpp sizes the SDL texture/logical-size/staging buffer to the internal
resolution and selects linear vs nearest present filtering from the config.
Verified on Tomba at supersampling=2: boots, renders a correct frame, holds
59.96 fps (pacing-limited, render headroom), dispatch_miss_total 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.
The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).
- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
bail aborts the interp run; tail transfers into compiled code now run
with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
headers; anomaly guard at exception entry; TCP_COMMANDS.md docs
Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The two pre-fix attract-idle 'degradations' (2026-06-10 dumps ...393 and
...707) were observer interference, not guest bugs:
- both wedge dumps show 8/8 main-thread stack samples inside WS2_32!send
(same call site) — the main-thread-pumped server blocked sending to a
trickle-draining client, throttling the loop to ~6 fps;
- the 'frame counter reset to 0' was a second process instance (no reset
path exists for s_frame_count or psx_cycle_count; dump ...707 has lower
cycles than ...393);
- the 'kernel-only dispatch loop' signature (0xB0/0x650, restore target
0x80068024, EPC 0x80000048, dispatch_count 0) is the normal Tomba
title-attract idle pattern — identical in a healthy 60-min post-fix soak.
Fix the loophole and make recurrence self-evident:
- send_all_blocking gains a 15 s TOTAL budget per response (the 2 s
SO_SNDTIMEO only bounded zero-progress chunks; a slow-but-alive client
could throttle the emulator indefinitely while heartbeating the
starvation watchdog). Over budget -> client dropped, runtime unharmed.
- cumulative tcp_send_stall_ms + tcp_clients_dropped exported in the
freeze heartbeat (top level + per-tick ring 'tcp_ms') and in every
wedge/fatal dump header, so a TCP-throttled window is attributable at
a glance. Documented in TCP_COMMANDS.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- psx_fatal_halt: deliberate fatal sites (unhandled MMIO access,
trap_crash) now write the structured crash report AND a full
freeze-style ring dump (wedge kind "fatal"), then keep pumping the
TCP debug server forever in debug-tools builds so every ring stays
queryable against the exact crash state (ring-buffer-first: crashes
are interrogated, not snapshotted). Release builds exit(1) as before.
Re-entry guarded; heartbeat JSON gains a "fatal" field.
- gp1_dump: dedicated always-on GP1 (0x1F801814 display control) ring,
512K entries (~15 min of gameplay) with frame-window server-side
filtering. The general MMIO ring rolls over in under a minute of SPU/
DMA traffic, which kept evicting boot-window display history before a
probe could ask "who toggled the display during the logo" (bug 1).
- mmio_dump/mmio_clear documented in TCP_COMMANDS.md.
The halt-and-serve path is what made the pig-crash root cause a
one-repro diagnosis.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pig-into-pig crash (TombaRecomp issue #1, v0.1.5): when a cached-DLL
native overlay function called uncached overlay code, the dirty interp's
overlay local-flow chained the callee's jr-ra straight past the nested
dispatch's return point (the target page is always dirty for overlays),
executing the native caller's tail in the interpreter. The suspended
native host frames then resumed and double-executed those tails against
a moved guest stack, restoring garbage callee-saved registers — main
loop's s1 mode-switch constant destroyed -> zombie loop (blue screen,
music on IRQs) or wild pointer -> fatal MMIO READ16.
Thread stop_addr (psx_dispatch_call's return_addr) through
dirty_ram_dispatch; local flow exits when any transfer or straight-line
fall-through reaches it, handing control back to the dispatch loop's
existing pc==stop_addr return. Signature change is deliberate: stale
generated dispatch fails to compile, forcing regen (BIOS + Tomba
regenerated; other titles regen on their own branches).
Sibling of the Bug A call-contract fix, on the return side. Root-caused
live via always-on rings on a wedged instance; user-verified fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- debug_server.c: dump_ram = alias of read_ram (one request, one response,
up to full 2 MB); screenshot = screenshot_file (24-bit BMP, optional
"path", default psx_screenshot.bmp); vram_peek cap 64->128; read_ram hex
encode via nibble table (was 2M snprintf calls stalling the main thread);
send_all_blocking bounded (1 MB chunks, SO_SNDTIMEO 2s, heartbeats the
starvation watchdog per chunk) and drops the client on send failure
instead of letting the watchdog kill the runtime; handle_quit tags exit
origin.
- beetle_debug_server.c: read_ram cap 64 KB->2 MB via heap+send_raw;
dump_ram/screenshot aliases; screenshot w/h->width/height parity;
set_input now holds until clear_input (frames param optional) matching
psx-runtime semantics — the old default expired after 1 frame and the
off-by-one in the getter cleared the override before it was ever read,
so scripted beetle navigation silently did nothing; wtrace_dump response
heap-built + send_raw (per-entry send_fmt dropped chunks on multi-MB
responses, producing broken JSON).
- crash_trace: new psx_crash_trace_set_exit_origin(); report gains
exit_origin (tcp_quit / sdl_window_close / unknown).
- debug_client.py: O(n) string-aware brace parser (old per-byte loop was
O(n^2) — minutes on a 2 MB response, which starved the server's send and
got the runtime killed); generic key=value passthrough for any server
command without a bespoke CLI mapping.
- TCP_COMMANDS.md updated for all of the above.
Verified: 2 MB read_ram round-trip 0.75 s; scripted beetle navigation
reaches the memcard file-select; wtrace_dump returns valid JSON at 21 MB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- First-run pickers now state exactly which user-supplied file is being
requested: an info box + step-numbered dialog title for the BIOS
(Step 1 of 2, SCPH1001.BIN) and the game disc (Step 2 of 2, named
game + accepted formats), each noting what it is NOT. Game name comes
from the loaded game config.
- event_ring recording is now compiled out under PSX_NO_DEBUG_TOOLS:
it is only readable via the TCP debug server (stripped in production)
and is not part of the freeze dump, so recording was pure hot-path
cost. All other rings were already no-op gated.
- Overlay autocapture (which writes the player-shareable
overlay_captures.json) is now enabled whenever overlay_cache is on,
no longer coupled to the dev-only autocompile command — player builds
produce the contribution file the README promises; the background
compile still requires a configured command and no-ops without one.
Production surface after this: no console (-mwindows), no TCP server,
no ring recording; keeps overlay cache + captures, freeze watchdog JSON,
and crash report.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause (freeze dump 1781046069, addr2line-verified): the wall-clock
pacing loop in sdl_vblank_present read the performance counter twice per
iteration — once in the while-condition, once to compute remaining time.
If the frame deadline was crossed between the two reads (a preemption
window), the unsigned subtraction underflowed, *1000 overflowed, and
SDL_Delay received ~2^64/10^4 ms mod 2^32 ≈ 24.7 days at the standard
10 MHz QPC. The app parks in one SDL_Delay with every counter frozen,
wedged between 'i_stat |= VBLANK' (interrupts.c:286) and exception
delivery — which is why dumps showed i_stat=1 pinned. VBLANK was being
ACK'd every frame right up to the wedge in every dump examined; the
'VBLANK-ACK reentry storm' framing was a symptom misread.
- frame_pacing.c/h: shared race-free pacer (single counter read per
decision, compare-before-subtract, sleep clamped to one period).
Both call sites switched: main.cpp sdl_vblank_present and
beetle_main.cpp. Unit test pins the underflow case.
- Vsync self-heal: the 'recovered storm' dump (1781045865) was 8/8
main-thread samples inside wglSwapBuffers — NV driver blocking ~1.5s
per present. After 3 presents >250ms, drop driver vsync (we pace to
59.94 Hz ourselves). Counters surfaced in the freeze dump.
- Watchdog classifier: 'reentry_storm' threshold (5K/sec) was below the
healthy baseline (~2K reentry blocks/frame × 30-60 fps), mislabeling
normal play and host stalls — that label misdirected two prior
investigations. Now normalized per frame (>20K/frame), plus a new
slow_frames kind (<5 fps) with multi-sample stack capture.
Verified: test_frame_pacing ALL PASS; live Tomba paced at 59.962 fps
over 20s (single-process measurement).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runtime detects an uncovered overlay/kernel variant by sustained dirty-RAM
interp pressure inside the capture windows (g_dirty_window_dispatches,
sampled every 600 vblanks; fires only at a coherent non-loading moment via
cdrom_load_in_progress, 60s cooldown, 16/session cap), writes the capture
JSON, and spawns the game.toml-configured compile command
([runtime] overlay_autocompile_cmd, cmd.exe /C, cwd=project root, child
output kept in an in-memory ring — no log files). On child exit 0 the emu
thread rescans the cache (overlay_loader_rescan: idempotent dir re-scan +
checked-regions reset) so the new DLL loads on the next dispatch and the
variant goes native in-session. Dedup is tool-side (existing-CRC skip);
pressure drops to zero once code is native, so the loop self-limits.
New TCP: autocompile_status / autocompile_run / overlay_rescan.
compile_dll now uses absolute native paths and reports exit codes.
VERIFIED live (user-played route): 2 auto-triggers fired, captures written,
child ran recompiler+audit (new variants 1E7DDE2C, A62D0A7C discovered and
audit-clean), rescan picked the compiled DLL up in-session. OPEN ISSUE: gcc
exits 1 with no output inside the spawned cmd.exe child (same absolute-path
command succeeds from an interactive shell) — compile step currently
completed manually; suspect child-env difference, to be isolated next
session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
While the game is loading, skip wall-clock pacing in sdl_vblank_present
so the guest runs at host speed (presents 1-in-30 for visual progress).
Gated three ways: game.toml [runtime] turbo_loads (OPT-IN — streaming
titles must stay off), post-BIOS-handoff only (fntrace_is_game_started),
and cdrom_load_in_progress() — a data-sector read stream is active or a
data sector landed within the burst-gap window (bridges per-file
Setloc/seek gaps); XA streaming (FMV/CD audio) is never a load. New
turbo_loads TCP command gets/sets the enable and reports load_active +
unpaced-vblank count.
Measured (Tomba boot, vs step-3 baseline; user-confirmed in play):
intro load 1066 sectors 7.59s -> 6.20s; later loads 3.3-5.1 ms/frame
vs 16.7 paced (3-5x); 704 vblanks ran unpaced; no watchdog events.
Compression ceiling is guest speed during loads, which is currently
interpreter-bound (the 0x11xxxx loader-region family, ~12M insns) —
step 2.8 variant-capture automation multiplies turbo directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CDROM_INSTANT_MAX_PER_FRAME becomes a runtime variable (default 32,
clamp [1,4096], period still floored at CDROM_MIN_DELAY). Three writers
share the knob: game.toml [runtime] instant_max_per_frame, the new
cdrom_instant_rate TCP command (in-session A/B without rebuilds), and
the upcoming turbo-through-loads predicate (step 4).
New always-on CD load-burst ring (ring-buffer rule): one record per
gap-separated run of delivered data sectors — start/end frame, host
wall ms, sector count, rate+divisor in effect. cdrom_bursts TCP command
dumps newest-first, making "load duration" a measured quantity.
MEASURED CONCLUSION (Tomba boot, identical deterministic workload):
the 32/frame cap is NOT the load-time bottleneck. Intro load at rate 32
= 1066 sectors / 456 frames / 7.59s; at rate 256 = 1064 / 449 / 7.48s
(identical within noise; actual delivery 2.2-4.8 sectors/frame, never
near either cap). Tomba's loads are paced by the game's own per-sector
processing at real-time speed — wall-clock load time is a job for
step 4 turbo (run the guest above real-time while loading), not for a
bigger IRQ budget. Default stays 32. No watchdog transient at either
rate (the old frame ~184-213 reentry storm did not reproduce — likely
calmed by step 2.7's native exception tail).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the overlay JIT cache below the 0x98000 floor to kernel RAM
[0, 0x10000) — the BIOS part-2 image + install-at-runtime stubs
(Rule 18), dominated by the SIO data-byte slot at 0xCF0 whose
install-slot hook tail-dispatches the whole exception-handler tail
into the interpreter on EVERY exception once installed.
Window model (dirty_ram_interp.h): two capture/candidate windows —
kernel [0, 0x10000) and overlay [0x98000, RAM); main-EXE text between
them is statically recompiled, never captured. Interp local-flow gates
intentionally stay overlay-only (kernel runs per-block in exception
context; named constant documents this).
- overlay_capture: per-window dirty-run enumeration, runs clamped at
window edges (kernel CPU-write pages and main-EXE DMA pages can be
adjacent; never merge them).
- overlay_loader: dispatch lazy-load gate covers both windows;
scan_cache_dir validates the <addr8>_<crc8>.dll shape explicitly
(kernel region_start 0 is legal, zero-parse is no longer a sentinel).
- classifier (compile_overlays.py): kernel-window-only DISPATCH_ROOT
promotion — an orphan dispatch interior no rooted walk covers has no
host to truncate and is a real execution root by construction (the
static install-slot hooks tail-dispatch into it). One orphan per
iteration so remaining orphans alias into the new host. Overlay
regions stay ineligible (cross-variant per-PC evidence: 0xE889C class).
- main_psx --overlay: 'dispatch_root' seed lines are trusted walk roots
(exempt from the boundary re-check); seed range is now the loaded
image's own bounds (the old hardcoded [0x80010000,0x80200000) window
silently dropped kernel seeds).
- overlay_api/loader/DLL preamble: route psx_restore_state_escape
through OverlayCallbacks (appended last, by-value copy keeps old
DLLs compatible) — exception-context kernel code longjmp-returns.
Tomba verified (user-played: title -> LOAD list renders -> save loads
-> overworld laps): kernel interp per-PC rows 3 -> 0 (was 0xCF0 =
61,775 dispatches / 2.71M insns at the LOAD screen alone, ~4.6M
insns/session); host func [0xCF0,0xEA0) + 0xDF8/0xE08 aliases, audit
clean; dispatch_native 357,763; dispatch_miss_total 0; interp aborts 0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two pieces from the variant-coverage investigation:
1. dirty_ram_stats per_pc rows gain an `entries` counter: dispatches
that arrived from NATIVE code (call / dispatch loop), distinguished
from the interpreter's own block-to-block chaining via a consumed
chain-target latch. `hits` conflates the two — below the local-flow
floor every taken branch re-dispatches, so chain blocks dominate and
misattribute hot families. First use REFUTED the main-EXE-interior-
entry hypothesis: the 0x1DFF8/0x1E0D8 family (~1M insns) has ZERO
external entries — it is purely downstream of overlay interp runs,
and collapsed to 57K insns once the overlay variant was compiled.
No main-EXE aliasing needed.
2. compile_overlays.py merges prior-manifest evidence on rebuild: every
F entry in an existing .ranges for the SAME image bytes was proven
compilable, so a fresh (poorer) capture cannot regress coverage.
Callable entries re-enter as captured function entries; non-callable
ones (alias wrappers from a prior build) re-enter as dispatch entries
so the classifier re-derives their interior disposition — feeding
them back as walk roots would truncate their hosts.
Village variant DLL (000E7000_9707AAB5) force-rebuilt with merged
evidence: 195 -> 244 functions incl. interior aliases. Verified on a
user-played village+overworld lap: 0x12A60C (~1M interp insns),
0x138084, 0xED2F4 now native and absent from per_pc; 0 dispatch misses,
0 aborts, 0 invalidations.
Remaining interp load is (a) the variant long tail — the same overlay
region holds scene-dependent variants where one variant's code address
is another variant's DATA (0x800E889C: code in the overworld-family
image, a pointer table in the village image); per-entry hashing
correctly refuses stale native code, and each variant gets covered when
captured — and (b) the kernel-RAM SIO stub at 0xCF0 (~4.6M insns/
session), a separate class.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Accumulated working-tree state for the overlay-JIT-cache phase, committed as a
checkpoint before the Bug A fix (next commit):
- event_ring.{c,h}: always-on unified event-timeline ring (IRQ deliver/gate/
raise, I_STAT changes, DMA kick/sched/done, ENQ/DEQ per source, exec-mode
tagged). Edge-recorded, eviction-bounded, survives a blue screen; dump via
event_ring_dump/_tail/_clear.
- cdrom.c: disc_speed="instant" no longer collapses every sector read to the
500cy floor (which flooded ~1100 DMA3 completions/frame and starved main
code). CDROM_INSTANT_PERIOD bounds sector IRQs to CDROM_INSTANT_MAX_PER_FRAME
(32)/frame. Known caveat: a transient ~1-2s reentry storm at boot frame ~210
still trips the watchdog once, then recovers.
- interrupts.c: claimed-IRQ cooldown is now per-source — SIO stays 0 (card
reads need back-to-back IRQs); DMA/VBLANK/timer get CLAIMED_PROGRESS_QUANTUM
(8) blocks of guaranteed main-code progress between deliveries. Root cause of
the instant-disc freeze: cooldown=0 let the block leader re-take the
exception before its body ran (pinned at the 0x8005A1D8 field loop).
- gpu/dma/memory/sio/timers/psx_cycles: event-ring instrumentation at raise/
kick/complete/schedule sites; g_event_step_conservative (single-cycle event
stepping A/B used to refute the event-granularity hypothesis for Bug A).
- overlay_capture.c, overlay_loader.h, crc32: capture/manifest plumbing for
per-entry validity (Inc3).
- recompiler: overlay-mode emission updates (code_generator, main_psx).
- tools: compile_overlays.py audit/patch updates; tomba_loadtest.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reload-on-return does not fire against the current merged-capture DLL. Added
code-span watch/hash (vs whole-region) + diagnostics, and the counters pinpoint
why:
r0_fn_lo=0x000E73C0 r0_fn_hi=0x0013B184 -> 343 KB function span
r0_writes_since_invalid=107222 -> below span/2 threshold (171 KB)
reval_attempts=0 -> never triggered
The DLL was compiled from the merged 1.1 MB roaming capture, so its functions
are scattered across the whole region with mutable data interleaved. There is no
coherent reload unit to threshold or hash: the re-validate trigger never fires,
and a 343 KB hash including data would never match anyway. OV-1 (correctness) is
unaffected — invalidation still fires (no blue screen); only the native speedup
on return is blocked.
Conclusion: reload-on-return requires coherent per-overlay segmentation (split
the merged region into real load units). The reload machinery stays in as inert
scaffolding until segmentation lands. Diagnostics exposed via
overlay_loader_status (r0_*, reval_attempts, reval_crc_miss, last_reval_crc).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two additions:
1. overlay_capture_dump TCP command — flush overlay_captures.json on demand,
so a freeze/kill can't lose a capture session (no clean window-close needed).
2. Reload-on-return: when an invalidated overlay region is loaded back with the
SAME content, re-enable its functions natively instead of staying interpreted.
- At register, hash the live region (crc_live).
- On invalidation, keep watching and count writes into the region (instead of
clearing the watch).
- When the region is substantially rewritten (>= size/2 bytes — a real reload,
not incidental data writes), the next dispatch re-hashes once; on a match to
crc_live, flip the region valid again. O(1) — dyntab entries never left, no
DLL reload. Cheap-gated so steady-state churn doesn't pay per-dispatch hashes.
- New counter: revalidations (in overlay_loader_status).
Ghidra note: searched the Tomba main EXE for the overlay load address (0x800E7000
virt / 0x000E7000 phys) — no flat overlay table; the address is computed and the
overlay is copied/decompressed to the code region by a BIOS routine. So overlay
identity must be a runtime content hash (this commit), not a static table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley