P3 step 1 (FAITHFUL_TIMING_PLAN.md): both backends now account guest CPU cycles
through ONE function, runtime/include/psx_instr_cost.h::psx_instr_base_cycles(insn),
so they can never disagree (the -8 drift class). Interp routes its 3 per-insn
charges (exec_delay_slot, dirty dispatch loop, precise slice) through it; the
recompiler sums it over each block (+ the outside delay-slot clone) to fold into
the compile-time block charge. Stage 1 = identity (1 cycle/insn) = the proven
baseline, so the seam is BEHAVIOR-PRESERVING: regen is byte-identical (verified,
full.c + dispatch.c) and Tomba 2 still reaches the intro FMV (i_stat 0x8D
streaming). Stage 2 edits ONLY psx_instr_base_cycles (transcribe real R3000A
costs from in-tree Beetle cpu.cpp/gte.cpp + psx-spx, verify each vs Beetle) and
both backends update together.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The multi-week Whoopee-Camp logo->FMV stall was a TIMING-FAITHFULNESS bug, not
game logic: the static recompiler charged cycles block-granular and a branch
whose delay slot is a SEPARATE block leader (branch at end_addr, delay slot
outside the block) undercharged by one cycle on every execution -- the
always-executed delay-slot clone was emitted but counted by neither the branch
block nor the (unentered) delay-slot block. ~8 such sites on the boot init path
= the measured -8 cycle drift that made native run behind the interpreter, so
the frame-1824 elapsed-Timer1 logo-delay loop exited after ~42 iterations
(native) instead of ~1557 (interp/HW) and never reached the FMV.
Fix (code_generator.cpp translate_basic_block, the faithful class fix -- every
dynamically executed instruction charged exactly once):
block_exec_cycles = instruction_count
+ (exit branch at end_addr with delay slot outside ? 1 : 0)
applied to both the block cycle charge and the slice budget. R3000 is MIPS-I so
the delay slot is unconditional; the not-taken path already falls through to
branch_pc+8 (no double-count). Result, screenshot-verified: native passes the
logo-delay loop, screen animates, CDROM+DMA stream active, the jungle intro FMV
plays.
Also in this checkpoint (all on wt/tomba2, not merged/pinned):
- PARKED precise interrupt-slicing (PRECISE_IRQ_SLICE.md): root-caused its pc=0
boot exit to a mid-function clean-text resume PC the dispatcher can't re-enter;
ChatGPT-validated fix is "every block leader = re-enterable CPS continuation"
(psx_game_is_function_entry predicate added; precise_pc_dispatchable +
irq_taken-once restructure in psx_run_precise). Default OFF (PSX_PRECISE_SLICE=1
to resume); inert on the baseline. Slice-trace diagnostics added to freeze_check.
- FAITHFUL_TIMING_PLAN.md: authoritative game plan (P1-P6) + standing directive
(build the faithful core, never hacks; ecosystem is being built, not preserved);
referenced from CLAUDE.md Rule -1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the fail-closed native entry guard from wt/tomba2 (b5a6940 / bd60381) to
master, where it had never been merged — which is why Tomba (and any master-built
title) still wedged. This is the long-standing "blue screen" / shared-RAM-
divergence-between-interp-and-native class.
Root cause: an overlay function dispatched at a FOREIGN INTERIOR PC (a range-
ownership mismatch routed a PC the function does not own into it) hit the CPS
entry-switch `default: break`, which fell through and ran the function FROM ITS
TOP — re-running its prologue and corrupting shared CPU/RAM state, so the guest's
main loop then waits forever on a condition that never gets set (the 0xCF0 SIO
spin / blue screen). Reproduced on Tomba's Village of All Beginnings: a native
0x000E7000-region shard function entered at a foreign PC wedged; the interpreter
ran the same bytes correctly.
Fix: the generated CPS entry-switch for OVERLAY functions (config_.overlay_mode)
now FAILS CLOSED on an unknown non-zero continuation PC — it restores the PC and
calls psx_native_bad_entry instead of running from the top. overlay_loader_dispatch
consumes g_native_bad_entry and routes the PC to the sanctioned dirty-RAM
interpreter (the live bytes run correctly — NOT a stub/HLE). True-prologue entry
still runs from the top. Static (BIOS/boot-EXE) functions keep the legacy fall-
through. ABI bumped 5->6 so stale ABI-5 caches are rejected and rebuilt with the
guard.
Minimal port: kept only the fail-closed mechanism + its ABI plumbing. The Tomba2
host_aliases interior-routing and the native-block / CPS-probe diagnostics (which
depend on the overlay-cache-v2 multi-range Candidate fields not on master) were
intentionally omitted — the fail-closed default alone is the safety net.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16:9 BG widen draws LEFT extra tile columns each side, but the engine's
streamer (FUN_800273e4 -> FUN_800274a0) only refreshes one column per side per
frame at the leading edge. On scene entry / fast scroll the inner reveal columns
held stale tiles from a prior scrollY -> real-but-misplaced tiles spilling
up-and-left (the "staircase"). Census confirmed this is NOT the BG packet-budget
overflow (674-816 prims/frame, well under the 1000-tile cap).
Fix (render window == refresh window): clone FUN_800274a0's inner metatile->ring
fill into mmx6_fill_column() and re-stream every column of the widened window for
all 3 layers each frame (psx_ws_mmx6_bg_refill_all), triggered once/frame from the
existing count_site hook (mmx6_bg_refill_tick, frame-gated). Uses each layer's own
scroll, omits the engine's scrollX wrap-bookkeeping writes (those drive the engine's
own incremental streamer), skips map-loop seams; off-map columns fill tile 0 (the
renderer skips them). Validated byte-exact against the engine's own streaming
(0 mismatches / 1113 cells x 3 frames; debug cmd mmx6_freshfix {validate:1}). Gated
native-wide <=384w so 4:3 and the 512 hi-res title stay byte-identical.
Also removes the 2-col reveal cap (WS_FULL2D_MAX_REVEAL): it never addressed the
staircase and cost screen real-estate; the full 53px/4-col reveal is restored.
Parked, gated-off experiments ride along (inert, default off): host-side reveal
(Attempt A: gpu_ws_mmx6_emit_reveal + sw_wide_emit_tile) and BG packet-buffer
relocation (Attempt B: [widescreen.bg2d] bufbase/cap recompiler hooks). These target
the separate dense-stage budget-overflow concern, not this fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The BG renderer widen (a8f37c1) draws extra tile columns on both sides, but the
engine's 64-column tile ring is only refreshed at its leading edge: the streamer
(FUN_800273e4) re-streams ONE column per side per frame at world-X scrollX-16
(left) and scrollX+16+width (right). So the columns the widened renderer reaches
were never populated — they showed empty (black margin) or stale (old tiles as
the camera scrolled), the "stale margins" the user saw in motion.
Add two more [widescreen.bg2d] sites (stream_left_site / stream_right_site) that
push the streamer's left/right leading-edge world-X out by LEFT*16 via gpu.c
psx_ws_mmx6_bg_stream_{left,right}, so the incremental streamer keeps the ring
valid across the widened window as the camera scrolls. Same number of streams
(the single leading-edge target just moves out LEFT columns) — no extra cost.
Identity at 4:3 / 512 hi-res.
Verified: the previously-black left reveal margin now fills with coherent
adjacent stage; blackL/blackR = 0 across scenes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure-2D sprite games (Mega Man X6) render a true 4:3 field of view with no
overscan to reveal, so 16:9 requires making the engine draw more. Two parts:
- [widescreen] full_2d opts a pure-2D game into the native-wide present path
(it never emits the 3D sprite-tag hook the gameplay detector keys on, so
every in-game frame was misclassified as a 4:3 menu and pillarboxed).
- [widescreen.bg2d] is a gen-time hook on the per-layer background tile
renderer (MMX6 FUN_800270d0) at its column-count / start-tile-column /
start-screen-x instructions, routed through gpu.c psx_ws_mmx6_bg_{cols,
startcol,startx}. It draws LEFT=ceil(nw_offset/16) extra 16px columns on
BOTH sides so the loop fills the 16:9 reveal margins with real adjacent
stage, existing columns staying pixel-aligned with objects. Identity at
4:3 and in the engine's own 512 hi-res mode (so the title is untouched).
Tooling: wide_full debug command + sw_wide_dump_full dump the entire
native-wide compositor surface (both vertical-double-buffer y-bands and both
reveal margins) for diagnosis.
Known follow-ups: object screen-X cull widen (actors in the margins);
top/bottom flicker + stale-margin refresh on the vertical double-buffer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The continuation-passing leak fix (RECURSION_BUG.md §25) is validated (Tomba boots+plays,
ce_profile flat through a 56.9k-frame soak, full overlay CPS). Flip the gate in all three
emitter sites so a normal regen emits CPS; PSX_CPS=0 is the legacy escape hatch. Verified:
default regen -> CPS markers present; PSX_CPS=0 -> none; default-CPS Tomba boots clean, leak flat.
Doc §25.6.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller (item 3): present an analog (DualShock) pad by default for
analog-capable games so the left stick gives variable run speed; fold the
physical D-pad / keyboard arrows onto the left analog axes at full deflection so
they still move the character (avoids the Special Edition full-throttle-D-pad
bug). Add a launcher-editable analog-stick deadzone option (settings.toml +
input.ini + game.toml [controller] default), modeled on snesrecomp.
Settings persistence (item 6): deterministic restore-at-init + save-on-exit for a
game's native OPTION-screen settings, declared per game in a dedicated
game_options.toml. The recompiler rewrites each setting's boot-init store (sb/sh)
to route its value through psx_game_option_store(addr,val), applying a value
persisted from a prior session over the game default (identity when none, so a
fresh install is byte-identical). The runtime saves the live config globals to
<memcard_dir>/<game_id>.options on exit (atexit). Adds a game_options debug
command. Requires a regen for the hook to emit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add [widescreen.cull] auto_screen_x — a gen-time pass that recognizes the GTE
per-vertex trivial-reject signature (a `sltiu …,0x140/0x141` width compare paired
with a `sltiu …,0xE0/0xF1` height compare in the same function) and emits every
WIDTH compare routed through the existing psx_ws_x_margin() helper, so the wider
16:9 field of view is submitted to the GPU instead of being culled at the 320
screen-edge boundary.
The width compare is sign-extended and shifted to the true wide window
(-margin <= SX < imm + margin) so BOTH the left and right margins reveal: SX is a
16-bit GTE coord loaded via lhu, so an off-left vertex is a large unsigned value
and a plain `SX <u (imm+2m)` widen only fixes the right edge. At margin 0 the
verdict reduces bit-for-bit to vanilla `SX <u imm`, so 4:3 / widescreen-off stays
byte-identical (ENHANCEMENTS Rule 1).
Replaces hand-listing the ~100 identical sites with one opt-in, per-game flag;
generic to any PSX GTE title (the 0x140/0xE0 reject pattern is universal). The
vertical (0xE0) bound is never touched. Tomba: 106 sites auto-widened across the
RTPT tri/quad and RTPS billboard render funnel.
Files: recompiler/{include,src}/code_generator.{h,cpp} (signature detect + emit),
config_loader.{h,cpp} (parse), main_psx.cpp (wire-through).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The far backdrop (ocean/cloud/distant mountain) is GTE-projected 3D drawn
by the main-EXE driver FUN_8004db3c (sole caller of the GTE emitter
FUN_80027600). The global widescreen GTE X-squash compresses it toward
centre, so its geometry falls short of the revealed 16:9 edges -> blue void
that pops as the camera moves.
Fix: bracket the backdrop driver with gte_ws_set_suppress(1)/(0) so its
draws skip the X-squash and fill the stretched frame (skybox treatment),
while the near 3D world keeps the wider FOV.
FUN_8004db3c draws a MIX (far backdrop + nearer props/scenery), so the
suppress is DEPTH-GATED in gte_rtps_internal: only vertices with projected
SZ >= s_ws_far_threshold (default 900, tunable live via the ws_far_threshold
TCP cmd) un-squash; nearer props stay squashed and aligned. Validated:
threshold 900 fills the void, keeps the foreground door + background
scenery (path/tree) aligned. Residual: the mountain grass-top straddles the
depth band (per-vertex limitation) -> flickers when the camera dips; this
is the checkpoint before exploring native-wide rendering to remove the
squash (and this whole artifact class) entirely.
Mechanism:
- recompiler: [widescreen.backdrop] unsquash_funcs emits
gte_ws_set_suppress(1) at entry + (0) before each jr-$ra return
(code_generator + config_loader + main_psx). Main-EXE -> regen-class.
- runtime: gte.cpp depth-gated squash + s_ws_far_threshold; cpu_state.h
decl; debug_server.c ws_aspect (live squash on/off) + ws_far_threshold
(set split + read SZ stats) diagnostics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Menu fix (user-validated): per-prim center-squash mangled composite 2D
UI (dialog boxes built from tiled cap/middle/cap SPRTs no longer
aligned). Root cause was decoupling — content was squashed while the
frame presented 4:3. Now squash applies IFF the frame is stretched:
gpu_ws_present_native_43() is the single predicate (FMV video OR a
full-2D screen = no character billboards this frame), shared by the
GTE squash, the GPU per-prim squash, and the present 4:3 pin. Full-2D
menus get zero squash + 4:3 pillarbox = pixel-native. Removed the old
menu-2D squash paths (lines/flat-rects/mono-sprites/polylines).
Cull-margin widening (recompiler feature): the game's per-object draw
classifier tests (objX-camX+BIAS) <u RANGE sized for the 4:3 screen;
the GTE squash shows ~33% more world so the margin collapses and
objects pop near the wide edges. New [widescreen.cull] config emits
those immediates with a runtime term psx_ws_x_margin() (bias +margin,
range +2*margin, plus a1+=margin for param-margin variants) — 0 at
4:3/boot/menu/FMV so the original cull is byte-identical, ~53px when
stretching 16:9. Site instruction types are verified at emit (fatal on
mismatch). Requires regen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User-validated on Tomba: world, characters and HUD all present at
native proportions on the 16:9 stretch.
Mechanism (all parameterized via game.toml [widescreen], inert by
default and at 4:3):
- recompiler: [widescreen] sprite_tag_funcs emits a
psx_ws_sprite_tag(cpu) callback at the entry of each listed
function (the game engine's shared per-prim helper). The runtime
records prim pointer ($a0) -> GTE-projected anchor (read from
sprite_anchor_addr scratchpad) in a frame-stamped hash.
- gpu.c: at GP0 execution, prims whose guest address (from the DMA
linked-list walk's source tracking) is tagged get every X
re-squashed around their own anchor — undoing the present stretch
for CPU-computed pixel offsets the GTE squash never saw. Covers
POLY_FT4 billboards and SPRT parts; SPRT texels keep full coverage
via gr_draw_textured_rect_scaled.
- hud_sprt_squash: untagged textured rects (SPRT family — pure
screen-space 2D: HUD, menus) center-squash around the display
centre. Untextured TILEs (fades) untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.
The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).
- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
bail aborts the interp run; tail transfers into compiled code now run
with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
headers; anomaly guard at exception entry; TCP_COMMANDS.md docs
Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Coverage campaign step 2. Interior code addresses reached through
function-pointer tables (e.g. Tomba's entity event pumps: 0x8004DF00
family inside func_8004DD1C, table at 0x80013F18) previously dispatch-
missed into the dirty-RAM interpreter, and seeding them truncated the
host function (mid-function-seed softlock class).
Discovery (main_psx): iterative loop classifying every candidate entry
against the current function set — function start: covered; interior:
ALIAS entry (host never capped); gap: forced entry (explicit seeds
trusted; scanned pointers need jr-ra/prologue boundary evidence). The
generic scanner sweeps all data words outside code ranges for text
pointers with table evidence (an adjacent text-pointer word).
Emission (code_generator/control_flow): aliases grouped per host into
ONE shared static body — entry switch + the union of blocks reachable
from any entry (successors + jr jump-table edges) — plus a dispatchable
wrapper per entry. Sibling entries are injected as block leaders so all
group CFGs are identical. Aliases are excluded from the split pre-pass
containment so they never truncate hosts; the pre-pass cap is raised
16 -> 256 with a loud non-convergence warning (16 no longer converged
once promotions widened the function set).
Tomba (SCUS-94236): 1441 alias entries in 108 host groups, 25 scan-
promoted entries; dispatch 2855 -> 5562; generated C 12 MB -> 15.3 MB
(naive per-alias duplication was 80 MB). Verified on the overworld
route: 0x4Dxxx family gone from dirty_ram_stats per-PC (was ~0.5M
insns/session), 0x8004DF00 called natively once/frame from
ra=0x800462C4 (fntrace), dispatch_miss_total=0, aborts=0, user-visible
gameplay clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Accumulated working-tree state for the overlay-JIT-cache phase, committed as a
checkpoint before the Bug A fix (next commit):
- event_ring.{c,h}: always-on unified event-timeline ring (IRQ deliver/gate/
raise, I_STAT changes, DMA kick/sched/done, ENQ/DEQ per source, exec-mode
tagged). Edge-recorded, eviction-bounded, survives a blue screen; dump via
event_ring_dump/_tail/_clear.
- cdrom.c: disc_speed="instant" no longer collapses every sector read to the
500cy floor (which flooded ~1100 DMA3 completions/frame and starved main
code). CDROM_INSTANT_PERIOD bounds sector IRQs to CDROM_INSTANT_MAX_PER_FRAME
(32)/frame. Known caveat: a transient ~1-2s reentry storm at boot frame ~210
still trips the watchdog once, then recovers.
- interrupts.c: claimed-IRQ cooldown is now per-source — SIO stays 0 (card
reads need back-to-back IRQs); DMA/VBLANK/timer get CLAIMED_PROGRESS_QUANTUM
(8) blocks of guaranteed main-code progress between deliveries. Root cause of
the instant-disc freeze: cooldown=0 let the block leader re-take the
exception before its body ran (pinned at the 0x8005A1D8 field loop).
- gpu/dma/memory/sio/timers/psx_cycles: event-ring instrumentation at raise/
kick/complete/schedule sites; g_event_step_conservative (single-cycle event
stepping A/B used to refute the event-granularity hypothesis for Bug A).
- overlay_capture.c, overlay_loader.h, crc32: capture/manifest plumbing for
per-entry validity (Inc3).
- recompiler: overlay-mode emission updates (code_generator, main_psx).
- tools: compile_overlays.py audit/patch updates; tomba_loadtest.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restores entry-scoped overlay analysis (analyze_exact_entries +
split_mid_function_targets=false) as the MANDATORY mode for overlay inputs,
renamed --overlay-exact -> --overlay: the ingester's contract, set
unconditionally by compile_overlays.py for every overlay, never a human toggle.
Evidence-scoped discovery compiles only proven seeds + reachable code and keeps
branch/jump-table targets as in-parent labels — no whole-byte sweep, so it no
longer decodes embedded data tables as code.
Effect on the captured Tomba overlay: the data-as-code explosion collapses from
thousands of UNSUPPORTED to 122 clean functions with 42 residual unsupported
opcodes + 8 bad targets. Those residuals are now a per-function-audit concern
(Inc1-B), not a wholesale region failure. Main-EXE codegen is unchanged (Tomba
still regenerates 2854 functions; split pre-pass still runs in non-overlay mode).
Also adds docs/overlay-recompilation-design.md — the converged design of record
(precision-over-recall; compiled!=registered!=callable; write-invalidation over
CRC; internal-labels-in-parent; the RAM-write-path audit result; the three
implementation increments).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rips out the --overlay-exact recompiler path (analyze_exact_entries,
split_mid_function_targets, the CLI flag, and the compile_overlays.py flag),
returning the recompiler to master's naive whole-blob analysis. Overlay
compilation now splits mid-function branch targets into standalone functions —
the original behavior that produced black-on-load / blue-on-new-game — so we
can faithfully reproduce the failure before fixing it.
Pivot in approach: rather than excluding mid-function seeds at scan time, the
plan is to capture everything (the full overlay image is already dumped to
overlay_captures.json) and have the ingester detect each seed's containing
parent function at encode time and merge/ignore mid-function seeds, while
flagging entries reached only via computed/indirect jumps as jump-risk.
The runtime overlay loader/capture system (b2929aa) is unchanged. The 5
recompiler files are restored byte-exact to master (d22c9ce); config_loader's
overlay_cache flag is retained.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.
Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).
Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
targets and set split_mid_function_targets=false, so overlay basic-block
targets are NOT split into bogus standalone functions (the mid-function-seed
failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
mid-function split pre-pass behind split_mid_function_targets (default true,
so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).
Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).
Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Framework work on codex/mmx6-support that makes Tomba build and play
(title screen, 0 dispatch misses) and MMX6 boot through BIOS to its
intro. Touches recompiler codegen (code_generator, function_analysis)
and runtime CD-ROM/DMA/MDEC/GPU/SIO timing (cdrom, dma, mdec, gpu,
psx_cycles, traps, fntrace, debug_server) + headers + runtime.cmake.
Milestone snapshot for collapsing onto master. Build dir excluded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Config loader (recompiler/src/config_loader.{h,cpp}): single source of
truth for both BIOS and game runtime settings (debug_port, window_title,
memcard_dir, disc path). psxrecomp-bios and psxrecomp-game both accept
--config <toml>; runtime takes --game <toml> (no more CLI overrides).
- runtime/src/main.cpp: replaced --disc/--memcard-dir/--game-root CLI
overrides with --game <toml>; game.toml is the single source of truth.
Discless launch (no args) goes to BIOS shell.
- runtime/src/dma.c: fixed update_master_irq() re-asserting I_STAT bit 3
on every DICR read. BIOS shell DMA handler at 0x80058628 only clears
DICR bit 27 (ch3 flag), so level-sensitive re-assert pinned the kernel
in the exception loop. Bit 3 is now edge-triggered (one assert per DMA
completion). Discless boot now reaches MAIN MENU / MEMORY CARD / CD
PLAYER. With-disc Tomba boots through to Whoopee Camp splash.
- Audit tooling: codegen_audit.py (BIOS), codegen_audit_game.py (game),
audit_config.py (config loader for tools), verify_audit_schema.py.
Replaces older tools/_codegen_audit.py probe.
- Docs: CURRENT_STATE.md (project state), UNIFICATION_PLAN.md (Phase C
translator merge plan), docs/config_schema.md (TOML schema reference),
docs/audit_inventory.md (audit tier inventory).
- .gitignore: tools/_*.py convention for ad-hoc scratch probes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Recompiler — `recompiler/src/function_analysis.cpp` now stops a
function's end_addr at its mapped `jr $ra` + delay slot instead of
stretching through trailing data. Fixes Tomba's 0x80076DB0, a real
136-byte function that was being emitted as a psx_unknown_dispatch
stub because data after it poisoned the boundary heuristic.
`code_generator.cpp` emits `func_<next>(cpu); return;` as a tail
call when a mid-function seed split leaves the physical fall-through
in a different C function, plus a `debug_server_log_call_entry` at
every function entry. `full_function_emitter.cpp` splits dispatch
into `psx_dispatch_impl(cpu, addr, stop_addr)` with public
`psx_dispatch` and new `psx_dispatch_call(cpu, addr, return_addr)`
so JAL/JALR into RAM stubs can return cleanly without growing the
native stack.
Runtime — `runtime/src/dma.c`/`include/dma.h`/`src/memory.c` now
expose `dma_write_masked()`; the 16-bit and 8-bit mmio paths route
DICR/DPCR/CHCR partial writes through it so a partial DICR write no
longer accidentally clears unrelated channels' pending IRQ flags,
and CHCR only triggers DMA when the written mask actually includes
the start bit. Unified `complete_transfer(ch)` across all channels,
removed `prev_master_flag` edge detection.
MDEC (`runtime/src/mdec.c`, `include/mdec.h`) gains a full debug
surface: `MDECDebugState` snapshot, an always-on 4096-entry
`MDECDebugEvent` ring with 11 event kinds (reset, ctrl_write,
cmd_begin/done, decode_done, dma_in/out start/end, output_drained,
read_underflow), and `mdec_state` / `mdec_trace` / `mdec_trace_clear`
debug-server commands. DMA ch0/1 emit start/end events around MDEC
transfers.
CDROM (`runtime/src/cdrom.c`, `include/cdrom.h`) — read stream
separated from one-shot pending state (`read_cmd`, `read_delay`,
`start_read_stream`, `stop_read_stream`, `deliver_read_sector`);
`cdrom_state`/`cdrom_trace` report the new fields.
Thread/fiber tracing — `traps.c`'s Win32-fiber TCB save/restore now
emits 12 thread-event kinds; new `ThreadTraceEntry` ring (64 K) plus
`thread_trace` / `thread_trace_clear` debug-server commands report
both live CPU regs and the target TCB's full saved register block.
wtrace / mmio rings — `WriteTraceEntry` grew (cpu_pc, sp, v0/v1/a0-a3/
t0/t1); capacity 1 M -> 4 M. `MmioTraceEntry` grew (pc, cpu_pc, sp,
a0-a3, sr, epc, i_stat, i_mask). Both dump commands accept
`count`/`newest=1` for reverse iteration. `restore_trace` and
`dirty_block_log` emit extended register fields (a2/a3/sp).
Misc — `interrupts.c` polls debug-server every 16K checks so it
stays responsive without a separate thread. `psx_runtime.h`'s
`call_by_address` routes through `psx_dispatch_call`. Dirty-RAM
interp JAL/JALR consult `psx_dispatch_game_compiled` when the game
is wired in. iso_reader gains the entry points needed by the game
disc path. New `tools/_tomba_probe.py` for port-4470 health checks.
Generated — regenerated `SCPH1001_dispatch.c` against the new
`psx_dispatch_impl` shape.
ISSUES.md — opened Issue #3 (BIOS PS-logo glyph missing on the
disc-detected screen, cosmetic).
.gitignore — added `recompiler/build-asan*/`, `build-checked/`,
`__pycache__/`.
Validated end-to-end: BIOS boots, disc detected, Whoopee Camp FMV
plays with audio. Frame count advances through ~40K frames without
freezes. `unknown_dispatch_log.total = 0`. Remaining FMV cluster
issues (1/4-size render, FMV-2 audio missing, no skip) tracked
under TombaRecomp/ISSUES.md.
Co-Authored-By: Codex (OpenAI) <noreply@openai.com>
Add Pass 2.6 to FunctionAnalyzer: detect JAL targets whose callee starts
with `sw $ra, imm($a0)` (SaveState-style context-save helpers) and force
the post-delay-slot PC into the dispatch table. Recovers continuation
entries that aren't reached by JAL prologue or jr $ra discovery.
Extend find_function_start to recognize `lui rt; lw rt, off(rt)` 8 bytes
before jr $ra as a prologue marker. Rewrite analyze_function block-build
from std::set boundaries to a sorted std::vector + binary_search.
main_psx CLI: --seeds, --out-dir, --strict, --inspect; out_dir support.
Add psxrecomp-game executable target alongside PSXRecomp.
code_generator: reg_name() now emits cpu->gpr[N] consistently (matching
what the rest of the codegen already emitted — the old reg_names[] table
returning cpu->v0/cpu->ra was unreachable because CPUState has no named
fields, only gpr[32]). LWL/LWR/SWL/SWR moved from cpu->lwl(...) member
form to psx_lwl(cpu, ...) free-function form for game-EXE codegen reuse.
Remove a stray fmt::print left in build_basic_blocks that spammed
stdout on every regen — per the no-stdio-debug rule (memory v4).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kernel's inline RestoreState (at BFC1094C inside the exception
handler func_00000C80) restored all registers and did `jr $ra`, but
the generated `return;` went back to psx_check_interrupts instead of
dispatching to the restored $ra (0xBFC421D8 — the VSync callback
loop). Then psx_check_interrupts overwrote all GPRs from the
pre-exception save, wiping RestoreState's changes. VSync counter
was permanently stuck at 0.
Fix has three parts:
1. runtime/src/interrupts.c: Add psx_restore_state_escape() which
longjmps with code 2 (vs code 1 for ReturnFromException).
psx_check_interrupts now loops: code 2 re-dispatches to cpu->pc
(still in exception context), code 1 exits, code 0 is normal.
2. recompiler full_function_emitter: Broaden longjmp-return detection
from first-instruction-only to scanning ALL instructions for the
exact encoding 0x8C9F0000 (lw $ra, 0($a0)). This catches both
standalone RestoreState (func_1FC0227C) and the kernel inline
copy (func_00000C80 at BFC10964).
3. Generated jr $ra terminators in matched functions now emit:
cpu->pc = cpu->gpr[31]; psx_restore_state_escape(); return;
Result: VSync counter increments steadily (~256/vblank), zero crashes,
shell renders Memory Card / CD Player screen with no ADEL faults.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Oracle interpreter:
- JR and JALR were missing delay slot execution (exec_one recursive call),
causing the entire boot sequence to malfunction — C0 table never populated,
oracle stuck at ~65K instructions. Now boots to shell and VSync incrementer
hit 272 times in 100M instructions (counter = 271).
- Added NULL-PC halt trap (jr to address 0 = corrupt function pointer).
- Reworked oracle main loop: counts VSync hits, steps past breakpoints,
dumps trace ring correctly (ring-relative indices), reads VSync counter
and init flags at end of run.
Code generator (longjmp-return):
- RestoreState (A0:0x14) loads $ra from a save buffer via $a0, so jr $ra
must set cpu->pc for the dispatch loop to tail-call to the restored
address. Detected by: first instruction is `lw $ra, 0($a0)` (0x8C9F0000).
- Applied in both code_generator.cpp (block-based) and full_function_emitter.cpp
(linear emitter used by psxrecomp-bios.exe).
Dispatch seeds (+5 new, 832→837):
- 0xBFC421D8: RestoreState longjmp target (mid-function in func_1FC42160)
- 0xBFC4231C: SaveState v0=0 init body (callback table setup)
- 0xBFC42370: func_1FC42160 epilogue (init flag gate branch target)
- 0xBFC42910: post-callback cleanup (was dispatch miss)
- 0xBFC42BF0: shell_register_callback (callback table was empty without it)
Runtime survivability:
- psx_unaligned_access: survivable ADEL with diagnostic dump (was exit(1))
- DMA ch2 linked-list: break on cycle instead of exit(1)
- NULL dispatch (addr=0): silently absorb in psx_unknown_dispatch
Regenerated BIOS: 1056 dispatch entries (was 1051).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The BIOS shell code (ROM 0xBFC18000+) runs from RAM at 0x80030000+.
Jump tables in shell functions store RAM-space target addresses, but
at recompile time only the ROM data is available.
Three fixes:
1. function_discovery.cpp: detect jump table pattern (SLTIU/SLL/LUI/
ADDU/LW/JR) during function scanning, translate RAM table addresses
to ROM, push case targets onto BFS work queue so case body code is
included in the function's instruction set.
2. full_function_emitter.cpp: detect same pattern at jr emission time,
read table from ROM with RAM-to-ROM mapping, emit C switch/goto
instead of cpu->pc=reg;return for resolved tables.
3. code_generator.cpp: add ram_to_rom() helper for the existing jump
table detection (used by a different code path).
Result: 13 jump tables resolved across the BIOS, +6 functions discovered,
+2825 instructions included. Dispatch miss at 0x80058564 (memcard SIO
handler) eliminated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Matthew Stanley