mingw gcc invoked by absolute path (the runtime's cmd.exe autocompile spawn,
or any launch without mingw64/bin on PATH) fails SILENTLY — exit 1, empty
stderr — because cc1/as/ld and the runtime DLLs resolve via PATH. _toolchain_env
auto-detects the bin dir ($PSX_MINGW_BIN, the --gcc path's dir, `which gcc`, then
common locations) and prepends it for the gcc subprocess. Fixes overlay-DLL
autocompile so it works regardless of launch environment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual PSX_OVERLAY_CODEGEN_VER must be hand-bumped on every emitter change;
when it isn't (e.g. between CPS emitter iterations), a stale-but-cgN DLL is reused
-> the v0.3.0 black screen. Now hash_codegen.cmake hashes the recompiler codegen
sources into a (gitignored) overlay_codegen_hash.h at build time; overlay_api.h
exposes PSX_OVERLAY_CODEGEN_HASH and the cache path becomes gcc/<arch-abi>/cg<N>_<hash>/.
The loader and compile_overlays.py read the same value, so any emitter change
auto-invalidates the cache (old dirs coexist, no stale load). Falls back to hash 0
when the header is absent (fresh checkout) — identical to today's single-cgN path.
Verified: build emits hash 0x347751c1; autocompile + loader both use cg4_347751c1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Vulkan backend alongside software + OpenGL: geometry/textured-prim
pipelines, GPU-authoritative VRAM image, deferred-submit + drain-at-present
perf model, GPU-direct present (vk_renderer_present_vram), and a
full-VRAM-pack readback path (ensure_cpu rect_adds the whole VRAM rect
before pack_flush so screenshots/guest VRAM reads reflect live content,
not just the incremental pack-dirty rect — readback-only, no per-frame cost).
Runtime plumbing: --debug-port / --renderer / --window-title CLI overrides
(multi-instance fleet), latency_ring, SPIR-V shaders + embed_spirv.py,
config_loader renderer selection, debug_server screenshot wiring.
All on feat/vulkan-renderer; snapshot in backup/vulkan-wip-20260617.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SLJIT.md §7 step 1. gcc-compiled overlay DLLs now live under
<cache>/<game_id>/gcc/<arch-abi>/ so a same-OS different-arch DLL (and,
later, sljit blobs under sljit/<arch-abi>/) can never be confused for one
another. The backend-neutral coverage currency stays overlay_captures.json.
- overlay_loader.c: canonical PSX_OVERLAY_ARCH_ABI tag ("<os>-<arch>");
scan_cache_dir() refactored to scan the namespaced gcc dir AND the legacy
flat layout (<game_id>/*.dll) for back-compat — existing caches keep
loading with no migration. The legacy scan path is a provably-equivalent
refactor (same indexed paths as before).
- compile_overlays.py: cache_arch_abi() (lockstep with the C macro) writes
DLLs to the namespaced gcc dir; prior-ranges coverage merge falls back to
the legacy flat dir so pre-namespacing coverage still merges forward.
Verified: C and Python tags both resolve to win-x64 on this host; clean
rebuild; the 16 existing flat-layout Tomba DLLs remain findable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Observability for the 16:9 backdrop-cull investigation (runtime-only,
4:3-identity-safe, no regen):
- gpu.c: always-on per-prim draw-census ring (ws_census). Every drawn GP0
primitive (0x20-0x7F) records {frame, src_addr, camX/camY, first-vertex
screen x/y, opcode} so object spawn/despawn and edge culls are observable
in data instead of screenshots. gpu_ws_census_dump() writes a frame range
to CSV (large dumps must not ride TCP).
- gpu.c: gpu_ws_set_margin_override() forces psx_ws_x_margin() to any value
for a live cull sweep at a fixed camera; gpu_ws_get_debug() snapshots live
widescreen state.
- debug_server.c: gpu_state now reports ws.{configured,active,game_mode,
present_native_43,x_margin,squash,cur_frame,last_tag_frame}; new ws_margin
and ws_census TCP commands.
- gpu.h: GpuWsDebug + census/override API.
- debug_client.py: ws_margin and ws_census subcommands.
Used to prove the shipped [widescreen.cull] sites are a no-op for 8C (their
4:3 margins already cover the 16:9 view) and that the real pop is an
overlay-resident backdrop layer. See TombaRecomp ISSUES.md #8C.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Overlay coverage (overlay_captures.json + the gcc-compiled cache DLLs) normally
lives only in gitignored build dirs, so a `cmake` clean or fresh worktree wipes
it (re-derivable only by replaying those areas). coverage_vault.py maintains one
additive vault OUTSIDE any build dir: `merge` unions captures by variant
(load_addr + bytes hash; same variant unions executed/entry PCs, distinct
variants all kept) and copies content-keyed cache DLLs (copy-if-absent/newer).
Wired into Tomba's overlay_autocompile_cmd so every compile backs up additively;
also runnable by hand to consolidate scattered build dirs. The vault holds
game-derived bytes, so it stays private/gitignored (this script does not).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The overlay cache keys DLLs by guest-bytes CRC, which is flavor-blind — the
same guest overlay yields the same filename whether compiled by the base
recompiler or a variant (e.g. widescreen, which emits psx_ws_*/GTE-adjust
symbols). Two flavors sharing a cache dir would collide: a base runtime could
pick up a widescreen DLL (or vice versa) with no guard, since both report ABI
v2.
Fold a codegen-flavor id into overlay_abi() (high 16 bits) and check the
combined tag in the loader's existing ABI gate. Base/master flavor is 0, so the
tag stays == PSX_OVERLAY_ABI_VERSION — fully backward compatible with every
existing base DLL (verified: an 8-DLL base cache loads with zero rejections).
A variant build sets PSX_OVERLAY_FLAVOR (e.g. widescreen: -DPSX_OVERLAY_FLAVOR=1
for the runtime, --flavor 1 for compile_overlays.py); its overlays then report a
distinct tag, so the base runtime rejects+regenerates them and vice versa. The
caches stay separate automatically, even in a shared directory — no new paths.
overlay_api.h: PSX_OVERLAY_FLAVOR (default 0) + PSX_OVERLAY_ABI_TAG.
overlay_loader.c: both ABI gates check the tag.
compile_overlays.py: --flavor arg -> -DPSX_OVERLAY_FLAVOR, baked into overlay_abi().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-player input routing with a real device dropdown (None / Keyboard /
each plugged-in SDL controller) and a digital-vs-DualShock pad-type toggle,
wired end-to-end through the runtime.
- sio.c/.h: per-slot pad state (pad_buttons[2]) + per-slot pad type. Analog
poll emulation (id 0x73 + 4 stick axes) when a slot is in DualShock mode;
digital (0x41) stays the default so existing games are unaffected. New
setters: sio_set_pad_state_slot / sio_set_pad_analog / sio_set_pad_connected,
getters sio_get_pad_{buttons,connected,analog}_slot.
- config_loader: settings.toml [controller] p1_device/p2_device (none/keyboard/
<SDL GUID>) + p1_analog/p2_analog; load + save round-trip.
- main.cpp: replaced the single auto-opened controller with a two-slot
PlayerInput model. Devices opened by GUID (with a first-free fallback so a
re-paired pad still works), keyboard or controller feeds each PSX port, and
analog stick axes drive the DualShock poll. Hot-plug re-resolves both slots.
Defaults: P1 keyboard/digital, P2 none.
- launcher: enumerates SDL controllers (name+GUID); each player card has a
custom click-to-open dropdown (option list injected via data-rml, the
data-for path being unreliable in this RmlUi build) and a DualShock switch
that drives the pad art (digital/analog) + status line. Zero-config default
selects the first plugged-in pad. A saved-but-unplugged controller is kept
as an "(offline)" option so the selection survives. Choices persist on launch.
- debug_server: pad_status now reports both slots' buttons + connected + analog.
UI polish from review:
- Removed the redundant disc-verdict widget (the green checks already convey it);
Change ISO now anchors to the disc panel instead of floating at the page top.
- Added a prominent "Settings" button in the top bar (renderer/SS/AA/etc. were
only reachable via the easily-missed footer link).
- Fixed the player-card asymmetry on toggle: status text no longer wraps and the
player card body is a fixed height, so the pad art renders at a consistent size
(measured 216x110 analog vs 219x110 digital) regardless of pad type.
Verified: device dropdown enumerates a real DualSense; selection + DualShock
toggle update the card; pad_status confirms per-slot connect + analog routing
(slot0 analog / slot1 digital and the reverse); default keyboard P1 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the procedural placeholder art (disc/controllers/memory card/
logo) with the real PlayStation hardware cropped straight out of the
reference mockup, with the dark background knocked out to transparent so
each asset blends seamlessly on the panels — no seams, no border-glow
artifacts.
tools/crop_launcher_assets.ps1: crops the five assets from the mockup
and removes the background. Photo assets use an edge flood-fill (flood
the dark background inward from the borders; bright object pixels are
walls, so interior dark detail — the disc's PlayStation logo and centre
hole, the controller buttons — is preserved). The logo uses a global
luminance key (thin bright glyphs on near-black). PowerShell [int] of a
division rounds, not floors — the flood uses [Math]::Floor for the row
index.
launcher.rml: add `contain` fit to the photo decorators so the crops
keep their aspect ratio, and nudge the art sizes closer to the mockup's
proportions (disc 216, pads 206x128, memcard 118).
The check/verdict icons remain the generated PNGs (already clean +
transparent). Pixel-verified: dashboard renders with authentic disc,
PS1 digital pad, DualShock, and memory cards, all seamless on the panels.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebuild the launcher as a card-based dashboard (n64recomp-style) per the
reference design, replacing the flat settings list.
Image pipeline: RenderInterface_GL3 only decodes TGA, so subclass it
(LauncherRenderInterface) and override LoadTexture to decode PNG via a
vendored stb_image (third_party/stb_image.h, STBI_ONLY_PNG, separate
impl TU), falling back to the base TGA path. This is done without
touching the pinned RmlUi submodule. RCSS now uses decorator:image(...)
for all art.
Layout (launcher.rml):
- Top bar: PS-glyph logo + "<game> Recomp".
- Disc Verification panel (Phase 3 — fully wired): disc art, FILE /
REGION / SERIAL, a three-check VERIFICATION RESULTS row
(Header Match / CRC-Hash Match / Disc Verified, driven by the
DiscIdentity fields) and a colour-coded verdict (icon + title +
detail). "Change ISO" re-runs verification.
- Player 1 / 2 and Memory Card 1 / 2 cards: full visual shell with
live enable toggles; controller enumeration (Phase 5) and memcard
introspection (Phase 4) wire in later — the rest is placeholder.
- Settings view (footer "Settings" <-> "Back"): the video / audio /
system controls moved off the home screen.
- Footer: Settings, recompiler-ready status, LAUNCH GAME.
launcher.cpp: model gains disc_file/region/serial + v_header/v_crc/
v_verified + verdict_{title,detail,state}, a view toggle, and card
enable bools; refresh_disc_status maps DiscIdentity onto the three
checks and the verdict; new events change_iso / show_settings /
show_dashboard / toggle_p1.p2.mc1.mc2.
Window bumped to 1280x924 for the dashboard. tools/gen_launcher_assets.ps1
generates the placeholder art (disc, controllers, memory card, PS-glyph
logo, check/verdict icons) into assets/img — swap for real renders later;
the PNG pipeline is identical.
Verified end-to-end on Tomba: dashboard renders with all three checks
green + "Tomba! disc verified"; Settings view and Back work; player /
memcard cards and toggles render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a shared disc-identity module (runtime/{include,src}/disc_identity)
that opens a .cue/.bin/.iso, checks for the ISO9660 PVD (raw 2352 and
cooked 2048 layouts), reads the PVD volume id, parses the boot serial
from SYSTEM.CNF, derives the region from the serial prefix, and
optionally streams a full-file CRC32 and compares against expected
serial/CRC. This is now the single source of truth for "is this the
right disc?" — main.cpp's launch-time validate_disc_image is rewritten
to delegate to it (dropping its duplicated raw-read logic and the now
dead cue_data_path/is_iso_pvd/psx_exe_id helpers) so the launch check
and the launcher badge can never drift.
game.toml [game] gains optional disc_crc / disc_sha1 (GameConfig +
config_loader). Left unset by default: the serial check is dump-
agnostic, whereas a baked CRC would flag every legitimate dump of a
different revision/region.
Launcher: GameInfo carries the expected serial + CRC; the launcher
recomputes verification on entry and on every Browse, showing a
colour-coded badge (Serial verified / Disc verified / CRC mismatch /
Wrong game? / Not a PlayStation disc) plus a detail line with the
detected serial · region · CRC. CRC is only hashed when an expected
CRC is configured.
Verified all three tiers on Tomba: Serial verified (no CRC),
CRC mismatch (wrong CRC), Disc verified (CRC B00ECB0D match).
tools/shot_launcher.ps1: bring the launcher GL window to front and
screenshot its client area (GL windows screenshot black via PrintWindow).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.
The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).
- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
bail aborts the interp run; tail transfers into compiled code now run
with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
headers; anomaly guard at exception entry; TCP_COMMANDS.md docs
Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- debug_server.c: dump_ram = alias of read_ram (one request, one response,
up to full 2 MB); screenshot = screenshot_file (24-bit BMP, optional
"path", default psx_screenshot.bmp); vram_peek cap 64->128; read_ram hex
encode via nibble table (was 2M snprintf calls stalling the main thread);
send_all_blocking bounded (1 MB chunks, SO_SNDTIMEO 2s, heartbeats the
starvation watchdog per chunk) and drops the client on send failure
instead of letting the watchdog kill the runtime; handle_quit tags exit
origin.
- beetle_debug_server.c: read_ram cap 64 KB->2 MB via heap+send_raw;
dump_ram/screenshot aliases; screenshot w/h->width/height parity;
set_input now holds until clear_input (frames param optional) matching
psx-runtime semantics — the old default expired after 1 frame and the
off-by-one in the getter cleared the override before it was ever read,
so scripted beetle navigation silently did nothing; wtrace_dump response
heap-built + send_raw (per-entry send_fmt dropped chunks on multi-MB
responses, producing broken JSON).
- crash_trace: new psx_crash_trace_set_exit_origin(); report gains
exit_origin (tcp_quit / sdl_window_close / unknown).
- debug_client.py: O(n) string-aware brace parser (old per-byte loop was
O(n^2) — minutes on a 2 MB response, which starved the server's send and
got the runtime killed); generic key=value passthrough for any server
command without a bespoke CLI mapping.
- TCP_COMMANDS.md updated for all of the above.
Verified: 2 MB read_ram round-trip 0.75 s; scripted beetle navigation
reaches the memcard file-select; wtrace_dump returns valid JSON at 21 MB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runtime detects an uncovered overlay/kernel variant by sustained dirty-RAM
interp pressure inside the capture windows (g_dirty_window_dispatches,
sampled every 600 vblanks; fires only at a coherent non-loading moment via
cdrom_load_in_progress, 60s cooldown, 16/session cap), writes the capture
JSON, and spawns the game.toml-configured compile command
([runtime] overlay_autocompile_cmd, cmd.exe /C, cwd=project root, child
output kept in an in-memory ring — no log files). On child exit 0 the emu
thread rescans the cache (overlay_loader_rescan: idempotent dir re-scan +
checked-regions reset) so the new DLL loads on the next dispatch and the
variant goes native in-session. Dedup is tool-side (existing-CRC skip);
pressure drops to zero once code is native, so the loop self-limits.
New TCP: autocompile_status / autocompile_run / overlay_rescan.
compile_dll now uses absolute native paths and reports exit codes.
VERIFIED live (user-played route): 2 auto-triggers fired, captures written,
child ran recompiler+audit (new variants 1E7DDE2C, A62D0A7C discovered and
audit-clean), rescan picked the compiled DLL up in-session. OPEN ISSUE: gcc
exits 1 with no output inside the spawned cmd.exe child (same absolute-path
command succeeds from an interactive shell) — compile step currently
completed manually; suspect child-env difference, to be isolated next
session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the overlay JIT cache below the 0x98000 floor to kernel RAM
[0, 0x10000) — the BIOS part-2 image + install-at-runtime stubs
(Rule 18), dominated by the SIO data-byte slot at 0xCF0 whose
install-slot hook tail-dispatches the whole exception-handler tail
into the interpreter on EVERY exception once installed.
Window model (dirty_ram_interp.h): two capture/candidate windows —
kernel [0, 0x10000) and overlay [0x98000, RAM); main-EXE text between
them is statically recompiled, never captured. Interp local-flow gates
intentionally stay overlay-only (kernel runs per-block in exception
context; named constant documents this).
- overlay_capture: per-window dirty-run enumeration, runs clamped at
window edges (kernel CPU-write pages and main-EXE DMA pages can be
adjacent; never merge them).
- overlay_loader: dispatch lazy-load gate covers both windows;
scan_cache_dir validates the <addr8>_<crc8>.dll shape explicitly
(kernel region_start 0 is legal, zero-parse is no longer a sentinel).
- classifier (compile_overlays.py): kernel-window-only DISPATCH_ROOT
promotion — an orphan dispatch interior no rooted walk covers has no
host to truncate and is a real execution root by construction (the
static install-slot hooks tail-dispatch into it). One orphan per
iteration so remaining orphans alias into the new host. Overlay
regions stay ineligible (cross-variant per-PC evidence: 0xE889C class).
- main_psx --overlay: 'dispatch_root' seed lines are trusted walk roots
(exempt from the boundary re-check); seed range is now the loaded
image's own bounds (the old hardcoded [0x80010000,0x80200000) window
silently dropped kernel seeds).
- overlay_api/loader/DLL preamble: route psx_restore_state_escape
through OverlayCallbacks (appended last, by-value copy keeps old
DLLs compatible) — exception-context kernel code longjmp-returns.
Tomba verified (user-played: title -> LOAD list renders -> save loads
-> overworld laps): kernel interp per-PC rows 3 -> 0 (was 0xCF0 =
61,775 dispatches / 2.71M insns at the LOAD screen alone, ~4.6M
insns/session); host func [0xCF0,0xEA0) + 0xDF8/0xE08 aliases, audit
clean; dispatch_native 357,763; dispatch_miss_total 0; interp aborts 0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two pieces from the variant-coverage investigation:
1. dirty_ram_stats per_pc rows gain an `entries` counter: dispatches
that arrived from NATIVE code (call / dispatch loop), distinguished
from the interpreter's own block-to-block chaining via a consumed
chain-target latch. `hits` conflates the two — below the local-flow
floor every taken branch re-dispatches, so chain blocks dominate and
misattribute hot families. First use REFUTED the main-EXE-interior-
entry hypothesis: the 0x1DFF8/0x1E0D8 family (~1M insns) has ZERO
external entries — it is purely downstream of overlay interp runs,
and collapsed to 57K insns once the overlay variant was compiled.
No main-EXE aliasing needed.
2. compile_overlays.py merges prior-manifest evidence on rebuild: every
F entry in an existing .ranges for the SAME image bytes was proven
compilable, so a fresh (poorer) capture cannot regress coverage.
Callable entries re-enter as captured function entries; non-callable
ones (alias wrappers from a prior build) re-enter as dispatch entries
so the classifier re-derives their interior disposition — feeding
them back as walk roots would truncate their hosts.
Village variant DLL (000E7000_9707AAB5) force-rebuilt with merged
evidence: 195 -> 244 functions incl. interior aliases. Verified on a
user-played village+overworld lap: 0x12A60C (~1M interp insns),
0x138084, 0xED2F4 now native and absent from per_pc; 0 dispatch misses,
0 aborts, 0 invalidations.
Remaining interp load is (a) the variant long tail — the same overlay
region holds scene-dependent variants where one variant's code address
is another variant's DATA (0x800E889C: code in the overworld-family
image, a pointer table in the village image); per-entry hashing
correctly refuses stale native code, and each variant gets covered when
captured — and (b) the kernel-RAM SIO stub at 0xCF0 (~4.6M insns/
session), a separate class.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Coverage campaign step 2.5 (classifier follow-up to step 2's alias
machinery). Hot overlay functions reached only through function-pointer
dispatch were excluded by classify_overlay_seeds (OBSERVED_PC_ONLY /
NO_ENTRY_BOUNDARY / BRANCH_TARGET_ONLY) and ran interpreted forever —
0x800E7C80/0x800E7C8C alone were ~14M de-facto interp insns/session.
compile_overlays.py: dispatch_entry_pcs that fail the callable-boundary
gate (or sit on a jump-table label) are now included as
DISPATCH_INTERIOR instead of excluded, and written to the seeds file as
'interior 0x...' lines. They are never walk roots — as roots they would
hard-cap (truncate) the sibling walk that owns them. entry_pc for the
fake PS-EXE header comes from the first ROOT seed.
main_psx --overlay: partitions seeds (interior-marked never roots;
unmarked re-verified against the same boundary rule as defense in
depth), walks the evidence root set, and attaches interior candidates
to their containing functions as overlapping-alias groups via the
step-2 emitter. Orphans (no covering function) stay with the
interpreter — an earlier backward-boundary-promotion attempt minted a
false root from data (0x800EC194) and failed the generated-C audit with
393 data-as-code TODOs; promotion was dropped, precision-first.
Tomba fresh overworld capture (000E7000_5D263ADE): 49 roots + 40
interior -> 67 compiled functions (26 alias wrappers, 14 orphans),
audit clean (0 TODOs, 0 unknown-bad). Verified on user-played overworld
with transitions: all four hot PCs gone from dirty_ram_stats per-PC,
796K native overlay dispatches, 0 invalidations, 0 dispatch misses,
0 interp aborts. Next dominant interp class is now main-EXE interior
entries via runtime-computed pointers (0x1DFF8/0x1E0D8 family, ~3.9M
insns) — needs runtime entry-PC capture evidence, not static scanning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inline multi-MB dumps wedge the single-threaded debug server (third
occurrence: dirty_block_log count=16384). Block ring joins the insn ring
and event ring in having a file-dump path; client plumbing included.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A bare `import tomllib` ahead of the try/except made the tomli fallback
unreachable on Python <3.11. Surfaced while compiling the overworld overlay
capture; with the fix, the dwarf-overworld variant (000E7000_6E1A219B, 115
functions, 0 unknown targets) compiled cleanly and the route's interpreter
load collapsed from 673M insns (99% overlay) to ~45M (main-EXE only),
valid candidates 14 -> 46.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the dwarf-village->overworld native-overlay blue screen
(deterministic, every run, at native call #70):
The dirty-RAM interpreter's jal/jalr handlers preserved the call's
return-address obligation for statically-compiled callees
(psx_dispatch_game_compiled -> *next_pc_out = return_pc) and for non-local
targets (dispatch_nonlocal_call passes return_pc as stop_addr), but a LOCAL
DIRTY target fell through to a bare `cpu->pc = target; return 1`. When that
target was a NATIVE overlay candidate, the dispatch loop ran it as a C
function; its pc==0 return made psx_dispatch_impl unwind to the C caller,
silently dropping the suspended interpreted caller's continuation.
Concrete case: main-EXE field dispatcher F=0x800338A8 (undiscovered entry —
discovery walked it from 0x338B0; reached via data-table jalr, so it runs
under the interpreter) does `jal 0x8011B0D4` per entity. Under native exec
F's epilogue never ran: guest sp leaked 0x18 per entity until the world
corrupted. Interp-only mode is immune (the callee's own jr ra is interpreted).
Evidence chain (full-register fingerprint diff ON vs OFF localized first
divergence to sp at exec #70; frozen dirty-block ring shows F re-entered at
sp-0x18 with no epilogue between) is in docs §9 and the session memory.
Fix: overlay_loader_call_native() — run a validated native candidate as a
unit and resume at return_pc — called from both jal and jalr handlers,
mirroring the psx_dispatch_game_compiled contract exactly. Side effect:
native overlay coverage rose from 86 calls per village->overworld run to
~112,000 (interpreted calls now route to native candidates).
Verified: dwarf->overworld renders and plays under native overlays.
Same-scene A/B: host CPU 7.31 (native) vs 7.83 (interp) CPU-s per 15s wall.
Timing hypotheses refuted by experiment before fixing: IRQ-check cadence
(suppress armed, identical failure) and device-event granularity
(conservative stepping, identical failure).
Also in this commit (the capture tooling that found it):
- fingerprint ring now stores full entry/exit register files + guest cycle
(FP_CAP 2^20 crc-only -> 2^16 full-reg); overlay_fp_dump takes a path.
- insn_freeze <addr> <nth>: one-shot latch freezing insn/block/flow/fntrace
rings immediately BEFORE the Nth dispatch of a watched candidate, so the
pre-divergence window survives until dumped (ring-first, no arm-then-hope).
- dirty_insn_gate <lo> <hi>: runtime-settable extra PC range for the per-insn
interp log; native entry/exit markers (transferred=200/201) in that ring.
- dirty_insn_dump_file: file dump for the insn ring — the 16MB inline TCP
dump wedged the single-threaded debug server.
- freeze gating in the flow-log path gates ONLY the ring write, never flow.
- debug_client plumbing for the new commands + fntrace_arm/fntrace_dump.
- docs/overlay-recompilation-design.md §9: the dispatch call-contract
invariant, the Bug A case study, verification numbers, and known residue
(data-jump-table discovery-miss class; one unreproduced black-screen
one-off; §8.3 tail-position caveat).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Accumulated working-tree state for the overlay-JIT-cache phase, committed as a
checkpoint before the Bug A fix (next commit):
- event_ring.{c,h}: always-on unified event-timeline ring (IRQ deliver/gate/
raise, I_STAT changes, DMA kick/sched/done, ENQ/DEQ per source, exec-mode
tagged). Edge-recorded, eviction-bounded, survives a blue screen; dump via
event_ring_dump/_tail/_clear.
- cdrom.c: disc_speed="instant" no longer collapses every sector read to the
500cy floor (which flooded ~1100 DMA3 completions/frame and starved main
code). CDROM_INSTANT_PERIOD bounds sector IRQs to CDROM_INSTANT_MAX_PER_FRAME
(32)/frame. Known caveat: a transient ~1-2s reentry storm at boot frame ~210
still trips the watchdog once, then recovers.
- interrupts.c: claimed-IRQ cooldown is now per-source — SIO stays 0 (card
reads need back-to-back IRQs); DMA/VBLANK/timer get CLAIMED_PROGRESS_QUANTUM
(8) blocks of guaranteed main-code progress between deliveries. Root cause of
the instant-disc freeze: cooldown=0 let the block leader re-take the
exception before its body ran (pinned at the 0x8005A1D8 field loop).
- gpu/dma/memory/sio/timers/psx_cycles: event-ring instrumentation at raise/
kick/complete/schedule sites; g_event_step_conservative (single-cycle event
stepping A/B used to refute the event-granularity hypothesis for Bug A).
- overlay_capture.c, overlay_loader.h, crc32: capture/manifest plumbing for
per-entry validity (Inc3).
- recompiler: overlay-mode emission updates (code_generator, main_psx).
- tools: compile_overlays.py audit/patch updates; tomba_loadtest.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A dispatched-to PC proves code reachability, not a callable function boundary.
Bare DISPATCH_ENTRY seeds were promoted to functions on first-word validity
alone, so jr-driven jump-table case labels and data-table addresses became
"functions" whose walk ran off into adjacent data tables -> 42 reserved-opcode
"UNSUPPORTED" + 8 bad targets across 3 false functions (800E9438/9EB8/9EF8),
which failed the region audit and emitted zero DLLs.
Fix: require real boundary evidence (jr $ra-preceded, prologue, or region
start -- the existing _callable_legacy_seed / vet_seeds heuristic) before
promoting a bare dispatch entry. Call-edge-proven reasons (DIRECT_JAL_TARGET,
FUNCTION_POINTER_TARGET, TOML_DECLARED_ENTRY) are exempt. Unproven dispatch
entries are left to the interpreter (self-heal via recapture).
Result on the Tomba 0xE7000 overlay: 122 -> 88 functions, 42 -> 0 unsupported,
8 -> 0 bad targets; a clean 274 KB DLL now emits. Also: always save the debug
<crc>_patched.c (even on audit failure) for opcode/boundary classification.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restores entry-scoped overlay analysis (analyze_exact_entries +
split_mid_function_targets=false) as the MANDATORY mode for overlay inputs,
renamed --overlay-exact -> --overlay: the ingester's contract, set
unconditionally by compile_overlays.py for every overlay, never a human toggle.
Evidence-scoped discovery compiles only proven seeds + reachable code and keeps
branch/jump-table targets as in-parent labels — no whole-byte sweep, so it no
longer decodes embedded data tables as code.
Effect on the captured Tomba overlay: the data-as-code explosion collapses from
thousands of UNSUPPORTED to 122 clean functions with 42 residual unsupported
opcodes + 8 bad targets. Those residuals are now a per-function-audit concern
(Inc1-B), not a wholesale region failure. Main-EXE codegen is unchanged (Tomba
still regenerates 2854 functions; split pre-pass still runs in non-overlay mode).
Also adds docs/overlay-recompilation-design.md — the converged design of record
(precision-over-recall; compiled!=registered!=callable; write-invalidation over
CRC; internal-labels-in-parent; the RAM-write-path audit result; the three
implementation increments).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rips out the --overlay-exact recompiler path (analyze_exact_entries,
split_mid_function_targets, the CLI flag, and the compile_overlays.py flag),
returning the recompiler to master's naive whole-blob analysis. Overlay
compilation now splits mid-function branch targets into standalone functions —
the original behavior that produced black-on-load / blue-on-new-game — so we
can faithfully reproduce the failure before fixing it.
Pivot in approach: rather than excluding mid-function seeds at scan time, the
plan is to capture everything (the full overlay image is already dumped to
overlay_captures.json) and have the ingester detect each seed's containing
parent function at encode time and merge/ignore mid-function seeds, while
flagging entries reached only via computed/indirect jumps as jump-risk.
The runtime overlay loader/capture system (b2929aa) is unchanged. The 5
recompiler files are restored byte-exact to master (d22c9ce); config_loader's
overlay_cache flag is retained.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.
Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).
Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
targets and set split_mid_function_targets=false, so overlay basic-block
targets are NOT split into bogus standalone functions (the mid-function-seed
failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
mid-function split pre-pass behind split_mid_function_targets (default true,
so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).
Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).
Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A default MinGW build dynamically imports three non-system DLLs
(SDL2.dll, libgcc_s_seh-1.dll, libstdc++-6.dll) that get bundled
side-by-side. On a user machine with a mismatched-architecture copy
of one of those earlier on the DLL search path, Windows loads the
wrong one and the process dies at launch with 0xc000007b
(STATUS_INVALID_IMAGE_FORMAT) before any of our code runs.
Add PSX_STATIC_RUNTIME (default ON for MinGW Release/MinSizeRel):
link -static -static-libgcc -static-libstdc++ and pull SDL2 from the
pkg-config --static line (libSDL2.a + its Windows system-lib chain)
instead of the import lib. The resulting exe imports only Windows
system DLLs, so it runs from any folder and 0xc000007b from a
mismatched runtime DLL becomes structurally impossible.
package_release.ps1 no longer bundles the runtime DLLs and instead
asserts self-containment via objdump (fails the build if any
non-system import remains).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The CD DMA hook (overlay_log.c) could not distinguish code overlays from
FMV streaming data — Tomba buffers FMV through the same physical address
range and DMA channel. Ripped it out entirely.
Replace with overlay_dump TCP command in debug_server.c: scans the
dirty_ram bitmap for contiguous dirty pages above a configurable physical
threshold (default 0x98000), writes <crc32>.bin files to a caller-supplied
directory, and returns a JSON manifest. dirty_ram IS the oracle for
"executed as code" — no false positives from data transfers.
tools/dump_overlays.py queries overlay_dump post-session and appends new
entries to logs/<game_id>/overlay_map.jsonl with (crc32, load_addr, size,
bytes_file). First run on Tomba captured 4 regions (8KB, 264KB, 356KB,
748KB) matching the known dirty_ram miss clusters.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
All rejected addresses now printed in the summary with their boundary
values, making false-reject investigation easy without re-running.
Boundary check now accepts jr $ra + any delay slot (not just nop), since
compilers often put a useful instruction in the delay slot. Non-nop delay
slots are noted in the verdict string for manual awareness.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Reads dirty_ram miss addresses and checks each against the game EXE
bytes directly (no Ghidra required). Two checks:
1. BOUNDARY: 8 bytes before the address must be jr $ra + nop.
Fail -> REJECT (mid-function).
2. PROLOGUE: first 4 instructions must include addiu $sp,$sp,-N.
Fail -> WARN (frameless: leaf or frame-borrower like 0x8004DF00,
which crashed on new-game load because it pops a frame it didn't
push, corrupting the stack when called from the wrong context).
Only ACCEPT seeds are safe to add to ghidra_funcs.txt without manual
Ghidra verification. WARN seeds require caller analysis first.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
docs/disc-speed.md: design doc for per-game disc speed configuration.
docs/overlay-discovery.md: design doc for B+A overlay pre-compilation
and runtime JIT caching (game-side, future work).
tools/dump_tomba_misses.py: one-shot dump of dirty_ram per_pc entries
in the game text range, formatted as seed lines.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
SIO handler alias (0x0CF0 → func_0000641C):
Runtime capture showed BIOS installs lui+addiu+jalr to RAM 0x641C at 0x0CF0.
New [[recompiler.bios_aliases]] TOML mechanism emits a one-liner wrapper and
registers it in the dispatch table. Eliminates 30K dirty_ram blocks/session.
Instant disc speed (disc_speed = "instant" in game.toml [runtime]):
cdrom_set_speed(0) collapses all seek/read delays to 1 cycle. Correct INT
sequence still fires; no artificial wait. "2x" and "4x" divisors available.
Default remains authentic 1x. Tomba set to instant.
Persistent game miss collector (tools/collect_game_misses.py):
Appends newly-seen dirty_ram dispatch misses in the game text range to a
file (TombaRecomp/seeds/dirty_ram_misses.txt) for later seeding.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Runtime measurement showed this function (RAM 0x641C) and its internal
branch targets (0x6430, 0x6444, 0x6458, 0x650C, 0x6514, 0x658C, 0x6594)
collectively generating ~50K dirty_ram_dispatch blocks/session. The address
is confirmed as a true function start via jalr call site at 0xBFC15F10.
Seeding the entry point only; CFG analysis discovers all internal labels.
Result: blocks_run dropped from 119,804 to 29,692 (75% reduction).
Also adds tools/probe_dirty_ram.py for live dirty_ram measurement via the
TCP debug server.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
The "v4" suffix was a holdover from iterative project restarts.
There is now one psxrecomp; all identifiers, include guards, cmake
variables, function names, log strings, schema tags, and tool paths
drop the v4 suffix.
Changes:
- PSXRECOMP_V4_ROOT → PSXRECOMP_ROOT (cmake)
- PSXRECOMP_V4_RUNTIME_SOURCES/_INCLUDE_DIRS/_BIOS_GENERATED → no V4
- psxrecomp_v4_add_runtime_target → psxrecomp_add_runtime_target
- PSXRECOMP_V4_*_H include guards → PSXRECOMP_*_H
- "psxrecomp-v4" log/window strings → "psxrecomp"
- Schema version strings in recompiler JSON output
- Tool script hardcoded absolute paths updated
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two distinct bugs were causing OPTIONS-black hard freezes after 12K-65K
frames:
1. interrupts.c + traps.c: exception_jmpbuf is a single global. setjmp
captures the running fiber, but longjmp may be requested from a
different fiber via psx_exception_longjmp / psx_restore_state_escape
during chain-handler dispatch. Cross-fiber longjmp restores RSP to
the wrong stack while Win32 still tracks the calling fiber, corrupting
fiber state until SwitchToFiber eventually hangs.
Fix: track g_exception_owner_fiber at setjmp time. New
deferred_exception_longjmp checks the current fiber; if it's not the
owner, sets a pending flag and SwitchToFiber's back to the owner.
psx_change_thread_fiber's post-SwitchToFiber path observes the flag
and executes the longjmp on the correct stack.
2. gpu.c: gpu_read_gpustat fired gpu_vblank_tick() after 1000 polls,
which invoked sdl_vblank_present (SDL_Delay -> Sleep) from inside
an MMIO read code path. That meant recompiled MIPS spent wall time
in Sleep deep inside the fiber call tree -- confusing to reason
about and a contributor to fiber/longjmp interactions.
Fix: synthetic VBLANK from a poll-storm only updates emulation state
(LCF + I_STAT). The host-side render+sleep happens only at the proper
cycle-paced VBLANK trigger in psx_check_interrupts.
Also adds: dirty_block ring slice to the freeze auto-dump, Win32 main-
thread stack capture for wedge_kind=1 hard freezes (SuspendThread +
StackWalk64 + SymFromAddr), and dbghelp link.
Validated: TombaRecomp psx-runtime ran 30+ minutes / 105K+ frames at
OPTIONS with no hard freeze. Previous freezes consistently hit at
11900, 20493, and 64596 frames.
Diagnostic tooling under tools/_options_session/ that produced this
diagnosis is included.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Config loader (recompiler/src/config_loader.{h,cpp}): single source of
truth for both BIOS and game runtime settings (debug_port, window_title,
memcard_dir, disc path). psxrecomp-bios and psxrecomp-game both accept
--config <toml>; runtime takes --game <toml> (no more CLI overrides).
- runtime/src/main.cpp: replaced --disc/--memcard-dir/--game-root CLI
overrides with --game <toml>; game.toml is the single source of truth.
Discless launch (no args) goes to BIOS shell.
- runtime/src/dma.c: fixed update_master_irq() re-asserting I_STAT bit 3
on every DICR read. BIOS shell DMA handler at 0x80058628 only clears
DICR bit 27 (ch3 flag), so level-sensitive re-assert pinned the kernel
in the exception loop. Bit 3 is now edge-triggered (one assert per DMA
completion). Discless boot now reaches MAIN MENU / MEMORY CARD / CD
PLAYER. With-disc Tomba boots through to Whoopee Camp splash.
- Audit tooling: codegen_audit.py (BIOS), codegen_audit_game.py (game),
audit_config.py (config loader for tools), verify_audit_schema.py.
Replaces older tools/_codegen_audit.py probe.
- Docs: CURRENT_STATE.md (project state), UNIFICATION_PLAN.md (Phase C
translator merge plan), docs/config_schema.md (TOML schema reference),
docs/audit_inventory.md (audit tier inventory).
- .gitignore: tools/_*.py convention for ad-hoc scratch probes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Recompiler — `recompiler/src/function_analysis.cpp` now stops a
function's end_addr at its mapped `jr $ra` + delay slot instead of
stretching through trailing data. Fixes Tomba's 0x80076DB0, a real
136-byte function that was being emitted as a psx_unknown_dispatch
stub because data after it poisoned the boundary heuristic.
`code_generator.cpp` emits `func_<next>(cpu); return;` as a tail
call when a mid-function seed split leaves the physical fall-through
in a different C function, plus a `debug_server_log_call_entry` at
every function entry. `full_function_emitter.cpp` splits dispatch
into `psx_dispatch_impl(cpu, addr, stop_addr)` with public
`psx_dispatch` and new `psx_dispatch_call(cpu, addr, return_addr)`
so JAL/JALR into RAM stubs can return cleanly without growing the
native stack.
Runtime — `runtime/src/dma.c`/`include/dma.h`/`src/memory.c` now
expose `dma_write_masked()`; the 16-bit and 8-bit mmio paths route
DICR/DPCR/CHCR partial writes through it so a partial DICR write no
longer accidentally clears unrelated channels' pending IRQ flags,
and CHCR only triggers DMA when the written mask actually includes
the start bit. Unified `complete_transfer(ch)` across all channels,
removed `prev_master_flag` edge detection.
MDEC (`runtime/src/mdec.c`, `include/mdec.h`) gains a full debug
surface: `MDECDebugState` snapshot, an always-on 4096-entry
`MDECDebugEvent` ring with 11 event kinds (reset, ctrl_write,
cmd_begin/done, decode_done, dma_in/out start/end, output_drained,
read_underflow), and `mdec_state` / `mdec_trace` / `mdec_trace_clear`
debug-server commands. DMA ch0/1 emit start/end events around MDEC
transfers.
CDROM (`runtime/src/cdrom.c`, `include/cdrom.h`) — read stream
separated from one-shot pending state (`read_cmd`, `read_delay`,
`start_read_stream`, `stop_read_stream`, `deliver_read_sector`);
`cdrom_state`/`cdrom_trace` report the new fields.
Thread/fiber tracing — `traps.c`'s Win32-fiber TCB save/restore now
emits 12 thread-event kinds; new `ThreadTraceEntry` ring (64 K) plus
`thread_trace` / `thread_trace_clear` debug-server commands report
both live CPU regs and the target TCB's full saved register block.
wtrace / mmio rings — `WriteTraceEntry` grew (cpu_pc, sp, v0/v1/a0-a3/
t0/t1); capacity 1 M -> 4 M. `MmioTraceEntry` grew (pc, cpu_pc, sp,
a0-a3, sr, epc, i_stat, i_mask). Both dump commands accept
`count`/`newest=1` for reverse iteration. `restore_trace` and
`dirty_block_log` emit extended register fields (a2/a3/sp).
Misc — `interrupts.c` polls debug-server every 16K checks so it
stays responsive without a separate thread. `psx_runtime.h`'s
`call_by_address` routes through `psx_dispatch_call`. Dirty-RAM
interp JAL/JALR consult `psx_dispatch_game_compiled` when the game
is wired in. iso_reader gains the entry points needed by the game
disc path. New `tools/_tomba_probe.py` for port-4470 health checks.
Generated — regenerated `SCPH1001_dispatch.c` against the new
`psx_dispatch_impl` shape.
ISSUES.md — opened Issue #3 (BIOS PS-logo glyph missing on the
disc-detected screen, cosmetic).
.gitignore — added `recompiler/build-asan*/`, `build-checked/`,
`__pycache__/`.
Validated end-to-end: BIOS boots, disc detected, Whoopee Camp FMV
plays with audio. Frame count advances through ~40K frames without
freezes. `unknown_dispatch_log.total = 0`. Remaining FMV cluster
issues (1/4-size render, FMV-2 audio missing, no skip) tracked
under TombaRecomp/ISSUES.md.
Co-Authored-By: Codex (OpenAI) <noreply@openai.com>
Root cause: BIOS music engine reads voice register 6 (byte offset 0x0C,
CURVOL / ADSR_LEVEL) to find a free voice; recomp returned the never-
written spu_regs value (always 0), so the engine saw every voice as
silent and over-recycled voices 0-3 instead of fanning across all 24.
After fix, recomp KEYON-per-voice counts match Beetle exactly (e.g.
v0 2/2, v18 2/2, v22 1/1).
Also:
- Full ADSR envelope (Attack/Decay/Sustain/Release) ported from Beetle's
CalcVCDelta/RunEnvelope. KEYOFF now triggers Release-phase decay
rather than immediate silencing.
- End-without-loop triggers Release (matches RunDecoder), not active=0.
- ENDX latch at 0x1F801D9C/9E now reports actual block-end-reached
state instead of active-mask.
- SPU event ring (KEYON/KEYOFF/END_STOP/END_LOOP) on both backends with
frame-stamped per-voice payload; 1 M entries each (32 MB) for chime +
game-scene headroom. Beetle-side env>0 filter so silent-loop noise
doesn't bury the audible-voice timeline (matches recomp behavior
where deactivated voices don't fire further events).
- spu_voices / spu_events / spu_events_reset debug-server commands on
both ports. send_fmt buffer kept single-shot to avoid extra newlines.
- tools/_spu_ring_diff.py: frame-aligned event-ring diff with per-voice
count summary; chunk-count cap removed (was aborting reads at 98 KB).
- tools/_spu_capture.py, _spu_probe.py, _probe_one.py: targeted probes
used during diagnosis.
Local-only beetle-psx change (gitignored): same env>0 filter at
mednafen/psx/spu.cpp around line 345/352. Rebuild beetle-psx static
lib if reapplying.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New runtime/src/crash_trace.c writes psx_last_run_report.json with:
reason / timestamp / frame / last_func_addr / last_store_pc
full CPU register state (all 32 GPRs + hi/lo/sr/cause/epc)
dispatch_ring tail (last 256 dispatched func entries)
unknown_dispatch tail (last 50 entries with addr/phys/ra/a0/a1)
dirty_block_log tail (last 100 entries)
Windows SEH info (when triggered from SEH filter)
Triggered from:
- SIGSEGV / SIGABRT signal handlers
- Windows SetUnhandledExceptionFilter (catches access violations)
- atexit
- psx_unknown_dispatch fail-fast path (set PSX_FAIL_FAST_UNKNOWN_DISPATCH=1)
- trap_crash (existing fatal trap path)
File overwrites per dump (last-write-wins). Per CLAUDE.md §3 this is not
a log — it's a one-shot final state snapshot for crashes the running
TCP server cannot intercept.
Pattern mirrors F:/Projects/SuperMarioWorldRecomp/src/post_mortem.c.
Used the new infrastructure to capture a controlled crash with
0xBFC0989C re-seeded. Crash dump revealed the chain:
- JAL at 0xBFC09788 is jalr $s3, NOT jalr $a0
- $s3 was set in func_1FC09598 prologue: $s3 = $a2 (callee saves arg)
- Caller at 0xBFC093C4 in func_1FC092EC: $a2 = $s3 - 128 (delay slot)
- func_1FC092EC at 0xBFC09314: $s3 = $a3 (callee saves arg)
So real bug is upstream-of-upstream caller passing $a3 = 131 (0x83)
to func_1FC092EC, propagating down to a misaligned function pointer.
The 0xA000E1E8 we suspected as a function-pointer table was a
red herring — $a0 in the loop body is loaded from $fp+s1*4 (the
ARG to pass to the called fn), not the call target.
0xBFC0989C reverted out of seed list (was diagnostic-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Modal close on YES/NO COPY confirm now works on recomp. Cause: function
discovery (Ghidra + transitive walk) missed the function at 0xBFC1A1BC
which the BIOS shell calls indirectly during modal close. The 5 nearby
"missing" PCs the diagnostic identified are all in the same gap from
0xBFC1A1B8 to 0xBFC1A3B8; seeding 0xBFC1A1BC discovers the rest
transitively.
Status: modal closes cleanly on X. Actual copy execution still has
downstream missing-function-discovery issues (3 more PCs hit
psx_unknown_dispatch on copy: 0xBFC09890, 0xBFC0989C, 0xBFC02918)
plus likely additional bugs that softlock when those are seeded.
Those remain for follow-up; this commit is the modal-close milestone.
This is a BANDAID per user permission. The proper fix is to improve
function discovery so missed functions like 0xBFC1A1BC are found
automatically. Ghidra didn't see this function, and our transitive
walk didn't reach it either.
Tooling additions:
- psx_unknown_dispatch_record() ring buffer + unknown_dispatch_log
debug command. Replaces the prior file-based miss log per
CLAUDE.md §3 (no log files, ever). Always-on; queryable summary
(per-target counts) or tail (last N entries with addr/phys/ra/a0/a1).
- WTRACE_MAX_RANGES bumped from 16 -> 64 (boot defaults were
crowding out investigative arms).
- DirtyRamBlockLogEntry +a0/a1 fields, ring CAP 16K -> 4M
(previous 28ms eviction made retroactive press-window analysis
impossible).
Diagnostic scripts (kept):
tools/_unknown_dispatch.py — query unknown_dispatch_log
tools/_dispatch_check_v3.py — was a PC ever dispatched (RAM->ROM map)
tools/_dispatch_tail.py — last N dispatched targets
tools/_freeze_diag2.py — softlock investigator
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switches diagnostic methodology from a single binary running both backends
in lockstep (psx-beetleoracle.exe) to two independent processes:
psx-runtime.exe (recomp only, port 4370)
psx-beetle.exe (Beetle libretro only, port 4380)
Both expose the same JSON wire protocol for the common debug commands
(read_ram, press, set_input, clear_input, pad_status, screenshot_file,
wtrace_*, fntrace_* on Beetle). Cross-process comparison is done by
querying both ports from a tool, NOT by sharing input or memory.
Why: the embedded-oracle setup required keyboard input to drive both
backends in lockstep. Once their state diverged, the same keypress
drove them to different screens and "compare on the same press" became
unreliable. Two independent processes lets each backend be navigated
to its own state and queried on its own timeline.
Removed:
- runtime/src/beetle_psx_bridge.cpp (cannibalised into beetle_libretro.cpp)
- runtime/src/duckstation_bridge.cpp
- runtime/src/psx_oracle_cmds.c
- runtime/src/psx_oracle_stub.c
- runtime/include/psx_oracle_backend.h
- psx-beetleoracle and psx-dsoracle cmake targets
- 22 dual-backend tools (emu_read_ram / find_first_divergence / etc)
Added:
- runtime/src/beetle_main.cpp (SDL window + libretro driver + keyboard)
- runtime/src/beetle_libretro.cpp (libretro callbacks + wtrace/fntrace/sio rings)
- runtime/src/beetle_debug_server.c (TCP server, port 4380, same wire protocol)
- psx-beetle cmake target
Also bumps tooling caps surfaced during modal investigation:
- DIRTY_RAM_BLOCK_LOG_CAP: 16K -> 4M (28ms eviction -> ~7s retention)
- WTRACE_MAX_RANGES: 16 -> 64 (boot defaults filled the prior cap)
- DirtyRamBlockLogEntry: added a0/a1 fields for richer dispatch context
CLAUDE.md section 16 rewritten for the new methodology. New memory note
locks the direction so future sessions don't reach for the deleted
embedded-oracle commands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matthew Stanley