Add an opt-in horizontal low-edge transform for subu rd,zero,rt classifiers, with matching native and dirty-RAM interpreter behavior and guarded overlay variants.
Co-authored-by: Opok <novelasevilliousesp@gmail.com>
Two provably-correct framework fixes (independent of the AOT work) plus the
play-free overlay-extraction spike tooling and design doc.
Framework fixes (accurate, safe, complete):
- ABI v14: add gte_precision_store_word to OverlayCallbacks + preamble shim +
runtime population. The emitter emits a direct call for every swc2 (GTE
store-word); it is stateful runtime code (gte.cpp) so overlay DLLs must
forward or fail to LINK (undefined reference). Play-captured shards rarely
compiled a swc2; fuller static coverage surfaced the gap. Same pattern as
the v13 ws_player_x_bound / v10 gte_read_ctrl shims.
- Gate splitgen OFF in overlay mode (main_psx.cpp): the current recompiler's
unconditional split-TU output deletes <stem>_full.c and emits only shards,
which broke ALL overlay compilation (compile_overlays.py reads _full.c).
Overlays are single small TUs with no parallel-compile benefit; emit the
monolith. Restores overlay compile on the current tree.
AOT spike (docs/AOT_OVERLAY_PLAN.md + tools/aot_overlay_spike/, WIP):
- Play-free static overlay extraction from the disc image. Validated:
Tomba 1 reproduces 90% of played functions + ~1964 unplayed, live-verified
in-game (Dwarf Village runs native, zero runtime compile). Tomba 2 yields
661 functions byte-identical to the shipped/played vault (provably real),
all from the self-describing MAIN.EXE producer; the header-table BIN
producer needs loader-table RE (base recovery unreliable). Generality is
producer-dependent. Spike scripts are Tomba-specific proofs of concept
with hardcoded paths, not a finished tool.
- Correctness is guaranteed by the existing per-function code_crc dispatch
guard: a mis-positioned shard fails audit at compile or never fires at
runtime -> coverage loss, never wrong execution. Autocompile self-heals gaps.
CROSS-CUTTING: the ABI v14 change is part of the overlay codegen-hash source
set, so it ROLLS the cg tag (1f2ad026 -> e8484213). On master + a fw bump,
every title's overlay cache invalidates and needs regen+reshard. Branch only;
NOT pushed; other games' caches untouched. Needs user validation before merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
28165d6 (Einhander native-wide player bounds) rewrites configured
[widescreen] signed_x_bound LUI sites into psx_ws_player_x_bound(vanilla)
calls in code_generator.cpp — and that site loop runs in overlay_mode too, so
the call lands in overlay-resident code. But the shard link contract never
forwarded the symbol: any shard containing a bound site fails to LINK
(undefined psx_ws_player_x_bound) and that variant runs interpreted forever —
the same silent class as v10 gte_read_ctrl and v12 psx_rfe_mark_escape (now
loud under the shard observability suite, but still a coverage hole).
The clamp computes against the host's live widescreen state (gpu.c), so a
local copy can't work: add the ws_player_x_bound callback (appended last),
the preamble forwarder (identity fallback => correct 4:3 on a host that
predates it), and the loader init wiring. Bump the ABI to 13 so stale DLL
mixes are rejected and regenerate.
This also unblocks removing the sljit tier: sljit's emit_ws_player_x_bound
was the only other native path carrying the clamp; with the gcc shard path
forwarding it (and dirty_ram_interp already handling the site), the feature
no longer depends on sljit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter emits psx_rfe_mark_escape() at every rfe — including in
overlay-resident exception-context code — but the dispatch preamble
never provided it, so any rfe-containing overlay failed to LINK
(undefined reference) and ran interpreted forever: the gte_read_ctrl
silent-coverage-blocker class again (11 of Tomba's vault shards hit
it). It mutates shared RFE-pending escape state (interrupts.c), so the
shim forwards unconditionally through OverlayCallbacks; ABI bump to 12
rejects pre-shim caches.
Found live during LOAD_TIME_ZERO E0 bring-up: the user's first pig
load triggered autocapture and the compile died on this link error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The emitter routes mfc2/cfc2/mtc2/ctc2 on flag/IR/derived GTE registers
through direct gte_read_data/gte_read_ctrl/gte_write_data/gte_write_ctrl
calls (code_generator.cpp COP2 case), but the overlay DLL preamble had no
shims for them — any GTE-heavy overlay failed to LINK (undefined
gte_read_ctrl) and its region ran interpreted forever. That silently
blocked all new native coverage for the lava/Trolley attract-demo regions
after the caches rotated.
GTE state is shared runtime state, so the shims forward unconditionally
through OverlayCallbacks (a local copy would fork the flag/IR timeline).
ABI bumped 9 -> 10; the loader gate rejects+deletes pre-shim DLLs and
autocompile self-heals; plain gte_data[]/gte_ctrl[] accesses stay direct
CPUState reads in emitted code, matching the emitter.
Smoke-validated: previously-failing GTE-heavy captures now compile clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cycle-accuracy emitter emits calls to 12 timing runtime functions
(psx_cyc_load_*, psx_icache_fetch, psx_muldiv_*, psx_gte_*, psx_slice_block,
...) + debug_server_cyc_observe + g_debug_last_store_pc. Overlay DLLs never
got these symbols, so every overlay failed to link -> 0 native shards ->
100% interpreter (unplayable). Fix (option A): expose the 12 via the overlay
callback ABI.
- overlay_api.h: +12 timing callbacks in OverlayCallbacks, ABI 8 -> 9.
- overlay_loader.c: populate the 12; add warn_on_cgtag_mismatch() (loud
warning when shards exist under the wrong cg-hash).
- compile_overlays.py: 12 preamble forwarders + local g_debug_last_store_pc
+ -DPSX_NO_DEBUG_TOOLS.
- cosim.c/main.cpp: input-cosim setpad + held pad (deterministic input).
- cosim.py: env-overridable COSIM_EXE/CWD/GAME.
KNOWN REGRESSION (patched on a follow-up branch): Tomba pause menu (triangle)
wedges. Root-caused: an IRQ taken mid-block resumes at the function entry
leader instead of the interrupted PC, re-zeroing the menu's VSync wait
counter (scratchpad 0x1F8001E8) every frame so it never reaches its target.
Not the scheduler (LLE reproduces) and not timing math -- interrupt-resume
re-entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In-progress investigation work snapshotted before rebasing onto the
tcc-refactored psxrecomp master. Includes overlay ABI v8
(check_interrupts_at resume-PC callback for async RFE recovery) and
code_generator/full_function_emitter delay-slot cycle accounting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root-caused the Tomba2 native logo-stuck bug: it is a guest-cycle-timeline
divergence between the compiled-overlay backend and the dirty-RAM interpreter,
surfacing through a Timer1 (0x1F801110) debounce loop at game pc 0x8008592C that
reads 1 (native) vs 2 (interp), forking logo->FMV. The handoff's "jalr target
divergence" was a benign store-PC artifact (read-watch proves a0=0xB870 and
t3=[a0]=0x800896E0 are identical in both backends); the scratchpad blind spot
was ruled out (separate sp fingerprint forks at the same frame).
First-divergence observability (debug_server.c, memory.c):
- frame_fingerprint: added separate sp (scratchpad) cumulative hash column +
per-entry cyc, so a scratchpad or cycle fork is classified, not hidden.
- Unified ordered access recorder: one buffer, kind = ramw/spw/mmiow/mmior/ramr,
array index = true execution order, per-entry cyc (psx_get_cycle_count).
record_frame_dump emits the unified stream; record_reads_dump filters reads.
- Targeted main-RAM read-watch (PSX_READ_WATCH="lo,hi" + g_ram_read_watch_active
guard in psx_read_word) — proves a load's value/address per backend.
- PSX_RECORD_FRAME env seed: arm the recorder from boot (race-free), never
arm-then-time.
Fix A (cycle-model unification, part 1): compiled-overlay code was built WITHOUT
PSX_ENABLE_BLOCK_CYCLES, so every psx_advance_cycles() in overlay code was
compiled out -> a function run as a native overlay charged ~0 cycles while the
same function via the interp charged per-instruction (~32x), the entire drift.
- overlay_api.h: OverlayCallbacks gains advance_cycles; ABI v6->v7 (rejects
stale zero-cycle DLLs).
- compile_overlays.py: build overlays with -DPSX_ENABLE_BLOCK_CYCLES=1 and
emit a psx_advance_cycles shim forwarding to the shared host timeline.
- overlay_loader.c: wire s_callbacks.advance_cycles = psx_advance_cycles.
Result: the first debounce now reads Timer1=2 in native (was 1); overlay cycle
charge went 9.5k -> 380k (interp 306k). Native still logo-stuck: a residual,
perfectly linear ~1 cycle/write over-charge remains (block.instruction_count
counts branch + delay-slot as 2 while the interp charges the pair as 1 step),
which flips a later debounce 3 vs 2. Fix A pt2 (cycle-exact block totals) is the
next step. wt/tomba2 only; no pin bump, no master merge, no other-game regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the fail-closed native entry guard from wt/tomba2 (b5a6940 / bd60381) to
master, where it had never been merged — which is why Tomba (and any master-built
title) still wedged. This is the long-standing "blue screen" / shared-RAM-
divergence-between-interp-and-native class.
Root cause: an overlay function dispatched at a FOREIGN INTERIOR PC (a range-
ownership mismatch routed a PC the function does not own into it) hit the CPS
entry-switch `default: break`, which fell through and ran the function FROM ITS
TOP — re-running its prologue and corrupting shared CPU/RAM state, so the guest's
main loop then waits forever on a condition that never gets set (the 0xCF0 SIO
spin / blue screen). Reproduced on Tomba's Village of All Beginnings: a native
0x000E7000-region shard function entered at a foreign PC wedged; the interpreter
ran the same bytes correctly.
Fix: the generated CPS entry-switch for OVERLAY functions (config_.overlay_mode)
now FAILS CLOSED on an unknown non-zero continuation PC — it restores the PC and
calls psx_native_bad_entry instead of running from the top. overlay_loader_dispatch
consumes g_native_bad_entry and routes the PC to the sanctioned dirty-RAM
interpreter (the live bytes run correctly — NOT a stub/HLE). True-prologue entry
still runs from the top. Static (BIOS/boot-EXE) functions keep the legacy fall-
through. ABI bumped 5->6 so stale ABI-5 caches are rejected and rebuilt with the
guard.
Minimal port: kept only the fail-closed mechanism + its ABI plumbing. The Tomba2
host_aliases interior-routing and the native-block / CPS-probe diagnostics (which
depend on the overlay-cache-v2 multi-range Candidate fields not on master) were
intentionally omitted — the fail-closed default alone is the safety net.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual PSX_OVERLAY_CODEGEN_VER must be hand-bumped on every emitter change;
when it isn't (e.g. between CPS emitter iterations), a stale-but-cgN DLL is reused
-> the v0.3.0 black screen. Now hash_codegen.cmake hashes the recompiler codegen
sources into a (gitignored) overlay_codegen_hash.h at build time; overlay_api.h
exposes PSX_OVERLAY_CODEGEN_HASH and the cache path becomes gcc/<arch-abi>/cg<N>_<hash>/.
The loader and compile_overlays.py read the same value, so any emitter change
auto-invalidates the cache (old dirs coexist, no stale load). Falls back to hash 0
when the header is absent (fresh checkout) — identical to today's single-cgN path.
Verified: build emits hash 0x347751c1; autocompile + loader both use cg4_347751c1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RECURSION_BUG.md §25.5. Brings the overlay native path under continuation-passing so the
idle-freeze leak fix holds with overlay_cache=true (the production gcc>sljit>interp path).
Runtime CPS-mode flag g_psx_cps_mode (overlay_loader.c) set by a __attribute__((constructor))
emitted in the BIOS + game CPS dispatch (full_function_emitter.cpp / main_psx.cpp). Gates all
overlay/sljit CPS behavior; legacy unaffected.
sljit JIT (overlay_sljit.c), scoped:
- jr $ra publishes cpu->pc (was a bare return). jr rX already tail-transferred.
- call-containing fragments DECLINE to the interpreter (CPS-safe, depth-1) — leaf shards still
JIT. (Full sljit call coverage / per-fragment entry-switch is future work.)
gcc-DLL path — FULL native overlay coverage (validated):
- compile_overlays.py --cps sets PSX_CPS for the recompiler -> CPS overlay C (tail-transfer +
per-function entry-switch; --overlay mode only affects widescreen, not CPS). DISPATCH_PREAMBLE
psx_syscall decl fixed void->int.
- PSX_OVERLAY_CODEGEN_VER 3->4: fresh cg4/ cache namespace; legacy unit-model DLLs (cg3) ignored.
- overlay_loader_dispatch: under CPS, when a dispatched addr is not an overlay ENTRY but lies
inside a registered candidate's code range (a tail-transferred call's continuation),
overlay_find_by_range() re-enters that function with cpu->pc=addr so its entry-switch routes
to the block. gcc DLLs are the trusted tier (diff-validated at entry) -> run native directly.
This is the CPS equivalent of the old unit-model that fixed the dwarf->overworld blue screen.
Validated: CPS overlay DLLs compile + load + dispatch_native>0; with continuation routing
dispatch_interp_fallback -> 0 (full native), ce_profile FLAT (6 KB), boots clean. Addresses are
KSEG0-consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The complete §25 fix: make BOTH recompiler emitters emit guest calls as
TAIL-TRANSFERS driven by the unified flat trampoline + the guest's own $ra/$sp,
instead of nesting one host C frame per guest call. The host stack can no longer
mirror unbounded guest re-entrancy — the architectural root of the ~14-min idle
freeze. Gated gen-time by PSX_CPS; legacy codegen byte-identical when unset.
VALIDATED (user): full-CPS Tomba boots BIOS -> FMV -> New Game -> village idle and
plays; ce_profile cur_max_kb FLAT at 5 KB from frame ~9.9k through 56,890 (turbo
soak) — blew past the ~50k-frame freeze threshold with the host stack bounded
(legacy climbed +1.17 KB/frame -> 48 MB -> freeze).
KEY CORRECTION to the prior session's §25.1: there are TWO emitters. The BIOS
(full_function_emitter.cpp) was done last session but the §8 leaking chain
(func_8001A954 ...) is GAME code, emitted by code_generator.cpp (via main_psx.cpp)
— never converted. Also, §25.1's "continuations not in the dispatch table"
hypothesis was wrong (they were always added); the real boot regressions were
found by dumping the always-on fntrace ring on the abnormal psx_dispatch return.
Emitter changes (gated PSX_CPS):
- full_function_emitter.cpp (BIOS): BIOS A0/B0/C0 vector runtime-fallback
tail-transfers instead of nested psx_dispatch_call (was the frame-0 exit).
- code_generator.cpp/.h (game): jal/jalr/jr-$ra/jr-table/branch-split/j-split ->
tail-transfer (jalr captures rs before the link, alias-safe); each call's
return point registered as a dispatchable continuation; entry-switch (keyed on
cpu->pc) at the top of every function AND every overlapping-alias body. 0
psx_call_contract / call_by_address emitted under CPS.
- main_psx.cpp: psx_dispatch_game_compiled gains continuation cases
(case cont: cpu->pc=cont; func_owner(cpu)); entries clear cpu->pc.
- strict_translator.cpp: syscall emits `if (psx_syscall(...)) return;` so a void
syscall falls through to the inline post-syscall jr $ra (real-HW EPC+4).
Runtime changes (the legacy "return via nested dispatch / cpu->pc=0" idioms that
exit the flat CPS trampoline):
- traps.c: psx_syscall returns int (0=void/fall-through, 1=transfer);
Enter/ExitCriticalSection return 0; case-3 ChangeThread-success returns 0 — the
fiber scheduler leaves cpu->pc=0 on resume, which was the frame-712 exit.
- cpu_state.h / overlay_api.h: psx_syscall signature void->int; overlay callback
ABI bumped 4->5 (stale overlay caches rejected).
- dirty_ram_interp.c: gate interp_enter_compiled behind PSX_HAS_GAME_DISPATCH
(un-breaks the BIOS-only build).
- main.cpp: dump the always-on fntrace ring tail + stack to psx_cps_exit_trace.json
on an abnormal psx_dispatch return (diagnostic; CLAUDE.md ring-buffer model).
Docs: RECURSION_BUG.md §25.2-§25.4. Remaining: dwarf->overworld playtest;
Stage 3 (sljit/overlay CPS parity); then consider making CPS default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
[widescreen.cull] auto_backdrop — generic far-backdrop column preload for
16:9. A shared detector (runtime/include/ws_backdrop_detect.h) finds each
scrolling-backdrop column-window generator by its /96-magic + 0x176-camX +
sra,5 divide-tail invariant and records the window's START/END bound
instructions + width (window_cols). All four execution paths route those
bounds through psx_ws_backdrop_value(orig, is_end, window_cols), which widens
the camera-tracked window by exactly the 16:9 reveal (margin scales with
window_cols; identity at 4:3 so the build stays byte-identical with widescreen
off). gcc emit + sljit + dirty-RAM interp + overlay ABI v4 / codegen ver 3.
Also: CodeGenConfig::overlay_mode — explicit [widescreen.*] per-address site
shape checks now SKIP (apply only where bytes match) instead of hard-exit in
overlay mode, because overlay regions reuse the same address for different
code across scene variants. Main-EXE mode keeps the hard error.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three coupled landings on top of the sljit merge:
1. Shared widescreen render-cull helper. auto_screen_x now emits a call to one
runtime helper psx_ws_cull_sltiu(sx,imm) (gpu.c) instead of an inline expr, and
the sljit JIT (overlay_sljit.c) + dirty-RAM interpreter (dirty_ram_interp.c)
route flagged render-funnel sltiu 0x140/0x141 through the SAME helper. So an
overlay's GTE render funnel widens identically whether it runs as a gcc DLL, a
sljit shard, or interpreted. Helper is byte-identical to vanilla at 4:3
(margin 0), so widescreen-off stays bit-exact. psx_ws_func_has_screen_cull
gates the widen to real render funnels (sltiu 0x140 + sltiu 0xE0 signature).
2. Versioned overlay cache path. gcc cache moves to <game_id>/gcc/<arch-abi>/cg<N>/
where N = PSX_OVERLAY_CODEGEN_VER (overlay_api.h), parsed identically by the
runtime (overlay_loader.c) and compile_overlays.py so they can't drift. A build
whose emitter output changed (e.g. this auto_screen_x change) reads+writes a
FRESH dir and never reuses a stale DLL; old versions coexist (no auto-delete).
Pre-1.0: legacy flat/unversioned fallbacks dropped.
3. sljit default-on (validated). gcc and sljit are complementary, not exclusive:
gcc fills the cache via prebuilt DLLs + async batch (compile_fragment NULL),
sljit JITs gcc-absent regions synchronously on-miss. New code_provider_sljit()
exposes sljit for the JIT-on-miss gap-fill independent of the "active" backend;
the dispatch gate fires whenever sljit is available (head<0 == gcc-absent), and
apply_live_policy defaults live ON when sljit is available. Priority gcc > sljit
> interp; shards still route through the same-state diff gate (device-touch +
diverging shards pinned to interp) so default-on does NOT reintroduce the
save-load wedge; a declined/failing shard falls to interp (never a crash).
Verified live: live=1, 62 shards compiled, 2 declines->interp, native:interp
ratio ~31%->~87%, gcc still primary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 16:9 edge pop-in is the parallax 2D BACKDROP layer (ocean/cloud/mountain/
grass), drawn by overlay actor handlers (FUN_801216bc family) that compute
screenX = (worldX - camX) >> parallax in pure integer math and store it to the
object's +0x12 field WITHOUT the GTE — so the GTE X-squash that gives 3D the
wider 16:9 FOV never reaches them and far pieces sit past the 320px edge,
clipped (the blue void / half-rectangles at the edges). See TombaRecomp
ISSUES.md #8C for the full root-cause trail.
Fix = psx_ws_backdrop_x() (gpu.c): squash screenX around screen-centre by the
same ws factor as the GTE, identity at 4:3. Applied on BOTH execution paths
because overlay code runs interpreted when no cache DLL is loaded (the common
case in dev — see #8G):
- Native: recompiler emits write_half(.., psx_ws_backdrop_x(rt)) at the
[widescreen.backdrop] x_sites store (config_loader + code_generator +
main_psx --ws-config so the overlay compile gets the sites). Forwarded into
overlay DLLs via OverlayCallbacks {ws_backdrop_x, ws_x_margin, ws_sprite_tag}
(overlay ABI v2->v3 as the cache discriminator); compile_overlays.py glue +
--ws-config passthrough.
- Interp: dirty_ram_interp SH hook keyed by psx_ws_is_backdrop_site(pc); the
runtime registers the x_sites from game.toml (main.cpp).
Verified in data: backdrop slot screenX squashes 371 -> 318 (pulled on-screen).
PARTIAL: visible culling persists -> remaining cause is the backdrop spawn/
despawn window (separate from screenX clip); see #8C "REMAINING CAUSE".
Also added (earlier this session): ws_census draw ring, ws_margin override,
gpu_state ws fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The overlay cache keys DLLs by guest-bytes CRC, which is flavor-blind — the
same guest overlay yields the same filename whether compiled by the base
recompiler or a variant (e.g. widescreen, which emits psx_ws_*/GTE-adjust
symbols). Two flavors sharing a cache dir would collide: a base runtime could
pick up a widescreen DLL (or vice versa) with no guard, since both report ABI
v2.
Fold a codegen-flavor id into overlay_abi() (high 16 bits) and check the
combined tag in the loader's existing ABI gate. Base/master flavor is 0, so the
tag stays == PSX_OVERLAY_ABI_VERSION — fully backward compatible with every
existing base DLL (verified: an 8-DLL base cache loads with zero rejections).
A variant build sets PSX_OVERLAY_FLAVOR (e.g. widescreen: -DPSX_OVERLAY_FLAVOR=1
for the runtime, --flavor 1 for compile_overlays.py); its overlays then report a
distinct tag, so the base runtime rejects+regenerates them and vice versa. The
caches stay separate automatically, even in a shared directory — no new paths.
overlay_api.h: PSX_OVERLAY_FLAVOR (default 0) + PSX_OVERLAY_ABI_TAG.
overlay_loader.c: both ABI gates check the tag.
compile_overlays.py: --flavor arg -> -DPSX_OVERLAY_FLAVOR, baked into overlay_abi().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug D root cause (Tomba chest freeze, also QA issue #1): the game's own
dispatcher func_80056E08 has a dead jumptable case (type 0x21 ->
jal 0x80120B3C) whose target is mid-function in EVERY overlay variant on
the disc (only the boot EXE references it; the real handler entry is
0x80120AC8). Entering mid-function skips the prologue, runs an epilogue
for a frame that was never pushed, and returns through garbage. Our C
model then resumed the suspended generated continuations on the moved
guest stack -> double-executed tails -> infinite zombie loop (~3K wild
calls/frame) -> MMIO fatal. Family root of Bugs A/C/D.
The contract: a generated C continuation may only run if the guest
actually arrived at it ($ra == call site return address, $sp == caller
sp at the call). Violations begin a "bail" unwind: cpu->pc := the wild
jr's true target, every generated frame returns immediately, and the
unwind resolves at the first enclosing site whose contract matches (or
flattens at the outermost dispatch loop, re-dispatching the wild target
on a clean host stack -- matching real-HW one-stack semantics).
- cpu_state.h: bail state + psx_call_contract() helper (+ overlay-DLL
pointer indirection under PSX_OVERLAY_DLL_BUILD)
- emitters: (ra, sp) checks at every BIOS and game jal/jalr site;
psx_dispatch_impl validates pc==0 and stop-hits against (stop, sp),
handles bail resolve/propagate/flatten
- dirty_ram_interp: surfaced calls validate the contract on return;
bail aborts the interp run; tail transfers into compiled code now run
with pc cleared (closes a latent double-run on plain C returns)
- overlay cache ABI v2: DLLs export overlay_abi(); loader rejects and
deletes stale DLLs (autocompile regenerates with contract checks)
- telemetry: bail_first/resolved/flattened/anomaly in heartbeat + dump
headers; anomaly guard at exception entry; TCP_COMMANDS.md docs
Verified: BIOS+Tomba regen, boot to intro at locked 60.0 fps, all bail
counters 0 through boot/title/attract (no false positives). Chest-area
gameplay verification pending (requires user play).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the overlay JIT cache below the 0x98000 floor to kernel RAM
[0, 0x10000) — the BIOS part-2 image + install-at-runtime stubs
(Rule 18), dominated by the SIO data-byte slot at 0xCF0 whose
install-slot hook tail-dispatches the whole exception-handler tail
into the interpreter on EVERY exception once installed.
Window model (dirty_ram_interp.h): two capture/candidate windows —
kernel [0, 0x10000) and overlay [0x98000, RAM); main-EXE text between
them is statically recompiled, never captured. Interp local-flow gates
intentionally stay overlay-only (kernel runs per-block in exception
context; named constant documents this).
- overlay_capture: per-window dirty-run enumeration, runs clamped at
window edges (kernel CPU-write pages and main-EXE DMA pages can be
adjacent; never merge them).
- overlay_loader: dispatch lazy-load gate covers both windows;
scan_cache_dir validates the <addr8>_<crc8>.dll shape explicitly
(kernel region_start 0 is legal, zero-parse is no longer a sentinel).
- classifier (compile_overlays.py): kernel-window-only DISPATCH_ROOT
promotion — an orphan dispatch interior no rooted walk covers has no
host to truncate and is a real execution root by construction (the
static install-slot hooks tail-dispatch into it). One orphan per
iteration so remaining orphans alias into the new host. Overlay
regions stay ineligible (cross-variant per-PC evidence: 0xE889C class).
- main_psx --overlay: 'dispatch_root' seed lines are trusted walk roots
(exempt from the boundary re-check); seed range is now the loaded
image's own bounds (the old hardcoded [0x80010000,0x80200000) window
silently dropped kernel seeds).
- overlay_api/loader/DLL preamble: route psx_restore_state_escape
through OverlayCallbacks (appended last, by-value copy keeps old
DLLs compatible) — exception-context kernel code longjmp-returns.
Tomba verified (user-played: title -> LOAD list renders -> save loads
-> overworld laps): kernel interp per-PC rows 3 -> 0 (was 0xCF0 =
61,775 dispatches / 2.71M insns at the LOAD screen alone, ~4.6M
insns/session); host func [0xCF0,0xEA0) + 0xDF8/0xE08 aliases, audit
clean; dispatch_native 357,763; dispatch_miss_total 0; interp aborts 0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reinstates the runtime-loaded overlay DLL cache (Layer A) and the offline
overlay compilation pipeline that were built on the pruned codex/cache-discovery
branch (tip 2bbafc2) and never merged. Ported onto current master; rebuilt
rather than history-restored.
Runtime (off by default, opt in via [runtime] overlay_cache = true):
- overlay_loader.c/.h, overlay_api.h: CRC32-keyed DLL cache at
cache/<game_id>/<region>_<crc>.dll. On a dirty-RAM dispatch miss the loader
LoadLibrary's the matching DLL, wires runtime callbacks via overlay_init(),
enumerates func_XXXXXXXX exports, and dispatches them BEFORE the interpreter.
- overlay_capture.c/.h: write-once capture set; records overlay DMA bytes and
writes overlay_captures.json (execution-verified seeds) at clean exit.
- dma.c: capture + cd_dma_log hook at CD DMA completion (async model).
- cdrom.c: cdrom_get_setloc_lba() for disc-position correlation.
- dirty_ram_interp.c: g_dirty_ram_exec_pc_table (exec-verified PCs), a
decodability guard on JAL-into-dirty targets, and the overlay dispatch hooks.
- debug_server.c: cd_read_log + overlay_loader_status TCP commands.
- config_loader: [runtime] overlay_cache flag (default false).
Recompiler (compile_overlays.py / extract_overlays.py drive these):
- main_psx.cpp --overlay-exact: analyze only explicit entries + direct JAL
targets and set split_mid_function_targets=false, so overlay basic-block
targets are NOT split into bogus standalone functions (the mid-function-seed
failure mode that produced black-on-load / blue-on-new-game).
- function_analysis.cpp: analyze_exact_entries(); code_generator.cpp: gate the
mid-function split pre-pass behind split_mid_function_targets (default true,
so normal main-EXE codegen is unchanged - Tomba still regens to 2854 funcs).
Deltas from the original: save states dropped (never worked); overlay enable is
runtime-config-gated (off by default) instead of a build-time define; loader
stderr logging converted to the overlay_loader_status TCP path (CLAUDE.md Rule 3).
Recovered from _backups/2026-06-06-consolidation/psxrecomp-all-PRE.bundle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matthew Stanley